zoukankan      html  css  js  c++  java
  • 启动Flink的yarnsession模式出错

      问题如下:bin/yarn-session.sh -d 报错

     The program finished with the following exception:
    
    org.apache.flink.client.deployment.ClusterDeploymentException: Couldn't deploy Yarn session cluster
        at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:425)
        at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:606)
        at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$4(FlinkYarnSessionCli.java:860)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
        at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
        at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:860)
    Caused by: org.apache.flink.configuration.IllegalConfigurationException: The number of requested virtual cores for application master 1 exceeds the maximum number of virtual cores 0 available in the Yarn Cluster.
        at org.apache.flink.yarn.YarnClusterDescriptor.isReadyForDeployment(YarnClusterDescriptor.java:326)
        at org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:534)
        at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:418)
        ... 7 more

      解析:日志中显示的大概原因就行启动失败,资源不够,但是我启动的时候并没有指定任何参数,使用的是默认值,申请的资源大小绝对不会超过可用的资源大小,完全够用,所以报错原因跟运行时指定的参数无关,那么我们该如何解决呢?

      解决:之前测试时启动太多次Filnk的yarn-session模式,所以断定是临时文件过多导致,故删除HDFS中的Flink的临时文件即可,你的文件存在哪就删除哪,我存在/flink/yarn/ha目录下,所以直接删除即可,可以在flink的conf目录下的flink-conf.yaml查看HDFS的存储路径

      再次启动,发现启动成功

    bin/yarn-session.sh -d

  • 相关阅读:
    堆排序的应用优先级队列
    php基础语法知识笔记(二)
    XP系统下用虚拟机安装centos 5.6(Linux系统)图文教程(一)—“快速安装”
    linux系统启动服务解释与优化(centos 5版本)
    XP系统下用虚拟机安装centos 5.6(Linux系统)图文教程(二)—“linux优化配置”
    php基础语法知识笔记(一)
    asp.net SQLite数据库操作类,
    SilverLight 相关资源网址 (更新中。。。)
    SPQuery查询语法简要说明
    asp.net SQLite数据库操作Demo
  • 原文地址:https://www.cnblogs.com/LzMingYueShanPao/p/15103250.html
Copyright © 2011-2022 走看看