zoukankan      html  css  js  c++  java
  • 启动Flink的yarnsession模式出错

      问题如下:bin/yarn-session.sh -d 报错

     The program finished with the following exception:
    
    org.apache.flink.client.deployment.ClusterDeploymentException: Couldn't deploy Yarn session cluster
        at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:425)
        at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:606)
        at org.apache.flink.yarn.cli.FlinkYarnSessionCli.lambda$main$4(FlinkYarnSessionCli.java:860)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
        at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
        at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:860)
    Caused by: org.apache.flink.configuration.IllegalConfigurationException: The number of requested virtual cores for application master 1 exceeds the maximum number of virtual cores 0 available in the Yarn Cluster.
        at org.apache.flink.yarn.YarnClusterDescriptor.isReadyForDeployment(YarnClusterDescriptor.java:326)
        at org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:534)
        at org.apache.flink.yarn.YarnClusterDescriptor.deploySessionCluster(YarnClusterDescriptor.java:418)
        ... 7 more

      解析:日志中显示的大概原因就行启动失败,资源不够,但是我启动的时候并没有指定任何参数,使用的是默认值,申请的资源大小绝对不会超过可用的资源大小,完全够用,所以报错原因跟运行时指定的参数无关,那么我们该如何解决呢?

      解决:之前测试时启动太多次Filnk的yarn-session模式,所以断定是临时文件过多导致,故删除HDFS中的Flink的临时文件即可,你的文件存在哪就删除哪,我存在/flink/yarn/ha目录下,所以直接删除即可,可以在flink的conf目录下的flink-conf.yaml查看HDFS的存储路径

      再次启动,发现启动成功

    bin/yarn-session.sh -d

  • 相关阅读:
    ORA-01940: cannot drop a user that is currently connected解决方法
    Git基本用法简介
    C 入门 第十节 存储区
    C 入门 第九节 结构体指针
    C 入门 第八节 指针
    C 入门 第七节 结构体
    C 入门 第六节 自定义函数
    C 入门 第五节 多维数组 字符串数组
    C 入门 第四节
    C 入门 第三节
  • 原文地址:https://www.cnblogs.com/LzMingYueShanPao/p/15103250.html
Copyright © 2011-2022 走看看