zoukankan      html  css  js  c++  java
  • Marking as slave lost.

    Spark on Yarn提交任务时报ClosedChannelException解决方案_服务器应用_Linux公社-Linux系统门户网站 http://www.linuxidc.com/Linux/2017-01/140068.htm

    yarn.nodemanager.pmem-check-enabled true Whether physical memory limits will be enforced for containers.
    yarn.nodemanager.vmem-check-enabled true Whether virtual memory limits will be enforced for containers.

    <property>
        <name>yarn.nodemanager.pmem-check-enabled</name>
        <value>false</value>
    </property>

    <property>
        <name>yarn.nodemanager.vmem-check-enabled</name>
        <value>false</value>
    </property>

    ssh://root@192.168.2.51:22/usr/bin/python -u /root/.pycharm_helpers/pydev/pydevd.py --multiproc --qt-support=auto --client '0.0.0.0' --port 40894 --file /home/data/crontab_chk_url/pyspark/pyspark_yarn_test.py
    pydev debugger: process 3778 is connecting

    Connected to pydev debugger (build 172.4343.24)

    Setting default log level to "WARN".
    To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
    17/12/03 21:30:45 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    17/12/03 21:30:46 WARN util.Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
    17/12/03 21:30:47 WARN yarn.Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
    17/12/03 21:30:55 ERROR client.TransportClient: Failed to send RPC 8836767686150811845 to /192.168.2.51:34970: java.nio.channels.ClosedChannelException
    java.nio.channels.ClosedChannelException
    at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source)
    17/12/03 21:30:55 WARN cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: Attempted to get executor loss reason for executor id 1 at RPC address 192.168.2.51:35068, but got no response. Marking as slave lost.

  • 相关阅读:
    DOM事件
    DOM样式操作
    asp.net-枚举绑定控件
    微信站点 点击 “退回” 按钮退回到主菜单
    阻止iOS中页面弹性回滚,只允许div.phone_body的区块有弹性
    asp.net 微信开发失效汇总
    ECharts使用心得
    PV、UPV、UV简介
    微信浏览器取消缓存的方法
    Visual Studio 2015简体中文企业版/专业版下载+有效激活密钥
  • 原文地址:https://www.cnblogs.com/rsapaper/p/7967459.html
Copyright © 2011-2022 走看看