zoukankan      html  css  js  c++  java
  • YARN调试过程中的常见问题

    执行操作:

    hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.0.jar wordcount  /user/today/input  /user/today/output

    报错:

    错误: 找不到或无法加载主类 org.apache.hadoop.mapreduce.v2.app.MRAppMaster

    解决:

    在mapred-site.xml 和 yarn-site.xml添加如下

    <property>

        <name>yarn.application.classpath</name>

        <value>

            /opt/hadoop-2.6.0/etc/hadoop,

            /opt/hadoop-2.6.0/share/hadoop/common/*,

            /opt/hadoop-2.6.0/share/hadoop/common/lib/*,

            /opt/hadoop-2.6.0/share/hadoop/hdfs/*,

            /opt/hadoop-2.6.0/share/hadoop/hdfs/lib/*,

            /opt/hadoop-2.6.0/share/hadoop/mapreduce/*,

            /opt/hadoop-2.6.0/share/hadoop/mapreduce/lib/*,

            /opt/hadoop-2.6.0/share/hadoop/yarn/*,

            /opt/hadoop-2.6.0/share/hadoop/yarn/lib/*

        </value>

      </property>

    报错:

    INFO mapreduce.Job: Task Id : attempt_1533612124021_0001_m_000000_2, Status : FAILED

    Container [……] is running 482732544B beyond the 'VIRTUAL' memory limit. Current usage: 34.4 MB of 1 GB physical memory used; 2.5 GB of 2.1 GB virtual memory used. Killing container.

    解决:

    在etc/hadoop/yarn-site.xml文件中,修改检查虚拟内存的属性为false

    <property>

        <name>yarn.nodemanager.vmem-check-enabled</name>

        <value>false</value>

    </property>

    执行成功

    web页面 ip:8088

     

  • 相关阅读:
    HTML Rendering Error
    PyCharm将选中的内容加上引号
    Scrapy中的Request和日志分析
    PyCharm关闭按两次Shift进入搜索框的功能
    博客园更换模板的链接
    Scrapy的Spider类和CrawlSpider类
    Scrapy安装和简单使用
    计算π的近似值
    Python用pip安装第三方库时换源下载
    网页解析 -- bs4 和 xpath 的简单使用
  • 原文地址:https://www.cnblogs.com/jin-nuo/p/9437186.html
Copyright © 2011-2022 走看看