zoukankan      html  css  js  c++  java
  • 根据硬件配置后mapred-site.xml和yarn-site.xml

    机器总共16G内存,sqoop导入数据时大约需要2G左右

    mapred-site.xml

    <configuration>
        <property>
            <name>mapreduce.framework.name</name>
            <value>yarn</value>
        </property>
    
        <property>
            <name>mapreduce.jobhistory.address</name>
            <value>master:10020</value>
            <description>MapReduce JobHistory Server IPC host:port</description>
        </property>
    
        <property>
            <name>mapreduce.jobhistory.webapp.address</name>
            <value>master:19888</value>
            <description>MapReduce JobHistory Server Web UI host:port</description>
    
        </property>
        <property>
            <name>mapred.child.java.opts</name>
            <value>-Xmx3000m -Xmx3000m -XX:-UseGCOverheadLimit</value>
        </property>
    
        <property>
            <name>mapreduce.reduce.memory.mb</name>
            <value>4096</value>
        </property>
    
        <property>
            <name>mapreduce.map.memory.mb</name>
            <value>4096</value>
    
        </property>
        <property>
            <name>mapreduce.task.timeout</name>
            <value>600000</value>
        </property>
    
    </configuration>

    yarn-site.xml

    
    <configuration>
        <property>
            <name>yarn.resourcemanager.hostname</name>
            <value>master</value>
        </property>
    
    
        <property>
            <description>The address of the applications manager interface in the RM.</description>
            <name>yarn.resourcemanager.address</name>
            <value>${yarn.resourcemanager.hostname}:8032</value>
        </property>
    
    
        <property>
            <description>The address of the scheduler interface.</description>
            <name>yarn.resourcemanager.scheduler.address</name>
            <value>${yarn.resourcemanager.hostname}:8030</value>
        </property>
    
    
        <property>
            <description>The http address of the RM web application.</description>
            <name>yarn.resourcemanager.webapp.address</name>
            <value>${yarn.resourcemanager.hostname}:8088</value>
        </property>
    
    
        <property>
            <description>The https adddress of the RM web application.</description>
            <name>yarn.resourcemanager.webapp.https.address</name>
            <value>${yarn.resourcemanager.hostname}:8090</value>
        </property>
    
    
        <property>
            <name>yarn.resourcemanager.resource-tracker.address</name>
            <value>${yarn.resourcemanager.hostname}:8031</value>
        </property>
    
    
        <property>
            <description>The address of the RM admin interface.</description>
            <name>yarn.resourcemanager.admin.address</name>
            <value>${yarn.resourcemanager.hostname}:8033</value>
        </property>
    
    
        <property>
            <name>yarn.nodemanager.aux-services</name>
            <value>mapreduce_shuffle</value>
        </property>
    
        <property>
            <name>yarn.scheduler.maximum-allocation-mb</name>
            <value>9216</value>
            <discription>每个节点可用内存,单位MB,默认8182MB</discription>
        </property>
    
        <property>
            <name>yarn.nodemanager.vmem-pmem-ratio</name>
            <value>2.1</value>
        </property>
    
        <property>
            <name>yarn.nodemanager.resource.memory-mb</name>
            <value>10240</value>
        </property>
    
    </configuration>
  • 相关阅读:
    VS2010 自动跳过代码现象
    Reverse Linked List II 【纠结逆序!!!】
    Intersection of Two Linked Lists
    Linked List Cycle II
    Remove Nth Node From End of List 【另一个技巧,指针的指针】
    Swap Nodes in Pairs
    Merge Two Sorted Lists
    Remove Duplicates from Sorted List
    Linked List Cycle
    Dungeon Game
  • 原文地址:https://www.cnblogs.com/ggzone/p/5170964.html
Copyright © 2011-2022 走看看