zoukankan      html  css  js  c++  java
  • Resourcemanager的高可用

    Resourcemanager的高可用:

    1. 架构




    2. 配置yarn-site.xml
      <configuration>
      <property>
      <!-- 1.开始yarn的高可用-->
      <name>yarn.resourcemanager.ha.enabled</name>
      <value>true</value>
      </property>
      <property>
      <!-- 2.指定RM的cluster id-->
      <name>yarn.resourcemanager.cluster-id</name>
      <value>yarn-ha</value>
      </property>
      <property>
      <!-- 3.指定开启高可用的RM别名-->
      <name>yarn.resourcemanager.ha.rm-ids</name>
      <value>krm1,krm2</value>
      </property>
      <property>
      <!-- 4.分别指定RM的地址-->
      <name>yarn.resourcemanager.hostname.krm1</name>
      <value>kmaster1</value>
      </property>
      <property>
      <name>yarn.resourcemanager.hostname.krm2</name>
      <value>kmaster2</value>
      </property>
      <property>
      <name>yarn.resourcemanager.recovery.enabled</name>
      <value>true</value>
      </property>
      <property>
      <name>yarn.resourcemanager.store.class</name>
      <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value>
      </property>
      <property>
      <!-- 5.指定zookeeper的地址-->
      <name>yarn.resourcemanager.zk-address</name>
      <value>kslave5:2181,kslave6:2181,kslave7:2181</value>
      </property>
      <property>
      <!-- 6.为 mapreduce 分配 yarn 服务-->
      <name>yarn.nodemanager.aux-services</name>
      <value>mapreduce_shuffle</value>
      </property>
      <property>
      <!-- 7.nodemanager 本机计算任务的临时文件-->
      <name>yarn.nodemanager.local-dirs</name>
      <value>file:///home/hadoop/disk1/nodemgr/local,file:///home/hadoop/disk2/nodemgr/local</value>
      </property>
      <property>
      <!-- 8.nodemanager 日志输出-->
      <name>yarn.nodemanager.log-dirs</name>
      <value>file:///home/hadoop/disk1/log/hadoop-yarn/containers</value>
      </property>
      <property>
      <!-- 9.远程任务的输出-->
      <name>yarn.nodemanager.remote-app-log-dir</name>
      <value>file:///home/hadoop/disk1/log/hadoop-yarn/apps</value>
      </property>
      <property>
      <!-- 10.日志汇集-->
      <name>yarn.log-aggregation-enable</name>
      <value>true</value>
      </property>
      </configuration>
    3. 然后两台机器分别启动resourcemanager即可

    4. 检查

      yarn rmadmin -getServiceState krm1
       active
      yarn rmadmin -getServiceState krm2
       standby

       高可用RM配置成功

    5. 学习博客
  • 相关阅读:
    maven工程的目录结构
    集合的区别
    名词解析
    1.(字符串)-判断字符串是否是子集字符串
    1.(字符串)-判断两字符串是否相等
    python max函数技巧
    1.(字符串)-子字符串位置查找
    numpy线性代数np.linalg
    Python图像库PIL 使用
    pyhthon-chr
  • 原文地址:https://www.cnblogs.com/Try-kevin/p/7880134.html
Copyright © 2011-2022 走看看