zoukankan      html  css  js  c++  java
  • Hadoop 管理工具HUE配置-Yarn Resource Manager HA配置

    安装HUE之后,需要配置很多东西才能将这个系统的功能发挥出来,因为Yarn是配置的HA模式,所以在配置HUE的时候,会有些不用,下面一段文字是官网拿来的

      # Configuration for YARN (MR2)
      # ------------------------------------------------------------------------
      [[yarn_clusters]]
     
        [[[default]]]
     
          # Whether to submit jobs to this cluster
          submit_to=True
     
          # Name used when submitting jobs
          logical_name=ha-rm
     
          # URL of the ResourceManager API
          resourcemanager_api_url=http://gethue-1.com:8088
     
          # URL of the ProxyServer API
          proxy_api_url=http://gethue-1.com:8088
     
          # URL of the HistoryServer API
          history_server_api_url=http://gethue-1.com:19888
     
        [[[ha]]]
          # Enter the host on which you are running the failover Resource Manager
          resourcemanager_api_url=http://gethue-2.com:8088
          logical_name=ha-rm
          submit_to=True

    这里需要说明一下,[[[default]]] 和 [[ha]]中各配置一个RM,logical_name名字就是yarn-site.xml中配置的

    <property> 
            <name>yarn.resourcemanager.ha.rm-ids</name>  
            <value>rm1,rm2</value> 
    </property>  
    URL of the ResourceManager API 这里配置资源管理的地址和端口,对应yarn-site.xml中的
    <property> 
            <name>yarn.resourcemanager.webapp.address.rm1</name>  
            <value>1423-0001:23188</value> 
      </property>  
      <property> 
            <name>yarn.resourcemanager.webapp.address.rm2</name>  
            <value>1423-0002:23188</value> 
      </property>
    URL of the HistoryServer API 这里配置历史记录资源管理的地址和端口,对应mapred-site.xml中的
    <property>
                  <name>mapreduce.jobhistory.webapp.address</name>
                  <value>1423-0003:19888</value>
    </property>

    我的配置文件如下:

      [[yarn_clusters]]
    
        [[[default]]]
          # Enter the host on which you are running the ResourceManager
          ## resourcemanager_host=localhost
    
          # The port where the ResourceManager IPC listens on
          ## resourcemanager_port=8032
    
          # Whether to submit jobs to this cluster
          submit_to=True
    
          # Resource Manager logical name (required for HA)
          logical_name=rm1
    
          # Change this if your YARN cluster is Kerberos-secured
          ## security_enabled=false
    
          # URL of the ResourceManager API
          resourcemanager_api_url=http://1423-0001:23188
    
          # URL of the HistoryServer API
          history_server_api_url=http://1423-0003:19888
    
          # In secure mode (HTTPS), if SSL certificates from YARN Rest APIs
          # have to be verified against certificate authority
          ## ssl_cert_ca_verify=True
    
        # HA support by specifying multiple clusters
        # e.g.
    
        [[[ha]]]
          # Resource Manager logical name (required for HA)
          logical_name=rm2
          resourcemanager_api_url=http://1423-0002:23188
          history_server_api_url=http://1423-0003:19888
          submit_to=True

    参考:

    http://gethue.com/hadoop-tutorial-yarn-resource-manager-high-availability-ha-in-mr2/

    http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_ig_hue_config.html

    http://cloudera.github.io/hue/docs-3.8.0/manual.html#_hadoop_configuration

  • 相关阅读:
    printf里的=、++
    线程也疯狂-----异步编程
    自己搭建node服务器环境(请求静态资源、get请求、post请求)
    React学习
    2020.10-2021-01总结
    接圈的作用和缺点
    CWnd,HWND; CDC,HDC
    Python通过requests模块处理form-data请求格式
    element-ui resetFields 无效的问题
    用python 将数字每三组分割
  • 原文地址:https://www.cnblogs.com/liuchangchun/p/4657520.html
Copyright © 2011-2022 走看看