zoukankan      html  css  js  c++  java
  • yum cdh4

    cdh4 install for Centos6
    那个最美的年代,最好的时光,一路梦想,一路流泪,流的不会是懦弱的泪,而是对奋斗的寄于;
    1,repo配置
    >>>/etc/yum.repo/wget http://archive.cloudera.com/cdh4/redhat/6/x86_64/cdh/cloudera-cdh4.repo
    或是wget archive.cloudera.com/cdh4/one-click-install/redhat/6/x86_64/cloudera-cdh-4-0.x86_64.rpm
    yum --nogpgcheck localinstall cloudera-cdh-4.x86_64.rpm

    2,添加Repository key for Centos6
    rpm --import http://archive.cloud.com/cdh4/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloudera

    3,yum 安装cdh相关daemons在相关master,slaves上命令,此时未考虑HA;
    默认MRv1,MRv2(YARN)未考虑

    where to install          install commands

    JobTracker:

    yum -y install hadoop-0.20-mapreduce-jobtracker;

    NameNode:

    yum -y install hadoop-hdfs-namenode

    SecondaryNameNode:

    yum -y install hadoop-hdfs-secondarynamenode

    DataNode,TaskTracker:

    yum -y install hadoop-0.20-mapreduce-tasktracker hadoop-hdfs-datanode

    Client hosts:

    yum -y install hadoop-client

    MRv1版本yum部署

    yum.repos.d]# yum -y install hadoop-0.20-mapreduce-jobtracker hadoop-hdfs-namenode hadoop-hdfs-secondarynamenode hadoop-0.20-mapreduce-tasktracker hadoop-hdfs-datanode hadoop-client

    MRv2 即 YARN:

    Resource Manguager(analogous MRv1 JobTracker):
    yum -y install hadoop-yarn-resourcemanager
    yum -y install hadoop-hdfs-namenode
    yum -y install yarn-nodemanager hadoop-hdfs-datanode hadoop-mapreduce
    yum -y install hadoop-mapreduce-historyserver hadoop-yarn-proxyserver


    MRv2(YARN)与MRv1 yum Install区别
    Resource Manager host
        yum -y install hadoop-yarn-resourcemanager
    NameNode host
        yum -y install hadoop-hdfs-namenode
    SecondaryName host
        yum -y install hadoop-hdfs-secondarynode
    All cluster host except Resource Manager
        yum -y install hadoop-yarn-nodemanager hadoop-hdfs-datanode hadoop-mapreduce

    one host in the cluster
        yum -y install hadoop-mapreduce-historyserver hadoop-yarn-proxyserver

    可选项:

    1,LZO(压缩算法,具体见google)

    wget http://archive.cloudera.com/gplextras/redhat/6/x86_64/gplextras/cloudera-gplextras4.repo 到/etc/yum.repo/

       yum -y install hadoop-lzo-cdh4

  • 相关阅读:
    TensorFlow进行简单的图像处理
    Python Numpy
    Python描述性统计
    对文件和文件夹操作的简单函数
    Tensorflow最简单的图像搭建识别系统
    Python大数据处理模块Pandas
    redis 小结
    datatable 和list 互转
    List集合去重的一种方法
    C#中TransactionScope的使用方法和原理
  • 原文地址:https://www.cnblogs.com/ruiy/p/yum_cdh4.html
Copyright © 2011-2022 走看看