zoukankan      html  css  js  c++  java
  • hadoop 2.8.5安装步骤

    1、创建hadoop用户,作为haoop的运行用户

    2、配置JAVA_HOME环境变量,修改/etc/profile
    export JAVA_HOME=/usr/java/jdk1.8.0_51
    export PATH=${JAVA_HOME}/bin:$PATH

    3、配置hadoop用户的PATH环境变量,修改.bash_profile
    $HOME/hadoop/bin:$HOME/hadoop/bin

    4、配置免密登录,需要配置master免密登录slave和master自身
    a、namenode上执行 ssh-keygen -t rsa 生成key
    b、cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
    c、将生成的公钥复制到机器namenode上的~/.ssh/authorized_keys中
    ssh-copy-id -i ~/.ssh/id_rsa.pub hadoop@192.168.1.100
    d、测试能登录到自身

    5、配置hadoop-env.sh及yarn-env.sh中的javahome
    export JAVA_HOME=/usr/java/jdk1.8.0_51
    export HADOOP_CONF_DIR=/home/hadoop/hadoop/etc/hadoop
    6、修改slave 文件
    7、创建目录
    mkdir tmp
    mkdir hdfs
    mkdir hdfs/name
    mkdir hdfs/data
    8、修改core-site.xml配置文件
    <configuration>
    <configuration>
    <property>
    <name>hadoop.tmp.dir</name>
    <value>/home/hadoop/tmp</value>
    <final>true</final>
    <description>A base for other temporary directories.</description>
    </property>
    <property>
    <name>fs.default.name</name>
    <value>hdfs://rhel01:9000</value>
    <final>true</final>
    </property>
    </configuration>

    9、修改hdfs-site.xml文件
    <configuration>
    <property>
    <name>dfs.http.address</name>
    <value>rhel01:50070</value>
    </property>
    <property>
    <name>dfs.replication</name>
    <value>1</value>
    </property>
    <property>
    <name>dfs.namenode.secondary.http-address</name>
    <value>rhel01:50090</value>
    </property>
    </configuration>

    10、修改mapred-site.xml
    <configuration>
    <property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
    </property>
    <property>
    <name>mapreduce.jobhistory.address</name>
    <value>rhel01:10020</value>
    </property>
    <property>
    <name>mapreduce.jobhistory.webapp.address</name>
    <value>rhel01:19888</value>
    </property>
    </configuration>

    11、修改yarn-site.xml文件
    <configuration>
    <property>
    <name>yarn.resourcemanager.hostname</name>
    <value>rhel01</value>
    </property>
    <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
    </property>
    <property>
    <name>yarn.resourcemanager.address</name>
    <value>rhel01:8032</value>
    </property>
    <property>
    <name>yarn.resourcemanager.scheduler.address</name>
    <value>rhel01:8030</value>
    </property>
    <property>
    <name>yarn.resourcemanager.resource-tracker.address</name>
    <value>rhel01:8031</value>
    </property>
    <property>
    <name>yarn.resourcemanager.admin.address</name>
    <value>rhel01:8033</value>
    </property>
    <property>
    <name>yarn.resourcemanager.webapp.address</name>
    <value>rhel01:8088</value>
    </property>
    </configuration>

    12、格式化namenode
    hdfs namenode -format

    13、启动dfs
    start-dfs.sh

    14、启动yarn
    start-yarn.sh

  • 相关阅读:
    哈夫曼编码拓展题
    TrieTree
    并查集
    hash一致性
    布隆过滤器
    如何计算完全二叉树的结点数?
    如何翻转单链表和双向链表
    如何判断是搜索二叉树与完全二叉树
    文本分类之特征描述vsm和bow
    文本分类概述
  • 原文地址:https://www.cnblogs.com/timlong/p/9802045.html
Copyright © 2011-2022 走看看