zoukankan      html  css  js  c++  java
  • hadoop安装

    1、tar -zxvf hadoop-2.8.0.tar.gz

    [root@localhost java-1.8.0]# useradd hadoop
    [root@localhost java-1.8.0]# passwd hadoop

    [root@localhost local]# chown hadoop:hadoop hadoop-2.8.0 -R

    [root@localhost hadoop-2.8.0]# mkdir tmp

    [hadoop@localhost hadoop-2.8.0]$ vim etc/hadoop/hadoop-env.sh

    export JAVA_HOME=/usr   //由于rpm装好 java已经分布在/usr/bin下面

    vim etc/hadoop/core-site.xml

    <configuration>
        <property>
            <name>fs.defaultFS</name>
            <value>hdfs://hadoop.master</value>
        </property>
            <property>
            <name>fs.default.name</name>
            <value>hdfs://hadoop.master:8020</value>
        </property>
    </configuration>
    

    [hadoop@localhost hadoop-2.8.0]$ vim etc/hadoop/hdfs-site.xml

    <configuration>
        <property>
            <name>dfs.replication</name>
            <value>1</value>
        </property>
    </configuration>

    ./bin/hadoop  namenode -format

     ./sbin/start-dfs.sh

     ./sbin/start-yarn.sh

    网页浏览

    http://127.0.0.1:50070/dfshealth.html#tab-overview

    如果网页打不开,说明执行不成功,可能哪里配置错误了,start-dfs.sh 的时候看下输出log,进行排查错误

    jps检查datanode 是否启动

    [root@hadoop hadoop-2.8.0]# yum list | grep jdk-devel

    [root@hadoop hadoop-2.8.0]# yum install java-1.8.0-openjdk-devel.x86_64

    想在宿主机浏览器访问hadoop web页面,开启某个端口

    [root@hadoop hadoop]# service iptables start
    Redirecting to /bin/systemctl start iptables.service
    Failed to start iptables.service: Unit not found.
    [root@hadoop hadoop]# firewall-cmd --zone=public --add-port=50070/tcp --permanent 
    success
    [root@hadoop hadoop]#   service iptables restart
    Redirecting to /bin/systemctl restart iptables.service
    Failed to restart iptables.service: Unit not found.
    [root@hadoop hadoop]#  
    [root@hadoop hadoop]# firewall-cmd --reload
    success

    在CentOS 7或RHEL 7或Fedora中防火墙由firewalld来管理,

    如果要添加范围例外端口 如 1000-2000

    语法命令如下:启用区域端口和协议组合

    firewall-cmd [--zone=] --add-port=[-]/ [--timeout=]

    此举将启用端口和协议的组合。端口可以是一个单独的端口 或者是一个端口范围 - 。协议可以是 tcp 或 udp。

    实际命令如下:

    添加

    firewall-cmd --zone=public --add-port=80/tcp --permanent (--permanent永久生效,没有此参数重启后失效)

    firewall-cmd --zone=public --add-port=1000-2000/tcp --permanent

    重新载入

    firewall-cmd --reload

    查看

    firewall-cmd --zone= public --query-port=80/tcp

    删除

    firewall-cmd --zone= public --remove-port=80/tcp --permanent



    重大bug,datanode服务连接不到namenode,查看namenode 8030端口是否有在监听
    显示
    127.0.0.1:8020 说明只能本台服务器本机请求
     
    需要修改外部服务器可以请求这个接口
    <property>
      <name>dfs.namenode.servicerpc-bind-host</name>
      <value>0.0.0.0</value>
      <description>
        The actual address the service RPC server will bind to. If this optional address is
        set, it overrides only the hostname portion of dfs.namenode.servicerpc-address.
        It can also be specified per name node or name service for HA/Federation.
        This is useful for making the name node listen on all interfaces by
        setting it to 0.0.0.0.
      </description>
    </property>
    
    <property>
      <name>dfs.namenode.http-bind-host</name>
      <value>0.0.0.0</value>
      <description>
        The actual adress the HTTP server will bind to. If this optional address
        is set, it overrides only the hostname portion of dfs.namenode.http-address.
        It can also be specified per name node or name service for HA/Federation.
        This is useful for making the name node HTTP server listen on all
        interfaces by setting it to 0.0.0.0.
      </description>
    </property>
    
    <property>
      <name>dfs.namenode.https-bind-host</name>
      <value>0.0.0.0</value>
      <description>
        The actual adress the HTTPS server will bind to. If this optional address
        is set, it overrides only the hostname portion of dfs.namenode.https-address.
        It can also be specified per name node or name service for HA/Federation.
        This is useful for making the name node HTTPS server listen on all
        interfaces by setting it to 0.0.0.0.
      </description>
    </property>


    Datanode denied communication with namenode

      

    dfs-site.xml - Need to add following line.

    <property> <name>dfs.namenode.datanode.registration.ip-hostname-check</name> <value>false</value></property>

  • 相关阅读:
    623. Add One Row to Tree 将一行添加到树中
    771. Jewels and Stones 珠宝和石头
    216. Combination Sum III 组合总数三
    384. Shuffle an Array 随机播放一个数组
    382. Linked List Random Node 链接列表随机节点
    向github项目push代码后,Jenkins实现其自动构建
    centos下安装Jenkins
    python提取批量文件内的指定内容
    批处理实现:批量为文件添加注释
    python抓取每期双色球中奖号码,用于分析
  • 原文地址:https://www.cnblogs.com/agang-php/p/10707272.html
Copyright © 2011-2022 走看看