zoukankan      html  css  js  c++  java
  • docker搭建hadoop2.8.5

    1.在宿主机上下载好安装包:hadoop-2.8.5.tar.gz  ; jdk1.8.0_131.tar.gz共两个包

    2.在Docker仓库中拉去镜像: 

                   docker pull centos

    3.创建容器: 

                  docker run -i -t -d --name centos_hdp centos:centos

    4.从宿主机拷贝hadoop-2.8.5.tar.gz  、 jdk1.8.0_131.tar.gz两个包到容器中:

                 docker cp jdk-8u131-linux-x64.tar.gz  centos_hdp:/usr/local

                 docker cp hadoop-2.8.5.tar.gz centos_hdp:/opt/

    5.进入后台运行的容器中:

               docker exec -it centos_hdp bash

               tar -zxvf /usr/local/jdk1.8.0_131.tar.gz -C /usr/local/

               tar -zxvf /opt/hadoop-2.8.5.tar.gz -C /opt/

               vim /etc/proflie

             source /etc/profile

                     vim  /opt/hadoop-2.8.5/etc/hadoop/core-site.xml

      

                  vim  /opt/hadoop-2.8.5/etc/hadoop/hdfs-site.xml

                vim /opt/hadoop-2.8.5/etc/hadoop/mapred-site.xml

             vim /opt/hadoop-2.8.5/etc/hadoop/yarn-site.xml

              vim /opt/hadoop-2.8.5/etc/hadoop/hadoop-env.sh

              cat /opt/hadoop-2.8.5/etc/hadoop/slave

    6.退出容器,把当前容器打包成镜像:

                    docker commit -a "mayunzhen" -m "hadoop base images" centos_hdp centos_hdp:2.8.5

    7.导出镜像成jar包

                   docker save centos_hdp:2.8.5 -o centos_hdp.jar 

    8.centos_hdp.jar镜像包传到其他宿主机上

                   scp centos_hdp.jar root@192.168.130.166:/

                   scp centos_hdp.jar root@192.168.130.167:/

                   scp centos_hdp.jar root@192.168.130.168:/

    9.分别在三台(192.168.130.166,192.168.130.167,192.168.130.168)宿主机上执行导入镜像

                  docker load -i centos_hdp.jar

    10.在每个宿主机上,配置docker weave ,让不同宿主机上的容器能通信

             详情见链接:https://www.cnblogs.com/kevingrace/p/6859173.html

    11.运行hadoop容器命令:

           (192.168.130.166主机上)docker run  -itd  -h hadoop-master --name hadoop-master  --net=hadoop -v /etc/localtime:/etc/localtime:ro  -p 50070:50070 -p 8088:8088 -p 9000:9000 iammayunzhen/hadoop:2.8.52

                                                        weave attach 192.168.1.11/24 hadoop-master

           (192.168.130.167主机上)docker run  -itd  -h hadoop-slave1 --name hadoop-slave1 --net=hadoop -v /etc/localtime:/etc/localtime:ro  iammayunzhen/hadoop:2.8.52

                                                        weave attach 192.168.1.12/24 hadoop-slave1

           (192.168.130.168主机上)docker run  -itd  -h hadoop-slave2 --name hadoop-slave2 --net=hadoop  -v /etc/localtime:/etc/localtime:ro  iammayunzhen/hadoop:2.8.52

                                                        weave attach 192.168.1.13/24 hadoop-slave2

            测试三个容器IP(192.168.1.1,192.168.1.2,192.168.1.3)互相能通信:

    12.分别在三个容器(192.168.1.1,192.168.1.2,192.168.1.3)上配置/etc/hosts

     

    13.分别在三个容器(192.168.1.1,192.168.1.2,192.168.1.3)上配置无密码登录

            详情:https://www.cnblogs.com/shuochen/p/10441455.html

    14.master容器中执行启动hadoop

             source /etc/profile

            start-all.sh

    15.查看集群状态:

     

     

     

     

     

    制作好的docker hadoop:2.8.5镜像已经上传到https://hub.docker.com/repository/docker/iammayunzhen/centos_hdp(或者可 docker pull  iammayunzhen/centos_hdp:2.8.5进行下载),有需要,可供大家下载,参考。

    Refereces:

    docker hadoop镜像制作:https://www.jianshu.com/p/bf76dfedef2f

    docker容器时间和宿主机同步:https://www.cnblogs.com/kevingrace/p/5570597.html 

    多宿主机容器之间通信:https://www.cnblogs.com/kevingrace/p/6859173.html

    docker容器中IP无法ssh登录:http://blog.chinaunix.net/uid-26168435-id-5732463.html

    安装netstat命令:yum install net-tools

    查看端口情况:netstat -tulnp |grep 22

    安装passwd命令:yum install -y passwd

    多容器IP无密码登录:https://www.cnblogs.com/shuochen/p/10441455.html 

     yum安装ssh客户端 : yum -y install openssh-clients     https://www.cnblogs.com/nulige/articles/9324564.html

  • 相关阅读:
    java 常用
    面试题目总结
    前端自动化构建工具gulp记录
    js面向对象学习笔记
    sass,compass学习笔记总结
    JS核心知识点:DOMBOMEVENT
    boost atomic
    boost thread
    boost function bind ref
    boost phoenix
  • 原文地址:https://www.cnblogs.com/yunger/p/12061173.html
Copyright © 2011-2022 走看看