zoukankan      html  css  js  c++  java
  • 基于Ambari Server部署HDP集群实战案例

              基于Ambari Server部署HDP集群实战案例

                                  作者:尹正杰

    版权声明:原创作品,谢绝转载!否则将追究法律责任。

    一.部署Ambari Server端

      博主推荐阅读:
        https://www.cnblogs.com/yinzhengjie2020/p/12219108.html

    二.时钟同步

      我们在生产环境中必须同步各个服务器时间是一致的,最好是有内网的时间服务器。
    
      配置时间同步的好处:
        可以定位集群故障发生的具体时间;
        解决HBase,Kudu等存储依赖于时间同步的问题(若时间不同步会报错);
    
      CentOS 6.x默认使用ntpd软件来同步时间的,但在CentOS 7.x版本之后就被更替为chrony来实现时间同步,因此咱们安装时间同步就是用CentOS 7.x推荐的Chrony来进行时间同步。

    1>.安装chrony

    [root@hdp101.yinzhengjie.org.cn ~]# yum -y install chrony
    Loaded plugins: fastestmirror
    Determining fastest mirrors
     * base: mirror.bit.edu.cn
     * extras: mirror.bit.edu.cn
     * updates: mirrors.huaweicloud.com
    ambari-repo                                                                                                                                                                                                                                            | 2.9 kB  00:00:00     
    base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
    extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
    mysql-connectors-community                                                                                                                                                                                                                             | 2.5 kB  00:00:00     
    mysql-tools-community                                                                                                                                                                                                                                  | 2.5 kB  00:00:00     
    mysql80-community                                                                                                                                                                                                                                      | 2.5 kB  00:00:00     
    updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
    (1/2): extras/7/x86_64/primary_db                                                                                                                                                                                                                      | 159 kB  00:00:00     
    (2/2): updates/7/x86_64/primary_db                                                                                                                                                                                                                     | 5.9 MB  00:00:00     
    Resolving Dependencies
    --> Running transaction check
    ---> Package chrony.x86_64 0:3.4-1.el7 will be installed
    --> Processing Dependency: libseccomp.so.2()(64bit) for package: chrony-3.4-1.el7.x86_64
    --> Running transaction check
    ---> Package libseccomp.x86_64 0:2.3.1-3.el7 will be installed
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    ==============================================================================================================================================================================================================================================================================
     Package                                                             Arch                                                            Version                                                              Repository                                                     Size
    ==============================================================================================================================================================================================================================================================================
    Installing:
     chrony                                                              x86_64                                                          3.4-1.el7                                                            base                                                          251 k
    Installing for dependencies:
     libseccomp                                                          x86_64                                                          2.3.1-3.el7                                                          base                                                           56 k
    
    Transaction Summary
    ==============================================================================================================================================================================================================================================================================
    Install  1 Package (+1 Dependent package)
    
    Total download size: 306 k
    Installed size: 788 k
    Downloading packages:
    (1/2): libseccomp-2.3.1-3.el7.x86_64.rpm                                                                                                                                                                                                               |  56 kB  00:00:00     
    (2/2): chrony-3.4-1.el7.x86_64.rpm                                                                                                                                                                                                                     | 251 kB  00:00:00     
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Total                                                                                                                                                                                                                                         957 kB/s | 306 kB  00:00:00     
    Running transaction check
    Running transaction test
    Transaction test succeeded
    Running transaction
      Installing : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
      Installing : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 
      Verifying  : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
      Verifying  : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 
    
    Installed:
      chrony.x86_64 0:3.4-1.el7                                                                                                                                                                                                                                                   
    
    Dependency Installed:
      libseccomp.x86_64 0:2.3.1-3.el7                                                                                                                                                                                                                                             
    
    Complete!
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# yum -y install chrony

    2>.禁用本地时间服务器从公网同步时间

    [root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf 
    server 0.centos.pool.ntp.org iburst
    server 1.centos.pool.ntp.org iburst
    server 2.centos.pool.ntp.org iburst
    server 3.centos.pool.ntp.org iburst
    driftfile /var/lib/chrony/drift
    makestep 1.0 3
    rtcsync
    logdir /var/log/chrony
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# sed -r -i 's@(^server)@#1@g' /etc/chrony.conf 
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
    driftfile /var/lib/chrony/drift
    makestep 1.0 3
    rtcsync
    logdir /var/log/chrony
    [root@hdp101.yinzhengjie.org.cn ~]# 

    3>.新增时间同步服务器为本机地址

    [root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
    driftfile /var/lib/chrony/drift
    makestep 1.0 3
    rtcsync
    logdir /var/log/chrony
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# vim /etc/chrony.conf 
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
    server hdp101.yinzhengjie.org.cn iburst
    driftfile /var/lib/chrony/drift
    makestep 1.0 3
    rtcsync
    logdir /var/log/chrony
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# 

    4>.设置允许来chronyd服务器端同步时间的客户端地址

    [root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
    server hdp101.yinzhengjie.org.cn iburst
    driftfile /var/lib/chrony/drift
    makestep 1.0 3
    rtcsync
    logdir /var/log/chrony
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# grep ^#allow /etc/chrony.conf 
    #allow 192.168.0.0/16
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# sed -r -i 's@#(allow) 192.168.0.0/16@1 172.200.0.0/21@' /etc/chrony.conf 
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# vim /etc/chrony.conf 
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
    server hdp101.yinzhengjie.org.cn iburst
    driftfile /var/lib/chrony/drift
    makestep 1.0 3
    rtcsync
    allow 172.200.0.0/21          #允许客户端IP地址同步
    allow 127/8                #允许本地同步时间
    logdir /var/log/chrony
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# 

    5>.设置chronyd服务器端允许返回本地时间

    [root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
    server hdp101.yinzhengjie.org.cn iburst
    driftfile /var/lib/chrony/drift
    makestep 1.0 3
    rtcsync
    allow 172.200.0.0/21
    allow 127/8
    logdir /var/log/chrony
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# grep ^#local /etc/chrony.conf
    #local stratum 10
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# sed -r -i 's@#(local)@1@' /etc/chrony.conf 
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
    server hdp101.yinzhengjie.org.cn iburst
    driftfile /var/lib/chrony/drift
    makestep 1.0 3
    rtcsync
    allow 172.200.0.0/21
    allow 127/8
    local stratum 10           #即使自己未能通过网络时间服务器同步到时间,也允许将本地时间作为标准时间授时给其它客户端,而我们上面设置了禁用默认公网的时间服务器进行同步(换句话说,这里表示不去同步任何服务器的时间)。
    logdir /var/log/chrony
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# 

    6>.启动chronyd并设置为开机自启动

    [root@hdp101.yinzhengjie.org.cn ~]# systemctl start chronyd
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# systemctl enable chronyd
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# systemctl list-unit-files | grep chronyd
    chronyd.service                               enabled 
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# 

    7>.查看chronyd正在使用哪个时间服务器进行时间同步

    [root@hdp101.yinzhengjie.org.cn ~]# chronyc sources
    210 Number of sources = 1
    MS Name/IP address         Stratum Poll Reach LastRx Last sample               
    ===============================================================================
    ^* hdp101.yinzhengjie.org.cn    10   6   377    39  -1609ns[-4236ns] +/-   16us
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# 

    8>.hdp102.yinzhengjie.org.cn客户端同步时间

    [root@hdp102.yinzhengjie.org.cn ~]# yum -y install chrony
    Loaded plugins: fastestmirror
    Determining fastest mirrors
     * base: mirrors.tuna.tsinghua.edu.cn
     * extras: mirrors.tuna.tsinghua.edu.cn
     * updates: mirror.bit.edu.cn
    base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
    extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
    updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
    (1/2): extras/7/x86_64/primary_db                                                                                                                                                                                                                      | 159 kB  00:00:00     
    (2/2): updates/7/x86_64/primary_db                                                                                                                                                                                                                     | 5.9 MB  00:00:01     
    Resolving Dependencies
    --> Running transaction check
    ---> Package chrony.x86_64 0:3.4-1.el7 will be installed
    --> Processing Dependency: libseccomp.so.2()(64bit) for package: chrony-3.4-1.el7.x86_64
    --> Running transaction check
    ---> Package libseccomp.x86_64 0:2.3.1-3.el7 will be installed
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    ==============================================================================================================================================================================================================================================================================
     Package                                                             Arch                                                            Version                                                              Repository                                                     Size
    ==============================================================================================================================================================================================================================================================================
    Installing:
     chrony                                                              x86_64                                                          3.4-1.el7                                                            base                                                          251 k
    Installing for dependencies:
     libseccomp                                                          x86_64                                                          2.3.1-3.el7                                                          base                                                           56 k
    
    Transaction Summary
    ==============================================================================================================================================================================================================================================================================
    Install  1 Package (+1 Dependent package)
    
    Total download size: 306 k
    Installed size: 788 k
    Downloading packages:
    (1/2): chrony-3.4-1.el7.x86_64.rpm                                                                                                                                                                                                                     | 251 kB  00:00:00     
    (2/2): libseccomp-2.3.1-3.el7.x86_64.rpm                                                                                                                                                                                                               |  56 kB  00:00:00     
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Total                                                                                                                                                                                                                                         1.5 MB/s | 306 kB  00:00:00     
    Running transaction check
    Running transaction test
    Transaction test succeeded
    Running transaction
      Installing : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
      Installing : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 
      Verifying  : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
      Verifying  : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 
    
    Installed:
      chrony.x86_64 0:3.4-1.el7                                                                                                                                                                                                                                                   
    
    Dependency Installed:
      libseccomp.x86_64 0:2.3.1-3.el7                                                                                                                                                                                                                                             
    
    Complete!
    [root@hdp102.yinzhengjie.org.cn ~]# 
    [root@hdp102.yinzhengjie.org.cn ~]# yum -y install chrony
    [root@hdp102.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
    server 0.centos.pool.ntp.org iburst
    server 1.centos.pool.ntp.org iburst
    server 2.centos.pool.ntp.org iburst
    server 3.centos.pool.ntp.org iburst
    driftfile /var/lib/chrony/drift
    makestep 1.0 3
    rtcsync
    logdir /var/log/chrony
    [root@hdp102.yinzhengjie.org.cn ~]# 
    [root@hdp102.yinzhengjie.org.cn ~]# sed -r -i 's@(^server)@#1@g' /etc/chrony.conf
    [root@hdp102.yinzhengjie.org.cn ~]# 
    [root@hdp102.yinzhengjie.org.cn ~]# vim /etc/chrony.conf
    [root@hdp102.yinzhengjie.org.cn ~]# 
    [root@hdp102.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
    server hdp101.yinzhengjie.org.cn iburst
    driftfile /var/lib/chrony/drift
    makestep 1.0 3
    rtcsync
    logdir /var/log/chrony
    [root@hdp102.yinzhengjie.org.cn ~]# 
    [root@hdp102.yinzhengjie.org.cn ~]# systemctl status chronyd
    ● chronyd.service - NTP client/server
       Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled)
       Active: inactive (dead)
         Docs: man:chronyd(8)
               man:chrony.conf(5)
    [root@hdp102.yinzhengjie.org.cn ~]# 
    [root@hdp102.yinzhengjie.org.cn ~]# systemctl start chronyd
    [root@hdp102.yinzhengjie.org.cn ~]# 
    [root@hdp102.yinzhengjie.org.cn ~]# systemctl enable chronyd
    [root@hdp102.yinzhengjie.org.cn ~]# 
    [root@hdp102.yinzhengjie.org.cn ~]# systemctl list-units -t service | grep chronyd
      chronyd.service                    loaded active running NTP client/server
    [root@hdp102.yinzhengjie.org.cn ~]# 
    [root@hdp102.yinzhengjie.org.cn ~]# systemctl status chronyd
    ● chronyd.service - NTP client/server
       Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled)
       Active: active (running) since Fri 2020-01-31 23:38:04 CST; 20s ago
         Docs: man:chronyd(8)
               man:chrony.conf(5)
     Main PID: 5424 (chronyd)
       CGroup: /system.slice/chronyd.service
               └─5424 /usr/sbin/chronyd
    
    Jan 31 23:38:04 hdp102.yinzhengjie.org.cn systemd[1]: Starting NTP client/server...
    Jan 31 23:38:04 hdp102.yinzhengjie.org.cn chronyd[5424]: chronyd version 3.4 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +SECHASH +IPV6 +DEBUG)
    Jan 31 23:38:04 hdp102.yinzhengjie.org.cn systemd[1]: Started NTP client/server.
    Jan 31 23:38:09 hdp102.yinzhengjie.org.cn chronyd[5424]: Selected source 172.200.1.101
    [root@hdp102.yinzhengjie.org.cn ~]# 
    [root@hdp102.yinzhengjie.org.cn ~]# 

    9>.hdp103.yinzhengjie.org.cn客户端同步时间

    [root@hdp103.yinzhengjie.org.cn ~]# yum -y install chrony
    Loaded plugins: fastestmirror
    Determining fastest mirrors
     * base: mirrors.tuna.tsinghua.edu.cn
     * extras: mirrors.tuna.tsinghua.edu.cn
     * updates: mirror.bit.edu.cn
    base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
    extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
    updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
    (1/2): extras/7/x86_64/primary_db                                                                                                                                                                                                                      | 159 kB  00:00:00     
    (2/2): updates/7/x86_64/primary_db                                                                                                                                                                                                                     | 5.9 MB  00:00:01     
    Resolving Dependencies
    --> Running transaction check
    ---> Package chrony.x86_64 0:3.4-1.el7 will be installed
    --> Processing Dependency: libseccomp.so.2()(64bit) for package: chrony-3.4-1.el7.x86_64
    --> Running transaction check
    ---> Package libseccomp.x86_64 0:2.3.1-3.el7 will be installed
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    ==============================================================================================================================================================================================================================================================================
     Package                                                             Arch                                                            Version                                                              Repository                                                     Size
    ==============================================================================================================================================================================================================================================================================
    Installing:
     chrony                                                              x86_64                                                          3.4-1.el7                                                            base                                                          251 k
    Installing for dependencies:
     libseccomp                                                          x86_64                                                          2.3.1-3.el7                                                          base                                                           56 k
    
    Transaction Summary
    ==============================================================================================================================================================================================================================================================================
    Install  1 Package (+1 Dependent package)
    
    Total download size: 306 k
    Installed size: 788 k
    Downloading packages:
    (1/2): chrony-3.4-1.el7.x86_64.rpm                                                                                                                                                                                                                     | 251 kB  00:00:00     
    (2/2): libseccomp-2.3.1-3.el7.x86_64.rpm                                                                                                                                                                                                               |  56 kB  00:00:00     
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Total                                                                                                                                                                                                                                         1.5 MB/s | 306 kB  00:00:00     
    Running transaction check
    Running transaction test
    Transaction test succeeded
    Running transaction
      Installing : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
      Installing : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 
      Verifying  : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
      Verifying  : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 
    
    Installed:
      chrony.x86_64 0:3.4-1.el7                                                                                                                                                                                                                                                   
    
    Dependency Installed:
      libseccomp.x86_64 0:2.3.1-3.el7                                                                                                                                                                                                                                             
    
    Complete!
    [root@hdp103.yinzhengjie.org.cn ~]# 
     
    [root@hdp103.yinzhengjie.org.cn ~]# yum -y install chrony
    [root@hdp103.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
    server 0.centos.pool.ntp.org iburst
    server 1.centos.pool.ntp.org iburst
    server 2.centos.pool.ntp.org iburst
    server 3.centos.pool.ntp.org iburst
    driftfile /var/lib/chrony/drift
    makestep 1.0 3
    rtcsync
    logdir /var/log/chrony
    [root@hdp103.yinzhengjie.org.cn ~]# 
    [root@hdp103.yinzhengjie.org.cn ~]# sed -r -i 's@(^server)@#1@g' /etc/chrony.conf
    [root@hdp103.yinzhengjie.org.cn ~]# 
    [root@hdp103.yinzhengjie.org.cn ~]# vim /etc/chrony.conf 
    [root@hdp103.yinzhengjie.org.cn ~]# 
    [root@hdp103.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
    server hdp101.yinzhengjie.org.cn iburst
    driftfile /var/lib/chrony/drift
    makestep 1.0 3
    rtcsync
    logdir /var/log/chrony
    [root@hdp103.yinzhengjie.org.cn ~]# 
    [root@hdp103.yinzhengjie.org.cn ~]# 
    [root@hdp103.yinzhengjie.org.cn ~]# systemctl status chronyd
    ● chronyd.service - NTP client/server
       Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled)
       Active: inactive (dead)
         Docs: man:chronyd(8)
               man:chrony.conf(5)
    [root@hdp103.yinzhengjie.org.cn ~]# 
    [root@hdp103.yinzhengjie.org.cn ~]# systemctl start chronyd
    [root@hdp103.yinzhengjie.org.cn ~]# 
    [root@hdp103.yinzhengjie.org.cn ~]# systemctl status chronyd
    ● chronyd.service - NTP client/server
       Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled)
       Active: active (running) since Fri 2020-01-31 23:42:09 CST; 1s ago
         Docs: man:chronyd(8)
               man:chrony.conf(5)
      Process: 5457 ExecStartPost=/usr/libexec/chrony-helper update-daemon (code=exited, status=0/SUCCESS)
      Process: 5453 ExecStart=/usr/sbin/chronyd $OPTIONS (code=exited, status=0/SUCCESS)
     Main PID: 5455 (chronyd)
       CGroup: /system.slice/chronyd.service
               └─5455 /usr/sbin/chronyd
    
    Jan 31 23:42:09 hdp103.yinzhengjie.org.cn systemd[1]: Starting NTP client/server...
    Jan 31 23:42:09 hdp103.yinzhengjie.org.cn chronyd[5455]: chronyd version 3.4 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +SECHASH +IPV6 +DEBUG)
    Jan 31 23:42:09 hdp103.yinzhengjie.org.cn systemd[1]: Started NTP client/server.
    [root@hdp103.yinzhengjie.org.cn ~]# 
    [root@hdp103.yinzhengjie.org.cn ~]# 
    [root@hdp103.yinzhengjie.org.cn ~]# systemctl start chronyd
    [root@hdp103.yinzhengjie.org.cn ~]# systemctl enable chronyd
    Created symlink from /etc/systemd/system/multi-user.target.wants/chronyd.service to /usr/lib/systemd/system/chronyd.service.
    [root@hdp103.yinzhengjie.org.cn ~]# 

    10>.chronyd服务端修改时间

    [root@hdp101.yinzhengjie.org.cn ~]# date 
    Fri Jan 31 23:44:30 CST 2020
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# date -s "2020-01-31 15:45:00"
    Fri Jan 31 15:45:00 CST 2020
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# hwclock -w
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# date 
    Fri Jan 31 15:45:02 CST 2020
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# chronyc sources
    210 Number of sources = 1
    MS Name/IP address         Stratum Poll Reach LastRx Last sample               
    ===============================================================================
    ^? hdp101.yinzhengjie.org.cn     0   7   377     -     +0ns[   +0ns] +/-    0ns
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# chronyc sourcestats -v
    210 Number of sources = 1
                                 .- Number of sample points in measurement set.
                                /    .- Number of residual runs with same sign.
                               |    /    .- Length of measurement set (time).
                               |   |    /      .- Est. clock freq error (ppm).
                               |   |   |      /           .- Est. error in freq.
                               |   |   |     |           /         .- Est. offset.
                               |   |   |     |          |          |   On the -.
                               |   |   |     |          |          |   samples. 
                               |   |   |     |          |          |             |
    Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
    ==============================================================================
    hdp101.yinzhengjie.org.cn   0   0     0     +0.000   2000.000     +0ns  4000ms
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# 

    11>.chronyd客户端同步时间

    [root@hdp102.yinzhengjie.org.cn ~]# date 
    Fri Jan 31 23:56:05 CST 2020
    [root@hdp102.yinzhengjie.org.cn ~]# 
    [root@hdp102.yinzhengjie.org.cn ~]# chronyc sourcestats -v
    210 Number of sources = 1
                                 .- Number of sample points in measurement set.
                                /    .- Number of residual runs with same sign.
                               |    /    .- Length of measurement set (time).
                               |   |    /      .- Est. clock freq error (ppm).
                               |   |   |      /           .- Est. error in freq.
                               |   |   |     |           /         .- Est. offset.
                               |   |   |     |          |          |   On the -.
                               |   |   |     |          |          |   samples. 
                               |   |   |     |          |          |             |
    Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
    ==============================================================================
    hdp101.yinzhengjie.org.cn   6   4   347     -0.085      3.307  -4050ns    64us
    [root@hdp102.yinzhengjie.org.cn ~]# 
    [root@hdp102.yinzhengjie.org.cn ~]# 
    [root@hdp102.yinzhengjie.org.cn ~]# systemctl restart chronyd
    [root@hdp102.yinzhengjie.org.cn ~]# 
    [root@hdp102.yinzhengjie.org.cn ~]# chronyc sourcestats -v
    210 Number of sources = 1
                                 .- Number of sample points in measurement set.
                                /    .- Number of residual runs with same sign.
                               |    /    .- Length of measurement set (time).
                               |   |    /      .- Est. clock freq error (ppm).
                               |   |   |      /           .- Est. error in freq.
                               |   |   |     |           /         .- Est. offset.
                               |   |   |     |          |          |   On the -.
                               |   |   |     |          |          |   samples. 
                               |   |   |     |          |          |             |
    Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
    ==============================================================================
    hdp101.yinzhengjie.org.cn   3   3     4    -20.657   1349.596   -382ns  6474ns
    [root@hdp102.yinzhengjie.org.cn ~]# 
    [root@hdp102.yinzhengjie.org.cn ~]# 
    [root@hdp102.yinzhengjie.org.cn ~]# date 
    Fri Jan 31 15:57:07 CST 2020
    [root@hdp102.yinzhengjie.org.cn ~]# 

    三.部署Ambari集群

    1>.浏览器访问"http://hdp101.yinzhengjie.org.cn:8080/",使用默认的用户名(admin)和密码(admin)登录成功后启动安装向导(如下图所示,点击"LAUNCH INSTALL WIZARD") 

    2>.自定义集群名称,并点击下一步

    3>.选择HDP版本,并选择本地软件源仓库

    4>.删除不相关操作系统的仓库信息,只保留部署Ambari的操作系统的仓库信息

    5>.注册主机

    6>.确认匹配的主机名称

    7>.等待自动注册

    8>.等待集群自

    9>.集群自检完成

      如果此步骤和上图一样直接跳过下面的阅读,如果你安装其它节点存在问题,如下图所示,别慌,查看日志解决问题才是正道。

      查看报错信息,如下图所示。

      根据报错信息将hdp101.yinzhengjie.org.cn与hdp[102-103].yinzhengjie.org.cn不同的文件同步后,之前两个失败的节点最终状态变为"Success"状态。

    10>.查看集群自建结果(此步骤可省略,直接进行下一步)

    11>.自定义需要安装的服务(服务勾选少了也没多大关系,后面可以手动添加服务)

    12>.勾选Ambari Metircs(会自动安装监控已经勾选服务的状态)服务并点击"NEXT",如果弹出如下图所示的提示,点击"PROCEED ANYWAY"

    13>.根据服务器实际资源分配Master服务角色安装到具体的主机

    14>.除了NFSGateway角色(如有需要后续可以自行安装)不勾选,其它的slave服务都勾选

    15>.为Grafana Admin和Activity Explorer's Admin服务设置密码

    16>.自定义各个服务对应数据的存储路径(由于我们使用的是root用户安装,因此尽管指定的目录不存在,Ambari会通过root用户自动创建的哟~)

    17>.确认各个服务在操作系统中以哪个用户身份运行(保持默认即可,直接下一步)

     18>.自定义安装服务的配置并点击下一步

    19>.在Review步骤无需做任何配置,而是对之前几个步骤的配置有一个大概的信息概览,如果确认配置五五后点击"DEPLOY"就正式部署啦

    20>.点击"DEPLOY"会开始初始化任务 

    21>.初始化完就开始部署了  

    22>.等待服务自动安装,启动和测试

    23>.服务安装成功

    24>.安装完成后点击上图的“COMPLETE”会自动进入到"Dashboard"的监控页面

    25>.手动测试HDFS服务是否安装成功

    [root@hdp101.yinzhengjie.org.cn ~]# hdfs dfs -ls /
    Found 6 items
    drwxr-xr-x   - hdfs   hdfs            0 2020-01-31 23:37 /atsv2
    drwxr-xr-x   - hdfs   hdfs            0 2020-01-31 23:37 /hdp
    drwxr-xr-x   - mapred hdfs            0 2020-01-31 23:37 /mapred
    drwxrwxrwx   - hdfs   hdfs            0 2020-01-31 23:39 /tmp
    drwxr-xr-x   - hdfs   hdfs            0 2020-01-31 23:37 /user
    drwxrwxrwx   - mapred hadoop          0 2020-01-31 23:40 /yinzhengjie
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# hdfs dfs -df -h
    Filesystem                              Size   Used  Available  Use%
    hdfs://hdp101.yinzhengjie.org.cn:8020  4.0 T  1.2 G      4.0 T    0%
    [root@hdp101.yinzhengjie.org.cn ~]# 
    [root@hdp101.yinzhengjie.org.cn ~]# 

  • 相关阅读:
    crunch--字典生成工具
    在LINUX上查询哪个用户从哪个IP登录,登录时间,执行了什么命令?
    关于jetbrains系列产品2018.1.5以后的使用(crack)方法
    vim 加密(crypt)文本文档
    ubuntu 安装 c语言的库函数man手册
    Ubuntu Desktop 编译 ffmpeg (简略的写写)
    统计php-fpm内存占用
    ffmpeg 视频 转 gif
    一条命令将windows下多个ts文件合并为一个ts文件
    CC攻击原理及防范方法
  • 原文地址:https://www.cnblogs.com/yinzhengjie2020/p/12227503.html
Copyright © 2011-2022 走看看