zoukankan      html  css  js  c++  java
  • kudu安装部署

    安装部署节点规划

    节点

    kudu-master

    kudu-tserver

    node01

    node02

    node03

    配置本地Yum的Repository

    下载kudu安装yum源。并配置安装yum源

    解压yum源文件到指定目录

    [root@hadoop03 soft]# tar zxvf cdh5.14.0-centos6.tar.gz -C /usr/local/

    镜像源是centos当中下载相关软件的地址,我们可以通过制作我们自己的镜像源指定我们去哪里下载kudu的rpm包,这里我们使用httpd这个软件来作为服务端,启动httpd的服务来作为我们镜像源的下载地址 这里我们选用第三台机器作为镜像源的服务端node3机器上执行以下命令:

    [root@hadoop03 soft]# yum install -y httpd

    [root@hadoop03 soft]# service httpd start

    [root@hadoop03 soft]# cd /etc/yum.repos.d

    [root@hadoop03 yum.repos.d]# vim localimp.repo

    [localimp]

    name=localimp

    baseurl=http://node03/cdh5.14.0

    gpgcheck=0

    enabled=1

      

    将node3上制作好的localimp配置文件发放到所有需要kudu的节点上去

    [root@hadoop03 yum.repos.d]# scp -r localimp.repo node01:$PWD

    [root@hadoop03 yum.repos.d]# scp -r localimp.repo node02:$PWD

    创建apache httpd的读取连接

    [root@hadoop03 ~]# ln -s /usr/local/cdh/5.14.0 /var/www/html/cdh5.14.0

    页面访问本地yum源,出现这个界面表示本地yum源制作成功 访问http://node03/cdh5.14.0

    安装kudu

    使用yum命令,在不同的服务器下载对应的服务

    [root@hadoop01 ~]# yum install -y kudu kudu-master kudu-tserver kudu-client0 kudu-client-devel

    [root@hadoop02 ~]# yum install -y kudu kudu-master kudu-tserver kudu-client0 kudu-client-devel

    [root@hadoop03 ~]# yum install -y kudu kudu-master kudu-tserver kudu-client0 kudu-client-devel

    命令说明:

    yum install kudu # Kudu的基本包

    yum install kudu-master # KuduMaster

    yum install kudu-tserver # KuduTserver

    yum install kudu-client0 #Kudu C ++客户端共享库

    yum install kudu-client-devel # Kudu C ++客户端共享库 SDK

      

    配置master和tserver节点

    [root@hadoop01 kudu]# cd /etc/kudu/conf/

      

    修改master.gflagfile

    [root@hadoop01 conf]# vim master.gflagfile

    # cat /etc/kudu/conf/master.gflagfile

    # Do not modify these two lines. If you wish to change these variables,

    # modify them in /etc/default/kudu-master.

    --fromenv=rpc_bind_addresses

    --fromenv=log_dir

    --fs_wal_dir=/export/servers/kudu/master

    --fs_data_dirs=/export/servers/kudu/master

    --master_addresses=node01:7051,node02:7051,node03:7051

      

    修改tserver.gflagfile

    [root@hadoop01 conf]# vim tserver.gflagfile

    # Do not modify these two lines. If you wish to change these variables,

    # modify them in /etc/default/kudu-tserver.

    --fromenv=rpc_bind_addresses

    --fromenv=log_dir

    --fs_wal_dir=/export/servers/kudu/tserver

    --fs_data_dirs=/export/servers/kudu/tserver

    --tserver_master_addrs=node01:7051,node02:7051,node03:7051

      

    修改 /etc/default/kudu-master

    [root@hadoop01 conf]# vim /etc/default/kudu-master

    export FLAGS_log_dir=/var/log/kudu

    #每台机器的master地址要与主机名一致,这里是在node01

    export FLAGS_rpc_bind_addresses=node01:7051

      

    修改 /etc/default/kudu-tserver

    [root@hadoop01 conf]# vim /etc/default/kudu-tserver

    export FLAGS_log_dir=/var/log/kudu

    #每台机器的tserver地址要与主机名一致,这里是在node01

    export FLAGS_rpc_bind_addresses=node01:7050

      

    修改完成后的上述配置文件分发到其他服务器上

    [root@hadoop01 default]# cd /etc/kudu/conf

    [root@hadoop01 conf]# scp -r master.gflagfile tserver.gflagfile node02:$PWD

    [root@hadoop01 conf]# scp -r master.gflagfile tserver.gflagfile node03:$PWD

      

    [root@hadoop01 conf]# cd /etc/default/

    [root@hadoop01 default]# scp -r kudu-master kudu-tserver node02:$PWD

    [root@hadoop01 default]# scp -r kudu-master kudu-tserver node03:$PWD

      

    修改node02服务器上的配置文件

    [root@hadoop02 conf]# cd /etc/default/

    [root@hadoop02 default]# vim kudu-master

    export FLAGS_log_dir=/var/log/kudu

    export FLAGS_rpc_bind_addresses=node02:7051

      

    [root@hadoop02 default]# vim kudu-tserver

    export FLAGS_log_dir=/var/log/kudu

    export FLAGS_rpc_bind_addresses=node02:7050

      

    修改node03服务其上的配置文件

    [root@hadoop03 ~]# cd /etc/default/

    [root@hadoop03 default]# vim kudu-master

    export FLAGS_log_dir=/var/log/kudu

    export FLAGS_rpc_bind_addresses=node03:7051

      

    [root@hadoop03 default]# vim kudu-tserver

    export FLAGS_log_dir=/var/log/kudu

    export FLAGS_rpc_bind_addresses=node03:7050

      

    在三台服务器上分别创建配置存储数据目录并修改权限

    [root@hadoop01 default]# mkdir -p /var/lib/kudu

    [root@hadoop01 default]# chmod –R kudu:kudu /var/lib/kudu

      

    [root@hadoop02 default]# mkdir -p /var/lib/kudu

    [root@hadoop02 default]# chmod –R kudu:kudu /var/lib/kudu

      

    [root@hadoop03 default]# mkdir -p /var/lib/kudu

    [root@hadoop03 default]# chmod –R kudu:kudu /var/lib/kudu

      

      

    注意

    kudu默认用户就是KUDU,所以需要将/var/lib/kudu权限修改成kudu

    [root@hadoop01 conf]# chown -R kudu:kudu /var/lib/kudu

      

    (如果使用的是普通的用户,那么最好配置sudo权限)/etc/sudoers文件中添加:

      

    kudu集群启动时需要ntp服务进行时间同步,在kudu启动前需要在三台服务器上安装ntp服务进行时间同步

    [root@hadoop01 ~]# yum -y install ntp #安装ntp服务

    [root@hadoop01 ~]# service ntpd start #启动ntp服务

    [root@hadoop01 ~]# chkconfig ntpd on #设置开机启动

      

    [root@hadoop02 ~]# yum -y install ntp #安装ntp服务

    [root@hadoop02 ~]# service ntpd start #启动ntp服务

    [root@hadoop02 ~]# chkconfig ntpd on #设置开机启动

      

    [root@hadoop03 ~]# yum -y install ntp #安装ntp服务

    [root@hadoop03 ~]# service ntpd start #启动ntp服务

    [root@hadoop03 ~]# chkconfig ntpd on #设置开机启动

      

    启动kudu集群

    [root@hadoop01 ~]# service kudu-master start

    [root@hadoop01 ~]# service kudu-tserver start

      

    [root@hadoop02 ~]# service kudu-master start

    [root@hadoop02 ~]# service kudu-tserver start

      

    [root@hadoop03 ~]# service kudu-master start

    [root@hadoop03 ~]# service kudu-tserver start

      

      

    启动集群后访问地址http://node01:8051/显示下图所示

      

      

    关闭kudu集群

    [root@hadoop01 ~]# service kudu-master stop

    [root@hadoop01 ~]# service kudu-tserver stop

      

    [root@hadoop02 ~]# service kudu-master stop

    [root@hadoop02 ~]# service kudu-tserver stop

      

    [root@hadoop03 ~]# service kudu-master stop

    [root@hadoop03 ~]# service kudu-tserver stop

      

      

    部署常见问题

    启动kudu的时候报错

    Failed to start Kudu Master Server. Return value: 1 [FAILED]

    去日志文件中查看:

    错误:F0810 09:04:08.354552 4866 master_main.cc:68] Check failed: _s.ok() Bad status:Service unavailable: Cannot initialize clock: Error reading clock. Clock considered

    Unsynchronized

      

    解决:

    第一步:首先检查是否有安装ntp:如果没有安装则使用以下命令安装:

    yum -y install ntp

    第二步:设置随机启动:

    service ntpd start

    chkconfig ntpd on

      

    启动过程中报错

    F0810 21:31:12.620932 20143 master_main.cc:71] Check failed: _s.ok() Bad status:

    Invalid argument: Unable to initialize catalog manager: Failed to initialize sys tables

    async: on-disk master list

    解决:

    (1):停掉master和tserver

    (2):删除掉之前所有的/export/servers/kudu/master/*和/export/servers/kudu/tserver/*

      

    启动过程中报错

      

    F0913 15:12:00.628237 20859 master_main.cc:74] Check failed: _s.ok() Bad status: IO

    error: Could not create new FS layout: unable to create file system roots: unable to

    write instance metadata: Call to mkstemp() failed on name template

    /export/servers/kudu/master/instance.kudutmp.XXXXXX: Permission denied (error 13)

      

    这是因为kudu默认使用kudu权限进行执行,可能遇到文件夹的权限不一致情况,更改文件夹权限即可

      

      

     

  • 相关阅读:
    PAT (Advanced Level) Practice 1055 The World's Richest (25 分) (结构体排序)
    PAT (Advanced Level) Practice 1036 Boys vs Girls (25 分)
    PAT (Advanced Level) Practice 1028 List Sorting (25 分) (自定义排序)
    PAT (Advanced Level) Practice 1035 Password (20 分)
    PAT (Advanced Level) Practice 1019 General Palindromic Number (20 分) (进制转换,回文数)
    PAT (Advanced Level) Practice 1120 Friend Numbers (20 分) (set)
    从零开始吧
    Python GUI编程(TKinter)(简易计算器)
    PAT 基础编程题目集 6-7 统计某类完全平方数 (20 分)
    PAT (Advanced Level) Practice 1152 Google Recruitment (20 分)
  • 原文地址:https://www.cnblogs.com/starzy/p/10553936.html
Copyright © 2011-2022 走看看