zoukankan      html  css  js  c++  java
  • 配置systemctl(zookeeper、hadoop、redis)

    ==背景==

    给公司搭建了一个建议的大数据平台,组件包括zookeeper、hadoop、flink、influxdb、redis、mysql、rocketmq

    因为zookeeper、hadoop是平台的基础组件,打算配置systemctl守护进程,用来挂掉自动重启,以减少运维负担。

    ==规划==

    1、每台机器设置service文件的管理文件夹:/home/radmin/rexel-dev

    2、文件组成,为每个service生成两个文件:xxx.service及xxx.service.run。.run是.service需要执行的脚本,负责环境依赖

    3、软连接:在systemctl目录下建立软连接,连接到rexel-dev下的具体文件

    4、所有.run文件赋予777权限。

    ==准备==

    1、创建pid文件存放路径

    for i in {1..3}; do ssh vm$i mkdir -p /home/radmin/data/pids/hadoop; done

    2、创建service文件存放路径

    for i in {1..3}; do ssh vm$i mkdir -p /home/radmin/rexel-dev; done

    3、配置系统环境变量

    export HADOOP_PID_DIR=/home/radmin/data/pids/hadoop
    export YARN_PID_DIR=/home/radmin/data/pids/hadoop

    ==zookeeper==

    1、文件规划

    rexel-zookeeper.service
    rexel-zookeeper.service.run

    2、配置run文件

    文件:rexel-zookeeper.service.run

    #!/bin/bash
    source /etc/profile
    /home/radmin/zookeeper-3.5.6/bin/zkServer.sh start

    3、配置service文件

    文件:rexel-zookeeper.service

    [Unit]
    Description=Zookeeper Service
    After=network.target
    ConditionPathExists=/home/radmin/zookeeper-3.5.6/conf/zoo.cfg
    
    [Service]
    Type=forking
    PIDFile=/home/radmin/data/zk/dataDir/zookeeper_server.pid
    ExecStart=/home/radmin/rexel-dev/rexel-zookeeper.service.run
    Restart=always
    
    [Install]
    WantedBy=multi-user.target

    4、配置软连接

    命令:

    ln -s /home/radmin/rexel-dev/rexel-zookeeper.service /usr/lib/systemd/system/rexel-zookeeper.service

    5、加载生效

    命令:

    systemctl daemon-reload

    6、启动服务

    命令:

    systemctl status rexel-zookeeper.service

    【小贴士】

    任何增加、修改或删除systemctl,都需要执行systemctl daemon-reload命令,使修改生效。

    【小插曲】

    service启动失败,通过journalctl -xe命令查看日志,发现提示

    Can't open PID file /home/radmin/data/pid/zookeeper.pid (yet?) after start: No such file or directory

    发现原因是配置的pid文件路径不正确,将pid文件的路径和zoo.cfg中的dataDir路径一致,重新启动service之后,恢复正常。

    ==hadoop==

    1、文件规划

    rexel-hdfs-journalnode.service
    rexel-hdfs-journalnode.service.run
    rexel-hdfs-namenode.service
    rexel-hdfs-namenode.service.run
    rexel-hdfs-zkfc.service
    rexel-hdfs-zkfc.service.run
    rexel-hdfs-datanode.service
    rexel-hdfs-datanode.service.run
    rexel-yarn-nodemanager.service
    rexel-yarn-nodemanager.service.run
    rexel-yarn-resourcemanager.service
    rexel-yarn-resourcemanager.service.run

    2、配置run文件

    文件1:rexel-hdfs-journalnode.service.run

    #!/bin/bash
    source /etc/profile
    /home/radmin/hadoop-2.8.3/sbin/hadoop-daemon.sh start journalnode

    文件2:rexel-hdfs-namenode.service.run

    #!/bin/bash
    source /etc/profile
    /home/radmin/hadoop-2.8.3/sbin/hadoop-daemon.sh start namenode

    文件3:rexel-hdfs-zkfc.service.run

    #!/bin/bash
    source /etc/profile
    /home/radmin/hadoop-2.8.3/sbin/hadoop-daemon.sh start zkfc

    文件4:rexel-hdfs-datanode.service.run

    #!/bin/bash
    source /etc/profile
    /home/radmin/hadoop-2.8.3/sbin/hadoop-daemon.sh start datanode

    文件5:rexel-yarn-resourcemanager.service.run

    #!/bin/bash
    source /etc/profile
    /home/radmin/hadoop-2.8.3/sbin/yarn-daemon.sh start resourcemanager

    文件6:rexel-yarn-nodemanager.service.run

    #!/bin/bash
    source /etc/profile
    /home/radmin/hadoop-2.8.3/sbin/yarn-daemon.sh start nodemanager

    3、配置service文件

    文件1:rexel-hdfs-journalnode.service

    [Unit]
    Description=HDFS Journalnode Service
    After=network.target
    
    [Service]
    Type=forking
    PIDFile=/home/radmin/data/pids/hadoop/hadoop-root-journalnode.pid
    ExecStart=/home/radmin/rexel-dev/rexel-hdfs-journalnode.service.run
    Restart=always
    
    [Install]
    WantedBy=multi-user.target

    文件2:rexel-hdfs-namenode.service

    [Unit]
    Description=HDFS Namenode Service
    After=network.target
    
    [Service]
    Type=forking
    PIDFile=/home/radmin/data/pids/hadoop/hadoop-root-namenode.pid
    ExecStart=/home/radmin/rexel-dev/rexel-hdfs-namenode.service.run
    Restart=always
    
    [Install]
    WantedBy=multi-user.target

    文件3:rexel-hdfs-datanode.service

    [Unit]
    Description=HDFS Datanode Service
    After=network.target
    
    [Service]
    Type=forking
    PIDFile=/home/radmin/data/pids/hadoop/hadoop-root-datanode.pid
    ExecStart=/home/radmin/rexel-dev/rexel-hdfs-datanode.service.run
    Restart=always
    
    [Install]
    WantedBy=multi-user.target

    文件4:rexel-hdfs-zkfc.service

    [Unit]
    Description=HDFS ZKFC Service
    After=network.target
    
    [Service]
    Type=forking
    PIDFile=/home/radmin/data/pids/hadoop/hadoop-root-zkfc.pid
    ExecStart=/home/radmin/rexel-dev/rexel-hdfs-zkfc.service.run
    Restart=always
    
    [Install]
    WantedBy=multi-user.target

    文件5:rexel-yarn-resourcemanager.service

    [Unit]
    Description=Yarn Resourcemanager Service
    After=network.target
    
    [Service]
    Type=forking
    PIDFile=/home/radmin/data/pids/hadoop/yarn-root-resourcemanager.pid
    ExecStart=/home/radmin/rexel-dev/rexel-yarn-resourcemanager.service.run
    Restart=always
    
    [Install]
    WantedBy=multi-user.target

    文件6:rexel-yarn-nodemanager.service

    [Unit]
    Description=Yarn Nodemanager Service
    After=network.target
    
    [Service]
    Type=forking
    PIDFile=/home/radmin/data/pids/hadoop/yarn-root-nodemanager.pid
    ExecStart=/home/radmin/rexel-dev/rexel-yarn-nodemanager.service.run
    Restart=always
    
    [Install]
    WantedBy=multi-user.target

    4、配置软连接

    命令:

    ln -s /home/radmin/rexel-dev/rexel-hdfs-journalnode.service /usr/lib/systemd/system/rexel-hdfs-journalnode.service
    ln -s /home/radmin/rexel-dev/rexel-hdfs-namenode.service /usr/lib/systemd/system/rexel-hdfs-namenode.service
    ln -s /home/radmin/rexel-dev/rexel-hdfs-zkfc.service /usr/lib/systemd/system/rexel-hdfs-zkfc.service
    ln -s /home/radmin/rexel-dev/rexel-hdfs-datanode.service /usr/lib/systemd/system/rexel-hdfs-datanode.service
    ln -s /home/radmin/rexel-dev/rexel-yarn-resourcemanager.service /usr/lib/systemd/system/rexel-yarn-resourcemanager.service
    ln -s /home/radmin/rexel-dev/rexel-yarn-nodemanager.service /usr/lib/systemd/system/rexel-yarn-nodemanager.service

    5、加载生效

    命令:

    systemctl daemon-reload

    6、启动服务

    命令:

    systemctl start rexel-hdfs-journalnode.service
    systemctl start rexel-hdfs-namenode.service
    systemctl start rexel-hdfs-zkfc.service
    systemctl start rexel-hdfs-datanode.service
    systemctl start rexel-yarn-resourcemanager.service
    systemctl start rexel-yarn-nodemanager.service

    【小插曲】

    和zookeeper一样,提示了找不到pid文件的错误。

    解决办法:在系统环境变量中配置HADOOP_PID_DIR,YARN_PID_DIR然后修改一下service文件中pid文件的路径,重新启动服务即可。

    ==Redis==

    1、文件规划

    rexel-redis.service
    rexel-redis.service.run

    2、配置run文件

    文件:rexel-redis.service.run

    #!/bin/bash
    source /etc/profile
    /home/radmin/redis-5.0.8/src/redis-server /home/radmin/redis-5.0.8/redis.conf

    3、配置service文件

    文件:rexel-redis.service

    [Unit]
    Description=Redis Service
    After=network.target
    
    [Service]
    Type=forking
    PIDFile=/home/radmin/data/pids/redis_8793.pid
    ExecStart=/home/radmin/rexel-dev/rexel-redis.service.run
    Restart=always
    
    [Install]
    WantedBy=multi-user.target

    【小贴士】

    redis设置了默认的pid文件路径:/var/run/redis_6379.pid,为了方便管理,我重新定义了pidfile的配置。

    相应的在rexel-redis.service文件中的

    4、配置软连接

    命令:

    ln -s /home/radmin/rexel-dev/rexel-redis.service /usr/lib/systemd/system/rexel-redis.service

    5、加载生效

    命令:

    systemctl daemon-reload

    6、启动服务

    命令:

    systemctl start rexel-redis.service

    --END--

  • 相关阅读:
    Linux系统常用目录操作函数 分类: B3_LINUX 20130218 16:44 486人阅读 评论(0) 收藏
    查找数组中最大的2个数 分类: B10_计算机基础 20130211 19:52 1041人阅读 评论(1) 收藏
    C语言程序的内存分配方式 分类: H_HISTORY 20130211 10:18 876人阅读 评论(3) 收藏
    sqlplus环境配置(login.sql) 分类: H2_ORACLE 20130517 09:24 1382人阅读 评论(0) 收藏
    GoldenGate系统二:快速搭建三方单向通信GoldenGate环境 分类: H2_ORACLE 20130516 10:54 827人阅读 评论(0) 收藏
    C语言声明数组变量时,在什么情况下,可不指定数组大小 分类: H_HISTORY 20130211 11:03 694人阅读 评论(0) 收藏
    【转载】linux中误删除oracle数据文件的恢复操作 分类: H2_ORACLE 20130614 12:07 353人阅读 评论(0) 收藏
    使用C语言读取properties文件V1.0 分类: H_HISTORY 20130226 12:37 946人阅读 评论(0) 收藏
    syslog()的基本用法 20130218 22:59 372人阅读 评论(0) 收藏
    使用异或加密数据 20130209 22:24 1043人阅读 评论(0) 收藏
  • 原文地址:https://www.cnblogs.com/quchunhui/p/13265747.html
Copyright © 2011-2022 走看看