zoukankan      html  css  js  c++  java
  • haproxy + keepalived + mycat 高可用与负载均衡集群配置 centos7

     

     架构如上,但是其实keepalived、haproxy、Mycat都可以多台(比如keepalived、haproxy、Mycat各3台,3台keepalived抢占vip,然后抢到vip的haproxy提供服务,每台haproxy后面挂3个mycat)

    集群部署图的理解:
    1、keepalived 和 haproxy 必须装在同一台机器上(如 172.17.210.210.83 机器上,keepalived 和
    haproxy 都要安装),keepalived 负责为该服务器抢占 vip(虚拟 ip),抢占到 vip 后,对该主机的访问可以通
    过原来的 ip(172.17.210.210.83)访问,也可以直接通过 vip(172.17.210.210.103)访问。
    2、172.17.210.64 上的 keepalived 也会去抢占 vip,抢占 vip 时有优先级,配置 keepalived.conf 中的
    (priority 150 #数值愈大,优先级越高,172.17.210.64 上改为 120,master 和 slave 上该值配置不同)决 定。
    但是一般哪台主机上的 keepalived 服务先启动就会抢占到 vip,即使是 slave,只要先启动也能抢到。(用高版本的keepalived可能抢占机制不一样,有待测试)
    3、haproxy 负责将对 vip 的请求分发到 mycat 上。起到负载均衡的作用,同时 haproxy 也能检测到 mycat
    是否存活,haproxy 只会将请求转发到存活的 mycat 上。
    4、如果一台服务器(keepalived+haproxy 服务器)宕机,另外一台上的 keepalived 会立刻抢占 vip 并接
    管服务。
    如果一台 mycat 服务器宕机,haporxy 转发时不会转发到宕机的 mycat 上,所以 mycat 依然可用。

     如下是我根据自己的开发环境,自己安装的步骤:

    keepalived-2.0.6.tar.gz
    haproxy-1.8.16.tar.gz
    Mycat-server-1.6.6.1-release-20181031195535-linux.tar.gz

    主机1:ip192.168.0.200 安装keepalived、haproxy、Mycat
    主机2:ip192.168.0.199 安装keepalived、haproxy、Mycat


    vip:192.168.0.206


    keepalived与haproxy需安装在同一台机器上

    运作理解:主机1与主机2上的keepalived会为该服务器抢占vip,抢占到vip后,对该主机的访问可以通过原来的ip(ip192.168.0.200或者192.168.0.199)访问,也可以直接通过vip(192.168.0.206)访问,当其中一台宕机时,vip会漂移到另一台机器上,当我们访问192.168.0.206时haproxy会根据配置将请求分发到两台Mycat上实现负载均衡,并且haproxy会检测mycat是否存活。

    一、haproxy安装

    1、安装haproxy(200和199都安装,安装方式一样)

    下载地址:

                             https://src.fedoraproject.org/repo/pkgs/haproxy/

    国内镜像地址:http://pkgs.fedoraproject.org/repo/pkgs/haproxy/
    查看内核版本

    uname -r
    
     

    [root@localhost haproxy]# uname -r
    3.10.0-957.5.1.el7.x86_64

     

    根据内核版本选择编译参数:

     

    这里内核是3.10的可以选择TARGET=linux310 或者 选择 linux2628,安装haproxy

    
    tar -zxvf haproxy-1.8.16.tar.gz
    cd  haproxy-1.8.16
    make TARGET=linux310 ARCH=x86_64 PREFIX=/usr/local/haproxy
    make install PREFIX=/usr/local/haproxy
    cd /usr/local/haproxy
    
    
    
    

    安装成功后,查看版本

    [root@localhost haproxy]# /usr/local/haproxy/sbin/haproxy -v
    HA-Proxy version 1.8.16-5c3f237 2018/12/21
    Copyright 2000-2018 Willy Tarreau <willy@haproxy.org>


     
    2、配置haproxy
    cd /usr/local/haproxy
    touch haproxy.cfg
    

    vim /usr/local/haproxy/haproxy.cfg
    写入一下内容:

    global
    #设置日志
    log 127.0.0.1 local0
    chroot /usr/local/haproxy
    #用户与用户组
    user root
    group root
    #定义每个haproxy进程的最大连接数 ,由于每个连接包括一个客户端和一个服务器端,所以单个进程的TCP会话最大数目将是该值的两倍。
    maxconn 4096
    # 以守护进程的方式运行
    daemon
    defaults
    log global
    #日志中不记录空连接
    option dontlognull
    # 定义连接后端服务器的失败重连次数,连接失败次数超过此值后将会将对应后端服务器标记为不可用
    retries 3
    option redispatch
    # 设置成功连接到一台服务器的最长等待时间,默认单位是毫秒
    timeout connect 5000
    # 设置连接客户端发送数据时的成功连接最长等待时间,默认单位是毫秒
    timeout client 50000
    # 设置服务器端回应客户度数据发送的最长等待时间,默认单位是毫秒
    timeout server 60000
    #统计页面
    listen admin_stats
    bind 192.168.0.206:48800
    mode http
    #采用http日志格式
    option httplog
    #统计页面自动刷新时间
    stats refresh 30s
    #统计页面url
    stats uri /admin_stats
    #统计页面密码框上提示文本
    stats realm Haproxy Manager
    #统计页面用户名和密码设置
    stats auth admin:admin
    #隐藏统计页面上HAProxy的版本信息
    stats hide-version
    listen mycat_service
    # 绑定192.168.0.206:8067端口访问mycat8066端口
    bind 192.168.0.206:8067
    # 定义为tcp模式
    mode tcp
    #采用http日志格式
    option tcplog
    # 开启对后端服务器的健康检测
    option httpchk OPTIONS * HTTP/1.1 Host: www
    # 设置haproxy的调度算法
    balance roundrobin
    #根据调度分配到真实的后台地址,参数解释:port 48700:检测端口48700, inter 5s:5秒检测一次,rise 2:检测成功2次表示服务器可用,fall 3:检测失败3次后表示服务器不可用
    server mycat_200 192.168.0.200:8066 check port 48700 inter 5s rise 2 fall 3
    server mycat_199 192.168.0.199:8066 check port 48700 inter 5s rise 2 fall 3
    #设置服务器端回应客户度数据发送的最长等待时间,默认单位是毫秒
    timeout server 60000
    listen mycat_admin
    #绑定192.168.0.206:9067端口访问mycat9066端口
    bind 192.168.0.206:9067
    mode tcp
    option tcplog
    option httpchk OPTIONS * HTTP/1.1 Host: www
    balance roundrobin
    server mycat_200 192.168.0.200:9066 cookie 1 check port 48700 inter 5s rise 2 fall 3
    server mycat_199 192.168.0.199:9066 cookie 2 check port 48700 inter 5s rise 2 fall 3
    timeout server 60000

    上面的配置中 timeout server   可以配置大一点,不然经常会出现有时候闪断连接mysql的情况

    启动和停止haproxy命令

    启动 haproxy 前必须先启动 keepalived,否则启动不了。
    /usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg #启动
    killall haproxy #停止


    启动的时候会报错如下,原因为:该机器没有抢占到 vip,如果另一台服务启动正常,这个错误可以忽略不管,如果另一台也一样,使
    用 ping 192.168.0.206 命令看看 vip 是否生效,或者用ip a 查看,如果没有生效,说明 keepalived 没有启动成功,回去检查 keepalived 的异
    常再说。

    [root@localhost haproxy]# /usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg
    [WARNING] 170/114135 (88204) : config : proxy 'mycat_admin' : ignoring cookie for server 'mycat_200' as HTTP mode is disabled.
    [WARNING] 170/114135 (88204) : config : proxy 'mycat_admin' : ignoring cookie for server 'mycat_199' as HTTP mode is disabled.
    [ALERT] 170/114135 (88204) : Starting proxy admin_stats: cannot bind socket [192.168.0.206:48800]
    [ALERT] 170/114135 (88204) : Starting proxy mycat_service: cannot bind socket [192.168.0.206:8067]
    [ALERT] 170/114135 (88204) : Starting proxy mycat_admin: cannot bind socket [192.168.0.206:9067]

    3、配置haproxy记录日志功能

    默认haproxy是不记录日志的,为了记录日志还需要配置syslog模块,在linux下是rsyslogd服务,需要先安装rsyslog

    yum install rsyslog -y


    [root@localhost haproxy]# yum install rsyslog -y
    Loaded plugins: fastestmirror, langpacks
    Determining fastest mirrors
    * base: mirrors.163.com
    * extras: mirrors.163.com
    * updates: mirrors.cn99.com
    base | 3.6 kB 00:00:00
    extras | 3.4 kB 00:00:00
    packages-microsoft-com-prod | 2.9 kB 00:00:00
    updates | 3.4 kB 00:00:00
    (1/3): extras/7/x86_64/primary_db | 205 kB 00:00:00
    (2/3): packages-microsoft-com-prod/primary_db | 170 kB 00:00:00
    (3/3): updates/7/x86_64/primary_db | 6.4 MB 00:00:01
    Package rsyslog-8.24.0-34.el7.x86_64 already installed and latest version
    Nothing to do





    如果没有目录
    /etc/rsyslog.d 就执行mkdir /etc/rsyslog.d

    cd /etc/rsyslog.d/ touch haproxy.conf

    vim haproxy.conf
    写入一下内容

    $ModLoad imudp
    $UDPServerRun 514
    local0.* /var/log/haproxy.log
    

    vim /etc/rsyslog.conf
    修改两处:1在#### RULES ####上面一行加入以下内容

    # Include all config files in /etc/rsyslog.d/
    $IncludeConfig /etc/rsyslog.d/*.conf
    
     

    第二处在local7.* /var/log/boot.log下面增加

    local0.*                                                /var/log/haproxy.log
    
     
     

    重启rsyslog服务

    systemctl restart rsyslog.service
    

    将rsyslog加入自动启动服务

    systemctl enable rsyslog.service
    

    二、配置监听mycat是否存活

    在Mycat server1 Mycat server2上都需要添加检测端口48700的脚本,为此需要用到xinetd,xinetd为linux系统的基础服务,在xinetd目录下面增加脚本与端口的映射配置文件。

    yum install xinetd -y
    cd /etc/xinetd.d
    touch mycat_status
    

    vim /etc/xinetd.d/mycat_status
    写入以下内容

     

    service mycat_status
    {
    flags = REUSE
    socket_type = stream
    port = 48700
    wait = no
    user = root
    server =/usr/local/bin/mycat_status
    log_on_failure += USERID
    disable = no
    }


    创建xinetd启动服务脚本
    vim /usr/local/bin/mycat_status

    #!/bin/bash
    #/usr/local/bin/mycat_status.sh
    # This script checks if a mycat server is healthy running on localhost. It will
    # return:
    #
    # "HTTP/1.x 200 OK
    " (if mycat is running smoothly)
    #
    # "HTTP/1.x 503 Internal Server Error
    " (else)
    mycat=`/root/mycat/bin/mycat status |grep 'not running'| wc -l`
    if [ "$mycat" = "0" ];
    then
    /bin/echo -e "HTTP/1.1 200 OK
    "
    else
    /bin/echo -e "HTTP/1.1 503 Service Unavailable
    "
    fi

    修改脚本文件权限

    chmod 777 /usr/local/bin/mycat_status
    chmod 777 /etc/xinetd.d/mycat_status
    

    将启动脚本加入服务
    vim /etc/services
    在末尾加入

    mycat_status    48700/tcp               # mycat_status
    

    重启xinetd服务

    systemctl restart xinetd.service
    

    验证mycat_status服务是否启动成功

    netstat -antup|grep 48700
    
     

    注意:/etc/xinetd.d/mycat_status里的service mycat_status(服务名)和port = 48700(端口)要与 /etc/services里的相同

    三、keepalived安装

    1、安装依赖
    yum install -y openssl-devel openssl libnl libnl-devel libnfnetlink-devel
    
    2、keepalived安装
    (安装keepalived的时候经常keepalived启动不了,或者说启动了但是haproxy切不了,后来试了keepalived2.0.6  2.0.16  1.4.5 都还是一样,最后按照mycat权威指南换成1.2.13还是不行,后来终于发现可能是
    我改了权威指南上写的/etc/keepalived/scripts/check_haproxy.sh 把两行的
    $STARTHAPROXY 改成了第二行是$STOPKEEPALIVED 导致每次keepalived一启动就被停了,后面我把两行都写成
    $STARTHAPROXY就没问题了,出问题的时候还可以多查看  /usr/local/keepalived/var/log/keepalived-haproxy-state.log 和/var/log/messages  看看是否check或者其他什么的。 还有就是下面这一段我试试了去掉后手动打字也有影响,但是可能也没影响,
    后续再安装一次验证一下

    vrrp_script chkhaproxy {
    script "/etc/keepalived/scripts/check_haproxy.sh"
    insterval 2
    weight 2
    }

    cd /root
    wget http://www.keepalived.org/software/keepalived-1.2.13.tar.gz (如果wget下载不了可以去keepalived官网下载 https://www.keepalived.org/download.html) tar -zxvf keepalived-1.2.13.tar.gz cd keepalived-1.2.13 ./configure --prefix=/usr/local/keepalived make make install cp /usr/local/keepalived/sbin/keepalived /usr/sbin/ cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
    cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/ (这里不同的keepalived版本路径可能不同) mkdir /etc/keepalived cd /etc/keepalived/ cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived mkdir -p /usr/local/keepalived/var/log
    2、keepalived配置

    a、查看网卡

    ifconfig
    
     
     

    我这里网卡是ens33所以配置里的interface为ens33,主机1与主机2配置基本一样,state与priority 不同。b、先配置主机2   192.168.0.199的keepalived
    vim /etc/keepalived/keepalived.conf

     

    ! Configuration Fileforkeepalived
    global_defs {
    router_id ip199
    }
    vrrp_script chkhaproxy {
    script "/etc/keepalived/scripts/check_haproxy.sh"
    insterval 2
    weight 2
    }
    #vrrp实例
    vrrp_instance VI_1 {
    state BACKUP
    #本机网卡
    interface ens33
    ##VRRP组名,两个节点的设置必须一样,以指明各个节点属于同一VRRP组
    virtual_router_id 51
    #优先级,数值愈大,优先级越高
    priority 120
    #检查间隔,默认1秒
    advert_int 1
    #设置认证
    authentication {
    #认证方式
    auth_type PASS
    #认证密码
    auth_pass 1111
    }
    #虚拟IP地址池,可以有多个IP,每个IP占一行
    virtual_ipaddress {
    192.168.0.206 dev ens33 scope global
    }
    track_script {
    #调用脚本check_haproxy.sh检查haproxy是否存活
    chkhaproxy
    }
    #指定当切换到master时,执行的脚本
    notify_master /etc/keepalived/scripts/haproxy_master.sh
    #指定当切换到backup时,执行的脚本
    notify_backup /etc/keepalived/scripts/haproxy_backup.sh
    #故障时执行的脚本
    notify_fault /etc/keepalived/scripts/haproxy_fault.sh
    notify_stop /etc/keepalived/scripts/haproxy_stop.sh
    }



    c、配置主机1   192.168.0.200的keepalived

    ! Configuration Fileforkeepalived
    global_defs {
    router_id ip200
    }
    vrrp_script chkhaproxy {
    script "/etc/keepalived/scripts/check_haproxy.sh"
    interval 2
    weight 2
    }
    vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 51
    priority 150
    advert_int 1
    authentication {
    auth_type PASS
    auth_pass 1111
    }
    virtual_ipaddress {
    192.168.0.206 dev ens33 scope global
    }
    track_script {
    chkhaproxy
    }
    notify_master /etc/keepalived/scripts/haproxy_master.sh
    notify_backup /etc/keepalived/scripts/haproxy_backup.sh
    notify_fault /etc/keepalived/scripts/haproxy_fault.sh
    notify_stop /etc/keepalived/scripts/haproxy_stop.sh
    }

    3、创建配置文件和脚本
    mkdir /etc/keepalived/scripts
    

    a、创建检查haproxy是否存活的脚本
    vim /etc/keepalived/scripts/check_haproxy.sh

    #!/bin/bash
    STARTHAPROXY="/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg"
    STOPKEEPALIVED="/etc/init.d/keepalived stop"
    LOGFILE="/usr/local/keepalived/var/log/keepalived-haproxy-state.log"
    echo "[check_haproxy status]" >> $LOGFILE
    A=`ps -C haproxy --no-header |wc -l`
    echo "[check_haproxy status]" >> $LOGFILE
    date >> $LOGFILE
    #如果没有haproxy进程存在,就启动haproxy,停止keepalived,并写入日志
    if [ $A -eq 0 ];then
    echo $STARTHAPROXY >> $LOGFILE
    $STARTHAPROXY >> $LOGFILE 2>&1
    sleep 5
    fi
    if [ `ps -C haproxy --no-header |wc -l` -eq 0 ];then
    exit 0
    else
    exit 1
    fi

    b、创建切换到MASTER与BACKUP脚本
    vim /etc/keepalived/scripts/haproxy_master.sh
    vim /etc/keepalived/scripts/haproxy_backup.sh
    在两个文件中写入以下内容

    #!/bin/bash
    STARTHAPROXY="/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg"
    STOPHAPROXY=`ps -ef | grep sbin/haproxy | grep -v grep | awk '{print $2}'| xargs kill -s 9`
    LOGFILE="/usr/local/keepalived/var/log/keepalived-haproxy-state.log"
    echo "[master]" >> $LOGFILE
    date >> $LOGFILE
    echo "Being master...." >> $LOGFILE 2>&1
    echo "stop haproxy...." >> $LOGFILE 2>&1
    $STOPHAPROXY >> $LOGFILE 2>&1
    echo "start haproxy...." >> $LOGFILE 2>&1
    $STARTHAPROXY >> $LOGFILE 2>&1
    echo "haproxy stared ..." >> $LOGFILE

    c、创建故障时执行的脚本
    vim /etc/keepalived/scripts/haproxy_fault.sh

    #!/bin/bash
    LOGFILE=/usr/local/keepalived/var/log/keepalived-haproxy-state.log
    echo "[fault]" >> $LOGFILE
    date >> $LOGFILE
    

    d、创建停止vrrp脚本
    vim /etc/keepalived/scripts/haproxy_stop.sh

    #!/bin/bash
    LOGFILE=/usr/local/keepalived/var/log/keepalived-haproxy-state.log
    echo "[stop]" >> $LOGFILE
    date >> $LOGFILE
    
    4、赋予脚本可执行权限
    chmod 777 /etc/keepalived/scripts/*
    
    5、启动keepalived
    systemctl start keepalived.service
    
    6、加入开机启动
    systemctl enable keepalived.service
    

    四、测试

    在浏览器输入http://192.168.0.206:48800/admin_stats

     

    查看8067与9067端口是否在监听

     

    连接8067端口
    mysql -uroot -p123456 -h192.168.0.206 -P8067


     
    可以连接,到此就搭建完成了
     
     
  • 相关阅读:
    rwkj 1337
    poj 1002
    map
    vector
    sort排序
    sort函数
    poj 2945
    poj2388
    rwkj 1422搜索(素数环)
    poj2503
  • 原文地址:https://www.cnblogs.com/xiaohanlin/p/11057408.html
Copyright © 2011-2022 走看看