haproxy+keepalived实现高可用负载均衡

环境四台机器都是CentOS5.5（32位）版本：

IP地址             用处

192.168.1.10       MASTER

192.168.1.11       BACKUP

192.168.1.101       负载A

192.168.1.102       负载B

192.168.1.20      VIP

1、MASTER上安装haproxy

检查主机上是否有yum  Extra Packages for Enterprise Linux (EPEL) 

#yum list |grep epel-release

epel-release.noarch                      5-4                        

如果有就安装：

#yum install epel-release

查看是否有yum haproxy

#yum list |grep haproxy

haproxy.i386                             1.3.26-1.el5

有就安装：

#yum install haproxy

 

编辑haproxy主配置文件

#vi /etc/haproxy/haproxy.cfg

#---------------------------------------------------------------------
# Example configuration for a possible web application. See the
# full configuration options online.
#
# http://haproxy.1wt.eu/download/1.3/doc/configuration.txt
#
#---------------------------------------------------------------------

#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
log 127.0.0.1 local2
chroot /var/lib/haproxy #安装目录
pidfile /var/run/haproxy.pid
maxconn 4000 #最大连接数
user nobody
group nobody
daemon #守护进程运行 

#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode http #7层,默认的模式mode {tcp|http|health},tcp是4层,http是7层
log global
option dontlognull #来防止记录 Alteo(4层负载均衡)发出的健康检测,如果一个 session 交互没有数据,这个 session就不会被记录 
option httpclose #主动关闭http通道,HA-Proxy不支持keep-alive模式
option httplog #http 日志格式
option forwardfor #后端服务器需要获得客户端的真实IP,将从Http Header中获得客户端IP 
option redispatch #serverId对应的服务器挂掉后,强制定向到其他健康的服务器
timeout connect 10000 # default 10 second time out if a backend is not found
timeout client 300000 #客户端超时(毫秒)
timeout server 300000 #服务器超时(毫秒)
maxconn 60000 #最大连接数
retries 3

#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend main *:80
# acl url_static path_beg -i /static /images /javascript /stylesheets
# acl url_static path_end -i .jpg .gif .png .css .js

# use_backend static if url_static
mode http
default_backend app

#---------------------------------------------------------------------
# static backend for serving up images, stylesheets and such
#---------------------------------------------------------------------
#backend static
#balance roundrobin
#server static 127.0.0.1:4331 check

#---------------------------------------------------------------------
# round robin balancing between the various backends
#---------------------------------------------------------------------
backend app
balance roundrobin
server app1 192.168.1.101:80 cookie 1 check inter 2000 rise 3 fall 3
server app2 192.168.1.102:80 cookie 1 check inter 2000 rise 3 fall 3

#---------------------------------------------------------------------
# check status
#---------------------------------------------------------------------
listen secure #自定义一个frontend，也可以放在listen或者backend中
bind *:8080 #监听的ip端口号
stats enable #开关
stats uri /admin?admin #访问的uri ip:8080/admin?admin
stats auth admin:admin #认证用户名和密码
stats hide-version #隐藏HAProxy的版本号
stats refresh 5s #统计页面自动刷新时间

多后端配置文件：

global

    log 127.0.0.1 local0 notice

    maxconn 20480

    ulimit-n 65535

    uid nobody

   gid nobody

   daemon

   nbproc 2

   pidfile /var/run/haproxy.pid

defaults

    log global

    mode http

    option httplog

    option httpclose

    option forwardfor

    option dontlognull

    option redispatch

    retries 3

    balance roundrobin

    timeout connect 5000

    timeout client 50000

    timeout server 50000

##frontend settings ######

frontend test

    bind 192.168.1.241:80

    mode http

    #capture request header Host len 32

    #log-format %hr %r %st %B %Tr

    #http-request add-header X-Req %[env(USER)]

    option httpclose

    option httplog

    option dontlognull

    option forwardfor

    default_backend x.yxpai.com

##setting ACLs ##

#acl ua_moz hdr_reg(User-Agent) -i ^iphone.*

acl host_x hdr_reg(host) -i ^(x.yxpai.com)$

#http-request deny if host_x

##applying ACLs#####

use_backend x.yxpai.com if host_x

#use_backend moz if ua_moz

#redirect code 301 prefix http://192.168.1.64 if ua_moz

###testing how to modify HTTP response HEADER##

##setting up backends###

backend x.yxpai.com

    option httplog

    option httpclose

    option forwardfor

    http-response set-header X-Server-Port %[dst_port]

    #http-response set-header X-Handled-By %[env(USER)]

    http-response add-header X-Cached %[env(HOSTNAME)]

    http-response replace-value Server ^nginx.*$ Apache2

    http-response replace-header X-Powered-By ^.*$ PHP

    http-response del-header X-Powered-By

    http-response del-header Server

    #http-response replace-value Cache-control ^public$ private

    cookie PHPSESSID insert indirect nocache ##setting session sticky

    server app1 192.168.1.102:80 maxconn 3000 cookie app1 check inter 5s rise 3 fall 3

backend moz

    balance roundrobin

    option httplog

    option httpclose

    option forwardfor

    server app1 192.168.1.101:80 maxconn 3000 cookie app1 check inter 5s rise 3 fall 3

# option httpchk GET /index.html

listen status *:8080

    stats enable

    stats uri /stats

    stats auth admin:123456

    #stats admin if TRUE

    stats realm (Haproxystatistic)

2、MASTER安装keepalived

#cd /home/sandea/

#wget http://www.keepalived.org/software/keepalived-1.1.17.tar.gz  

#tar zxvf keepalived-1.1.17.tar.gz

#cd keepalived-1.1.17

#./configure prefix=/

#make&&make install

#vi /etc/keepalived/keepalived.conf内容如下：

! Configuration File for keepalived
global_defs {
   router_id LVA_DEVEL
}
vrrp_script chk_http_port {
   script "/etc/keepalived/check_haproxy.sh"
   interval 2
   weight  2
}
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
track_script {
    chk_http_port
}
    virtual_ipaddress {
        192.168.1.20
    }
}

 

创建上面调用了一个脚本check_haproxy.sh：

#vi /etc/keepalived/check_haproxy.sh

上面调用了一个脚本check_haproxy.sh，内容如下：

#!/bin/bash
A=`ps -C haproxy --no-header | wc -l`
if [ $A -eq 0 ];then
/usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg
echo "haproxy start"
sleep 3
if [ `ps -C haproxy --no-header | wc -l` -eq 0 ];then
/etc/init.d/keepalived stop
echo "keepalived stop"
fi
fi

3、BACKUP上安装haproxy，步骤就不详细介绍，和MASTER上面一样的。

haproxy.conf内容如下

#---------------------------------------------------------------------
#
# http://haproxy.1wt.eu/download/1.3/doc/configuration.txt
#
#---------------------------------------------------------------------

#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
pidfile /var/run/haproxy.pid
maxconn 4000
user nobody
group nobody
daemon

#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode http
log global
option dontlognull
option httpclose
option httplog
option forwardfor
option redispatch
maxconn 60000
retries 3

#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend main *:80
# acl url_static path_beg -i /static /images /javascript /stylesheets
# acl url_static path_end -i .jpg .gif .png .css .js

# use_backend static if url_static
mode http
default_backend app

#---------------------------------------------------------------------
# static backend for serving up images, stylesheets and such
#---------------------------------------------------------------------
#backend static
#balance roundrobin
#server static 127.0.0.1:4331 check

#---------------------------------------------------------------------
# round robin balancing between the various backends
#---------------------------------------------------------------------
backend app
balance roundrobin
server app1 192.168.1.101:80 cookie 1 check inter 2000 rise 3 fall 3
server app2 192.168.1.102:80 cookie 1 check inter 2000 rise 3 fall 3

#---------------------------------------------------------------------
# check status
#---------------------------------------------------------------------
listen secure #自定义一个frontend，也可以放在listen或者backend中
bind *:8080 #监听的ip端口号
stats enable #开关
stats uri /admin?admin #访问的uri ip:8080/admin?admin
stats auth admin:admin #认证用户名和密码
stats hide-version #隐藏HAProxy的版本号
stats refresh 5s #统计页面自动刷新时间

4、BACKUP上安装keepalived，步骤也不多介绍，keepalived.conf文件内容就两处有变化，红色字体标出

! Configuration File for keepalived
global_defs {
router_id LVA_DEVEL
}
vrrp_script chk_http_port {
script "/etc/keepalived/check_haproxy.sh"
interval 2
weight 2
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
track_script {
chk_http_port
}
virtual_ipaddress {
192.168.1.20
}
}

 

创建上面调用了一个脚本check_haproxy.sh：

#vi /etc/keepalived/check_haproxy.sh

调用脚本check_haproxy.sh内容：

#!/bin/bash
A=`ip a | grep 192.168.1.20 | wc -l`
B=`ps -ef | grep haproxy | grep -v grep | awk '{print $2}'`
if [ $A -gt 0 ];then
/usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg
else
kill -9 $B
fi
5、两台负载机器我就不多介绍了，用的是系统自带的nginx+PHP

可以查看：http://www.cnblogs.com/sandea/p/4557540.html

6、测试步骤

启动MASTER上的keepalived服务，再启动BACKUP上的keepalived服务。

#ip add 

1.再两台机器上分别执行ip add
主: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:0c:29:98:cd:c0 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.10/24 brd 192.168.1.255 scope global eth0
inet 192.168.1.20/32 scope global eth0
inet6 fe80::20c:29ff:fe98:cdc0/64 scope link
valid_lft forever preferred_lft forever

 

备: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:0c:29:a6:0c:7e brd ff:ff:ff:ff:ff:ff
inet 192.168.1.11/24 brd 255.255.255.254 scope global eth0
inet6 fe80::20c:29ff:fea6:c7e/64 scope link
valid_lft forever preferred_lft forever

 

确定MASTER上是否有192.168.1.20地址！

 

2.停掉主上的haproxy，3秒后keepalived会自动将其再次启动

3.停掉主的keepalived，备机马上接管服务
备: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:0c:29:a6:0c:7e brd ff:ff:ff:ff:ff:ff
inet 192.168.1.11/24 brd 255.255.255.254 scope global eth0
inet 192.168.1.20/32 scope global eth0
inet6 fe80::20c:29ff:fea6:c7e/64 scope link
valid_lft forever preferred_lft forever

 

4、在浏览器地址栏输入：

http://192.168.1.20

看访问是否成功

5、监控haproxy

http://192.168.1.10:8080/admin?admin

或者

http://192.168.1.11:8080/admin?admin