zoukankan      html  css  js  c++  java

  • Nginx+keepalived做双机热备加tomcat负载均衡

    Nginx+keepalived做双机热备加tomcat负载均衡

    环境说明：

    nginx1:192.168.2.47
nginx2:192.168.2.48
tomcat1:192.168.2.49
tomcat2:192.168.2.50
vip:192.168.2.51

    一.Nginx配置
    1.安装Nginx所需pcre库
    wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.10.tar.gz

    tar -zxvf pcre-8.10.tar.gz
cd pcre-8.10
./configure
make
make install

    2.安装Nginx
    wget http://nginx.org/download/nginx-0.8.52.tar.gz

    groupadd www
useradd -g www www
tar zxvf nginx-0.8.52.tar.gz
cd nginx-0.8.52/
./configure --user=www --group=www --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_ssl_module
make
make install

    注：如果出现以下错误

    ./configure: error: SSL modules require the OpenSSL library.
Centos需要安装openssl-devel
Ubuntu则需要安装:sudo apt-get install libssl-dev

    3.修改配置文件为以下内容：

    user  www www;
worker_processes 2;
pid        logs/nginx.pid;
worker_rlimit_nofile 51200;
 
events
{
 use epoll;
 worker_connections 51200;
}
 
http
{
 include       mime.types;
 default_type  application/octet-stream;
 keepalive_timeout 120;
 server_tokens off;
 send_timeout 60;
 tcp_nodelay on;
 
 upstream  tomcats  {
 server 192.168.2.50:8080;
 server 192.168.2.49:8080;
 #ip_hash;       #在没有做共享session的情况下ip_hash可以解决session问题
 
 }
 
 server
 {
 listen  80;
 server_name  192.168.2.48;
 
 location / {
 proxy_pass        http://tomcats;
 proxy_set_header   Host             $host;
 proxy_set_header   X-Real-IP        $remote_addr;
 proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
 }
 
 log_format access_log  '$remote_addr - $remote_user [$time_local] $request '
 '"$status" $body_bytes_sent "$http_referer" '
 '"$http_user_agent" "$http_x_forwarded_for"';
 access_log  /usr/local/nginx/logs/access.log  access_log;
 }
 
}

    4.测试配置文件

    /usr/local/nginx/sbin/nginx -t

    如果出现以下情况

    /usr/local/nginx/sbin/nginx: error while loading shared libraries: libpcre.so.0: or directory

    解决方法：

    sudo ln -s /usr/local/lib/libpcre.so.0 /usr/lib/libpcre.so.0

    /usr/local/nginx/sbin/nginx -t
    显示以下信息为正确的

    the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
configuration file /usr/local/nginx/conf/nginx.conf test is successful

    5.优化内核参数
    vim /etc/sysctl.conf在最后添加

    net.ipv4.tcp_max_syn_backlog = 65536
net.core.netdev_max_backlog = 32768
net.core.somaxconn = 32768
net.core.wmem_default = 8388608
net.core.rmem_default = 8388608
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 2
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_mem = 94500000 915000000 927000000
net.ipv4.tcp_max_orphans = 3276800
net.ipv4.ip_local_port_range = 1024  65535

    保存退出后执行

    sysctl -p

    6.切割Nginx日志脚本

    #!/bin/bash
PATH_LOGS="/usr/local/nginx/logs"
YEAR=`date -d "-1 days" +"%Y"`
MONTH=`date -d "-1 days" +"%m"`
mkdir -p $PATH_LOGS/$YEAR/$MONTH
mv $PATH_LOGS/access.log $PATH_LOGS/$YEAR/$MONTH/access_$(date -d "-1 days" +"%Y%m%d").log
kill -USR1 `cat $PATH_LOGS/nginx.pid`

    把该脚本加到crontab每天00点执行
    注：备机的Nginx和以上安装步骤一样

    二.安装配置Keepalived
    1.下载所需要的软件
    wget http://keepalived.org/software/keepalived-1.1.19.tar.gz
    wget http://rpm5.org/files/popt/popt-1.16.tar.gz
    2.安装popt
    编译keepalived时需要popt，否则会报以下错误：

    configure: error: Popt libraries is required
    tar -zxvf popt-1.16.tar.gz
cd popt-1.16
./configure
make
make install

    3.安装keepalived

    tar -zxvf keepalived-1.1.19.tar.gz
cd keepalived-1.1.19
./configure --prefix=/usr/local/keepalived
make
make install

    4.修改配置文件为以下内容:

    vim /usr/local/keepalived/etc/keepalived/keepalived.conf

    ! Configuration File for keepalived
 
global_defs {
 router_id LVS_DEVEL
}
vrrp_script Monitor_Nginx {
 script "/root/scripts/monitor_nginx.sh" #根据自己的实际路径放置monitor_nginx.sh    
 interval 2
 weight 2
}
vrrp_instance VI_1 {
 state MASTER
 interface eth0
 virtual_router_id 51
 priority 100
 advert_int 1
 authentication {
 auth_type PASS
 auth_pass 1234
}
 track_script {
 Monitor_Nginx
}
 virtual_ipaddress {
 192.168.2.51
 }
}

    注：monitor_nginx.sh为监控nginx进程的脚本,内容如下

    #!/bin/bash
if [ "$(ps -ef | grep "nginx: master process"| grep -v grep )" == "" ]
then
 /usr/local/nginx/sbin/nginx
 sleep 5
 if [ "$(ps -ef | grep "nginx: master process"| grep -v grep )" == "" ]
 then
 killall keepalived
 fi
fi

    5.启动keepalived

    /usr/local/keepalived/sbin/keepalived -D -f /usr/local/keepalived/etc/keepalived/keepalived.conf

    注:备机的keepalived的安装和上面一样，只要把配置文件改为以下(把MASTER改为BACKUP)

    ! Configuration File for keepalived
 
global_defs {
 router_id LVS_DEVEL
}
vrrp_script Monitor_Nginx {
 script "/root/scripts/monitor_nginx.sh"
 interval 2
 weight 2
}
vrrp_instance VI_1 {
 state BACKUP            #改为BACKUP
 interface eth0
 virtual_router_id 51
 priority 100            #比MASTER数值要低
 advert_int 1
 authentication {
 auth_type PASS
 auth_pass 1234
}
 track_script {
 Monitor_Nginx
}
 virtual_ipaddress {
 192.168.2.51
 }
}

    三．测试步骤

    1．    访问VIP看是否能够正常访问后端的tomcat
    2．    停止其中一个tomcat看是否能将访问转到另一台上
    3．    停止两台nginx上任何一个nginx进程看监控进程脚本是否会自动启动nginx
    4．    停止任何一台nginx上的keepalived进程看另一台是否接管vip
    比如停止Master上的keepalived,例如如下killall keepalived,查看BACKUP机器是否已经接管，如果BACKUP接管后，BACKUP机器日志会是出下情况
    tail  /var/log/syslog

    Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE
Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATE
Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.2.51

    MASTER机器上日志会显示

    Keepalived_vrrp: Terminating VRRP child process on signal
Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.

    现在把MASTER上的Keepalived重新启动，会看到MASTER重新接管VIP，并对外提供服务，BACKUP仍旧回到BACKUP STATE,如果不是这种情况，请检查配置文件和步骤.

    现在的BACKUP日志如下：

    Keepalived_vrrp: VRRP_Instance(VI_1) Received higher prio advert
Keepalived_vrrp: VRRP_Instance(VI_1) Entering BACKUP STATE
Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.

    Master日志如下：

    Keepalived_vrrp: VRRP_Script(Monitor_Nginx) succeeded
Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE
Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATE
Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.2.51

    ------------------------------------------------------

    nginx可以根据客户端IP进行负载均衡，在upstream里设置ip_hash，就可以针对同一个C类地址段中的客户端选择同一个后端服务器，除非那个后端服务器宕了才会换一个。

    nginx的upstream目前支持的5种方式的分配


    1、轮询（默认） 
    每个请求按时间顺序逐一分配到不同的后端服务器，如果后端服务器down掉，能自动剔除。 
    upstream backserver { 
    server 192.168.0.14; 
    server 192.168.0.15; 
    } 

    2、指定权重 
    指定轮询几率，weight和访问比率成正比，用于后端服务器性能不均的情况。 
    upstream backserver { 
    server 192.168.0.14 weight=10; 
    server 192.168.0.15 weight=10; 
    } 

    3、IP绑定 ip_hash 
    每个请求按访问ip的hash结果分配，这样每个访客固定访问一个后端服务器，可以解决session的问题。 
    upstream backserver { 
    ip_hash; 
    server 192.168.0.14:88; 
    server 192.168.0.15:80; 
    } 

    4、fair（第三方） 
    按后端服务器的响应时间来分配请求，响应时间短的优先分配。 
    upstream backserver { 
    server server1; 
    server server2; 
    fair; 
    } 

    5、url_hash（第三方） 
    按访问url的hash结果来分配请求，使每个url定向到同一个后端服务器，后端服务器为缓存时比较有效。 
    upstream backserver { 
    server squid1:3128; 
    server squid2:3128; 
    hash $request_uri; 
    hash_method crc32; 
    } 

    在需要使用负载均衡的server中增加 

    proxy_pass http://backserver/; 
    upstream backserver{ 

    ip_hash; 
    server 127.0.0.1:9090 down; (down 表示单前的server暂时不参与负载) 
    server 127.0.0.1:8080 weight=2; (weight 默认为1.weight越大，负载的权重就越大) 
    server 127.0.0.1:6060; 
    server 127.0.0.1:7070 backup; (其它所有的非backup机器down或者忙的时候，请求backup机器) 
    } 

    max_fails ：允许请求失败的次数默认为1.当超过最大次数时，返回proxy_next_upstream 模块定义的错误 
      

    fail_timeout:max_fails次失败后，暂停的时间
  • 相关阅读:
    解决 vs2010 安装过程 提示序列号非法问题
    下载文件（弹出迅雷来下载）
    UrlRewriter URL重写
    C#加密算法汇总
    js 平时经常用的
    c# 解析txt 统计
    漂亮的后台 模板
    无限级分类 父节点 子节点
    FlyTreeView for asp.net （4.4.1.2最新破解版）
    jQueryJSON 无刷新三级联动
  • 原文地址：https://www.cnblogs.com/fx2008/p/4071819.html
Copyright © 2011-2022 走看看