实验: Keepalived + Nginx + Tomcat 搭建高可用主从模式Web服务器
1. 安装必要组件
实验环境
IP | 应用 | 组件 |
---|---|---|
10.192.15.125(ubuntu-18.04.6) | nginx-master | nginx,keepalived |
10.192.15.126 (ubuntu-18.04.6) | nginx-slaver | nginx,keepalived |
10.192.15.124 | tomcat | tomcat1占用8082端口 tomcat2占用8083端口 |
Tomcat安装省略...........................
两台nginx服务器相同操作
安装 gcc g++依赖库
apt-get install build-essential -y
apt-get install libtool -y
安装 pcre依赖库
sudo apt-get update
sudo apt-get install libpcre3 libpcre3-dev -y
安装zlib
apt-get install zlib1g-dev
安装openssl
apt-get install openssl
2. 安装nginx
安装nginx
apt-get install nginx -y
nginx
配置nginx
rm /etc/nginx/sites-enabled/default
touch /etc/nginx/sites-enabled/tomcat.conf
vim /etc/nginx/sites-enabled/tomcat.conf
nginx-master tomcat.conf配置
include /etc/nginx/mime.types;
include /etc/nginx/conf.d/*.conf;
#########################
upstream 10.192.15.125 {
server 10.192.15.124:8083 weight=1;
server 10.192.15.124:8082 weight=1;
}
server {
listen 80;
server_name localhost;
location / {
proxy_pass http://10.192.15.125;
proxy_set_header X-NGINX "NGINX-master"; #区分两台nginx请求
}
}
nginx-slaver tomcat.conf配置
include /etc/nginx/mime.types;
include /etc/nginx/conf.d/*.conf;
#########################
upstream 10.192.15.126 {
server 10.192.15.124:8083 weight=1;
server 10.192.15.124:8082 weight=1;
}
server {
listen 80;
server_name localhost;
location / {
proxy_pass http://10.192.15.126;
proxy_set_header X-NGINX "NGINX-slaver";
}
}
使新的配置生效
nginx -s reload
3. 安装keepalived
安装keepalived
apt-get insall keepalived
配置keepalived
cd /etc/keepalived/
touch check_nginx.sh keepalived.conf
check_nginx.sh
#!/bin/bash
#代码一定注意空格,逻辑就是:如果nginx进程不存在则启动nginx,如果nginx无法启动则kill掉keepalived所有进程
A=`ps -C nginx --no-header |wc -l`
if [ $A -eq 0 ];then
/etc/init.d/nginx start
sleep 3
if [ `ps -C nginx --no-header |wc -l`-eq 0 ];then
killall keepalived
fi
fi
两台nginx的keepalived.conf有所不同
配置nginx-master上 /etc/keepalived/keepalived.conf
global_defs {
notification_email {
//可以添加邮件提醒
}
}
vrrp_instance VI_1 {
state MASTER //主服务器
interface enp0s3
virtual_router_id 51
mcast_src_ip 10.192.15.125 //真实ip
priority 250
advert_int 1
authentication {
auth_type PASS
auth_pass 123456
}
track_script {
chk_nginx
}
virtual_ipaddress {
10.192.15.130 # 虚拟ip 主从配置一致
}
}
配置nginx-slaver上 /etc/keepalived/keepalived.conf
global_defs {
notification_email {
//可以添加邮件提醒
}
}
vrrp_instance VI_1 {
state BACKUP //从服务器
interface enp0s3
virtual_router_id 51 //虚拟router id 主从一致
mcast_src_ip 10.192.15.126 //真实ip
priority 240
advert_int 1
authentication {
auth_type PASS
auth_pass 123456
}
track_script {
chk_nginx
}
virtual_ipaddress {
10.192.15.130 #虚拟ip 主从配置一致
}
}
tips
- state 主为 MASTER 从为BACKUP
- virtual_router_id 虚拟路由id主从需要保持一致
- mcast_src_ip nginx主或从服务器真实ip
- interface 使用ifconfig 查看2 里ip对应的网卡 我这里vbox 第一块网卡默认是enp0s3 第二块是enp0s8
- virtual_ipaddress{ } 这里是虚拟出来的ip,应该是同一网段未被占用的ip
- authentication{ } 主从通信验证的密钥 ,主从两边需要一致
启动keepalived
/etc/init.d/keepalived start
查看 keepalived log
tail -f /var/log/syslog
检查keepalived 及 nginx运行是否正常
ps -ef | grep keepalived
ps -ef | grep nginx
4. 验证高可用主备模式是否正常
正常状态
检查虚拟ip绑定
ip add
----------------
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 08:00:27:55:7b:bf brd ff:ff:ff:ff:ff:ff
inet 10.192.15.125/24 brd 10.192.15.255 scope global enp0s3
valid_lft forever preferred_lft forever
inet 10.192.15.130/32 scope global enp0s3 ## 可以看到虚拟ip现在绑定再 主nginx的网卡上
valid_lft forever preferred_lft forever
inet6 fe80::a00:27ff:fe55:7bbf/64 scope link
valid_lft forever preferred_lft forever
master down机
在master上执行
killall keepalived
killall nginx
ip add
--------------------------
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 08:00:27:55:7b:bf brd ff:ff:ff:ff:ff:ff
inet 10.192.15.125/24 brd 10.192.15.255 scope global enp0s3
valid_lft forever preferred_lft forever
inet6 fe80::a00:27ff:fe55:7bbf/64 scope link
valid_lft forever preferred_lft forever
在slaver上执行
ip add
---------------------------------------
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 08:00:27:e1:1c:42 brd ff:ff:ff:ff:ff:ff
inet 10.192.15.126/24 brd 10.192.15.255 scope global enp0s3
valid_lft forever preferred_lft forever
inet 10.192.15.130/32 scope global enp0s3 # 可以看到 130这个ip 帮到 126这个台机器的网卡上了
valid_lft forever preferred_lft forever
inet6 fe80::a00:27ff:fee1:1c42/64 scope link
valid_lft forever preferred_lft forever
5. 非抢占模式
非抢占模式指master down机重启后不会和slaver争抢虚拟ip地址。 除非slaver也down机或者不能正常使用
master的 keepalived.conf配置
vrrp_script chk_nginx {
script "/etc/keepalived/check_nginx.sh" //检测nginx进程的脚本
interval 2
weight -20
}
global_defs {
notification_email {
//可以添加邮件提醒
}
}
vrrp_instance VI_1 {
state BACKUP //主服务器(非抢占模式需要配置成BACKUP)
interface ens33
virtual_router_id 51
mcast_src_ip 10.192.15.125
priority 250
advert_int 1
nopreempt //非抢占模式
authentication {
auth_type PASS
auth_pass 123456
}
track_script {
chk_nginx
}
virtual_ipaddress {
10.192.15.130
}
}
slaver的keepalived.conf配置
vrrp_script chk_nginx {
script "/etc/keepalived/check_nginx.sh" //检测nginx进程的脚本
interval 2
weight -20
}
global_defs {
notification_email {
//可以添加邮件提醒
}
}
vrrp_instance VI_1 {
state BACKUP //从服务器
interface ens33
virtual_router_id 51
mcast_src_ip 10.192.15.126
priority 240
advert_int 1
nopreempt //非抢占模式
authentication {
auth_type PASS
auth_pass 123456
}
track_script {
chk_nginx
}
virtual_ipaddress {
10.192.15.130
}
}
tips 非抢占模式的和抢占模式的区别
- state 主从服务器 都是BACKUP
- vrrp_instance 下增加了nopreempt项
测试结果:
1. 强占模式下 重起 master服务器上keepalived和nginx 虚拟ip 130 重新绑定到master服务器(125)网卡
2. 非抢占模式 重启 master后 虚拟ip 130仍旧绑定在slaver服务器(126)上未变
参考资料 : https://blog.csdn.net/apei830/article/details/78685855