Keepalived

系统可用性评价指标SLA对服务可用性提供了一定程度的明确指标，如果要达到99.5%的月服务可用性，意味着每月停服（计划内或计划外）时间不能超过21.6min
A = MTBF / (MTBF+MTTR） 60x24x30 x (1-99.5%) = 21.6min
提升系统高用性的解决方案：建立冗余机制，在服务层面常用keepalived技术

keepalived功能

1. 基于vrrp协议完成地址流动
2. 为vip地址所在的节点生成ipvs规则(在配置文件中预先定义)
3. 为ipvs集群的各RS做健康状态检测
4. 基于脚本调用接口完成脚本中定义的功能，进而影响集群事务，以此支持nginx、haproxy等服务

Keepalived架构

• 用户空间核心组件：
  vrrp stack：VIP消息通告
  checkers：监测real server
  system call：实现 vrrp 协议状态转换时调用脚本的功能
  SMTP：邮件组件
  IPVS wrapper：生成IPVS规则
  Netlink Reflector：网络接口
  WatchDog：监控进程
• 控制组件：提供keepalived.conf 的解析器，完成Keepalived配置
• IO复用器：针对网络目的而优化的自己的线程抽象
• 内存管理组件：为某些通用的内存管理功能（例如分配，重新分配，发布等）提供访问权限

---

Keepalived 相关文件

1. 软件包名：keepalived
2. 主程序文件：/usr/sbin/keepalived
3. 主配置文件：/etc/keepalived/keepalived.conf
4. 配置文件示例：/usr/share/doc/keepalived/
5. Unit File：/lib/systemd/system/keepalived.service
6. Unit File的环境配置文件：
   /etc/sysconfig/keepalived CentOS
   /etc/default/keepalived Ubuntu

Installing Keepalived

https://keepalived.org/doc/installing_keepalived.html

yum -y install kernel-devel openssl-devel popt-devel
ls keepalived-1.2.13.tar.gz 
keepalived-1.2.13.tar.gz
tar zxvf keepalived-1.2.13.tar.gz -C /usr/src/
cd /usr/src/keepalived-1.2.13/
./configure --prefix=/usr/local/keepalived
make && make install
cd
mkdir -p /etc/keepalived
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
chmod 755 /etc/init.d/keepalived

verification

systemctl status keepalived.service
rpm -ql keepalived

dpkg -s keepalived
ps auxf |grep keepalived
/usr/local/keepalived/sbin/keepalived -v

configuration synopsis

/etc/keepalived/keepalived.conf 配置组成
man keepalived.conf

1. GLOBAL CONFIGURATION
   Global definitions：定义邮件配置，route_id，vrrp配置，多播地址等
2. VRRP CONFIGURATION
   VRRP instance(s)：定义每个vrrp虚拟路由器
3. LVS CONFIGURATION
   Virtual server group(s)
   Virtual server(s)：LVS集群的VS和RS

- 常规配置要点

# router ID 配置
vrrp_instance virtual_router_id 51
#实例路由生效优先级别
vrrp_instance priority 80
# stata配置
MASTER or BACKUP

! Configuration File for keepalived

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_iptables
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 66
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1122
    }
    virtual_ipaddress {
        192.168.200.100 dev eth0 label eth0:0
        192.168.200.200 dev eth0 label eth0:1
    }
}

- 全局配置

#/etc/keepalived/keepalived.conf
global_defs {
   notification_email {
     acassen@firewall.loc    #发生故障切换时邮件发送的目标邮箱，可以按行区分写多个
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc  #发邮件的地址
   smtp_server 192.168.200.1   #邮件服务器地址
   smtp_connect_timeout 30     #邮件服务器连接timeout
   router_id LVS_DEVEL         #每个keepalived主机唯一标识，建议使用当前主机名，但多节点重名不影响        
   vrrp_skip_check_adv_addr   #对所有通告报文都检查，会比较消耗性能，启用此配置后，如果收到的通告报文和上一个报文是同一个路由器，则跳过检查，默认值为全检查
   vrrp_strict  #严格遵守VRRP协议,启用此项后以下状况将无法启动服务:1.无VIP地址 2.配置了单播邻居 3.在VRRP版本2中有IPv6地址，开启动此项并且没有配置vrrp_iptables时会自动开启iptables防火墙规则，默认导致VIP无法访问,建议不加此项配置
   vrrp_garp_interval 0   #gratuitous ARP messages 报文发送延迟，0表示不延迟
   vrrp_gna_interval 0    #unsolicited NA messages （不请自来）消息发送延迟

   #vrrp_mcast_group4 224.0.0.18   #指定组播IP地址范围：224.0.0.0到239.255.255.255,默认值：224.0.0.18
   #vrrp_iptables     #此项和vrrp_strict同时开启时，则不会添加防火墙规则,如果未配置vrrp_strict项,则无需启用此项配置
}

- 虚拟路由器配置

vrrp_instance  {  #VI_1为vrrp的实例名,一般为业务名称配置参数
    state MASTER  #MASTER为当前节点在此虚拟路由器上的初始状态，状态为MASTER或者BACKUP
    interface eth0  #绑定为当前虚拟路由器使用的物理接口，如：eth0,bond0,br0,可以和VIP不在一个网卡
    virtual_router_id 51  # 每个虚拟路由器惟一标识，范围：0-255，每个虚拟路由器此值必须唯一，否则服务无法启动，同属一个虚拟路由器的多个keepalived节点必须相同,务必要确认在同一网络中此值必须唯一
    priority 100  #当前物理节点在此虚拟路由器的优先级，范围：1-254，每个keepalived主机节点此值不同
    advert_int 1  #vrrp通告的时间间隔，默认1s
    authentication {  #认证机制
        auth_type auth_pass   #AH为IPSEC认证(不推荐),PASS为简单密码(建议使用)
        auth_pass 1111    #预共享密钥，仅前8位有效，同一个虚拟路由器的多个keepalived节点须一致
    }
    virtual_ipaddress {   ##虚拟IP地址
    # <IPADDR>/<MASK> brd <IPADDR> dev <STRING> scope <SCOPE> label <LABEL>
    # 192.168.200.100 #指定VIP，不指定网卡，默认为eth0,不指定/prefix,默认为/32
    # 192.168.200.101/24 dev eth1  #指定VIP的网卡，建议和interface指令指定的岗卡不在一个网卡
    # 192.168.200.102/24 dev eth2 label eth2:1 #指定VIP的网卡label
        192.168.200.16
        192.168.200.17
        192.168.200.18
    }
    #track_interface { #配置监控网络接口，一旦出现故障，则转为FAULT状态实现地址转移
    #eth0
    #eth1
}

---

- Custom Configuration

- 开启日志功能

egrep -v '^$|#' /etc/sysconfig/keepalived
KEEPALIVED_OPTIONS="-D -S 6"

cat >> /etc/rsyslog.conf << eof
local6.*                        /var/log/keepalived.log
eof

systemctl restart rsyslog
tailf /var/log/keepalived.log

- 子配置文件

include /path/file

cat /etc/keepalived/keepalived.conf
global_defs {
...
}
include /etc/keepalived/conf.d/*.conf   #将VRRP相关配置放在子配置文件中

- Functional configuration

Question：

1. keepalived的VIP无法访问

• global_defs vrrp_strict #开启此配置，会自动生效防火墙设置，导致无访问VIP
• 或可添加vrrp_iptables配置不生成iptables配置
• 或在源码安装时指定 ./configure --prefix=/usr/local/keepalived #--disable-fwmark 可用于禁用iptables规则,可访止VIP无法访问,无此选项默认会启用

PS:

SLA
keepalived.org
VRRP概述
华为VRRP原理讲解