Solaris 11中配置基于link的IPMP

http://blog.itpub.net/29960937/viewspace-1347901/

Tips: Record and feedback errors you have encountered.

Welcome to umap  from10.204.205.253,杩炴帴鍫″瀿鎴愬姛!
Trying 10.204.197.128 ...

Last login: Tue May 14 10:30:13 2019 from 10.204.205.5
Authorized users only. All activity may be monitored and reported       Aug 2018
You have new mail.
-bash-4.4$ 
-bash-4.4$ uname
SunOS
-bash-4.4$ su -
Password: 
Authorized users only. All activity may be monitored and reported       Aug 2018
You have new mail.
root@lnltedmr-tds:~# ipmpstat
usage: ipmpstat [-n] [-o <field> [-P]] -a|-g|-i|-p|-t

  output modes:  -a  display IPMP data address information
                 -g  display IPMP group information
                 -i  display IPMP-related IP interface information
                 -p  display IPMP probe information
                 -t  display IPMP target information

       options:  -n  display IP addresses numerically
                 -o  display only the specified fields, in order
                 -P  display using parsable output mode
root@lnltedmr-tds:~# ipmpstat -p
ipmpstat: probe-based failure detection is disabled
root@lnltedmr-tds:~# Linu
root@lnltedmr-tds:~# 
root@lnltedmr-tds:~# 
root@lnltedmr-tds:~# netadm enable -p ncp DefaultFixed
-p: illegal option -- p
usage: enable <enm-name>
        Enable the specified ENM.
root@lnltedmr-tds:~# netadm list
root@lnltedmr-tds:~# netadm list
root@lnltedmr-tds:~# ipmpstat -p
ipmpstat: probe-based failure detection is disabled
root@lnltedmr-tds:~# ipmpstat -g
root@lnltedmr-tds:~# ipadm create-ipmp ipmp1
root@lnltedmr-tds:~# ipmpstat -g
GROUP       GROUPNAME   STATE     FDT       INTERFACES
ipmp1       ipmp1       failed    --        --
root@lnltedmr-tds:~# ipadm create-ip  net2
root@lnltedmr-tds:~# ipmpstat -g
GROUP       GROUPNAME   STATE     FDT       INTERFACES
ipmp1       ipmp1       failed    --        --
root@lnltedmr-tds:~# ipadm add-ipmp -i net2 ipmp1
root@lnltedmr-tds:~# ipmpstat -g
GROUP       GROUPNAME   STATE     FDT       INTERFACES
ipmp1       ipmp1       failed    --        [net2]
root@lnltedmr-tds:~# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000 
net0: flags=100001000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,PHYSRUNNING> mtu 1500 index 3
        inet 10.204.197.128 netmask ffffff00 broadcast 10.204.197.255
        ether 0:10:e0:3d:24:88 
net2: flags=11000803<UP,BROADCAST,MULTICAST,IPv4,FAILED> mtu 1500 index 6
        inet 0.0.0.0 netmask ff000000 
        groupname ipmp1
        ether 0:10:e0:3d:24:8a 
sp-phys0: flags=100001000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,PHYSRUNNING> mtu 1500 index 4
        inet 169.254.182.77 netmask ffffff00 broadcast 169.254.182.255
        ether 2:21:28:57:47:17 
ipmp1: flags=8011000802<BROADCAST,MULTICAST,IPv4,FAILED,IPMP> mtu 1500 index 5
        inet 0.0.0.0 netmask 0 
        groupname ipmp1
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
        inet6 ::1/128 
net0: flags=120002004841<UP,RUNNING,MULTICAST,DHCP,IPv6,PHYSRUNNING> mtu 1500 index 3
        inet6 fe80::210:e0ff:fe3d:2488/10 
        ether 0:10:e0:3d:24:88 
net2: flags=20012000801<UP,MULTICAST,IPv6,FAILED> mtu 1500 index 6
        inet6 ::/0 
        groupname ipmp1
        ether 0:10:e0:3d:24:8a 
sp-phys0: flags=120002000840<RUNNING,MULTICAST,IPv6,PHYSRUNNING> mtu 1500 index 4
        inet6 ::/0 
        ether 2:21:28:57:47:17 
ipmp1: flags=28012000800<MULTICAST,IPv6,FAILED,IPMP> mtu 1500 index 5
        inet6 ::/0 
        groupname ipmp1
root@lnltedmr-tds:~# ipmpstat -i
INTERFACE   ACTIVE  GROUP       FLAGS     LINK      PROBE     STATE
net2        no      ipmp1       -------   down      disabled  failed
root@lnltedmr-tds:~# 

　　在远程不好搞。得去机房了。

IPMP(IP Multipathing)是Solaris操作系统中重要特性，它将几个网络接口绑定在一起作为一个IPMP组，做到网络接口的冗余。即使其中的一个网络接口发生故障，IPMP上的业务IP不会丢失，会failover到其他接口上，避免了单点故障。IPMP可用于Oracle的RAC的公用网络或私有网络上。

基于link的IPMP(Link Based IPMP)

在Solaris中mpathd守护进程负责探测网络接口的故障与修复，它所用的参数配置文件为/etc/default/mpathd。IPMP根据mpathd探测网络接口故障的方法不同，可分为基于link的IPMP和基于probe的IPMP。

基于link的IPMP具有以下特点

–mpathd利用网络接口驱动程序进行探测(需要接口硬件支持)

–不需要额外的测试IP地址，节省IP资源

–默认可用(如果接口硬件支持)

在配置IPMP时，有两种模式：Active-Active模式和Active-Standby模式。在Active-Active模式中所有网络接口都要在线处于active状态；在Active-Standby模式中，至少有一个网络接口在线处于active状态，其他网络接口处于standby状态。查看IPMP组中各个网络接口的状态信息可用ipmpstat命令。

# ipmpstat
usage: ipmpstat [-n] [-o [-P]] -a|-g|-i|-p|-t
  output modes:  -a  display IPMP data address information
                 -g  display IPMP group information
                 -i  display IPMP-related IP interface information
                 -p  display IPMP probe information
                 -t  display IPMP target information
       options:  -n  display IP addresses numerically
                 -o  display only the specified fields, in order
                 -P  display using parsable output mode

在Solaris 11上配置Link Based IPMP示例

1. 设置NCP (network configuration profile) 为DefaultFixed.

# netadm enable -p ncp DefaultFixed
查看ncp是否为DefaultFixed. # netadm list
TYPE        PROFILE        STATE
ncp         DefaultFixed   online
ncp         Automatic      disabled
loc         Automatic      offline
loc         NoNet          offline
loc         DefaultFixed   online

2. 设置IPMP的transitive-probing属性为false

#svccfg -s svc:/network/ipmp setprop config/transitive-probing=false

#svcadm refresh svc:/network/ipmp:default

查看transitive-probing属性

# svccfg -s svc:/network/ipmp listprop config/transitive-probing

config/transitive-probing boolean false

# ipmpstat -p

ipmpstat: probe-based failure detection is disabled

3. Active-Active模式 (以4个网络接口为例)

创建IPMP组ipmp1

# ipadm create-ipmp ipmp1

创建网络接口net2, net3, net9和net10

#ipadm create-ip net2

#ipadm create-ip net3

#ipadm create-ip net9

#ipadm create-ip net10

添加网络接口到ipmp1组中

# ipadm add-ipmp -i net2 -i net3 -i net9 -i net10 ipmp1

查看ipmp1信息

# ipmpstat -g
GROUP       GROUPNAME   STATE     FDT       INTERFACES
ipmp1           ipmp1                 ok            --           net10 net9 net3 net2
# ipmpstat -i
INTERFACE   ACTIVE  GROUP       FLAGS     LINK      PROBE     STATE
net10               yes         ipmp1          -------         up          disabled    ok
net9                 yes         ipmp1          -------         up          disabled    ok
net3                 yes         ipmp1          -------         up          disabled    ok
net2                 yes         ipmp1          --mbM--     up          disabled    ok

为ipmp1设置IP地址

# ipadm create-addr -T static -a 192.168.57.142/24 ipmp1/v4

# ipmpstat -a
ADDRESS                   STATE  GROUP       INBOUND     OUTBOUND
192.168.57.142            up         ipmp1          net2              net10 net9 net3 net2
查看ipmp1的IP地址

# ipadm show-addr

ADDROBJ   TYPE     STATE     ADDR

lo0/v4           static      ok            127.0.0.1/8

ipmp1/v4      static       ok           192.168.57.144/24

4. Active-Standby模式

继续上面的例子，把net9和net10转换为standby

# ipadm set-ifprop -p standby=on -m ip net9

# ipadm set-ifprop -p standby=on -m ip net10

# ipmpstat -i
INTERFACE   ACTIVE  GROUP       FLAGS     LINK      PROBE     STATE
net10              no           ipmp1           is-----        up          disabled    ok
net9                no           ipmp1           is-----        up          disabled    ok
net3                yes         ipmp1           -------         up          disabled    ok
net2                yes         ipmp1           --mbM--     up          disabled    ok

从ifconfig命令输出中可以看到net9和net10有STANDBY标志，处于inactive状态。

# ifconfig -a inet

ipmp1: flags=108001000843<up,broadcast,running,multicast,ipv4,ipmp,physrunning> mtu 1500 index 9
        inet 192.168.57.142 netmask ffffff00 broadcast 192.168.57.255
        groupname ipmp1
net2: flags=100001000843<up,broadcast,running,multicast,ipv4,physrunning> mtu 1500 index 10
        inet 0.0.0.0 netmask ff000000 broadcast 0.255.255.255
        groupname ipmp1
net3: flags=100001000843<up,broadcast,running,multicast,ipv4,physrunning> mtu 1500 index 11
        inet 0.0.0.0 netmask ff000000 broadcast 0.255.255.255
        groupname ipmp1
net9: flags=100061000843<up,broadcast,running,multicast,ipv4,standby,inactive,physrunning> mtu 1500 index 12
        inet 0.0.0.0 netmask ff000000 
        groupname ipmp1
net10: flags=100061000843<up,broadcast,running,multicast,ipv4,standby,inactive,physrunning> mtu 1500 index 8
        inet 0.0.0.0 netmask ff000000 
        groupname ipmp1

官网原版：

Commonly Used Network Administration Commands

https://docs.oracle.com/cd/E37838_01/html/E61478/index.html

About IPMP Administration

https://docs.oracle.com/cd/E37838_01/html/E60991/gfkcy.html#scrolltoc

Administering IPMP

 https://docs.oracle.com/cd/E37838_01/html/E60991/gdysw.html#scrolltoc

How to Configure an Active-Standby IPMP Group

https://docs.oracle.com/cd/E37838_01/html/E60991/gfyeh.html#scrolltoc

Example 15  Configuring an Active-Standby IPMP Group

This example shows how to create an active-standby IPMP configuration.

$ ipadm create-ipmp ipmp0

$ ipadm create-ip net0
$ ipadm create-ip net1
$ ipadm create-ip net2
$ ipadm add-ipmp -i net0 -i net1 -i net2 ipmp0

$ ipadm create-addr -a 192.0.2.5/24 ipmp0
ipadm: ipmp0/v4
$ ipadm create-addr -a 192.0.2.10/24 ipmp0
ipadm: ipmp0/v4a

$ ipadm create-addr -a 192.0.2.15/27 net0
ipadm: net0/v4
$ ipadm create-addr -a 192.0.2.20/27 net1
ipadm: net1/v4
$ ipadm create-addr -a 192.0.2.25/27 net2
ipadm: net2/v4

$ ipadm set-ifprop -p standby=on -m ip net2

$ ipmpstat -g
GROUP     GROUPNAME   STATE      FDT        INTERFACES
ipmp0     ipmp0      ok         10.00s     net0 net1 (net2)

$ ipmpstat -t
INTERFACE  MODE     TESTADDR        TARGETS
net0       routes   192.0.2.15/27   192.0.2.2/27
net1       routes   192.0.2.20/27   192.0.2.2/17
net2       routes   192.0.2.25/27   192.0.2.5/27