http://www.tokiwinter.com/configuring-transitive-ipmp-on-solaris-11/
We all know the pain of configuring probe-based IPMP under Solaris, with a slew of test addresses being required, and a long line of ifconfig configuration in our /etc/hostname.<interface> files.
With Solaris 11, there is a new type of probe-based IPMP called transitive probing. This new type of probing does not require test addresses, as per the documentation: "Transitive probes are sent by the alternate interfaces in the group to probe the active interface. An alternate interface is an underlying interface that does not actively receive any inbound IP packets".
In this article, I will configure failover (active/passive) IPMP on clusternode1 (the first node of a Solaris Cluster I'm building). Interface net0 has an address of 10.1.1.80 (configured at install time), and I'll be adding this into an IPMP group ipmp0 along with a standby interface, net1. Make sure you are performing these steps via a console connection, as the original address associated with net0 will need to be removed before attempting to add it to an IPMP group.
The first step, ensure that there is an entry in /etc/hosts for the IP address you're configuring IPMP for:
# grep '^10.1.1.80' /etc/hosts
10.1.1.80 clusternode1
Next, ensure that automatic network configuration is disabled. In my case it was as I'd configured networking manually during the installation of Solaris 11:
# netadm list -p ncp –x
TYPE PROFILE STATE AUXILIARY STATE
ncp Automatic disabled disabled by administrator
ncp DefaultFixed online active
Verify that the appropriate physical interfaces are available. In the following output, I'll be bonding e1000g0 (net0) and e1000g1 (net1) into a failover IPMP group.
# dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net1 Ethernet unknown 0 unknown e1000g1
net2 Ethernet unknown 0 unknown e1000g2
net3 Ethernet unknown 0 unknown e1000g3
net0 Ethernet up 1000 full e1000g0
List the current addresses – from the output of ipadm show-addr I can see that I'll need to delete net0/v4 and net0/v6, otherwise I'll be unable to add net0 to the IPMP group.
# ipadm delete-addr net0/v4
# ipadm delete-addr net0/v6
As the net0 IP interface is already created, I only need to create the net1 interface:
# ipadm create-ip net1
I can then create the IPMP group, which I'll call ipmp0:
# ipadm add-ipmp -i net0 -i net1 ipmp0
Next, enable transitive probing, which is disabled by default:
# svccfg -s svc:/network/ipmp setprop config/transitive-probing=true
# svccfg -s svc:/network/ipmp listprop config/transitive-probing
config/transitive-probing boolean true
# svcadm refresh svc:/network/ipmp:default
And configure the appropriate interface (in my case net1) to be a standby interface (as I'm using failover):
# ipadm set-ifprop -p standby=on -m ip net1
Now I can create my IPv4 address on the IPMP group:
# ipadm create-addr -T static -a clusternode1/24 ipmp0/v4
# ipadm show-addr ipmp0
ADDROBJ TYPE STATE ADDR
ipmp0/v4 static ok 10.1.1.80/24
Finally, fix the default route. I removed the existing route and added a new default route using the new and correct interface – ipmp0:
# route -p delete default 10.1.1.1
# route -p add default 10.1.1.1 -ifp ipmp0
# netstat -rn -f inet
Routing Table: IPv4
Destination Gateway Flags Ref Use Interface
-------------------- -------------------- ----- ----- ---------- ---------
default 10.1.1.1 UG 1 0 ipmp0
10.1.1.0 10.1.1.80 U 8 388 ipmp0
127.0.0.1 127.0.0.1 UH 2 554 lo0
You can use ipmpstat to verify the configuration and health of the IPMP group:
# ipmpstat -g
GROUP GROUPNAME STATE FDT INTERFACES
ipmp0 ipmp0 ok 10.00s net0 (net1)
# ipmpstat -a
ADDRESS STATE GROUP INBOUND OUTBOUND
:: down ipmp0 -- --
clusternode1 up ipmp0 net0 net0
# ipmpstat -t
INTERFACE MODE TESTADDR TARGETS
net1 transitive <net1> <net0>
net0 routes clusternode1 10.1.1.1
Let's perform a failover test. I'll disable net0 and ensure that the clusternode1 address fails over:
# ipadm disable-if -t net0
# ipmpstat -t
INTERFACE MODE TESTADDR TARGETS
net1 routes clusternode1 10.1.1.1
It works! (and my SSH connection is still active…) – net1 is now active with the correct IP address. Let's fail it back:
# ipadm enable-if -t net0
# ipmpstat -t
INTERFACE MODE TESTADDR TARGETS
net0 routes clusternode1 10.1.1.1
net1 transitive <net1> <net0>
The address has failed back to net0, and again my SSH connection is still active. I can now continue with clusternode2, and the rest of the cluster install.