创建tenant1虚机vm1
1）在网络节点添加tenant1流表
ovs-ofctl mod-flows br-tun "table=21,dl_vlan=1,actions=strip_vlan,set_tunnel:1,output:2" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=1,table=2,tun_id=1,actions=mod_vlan_vid:1,resubmit(,10)"
2）在网络节点添加tenant1的dhcp agent服务
ovs-vsctl -- --if-exists del-port tap01 -- add-port br-int tap01 -- set interface tap01 type=internal ovs-vsctl --timeout=10 set Port tap01 tag=1 ip netns add qdhcp01 ip netns exec qdhcp01 ip link set lo up ip link set tap01 netns qdhcp01 ip netns exec qdhcp01 ip link set tap01 up ip netns exec qdhcp01 ip -4 addr add 10.0.0.3/24 brd 10.0.0.255 scope global dev tap01 ip netns exec qdhcp01 ip route replace default via 10.0.0.1 dev tap01 ip netns exec qdhcp01 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tap01 --except-interface=lo --dhcp-range=set:tag0,10.0.0.6,10.0.0.90,86400s --dhcp-lease-max=256 --dhcp-option=3,10.0.0.1
3）在计算节点添加虚机tenant1流表
ovs-ofctl mod-flows br-tun "table=21,dl_vlan=1,actions=strip_vlan,set_tunnel:1,output:2" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=1,table=2,tun_id=1,actions=mod_vlan_vid:1,resubmit(,10)"
4）创建vm1挂接的网桥设备
brctl addbr qbr01 ip link set qbr01 up ip link add qvo01 type veth peer name qvb01 brctl addif qbr01 qvb01 ovs-vsctl add-port br-int qvo01 ip link set qvb01 up ip link set qvo01 up ovs-vsctl set port qvo01 tag=1 ip tuntap add tap01 mode tap ip link set dev tap01 up brctl addif qbr01 tap01
5）准备虚机映像文件
cp cirros-0.3.4-x86_64-disk.img cirros-vm1.img cp cirros-0.3.4-x86_64-kernel cirros-kernel-vm1
6）通过qemu启动虚机vm1
qemu-system-x86_64 -nographic -kernel ./cirros-kernel-vm1 -hda cirros-vm1.img -append "root=/dev/sda1 console=ttyS0" -net nic,macaddr=52:54:00:12:34:60 -net tap,ifname="tap01",script=no,downscript=no
7)虚机vm1启动过程中可以看到dhcp获取的ip信息
udhcpc (v1.20.1) started Sending discover... Sending select for 10.0.0.81... Lease of 10.0.0.81 obtained, lease time 86400
8）登录进vm1虚机
login as 'cirros' user. default password: 'cubswin:)'. use 'sudo' for root. cirros login: 输入用户名cirros，密码cubswin:) $ ip addr|grep eth0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 inet 10.0.0.81/24 brd 10.0.0.255 scope global eth0 $ ip route list default via 10.0.0.1 dev eth0 10.0.0.0/24 dev eth0 src 10.0.0.81
5.2.2创建tenant1虚机vm2
1）创建vm2挂接的网桥设备
brctl addbr qbr02 ip link set qbr02 up ip link add qvo02 type veth peer name qvb02 brctl addif qbr02 qvb02 ovs-vsctl add-port br-int qvo02 ip link set qvb02 up ip link set qvo02 up ovs-vsctl set port qvo02 tag=1 ip tuntap add tap02 mode tap ip link set dev tap02 up brctl addif qbr02 tap02
2）准备虚机映像文件
cp cirros-0.3.4-x86_64-disk.img cirros-vm2.img cp cirros-0.3.4-x86_64-kernel cirros-kernel-vm2
3）通过qemu启动虚机vm2
qemu-system-x86_64 -nographic -kernel ./cirros-kernel-vm2 -hda cirros-vm2.img -append "root=/dev/sda1 console=ttyS0" -net nic,macaddr=52:54:00:12:34:61 -net tap,ifname="tap02",script=no,downscript=no
4）虚机vm2启动过程中可以看到dhcp获取的ip信息
udhcpc (v1.20.1) started Sending discover... Sending select for 10.0.0.82... Lease of 10.0.0.82 obtained, lease time 86400
5.2.3创建tenant2虚机vm3
1）在网络节点添加tenant2流表
ovs-ofctl mod-flows br-tun "table=21,dl_vlan=2,actions=strip_vlan,set_tunnel:2,output:2" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=1,table=2,tun_id=2,actions=mod_vlan_vid:2,resubmit(,10)"
2）在网络节点添加tenant2的dhcp agent服务
ovs-vsctl -- --if-exists del-port tap02 -- add-port br-int tap02 -- set interface tap02 type=internal ovs-vsctl --timeout=10 set Port tap02 tag=2 ip netns add qdhcp02 ip netns exec qdhcp02 ip link set lo up ip link set tap02 netns qdhcp02 ip netns exec qdhcp02 ip link set tap02 up ip netns exec qdhcp02 ip -4 addr add 10.0.1.3/24 brd 10.0.1.255 scope global dev tap02 ip netns exec qdhcp02 ip route replace default via 10.0.1.1 dev tap02 ip netns exec qdhcp02 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tap02 --except-interface=lo --dhcp-range=set:tag0,10.0.1.6,10.0.1.90,86400s --dhcp-lease-max=256 --dhcp-option=3,10.0.1.1
3）在计算节点添加虚机tenant2流表
ovs-ofctl mod-flows br-tun "table=21,dl_vlan=2,actions=strip_vlan,set_tunnel:2,output:2" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=1,table=2,tun_id=2,actions=mod_vlan_vid:2,resubmit(,10)"
4）创建vm3挂接的网桥设备
brctl addbr qbr03 ip link set qbr03 up ip link add qvo03 type veth peer name qvb03 brctl addif qbr03 qvb03 ovs-vsctl add-port br-int qvo03 ip link set qvb03 up ip link set qvo03 up ovs-vsctl set port qvo03 tag=2 ip tuntap add tap03 mode tap ip link set dev tap03 up brctl addif qbr03 tap03
5）准备虚机映像文件
cp cirros-0.3.4-x86_64-disk.img cirros-vm3.img cp cirros-0.3.4-x86_64-kernel cirros-kernel-vm3
6）通过qemu启动虚机vm3
qemu-system-x86_64 -nographic -kernel ./cirros-kernel-vm3 -hda cirros-vm3.img -append "root=/dev/sda1 console=ttyS0" -net nic,macaddr=52:54:00:12:34:62 -net tap,ifname="tap03",script=no,downscript=no
7）虚机vm3启动过程中可以看到dhcp获取的ip信息
udhcpc (v1.20.1) started Sending discover... Sending select for 10.0.1.83... Lease of 10.0.1.83 obtained, lease time 86400
5.2.4创建tenant3虚机vm4
1）在网络节点添加tenant3流表
ovs-ofctl add-flow br-int "hard_timeout=0,idle_timeout=0,priority=3,in_port=2,dl_vlan=103,actions=mod_vlan_vid:3,NORMAL" ovs-ofctl add-flow br-eth2 "hard_timeout=0,idle_timeout=0,priority=4,in_port=2,dl_vlan=3,actions=mod_vlan_vid:103,NORMAL"
2）在网络节点添加tenant3的dhcp agent服务
ovs-vsctl -- --if-exists del-port tap03 -- add-port br-int tap03 -- set interface tap03 type=internal ovs-vsctl --timeout=10 set Port tap03 tag=3 ip netns add qdhcp03 ip netns exec qdhcp03 ip link set lo up ip link set tap03 netns qdhcp03 ip netns exec qdhcp03 ip link set tap03 up ip netns exec qdhcp03 ip -4 addr add 10.0.2.3/24 brd 10.0.2.255 scope global dev tap03 ip netns exec qdhcp03 ip route replace default via 10.0.2.1 dev tap03 ip netns exec qdhcp03 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tap03 --except-interface=lo --dhcp-range=set:tag0,10.0.2.6,10.0.2.90,86400s --dhcp-lease-max=256 --dhcp-option=3,10.0.2.1
3）在计算节点添加tenant3流表
ovs-ofctl add-flow br-int "hard_timeout=0,idle_timeout=0,priority=3,in_port=2,dl_vlan=103,actions=mod_vlan_vid:3,NORMAL" ovs-ofctl add-flow br-eth2 "hard_timeout=0,idle_timeout=0,priority=4,in_port=2,dl_vlan=3,actions=mod_vlan_vid:103,NORMAL"
4）创建vm4挂接的网桥设备
brctl addbr qbr04 ip link set qbr04 up ip link add qvo04 type veth peer name qvb04 brctl addif qbr04 qvb04 ovs-vsctl add-port br-int qvo04 ip link set qvb04 up ip link set qvo04 up ovs-vsctl set port qvo04 tag=3 ip tuntap add tap04 mode tap ip link set dev tap04 up brctl addif qbr04 tap04
5）准备虚机映像文件
cp cirros-0.3.4-x86_64-disk.img cirros-vm4.img cp cirros-0.3.4-x86_64-kernel cirros-kernel-vm4
6）通过qemu启动虚机vm4
qemu-system-x86_64 -nographic -kernel ./cirros-kernel-vm4 -hda cirros-vm4.img -append "root=/dev/sda1 console=ttyS0" -net nic,macaddr=52:54:00:12:34:63 -net tap,ifname="tap04",script=no,downscript=no
7）虚机vm3启动过程中可以看到dhcp获取的ip信息
udhcpc (v1.20.1) started Sending discover... Sending select for 10.0.2.84... Lease of 10.0.2.84 obtained, lease time 86400
5.3各个租户虚机创建完成后网络功能验证
5.3.1租户虚机网络环境验证
1）在vm1中ping dhcp
$ ping 10.0.0.3 PING 10.0.0.3 (10.0.0.3): 56 data bytes 64 bytes from 10.0.0.3: seq=0 ttl=64 time=10.461 ms 64 bytes from 10.0.0.3: seq=1 ttl=64 time=3.099 ms 64 bytes from 10.0.0.3: seq=2 ttl=64 time=2.730 ms
同时在网络节点qdhcp01命名空间中tap01设备上抓包
[root@ofs-network ~]# ip netns exec qdhcp01 tcpdump -i tap01 -enf 13:29:21.606157 92:7c:c4:bf:6c:20 > 52:54:00:12:34:60, ethertype ARP (0x0806), length 42: Request who-has 10.0.0.81 tell 10.0.0.3, length 28 13:29:21.610128 52:54:00:12:34:60 > 92:7c:c4:bf:6c:20, ethertype ARP (0x0806), length 42: Reply 10.0.0.81 is-at 52:54:00:12:34:60, length 28 13:29:21.620086 52:54:00:12:34:60 > 92:7c:c4:bf:6c:20, ethertype IPv4 (0x0800), length 98: 10.0.0.81 > 10.0.0.3: ICMP echo request, id 23553, seq 5, length 64 13:29:21.620199 92:7c:c4:bf:6c:20 > 52:54:00:12:34:60, ethertype IPv4 (0x0800), length 98: 10.0.0.3 > 10.0.0.81: ICMP echo reply, id 23553, seq 5, length 64 13:29:22.623044 52:54:00:12:34:60 > 92:7c:c4:bf:6c:20, ethertype IPv4 (0x0800), length 98: 10.0.0.81 > 10.0.0.3: ICMP echo request, id 23553, seq 6, length 64 13:29:22.623224 92:7c:c4:bf:6c:20 > 52:54:00:12:34:60, ethertype IPv4 (0x0800), length 98: 10.0.0.3 > 10.0.0.81: ICMP echo reply, id 23553, seq 6, length 64
2）在vm2中ping dhcp
$ ping 10.0.0.3 PING 10.0.0.3 (10.0.0.3): 56 data bytes 64 bytes from 10.0.0.3: seq=0 ttl=64 time=6.101 ms 64 bytes from 10.0.0.3: seq=1 ttl=64 time=2.937 ms 64 bytes from 10.0.0.3: seq=2 ttl=64 time=2.863 ms
同时在网络节点qdhcp01命名空间中tap01设备上抓包
[root@ofs-network ~]# ip netns exec qdhcp01 tcpdump -i tap01 -enf 13:28:06.326290 52:54:00:12:34:61 > 92:7c:c4:bf:6c:20, ethertype ARP (0x0806), length 42: Request who-has 10.0.0.3 tell 10.0.0.82, length 28 13:28:06.326329 92:7c:c4:bf:6c:20 > 52:54:00:12:34:61, ethertype ARP (0x0806), length 42: Reply 10.0.0.3 is-at 92:7c:c4:bf:6c:20, length 28 13:28:23.593235 52:54:00:12:34:61 > 92:7c:c4:bf:6c:20, ethertype IPv4 (0x0800), length 98: 10.0.0.82 > 10.0.0.3: ICMP echo request, id 21761, seq 0, length 64 13:28:23.593297 92:7c:c4:bf:6c:20 > 52:54:00:12:34:61, ethertype IPv4 (0x0800), length 98: 10.0.0.3 > 10.0.0.82: ICMP echo reply, id 21761, seq 0, length 64 13:28:24.595536 52:54:00:12:34:61 > 92:7c:c4:bf:6c:20, ethertype IPv4 (0x0800), length 98: 10.0.0.82 > 10.0.0.3: ICMP echo request, id 21761, seq 1, length 64 13:28:24.595720 92:7c:c4:bf:6c:20 > 52:54:00:12:34:61, ethertype IPv4 (0x0800), length 98: 10.0.0.3 > 10.0.0.82: ICMP echo reply, id 21761, seq 1, length 64
3）在vm1中ping vm2
$ ping 10.0.0.82 PING 10.0.0.82 (10.0.0.82): 56 data bytes 64 bytes from 10.0.0.82: seq=0 ttl=64 time=61.110 ms 64 bytes from 10.0.0.82: seq=1 ttl=64 time=7.910 ms 64 bytes from 10.0.0.82: seq=2 ttl=64 time=4.795 ms
同时在计算节点上vm2虚机qbr02安全组桥上抓包
[root@ofs-compute ~]# tcpdump -i qbr02 -enf 13:31:57.142529 52:54:00:12:34:61 > 52:54:00:12:34:60, ethertype ARP (0x0806), length 42: Request who-has 10.0.0.81 tell 10.0.0.82, length 28 13:31:57.143956 52:54:00:12:34:60 > 52:54:00:12:34:61, ethertype ARP (0x0806), length 42: Reply 10.0.0.81 is-at 52:54:00:12:34:60, length 28 13:32:04.650250 52:54:00:12:34:60 > 52:54:00:12:34:61, ethertype IPv4 (0x0800), length 98: 10.0.0.81 > 10.0.0.82: ICMP echo request, id 24065, seq 0, length 64 13:32:04.650929 52:54:00:12:34:61 > 52:54:00:12:34:60, ethertype IPv4 (0x0800), length 98: 10.0.0.82 > 10.0.0.81: ICMP echo reply, id 24065, seq 0, length 64 13:32:05.652430 52:54:00:12:34:60 > 52:54:00:12:34:61, ethertype IPv4 (0x0800), length 98: 10.0.0.81 > 10.0.0.82: ICMP echo request, id 24065, seq 1, length 64 13:32:05.653128 52:54:00:12:34:61 > 52:54:00:12:34:60, ethertype IPv4 (0x0800), length 98: 10.0.0.82 > 10.0.0.81: ICMP echo reply, id 24065, seq 1, length 64
其他租户虚机同样方法可以验证。
参考资料：
SammyLiu的《Neutron 理解》系列 http://www.cnblogs.com/sammyliu/p/4622563.html
深入理解Neutron -- OpenStack 网络实现 https://www.gitbook.com/book/yeasy/openstack_understand_neutron/details