1. 环境
一台物理服务器 9.115.251.86，上面创建两个虚机，每个虚机两个网卡：
- vm1：eth0 - 9.*.*.232 eth1：10.0.0.14
- vm2: eth0 - 9.8.*.219 eth1:10.0.0.10，上面运行DHCP Agent，管理 dnsmasq，提供 DHCP 服务
两块 eth1 连到物理机上的一个 bridge 上：
bridge name bridge id STP enabled interfaces mgtbr0 8000.fa8013216b56 no tap0 vnet1 vnet12 vnet13 vnet14 vnet16 vnet18 vnet2 vnet5
2. 错误
vm1 上创建一个虚机 vmchild1，启动时发出 BOOTP 请求去向 DHCP Agent 获取 IP 地址。DHCP Agent 管理的 dnsmasq 可以正常收到 BOOTP 请求，而且发回了请求，但是 vm1 无法收到。
vm2 上 tcpdump：
15:30:14.135874 IP (tos 0x0, ttl 64, id 46594, offset 0, flags [none], proto UDP (17), length 401) 10.0.0.10.40589 > 10.0.0.14.8472: OTV, flags [I] (0x08), overlay 0, instance 1027 IP (tos 0xc0, ttl 64, id 57625, offset 0, flags [none], proto UDP (17), length 351) 50-0-0-10.static.sonic.net.bootps > 50-0-0-15.static.sonic.net.bootpc: BOOTP/DHCP, Reply, length 323, xid 0x11f7631f, secs 60, Flags [none] Your-IP 50-0-0-15.static.sonic.net Server-IP 50-0-0-10.static.sonic.net Client-Ethernet-Address fa:16:3e:ba:43:4c (oui Unknown) Vendor-rfc1048 Extensions Magic Cookie 0x63825363 DHCP-Message Option 53, length 1: Offer Server-ID Option 54, length 4: 50-0-0-10.static.sonic.net Lease-Time Option 51, length 4: 86400 RN Option 58, length 4: 43200 RB Option 59, length 4: 75600 Subnet-Mask Option 1, length 4: 255.255.255.0 BR Option 28, length 4: 50-0-0-255.static.sonic.net Domain-Name Option 15, length 14: "openstacklocal" Domain-Name-Server Option 6, length 4: 50-0-0-10.static.sonic.net Default-Gateway Option 3, length 4: 50-0-0-1.static.sonic.net Classless-Static-Route Option 121, length 13: (60.0.0.0/24:0.0.0.0),(default:50-0-0-1.static.sonic.net) 15:30:14.136118 IP (tos 0xc0, ttl 64, id 10816, offset 0, flags [none], proto ICMP (1), length 429) 10.0.0.100 > 10.0.0.10: ICMP host 10.0.0.14 unreachable - admin prohibited, length 409 IP (tos 0x0, ttl 63, id 46594, offset 0, flags [none], proto UDP (17), length 401) 10.0.0.10.40589 > 10.0.0.14.8472: OTV, flags [I] (0x08), overlay 0, instance 1027
而且发现另外的问题：
root@controller:~/s1# ping 10.0.0.13 PING 10.0.0.13 (10.0.0.13) 56(84) bytes of data. 64 bytes from 10.0.0.13: icmp_seq=1 ttl=64 time=0.630 ms From 10.0.0.13: icmp_seq=2 Redirect Host(New nexthop: 10.0.0.13) 64 bytes from 10.0.0.13: icmp_seq=2 ttl=64 time=0.628 ms From 10.0.0.13: icmp_seq=3 Redirect Host(New nexthop: 10.0.0.13)
3.定位
初步推断是某处防火墙阻止了 BOOTP 的包，于是将 vm1，vm2 上的防火墙全部关掉，无效。最后，查看物理机 iptables：
[root@rh65 ~]# iptables -L Chain INPUT (policy ACCEPT) target prot opt source destination ACCEPT udp -- anywhere anywhere udp dpt:domain ACCEPT tcp -- anywhere anywhere tcp dpt:domain ACCEPT udp -- anywhere anywhere udp dpt:bootps ACCEPT tcp -- anywhere anywhere tcp dpt:bootps ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED ACCEPT icmp -- anywhere anywhere ACCEPT all -- anywhere anywhere ACCEPT all -- anywhere anywhere ACCEPT all -- anywhere anywhere ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:domain ACCEPT udp -- anywhere anywhere state NEW udp dpt:domain ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ftp ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:16509 ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:16514 ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:http ACCEPT tcp -- anywhere anywhere state NEW tcp dpts:vnc-server:cvsup REJECT all -- anywhere anywhere reject-with icmp-host-prohibited Chain FORWARD (policy ACCEPT) target prot opt source destination ACCEPT all -- anywhere 192.168.122.0/24 state RELATED,ESTABLISHED ACCEPT all -- 192.168.122.0/24 anywhere ACCEPT all -- anywhere anywhere REJECT all -- anywhere anywhere reject-with icmp-port-unreachable REJECT all -- anywhere anywhere reject-with icmp-port-unreachable ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED ACCEPT icmp -- anywhere anywhere ACCEPT all -- anywhere anywhere ACCEPT all -- anywhere anywhere ACCEPT all -- anywhere anywhere REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
发现 FORWARD 阻止了 ICMP 相关的包。
4. 解决
在物理机上运行 service iptables stop 将 iptables 关闭，问题解决。