执行ifconfig eth2 up命令报错eth2: unknown interface: No such device的解决思路

排查问题思路

一般出现这种状况都是网卡mac地址错误引起的！要么网卡配置文件中的mac地址不对，要么/etc/udev/rules.d/70-persistent-net.rules文件中的mac地址不对！！！

问题现象描述

• bond网卡地址ping不通；
• 交换机侧看对应端口状态如下(无关信息省略)

<CL202-R04F02-H3CS7610-SW01>display interface Ten-GigabitEthernet 1/2/0/4
Ten-GigabitEthernet1/2/0/4
Current state: UP
Line protocol state: UP
IP packet frame type: Ethernet II, hardware address: 7057-bf25-8a00
......

<CL202-R04F02-H3CS7610-SW01>display interface Ten-GigabitEthernet 2/2/0/4
Ten-GigabitEthernet2/2/0/4
Current state: UP
Line protocol state: DOWN(LAGG)
IP packet frame type: Ethernet II, hardware address: 7057-bf24-b800
......

• 在配置bond的两张网卡上执行ifconfig eth2 up和ifconfig eth3 up都报类似的错：eth2: unknown interface: No such device；

故障分析定位

• 从故障现象描述第3条手动UP网卡的报错信息以及交换机侧看对应端口的信息，基本可以排除是交换机侧的故障和物理链路故障，主要排查服务器侧的故障；一般此问题是服务器网卡的MAC地址不对造成的。

故障排查过程

• 查看网卡
  如下，我们可以看到系统中有4张网卡，eth0、eth1、eth2和eth3：

[root@CL202-R04F05-NRGL-MONGODB-SLAVE-NF5180M5-SV01 network-scripts]# ll ifcfg-*
-rw-r--r--. 1 root root 196 Mar 23 15:34 ifcfg-bond0
-rw-r--r--  1 root root 328 Mar 23 21:02 ifcfg-eth0
-rw-r--r--. 1 root root 212 Mar 23 15:30 ifcfg-eth1
-rw-r--r--  1 root root 117 May  7 16:58 ifcfg-eth2
-rw-r--r--  1 root root 117 May  7 16:58 ifcfg-eth3
-rw-r--r--. 1 root root 254 Apr 27  2018 ifcfg-lo
[root@CL202-R04F05-NRGL-MONGODB-SLAVE-NF5180M5-SV01 network-scripts]# 

• 查看/etc/udev/rules.d/70-persistent-net.rules文件内容如下

[root@CL202-R04F05-NRGL-MONGODB-SLAVE-NF5180M5-SV01 network-scripts]# more /etc/udev/rules.d/70-persistent-net.rules
# This file was automatically generated by the /lib/udev/write_net_rules
# program, run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single
# line, and change only the value of the NAME= key.

# PCI device 0x8086:0x1521 (igb)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="50:af:73:2e:5c:37", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

# PCI device 0x8086:0x1521 (igb)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="50:af:73:2e:5c:38", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

# PCI device 0x8086:0x37d3 (i40e)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="6c:92:bf:c5:a8:28", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"

# PCI device 0x8086:0x37d3 (i40e)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="6c:92:bf:c5:a8:29", ATTR{type}=="1", KERNEL=="eth*", NAME="eth3"

# PCI device 0x8086:0x37d3 (i40e)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="6c:92:bf:a3:ac:49", ATTR{type}=="1", KERNEL=="eth*", NAME="eth4"

# PCI device 0x8086:0x37d3 (i40e)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="6c:92:bf:a3:ac:48", ATTR{type}=="1", KERNEL=="eth*", NAME="eth5"
[root@CL202-R04F05-NRGL-MONGODB-SLAVE-NF5180M5-SV01 network-scripts]#

• 发现的问题：在网卡配置文件目录下只有eth0、eth1、eth2和eth3这4张网卡，但是在/etc/udev/rules.d/70-persistent-net.rules文件中发现竟然多了eth4和eth5这2张网卡；并且查看eth2和eth3网卡配置文件时发现其mac地址和/etc/udev/rules.d/70-persistent-net.rules文件中显示的eth2和eth3文件的mac地址不一样；eth2和eth3配置文件内容如下：

[root@CL202-R04F05-NRGL-MONGODB-SLAVE-NF5180M5-SV01 network-scripts]# cat ifcfg-eth2 
DEVICE="eth2"
#HWADDR="6c:92:bf:c5:a8:28"
ONBOOT=yes
BOOTPROTO=none
TYPE=Ethernet
NAME="eth2"
MASTER=bond0
SLAVE=yes
[root@CL202-R04F05-NRGL-MONGODB-SLAVE-NF5180M5-SV01 network-scripts]# 
[root@CL202-R04F05-NRGL-MONGODB-SLAVE-NF5180M5-SV01 network-scripts]# cat ifcfg-eth3
DEVICE="eth3"
#HWADDR="6c:92:bf:c5:a8:29"
ONBOOT=yes
BOOTPROTO=none
TYPE=Ethernet
NAME="eth3"
MASTER=bond0
SLAVE=yes
[root@CL202-R04F05-NRGL-MONGODB-SLAVE-NF5180M5-SV01 network-scripts]# 

• 可以从以上信息看出，eth2和eth3网卡配置文件中的mac地址和/etc/udev/rules.d/70-persistent-net.rules中eth2和eth3中的mac地址不一样；

• 远程登录IPMI查看主机mac地址信息如下图：
  

• 从上述信息可以判定配置文件中eth2和eth3的mac地址信息是错的

造成mac地址错误的原因

之前这台设备报修过，更换过网卡文件，所以网卡的mac地址变了；但是/etc/udev/rules.d/70-persistent-net.rules和网卡配置文件中eth2和eth3的mac地址没有对应更新，而是异常新增了并不存在的eth4和eth5网卡，而实际的bond配置还是使用的eth2和eth3网卡，所以网络异常，手动UP网卡报错unknown interface: No such device。

解决办法

修改网卡配置文件和/etc/udev/rules.d/70-persistent-net.rules，修改后正确配置如下：

• /etc/udev/rules.d/70-persistent-net.rules

[root@CL202-R04F05-NRGL-MONGODB-SLAVE-NF5180M5-SV01 network-scripts]# cat /etc/udev/rules.d/70-persistent-net.rules 
# This file was automatically generated by the /lib/udev/write_net_rules
# program, run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single
# line, and change only the value of the NAME= key.

# PCI device 0x8086:0x1521 (igb)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="50:af:73:2e:5c:37", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

# PCI device 0x8086:0x1521 (igb)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="50:af:73:2e:5c:38", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

# PCI device 0x8086:0x37d3 (i40e)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="6c:92:bf:a3:ac:48", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"

# PCI device 0x8086:0x37d3 (i40e)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="6c:92:bf:a3:ac:49", ATTR{type}=="1", KERNEL=="eth*", NAME="eth3"
[root@CL202-R04F05-NRGL-MONGODB-SLAVE-NF5180M5-SV01 network-scripts]# 

• ifcfg-eth2

[root@CL202-R04F05-NRGL-MONGODB-SLAVE-NF5180M5-SV01 network-scripts]# cat ifcfg-eth2 
DEVICE="eth2"
#HWADDR="6c:92:bf:a3:ac:48"
ONBOOT=yes
BOOTPROTO=none
TYPE=Ethernet
NAME="eth2"
MASTER=bond0
SLAVE=yes
[root@CL202-R04F05-NRGL-MONGODB-SLAVE-NF5180M5-SV01 network-scripts]# 

• ifcfg-eth3

[root@CL202-R04F05-NRGL-MONGODB-SLAVE-NF5180M5-SV01 network-scripts]# cat ifcfg-eth3
DEVICE="eth3"
#HWADDR="6c:92:bf:a3:ac:49"
ONBOOT=yes
BOOTPROTO=none
TYPE=Ethernet
NAME="eth3"
MASTER=bond0
SLAVE=yes
[root@CL202-R04F05-NRGL-MONGODB-SLAVE-NF5180M5-SV01 network-scripts]# 

关键最后一步 --- 重启主机

修改配置文件后，尝试过重启网卡，但是依旧未成功，所以尝试了重启主机后世界豁然开朗，网络马上ojbk。
注：没修改mac地址之前重启网卡也是无效的。