linux bonding 原理,Linux bonding介绍

网卡bond是通过把多个物理网卡绑定为一个逻辑网卡，实现本地网卡的冗余，带宽扩容和负载均衡。

bonding技术的最早应用是在集群——beowulf上，为了提高集群节点间的数据传输而设计的。下面我们讨论一下bonding 的原理, 什么是bonding需要从网卡的混杂(promisc)模式说起。我们知道，在正常情况下，网卡只接收目的硬件地址(MAC Address)是自身Mac的以太网帧，对于别的数据帧都滤掉，以减轻驱动程序的负担。但是网卡也支持另外一种被称为混杂promisc的模式，可以接收网络 上所有的帧，比如说tcpdump，就是运行在这个模式下。bonding也运行在这个模式下，而且修改了驱动程序中的mac地址，将两块网卡的Mac地址改成相同，可以接收特定mac的数据帧。然后把相应的数据帧传送给bond驱动程序处理。

可以通过以下命令确定内核是否支持bonding：

(ENV) [root@ceph-2 ~]# cat /boot/config-`uname -r` | grep -i bonding

CONFIG_BONDING=m

bond配置后网卡信息如下，从输出可以看出系统中配置了两个bond，分别为bond0和bond1，bond0由enp103s0f0和enp103s0f1组成，bond1由ens1f0和ens1f1组成。bond中的网卡具有相同的mac地址。

(ENV) [root@ceph-2 ~]# ip a

1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000

link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

inet 127.0.0.1/8 scope host lo

valid_lft forever preferred_lft forever

inet6 ::1/128 scope host

valid_lft forever preferred_lft forever

2: enp103s0f0: mtu 1500 qdisc mq master bond0 state UP group default qlen 1000

link/ether 5c:c9:99:6a:dc:64 brd ff:ff:ff:ff:ff:ff

3: enp103s0f1: mtu 1500 qdisc mq master bond0 state DOWN group default qlen 1000

link/ether 5c:c9:99:6a:dc:64 brd ff:ff:ff:ff:ff:ff

4: ens1f0: mtu 1500 qdisc mq master bond1 state UP group default qlen 1000

link/ether 88:df:9e:35:bb:e8 brd ff:ff:ff:ff:ff:ff

5: ens1f1: mtu 1500 qdisc mq master bond1 state UP group default qlen 1000

link/ether 88:df:9e:35:bb:e8 brd ff:ff:ff:ff:ff:ff

6: bond0: mtu 1500 qdisc noqueue state UP group default qlen 1000

link/ether 5c:c9:99:6a:dc:64 brd ff:ff:ff:ff:ff:ff

inet 10.0.11.18/24 brd 10.0.11.255 scope global bond0

valid_lft forever preferred_lft forever

inet6 fe80::5ec9:99ff:fe6a:dc64/64 scope link

valid_lft forever preferred_lft forever

7: bond1: mtu 1500 qdisc noqueue state UP group default qlen 1000

link/ether 88:df:9e:35:bb:e8 brd ff:ff:ff:ff:ff:ff

inet 10.0.21.18/24 brd 10.0.21.255 scope global bond1

valid_lft forever preferred_lft forever

inet6 fe80::8adf:9eff:fe35:bbe8/64 scope link

valid_lft forever preferred_lft forever

我们也可以通过查看/sys/class/net/目录下查看网卡信息。其目录下有个常规文件bonding_masters，记录bonding信息。其他的网卡文件都为链接文件。

sysfs is a virtual file system that represents kernel objects as directories, files and symbolic links. sysfs can be used to query for information about kernel objects, and can also manipulate those objects through the use of normal file system commands. The sysfs virtual file system is mounted under the /sys/ directory. All bonding interfaces can be configured dynamically by interacting with and manipulating files under the /sys/class/net/ directory.

(ENV) [root@ceph-2 net]# pwd

/sys/class/net

(ENV) [root@ceph-2 net]# ls

bond0 bond1 bonding_masters enp103s0f0 enp103s0f1 ens1f0 ens1f1 lo

(ENV) [root@ceph-2 net]# cat bonding_masters

bond0 bond1

访问任一bond所对应的目录，可以看到其对应的sys文件系统信息。包括link_mode, operstate, type, speed, address等。如下所示为bond0的信息，也可以查看其下bonding目录下的信息，例如slaves, mode, miimon, mii_status等等。例子中mode=1 表示 fault-tolerance (active-backup)提供冗余功能，工作方式是主从的工作方式，也就是说默认情况下只有一块网卡工作，另一块做备份。

(ENV) [root@ceph-2 bond0]# pwd

/sys/class/net/bond0

(ENV) [root@ceph-2 bond0]# ls

addr_assign_type carrier duplex iflink netdev_group power speed uevent

address carrier_changes flags link_mode operstate proto_down statistics

addr_len dev_id gro_flush_timeout lower_enp103s0f0 phys_port_id queues subsystem

bonding dev_port ifalias lower_enp103s0f1 phys_port_name slave_enp103s0f0 tx_queue_len

broadcast dormant ifindex mtu phys_switch_id slave_enp103s0f1 type

(ENV) [root@ceph-2 bond0]# cat operstate

up

(ENV) [root@ceph-2 bond0]# cat address

5c:c9:99:6a:dc:64

(ENV) [root@ceph-2 bond0]# cat speed

10000

(ENV) [root@ceph-2 bond0]# cat type

1

(ENV) [root@ceph-2 bond0]# cat link_mode

0

(ENV) [root@ceph-2 bond0]# cat iflink

6

(ENV) [root@ceph-2 bond0]# cat bonding/slaves

enp103s0f0 enp103s0f1

(ENV) [root@ceph-2 bond0]# cat bonding/mode

active-backup 1

(ENV) [root@ceph-2 bond0]# cat bonding/active_slave

enp103s0f0

(ENV) [root@ceph-2 bond0]# cat bonding/miimon

100

(ENV) [root@ceph-2 bond0]# cat bonding/mii_status

up

miimon=time_in_milliseconds

Specifies (in milliseconds) how often MII link monitoring occurs. This is useful if high availability is required because MII is used to verify that the NIC is active. To verify that the driver for a particular NIC supports the MII tool, type the following command as root:

~]# ethtool | grep "Link detected:"

In this command, replace interface_name> with the name of the device interface, such as eth0, not the bond interface. If MII is supported, the command returns:

Link detected: yes

If using a bonded interface for high availability, the module for each NIC must support MII. Setting the value to 0 (the default), turns this feature off. When configuring this setting, a good starting point for this parameter is 100.

(ENV) [root@ceph-2 bond0]# ethtool enp103s0f0

Settings for enp103s0f0:

Supported ports: [ FIBRE ]

Supported link modes: 10000baseT/Full

1000baseX/Full

10000baseSR/Full

10000baseLR/Full

Supported pause frame use: Symmetric

Supports auto-negotiation: Yes

Supported FEC modes: Not reported

Advertised link modes: 10000baseT/Full

1000baseX/Full

10000baseSR/Full

10000baseLR/Full

Advertised pause frame use: No

Advertised auto-negotiation: Yes

Advertised FEC modes: Not reported

Speed: 10000Mb/s

Duplex: Full

Port: FIBRE

PHYAD: 0

Transceiver: internal

Auto-negotiation: off

Supports Wake-on: g

Wake-on: g

Current message level: 0x00000007 (7)

drv probe link

Link detected: yes

另外我们也可以通过proc文件系统来查看bond的信息

(ENV) [root@ceph-2 bond0]# ll /proc/net/bonding

total 0

-r--r--r-- 1 root root 0 Mar 14 23:56 bond0

-r--r--r-- 1 root root 0 Mar 14 23:56 bond1

(ENV) [root@ceph-2 bonding]# cat bond0

Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: fault-tolerance (active-backup)

Primary Slave: None

Currently Active Slave: enp103s0f0

MII Status: up

MII Polling Interval (ms): 100

Up Delay (ms): 0

Down Delay (ms): 0

Slave Interface: enp103s0f0

MII Status: up

Speed: 10000 Mbps

Duplex: full

Link Failure Count: 0

Permanent HW addr: 5c:c9:99:6a:dc:64

Slave queue ID: 0

Slave Interface: enp103s0f1

MII Status: down

Speed: Unknown

Duplex: Unknown

Link Failure Count: 0

Permanent HW addr: 5c:c9:99:6a:dc:65

Slave queue ID: 0

网卡的统计信息可以通过/proc/net/dev查看

(ENV) [root@ceph-2 bonding]# cat /proc/net/dev

Inter-| Receive | Transmit

face |bytes packets errs drop fifo frame compressed multicast|bytes packets errs drop fifo colls carrier compressed

bond1: 218380932618639 150903303281 0 254531 0 0 0 102 202196913483854 141663976093 0 0 0 0 0 0

bond0: 4601316958197 4030868434 0 822 0 0 0 4681298 4903772722520 3881581069 0 0 0 0 0 0

enp103s0f0: 4601316958197 4030868434 0 0 0 0 0 4681298 4903772722520 3881581069 0 0 0 0 0 0

lo: 13220872268767 2741169953 0 0 0 0 0 0 13220872268767 2741169953 0 0 0 0 0 0

ens1f0: 205136338566527 141871069749 0 37923 0 0 0 51 188931569389247 132611955898 0 0 0 0 0 0

enp103s0f1: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

ens1f1: 13244594130872 9032233628 0 0 0 0 0 51 13270685429231 9052021096 0 0 0 0 0 0

docker0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

References