1、原理解析
1.1 LACP模式链路聚合简介
以太网链路聚合是指将多条以太网物理链路捆绑在一起成为一条逻辑链路,从而实现增加链路带宽的目的。链路聚合分为手工模式和LACP模式。
LACP模式需要有链路聚合控制协议LACP的参与。当需要在两个直连设备间提供一个较大的链路带宽而设备支持LACP协议时,建议使用LACP模式。LACP模式不仅可以实现增加带宽、提高可靠性、负载分担的目的,而且可以提高Eth-Trunk的容错性、提供备份功能。
LACP模式下,部分链路是活动链路,所有活动链路均参与数据转发。如果某条活动链路故障,链路聚合组自动在非活动链路中选择一条链路作为活动链路,参与数据转发的链路数目不变。
1.2 bonding技术
bonding(绑定)是一种linux系统下的网卡绑定技术,可以把服务器上n个物理网卡在系统内部抽象(绑定)成一个逻辑上的网卡,能够提升网络吞吐量、实现网络冗余、负载等功能,有很多优势。
bonding技术是linux系统内核层面实现的,它是一个内核模块(驱动)。使用它需要系统有这个模块, 我们可以modinfo命令查看下这个模块的信息, 一般来说都支持.
1.3 华为交换机和Linux服务器对接采用的方式
服务器网卡绑定模式 | 交换机对接方式 | 说明 |
---|---|---|
round-robin | 配置手工模式链路聚合 | 服务器所绑定的网卡被修改成相同的MAC地址,需要交换机通过手工模式链路聚合与之对接。 |
active-backup | 配置对接接口在同一个VLAN | 服务器采用双网卡时,一个处于主状态,一个处于从状态,所有数据都通过主状态的端口传输。当主状态端口对应链路出现故障时,数据通过从状态端口传输。因此交换机对应的两个端口建议配置在同一个VLAN。 |
load balancing | 配置手工模式链路聚合 | 服务器的多网卡基于指定的HASH策略传输数据包,需要交换机配置手工模式链路聚合与之对接。 |
broadcast | 采用两台交换机对接且配置在不同VLAN | 服务器的多网卡对于同一份报文会复制两份分别从两个端口传输,建议使用两台交换机,且配置不同VLAN与之对接。 |
lacp | 配置LACP模式链路聚合 | 服务器的多网卡采用LACP模式绑定,对应交换机应采用LACP模式的链路聚合与之对接。 |
transmit load balancing | 采用两台交换机对接 | 服务器的多网卡采用适配器传输负载均衡,交换机上不需进行配置。 |
adaptive load balancing | 配置手工模式链路聚合 | 服务器的多网卡采用适配器适应性负载均衡,需要交换机配置手工模式链路聚合与之对接。 |
2、S5720交换机配置
2.1 创建Eth-Trunk 15,配置工作模式为LACP
[Sw_hexin]interface Eth-Trunk 15
[Sw_hexin-Eth-Trunk15]bpdu enable
[Sw_hexin-Eth-Trunk15]mode lacp
[Sw_hexin-Eth-Trunk15]quit
2.2 将接口加入Eth-Trunk 15
[Sw_hexin]interface GigabitEthernet 0/0/43
[Sw_hexin-GigabitEthernet0/0/43]eth-trunk 15
[Sw_hexin-GigabitEthernet0/0/43]quit
[Sw_hexin]interface GigabitEthernet 0/0/44
[Sw_hexin-GigabitEthernet0/0/44]eth-trunk 15
[Sw_hexin-GigabitEthernet0/0/44]quit
[Sw_hexin]interface Eth-Trunk 15
[Sw_hexin-Eth-Trunk15] port link-type access
[Sw_hexin-Eth-Trunk15] port default vlan 15
[Sw_hexin-Eth-Trunk15] traffic-policy Security_Policy_15 inbound //加入定义的策略,可以省略
2.3 交换机配置信息
[Sw_hexin]display current-configuration
!Software Version V200R010C00SPC600
#
interface Eth-Trunk15
port link-type access
port default vlan 15
traffic-policy Security_Policy_15 inbound
mode lacp
#
interface GigabitEthernet0/0/43
eth-trunk 15
#
interface GigabitEthernet0/0/44
eth-trunk 15
#
2.4 验证交换机配置
查看Switch设备的Eth-Trunk信息,查看链路是否协商成功;
[Sw_hexin]display eth-trunk 15
Eth-Trunk15's state information is:
Local:
LAG ID: 15 WorkingMode: LACP
Preempt Delay: Disabled Hash arithmetic: According to SIP-XOR-DIP
System Priority: 32768 System ID: 80b5-7587-c2c0
Least Active-linknumber: 1 Max Active-linknumber: 8
Operate status: up Number Of Up Port In Trunk: 2
--------------------------------------------------------------------------------
ActorPortName Status PortType PortPri PortNo PortKey PortState Weight
GigabitEthernet0/0/43 Selected 1GE 32768 1 3889 10111100 1
GigabitEthernet0/0/44 Selected 1GE 32768 2 3889 10111100 1
Partner:
--------------------------------------------------------------------------------
ActorPortName SysPri SystemID PortPri PortNo PortKey PortState
GigabitEthernet0/0/43 65535 dcf4-01e7-e6ce 255 2 9 11111100
GigabitEthernet0/0/44 65535 dcf4-01e7-e6ce 255 1 9 11111100
3、 服务器配置bonding
系统:CentOS Linux release 7.7.1908
网卡:ifcfg-em3 、ifcfg-em4
bond0:192.168.15.15
负载模式: mode4(802.3ad 动态链路聚合)
3.1 关闭和停止NetworkManager服务
[root@svn-zt:/root]# systemctl start NetworkManager.service
[root@svn-zt:/root]# systemctl enable NetworkManager.service
3.2 加载bonding模块
[root@svn-zt:/root]# modprobe bonding
查看模块是否被加载
[root@svn-zt:/root]# lsmod | grep bonding
bonding 152979 0
3.3 创建基于bond0接口的配置文件
[root@svn-zt:/etc/sysconfig/network-scripts]# vim ifcfg-bond0
DEVICE=bond0
NAME=bond0
TYPE=Bond
BONDING_MASTER=yes
IPADDR=192.168.15.15
GATEWAY=192.168.15.1
NETMASK=255.255.255.0
PEERDNS=yes
ONBOOT=yes
BOOTPROTO=static
BONDING_OPTS="resend_igmp=1 updelay=0 use_carrier=1 miimon=100 downdelay=0 xmit_hash_policy=layer3+4 primary_reselect=0 fail_over_mac=0 arp_validate=0 mode=4 lacp_rate=1 arp_interval=0 ad_select=0"
##上面的BONDING_OPTS="mode=4 miimon=100" 表示这里配置的工作模式是802.3ad 动态链路聚合, miimon表示监视网络链接的频度 (毫秒), 我们设置的是100毫秒, 根据你的需求也可以指定mode成其它的负载模式。
3.4 修改ifcfg-em3和ifcfg-em4接口的配置文件
[root@svn-zt:/etc/sysconfig/network-scripts]# vim ifcfg-em3
TYPE=Ethernet
BOOTPROTO=none
NAME=em3
UUID=5737ffd3-4a8a-46b1-ae2e-603127fdf5a4
DEVICE=em3
ONBOOT=yes
MASTER=bond0
SLAVE=yes
-------------------------------------------------------------------------------------
[root@svn-zt:/etc/sysconfig/network-scripts]# vim ifcfg-em4
TYPE=Ethernet
BOOTPROTO=none
NAME=em4
UUID=f0393551-fea5-4ad8-a247-802eae29c2ec
DEVICE=em4
ONBOOT=yes
MASTER=bond0
SLAVE=yes
3.5 编辑/etc/modprobe.d/bond.conf文件
[root@svn-zt:/root]# vim /etc/modprobe.d/bond.conf
alias bond0 bonding
options bond0 miimon=100 mode=4 lacp_rate=1
3.6 重启网络服务
systemctl restart network
或 直接重启系统
reboot
3.7 查看bond0的接口状态信息
[root@svn-zt:/root]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation //绑定模式: 当前是ald模式(mode 4), 也就是802.3ad 动态链路聚合
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
802.3ad info //802.3ad 信息
LACP rate: fast
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: dc:f4:01:e7:e6:ce
Active Aggregator Info:
Aggregator ID: 1
Number of ports: 2
Actor Key: 9
Partner Key: 3889
Partner Mac Address: 80:b5:75:87:c2:c0
Slave Interface: em3
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 4
Permanent HW addr: dc:f4:01:e7:e6:ce
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
system priority: 65535
system mac address: dc:f4:01:e7:e6:ce
port key: 9
port priority: 255
port number: 1
port state: 63
details partner lacp pdu:
system priority: 32768
system mac address: 80:b5:75:87:c2:c0
oper key: 3889
port priority: 32768
port number: 2
port state: 61
Slave Interface: em4
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 3
Permanent HW addr: dc:f4:01:e7:e6:cf
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
system priority: 65535
system mac address: dc:f4:01:e7:e6:ce
port key: 9
port priority: 255
port number: 2
port state: 63
details partner lacp pdu:
system priority: 32768
system mac address: 80:b5:75:87:c2:c0
oper key: 3889
port priority: 32768
port number: 1
port state: 61
3.8 通过ifconfig命令查看下网络的接口信息
[root@svn-zt:/root]# ifconfig
bond0: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST> mtu 1500
inet 192.168.15.15 netmask 255.255.255.0 broadcast 192.168.15.255
inet6 fe80::def4:1ff:fee7:e6ce prefixlen 64 scopeid 0x20<link>
ether dc:f4:01:e7:e6:ce txqueuelen 1000 (Ethernet)
RX packets 17673 bytes 2172638 (2.0 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2003 bytes 240578 (234.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
em1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether dc:f4:01:e7:e6:cc txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 32
em2: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether dc:f4:01:e7:e6:cd txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 54
em3: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST> mtu 1500
ether dc:f4:01:e7:e6:ce txqueuelen 1000 (Ethernet)
RX packets 6352 bytes 806388 (787.4 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 380 bytes 42416 (41.4 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 55
em4: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST> mtu 1500
ether dc:f4:01:e7:e6:ce txqueuelen 1000 (Ethernet)
RX packets 11321 bytes 1366250 (1.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1623 bytes 198162 (193.5 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 61
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 40 bytes 3320 (3.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 40 bytes 3320 (3.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0