linux平台enthercat,Linux Etherchannel

As always I'm long overdue for an update. I've been busy with other more important things, but I figured since I had to document this for others within my group, here would be the best place since it might benefit others as well.

For today's exercise we'll be creating an EtherChannel between a Red Hat Enterprise 5 server, and a Cisco Catalyst 3750 Switch. This is actually far simpler then it sounds, and can be completed in about ten minutes.

We'll begin with configuring the IEEE 802.3ad Dynamic link aggregation (AKA EtherChannel) on the Red Hat Enterprise Linux server. Begin by logging in via SSH, Telnet or directly on the console itself. I do recommend having access to the console directly, so should anything go wrong and you lose network connectivity you'll be able to easily change things back.

Once logged into the server, switch user to "root" if you're not already logged in as root. Change directory to "/etc" and modify the "modprobe.conf" file using your favorite text editor such as "vi". I personally like using "nano". Add the lines in bold from the example "modprobe.conf" below to your file. Then save your changes and return to the bash prompt.

Sample /etc/modprobe.conf

alias scsi_hostadapter megaraid_sas

alias scsi_hostadapter1 usb-storage

alias eth0 bnx2

alias eth1 bnx2

alias bond0 bonding

options bond0 miimon=100 mode=4 lacp_rate=1

Next we need to create a network script for the "bond0" interface that we defined above in the "modprobe.conf" file. This will be used to configure the network properties for the virtual adapter. Once again, use your favorite text editor to create a new file called "ifcfg-bond0" in the "/etc/sysconfig/network-scripts" directory. In this file you will define the device name used above"bond0", IP address, gateway, network mask etc for the virtual adapter. Below is an example.

Sample /etc/sysconfig/network-scripts/ifcfg-bond0

DEVICE=bond0

BOOTPROTO=none

ONBOOT=yes

NETWORK=192.168.0.0

NETMASK=255.255.255.0

IPADDR=192.168.0.25

USERCTL=no

GATEWAY=192.168.0.1

TYPE=Ethernet

IPV6INIT=no

PEERDNS=yes

When you're done configuring the properties of the virtual adapter, save your changes and exit the editor.

The next step is to modify the network script for each adapter that will be added to the EtherChannel. The adapters that we'll be using in this server are eth0 and eth1. Please note your interfaces may be different, so check before continuing.

Start by modifying "ifcfg-" using your text editor, where is the interface name. In this case my file name is "ifcfg-eth0". Add the proper references to the virtual adapter created above "bond0" and remove any IP information such as IP address, gateway, netmask etc since that information will be handled by the virtual adapter. Below is an example of the "ifcfg-eth0" file. Note the bold items are required for the EtherChannel to function properly.

Sample /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0

BOOTPROTO=none

HWADDR=00:11:22:33:44:55

ONBOOT=yes

MASTER=bond0

SLAVE=yes

TYPE=Ethernet

USERCTL=no

IPV6INIT=no

PEERDNS=yes

Repeat the steps above for each additional interface you add to the Etherchannel.

Sample /etc/sysconfig/network-scripts/ifcfg-eth1

DEVICE=eth1

HWADDR=66:77:88:99:aa:bb

ONBOOT=yes

MASTER=bond0

SLAVE=yes

BOOTPROTO=none

TYPE=Ethernet

USERCTL=no

Now that each physical adapter has been associated to the virtual adapter, it's now time to create the EtherChannel on the Cisco switch that this server is connected to. In this example I'll be using two Cisco Catalyst 3750G switches in a stack. In addition to the added benefit of increased bandwidth that the EtherChannel provides, having multiple switches will add switch level redundancy to the EtherChannel.

Begin by logging into the switch, entering configuration mode, and creating the EtherChannel interface. The EtherChannel interface, also known as a "Port-channel", is a virtual interface. You can use any number from 1-48 that isn't already assigned to another EtherChannel. In this example I'll be using "28". Configure all the properties such as VLAN, switchport mode etc. that you would normally define on the physical interfaces. Please note that the properties defined in the EtherChannel will override any interface configuration you have defined on the physical interfaces this Etherchannel is assigned to.

Switch# config terminal

Switch(config)# interface Port-channel28

Switch(config-if)# description SERVER

Switch(config-if)# switchport access vlan 123

Switch(config-if)# switchport mode access

Switch(config-if)# spanning-tree portfast

Next we'll need to configure the physical interfaces by associating them to the EtherChannel we created above. I like to mirror the configuration of the EtherChannel on the physical interfaces, so in the event the EtherChannel on the far end of the link becomes misconfigured the server will still be accessible. The line in bold is the only required change that needs to be made on the physical interfaces.

Switch(config)# interface GigabitEthernet1/0/8

Switch(config-if)#description SERVER-1

Switch(config-if)#switchport access vlan 123

Switch(config-if)#switchport mode access

Switch(config-if)#channel-group 28 mode passiveSwitch(config-if)#spanning-tree portfast

Repeat the process for each additional interface in the EtherChannel.

Switch(config)# interface GigabitEthernet2/0/8

Switch(config-if)#description SERVER-2

Switch(config-if)#switchport access vlan 123

Switch(config-if)#switchport mode access

Switch(config-if)#channel-group 28 mode passiveSwitch(config-if)#spanning-tree portfast

It's now time to restart the network service on the Linux server. Optionally you can reboot the Linux server if that's easier.

[root@SERVER network-scripts]# service network restart

Shutting down interface eth0:  /etc/sysconfig/network-scripts/ifdown-eth: [  OK  ]

Shutting down interface eth1:  /etc/sysconfig/network-scripts/ifdown-eth: [  OK  ]

Shutting down loopback interface:  [  OK  ]

Bringing up loopback interface:  [  OK  ]

Bringing up interface bond0:  [  OK  ]

Check the status of the EtherChannel on the Linux server. If the EtherChannel is operating properly you should see "Bonding Mode: IEEE 802.3ad Dynamic link aggregation" when you concatenate "/proc/net/bonding/bond0"

[root@SERVER ~]# cat /proc/net/bonding/bond0

Ethernet Channel Bonding Driver: v3.2.4 (January 28, 2008)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation

Transmit Hash Policy: layer2 (0)

MII Status: up

MII Polling Interval (ms): 100

Up Delay (ms): 0

Down Delay (ms): 0

802.3ad info

LACP rate: fast

Active Aggregator Info:

Aggregator ID: 1

Number of ports: 2

Actor Key: 17

Partner Key: 28

Partner Mac Address: 01:23:45:67:89:ab

Slave Interface: eth0

MII Status: up

Link Failure Count: 0

Permanent HW addr: 00:11:22:33:44:55

Aggregator ID: 1

Slave Interface: eth1

MII Status: up

Link Failure Count: 0

Permanent HW addr: 66:77:88:99:aa:bb

Aggregator ID: 1

To check the status of the etherchannel on the Cisco switch, use the "show ether-channel" command.

Switch# show etherchannel port-channel

Group: 28

----------

Port-channels in the group:

---------------------------

Port-channel: Po28    (Primary Aggregator)

------------

Age of the Port-channel   = 12d:18h:33m:47s

Logical slot/port   = 10/28          Number of ports = 2

HotStandBy port = null

Port state          = Port-channel Ag-Inuse

Protocol            =   LACP

Port security       = Disabled

Ports in the Port-channel:

Index   Load   Port     EC state        No of bits

------+------+------+------------------+-----------

0     00     Gi1/0/8  Passive            0

0     00     Gi2/0/8  Passive            0

Time since last port bundled:    7d:07h:50m:28s    Gi2/0/8

Time since last port Un-bundled: 7d:07h:51m:31s    Gi1/0/8

At this point you should have a fully functional EtherChannel. If you have any questions feel free to post comments, or contact me directly.