VDS Best Practices – Blade Server Deployments (Part 5 of 6)

Blade Server in Example Deployment

Blade servers are server platforms that provide higher server consolidation per rack unit along with benefits of lower power and cooling costs. Blade chassis that hosts the blade servers have proprietary architectures and each vendor has its own way of managing resources in the blade chassis. It is difficult to talk about all different blade chassis available in the market and explain their deployments in this document. In this section, we will focus on some generic parameters that customers should consider while deploying VDS in a blade chassis environment.

From the networking point of view all blade chassis provide the following two options:

• Integrated switches: In this option, the blade chassis allows built-in switches to control traffic flow between blade servers within the chassis and external network.
• Pass – Through Technology: This is an alternative method of network connectivity that allows the individual blade servers to communicate directly with external network.

In this design, the integrated switch option is described where the blade chassis has a built-in Ethernet switch. This Ethernet switch acts as an Access layer switch as shown in Figure 1.

This section discusses the deployment where the ESXi host is running on Blade server. Two types of Blade server configuration will be described in the following section

• Blade Server with Two 10 Gigabit Ethernet network adapters
• Blade Server with hardware assisted multiple logical network adapters

For each of the above two configurations, the different VDS design approaches will be discussed.

 

Blade Server with Two 10 Gigabit Ethernet network adapters

This deployment is quite similar to the Rack Server with two 10 Gigabit Ethernet network adapters deployment where each ESXi host was presented with two 10 Gigabit network adapters. As shown in Figure 1, the ESXi host running on a blade server in the blade chassis is also presented with two 10 Gigabit Ethernet network adapters.

Figure 1 Blade Server with 2- 10 Gig NICs

In this section two design options are described; one is a traditional static approach and other one is a VMware recommended dynamic configuration with NIOC and LBT features enabled. These two approaches are exactly similar to the deployment described under Rack Server with two 10 Gigabit network adapter section. Only blade chassis specific design decisions will be discussed as part of this section. For all other VDS and switch related configuration, readers are encouraged to refer the Rack sever with two 10 Gigabit network adapter section of this document.

Design Option 1 – Static Configuration

The configuration of this design approach is exactly same as described in the Design option1 section under Rack server with two 10 Gigabit network adapters. Please refer to the Table 1 below for the dvportgroup configuration details. Let’s take a look at the blade server specific parameters that need attention during the design.

Table 1 Static design configuration

Traffic Type	Port Group	Teaming Option	Active Uplink	Standby Uplink	Unused Uplink
Management	PG-A	Explicit Failover	dvuplink1	dvuplink2	None
vMotion	PG-B	Explicit Failover	dvuplink2	dvuplink1	None
FT	PG-C	Explicit Failover	dvuplink2	dvuplink1	None
iSCSI	PG-D	Explicit Failover	dvuplink1	dvuplink2	None
Virtual Machine	PG-E	LBT	dvuplink1/ dvuplink2	None	None

 

The network and hardware reliability considerations should be incorporated during the blade server design as well. In these blade server designs, customers have to focus on the following two areas:

• High availability of blade switches in the blade chassis
• Connectivity of the blade server network adapters to internal blade switches.

High availability of blade switches can be achieved by having two Ethernet switching modules in the blade chassis. And the connectivity of two network adapters on the blade server should be such that one network adapter is connected to first Ethernet switch module and other network adapter is hooked to the second switch module in the blade chassis.

Another aspect that needs attention in the blade server deployment is the network bandwidth availability across the mid plane of blade chassis and between blade switches and aggregation layer. If there is oversubscription scenario in the deployment then customers have to think about utilizing traffic shaping and prioritization (802.1p tagging) features available in the vSphere platform. The prioritization feature allows customer to tag the important traffic coming out of the vSphere platform. These high priority tagged packets are then treated according to priority by the external switch infrastructure. During congestion scenarios, the switch will drop lower priority packets first and avoid dropping the important high priority packets.

This static design option provides the flexibility to the customers of choosing different network adapters for different traffic types. However, while doing the traffic allocation on limited two 10 Gigabit network adapters administrators end up scheduling multiple traffic types on a single adapter. As multiple traffic types flow through one adapter, the chances of one traffic dominating others goes up. To avoid the performance impact due to the noisy neighbors (dominating traffic type), customers have to utilize the traffic management tools provided in the vSphere platform. One of the traffic management features is NIOC, and that feature is utilized in the design option 2 described below.

 

Design Option 2 – Dynamic Configuration with NIOC and LBT

This Dynamic configuration approach is exactly same as described in the Design option2 section under Rack server with two 10 Gigabit Ethernet network adapters. Please refer to the Table 2 below  the dvportgroup configuration details and NIOC settings. The physical switch related configuration in the blade chassis deployment is the same as described in the rack server deployment. For the blade center specific recommendation on reliability and traffic management please refer to previous section.

Table 2 Dynamic design configuration

Traffic Type	Port Group	Teaming Option	Active Uplink	Standby Uplink	NIOC Shares	NIOC Limits
Management	PG-A	LBT	dvuplink1, 2	None	5	-
vMotion	PG-B	LBT	dvuplink1, 2	None	20	-
FT	PG-C	LBT	dvuplink1, 2	None	10	-
iSCSI	PG-D	LBT	dvuplink1, 2	None	20	-
Virtual Machine	PG-E	LBT	dvuplink1, 2	None	20	-

 

VMware recommends this design option that utilizes the advanced VDS features and provides customer with a dynamic and flexible design approach. In this design, I/O resources are utilized effectively and Service Level Agreements are met based on the shares allocation.

Blade Server with Hardware assisted Logical network adapters (HP-Flex10 like deployment)

Some of the new blade chassis supports traffic management capabilities that allow customers to carve I/O resources. This is achieved by presenting logical network adapters to the ESXi hosts. Instead of two 10 Gigabit Ethernet network adapters, ESXi host now sees multiple physical network adapters that operate at different configurable speeds. As shown in Figure 2, each ESXi host is presented with eight Ethernet network adapters that are carved out of two 10 Gigabit Ethernet network adapter.

Figure 2 Multiple Logical network adapters

This deployment is quite similar to the Rack server with eight 1 Gigabit Ethernet network adapter deployment. However, instead of 1 Gigabit network adapters the capacity of each network adapter is configured at the blade chassis level. In the blade chassis, customers can carve out different capacity network adapters based on the need of each traffic types. For example, if iSCSI traffic needs 2.5 Gigabit of bandwidth, a logical network adapter with that amount of I/O resources can be created on the blade chassis and presented to the blade server.

As for the configuration of the virtual switch VDS and blade chassis switch infrastructure goes, the configuration described in the design option 1 under the Rack server with eight 1 Gigabit network adapters is more relevant for this deployment. The static configuration option described in that design can be applied as is in this blade server environment. Please refer to Table 3 for dvportgroup configuration details and the switch configurations descried in that section for physical switch configuration details.

Table 3 Static Design configuration

Traffic Type	Port Group	Teaming Option	Active Uplink	Standby Uplink	Unused Uplink
Management	PG-A	Explicit Failover	dvuplink1	dvuplink2	3,4,5,6,7,8
vMotion	PG-B	Explicit Failover	dvuplink3	dvuplink4	1,2,5,6,7,8
FT	PG-C	Explicit Failover	dvuplink4	dvuplink3	1,2,5,6,7,8
iSCSI	PG-D	Explicit Failover	dvuplink5	dvuplink6	1,2,3,4,7,8
Virtual Machine	PG-E	LBT	dvuplink7/ dvuplink8	None	1,2,3,4,5,6

 

Now the question is whether NIOC capability adds any value in this specific blade server deployment. NIOC is a traffic management feature that helps in scenarios where multiple traffic types flow through one uplink or network adapter. If in this particular deployment only one traffic type is assigned to a specific Ethernet network adapter then the NIOC feature will not add any value. However, if multiple traffic types are scheduled over one network adapter then customers can make use of NIOC to assign appropriate shares to different traffic types. This NIOC configuration will make sure that the bandwidth resources are allocated to the traffic types and SLA is met.

To illustrate this through an example, Let’s consider a scenario where vMotion and iSCSI traffic is carried over one 3 Gigabit logical uplink. To protect the iSCSI traffic from network intensive vMotion traffic, administrators can configure NIOC and allocate shares to each traffic type. If both traffics are equally important then administrators can configure shares with equal values (10 each). With this configuration, when there is a contention scenario, NIOC will make sure that iSCSI process will get half of the 1Gigabit uplink bandwidth and avoid any impact of vMotion process.

VMware recommends that the Network and Server administrators work closely together while deploying the traffic management features of the VDS and Blade Chassis. A lot of co-ordination is required during the configuration of the traffic management features to achieve the best end-to-end QoS result.

This concludes the different design options for the Rack and Blade server deployments with different network adapter configurations. Would love to get your feedback on these different design options and design guidelines. In the next blog entry I will talk about some operational aspect of VDS. Please stay tuned.