TCP Offload Engine (TOE), is gaining increased attention among users seeking to get the most performance and scalability out of their servers. Microsoft’s release of TCP Chimney Offload1 has helped fuel this awareness. To date, the industry response to Chimney TOE compatible products has been robust. As TOE enters the mainstream, there is a desire to better understand what it is and its capabilities from a basic framework perspective. First, we will look at what TOE is and the factors that shaped its development. We will look at the anatomy of a server and why TOE is of benefit in shaping performance and scalability even in today’s multi-processor, multi-core server architectures. And, with iSCSI’s growing popularity and anticipated growth, we will discuss how TOE is used today in iSCSI deployments and its significance going forward. We will close with a discussion on bandwidth, specifically just how much can be achieved with gigabit Ethernet and contrasting this with 10 gigabit Ethernet.
Introduction to TCP/IP Offload
TCP/IP (Transmission Control Protocol/Internet Protocol) is clearly the defacto standard protocol used by computers throughout the world to communicate. IP contains the necessary information to connect two computers much like a phone number is the information necessary to connect two phones. The network infrastructure uses IP to route data from one computer to another and TCP guarantees the integrity of that data. TCP breaks the data into segments for transmission, and reassembles the data at the receiving end, making sure that the data is in order and error free. TCP requires significant processing to perform these tasks and because of this, by way of background, UDP (User Datagram Protocol) was often used in the local area. Unlike TCP, UDP does not guarantee delivery of the data, but it is much more efficient than TCP. For that reason, many data intensive applications made use of UDP, resulting in a much more efficient transfer of data in noise-free environments. However, UDP failed miserably in noisy error-prone environments. As the Internet grew, it became increasingly difficult to differentiate between data that could be isolated to the local area and data destined for the wide area network. As the processors became more powerful, TCP became the standard method for transporting data for both the local and wide area network.
The availability of TOE has largely negated the need for lightweight protocol alternatives to TCP. TOE offloads TCP processing from the CPU, freeing it to do more of the tasks that actually generate the data to be transferred. TOE is frequently considered to be valuable only when the CPU is not sufficiently powerful to saturate the network connection. While TOE is certainly useful in this regard, there are many other reasons one might want to specify TOE for their servers. To better understand this, we first need to delve a bit further into the anatomy of the server.
The Anatomy of Today’s Server: From TCP to TOE
The Internet has driven major changes in the appearance of the data center. Prior to the Internet, most of the data stored on disk drives was “data processing” data. This data had no use until it was further processed by the CPU. This might include word processing data, database information, etc. All of this data was sent to the CPU, processed (sometimes including interaction with the user) and then returned to the disk drive. With the advent of the Internet and HTTP, the amount of data stored on disk drives has grown by leaps and bounds. All of this data is designed exclusively to be viewed, that is it is not data that requires processing by a CPU. In addition to driving this huge growth in data, the Internet has driven equivalent growth in the data delivery infrastructure. Today, we move orders of magnitude more data than we did prior to the Internet. TCP protocol processing has grown proportionally with data growth as TCP is responsible for transporting this data. So while CPUs have gotten faster, we now have such a large amount of TCP to process that a more efficient method than a general purpose processor is needed. TOE is a method that can save significant cost and energy, both in power and human resource.
A generally understood rule of thumb is that it takes one CPU instruction per bit to process TCP. Stated differently, a one gigahertz CPU would be saturated as it tried to process a one gigabit Ethernet connection. Or, a three gigahertz processor would use 33 percent of its capability to support the same Ethernet connection. If this capability could be returned to the CPU, it could support 33 percent more users before it ran out of gas. TOE can allow the entire computer, including the processor, associated storage, etc. to support significantly more users.
A different way to look at the value of TOE is to consider the function the server is performing and how the application running on the server does its job. For instance, an efficient file server application will spend much more time processing metadata than it will to process TCP protocol. By utilizing TOE, the processor can process twice as much metadata, and, as long as sufficient network bandwidth exists, the server can support twice as many clients. Backup can be made even more efficient than a file server application, due to the orderly processing of data and the efficient metadata processing that backup allows.
With the introduction of multi-processor and multi-core architectures, there is a common misperception that the need for TOE in gigabit environments has passed. After all, if one has spare processor capacity to throw at TCP/IP processing, why invest in TOE? It is certainly true that if a server has substantially more capacity than is required to provide the needed functionality, adding more capability would be unnecessary. However, for all applications where multiple servers are needed to provide the required function, TOE can significantly reduce the number of servers required. Many large I/O intensive jobs fall in this category including file serving, web serving and backup applications. For these applications, TOE can provide significant power, space and capital expense savings. TOE provides value for any application that requires significant TCP/IP processing. Many of these applications result in 50 percent or higher total CPU utilization when allocated to TCP.
Implications for iSCSI and the Path to the Enterprise
As more and more users move to iSCSI for their block-level storage requirements, the requirement for TCP processing is driven to a new level. When iSCSI was first introduced, many in the storage community dismissed it, largely based on the fact that the TCP processing burden on the CPU would keep it from performing adequately. Today, in many areas where iSCSI has been introduced, the processor itself is sufficient to provide excellent performance. However, as iSCSI moves to the enterprise level, TOE will be required.
Consider an enterprise-class file server head connected to a large storage array via iSCSI. With each client request for data, if the data is not cached, there is twice the TCP processing required compared to that of a server connected via Fibre Channel or directly to a set of disk drives. This file server head can easily double its ability to supply data to clients by adding TOE capability. For a large data center with significant file serving requirements, TOE can provide huge savings.
While Microsoft started out providing a desktop operating system, over time it has also provided powerful operating systems that are becoming a significant factor in the enterprise server space. Just witness the market share and growth that Microsoft has captured in the database space with SQL. With the addition of TCP Chimney Offload to Windows, Microsoft is providing levels of performance not available in UNIX or Linux. This will accelerate the adoption ramp of Windows and iSCSI in the enterprise. While it is unlikely that competing operating systems will wait too long to add this functionality in order to maintain their competitive position, Microsoft has already gained a large lead and will undoubtedly be able to capitalize on it.
TOE Adoption Ramps Up
Today we are seeing servers with TOE capability built into the server motherboards. Large server manufacturers are making it widely available to customers as they clearly understand the benefits of TOE and the enterprise performance benefits when combined with Windows TCP Chimney Offload. While this currently requires a separate integrated circuit on the board to provide the function, this capability will soon be included in the Southbridge chip2, which will further significantly reduce the cost to the end user. Systems designers are seeing the benefit of TOE and its importance to server scalability and performance, and recognize its ability to improve the overall efficiency of the server.
Thoughts on Bandwidth
As servers become larger and processors become more powerful, 10 gigabit Ethernet (10GbE) will become increasingly important. Today, however, most servers can only make use of a few gigabits per second of bandwidth, even if the TCP processing requirement has been removed. Through the TCP-provided port aggregation functionality, a few gigabit Ethernet connections (which can be obtained on a single PCIe card) can supply all the capability that can be achieved with a single 10 gigabit Ethernet. For the next few years, until the price of 10GbE comes down, this combination of aggregated gigabit connections and TOE will provide the most cost-effective way of moving large amounts of data. We expect that TOE and iSCSI will continue to be the fastest growing new technologies in the networking space.
In summary, TOE can add tremendous value to users seeking to get the most performance out of their servers. Driven by the growth of the Internet, the load on the server has changed. With the addition of TCP/IP processing and the adoption of TOE, valuable cycles from the server’s CPU can be reduced, allowing the server to support other functions (such as metadata processing in the file server) and enabling it to scale and support more users. Even in today’s multi-processor, multi-core server environments, TOE provides significant benefits. The adoption of iSCSI will continue to increase, fueled in part by Microsoft’s continued success in displacing UNIX servers in the data center as well as the attractiveness that iSCSI offers in price, performance, and simplicity. TOE will increasingly be integrated into servers in the form of dedicated chip sets laid down on the motherboard (LOM), and eventually integrated with the Southbridge itself; supporting front-end, client-facing and back-end, iSCSI SAN-facing TCP offload. Lastly, users do not need to immediately jump to 10GbE in order to achieve higher bandwidth. While 10GbE will certainly contribute to iSCSI adoption, users can benefit from high bandwidth gigabit performance today through the use of multi-port PCI-e TOE adapters that support link aggregation.
Larry Boucher is founder, president and chief executive officer of Alacritech.
www.Alacritech.com
1TCP Chimney Offload is the process of offloading the TCP protocol stack in Microsoft Windows to a network interface card (NIC) while preserving customer investments in applications, manageability and security.
2 The Southbridge is a chip that implements the "slower" capabilities of the motherboard in the construct of a Northbridge/Southbridge architecture delineation. The Southbridge is generally distinguished from the Northbridge by not being directly connected to the processor. Rather, the Northbridge ties the Southbridge to the CPU.
6517
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
