myrinet简介

Myrinet is a cost-effective, high-performance, packet-communication and switching technology that is widely used to interconnect clusters of workstations, PCs, servers, blade servers, or single-board computers. Clusters provide an economical way of achieving:

• high performance, by distributing demanding computations across an array of cost-effective hosts. For "tightly coupled" distributed computations, the interconnect must provide high-data-rate and low-latency communication between host processes.
• high availability, by allowing a computation to proceed with a subset of the hosts. The interconnect should be capable of detecting and isolating faults, and of using alternative communication paths.

Conventional networks such as Ethernet can be used to build clusters, but do not provide the performance or features required for high-performance or high-availability clustering. Characteristics that distinguish Myrinet from other networks include:

• Flow control, error control, and "heartbeat" continuity monitoring on every link.
• Low-latency, cut-through switches, with monitoring for high-availability applications.
• Switch networks that can scale to tens of thousands of hosts, and that can also provide alternative communication paths between hosts.
• Network Interface Cards (NICs) that execute firmware to offload protocol processing from the host computer. The firmware interacts directly with host processes ("OS bypass") for low-latency communication, and directly with the network to send, receive, and buffer packets.

Myrinet is an ANSI (American National Standards Institute) Standard -- ANSI/VITA 26-1998. The link and routing specifications are public, published, and open.

Myrinet Components and Software

Myricom supplies Myrinet components and software in two series: Myrinet-2000 and Myri-10G. Myrinet-2000 is a superior alternative to Gigabit Ethernet for clusters, whereas Myri-10G offers performance and cost advantages over 10-Gigabit Ethernet. Myri-10G uses the same physical layers (PHYs: cables, connectors, signaling) as 10-Gigabit Ethernet, and is highly interoperable with 10-Gigabit Ethernet. In fact, Myri-10G NICs are both 10-Gigabit Myrinet NICs and 10-Gigabit Ethernet NICs.

Product series	Myrinet-2000	Myri-10G
Full-duplex data rate for the links, NIC ports, and switch ports	2+2 Gigabits/s	10+10 Gigabits/s
Link cables	LC-connectorized duplex multimode fiber to 200m	Selected 10-Gigabit Ethernet cables, copper and fiber
NICs	Single-port and dual-port PCI-X	Single-port PCI-Express, dual-protocol 10G Myrinet or 10G Ethernet
Switches	Based on 16-port and 32-port crossbar switches	Based on 16-port crossbar switches
Switch networks	Up to 256 host ports with a single "Network in a Box" component, and up to tens of thousands of hosts by combining these components	Up to 128 host ports with a single "Network in a Box" component, and up to tens of thousands of hosts by combining these components
Interoperability	Gigabit Ethernet	10-Gigabit Ethernet
Myrinet software support	Myrinet Express (MX-2G) or GM-2	Myrinet Express (MX-10G)
MX or MPI latency	2.6µs–3.2µs	2µs
MX unidirectional data rate	247 MBytes/s (one-port NICs) 495 MBytes/s (two-port NICs)	1.2 GBytes/s
TCP/IP (MX ethernet emulation) data rate	1.98 Gbits/s (one-port NICs) 3.95 Gbits/s (two-port NICs)	9.6 Gbits/s

Myrinet-2000 and Myri-10G employ the same network architecture and protocols, and, when used with MX software support, are fully compatible with respect to applications and application-programming interfaces.

Myricom supplies Myrinet software support for most common hosts and operating systems. The software is supplied "open source," and other Myrinet software is available from third parties.

You or an integrator install the NICs and software in the hosts, and connect the network with cables and switches. The software maps the network, and uses whatever communication paths are available from host to host. No switch programming or routing-table configuration is necessary.

Software Interfaces

Myrinet packets may be of any length, and thus can encapsulate other types of packets, including IP packets, without an adaptation layer. Each packet is identified by type, so that a Myrinet, like an Ethernet, may carry packets of many types or protocols concurrently. Thus, Myrinet supports several software interfaces.

Myricom supplies two alternative low-level message-passing systems for Myrinet: GM, which is used in most Myrinet clusters today, and a new system called MX (Myrinet Express). Other software interfaces such as MPI, Sockets, DAPL, VI, and PVM are layered efficiently over GM or MX, and are available from Myricom and from third parties. Both GM and MX provide "ethernet emulation," in which any protocols carried over ethernet, typically TCP/IP and UDP/IP, can be carried over Myrinet. The GM and MX systems provide protected user-level access to the Myrinet (secure in multi-user, multiprogramming environments); reliable, ordered delivery of messages; network mapping and route computation; and other features that support robust and error-free communication. The Software and Customer Support page provides a complete outline of Myrinet software, with links to documentation, and the Performance Measurements page shows the performance achieved with the various software interfaces.

Technology and Reliability

Myrinet components are implemented with the same advanced technology -- full-custom-VLSI CMOS chips -- as today's workstations, servers, PCs, and single-board computers. This use of CMOS technology is one reason why Myrinet performance has advanced and will continue to advance in step with advances in the hosts, without changes to the network architecture and software interfaces.

These CMOS-based Myrinet components are also extremely reliable. The MTBF of current-production Myrinet switches and NICs exceeds 5 million hours per port. Myrinet exhibits a very low bit-error rate, and is highly robust with respect to host, switch, and cable faults. Myrinet can map itself continuously, and use alternate routes to circumvent faults (e.g., disconnections and powering-down). The hardware computes and checks a CRC for each packet on each link. The NICs provide parity checking both in their memory and across host IO buses.

Reality

Myrinet is the clear market leader in high-performance, high-availability, cluster interconnect. Myricom shipped its first Myrinet products in August 1994. Including the installations supplied by Myricom's OEM customers and by Myrinet resellers and integrators, there are now many thousands of Myrinet installations, ranging in size to more than 2,000 hosts and more than 4,000 processors. These sites include many of the world's premier cluster-computing systems. A total of 141 (28.2%) of the June-2005 TOP500 supercomputers use Myrinet technology.