Technology Decription

Applications

New Applications

Gigabit Ethernet vs. ATM

Products

Product Report: chartware, vaporware or hardware?

Caveat

Gigabit Ethernet - A Core Competence Industry Report

by David M. Piscitello

This white paper is revised periodically. Recipients of CORnerstone are automatically notified. To receive CORnerstone, please contact us.

Overview of Gigabit Ethernet

Gigabit Ethernet uses the same frame format, frame size, and when shared, the same (CSMA/CD) protocol as 10 Megabit and 100 Megabit (Fast) Ethernet. The same Simple Network Management Protocol managed objects defined for 10 Mbps Ethernet also apply to Gigabit Ethernet, including remote monitoring (RMON) objects. Proponents of Gigabit Ethernet identify the commonalities of protocol, framing, management and scalability as key factors to consider when evaluating an Ethernet or alternative (e.g., ATM) LAN solutions.

Gigabit Ethernet Technology Description

Gigabit Ethernet operates either switched (full-duplex from switch to switch or switch port to end node) or shared (half-duplex using CSMA/CD). Switched Gigabit Ethernet is appealing for its very high aggregation and low latency characteristics. Shared Gigabit Ethernet may be attractive for very high-performance applications, e.g., collaborative computing or parallel tasking across an Ethernet workgroup, rather than a HIPPI, environment.

Gigabit Ethernet can be operated over Category 5 Unshielded twisted pair, coax, multi-mode or single-mode fiber. The following table describes the maximum recommended cable/fiber distances for what is expected to emerge as the industry (IEEE 802.3z) standard.

Medium
Maximum

distance
Comment
CAT5 UTP
100 meters
Coax
25 meters
wiring closet applications
Multi-mode fiber (50 micron)
300 meters
Use same 850 nm short wavelength optical components, yield 1.063 Gbps.
Multi-mode fiber (62.5 micron)
550 meters
Use long wavelength (1300 nm) optical components, yields 1.250 Gbps
Single mode fiber
2-3 Km
Use long wavelength (1300 nm) optical components, yields 1.250 Gbps

Some companies operate proprietary Gigabit Ethernet at 10 kilometers, and have indicate they expect to push the distance to perhaps 20 kilometers by 1998.

To increase throughput and reduce CPU processing, some companies offer extended or jumbo frames. This feature is proprietary, and usually requires that you use switches and network interfaces from a single vendor, but for certain applications (e.g., NFS operating over unfragemented UDP datagrams) the performance gains realized using jumbo frames can be significant.

Gigabit Applications

The initial applications for, and early deployment experience with Gigabit Ethernet included the following:

  • Switch-to-switch or switch-to-router, full-duplex connections to support considerable aggregation in environments where 10/100 Ethernet switching is prominent and 100 Mbps, FDDI, or OC-3 ATM campus backbone capacities are (nearly) exhausted.
  • High-bandwidth connectivity for server farms, where servers each have a shared or dedicated 100 Mbps Ethernet (or ATM or FDDI) connection, and backbone access to the server farm is operating at or near capacity.
  • High-throughput bridging between the HIPPI workgroup (high-performance computing environments) and the desktop workgroup traditionally connected via Ethernet.

National Network Access Points (NAPs) and similar co-location points for Internet Service Providers served by ATM, FDDI, and SMDS, are likely applications for Gigabit Ethernet.

A Wave of New Gigabit Applications

Gigabit Ethernet is an attractive technology on which to deploy Intranets, Extranets, and enterprise client-server applications on a very large scale. Such environments demand enormous CPU cycles from servers, and vast amounts of storage and bus-like characteristics from LAN technologies. Applications of Gigabit Ethernet in such environments have produced a new wave of acronyms and terminology. Here are three of the more interesting terms you'll come across.

Server-switching is a term used to describe a highly optimized Gigabit Ethernet LAN interconnect for servers that comprise a server farm. By analogy, if conventional LAN switching were a mountain bike, server switching might be a mountain bike customized for Exteme Team bikers. Your basic and Extreme Team mountain bikes might begin with the same standard frame, but the latter is outfitted for the specific purpose of not merely enduring, but excelling, in a race down a mountain at a breakneck pace. If it works for bikes, why not for switching? Server-switching removes the general-purpose features of a conventional LAN switch (multiprotocol routing, packet filtering, and other level 3 functions), replaces them with functions that can enhance server performance (large Ethernet frame support, server load-balancing), and provides NICs for your servers that support these features and offload CPU in the process.

Layer-4 switching? You bet. Alteon Networks uses this term to describe how a server-switch can use layer 3 and layer 4 protocol information to implement a variety of load-balancing schemes. In Alteon's model, the switch behaves like a front-end processor for a group of servers that support a common application. A special IP address that is treated much like a "hunt group address" is DNS-registered. The switch maintains a state-full relationship with all the servers, and establishes connections to the least-busy server based on load information. Curious? Read the white paper at Alteon.

I/O switching is a "NIC-bypass" technique. The conventional LAN I/O path for data on a server is from system bus to a LAN NIC, where the data are Ethernet-framed and forwarded to a switch. I/O switching basically eliminates Ethernet framing between server and switch. A special PCI or SBus NIC is used on the server, and data are passed between server and switch at native bus rates in native I/O requests, i.e., without Ethernet framing. A Gigabit switch that supports I/O switching only Ethernet-frames data emitted by the switch onto LAN media.

The people at Gadzoox define a Storage Area Network (SAN) as "a highly scalable, managed server-storage infrastructure that offers gigabit speed data connectivity, high system availability, extensive fault tolerance and low cost of ownership." SANs provide storage access, server networking and server-storage clustering at gigabit rates over Fibre-Channel Arbitrated Loop (where both SCSI and IP operate over the same medium). From an architectural perspective, I/O switching and SAN have a lot in common, and it's hard to imagine that only Fibre-Channel can accommodate SANs.

Gigabit Ethernet vs. ATM

When asked to summarize the pro's and con's of these technologies, I find the following comparisons between Gigabit Ethernet and ATM most useful:

  • Full-duplex Gigabit Ethernet offers 1.250 Gbps, so you must compare cost per port and service against ATM OC-24.
  • Gigabit Ethernet per port costs of ownership are already lower lower than ATM, primarily due to the ease and familiarity of managing Ethernet.
  • ATM is largely distance-insensitive, whereas Gigabit Ethernet is distance-constrained.
  • The campus that requires substantial voice integration with data may find ATM better suited to meet the task.
  • ATM deployment is unnecessarily complicated by the need to sort through and determine which among the alternatives for operating IP over ATM is appropriate (classic ATM ARP, NHRP, LAN Emulation, MPOA).

ATM gets most of the attention in discussions concerning differentiated communications services, but ATM is not necessarily an entirely end-to-end solution for all campus internets. Switched Ethernet environments, when complemented with sophisticated routing and other end-to-end transport services (RSVP, RTP, IPv6) may perform equally well. Several switch manufacturers offer class of service switching on Ethernet switches. These features allow priority queueing, type of switching selection (cut-through, store-and-forward), configurable back pressure, and latency control.

There is neither sufficient information to dismiss Ethernet and general topology internetworks entirely as media for integrated services, nor sweeping evidence that integrated services can be supported in a scalable and tractable manner.

Adopting the posture of technology agnostic is appropriate when considering the relative merits of ATM and Gigabit Ethernet. (You may find my column at Interop OnLine relevant.)

Gigabit Ethernet Products

Gigabit Ethernet is available as an interface on the following hardware:

  • one or several Gigabit Ethernet backbone ports can be used as uplinks for switch-to-switch connections. Switches having such backbone ports come in (modular) chassis and stackable forms; offer 10 Mbps, 100 Mbps, and other legacy LAN interfaces including HIPPI; and offer either level 2 or level 3 VLAN capabilities, RMON and SNMP support. Some switches also support level 3 routing and flow support.
  • a full-duplex repeater of Gigabit Ethernet ports provides high-performance workgroup connections, with SNMP management
  • an extension of server or storage I/O system into a switch backplane (to accommodate higher bandwidth and reduced latency, data are transferred directly from a server/storage I/O bus onto the switch fabric)
  • a PCI and SBus network interface card(s)
  • a router interface card(s) or route switching modules(RSMs)
  • I/O NICs for I/O switching

Products available today are pre-standard IEEE 802.3z. Many vendors offer both a pre-standard and proprietary interface (e.g., an I/O NIC, or server adapter for server-switching). Many vendors commit to compatibility with the eventual standard through an upgrade or replacement guarantee.

Gigabit Ethernet Products: chartware, vaporware or hardware?

The pace of product announcements for Gigabit Ethernet switching is fast and furious. Whereas the products available in May 1997 were of the multi-port repeater and level 2 variety, 4Q97 products are specialized, catering to server-switching, or enterprise-class iron, providing level 3 switching, VLAN capabilities, and fault tolerance to compete with the ATM-centric switches in this class. Modular Gigabit Ethernet Adapters are available for some stackable switches as well.

Gigalabs offers a modular chassis backbone switch product (the Gigastar 3000, a new product evaluation on this product for InfoWorld can be found online. Gigalabs has a companion switch, the Gigastar 100, that can be used to connect servers and systems with HIPPI into the backplane of the Gigastar 3000 using their Gigabit I/O switching interface. Gigalabs has installed Gigabit Ethernet using single-mode fiber over 3 Km distances, and claims to be developing optical-electronics necessary to extend the maximum distance over fiber to between 10-20 Km using their proprietary gigabit Ethernet interface. The most recent product from Gigalabs is an enterprise switch (Gigastar 8000) that supports multicasting in hardware, layer 3 switching and VLAN capabilities.

The ACEswitch100 and compatible Gigabit Ethernet adapters from Alteon Networks fall into the server-switching solution category previously described. The Alteon "network switch/adapter combo" solution provides load-balancing across servers (see layer 4 switching). ACEswitch 100's have eight full- or half-duplex 10/100 Mbps Ethernet ports and two full-duplex Gigabit Ethernet ports and can be cascaded.

Prominet has recently been acquired by Lucent Technologies. The Prominet P550™ Cajun™ switch has performed well in testing by Strategic Networks and Network Computing. This enterprise class switch is VLAN level 3 capable. Hardware integrated multiprotocol routing and multicast support will be available 1Q98. P550™ VLAN implementation is compatible with 3Com.

Extreme Networks has a level 3 VLAN stackable switch line (Summit) that supports multicast routing protocols, RSVP and RIP. Two Summit models support ExtremeWare™, the software through which Extreme Network's flavor of Policy-Based QoS is implemented. Policies are based on user-defined traffic groups (e.g., TCP sessions, UDP/RSVP flows) and QoS profiles for those groups (minimum/maximum bandwidth allowed, relative priorities). Extreme recently announced the Summit Virtual Chassis, a high-speed backplane that connects Summit stackable switches. Virtual chassis is an odd monicker for this product: it's a freestanding unit that provides eight channels, to which you connect Summit switches. What's very newsworthy about this product is the announced price ($8995), a very modest outlay for an external backplane.

3Com has announced a Gigabit Ethernet module for its stackable switches in the SuperStack II series. The module installs into the SuperStack II Switch 3000 Fast Ethernet Switch and operates full-duplex over multi-mode fiber interface. A SuperStack II Switch 9000 SX will 8 full duplex multi-mode fiber switched Gigabit-Ethernet ports. The SuperStack series provides class of service switching and switch method selection (cut-through, store-and-forward, fragment-free, and intelligent). We have evaluated the Switch 1000 and Desktop Switch, and hope to have 3000's to evaluate in the future. A nice feature of the SuperStack series is support for all 8 groups of RMON (many vendors only support RMON-1, four groups).

Packet Engines began shipping a 12-port shared Gigabit Ethernet repeater (FDR12™) and companion NIC's mid-summer. Packet Engines' PE-4884™ Gigabit Routing Switch is an enterprise class switch (first customer ship 1Q98) and claims to provide "wire-speed Layer 3 routing, wire-speed layer 4 filtering and forwarding" of IPv4, IPX, and IPv6 packets based on a set or matrix of layer 3 and 4 attributes (a policy). I can't imagine who in world is pushing information at gigabit rates using IPv6, and to whom, but I'm comforted to hear that switching will be ready when the demand appears:-)

Cisco Systems describes its 1998 rollout of Gigabit Ethernet in a public statement of direction The (ASIC)-based Catalyst LAN switch architecture is expected to provide 100+ Gbps of switching capacity, wireline (gigabit) forwarding using Cisco's NetFlowTM LAN Switching and Cisco IOS™ software technologies. Gigabit Ethernet uplinks, switching modules, and high performance route switch modules will be available for the Catalyst 5000 LAN switch family. Gigabit Ethernet interfaces for the Cisco 7500 and 12000 series router product families will also be available in 1998. Visit Cisco and download a copy of their detailed white paper on Gigabit Networking. The paper discusses the roles of Gigabit Ethernet and ATM as highly complementary. Cisco's position regarding Gigabit Ethernet and ATM is representative of companies that have invested in both technologies and is an interesting contrast to position papers you will find at web sites of companies whose product line is exclusively Gigabit Ethernet.

PCI and SBus NIC's are available from several manufacturers (Extreme, Packet Engines, Gigalabs, Alteon). With pre-standard Gigabit Ethernet, and proprietary extensions for specialized applications (see server-switching and I/O switching) interoperability is a question mark, so early adopters should choose the safe course and use NICs and switches from a single manufacturer.

It still appears that no manufacturer will ship a router interface in 1997.

Caveat

There are now three dozen manufacturers who offer or claim to be developing Gigabit Ethernet solutions. I have not attempted to include every vendor, nor did every vendor I contacted communicate sufficient information to merit discussion.

If you feel your company's products merit discussion, please contact us . We will be only too happy to discuss Gigabit Ethernet technology with you.

The extent to which I can assert accuracy regarding product specifications and product availability is limited to the information provided to me by manufacturers.

Back to the Technology Corner