This is intended to be a table that fairly compares modern high speed interconnects that can span distances exceeding a meter. At present, that includes ATM, FibreChannel, HIPPI, Serialbus, and SCI/LAMP, and soon the emerging interconnects SuperHIPPI and SerialPlus.

Comments and suggestions are welcome. Partially updated 990826.Dave Gustavson

Technology:
  • Pros
  • Cons
ATM
  • Major telecom interest surely will make it the preferred wide area digital interconnect in a few years.
  • Standards will result.
  • High speeds are possible in future.
  • No real standards yet.
  • No low-latency switches, in practice, as telecom doesn't mind 10 microsecond latencies.
  • Serious inefficiencies and latencies arise when an intermediate switch drops a packet due to transient overload if the packet is part of a block of digital data.
  • No low-latency interfaces, all have software driver overheads.
  • No support for caching data to reduce apparent latency.
  • Low speed (155 Mbits typical max today, with a major push now toward slower speeds of 25 or 50 Mbits to be more cost-effective).
HIPPI
  • Interfaces are already available for several computer systems.
  • Switches exist.
  • Serial HIPPI works well (same hardware as serial SCI).
  • Optimized for streaming large volumes of data, not for low-latency dynamic interactive usage.
  • No low-latency interfaces, all have software driver overheads.
  • No support for caching data to reduce apparent latency.
  • Low speed (100 MBytes/s).
SuperHIPPI
  • 8 times faster than HIPPI.
  • Much lower latency than HIPPI (uses short blocks).
  • Los Alamos and SGI are talking about the design.
  • Designed by current HIPPI users.
  • Fresh start without backward compatibility constraints.
  • Different from HIPPI.
FibreChannel
  • There are some disk drives with FibreChannel interfaces.
  • Likely will be a wide variety of computers with FibreChannel interfaces.
  • Switches do exist.
  • Lowest layers of protocol are already standardized, others in progress and likely will become standards soon.
  • Profiles exist to reduce the number of incompatible combinations of options.
  • There is an FC-AE (Avionics Environment) working group.
  • Optimized for streaming large volumes of data, not for low-latency dynamic interactive usage.
  • No low-latency interfaces, all have software driver overheads.
  • No support for caching data to reduce apparent latency.
  • No low-latency switches yet.
  • Very complex protocols.
  • Few Gbit links actually available, since overheads dominate performance already at 255 Mbits.
  • Many "existing" devices still won't communicate because built to different drafts or to different sets of options.
  • Large number of "profiles" to reduce incompatibilities.
SerialBus IEEE Std 1394
(FireWire (tm))
  • Architecture of shared memory makes interfaces low-latency.
  • Address model compatible with SCI's, easy to bridge.
  • Presumed high volumes have set chip prices low.
  • Thin flexible copper cables also carry power for remote small devices.
  • Provides an "isochronous" transfer for audio/video requirements that need predictable latency and bandwidth.
  • AC-coupling breaks ground loops inexpensively.
  • Used commercially in digital video cameras (Sony) and should soon replace SCSI in desktop systems (200 Mbit/s).
  • Disk Drives and other devices are available.
  • Connected system acts like one single bus, multiple links are not independent so do not increase throughput.
  • Speed and distance are bounded (400 Mbits someday, 10's of meters) because of bus arbitration and other architectural requirements.
  • Signal isolation requires a separate "phy" chip, increasing board area and cost.
  • No support for caches.
  • No support for fiber.
SerialPlus
(was "Serial Express")
  • Extends Serialbus to higher speed, longer distances.
  • Extends Serialbus bandwidth using concurrency, scales up like SCI.
  • Transports Serialbus packets unmodified, easy to interface.
  • AC-coupled to eliminate ground current problems.
  • No separate "phy" chip necessary, easily integrated with other circuitry.
  • Arbitrary cabling is allowed, including redundant cables--customer friendly. As long as any device is connected by at least one cable, it will work.
  • Mixed speeds within a connected system are allowed, simplifies product migration/upgrading.
  • Extends SCI to RealTime.
  • Extends SCI to Isochronous.
  • Supports hardware fault retry
  • Supports fail-over, redundant routing paths, redundant cables, hot plugging, nonstop systems.
  • Extensible for cache coherence where needed.
  • Protocols completely compatible with high speed parallel links like SCI's, no speed limits.
  • Common future path for Serialbus and SCI, high volumes bring lower prices.
  • Protocols simpler to implement than either SCI or Serialbus (experience helps the third-generation design).
  • Packet header overhead a few bytes larger than for SCI.
  • 32-bit CRC includes local flow-control bits (unlike SCI, which masks those off), which requires more care in designs to preserve end-to-end CRC coverage.
SCI/LAMP ANSI/IEEE
Std 1596
  • Optimized for random low-latency dynamic traffic, with multiple requesters and multiple responders.
  • Guaranteed delivery.
  • Protocol eliminates deadlock and starvation, guarantees forward progress, while using shared queues/buffers.
  • Distributed-memory architectural model cuts overheads down to fractional instruction compared with other models' multiple instructions or library calls (transfer is handled in hardware as part of a normal load or store instruction).
  • Fully supports use of caches to reduce effective latency, automatically keeps multiple cached copies from being inconsistent.
  • RISC-like protocols, integral transceivers, and narrow links make interfaces simpler than buses, will be very low cost when produced in volume.
  • Fully standardized, stable since 1991, ANSI, IEEE, ISO (international publication awaiting editorial changes).
  • Protocols support interconnecting arbitrary heterogeneous machines, not optimized to one particular architecture.
  • Protocols designed to work equally well for fiber optics or copper, bit serial or any width parallel links, can be intermixed freely.
  • Architecturally friendly to Serialbus and SerialPlus.
  • Serial fiber (125 MBytes/s) G-Link chips easy to use, reliable, 44 km demonstrated in single-mode.
  • Fiber ribbon demonstrated at 1 GByte/s, available commercially though perhaps at lower speeds.
  • There are an IEEE SCI-RT group and an SAE group extending SCI for avionics purposes.
  • Less efficient than FibreChannel or HIPPI for long blocks, especially over long distances.
  • A paradigm shift is required--to think of I/O and networking as memory transfers to possibly remote memory is very difficult for current I/O designers.
  • Only a few vendors have products yet.
  • No early adopters have used SCI over its full range--they either use it as the processor/cache/cluster/memory interface for supercomputer-class MPPs (Sequent, Convex/HP) or as an I/O interconnect (Cray Research, Siemens Nixdorf), nobody has yet done both (though Data General says it is heading in that direction).
  • All vendors to date have positioned SCI for the high performance market instead of the low-cost market.
  • Many implementations appear to be closed, to prevent interconnection by others.
  • Multiple paralleled links and interface chips have to be used if a single node needs high bandwidth (e.g. more than 2 GBytes/s in BiCMOS or 500 MByte/s in CMOS).