Fibre Channel vs. ATM Executive Summary Both Fibre Channel and ATM offer solutions for solving current and future network bandwidth problems. The technology that a network manager selects should be based on a careful evaluation of not only their cur rent needs, but their future needs. For those users that transmit mostly voice and video ATM is the best choice. For those users that require high raw throughput, and can accept reasonable efficiency in the trans mission of voice and video, Fibre Channel is the better choice. Background Current shared media is beginning to show signs of aging. Currently there is a race between Moore's Law and Amdahl's Law. Moore's Law states that processing power will double every two years while Amdahl's Law states that one Mbps of I/O is required for every MIP of computing power. As more nodes are added to a network using shared media, the bandwidth must not only be shared with the new participants, but the over all bandwidth may actually be reduced due to collisions. Switch technologies avoid this problem by essen tially connecting each node independently. Fibre Channel and ATM are two current technologies which avoid the problem of shared media by using switches. Fibre Channel What is it? Fibre Channel was developed as a general transport vehicle for various protocols (IPI, SCSI, HIPPI, IP, and even ATM) and became an official ANSI standard in 1988. The Fibre Channel standard grew out of the need to support several device protocol communications over channels and networks. Channels are normally be tween a host computer and a device such as a disk drive. These channels normally incur little software over head once the data transmissions start. The main requirement for a channel is that it can transfer a large amount of data at a very high rate over small distances. Networks, on the other hand, normally interface to several users over long distances transferring small amounts of data. A network incurs a higher software overhead. Fibre Channel combines both channel and network protocols into a single protocol. Unlike current networks which are software intensive, Fibre Channel is quite hardware intensive. With trans mission speeds greater than 100 Mbs there is very little time to make routing decisions unless buffers are add ed, which increase congestion. Fibre Channel relies on the header to trigger actions such as routing arriving data to the proper buffer. For Layer 4, see below, requests/responses are stored in the Layer 4 buffers while the data is stored in memory allocated by the task which made the request. Layers Fibre Channel protocol is divided into five layers. These five layers define the transmission rates, encoding scheme, framing protocol, common services, and interfaces to the upper level. Layer 0 - Defines the media, transmitters, receivers and connectors that can be used with Fibre Channel. This layer covers a wide range of performance and cost alternatives which offers designers the ability to select el ements that meet their current need. Layer 1 - Defines the encoding/decoding scheme used to transmit/receive the data. Fibre Channel uses 8b/ 10b as its scheme to transmit the data. Layer 2 - Defines the rules for framing the data. Layer 3 - Provides common services required for advance features such as hunt groups (more than one port can respond to the same alias address). Layer 4 - Provides seamless integration of existing standards. This layer offers Fibre Channel one of its great est strength, its ability to integrate with existing standards. Current communication protocols supported are FDDI, HIPPI, IPI, SCSI, IP, Token Ring, and ATM. Service Fibre Channel was designed to meet a wide range of communication needs and offers three classes of service. Class 1 - Provides a circuit connection. This mode is often referred to as the `selfish mode' since once two node connect, that path cannot be used by another node. Class 2 - Provides a connectionless, frame-linked switch. This mode is often referred to as the `unselfish mode' since other nodes can share the same path. Data delivery is guaranteed with acknowledgment of re ceipt. If delivery cannot be made, a `busy' signal is returned and the host can retry. This is a very nice feature since the host does not have to wait for the transmission to time out before learning of the problem. Class 3 - Provides a connectionless, frame-linked switch with no confirmation. This mode is often referred to as the `hopeful mode'. Intermix is an optional mode which was developed to provide the full Fibre Channel bandwidth offered by a Class 1 connection but also allows for connectionless traffic if their is spare capacity. Finally, a Class 4 is currently being developed to provide for isochronous transmissions (voice, video). Topology Fibre Channel offers three connection methods; Point-to-Point, Switched Fabric, and Arbitrated Loop. Point-to-Point is the simplest connection of the three which uses a single, full duplex cable between two points. This topology offers the greatest possible bandwidth and the lowest latency since there are no inter mediate devices. Switched Fabric offers the greatest connection capability and the largest aggregate throughput of the three topologies. Here each device is connected to a switch and receives a non-blocking path to any other connec tion on the switch. This provides for a dedicated connection to every device. As the number of devices in crease, switches can be connected together. The switched fabric allows the interconnection of a large number of systems, the ability to sustain high bandwidth requirements, and the ability to match speeds between dif ferent connections. The main disadvantages of the switched fabric is the cost and latency of the switch. Arbitrated Loop (FC-AL) offers the logic of the switched topology and distributes it to all devices in the loop. Using this topology up to 126 devices can be connected. In this topology, each device arbitrates for loop ac cess, and once granted, has a dedicated connection between sender and receiver. Available bandwidth is shared between all devices. The main advantage for using FC-AL is cost since no switches are required. The main disadvantages are that each device must operate at the same link speed, and they are inherently ineffi cient in large configurations. The inefficiency arises out of the fact that only two devices can exchange in formation at a time. The main advantage with Fibre Channel is its absence of topology dependencies. Current topologies like To ken Ring, Ethernet, and FIDDI cannot share the same medium since their topologies are different. Fibre Channel avoids the topology dependency problem by operating under a closed system where ports log in with each other and exchange information on attributes and characteristic which allow them to determine commu nication compatibility. If compatible, they will define the communication rules with each other. The man agement issues are no longer the node's responsibility but have been migrated to the switch fabric. ATM What is it? Asynchronous Transfer Method (ATM) has been around since the early 1980's and offers a connection ori ented high performance network. It was originally developed as a standard for WANs. ATM is a switched network capable of supporting multiple connections as a time. At the current time, ATMs main advantage over other technologies is its ability to deliver voice and video at a constant rate. In order provide this high level of isochronous service, data sent over an ATM network is broken up into to cells each consisting of 53 bytes (5 byte header, and a 48 byte payload). By sending fixed packets, switches can be simplified since the size of the incoming/outgoing data is predefined. ATM is connection based and once a connection is made, it remains opened until the connection is broken. The header of an ATM message contains a Virtual Path Identifier (VPI) and a Virtual Circuit Identifier (VCI). In the case of a switch connecting to another switch, there is only a VCI, see topology below. When a con nection is made, a virtual path is setup. This path can either be private (PVC) or shared (SVC). A PVC must be physically setup beforehand and quickly becomes unmanageable as the number of nodes increases. Layers The protocol reference is divided into three layers each of which is discussed below. Physical Layer - defines the bit timing and other characteristics needed for encoding/decoding data. Several media are supported such as optical and twisted pair. ATM Layer - Provides the interface between the adaptation layer, discussed below, and the physical layer. ATM Adaptation Layer - Provides interfaces to higher level protocols. This layer consists of two sub layers, the first, Segmentation and Reassemble (SAR), is responsible for segmenting data from the upper levels into ATM cells and reassembling ATM cells into data for the upper levels. The second, Convergence Layer per forms functions like message identification and clock recovery. The ALL also provides several services dis cussed below. Service The ALL support four classes of service: Class A - Constant Bit Rate (CBR) service. Connection oriented service which has specific timing and delay requirements. Supported by AAL1. Class B - Variable Bit Rage (VBR) service. Connection oriented service in which the bit rate is variable but the delay is bounded. The bounding is required so the recipient can reconstruct the original voice or video data. Supported by AAL2 (still in work). Class C - Connection oriented data service. Variable bit rate and does not require bounded delivery. Sup ported by AAL3/4 and AAL5. Class D - Connectionless data Service. No connection is set up before transfer. Supported by AAL3/4 and AAL5. Topology ATM supports a single topology. A node connects to the ATM network through the user network interface (UNI). Internal nodes connect with other using the network-to-node interface (NNI). Key Issues Although the issues discussed below are not entirely inclusive for making a decision on which technology is better, it does offer significant insight. Bandwidth Bandwidth is an important criteria since it determines how much data a node can transmit. With today's ap plication demanding more and more data throughput, bandwidth is a very important element. Latency Latency is also important since it introduces delay in the transfer of data from one point to the next. If latency is too large many time critical application may be impacted. Fault Tolerance Fault tolerance is important since we are now a 24 hour a day society. In the event of an error, how does the network respond? Cost Cost is always an issue. Does the new technology make fiscal sense? Compatibility With Existing Protocols This is a very important criteria since there is such an abundance of existing network hardware and software. It would probably be infeasible to require everyone to get rid of their existing network hardware and rewrite their existing applications to upgrade to a faster network. Future Expansion Growth rates for networking have been explosive and any new technology must be scalable to meet the new demands. Analysis This section will compare both Fibre Channel and ATM using the key issues discussed above. Bandwidth Fibre Channel operates at speeds up from 132 to 1.062 Gbps with faster speeds (2-4 Gbps currently being reviewed). In contrast, ATM operates at speeds from 51 to 622 Mbps. The current standard for ATM is scal able to 2.5 Gbps (OC-12). Fibre Channel also has the advantage of being full duplex. Each Fibre Channel port has a separate transmit and receive line. Both can effectively operate at the bandwidth of the network essentially doubling the bandwidth. Latency Both Fibre Channel and ATM have very low latency characteristics ( < 30 micro seconds ). However, latency for ATM switches can rise to several hundred microseconds when heavily congested. In a test conducted at the University of Minnesota, Fibre Channel was found to have a 20.6% lower latency than ATM for small message sizes. In the test, end-to-end latency between two nodes was measured. The data packets ranged from 16 bytes to 64 KB. Fault Tolerance Fibre Channel was designed with fault tolerance from the start. Fibre Channel does not rely on an upper layer protocol to guarantee data delivery. Instead it has a built in mechanism that prevents undetected errors and guarantees data reliability and delivery. AMT, on the other hand, has no built in mechanism for guaranteeing delivery, when congestion occurs packets are dropped on a priority bases. If a cell is lost in an ATM network, the entire message must be retransmitted, which leads to higher-level packet retransmissions, which could lead to an exponential increase in traffic through the switch. As an example, suppose we want to send a 1K (21 cells) message across an ATM port and there is a 1% chance that a cell will be lost. Using TCP/IP over ATM, one out of every 5 messages would have to retransmitted. In order to reduce congestion both technologies employ a method for flow control. Fibre Channel uses a cred it based scheme (sliding window),while ATMs is rate based. The credit based scheme ensures that there is enough space at the receiver's end to accept the data. By operating in this manner zero cell loss can be ob tained. ATM, on the other hand, has the source and destination exchange information via a Resource Man agement cell which instructs the source to increase or decrease its data rate when congestion is detected. The problem here is that in order to detect congestion, a cell has to be dropped. As mentioned above, this method may pose problems for TCP/IP. Cost Although not as inexpensive as technologies as Ethernet, the prices for both Fibre Channel and ATM are decreasing every year. Although I do not have current prices, in 1996, for approximately $1500-$1600 per port you could get either ATM running at 155 Mbps or you could get Fibre Channel running at 266 Mbps. For an $2500 per port you could get Fibre Channel running at 1.06 Gbps. Frymoyer notes that by the year two thousand, Fibre Channel switches should be delivering gigabit data rates for about the same price as to day's 10-Mbps Ethernet equipment. If one considers the number of vendors and growth rate as a criteria for which technology will be the most inexpensive (due to competition from various vendors) and wide spread in the future, ATM is the correct choice. In 1995 there were more than 150 ATM equipment vendors compared to Fibre Channel's 50. There were also 33,000 new ATM nodes installed in 1995 compared to 5000 for Fibre Channel. The growth rate for ATM is also increasing faster than Fibre Channel. In 1997, new nodes are predicted to be 440,000 the ATM and only 35,000 for Fibre Channel. Today, over 50% of the Fortune 500 companies have developed ATM implementation plans and almost all major carriers are planning to deploy ATM in their backbone network. Projected revenues are likely to reach $12 billion by 2002. In comparison, Fibre Channel is projected to become a $10 billion/year industry by 2002. Physical connectors are also a factor, Fibre Channel is very flexible with respect to the media it operates on. Although not as flexible, ATM can be engineered to also operate over several medias. Other costs to consider are the increased functionality that both Fibre Channel and ATM offer. Besides data, each can also carry voice and video. Fibre channel can offer additional cost savings since it can also connect to I/O channel such as SCSI and HIPPI. With Fibre Channel, one network can satisfy a companies I/O and networking needs. Compatibility With Existing Protocols If either Fibre Channel or ATM expect to find a way into networking mainstream support for existing proto cols such as IP must exist. Fibre Channel by design does not care what the data is (packet, frame, etc.), it just transports it. As such, Fibre Channel supports many high level protocols for both networks (IP, ATM) and I/O (SCSI, IPI). ATM also supports IP using either Classical IP or LAN Emulation. At this time, LAN Emulation is the meth od of choice. Since ATM is a connection oriented protocol, scalability may become a problem as a greater burden is placed on the setup and tear-down of the connections. ATM working groups are currently defining standards for IP over ATM. A study at the University of Kansas studied how well TCP operated over ATM. The results found that default TCP/IP performance over congested ATM networks was poor. As mentioned earlier, when an ATM network becomes congested, cells start being dropped, at this point TCP/IP will re transmit the message and congestion can become worse. The study did find that strict level ATM flow control will avoid congestion induced cell loss and thus provide excellent throughput on in a LAN. The ATM forum is currently looking at different methods of flow control. Future Expansion Future expansion will come from more users, faster processors, more bandwidth hungry applications, and more distributed computing. With the increase in multimedia applications, networks must also handle voice and video. Both ATM and Fibre Channel offer a much greater bandwidth compared to today's Ethernet so lution and also offer voice and video. ATM already provides for time critical applications while Fibre Chan nel offers a pseudo solution from its intermix service. A problem with this service is that although data is sent out over a circuit service, large packets can be interleaved between the gaps causing delays in transmis sion of the ischronous data. Clearly for this service not to interfere with the class 1 service, class 2 and class 3 message lengths would have to be limited. This problem should be corrected with the proposed level four service dedicated to isochronous data transfers. Finally, in the next few years I/O devices which once required a server will connect directly into the network. Since Fibre Channel can already support channel devices, it is ready and waiting. Conclusion It is evident that a high bandwidth network is required for the future. From my readings, I have determined that Fibre Channel and ATM are not competing technologies, but complementary. Fibre Channel offers a complete I/O and network solution. Applications which once had to keep their SCSI disk within several feet of the server can now be placed at network lengths. Fibre channel has been optimized for high performance data delivery and can also deliver voice and video at a reasonable rate. ATM, on the other hand, has been optimized for voice and video, and also offers reasonable data delivery.