Comparison of Two Shortcut Routing Schemes: Multi-Protocol Over ATM vs. Fast IP 1 Table of Contents 1 Table of Contents 1 2 Executive Summary 1 3 Background 2 4 Architecture Details 2 4.1 Multi-protocol Over ATM (MPOA) 2 4.1.1 LAN Emulation (LANE) 3 4.1.2 Next Hop Resolution Protocol (NHRP) 4 4.1.3 How MPOA uses LANE and NHRP 4 4.2 Fast IP 5 5 Comparative Analysis 5 5.1 Quality of Service 6 5.2 Price / Performance Benefit 6 5.3 Migration Path / Standards 6 5.4 Protocol Support 7 5.5 Scalability 7 5.6 Filtering / Firewalls 8 6 Conclusions 8 7 References 8 2 Executive Summary During the past year a number of schemes have emerged to solve the router bottleneck problem by avoiding or minimizing packet by packet handling of data by routers. In this paper we study two such schemes: ATM Forum's Multi-Protocol Over ATM and 3Com's Fast IP. Both MPOA and Fast IP are flow based schemes, i.e. shortcut is set up based on the volume of traffic going to a particular end station. Both MPOA and Fast IP are primarily meant for campus LAN environment. MPOA is an ATM only solution while Fast IP also works with packet switched networks. MPOA is based on two core technologies, LANE and NHRP. LANE provides emulated LANs that emulate the services of Ethernet and token ring LANs across an ATM network. NHRP is an address resolution protocol that permits queries between different emulated LANs. NHRP allows MPOA devices in one subnet (VLAN) to set up direct (layer-2) connections to MPOA devices in another subnet thus avoiding hop by hop handling of packets by routers. Like MPOA, Fast IP also uses a variation of NHRP to enable two systems on different VLANs to communicate directly. Both MPOA and Fast IP currently do not have any explicit support for Quality of Service, however both have plans in place for future support. There are some concerns about the scalability of these schemes to WANs which haven't been fully addressed yet. Both these schemes offer significant cost benefits in the campus LAN environment as they can simplify inter-VLAN communications and reduce the role of routers, which should translate into a need for fewer router ports. In both these schemes no changes are required at the core of the network and only the edge devices are affected. Initially both these schemes will only support IP, however they can be extended to use any other network protocol. Networks with filtering and firewalls have to do the "policing" during the setup of the shortcut as there is no way to monitor the network once the shortcut has been established. MPOA , Fast IP, and other shortcut routing schemes are in their evolutionary stages and it will be some time before these schemes stabilize and industry has enough experience to determine which scheme lives up to its promise. 3 Background Over the last year a number of schemes have emerged to tackle the issue of how to deliver the functions traditionally associated with routers in a way that's optimized for switched networks. The goal is to exploit the speed of switches and control function of routers. In networks where layer-2 switches operate at a very high speed, layer-3 slow routers can be a potential bottleneck. To solve this router bottleneck problem many vendors initially introduced schemes that blurred the lines between layer-2 switches and layer-3 routers by providing LAN switches with on-board routing. However during the past year a number of schemes have emerged that take a different approach by avoiding or minimizing packet by packet handling of data by routers. In spring of 1996 Ipsilon Networks introduced IP switching which exploits ATM switching fabric with a new set of IP-oriented protocols. This scheme has a potential of delivering millions of packets per second as compared to hundred of thousands per second throughput of traditional routers. In response to Ipsilon's IP switching several vendors introduced their own solutions all based on the idea of minimizing handling of data packets by routers. Although the ATM Forum has been working on its own solution for some time the announcement by Ipsilon forced them to speed up their work and they came up with the scheme known as Multi-Protocol over ATM (MPOA). Similarly Cisco Systems response to IP switching is Tag switching whereas 3Com has its own solution known as Fast IP. In this paper we will look at MPOA in detail and compare it to the 3Com's Fast IP. We are comparing these two schemes because of their similarities in that both MPOA and Fast IP can simplify inter-VLAN communications at the campus LAN level and reduce the role of routers. Fast IP uses a core protocol (NHRP) used in MPOA. 4 Architecture Details 4.1 Multi-protocol Over ATM (MPOA) Multi-protocol Over ATM (MPOA) is an address-resolution scheme whereby a sending station discovers the ATM address of either the end host or an edge device closest to the destination host so that it can establish a switched connection to the destination bypassing the routers on the way. MPOA supports both layer-2 and layer-3 connectivity between two ATM-attached devices across an ATM fabric. Hence it enables both bridging and routing information to be used to connect to the device closest to the destination. For bridging, MPOA uses a Revised Version of ATM Forum's LAN emulation specification (LANE). LANE allows the bridging of traffic between ATM-attached devices on the same subnetwork as well as supports emulated LANs (VLANs). For routing, MPOA uses the Next Hop Resolution Protocol (NHRP) which is a request response mechanism that allows a source station to determine the network layer (Layer-3) address of the destination device. The primary goal of MPOA is the efficient transfer of inter-subnet unicast data in a LAN environment. MPOA integrates LANE and NHRP to preserve the benefits of LAN Emulation, while allowing intersubnet, network layer protocol communication over ATM virtual channels without requiring routers in the data path. MPOA provides a framework for effectively integrating bridging and routing with ATM in an environment of diverse protocols, network technologies, and VLANs. This framework is intended to provide a unified model for overlaying network layer protocols on ATM. MPOA allows the physical separation of network layer route calculation and forwarding, a technique known as Virtual Routing. Virtual routers are a set of MPOA devices operating over an ATM fabric that collectively provide the functionality of a multi-protocol router. MPOA splits the routing functionality between the route server and edge devices. Edge devices examine the destination address of packets received on legacy LAN segments and decide how to forward those packets. If the packet doesn't need to go outside the subnet, the work of the edge device is done ? it merely bridges the packet, using LAN emulation to resolve the ATM address and establish a virtual circuit to the destination. If the packet must be routed, the edge device examines the destination network layer address of the edge device and looks up the ATM address corresponding to that network layer address. The edge device then establishes a direct virtual circuit to the appropriate destination. The edge device gets the destination ATM address from the route server, which can use various protocols to discover the ATM address of any device in the network. To minimize the number of times an edge device visits the route server, edge devices also maintain their own cache of addresses. This separation of edge devices, which forward packets, and route servers, which supply routing information, has the following key benefits: ? It allows efficient inter-subnet communication. ? It increases manageability by decreasing the number of devices that must be configured to perform network layer route calculation. ? It increases scalability by reducing the number of devices participating in network layer route calculation. ? It reduces the complexity of edge devices by eliminating the need to perform network layer route calculation. The goal of MPOA as mentioned above is to remove the router bottleneck problem. It achieves this by allowing MPOA devices in one subnet (VLAN) to setup direct connections with MPOA devices in another MPOA subnet thus obviating hop by hop handling of packets by routers. The support for VLANs is inherent in the design of MPOA since the ATM fabric is treated like a cloud between the source and destination devices which allows the layer-3 topology to differ from layer-2 topology. This allows network managers to build virtual subnets so users can be grouped together as part of a virtual network regardless of where they are physically located in the network. The next two sections give a brief description of MPOA's core technologies, LANE and NHRP. 4.1.1 LAN Emulation (LANE) The ATM Forum's LAN Emulation (LANE) provides emulated LANs that emulate the services of Ethernet and token ring LANs across an ATM network. LANE provides many benefits including interoperation with Ethernet and token ring hardware and software, allowing a subnet to be bridged across an ATM/LAN boundary. LANE allows a single ATM network to support multiple emulated LANs. By utilizing emulated LANs, network layer protocols may operate over an ATM network in essentially the same way that they operate over Ethernet and token ring LANs. LANE works at the MAC layer and enables legacy Ethernet, token ring, or FDDI traffic to run over an ATM network with no changes to applications, operating system, or network adapters. In version 1.0 of the specification, the ATM Forum defined how a LAN emulation client (LEC) interacts with a LAN emulation service across the User-to-Network Interface (UNI). LANE sits above the ATM adaptation layer (AAL). It masks the connection setup and handshaking functions required by the ATM network from the higher protocol layers, enabling LANE to be independent of upper-layer protocols, services, and applications. It also maps the MAC address based data networking protocols, such as Ethernet and token ring, into ATM virtual connections so that the higher-layer protocols think they are operating on a connectionless LAN. An emulated LAN consists of multiple LECs communicating through the LAN emulation UNI (LUNI) to a single LAN emulation service. A LEC is a combination software and hardware agent within a networking device that handles data forwarding, address resolution, packet-to-cell segmentation and reassembly (SAR), signaling, and other control functions. Each network component, including servers, can support multiple instances of a LEC, allowing multiple emulated LANs to exist simultaneously on the same physical network. The LANE service consists of a LAN emulation server (LES), a broadcast and unknown server (BUS), and a LANE configuration server (LECS). Unlike legacy LANs, which support multipoint-to-mulitpoint broadcast, ATM supports only point-to-point (unicast) and point-to-multipoint (multicast or broadcast) connections. The LES and BUS, which are typically co-located, work together to transfer unicast and broadcast traffic. The LES handles address resolution and control information. Its primary job is to register and resolve MAC addresses to ATM addresses. The LES also enables LECs to join an emulated LAN and register/unregister multiple LAN destinations. The BUS is designed for carrying broadcast data, such as TCP/IP address resolution broadcasts. It also handles all multicast traffic and broadcasts the initial unicast frames sent by the LEC, while the LES works in tandem to provide the appropriate ATM address for establishing a direct virtual channel connection, either switched (SVCs) or permanent (PVCs). The LECS provides connectivity for LECs so they can obtain configuration information, and assigns each LEC to a LES. 4.1.2 Next Hop Resolution Protocol (NHRP) NHRP is an address resolution protocol that permits queries between different subnetworks. It reduces extra router hops required by traditional IP routers. A source station, such as a host or a router, connected to a MPOA subnetwork can use NHRP to determine the layer-3 address and ATM address of the next hop towards a destination station. Like LANE, NHRP is based on a client-server model. An NHRP client initiates NHRP requests to obtain NHRP services. NHRP server provides NHRP services within the ATM cloud. NHRP works as follows. Station "A" (which could be the source or an edge device) wants to send data to station "B" and so needs to resolve station B's layer-3 (IP) address to an ATM address. If B's address resolution information is in A's cache it uses that, otherwise A constructs an NHRP request packet and emits towards the NHRP server. When the NHRP server receives the packet it checks to see if it serves station B. If it does not server station B it forward the request to another NHRP server. The NHRP server which serves station B resolves B's ATM address and generates a positive NHRP reply on B's behalf. If B is not on the ATM network, the next hop layer-3 address will be that of the nearest edge device/router to B through which packets for station B are forwarded. 4.1.3 How MPOA uses LANE and NHRP Devices acting as MPOA clients will initiate communication using LANE. During the normal LANE address resolution process, a LANE client uses the LAN Emulation address resolution protocol (LE-ARP) to gain the information it needs to associate a LAN destination with the ATM address of another client. The LE-ARP response also contains information indicating whether a destination device is MPOA capable or not. If the destination device is MPOA capable, the sending MPOA client will monitor the amount of traffic it's sending to the destination. If the volume of the traffic reaches a pre-defined threshold, the sending MPOA will initiate an NHRP request in order to set up a direct virtual circuit. By tracking the number of frames to a destination over a predetermined timeframe, the client avoids the overhead of setting up shortcuts for short-lived traffic. A number of vendors, including Fore Systems, Cisco Systems, Madge Networks, and IBM, have indicated that they plan to implement MPOA. Fore has a prototype of MPOA implemented at a customer site already and expects to offer customers a full implementation in its ForeThought 5.0 release. Cisco plans to deliver MPOA support in the next major release of its Internetwork Operating System (IOS). Madge will deliver MPOA support in the second half of 1997. 4.2 Fast IP Fast IP provides a shortcut routing solution for packet only networks as well as a solution that will work for both packet and cell based networks. Despite its name, Fast IP is not IP-specific and can be extended to other protocols. Like MPOA, Fast IP will enable two systems on different VLANs to communicate directly, without going through a router, once the initial NHRP request-response is completed. Unlike MPOA, which is an ATM specific solution, Fast IP will work with FDDI, Ethernet, token ring and ATM. NHRP, which was primarily designed for non broadcast networks like ATM, has been changed to work on broadcast networks. Both MPOA and Fast IP are based on NHRP. However in Fast IP when end systems or edge switches issue an NHRP request, the request will include the originating system's MAC address as well as its VLAN ID in the request packet. When the destination system receives it, it will issue an NHRP response directly to the originating system using the originating system's MAC address and VLAN ID found in the NHRP request. Switches along the path of the NHRP response will forward the packet based on either the originating system's MAC address or VLAN ID. In returning the NHRP response, switches in the data path will learn and map the address of the destination system. Like the NHRP, once the originating system receives an NHRP response, it will redirect data packets directly to the destination system using its MAC address, thus bypassing the router. In Fast IP, the intra-subnet communication stays the same. The inter-subnet communication out the same, using the normal controlled layer 3 path. However, once the controlled layer 2 communication is established, Fast IP desktops and servers investigate whether there is a faster, lower latency, layer 2 path available. A distributed Next Hop Resolution Protocol (dNHRP), that unlike MPOA requires no dedicated NHRP server, is supported by peer Fast IP desktops and servers over the controlled layer 3 path. If both end systems support Fast IP and an end-to-end layer 2 path is discovered, the desktop to server communication automatically moves over to the faster lower-latency layer 2 path. If no layer 2 path is discovered, communication continues undisturbed over the original layer 3 path. 5 Comparative Analysis Both MPOA and Fast IP discussed above attempt to address one or more of the shortcomings of traditional routing. Let's look at the similarities and differences of these two approaches. In terms of similarities, both MPOA and Fast IP are flow based schemes, i.e. a shortcut is set up based on the volume of traffic going to a particular end station. Both MPOA and Fast IP have value in the campus LAN environment, and where MPOA will appeal to customers with ATM backbones, Fast IP will appeal to customers with pure packet-based networks. Because it's based on NHRP, a key component of MPOA, Fast IP shares some of the same characteristics as MPOA. For example, Fast IP inherently supports VLANs and is multiprotocol. Unlike MPOA, however, Fast IP can be deployed with legacy packet-based technologies such as Ethernet and token ring. The fact that Fast IP is implemented on packet-based hardware is a plus, since some customers may feel more comfortable maintaining a pure packet-based network. However, the ATM camp is more bullish on connection-oriented switching and featureslike traffic and congestion control, associated with ATM. Although ATM has not turned out to be the "one-size-fits- all" technology that some envisioned, it is a credible alternative to traditional router-based networks. For corporate customers who have deployed ATM backbones, MPOA has clear benefits. The most obvious of these benefits is the ability to communicate between subnetworks using switched connections rather than the traditional hop-by-hop, packet-by-packet router scheme. 5.1 Quality of Service MPOA version 1.0 does not have specifically-defined QoS support. LANE version 2.0 does have mechanisms for associating QoS with LANE connections, and MPOA implementations built on LANE v2 will "inherit" this capability. The MPOA working group expects to address the specific tie-in between MPOA and the IP-based Resource Reservation Protocol (RSVP) in the future. Similarly Fast IP has no explicit QoS support. However, 3Com claims that with Fast IP it has laid the foundation for future support for polices such as QoS. The claim is that functionality should not be added to the devices at the network edge and data centers at the network core to let them monitor traffic flows across the network and make educated guesses based on those traffic flows about what type of service is needed. Instead, desktops and servers should be equipped with the means to tell the network what they need and when they need it and then explicitly tag the associated frames. This will allow networks to implement required policies without adding the complexity of guessing or compromising performance by having to carefully examine details in frames. Fast IP involves desktops and servers to improve the speed and lower the latency of networks by reducing layer 3 routing hops whenever possible. 3Com claims that defining an active role for desktops and servers, as well as streamlining the communication paths through the infrastructure, forms a powerful foundation for adding support for policies such QoS guarantees. Combined with WAN specific schemes like Cascade Corporation's IP Navigator, Fast IP can be extended to provide complete end to end support for QoS. 5.2 Price / Performance Benefit As with all technology decisions, network managers and planners need to weigh the benefits of shortcut routing against the costs. Do these shortcut routing schemes offer cost or feature benefits that make them more attractive than next-generation multi-gigabit routers? High-performance routers are currently being developed by various organizations, including Cisco, start-up companies such as Juniper Networks, and some research organizations. For example, BBN Research is building a 50 gigabit-per-second (Gbps) router that it expects to deliver to the Defense Advanced Research Projects Agency this summer. These high-end routers should solve the congestion problems posed by today's lower-performance routers, and could obviate the need for shortcut routing schemes like MPOA and Fast IP. However, such high-performance routers are likely to cost substantially more per port than shortcut routing alternatives such as Fast IP. If shortcut routing alternatives continue to be more price competitive than standard routers, there will be a market for them. Likewise these shortcut routing schemes offer other benefits that may make them more attractive than standard routers, regardless of their performance. Both MPOA and Fast IP can simplify inter-VLAN communications at the campus LAN level and reduce the role of routers in traffic forwarding, which should translate into a need for fewer router ports. 5.3 Migration Path / Standards In a traditional collapsed backbone based on routers, MPOA doesn't offer significant performance benefits unless you replace your existing routers with ATM switches. On the other hand, if you've already migrated to a backbone based on ATM switches, MPOA offers significant benefits. For organizations that are evolving their campus networks to include a combination of hubs, LAN switches, and routers, Fast IP is worth investigating, particularly if you already use 3Com network interface cards and switches. MPOA requires changes only at the edge but not at the core of the network backbone. MPOA works by moving traffic from an edge device at the entrance to the backbone to the right exit device, rather than making traffic cross the backbone in a hop-by-hop fashion from router to router. Similarly in Fast IP, since the NHRP protocol is supported by Fast IP desktops and servers, no changes to switches and routers are required. Only the Fast IP software needs to be installed on the desktops and servers. Desktops and servers with 3Com's Fast IP software will work with other vendor's switches and routers. Also the risk involved in deploying a given solution should be taken into consideration. Although the specification is just nearing completion, MPOA can be considered moderately stable in that all the leading ATM vendors have LAN emulation (LANE) implementations, and the upgrades to LANE specified in MPOA aren't major. In addition, some vendors, such as Cisco and Fore, already have implemented the Next Hop Resolution Protocol (NHRP), the other major component of MPOA. Fast IP's stability is unknown since there is no wide scale deployment at this point. Organizations evaluating shortcut routing schemes also need to decide how important standards are to them. In general, a standards-based approach offers network managers the best investment protection, since multiple vendors typically support any given standard. Network managers therefore have the flexibility to buy standards-based products from more than one vendor with reasonable assurance they will interoperate. However, due to the explosion of new standards, network managers may find that vendors choose to implement different subsets of standards and that interoperability is compromised. MPOA is clearly standards-based, with a straw ballot by ATM Forum members due in early 1997. 3Com's Fast IP combines three standard technologies, including a core protocol used in MPOA. However, there is no other vendor that has committed to using the same set of standards in quite the same way as 3Com. Officials at 3Com and IBM have indicated that they will provide interoperability between their LAN switches based on at least one of the standards encompassed under Fast IP, but it's unclear what degree of interoperability will be provided. 5.4 Protocol Support MPOA can be viewed as a multilayer switching solution because it encompasses both bridging and routing functions. As a result, MPOA supports both routable and non-routable protocols such as IP, IPX, DECnet, AppleTalk, NetBIOS, and SNA/DLC, without needing to encapsulate them in IP. MPOA allows subnets to extend across an ATM backbone, enabling moves and changes to be carried out without the need for new IP addresses to be assigned. Similarly Fast IP is not IP-specific and can be extended to other protocols. 5.5 Scalability MPOA relies on standard ATM signaling for connections to be established. In addition, address resolution via NHRP must occur before an MPOA-based shortcut can be set up and traffic begin to flow across the backbone. So MPOA is theoretically constrained by the fact that one VC must be set up per shortcut connection. Thus many vendors characterize MPOA as campus-level solutions due to concerns about its ability to scale to very large networks. However, several vendors, including Fore Systems and General DataComm, believe that MPOA can scale to the wide area. For example one solution that gets around the VC problem is to have multiple flows aggregated onto a single connection thus conserving VC resources. Like MPOA, Fast IP is designed for building and campus backbone. To extend Fast IP benefits to Wide Area Networks, 3Com suggests synergy between Fast IP and other schemes like Cascade Corporation's IP Naviagor which address wide area scalability. IP Navigator shares the Fast IP objective of creating fast paths by removing layer 3 routing hops across the network, while maintaining layer 3 controls. IP Navigator addresses the VC scaling issue at layer 1, defining a multipoint-to-point (MPT) tree VC for each site, significantly reducing the number of layer 3 routing hops across the WAN. Therefore, by combining Fast IP and IP Navigator, it should be possible to maintain the control of routing while capturing much more of the speed and low latency of layer 2 and layer 1 switching from the desktop all the way across the WAN to a server at the other end. The debate about which solution is the most scalable will only be settled when proponents of each approach deploy them in WAN applications. Such deployments could begin this year, but it will be 1998 before a clearer picture begins to emerge. 5.6 Filtering / Firewalls Because shortcut routing approaches attempt to limit or avoid packet-by-packet processing, network planners should be aware of how these schemes may affect their use of routing filters for traffic control. Filtering requires a router or other layer-3 device to examine the layer-3 header of a packet. With MPOA, there is no way to "police" a VC once it's been established. Consequently, any filtering must be handled at the edge of the network. For example, an MPOA server (by definition a router) can refuse to forward an NHRP query if the sending station isn't allowed to talk to a particular destination. Similarly in Fast IP inter-subnet communication between two subnets starts out the same, using the normal, controlled layer 3 path. Thus, desired broadcast domain containment and firewall safeguards are preserved across the network. 6 Conclusions Both MPOA and Fast IP provide an easy way to optimize inter-VLAN routing. While MPOA is an ATM specific solution, Fast IP also provides a solution for packet switched networks which allows it to work with FDDI, Ethernet, token ring, and ATM. Both MPOA and Fast IP have value in the campus LAN environment. MPOA will appeal to customers with ATM backbones and Fast IP will appeal to customers with pure packet based networks. Both MPOA and Fast IP are flow based schemes, i.e. a shortcut is set up based on the volume of traffic going to a particular end station. There are some concerns about the scalability of these schemes to Wide Area Networks that haven't been fully addressed yet. Before adopting any of these schemes network managers have to look at a number of factors including support for virtual LANs, quality of service support, protocol support, traffic management, LAN-WAN integration and how these schemes meet future network design requirements. MPOA and Fast IP are two of the many schemes being developed for solving the router bottleneck problem. It will be some time before these schemes stabilize and industry has enough experience to determine which is most viable. Since the industry is in such a volatile state, it's also possible that new routing alternatives will emerge that supersede the current crop of solutions. Alternately, several of these approaches could be collapsed into one hybrid approach. 7 References 1. Multiprotocol Sub-Working Group: "Multiprotocol Over ATM Version 1.0 - Straw Ballot" http://www.cisco.com/public/rfc/ATM/atm-forum/mpoa/ 2. Petrosky, Mary: "The Burton Group Network Strategy Report. Shortcut Routing." 3. Hart, John: "Fast IP: The Foundation for 3D Networking," http://www.3com.com/FastIP/501312.html 4. Andres, Eric: "MPOA Ties It All Together," http://www.data.com/Tutorials/MPOA_Ties_It_All_Together.html