Method for recursive BGP route updates in MPLS networks

[0001] The present invention relates generally to communication networks, and more specifically, to a method and system for handling recursive BGP (Border Gateway Protocol) route updates in MPLS (MultiProtocol Label Switching) networks.

[0002] The rapid growth of the Internet and the widespread deployment of networks built around the Internet Protocol suite are creating a demand for new capabilities in IP (Internet Protocol) networks. MPLS provides a number of powerful capabilities such as traffic engineering, etc. As with IP routers, MPLS nodes use a routing protocol such as OSPF or IS-IS to calculate network paths and establish reachability. Multiprotocol Label Switching is an IETF initiative that integrates Layer 2 information about network links (bandwidth, latency, utilization) into Layer 3 (IP) within a particular autonomous system in order to simplify and improve IP packet exchange. MPLS provides network operators a great deal of flexibility to divert and route traffic around link failures, congestion, and bottlenecks. MPLS based networks are becoming increasingly important for today's backbone Internet.

[0003] In MPLS, data transmission occurs on label-switched paths (LSPs). LSPs are a sequence of labels at each and every node along the path from the source to the destination. LSPs are established prior to data transmission (control-driven). The labels, which are underlying protocol-specific identifiers, are distributed using label distribution protocol (LDP) or RSVP or multiprotocol border gateway protocol (MPBGP). Each data packet encapsulates and carries the labels during their journey from source to destination. High-speed switching of data is possible because the fixed-length labels are inserted at the very beginning of the packet or cell and can be used by hardware to switch packets quickly between links.

[0004] With MPLS, incoming packets (predominantly IP packets) are assigned with a label when the packets enter the MPLS network. Packet forwarding is done based on label information within the MPLS core network. In today's routers, packets are typically forwarded with the assistance of ASIC hardware. After an IP packet enters a router, the router decides the next hop or path the packet needs to go by looking up the routing table, typically using the destination address of the IP packet.

[0005] MPLS enabled networks are becoming increasingly important for today's service providers (SPs) in the design and deployment of current and future networks. A primary contributing factor is that MPLS, as an enabling technology, has the capability of converging not only voice, data, and video, but also frame, cell, and packet networks into a single network. Deploying and managing a single and scaleable network is a great benefit to service providers. Furthermore, MPLS networks also allow service providers to do traffic engineering, and quickly reroute customer traffic upon identification of a link failure within their network. This is an important feature for carrier class networks.

[0006] One of the key advantages that MPLS provides is that the vast amount of BGP routes that must be available in every router in the SP network is only needed at the provider edge router (PE). This is accomplished by the combination of BGP (MP-BGP for MPLS/VPN) and LDP (Label Distribution Protocol) running in the core. BGP is a routing protocol that can be used to exchange routing information between different or same autonomous systems. Customer networks usually use an Interior Gateway Protocol (IGP) such as RIP or OSPF for the exchange of routing information within their networks. Customers connect to ISPs and ISPs use BGP to exchange customer and ISP routes. BGP neighbors exchange full routing information when the TCP connection between neighbors is first established. When changes to the routing table are detected, the BGP routers send to their neighbors only those routes that have changed. BGP routers do not send periodic routing updates, and BGP routing updates advertise only the optimal path to a destination network.

[0007] Routers which have been configured to serve the SP network must be capable of switching data traffic at very high rates and processing a large amount of routing information very efficiently. Routers are often expected to recover from a link failure in a very short period of time (e.g., 50 msec) in order to serve the carrier network space. Fast ReRoute is one application of MPLS Traffic Engineering which enables customers to set up a backup network path with certain bandwidth requirements and explicitly route traffic down that path in the event of an interface or link failure.

[0008] Currently, high speed data switching is typically accomplished with the assistance of ASIC based switching engines. When an IP packet enters the PE router, the switching engine looks up its routing table, which may contain several hundred thousand routes. For MPLS/VPN case, the switching engine then pushes two labels (a top label and a bottom label) on top of the IP packet. The top label is an IGP (Interior Gateway Protocol) label for routing the packet across the network core. The bottom label is a VPN label for routing the packet to its final destination when it enters an egress PE router.

[0009] When a customer's IP packet enters the ingress PE of a SP's MPLS/VPN network, the PE's forwarding engine needs to lookup its routing table (using the packet's IP destination address). If there is a match, the packet will carry two labels when it leaves the ingress PE and enters the MPLS core network. Forwarding in ASIC for this scenario typically involves a hardware assisted lookup using the packet's IP destination address. If there is a match, the lookup result provides the forwarding engine with the location of a hardware adjacency entry, where the two labels and outgoing interface information are stored, to rewrite the packets with the correct label information before sending out the packet.

[0010] While traversing the core network to reach its final destination, the packet's IP destination address is no longer used. Instead the top (IGP) label is used to transmit the packet to the egress PE and the bottom (BGP/VPN) label is used for the egress PE to get to the final destination within the customer network. For example, if there are 500,000 different BGP/VPN routes, there will be 500,000 different BGP/VPN labels while there are only a few routes (if load sharing paths are considered) for the ingress PE to get the customer packets to the egress PE.

[0011] The conventional approach for the ingress PE to forward the packet is as follows. Each of the customer routes (BGP/VPN) is programmed in its ASIC FIB (Forwarding Information Base) table. Each route is then associated with an ASIC adjacency entry, where the IGP label, BGP/VPN label, and source/destination MAC are rewritten to the outgoing packet. Their relationship and dependency are summarized in FIG. 1. Prefixes 100.1.1.0/24, 101.1.2.0/24, etc. are BGP routes. There are typically several hundred thousand routes. Labels 10001 and 10002 are BGP/VPN labels. Labels 100 and 200 are IGP labels. In this example, there are two loadsharing paths for each BGP route, represented by label 100 and 200, respectively.

[0012] The conventional approach described above works well for switching packets very fast, even when a large size routing table is present. However, it does not scale when the link between the PE and one of its immediate provider routers fails. This is because it takes a considerable amount of time to reprogram the ASIC switching engine to use a new IGP label to get to an alternative provider router for each of the BGP routes in it its routing table. The size of the BGP table may contain as many as several hundred thousand routes. Thus, latency in updating the ASIC forwarding engine will typically result in a long disruption of customer traffic.

[0013] There is, therefore, a need for a scaleable scheme for a forwarding engine to handle, for example, when a link between a provider edge router and its neighboring provider routers goes down or up and only the IGP routes for the network core (typically a small number of the BGP routes) need to be reprogrammed in the forwarding engine.

[0014] A method for forwarding packets in an MPLS network is disclosed. The method generally comprises receiving a packet at a router and inserting a BGP/VPN label and an internal label which identifies an IGP route. The method further includes creating an entry in a forwarding information table and creating at least one adjacency associated with the internal label in an adjacency table. The packet is recirculated with the BGP/VPN label and internal label. A lookup is then performed using the internal label and the internal label is replaced with an IGP label. The packet is then sent out from the router.

[0015] A method for providing BGP route updates in an MPLS network is also disclosed. The route update is performed at a router having a forwarding information table containing BGP routes and an internal label, and an adjacency table containing BGP/VPN labels and the internal label. The internal label corresponds to at least one IGP route and has an adjacency associated therewith. The method includes updating the adjacency associated with the internal label following an IGP route change.

[0016] In another aspect of the invention, a system for forwarding packets in an MPLS network is configured to reduce route updates. The system generally comprises a forwarding engine having a forwarding information table containing BGP routes and an internal label, and an adjacency table containing BGP/VPN labels and the internal label. The internal label corresponds to at least one IGP route and has an adjacency associated therewith. The forwarding engine is configured to insert a BGP/VPN label and the internal label to incoming packets, recirculate packets containing the BGP and internal labels, perform a lookup using the internal label, replace the internal label with an IGP label, and send out the packet.

[0017] The above is a brief description of advantages of the present invention. Other features, advantages, and embodiments of the invention will be apparent to those skilled in the art from the following description, drawings, and claims.

[0018] FIG. 1 is a diagram illustrating a prior art packet forwarding scheme.

[0019] FIG. 2 is a network diagram illustrating an exemplary MPLS/VPN network topology.

[0020] FIG. 3 is a diagram illustrating labels of an IP packet encapsulated with VPN label and internal label.

[0021] FIG. 4 is a block diagram illustrating a hardware forwarding engine of one of the routers of FIG. 2.

[0022] FIG. 5 is a diagram illustrating a packet forwarding scheme of the present invention.

[0023] FIG. 6 is a flowchart illustrating a method for performing route updates in an MPLS network.

[0024] FIG. 7 is a flowchart illustrating data packet flow while the packet is being switched at the PE router.

[0025] Corresponding reference characters indicate corresponding parts throughout the several views of the drawings.

[0026] The following description is presented to enable one of ordinary skill in the art to make and use the invention. Descriptions of specific embodiments and applications are provided only as examples and various modifications will be readily apparent to those skilled in the art. The general principles described herein may be applied to other embodiments and applications without departing from the scope of the invention. Thus, the present invention is not to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features described herein. For purpose of clarity, details relating to technical material that is known in the technical fields related to the invention have not been described in detail.

[0027] The present invention operates in the context of a data communication network including multiple network elements. Some of the elements in a network that employs the present invention may be network devices such as routers and switches. For example, some of the nodes may be specially configured routers such as those available from Cisco Systems, Inc. of San Jose, Calif. As used herein the term router is used to refer to devices that forward packets based on network and higher layer information. The router may include, for example, a master central processing unit (CPU), interfaces, a hardware assisted forwarding engine, and a bus (e.g., a PCI bus). The CPU preferably includes a memory and a processor. When acting under the control of appropriate software or firmware, the CPU is responsible for such router tasks as routing table computations, network management, and processing of protocol packets. It preferably accomplishes all these functions under the control of software including an operating system (e.g., a version of the Internetwork Operating System (IOS) of Cisco Systems, Inc.) and any appropriate applications software. The CPU may include one or more processors such as a processor from the Motorola family or microprocessors of the MIPS family of microprocessors. In an alternative embodiment, the processor is specially designed hardware for controlling operations of the router. Memory can be non-volatile RAM and/or ROM. However, there are many different ways in which memory could be coupled to the system. In an alternative embodiment, a router or switch may be implemented on a general purpose network host machine such as a computer system.

[0028] The present invention provides a scaleable method and system for hardware recursive BGP route updates in MPLS networks. As previously discussed, with the conventional approach if there is any change in IGP path (e.g., a path change due to FRR or path number change for load sharing paths) the number of hardware adjacency entries for all of the BGP/VPN routes affected need to be updated. In some cases, the number of BGP routes in the PE may be a half million or higher. In this case, the conventional approach would require a large amount of time to update all of the adjacency entries for these routes, resulting in serious customer traffic disruption. As described in detail below, the present invention maps a large amount of BGP routes to a few internal labels so that forwarding entries for BGP routes do not need to be changed when an IGP route change occurs. Thus reducing the time required to perform BGP route updates.

[0029] Referring to FIG. 2, a network that may use a system and method of the present invention is shown. The network includes three customer edge routers (CE1, CE2, CE3) 36, three associated provider edge routers (PE1, PE2, PE3) 38, and five provider routers (P) 42. Each CE 36 is associated with a VPN (VPN1, VPN2, VPN3) 44. In MPLS, data transmission occurs on label-switched paths (LSPs). LSPs are a sequence of labels at each and every node along the path from the source to the destination. Fixed-length labels are inserted at the very beginning of the packet or cell and can be used by hardware to switch packets quickly between links. The routers may be, for example, an edge router 38 or a label switching router (LSR). The LSRs participate in the establishment of LSPs using the appropriate label signaling protocol and high-speed switching of the data traffic based on the established paths. Edge routers 38 operate at the edge of the access network and MPLS network and support multiple ports connected to dissimilar networks (such as frame relay, ATM, and Ethernet) and forwards this traffic on to the MPLS network after establishing LSPs, using the label signaling protocol at the ingress and distributing the traffic back to the access networks at the egress. A label identifies the path a packet should traverse and is carried or encapsulated in a Layer-2 header along with the packet. The receiving router examines the packet for its label content to determine the next hop. Once a packet has been labeled, the travel of the packet through the network backbone is based on label switching.

[0030] It is to be understood that FIG. 2 is only one example illustrating MPLS/VPN network technology and that other types of networks may be used without departing from the scope of the invention. For example, instead of a MPLS/VPN route, a pure BGP route may be implemented using this scheme to achieve high scalability during IGP route change. In this case, there is no VPN label. As described in detail below, the present invention provides a highly scaleable scheme for hardware recursive BGP route updates in MPLS networks.

[0031] The invention described herein may be implemented in dedicated hardware, microcode, software, or photonic (optical) logic. The following describes a hardware implementation utilizing a ASIC based forwarding engine 48 (FIG. 4).

[0032] Hardware forwarding engine 48 uses hardware to forward data packets to achieve high performance. The hardware routing table and adjacency table are programmed with the assistance of software prior to data transmission. When data packets arrive, the hardware forwarding engine 48 looks up the hardware routing table, finds the route entry and adjacency to route the packet and forward the packets accordingly. An IP path for a given route within a router is typically represented by the router's next hop IP address, which is a four-byte value. An MPLS path for a given route is typically represented by an encapsulated byte string, which contains the destination and source MAC addresses and the variable sized MPLS label stack. When an IP packet enters one of the PE routers 38, the forwarding engine 48 looks up its routing table. The forwarding engine 48 then pushes two labels on top of the IP packet 52, as shown in FIG. 3. The top label is a fixed internal label 56 (which replaces the prior art IGP labels) and identifies an IGP route. The bottom label is a BGP/VPN label 54 used for routing the packet to its final destination when it enters the egress PE router 38.

[0033] FIG. 5 illustrates a FIB (Forwarding Information Base) table 60, an adjacency table 62, and a route and adjacency relationship for one embodiment of the forwarding scheme of the present invention. Each of the customer routes (BGP/VPN) is programmed in its FIB table. Each route is then associated with an adjacency entry. Prefixes 100.1.1.0/24, 101.1.2.0/24, and 110.1.2.0/24 are BGP routes. As described above, there may be several hundred thousands of such routes. Labels 10001, 10002, and 12002 are BGP/VPN labels. In the present scheme, conventional IGP labels (100 and 200) (FIG. 1) are replaced with a fixed internal MPLS label 1000000 in each hardware adjacency entry for BGP routes. The internal label is not given to PE's MPLS neighbors for use in normal forwarding purposes.

[0034] In the hardware FIB table, a single entry having the internal label as the lookup key is created. Another hardware adjacency (or multiple adjacency entries for loadsharing paths) is created where the IGP label and MAC information of the PE's neighboring provider router are stored. This hardware adjacency is associated with the internal label based hardware FIB entry.

[0035] Each hardware adjacency of a BGP route tells the forwarding engine 48 to push the BGP/VPN label plus the internal label and then recirculate the packet (FIGS. 2 and 5). This takes place instead of having the adjacency instruct the forwarding engine 48 to forward the packet out of the PE router (as shown in the conventional forwarding scheme of FIG. 1). Packet recirculation can be completed at very high speed with ASIC assistance.

[0036] The packet is now labeled with the internal label sitting on top and the BGP/VPN label in the bottom of the label stack (FIGS. 3 and 5). The labeled packet re-enters the forwarding engine 48 after recirculation and packet lookup matches the internal label. The forwarding engine 48 then uses its adjacency entry to swap the internal label with the IGP label and send out the packet.

[0037] When an IGP route change occurs, only the hardware adjacency associated with the internal label needs to be updated while BGP routes and their corresponding hardware adjacency entries remain the same. Updating a single entry versus hundred thousands of entries becomes very scaleable and as such, SP stringent requirements to restore all network traffic in a fraction of seconds can be met. Thus, each hardware adjacency for BGP routes takes only one hardware adjacency entry even if loadsharing paths are considered. Hardware adjacency for internal label can take multiple hardware adjacency entries.

[0038] FIG. 6 is a flowchart illustrating the above described process for route updates in an MPLS network. At step 100, a BGP/VPN route update arrives. A hardware adjacency associated with this route is created (step 102). The adjacency contains one VPN label and one internal label. Next, an entry for the VPN route in the hardware lookup table is created (step 104). This entry points to the adjacency created at step 102. The internal label is programmed to the hardware lookup table if it is not already there (step 106). Also, an adjacency entry is created for this label to swap the internal label with the IGP label (step 108). The IGP route change arrives at step 110. The adjacency entry is then updated for the internal label (step 112).

[0039] FIG. 7 is a process flowchart illustrating data packet flow while the packet is being switched at one of the PE routers 38. A customer's IP packet enters an ingress PE of a SP's MPLS/VPN network at step 80. The PE's forwarding engine looks up its routing table and finds a match (step 82). Each adjacency of a BGP route then instructs the forwarding engine to push BGP/VPN label and internal label (step 90) and recirculate the packet (step 92). Packet lookup then matches the internal label (step 94). The forwarding engine uses its adjacency entry to swap the internal label with IGP label (step 96) and sends the packet out (step 98).

[0040] It is to be understood that although the above example described how BGP recursive routes for IP to MPLS paths can be handled in a highly scaleable manner, this scheme is also applicable to MPLS to MPLS paths (e.g., as is the case at a carrier's carrier PE), where the interface between PE and CE is MPLS enabled.

[0041] As can be observed from the foregoing, the above described invention provides increased network reliability, availability, and high scalability in MPLS networks. For example, increased reliability, availability and much shorter down time is obtained in the case of link failures, as only hardware adjacency entries for IGP routes need to be updated in the case of IGP route changes. As compared to conventional schemes, which require updates to adjacency entries of all BGP routes, the invention described herein greatly reduces the time required to update the hardware forwarding table. The network traffic is thus disrupted for a small amount of time. This low latency advantage is important in time critical applications such as Fast Reroute.

[0042] Furthermore, the above described invention provides high scalability since it maps large amounts of BGP routes to a few internal labels which correspond to the IGP routes so that hardware forwarding entries for BGP routes are intact during IGP route change. This makes the hardware table update time independent of the number of BGP routes in the device, thus, providing a high degree of scalability.

[0043] The invention further provides hardware resource savings in loadsharing cases. In a router where there are multiple paths to its BGP neighbor (a common scenario in SP networks) the conventional scheme consumes multiple hardware adjacency entries for each BGP route. The present invention hides the IGP loadsharing paths behind the internal MPLS label which is shared by all the BGP routes, so that each BGP route only needs to use one hardware adjacency entry. In a service provider or enterprise edge router where there are hundreds of thousands of BGP routes, this provides large savings in hardware resource usage. Also, less CPU processing time is required since only a limited number of IGP routes need to be updated. This provides high stability during converging time.

[0044] Although the present invention has been described in accordance with the embodiments shown, one of ordinary skill in the art will readily recognize that there could be variations made to the embodiments without departing from the scope of the present invention. Accordingly, it is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative and not in a limiting sense.