Load balancing network traffic using race methods

[0001] This application claims benefit of Provisional Appln. 60/627,798, filed Oct. 29, 2004, the entire contents of which is hereby incorporated by reference as if fully set forth herein, under 35 U.S.C. §119(e).

[0002] The present invention relates to using the mobility support features of Internet Protocol version 6 (IPv6) to allow a client and a server to communicate without the continuing intervention of a load-balancing mechanism that selected the server.

[0003] It is commonplace nowadays for a Web browser (“client”) to access content that is stored on a remote server by sending a request to the remote server using the remote server's Universal Resource Locator (URL) and receiving the content in response. Web sites associated with very popular URLs receive an extremely large volume of such requests from separate clients. In order to handle such a large volume of requests, these Web sites sometimes make use of a proxy device that initially receives requests and distributes them, according to some scheme, among multiple servers.

[0004] One such scheme attempts to distribute requests relatively evenly among servers that are connected to the proxy device. A proxy device employing this scheme is commonly called a “load balancer.” When successful, a load balancer helps to ensure that no single server in a server “farm” becomes inundated with requests.

[0005] When a proxy device receives a request from a client, the proxy device determines to which server, of many servers, the request should be directed. For example, a request might be associated with a session that is associated with a particular server. In that case, the proxy device might need to send the request to the particular server with which the session is associated.

[0006] A proxy device typically communicates with servers using a suite of protocols. For example, the proxy device may use the Internet Protocol (IP), Transport Control Protocol (TCP), and Hypertext Transfer Protocol (HTTP) in combination to communicate with a server. Some of these protocols are connection-oriented. Connection-oriented protocols typically require the negotiation of connection-related parameters between the nodes that are to be involved in the connection before the connection is “opened.” Data can be sent through the connection only after these parameters have been negotiated and the connection has been opened.

[0007] Parameter negotiation is often achieved through a “handshake” phase of a protocol. A handshake phase usually requires a significant amount of communication between the nodes. When encryption is involved, a handshake phase also may require a significant amount of processing by the nodes.

[0008] After a connection has been opened between the proxy device and a particular server, the proxy device receives a request from a client and forwards that request to the particular server through the connection. The particular server sends a response to the proxy device through the connection. The proxy device forwards the response to the client. Several requests and responses might be sent through the connection during the course of a transaction between the client and the particular server.

[0009] The proxy device acts as an intermediary throughout the communications between the client and the server. Data packets sent from the client pass through the proxy device on the way to the server. Data packets sent from the server pass through the proxy device on the way to the client. This can produce a communications “bottleneck” at the proxy device. When a single proxy device acts as an intermediary for many clients and servers, communications between those clients and servers may be delayed significantly.

[0010] Furthermore, when all communications that transpire between a client and server must pass through a proxy device, the client and the server become absolutely dependent upon the proxy device. If the proxy device fails for any reason, then the connection path between the client and the server is severed. The client and the server might not be able to resume communications at all until the proxy device becomes operational again. Even after the proxy device becomes operational again, client-to-proxy device and proxy device-to-server connections will need to be re-established via another round of time-consuming handshake phases.

[0011] In addition to causing communications between a client and a server to be interrupted, the failure of a proxy device can result in the loss of session state information that pertains to client-server sessions. Under some current approaches, session state information is stored at a proxy device rather than a server, so that if the server fails, then the proxy device can seamlessly continue the session with another server. However, viewed from one perspective, this approach merely pushes vulnerabilities from the server to the proxy device. If the proxy device suffers data loss, then all sessions that were being maintained by the proxy device may be irreparably lost also.

[0012] Some approaches attempt to compensate for this vulnerability by providing a “backup” proxy device, to which the “primary” proxy device periodically transmits updated session state information. If the “primary” proxy device fails, then the “backup” proxy device can be made, quickly, to function as a substitute for the “primary” proxy device. Unfortunately, if the “primary” proxy device fails soon before transmitting updated session state information to the “backup” proxy device, then the session state information that the “backup” proxy device will use to resume the client-server sessions may be stale, outdated, and inaccurate. The transmission of updated session state information between “primary” and “backup” proxy devices also may consume a substantial amount of network bandwidth.

[0013] Consequently, a technique that enables load-balanced communications between clients and servers, and which does not suffer from the disadvantages of some of the approaches discussed above, is desirable. The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.

[0021] In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of invention described herein. It will be apparent, however, that embodiments of the invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the description of embodiments of the invention.

[0022] Techniques are discussed herein for using the mobility support features of IPv6 to allow a client node and a server node to communicate without the continuing intervention of a load-balancing node that selected the server node. According to some of these techniques, a virtual IP address is associated with an entire server “farm” or “cluster.” A client node sends an IP packet toward the virtual IP address. Typically, one or more routing nodes lie between the client node and the server cluster. A load-balancing node advertises, to the routing nodes, that the virtual IP address is reachable through the load-balancing node. As a result, the routing nodes eventually route, to the load-balancing node, IP packets that are addressed to the virtual IP address. The load-balancing node intercepts such IP packets. An intercepted IP packet may contain a TCP packet that signals the recipient to engage in a handshake phase with the client node. TCP is described in the Internet Engineering Task Force (IETF) Request For Comments (RFC) 793, which is incorporated by reference in its entirety for all purposes as though fully disclosed herein.

[0023] However, according to one technique, rather than engaging in a handshake phase with a client node that sent such an intercepted packet, the load-balancing node selects a server node from among multiple server nodes that exist in a server cluster that is associated with the virtual IP address, and sends the intercepted packet toward the selected server node. In response to receiving the TCP packet, the selected server node completes a handshake phase with the client node, thereby establishing a connection path between the client node and the server node. The connection path does not need to include the load-balancing node.

[0024] During the course of the handshake phase, the server node sends an IPv6 packet toward the client node. The IPv6 packet contains an IPv6 Mobility Header. The Mobility Header contains a Binding Update option. The Binding Update option specifies a Home Address option that indicates the virtual IP address. As a source IP address, the IPv6 packet indicates the “physical,” or “care-of” IP address of the selected server—which differs from the virtual IP address. IPv6 mobility features are described in the IETF RFC 3775, which is incorporated by reference in its entirety for all purposes as though fully disclosed herein.

[0025] The client node receives the IPv6 packet. The information in the Mobility Header causes the client node to perform IP address replacement operations so that IP packets that the client node otherwise would have addressed to the virtual IP address are addressed to the physical IP address of the selected server node instead. Consequently, the IP packets are routed toward the selected server node instead of the load-balancing node. Thereafter, IP packets that the selected server sends toward the client node contain an IPv6 Mobility Header such as the one described above, which causes the client node to continue to perform such IP address replacement operations. As a result, the client node and the server node may communicate without the continuing intervention of the load-balancing node that selected the server node.

[0026] Variations of, alternatives to, and enhancements to the foregoing technique are described in greater detail below.

[0027] FIG. 1 is a block diagram that illustrates a system 100 in which techniques for enabling more direct client-server communication may be practiced. According to one embodiment, system 100 comprises load-balancing nodes 102A-N, server clusters 110A and 110B, network 106, and client nodes 108A-N. Server cluster 110A comprises server nodes 104A-N. Server cluster 110B comprises server nodes 105A-N.

[0028] Each of server nodes 104A-N and 105A-N is coupled communicatively with network 106. Each of server nodes 104A-N and 105A-N may be a separate computer. Alternatively, each of server nodes 104A-N and 105A-N may be a separate process executing on the same or on separate computers. For example, each of server nodes 104A-N and 105A-N may include a separate HTTP server (or “Web server”) process. Server nodes 104A-N and 105A-N receive requests and send responses through network 106. Such requests and responses may be, for example, HTTP requests and responses carried over TCP and IP. Each of server nodes 104A-N and 105A-N is associated with a different IP address.

[0029] Each of load-balancing nodes 102A-N is also coupled communicatively with network 106. Load-balancing nodes 102A-N also receive requests through network 106. In response to receiving IP packets, load-balancing nodes 102A-N select server nodes and send the IP packets toward the selected server nodes. Load-balancing nodes 102A-N may be implemented within separate network routing devices, for example. Each of load-balancing nodes 102A-N is associated with a different IP address.

[0030] Network 106 is a computer network, such as, for example, a local area network (LAN), wide area network (WAN), or internetwork such as the Internet. Network 106 may comprise multiple interconnected routing nodes, such as network routers, which use routing protocols and routing tables to forward IP packets toward destination IP addresses that are specified within those IP packets.

[0031] Client nodes 108A-N also are coupled communicatively with network 106. Each of client nodes 108A-N may be a separate computer. Alternatively, each of client nodes 108A-N may be a separate process executing on the same or on separate computers. For example, each of client nodes 108A-N may include a separate HTTP client (or “Web browser”) process. Each of client nodes 108A-N is associated with a different IP address.

[0032] Each of server clusters 110A and 110B is associated with a separate virtual IP address; the virtual IP addresses are not associated with any individual physical machine or computer. Client nodes 108A-N initially send requests through network 106 toward such virtual IP addresses. Due to advertising by load-balancing nodes 102A-N, these requests are routed to and intercepted by load-balancing nodes 102A-N. Load-balancing nodes 102A-N send intercepted requests to various ones of server nodes 104A-N and 105A-N. Server nodes 104A-N and 105A-N send responses to the requests that they receive. However, instead of sending responses to load-balancing nodes 102A-N, server nodes 104A-N and 105A-N may send responses toward client nodes 108A-N via routes that do not include load-balancing nodes 102A-N, using a technique that is described in greater detail below.

[0033] In one embodiment, one or more entities (e.g., individuals, corporations, partnerships, etc.) each pay a fee to the service provider of server nodes 104A-N and/or 105A-N in order to have content (e.g., files, Web pages, etc.) hosted on server nodes 104A-N and/or 105A-N. In one embodiment, one or more entities each pay a fee to such a service provider in order to have server nodes 104A-N and/or 105A-N provide specified services for the one or more entities.

[0034] FIG. 2 is a flowchart that illustrates a technique by which a load-balancing node enables a client node and a server node to communicate without further intervention from the load-balancing node after the load-balancing node has selected the server node, according to one embodiment of the invention. In order to better describe the technique, examples given below refer to entities shown within FIG. 1. For example, the technique may be performed by any of load-balancing nodes 102A-N.

[0035] Referring to FIG. 2, in block 202, a load-balancing node advertises, to one or more routing nodes, that a particular virtual IP address is reachable through the load-balancing node. For example, using a routing protocol, load-balancing node 102A may advertise, to one or more network routers within network 106, that a particular virtual IP address, which is associated with server cluster 110A, is reachable through load-balancing node 102A. As a result, the network routers may send, toward load-balancing node 102A, IP packets that indicate the particular virtual IP address as a destination IP address.

[0036] In block 204, a first IP packet is received at the load-balancing node. The first IP packet contains a TCP SYN packet. The first IP packet also indicates the particular virtual IP address as a destination IP address. For example, load-balancing node 102A may receive an IP packet that client node 108A addressed to the particular virtual IP address that is associated with server cluster 110A.

[0037] In block 206, a server node is selected from among a plurality of server nodes. For example, load-balancing node 102A may select server node 104A from among server nodes 104A-N. The selection may be based on the contents of the first IP packet as well as other criteria, some of which are described in greater detail below. Inasmuch as each server node is associated with its own distinct IP address, the selected server node is associated with an IP address that differs from the virtual IP address.

[0038] In block 208, the first IP packet is encapsulated within a second IP packet. While the first IP packet indicates the virtual IP address as a destination IP address, the second IP packet indicates the selected server node's IP address as a destination IP address. As a result, routing nodes that receive the second IP packet will forward the second IP packet toward the selected server. For example, load-balancing node 102A may encapsulate the first IP packet within a second IP packet, as described above, using IPv6 encapsulation mechanisms.

[0039] In block 210, the second IP packet is sent toward the selected server node. For example, load-balancing node 102A may send the second IP packet through network 106 toward server node 104A. Because load-balancing node 102A forwards the TCP SYN packet toward server node 104A instead of responding to the TCP SYN packet with a TCP SYN/ACK packet, client node 108A and server node 104A may engage in a TCP handshake phase to establish a TCP connection that does not need to involve load-balancing node 102A. Thus, according to this particular technique, load-balancing node 102A does not “delay” the binding between client node 108A and server node 104A.

[0040] Once the selected server node receives the second IP packet, which contains the first IP packet, which contains the TCP SYN packet, the selected server node may perform a technique that enables the client node and the selected server node to communicate without the intervention of the load-balancing node. Such a technique is described below with reference to FIG. 3.

[0041] FIG. 3 is a flowchart that illustrates a technique by which a server node that was selected by a load-balancing node engages in a handshake phase with a client node that sent a connection request toward the load-balancing node, according to one embodiment of the invention. In order to better describe the technique, examples given below refer to entities shown within FIG. 1. For example, the technique may be performed by any of server nodes 104A-N or 105A-N.

[0042] Referring to FIG. 3, in block 302, a first IP packet is received at a server node. The first IP packet indicates the server node's IP address as a destination IP address. The first IP packet contains a second IP packet. For example, server node 104A may receive a particular IP packet that load-balancing node 102A sent toward server node 104A. The particular IP packet may contain another IP packet that load-balancing node 102A intercepted and encapsulated within the particular IP packet, as described above.

[0043] The second IP packet indicates, as a source IP address, an IP address of a client node from which the second IP packet originated. The second IP packet also indicates, as a destination IP address, a virtual IP address that is associated with a server cluster to which the server node belongs. A selected server node has the virtual IP address on an interface (possibly on a loopback or physical) so it can receive the IP packets destined for the virtual IP address. The second IP packet contains a TCP SYN packet, which signals the server node to engage in a handshake phase with the TCP SYN packet's sender.

[0044] For example, the source IP address of the second IP packet may be an IP address that is associated with client node 108A, which originally sent the second IP packet toward the virtual IP address. The destination IP address of the second IP packet may be the virtual IP address that is associated with server cluster 110A.

[0045] In block 304, in response to receiving the TCP SYN packet, a third IP packet is sent toward the client node that is associated with the second packet's indicated source IP address. The third IP packet contains a TCP SYN/ACK packet that indicates, to the client node, which of the TCP parameters proposed in the TCP SYN packet the server node has accepted for use in the forthcoming TCP connection between the client node and the server node. For example, server node 104A may send, toward client node 108A, an IP packet that contains such a TCP SYN/ACK packet. Server node 104A does not need to send this IP packet via load-balancing node 102A. Thus, server node 104A engages in a handshake phase with client node 108A, without the continuing intervention of load-balancing node 102A.

[0046] The third IP packet also contains an IPv6 Mobility Header, such as is described in the IETF RFC 3775. The Mobility Header contains a Binding Update option. The Binding Update option specifies a Home Address option that indicates the same virtual IP address that the second IP packet indicated as a destination IP address. Continuing the above example, the IP packet that server node 104A sends toward client node 108A may contain an IPv6 Mobility Header that specifies, using the Home Address option, the virtual IP address that is associated with server cluster 110A.

[0047] In one embodiment, the Binding Update option in the Mobility Header also indicates, in a Lifetime field, a value that is greater than zero. The presence of this value causes a cache entry to be established at the node for which the Mobility Header is destined, as is described below.

[0048] When the client node receives the IP packet that the server node sent, the Home Address option in the Binding Update option causes the client node to replace the IP packet's source IP address (i.e., the server node's IP address) with the IP address specified in the Home Address option (i.e., the virtual IP address). As a result of this replacement, the IP packet appears to have originated from the virtual IP address rather than the server node's IP address. This replacement may be performed at the IP layer, for example, before the TCP SYN/ACK packet contained within the IP packet is passed “upward” to the TCP layer.

[0049] Because the IP packet appears to have originated from the virtual IP address, the client node reacts to the TCP SYN/ACK packet as though the TCP SYN/ACK packet came in response to the TCP SYN packet that the client node originally sent toward the virtual IP address; without the earlier IP address replacement, the TCP SYN/ACK packet would appear to have come from the server node's IP address, and, consequently, the client node would not be able to “match up” the TCP SYN packet—which the client node did not sent toward the server node's IP address—with the TCP SYN/ACK packet.

[0050] The presence of a non-zero value in the Lifetime field of the Binding Update option causes the client node to establish a cache entry locally within an IP layer cache. The cache entry indicates a binding between the IP packet's pre-replacement source IP address (i.e., the server node's IP address) and the IP address specified in the Home Address option (i.e., the virtual IP address). The cache entry remains in the IP layer cache for a period of time commensurate with the value indicated in the Lifetime field.

[0051] The presence of the cache entry within the client node's IP layer cache causes the client node to perform, at the IP layer, a destination IP address replacement operation on each outgoing IP packet that indicates the virtual IP address as a destination IP address. Because the cache entry binds the virtual IP address to the server node's IP address, the client node changes the destination IP addresses of such outgoing IP packets from the virtual IP address to the server node's IP address. As a result, IP packets that the client node otherwise would have sent toward the virtual IP address get routed toward the server node's IP address instead. Thus, such IP packets do not need to pass through the load-balancing node on their way to the server node. This helps to reduce the extent to which the load-balancing node becomes a “bottleneck.”

[0052] The presence of the cache entry within the client node's IP layer cache also causes the client node to insert an IPv6 type 2 routing header, such as is described in the IETF RFC 3775, into each outgoing IP packet that indicates, pre-replacement, the virtual IP address as a destination IP address. The IPv6 type 2 routing header indicates the virtual IP address in the Home Address field. The client node sends such IPv6 packets, with IPv6 type 2 routing headers, toward the server node.

[0053] One such packet typically contains a TCP ACK packet that concludes the client-server handshake phase upon receipt; at the conclusion of the handshake phase, a TCP connection is established between the client node and the server node.

[0054] In block 306, an IPv6 packet, which indicates the client node's IP address as a source IP address, is received. The IPv6 packet contains an IPv6 type 2 header that indicates the virtual IP address, as described above. For example, server node 104A may receive such an IPv6 packet that client node 108A sent toward server node 104A.

[0055] The receipt of the IPv6 type 2 header causes the server node to continue to place an IPv6 Mobility Header, as described above with reference to block 304, into each IP packet that the server node sends toward the client node. The presence of the IPv6 Mobility Header in such IP packets causes the client node to continue to perform IP address replacement operations, so that the client node and the server node can communicate without the intervention of the load-balancing node.

[0056] The technique described above with reference to FIG. 2 is especially useful when all of the server nodes within a particular server cluster provide the same services. However, in some systems, the server nodes within a particular server cluster might offer different services. For example, one server node might offer HTTP services, while another server node might offer File Transfer Protocol (FTP) services, while yet another server node might offer Simple Mail Transfer Protocol (SMTP) services.

[0057] In such a scenario, some benefit may be derived from having the load-balancing node select a server node based on the contents of a message that is collectively contained within payload portions of the IP packets that the load-balancing node intercepts. For example, a load-balancing node might make a load-balancing decision based on a Uniform Resource Locator (URL) or a cookie that is embedded within a message. Because a single message might be split among multiple separate IP packets, the load-balancing node might need to accumulate and assemble the contents of multiple IP packets prior to inspecting the message and selecting the server node to which those IP packets are to be forwarded.

[0058] In order to accumulate packets, the load-balancing node typically needs to assume the burden of guaranteeing, to client nodes, that the data that the client nodes have sent has successfully reached the load-balancing node. Usually, this means that the load-balancing node needs to establish, at least initially, TCP connections with the client nodes. Through such TCP connections, the load-balancing node sends TCP ACK packets to the client nodes as an assurance that client-sent data has been received and does not need to be re-sent. The load-balancing node also typically assumes, at least temporarily, the responsibility for receiving TCP ACK packets from the selected server node, and re-sending data packets to the selected server node when necessary.

[0059] However, once the accumulated IP packets have been successfully transmitted between the load-balancing node and the selected server node, there is no real need for client-server communications to continue to pass through the load-balancing node.

[0060] FIG. 4 is a flowchart that illustrates a technique by which a load-balancing node enables a client node and a server node to communicate without further intervention from the load-balancing node after the load-balancing node has selected the server node based on the collective contents of accumulated packets, according to one embodiment of the invention. In order to better describe the technique, examples given below refer to entities shown within FIG. 1. For example, the technique may be performed by any of load-balancing nodes 102A-N.

[0061] Referring to FIG. 4, in block 402, a load-balancing node advertises, to one or more routing nodes, that a particular virtual IP address is reachable through the load-balancing node. For example, using a routing protocol, load-balancing node 102A may advertise, to one or more network routers within network 106, that a particular virtual IP address, which is associated with server cluster 110A, is reachable through load-balancing node 102A. As a result, the network routers may send, toward load-balancing node 102A, IP packets that indicate the particular virtual IP address as a destination IP address.

[0062] In block 404, a first IP packet is received at the load-balancing node. The first IP packet contains a TCP SYN packet. The first IP packet also indicates the particular virtual IP address as a destination IP address. For example, load-balancing node 102A may receive an IP packet that client node 108A addressed to the particular virtual IP address that is associated with server cluster 110A.

[0063] In block 406, a second IP packet is sent toward the source IP address that is indicated in the first IP packet. The second IP packet contains a TCP SYN/ACK packet that indicates the TCP parameters by which the load-balancing node has agreed to abide. For example, load-balancing node 102A may send such an IP packet toward client node 108A. Upon receiving the TCP SYN/ACK packet, client node 108A may send a TCP ACK packet toward load-balancing node 102A.

[0064] In block 408, a third IP packet is received. The third IP packet contains a TCP ACK packet. For example, load-balancing node 102A may receive such an IP packet that client node 108A sent toward load-balancing node 102A in response to receiving the TCP SYN/ACK packet. Thus, the handshake phase between load-balancing node 102A and client node 108A is concluded, and a TCP connection between the two is established. Client node 108A may send one or more IP packets through the TCP connection.

[0065] In block 410, one or more IP packets are received through the TCP connection. For example, load-balancing node 102A may receive one or more IP packets that client node 108A sent through the TCP connection. The payload portions of the IP packets may collectively contain at least part of a message.

[0066] In block 412, at least part of a message collectively contained within the one or more IP packets is assembled. For example, load-balancing node 102A may accumulate the one or more IP packets and re-order and assemble their contents to construct at least a portion of a message.

[0067] In block 414, a server node is selected from among a plurality of server nodes based on the assembled message or portion thereof. For example, based on the contents of such a message (e.g., a URL, a cookie, etc.) or portion thereof, load-balancing node 102A may select server node 104A from among server nodes 104A-N. The selection also may be based on other criteria, some of which are described in greater detail below. Inasmuch as each server node is associated with its own distinct IP address, the selected server node is associated with an IP address that differs from the virtual IP address.

[0068] In block 416, the first IP packet is encapsulated within a fourth IP packet. While the first IP packet indicates the virtual IP address as a destination IP address, the fourth IP packet indicates the selected server node's IP address as a destination IP address. As a result, routing nodes that receive the fourth IP packet will forward the fourth IP packet toward the selected server. For example, load-balancing node 102A may encapsulate the first IP packet within a fourth IP packet, as described above, using IPv6 encapsulation mechanisms.

[0069] In one embodiment, the original TCP SYN packet from the client node is contained within the first IP packet. As a result, the TCP sequence and acknowledgement numbers that are represented in the TCP SYN packet are used in both the client-to-load balancer connection and the forthcoming load balancer-to-server connection.

[0070] In block 418, the fourth IP packet is sent toward the selected server node. For example, load-balancing node 102A may send the fourth IP packet through network 106 toward server node 104A. Server node 104A may receive the fourth IP packet, extract the TCP SYN packet encapsulated therein, and send, toward load-balancing node 102A, a TCP SYN/ACK packet that indicates TCP sequence and acknowledgement numbers that are based on those contained in the TCP SYN packet. The TCP SYN/ACK packet may be carried within an IP packet.

[0071] In block 420, a fifth IP packet, which was sent from the selected server node, is received. The fifth IP packet contains a TCP SYN/ACK packet. For example, load-balancing node 102A may receive such an IP packet that server node 104A sent toward load-balancing node 102A.

[0072] In block 422, a sixth IP packet is sent toward the selected server node. The sixth IP packet contains a TCP ACK packet. For example, load-balancing node 102A may send such an IP packet toward server node 104A. The receipt of the TCP ACK packet by server node 104A concludes the handshake phase between load-balancing node 102A and server node 104A. As a result, a TCP connection is established between load-balancing node 102A and server node 104A.

[0073] In block 424, the one or more IP packets that were received through the client-to-load balancer TCP connection in block 410 are modified. An IPv6 Mobility Header is added to each such packet. Each such Mobility Header contains a Binding Update option, and each such Binding Update option contains a Home Address option that indicates the load-balancing node's IP address. For example, to each such IP packet, load-balancing node 102A may add an IPv6 Mobility Header that indicates, in the Home Address option, an IP address that is associated with load-balancing node 102A. The load-balancing node's IP address typically differs from the virtual IP address.

[0074] Each such IP packet's source IP address is the IP address of the client node that originally sent the IP packet toward the virtual IP address. For example, the source IP address in each such IP packet may be the IP address that is associated with client node 108A.

[0075] In block 426, the modified IP packets are sent toward the selected server node through the TCP connection established between the load-balancing node and the selected server node. For example, load-balancing node 102A may send the modified IP packets toward server node 104A.

[0076] Once the selected server node receives the modified IP packets, which contain the IPv6 Mobility Headers, the selected server node may perform a technique that enables the client node and the selected server node to communicate without the intervention of the load-balancing node. Such a technique is described below with reference to FIG. 5.

[0077] FIG. 5 is a flowchart that illustrates a technique by which a server node, which was selected by a load-balancing node based on the collective contents of accumulated packets, enables communication with a client node that sent the packets, without further intervention from the load-balancing node, according to one embodiment of the invention. In order to better describe the technique, examples given below refer to entities shown within FIG. 1. For example, the technique may be performed by any of server nodes 104A-N or 105A-N.

[0078] Referring to FIG. 5, in block 502, a first IP packet is received at a server node. The first IP packet indicates, as a destination IP address, a virtual IP address that is associated with a server cluster to which the server node belongs. The first IP packet indicates the IP address of a client node as a source address. For example, server node 104A may receive a modified IP packet that load-balancing node 102A sent toward server node 104A through a TCP connection. The modified IP packet may be, for example, one of the packets that are described above in relation to blocks 424 and 426 of FIG. 4. Continuing the example provided above with reference to FIG. 4, the modified IP packet's source IP address may be the IP address that is associated with client 108A, and the modified IP packet's destination IP address may be the IP address that is associated with the virtual IP address that is associated with server cluster 110A.

[0079] The first IP packet also contains an IPv6 Mobility Header, such as is described above with reference to block 424 of FIG. 4. The Mobility Header contains a Binding Update option. The Binding Update option specifies a Home Address option that indicates the IP address that is associated with a load-balancing node that sent the first IP packet toward the server node. Continuing the above example, the first IP packet may contain an IPv6 Mobility Header that specifies, using the Home Address option, the IP address that is associated with load-balancing node 102A.

[0080] The first IP packet also contains at least a portion of a request. For example, the first IP packet may contain an HTTP request, or a portion thereof.

[0081] In block 504, the source IP address in the first IP packet (i.e., the client node's IP address) is replaced with the IP address that is specified in the Home Address option of the IPv6 Mobility Header in the first IP packet (i.e., the load-balancing node's IP address). For example, server node 104A may perform this replacement in response to the presence of the Home Address option in the IPv6 Mobility Header of the first IP packet.

[0082] In block 506, a second IP packet is sent toward the client node that is associated with the pre-replacement source IP address indicated in the first IP packet. The second IP packet contains at least a portion of a response to the request that was at least partially contained in the first IP packet. For example, server node 104A may send, toward client node 108A, an IP packet that contains at least a portion of an HTTP response to the HTTP request that client node 108A previously sent toward the virtual IP address.

[0083] The second IP packet also contains an IPv6 Mobility Header. The Mobility Header contains a Binding Update option. The Binding Update option specifies a Home Address option that indicates the same virtual IP address that the first IP packet indicated as a destination IP address. Continuing the above example, the IP packet that server node 104A sends toward client node 108A may contain an IPv6 Mobility Header that specifies, using the Home Address option, the virtual IP address that is associated with server cluster 110A.

[0084] In one embodiment, the Binding Update option in the Mobility Header also indicates, in a Lifetime field, a value that is greater than zero. The presence of this value causes a cache entry to be established at the node for which the Mobility Header is destined, as is described below.

[0085] When the client node receives the IP packet that the server node sent, the Home Address option in the Binding Update option causes the client node to replace the IP packet's source IP address (i.e., the server node's IP address) with the IP address specified in the Home Address option (i.e., the virtual IP address). As a result of this replacement, the IP packet appears to have originated from the virtual IP address rather than the server node's IP address.

[0086] Because the IP packet appears to have originated from the virtual IP address, the client node reacts to the IP packet as though the IP packet came in response to the IP packets that the client node originally sent toward the virtual IP address.

[0087] The presence of a non-zero value in the Lifetime field of the Binding Update option causes the client node to establish a cache entry locally within an IP layer cache. The cache entry indicates a binding between the IP packet's pre-replacement source IP address (i.e., the server node's IP address) and the IP address specified in the Home Address option (i.e., the virtual IP address). The cache entry remains in the IP layer cache for a period of time commensurate with the value indicated in the Lifetime field.

[0088] The presence of the cache entry within the client node's IP layer cache causes the client node to perform, at the IP layer, a destination IP address replacement operation on each outgoing IP packet that indicates the virtual IP address as a destination IP address. Because the cache entry binds the virtual IP address to the server node's IP address, the client node changes the destination IP addresses of such outgoing IP packets from the virtual IP address to the server node's IP address. As a result, IP packets that the client node otherwise would have sent toward the virtual IP address get routed toward the server node's IP address instead. Thus, such IP packets do not need to pass through the load-balancing node on their way to the server node. This helps to reduce the extent to which the load-balancing node becomes a “bottleneck.”

[0089] Thenceforth, the client node and the server node can communicate with each other as mobile devices using the mobility features of IPv6. The client node uses the load-balancing node's IP address as its “home address,” and the server node uses the virtual IP address as its “home address.”

[0090] In one embodiment, instead of sending an intercepted TCP SYN packet toward a single server node in the manner described above with reference to FIG. 2, a load-balancing node sends intercepted TCP SYN packets to multiple server nodes within a server cluster. The load-balancing node may send a copy of an intercepted TCP SYN packet to each server node that is qualified to respond to a client request; in one embodiment, only those server nodes that satisfy specified criteria, some of which are described below, are selected to receive a copy of the TCP SYN packet.

[0091] In one embodiment, each of the selected server nodes receives a copy of the TCP SYN packet and responds to the client node with a TCP SYN/ACK packet, in the same manner as is described above with reference to FIG. 3. The TCP SYN/ACK packet that the client node receives first is deemed to be from the server node that is closest to, or has the fastest communications with, the client node. The client node sends an ACK packet only to the server node from which the first-received TCP SYN packet was received. As a result, a TCP connection is established between that server node and the client node.

[0092] The client node disregards the TCP SYN/ACK packets that the other server nodes sent. Upon detecting that a specified amount of time has passed without receiving a corresponding TCP ACK packet from the client node, these other server nodes abandon the attempt to connect to the client node.

[0093] For example, referring to FIG. 1 for sake of illustration, load-balancing node 102A may intercept a TCP SYN packet that client node 108A sent toward the virtual IP address that is associated with server cluster 110A. Based on specified criteria, load-balancing node 102A may determine that, among server nodes 104A-N, only server nodes 104A and 104B are qualified to provide the service that client node 108A requires. Consequently, load-balancing node 102A may send a copy of the TCP SYN packet to both server node 104A and server node 104B, in the manner described above with reference to FIG. 2.

[0094] Each of server nodes 104A and 104B receive a copy of the TCP SYN packet and send a corresponding TCP SYN/ACK packet toward client node 108A in the manner described above with reference to FIG. 3. If the connection path between server node 104A and client node 108A has less latency than the connection path between server node 104B and client node 108A, then the TCP SYN/ACK packet from server node 104A will reach client node 108A first. When client node 108A receives the TCP SYN/ACK packet from server node 104A, client node 108A sends a corresponding TCP ACK packet toward server node 104A. Thus, a TCP connection is established between client node 108A and server node 104A; these nodes may communicate with each other through this TCP connection.

[0095] Later, when client node 108A receives the TCP SYN/ACK packet that server node 104B sent, client node 108A disregards the TCP SYN/ACK packet and does not send a corresponding TCP ACK packet toward server node 104B. After waiting for a specified amount of time, server node 104B determines that a TCP connection with client node 108A will not be established, and abandons the attempt.

[0096] As a result of the foregoing technique, the best pairings of client and server nodes is achieved, so that communications between client and server nodes have as little latency as possible. Even if the network topology changes, the above technique still functions to select the best pairings in a real-time manner. The latency measurement achieved by using the above “SYN/ACK race” technique is more accurate and adaptable than other latency-measuring mechanisms.

[0097] As is described above, the SYN/ACK race technique can be used to determine the relative latency “distances” between client and server nodes. This distance information can be used to generate a network map. The network map, in turn, can be used to make load-balancing decisions. For example, based on such a network map, load-balancing node 102 may select, from among server nodes 104A-N, a server node that has the least latency relative to client node 108A.

[0098] For example, a network map can be generated by turning on the SYN/ACK race for a predetermined amount of time. As the SYN/ACK race proceeds, the servers log the results of the race, e.g., the server records the clients that it establishes a connection with.

[0099] A load-balancer or centralized server can collect logs from the servers in the server clusters. A client IP address is listed in a log entry when a server establishes a connection with the client. Note that the server that establishes a connection with a client can be considered as the closest server to the client since the server won the SYN/ACK race. Given the log entry information, the load-balancer or centralized server can correlate the server and client information to create a map of major clients and their corresponding servers that are considered the closest to the clients.

[0100] The size of the generated network map can become large because the number of clients may grow to be unwieldy. One method of reducing the size of the network map is to use the subnet addresses of the clients instead of using each client's IP addresses. This brings the network map to a higher level of abstraction where the servers closest to the subnets are recorded.

[0101] The network map can be used by the load-balancers to make server selection decisions. The network map can be periodically refreshed by obtaining logs from the servers after the SYN/ACK race has been running for a predetermined time.

[0102] As is described above, in one embodiment, the Binding Update option in the Mobility Header indicates, in a Lifetime field, a value that is greater than zero. The presence of this value causes a cache entry to be established at the node for which the Mobility Header is destined, as is described below.

[0103] As long as the cache entry remains in a client node's cache, the client node will continue to send IP packets to the same selected server node. This allows a session between the client node and the selected server node to persist. Because the client node communicates with the selected server node without intervention from the load-balancing node after the load-balancing node has selected the server node, there is no need to store session mapping information at the load-balancing node.

[0104] The cache entry expires after a period of time that is commensurate with the value of the Lifetime field. At this point, the cache entry is expelled from the cache, and the client node once again sends IP packets toward the virtual IP address; such packets get routed toward the load-balancing node again.

[0105] One advantage of this technique is that it is effective even when multiple client nodes are separated from the rest of a network environment by a proxy device. Another advantage of this technique is that it can be used even with non-HTTP client nodes and with HTTP client nodes that do not have cookies enabled.

[0106] At times, a server node may fail in some way and become at least temporarily incapable of servicing client requests. If a cache entry in a client node's cache binds the virtual IP address to the failed server node's IP address, then the client node will continue to send IP packets toward the failed server node's IP address. This is an undesirable result.

[0107] In order to compensate for the possibility of server node failure, in one embodiment, a load-balancing node periodically determines whether the server nodes in a server cluster remain capable of servicing client requests. For example, load-balancing node 102 can periodically “ping” server nodes 104A-N (such as the method described in U.S. patent application Ser. No. 09/909,711, owned by the Applicant, the entire contents of which is hereby incorporated by reference as if fully set forth herein); if a server node does not respond to the “ping” in a timely manner, then load-balancing node 102 may conclude that the server node has suffered some kind of failure.

[0108] In response to determining that a server node has failed, the load-balancing node can “assume” the failed server node's IP address by advertising, to one or more routing nodes, that the failed server node's IP address is associated with, or is reachable through, the load-balancing node. This causes the routing nodes to route, toward the load-balancing node, IP packets that are addressed to the failed server node. The load-balancing node therefore intercepts such IP packets.

[0109] When the load-balancing node intercepts such an IP packet, the load-balancing node determines whether the IP packet is addressed to the failed server node. If the IP packet is addressed to the failed server node, then the load-balancing node sends, toward the client node whose IP address is indicated as a source IP address in the IP packet, another IP packet that contains a TCP RST packet and an IPv6 Mobility Header.

[0110] The TCP RST packet instructs the client node to reset the TCP connection that the client node had established with the failed server node. The Mobility Header contains a Binding Update option that indicates, in the Lifetime field, a value of zero. The presence of the zero value in the Lifetime field causes the client node to remove, from the client node's cache, the cache entry that binds the virtual IP address to the failed server node's IP address. With the TCP connection reset and the cache entry removed, the client node will, once again, send IP packets toward the virtual IP address; the load-balancing node intercepts such packets.

[0111] The load-balancing node does not send intercepted packets to server nodes that are known to be in a non-operational state. Therefore, the load-balancing node selects a different server node, and sends the intercepted packets toward that server node. In one embodiment, the newly selected server node communicates with the client node using IPv6 mobility support features as described above. If and when a previously failed server node becomes operational again, then the load-balancing node may detect this (such as through the pinging technique) and once again include the recovered server node as a candidate when making a load-balancing decision.

[0112] In a traditional load-balancing cluster, a standby backup load-balancing node is provided for high availability in case a primary load-balancing node fails. The primary load-balancing node periodically transfers persistent session state information to the standby load-balancing node. If the primary load-balancing node fails, then the standby load-balancing node assumes the role of the primary load-balancing node, at least until the primary load-balancing node can be brought back online. Unfortunately, under this traditional approach, a “gap” of unavailability could occur during the switchover.

[0113] However, using the techniques described herein, client-server connections do not need to be interrupted, since client nodes and server nodes are able to communicate without the further intervention of the load-balancing node. Because the client node maintains a binding to the server node in the form of the cache entry discussed above, the client node continues to communicate with the same server node, without losing persistence, even when the load-balancing node that selected the server node fails.

[0114] In the past, each server cluster would be assigned one or more dedicated load-balancing nodes. A particular load-balancing node would only select from among server nodes in the server cluster to which the particular load-balancing node belonged. The fact that the load-balancing node always acted as an intermediary throughout all communications between client nodes and server nodes made it necessary for the load-balancing node to be in close proximity, in the network, to the load-balancing node's corresponding server cluster. If the load-balancing node were located too remotely from the server nodes, then the latency would become unacceptably high.

[0115] Fortunately, using to the techniques discussed herein, this proximity limitation is removed. As is described above, once a load-balancing node has selected a server node, the load-balancing node does not need to have any more participation in the communications between that server node and a client node. As a result, the load-balancing node does not need to be located in close proximity, in the network, to any particular server node. Load-balancing nodes do not even need to be in the same LAN or WAN as the server nodes from which the load-balancing nodes choose. Using the techniques described herein, a load-balancing node may be placed close to one or more client nodes, or anywhere in the Internet, for that matter. Using the SYN/ACK race features described above, a qualified server node having the least latency related to a client node can be selected regardless of the position of the load-balancing node relative to the server node.

[0116] According to one embodiment, a single server cluster serves requests that originate from anywhere in the world.

[0117] However, according to an alternative embodiment, there are multiple separate server clusters (e.g., server clusters 110A and 110B shown in FIG. 1). Each server cluster may be associated with a different geographical region or location. Furthermore, each server cluster may be associated with the same virtual IP address—a uniform, global, “anycast” address. Each load-balancing node, regardless of that load-balancing node's location, advertises to one or more routing nodes that the anycast address is reachable through that load-balancing node.

[0118] As a result of both this advertising and the routing algorithms that are employed by the routing nodes, the routing nodes route each client node's requests (which are addressed to the “anycast” address) to the load-balancing node that is nearest to, or has the lowest latency relative to, that client node. The IPv6 Binding Update options discussed above, which create bindings between client nodes and server nodes, eliminate the possibility that a routing node will route requests from a particular client node to more than one server cluster, so long as the binding remains valid.

[0119] As is discussed above in relation to block 206 of FIG. 2, load-balancing node may select a server node from among a plurality of server nodes based on specified criteria. An example technique for selecting a server node is now described.

[0120] In one embodiment, a load-balancing device selects, from among a set of server nodes in a server cluster, a first subset of server nodes that are functional. For example, load-balancing node 102 may “ping” each of server nodes 104A-N (such as the method described in U.S. patent application Ser. No. 09/909,711, owned by the Applicant). Those of server nodes 104A-N that respond to the “ping” in a timely manner are included within the first subset.

[0121] From among the first subset, the load-balancing node selects a second subset of server nodes that are capable of providing a service that the client node is requesting. For example, load-balancing node 102 may inspect a destination TCP port that is indicated in an intercepted TCP SYN packet. Load-balancing node 102 may determine a particular service (e.g., HTTP, FTP, SMTP) that is associated with the destination TCP port. Load-balancing node 102 may maintain a mapping between services and the server nodes that provide those services. If the destination TCP port is port 80, for example, then load-balancing node 102 may select, from among the first subset, one or more server nodes that provide HTTP services.

[0122] From among the second subset, the load-balancing node selects a third subset of server nodes that possess the resource that the client node is requesting. For example, if an HTTP request is for a particular file or Web page, then load-balancing node 102 may determine which of the server nodes in the second subset possess the particular file or Web page; the file or page might be present on some server nodes but not on others.

[0123] If the request is associated with a particular session, and if the server node that is associated with the particular session is contained in the third subset, then the load-balancing node selects that server node. If that server node is not currently operational, then it will not be in the third subset, and the load-balancing node will attempt to select another server node.

[0124] If more than one server node is present in the third subset, then the load-balancing node may select from among them based on one or more factors. For example, load-balancing node 102 may select, from among multiple server nodes, the server node that has the least latency, or which is closest to, the client node that sent the request. For another example, load-balancing node 102 may select, from among multiple server nodes, the server node that is least loaded. Each server node's load may be determined based on that server node's processing capacity, memory capacity, secondary storage capacity, and/or the number of client nodes with which that server node is concurrently communicating. While the foregoing list is illustrative, it is not exhaustive. For another example, load-balancing node 102 may base its selection upon both the latency and load factors discussed above.

[0125] According to certain embodiments of the invention, a load-balancing node and a server node each may be implemented on a computer system. FIG. 6 is a block diagram that illustrates a computer system 600 upon which an embodiment of the invention may be implemented. Computer system 600 includes a bus 602 or other communication mechanism for communicating information, and a processor 604 coupled with bus 602 for processing information. Computer system 600 also includes a main memory 606, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 602 for storing information and instructions to be executed by processor 604. Main memory 606 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 604. Computer system 600 further includes a read only memory (ROM) 608 or other static storage device coupled to bus 602 for storing static information and instructions for processor 604. A storage device 610, such as a magnetic disk or optical disk, is provided and coupled to bus 602 for storing information and instructions.

[0126] Computer system 600 may be coupled via bus 602 to a display 612, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device 614, including alphanumeric and other keys, is coupled to bus 602 for communicating information and command selections to processor 604. Another type of user input device is cursor control 616, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 604 and for controlling cursor movement on display 612. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane.

[0127] The invention is related to the use of computer system 600 for implementing the techniques described herein. According to one embodiment of the invention, those techniques are performed by computer system 600 in response to processor 604 executing one or more sequences of one or more instructions contained in main memory 606. Such instructions may be read into main memory 606 from another machine-readable medium, such as storage device 610. Execution of the sequences of instructions contained in main memory 606 causes processor 604 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and software.

[0128] The term “machine-readable medium” as used herein refers to any medium that participates in providing data that causes a machine to operation in a specific fashion. In an embodiment implemented using computer system 600, various machine-readable media are involved, for example, in providing instructions to processor 604 for execution. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device 610. Volatile media includes dynamic memory, such as main memory 606. Transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 602. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infra-red data communications.

[0129] Common forms of machine-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punchcards, papertape, any other physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read.

[0130] Various forms of machine-readable media may be involved in carrying one or more sequences of one or more instructions to processor 604 for execution. For example, the instructions may initially be carried on a magnetic disk of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 600 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 602. Bus 602 carries the data to main memory 606, from which processor 604 retrieves and executes the instructions. The instructions received by main memory 606 may optionally be stored on storage device 610 either before or after execution by processor 604.

[0131] Computer system 600 also includes a communication interface 618 coupled to bus 602. Communication interface 618 provides a two-way data communication coupling to a network link 620 that is connected to a local network 622. For example, communication interface 618 may be an integrated services digital network (ISDN) card or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 618 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 618 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.

[0132] Network link 620 typically provides data communication through one or more networks to other data devices. For example, network link 620 may provide a connection through local network 622 to a host computer 624 or to data equipment operated by an Internet Service Provider (ISP) 626. ISP 626 in turn provides data communication services through the worldwide packet data communication network now commonly referred to as the “Internet” 628. Local network 622 and Internet 628 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 620 and through communication interface 618, which carry the digital data to and from computer system 600, are exemplary forms of carrier waves transporting the information.

[0133] Computer system 600 can send messages and receive data, including program code, through the network(s), network link 620 and communication interface 618. In the Internet example, a server 650 might transmit a requested code for an application program through Internet 628, ISP 626, local network 622 and communication interface 618.

[0134] The received code may be executed by processor 604 as it is received, and/or stored in storage device 610, or other non-volatile storage for later execution. In this manner, computer system 600 may obtain application code in the form of a carrier wave.

[0135] In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from implementation to implementation. Thus, the sole and exclusive indicator of what is the invention, and is intended by the applicants to be the invention, is the set of claims that issue from this application, in the specific form in which such claims issue, including any subsequent correction. Any definitions expressly set forth herein for terms contained in such claims shall govern the meaning of such terms as used in the claims. Hence, no limitation, element, property, feature, advantage or attribute that is not expressly recited in a claim should limit the scope of such claim in any way. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.