US 8,645,517 B2Grant
Policy-based content delivery network selection
Issue Date:2014-02-04
•20 Claims
•32 Drawing Sheets
Abstract
In a framework wherein resources of a content provider may be delivered to clients from different domains, a method distributes the requests based on content-provider policies. In some cases, the domains include at least two distinct content delivery network (CDN) domains. The domains may include a content provider domain. Responsive to a request, either the content provider domain or one of the two CDN domains is selected, the selection being based at least in part on one or more policies set by the content provider.
Metadata
Assignee
- Level 3 Communications, LLC
Inventors
- Paul E. Stolorz
- John K. Salmon
- Michael S. Warren
- Jeffrey G. Koller
- Maksim Yevmenkin
- Mark Brady
- David Pfitzner
- Ted Middleton
- Aric Hagberg
Application Information
Application Number:US 12/978,537
Filing Date:2010-12-24
Priority Date:2001-09-28
Art Unit:2451
Classifications
IPC:
G06F15/173G06F15/16
Patent Drawings (32 sheets)
Description
RELATED APPLICATIONS
[0001] This application is a continuation of U.S. application Ser. No. 11/976,648, filed Oct. 26, 2007, titled, “Policy-Based Content Delivery Network Selection,” (which will issue on Dec. 28, 2010 as U.S. Pat. No. 7,860,964) and which is a continuation-in-part (CIP) of U.S. application Ser. No. 10/259,497, titled “Configurable adaptive global traffic control and management,” filed Sep. 30, 2002, now U.S. Pat. No. 7,822,871, the entire contents of which are hereby incorporated herein by reference for all purposes. Application Ser. No. 10/259,497 is related to and claims priority from provisional U.S. Patent Application No. 60/325,177, titled “Configurable Adaptive Global Traffic Control and Management,” filed Sep. 28, 2001, the entire contents of which are incorporated herein by reference for all purposes. This application is also related to and claims priority from U.S. application Ser. No. 11/932,162, filed Oct. 31, 2007, the entire contents of which are incorporated herein by reference for all purposes.
RESERVATION OF COPYRIGHT
[0002] This patent document contains information subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent, as it appears in the U.S. Patent and Trademark Office files or records but otherwise reserves all copyright rights whatsoever.
FIELD OF THE INVENTION
[0003] Aspects of the present invention relate to network traffic management. Other aspects of the present invention relate to configurable, adaptive, global traffic control and management in networks such as the Internet.
BACKGROUND & SUMMARY
[0004] As the volume of Internet traffic grows, providers of web content and applications increasingly need to deliver content from multiple servers at widely-separated locations in order to sustain a good end-user experience under high traffic loads. This need generates several difficult challenges, including, among others:
-
- [0005] how to guarantee the fault-tolerance of such a multiple-server system in the face of failure of one or more individual servers;
- [0006] how to control the way in which requests from end-users are distributed to each server according to important content provider policy constraints imposed for economic, contractual or other reasons; and
- [0007] how to guarantee high performance as experienced by end-users as network conditions change.
[0008] This invention solves these and other problems by providing a means to distribute network (e.g., Internet) traffic according to a configurable set of rules. The rules can be configured to take into account key factors such as:
-
- [0009] server availability.
- [0010] specific requirements of content providers who deploy the invention, e.g., distribution based upon geography, position in IP address space, load share, etc.
- [0011] state of the network (Internet) at any given moment, including measures of network latency.
[0012] These rules together provide an extremely fine-grained level of network Internet traffic control to providers of Internet content and applications, enabling them to dramatically improve the end-user experience (measured by speed of request resolution, associated download time, and the availability of servers) over that provided by conventional web servers and mirrored server farms.
[0013] There are many potential uses for the invention. One use is to provide a stand-alone service directing traffic exclusively to a set of designated servers managed by a single organization. The invention may also be used in more general ways—for example, one or more of the designated destinations can refer to servers (or server collections) outside the organization's control. The latter case includes, for example, Content Delivery Networks (CDNs), as well as local load-balancing servers, as potential destinations. The invention can also be used, e.g., to provide the DNS (Domain Name Service) component of a Content Delivery Network itself It can be deployed as a service on behalf of subscribers, or it can be deployed as software to be used directly by subscribers themselves.
BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The present invention is further described in terms of exemplary embodiments, which will be described in detail with reference to the drawings. These embodiments are non-limiting exemplary embodiments, in which like reference numerals represent similar parts throughout the several views of the drawings, and wherein:
[0015] FIG. 1 depicts a configurable adaptive global traffic control and management framework;
[0016] FIG. 2 describes exemplary types of policies;
[0017] FIG. 3 shows an exemplary hierarchy or a decision tree built based on subscriber policies;
[0018] FIG. 4 shows an exemplary relationship between a subscriber's network hierarchy and subscriber policies;
[0019] FIG. 5 depicts a high-level architecture of an adaptive traffic control framework;
[0020] FIG. 6 depicts a high level functional block diagram of an administrative master agent (AMA);
[0021] FIG. 7 depicts the internal functional block diagram of a monitoring mechanism;
[0022] FIG. 8( a) is an exemplary flowchart of a process, in which a content delivery framework provides adaptive policy-based domain name service;
[0023] FIG. 8( b) is an exemplary flowchart of a process, in which a domain name server resolves a hostname based on policies;
[0024] FIG. 8( c) is an exemplary flowchart of a process, in which a monitoring mechanism of a domain name server dynamically monitors the name service operations as well as the availability and the load share status of servers;
[0025] FIG. 8( d) is an exemplary flowchart of a process, in which an ATC administrative network maintains dynamic policies and monitors the operations of a content delivery framework;
[0026] FIG. 8( e) is an exemplary flowchart of a process, in which an ATC network monitoring mechanism traps events from different domain name servers and generates alerts when necessary;
[0027] FIG. 9( a) shows an exemplary secure web based graphical interface, through which a subscriber may define load share and shed fraction policies among static resources;
[0028] FIG. 9( b) shows an exemplary interface for defining policies for dynamic servers;
[0029] FIG. 9( c) shows an exemplary graphical user interface through which an overflow server may be defined using a canonical name;
[0030] FIGS. 10( a)-10(o) show exemplary subscriber policies; and
[0031] FIGS. 11( a)-11(c) show exemplary archived log information that can be displayed and viewed through a graphical user interface.
DETAILED DESCRIPTION
[0032] The invention is described below, with reference to detailed illustrative embodiments. It will be apparent that the invention can be embodied in a wide variety of forms, some of which may be quite different from those of the disclosed embodiments. Consequently, the specific structural and functional details disclosed herein are merely representative and do not limit the scope of the invention.
[0033] Although there are a number of different scenarios in which the invention might be deployed, this description will focus, for clarity and example only, on a scenario in which DNS service is provided by a third party on behalf of a content or applications provider.
[0034] FIG. 1 depicts a high-level of a framework or system 100 according to embodiments of the present invention. A subscriber 102 provides content via a subscriber server network 104 which is made up of one or more server groups 106-1, . . . , 106-k (collectively 106).
[0035] As used in this description, the framework according to embodiments of this invention contemplates three groups of users:
[0036] An end-user who wishes to access content and/or applications over the Internet.
[0037] An operator who offers and supports the service on a managed network of computers.
[0038] A subscriber (typically a content or applications provider) who subscribes to the operator's service to help to support the needs of the end user.
[0039] The terminology end-user, operator and subscriber are used throughout this description to distinguish these three roles, although there are many scenarios in which more than one role can be taken by a single entity. Such scenarios are contemplated by this invention.
[0040] Each server group 106 may correspond to a server hierarchy which includes one or more tiers of servers. For example, the first tier of a server hierarchy may comprise one or more primary servers and the second tier of the server hierarchy may comprise possibly one or more overflow servers that are used when the primary servers in the first tier fail to function properly. In general, for the j-th server group, the primary servers at the first tier of server group 106-j are denoted 108-j, and the overflow servers at the second tier of server group 106-j are denoted 110-j.
[0041] Each server in a server group is any process or collection of processes that provide resources in response to requests, e.g., from a client. A server can be any off-the-shelf Web server. In some embodiments, servers are typically a Web server such as the Apache server or Netscape Communications Corporation's Enterprise™ server.
[0042] Client 112 accesses the subscriber server network 104 in order to obtain content from the subscriber. Content includes any kind of data, including, without limitation, video and audio data and the like. To achieve this access, a user at client 112 enters a resource locator, e.g., a Universal Resource Locator (“URL”), into a browser 114 on client 112. URLs specify the location of resources (information, data files, etc.) on the network. URLs are defined in detail in T. Berners-Lee et al, Uniform Resource Locators (URL), Network Working Group, Request for Comments: 1738, Category: Standards Track, December 1994, located at “http://ds.internic.net/rfc/rfc1738.txt”, which is hereby incorporated herein by reference. URLs generally have the following form:
-
- [0043] scheme://host[:port]/url-path
where “scheme” can be a symbol such as “file” (for a file on the local system), “ftp” (for a file on an anonymous FTP file server), “http” (for a file on a file on a Web server), and “telnet” (for a connection to a Telnet-based service). Other schemes can also be used and new schemes are added every now and then. The port number is optional, the system substituting a default port number (depending on the scheme) if none is provided. The “host” field (the hostname) maps to one or more particular network addresses for particular computer(s). The “url-path” is relative to the computer specified in the “host” field. A url-path is typically, but not necessarily, the pathname of a file in a web server directory. Those skilled in the art will realize and understand, upon reading this description, that the client may not actually enter a URL, but needs only ran an application that needs to look up a domain name (i.e., perform a DNS query). A browser does this when a client enters a URL, but other applications may perform DNS queries in other manners.
- [0043] scheme://host[:port]/url-path
[0044] The framework or system 100 includes at least one Domain Name Service (DNS) name server 118-1. In preferred embodiments, the system 100 also includes DNS name servers 118-1, 118-2, . . . , 118-n, (collectively referred to as name servers 118) all operated by a single particular entity. In the embodiment shown in FIG. 1 , one of the name servers 118-1 serves the U.S.A., another name server 118-2 serves the U.K., another name server 118-3 serves Japan, and so on. The various name servers 118-1, 118-2, . . . , 118-n can be configured (collectively or individually) to serve geographical regions, geopolitical regions, corporate structures or any other logical organizational structure. The structure and operation of each name server 118 is described below.
[0045] When the client's browser 114 obtains a request (e.g., in the form of a URL), the browser queries its resolver 116 for an address for the hostname specified in the requested URL. The resolver 116 eventually queries a particular name server (e.g., name server 118-1). The name server 118-1 returns (provides or attempts to provide) the IP (Internet Protocol) address (or addresses) of a server (or servers) in the subscriber server network or a CNAME (a domain name). Other DNS answer types (e.g., MX record, NS record, etc.) are also possible, as described below. The determination of the particular IP address returned to the resolver 116 may be based on a number of factors, including the resolver's location (e.g., as determined from the resolver's IP address) and various policies (e.g., subscriber policies 120, other policies 122) in a policy database 124. The client's browser 114 is then able to communicate with the selected server in the subscriber server network in order to obtain the desired resource.
[0046] Name servers 118-1, 118-2, . . . , 118-n, according to embodiments of the present invention, include an adaptive traffic control (ATC) mechanism 126 which provides domain name service to a client based on policies in the policy database 124. Each name server 118 includes or has access to a location determination mechanism 128 for associating the client's request with the client's location. Any program that can determine a requestor's location may be used.
[0047] The subscriber server network 104 represents a network of servers that provides, on behalf of an underlying subscriber, Internet content or services. For example, a subscriber may be a content provider, which has its own network of servers that deliver content to end users' browsers via the Internet. The subscriber server network 104 may be configured in such a way that the processing of the service requests may be reasonably distributed among all the servers in the server network 104, according to some criteria. To achieve that distribution, the subscriber server network 104 may be organized, for example, to have one or more server groups (e.g., server group 1 106-1, . . . , server group k 106-k), each of which may be responsible for processing a portion of the service requests. For example, service requests coming from Finland may be routed to a server group that is physically located in Europe.
[0048] Each server group 106 in the subscriber server network 104 may comprise a plurality of servers to further share the load. For example, a service request from Finland may be subsequently routed to a server located in Scandinavia. Routing service requests to different server groups and subsequently to different servers may be based on various criteria. For example, such routing may be based on the distance between the origin of the request and the location of the server. For example, for a service request originated from Finland, it may be more effective and efficient to direct the request to a server located in Norway instead of directing it to a server located in the U.S. Server load may also be used to determine where to route a service request. For example, a service request originated from the U.S. may be routed to a server group in Mexico if the server group in the U.S. is overloaded while the server group in Mexico is relatively idle.
[0049] Servers in a server group may be organized into a hierarchy with one or more tiers of servers. Servers at different tiers may have different designated purposes. For instance, servers at the first tier of a server hierarchy may include servers that are primary functioning servers, servers at the second tier may include servers that are used as overflow servers which become active only when the primary servers at the first tier become unavailable or overloaded, and servers at the third tier that are used as second layer overflow servers that become active only when the primary servers and the overflow (or first layer overflow) servers at the second tier become unavailable or overloaded, etc.
[0050] The first tier of a server group hierarchy may include one or more primary servers. When multiple primary servers are present, they may be configured in such a way that they share the load. For example, if there are five primary servers at the first tier, they may be configured so that each takes 20% of the total service requests routed to the server group. For each of such primary servers, it may be further configured so that when a particular server is overloaded or fails, the load share originally designated to this server may be shed or directed to other server(s). The load shed may also be configured when the server is not overloaded. The portion to be shed to other server(s) may be governed by certain pre-defined policies.
[0051] As discussed above, servers in a server group 106 may be classified into different categories based on specific functionality of the underlying servers. For example, a server in a server group 106 may be a primary server, a first layer overflow server, or a second layer overflow server, etc. A server may also be classified in terms of whether and how a server is to be dynamically monitored. For instance, in some embodiments, a server can be classified as either a monitored server or a managed server.
[0052] A classification of a monitored server indicates that the underlying server is to be monitored dynamically for its availability. In this case, the server may be probed for its availability according to some pre-determined schedule. Such a probe may be sent to the underlying server from different locations of the network so that the availability can be detected accurately. That is, if the underlying server failed to respond to one probe sent from one location, it does not necessarily indicate that the server is no longer available (could be due to that only part of the network is congested).
[0053] A probe can be realized in different fashion. It can be simply a signal sent to the server to request an acknowledgement. It can also be a poll operation in which a file stored at a designated location on the underlying server is polled. If the file can be successfully polled, the underlying server is considered to be available. The detected availability may be used to adjust or update policies associated with the server so that the network traffic management will respond to the dynamics related to the server.
[0054] A server that is classified as a managed server may be actively participating the adaptive policy-based management scheme. Similar to a monitored server, a managed server may be regularly probed for its availability. In addition, a managed server may dynamically provide information related to its load share or load shed. A managed server may update its load share or load shed fraction according to its changed capacity or its current load. For example, when a managed server is upgraded, its capacity may be increased so that it may increase its load share or decrease its load shed fraction. On the other hand, when a managed server is overloaded, it may revise its load share to a lower level or increase its load shed fraction to prevent failure. A managed server may revise its load share or load shed by updating the corresponding load share or load shed information in a designated file stored on the server and this file may be polled by a name server so that the dynamically changed load share and load shed information can be used to direct traffic accordingly.
[0055] The distribution of service requests (from a client 112) within the subscriber server network 104 may be controlled through a set of ATC policies (120, 122) stored in the policy database 124. Various authorities may influence the ATC policies and may create policies in the policy database 124. For example, the subscriber may set up policies to direct traffic with respect to considerations such as the geographical locations and the capacities of the underlying servers in the subscriber's network 104. Other policy entities 138, including, for example, the operator of the name servers 118 or various geo-political entities may also have policies regarding how the network traffic should be managed and directed. For example, governmental or some organizational agencies may regulate some aspects of network traffic policies. Such regulation policies may be required to be incorporated so that a service request from the client 112 can be routed in a manner that satisfies regulatory policies.
[0056] Thus, the ATC policies in the policy database 124 may be a combination of subscriber policies 120 and other policies 122 from different sources such as subscriber 102 and policy entity 138. Policies from different sources may be accessed by the ATC mechanism 126 from the policy database 124. From the point of view of the ATC mechanism 126, the source of a policy may not be relevant or even determinable. For example, policies may be defined in a textual file stored at a designated location, which may be downloaded to an ATC policy management mechanism 152 in an ATC administrative framework 142 and then broadcast to the database manager 132 located in each of the name servers in the ATC name server network 140. The download may be via either a graphical user interface (GUI), a file transfer protocol (FTP), or some other mechanism. Policy makers may also enter policies directly via a web-based GUI. For example, the subscriber 102 may enter subscriber policies 120 via a browser interface 156 connected with the ATC administrative framework via, preferably a secure interface (e.g., implemented using the “https” protocol).
[0057] The policies from the policy database 124 used by the ATC mechanism 126 are collectively referred to as ATC policies, which may include, not is not limited to, the subscriber policies 120 as well as other policies 122. The ATC policies may be organized in a manner that is appropriate to govern and/or control the traffic at different levels of the subscriber server network 104. FIG. 2 shows exemplary types of ATC policies 200 according to embodiments of the present invention.
[0058] The ATC policies (200) may be classified into different types such as geo-political policies 215, load share policies 240, failover policies 245, tiered failover policies 250, shedding policies 255, regulatory policies 260, and Classless Inter-Domain Routing (CIDR) block policies 210.
[0059] These exemplary policies in the policy database 124 are described in greater detail below:
[0060] Geographic Policy (215): Decisions are based on location of the end-user or an approximate thereof, e.g., using the IP address of an end-user's resolver. For example, if the IP address is within the U.S., the request may be directed to a “domestic” group of servers in the subscriber server network 104, otherwise it may be directed to “international” servers in the network. The request may be directed to a proprietary Content Delivery Network (CDN) or to another service specified by the subscriber.
[0061] Load Share Policy (240): The subscriber can explicitly specify the amount of traffic (load share) to be directed to each of their servers within a defined server set in the subscriber server network 104. Typically the load may be specified according to the capacity of each server. Based on such specification, the load share of each server may be derived as a percentage of the total load.
[0062] Failover policy (245): The subscriber may specify policies regarding a failover situation where some of the primary servers fail to function. To take care of such a scenario, a failover policy may instruct the ATC mechanism in terms of, for example, how often to monitor the availability of the servers and what strategy to adopt when a partial set of the primary servers are detected to be unavailable. For instance, a failover policy may specify to distribute the load of a failing server to other primary servers. It may also alternatively instruct the ATC framework to direct traffic to servers other than the primary servers.
[0063] Tiered Failover Policy (250): The subscriber may specify a strategy through tiered failover policies by which the load should be re-directed to servers at a next tier when one or more subscriber servers in a previous tier fail to function. For example, in defining the subscriber server network, each server group may be configured as a hierarchy, having the first tier of primary servers, the second tier of first layer overflow servers, and the third tier of second layer overflow servers, etc. In this case, a tiered failover policy may be defined to indicate when the load should be re-directed from the primary servers to the overflow servers. For instance, a tiered failover policy may indicate that when all the primary servers fail, the load should be directed to the second tier, and when servers at both the first tier and the second tier fail, the load should be directed to the overflow servers at the third tier. The traffic may also be re-directed to some other servers. For example, when a content delivery network (CDN) is available, the traffic may be re-directed to the entire CDN.
[0064] Shedding Policy (255): In some circumstances, a fraction of the “load” originally designated to a server may be shed or re-directed to one or more different servers. This may occur when the amount of traffic directed to the subscriber server exceeds a prescribed level. In these cases, a subscriber-specified fraction of traffic (shed fraction) that would otherwise be directed to the server may be shed to one or more other different servers. Such strategy may be adopted to prevent catastrophic failure due to overload. The servers that take the shed load may be an overflow server or some other servers such as a content delivery network. An overload situation may be detected according to the response time of the server. For example, if the response time from a server becomes long, it may indicate that the server is overloaded. In this case, shedding policies 255 may be invoked to re-direct the traffic elsewhere. For instance, if there are a total of 3 primary servers in a server group with load share of (0.3, 0.3, 0.4) and the primary server that is designated to take 40% of the total load is completely overloaded, a shedding policy may specify to shed the load of this server by re-directing 50% of its original load to, for example, the servers located in the CDN 105. A shedding policy may also specify a condition upon which the traffic will be directed again to the shedding server. Such a condition may relate to a desirable level of performance of the shedding server.
[0065] CIDR Policy (210): Policy decisions are supported based upon CIDR blocks of IP address space. CIDR denotes Classless Inter-Domain Routing, an IP addressing scheme that replaces the system based on classes A, B, and C. With CIDR, a single IP address can be used to designate many unique IP addresses.
[0066] Regulatory policy (260): Certain policies may be specified by some policy entities to control network traffic. Such policies may be enforced in system 100.
[0067] A policy may be static or dynamic. Selection of a server from the subscriber server network 104 may be based on an adaptive, regularly updated map of the state of the Internet as well as adaptively updated policies. The map may cluster IP addresses together according to their network latency to a selected set of network agents. This enables the subscriber servers to be selected according to their “network proximity” to an end user's browser, optimizing resulted download time. The policies may be defined in such an adaptive manner that they reflect the dynamic status of the servers such as the availability and load.
[0068] Geo-political policies 215 may govern the selection of a server according to where the client is located. As shown in FIG. 2 , the geo-political policies 215 may be further classified into continental based distribution policies 220, country based distribution policies 225, region based distribution policies 230 and time zone based distribution policies 235. These policies may be set up to govern the process of selecting a particular server group according to different geographical or political criteria.
[0069] Geo-political policies differ from load based policies (described above). The former is designed to guide selections based on geographical criteria or time criteria. The latter concerns the selection process with respect to the dynamic capacity and load of the underlying servers. The load share policies 240 govern the selection according to the capacities of the servers. Partial failover or tiered failover policies (245 and 250) govern the traffic re-direction process when functioning servers in a server group are overloaded or failed.
[0070] The ATC policies 200 may be hierarchically constructed to form a decision tree. FIG. 3 depicts an exemplary subscriber policy hierarchy or a decision tree built based on various types of subscriber policies, according to an embodiment of the present invention. In FIG. 3 , the continental based distribution policies 220 are used to first direct a selection process limited to a particular continent. For example, if the requesting client 112 is located in Asia, the continental based distribution policies 220 may direct the further selection within servers that are located in Asia. Similarly, if the client 112 is located in either North America or Europe, the further selection process may be directed to the appropriate servers located in the corresponding continent.
[0071] As noted, a policy may be created by generating a decision tree representing the ATC rules that the user (content provider/subscriber) wants to apply. At the leaves of this tree (referred to as resource nodes) a user specifies the answers (IP addresses or CNAMEs) that the user wants ATC to provide in response to a DNS request. At the branches of this tree (referred to as branch nodes) the user specifies the various decision criteria that the user wants to apply. Those skilled in the art will realize and understand that, in this discussion about policy setting and control of the ATC, the term “user” (as opposed to “end user”) generally refers to the content provider/subscriber whose content is being delivered.
[0072] Branches within a policy decision tree enable ATC decisions based, e.g., upon the IP address of the requesting resolver. Branches can be selected based upon different criteria. In a presently preferred exemplary embodiment, branches can be selected based on five different criteria:
-
- [0073] Split on World zone
- [0074] Split on country
- [0075] Split on U.S. state
- [0076] Split on U.S. time zone
- [0077] Split on block of IP addresses—specified, e.g., using CIDR classless notation: e.g. 1.2.0.0/16, 1.2.3.4/32 (single IP), or 0.0.0.0/0 (any IP).
[0078] Those skilled in the art will understand, upon reading this description, that different and/or other criteria may be use to make branch selections. Examples of such criteria include, without limitation, time of day, day of week, etc.
[0079] Resources are selected to occupy the leaves of a policy decision tree. Resources are used to specify the ATC answers to DNS queries. ATC offers substantial decision making power during the resource selection process. It is here, for example, that traffic can be distributed according to load share and shed fraction criteria. In a presently preferred exemplary implementation there are five types of resource nodes that can be selected in the policy decision tree:
-
- [0080] IP addresses (Internet Protocol addresses)
- [0081] CNAME records (Canonical Name records)
- [0082] MX records (Mail Exchange records)
- [0083] NS records (Name Server records)
- [0084] Load sharing server sets (LSSS)—consisting of Managed, Dynamic, Static and Overflow servers.
[0085] Those skilled in the art will understand, upon reading this description, that different and/or other resource nodes may be used.
[0086] IP address(es) are used when the current decision tree leaf points to one or more fixed IP addresses. CNAME records are used when the current decision tree leaf points to a destination whose CNAME is known.
[0087] MX Records are used when the current decision tree leaf points to one or more MX (Mail Exchange) records. MX records may be combined with IP address or Load Sharing Server Set resources. To select this option, the user firsts enters the Exchange, Pref, and TTL (Time To Live) details for the mail exchange. The Exchange field holds the domain name of the host which will receive mail. If the Exchange name lies under the Domain Name Origin, then the policy must specify an IP address for the name by means of another node label and appropriate resource (IP address or Load Sharing Server Set). A Pref field holds the preference order of the host; lower numbers are preferred (used first) over large numbers.
[0088] NS Records are used when the current decision tree leaf points to one or more NS (Name Server) records. NS records are used to specify a delegation from the ATC name servers to another set of name servers.
[0089] Load Sharing Server Set (LSSS) option is used when the current decision tree leaf points to a number of destinations among which the user wants to share load (including failover) according to arbitrary criteria. In presently preferred exemplary implementations, this decision takes place only after any decisions that the user wants to make via branch nodes based on the IP address of the requesting resolver. There are presently four different types of load sharing server resources, namely static servers, managed servers, dynamic servers, and overflow servers.
[0090] In a present implementation, managed Servers are preferred over the other server types.
[0091] Load sharing server sets (LSSS) are necessary to utilize the load share, tiered/overflow and failover rules. When an ATC name server encounters a server set, one or more answers are chosen based on several criteria, including, e.g., the load share of each server, the tier in which each server sits, the shed fraction of each server, and the monitoring results of each server.
[0092] Static, dynamic and managed servers have load share, shed fraction and, when in a set, tier parameters. An overflow server is, by definition, at the lowest tier and cannot shed; thus it has only the load share parameter. Dynamic and managed servers each may have the load share and shed fraction values updated continuously and may be deemed on or off-line due to monitoring.
[0093] When answering a DNS query from a LSSS, ATC preferably selects answers (IP addresses or a CNAME) from servers included in the set. ATC examines servers in tier order (tier 1 first, tier 2 next, and so on, with overflow servers last), selecting servers based on load share, shed fraction and monitoring results. Online servers are selected within a tier according to their load share values relative to other servers in the same tier. If a candidate server is shedding, e.g., 10% (i.e., has a shed fraction of 0.1), ATC will keep that server in the DNS reply nine tenths of the time, and look in the next tier for a server one tenth of the time. If there is only one tier, the shed traffic returns to the same tier and another server is selected. If the LSSS specifies that multiple answers should be returned, ATC will look for each answer starting from the first tier. If any server selection falls to the overflow servers, one of these is selected as the sole answer to the DNS reply. Typically, the overflow server is a CNAME, and DNS answers of this type should not be combined with any other record type.
[0094] In a presently preferred implementation, the number of answers chosen from Load Sharing Server Sets depends on two more user-settable parameters, the “number of answers desired” (NAD) and the “number of answers required” (NAR). These parameters control how many answers are picked from non-overflow servers.
[0095] The user sets the NAD parameter to the typical number of answers to be selected by ATC and returned in DNS queries. The NAR parameter, which must be less than or equal to the NAD, comes into play when the normal ATC decision making process cannot select the desired number of servers specified in NAD. This server count shortfall situation can arise when one or more servers are shedding or offline. Setting NAD to one (1) or more guarantees that ATC will never reply to a DNS query with no servers. When NAD is greater than one, ATC may deviate from the desired load share ratios, e.g., if there are three servers and NAD is three then DNS replies will typically have all three servers, irrespective of their load share values.
[0096] If the number of servers selected is less than NAD, but greater than or equal to NAR, then server selection is finished. However, if the number of servers selected is less than NAD and less than NAR, then ATC progressively ignores shed fraction and load share values until it is able to select NAR servers. As a special case, if ATC cannot select any servers because they are all offline, all servers are returned in the DNS reply (subject to size constraints of the DNS packet).
[0097] As an example, consider a LSSS with two servers in tier 1, two servers in tier 2, and two answers desired and required (i.e., NAD=2, NAR=2). If both servers in tier 1 are online and not shedding, those servers will be returned. If one of the tier 1 servers has 10% shed, then 10% of the answers contain one server from tier 1 and one server from tier 2. If both servers from tier 1 are offline, the two servers from tier 2 are returned. If all four servers are offline, then all four are returned (the special case).
[0098] In some implementations, the LSSS may also have a Style parameter which may be used to control where traffic goes when it is directed away from a shedding server, or from an offline server, according to the following table.
[0099]
| Style | Shed Traffic | Off-line Traffic | |
|---|---|---|---|
| Constant Tier Load Ratio | Next Tier | Same Tier | |
| (default) | |||
| Constant Server Load | Next Tier | Next Tier | |
| Ratio | |||
| Maximal Tier Load | Same Tier | Next Tier | |
[0100] Consider, e.g., a LSSS which contains a few servers in Tier 1 and an overflow server sending traffic to a CDN. The three styles above relate to three underlying motivations for how to split traffic between the Tier 1 servers and the CDN.
[0101] If the Tier 1 servers should always take a specified percentage of traffic, as a whole, then the Constant Tier Load Ratio style is best. For example, for the CDN to always take 50% of the traffic, each of the Tier 1 servers would have their shed fraction set to 0.5. This way, even if servers are added or removed from Tier 1, that tier and the CDN will always get 50% of the traffic. This, of course, means that the load on the Tier 1 servers will vary as servers are added or removed.
[0102] If, instead, it is not acceptable for the Tier 1 servers to increase their load upon the failure of another Tier 1 server, the Constant Server Load Ratio style is best. With this style, both shed traffic and traffic that would have gone to an off-line Tier 1 server are directed to the next tier (in the example above, the CDN). This is a preferable scenario if the Tier 1 servers are generally under high load. Lastly, if the Tier 1 servers are generally under low load and the CDN is to be considered a last resort, the Maximal Tier Load style should be used. With this style, both shed traffic and traffic that would have gone to an off-line Tier 1 server are instead directed to other servers in Tier 1. Traffic only gets to the CDN if all servers in Tier 1 are offline or shedding.
[0103] The Static Server option is used to share load at the current decision tree leaf among a set of servers that the user does not expect to change often over time. The load share for a given server represents the amount of traffic that will be sent to that server within the same tier of the server set. The fraction of traffic sent to that server is computed by summing the load share numbers for all servers in the same tier of the server set and dividing the load share by that total. The shed fraction for a given server represents the fraction (from 0 to 1) of traffic that should be redirected away from a given server, after the initial decision based upon load share. The online flag can be used to remove a server from the list of ATC answers if the user wishes to take a server off-line for maintenance. (If a server has been switched off-line, end users may continue to access it for the duration of the TTL, or perhaps longer if the end user's resolver or browser ignores the TTL.)
[0104] In addition to default values for load share and shed fraction, the managed server may optionally be monitored to determine the on/off-line status, and polled for load feedback information (updated values for the load share and shed fraction). Furthermore, a managed server may hold either an IP address or a CNAME. A managed server holding an IP address without monitoring or load feedback becomes equivalent in functionality to the static server. Managed servers with monitoring only (load feedback not configured) are used when the user expects the load share and shed fraction of the servers to change infrequently, but want an instant reaction if the user's server becomes unavailable. In a presently preferred implementation, ATC monitors the user's server by polling a user-specified resource (denoted by a user-specified URL) at regular intervals (e.g., every 30 seconds) from each ATC name server. The URL scheme may be, e.g., HTTP, HTTPS or TCP. With HTTP(S), ATC name servers independently send a HEAD request, optionally with HTTP's basic authentication, and examine the response code. Response codes from 200 to 399, received within 15 seconds of the probe, put the server in an online state as viewed by the probing name server; all other conditions (connection refused, server 400 or 500 error, 15 second time, etc.) lead to an offline state for the server until the next probe.
[0105] Alternatively, ATC can monitor the user's server by checking the ability to connect to a specified port. In this case, the URL format is tcp://HOST[:port], and the ATC name servers simply attempt to establish a TCP connection to determine the online state of the user's server. The same probing rate and timeout durations apply.
[0106] Suppose, for example, that a user wishes to use ATC for name service for download.example.com. Suppose too that the user has two servers and wants ATC to monitor them both and only return the IP address of an available server. In this example, the user could define two Managed servers, each with a Monitored URL as shown in the following table:
[0107]
| Server nick name | IP Address | TTL | Monitored URL | |
|---|---|---|---|---|
| SF server | 10.0.0.1 | 2 m | http://10.0.0.1 | |
| NY server | 10.0.0.2 | 2 m | http://10.0.0.2 | |
[0108] The Monitored URL can point to any resource available to the web server, in this case the web server's root. When these servers are used in a server set, ATC continuously polls the monitoring URLs to determine the on/off-line state of the user's servers.
[0109] A user may specify the load share and/or shed fraction of the user's servers dynamically, using a Load Feedback URL to specify an XML resource file (at a location of the subscriber's choice) in the following form (preferably plain text):
-
- [0110] <ATCServerStatus
- [0111] loadShare=“LS”
- [0112] shedFraction=“SF”
- [0113] onLine=“OL”
- [0114] expireTime=“ET”/>
[0115] The file must be made accessible by the subscriber via http or https, optionally with basic authentication. In a presently preferred implementation, the onLine field is required, and all others are optional. LS is a number greater than (or equal to) zero, SF is a number from 0 to 1, OL is either True or False, and ET is a time (either absolute GMT or relative).
[0116] The expireTime parameter is the absolute time at which the current XML file expires, or the relative amount of time, post-probe, at which the current XML file expires. After a retrieved XML file expires, the loadShare and shedFraction return to their default values. In this manner a user (i.e., content provider) can temporarily (or permanently) override default load share and/or shed values.
[0117] The Managed Server specification with both Monitoring and Load Feedback may then look like the following table:
[0118]
| Monitoring | ||||
|---|---|---|---|---|
| Server Name | IP | TTL | URL | Load Feedback URL |
| SF Server | 10.0.0.1 | 2 m | http://10.0.0.1 | http://10.0.0.1/atc- |
| feedback.xml | ||||
| NY Server | 10.0.0.2 | 2 m | http://10.0.0.2 | http://10.0.0.2/atc- |
| feedback.xml | ||||
[0119] The load feedback resource does not need to be on the server itself. The user may wish to use an independent server to control the load share and shed fraction of some or all of the user's servers. The state of the user's managed servers is maintained by each ATC name server as follows:
[0120] If the user's server uses neither a monitored URL nor a load feedback URL, it is considered to be always on-line with default load share and shed fraction.
[0121] If the user's server uses a monitored URL but not a load feedback URL, it is considered to be on-line when the monitored URL is successfully retrieved (HTTP response code from 200 to 399 within 15 seconds), otherwise it is considered to be off-line.
[0122] If the user's server uses both a monitored URL and a load feedback URL, the server status is preferably given by the following table:
[0123]
| Load Feedback URL | ||
|---|---|---|
| Contents | Monitoring Down | Monitoring Up |
| No XML | Off-line | On-line with defaults |
| No XML, last known still | Off-line | Last known |
| valid | ||
| On-line | Off-line | On-line with defaults |
| On-line, LS and SF | Off-line | On-line with LS and SF |
| Off-line | Off-line | Off-line |
| Off-line, LS and SF | Off-line | Off-line |
[0124] In the above table, “No XML” means that either the XML file could not be retrieved within the 15 second time-out period (e.g., due to network error), the XML file was retrieved but could not be parsed, or the XML file had expired. “Last known” refers to the full contents of the last successfully retrieved XML file. Last known values expire and cease to exist at the expire time if the expire time is specified in the XML file. The term “defaults” refers to the default load share and shed fraction values specified in the Managed resource server.
[0125] If the user's server uses the Load Feedback URL but no Monitored URL, the user can achieve an N minute delayed failover by putting in the XML snippet a relative expireTime of N minutes (+Nm) and a loadshare of one, while having in the policy a default Load Share of 0.
Dynamic Server
[0126] The Dynamic Server option may be used to share load at the current decision tree leaf among a set of servers when the user expect that the load share, shed fraction, or server availability may change reasonably often. (The managed server option is preferred over the dynamic server option.) Load share and shed fraction are defined for this case as they are for Static Servers. However, the Dynamic Servers selection offers flexibility in exactly how they are specified. The Dynamic Server option supports specification of load share, shed fraction and the On line flag dynamically, allowing them to be defined by the subscriber via a short XML status file at a location of the subscriber's choice (as described above).
[0127] The subscriber specifies the location of this file to ATC, including any authentication requirements desired. ATC polls the file at predetermined fixed intervals, e.g., every 30 seconds, and updates the current ATC policy to reflect its contents. Subscribers can easily specify changes to load share, shed fraction and the On line flag by editing the file, either manually or by updating it using automated modules, without requiring updates to an entire ATC policy.
[0128] In order to maintain flexibility, values for the “URL Down load share” and the “URL Down shed fraction” parameters can be set in the ATC policy. They are used to control the behavior of ATC under the circumstance that the ATC polling mechanism is unable to retrieve the status file for a server, or is unable to parse its contents.
Case 1
[0129] Subscriber specifies all three ATCServerStatus attributes in the status file, with the “URL Down load share” parameter set to zero. In this case the subscriber's server will not be returned as an answer under any conditions until the polling mechanism retrieves a valid XML file.
Case 2
[0130] Subscriber specifies all three ATCServerStatus attributes in the status file, with the “URL Down load share” parameter set to one. In this case, ATC treats a failure to return valid XML according to the “Down load share” and “Down shed fraction” specified in the policy.
Case 3
[0131] Subscriber specifies only the on line ATCServerStatus attribute in the status file. In this case, ATC uses the “Default load share” and “Default shed fraction” specified in the policy if the “On line” flag in the ATCServerStatus file is set to true.
[0132] Below is a table detailing all of the possible cases. In the table, a dash represents ATC taking the dynamic server out of consideration. LS and SF should be replaced by actual numbers.
Dynamic Server Cases
[0133]
Overflow Server
| Form of retrieved | ||
|---|---|---|
| ATCServerStatus file | LoadShare | ShedFraction |
| <ATCServerStatus | LS | SF |
| onLine=“True” | ||
| loadShare=“LS” | ||
| shedFraction=“SF”/> | ||
| <ATCServerStatus | Default load share | Default shed fraction |
| onLine=“True”/> | ||
| <ATCServerStatus | — | — |
| onLine=“False” | ||
| loadShare=“LS” | ||
| shedFraction=“SF”/> | ||
| <ATCServerStatus | — | — |
| onLine=“False”/> | ||
| Bad format or un- | Down load share | Down shed fraction |
| retrievable: | ||
Overflow Server
[0134] A user uses the Overflow Server option if the user would like to specify what should happen if, for some reason (e.g., server unavailability), no answers are selected by the Static or Managed (or Dynamic) Server selection process. Typically, the overflow server will be a CName to a service, such as Level 3 Communications' Content Delivery Network (CDN), that should be available in the case that the user's individual Static or Managed (or Dynamic) servers are unavailable.
Managed and Dynamic Server Status Alerts
[0135] The global ATC network preferably maintains an overall state of a user's Managed and Dynamic servers using Monitored or Load feedback URLs. A Server Status page, available from the subscriber web interface, continuously updates with the status (up or down) of the user's URLs and, if down, the relevant error condition (time-out, connection failure, file not found, etc.). Additionally, the overall server state is displayed (either on-line or off-line) along with a relevant status message. If a server changes state, an email message is sent to the user detailing the event. Email messages are preferably not sent more frequently than once in 15 minutes per domain name origin.
[0136] Once a DNS request is directed to an appropriate continent, the country based distribution policies 225 may further constrain the selection to particular servers that are in the same or close-by countries where the client 112 is located. For example, if the client 112 is located in the U.S., the country based policies 225 may direct the selection from the servers located in the U.S. Similarly, the region based distribution policies 230 may further constrain the selection to, for example, the west coast or east coast depending on where the client 112 is located.
[0137] When a particular server group is selected (e.g., after a hierarchical decisions based on the continent-based policies 220 and the region-based policies 230), the load share policies 240 govern the process whereby servers in a given server group should be selected. Such policies may be determined based on the servers' capacities or may be adaptively revised based on the dynamic performance or load of the underlying servers. The load share policies 240 may specify the percentage (share) of the total requests that each server in a server group should handle. For example, if a server group comprises a total of three primary servers (server 1, server 2, server 3), a load share policy for this server group may specify the load share as (0.3, 0.5, 0.2), indicating that server 1 should take 30% of the total load, server 2 should take 50% of the load, and server 3 should take 20% of the total load.
[0138] The tiered failover policies 250 govern the selection of a server when a particular default set of servers is no longer functioning or available. For example, primary servers in a server group may be considered as a default set of servers that provide service when operation is normal. Unavailability of such primary servers may be detected according to the response time of the server. For example, if a server is simply not responding, the server may be considered as not available. When all the primary servers are down, the tiered failover policies 250 govern where the traffic should be directed. For instance, the tiered failover policies 250 may specify to direct all traffic to the overflow servers at the next tier.
[0139] FIG. 4 depicts an exemplary relationship between the subscriber server network 104 and the subscriber policies 120, according to embodiments of the present invention. The subscriber server network 104 is depicted in FIG. 4 as a network hierarchy or a tree in which the subscriber server network 104 includes server group 1 106-1 through server group k 106-k, and each server group may include its own primary server group 108 in the first tier and overflow server group 110 in the second tier (e.g., server group 1 106-1 has primary server group 108-1 and overflow server group 110-1 associated therewith, while server group k 106-k has primary server group 108-k and overflow server group 110-k associated therewith).
[0140] The subscriber policies 120 govern selection of one or more servers within the subscriber server network 104 (or in some other network of servers) so that their IP addresses may be returned in response to DNS requests from the client 112 (resolver 116). To determine or to select appropriate servers in the subscriber server network 104, the decisions may be hierarchical. For example, as shown in FIG. 4 , at the level of the subscriber server network 104, the ATC mechanism 126 may make geo-political distribution decisions 410 to determine which server group should be selected. Once a particular server group is selected, the ATC mechanism 126 may further determine which particular servers in the group should be selected according to, for example, how the traffic load should be shared among the servers within the server group. Decisions at server group level may be based on the load share policies 240. This decision making process illustrates that, at each branch node in the server hierarchy, appropriate ATC policies may be applied to govern the selection of an appropriate server or servers.
[0141] As illustrated in FIG. 4 , the geo-political policies 215 may be applied to control the selection of a particular server group, the load share policies 240, the tiered failover policies 245 and 250, and the shedding policies 255 may be applied to re-direct traffic. That is, the subscriber server network 104 forms a tree and the ATC policies 200 govern the flow from the top of the tree to one or more leaves of the tree. The ATC mechanism 126 controls the flow by applying the ATC policies 200.
[0142] FIG. 5 depicts a high level architecture of an adaptive traffic control (ATC) framework according to embodiments of the present invention. The ATC framework comprises an ATC administrative framework 142 and an ATC name server network 140. The ATC administrative framework 142 is responsible for various administrative tasks associated with subscribers or other policy making entities (138), include manipulating and storing the ATC policies 200, propagating or broadcasting the ATC policies 200 to name servers in the ATC name server network 140, monitoring name server behavior, generating status reports to display monitoring results on appropriate media, and sending alert to a network operation center (NOC) 148 (see FIG. 1 ) and the subscriber 102.
[0143] The ATC name server network 140 is responsible for responding to DNS requests, including processing DNS requests, applying the ATC policies 200 to select one or more servers from the subscriber server network (or a different designated network), and replying with the IP address(es) of the selected server(s). The ATC name server network 140 dynamically maintains the ATC policies 200, that are either received from the ATC administrative framework 142 or updated according to the dynamic operational status of the servers. The ATC name server network 140 provides domain name-IP address resolutions based on dynamically updated ATC policies 200. The ATC name server network 140 may also monitor the operational status of individual name servers within the network and supply logging and monitoring data to the ATC administrative framework 142.
[0144] The ATC administrative framework 142 may be designed to have fault-tolerance. For example, as depicted in FIGS. 1 and 5 , the ATC administrative framework 142 may include an administrative master agent (AMA) 144, and one or more AMA backups 146-1, . . . , 146-m (collectively 146). All AMA agents, including the master agent 144 and the backups 146 may be capable of performing the exact functions except that one of them (e.g., master AMA 144) may be a central or primary administrative master agent and others (e.g., AMA backups 146) may be backup or secondary or redundant master agents. The central or primary AMA 144 may be responsible for regularly backing up the AMA backup agents 146. When, for whatever reason, the primary AMA 144 is no longer functioning properly, one of the AMA backups 146 may take on the role of the central or master AMA 144.
[0145] The ATC name server network 140 may comprise a plurality of name server agents 118-1, 118-2, . . . , 118-k, each of which may be designed to be responsible for the DNS requests of a particular geographical (or any other administrative or functional) region. For example, the name server agent 118-1 may be responsible for processing all the DNS requests from North America, the name server 118-2 may be responsible for Europe's DNS requests, and the name server 118-3 may be responsible for DNS requests from Japan. In addition, a name server agent may also serve as a back up name server agent for other name server agents in the network. For example, if the North America name server agent is not functioning properly, the Europe name server agent may be temporarily assigned to handle the DNS requests from North America. For that purpose, all ATC policies may be propagated to all of the name server agents in the ATC name server network 140.
[0146] FIG. 6 depicts a high level functional block diagram of an administrative master agent (AMA) 144 according to embodiments of the present invention. The AMA 144 comprises a secure web-based graphical user interface 160, a policy editing mechanism 162, an ATC policy database 124, an administrative policy update mechanism 164, an ATC administrative browser 166, an ATC policy management mechanism 152, a report generation mechanism 168, an ATC network monitoring mechanism 150, and an administrative master backup mechanism 170.
[0147] The ATC policy database 124 stores the ATC policies 200. As discussed above, the ATC policies 200 may include policies from different sources (e.g., from subscribers and from other entities that control network traffic). Both the subscriber policies 120 and other policies 122 may be defined and provided to the AMA 144. In the exemplary configuration shown in FIG. 6 , the policies may be received at the AMA 144 through the secure web based GUI 160. There may be other means through which the policies can be delivered to the AMA 144 and will be discussed later. The policy editing mechanism 162 organizes the received policies to form the ATC policies 200 and then stores them in the ATC policy database 124.
[0148] The ATC policy management mechanism 152 may broadcast or propagate the ATC policies 200 to relevant name servers in the ATC name server network 140 so that the ATC policies 200 may be used to control the domain name translation service. The stored ATC policies may be dynamically updated via different means. For example, the policies may be updated through the ATC administrative browser 166, or the administrative policy update mechanism 164 may revise existing ATC policies.
[0149] Alternatively, the AMA 144 may also be provided with policies from different sources through the ATC policy management mechanism 152, which may regularly poll dynamically updated policies from different locations. Such locations may include designated network locations that are designated to provide dynamic policy related information or servers (either in the subscriber server network 104 or in the CDN 105) that are classified as managed servers. For instance, a server may dynamically specify its load share via a designated file stored on the server or at some other location (the file may reside on the server being monitored or at some other location, e.g., a central load controlling server). To retrieve such dynamically defined load share information from a managed server, the ATC policy management mechanism 152 may poll the designated file stored on the managed server to obtain relevant load share information. Dynamic policies may also be polled from other policy making entities.
[0150] Broadcasting ATC policies may take place periodically according to some pre-defined interval or may be triggered whenever the stored ATC policies are updated. The ATC policy management mechanism 152 may monitor changes made to the existing ATC policies. The ATC policy management mechanism 152 may poll the ATC policies stored in the ATC policy database and see whether there are changes. On the other hand, whenever the ATC policy management mechanism 152 polls dynamic policies from specified locations (such locations may be specified in existing ATC policies), it may determine whether the dynamically polled policies differ from existing ATC policies. In the event that updated policies are different from the existing ATC policies, the ATC policy management mechanism 152 may re-broadcast the updated ATC policies to the ATC name server network 140.
[0151] The ATC network monitoring mechanism 150 may collect DNS log summaries from different name servers in the ATC name server network 140. Such summary log data may be received in the form of events that provide information such as, for example, the number of requests directed to particular servers in a given time period. The ATC network monitoring mechanism 150 may collectively processes such DNS log summaries (or events) from the entire ATC system. The report generation mechanism 168 may generates monitoring status reports from these summaries and makes such reports available to the subscriber 102 via the secure web-based GUI 160.
[0152] The administrative master backup mechanism 170 may periodically update the AMA backups 146-1, . . . , 146-m to ensure that all the backup agents are current. This may include replicating the ATC policies, the operational status of various control mechanisms (including the ATC policy management mechanism 152), the policy editing mechanism 162, and the administrative policy update mechanism 164, and providing the up-to-date information to the AMA backups.
[0153] A major function of an administrative master agent is to manage the ATC policies, to make sure that updated ATC policies are supplied to the name server agents in the ATC name server network 140, to monitor the various name servers' performance, to generate dynamic monitoring status report of system performance, and to maintain a connection through which policies may be updated dynamically and monitoring report can be examined.
[0154] Information flagging system errors and other anomalous conditions is collected by the ATC network monitoring mechanism or agent 150. FIG. 7 depicts an internal functional block diagram of an ATC network monitoring mechanism 150, according to embodiments of the present invention. The ATC network monitoring mechanism 150 includes a trap handler 176, a processing mechanism 182, and an alert generation mechanism 184. The trap handler 176 traps events from the name servers in the name server network 140. The processing mechanism 182 analyzes both the collected trapped events, and, based on analyzed information, the alert generation mechanism 184 generates alerts when necessary, and reports such alerts to, for example, the ATC's network operation center (NOC) 148 and the subscriber 102 (FIG. 1 ).
[0155] The trap handler 176 further comprises an event receiver 178 that intercepts trap events from the name servers and an event consolidation mechanism 180 which may classify the trapped events and organize them in a reasonable and appropriate fashion. The processing mechanism 182 may process the consolidated events to identify useful or informative patterns which may be further used, by the alert generation mechanism 184 to identify problematic patterns which may significantly affect the system performance.
[0156] The Adaptive Traffic Control (ATC) framework according to the present invention may be deployed as a stand-alone service directing traffic solely to the subscriber's servers, in conjunction with another content delivery network (CDN) provider, or in conjunction with any other service.
[0157] Each domain name server in the ATC name server network 140 may include (FIG. 1 ) a location determiner 128, an adaptive traffic control (ATC) mechanism 126, a monitoring agent or mechanism 130, a database manager 132, and a report generator 134. When a name server (e.g., 118-1) receives a request from the resolver 116 of the client 112, the location determiner 128 determines the location of the resolver and sends such location information to the ATC mechanism 126. Based on the location information, the ATC mechanism 126 retrieves relevant ATC policies from the policy database 124 (e.g., the subscriber policies 120 or the other policies 122) and selects one or more servers in the subscriber server network 104 according to the retrieved relevant policies. The corresponding IP address(es) or the CNAME of the selected servers are then returned to the resolver 116.
[0158] The database manager 132 maintains the policy database 124. It received policies broadcast from the ATC policy management mechanism 152 in the ATC administrative framework 142 and populates the policies in the policy database 124. The ATC policies received from the ATC administrative framework 142 may also include information that defines or classifies servers in the subscriber server network 104 (or in the CDN 105). For example, some servers may be defined as monitored servers and some may correspond to managed servers.
[0159] When the database manager receives such information, it may inform the monitoring mechanism 130 of the classification of the underlying servers so that the monitoring mechanism 130 can monitor each server according to its status. When the ATC policy management mechanism 152 broadcasts updated policies, the database manager 132 accordingly updates the relevant policies stored in the policy database 124.
[0160] The monitoring mechanism 130 monitors the operational status of the name server 118-1 and one or more servers in the subscriber server network. It may collect events occurred in the name server 118-1 during operations and send such events to the ATC network monitoring mechanism 150 in the ATC administrative framework 142. On the other hand, it may also monitor the operations of various servers in the subscriber server group 104 (or in the CDN 105) according to how each server is defined (monitored or managed server).
[0161] If a server is defined as a monitored server, the monitoring mechanism 130 may dynamically probe the server (as discussed earlier) to determine its availability. If a server is defined as a managed server, the monitoring mechanism 130 may monitor its availability during operation. The monitoring mechanism 130 may also poll dynamic load share information from the server. When the monitoring mechanism 130 detects that a server is no longer available, it may inform the database manager 132 to create a local policy that indicate that the server is no longer available so that the ATC mechanism 126 can take into account when resolving a hostname.
[0162] When the monitoring mechanism 130 polls the dynamic load share information from the server, it may inform the database manager 132 to update the load share policies in the policy database 124 that are affected by the dynamics of the polled load share. For example, if three primary servers in a server group originally have load share (0.3, 0.3, 0.4) and the third primary server now changes its load share to 0.2, the database manager 132 may accordingly update the load share among these three primary servers into (0.4, 0.4, 0.2).
[0163] The report generator 134 generates reports related to the operations of the underlying name server based on log information 136 recorded. Such generated reports may be sent to a report consolidator 154 in the ATC administrative framework 142 so that reports from different name servers may be consolidated.
[0164] A subscriber may activate (turn up) the ATC system (DNS servers) in one of two ways: using a DNS CNAME or using NS delegation. Instead of using ATC to direct traffic for a single DNS hostname, the subscriber may have many different subdomains that it would like to direct to ATC. For example, the subscriber might want all downloads from dl.customer.com handled by ATC, together with all of its subdomains, but have all other domains that it controls, such as www.customer.com, resolved by its own name server:
-
- [0165] dl.customer.com.=>ATC
- [0166] any.thing.dl.customer.com.=>ATC
- [0167] www.customer.com.=>not ATC
- [0168] customer.com.=>not ATC
[0169] In this case, instead of adding the CNAME record in the subscriber's DNS zone file, it simply delegates the dl.customer.com. name to ATC via NS records in its zone file.
[0170] The delegation (using NS-records) method is presently preferred as it is more flexible and offers all of the reliability, scalability and flexibility of ATC. After the initial contact for delegation, a properly operating resolver making occasional queries will not contact the subscriber's own name server or name servers again. The result provides much better name resolution performance since the unnecessary overhead of routing each fresh DNS request through the subscriber's name server is eliminated. The CNAME method keeps the subscriber's own name server in the loop. That is, whenever the CNAME TTL expires, client resolvers will return to the subscriber's name servers to refresh the record.
[0171] For both CNAME and NS delegation methods, the procedure to turn off the ATC switch is the same—the subscriber edits its DNS zone files to remove the delegation authority to ATC. DNS requests will continue to be served in accordance with the subscriber's defined ATC policies until the TTLs have expired on the appropriate delegations.
The System in Operation
[0172] FIG. 8( a) is an exemplary flowchart of a process, in which a name server resolves a DNS request based on ATC policies, according to an embodiment of the present invention. First, a user enters a URL into the user's browser 114 (or into any application that accepts URLs as input and obtains the corresponding resource for the client) (at 802). The client's resolver 116 attempts to resolve the hostname of the URL in order to obtain an IP address of a server from which the resource identified by the URL can be obtained (at 804). The resolver 116 will be directed by the client's DNS (not shown) to a DNS Name Server 118 in the ATC name server network 134 (at 806). The resolver 116 provides the name server 118 with the hostname it is attempting to resolve.
[0173] The name server 118 receives the request to resolve the hostname (at 808) and determines one or more servers in the subscriber server network 104 or in the CDN 105 that can process the client's request according to the location of the resolver 116 as well as relevant ATC policies retrieved from the policy database 124 (at 810). Details of this operation are described with reference to FIG. 8( b). The IP address(es) of the selected server(s) are returned to the requesting resolver 116 (at 812). The browser 114 then connects to one of the servers (at 814) in order to obtain the requested resource.
[0174] FIG. 8( b) is a flowchart of an exemplary process, in which a domain name server selects one or more servers according to location of the client and relevant ATC policies. The location of the resolver 116 (or client) is first determined (at 816). Relevant ATC policies are then retrieved (at 818) from the policy database 124. One or more servers in either the subscriber server network 104 or the CDN 105 are selected according to the determined location of the client and the relevant ATC policies (at 820).
[0175] FIG. 8( c) is a flowchart of an exemplary process, in which the monitoring mechanism 130 in a domain name server monitors the operations of the name server as well as one or more servers in the subscriber server network 104 or the CDN 105. Events occurring during domain name service are monitored (at 822). Such events are sent to the ATC network monitoring mechanism 152 (at 824). In addition, the availability of the one or more servers are also monitored (at 826). Furthermore, if any of the one or more servers is defined as a managed server (determined at 828), dynamic load share information is polled (at 830). Both the availability information and the dynamic load share information (if any) are used to update some ATC policies that are local to the name server (at 832).
[0176] FIG. 8( d) is a flowchart of an exemplary process, in which the ATC policy management mechanism 152 dynamically maintains and broadcasts the ATC policies. Initially, policies from different sources are received (at 834) and broadcast to the name servers 118 (at 836). If backup is necessary (determined at 838), the ATC policy management mechanism 152 sends current policies to the master backup agents 146 (at 840).
[0177] The ATC policy management mechanism 152 also performs dynamic policy maintenance. It polls dynamic policy information (at 842) at certain defined intervals and uses such polled dynamic policy information to update existing policy (at 844). The updated policies are then broadcast to the name servers (at 846). If the updated policies need to be propagated to the backup agents (determined at 848), they are sent to the master backup agents (at 850).
[0178] FIG. 8( e) is a flowchart of an exemplary process, in which the ATC network monitoring mechanism 150 monitors operations of the name servers and sends alert to the NOC 148 and the subscriber 102. Events sent from monitoring mechanisms of different name servers are trapped (at 852). Such trapped events from different sources are then consolidated (at 854) and processed (at 856). If there is any alarming situation (determined at 858), the ATC network monitoring mechanism 150 generates an alert (at 860). The generated alert is then sent to both the NOC 148 and the subscriber 102 (at 862).
Policy Administration
[0179] The ATC policies may be initially set up and later adaptively updated according to servers' dynamic operational status. The ATC policies may be formed via various means. The described approaches may also be applied to form other policies. A GUI approach or a file-based approach (or both) may be employed to set up subscriber policies. Through the GUI approach, different means to form subscriber policies may be adopted. For example, a browser may be used so that a subscriber can enter policies directly. An XML file containing descriptions of the subscriber policies may also be loaded using GUI approach so that the policies may be parsed and stored. As another alternative, a file containing descriptions of the subscriber policies may also be loaded in a similar fashion and parsed accordingly. When a file based method is used, a file containing descriptions of the subscriber policies, either constructed based on XML or some other structure, may be accessed via other means such as FTP.
[0180] In some preferred embodiments, a subscriber accesses a secure web-based GUI 160 (FIG. 6 ) using subscriber's secure browser interface 156.
[0181] FIGS. 9( a)-9(c) show exemplary secure web based graphical interfaces, through which a subscriber may define load share policies and overflow policies with respect to specified network resources, according to embodiments of the present invention. There may be different types of load sharing servers: (1) static servers, (2) dynamic servers, and (3) overflow servers. A dynamic server is one that is specified as either a monitored or a managed server whose dynamic availability may be monitored and whose load may be re-directed when it becomes unavailable. In the case of a monitored server, both of its load share and load shed fraction may also be made dynamic.
[0182] Correspondingly, the policies governing routing requests to different types of servers may also be defined accordingly. First, the policies that govern static servers are applied when servers are relatively static and do not change often over time. This may mean that the availability of the servers is fairly stable and load sharing among different servers is also relatively stable. The policies that control dynamic servers are applied when servers are expected to change frequently. Such policies include failover policies, shedding policies, and tiered failover policies (described earlier). The overflow policies control the change of flow of the requests when, for some reason, primary servers, either static or dynamic, become unavailable or overloaded. In this case, relevant overflow policies determine to which overflow server a request for name service should be directed.
[0183] The load share with respect to a given server represents the amount of traffic that will be sent to that server within a static server setting. An exemplary method to compute the load share of the traffic sent to the server is to sum the load share amount of all servers in a current static server setting and divide the load share for that server by the summed total to derive the load share.
[0184] The shed fraction for a given server represents the percentage of traffic that should be redirected away from the server. The redirection may be carried out after load share is made based upon load share policies. FIG. 9( a) gives an example interface, in which each of the servers listed is assigned various selection related parameters. For example, for each server, a TTL is specified. In addition, a load share and a shed fraction may also be specified. For example, both servers New York and London have load share of 1.0. The shed fraction assigned to server New York is 0.2, meaning that 20% of its load is re-directed to some other server, and the shed fraction assigned to server London is 0.3, meaning that 30% of its load is re-directed to some other server.
[0185] Shedding fractions may also be computed automatically on the fly. For example, when a server is detected to have slow response, indicating that it may be overloaded, a shedding fraction may be computed according to the discrepancy between its expected response time and the actual response time. Such an automatically computed shedding fraction can then be applied to re-direct the newly computed fraction of the traffic to a different server to unload some of the traffic originally intended to be directed to the overloaded server.
[0186] For each resource server, an on-line flag may also be set to indicate whether the server is currently available. This flag is shown in FIG. 9( a) in the rightmost column. With this flag, a server may be temporarily removed from the service, if, for instance, a server needs to be taken down for maintenance.
[0187] Since managed and dynamic servers are expected to change relatively frequently, policies that govern their selection may be defined in two stages. The policies for managed and dynamic servers may be initially specified in a similar fashion as for static servers. For instance, load share and shed fraction can be defined for dynamic servers New York and London, as shown in FIG. 9( a). During operation, however, the policies that govern the selection of managed and dynamic servers may be established dynamically. For example, initially defined policies (e.g., load share, shed fraction, and on-line flag) may be adaptively revised based on, for instance, an on-line status report retrieved from one or more specified locations on the network. Dynamic policies provide considerable flexibility in how the selection may be conducted in a manner that is adaptive to the network health or any other network performance related factors (e.g., maintenance).
[0188] The on-line status report may be provided at one or more network locations specified by the subscriber. The locations may be specified as a Uniform Resource Locator (URL) and may be accessed through an HTTP request. A location of the status report may be specified in the initial policy, providing a constant link to the location. FIG. 9( b) shows an exemplary GUI for specifying an URL link to a status report. An URL link for the location of the status report is specified as “http://server” which corresponds to a server named London with an IP address of 10.0.0.3. The access may be authenticated and such authentication requirements may also be specified in the initial policies. The on-line accessed status report may be constructed based on information gathered during monitoring the servers' performance. For instance, each server in the subscriber server network may provide a status report containing information related to its performance. Name servers in the ATC name server network may poll such information from such servers and such information may be used to update policies.
[0189] The frequency with which the status report is accessed may also be specified explicitly. For example, it may require the relevant mechanisms (specifically, the ATC policy management mechanism or the monitoring mechanism of each name server) to poll the status report at a regular time, e.g., every thirty seconds.
[0190] A status report may be constructed using some standard language such as extendible Markup Language (XML). Such a status report may contain revised policies, which may be determined by the subscriber manually based on network performance, devised by an automated process based on network performance, or may be generated by an individual managed server. In such cases, the status report includes updated policies and when it is polled, the updated policies are used in future traffic control. For example, a managed server may re-define its load share of according to its dynamic capacity. When such dynamically defined load share information is accessed and used in enforcement, the traffic is controlled in a manner that is adaptive to the network dynamics.
[0191] When a status report is accessed, the relevant mechanisms (e.g., the ATC policy management mechanism in the ATC administrative framework or the ATC mechanism in corresponding name server) updates the associated policies based on the information contained in the status report. For example, a status report may indicate that the current load share for server New York should be 0.4 (down from 1.0 previously) with the same shed fraction (0.2) and the current load share for server London should be 0.5 (down from previous 1.0) with 0.4 shed fraction (40% shed fraction). An alternative scenario may be that the status report provides network performance data about each dynamic server and the ATC mechanism, after accessing the status report, determines how the current policies may be revised accordingly.
[0192] When the status report provides dynamic policies, it may, in general include, for each dynamic server, the following information:
[0193] <ServerStatus loadShare=“LS” shedFraction=“SF” online=“Boolean”/>
where LS and SF represent numerical numbers and “Boolean” represents a logical value of either “true” or “false”.
[0194] With respect to dynamic servers, with the mechanism of dynamic policies described above, a subscriber or a server can easily specify changes to existing policies (e.g., changes to load share, shed fraction policies and the On-line flag) without having to update an entire ATC policy hierarchy. In addition, a parameter (called “Down on line” in some embodiments) can be set in an ATC policy to control the behavior of the ATC mechanism under the circumstance that the ATC policy management mechanism is unable to properly access the status report due to reasons such as a failed polling or retrieval or failing to parse the status report's content. According to some embodiments of the present invention, three different exemplary approaches may be applied to handle the situation.
[0195] With the first approach, a subscriber may instruct an ATC mechanism in a name server not to select a particular server if the status report for that server can not be properly obtained. The subscriber may specify this policy by setting parameter “Down on line” to false. In this case, the ATC mechanism will not consider the underlying server as a candidate for selection process until the polling mechanism retrieves a valid status report.
[0196] The second approach to deal with a polling failure is to allow the ATC mechanism to select the underlying dynamic server according to its default policies (or original policies). To specify this solution, the subscriber may set the “Down on line” parameter true.
[0197] The third approach to handle the situation where the status report can not be properly retrieved is to allow the ATC mechanism to select the underlying server if its “on line” flag is on (it is available). To achieve this, a subscriber may specify only the “on line” ServerStatus in the status report, with parameter “Down on line” set to either true or false. In this case, the ATC mechanism will use a load share and a shed fraction specified in the policy when the “On line” flag is set to true. The load share and the load shed fraction may both be dynamically determined (either broadcasted from the ATC policy management mechanism or polled by the ATC mechanism 130 from the underlying server.
[0198] Static, dynamic and managed servers are primary servers, although their selection may be controlled by operationally different policies. The third type of servers are called overflow servers. They provide alternatives when primary servers, for some reason, can not be selected. Typically, an overflow server corresponds to a Canonical name or CName, pointing to a service providing a CDN (such as, e.g., the CDN service offered by Level 3 Communications). The choice of overflow servers may be determined based on the belief that they are in general always available. An overflow server may be defined through a window illustrated in FIG. 9( c). In FIG. 9( c), overflow servers are defined by a CName, which points to the CDN server address “customer.footprint.net” (e.g., the CDN 105, shown in FIG. 1) .
EXAMPLES
Example A
[0199] FIGS. 10( a)-10(b) show example subscriber policies, defined based on a set of resource servers, that govern the selection of the servers according to different criteria such as geographical location of an incoming request and the time zone of each of the locations. In FIG. 10( a), three different resource servers are defined, including an “eastserver” with IP address 10.0.0.2, a “westserver” with IP address 10.0.0.1, and a service network with CName “customer.footprint.net”. Here, the service network may include more than one server.
[0200] FIG. 10( b) shows an exemplary decision tree embedded in a set of geographical policies that guide how the traffic should be directed to a set of servers. In the decision tree depicted in FIG. 10( b), the selection is first directed to different resources at the top level according to a split between the United States and rest of the world. A resource may correspond to either a single server (or CDN) or a group of servers (or CDNs). A request initiated from non-U.S. geographical regions is directed to a resource named “London”. Within the United States, time zone based policies are further applied. A request initiated from a region within the central time zone is directed to a resource named “New Work”. A request initiated from a region within Alaska time zone is directed to the resource “London”. Finally, a request initiated from a region within the United States that does not fall in any of these time zones (i.e., Alaska or Central) can be directed to either “London” or “New York”.
Example B
[0201] FIGS. 10( c)-10(d) show another example decision tree constructed based on a set of resources (servers or CDNs) and subscriber policies that are defined based on geographical locations of an incoming request with overflow policies that allow the ATC mechanism to direct traffic to pre-defined overflow servers when primary servers are not available. In the Resource window shown in FIG. 10( c), three resources are defined: static resources “London” and “New York”, each defined based on its IP address, a service network named “Sydney” with CName “customer.sandpiper.net”. Each resource may correspond to a set of servers such as a server group with certain hierarchy. Resource “London” is further defined as a dynamic resource and resource “Sydney” is further defined as an overflow server network nicknamed “cdn_service”. FIG. 10( d) illustrates an example decision tree built based on these resources.
[0202] The decision tree in FIG. 10( d) first splits two ways at the top level based on whether a request is from the United States. When a request is initiated from the United States, it is directed, according to the policies illustrated in FIG. 10( d), to the dynamic servers at the first tier of resource “London”. A request from the United States may also directed to one of the overflow servers in resource “cdn_service”. This may happen when either all the primary servers at the first tier of resource “London” fail to function or when such primary servers are overloaded.
[0203] The next four examples (Examples 3-6) are based on an example subscriber which has ATC handling traffic for www.subscriber.com with a CNAME hand-off to www.subscriber.com.nsatc.net. The example subscriber has three datacenters, one in San Francisco, New York and London. The set of resources used by these four examples is shown in FIG. 10( e). Managed servers are used with monitoring since the example provides failover from one server to another. The default load share values vary: the San Francisco server can handle 2.5 times the traffic of the London or New York servers. The overflow server is utilized in only two of the example policies. It is a CNAME specification to a CDN—the Level 3 CDN.
Example C
[0204] With this policy (FIG. 10( f)), the example subscriber does not utilize any geographic ATC rules, and the London server is not in use. End users are simply directed to one of the two servers, provided the ATC monitoring of that server indicates the server is up and functioning Unlike standard DNS round-robin techniques, ATC directs end users to the two servers in exactly the ratios specified by the load share values in the Resources Window (FIG. 10( e)). ATC will return the IP address of the San Francisco server 2.5 times as often as the New York server. If ATC monitoring detects that the New York server is offline, ATC name servers will immediately stop returning the IP address of the New York server and only return the San Francisco server IP address until the monitoring indicates the New York server has returned. If the example subscriber determines that any one of the servers has too much traffic, the load share of that server should be adjusted in one of two ways:
-
- [0205] the subscriber can login to the ATC Admin GUI and change the load share value manually, or
- [0206] the subscriber can configure ATC to poll a load feedback URL for the load share value, and change that value on the fly
[0207] Changing the shed fraction for any one server is not appropriate in this example because there is no defined lower tier or overflow server to receive shed traffic. Both servers are in the same tier.
[0208] A variation of this example is to put the New York server in Tier 2, i.e., have New York function as a backup. In this configuration, all traffic would be directed to the San Francisco server unless it was monitored as down or shedding in which case traffic would then go to New York. That is, if the San Francisco server had a shed fraction of, e.g., 10%, then ATC would return the New York server 10% of the time. When there is only one server in a Tier, there is no load share balancing for ATC to do, and the load share value becomes unimportant (unless zero).
[0209] This example demonstrates load share variation and server failover handling within a load sharing server set consisting only of managed servers.
Example D
[0210] This policy (FIG. 10( g)) adds an overflow server to the server set of the previous policy example. The example subscriber has two methods to inform ATC that any one server should shed some of its traffic to the overflow server, in this case the Level 3 CDN. The overflow server is intended to be an effective large supply of bandwidth, to be used when the subscriber's servers are at capacity or when the lowest latency is required.
[0211] The shed fractions can be modified in one of two ways:
-
- [0212] the subscriber can login to the ATC Admin GUI and change the shed fraction value manually; or
- [0213] the subscriber can configure ATC to poll a load feedback URL for the shed fraction value, and change that value on the fly
[0214] The subscriber may maintain an even load across all subscriber servers by setting the load share. This value typically does not need continuous adjustment. Then, if traffic levels increase and all servers are nearing their capacity, all servers begin shedding traffic to the Overflow server(s) (in this example, “dl.subscriber.com.c.footprint.net”, a CName for a CDN).
Example E
[0215] This example policy (FIG. 10( h)) utilizes geographic control to direct end users to the subscriber servers nearby, on a continental scale, improving end-user response time (latency). A variation in tiers is used to specify preferential and failover server selection. Within a load sharing server set, all servers in Tier 1 are considered first. Only if a server cannot be selected from Tier 1 (due to shedding or because it is off-line) is Tier 2 considered, and so on for Tier 3. Requests with end-user resolvers in Europe (and Africa and the Middle East) are directed to the London server, unless it is down or shedding, in which case traffic is sent to New York, and then San Francisco. Similarly, requests from Asia and Australia are directed to San Francisco, unless that server is down or shedding, in which case the lower tiers are used.
[0216] In this case, the subscriber does not have an overflow server defined. If the load on any server becomes too high, the shed fraction can be increased on that server and traffic will be sent to the lower tiers defined within each geographic region.
[0217] If both the San Francisco and New York servers are offline, all traffic is directed to the London server until the U.S. servers return. If all three servers are offline, ATC is unable to determine which servers are preferred and thus selects all three servers, with the hope that the end user may be able to connect to one of them.
Example F
[0218] This policy (FIG. 10( i)) demonstrates finer geographic control. End users with resolvers on the West coast are directed to San Francisco rather than New York. Likewise, end users with resolvers in the Eastern U.S. are directed to New York rather than San Francisco. The overflow server, in this case “dl.subscriber.com.c.footprint.net”, a CName for a CDN, is again in use. If all servers are offline or shedding (which is considered unlikely), traffic will be directed to the CDN.
[0219] Note: The Other label is used, as before, to catch all non-specified world zones, and again within the U.S. time zone split to catch all remaining regions (in this case the western time zones of the U.S. and the rest of the Americas).
[0220] The following three examples (Examples 7-9) are based on an example subscriber which has ATC handling traffic for an origin server demo.nsatc.net and for the domain nsatc.net. The subscriber has three data centers in New York, San Francisco, and London, UK, with managed servers at each center. As shown in FIG. 10( j), the load shares for the managed servers are (NY: 1, SF: 2, and London: 1).
Example G
Origin Plus CDN
[0221] FIG. 10( k) shows a policy for the two sub-domains “images.nsatc.net” and “www.nsatc.net”. For the “images” sub-domain, end users are directed to servers in New York or San Francisco, in the ratio 1:2. That is, the ATC mechanism will return the IP address of a San Francisco server twice as often as that of a NY server. If the ATC mechanism detects that, e.g., the San Francisco server is down, it will stop return the IP address for the SF server and will only return the IP address for the NY server (until monitoring indicates that the SF server has returned). The subscriber may adjust the load share of the servers as in the earlier examples (either by logging in to the ATC to change the load share values manually, or by configuring ATC to poll a particular location for a configuration file). In case of overflow, the requests go to the CDN (with the CName “demo.c.footprint.net”).
[0222] The “www” sub-domain is handled somewhat differently in that the overflow for “www” will go to either of two CDNs. In this example, as shown in FIG. 10( l), “Level 3 CDN” is a CNAME for a CDN at “demo.c.footprint.net”, and “Other CDN” is a CNAME for another CDN at “demo.othercdn.net”. The ATC is configured to share the load between the two CDNs in the ratio 1.5 to 1. In other words, as configured in FIG. 10( l), ATC will return the CNAME of the Level 3 CDN 1.5 times as often as it returns the CNAME of the “Other CDN”.
[0223] As with the load share values for the other servers, the load share for the CDNs can also be adjusted by the subscriber on the fly or by logging in to the ATC.
[0224] Those skilled in the art will realize and understand, upon reading this description, that a subscriber may wish to adjust load share values for a CDN for performance and/or business reasons. E.g., a subscriber may have to pay a higher rate for one CDN than for another.
[0225] Those skilled in the art will realize and understand, upon reading this description, that the two CDNs may be owned and operated by different entities and may be in different domains.
[0226] Using this example, a subscriber can have its traffic handled initially by its origin servers (in NY or SF), with overflow being handled by a CDN. Overflow for the “images” sub-domain is handled by a single CDN, whereas overflow for the “www” sub-domain is handled by two CDNs.
Example H
[0227] FIGS. 10( m)-10(n) show a policy in which traffic to the sub-domain “www” is split equally between three CDNs. Each CDN has a load share of 1 and so an end-user will be returned a CNAME of one of the three CDNs, with each CDN having one third of the load. Again, the subscriber can modify the load share on the fly or by logging in to the ATC.
Example I
[0228] FIG. 10( o) shows an exemplary policy with different rules for the sub-domains “ftp”, “download”, “www” and “images.”
[0229] End-user requests for the “download” sub-domain (download.nsatc.net) are handled by the subscriber's origin servers in NY and SF (in the ratio 1:2 (per FIG. 10( l)). Overflow from the “download” sub-domain is handled by the CDN “demo.cdn.com” (a CNAME for the CDN). In other words, overflow from the “download” sub-domain will be directed via a CNAME demo.cdn.com to the CDN (i.e., ATC will return the CNAME demo.cdn.com for overflow cases to the sub-domain “download”).
[0230] The “ftp” sub-domain is split based on country (i.e., on the country in which the end-user's request is believed to originate). End-user requests from the US are handled by the domain “us.demo.com” (a CNAME), and all other requests are handled by “international.demo.com” (also a CNAME). In this manner, ftp requests can be directed to different networks based on their country of origin.
[0231] The “images” sub-domain is split initially on three geographic zones:
[0232] I. Europe
[0233] II. Asia and Australia
[0234] III U.S. and other
[0235] Within each zone, end-user requests are further directed based on server load. E.g., in zone I (for Europe), end-user requests are handled by the tier 1 server(s) in the UK. If these fail (or are overloaded and need to shed load), requests then go to the tier 2 server(s) in NY. And if these fail (or are overloaded and must shed), requests are directed to the tier 3 server(s) in SF. Thus, in response to an end-user request for the sub-domain “images”, the request originating in Europe, the ATC will return the IP address of a server in the U.K. But if that server in the UK is overloaded, the ATC will return the IP address of a server in N.Y. And if that server is also overloaded, the ATC will return the IP address of a server in S.F.
[0236] The Asia/Australia zone operates similarly, except that the servers are put into different tiers, favoring the order SF, NY, UK. All other zones favor the order NY, SF, UK.
[0237] The “www” sub-domain is not split based on geographic location of the end-user. Traffic is handled by SF first, with NY as an overflow.
[0238] In addition to the various web-based interfaces described herein, there are also other means through which subscriber policies may be defined. As discussed earlier, for example, subscriber policies may also be constructed or specified in an XML file which can be downloaded by an ATC mechanism and used to control the traffic.
[0239] Policies defined via different means (e.g., through web based GUI or XML file) may be converted into some pre-defined format within the ATC mechanism. Such pre-defined format may be designed for efficiency in manage and handling the ATC policies. For example, the internal format for ATC policies may be designed so that, internally, the AMAs can conveniently store, access, and broadcast the ATC policies to the name server agents and the name server agents can efficiently apply the policies.
[0240] As mentioned earlier, in addition to management of the ATC policies, the AMA may also monitor the performance of name servers and generates viewable DNS log reports. The monitoring mechanism may gather performance information from either the DNS logs of the name servers or the events trapped from the name servers. Such gathered information may be used by the report generation mechanism to construct informative reports. The report generation mechanism may also make such reports available to the subscribers via the secure web-based GUI. FIGS. 11( a)-11(c) show exemplary types of report information that is accessible via the secure web-based GUI. FIG. 11( a) depicts a web interface that allows a subscriber to view DNS log data of a specified resource server. A subscriber can specify the name of the server under review (e.g., “download.subscriber.com”), the form in which the log data is to be organized (e.g., specify output style of “Separate”, meaning to display IP and CName queries and replies with answers as separate items in the output), the time period under review (e.g., from Sep. 1, 2002 to Sep. 24, 2002), and the time resolution used in displaying the log data (e.g., automatically select time resolution).
[0241] FIG. 11( b) shows a plot of log data related to a specified server. The log data is presented in a plot, generated based on the log entries of a particular server against a specified period of time (X-axis) with certain resolution. The exemplary plot reflects the volume of the traffic directed to the underlying server during a period time between 2:30 pm and 3:30 pm on Sep. 24, 2002. The traffic volume is described in terms of number of replies generated by the underlying server per minute (Y-axis). It can be seen in this example plot that there is a surge in traffic volume between 2:36 pm and 2:38 pm. FIG. 11( c) shows a graphical display for the same log data in FIG. 11( b) in a table form, in which the log data is listed in a chronicle order from top to the bottom of the table. For instance, each row in the left column lists a period of time. In the exemplary table, the log data within the one hour period (2:30 pm to 3:30 pm) is divided into a plurality of sub-periods, each of which is 15 minutes. For each sub-period, the third column provides the number of replies from the corresponding sub-period. For instance, during the sub-period of 3:15 pm to 3:30 pm, there are 13 replies and there is no reply in other sub-periods (consistent with the plot illustrated in FIG. 11( b)).
[0242] In some implementations, end-user resolvers may use a preferred subset of the ATC nameservers. This may be implemented by providing an initial pool of ATC name servers which respond to new requests from new end-user resolvers. When an end-user resolver first requires name resolution from an ATC nameserver, that resolver is directed (e.g., by the DNS system) to one of the pool of ATC nameservers. When that ATC nameserver provides an answer to the end-user resolver, it also gives that resolver a list of preferred ATC nameservers for that resolver, i.e., one or more ATC nameserver that are better suited to handle name resolution for that end-user resolver. An ATC nameserver may be better suited to handle requests from an end-user resolver for a number of reasons, e.g., because it is electronically closer to the resolver, because it is geographically appropriate for that resolver, etc. Those skilled in the art will realize, upon reading this description, that different and/or other measures of suitability may be used to provide an end-user resolver with a list of preferred ATC nameservers. Once an end-user resolver has a list of preferred ATC nameservers, that resolver will make future requests of nameservers in its list (until the entries for those nameservers expire or are replaced).
[0243] The various mechanisms described herein, including, without limitation, the adaptive traffic control (ATC) mechanism, the location determination mechanism, policy editing mechanism, administrative policy update mechanism, ATC policy management mechanism, report generation mechanism, a monitoring mechanism, and an administrative master backup mechanism may be implemented in hardware, software or a combination thereof. When implemented in software, they may be implemented in any type of appropriate interpreted or compiled programming language. When implemented fully or partially in software, aspects of the invention can reside on any memory or storage medium, including but not limited to a ROM, a disk, an ASIC, a PROM and the like. While the invention has been described with reference to particular mechanisms (algorithms, processes and functions) and architectures, one skilled in the art would realize that other mechanisms and/or architectures could be used while still achieving the invention.
[0244] When the various mechanisms of the present invention are running on a particular machine (e.g., at a client or on a server), they may reside in the memory of the machine or on a storage device or in a combination. Further, while many of the operations have been shown as being performed in a particular order, one skilled in the art would realize that other orders, including some parallelization of operations, are possible and are considered to be within the scope of the invention.
[0245] The present invention has been described above in connection with a preferred embodiment thereof; however, this has been done for purposes of illustration only, and the invention is not so limited. Indeed, variations of the invention will be readily apparent to those skilled in the art. Such variations also fall within the scope of the invention. Thus, while the invention has been described with reference to the certain illustrated embodiments, the words that have been used herein are words of description, rather than words of limitation. Changes may be made, within the purview of the appended claims, without departing from the scope and spirit of the invention in its aspects. Although the invention has been described herein with reference to particular structures, acts, and materials, the invention is not to be limited to the particulars disclosed, but rather extends to all equivalent structures, acts, and, materials, such as are within the scope of the appended claims.
[0246] The processing described may be performed by a properly programmed general-purpose computer alone or in connection with a special purpose computer. Such processing may be performed by a single platform or by a distributed processing platform. In addition, such processing and functionality can be implemented in the form of special purpose hardware or in the form of software being run by a general-purpose computer. Any data handled in such processing or created as a result of such processing can be stored in any memory as is conventional in the art. By way of example, such data may be stored in a temporary memory, such as in the RAM of a given computer system or subsystem. In addition, or in the alternative, such data may be stored in longer-term storage devices, for example, magnetic disks, rewritable optical disks, and so on. For purposes of the disclosure herein, a computer-readable media may comprise any form of data storage mechanism, including such existing memory technologies as well as hardware or circuit representations of such structures and of such data.
Claims
What is claimed is:
1. A method, operable in a framework in which an adaptive traffic control name server network implements policy-based traffic direction, the name server network comprising at least one domain name server comprising hardware in combination with software and constructed and adapted to provide adaptive policy-based domain name service, the method comprising:
(A) providing a graphical user interface (GUI);
(B) using said GUI to obtain at least one policy for direction of network traffic, wherein the GUI supports the setting of said at least one policy using a decision tree representing rules; and
(C) providing said at least one policy to the name server network, wherein the decision tree comprises one or more resource nodes, and one or more branch nodes, wherein the one or more resource nodes specify one or more answers to be provided in response to a Domain Name Service (DNS) request, and wherein the one or more branch nodes specify one or more decision criteria to be applied, and wherein the GUI supports the specification of:
(i) one or more answers for the one or more resource nodes, and
(ii) one or more decision criteria for the one or more branch nodes, wherein the one or more criteria are selected from criteria related to: world zones, countries, states, time zones, and blocks of Internet Protocol (IP) addresses, and wherein the one or more resource nodes are selected from: IP addresses, canonical name (CNAME) records, mail exchange (MX) records, name server (NS) records, and load sharing server sets; and wherein the answers are selected from: IP addresses and CNAMEs.
2. The method of claim 1 wherein the at least one policy relates to direction of network traffic based on sub-domains of a domain of a subscriber.
3. The method of claim 2 wherein the at least one policy that relates to direction of network traffic based on sub-domains of a domain of the subscriber relates to one or more of the sub-domains: “ftp”, “download”, “www”, and “images”.
4. The method of claim 1 further comprising:
(D) generating reports based on information gathered from the name server network; and
(E) providing said reports.
5. The method of claim 4 wherein the information gathered from the name server network comprises: DNS log summaries from the at least one domain name server in the name server network.
6. The method of claim 5 wherein the information includes a number of requests directed to particular servers in a given time period.
7. The method of claim 4 wherein the reports are made available via a secure web-based GUI.
8. The method of claim 1 wherein the act (C) comprises:
providing said at least one policy to the name server network as an XML file.
9. An administrative agent, operable in a framework in which an adaptive traffic control name server network implements policy-based traffic direction, the name server network comprising at least one domain name server comprising hardware in combination with software and constructed and adapted to provide adaptive policy-based domain name service, said administrative agent comprising hardware in combination with software and constructed and adapted to:
(a) provide a graphical user interface (GUI);
(b) use said GUI to obtain at least one policy for direction of network traffic, wherein the GUI supports the setting of said at least one policy using a decision tree representing rules; and
(c) provide said at least one policy to the name server network, wherein the decision tree comprises one or more resource nodes, and one or more branch nodes, wherein the one or more resource nodes specify one or more answers to be provided in response to a Domain Name Service (DNS) request, and wherein the one or more branch nodes specify one or more decision criteria to be applied, and wherein the GUI supports the specification of:
(i) one or more answers for the one or more resource nodes, and
(ii) one or more decision criteria for the one or more branch nodes,
wherein the one or more criteria are selected from criteria related to: world zones, countries, states, time zones, and blocks of Internet Protocol (IP) addresses, and
wherein the one or more resource nodes are selected from: IP addresses, canonical name (CNAME) records, mail exchange (MX) records, name server (NS) records, and load sharing server sets; and
wherein the answers are selected from: IP addresses and CNAMEs.
10. The administrative agent of claim 9 wherein the at least one policy relates to direction of network traffic based on sub-domains of a domain of a subscriber.
11. The administrative agent of claim 10 wherein the at least one policy that relates to direction of network traffic based on sub-domains of a domain of the subscriber relates to one or more of the sub-domains: “ftp”, “download”, “www”, and “images”.
12. The administrative agent of claim 9 wherein said administrative agent comprises hardware in combination with software and constructed and adapted to:
(d) generate reports based on information gathered from the name server network; and
(e) provide said reports.
13. The administrative agent of claim 12 wherein the information gathered from the name server network comprises: DNS log summaries from the at least one domain name server in the name server network.
14. The administrative agent of claim 12 wherein the information includes a number of requests directed to particular servers in a given time period.
15. The administrative agent of claim 12 wherein the reports are made available via a secure web-based GUI.
16. The administrative agent of claim 9 wherein said at least one policy is provided to the name server network as an XML file.
17. A system comprising:
(A) an adaptive traffic control name server network comprising at least one domain name server comprising hardware in combination with software and constructed and adapted to provide adaptive policy-based domain name service, and
(B) an administrative agent comprising hardware in combination with software and constructed and adapted to:
(a) provide a graphical user interface (GUI);
(b) use said GUI to obtain at least one policy for direction of network traffic, wherein the GUI supports the setting of said at least one policy using a decision tree representing rules; and
(c) provide said at least one policy to the name server network,
wherein the decision tree comprises one or more resource nodes, and one or more branch nodes,
wherein the one or more resource nodes specify one or more answers to be provided in response to a Domain Name Service (DNS) request, and
wherein the one or more branch nodes specify one or more decision criteria to be applied, and
wherein the GUI supports the specification of:
(i) one or more answers for the one or more resource nodes, and
(ii) one or more decision criteria for the one or more branch nodes,
wherein the one or more criteria are selected from criteria related to: world zones, countries, states, time zones, and blocks of Internet Protocol (IP) addresses, and
wherein the one or more resource nodes are selected from: IP addresses, canonical name (CNAME) records, mail exchange (MX) records, name server (NS) records, and load sharing server sets; and
wherein the answers are selected from: IP addresses and CNAMEs.
18. The system of claim 17 wherein said administrative agent comprises hardware in combination with software and constructed and adapted to:
(d) generate reports based on information gathered from the name server network; and
(e) provide said reports.
19. The system of claim 18 wherein the information gathered from the name server network comprises: DNS log summaries from the at least one domain name server in the name server network.
20. The system of claim 18 wherein the information includes a number of requests directed to particular servers in a given time period.
Patent Citations (429)
| Patent | Date | Inventor | Cited By |
|---|---|---|---|
| US4495570(A) | 1985-01-01 | Kitajima et al. | Applicant |
| US4591983(A) | 1986-05-01 | Bennett et al. | Applicant |
| US4594704(A) | 1986-06-01 | Ollivier | Applicant |
| US4726017(A) | 1988-02-01 | Krum et al. | Applicant |
| US4803641(A) | 1989-02-01 | Hardy et al. | Applicant |
| US4839798(A) | 1989-06-01 | Eguchi et al. | Applicant |
| US4847784(A) | 1989-07-01 | Clancey | Applicant |
| US4920432(A) | 1990-04-01 | Eggers et al. | Applicant |
| US4922417(A) | 1990-05-01 | Churm et al. | Applicant |
| US4943932(A) | 1990-07-01 | Lark et al. | Applicant |
| US4949187(A) | 1990-08-01 | Cohen | Applicant |
| US4949248(A) | 1990-08-01 | Caro | Applicant |
| US5029232(A) | 1991-07-01 | Nall | Applicant |
| US5130792(A) | 1992-07-01 | Tindell et al. | Applicant |
| US5132992(A) | 1992-07-01 | Yurt et al. | Applicant |
| US5136716(A) | 1992-08-01 | Harvey | Applicant |
| US5172413(A) | 1992-12-01 | Bradley et al. | Applicant |
| US5191573(A) | 1993-03-01 | Hair | Applicant |
| US5253275(A) | 1993-10-01 | Yurt et al. | Applicant |
| US5253341(A) | 1993-10-01 | Rozmanith et al. | Applicant |
| US5287499(A) | 1994-02-01 | Nemes | Applicant |
| US5287537(A) | 1994-02-01 | Newmark et al. | Applicant |
| US5291554(A) | 1994-03-01 | Morales | Applicant |
| US5341477(A) | 1994-08-01 | Pitkin et al. | Applicant |
| US5371532(A) | 1994-12-01 | Gelman et al. | Applicant |
| US5410343(A) | 1995-04-01 | Coddington et al. | Applicant |
| US5414455(A) | 1995-05-01 | Hooper et al. | Applicant |
| US5442389(A) | 1995-08-01 | Blahut et al. | Applicant |
| US5442390(A) | 1995-08-01 | Hooper et al. | Applicant |
| US5442749(A) | 1995-08-01 | Northcutt et al. | Applicant |
| US5471622(A) | 1995-11-01 | Eadline | Applicant |
| US5475615(A) | 1995-12-01 | Lin | Applicant |
| US5508732(A) | 1996-04-01 | Bottomley et al. | Applicant |
| US5515511(A) | 1996-05-01 | Nguyen et al. | Applicant |
| US5519435(A) | 1996-05-01 | Anderson | Applicant |
| US5522070(A) | 1996-05-01 | Sumitomo | Applicant |
| US5528281(A) | 1996-06-01 | Grady et al. | Applicant |
| US5539621(A) | 1996-07-01 | Kikinis | Applicant |
| US5542087(A) | 1996-07-01 | Neimat et al. | Applicant |
| US5544313(A) | 1996-08-01 | Shachnai et al. | Applicant |
| US5544327(A) | 1996-08-01 | Dan et al. | Applicant |
| US5550577(A) | 1996-08-01 | Verbiest et al. | Applicant |
| US5550863(A) | 1996-08-01 | Yurt et al. | Applicant |
| US5550982(A) | 1996-08-01 | Long et al. | Applicant |
| US5557317(A) | 1996-09-01 | Nishio et al. | Applicant |
| US5572643(A) | 1996-11-01 | Judson | Applicant |
| US5590288(A) | 1996-12-01 | Castor | Applicant |
| US5592611(A) | 1997-01-01 | Midgely | Applicant |
| US5594910(A) | 1997-01-01 | Filepp et al. | Applicant |
| US5603026(A) | 1997-02-01 | Demers et al. | Applicant |
| US5619648(A) | 1997-04-01 | Canale | Applicant |
| US5623656(A) | 1997-04-01 | Lyons | Applicant |
| US5625781(A) | 1997-04-01 | Cline | Applicant |
| US5627829(A) | 1997-05-01 | Gleeson et al. | Applicant |
| US5630067(A) | 1997-05-01 | Kindell | Applicant |
| US5633999(A) | 1997-05-01 | Clowes | Applicant |
| US5634006(A) | 1997-05-01 | Baugher et al. | Applicant |
| US5638443(A) | 1997-06-01 | Stefik et al. | Applicant |
| US5644714(A) | 1997-07-01 | Kikinis | Applicant |
| US5646676(A) | 1997-07-01 | Dewkett et al. | Applicant |
| US5649186(A) | 1997-07-01 | Ferguson | Applicant |
| US5659729(A) | 1997-08-01 | Nielsen | Applicant |
| US5666362(A) | 1997-09-01 | Chen | Applicant |
| US5671279(A) | 1997-09-01 | Elgamai | Applicant |
| US5675734(A) | 1997-10-01 | Hair | Applicant |
| US5682512(A) | 1997-10-01 | Tetrick | Applicant |
| US5699513(A) | 1997-12-01 | Feigen et al. | Applicant |
| US5708780(A) | 1998-01-01 | Levergood et al. | Applicant |
| US5712979(A) | 1998-01-01 | Graber et al. | Applicant |
| US5715453(A) | 1998-02-01 | Stewart | Applicant |
| US5721914(A) | 1998-02-01 | DeVries | Applicant |
| US5734831(A) | 1998-03-01 | Sanders | Applicant |
| US5740423(A) | 1998-04-01 | Logan et al. | Applicant |
| US5742762(A) | 1998-04-01 | Scholl | Applicant |
| US5751961(A) | 1998-05-01 | Smyk | Applicant |
| US5761507(A) | 1998-06-01 | Govett | Applicant |
| US5761663(A) | 1998-06-01 | Largarde et al. | Applicant |
| US5764906(A) | 1998-06-01 | Edelstein et al. | Applicant |
| US5774660(A) | 1998-06-01 | Brendel et al. | Applicant |
| US5774668(A) | 1998-06-01 | Choquier et al. | Applicant |
| US5777988(A) | 1998-07-01 | Cisneros | Applicant |
| US5777989(A) | 1998-07-01 | McGarvey | Applicant |
| US5784058(A) | 1998-07-01 | LaStrange et al. | Applicant |
| US5796952(A) | 1998-08-01 | Davis | Applicant |
| US5799141(A) | 1998-08-01 | Galipeau et al. | Applicant |
| US5802106(A) | 1998-09-01 | Packer | Applicant |
| US5802291(A) | 1998-09-01 | Balick et al. | Applicant |
| US5812769(A) | 1998-09-01 | Graber et al. | Applicant |
| US5815664(A) | 1998-09-01 | Asano | Applicant |
| US5826031(A) | 1998-10-01 | Nielsen | Applicant |
| US5828847(A) | 1998-10-01 | Gehr | Applicant |
| US5832506(A) | 1998-11-01 | Kuzma | Applicant |
| US5832514(A) | 1998-11-01 | Norin et al. | Applicant |
| US5835718(A) | 1998-11-01 | Blewett | Applicant |
| US5838906(A) | 1998-11-01 | Doyle et al. | Applicant |
| US5845303(A) | 1998-12-01 | Templeman | Applicant |
| US5856974(A) | 1999-01-01 | Gervais et al. | Applicant |
| US5862339(A) | 1999-01-01 | Bonnaure | Applicant |
| US5867706(A) | 1999-02-01 | Martin et al. | Applicant |
| US5867799(A) | 1999-02-01 | Lang et al. | Applicant |
| US5870546(A) | 1999-02-01 | Kirsch | Applicant |
| US5870559(A) | 1999-02-01 | Leshem et al. | Applicant |
| US5878212(A) | 1999-03-01 | Civanlar et al. | Applicant |
| US5884038(A) | 1999-03-01 | Kapoor | Applicant |
| US5890171(A) | 1999-03-01 | Blumer et al. | Applicant |
| US5893116(A) | 1999-04-01 | Simmonds et al. | Applicant |
| US5894554(A) | 1999-04-01 | Lowery et al. | Applicant |
| US5896533(A) | 1999-04-01 | Ramos et al. | Applicant |
| US5903723(A) | 1999-05-01 | Beck et al. | Applicant |
| US5907704(A) | 1999-05-01 | Gudmundson et al. | Applicant |
| US5913028(A) | 1999-06-01 | Wang et al. | Applicant |
| US5913033(A) | 1999-06-01 | Grout | Applicant |
| US5918010(A) | 1999-06-01 | Appleman et al. | Applicant |
| US5919247(A) | 1999-07-01 | Van Hoff et al. | Applicant |
| US5920701(A) | 1999-07-01 | Miller et al. | Applicant |
| US5931904(A) | 1999-08-01 | Banga | Applicant |
| US5933832(A) | 1999-08-01 | Suzuoka et al. | Applicant |
| US5935207(A) | 1999-08-01 | Logue et al. | Applicant |
| US5944780(A) | 1999-08-01 | Chase | Applicant |
| US5945989(A) | 1999-08-01 | Freishtat et al. | Applicant |
| US5956489(A) | 1999-09-01 | San Andres et al. | Applicant |
| US5956716(A) | 1999-09-01 | Kenner et al. | Applicant |
| US5958008(A) | 1999-09-01 | Pogrebisky et al. | Applicant |
| US5961596(A) | 1999-10-01 | Takubo et al. | Applicant |
| US5966440(A) | 1999-10-01 | Hair | Applicant |
| US5968121(A) | 1999-10-01 | Logan et al. | Applicant |
| US5973696(A) | 1999-10-01 | Agranat et al. | Applicant |
| US5978791(A) | 1999-11-01 | Farber et al. | Applicant |
| US5983214(A) | 1999-11-01 | Lang et al. | Applicant |
| US5983227(A) | 1999-11-01 | Nazem et al. | Applicant |
| US5987606(A) | 1999-11-01 | Cirasole et al. | Applicant |
| US5991809(A) | 1999-11-01 | Kriegsman | Applicant |
| US5996025(A) | 1999-11-01 | Day | Applicant |
| US6002720(A) | 1999-12-01 | Yurt et al. | Applicant |
| US6003030(A) | 1999-12-01 | Kenner et al. | Applicant |
| US6006264(A) | 1999-12-01 | Colby et al. | Applicant |
| US6012090(A) | 2000-01-01 | Chung et al. | Applicant |
| US6014686(A) | 2000-01-01 | Elnozahy et al. | Applicant |
| US6014698(A) | 2000-01-01 | Griffiths | Applicant |
| US6018516(A) | 2000-01-01 | Packer | Applicant |
| US6021426(A) | 2000-02-01 | Douglis | Applicant |
| US6026440(A) | 2000-02-01 | Sharder et al. | Applicant |
| US6029175(A) | 2000-02-01 | Chow et al. | Applicant |
| US6029176(A) | 2000-02-01 | Cannon | Applicant |
| US6035332(A) | 2000-03-01 | Ingrassia, Jr. et al. | Applicant |
| US6038216(A) | 2000-03-01 | Packer | Applicant |
| US6038310(A) | 2000-03-01 | Hollywood et al. | Applicant |
| US6038610(A) | 2000-03-01 | Belfiore et al. | Applicant |
| US6041307(A) | 2000-03-01 | Ahuja et al. | Applicant |
| US6041324(A) | 2000-03-01 | Earl et al. | Applicant |
| US6044405(A) | 2000-03-01 | Driscoll, III et al. | Applicant |
| US6046980(A) | 2000-04-01 | Packer | Applicant |
| US6049831(A) | 2000-04-01 | Gardell et al. | Applicant |
| US6052718(A) | 2000-04-01 | Gifford | Applicant |
| US6052730(A) | 2000-04-01 | Feliciano et al. | Applicant |
| US6052788(A) | 2000-04-01 | Wesinger et al. | Examiner |
| US6065051(A) | 2000-05-01 | Steele et al. | Applicant |
| US6065062(A) | 2000-05-01 | Periasamy et al. | Applicant |
| US6070191(A) | 2000-05-01 | Narendran et al. | Applicant |
| US6081829(A) | 2000-06-01 | Sidana | Applicant |
| US6092112(A) | 2000-07-01 | Fukushige | Applicant |
| US6092178(A) | 2000-07-01 | Jindal et al. | Examiner |
| US6092204(A) | 2000-07-01 | Baker | Applicant |
| US6098078(A) | 2000-08-01 | Gehani | Applicant |
| US6105028(A) | 2000-08-01 | Sullivan et al. | Applicant |
| US6108673(A) | 2000-08-01 | Brandt et al. | Applicant |
| US6108703(A) | 2000-08-01 | Leighton et al. | Applicant |
| US6112231(A) | 2000-08-01 | DeSimone et al. | Applicant |
| US6112239(A) | 2000-08-01 | Kenner et al. | Applicant |
| US6112240(A) | 2000-08-01 | Pogue et al. | Applicant |
| US6115357(A) | 2000-09-01 | Packer et al. | Applicant |
| US6115752(A) | 2000-09-01 | Chauhan | Applicant |
| US6119143(A) | 2000-09-01 | Dias et al. | Applicant |
| US6125388(A) | 2000-09-01 | Reisman | Applicant |
| US6125394(A) | 2000-09-01 | Rabinovich | Applicant |
| US6128601(A) | 2000-10-01 | Van Horne et al. | Applicant |
| US6128660(A) | 2000-10-01 | Grimm et al. | Applicant |
| US6130890(A) | 2000-10-01 | Leinwand et al. | Applicant |
| US6134583(A) | 2000-10-01 | Herriot | Applicant |
| US6144375(A) | 2000-11-01 | Jain et al. | Applicant |
| US6144702(A) | 2000-11-01 | Yurt et al. | Applicant |
| US6144996(A) | 2000-11-01 | Starnes et al. | Applicant |
| US6151624(A) | 2000-11-01 | Teare et al. | Applicant |
| US6154738(A) | 2000-11-01 | Call | Applicant |
| US6154744(A) | 2000-11-01 | Kenner et al. | Applicant |
| US6154753(A) | 2000-11-01 | McFarland | Applicant |
| US6154777(A) | 2000-11-01 | Ebrahim | Applicant |
| US6157648(A) | 2000-12-01 | Volt et al. | Applicant |
| US6163779(A) | 2000-12-01 | Mantha et al. | Applicant |
| US6167427(A) | 2000-12-01 | Rabinovich et al. | Applicant |
| US6173311(B1) | 2001-01-01 | Hassett et al. | Applicant |
| US6173322(B1) | 2001-01-01 | Hu | Applicant |
| US6178160(B1) | 2001-01-01 | Bolton et al. | Applicant |
| US6181867(B1) | 2001-01-01 | Kenner et al. | Applicant |
| US6185598(B1) | 2001-02-01 | Farber | Applicant |
| US6185619(B1) | 2001-02-01 | Joffe et al. | Applicant |
| US6189030(B1) | 2001-02-01 | Kirsch et al. | Applicant |
| US6189039(B1) | 2001-02-01 | Harvey | Applicant |
| US6195680(B1) | 2001-02-01 | Goldszmidt et al. | Applicant |
| US6205120(B1) | 2001-03-01 | Packer et al. | Applicant |
| US6226618(B1) | 2001-05-01 | Downs | Applicant |
| US6226642(B1) | 2001-05-01 | Beranek et al. | Applicant |
| US6230196(B1) | 2001-05-01 | Guenthner et al. | Applicant |
| US6243760(B1) | 2001-06-01 | Armbruster et al. | Applicant |
| US6249810(B1) | 2001-06-01 | Kiraly | Applicant |
| US6256675(B1) | 2001-07-01 | Rabinovich | Applicant |
| US6263313(B1) | 2001-07-01 | Milsted | Applicant |
| US6266699(B1) | 2001-07-01 | Sevcik | Applicant |
| US6269394(B1) | 2001-07-01 | Kenner et al. | Applicant |
| US6272566(B1) | 2001-08-01 | Craft | Applicant |
| US6275470(B1) | 2001-08-01 | Ricciulli | Applicant |
| US6282569(B1) | 2001-08-01 | Wallis et al. | Applicant |
| US6282574(B1) | 2001-08-01 | Voit | Applicant |
| US6286045(B1) | 2001-09-01 | Griffiths et al. | Applicant |
| US6298041(B1) | 2001-10-01 | Packer | Applicant |
| US6311214(B1) | 2001-10-01 | Rhoads | Applicant |
| US6314565(B1) | 2001-11-01 | Kenner et al. | Applicant |
| US6327622(B1) | 2001-12-01 | Jindal et al. | Applicant |
| US6330602(B1) | 2001-12-01 | Law et al. | Applicant |
| US6332195(B1) | 2001-12-01 | Green et al. | Applicant |
| US6338044(B1) | 2002-01-01 | Cook et al. | Applicant |
| US6347085(B2) | 2002-02-01 | Kelly | Applicant |
| US6360256(B1) | 2002-03-01 | Lim | Applicant |
| US6370571(B1) | 2002-04-01 | Medin, Jr. | Applicant |
| US6370580(B2) | 2002-04-01 | Kriegsman | Applicant |
| US6398245(B1) | 2002-06-01 | Gruse | Applicant |
| US6405252(B1) | 2002-06-01 | Gupta et al. | Applicant |
| US6405257(B1) | 2002-06-01 | Gersht et al. | Applicant |
| US6412000(B1) | 2002-06-01 | Riddle et al. | Applicant |
| US6415280(B1) | 2002-07-01 | Farber et al. | Applicant |
| US6418421(B1) | 2002-07-01 | Hurtado | Applicant |
| US6421726(B1) | 2002-07-01 | Kenner et al. | Applicant |
| US6430618(B1) | 2002-08-01 | Karger et al. | Applicant |
| US6442549(B1) | 2002-08-01 | Schneider | Applicant |
| US6456630(B1) | 2002-09-01 | Packer et al. | Applicant |
| US6460082(B1) | 2002-10-01 | Lumelsky | Applicant |
| US6460085(B1) | 2002-10-01 | Toporek et al. | Applicant |
| US6463454(B1) | 2002-10-01 | Lumelsky | Applicant |
| US6463508(B1) | 2002-10-01 | Wolf | Applicant |
| US6470389(B1) | 2002-10-01 | Chung et al. | Applicant |
| US6473405(B2) | 2002-10-01 | Ricciulli | Applicant |
| US6480893(B2) | 2002-11-01 | Kriegsman | Applicant |
| US6484143(B1) | 2002-11-01 | Swildens et al. | Applicant |
| US6484204(B1) | 2002-11-01 | Rabinovich | Applicant |
| US6484261(B1) | 2002-11-01 | Wiegel | Examiner |
| US6490580(B1) | 2002-12-01 | Dey et al. | Applicant |
| US6493707(B1) | 2002-12-01 | Dey et al. | Applicant |
| US6496856(B1) | 2002-12-01 | Kenner et al. | Applicant |
| US6502125(B1) | 2002-12-01 | Kenner et al. | Applicant |
| US6502215(B2) | 2002-12-01 | Raad et al. | Applicant |
| US6505248(B1) | 2003-01-01 | Casper et al. | Applicant |
| US6529477(B1) | 2003-03-01 | Toporek et al. | Applicant |
| US6553413(B1) | 2003-04-01 | Leighton et al. | Applicant |
| US6553420(B1) | 2003-04-01 | Karger et al. | Applicant |
| US6557054(B2) | 2003-04-01 | Reisman | Applicant |
| US6564251(B2) | 2003-05-01 | Katariya et al. | Applicant |
| US6577595(B1) | 2003-06-01 | Counterman | Applicant |
| US6581090(B1) | 2003-06-01 | Lindbo et al. | Applicant |
| US6584083(B1) | 2003-06-01 | Toporek et al. | Applicant |
| US6587837(B1) | 2003-07-01 | Spagna | Applicant |
| US6591299(B2) | 2003-07-01 | Riddle et al. | Applicant |
| US6611862(B2) | 2003-08-01 | Reisman | Applicant |
| US6625643(B1) | 2003-09-01 | Colby et al. | Applicant |
| US6654344(B1) | 2003-11-01 | Toporek et al. | Applicant |
| US6654807(B2) | 2003-11-01 | Farber et al. | Applicant |
| US6658464(B2) | 2003-12-01 | Reisman | Applicant |
| US6665706(B2) | 2003-12-01 | Kenner et al. | Applicant |
| US6665726(B1) | 2003-12-01 | Leighton et al. | Applicant |
| US6691148(B1) | 2004-02-01 | Zinky et al. | Applicant |
| US6694358(B1) | 2004-02-01 | Swildens et al. | Applicant |
| US6699418(B2) | 2004-03-01 | Okada et al. | Applicant |
| US6708137(B2) | 2004-03-01 | Carley | Applicant |
| US6718328(B1) | 2004-04-01 | Norris | Applicant |
| US6741563(B2) | 2004-05-01 | Packer | Applicant |
| US6751673(B2) | 2004-06-01 | Shaw | Applicant |
| US6754699(B2) | 2004-06-01 | Swildens et al. | Applicant |
| US6754706(B1) | 2004-06-01 | Swildens et al. | Applicant |
| US6763377(B1) | 2004-07-01 | Belknap et al. | Applicant |
| US6763388(B1) | 2004-07-01 | Tsimelzon | Applicant |
| US6778502(B2) | 2004-08-01 | Ricciulli | Applicant |
| US6785704(B1) | 2004-08-01 | McCanne | Applicant |
| US6799221(B1) | 2004-09-01 | Kenner et al. | Applicant |
| US6801576(B1) | 2004-10-01 | Haldeman et al. | Applicant |
| US6834306(B1) | 2004-12-01 | Tsimelzon | Applicant |
| US6842604(B1) | 2005-01-01 | Cook et al. | Applicant |
| US6859791(B1) | 2005-02-01 | Spagna | Applicant |
| US6870851(B1) | 2005-03-01 | Leinwand et al. | Applicant |
| US6874032(B2) | 2005-03-01 | Gersht et al. | Applicant |
| US6901604(B1) | 2005-05-01 | Kiraly | Applicant |
| US6915329(B2) | 2005-07-01 | Kriegsman | Applicant |
| US6928442(B2) | 2005-08-01 | Farber et al. | Applicant |
| US6934255(B1) | 2005-08-01 | Toporek et al. | Applicant |
| US6950623(B2) | 2005-09-01 | Brown et al. | Applicant |
| US6963910(B1) | 2005-11-01 | Belknap | Applicant |
| US6963980(B1) | 2005-11-01 | Mattsson | Applicant |
| US6963981(B1) | 2005-11-01 | Bailey et al. | Applicant |
| US6965890(B1) | 2005-11-01 | Dey et al. | Applicant |
| US6970432(B1) | 2005-11-01 | Hankins et al. | Applicant |
| US6973485(B2) | 2005-12-01 | Ebata et al. | Applicant |
| US6973490(B1) | 2005-12-01 | Robertson et al. | Applicant |
| US6976090(B2) | 2005-12-01 | Ben-shaul | Applicant |
| US6981050(B1) | 2005-12-01 | Tobias et al. | Applicant |
| US6981180(B1) | 2005-12-01 | Bailey et al. | Applicant |
| US6996616(B1) | 2006-02-01 | Leighton et al. | Applicant |
| US7003572(B1) | 2006-02-01 | Lownsbrough et al. | Applicant |
| US7007089(B2) | 2006-02-01 | Freedman | Applicant |
| US7010578(B1) | 2006-03-01 | Lewin et al. | Applicant |
| US7012900(B1) | 2006-03-01 | Riddle | Applicant |
| US7039633(B1) | 2006-05-01 | Dey et al. | Applicant |
| US7047300(B1) | 2006-05-01 | Oehrke et al. | Applicant |
| US7054935(B2) | 2006-05-01 | Farber et al. | Applicant |
| US7058706(B1) | 2006-06-01 | Iyer et al. | Applicant |
| US7069177(B2) | 2006-06-01 | Carley | Applicant |
| US7096266(B2) | 2006-08-01 | Lewin et al. | Applicant |
| US7103564(B1) | 2006-09-01 | Ehnebuske | Applicant |
| US7103645(B2) | 2006-09-01 | Leighton et al. | Applicant |
| US7110984(B1) | 2006-09-01 | Spagna | Applicant |
| US7117259(B1) | 2006-10-01 | Rohwer | Applicant |
| US7185052(B2) | 2007-02-01 | Day | Applicant |
| US7188085(B2) | 2007-03-01 | Pelletier | Applicant |
| US7206748(B1) | 2007-04-01 | Gruse | Applicant |
| US7219153(B1) | 2007-05-01 | Day | Applicant |
| US7254645(B2) | 2007-08-01 | Nishi | Applicant |
| US7373644(B2) | 2008-05-01 | Aborn | Applicant |
| US7577754(B2) | 2009-08-01 | Garcia-Luna-Aceves et al. | Applicant |
| US2001/0029525(A1) | 2001-10-01 | Lahr | Applicant |
| US2001/0056500(A1) | 2001-12-01 | Farber et al. | Applicant |
| US2002/0018449(A1) | 2002-02-01 | Ricciulli | Applicant |
| US2002/0023164(A1) | 2002-02-01 | Lahr | Applicant |
| US2002/0023165(A1) | 2002-02-01 | Lahr | Applicant |
| US2002/0040404(A1) | 2002-04-01 | Lahr | Applicant |
| US2002/0042817(A1) | 2002-04-01 | Lahr | Applicant |
| US2002/0046273(A1) | 2002-04-01 | Lahr et al. | Applicant |
| US2002/0046405(A1) | 2002-04-01 | Lahr | Applicant |
| US2002/0049857(A1) | 2002-04-01 | Farber et al. | Applicant |
| US2002/0059592(A1) | 2002-05-01 | Kiraly | Applicant |
| US2002/0066038(A1) | 2002-05-01 | Mattsson | Applicant |
| US2002/0073199(A1) | 2002-06-01 | Levine et al. | Applicant |
| US2002/0078233(A1) | 2002-06-01 | Biliris | Applicant |
| US2002/0082999(A1) | 2002-06-01 | Lee et al. | Applicant |
| US2002/0083124(A1) | 2002-06-01 | Knox et al. | Applicant |
| US2002/0087684(A1) | 2002-07-01 | Foth | Examiner |
| US2002/0099850(A1) | 2002-07-01 | Farber et al. | Applicant |
| US2002/0116444(A1) | 2002-08-01 | Chaudhri et al. | Applicant |
| US2002/0124080(A1) | 2002-09-01 | Leighton et al. | Applicant |
| US2002/0129134(A1) | 2002-09-01 | Leighton et al. | Applicant |
| US2002/0131645(A1) | 2002-09-01 | Hamilton | Applicant |
| US2002/0143798(A1) | 2002-10-01 | Lisiecki et al. | Applicant |
| US2002/0143888(A1) | 2002-10-01 | Lisiecki et al. | Applicant |
| US2002/0145975(A1) | 2002-10-01 | MeLampy et al. | Examiner |
| US2002/0147774(A1) | 2002-10-01 | Lisiecki et al. | Applicant |
| US2002/0163882(A1) | 2002-11-01 | Bornstein et al. | Applicant |
| US2002/0199016(A1) | 2002-12-01 | Freedman | Applicant |
| US2003/0009444(A1) | 2003-01-01 | Eidler et al. | Applicant |
| US2003/0018966(A1) | 2003-01-01 | Cook et al. | Applicant |
| US2003/0028623(A1) | 2003-02-01 | Hennessey et al. | Applicant |
| US2003/0028626(A1) | 2003-02-01 | Hennessey et al. | Applicant |
| US2003/0028777(A1) | 2003-02-01 | Hennessey et al. | Applicant |
| US2003/0055972(A1) | 2003-03-01 | Fuller et al. | Applicant |
| US2003/0061263(A1) | 2003-03-01 | Riddle | Applicant |
| US2003/0061280(A1) | 2003-03-01 | Bulson et al. | Applicant |
| US2003/0078888(A1) | 2003-04-01 | Lee et al. | Applicant |
| US2003/0078889(A1) | 2003-04-01 | Lee et al. | Applicant |
| US2003/0105604(A1) | 2003-06-01 | Ash et al. | Applicant |
| US2003/0149581(A1) | 2003-08-01 | Chaudhri | Applicant |
| US2004/0010588(A1) | 2004-01-01 | Slater et al. | Applicant |
| US2004/0022194(A1) | 2004-02-01 | Ricciulli | Applicant |
| US2004/0039798(A1) | 2004-02-01 | Hotz et al. | Applicant |
| US2004/0064832(A1) | 2004-04-01 | Tsukidate et al. | Applicant |
| US2004/0139097(A1) | 2004-07-01 | Farber et al. | Applicant |
| US2004/0177148(A1) | 2004-09-01 | Tsimelzon | Applicant |
| US2005/0033858(A1) | 2005-02-01 | Swildens et al. | Applicant |
| US2005/0038851(A1) | 2005-02-01 | Kriegsman | Applicant |
| US2005/0060493(A1) | 2005-03-01 | Krissell et al. | Applicant |
| US2005/0100027(A1) | 2005-05-01 | Leinwand et al. | Applicant |
| US2005/0114296(A1) | 2005-05-01 | Farber et al. | Applicant |
| US2005/0240966(A1) | 2005-10-01 | Hindle et al. | Applicant |
| US2005/0262104(A1) | 2005-11-01 | Robertson et al. | Applicant |
| US2006/0143293(A1) | 2006-06-01 | Freedman | Applicant |
| US2007/0055764(A1) | 2007-03-01 | Dilley et al. | Applicant |
| US2009/0254661(A1) | 2009-10-01 | Fullagar et al. | Applicant |
| US2010/0169772(A1) | 2010-07-01 | Stallings et al. | Applicant |
| US2010/0312861(A1) | 2010-12-01 | Kolhi et al. | Applicant |
| US2010/0332595(A1) | 2010-12-01 | Fullagar et al. | Applicant |
| AU763380 | 2003-11-01 | Applicant | |
| CA2202572 | 1998-10-01 | Applicant | |
| CA2335661 | 2001-09-01 | Applicant | |
| CA2335662 | 2001-09-01 | Applicant | |
| CA2467998 | 2006-04-01 | Applicant | |
| CNZL99810853.7 | 2004-08-01 | Applicant | |
| EP800143(A) | 1997-10-01 | Applicant | |
| EP801487(A2) | 1997-10-01 | Applicant | |
| EP817020(A2) | 1998-01-01 | Applicant | |
| EP817444 | 1998-01-01 | Applicant | |
| EP824236(A2) | 1998-02-01 | Applicant | |
| EP865180(A2) | 1998-09-01 | Applicant | |
| EP1104555 | 2001-06-01 | Applicant | |
| GB2281793 | 1995-03-01 | Applicant | |
| GB2353877 | 2004-03-01 | Applicant | |
| IL140935 | 2006-03-01 | Applicant | |
| JP5-130144 | 1993-05-01 | Applicant | |
| JP5162529 | 1993-06-01 | Applicant | |
| JP7066829 | 1995-03-01 | Applicant | |
| JP8328583 | 1996-09-01 | Applicant | |
| JP10-027148 | 1998-01-01 | Applicant | |
| JP10-70571 | 1998-03-01 | Applicant | |
| JP10-093552 | 1998-04-01 | Applicant | |
| JP10-126445 | 1998-05-01 | Applicant | |
| JP2000259539 | 2000-09-01 | Applicant | |
| JP2001-053793 | 2001-02-01 | Applicant | |
| JP2002522995 | 2002-07-01 | Applicant | |
| JP3566626 | 2004-06-01 | Applicant | |
| JP2005124165 | 2005-05-01 | Applicant | |
| JP3762649 | 2006-01-01 | Applicant | |
| KR1999-72167 | 1999-09-01 | Applicant | |
| NI176482 | 2003-08-01 | Applicant | |
| WO96/42041 | 1996-12-01 | Applicant | |
| WO97/11429 | 1997-03-01 | Applicant | |
| WO97/29423 | 1997-08-01 | Applicant | |
| WO98/04985 | 1998-02-01 | Applicant | |
| WO98/06033 | 1998-02-01 | Applicant | |
| WO9909726(A) | 1999-02-01 | Applicant | |
| WO9929083(A) | 1999-06-01 | Applicant | |
| WO/14633 | 2000-03-01 | Applicant | |
| WO52594(A) | 2000-09-01 | Applicant | |
| WO-00/52594(A2) | 2000-09-01 | Applicant | |
| WO/62502 | 2000-10-01 | Applicant | |
| WO1/35601 | 2001-05-01 | Applicant | |
| WO1/52497 | 2001-07-01 | Applicant |
Non-Patent Literature (108)
- Chinese Reexamination Notification, dated Jul. 5, 2012, Appl. No. 02821371.8, 5 pgs.Applicant
- “International Search Report, dated Jan. 23, 2012, Intl Appl. No. PCT/US11/51252”, Intl Filing Date Sep. 12, 2011, 3 pgs.Applicant
- “Written Opinion, dated Jan. 23, 2012, Intl Appl No. PCT/US11/51252”, Intl Filing Date Sep. 12, 2011, 5 pgs.Applicant
- Chinese Reexamination Notification, dated Sep. 13, 2012, Chinese App. No. 02821371.8, including translation, 10 pgs.Applicant
- European Examination Report dated Sep. 6, 2012, EP Application No. 10178695.2, 4 pgs.Applicant
- “Content Management Technology/Industry News,” Content Technologies Trends and Advice, Gilbane Report, News for Jun. 1999 [21 pages].Applicant
- “Exporting Web Server Final Report,” http://www.cs.technion.ac.il/Labs/Lccn/projects/spring97/project4/final—report.html, Spring 1997 (downloaded Jul. 7, 2007).Applicant
- “Local Area Network Server Replacement Procedure”, IBM Technical Disclosure Bulletin, vol. 38, No. 1, (Jan. 1995), 235-236.Applicant
- “Patent Abstracts of Japan, Electronic Mail Multiplexing System and Communication Control Method in the System” (Appln. No. JP19930162529), (Jun. 30, 1993) (Pub. No. JP 7066829).Applicant
- “Patent Abstracts of Japan, Method and Device for Repeating and Converting Information”, (Appln. No. JP1996 0328583) (Pub. No. JP10171727), Jun. 26, 1998.Applicant
- Adler, R. M., “Distributed Coordination Models for Client/Server Computing,” Computer 28, 4 (Apr. 1995), 14-22.Applicant
- Aggarwal, A. et al., “Performance of Dynamic Replication Schemes for an Internet Hosting Service”. Technical Report, AT&T Labs, Oct. 1998.Applicant
- Andresen et al., “SWEB: Towards a Scalable World Wide Web Server on Multicomputers”, Proc. IPPS, (Apr. 15, 1996), 850-856.Applicant
- Andresen, D., et al., Multiprocessor scheduling with client resources to improve the response time of WWW applications, Proc. 11th Int'l Conf. on Supercomputing (Austria, Jul. 1997). ICS '97. ACM Press, NY, NY, 92-99.Applicant
- Awerbuch, B. et al., Distributed Paging for General Networks. In Proc. of the 7th ACM-SIAM Symposium on Discrete Algorithms, pp. 574-583, Jan. 1996.Applicant
- Awerbuch, et al., Competitive Distributed File Allocation. in Proc. of the 25th Ann. ACM Symp. on Theory of Computing, pp. 164-173, May 1993.Applicant
- Baentsch, M, et al. “Enhancing the Web's Infrastructure: From Caching to Replication.” IEEE Internet Computing, 1(2):18-27, Mar. 1997.Applicant
- Bartal, Y., et al., “Competitive Algorithms for Distributed Data Management”, 24th Annual ACM STOC, 5/92, Victoria, B.C. Canada.Applicant
- Basturk, E., et al., “Using network layer anycast for load distribution in the Internet,” Tech. Rep., IBM T.J. Watson Research Center, Jul. 1997 (21 pgs.).Applicant
- Berners-Lee, T., et al., RFC 1738—Uniform Resource Locators, Dec. 1994.Applicant
- Bestavros, A., “Speculative Data Dissemination and Service to Reduce Server Load, Network Traffic and Service Time in Distributed Information Systems”, In Proc. ICDE '96: The 1996 Int'l Conf. on Data Engineering, (Mar. 1996), 4 pages.Applicant
- Bestavros, et al., “Server-Initiated Document Dissemination for the WWW,” IEEE Data Engineering Bulletin 19(3):3-11, Sep. 1996.Applicant
- Bhattacharjee et al., “Application-layer anycasting,” in Proc. IEEE INFOCOM '97, Apr. 1997.Applicant
- Braun, H., et al., “Web traffic characterization: an assessment of the impact of caching documents from NCSA's web server,” Comput. Netw. ISDN Syst. 28, 1-2 (Dec. 1995), 37-51.Applicant
- Brisco, T. P. RFC 1794: DNS support for load balancing, Apr. 1995.Applicant
- Carter et al., “Dynamic server selection using bandwidth probing in wide-area networks,” Tech. Rep. BU-CS-96-007, Comp. Sci. Dept., Boston University, Mar. 1996.Applicant
- Carter et al., Server selection using dynamic path characterization in Wide-Area Networks, IEEE INFOCOM '97, Apr. 7-12, 1997 (pp. 1014-1021).Applicant
- Carter, J. Lawrence et al., “Universal Classes of Hash Functions”, Journal of Computer and System Sciences, vol. 18, No. 2, (Apr. 1979), 143-154.Applicant
- Cate, V. “Alex: a global file system”, in Proc. Usenix Conf. on File Systems, May 1992, pp. 1-11.Applicant
- Chankhunthod, A. et al., “A Hierarchical Internet Object Cache”, Proc. of the 1996 USENIX Technical Conf., Jan. 1996, pp. 153-163.Applicant
- Cisco Systems, Inc., Cisco DistributedDirector 2500 Series Installation and Configuration Guide, pp. xix-xxii; 1-1 to 1-12; 6-1 to 6-18; 7-1 to 7-18; 8-1 to 8-24, Dec. 1997, downloaded Apr. 2007: http://www.cisco.com/univercd/cc/td/doc/product/iaabu/distrdir/dd2501/.Applicant
- Cisco Systems, Inc., Cisco DistributedDirector 4700-M Installation and Configuration Guide, pgs. xix-xxii; 1-1 to 1-14; 7-1 to 7-18, 8-1 to 8-20; Dec. 1997, [downloaded Apr. 2007 from http://www.cisco.com/univercd/cc/td/doc/product/iaabu/distrdir/dd4700m/].Applicant
- Claims (amended) filed by applicant in Japanese Patent Application No. 2003-531370, Jan. 17, 2008 (English translation provided by foreign associate) [12 pgs.].Applicant
- Claims (amended) filed by applicant in KIPO for Korean Patent Application No. KR 10 2004 7004613, Jul. 24, 2009 (English translation provided by foreign associate) [10 pgs.].Applicant
- CN Appln. No. 02821371.8—Jan. 30, 2011 SIPO Office Action.Applicant
- Cohen, J., et al., “Cache Array Routing Protocol v1.1”, Sep. 29, 1997; http://tools.ietf.org/id/draft-vinod-carp-v1-01.txt (Last-Modified: Wed, Oct. 1, 1997).Applicant
- Colajanni, M. and Yu, P. S. 1997. Adaptive TTL schemes for load balancing of distributed Web servers. SIGMETRICS Perform. Eval. Rev. 25, 2 (Sep. 1997), 36-42.Applicant
- Communication (1 page) and European Search Report (with Annex) for Appln. No. EP 02799672.7, dated Apr. 17, 2008.Applicant
- Cormen, T. H., et al., “Introduction to Algorithms”, The MIT Press, Cambridge, Massachusetts, (1994), 219-243, 991-993.Applicant
- Crovella et al., Dynamic server selection in the Internet, 3rd IEEE Workshop on the Arch. and Implementation of High Performance Computer Sys. '95, pp. 158-162, Aug. 1995.Applicant
- Danzig, P. B., et al., “An analysis of wide-area name server traffic: a study of the Internet Domain Name System,” Conf. Proc. Communications Architectures & Protocols (Aug. 1992). D. Oran, Ed. SIGCOMM '92. ACM Press, New York, NY, 281-292.Applicant
- De Bra, P.M.E., et al., “Information Retrieval in the World Wide Web: Making Client-Based Searching Feasible”, Computer Networks and ISDN Systems, NL, North Holland Publishing, Amsterdam, vol. 27, No. 2, ISSN: 0169-7552, (Nov. 1, 1994), 183-192.Applicant
- Deering, S. E., et al, “Multicast routing in datagram internetworks and extended LANs,” ACM Trans. Comput. Syst. 8, 2 (May. 1990), 85-110.Applicant
- Devine, R., “Design and Implementation of DDH: A Distributed Dynamic Hashing Algorithm”, In Proc. 4th Int'l Conf. on Foundations of Data Organizations and Algorithms, (Oct. 1993), 101-114.Applicant
- Doi, K. “Super Proxy Script—How to make distributed proxy servers by URL hashing,” Sharp Corp., http://naragw.sharp.co.jp/sps/, dates unknown (1996-2000), download Jul. 7, 2007.Applicant
- European Patent Application No. 02799672.7, file history as of Apr. 23, 2010 (downloaded from EPO).Applicant
- Feeley, M., et al., “Implementing Global Memory Management in a Workstation Cluster”, In Proc. 15th ACM Symp. on Operating Systems Principles, (Dec. 1995), 201-212.Applicant
- Floyd, S., et al., “A Reliable Multicast Framework for Light-Weight Sessions and Application Level Framing”, In Proc. of ACM SIGCOMM '95, 342-356, Aug. 1995.Applicant
- Fox, A., “A Framework for Separating Server Scalability and Availability from Internet Application Functionality,” PhD thesis, University of California, Berkeley, Dec. 1998.Applicant
- Fox, A., et al,, “Cluster-based scalable network services”, Proc. 16th ACM Symp. on Operating Systems Principles (Saint Malo, France, Oct. 5-8, 1997). W. M. Waite, Ed. SOSP '97. ACM Press, New York, NY, 78-91.Applicant
- Fredman, M., et al., “Storing a Sparse Table with 0(1) Worst Case Access Time”, J. ACM, vol. 31, No. 3, (Jul. 1984), 538-544.Applicant
- Gadde, S., et al., “Reduce, reuse, recycle: An approach to building large internet caches,” in Workshop on Hot Topics in Operating Systems (HotOS), Apr. 1997, pp. 93-98.Applicant
- Goldszmidt, et al., “Load Distribution for Scalable Web Servers: Summer Olympics 1996—A Case Study,” In Proc. 8th IFIP/IEEE Int'l Workshop on Distributed Systems: Operations and Management, Sydney, Australia, Oct. 1997.Applicant
- Grigni, M., et al., “Tight Bounds on Minimum Broadcasts Networks”, SIAM J. Disc. Math. 4 (May 1991), 207-222.Applicant
- Gulbrandsen, A., et al., “A DNS RR for specifying the location of services (DNS SRV)”, Network Working Group, RFC 2052, Oct. 1996.Applicant
- Guyton et al., “Locating nearby copies of replicated Internet servers,” Proc. ACM SIGCOMM '95, pp. 288-298, Oct. 1995.Applicant
- Gwertzman, J., et al., “The Case for Geographical Push-Caching”, Proc. Workshop on Hot OS '95, (May 4, 1995), 51-55.Applicant
- Gwertzman, J., et al., “World-Wide Web Cache Consistency”, Proc. 1996 USENIX Tech. Conf., pp. 141-151, San Diego, CA, Jan. 1996.Applicant
- International Preliminary Examination Report (IPER), Form PCT/IPEA/409, for PCT/US02/30921, Oct. 2005 [5 pgs.].Applicant
- International Search Report (ISR), Form PCT/ISA/210, for PCT/US02/30921, Jan. 28, 2004 [5 pgs.].Applicant
- Japanese Patent Office (JPO) “Notice of Reasons for Rejection”, for Japanese Patent Application No. 2003-531370, Jul. 17, 2007, “Organized Translation” provided by foreign associate [2 pgs.].Applicant
- Japanese Patent Publication JP 10-70571,Jun. 1997, International Business Machines Corporation, English translation provided with Office Action [34 pgs.].Applicant
- Japanese Patent Publication JP 5-130144, Published May 25, 1993, Fujitsu Ltd., English translation provided with Office Action [9 pgs.].Applicant
- Jeffery, C., et al., “Proxy sharing proxy servers.” In Proc. IEEE etaCOM Conf., pp. 116-119, May 1996.Applicant
- Karger, D., et al., “Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web”, in Proc. 29th Annual ACM Symp. on Theory of Computing, (May 1997), 654-663.Applicant
- Korean Intellectual Property Office (KIPO), Notice of Preliminary Rejection (Non-Final)) for Korean Patent Application No. KR 10 2004 7004613, Apr. 24, 2009, English translation (provided by foreign associate) [5 pgs.].Applicant
- Korkea-aho, M. (Nov. 5, 1995). Scalability in Distributed Multimedia Systems, Technical report TKO-B128, Helsinki University of Technology.Applicant
- Krishnamurthy, B. et al., Study of piggyback cache validation for proxy caches in the World Wide Web, in: Symp. on Internet Technology and Systems, USENIX Association, Dec. 1997.Applicant
- Kwan et al., NCSA's World Wide Web Server: Design and Performance, IEEE, pp. 68-74, Nov. 1995.Applicant
- Litwin, W., et al., “LH*—A Scalable, Distributed Data Structure”, ACM Trans. on Database Systems, vol. 21, No. 4, pp. 480-525, Dec. 1996.Applicant
- Luotonen et al., World-Wide Web Proxies, CERN, Apr. 1994 (modified May 24, 1994).Applicant
- Malpani, R., et al., “Making World Wide Web Caching Servers Cooperate”, in Proc. 4th Int'l. World Wide Web Conf. (Dec. 1995), 10 pages (downloaded from http://www.w3.org/Conferences/WWW4/Papers/59/ on Jul. 7, 2007).Applicant
- Mockapetris et al., “Development of the Domain Name System,” Proc. SIGCOMM '88 Computer Communications Review, vol. 18, No. 4, Aug. 1988.Applicant
- Mockapetris, P., RFC 1034: Domain Names—Concepts and Facilities, Nov. 1987.Applicant
- Mockapetris, P., RFC 1035: Domain Names—Implementation and Specification, Nov. 1987.Applicant
- Mourad et al., “Scalable Web Server Architectures,” iscc, 2nd IEEE Symposium on Computers and Communications (ISCC '97), Jul. 1997, pp. 12-16.Applicant
- Nisan, N. 1990. Pseudorandom generators for space-bounded computations. In Proc. 22nd Annual ACM Symp. on theory of Computing (Baltimore, MD, U.S., May 13-17, 1990). H. Ortiz, Ed. STOC '90. ACM Press, New York, NY, 204-212.Applicant
- Office Action dated Dec. 8, 2009 in Japanese Application No. 2001-504633 (based on PCT/US001/16732, titled “On-Demand Overlay Routing for Computer-based communication networks,” Inventor: Ricciulli, Livio) [in Japanese, 2 pgs., with 2 page English translation].Applicant
- Office Action for Chinese (PRC) patent application No. 02821371.8, from the Patent Office of the People's Republic of China, dated Aug. 3, 2010 (non-official translation, 9 pgs., including 2 pg. summary).Applicant
- Office Action from Chinese Patent Office for application non. 02821371.8, Jan. 29, 2010, in Chinese with translation.Applicant
- Office Action from Chinese Patent Office for application non. 02821371.8, Jan. 4, 2008, in Chinese with translation.Applicant
- Office Action from Chinese Patent Office for application non. 02821371.8, May 8, 2009, in Chinese with translation.Applicant
- Oguchi et al., A Study of Caching Proxy Mechanisms Realized on Wide Area Distributed Networks, High Performance Distributed Computing, 5th Intl Symp., pp. 443-449, Aug. 1996.Applicant
- Palmer, M., et al., “Fido: A Cache that Learns to Fetch”, In Proc. the 17th Int'l Conf. on Very Large Data Bases, (Sep. 1991), 255-264.Applicant
- Panigrahy, R., “Relieving Hot Spots on the World Wide Web”, Master's thesis, MIT EECS, Jun. 1997, pp. 1-66.Applicant
- Patent Abstracts of Japan, “Server System for Internet”, Pub. No. 10-027148, pub. date Jan. 27, 1998, Applicant: Hitachi, computer translation, 12 pgs.Applicant
- Patent Abstracts of Japan, “Communication Connection Method With a Plurality of Hosts having Common Identifier”, Appln. No. JP 08-20114, Pub. No. 10-093552, Applicant Nippon Telegraph & Telephone, published Apr. 1998.Applicant
- Patent Abstracts of Japan, “Domain Name Solving Method and Domain Name Solving System”, Appln. No. JP19990223763, Pub. No. 2001-053793, Applicant Nippon Telegraph & Telephone, published Feb. 23, 2001.Applicant
- Peleg, D., et al., “The Availability of Quorum Systems”, Information and Computation, 123, (Dec. 1995), 210-223.Applicant
- Petri S., et al., “Load Balancing and Fault Tolerance in Workstation Clusters. Migrating Groups of Communicating Processes.”, Operating Systems Review, vol. 29, No. 4, Oct. 1995, pp. 25-36.Applicant
- Plaxton, G. C., et al., “Fast Fault-Tolerant Concurrent Access to Shared Objects”, In Proc. 37th IEEE Symp. of Foundations of Computer Science, (Oct. 1996), 570-579.Applicant
- Postel, RFC 1591—Domain Name System Structure and Delegation, Mar. 1994.Applicant
- Rabin, M. O., 1989, “Efficient dispersal of information for security, load balancing, and fault tolerance,” J. ACM 36, 2 (Apr. 1989), 335-348.Applicant
- Rabinovich, M. et al., “Dynamic Replication on the Internet Work Project No. 3116-17-7006”, AT&T Labs Research, Mar. 5, 1998.Applicant
- Rabinovich, M. et al., RaDaR: A Scalable Architecture for a Global Web Hosting Service, WWW8, May 1999.Applicant
- Ross, K.W., “Hash-Routing for Collections of Shared Web Caches”, IEEE Network Magazine, 11, 7:37-44, Nov.-Dec. 1997.Applicant
- Schemers, R., “Ibnamed—A load balancing name server written in Perl,” 1995 LISA IX—Sep. 17-22, 1995—Monterey, CA.Applicant
- Schuba, Christoph; “Addressing Weaknesses in the Domain Name System Protocol,” COAST Laboratory, Dept. of Computer Sciences, Purdue University; West Layfayette, IN; Aug. 1993, p. 1-87.Applicant
- Smith, “What can Archives offer the World Wide Web?”, Technical Report 11, University of Kent, Computing Laboratory, University of Kent, Canterbury, UK, Mar. 1994.Applicant
- SNMPV2 Working Group J Case SNMP Research et al: “Management Information Base for Version 2 of the Simple Network Management Protocol (SNMPv2); rfc1907.txt” IETF Standard, Internet Engineering Task Force, IETF, CH, Jan. 1996.Applicant
- Tarjan, Robert E., et al., “Storing a Sparse Table”, Commun.ACM, 22, 11, (Nov. 1979), 606-611.Applicant
- Thaler, D. G. and Ravishankar, C. V. 1998. Using name-based mappings to increase hit rates. IEEE/ACM Trans. Netw. 6, 1 (Feb. 1998), 1-14.Applicant
- U.S. P.T.O. Non-Final Office Action in U.S. Appl. No. 11/980,689, May 20, 2009.Applicant
- Vitter, J. S., et al., “Optimal Prefetching via Data Compression,” Proc. 32nd Annual IEEE Symposium on Foundations of Computer Science (Oct. 1991).Applicant
- Vixie, Paul; “Name Server Operations Guide for BIND,” Internet Software Consortium; La Honda, CA; p. SMM:10-2—SMM:10-30 (undated, 1996).Applicant
- Wessels, Intelligent Caching for World-Wide Web Objects, Masters Thesis, University of Colorado, Jan. 1995 (also presented at INET '95 in Jun. 1995).Applicant
- Yao, A. C. 1981. Should Tables Be Sorted?. J. ACM 28, 3 (Jul. 1981), 615-628.Applicant
- “Extended European Search Report dated Dec. 6, 2011”, App. No. 10178695.2, 7 pgs.Applicant