Multi-level cache architecture and cache management method for peer-to-peer name resolution protocol

[0001] This invention relates generally to cache memory data structures and memory management methods and, more particularly, to cache memory storage architectures and methods for storage and sorting of peer-to-peer (P2P) name resolution information.

[0002] Peer-to-peer (P2P) networks, which served as the foundation of the Internet, are currently experiencing a resurgence in popularity and use. The departure from the most recent server-centric driven networks, however, requires that each individual peer acquire and maintain contact and location information for the other peers and discoverable resources available on the network. While some hybrid P2P systems exist that utilize a server to act as an index or central address depository of available peers and/or resources, true P2P systems do not rely on any such servers. As such, and with the shear number of peers and discoverable resources available on a large scale P2P network, internal management of the stored contact and directory information within each peer becomes a critically important task. Without such a management system, the peer's own memory can quickly become consumed with this contact and directory information.

[0003] Many distributed resolution mechanisms for P2P networks are designed around a structured knowledge of the name space in which peer and resource resolution is being performed. The name space knowledge that is developed and maintained is usually structured around each name that is locally registered in the name space. That is to say, information about other peers within a certain distance of the registered name is maintained in the knowledge base for each registered name. When two or more names are registered locally, there can be significant overlap between the knowledge bases maintained for each name, especially when the names are registered in close proximity to one another. This redundant knowledge consumes additional resources at each node and throughout the network. Considering that a typical peer will have several, possibly in the hundreds, of names registered (to account for each discoverable resource, file, service, application, etc. available at that peer), the resulting redundancy in knowledge bases becomes intolerable. In some cases, the pieces of information stored on a single host can exceed the number of active pieces of information in the entire name space due to the duplication of this information in the overlapping knowledge bases.

[0004] There exists a need, therefore, for a memory structure and information maintenance system that allows efficient P2P name resolution through the acquisition and maintenance of a knowledge base for registered names in the P2P namespace that does not consume excessive peer resources through the duplication of information for multiple name/resource registrations.

[0005] In view of the above, it is an objective of the present invention to provide a new and improved cache memory structure and information management system that will allow efficient name resolution in a peer-to-peer cloud and that will not consume excessive resources even, and particularly, when there are multiple resources and/or services registered at the peer.

[0006] The segmented cache architecture of one embodiment of the present invention stores PNRP IDs in different segments according to the number of PNRP IDs being stored and their relative distance in the PNRP name space. Underlying the present invention is the statistical assumption that the probability of knowing a given PNRP ID is inversely proportional to the distance of that ID from the locally registered PNRP ID. By storing PNRP IDs in accordance with the system and method of the present invention, storing duplicate information in different caches is reduced and the number of hops required for name resolution can be reduced.

[0007] Proceeding with the statistical assumption that the peers in the name space are approximately evenly distributed throughout the name space, and knowing the number of instantiated IDs, the average spacing between peers can be determined. Based on this information, the cache regions or levels in accordance with one embodiment of the present invention may be determined relative to each locally registered PNRP ID based on this average spacing between peers. A first or lowest cache region or level will include PNRP IDs at a granularity of the average distance, and will span the range of / ten times the average distance from the locally registered ID. The next cache region will have PNRP IDs at a granularity of ten times the average distance, and will span the range / one hundred times the average distance from the locally registered ID, but excluding the first region. This magnitude increase in granularity and range will increase until the N^th cache region. This N^th region will contain IDs at a granularity of 10^N1 times the average distance, and will span the range /10^N times the average distance from the locally registered ID, but excluding the previous N1 regions. The number of cache levels required, N, is approximately equal to log( of instantiated IDs).

[0008] In a large peer-to-peer cloud, the number of instantiated PNRP IDs may not be known or even knowable due to the dynamic nature of the peers themselves. Therefore, in an alternate embodiment of the present invention a cache segment splitting policy is implemented. In this embodiment, the highest cache region is established to cover the entire name space and to hold twenty PNRP IDs. Once this highest region is filled, the receipt of another peer address certificate (PAC) will result in a lower level cache region being instantiated. This region will cover ten percent of the next higher region centered at the locally registered ID. All information from the higher cache region within this range will be migrated to this lower cache region to eliminate any duplication of information. Once this new region is filled with twenty entries, the receipt of another PAC that fits in this region will cause the instantiation of a lower cache region as ten percent of that newly filled region. All information from the higher cache region within this range will be migrated to this lower cache region to eliminate any duplication of information. This process will continue each time the lowest level cache region becomes full and another PAC is received that fits in that region. However, if PACs are received that belong in the upper level cache regions, a cached PAC from that region must be deleted before the new PAC can be added.

[0009] Recognizing that many peers will have multiple resources and services discoverable in the P2P cloud, the cache architecture and method of an embodiment of the present invention supports multiple locally registered PNRP IDs without duplicating information in the cache. This is accomplished according to one embodiment by combining overlapping cache segments within a region since they each have the same granularity within that particular cache region. This combination does not affect the instantiation of lower level cache regions where there is no overlap.

[0010] One embodiment of the present invention utilizes a PAC cache tree, an instantiated segment tree, and an uninstantiated segment tree. The cache tree is a collection of PACs sorted by PNRP ID and organized into a binary tree, e.g. a red-black tree in one embodiment, to simplify insertion, location, and deletion of PACs. The instantiated segment tree is a collection of pointers indicating the starting address of each instantiated segment in the cache architecture. This instantiated segment tree is also preferably organized into a red-black tree to simplify searching for the appropriate cache segment into which a newly received PAC belongs. The uninstantiated segment tree contains the next cache segments to be instantiated if the next higher cache segment with which it overlaps becomes full. In one embodiment, only the cache segments that are in the next immediate cache region are included in the uninstantiated segment tree.

[0011] Additional features and advantages of the invention will be made apparent from the following detailed description of illustrative embodiments which proceeds with reference to the accompanying figures.

[0012] While the appended claims set forth the features of the present invention with particularity, the invention, together with its objects and advantages, may be best understood from the following detailed description taken in conjunction with the accompanying drawings of which:

[0013] FIG. 1 is a block diagram generally illustrating an exemplary computer system on which the present invention resides;

[0014] FIG. 2 is a peer-to-peer number space line diagram illustrating the concept of distance between peers modulo ID_MAX;

[0015] FIG. 3 is a graphical illustration of cache regions and segments instantiated in accordance with an embodiment of the present invention in relation to a locally registered PNRP ID in a simplified peer-to-peer number space;

[0016] FIGS. 4A-D are graphical illustrations of the successive instantiation of cache regions and segments in accordance with a segment splitting policy of an embodiment of the present invention as additional peer address certificates (PACs) are received in relation to a single locally registered PNRP ID in a simplified peer-to-peer number space;

[0017] FIGS. 5A-D are graphical illustrations of the successive instantiation of cache regions and segments in accordance with a segment splitting policy of an embodiment of the present invention as additional peer address certificates (PACs) are received in relation to multiple locally registered PNRP IDs in a simplified peer-to-peer number space and of the surrender of cache regions and segments upon the removal of a locally registered PNRP ID; and

[0018] FIG. 6 is a graphical illustration of a relationship between a PAC segment tree, an instantiated segment tree, and an uninstantiated segment tree at one stage of the successive instantiation illustrated in FIG. 5B in accordance with one embodiment of the present invention.

[0019] Turning to the drawings, wherein like reference numerals refer to like elements, the invention is illustrated as being implemented in a suitable computing environment. Although not required, the invention will be described in the general context of computer-executable instructions, such as program modules, being executed by a personal computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the invention may be practiced with other computer system configurations, including hand-held devices, multi-processor systems, microprocessor based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, and the like. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0020] FIG. 1 illustrates an example of a suitable computing system environment 100 on which the invention may be implemented. The computing system environment 100 is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the invention. Neither should the computing environment 100 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary operating environment 100.

[0021] The invention is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known computing systems, environments, and/or configurations that may be suitable for use with the invention include, but are not limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.

[0022] The invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

[0023] With reference to FIG. 1, an exemplary system for implementing the invention includes a general purpose computing device in the form of a computer 110. Components of computer 110 may include, but are not limited to, a processing unit 120, a system memory 130, and a system bus 121 that couples various system components including the system memory to the processing unit 120. The system bus 121 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Associate (VESA) local bus, and Peripheral Component Interconnect (PCI) bus also known as Mezzanine bus.

[0024] Computer 110 typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by computer 110 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computer 110. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term modulated data signal means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer readable media.

[0025] The system memory 130 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 131 and random access memory (RAM) 132. A basic input/output system 133 (BIOS), containing the basic routines that help to transfer information between elements within computer 110, such as during start-up, is typically stored in ROM 131. RAM 132 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 120. By way of example, and not limitation, FIG. 1 illustrates operating system 134, application programs 135, other program modules 136, and program data 137.

[0026] The computer 110 may also include other removable/non-removable, volatile/nonvolatile computer storage media. By way of example only, FIG. 1 illustrates a hard disk drive 141 that reads from or writes to non-removable, nonvolatile magnetic media, a magnetic disk drive 151 that reads from or writes to a removable, nonvolatile magnetic disk 152, and an optical disk drive 155 that reads from or writes to a removable, nonvolatile optical disk 156 such as a CD ROM or other optical media. Other removable/non-removable, volatile/nonvolatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and the like. The hard disk drive 141 is typically connected to the system bus 121 through a non-removable memory interface such as interface 140, and magnetic disk drive 151 and optical disk drive 155 are typically connected to the system bus 121 by a removable memory interface, such as interface 150.

[0027] The drives and their associated computer storage media discussed above and illustrated in FIG. 1, provide storage of computer readable instructions, data structures, program modules and other data for the computer 110. In FIG. 1, for example, hard disk drive 141 is illustrated as storing operating system 144, application programs 145, other program modules 146, and program data 147. Note that these components can either be the same as or different from operating system 134, application programs 135, other program modules 136, and program data 137. Operating system 144, application programs 145, other program modules 146, and program data 147 are given different numbers hereto illustrate that, at a minimum, they are different copies. A user may enter commands and information into the computer 110 through input devices such as a keyboard 162 and pointing device 161, commonly referred to as a mouse, trackball or touch pad. Other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, or the like. These and other input devices are often connected to the processing unit 120 through a user input interface 160 that is coupled to the system bus, but may be connected by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB). A monitor 191 or other type of display device is also connected to the system bus 121 via an interface, such as a video interface 190. In addition to the monitor, computers may also include other peripheral output devices such as speakers 197 and printer 196, which may be connected through a output peripheral interface 195.

[0028] The computer 110 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 180. The remote computer 180 may be another personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the personal computer 110, although only a memory storage device 181 has been illustrated in FIG. 1. The logical connections depicted in FIG. 1 include a local area network (LAN) 171 and a wide area network (WAN) 173, but may also include other networks. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet.

[0029] When used in a LAN networking environment, the personal computer 110 is connected to the LAN 171 through a network interface or adapter 170. When used in a WAN networking environment, the computer 110 typically includes a modem 172 or other means for establishing communications over the WAN 173, such as the Internet. The modem 172, which may be internal or external, may be connected to the system bus 121 via the user input interface 160, or other appropriate mechanism. In a networked environment, program modules depicted relative to the personal computer 110, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation, FIG. 1 illustrates remote application programs 185 as residing on memory device 181. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.

[0030] In the description that follows, the invention will be described with reference to acts and symbolic representations of operations that are performed by one or more computer, unless indicated otherwise. As such, it will be understood that such acts and operations, which are at times referred to as being computer-executed, include the manipulation by the processing unit of the computer of electrical signals representing data in a structured form. This manipulation transforms the data or maintains it at locations in the memory system of the computer, which reconfigures or otherwise alters the operation of the computer in a manner well understood by those skilled in the art. The data structures where data is maintained are physical locations of the memory that have particular properties defined by the format of the data. However, while the invention is being described in the foregoing context, it is not meant to be limiting as those of skill in the art will appreciate that various of the acts and operation described hereinafter may also be implemented in hardware.

[0031] A truly server-less or peer-to-peer (P2P) name resolution protocol for which the present invention is particularly well suited is described in co-pending application Ser. No. 09/942,164, entitled PEER-TO-PEER NAME RESOLUTION PROTOCOL (PNRP) AND MULTILEVEL CACHE FOR USE THEREWITH, filed Aug. 29, 2001, and assigned to the assignee of the instant application, the teachings and disclosure of which are hereby incorporated in their entireties by reference thereto. This PNRP ensures convergence in large P2P networks through two mechanisms: a multilevel cache and a proactive cache initialization strategy. The multilevel cache allows the protocol to adapt to networks of various sizes, and grows only as the logarithm of the size of the network (not linearly as required by prior peer-to-peer protocols). The multilevel cache is built based on an underlying concept of a circular number space. Each level in the cache contains information from different levels of slivers of the circular space. The number of levels in the cache is dependent on the size of the network to which it is attached. However, since this size is not known, a mechanism is included to add a level to the multilevel cache when the node determines that the last level is full. In this way, rapid convergence is assured.

[0032] Generally, the PNRP cache is a collection of peer address certificates (PACs) representing knowledge about select participants in the PNRP cloud. From the PNRP cache perspective, each PAC in the cache pairs a PNRP ID with a PNRP network endpoint. PNRP IDs are 256-bit unsigned integers composed of two fields, a P2P ID and a service location. As discussed in the above identified application, the P2P ID is a cryptographic hash of a P2P name, and is the most significant 128 bits of a PNRP ID. The cryptographic hash ensures P2P IDs are randomly distributed in the 2¹²⁸ number space. The service location is the least significant 128 bits of a PNRP ID. Its most significant 64 bits are a 0-padded IPv4 address, or the aggregator from an IPv6 address. The remaining 64 bits are a hash of the IP address, port, protocol, and public key which will be used to create a PAC for this ID. The IP address in service location induces structure in this portion of a PNRP ID. However, the association of P2P IDs with service locations may be considered randomly distributed, making PNRP IDs as a whole randomly distributed.

[0033] Since PNRP IDs are unsigned integers, IDs may be arithmetically compared. Distance is measured by the arithmetic difference between two IDs modulo the PNRP ID number space, or 2²⁵⁶. DistID( ), the distance between two IDs, can be calculated in modulo arithmetic as the minimum arithmetic distance between the IDs modulo ID_MAX as follows:

[0034] For example, suppose there are three PNRP IDs, X 200, Y 202, and Z 204 as shown in FIG. 2. In this diagram, X 200 is closer to Z 204 than it is to Y 202. Mathematically, the DistID(X,Z)0.2*ID_MAX, while DistID(X,Y)0.3*ID_MAX.

[0035] PNRP relies on random distribution of PNRP IDs through the ID number space. Assuming such a random distribution with a number space that is 2²⁵⁶ wide, if there were, e.g., 2⁴ instantiated PNRP IDs, then on average the randomly instantiated IDs will be (2²⁵⁶/2⁴)2²⁵² away from the next closest instantiated ID. This is not a requirement for every PNRP ID, but rather it is an expectation for typical ID distribution under PNRP. Despite the fact that the PNRP number space has a width of 2²⁵⁶, to aid in the understanding of the cache management of the present invention, the remaining discussion will describe algorithms and functionality in a simplified PNRP ID space. This simplified space will consist of IDs between 0 and 9999 (instead of 0 to 2²⁵⁶), i.e. the group of integers modulo 10000. This will make the mathematics easier to understand. The first two digits correspond to P2P ID, and the last two digits correspond to service location.

[0036] PNRP is an application-level routing protocol. The protocol is designed to route a resolution request (RESOLVE) to the owner of the target ID being resolved. In the ideal case, each PNPR host which touches a RESOLVE request is able to reduce the distance to the target ID by an order of magnitude. To achieve this order of magnitude reduction it is recognized that the magnitude of reduction provided is log_{(base K/2)}log_{((base of entries in cache)/2)}. Therefore, to achieve an order of magnitude reduction at each level, the cache is populated with K20 entries. In general, however, the system of the present invention may provide a user selectable magnitude of reduction by selecting K in accordance with the logarithmic function just discussed, and having each cache region cover 1/(K/2) of the number space that the previous region covered allowing K entries to be stored therein.

[0037] For example, suppose that the owner of PNRP ID J1234 wants to resolve PNRP ID Z5432. J is 4198 away from Z, as measured by DistID(J,Z)MIN(1234-5432 mod 10000, 5432-1234 mod 10000)MIN(5802, 4198)4198. J probably will not know exactly where to find Z, but should be able to forward the RESOLVE to a peer K stored in its cache that has a PNRP ID no further than (4198/10)420 away from Z. K in turn forwards the RESOLVE to L, which is no further than (420/10)42 away from Z. L forwards the RESOLVE to M, no further than (42/10)4 away from Z. M then passes the RESOLVE directly to Z, who answers the RESOLVE with a RESPONSE. In this example, the number of hops is log(ID_MAX). In one embodiment, the number of hops is actually log( of instantiated IDs). If there are only 100 active PNRP IDs in this simplified number space, resolution should take log(100)2 hops, not log(10,000)4 hops. This is counter-intuitive, but the example below should help explain.

[0038] Suppose there are only 100 active PNRP IDs instantiated in the P2P cloud. With an expectation that these IDs are evenly distributed in the PNRP ID space, it may be assumed that there exists an average spacing of 100 between each peer as calculated by the equation: $\begin{array}{cc}\mathrm{AverageSpacing}=\frac{\left(\mathrm{ID\_MAX}-\mathrm{ID\_MIN}+1\right)}{\mathrm{NumActiveIDs}}=\frac{10000}{100}=100& \left(2\right)\end{array}$

[0039] In the above example, there probably are not IDs L and M because statistically IDs within 42 or 4 of Z are unlikely to exist. Instead, K would know Z's address because Z's relative proximity to K. Another way to think about this is if there are 100 IDs in N₁₀₀₀₀ which are 100 apart each, these can be mapped to 100 IDs in N₁₀₀₀ 10 apart, or even to 100 IDs in N₁₀₀ 1 apart. Since resolution in N₁₀₀ should take two hops (log(100)2), as the example in N₁₀₀₀₀ with 100 instantiated IDs did above.

[0040] In order to have these properties, the cache needs to be carefully structured. An ideal cache has a distribution of knowledge such that the probability of knowing a given PNRP ID is inversely proportional to the distance of that ID from the locally registered PNRP ID. The instantiation of the cache of the present invention applies a stepping function to this requirement. Suppose there are 1000 instantiated PNRP IDs, each an average of DIST_AVG10 apart. A cache should have PACs for the ten closest instantiated IDs in each direction (i.e. less than, greater than) from itself in the region

[0041] This implies an average distance between PACs in this level of DIST_AVG*10⁰. This first region is populated with 20 PACs. The next level of granularity is the region

[0042] but excluding the first region. The exclusion sums to {fraction (1/10)}^th of the range of this second region. Since 10% of the area of this cache region is excluded, the cache should have PACs for 90% of 2018 IDs in this region. This implies an average distance between those PACs of (DIST_AVG*10¹). This process continues until the least granular region N is reached. This last level N has PACs in the region

[0043] but excluding the first through the N1^st regions. These exclusions sum to {fraction (1/10)}^th of the range of this N^th region. As such, the N^th region cache should have PACs for 90% of 2018 IDs in this range, which implies an average distance between those PACs of (DIST_AVG*10^N1). In the complete PNRP number space of 2²⁵⁶, the number of cache regions would be log(2²⁵⁶), which is approximately equal to 78 cache regions. However, the existence of a statistical phenomenon known as the Birthday Paradox applied to the PNRP number space reduces the actual number of active PNRP names that will likely exist to approximately the square root of 2²⁵⁶, or only about 2¹²⁸ active names. As a result, the number of cache regions would actually be based on the log(2¹²⁸), which is approximately equal to 38 cache regions.

[0044] This cache structure enables efficient RESOLVE routing based on PNRP ID proximity. For each locally registered PNRP ID, a peer should be able to forward the RESOLVE request an order of magnitude closer to its destination. The rules above are almost enough to ensure this, However, in order for resolution to converge, each region of cache should have its entries approximately evenly spaced. The even spacing allows statistical guarantees to be made about maximum distance an ID to resolve is from a cache entry. Specifically, the regions are structured such that any ID randomly picked in any region will be no more than {fraction (1/10)}^th the region size from a cached PAC entry in that region. In the ideal case having 20 evenly spaced cached PAC entries, any ID randomly picked in the region will be no more than {fraction (1/40)}^th the region size from a cached PAC entry in that region. That is, because there are 20 evenly spaced entries in a region (one every {fraction (1/20)}^th of the region) and because any randomly picked ID cannot be further away from an entry by the distance between two consecutive entries, no randomly picked ID can be more than {fraction (1/20)}*{fraction (1/40)}the region size from a cached PAC entry in that region. Cache regions are organized according to the present invention into segments to meet the cache content requirements, and to efficiently handle the case where multiple local PNRP IDs are registered and supported by a single cache as will be discussed more fully below.

[0045] A peer preferably includes a supporting cache for each one of its published PNRP IDs. The cache contains PACs with strategically selected PNRP IDs. The cache is composed of a list of segments which are non-overlapping intervals that cover the entire ID space. Each segment contains a list of PACs sorted by PNRP ID. As shown in FIG. 3, a sample PNRP ID number space 206 from 0 to 9999 is divided into cache segments A through E. The union of segments displayed on the same horizontal plane correspond to a single region as defined above. Segment C 208 is corresponds to region 1, segment B 210segment D 212 corresponds to region 2, and segment A 214segment E 216 corresponds to region N3. Consequently, the proportions for segments in this Figure are (AE)10*(BD)111*C. C 208 has a different size because while region 3 and region 2 each have 10% of their breadth taken by exclusions for lower regions, region C 208 has no such exclusions. As such, it is has a relative size of (100/90)1.11 compared to the other regions.

[0046] Each segment in the cache of the present invention is characterized by two attributes, range and ideal spacing (or just spacing). Range is the starting and ending point of a segment. Spacing is the ideal PNRP ID distance between adjacent PACs in the segment. Range and ideal spacing are used to calculate the number of PACs allowed in a segment. The formula is $\begin{array}{cc}\mathrm{PACCapacity}=\frac{\mathrm{SegmentEnd}-\mathrm{SegmentStart}+1}{\mathrm{IdealSpacing}}& \left(6\right)\end{array}$

[0047] For example, a segment for IDs 5000 to 5449 with an ideal spacing of 50 could contain up to (544950001)/509 PACs.

[0048] In one embodiment of the present invention, the cache is preferably structured according to the number of instantiated IDs, not the total number space. The number of instantiated IDs determines how many regions are needed, and therefore how many segments are needed. For a cache servicing a single locally registered ID, the number of regions will be log( instantiated IDs), and the number of segments will be 2*(regions)1. As an example, assume that there are 2,000 instantiated PNRP IDs in the simplified number space of 0-9999. Then the number of regions are log(2000)3 regions, and the number of segments is 2*(3)15.

[0049] If a local ID 218 is registered at ID5459, the first region (segment C 208 in FIG. 3) spans IDs /DIST_AVE*10¹ in accordance with equation (3), with an average spacing within the cache region of DIST_AVE*10⁰. With a number space of 0-9999 having 2000 instantiated IDs, the DIST_AVE10,000/20005. Therefore, segment C 208 will span IDs 5495505445 to 5495505545, with an average spacing between IDs of 5, and a PAC capacity of 20 IDs within this region according to equation (6) above.

[0050] The next region, in accordance with equation (4) above, will range from 54955004995 to 54955005995, less segment C 208, with an average spacing between IDs of 50. Therefore, segment B 210 will range from 4995 to 5444, with a PAC capacity of 9 in accordance with equation (6) above, and segment D 212 will range from 5546 to 5995, with a PAC capacity of 9.

[0051] The third and final region, in accordance with equation (5) above, will range from 54955000495 to (54955000) mod 10,000495 less segments B 210, C 208, and D 212. However, since this region wraps around the circular number space (mod 10,000), segment A 214 may be thought of as covering the range from 0 to 4994 while segment E 216 may be thought of as covering the range from 5996 to 9999. As a result, segment A 214 will have a PAC capacity of 10, while segment E 216 will only have a PAC capacity of 8, each with an average spacing of 500.

[0052] Unfortunately, in a large scale P2P network cloud there is no way to know the number of instantiated ID. This is especially true when the dynamic nature of the peers in such a cloud is considered. Instead, the system of the present invention relies on the random distribution of PNRP IDs and a segment splitting policy to structure the cache appropriately. In accordance with this policy, an initial cache segment A 218, is illustrated in FIG. 4A created that covers the entire number space and can hold up to 20 PACs. If less than 20 IDs are active in the cloud, segment A 218 will never overflow, and the cache will stay at one region 222. FIG. 4A shows segment A 218 with 20 PACs evenly distributed throughout the number space, and a vertical dashed line 220 corresponding to the locally registered PNRP ID of 5495.

[0053] When a new PAC with a PNRP ID not already in the cache is received, PNRP knows that the information space requires service instances to specialize with at least two regions. There may ultimately be a need for more regions, but the cache management method of the present invention will wait for that need to be proven before acting upon it. This method utilizes three data structures organized into red-black trees (symmetric binary B-trees) to simplify this procedure. These data structures include a PAC cache tree, an instantiated segment tree, and an uninstantiated segment tree. The PAC cache tree is simply a collection of the PACs, sorted by the PNRP ID and arranged into a red-black tree. This allows easy insertion, location, and deletion of PACs within O(lg₂N). Since the PAC cache tree is sorted by PNRP ID, PACs corresponding to each segment are continuous as encountered while traversing the tree.

[0054] An instantiated segment structure is also maintained for each logical cache segment that has been instantiated. This segment list includes the segment minimum and maximum bounds (range), the expected spacing between entries, and a red-black instantiated segment tree. This instantiated segment tree contains pointers to nodes in the PAC cache tree corresponding to the PNRP ID of the PAC that is the start of that segment. This instantiated segment tree is used to determine in which segment a newly acquired PAC belongs.

[0055] An uninstantiated segment tree is also maintained by the system of the present invention. This uninstantiated segment tree contains a listing of the next segment(s) in the next region that could be instantiated if the next higher region is filled and another PAC is received for that region. That is, the system of the present invention checks the uninstantiated segment tree if a PAC is received which would belong in a full segment to see if there exists an uninstantiated segment(s) in the next lower region that overlaps the full segment. If so, then the system of the present invention instantiates the new segment(s) as will be described more fully below, moves the new segment from the uninstantiated segment tree to the instantiated segment tree, and calculates the next uninstantiated segment(s) for the uninstantiated segment tree.

[0056] Returning to the present example, the system of the present invention creates a second region 224 when region 1 222 is full and another PAC is received. This is determined by looking to the uninstantiated segment tree and noting that segment B 226 overlaps the now full segment A 218. As such, the number space previously taken by segment A 218 in FIG. 4A is subdivided into segments A 228, B 226, and C 230 illustrated in FIG. 4B. Segments A 228 and C 230 make up region 1 222, and segment B 226 makes up region 2 224. The endpoints for segment B 226 are determined by taking 10% of the number space of region 1 222, centered at the locally registered ID 220. In this exemplary number space, segment B 226 would span 5495/1000 or from 4495 to 6495, segment A 228 would span 0 to 4494, and segment C 230 would span 6496 to 9999. Segments are non-overlapping, and all PACs are migrated into the segment which covers their portion of the PNRP ID space 206.

[0057] Now assume that segments A 228 and C 230 are each full. If a new PAC is added in the range of either of those segments, it will have to replace a PAC already cached in that segment in accordance with the PNRP management described in the application discussed above because the uninstantiated segment tree will indicate that there are no uninstantiated segments that overlap these segments. Segment B 226 is in region 2 224, which means that its spacing is {fraction (1/10)}^th that of segments A 228 and C 230. Consequently, region B 224 has a capacity of 20 PACs. Since it is only holding 2 PACs after its instantiation, it can accept another 18 before becoming full. If PNRP IDs are well distributed, there would need to be at least 200 instantiated IDs for segment B 226 to fill up because B 226 covers {fraction (1/10)}^th of the full ID number space 206, and {fraction (1/10)}^th of all registered PNRP IDs can potentially fit in its range.

[0058] If the second region 224 becomes full and another PAC is received that fits in that second region 224, then the system of the present invention will instantiate another cache region 232 listed in the uninstantiated segment tree as 10% of region 2 224, centered at the local ID 220 as illustrated in FIG. 4C. The middle {fraction (1/10)}^th of the region 224 will be split into a new region 232, and the remaining pieces of region 2 224 will form segments B 224 and E 236. This split is analogous to the split which created segments 226, 230 from the original region A 218. Specifically, segment D 238 will span 5495/100 or from 5395 to 5595, segment B 234 will now span 4495 to 5394, and segment E 236 will span 5596 to 6495.

[0059] Now assume that segments B 234 and E 236 are each full. If a new PAC is added in the range of either of those segments, it will have to replace a PAC already cached in that segment in accordance with the PNRP management described in the application discussed above because there are no uninstantiated segments that overlap these segments. Segment D 238 is in region 3 232, which means that its spacing is {fraction (1/10)}^th that of segments B 234 and E 236. Consequently, segment D 238 has a capacity of 20 PACs. Since it is only holding 2 PACs after its instantiation, it can accept another 18 before becoming full. If PNRP IDs are well distributed, there would need to be at least 2000 instantiated IDs for segment D 238 to fill up because it covers {fraction (1/100)}^th of the full ID number space 206, and {fraction (1/100)}^th of all registered PNRP IDs can potentially fit in its range.

[0060] If region 232 becomes full and another PAC is received, then the system of the present invention will check the uninstantiated segment tree to determine if an additional region should be instantiated. Recognizing that such a segment exists, the system will instantiate another cache region 240 as 10% of region 232, centered at the local ID 220 as illustrated in FIG. 4D. The middle {fraction (1/10)}^th of the region 232 will be split into a new region 240, and the remaining pieces of region 232 will form segments D 242 and G 244. This split is analogous to the split which created segments 238, 236 from region 224. Specifically, segment F 246 will span 5495/10 or from 5485 to 5505, segment D 242 will now span 5395 to 54844, and segment G 244 will span 5506 to 5595.

[0061] Now assume that segments D 242 and G 244 are each full. If a new PAC is added in the range of either of those segments, it will have to replace a PAC already cached in that segment in accordance with the PNRP management described in the application discussed above because there are no uninstantiated segments that overlap these segments. Segment F 246 is in region 4 240, which means that its spacing is {fraction (1/10)}^th that of segments D 242 and G 244. Consequently, region 240 has a capacity of 20 PACs. Since it is only holding 2 PACs after its instantiation, it can accept another 18 before becoming full. If PNRP IDs are well distributed, there would need to be at least 20000 instantiated IDs for segment F 246 to fill up because F 246 covers {fraction (1/1000)}^th of the full ID number space 206, and {fraction (1/1000)}^th of all registered PNRP IDs can potentially fit in its range. However, since this exemplary number space 206 is only 10000 wide and can therefore only have 10000 registered PNRP IDs, it is statistically unlikely in a well distributed cloud that this region will become full.

[0062] The preceding discussions focused on how to structure a cache around a single locally registered PNRP ID to provide an understanding of the concepts of the present invention. In practice each cache will typically need to support anywhere from 10 to 100 or more locally registered PNRP IDs. This is because each resource that the peer allows to be discoverable in the cloud will be registered with a PNRP ID. Indeed, each service provided by the peer will also be registered with a PNRP ID. The rules for creating segments discussed above with regard to a single PNRP ID change only slightly when multiple IDs are supported.

[0063] To illustrate this multi-ID cache management embodiment of the present invention, suppose there are three locally registered IDs, e.g. X 248, Y 250, and Z 252, supported by the same cache as illustrated in FIG. 5A. As with the previous embodiment, the cache starts out with a single segment, which in this example is now full. A 254 is the only instantiated segment in the cache at this point, and the uninstantiated segment tree lists segments H 256, I 258, and J 260 (the next uninstantiated segments). The range of the number space potentially covered by segments H-J 256-260 in region two 262 and P-R 264-268 in region three 270 are shown in FIG. 5A with dashed lines. Segments H 256, 1 258, J 260, P 264, Q 266, and R 268 do not exist yet, they simply illustrate where additional segments could eventually be added if necessary. However, as discussed above, only segments H 256, I 258, and J 260 are included in the uninstantiated segment tree. Segments P, Q, and R will be added to the uninstantiated segment tree once segments H, I, and J are instantiated.

[0064] As may be seen from this particular example, the local ID X 248 is so close to ID_MIN that its region two segment, if instantiated, wraps around ID_MAX slightly as a result of the modular arithmetic (i.e. ID_LOCAL/{fraction (1/10)}*ID_MAXID_MIN1 mod ID_MAX). To simplify the search in the segment tree discussed above, this second region is split into two segments H 256 and J 260. In this way, each of the segments H 296 and J 260 have starting addresses that are lower than their ending addresses, enabling simple linear arithmetic to be used in the search process instead of requiring modular arithmetic (mod ID_MAX).

[0065] As may also be seen from this example, the local PNRP IDs Y 250 and Z 252 are within {fraction (1/10)}^th of the number space of each other. As such, their potential region two 262 segments would overlap if both were instantiated. If both such segments were instantiated and separately maintained, a duplication of information would result, thus wasting resources. Since these segments will have the same granularity, the system of this embodiment of the present invention combines them into a single segment I 258. This segment I has the starting address of the start of the segment that would have been instantiated based on the local PNRP ID Y 250 (i.e. ID_LOCAL_Y{fraction (1/10)}*ID_MAXID_MIN1 mod ID_MAX), and has the ending address of the end of the segment that would have been instantiated based on the local PNRP ID Z 252 (i.e. ID_LOCAL_Z{fraction (1/10)}*ID_MAXID_MIN1 mod ID_MAX). Unlike the typical splitting of segments in a cache under the system of the present invention, the single top segment in this embodiment was split into three region-2 segments covering a little less than 30% of the number space. They cover less than 30% because Y and Z's region two segments overlap somewhat to form segment I 258.

[0066] Each segment is characterized by its range and its spacing (determined by its region). Each time a PAC is added to a segment, the segment is examined first. If the segment has room for the PAC, it is simply added. If the segment is full, then PNRP checks to see whether there are any potential (uninstantiated) segments in the next region which overlap this segment's range. In the current example, adding a PAC to segment A 254, which is full, would result in instantiation of segments H 256, I 258, and J 260, as illustrated in FIG. 5B, because each overlaps with the original segment A 254 and are in the next region 262. All instantiated segments obey the rules for segments. First, the union of all segments covers the entire number space. Second, none of the segments overlap. Segments A 272 and B 274 are full, and no potential segments overlap with them. If a PAC is to be added in either of these segments A 272 and B 274, then an entry must be deleted before the new entry can be added.

[0067] Segments H 256, I 258, and J 260 can each hold several more entries before any further splits need to be made. If segment J 260 fills up, an entry will have to be deleted in it before a new one can be added because there are no uninstantiated regions that overlap with segment J 260 in the next region. If segment H 256 fills up, however, attempting to add an additional entry to H 256 will result in instantiation of segment P 264 because uninstantiated segment P 264 overlaps segment H 256 and is in the next region. If segment I 258 fills up, attempting to add an additional entry will result in both segments Q 266 and R 268 being instantiated. This is because both of these two segments overlap with I 258 and are in the next region 270.

[0068] To illustrate this point graphically, reference is directed briefly to FIG. 6, which depicts the relationship between the PAC cache tree 290, the instantiated segment tree 292, and the uninstantiated segment tree 294. As discussed above, the PAC cache tree 290 is a red-black tree that is sorted by the PNRP IDs of the PACs stored in the cache. As illustrated, each PAC currently stored in the cache is represented in the PAC cache tree 290. The instantiated segment tree 292 is also arranged into a red-black tree, but instead of containing PACs it contains pointers to the starting address of each segment instantiated in the cache. Organization into a red-black tree aids in the searching of the proper segment into which a newly received PAC should be placed. The uninstantiated segment tree 294 is also arranged into a red-black tree to simply searching, and contains only the segments in the next region of the cache that could be instantiated if the regions that they overlap become full.

[0069] Assuming that segment I 258 is full and another PAC is received having a PNRP ID that would place it in segment I 258, segments Q 266 and R 268 will be instantiated as illustrated in FIG. 5C. As a result of instantiating segments Q 266 and R 268, segment I is broken into segments I 276, K 278, and L 280.

[0070] If segments Q 266 and R 268 have been instantiated as illustrated in FIG. 5C and the resource or service registered at PNRP ID Y 250 is removed, the system of the present invention will relinquish the cache segment Q 266 to the higher level I 276. Unfortunately, since this cache segment I 276 was formed as a combination of the two overlapping segments that would have been instantiated for the close IDs Y 250 and Z 252, a portion of segment I 276 will also be relinquished to the higher segment A 284. As illustrated in FIG. 5D the remaining segments K 282 and L 280 will now only cover the range ID_LOCAL_Z/(ID_MAXID_MIN1) mod ID_MAX. Because the higher segments are already full, the PACs in the portions of the lower regions that are being relinquished are treated as new information to the higher segments As such, their addition to that higher segment will be dependent on the PNRP rules governing addition of information to a full segment discussed above.

[0071] If a new resource or service is registered once multiple cache segments have been instantiated, the uninstantiated segment tree is populated with the next segment that would need to be instantiated if a new PAC is received in a full region below which this new uninstantiated region overlaps. When a new PAC is received, the uninstantiated segment tree is checked as described above, and the lower level region will be instantiated if appropriate. If any overlaps with other instantiated segments exists, the segments will be combined as discussed above.

[0072] All of the references cited herein, including patents, patent applications, and publications, are hereby incorporated in their entireties by reference.

[0073] In view of the many possible embodiments to which the principles of this invention may be applied, it should be recognized that the embodiment described herein with respect to the drawing figures is meant to be illustrative only and should not be taken as limiting the scope of invention. For example, those of skill in the art will recognize that the elements of the illustrated embodiment shown in software may be implemented in hardware and vice versa or that the illustrated embodiment can be modified in arrangement and detail without departing from the spirit of the invention. Therefore, the invention as described herein contemplates all such embodiments as may come within the scope of the following claims and equivalents thereof.