Client controlled pre-fetching of resources

[0002] The present invention relates generally to computer systems and more particularly to client controlled pre-fetching of resources in computer systems.

[0003] In conventional systems, users must often wait a significant period of time between when the users request web pages from web servers and when the users receive the web pages. As a result, some conventional systems have begun to employ caches. Each cache holds selected web pages and is able to return the selected web pages to the users more quickly than the web servers that hold the web pages. When a user wishes to obtain a copy of a selected web page, a cache is checked to determine whether a web page with a matching uniform resource locator (URL) is resident within the cache. If no matching web page is resident within the cache, a request is sent to a remote web server to obtain a copy of the web page.

[0004] The resulting increase in performance due to the use of caching is largely dependent upon what web pages are cached. Numerous conventional approaches have been devised for “pre-fetching” web pages in anticipation that the user will next request the web pages that are pre-fetched. For example, in some conventional systems, when a web page is retrieved from a web server, all of the web pages referenced by hyperlinks within the retrieved web page are pre-fetched for storage within the cache. Other conventional systems pre-fetch web pages resident on the same web site as the web page that was most recently retrieved.

[0005] Unfortunately, the pre-fetching strategies of conventional systems have not predicted user activity well. In other words, with such strategies, web pages have been pre-fetched that are not of interest to the user. This results in unnecessary overhead, the waste of precious cache space and unnecessary traffic and congestion. This problem is especially magnified when a significant number of web pages or very large web pages are pre-fetched. An additional problem with conventional systems is that web pages may be pre-fetched that are already resident within the cache. Still further, web pages of small size may be pre-fetched; hence, the caching results in only a minimum overhead savings or even an overhead loss.

[0006] The present invention addresses the above-described limitations of conventional systems. In particular, the present invention provides client controlled pre-fetching of identifying function values for resources and/or resources for a cache. The resources may be, for example, web pages, audio content, video content, data files or even computer programs. The client may control pre-fetching based upon a number of different criteria. For example, an identifying property value may be examined for a desired resource to determine whether a copy of the resource is already resident within the cache. The identifying property value is intrinsically tied to the contents of the resource so that resources with different names, addresses or URLs may be compared to determine if they have the same content. Information regarding the size of a resource may be obtained prior to pre-fetching the resource to determine whether the resource is of sufficient size to result in a significant enough savings to warrant pre-fetching. Other criteria, such as statistical information, historical information, state of the network and the like, may be used as part of the decision to determine whether to pre-fetch a resource or not.

[0007] In accordance with one aspect of the present invention, a selected identifying property value for a selected resource is obtained in an electronic device, such as a computer system, that has a cache for caching resources. The selected identifying property value is compared with identifying property values for the resources in the cache to determine whether the selected resource is resident within the cache. Where the selected resource is not resident within the cache, the selected resource is pre-fetched to obtain and store a copy of the selected resource in the cache. In the case where the selected resource is determined to be resident in the cache, the pre-fetching of the selected resource may be prevented. The resource may be a web page, for example. The identifying property value may be a hash function value that is calculated by applying a hash function to the respective resources. This method may be performed by a proxy or a web browser.

[0008] In accordance with another aspect of the present invention, information regarding a selected resource is requested by a device that has a cache for caching resources. An identifying property value for the selected resource and information regarding the size of the selected resource are received in response to the request. Where the size of the selected resource fails to exceed a threshold, the selected resource is not pre-fetched. Where the selected resource exceeds the threshold, the identifying property value may be used to determine whether the selected resource is already resident within the cache. If the selected resource is already resident in the cache, pre-fetching of the selected resource is prevented. In contrast, where it is determined that the selected resource is not already resident in the cache, the pre-fetching of the selected resource may be initiated.

[0009] In accordance with an additional aspect of the present invention, a method is practiced in a device that has a cache for caching resources. In accordance with this method, a selected value is obtained for a selected resource. The selected value is intrinsically tied to the contents of the resource so as to identify the resource. Based on the selected value, it is determined whether the selected resource is already resident in the cache. If it is determined that the selected resource is not already resident in the cache, a check is made whether an additional criteria is met. If the additional criteria is met and it is determined that the selected resource is not already resident in the cache, the selected resource is pre-fetched. The additional criteria may be that the resource is of a size that is large enough to warrant pre-fetching.

[0010] An illustrative embodiment, consistent with the principles of the present invention, will be described in more detail below relative to the following drawings.

[0011] FIG. 1 is a block diagram that illustrates the communications between a first device and a second device to pass information regarding a resource prior to pre-fetching the resource.

[0012] FIG. 2 is a block diagram illustrating the communications between a first device and second device in which information regarding a resource is obtained and the resource is pre fetched.

[0013] FIG. 3 is a flow chart illustrating the steps that are performed by a device in determining whether to pre-fetch a resource based on whether the resource is already cached and whether the resource is of sufficient size to warrant pre-fetching.

[0014] FIG. 4 is a flow chart illustrating the steps that are performed by a device in which multiple criteria are applied to determine whether to pre-fetch a resource.

[0015] FIG. 5 is a flow chart illustrating the steps that are performed by a device holding a copy of a resource to provide a requester with information regarding the resource to guide pre-fetching decisions.

[0016] FIG. 6 is a flow chart illustrating the steps that are performed by a device holding a copy of a resource when the resource is to be pre-fetched.

[0017] FIG. 7 is a block diagram illustrating communications between a proxy and a server when performing pre-fetching in accordance with the illustrative embodiment.

[0018] The illustrative embodiment, which is consistent with the principles of the present invention, provides a mechanism wherein a device may control whether and what resources are pre-fetched for storage within a cache. The device may obtain information from the source of the resource (e.g. a web server) to help provide guidance as to whether to pre-fetch the resource. For example, information regarding the contents of the resource may be obtained so that the requesting device may determine whether a copy of the resource is already cached prior to pre-fetching the resource. Similarly, size information regarding the size of the resource may be obtained to determine whether the resource is of sufficient size as to warrant pre-fetching. The requesting device may also examine other information, such as statistical information, historical information, the current state of the relevant network and other information, to determine whether pre-fetching of the resource is warranted.

[0019] The resources that are to be pre-fetched may take many different forms. For instance, the resources may include web pages, markup language document pages such as hypertext mark-up language (HTML) pages, or extensible mark-up language (XML) pages, or other types of pages. In addition, the resources may be data files, audio content, video content or even executable content, such as computer programs or scripts.

[0020] The illustrative embodiment employs identifying property values. Each resource has an associated identifying property value that is intrinsically tied to the content of the resource. If the content of the resource changes, the identifying property value is very likely to change. One suitable identifying property value for practicing the illustrative embodiment is a hash function value that is calculated by applying a hash function to at least the portion of the contents of the resource. A suitable hash function is the MD5 hash function developed by Ron Rivest. Those skilled in the art will appreciate that other hash functions may be utilized in practicing the present invention. For example, the MD2, MD4, Secure Hash Algorithm (SHA), SNERFU, HAVAL or RIPE-MD hash functions may be utilized in practicing the present invention. The hash function does not need to be a “strong” hash function. Moreover, identifying property values are not limited to hash functions but may be checksums or other functions that are intrinsically tied to the contents of the resources.

[0021] For purposes of the discussion below, it is helpful to define a few terms. A “document” is a self-contained referenceable grouping of information. The term “document” is intended to encompass but is not limited to word processing documents, web pages, images, audio files, and other varieties of textual files.

[0022] A “web page” is a page of a document or a file that may be displayed by a web browser as a single page. Examples of a “web page” include a hypertext markup language (HTML) file, an extensible markup language file (XML) file or other encoding format suitable for encoding a web page. A web page is a page containing content in a format that is renderable by a web browser.

[0023] “Content” refers to any information or data that may be incorporated into a destination by a data sink.

[0024] A “resource” refers to a web page, program, data file or other entity that may be used or consumed by a client.

[0025] “Pre-fetching” is the retrieving of a resource for caching prior to an explicit request by a client to obtain the resource.

[0026] FIG. 1 depicts an environment suitable for practicing the illustrative embodiment. In this environment, device 10 communicates with device 12. Device includes a web browser 14 for enabling a user of the device 10 to browse the Internet, an intranet, an extranet or the like. The web browser 14 may be, for example, the Netscape Navigator™ web browser or the Netscape Communicator™ web browser. Those skilled in the art will appreciate that other web browsers 14 may be used to practice the present invention. More generally, the present invention may be practiced with any program (not just a web browser) that is able to view or utilize resources that are to be obtained from a second device and cached.

[0027] The web browser 14 includes a cache 16 for caching resources 18. The web browser 14 includes an add-on 20 that provides the added functionality necessary for practicing the illustrative embodiment. Threshold information 22 may also be stored on the device 10. The add-on 20, the web browser 14 and the threshold information 22 are stored in a storage medium. The storage may include both primary memory and secondary memory.

[0028] Device 12 holds a copy of a resource 24 and may also hold additional resources. An identifying property value 26 for the resource 24 is stored on the device 12. Alternatively, the identifying property value may be calculated on the fly by the device 12 when needed. Size information 28 is also stored on the device 12 for the resource 24. Those skilled in the art will appreciate that the size information 28 may be directly integrated into the resource 24 in some instances rather than being stored as a separate piece of data.

[0029] Devices 10 and 12 may take many different forms. For example, the devices 10 and 12 may be computer systems, such as personal computer systems, workstations, network computers, laptop computers, personal digital assistants or other varieties of computer systems. More generally, devices 10 and 12 may be any electronic devices that are capable of communicating to exchange resources that are to be cached. As such, devices 10 and 12 may be intelligent appliances (i.e. electronic devices suitable for household use that have intelligence for communicating with other devices), printers, facsimile machines, settop boxes for televisions or even two-way pagers. Those skilled in the art will appreciate that the listed varieties of electronic devices is intended to be merely illustrative and not exhaustive.

[0030] The discussion will now focus on activities performed by device 10 in practicing the present invention. Before device 10 pre-fetches resource 24 from device 12, device submits a request 30 for information regarding the resource (step 40 in FIG. 3). An identifying property value 26 for the resource 24 and size information 28 regarding the resource are received by the device 10 after the request has been fulfilled (step 42 in FIG. 3). As mentioned above, the identifying property value 26 for the resource 24 may be, for example, a hash function value that is calculated by passing the contents of the resource through a hash function. Thus, the identifying property value may be “pre-fetched.” The identifying property value 26 is used to determine whether a copy of the resource 24 is already resident within the cache 16 on device 10. Specifically, device 10 compares the identifying property value 26 for the resource 24 with identifying property values for the resources 18 resident within the cache to see if there are any matches (step 44 in FIG. 3). The identifying property values for the resources 18 may be stored on device 10 or may be calculated on the fly. The resources 18 may be indexed by the identifying property values so as to make a quick determination of whether there is a match or not. A match implies that a copy of the resource is already resident within the cache. If such a match is found, the resource is not pre-fetched (step 46 in FIG. 3). In contrast, if there is not matching identifying property value, it is indication that a copy of the resource is not already resident within the cache 16.

[0031] In the case where a copy of the resource is not already resident in the cache, device 10 may make an additional check to determine whether it is desirable to pre-fetch the resource or not. In particular, device 10 may determine whether the resource 24 is of sufficient size to warrant pre-fetching. The size information 28 that is received by device 10 is compared to the threshold 22 to determine whether the size of the resource exceeds the threshold (step 48 in FIG. 3). If the size of the resource is less than or equal to the threshold, there is no need to pre-fetch the resource because the pre-fetching would result in only minimal latency savings and would cost computational and memory overhead. Those skilled in the art will appreciate that the comparison to the threshold need not look purely to see whether the size of the resource exceeds the threshold. There may be instances wherein if the size of the resource is equal to the threshold, the resource is still pre-fetched. Moreover, the size of the resource and the threshold need not be calculated on a positive scale where a greater than determination is made. Instead a negative scale may be used where a less than comparison results in pre-fetching of the resource. More generally, the determination is based upon the magnitude of the resource and whether, based on the magnitude of the resource, it is desirable to pre-fetch the resource or not. Other criteria may also be taken into account in deciding whether to pre-fetch the resource. For example, available network bandwidth, cache space available and other factors may be considered in deciding whether to pre-fetch the resource.

[0032] Where it is determined that it is desirable to pre-fetch the resource, device 10 submits a resource request as shown in FIG. 2 (step 50 in FIG. 3). Device 10 receives the resource 24 and stores the resource within the cache 16 (step 52 in FIG. 3).

[0033] Those skilled in the art will appreciate that additional criteria may also be applied in determining whether to pre-fetch the resource 24. FIG. 4 shows a flow chart of such a case. The steps 40, 42, 44, 46, 48, 50 and 52 are identical to those described above relative to FIG. 3. The flow chart of FIG. 4 differs in that when it is determined that the size of the resource exceeds the threshold in step 48, a check is made whether other criteria are met (step 60 in FIG. 4). As mentioned above, additional criteria may be employed in determining whether to pre-fetch the resource. The other criteria may relate to the state of the network that facilitates communication between device 10 and 12, whether the resource has been successfully pre-fetched in the past and whether other certain statistical criteria are met. Those skilled in the art will appreciate that a number of different types of criteria may be employed. If such other criteria are not met, the resource is not pre-fetched (step 46 in FIG. 4). Otherwise the resource is pre-fetched by performing steps 50 and 52, which have been described above relative to FIG. 3.

[0034] The above discussion has focused on activities performed by device 10. The discussion below focuses on activities performed by device 12.

[0035] FIG. 5 is a flow chart that illustrates the steps that are performed by device 12 in providing information to device 10 to guide the decision whether to pre-fetch a resource. Initially the request 30 for information is received by device 12 from device (step 62 in FIG. 5). The device 12 retrieves the identifying property value 26 and the size information 28 (step 64 in FIG. 5). As was described above, the identifying property value 26 and the size information 28 are returned by device 12 to device 10.

[0036] Subsequently, device 10 may submit a resource request 32 that is received as a pre-fetch request by device 12 (step 70 in FIG. 6). In response, the device 12 simply retrieves the resource 24 (step 72 in FIG. 6) and returns a resource to the device 10 (step 74 in FIG. 6).

[0037] The above discussion has assumed that the device 10 directly communicates with the device 12. In many instances, a proxy may actually perform the interactions that have been described above relative to the flow charts in FIGS. 3 and 4. FIG. 7 depicts an instance wherein a proxy 80 communicates with a server 82 that holds a copy of a resource 84. The server 82 also holds an identifying property value 92 for the resource, and holds information 94 regarding the resource size. The proxy 80 holds a cache 86. The cache is resources 88. Threshold information 89 may be stored on the proxy 80. The protocol described above relative to FIGS. 3 and 4 may be applied in the case where a proxy is used. Specifically, information request 90 is submitted to the server 82 which returns the identifying property value 92 and the resource size information 94. If the proxy 80 determines that it is desirable to pre-fetch the resource 84, the proxy submits a request 96 and the server 82 returns the resource 98 for storage within the cache 86. The proxy 80 may be shared by multiple users.

[0038] While the present invention has been described with reference to an illustrative embodiment thereof, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the intended scope of the present invention as defined in the appended claims.