Method for Accessing and Retrieving Information From a Source Maintained by a Network Server

This invention relates to data communications networks and, in particular, to methods and apparatus for accessing and retrieving information from a database, documents, files or web pages maintained by a network server. The methods and apparatus of this invention are particularly useful for scheduling the data download from the World Wide Web (WWW) without keeping the requesting computer system power on all the time till the upcoming download activites. The methods and apparatus of this invention also allow the user to download data from web sites requiring user id and password by supplying the previous stored user id and password on the requesting computer system.

An internet user typically employs a browser to access the WWW. The most popular browsers are Netscape's Navigator and Microsoft's Internet Explorer. Often a user wishes to download data from various sites. The user may be linked to these sites during certain periods of the day in which either the internet of the server at the linked sites may be very busy. Downloading at such times could take a very long time. The user may not wish to tie up his browser, or the computer in which the browser is housed, for such a long time. In addition, the connections may require connect charges billed on a per minute basis.

The widespread availability of WWW phones, Personal Data Assistants (PDAs), and Windows-based CE machines with internet connectivity are expected to soon provide internet access capability to larger portions of the earth's population, thereby making efficient techniques to access WWW pages (web pages) even more desirable.

For many users an internet connection made at home some time in the middle of the night may be lower in cost than a connection made during peak phone-rate hours while travelling.

At present there exist several techniques that are known to the inventor for indicating specific web pages to be downloaded at a later time. These techniques download the requested pages to the same (requesting) machine at a later time, for example at night when phone rates are lower and internet traffic is reduced.

There also exist so-called push technology schemes, such as one known as Pointcast.TM., that periodically download information from certain sites to a given data processor. A user can schedule, for example, news, stock, and/or weather information to be downloaded at specific times or at specific intervals. However, these techniques also download the requested information to the requesting data processor.

Other techniques, such as one known as Webwhacker.TM., enable a user to make a local copy of a web site, and allow the user to specify a number of links (i.e., Hyperlinks) to follow and download.

A technology available from the assignee of this patent application, referred to as ARTour WebExpress.TM., allows a user to browse the web more asynchronously than is possible with current browsers. For example, using conventional WWW browsers such as Netscape Navigator.TM. 3.0 or Internet Explorer.TM. 3.0 the user can scroll a current page while a next page is being downloaded, thereby providing a degree of asynchronous access. The WebExpress.TM. technique takes this one level further by allowing the user to continue to specify links (Hyperlinks) to fetch while previously specified pages are being fetched. These requests are queued by in a local buffer and the pages are fetched in a sequential manner. When the requested pages are available on the local machine, the user is made aware of it by a suitable signaling mechanism.

A proxy server is a World Wide Web server that acts as the sole web server for an entire domain, or for those client computers that are placed behind a firewall (i.e., a logical block between the clients and the rest of the internet). The proxy server typically resides at the firewall and intercepts all web requests originating from clients within the firewall. If a given web page request is not in the proxy server's access control list, the request is processed normally and the retrieved web page is sent back to the requesting client. If, however, the requested web page or web site is on the control list, the client instead receives a message indicating that the URL is not accessible or is not valid.

A proxy server can improve a network's performance by functioning as a caching server. Using its cached web pages, the proxy server will serve already-accessed web pages to requesting clients without requiring outside access to the internet. For example, consider a case of an environment where n client computers access the same web page, wherein each client computer outputs the address (URL) of the web page to be accessed. Without the use of the proxy server, n separate requests for the web page are initiated, and n separate copies of that same web page are retrieved and returned to the client computers.

Using a proxy server, however, the same n web page requests are handled more efficiently. Only the first request to reach the proxy server actually causes that web page to be retrieved from the WWW server, and only if that web page is not already stored in the proxy server's cache. When retrieved, the web page is sent back to the requesting client computer, and is also cached on the proxy server's hard disk. The remaining n-1 clients that request that same web page are then served instead from the proxy server's cache, thus avoiding unnecessary duplicated requests and delays.

An article by L. B. Rosenfeld and M. P. Holland (Online, Vol. 18, no. 3, pp 27-30, May 1994) discusses an internet filtering system which automatically downloads postings daily.

An article by T. Jaeger and A. D. Rubin (Proc. of Symp. on Network and Distributed System Security, IEEE Computer Soc. Press, pp 53-63, 1996) discusses an automated file location and retrieval system which retrieves files and software across the internet.

A web page from Traveling Software (http://www.travsoft.com/press.sub.-- room/webex.htm) discloses a software downloading package known as WebEx which allows a user to select which web sites to download as well as when to begin the downloading operation. For this method to work, the receiving client must be powered on. Also the downloading will occur no matter what bandwidth is established between the server and the receiving client. This may not be desirable, as in the case where costs are proportional to the time spent for the link to be operational, and when the link is of very low bandwidth, thereby making the download expensive.

However, none of the existing techniques that are known to the inventors enable web pages and other data to be downloaded to this or anoter machine at some other specified time, whether on not that machine is powered on or not. Nor do any of the existing techniques that are known to the inventors enable data to be downloaded only if a minimum prescribed bandwidth between the server and the receiving client is obtained.

It is an object and advantage of this invention to provide an improved method and apparatus for downloading information from a server that overcomes the foregoing and other problems.

It is an object and advantage of this invention to provide a method and system for scheduling the data download from the World Wide Web (WWW) without keeping the requesting computer system power on all the time till the upcoming download activites. The methods and apparatus of this invention also allow the user to download data from web sites requiring user id and password by supplying the previous stored user id and password on the requesting computer system.

The foregoing and other problems are overcome and the objects of the invention are realized by methods and apparatus in accordance with embodiments of this invention.

A method and system is disclosed for scheduling data download, such as web pages, databases or softwares, over a network such as the internet without keeping the computer system power on all the time till the upcoming data download activities. The method includes the steps of (a) initiating the data download request and the user input interfaces; saving the requesting computer system's network address, the requesting computer system being either connected to or disconnected from the network; (b) fetching from the internet (the network connected requesting computer system) or from the user's input (the network disconnected requesting computer system) and saving those web pages, databases or softwares' source entities and their corresponding network addresses (e.g. the web site URL on the distant server) for the upcoming data download (which web pages and their linked sub level pages are to be downloaded); (c) fetching and saving from the user's input on the data download schedules (when the data download is to take place) and user id and password if required for those web pages, databases or softwares for the upcoming download; and other user's options and choices, such as where the downloaded data is to be stored (i.e. cache or disk directory), limits on downloading time and allocated storage space, and automatically rescheduling another data download for previous unsuccessful data download; data downloading either scheduled all at the same time in sequent or individually at different time; (d) setting the system timer at the wake-up time according to the data download schedules; (e) automatically turning on the requesting computer system according to the system timer if the requesting computer system is not turned on, and dialing up to connect to the network; generating a network busy signal to the Internal Data Download Schedule for rescheduling the data download according to the user's input of options and choices if the network is busy; (f) accessing the download data's network address and transmitting the data to the requesting computer system; providing the user id and password if required; (g) receiving and storing the requested web pages, databases or softwares in the requesting computer system for subsequent use by the user; transmitting successful and unsuccessful download messages from the destination entity to the requesting computer system; accordingly generating successful or unsuccessful data download messages for the user and unsuccessful data download signals to the Internet Data Download Scheduler to schedule another try of data download according to the user's input of options and choices; the user can access all these messages when the user logs on the requesting computer system; (h) interrupting a particular data downloading (transmitting and receiving) if the downloading time (estimated or actual) exceeding the user's earlier input downloading time limit or if the allocated download storage space reached the user specified limit; (i) automatically disconnecting from the network; (j) automatically rescheduling the data download in some other time, such as every hour afterwards, according to the user's earlier input choice if the previous data download is not successful or the network is too busy; (k) automatically turning off the requesting computer system. In this manner the requester doesn't need to keep the requesting computer system power on for the upcoming download actvities.

The most important advantage of this invention is that the requesting computer system does not have to be power on all the time till the upcoming download activities.

Other advantages of this invention include the abilities of allowing the user to schedule data download from those web sites requiring an user id and password, and to specifying the upper limits on the download time and the allocated storage space for downloaded data.

In one embodiment the step of initiating includes steps of generating a web page download command and transmitting the web page download command to the destination entity. This may be accomplished over the network, or over another network, such as an intranet, that connects the initiating and destination entities. In this case the step of fulfilling includes initial steps of formulating, in response to receiving the web page download command at the destination entity, a network web page request message and transmitting the network web page request message from the destination entity to the web page source entity. A confirmation message may be sent to the initiating entity to confirm the receipt of the web page download command.

In another embodiment of this invention the step of initiating includes steps of generating the network web page request message that includes the user id and password if required and transmitting the network web page request message from the initiating entity to the web page source entity. In this case the web page source entity checks the user id and password if required before transmits the requested web page(s) to the destination entity at the requesting computer system network address.

The step of initiating includes a preliminary step of responding to a signal from a user through a user interface, such as by redefining mouse clicks when interacting with a web browser, such that the signal indicates that a specified web page is to be downloaded to and stored in the destination entity, as opposed to being fetched and displayed in a conventional manner. In this embodiment the step of responding includes a step of prompting the user to enter information for specifying at least one parameter related to downloading the web page, and/or includes a step of retrieving at least one user default parameter related to downloading the web page.

In a preferred embodiment the web page download command sent by the initiating entity includes a plurality of fields, including fields intended to specify: at least one user download preference; and at least one postprocessing operation to be performed on a received web page. The at least one user download preference includes at least one of: a number of web page levels to download; whether to download graphical data; a number of permissible retries to download a web page; and an interval between the retries. The at least one postprocessing operation includes at least one of: whether to decompress a received web page; whether to virus scan a received web page; and whether to print a received web page.

The step of receiving and storing the requested web page in the destination entity includes a step of writing data into an index entry associated with the received web page. The index entry is comprised of a plurality of fields, including fields intended to specify: the first and second network addresses, and a link summary of the web page. The index entry fields further specify at least one of: a time that the web page was downloaded; a number of bytes that were downloaded; a time that the web page download command was received by the destination entity; a number of retries that were required, if any, to download the web page; and an error report.

In a preferred embodiment of this invention the method includes a capability to transmit a cancellation message from the initiating machine to the destination machine. In response to receiving a cancellation message the destination machine one of terminates an on-going web page download, or deletes an already downloaded and stored web page, as well as the index information associated with the stored web page.

In a preferred, but not limiting embodiment, the network includes the internet, and the web page source entity is a WWW server compliant with conventional and/or extended HTTP protocols.

The above set forth and other features of the invention are made more apparent in the ensuing Detailed Description of the Invention when read in conjunction with the attached Drawings, wherein:

FIG. 1 is a logic flow diagram showing how to set up the Internet Data Download Scheduler;

FIG. 2 is a logic flow diagram showing a method executed by the Internet Data Download Scgeduler when the system timer is up at the scheduled wake-up time. FIG. 2 is a logic flow diagram showing a method executed by the IDP of FIG. 1; and

The following description of this invention is made in the context of methods and apparatus that use a browser software entity for specifying and downloading web pages from an internet-connected server. It should be realized, however, that the teachings of this invention apply as well to accessing and retrieving other data, such as database information, documents, and files, that is available through a network-connected server.

A method and system is disclosed for scheduling data download, such as web pages, databases or softwares, over a network such as the internet without keeping the computer system power on all the time till the upcoming data download activities. In FIG. 1 it shows when and how to set up the Internet Data Download Scheduler. It describes the following method from steps (a) to (d). In FIG. 2 it shows a method to be executed by the Internet Data Download Scheduler when the system timer is up at the scheduled wake-up time. It descriv=bes the following method from steps (e) to (k).

The method includes the steps of (a) initiating the data download request and the user input interfaces; saving the requesting computer system's network address, the requesting computer system being either connected to or disconnected from the network; (b) fetching from the internet (the network connected requesting computer system) or from the user's input (the network disconnected requesting computer system) and saving those web pages, databases or softwares' source entities and their corresponding network addresses (e.g. the web site URL on the distant server) for the upcoming data download (which web pages and their linked sub level pages are to be downloaded); (c) fetching and saving from the user's input on the data download schedules (when the data download is to take place) and user id and password if required for those web pages, databases or softwares for the upcoming download; and other user's options and choices, such as where the downloaded data is to be stored (i.e. cache or disk directory), limits on downloading time and allocated storage space, what the minimum bandwidth is between the requesting computer system and the distant server, and automatically rescheduling another data download for previous unsuccessful data download; data downloading either scheduled all at the same time in sequent or individually at different time; (d) setting the system timer at the wake-up time according to the data download schedules; (e) automatically turning on the requesting computer system according to the system timer if the requesting computer system is not turned on, and dialing up to connect to the network; generating a network busy signal to the Internal Data Download Schedule for rescheduling the data download according to the user's input of options and choices if the network is busy; (f) accessing the download data's network address and transmitting the data to the requesting computer system; providing the user id and password if required; (g) receiving and storing the requested web pages, databases or softwares in the requesting computer system for subsequent use by the user; transmitting successful and unsuccessful download messages from the destination entity to the requesting computer system; accordingly generating successful or unsuccessful data download messages for the user and unsuccessful data download signals to the Internet Data Download Scheduler to schedule another try of data download according to the user's input of options and choices; the user can access all these messages when the user logs on the requesting computer system; (h) interrupting a particular data downloading (transmitting and receiving) if the downloading time (estimated or actual) exceeding the user's earlier input downloading time limit or if the allocated download storage space reached the user specified limit; (i) automatically disconnecting from the network; (j) automatically rescheduling the data download in some other time, such as every hour afterwards, according to the user's earlier input choice if the previous data download is not successful or the network is too busy; (k) automatically turning off the requesting computer system. In this manner the requester doesn't need to keep the requesting computer system power on for the upcoming download actvities.

The most important advantage of this invention is that the requesting computer system does not have to be power on all the time till the upcoming download activities. Other advantages of this invention include the abilities of allowing the user to schedule data download from those web sites requiring an user id and password, and to specifying the upper limits on the download time and the allocated storage space for downloaded data.

In one embodiment the step of initiating includes steps of generating a web page download command and transmitting the web page download command to the destination entity. This may be accomplished over the network, or over another network, such as an intranet, that connects the initiating and destination entities. In this case the step of fulfilling includes initial steps of formulating, in response to receiving the web page download command at the destination entity, a network web page request message and transmitting the network web page request message from the destination entity to the web page source entity. A confirmation message may be sent to the initiating entity to confirm the receipt of the web page download command.

In another embodiment of this invention the step of initiating includes steps of generating the network web page request message that includes the user id and password if required and transmitting the network web page request message from the initiating entity to the web page source entity. In this case the web page source entity checks the user id and password if required before transmits the requested web page(s) to the destination entity at the requesting computer system network address.

The step of initiating includes a preliminary step of responding to a signal from a user through a user interface, such as by redefining mouse clicks when interacting with a web browser, such that the signal indicates that a specified web page is to be downloaded to and stored in the destination entity, as opposed to being fetched and displayed in a conventional manner. In this embodiment the step of responding includes a step of prompting the user to enter information for specifying at least one parameter related to downloading the web page, and/or includes a step of retrieving at least one user default parameter related to downloading the web page.

In a preferred embodiment the web page download command sent by the initiating entity includes a plurality of fields, including fields intended to specify: at least one user download preference; and at least one postprocessing operation to be performed on a received web page. The at least one user download preference includes at least one of: a number of web page levels to download; whether to download graphical data; a number of permissible retries to download a web page; and an interval between the retries. The at least one postprocessing operation includes at least one of: whether to decompress a received web page; whether to virus scan a received web page; and whether to print a received web page.

The step of receiving and storing the requested web page in the destination entity includes a step of writing data into an index entry associated with the received web page. The index entry is comprised of a plurality of fields, including fields intended to specify: the first and second network addresses, and a link summary of the web page. The index entry fields further specify at least one of: a time that the web page was downloaded; a number of bytes that were downloaded; a time that the web page download command was received by the destination entity; a number of retries that were required, if any, to download the web page; and an error report.

In a preferred embodiment of this invention the method includes a capability to transmit a cancellation message from the initiating machine to the destination machine. In response to receiving a cancellation message the destination machine one of terminates an on-going web page download, or deletes an already downloaded and stored web page, as well as the index information associated with the stored web page.

In a preferred, but not limiting embodiment, the network includes the internet, and the web page source entity is a WWW server compliant with conventional and/or extended HTTP protocols.