Method and apparatus for populating a form with data

[0001] This is a continuation of application Ser. No. 09/430,535 filed Oct. 29, 1999 now U.S. Pat. No. 6,589,290.

[0002] 1. Field of the Invention

[0003] The present invention relates to computer networks and more particularly to a method and apparatus for populating a form with data.

[0004] 2. Background Art

[0005] Computer users can shop for merchandise using the Internet. A number of companies maintain web sites where customers can purchase books, records, videos, and many other products. Other web sites provide subscription services that provide the user with access to a service or product. Typically, users make purchases or sign up for services by completing forms. A problem with current Internet schemes is that each field on a form must be filled out separately and manually. This is a time consuming and often frustrating process for the user.

[0006] Current systems do not provide a way for users to quickly complete a form (referred to as “populating” a form) with data. For example, when a user wishes to purchase a product online, the user typically supplies a credit card number, a shipping address, and any other personal information that may be needed to complete the transaction. The user supplies this information using an input device such as a keyboard to manually enter data into the fields of a form. However, after buying goods from one vendor, for example, the user may wish to purchase goods from a second vendor. The user must manually complete the second vendor's form even though the second vendor may require the purchaser to provide the same kind of information the user provided to the first vendor. This presents a problem to the user because it forces the user to manually enter the same information into different vendor's forms multiple times.

[0007] Another problem presented by the prior art is that users do not have a way to securely store sensitive information and then later use that information to populate a form. Current systems do not provide a way to prevent unauthorized users from obtaining access to sensitive information stored on a user's computer. Instead such systems leave data that is entered into a form readily accessible. For example, data that is entered into a form may be stored locally using a technology called cookies. A cookie is a local representation of information related to a particular web page that was previously visited by the user. Cookies are not encrypted and may be read by anyone who can obtain them. The term anyone also includes any computer program that knows where the cookies are stored. This is problematic because it unnecessarily exposes sensitive information to unauthorized users. Therefore, there is a need for a method and apparatus for populating forms with data where that data is accessible only to authenticated users.

[0008] The problems associated with form population may be better understood by the following discussion of the Internet/World Wide Web, web page creation, embedded forms, cookies, and web browser technology.

The Internet/World Wide Web:

[0009] A web browser is a type of computer program that provides users with a mechanism for accessing the World Wide Web (WWW). The WWW is a segment of the Internet comprised of numerous web clients and web servers that communicate with one another using a standard set of protocols. A web server is a computer configured to provide web pages to the web client upon request. A web client typically utilizes the web browser application to request web pages from the web server.

[0010] The Internet is a global computer network comprised of an amalgamation of interconnected networks that are capable of readily communicating with one another using a standardized set of protocols. Protocols provide a uniform set of communication parameters that enable computers to effectively transmit and receive data. Most computer networks, including the Internet, utilize several different layers of protocols. Each layer provides the network with different functionality.

[0011] The WWW is a segment of the Internet that utilizes an application layer protocol called the HyperText Transfer Protocol (HTTP) to disseminate and to obtain information from users. HTTP is a request/response protocol used with distributed, collaborative, hypermedia information systems. In operation, HTTP enables one computer to communicate with another. For example referring now to FIG. 1, web client 100 can use HTTP to communicate with web server 150 via network 125. In this scenario the web server acts as a repository for files 160 and is capable of processing the web client's requests for such files. The files 160 stored on the web server may contain any type of data. For example, the files may contain data used to construct a form, image data, text data, or any other type of data.

[0012] HTTP has communication methods that allow web client 100 to request or send data to web server 150. The web client 100 may use web browser 110 to initiate request 115 and receive one or more files from file repository 160. Typically, web browser 110 sends a request 115 for at least one file to web server 150 and the web server forwards the requested file to web client 100 in response 120. The connection is then terminated between web client 100 and web server 150. Web client 100 then uses web browser 110 to display the requested file. A client request 115 therefore, consists of establishing a connection between the web client and the web server using network 125, issuing a response 120 to the request 115, and terminating the connection.

[0013] Web server 150 does not utilize state information about the request once the connection is terminated. HTTP is, therefore, a stateless application protocol. That is, a web client can send several requests to a web server, but each individual request is treated independent of any other request. In some instances, the web server maintains a record of such requests, but the web server does not use that information to process later requests. Thus, for example, if a form is completed by the user and submitted to the web server for processing, the web server may maintain a record of the data entered into the form, but that record will not be used to later influence another request from the same client. In other words, web server 150 may record a request in a log file, but it does not later read from the log to determine how to respond to another request from the same client.

[0014] Once a file is sent from web server 150 to web client 100 it becomes ready for display. The web client's 100 web browser 110 is typically used to format and display files. Web browser 110 allows the user to request and view a file without having to learn a complicated command syntax. Examples of several widely used web browsers include Netscape Navigator, Internet Explorer, and Opera. Some web browsers can display several different types of files. For example, files written using the HyperText Markup Language (HTML), the JavaScript programming language, the ActiveX programming language, or the Portable Document Format (PDF) may be displayed using a web browser. It is also possible to display various other types of files using language such as Standard Generalized Markup Language (SGML) or eXtensible Markup Language (XML).

Creating a Web Page:

[0015] A form, which provides one or more places for a user to enter data, can be embedded inside of a web page. A web page may be created using a variety of different data formats and/or programming languages. Most web pages, and as a result most forms, are created using the HyperText Markup Language (HTML). The techniques used to create a web page will now be discussed in further detail.

[0016] HTML is a language that may be used to specify the contents of a web page (e.g. web page 220). An HTML description is typically comprised of a set of markup symbols which are described in more detail below. HTML file 250 or any type of data file that contains the markup symbols for web page 220 may be sent to web browser 210. Web browser 210 executing at web client 200 parses the markup symbols in HTML file 250 and produces web page 220, which is then displayed, based on the information in HTML file 250. Web page 220 may contain text, pictures, or forms comprised of embedded text fields, checkboxes, or other types of data that is to be displayed on the web client using web browser 210. Consequently, HTML document 250 defines the web page 220 that is rendered by web browser 210. For example, the following set of markup symbols directs web browser 210 to display a title, a heading, and an image called “image.jpg”:

[0017]

	<HTML>
	<HEAD>
	<TITLE> This is a document title </TITLE>
	</HEAD>
	<BODY>
	<H1> This text uses heading level one </H1>
	<IMG SRC=“http://www.idealab.com/image.jpg”>
	</BODY>
	</HTML>

[0018] In the above example, markup symbols (e.g. “<” and “>”) indicate where each HTML command (e.g. TITLE) begins and ends. An HTML command, which is typically surround by markup symbols, provides the web browser with instructions to execute. Markup symbols typically surround an HTML command. The “<” symbol indicates the start of an HTML command an the “</” symbol indicates the end of an HTML command. Each start or end command has a corresponding “>” to indicate the close of that particular command. Information associated with the HTML command may be contained within the HTML command's start and end symbols. An HTML command is used by the web browser 210 to determine how to process the block of information associated with the two commands.

[0019] In the above example, “<TITLE>”, and “</TITLE>” are examples of HTML commands surrounded by markup symbols. The “</TITLE>” HTML command directs web browser 210 to place the text “This is a document title” in the title bar of web browser 210.

[0020] Some HTML commands have attribute names associated with the command. For example, HTML command “<IMG>”, directs web browser 210 to display an image. A “SRC=” attribute identifies the location and name of the image to be displayed. In the above example, the statement “<IMG SRC=“http://www.idealab.com/image.jpg”>” tells the web browser to display an image named “image.jpg” that can be obtained from the web server located at “http://www.idealab.com.”

Embedding a Form into a Web Page

[0021] An HTML file may also contain HTML commands that cause the web browser to render a web page that contains fields for entering data. The portion of the web page that contains the data entry fields may be referred to as a form (e.g. the portion enclosed in the HTML tags <FORM></FORM>). When the web page is comprised of primarily data entry fields the entire web page is sometimes also referred to as a form. As is discussed below, HTML includes an HTML form command that may cause the browser to display data entry fields. A text box, a drop down menu, a check box, a command button, a toggle button, or any other kind of interface component capable of receiving input are some examples of the type of data entry fields that may be placed in a form. Data is manually entered into the fields by the user and then submitted to a web server for processing. As previously discussed, when a user wishes to use a form to purchase a product, for example, the user manually fills in text boxes located on the form with the type of information needed to carry out the transaction. Other types of data entry fields require the user to select from one or more options such as in the case of a menu, a toggle button, or a radio button, for example. When the form is complete the user submits the completed form to the vendor's web server by pressing a command button, for example.

[0022] A problem with collecting information from the user in this manner is that the user is required to manually enter data into the fields of a form. This takes time and becomes unnecessarily burdensome when the user wishes to complete more than one form with the same information.

[0023] FIG. 3 provides an example of, a form created using the HTML definition language. Code block 310 contains HTML command examples. When a document comprising code block 310 is transmitted to web browser 300 executing on web client 330, it causes form 305 to be displayed. Web browser 300 displays form 305 by parsing the HTML commands contained in code block 310 and then using the information obtained to format form 305. Once the user finishes entering information into form 305, the user may submit the form by pressing command button 331. Pressing command button 331 sends the information entered by the user in form 305 to web server 320 using the POST method. However, one of ordinary skill in the art could modify code block 310 to use the GET method instead of the POST method.

[0024] The <FORM> command shown in code block 310 indicates the beginning of a form. Once the initial FORM command is placed into the HTML document other HTML commands may be entered between the initial FORM command and the closing FORM command (e.g. </FORM>) that represent, for example, one or more data entry fields such as a text-box, drop-down lists, check boxes, radio buttons, and input buttons. The INPUT command is used to specify different types of data entry fields. The type of data entry field created is dependant upon the value assigned to the INPUT command's TYPE attribute. For example, the INPUT command can create a text-box, a password field, a hidden field, a button, a radio button, or a checkbox. A “text” value assigned to the TYPE attribute of an INPUT command creates a text box, for example. Similarly, a TYPE of “checkbox” creates a checkbox. The following code, if inserted after the FORM element, creates a text box and a checkbox:

[0025]

	<INPUT TYPE=“text” NAME=“user_name”>
	<INPUT TYPE=“checkbox” NAME=“user_item1”>

[0026] The HTML tags and text contained in code block 310, for example, create form 305 when displayed using web browser 300.

[0027] A <FORM> tag may contain an ACTION and/or METHOD attribute. The ACTION attribute specifies what program on the server to execute when form 305 is submitted for processing. The METHOD attribute describes how data entered into the form is handled. There are two METHOD choices: GET and POST. Each one of them is capable of processing the data entered into a form. To illustrate, if a user named Bill who resides at 130 West Union Street, Pasadena, Calif. 91103 with a credit card number of 1111222233334444 completes the Name Address, and Credit Card number fields shown on form 305 by filling in spaces 345–347, then the following data string 350 is created and sent to web server 320.

[0028]

	user_name=Bill&user_address=130 West Union Street
	Pasadena CA 91103&user_cc=1111222233334444

This data string may be handled by either the GET method or the POST method. However, in FIG. 3 it is handled by the POST method. The GET method appends information entered into form 305 by the user onto the end of a Uniform Resource Locator (URL 335) or in a QUERY_STRING environment variable. URL 335 is submitted to web server 320 by web client 330 in an HTTP request 315. Web server 320 is responsible for processing the submitted information. A problem with the GET method is that the information contained in URL 335 is not stored on web client 330 for later use and cannot therefore be later used to populate form 305. With the POST method, the information placed into form 305 by the user is placed into the data stream of the HTTP protocol. This allows web server 320 to process form 305 data using the standard input data stream to a separate program or script on the server. Standard input is a method for providing a program or script with data that is well know to programmers skilled in the art. The POST method also does, not provide a way to store data on web client 330 for later use.

[0029] The ACTION attribute of the FORM tag indicates where the computer program tasked with processing data string 350 resides. For example, the following portion of code block 310 specifies that a computer program (e.g. a PERL script) named form_processor.pl is located in cgi-bin 321 of sever 320.

[0030]

	<FORM METHOD=POST ACTION=http://server320
	/form_processor.p1>

The form_processor.pl program is responsible for processing data string 350. Programs that utilize the Common Gateway Interface (CGI) standard are typically located in a common directory (e.g. cgi-bin 321). CGI is a specified standard for communicating between HTTP servers and server-side gateway programs. A CGI program is capable of storing data supplied by the user, but it cannot determine whether the user asking for the form is the same user that previously submitted data to the CGI program. If the user has never visited the web site before, the CGI program does not have any information about that user. CGI programs can collect information about users who visit a web site, but such programs cannot collect information about users who have never visited the web site.

[0031] The program “form_processor.pl” used in the above example is an example of a server-side program that processes data input by the user via form 305, for example. When form 305 is submitted web server 320 executes the “form_processor.pl” program and passes the data that was entered by the user and then sent from web client 330 to server 321. That is, data string 350 is passed to “form_processor.pl” when a form is submitted. When the gateway program is done processing data string 350 the result is sent to server 320 and a response may be forwarded back to web client 330. Gateway programs can be compiled programs written in languages such as C or C++ or they can be executable scripts written using a scripting language such as PERL, TCL, or various other language. Once data string 350 is processed it may be stored on server 320 in data store 322. A problem with using CGI programs is that since they are located on the server, information cannot easily be saved on the web client for later use.

[0032] Thus, existing mechanisms for processing forms reside on the server and are unable to populate a form on the client with user data from users that have never visited the web site having the form (e.g. the data is only available to web servers the user has previously visited).

HTTP Cookies

[0033] One example of data that may be stored on the client is a cookie. A cookie is an HTTP header that consists of a text-only string. The text-string is entered into the memory of a web client and accessible by the web browser. This text-string is initially set by the computer serving the web page and consists of the domain name, path, lifetime, and value of a variable that the server sets. If the lifetime of this variable is longer than the time the user spends at that site, then the text-string is saved on the web client for later use. As a result, cookies provide a way to store and later recall values entered into a web page by the user. For example, cookies allow a web server to individually customize a web page for each user who visits the page.

[0034] A cookie may be used to populate the elements of a form that a user has previously completed. For example, referring now to FIG. 9, if a user completes field 945–947 of form 905 and submits it to server 920 via submission path 915, server 920 can place the data submitted by the user in cookie 931 by sending web client 910 data via path 925. If the user returns to form 905 again Web browser 930 executing at web client 910 may use the data saved in cookie 931 populate fields 945–947 with the data previously entered by the user. Thus, a cookie can free the user from having to complete form 905 more than once. Cookies can be used to fill forms the user has previously visited. However, the data that is stored by the cookie cannot be used to populate other forms. That is, a problem with cookies is that forms located on different servers are not provided access to the same cookie. Only web servers that are listed as having permission to access a particular cookie may obtain such access.

[0035] A cookie is only accessible to the server that set, or created it. Thus, a second form cannot use the first form's cookies to obtain the user's name and address. For example, the web server located at www.merchantA.com may be allowed to read a cookie, but the web server located at www.merchantB.com may not be allowed to read the cookie created by www.merchantA.com

[0036] Referring now to FIG. 4, the process used to create and retrieve a cookie is illustrated. Initially, web browser 401 which resides on web client 400 issues a request 410 for web page 415 from web server 450. Web server 450 responds by sending a copy of web page 415 to web client 400 via response 411. Web client 400 uses web browser 401 to display web page 415 to the user. To create a cookie 460, web server 450 sends a “Set-Cookie” command in the header line contained in response 411. For example, in response to a request 410, web server 450 may send the following command in HTTP response 411:

[0037]

	Set-Cookie: NAME=VALUE; expires=DATE; path=PATH;
	domain=DOMAIN_NAME; secure.

The NAME and VALUE parameter contain the information included in the cookie. DATE is the time at which the cookie expires and is thus no longer saved on web client 210. DOMAIN is the host address or domain name for which the cookie is valid. For example, if a cookie is set by a computer having a domain name of www.merchantA.com, only the server responsible for that particular domain name is provided access to the cookie. The PATH parameter specifies a subset of URL's at the appropriate domain for which the cookie is valid. If the keyword secure is used then the cookie is only transmitted over a secure connection. All of these parameters except the NAME=VALUE are optional to set a cookie. Once the Set-Cookie command is sent to web client 400 a cookie 460 is placed on web client 400.

[0038] To create a cookie 460 using the UNIX operating system, for example, web server 450 could execute the following shell script in response to a request 410:

[0039]

	#!/bin/sh
	echo Content-type: text/html
	echo Set-cookie: FooBar=foo; expires=Wednesday,
	02-11-99 12:00:00 GMT
	echo
	echo <H1>A Cookie named Foobar containing the text
	foo is now present on your computer</H>

This script stores “FooBar=foo” on web client 400. The following script allows web server 450 to read the HTTP cookie:

[0040]

	#!/bin/sh
	echo Content-type: text/html
	echo
	echo The data supplied here was obtained from a
	cookie: <P>
	echo $HTTP_COOKIE

Once cookie 460 is created, the information stored in the cookie can be used in many different ways. Information in cookie 460 may be accessed by a script that has authorization to insert cookie 460 data into a form, for example. However, a problem with cookies that no authentication mechanism is required to obtain the information they store. A cookie is a text file and is not stored in encrypted form. While a browser may limit access to a cookie, the text that is stored in the cookie may be accessed or read by any program that can read a text file. Therefore, it is not wise to store any sensitive data in a cookie. An additional problem with cookies is that users do not have to enter a user name and password to obtain access to the cookie.
Predefined Paste Operation

[0041] Some web browsers provide users with a mechanism for pasting predefined data into a form. Under the generic preferences menu of the Opera web browser, for example, the user can select a command button labeled personal information. When this command button is selected a dialog box that contains fields for entering personal information opens. If the user desires, the user can enter a name, an address, a phone number, an e-mail address, and/or any other kind of information into the text fields of the dialog box.

[0042] Once the user enters information into the fields of the dialog box and clicks “OK” the information that was entered can be pasted into the fields of a form by performing a right click operation. For example, if the user encounters a form that has a place for entering an address, the user can elect to paste the address previously specified in the dialog box by first selecting the address field and then right-clicking on a pointing device such as a mouse and selecting from a drop down menu the item titled “insert address”.

[0043] A problem with pasting information into the fields of a form in this manner is that the user is required to manually select the place to insert the information and the type of information to insert into that place. Another problem is that unauthorized users are not prevented from accessing the information entered into the dialog box.

[0044] A method and apparatus for populating a form with data is described. One embodiment of the present invention executes on a web client that is connected to a computer network such as the Internet. The web client is capable of obtaining web pages that contain forms from the Internet. The web client contains computer code, such as a form completion program, configured to enable a user to populate a form with data. The form completion program interacts with a target application (e.g. a web browser) installed at the web client to provide the user with a mechanism for filling out multiple forms. The form completion program, for example, provides the user with a mechanism to complete a form by dragging a graphical representation of a data set over to the form and then dropping it.

[0045] In one embodiment of the invention, a form is displayed to the user via the target application. Each form has one or more data receptacles. The data receptacles of a form are filled with data when the user executes a data population command. The form completion program executes the data population command when a graphical representation of a particular data set is placed over the form. The form completion program collects data from the user via a data collection interface. The data collection interface provides data entry fields for the user to enter a particular type of information (e.g. a data set). Each data set that is entered using the data collection interface is stored in an encrypted manner and is accessible to users who enter the appropriate information into an authentication mechanism.

[0046] To populate a form with data the form completion program obtains an image of the form and then searches for a template file that resembles the form image to within a certain threshold. The template files are typically stored on the computer hosting the target application in a template directory that is arranged according to a predefined structure. The form completion program is configured to search for templates that resemble the form image in the template directory to within a certain threshold. However, the invention also contemplates searching for templates at alternate locations. Templates, for example, may reside on any other computer that is accessible via a telecommunication medium such as a computer network.

[0047] The form completion program examines each template file in order to determine if one or more of the template files resembles the form image to within a certain threshold. If a template file that resembles the form image to within a certain threshold is located, then the form completion acknowledges that a match occurred. When a match occurs the form completion program utilizes the template file to identify what kind of data to insert into each of the form's data receptacles. For example, the template file allows the form completion program to determine which of the data receptacles contain personal information and which data receptacles contain payment information. Once the form completion program successfully identifies what kind of data to insert into each data receptacle the program begins to input the appropriate kind of data into the appropriate data receptacle.

[0048] The present invention contemplates the use of multiple techniques to insert information into the data receptacles. The technique used is largely dependent upon the type of target application used to display the form.

[0060] A method and apparatus for populating a form with data is described. In the following description numerous specific details are set forth in order to provide a more thorough understanding of the present invention. It will be apparent, however, to one skilled in the art, that the present invention may be practiced without these specific details. In other instances, well-known features have not been described in detail so as not to obscure the invention.

[0061] One embodiment of the present invention executes on a web client that is connected to a computer network such as the Internet. The web client is capable of obtaining forms from the Internet and contains computer code configured to allow a user to populate a form with data. In one embodiment of the invention, a form completion program is created to execute the invention.

[0062] The form completion program interacts with a target application (e.g. a web browser) installed at the web client to provide the user with a mechanism for filling out multiple forms. The target application displays forms to the user. The form completion program may have multiple interfaces that each provide the web client 1015 with additional functionality. For example, referring now to FIG. 10, a data collection interface 1000 is shown. The data collection interface 1000 may be used to collect and store multiple data sets from multiple users. A data set is a collection of related information (e.g. payment information or personal information) that represents any kind of information likely to be used to complete a form.

[0063] In one embodiment of the invention, the data collection interface collects a data set via a data collection form 1005. Data entered into the data collection fields 1006–1009 of the data collection form 1005 may be stored in secure storage medium 1010. The data stored in storage medium 1010 is utilized to populate a form when the user issues a form population command. The data held in secure storage medium 1010 may be encrypted and reside on web client 1015 and/or any other computational device available via a network communication medium. A user is not permitted to access secure storage medium 1010 until properly authenticated.

[0064] Each data set stored in secure storage medium 1020, may be associated with a graphical representation 1120. Referring now to FIG. 11, for example, a data set comprising payment information may be represented by a credit card graphic. A data set comprising personal information may be represented by an identification card graphic. A data set may be associated with any type of graphic. In one embodiment of the invention, the graphical representations of each data set are displayed to the user via a shopping interface 1125. The shopping interface may be used to execute a form population command. When executed, a form population command completes form 1135. Form 1135 is displayed via a target application 1103 such as a web browser or other computer program.

[0065] The form completion program executes the data population command when a graphical representation 1120 is moved from the shopping interface 1120 to form 1135. The data population command fills data receptacles 1136–1139 with data taken from secure storage medium 1010. For example, if the user encounters a form 1135 that prompts the user for personal information (e.g. name, address, and phone number), then the user could provide that information to form 1135 by dragging a graphic (e.g. graphic 1120) that represents the user's personal information from form completion program to form 1135. When the user drops the graphical representation 1120 of the information onto the form 1135, the data population command places data into the appropriate place of the form by obtain the data from secure storage medium 1010, identifying what type of form 1135 is to be populated, and then using this information to fill the fields 1136–1139 of form 1135 with data. In one embodiment of the invention, the type of form 1135 that is to be populated is identified by a template file. However, the manner with which a form is populated is dependent upon the type of form that is being populated.

[0066] The form completion program, for example, provides the user with a mechanism to complete a form by dragging a graphical representation of a data set over to the form and then dropping it. Anytime the user encounters a form the user may elect to populate the form by performing this drag and drop operation.

[0067] In one embodiment of the invention, the data receptacles of a form are filled with data when the user executes a data population command.

Creating a Form

[0068] A form may be created using HTML or any other programming language capable of generating a document that provides a place for the user to enter data (e.g. a data receptacle). For example, a spreadsheet, a word processing document, a document created using PDF, a database table, or any other type of document having one or more data receptacles may be referred to as a form. A data receptacle is a field or place for holding and/or receiving data.

Data Population Command

[0069] In one embodiment of the invention, the data population command is executed by a form completion program. The form completion program may be configured to collect data from the user and execute a data population command when the user performs a drag and drop operation. Other types of computer programs, however, may also collect data from the user and perform a data population command. For example, an operating system, a dynamic link library, a Java applet, or any other type of executable computer program may be configured to execute a data population command.

[0070] The form completion program aids in the process of collecting data from the user by prompting the user for information likely to be required to complete a form. For example, the user might be prompted to enter personal information (e.g. name, address, phone number, etc.) and payment information (e.g. American Express number, Visa card number, MasterCard number). Once information is gathered, the form completion program creates a graphic that is representative of the type of information collected. For example, if the user provides the form completion program with payment information such as a Visa Card or MasterCard number, then a graphic is created that represents the form of payment information the user supplied. For example, an icon depicting a Visa Card, MasterCard or other such card might be displayed. If the user provides personal information, such as a name and address, then a graphic is created that identifies the data associated with that particular personal information. For example, an image resembling a drivers license or identification card might be displayed. Each graphical representation that is created is associated with a different data set. The user may elect to create additional data sets that have a corresponding graphical representation. The user may customize the type of data set collected and the graphics associated with that data set. A graphic may represent any kind of data. The form completion program utilizes the information the user supplies (e.g. the data set) to populate form. For example, when a graphic that is representative of a particular type of data is dragged from the form completion program and dropped on a form the data population command is executed, and the form is completed without requiring more input from the user.

[0071] When the user executes the data population command, data is placed inside the data receptacles of the form on which the user executed the data population command. Multiple data sets/graphics may be used to populate the same form. For example, a graphic representative of a credit card may be used to populate the form with credit card data and a graphic representative of a user's personal information may be used to populate the form with personal information.

[0072] Various types of data may be placed into each data receptacle. For example, executing a data population command may insert text data, image data, and/or any other type of data into one or more data receptacles. In one embodiment of the invention, the data utilized to populate the form is supplied by the user and stored on the user's computer before execution of the data population command. However, it is possible to obtain data from other sources. For example, the data population command may obtain data from a memory device that is available via a communication medium such as a computer network. The user executes a data population command by moving a graphical representation of the data from a first location to a second location.

[0073] Referring now to FIG. 5, an illustration of the process used by one embodiment of the invention to populate a form is shown. The user initiates a data population command at step 500 by performing a drag and drop operation. Once the user issues a data population command the form completion program proceeds to step 503 where an attempt is made to determine what type of target application is displaying the form that is to be populated. For example, the form completion program may determine that the user is attempting to populate a form displayed using the Internet Explorer web browser. The type of application displaying the form to the user is referred to as the target application.

[0074] When the target application type (e.g. Internet Explorer) is determined, then the form completion program proceeds to step 506. At step 506, the form completion program determines whether it recognizes the target application. The form completion program may be configured to support a variety of different target applications.

[0075] If the target application is recognized the form completion program proceeds to step 509 where it begins to communicate with the target application in a manner understood by the target application. If the target application is not recognized, then a return value is sent, at step 519, and the form completion program informs the user that this type of target application is not yet supported. At step 509, the form completion program is configured to interact with different target applications using a variety of different communication techniques. A form displayed using Internet Explorer may be dealt with in a different way than a form displayed using Netscape Communicator (e.g. a different interface protocol may be used). The COM interface, for example, is used to communicate with Internet Explorer 4. However, the present invention also contemplates the use of other interfaces and API's such as ActiveX or Javascript. The type of protocol utilized by the form completion program is dictated by the target application. The present invention contemplates the use of a protocol capable of communicating with the target application. It is possible to utilize a variety of different protocols to open a communication channel between the form completion program and the target application.

[0076] Once step 509 is complete, the communication channel between the form completion program and the target application is open. This allows the form completion program to proceed to step 511 where it captures the location of the form. For example, if the form is displayed using a web browser, the form completion program will capture the Uniform Resource Locator (URL) that identifies where the form is located. The URL provides the form completion program with the address information needed to determine where the form being populated was obtained. For example, the URL may indicate that the user obtained the form from another computer accessible via a communication medium such as a network. At step 513, the form completion program builds an image of the form the target application displays to the user. In one embodiment of the invention, the form completion program builds the form image by querying the computer (e.g. the server) that originally supplied the form to the target application. However, the form completion program may build a form image by querying any computer that has a version of the form that is displayed to the user. For example, the form image could be built by contacting a mirror server instead of the original server. The form completion program may also build the form image by querying the target application about its form properties or the form completion program may elect to utilize a local version of the form that is displayed to the user. For example, the form completion program may utilize a cached version of the form to build a form image. The present invention contemplates various techniques for building a form image. It is possible to utilize any version of the form the user is attempting to populate in order to build the form image.

[0077] Once an image of the form the user is attempting to populate is produced, the form completion program proceeds to step 515 where it searches for a template file that resembles the form image. The template file comprises a collection of form descriptions. In one aspect of the invention, the template file is a text file that contains one or more form descriptions. Each form description is typically associated with a particular form. However, one description may represent multiple forms if the forms are substantially similar to one another. Each form description in the template file is associated with a regular expression and a list of controls. Each control has an index, a symbol, and a control type description.

[0078] The index associates a number with each control that resides on a form. For example, if a form contains a button control and a select box control the template file may identify information about each of these controls by associating the number zero with the button control and the number one with the select box control. The template file may also contain a plain text name for each control. In one embodiment of the invention, each control is associated with a symbol. The symbol represents the data that is to be placed inside the fields of the form. For example, the symbol can represent a name, date, or gender. Each control also has a control type description that identifies what type of control is present. The control type description, for example, may be utilized by the form completion program to determine what steps are required to populate the control with data. The way a text box is filled, for example, may be different from the way a check box is completed. A text box may accept raw textual data whereas a check box may accept a Boolean value.

[0079] In one embodiment of the invention, each form in the template file has a regular expression associated with it. The regular expression describes a set of one or more forms that a particular template file resembles (e.g. by using a string of text, such as a URL, that is contained within a form) and thereby provides information that may be used to perform a string matching operation. String matching provides the form completion program with a way to determine whether the template file resembles a form to within a certain threshold. For example, if the URL of a form contains a word that starts with W followed by a period and three more letters, this information can be stored in the regular expression and used to verify that the correct template file is being used to complete the form.

[0080] If the form completion program attempts to complete a form that is not represented in the template file, a list of the data that was collected from the user is displayed. When the list is displayed, the user may elect to manually intervene by dragging information from the list onto the form. If, for example, the user wishes to place address information into one of the data receptacles contained on a form, the user can drag the address data from the list to the form. At this point, the information placed over the form is inserted into the appropriate data receptacle. The user may repeat this process to fill-in the remaining data receptacles. In one embodiment of the invention, the actions of the user may be used to generate a form description. For example, if a user manually completes a form, the form completion program may track the information the user elected to place in the forms various data receptacles. This information may then be transmitted to a third party for verification and utilized to generate a new form description for the template file. In one embodiment of the invention, a different template file is created for each form. However, a single template file may represent more than one form. For example, a generic template that operates on forms that adhere to a set of predefined standards may be created and later verified by an independent party. The template may store information using a variety of formats such as binary, hexadecimal, or text (e.g. ASCII or XML). It is also possible to use other methods for representing information. A template file, for example, may be an executable file, a compressed file, and/or any other form of storing digital information. For example, a template file may be included in the form (using ECML, inband tems).

[0081] The template files are typically stored on the computer hosting the target application in a template directory. The form completion program is configured to search for templates that resemble the form image in the template directory to within a certain threshold. However, the invention also contemplates searching for templates at alternate locations. Templates, for example, may reside on any other computer that is accessible via a telecommunication medium such as a computer network.

[0082] The form completion program examines each template file in order to determine if one or more of the template files resembles the form image to within a certain threshold. For example, if a percentage of the template file is similar to the form image then the form completion program may assume that a match occurred. In one embodiment of the invention, the threshold percentage is thirty percent. However, the present invention contemplates the use of any percentage indicative of a match. For example, in some instances a threshold percentage lower than thirty percent indicates a match whereas in other instances a percentage greater than thirty percent indicates a match. The percentage chosen is largely a result of how much the form has changed since the template file was created. If, for example, a form has added new graphics, but has substantially the same text as when the template was created, then the template file will resemble the form image to a greater percent than it would if the text or number of data receptacles on the form were changed.

[0083] If a template file that resembles the form image to within a certain threshold is located, then the form completion program proceeds to step 517 where it acknowledges that a match occurred. If no match is found the form completion program informs the user that the data population command was not successfully completed. At this point the program may execute step 519 and exit. Otherwise, the form completion program executes step 518 where the information obtained from the template file is utilized to populate the form.

[0084] The form completion program utilizes the template file to identify what kind of data to insert into each of the form's data receptacles. For example, the template file allows the form completion program to determine which of the data receptacles contain personal information and which data receptacles contain payment information. Once the form completion program successfully identifies what kind of data to insert into each data receptacle the program begins to input the appropriate kind of data into the appropriate data receptacle. The user's name, for example, is placed in all data receptacles that expect such information and the user's credit card number is placed in all data receptacles that expect credit card information.

[0085] The present invention contemplates the use of multiple techniques to insert information into the data receptacles. The technique used is largely dependent upon the type of target application used to display the form. If, for example, the form is displayed using the Internet Explorer web browser the form completion program populates the data receptacles by sending a command to the browser that tells the browser which string belongs in which data receptacle (e.g. insert name into data receptacle 1). If the form is displayed with the Netscape web browser, the technique for inserting information into the data receptacles differs. When Netscape is used, data is inserted by grabbing standard input and emulating keystrokes. Each keystroke inserts a character of information into the appropriate data receptacle. To move from one data receptacle to the next the set focus system message is sent (WM_SETFOCUS). However, the invention contemplates using other methods such as emulating the tab key. Once information is properly inserted into the appropriate data receptacles, the data population command is complete.

The Form Completion Program:

[0086] In one embodiment of the invention, the form completion program uses a registry with the operating system (OS) to assure proper execution. The registry contains multiple fields. Each field contains information that may be utilized by the form completion program during execution. An example of the kind of fields that may be placed in the registry follows.

[0087]

	Field Name	Description
	InstallPath	Contains the path of installed program
	UpdateFile	Contains the name and path to copy to be deleted
	LastUser	Contains the name of last user
	Coordinate	Contains the horizontal coordinate (in pixel), of
	X	the program window
	Coordinate	Contains the vertical coordinate (in pixel), of the
	Y	program window
	Server URL	Contains URL(s) where update routine should
		check for a new version of the program

[0088] When the form completion program is installed, a directory structure is created that contains files used by the form completion program during execution. This directory contains the form completion program and two sub-directories: “user” and “data”. The “user” sub-directory contains as many sub-directories as there are users. Each sub-directory contains one file called by the login name of the user, followed by the extension “.dat”. This file contains encrypted data about the user (e.g. address, credit card information, electronic money, etc.,). The “data” sub-directory contains general information utilized by the form completion program during execution. For example, graphical images that represent various kinds of data, web site information, and other information useful to the form completion program is placed in the “data” sub-directory. The information placed in the “data” sub-directory is organized in a tree of one or more additional sub-directories.

[0089] The “domain” sub-directory, for example, is located within the “data” sub-directory. In one embodiment of the invention, the “domain” sub-directory contains a database of information that is used by the form completion program to populate a form. For example, the database may contain a list of web sites (e.g. domain names) having forms that can be populated. Template files associated with each domain name are also placed in the database.

[0090] The database contains sub-directories listing each of the top-level domain names (e.g. .com, .net, .org, etc.). Each sub-directory of top-level domain names contains a tree of sub-directories organized in alphabetical order. When the form completion program initially launches, a startup routine begins to execute. The startup routine performs a number of checks to ensure that the program is running properly. The form completion program then launches a separate process, called a thread, to determine if the form completion program that is running is the most current version available. The process determines if the program is current by contacting a server. The server responds by transmitting an update signal to the form completion program that indicates whether or not the program is indeed out of date. If the form completion program is out of date, the program starts the download-update process.

[0091] After running the start-up checks the form completion program enters a listening mode. While in listening mode the program listens for any action that occurs within the display space of the form completion program, or hot keys. For example, the program listens for actions received from the operating system (e.g. mouse pointer selection, keyboard key stroke, voice commands, etc.). These actions are handled by the application action listener which generates commands according to an action-to-command map.

[0092] FIG. 6 illustrates some of the actions the form completion program listens for when it is in the listening mode. When an action is triggered that is bound to a command (e.g. those being listened for), the form completion program runs the segment of code responsible for performing the corresponding action. If, for example, the form completion program is in listening mode 600 and the user initiates a login action 601, then the program performs a login operation 602. In one embodiment of the invention, the login operation ensures that users are properly authenticated before being allowed access to sensitive data. For example, users are not permitted to access personal information such as a credit card number without first being authenticated. A user is authenticated when the user enters the correct login and password information.

[0093] However, in one embodiment of the invention, the user's password information is not stored by the form completion program. To determine if the user's login and password information is correct, the form completion program determines if the user has logged in before by examining whether a directory having the name entered by the user as a login parameter exists. If the user has not previously logged in, the user is asked to provide a login name and a password. If the user has previously logged in (e.g. the user's directory exists), then the program generates a 32-bit number using a CRC Algorithm. A random number key is generated using the CRC output as a seed. If the correct random number is generated, the user is authenticated. At this point, the user is permitted to access the user's sensitive data. This technique is beneficial because it enables the form completion program to authenticate a user without having to permanently store the user's password information. The present invention also contemplates the use of various authentication schemes that are known in the art and is not limited to the process discussed herein.

[0094] In one embodiment of the invention, after the login operation is performed the form completion program proceeds to step 603 where it determines if the user is a new user. If the user is new, the form completion program advances to step 604 where it shows the user a data collection interface. Once the user enters data into the data collection interface, the program executes step 605 where it displays a shopping view to the user. The shopping view is comprised of one or more graphical representations of the user's sensitive information.

[0095] It is possible for other types of actions to transpire while the form completion program is in listening mode. For example, the user might wish to perform a shopping action 606. When a shopping action 606 occurs the form completion program performs a shopping operation 607. In one embodiment of the invention, a shopping action occurs when the user selects an image indicative of a particular shopping destination. For example, if the form completion program displays a list of web sites to the user, a shopping action 606 occurs when the user selects one of the web sites on the list. When the user initiates the shopping action 606 it causes the form completion program to perform step 607 where a web browser window is displayed to the user containing the web page the user selected from the list.

[0096] When a search action 608 occurs, the form completion program performs a search 609. For example, in one embodiment of the invention the form completion program contains a location for the user to enter a search query and a way to identify a preferred search engine. If the user enters a search query and submits the query a search 609 is performed. The form completion program responds by passing the search parameter entered by the user to the preferred search engine identified by the user. The results of the search are displayed inside of a web browser window that runs external to the form completion program.

[0097] If an update action 610 occurs, the form completion program performs an update 611. During an update operation 611 the form completion program checks to see if the version of the program currently executing is the most recent version available. In one embodiment of the invention, the update operation 611 queries a server via a network to determine the latest available version is. If the form completion program currently executing is in need of a version update, the program starts an update thread to handle the update process. When the execution of the update process is complete, the form completion program running is the most recent version. The update operation 611, for example, may be utilized to add functionality to the form completion program.

[0098] The form completion program may be configured to listen for actions other than the ones identified in FIG. 6. For example, the user may customize the program to listen for a specific type of action. The present invention also contemplates action added by the update operation 611.

[0099] When a FillOut Information action 612 is performed the form completion program executes step 613. At step 613, the user is prompted for information. In one embodiment of the invention, the user is shown a data collection interface that provides locations for the user to enter a particular type of information. For example, the user may be prompted for personal information, payment information, and/or any other type of information deemed necessary to populate a form. This information may be associated with an image file, text file, Multimedia file, or other representation of the data entered into the data collection interface.

[0100] If a FillOut Form action 614 occurs, the form completion program executes a data population command 615. The data population command 615 provides the user with a mechanism for populating a form. When the user elects to stop running the form completion program, a quit action 616 occurs. When a quit action 616 executes the form completion program returns to an inactive state.

[0101] Referring now to FIG. 7, an example of several of the interfaces presented to the user by the form completion program is shown. When the user initially opens the form completion program, a login interface 715 is displayed which provides the user with locations for a user name 711 and password 712. In one embodiment of the invention, a login action 601 occurs when the user enters the appropriate information into the login interface 715 and presses the return key.

[0102] If the user is properly authenticated the form completion program displays a data collection interface 720. The data collection interface 720 contains one or more fields 705–709 for entering data. When the user enters data into these fields 705–709 the data is stored for later use during a form population command. There are several different types of data collection interfaces 720. Each type is used to obtain a different data set from the user. For example, the user is presented one interface for entering payment information and another interface for entering personal information.

[0103] If, for example, the data collection interface 720 intended for collecting payment information is displayed, the user is provided with one or more fields 705–709 for entering data. Each field is intended to store a different type of information. For example, field 705 is used to collect the form of payment (e.g. American Express) the user wishes to utilize for later populating a form. Fields 706–709 provides a place for the user to enter information related to the form of payment identified. The name of the issuing bank is entered into field 706. The account number of the identified card is entered into field 707 and the expiration date and name on the card are entered into fields 708 and 709.

[0104] Once the user enters such information into fields 705–709 it is stored by the form completion program for later use. In one embodiment of the invention, the data collected via the data collection interface is stored on the computer having the form completion program in encrypted form and protected from unauthorized users by an authentication mechanism. For example, data about a particular user may be stored in a directory bearing the user's login name. Before access is granted to the information in that directory the user is prompted for a name and password. If the appropriate user name and password is entered, then access is granted to the specified user's data. When the correct information is not entered access is denied. This permits multiple users to make use of the form completion program and prevents unauthorized users from inappropriately obtaining access to another users data.

[0105] In one embodiment of the invention, each collection of data entered into the data collection interface 720 is associated with a representation of that data. The representation may be in the form of image data, text data, multimedia data, and/or any other kind of data. For example, if data collection interface 720 prompts the user for payment information, a graphic representing that payment information (e.g. a Visa Card graphic) will be displayed to the user.

[0106] For example, the shopping interface 730 displays numerous representations 701–702 of each data set entered by the user via the data collection interface 720. Each representation 701–702 may represent a different data set. For example, representation 701 might represent a form of payment whereas representation 702 might represent a user's personal information. Additional representations may be displayed when the user scrolls either up or down using command buttons 731 and 732.

[0107] In one embodiment of the invention, a portion of the shopping interface 730 is dedicated to displaying a list 710 of web sites where the user can purchase goods and/or services. This list 710 can be changed and modified by the update action 610. The list 710 may be organized in any manner preferred by the user. In one embodiment of the invention, the list 710 is divided into categories. When the user selects a link to a particular web site from list 710, the form completion program passes the URL information identified by the users action to a web browser program. The browser program, for example, accesses the web site identified in the URL to obtain the web page identified in the URL. If after perusing through the web site the user needs to complete a form to purchase an item, the user may use the form completion program to populate the form.

[0108] To illustrate, when the user encounters a form (e.g. after shopping), the user may elect to populate the data receptacles 751–755 of that form with the information associated with a particular representative graphic. If, for example, the user wishes to fill-out form 750 with payment information, the user may cause a data population command to execute by utilizing a pointing device to drag representation 701 along path 725 until the pointing device resides over form 750. When the pointing device is located over form 750 the data population command executes. The data population command causes data receptacles 751–755 to fill with data.

[0109] In one embodiment of the invention, the user is not required to drag representation 701 over to form 750, but can instead drag an iconosized version of representation 701 to form 750 in order to initiate a data population command. The iconosized version of representation 701 is a smaller image of graphical representation 701 that requires less processor overhead to rapidly move about the display region. The invention also contemplates the use of other mechanisms to initiate a data population command. The form completion program can, for example, be configured to perform a data population command whenever the user performs an event that discernibly indicates the user wishes to populate a form (e.g. a click event or a menu event).

Embodiment of Computer Execution Environment (Hardware)

[0110] An embodiment of the invention can be implemented as computer software in the form of computer readable code executed on a general purpose computer such as computer 800 illustrated in FIG. 8, or in the form of bytecode class files executable within a Java™ runtime environment running on such a computer, or in the form of bytecodes running on a processor (or devices enabled to process bytecodes) existing in a distributed environment (e.g., one or more processors on a network). A keyboard 810 and mouse 811 are coupled to a system bus 818. The keyboard and mouse are for introducing user input to the computer system and communicating that user input to processor 813. Other suitable input devices may be used in addition to, or in place of, the mouse 811 and keyboard 810. I/O (input/output) unit 819 coupled to system bus 818 represents such I/O elements as a printer, A/V (audio/video) I/O, etc.

[0111] Computer 800 includes a video memory 814, main memory 815 and mass storage 814, all coupled to system bus 818 along with keyboard 810, mouse 811 and processor 813. The mass storage 814 may include both fixed and removable media, such as magnetic, optical or magnetic optical storage systems or any other available mass storage technology. Bus 818 may contain, for example, thirty-two address lines for addressing video memory 814 or main memory 815. The system bus 818 also includes, for example, a 64-bit data bus for transferring data between and among the components, such as processor 813, main memory 815, video memory 814 and mass storage 814. Alternatively, multiplex data/address lines may be used instead of separate data and address lines.

[0112] In one embodiment of the invention, the processor 813 is a microprocessor manufactured by Motorola, such as the 680X0 processor, or a microprocessor manufactured by Intel, such as the 80X86, or Pentium processor. However, any other suitable microprocessor or microcomputer may be utilized. Main memory 815 is comprised of dynamic random access memory (DRAM). Video memory 814 is a dual-ported video random access memory. One port of the video memory 814 is coupled to video amplifier 816. The video amplifier 816 is used to drive the cathode ray tube (CRT) raster monitor 817. Video amplifier 816 is well known in the art and may be implemented by any suitable apparatus. This circuitry converts pixel data stored in video memory 814 to a raster signal suitable for use by monitor 817. Monitor 817 is a type of monitor suitable for displaying graphic images.

[0113] Computer 800 may also include a communication interface 840 coupled to bus 818. Communication interface 840 provides a two-way data communication coupling via a network link 841 to a local network 844. For example, if communication interface 840 is an integrated services digital network (ISDN) card or a modem, communication interface 840 provides a data communication connection to the corresponding type of telephone line, which comprises part of network link 841. If communication interface 840 is a local area network (LAN) card, communication interface 840 provides a data communication connection via network link 841 to a compatible LAN. Wireless links are also possible. In any such implementation, communication interface 840 sends and receives electrical, electromagnetic or optical signals which carry digital data streams representing various types of information.

[0114] Network link 841 typically provides data communication through one or more networks to other data devices. For example, network link 841 may provide a connection through local network 844 to local server computer 843 or to data equipment operated by an Internet Service Provider (ISP) 844. ISP 844 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the “Internet” 845. Local network 844 and Internet 845 both use electrical, electromagnetic or optical signals which carry digital data streams. The signals through the various networks and the signals on network link 841 and through communication interface 840, which carry the digital data to and from computer 800, are exemplary forms of carrier waves transporting the information.

[0115] Computer 800 can send messages and receive data, including program code, through the network(s), network link 841, and communication interface 840. In the Internet example, remote server computer 846 might transmit a requested code for an application program through Internet 845, ISP 844, local network 844 and communication interface 840.

[0116] The received code may be executed by processor 813 as it is received, and/or stored in mass storage 814, or other non-volatile storage for later execution. In this manner, computer 800 may obtain application code in the form of a carrier wave.

[0117] Application code may be embodied in any form of computer program product. A computer program product comprises a medium configured to store or transport computer readable code, or in which computer readable code may be embedded. Some examples of computer program products are CD-ROM disks, ROM cards, floppy disks, magnetic tapes, computer hard drives, servers on a network, and carrier waves.

[0118] The computer systems described above are for purposes of example only. An embodiment of the invention may be implemented in any type of computer system or programming or processing environment.

[0119] Thus, a method and apparatus for populating forms is described.