Friendly URLs

[0001] The present invention generally relates to accessing dynamically generated data with a static universal resource locator (URL), and more specifically, pertains to accessing data that are dynamically generated with one or more query parameters determined from elements of a static URL, instead of including the query parameters in a conventional URL.

[0002] Searching for information is one of the most prevalent activities on the Internet. Individuals increasingly use search engines that show lists of web sites potentially related to topics of interest. However, finding relevant information is difficult, because it is difficult for individual users and conventional search engines to identify relevant content among the enormous volume of accessible data. Further, most substantive content is not readily accessible, because the content is stored in databases that are used to dynamically generate documents and other data instead of being stored as static files. To access database content, a query string is typically specified in a URL that also identifies a network address of the database, so that the specific database to be searched for the desired content is identified in the query. URLs with query strings frequently become very long and complex. As the complexity of database driven Web sites has increased, it has become highly unlikely that a typical user will correctly type a URL and a query string to access content of possible interest on even a known Web site.

[0003] Similarly, secondary users of the Internet, such as conventional search engines and dynamic web crawlers, are often set up to avoid links that include URLs with a query string. Consequently, a large amount of the most valuable and substantive information that is available is not reported. To counteract this problem, some Web sites have been designed specifically for Web crawlers and search engines instead of for individual users. To ensure that their content is indeed identified in search results, such Web sites may use “cloaked URLs” to present different Web pages to Web crawlers than they do to individuals using browser programs to explore the Web sites. These Web sites detect when a request is from a Web crawler and redirect the request to a Web page that is specifically designed to maximize a search ranking in the reported search results, rather than to provide the page that a user would view in a browser. However, many search engine operators disapprove of this practice of cloaking URLs. Search engine operators try to ensure that their search engines return lists of Web pages that would actually be seen by their customers, if the customers were browsing the Web sites represented in the lists. Search engines are thus frequently set up to consider cloaked URLs as a form of spam and to make special efforts to eliminate cloaked URLs from their search results.

[0004] Another approach used to identify and access information on database driven Web sites evaluates a user's keywords, and attempts to match those keywords to a relevant search engine that is closely tied to one or a few database driven Web sites. This approach is sometimes discussed in terms of mining the “deep Web” or the “invisible Web.” Based on the keywords, a query string is created, which is syntactically valid for a selected search engine that is closely tied to one or more relevant databases. This preprocessing approach of selecting relevant Web sites and creating custom queries for their corresponding database(s) provides an alternative to using a conventional, “all-purpose” search engine, such as GOOGLE™, ALTAVISTA™, etc. However, it would be preferable to utilize these conventional, all-purpose search engines to find the desired content in databases on the Web, because the conventional search engines are so widely used. Thus, it is preferable to provide a way for Web site operators to make their database content accessible and searchable using conventional search engines and Web crawlers. It is further desirable to enable users to search database content directly from a browser when the user already knows one or more database driven Web sites that likely contain relevant information.

[0005] The present invention provides a method and system for enabling a Web crawler, search engine, browser or other client application to access data by using “friendly” URLs that do not require conventional query parameters or other non-intuitive coding, and which are dynamically obtained and/or created. Such URLs are friendly because they are easy to read and understand by humans and are in a form that is readily searchable by conventional search engines, Web crawlers, browsers, and other client applications. A friendly URL comprises a static URL that appears to identify a location and name of a static resource such as a conventional hypertext markup language (HTML) document. A friendly URL does not include query strings, and preferably does not directly refer to any active server page (ASP) script, common gateway interface (CGI) script, or other executable module. The present invention enables a friendly URL to be mapped to a data source, such as a database driven Web site, and causes the data source to dynamically access or generate data associated with an easily understood data key provided in the friendly URL. The data key can refer to specific information known to be available from the data source, and/or indicate a keyword that can be used to search the data source for related information of possible interest.

[0006] A friendly URL can be incorporated into a link in a source document that is accessible to a conventional search engine, Web crawler, browser, or other client application. Alternatively, a friendly URL can be entered by a user into an address data field of a browser or other communication application. The friendly URL is communicated to a server or other communication module that incorporates the present invention and that is in communication with a desired data source. The communication module receives the friendly URL just as it would any other static URL, but does not immediately attempt to access the static resource indicated in the friendly URL. Instead, the friendly URL is parsed to determine whether the friendly URL includes a data type and a data key at predefined locations within the URL string. The data type preferably refers to a text data source, an image data source, a media data source, or other data source that is dynamically accessible. The data key can comprise a specific data identifier or a search term. If a recognized data type is found in the received friendly URL, a query URL is constructed with a data source identifier that is mapped to the data type. The data source identifier preferably refers to a script or other function that can dynamically access data from the data source. This internal mapping of friendly URLs to data sources enables data providers to change the mappings whenever internal data locations change and does not require updating links to the data on existing Web pages. The internal mapping also provides a layer of security by not directly disclosing the location, data access function, and other details to the client application and/or user, as occurs when using a real URL. The data key is also incorporated into the query URL as a query parameter with any other code characters needed by the data source to dynamically access and/or generate data associated with the data key. The query URL can be communicated directly to the data source or internally redirected to the data source. In any case, the resulting dynamically accessed data are then communicated back to the requesting client application, but are identified only by reference to the original friendly URL.

[0007] Another aspect of the present invention is directed to a memory medium that stores machine instructions for carrying out the steps described above and as discussed in further detail below.

Exemplary System for Implementing Present Invention

[0015] FIG. 1 illustrates an exemplary system suitable for implementing various portions of the present invention, including executing server software or executing a Web browser such as Microsoft Corporation's INTERNET EXPLORER™ for accessing Web pages over the Internet. The system includes a general purpose computing device in the form of a conventional PC 20, provided with a processing unit 21, a system memory 22, and a system bus 23. The system bus couples various system components including the system memory to processing unit 21 and may be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. The system memory includes read only memory (ROM) 24 and random access memory (RAM) 25. A basic input/output system 26 (BIOS), containing the basic routines that help to transfer information between elements within the PC 20, such as during start up, is stored in ROM 24. PC 20 further includes a hard disk drive 27 for reading from and writing to a hard disk (not shown), a magnetic disk drive 28 for reading from or writing to a removable magnetic disk 29, and an optical disc drive 30 for reading from or writing to a removable optical disc 31, such as a compact disc-read only memory (CD-ROM) or other optical media. Hard disk drive 27, magnetic disk drive 28, and optical disc drive 30 are connected to system bus 23 by a hard disk drive interface 32, a magnetic disk drive interface 33, and an optical disc drive interface 34, respectively. The drives and their associated computer readable media provide nonvolatile storage of computer readable machine instructions, data structures, program modules, and other data for PC 20. Although the exemplary environment described herein employs a hard disk, removable magnetic disk 29, and removable optical disc 31, it will be appreciated by those skilled in the art that other types of computer readable media, which can store data and machine instructions that are accessible by a computer, such as magnetic cassettes, flash memory cards, digital video disks (DVDs), Bernoulli cartridges, RAMs, ROMs, and the like, may also be used in the exemplary operating environment.

[0016] A number of program modules may be stored on the hard disk, magnetic disk 29, optical disc 31, ROM 24, or RAM 25, including an operating system 35, one or more application programs 36, other program modules 37, and program data 38. A user may enter commands and information in to PC 20, and provide control input through input devices such as a keyboard 40 and a pointing device 42. Pointing device 42 may include a mouse, stylus, wireless remote control, or other pointer. As used hereinafter, the term “mouse” is intended to encompass virtually any pointing device that is useful for controlling the position of a cursor on the screen. Other input devices (not shown) may include a microphone, joystick, haptic joystick, yoke, foot pedals, game pad, satellite dish, scanner, or the like. These and other input/output (I/O) devices are often connected to processing unit 21 through an I/O interface 46 that is coupled to system bus 23. The term I/O interface is intended to encompass each interface specifically used for a serial port, a parallel port, a game port, a keyboard port, and/or a universal serial bus (USB). A monitor 47 or other type of display device is also connected to system bus 23 via an appropriate interface, such as a video adapter 48. In addition to the monitor, PCs are often coupled to other peripheral output devices (not shown), such as speakers (through a sound card or other audio interface—not shown), and printers.

[0017] The present invention may be practiced on a single machine, however, PC 20 can also operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 49. Remote computer 49 may be another PC, a server (which is typically generally configured much like PC 20), a router, a network PC, a peer device, or a satellite or other common network node, and typically includes many or all of the elements described above in connection with PC 20, although only an external memory storage device 50 has been illustrated in FIG. 1. The logical connections depicted in FIG. 1 include a local area network (LAN) 51 and a wide area network (WAN) 52. Such networking environments are common in offices, enterprise wide computer networks, intranets, and the Internet.

[0018] When used in a LAN networking environment, PC 20 is connected to LAN 51 through a network interface or adapter 53. When used in a WAN networking environment, PC 20 often will typically include a modem 54, or other means such as a cable modem, Digital Subscriber Line (DSL) interface, or an Integrated Service Digital Network (ISDN) interface for establishing communications over WAN 52, such as the Internet. Modem 54, which may be internal or external, is connected to system bus 23 or coupled to the bus via I/O device interface 46, i.e., through a serial port. PC 20 may also communicate over a WAN through comparable network interfaces that are provided on the LAN to which the PC is coupled. In a networked environment, program modules depicted relative to PC 20, or portions thereof, may be stored in the remote memory storage device. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used, such as wireless communication and wide band network links.

Request Specific Dynamic Data with Friendly URL

[0019] FIG. 2A comprises a block diagram 100 of software modules that illustrate, in an exemplary manner, the processes involved in handling a request for a specific dynamically generated document using a friendly URL. Block diagram 100 shows a browser or Web crawler 102 accessing a source Web page 104 that includes a hyperlink specifying a friendly URL 106. URL 106 is considered friendly because the URL does not include cryptic query parameters and/or query delimiters. Instead, friendly URL 106 is easily human readable because it clearly specifies a static hypertext markup language (HTML) document from a recognizable data source, such as an online encyclopedia. If a user selects the hyperlink specifying friendly URL 106 that is displayed in the browser program, or if a Web crawler accesses the hyperlink, friendly URL 106 is incorporated into a request message 108 that is sent to a computing platform executing server communication software 110. An example of such software is Microsoft Corporation's Internet Information Services (IIS) module.

[0020] As in the example of FIG. 2A, request message 108 preferably comprises a hypertext transfer protocol message to a known domain name and known directory such as the root directory of Microsoft Corporation's “encarta.msn.com/” domain. Friendly URL 106 further indicates a data type, such as “encyclopedia,” and a data key, such as “1861595373.” The data type and data key are separated by a non-query separator such as an underscore character (“_”). The data type preferably refers to the form of the requested information—in this case, an encyclopedia article. Those of ordinary skill in the art will recognize that the data type could be a picture file, a video file, an audio file, or almost any kind of electronically transmittable file. The data key preferably refers to a specific document or other item of information that is to be dynamically generated from the storage being accessed. The document name “camel.html” indicates the topic of the dynamically generated data, but is expressed in static form with the “.html” extension.

[0021] An intercept module 112 of server communication software 110 recognizes the “.html” extension and stores the friendly URL. Intercept module 112 passes received friendly URL 114 to an application extension 116, such as an Internet Server Application Programming Interface (ISAPI) Extension. The application extension evaluates the friendly URL and generates an internal real URL 118a. Internal real URL 118a specifies the query parameters and/or other coding needed to dynamically access or generate the desired encyclopedia article or other information. In general, the internal real URL is in a form that contains a source identifier, a data key, and a query indicator. In the example of FIG. 2A, “/encnet/refpages/RefArticle.aspx” of internal real URL 118a comprises a source identifier that specifies a source of the desired dynamically generated information. In this case, the source identifier refers to an internal directory location and name of a script that will construct the encyclopedia article as a web page. This source identifier is mapped from the data type “encyclopedia” received in the friendly URL. Because the internal directory location and script name are not included in the original friendly URL, there is an added level of security introduced by using a friendly URL. The data key “1861595373” is also incorporated into the internal real URL for use by the script. As a conventional query string, the internal real URL includes any necessary query indicators, such as a question mark “?.”

[0022] Application extension 116 communicates internal real URL 118a to an internal control transfer module 120, which communicates a redirected internal real URL 118b to a database lookup script 124. The database lookup script retrieves data associated with the data key and/or dynamically generates a Web page, and passes results data 126 to a return module 128 controlled by the IIS server software. Return module 128 creates a return message 130 comprising the dynamically generated Web page (or other data) and friendly URL 106 that was previously stored and then communicates return message 130 to the requesting browser or Web crawler.

Requesting a Search with the Friendly URL

[0023] FIG. 2B is a block diagram 140 of software modules and illustrates, in an exemplary manner, the processes involved in handling a search request made with a friendly URL. Through the address bar of a browser program 142, a user can enter a search request in the form of a friendly URL 144. The friendly URL is incorporated into a search request message 146 that the browser program sends to server communication software 110, which is executed by a server computing platform. As in the hyperlink example of FIG. 2A, the friendly URL in search request message 146 of FIG. 2B includes a data type, a separator, and a data key. However, in this search example, the data key specifies a search term rather than referring to a particular dynamically generated Web page or other particular dynamic data.

[0024] Also, similar to the hyperlink example, search request message 146 conforms to the hypertext transfer protocol and specifies a known domain name and known directory (e.g., “encarta.msn.com/”). Search request message 146 also includes the data type “encyclopedia” and a separator “/” (i.e., a forward slash). Again, the data type preferably refers to the form of the requested information, such as an encyclopedia article. The server software uses the separator to distinguish between the data type and data key and can comprise any predefined character other than a query character. The data key in this search example is the name of a static HTML file “camel” that can be easily specified by a user, rather than a reference number that is predefined in a hyperlink. The “.html” extension again indicates a simple static document, but the server software is designed to use the data type and data key of the static friendly URL to access and/or generate the dynamic data.

[0025] The remaining processing is very similar to the hyperlink example; however, a keyword search is performed rather than a direct dynamic access based on a predefined reference identifier. More than one search result may also be returned. In further detail, intercept module 112 of server communication software 110 recognizes the “.html” extension and stores the friendly URL. Intercept module 112 passes a received friendly URL 148 to application extension 116, which evaluates the received friendly URL and generates an internal real URL 119a. Internal real URL 119a specifies the query parameters and/or other coding needed to perform the keyword search. In general, the internal real URL is still in a form that contains a source identifier, a data key, and a query indicator. In particular, “/encnet/refpages/RefArticle.aspx” of internal real URL 119a still comprises the source identifier that specifies the source to be searched. In this case, the source identifier refers to an internal directory location and name of a script that will construct a search for one or more encyclopedia articles that will be generated based on the data key received. The data key “camel” is incorporated into the internal real URL for use by the script.

[0026] Application extension 116 communicates internal real URL 119a to internal control transfer module 120, which communicates a redirected internal real URL 119b to database lookup script 124. The database lookup script performs a search based on the data key and generates a search results page, encyclopedia article, or other appropriate search results. The database lookup script then passes search results data 127 to return module 128, which is controlled by the IIS server software. Return module 128 creates a return message 132 comprising the dynamically generated search results and friendly URL 144 that was previously stored. Return module 128 then communicates return message 132 to the requesting browser.

Providing Dynamically Accessed Data in Response to a Request

[0027] FIG. 3 is a flow diagram illustrating the process of providing dynamically accessed data to a browser program or Web crawler, based on a friendly URL. As discussed above, the intercept module of the IIS receives and parses the URL of a request message at a step 154. In a decision step 156, the IIS determines whether the URL includes a predefined filename extension, such as .html. If the URL does not include the predefined filename extension, the URL is not a friendly URL. Accordingly, the IIS routes the URL to an appropriate script at a step 158, which generates and returns a standard results page. However, if the URL does include a predefined filename extension, such as “.html”, then the URL is a friendly URL. In that case, the IIS stores the URL, at a step 162, for later use with a return message. The IIS then passes the URL to the ISAPI Extension and also transfers control to the ISAPI extension, at a step 164.

[0028] In a step 166, the ISAPI extension generates the internal real URL and returns the internal real URL to the control transfer module of the IIS. As discussed in further detail below with regard to FIG. 4, the ISAPI extension uses the data type and data key of the friendly URL to construct an internal URL that contains a location, a script name, and query parameters. In a step 168, the control transfer module of the IIS redirects the internal real URL to the script defined in the internal real URL. This internal redirection process can be implemented with a child request of the original request. For instance, the ISAPI extension can call an HSE_REQ_EXEC_URL function of IIS version 6.0 to rewrite the received URL request to the script mapped to the data type. In a step 170, the script accesses the desired data associated with the data key and/or generates a results page. As noted above, the desired data are typically stored in a database and are readily accessible when accessed using scripts that are controlled by the sever software. The script passes the results data to the return module of the IIS. In a step 172, the return module of the IIS accesses the friendly URL that was stored at step 162 when the request message was received by the intercept module of the IIS. The return module constructs a return message with the friendly URL as a header and incorporates the results data (e.g., dynamically generated Web page, search results, etc.) into the return message. In a step 174, the IIS sends the return message to the requesting browser program or Web crawler. Thus, the requesting entity receives the requested information using an easily readable URL that does not include any query parameters. Also, by using the friendly URL in the return message, details of the internal real URL are not discoverable by the requesting browser program or Web crawler.

Constructing a URL for Internal Processing

[0029] Further detail is now provided regarding steps employed for generating a real URL from a friendly URL as indicated in step 166 above. FIG. 4 is a flow diagram illustrating a method for constructing a corresponding internal URL for internal processing. At a step 184, the ISAPI extension receives the friendly URL from the IIS and parses it. In a decision step 186, the ISAPI extension examines the friendly URL to determine whether it includes a component followed by a predefined separator, such as an underscore or slash. If a separator component is present, the ISAPI extension determines whether the component matches a known data type. A known data type can include a text encyclopedia article, a dictionary entry, an image, a multimedia presentation, or other digital file. If a known data type is not found or an appropriate separator is not identified, the ISAPI extension uses a default mapping algorithm to construct the internal real URL at a step 188. The default mapping algorithm preferably processes conventional static URLs and/or other URLs that do not include a known data type.

[0030] However, if a known data type is found, the ISAPI extension uses the data type, at a step 192, to ascertain the directory location and the script required to dynamically access and/or generate the requested information. Table 1, which is shown in FIG. 5, illustrates some of the data types and corresponding paths to data source scripts used to dynamically access and/or generate data from an online encyclopedia Web site. After detecting a separator, the ISAPI extension further analyzes the friendly URL in a step 194, to identify a data key and any other parameters expected to be associated with the data type. These parameters are sometimes referred to as tokens. Those skilled in the art will recognize that different separator characters can also be used to help identify different data keys and/or associated parameters. The separator can also be used to help distinguish between a friendly URL that identifies a specific data item (e.g., a specific Web page) and a friendly URL that indicates a keyword search.

[0031] In a step 196, the ISAPI extension constructs an internal real URL by replacing the data type of the friendly URL with the directory location and script name of the corresponding data source. The ISAPI extension also replaces the separator of the friendly URL with the appropriate query character(s) and places the data key and token(s) in the appropriate locations to complete a valid query string for use in the internal real URL. In a step 198, the ISAPI extension returns the internal real URL to the IIS software for internally redirecting to the data source script.

[0032] Although the present invention has been described in connection with the preferred form of practicing it and modifications thereto, those of ordinary skill in the art will understand that many other modifications can be made to the present invention within the scope of the claims that follow. Accordingly, it is not intended that the scope of the invention in any way be limited by the above description, but instead be determined entirely by reference to the claims that follow.