System and method for locating on electronic documents items referenced in a physical document

[0001] This application claims foreign priority benefits under 35 U.S.C. §119(a)–(d) of European Patent Application No. 02368026.7, filed Mar. 21, 2002.

[0002] The present invention relates to interactive hypermedia systems and more particularly to a method, system and computer program for locating on electronic documents items referenced in a physical document.

[0003] The Internet is a global network of computers and computer networks (the “Net”). The Internet connects computers that use a variety of different operating systems or languages, including UNIX, DOS, Windows, Macintosh, and others. To enable communication among these various systems and languages, the Internet uses a language referred to as TCP/IP (“Transmission Control Protocol/Internet Protocol”). TCP/IP protocol supports three basic applications on the Internet: transmitting and receiving electronic mail, logging into remote computers (the “Telnet”), and transferring files and programs from one computer to another (“FTP” or “File Transfer Protocol”).

[0004] With the increasing size and complexity of the Internet, tools have been developed to help find information on the network. These tools are often called navigators or navigation systems. Navigation systems that have been developed include standards such as Archie, Gopher and WAIS. The World Wide Web (“WWW” or “the Web”) is a recent and superior navigation system. The Web is: an Internet-based navigation system, an information distribution and management system for the Internet, and a dynamic format for communicating on the Web.

[0005] The Web seamlessly integrates various kinds of information, including still images, text, audio and video. A Web user with a graphical user interface (“GUI”, pronounced “gooey”) may transparently communicate with different host computers on the system, different system applications (including FTP and Telnet), and different information formats for files and documents including, for example, text, sound and graphics.

[0006] The Web uses hypertext and hypermedia. Hypertext is a subset of hypermedia and refers to computer-based “documents” in which readers move from one place to another within a document, or from one document to another, in a non-linear manner. To do this, the Web uses a client-server architecture. The Web servers enable the user to access hypertext and hypermedia information through the Web and the user's computer. (The user's computer is referred to as a client computer of the Web Server computers.) The clients send requests to the Web Servers, which react, search and respond. The Web allows client application software to request and receive hypermedia documents (including formatted text, audio, video and graphics) with hypertext link capabilities to other hypermedia documents, from a Web file server.

[0007] The Web, then, can be viewed as a collection of document files residing on Web host computers that are interconnected by hyperlinks using networking protocols, forming a virtual “web” that spans the Internet. A resource of the Internet is unambiguously identified by a Uniform Resource Locator (URL), which is a pointer to a particular resource at a particular location. A URL specifies the protocol used to access a server (e.g. HTTP, FTP, and so forth), the domain name of the server, the port address to be used for communication, and the location of a file on that server. Thus, when a user selects a hyperlink, the system uses the URL to establish communication with a server computer designated in the URL and displays a document or content retrieved from the server.

[0008] Each Web page may appear as a complex document that integrates, for example, text, images, sounds and animation. Each such page may also comprise hyperlinks to other Web documents so that a user at a client computer may click on icons with a mouse and thereby activate hyperlink jumps to a new page on the same or a different Web server.

[0009] A Web server is a software program on a Web host computer that answers requests from Web clients, typically over the Internet. All Web servers use a language or protocol to communicate with Web clients called Hyper Text Transfer Protocol (“HTTP”). All types of data can be exchanged among Web servers and clients using this protocol, including Hyper Text Markup Language (“HTML”), graphics, sound and video.

[0010] On the Web, documents are generally created in HTML. HTML describes the layout, contents and hyperlinks of the documents and pages. Each HTML document supports embedded hyperlinks that reference other locations (folder, FTP site, other HTML documents, etc.) allowing the client computer to jump to another location when the user selects a hyperlink by clicking on the hyperlink or pressing an appropriate keystroke.

[0011] HTML allows, by example, to embed graphical images in HTML documents. For embedding a digital image in an HTML document, the dimensions of the embedded digital image can be specified with HTML statements. For example, an embedded graphical image may be defined in HTML by:

• • [0012] <IMG SRC=“image_file.gif” WIDTH=X HEIGHT=Y>
    where:
  • [0013] IMG is an HTML tag that references an embedded image,
  • [0014] SRC=“image_file.gif” specifies the path and the name of the file comprising the digital image embedded in the HTML document, (e.g., a Graphics Information file or *.GIF), and
  • [0015] X,Y are the width and height, respectively, of the digital image measured in pixels.
  • [0016] When browsing, Web clients convert user specified commands into HTTP GET requests, connect to the appropriate Web server to get information, and wait for a response.

[0017] The response from the server can be the requested document or an error message. After the document or an error message is returned, the connection between the Web client and the Web server is closed.

[0018] After receipt, the Web client formats and presents the data. To present the data, the Web client can activate an ancillary application such as a sound player according to the various types of data received. The Web Client is also referred to as the Web Browser, since it browses documents retrieved from the Web Server.

[0019] Interactive electronic services, video-on-demand, and the World Wide Web are providing access to an increasing offering of movies, shopping information, games, multimedia documents, electronic commerce and. many other services. During these last years, due mainly to the general use of personal computers and the capability of millions of users to access the World Wide Web, “multimedia publishing” has veritably exploded. Due to the widespread penetration of CD-ROM drives, an enormous number of multimedia works combining text, images and sounds, are now accessible to owners of personal computers. Furthermore, a vast choice of hypermedia information is today accessible via the Internet on the World Wide Web.

[0020] A major problem in using these multimedia systems is to browse the enormous diversity and quantity of information, to discover what is available, and to make a selection among all the possible choices. For instance, when surfing on the Web, a conventional method of navigating across the pages of hypertext documents consists in using search tools or invoking bookmarked links. When surfing on video-on-demand services, a conventional navigation method is to pass from channel to channel. Advertisements on preview channels are used as entry points to various movies. Users can navigate and make selections using hierarchical menus. Obviously, these navigating means do not enable a simple user to access and browse the thousands of multimedia documents that are available on the Web or interactive TV.

[0021] For all kind of reasons, people more and more often want to be rapidly and easily informed about their environment. They need to have information about the resources or services located in different regions of the world, in rural, industrial or urban areas. This common need was already identified a long time ago by Geographic Information Systems (GIS) providers, cartographic information providers and, more recently, by many Internet information providers.

[0022] These institutions and many others (e.g.: travel agencies, government agencies, local authorities, etc.) today provide geographic and cartographic information in the Web by means of various services. Basically, these services enable the user to access, browse or download many different types of digitized maps covering practically all regions of the world. Recently, due mainly to the widespread use of the Global Positioning System (GPS) and in-vehicle computerized navigation systems, and due to the capability of millions of users to access the World Wide Web, “digital map publishing” has veritably exploded. Due to the widespread use of CD-ROMs, a large number of geographic maps are now available to owners of personal computers. Because Internet users can access GIS (Geographic Information System) applications from their browsers without purchasing proprietary GIS software, WebGIS has the potential to make Distributed Geographic Information (DGI) available to a very large worldwide audience. Due to this rapid evolution, a vast choice of geographic and cartographic information is today accessible by means of Internet, and more particularly by means of the World Wide Web. Furthermore, WebGIS allows to use the functionality of the many GIS applications with a wide range of network-based applications in the field of business, administration, or education.

[0023] On the other hand, people are used to browsing through paper catalogs, magazines, newspapers, and books by flipping through the pages and glancing at pictures and text. Even if the enthusiasm of the public for new computer-based multimedia services can be considered as a threat to the conventional forms of hard-copied publishing, particularly book publishing, the reality is that reading a book cannot be compared with reading electronic media. Even when many electronic systems attempt to replace paper by providing many advantages such as, for example, a better access to multimedia services, reading paper remains today preferable for most people, whether they are familiar with computers or not.

[0024] A Publication entitled “The Last Book”, (IBM Systems Journal, Vol 36, No. 3 Vol 36, No. 3-1997, by J. Jacobson, et al.), clearly illustrates the differences between printed books and computer screens in the following terms:

• • [0025] “A book represents a fundamentally different entity than a computer screen in that it is a physical embodiment of a large number of simultaneous high-resolution displays. When we turn the page, we do not lose the previous page. Through evolution the brain has developed a highly sophisticated spatial map. Persons familiar with a manual or textbook can find information that they are seeking with high specificity, as evidenced by their ability to remember whether something that was seen only briefly was on the right side or left side of a page, for instance. Furthermore their haptic connection with the brain's spatial map comprises a highly natural and effective interface, when such information is embodied on actual multiple physical pages.
  • [0026] Another aspect of embodying information on multiple, simultaneous pages is that of serendipity and comparison. We may leaf through a large volume of text and graphics, inserting a finger bookmark into those areas of greatest interest. Similarly, we may assemble a large body of similar matter in order to view elements in contrast to one another, such as might be done to determine which of a particular set of graphical designs is most satisfying”.

[0027] So even if the enthusiasm of the public for computer-based digital technology might be seen as a threat to conventional paper documents, in reality the use of physical documents remains preferable to most people, whether they are skilled or not in using computers.

[0028] Nevertheless, physical documents lack the many advantages provided by today's computer technology. Thus, there is a need for new ways to enhance the usefulness of physical documents by providing the documents the advantages of computer-based technology.

[0029] To make additional information directly accessible from printed publications, U.S. patent application Ser. No. 09/841,930, entitled “Method and system for accessing interactive multimedia information or services by touching highlighted items on physical documents” discloses a system and method for selecting and accessing multimedia information and/or services located on a user workstation (or on one or a plurality of servers connected to a communication network) simply by touching with a finger, items (words, letters, symbols, pictures, icons, . . . ) that are electronically illuminated over the surface of a hard-copy document (or any other physical surface) by means of an opto-touch foil. The referenced system includes: an opto-touch foil preferably transparent, placed by the user over (or under) the document (or a portion of the document). This opto-touch foil is used: to illuminate and highlight hyperlinked items over the surface of the physical document (or portion of this document), and to read coordinates of these hyperlinked items, an user workstation for accessing and displaying the information and/or the service associated with the hyperlinked items.

[0030] The hyperlinked items are identified by means of a luminous signal (or light spot) generated by the opto-touch foil. The opto-touch foil operates under the control of the user workstation. Illuminated items are selected by pressing the opto-touch foil. When the user selects an item among all illuminated items, the user workstation receives from the opto-touch foil a signal indicating the position of this selected item. The user workstation identifies and locates referring to a hyperlink table the information and/or the service associated with the position of the selected item. If the information and/or service is located in a remote server, a request is sent to this server. If the information and/or the service is stored in the user workstation, then this information and/or service is accessed locally.

[0031] Using the same opto-touch foil, U.S. patent application Ser. No. 09/892,399, entitled “System and method for locating on a physical document items referenced in a electronic document”, discloses a method and system for using the same opto-touch foil for locating on a physical document items referenced in an electronic document. In an illustrative embodiment, this invention enables to highlight on a paper map the geographic position of places referenced in a Web page.

[0032] Also, using opto-touch foils, U.S. patent application Ser. No. 09/923,150, entitled: “System and method for locating on a physical document items referenced in another physical document”, discloses a method and system for creating hyperlinks from items (e.g. words, pictures, foot notes, symbols, icons) on a first physical document to particular points on a second physical document (manuscript or printed document), for activating these hyperlinks simply by touching the first document, and for highlighting, by means of a light emitting source, the position of the items on the second document. In an illustrative embodiment, the invention enables to highlight on a hard-copy map the geographic positions of places referenced in a hard-copy document.

[0033] None of the methods described in the prior art discloses a method or system that would enable a user, simply by touching with a fingertip, an item (i.e., word, icon, figure, foot note, etc.) printed on a physical (i.e., hard-copy) document, to highlight the position of this item (or the location of the information related to this items) on an electronic document.

[0034] Generally, there is a real need to provide new systems and methods for enriching the static information comprised in conventional paper documents.

[0035] There is a need, using items printed on a physical document, to create hyperlinks with electronically stored documents and to automatically identify and locate the, position of the hyperlinked items on the electronic documents.

[0036] In particular, there is a need to provide mobile users with additional information directly accessible from hard-copy documents and more particularly to illuminate on digital maps, places referenced on hard-copy documents.

[0037] It is an object of the invention to combine the advantages of searching and browsing textual information on hard-copy documents with the advantages of displaying graphic elements on electronic documents accessed and retrieved on a network.

[0038] It is another object of the present invention to highlight on a digital (electronic) document, information related to items selected in a physical document (or more generally on a physical surface).

[0039] It is a further object of the present invention to enable a user, when browsing a printed document comprising references to locations in the world (e.g., names of towns, oil posts, motels, buildings, hospitals, monuments, etc.), to receive and display a digital map and to show the position of the locations illuminated over the digital map, even when the locations are not initially represented on it.

[0040] The present invention generally relates to interactive hypermedia systems and more particularly to a method, system and computer program for locating on electronic documents items referenced on physical documents. More particularly, the present invention discloses a method, system and computer program for creating hyperlinks from one or a plurality of items (e.g. words, pictures, foot notes, symbols, icons) on a physical document to electronic documents and for activating anyone of these hyperlinks simply by touching a selected item on the physical document, for accessing, retrieving and displaying the hyperlinked electronic document and for pointing to the position (or positions) of the selected item on the displayed electronic document. In a particular embodiment, the present invention discloses a method, system and computer program for locating on digital images (e.g., digital maps) the positions of items (e.g., locations names) referenced in a hard-copy document.

[0041] According to one aspect of the invention, a method of creating hyperlinks from hyperlinked items referenced in a physical document to an electronic document comprises: (a) creating a hyperlink table for the physical document, the physical document having at least one page; (b) for each page of the physical document, storing in the hyperlink table means for identifying a page and means for identifying at least one hyperlinked item defined on the page; (c) determining the position of points pressed on an opto-touch foil aligned with the page of the physical document; the points corresponding to the positions of the hyperlinked items; (d) storing in the hyperlink table the position of the points pressed, the hyperlink table comprising for each hyperlinked item, an indication of a position on the page; and (e) for each hyperlinked item defined on the page: (i) associating an electronic document; (ii) storing in the hyperlink table data identifying and locating the electronic document; (iii) associating one or a plurality of hyperlinked points located on the electronic document; and (iv) storing in the io hyperlink table the position of each hyperlinked point on the electronic document.

[0042] According to a second aspect of the invention, a method for activating hyperlinks from one or a plurality of items referenced in a physical document to one or a plurality of points in electronic documents, comprises: (a) identifying a physical document; (b) identifying a page in the physical document; (c) determining the position of a point pressed on an opto-touch foil placed and aligned over or under the identified page, the opto-touch foil being pressed at a point corresponding to a selected hyperlinked item; (d) identifying the selected hyperlinked item by referring to a hyperlink table associated with the physical document, the hyperlink table comprising an indication of the position on the identified page of the physical document of each referenced hyperlinked item; (e) identifying and locating the electronic document associated with the selected hyperlinked item referring to the hyperlink table, the hyperlink table comprising for each hyperlinked item of each page of the document the identification and location of the electronic document associated with the hyperlinked item; (f) accessing and displaying the electronic document associated with the selected hyperlinked item; (g) identifying the position on the electronic document of each point hyperlinked with the selected hyperlinked item referring to the hyperlink table, the hyperlink table comprising for each hyperlinked item referenced in the physical document, the position of one or a plurality of points on the electronic document; and (h) designating the position on the displayed electronic document, of each point hyperlinked with the selected item.

[0043] According to a third aspect of the invention, a computer program product for use with a computer system capable of creating hyperlinks from hyperlinked items referenced in a physical document to an electronic document, the computer program product comprising a computer useable medium having embodied therein program code comprising: (a) program code for creating a hyperlink table for the physical document, the physical document having at least one page; (b) for each page of the physical document, program code for storing in the hyperlink table means for identifying a page and means for identifying at least one hyperlinked item defined on the page; (c) program code for determining the position of points pressed on an opto-touch foil aligned with the page of the physical document; the points corresponding to the positions of the hyperlinked items; (d)program code for storing in the hyperlink table the position of the points pressed, the hyperlink table comprising for each hyperlinked item, an indication of a position on the page; and (e) for each hyperlinked item defined on the page, program code for: (i) associating an electronic document; (ii) storing in the hyperlink table data identifying and locating the electronic document; (iii) associating one or a plurality of hyperlinked points located on the electronic document; and (iv) storing in the hyperlink table the position of each hyperlinked point on the electronic document.

[0044] According to a fourth aspect of the invention, a computer program product for use with a computer system capable of activating hyperlinks from one or a plurality of items referenced in a physical document to one or a plurality of points in electronic documents, the computer program product comprising a computer useable medium having embodied therein program code comprising:(a) program code for identifying a physical document; (b) program code for identifying a page in the physical document; (c) program code for determining the position of a point pressed on an opto-touch foil placed and aligned over or under the identified page, the opto-touch foil being pressed at a point corresponding to a selected hyperlinked item; (d) program code for identifying the selected hyperlinked item by referring to a hyperlink table associated with the physical document, the hyperlink table comprising an indication of the position on the identified page of the physical document of each referenced hyperlinked item; (e) program code for identifying and locating the electronic document associated with the selected hyperlinked item referring to the hyperlink table, the hyperlink table comprising for each hyperlinked item of each page of the document the identification and location of the electronic document associated with the hyperlinked item; (f) program code for accessing and displaying the electronic document associated with the selected hyperlinked item; (g) program code for identifying the position on the electronic document of each point hyperlinked with the selected hyperlinked item referring to the hyperlink table, the hyperlink table comprising for each hyperlinked item referenced in the physical document, the position of one or a plurality of points on the electronic document; and (h) program code for designating the position on the displayed electronic document, of each point hyperlinked with the selected item.

[0045] The foregoing, together with other objects, features, and advantages of this invention can be better appreciated with reference to the following specification, claims and drawings.

[0059] The present invention relates to the creation of hyperlinks from printed items (e.g. words, pictures, foot notes, symbols, icons) on a physical document to other items or particular points located on electronic documents. The hyperlinks are used to locate on the electronic documents, the items referenced in the physical document.

[0060] More particularly, by assigning display coordinates to geographic locations (for example places like towns, motels, public buildings, water springs, historic monuments, factories, etc.) cited or referenced in a physical document (for example, cited in a hard-copied travel guide), the invention provides means for highlighting on a digital map the geographic location of the places cited or referenced in the physical document. The activation of a “geographic link” associated with a particular printed item on a physical document (for example a directory of hotels), results in highlighting the position of this item on a digital map.

[0061] In a particular embodiment, the system comprises a transparent opto-touch foil placed aligned over the physical document and connected to a user workstation. The opto-touch foil generates optical signals highlighting the hyperlinked items referenced on the physical document. The system includes logic for highlighting on the hyperlinked electronic documents, the locations of the hyperlinked items referenced on the physical document.

[0062] As shown in FIG. 1, the system according to the present invention comprises an opto-touch foil (100), preferably transparent, placed by the user over (or under) a physical document (101), or a portion/page of this document. This opto-touch foil is used to illuminate and highlight hyperlinked items over the surface of the physical document, and to read coordinates of these hyperlinked items; and an user workstation (102) for accessing, retrieving and displaying electronic documents associated with the hyperlinked items referenced on the physical document and for highlighting the locations of the hyperlinked items on the electronic documents. Note in the following description the expressions “portion of a physical document” or “page of a physical document” will be used indifferently.

Physical Document

[0063] A physical documents (101) can be of any kind. It can be, for instance, a newspaper, a travel book, a novel book, a text book, a technical book, a commercial catalog or even any other type of hard-copy, engraved, written, or printed surfaces (e.g., paintings in a museum of art). The material of the physical documents can be paper, plastic, wood or any other material.

Opto-Touch Foil

[0064] In an illustrative embodiment, the opto-touch foil (100) comprises two, functionally independent transparent foils, namely: a touch foil, and a light emitting foil (opto foil).

[0065] FIG. 2 shows the cross section of an opto-touch foil (200) comprising a transparent resistive or capacitive touch foil (201), of the type used commonly to manufacture touch screens. The generated signal is generally proportional to the coordinates (X, Y) of the point that is pressed or touched (203). The opto-touch foil (200) further comprises a transparent light emitting foil (202), which is a transparent, bright, self-emitting display that can emit light (204) from either one or both surfaces.

[0066] The combination of both foils (i.e., the touch foil stacked over the light emitting foil) forms an opto-touch foil (200). FIG. 2 represents an opto-touch foil placed and aligned over a physical document (205) comprising a plurality of items (206) (i.e., words, pictures, icons, etc.) printed (or written, painted, engraved . . . ) on its surface.

Transparent Touch Foil Technology

[0067] The touch foil component (201) can be made of transparent resistive or capacitive films of the type used commonly to manufacture touch screens. One example of appropriate touch foil technology is the TouchTek4 (4-wire Analog Resistive Touchscreens) of MICRO TOUCH company (TouchTek™ is a trademark of the MICRO TOUCH Company). The TouchTeck4 technology is fully described in the Web site of MICRO TOUCH Company.

[0068] TouchTek4 touchscreens are commonly used in hand-held personal information management systems, PDAs, mobile computing systems, automotive, diagnostics and telecom devices, and Internet appliances. TouchTek4 touchscreens are engineered to accept more than three million touches to any area of the screen.

Light Emitting Foil (Opto-foil)

[0069] The light emitting foil (202) can be made of an array of TOLED's (Transparent Organic Light Emitting Devices), of the type used today to create vision area displays on windshields, cockpits, helmets and eyeglasses. TOLED displays, which can be up to 85% transparent when not energized are today manufactured with standard silicon semiconductors. One example of light emitting foil technology it would be possible to use is the technology used for the TOLEDs manufactured by UNIVERSAL DISPLAY CORPORATION. The TOLED technology is fully described in the Web site of UNIVERSAL DISPLAY CORPORATION, Ewing, N.J. at http://www.universaldisplay.com/

[0070] Since TOLEDs are thin-film, solid-state devices, they are very thin, lightweight and durable, ideal for portable applications, like the one disclosed in the present invention. TOLEDs can be bottom, top, or both bottom and top emitting. Also, TOLEDs technology has attractive advantages regarding, transparency (TOLED displays can be nearly as clear as the glass or substrate they're on and when built between glass plates, TOLEDs are >80% transparent when turned off), energy efficiency (for longer battery life), full viewing angle, bright and high contrast light emission, fast response time, and environmental robustness. Thus, TOLEDs are well suited for manufacturing the light emitting foil component used jointly with the touch foil (201) according to the present of this invention.

User Workstation

[0071] The user workstation (102) is used to receive from the opto-touch foil (100) the foil coordinates of the points pressed on the physical document (101), to send to the opto-touch foil (100) the foil coordinates of the hyperlinked items to highlight, and to access, retrieve and display the electronic documents locally stored or located on one or a plurality of servers (103) connected to the network (104).

[0072] Preferably, the user workstation (102) is connected to the Internet network (104) and comprises a Web Browser application. Servers (103) are Web servers. In an illustrative embodiment, electronic documents are digital maps (e.g., HTML Web pages representing geographic maps) hyperlinked to selected items printed or engraved on the physical document (101).

[0073] The user workstation can be, for example an Internet enabled cell phone (e.g., the NOKIA 7110 or the 9110i Communicator), a PDA, an onboard computer, a network computer, an Internet appliance, a multimedia Personal Computer (PC), a set-top box, a WebTV, a game console or any wireless IP enabled device communicated with the opto-touch foil (100). The opto-touch (100) foil can communicate with the user workstation (102) by means of a cable, a wire pair, an infrared link, or a wireless radio link.

Information Access and Display

[0074] As illustrated by FIG. 3, to select a particular portion of a physical document (301) (a page of a book for example), the user places over (or under) this portion the opto-touch foil (300) and enters in the workstation (302) the reference of this portion (the page number for example). In an illustrative embodiment, the page number can be entered simply by touching some buttons (303) marked on the top (or bottom) of the opto-touch foil (300). The page number can be also entered directly by means of the keyboard, the mouse or the touch screen of the user workstation or by any other means such as a bar code reader.

[0075] In an illustrative embodiment, hyperlinked items (304) on a page are automatically illuminated and highlighted by the opto-touch foil (300) placed on the surface of this page. The light emitting foil operates under the control of the user workstation (302). This illumination and highlighting allows the localization and identification of all hyperlinked items on the page selected by the user.

[0076] In order to access to a digital map (307) and to highlight on the map the location (308) of an hyperlinked item (305) printed on the physical document (301), the user touches with its finger or exercises a pressure on the portion of the opto-touch foil (300) placed over (or under) the hyperlink item (305) he wants to select. The position of the hyperlinked item (305) selected on the opto-touch foil (300) is used to identify and locate the server (103) and the information within this server (e.g., a Web page that comprises a digital map) the user wants to access. The user workstation (302) then accesses the identified server (103), and retrieves and displays the information (306) associated with the selected item (305). The position of the hyperlinked item (305) selected on the opto-touch foil (300) is also used to determine the points (or points) (308) hyperlinked to this item, on the retrieved displayed document (306) (preferably a digital map (307)). In an illustrative embodiment, the position corresponding to the item (305) selected in the physical document is automatically highlighted (308) or marked on the retrieved electronic document (306) by means a luminous signal or a pointer generated by the user workstation (302).

Method for Creating Hyperlinked Items

[0077] In general terms, the method of creating hyperlinks from items (304) on a physical document (301) to particular points (308) on electronic documents (306), comprises the steps of:

• • [0078] creating an hyperlink table (407) for the physical document (401); the physical document comprising one or a plurality of pages;
  • [0079] receiving and storing in the hyperlink table (407) an identification of the physical document (401);
  • [0080] for each page of the physical document (401):
    • [0081] receiving and storing in the hyperlink table (407) an identification of the page and an identification of one or a plurality of hyperlinked items (405) defined on the page;
  • [0082] for each hyperlinked item (304) defined on the page, associating a hyperlinked document, the step comprising the further steps of receiving and storing in the hyperlink table (407) an identification and a location of a hyperlinked digital document located on the user system (402) or on one or a plurality of servers (403);
  • [0083] for each hyperlinked item (305) defined on the page, associating one or a plurality of hyperlinked points (308) in the associated hyperlinked electronic document (306);
  • [0084] for each hyperlinked item (305) defined on the page, storing in the hyperlink table (407), the pixel coordinates (i.e., scroll and screen coordinates) of the one or plurality of associated hyperlinked points (308) when displayed on the hyperlinked electronic document (306);
  • [0085] placing and aligning the opto-touch foil (400) over (or under) the selected page of the physical document (401);
  • [0086] determining the foil position of each hyperlinked item (405) defined on the page by pressing the opto-touch foil (400) over the hyperlinked item (406); and
  • [0087] for each hyperlinked item (405) defined on the page, storing in the hyperlink table (407) the corresponding foil position sensed by the opto-touch foil.

[0088] FIG. 5 illustrates the method of creating hyperlinks from a physical document (hand written or printed document like the document shown in FIG. 6 entitled “Travel Guide to the Balkans”), to an electronic document (for example, a digital map). The method comprises the steps of:

• • [0089] (501) assigning a reference number (identifier) to the physical document; and for each page of the physical document document (or portion of the document):
  • [0090] (502) selecting one or a plurality of items on the page;
  • [0091] (503) for each selected item, creating an hyperlink between the item and an electronic document;
  • [0092] (504) on each hyperlinked electronic document, selecting one or a plurality of hyperlinked points;
  • [0093] (505) storing in an hyperlink table, scroll and screen coordinates of the hyperlinked points displayed over the hyperlinked electronic documents;
  • [0094] (506) placing and aligning an opto-touch foil over the page;
  • [0095] (507) pressing the opto-touch foil over the selected items;
  • [0096] (508) reading and storing in the hyperlink table, the position (foil coordinates) on the opto-touch foil of the selected items on this page.

[0097] Assigning a Reference Number to the Document (501)

[0098] As shown in FIG. 6, for each physical document (600) received, the user: assigns a reference number (601) to this document (e.g., 387) for identifying the document; writes this reference number (601) on the document, and creates an hyperlink table associated with the document and accessible from the user workstation, the hyperlink table comprising the reference number of the document and other relevant information related to the document such as title, author, ISBN (International Standard Book Number), date, etc . . . The hyperlink table may be stored within the user workstation or may be stored in an external memory accessible from the user workstation.

[0099] The header of the new created hyperlink table associated with the physical document shown in FIG. 6, can be built as follows:

[0100]

Doc: 0387	Date:	Title:	ISBN:	Author:
	28/01/2001	“Travel Guide to the	84-344-0856-2	David Nash Ford, Editor
		Balkans”		Web Atlas Association,
				Ltd., 2001

Selecting Items on Pages of the Physical Document (502) and Creating Hyperlinks to Electronic Documents (503)

[0101] As shown in FIG. 7, each time the user wants to create a hyperlink for an item (702) located on the page of the physical document (700), the user enters the page number (701) where the item is referenced in the hyperlink table of the physical document (e.g., 16). The user then associates with this item on this page, means for accessing an electronic document, for instance a destination address within a communication network (e.g., an URL address for example), this destination address identifying a server connected to the communication network and within this server, the electronic document the user wants to access. The user then enters the means for accessing the electronic document in the hyperlink table.

[0102]

In an	Date:	Title:	ISBN:	Author:
illustrative	28/01/2001	“Travel Guide to the	84-344-0856-2	David Nash Ford, Editor
embodiment,		Balkans”		Web Atlas Association,
the				Ltd., 2001
hyperlink
table
associated
with the
page shown
in FIG. 7,
appears as
follows:Doc:
0387
Pg: 16
	FOIL X/Y	ELECTRONIC DOC.	SCROLL POS.	SCREEN POSITION
LINKED ITEM	mm.	URL	Ys	(Xp, Yp)
Zagrev		http://www.interconti.com/croa-
		tia/zagreb/hotel_zagic.html
Karlovak		http://www.kakarigi.net/manu/
		briefhis.htm
Knin		http://www.kakarigi.net/manu/
		briefhis.htm
Hotel Bellevue		http://www.kakarigi.net/manu/
		briefhis.htm
Slunj		http://www.kakarigi.net/manu/
		briefhis.htm
Plitvice		http://www.kakarigi.net/manu/
		briefhis.htm
Udbina		http://www.kakarigi.net/manu/
		briefhis.htm
Gracac		http://www.kakarigi.net/manu/
		briefhis.htm

[0103] The electronic document associated with the hyperlinked item can be stored locally in the user workstation. In this case, an identification of the electronic document (local address, file name, file type . . . ) within the user workstation, is entered in the hyperlink table.

[0104] The hyperlink table shows that, for example, the electronic document stored on the URL: http://www.kakarigi.net/manu/briefhis.htm has been associated with the item named “Hotel Bellevue”. FIG. 8 shows the content of this hyperlinked electronic document (800), which comprises a digital map (801). FIG. 9 shows in detail the digital map. Note that the locations of the hyperlinked items named “Plitvice” and “Hotel Bellevue” on physical document (700) are not referenced nor represented on this digital map

Selecting Hyperlinked Points on Electronic Documents (504) and Storing Hyperlinked Points Coordinates (505)

[0105] For each hyperlinked item (305) (e.g., “Hotel Bellevue”) located on the page of the physical document (301) the user accesses, retrieves and displays on the user workstation (e.g., by means of a Web browser program) the hyperlinked electronic document (800) (e.g., http://www.kakarigi.net/manu/briefhis.htm). The user then

• • [0106] scrolls the screen to conveniently display the area of the electronic document (e.g., the digital map (801)) where the hyperlinked point is located. The scrolling action performed by the user is illustrated in the example shown in FIG. 10. The hyperlinked document (1000) is scrolled on the screen of the workstation, from an initial display window (1001) to a final display window (1002), so that, in the final scroll position (1005), the digital map (1003) comprised in the hyperlinked document (1000), can be fully displayed and conveniently viewed by the user. The user then identifies on the displayed area (1002) of the electronic document (1000), the position (1004) of the hyperlinked point, and moves the pointer to this position.

[0107] The user then clicks with the pointer on the selected point. The scroll position of the screen, Ys (1005) and the position of the point on the screen (Xp, Yp) (1006) (coordinates preferably measured in pixels) are computed (409) and copied into the corresponding entry of the hyperlink table (407).

[0108]

Once	Date:	Title:	ISBN:	Author:
completed, the	28/01/2001	“Travel Guide to the	84-344-0856-2	David Nash Ford, Editor
process of		Balkans”		Web Atlas Association,
assigning and				Ltd., 2001
entering scroll
and screen
coordinates to
the locations
of the
hyperlinked
points on the
hyperlinked
electronic
documents,
the hyperlink
table is
updated as
follows:Doc:
0387
Pg: 16
	FOIL X/Y	ELECTRONIC DOC.	SCROLL POS.	SCREEN POSITION
LINKED ITEM	mm.	URL	Ys	(Xp, Yp)
Zagrev		http://www.interconti.com/croa-	619	578
		tia/zagreb/hotel_zagic.html		421
Karlovak		http://www.kakarigi.et/manu/	1457	78
		briefhis.htm		154
Knin		http://www.kakarigi.et/manu/	1457	312
		briefhis.htm		473
Hotel Bellevue		http://www.kakarigi.et/manu/	1457	253
		briefhis.htm		345
Slunj		http://www.kakarigi.et/manu/	1457	92
		briefhis.htm		202
Plitvice		http://www.kakarigi.et/manu/	1457	128
		briefhis.htm		247
Udbina		http://www.kakarigi.et/manu/	1457	149
		briefhis.htm		293
Gracac		http://www.kakarigi.et/manu/	1457	206
		briefhis.htm		331

Placing and Aligning the Opto-touch Foil over a Page of the Document (506)

[0109] As shown in FIG. 4, after each hyperlinked item of a page has been defined in the hyperlink table and has been associated with an electronic document, the opto-touch foil is placed over (or under) the page, and aligned with the borders of this page using some conventional means (e.g., by adjusting the upper left corner of the opto-touch foil with the upper left corner of the page).

Pressing the Opto-touch Foil over the Selected Items and (508) Reading the Foil Coordinates (507)

[0110] As shown in FIG. 4, for each item (405) previously selected on the page (“Zagreb”, “Knin”, “Hotel Bellevue”, “Slunj”, “Plitvice”, “Udvina”, “Gracac”), the user presses (e.g., using his finger tip) the opto-touch foil (400) to determine the position (for instance, the coordinates) of this item on the opto-touch foil.

[0111] Once the foil coordinates of each selected item (hyperlinked item) on the page have been measured, the hyperlink table is updated as follows:

[0112]

Doc: 0387	Date:	Title:	ISBN:	Author:
	28/01/2001	“Travel Guide to the	84-344-0856-2	David Nash Ford, Editor
		Balkans”		Web Atlas Association,
				Ltd., 2001
Pg: 16
	FOIL X/Y	ELECTRONIC DOC.	SCROLL POS.	SCREEN POSITION
LINKED ITEM	mm.	URL	Ys	(Xp, Yp)
Zagrev	125	http://www.interconti.com/croa-	619	578
	110	tia/zagreb/hotel_zagic.html		421
Karlovak	170	http://www.kakarigi.net/manu/	1457	78
	110	briefhis.htm		154
Knin	190	http://www.kakarigi.net/manu/	1457	312
	110	briefhis.htm		473
Hotel Bellevue	35	http://www.kakarigi.net/manu/	1457	253
	155	briefhis.htm		345
Slunj	145	http://www.kakarigi.net/manu/	1457	92
	180	briefhis.htm		202
Plitvice	170	http://www.kakarigi.net/manu/	1457	128
	180	briefhis.htm		247
Udbina	195	http://www.kakarigi.net/manu/	1457	149
	180	briefhis.htm		293
Gracac	215	http://www.kakarigi.net/manu/	1457	206
	180	briefhis.htm		331

[0113] For example, on page 16 of document 387 entitled “Travel Guide to the Balkans” edited by “David Nash Ford, Web Atlas Association, Ltd.”, ISBN: 84-344-0856-2, the foil coordinates of the hyperlinked item “Slunj” on the document are X=145/Y=180. This hyperlinked word “Slunj” points to the electronic document at URL address: http:// www.kakarigi.net/manu/briefhis.htm. This item also points to a point in this electronic document (where the place named “Slunj” is located) having a scroll coordinate Ys=1457 pixels and screen coordinates Xp=92, Yp=202 pixels.

[0114] The hyperlink table of the complete document is built using the herein described method for the different pages of a same document.

Method for Activating Hyperlinked Items

[0115] The user receives a hand written or printed document, like the document entitled “Travel Guide to the Balkans”, (Doc: 387) shown in FIG. 6. This document comprises hyperlinks previously defined between items selected on the pages of this physical document and points (locations) on a digital map included in an electronic document. As shown in FIG. 11, the method of activating these hyperlinks for visualizing and highlighting the hyperlinked points (locations) on the digital map comprises the steps of:

• • [0116] (1101) selecting a physical document (by entering the reference number of this document);
  • [0117] (1102) selecting a page of this physical document;
  • [0118] (1103) placing and aligning the opto-touch foil over (or under) the selected page;
  • [0119] (1104) identifying the hyperlinked items defined on the selected page, identifying the position of these hyperlinked items on the opto-touch foil, and highlighting these hyperlinked items on the selected page by means of the opto-touch foil;
  • [0120] (1105) selecting an hyperlinked item on this page by pressing the opto-touch foil on the point (light spot) corresponding to the item;
  • [0121] (1106) identifying the electronic document associated with the selected hyperlinked item;
  • [0122] (1107) accessing, retrieving and displaying the electronic document associated with the selected hyperlinked item;
  • [0123] (1108) identifying scroll and screen coordinates of the hyperlinked point;
  • [0124] (1109) scrolling the electronic document to the identified scroll coordinate;
  • [0125] (1110) highlighting hyperlinked point on identified screen coordinates.
    Selecting a Document Entering the Reference Number (1101)

[0126] The user selects a physical document (600) and, by means of any user interface (keyboard, mouse, touch screen, . . . ) or any reading means (bar code reader . . . ), enters the reference number or identifier (601), (e.g.: document 387) of the document. In the particular embodiment shown in FIG. 3, the user enters the reference number of the document (e.g.: Doc: 387) by means of an array of pressure sensible “touch buttons” (303) printed on the bottom of the opto-touch foil (300). The user presses in the following order:

• • [0127] a touch button marked “Doc, and then
  • [0128] numerical touch buttons corresponding to each digit of the document reference number (e.g., “3”, “8” and “7”).

[0129] This procedure gives access to the hyperlink table (407) associated with this selected document.

Selecting a Page (1102)

[0130] The user selects a page (701) of the physical document (700) and, by means of any user interface (keyboard, mouse, touch screen . . . ) or any reading means (bar code reader . . . ), the user enters the page number (or the portion of the document) (e.g.: page 16) of selected page. In the particular embodiment shown in FIG. 3, by means of the array of pressure sensible touch buttons (303) printed on the bottom of the opto-touch foil (300), the user enters the page number to select (e.g.: Pg. 16). The opto-touch foil (300) can be used in any position (normally the opto-touch foil is placed over the page comprising the hyperlink to activate). The user presses in the following order: a touch button marked as “Pg”, and then numerical touch buttons corresponding to each digit of the page number (e.g., “1” and “6”),to select.

[0131] This step gives access to the selected page (e.g., Pg. 16) within the hyperlink table (407) associated with the selected document (e.g., Doc: 387), the hyperlink table comprising, for each hyperlinked item of each page of the document, the identification and position of the hyperlinked item.

Placing and Aligning an Opto-touch Foil (1103)

[0132] As shown also on FIG. 3, the opto-touch foil is placed over (or under) the page, and aligned with the borders of the selected page by some conventional means (e.g., by adjusting the upper left corner of the touch foil with the upper left corner of the page).

Highlighting Hyperlinks on the Selected Page (1104)

[0133] The position (coordinates X and Y) of all the hyperlinked items previously defined for the page selected by the user are retrieved from the hyperlink table (407) and sent by the user workstation (402) to the opto-touch foil (400). The opto-touch foil (400) receives the information sent by the user workstation (402). This information comprises the position (coordinates X and Y) of each hyperlinked item defined on the selected page. The opto-touch foil (400) decodes this information and visualizes (highlights or illumines) (405) the position of each of the hyperlinked items.

[0134] Once a document (601) and a page (701) have been selected, the opto-touch foil (400) generates one or a plurality of light spots (405) signaling that one or more hyperlinks have been defined for this page.

[0135] When the opto-touch foil (300) is placed and aligned over the selected page, the position of the light spots (304) emitted by the opto-touch foil corresponds to the position of the hyperlinked items (702) defined for this page. The hyperlinked items appear through the transparent opto-touch foil (300) and are identified by the user thanks to the light spots emitted by the opto-touch foil.

Selecting an Hyperlinked Item on this Page (1105)

[0136] As shown in FIG. 4, the user selects an hyperlinked item (406) (“Hotel Bellevue”) on the page by pressing (e.g., by means of his fingertip) the opto-touch foil (400) on the (illuminated) point corresponding to the hyperlinked item.

[0137] By means of this action, the opto-touch foil (400) sends a signal to the user workstation (402) to identify the selected item. This signal indicates the position on the page of the point that has been pressed by the user on the opto-touch foil. The generated signal is generally proportional to the coordinates (X/Y) of the point that has been pressed. In our example, the opto-touch foil measures the position of the point that has been pressed on the page (near the light spot over the word “Hotel Bellevue”) by the user. The coordinates measured at this point are around X=35 and Y=155.

[0138] The item that has been selected (406) on the opto-touch foil (400) by the user is then identified thanks to the hyperlink table (407). The hyperlink table comprises an indication of the position (coordinates X and Y) of each illuminated item on each page of the document. In our example, the coordinates measured by the opto-touch foil are around (close to) X=35 and Y=155. They corresponds, in the hyperlink table (407), to the hyperlinked item “Hotel Bellevue”.

[0139] The hyperlink table is either stored locally in the user workstation (402), or is retrieved from a remote server (403) and then stored locally in the user workstation or is stored in a remote server and is accessed remotely.

Identifying the Electronic Document Associated with the Selected Item (1106)

[0140] The hyperlink table (407) comprises for each hyperlinked (illuminated) item (405) of each page of the physical document (401) means for locating within the user workstation or within the network, an associated electronic document. The electronic document can be located by means of a destination address. In the Internet network, Web pages in Web Servers (403) are identified by an URL (Uniform resource Locator).

[0141] In our example, the hyperlink table associates the illuminated item “Hotel Bellevue” with the URL: http://www.kakarigi.net/manu/briefhis.htm.

Accessing, Retrieving and Displaying the Associated Electronic Document (1107)

[0142] The user workstation (402) Web activates the hyperlink (destination address, URL, . . . ) associated in the hyperlink table (407), with the identified selected item (406).

[0143] The system searches in the hyperlink table (407), the coordinates of the hyperlinked item that is the nearest to the point pressed by the user. In our example, using the hyperlink table (407), for this page (i.e., Pg. 16), the hyperlink labeled “Hotel Bellevue” with the coordinates X=35, Y=155 is identified and activated. Thus, a simple pressure near the illuminated item “Hotel Bellevue” will automatically activate the following hyperlink on the Web: http://www.kakarigi.net/manu/briefhis.htm.

[0144] The Web Browser program (408) sends a HTTP request (410) to the identified Web Server (411). The reply to the request (HTTP response) (412) is sent by the Web Server (411) in the reverse direction, to the Web Browser (408). The HTTP reply comprises the requested Web page associated with the selected illuminated item (406). The electronic document (Web page) at the selected destination address (URL) is then retrieved through the Internet network (404) and is displayed by the Web browser (408) on the user workstation (402).

[0145] In our example, as illustrated in FIGS. 8 and 10, immediately after its retrieval by the user workstation, the Web page is automatically displayed by the Web browser starting from the beginning. Therefore, only the first portion (802) (1001) of the Web page is displayed on the screen of the user workstation.

Identifying Scroll and Screen Coordinates of the Hyperlinked Point (1108)

[0146] The user workstation (402) identifies the coordinates (i.e., scroll and screen pixel coordinates) (409) of the hyperlinked point associated, in the hyperlink table (407), with the identified selected item (406), the hyperlink table comprising for each hyperlinked item (405) referenced in the physical document (401), the coordinates (1005), (1006) of one or a plurality of hyperlinked points in the associated electronic document (1000);

[0147] In our example, the hyperlink labeled “Hotel Bellevue” has been activated on page 16 of the physical document. From the information contained in the hyperlink table (407), the system determines that:

• • [0148] Ys=1457 (pixels) is the scroll position coordinate (1005), and
  • [0149] Xp=253, Yp=345 (pixels) are the screen position (1006) coordinates of the hyperlinked point on the Web page retrieved from the URL address: http://www.kakarigi.net/manu/briefhis.htm.
    Scrolling the Electronic Document to the Identified Scroll Coordinate (1109)

[0150] The user workstation (402) scrolls the electronic document to the scroll coordinate Ys previously identified to display on the screen, the area of the electronic document (e.g., the digital map (801)) comprising the point hyperlinked (1004) to the selected item (406).

[0151] The scrolling action performed In our example, by the user workstation is illustrated in FIG. 10. The Web page (1000), (800) retrieved from the URL address: http://www. kakarigi.net/manu/briefhis.htm.

[0152] is scrolled on the workstation screen, from an initial display window (1001) to a final display window (1002). At the final scroll position (1005) (Ys=1457 pixels), the digital map (1003) comprised in the hyperlinked document (1000) (where the hyperlinked point will be located) is displayed on the screen.
Highlighting Hyperlinked Point on Identified Screen Coordinates (1110)

[0153] The user workstation (402) positions a cursor or highlights by any other means (e.g., by a special symbol) the screen point corresponding to the identified screen coordinates (Xp, Yp).

[0154] In our example, as illustrated by FIG. 10, to highlight the hyperlinked point (i.e. the location of “Hotel Bellevue”) on the Web page (1000), (800) retrieved from the URL address: http://www.kakarigi.net/manu/briefhis.htm, the point with screen coordinates Xp=253, Yp=345 (pixels) is signaled by means of a pointer (1004). FIG. 3 shows with more detail the relationship between the item selected by the user (305) on the physical document (301) and the position (location) (308) corresponding to this item on the displayed electronic document (306) associated with this selected item. Observe in FIG. 9 that the digital map included in the displayed Web page, do not references nor represents the location of “Hotel Bellevue”.

Typical Applications

[0155] The present invention can be used in many applications. The differences between these applications are mainly due to the type of graphical information shown on the electronic documents and to the type of physical (i.e., hard-copy) document used to activate the hyperlinks.

[0156] Yellow Pages

[0157] In this particular application the user places an opto-touch foil over a page of the “Yellow Pages” of a city, (e.g., over a page with a list of “Restaurants” in the “Yellow Pages” of the city of Paris). The opto-touch foil is connected to an user workstation (e.g., to a WAP enabled cell phone or a PDA).

[0158] When the user presses the touch-foil on a printed item (e.g., the name of a restaurant), the geographic coordinates of this restaurant are retrieved from the Web and are highlighted over a digital map of the city of Paris displayed on the screen of the user workstation.

[0159] Installation and Engineering Instructions

[0160] The purpose of the present system is to help an technician to identify a part in a complete assembly, and to help him to see how this part interacts with the other parts.

[0161] A technician inspects a complex mechanical equipment, with an installation and service handbook comprising explanatory descriptions referencing parts and subassemblies represented on separate digital drawings and schemes. With an opto-touch foil placed on the installation and service handbook, the technician can immediately see items highlighted by luminous spots. These illuminated items are identified as hyperlinked items and can be used to access additional information on a remote Web server or on a local computer.

[0162] When the technician touches with a fingertip one of these illuminated items (e.g., the name of a part), multimedia instructions are instantly displayed on the screen of his workstation to help him to install the part. At the same time the location of the part appears pointed on an auxiliary digital drawing or scheme on the same screen.

[0163] There is no need to reference the different parts on the printed handbook or on the drawings by means of specific names, numerals or pointers (as is the common practice with engineering documentation). The technician has just to press with his finger on the item illuminated on the installation handbook to see automatically illuminated the position of this item over a digital graphical representation of the equipment to install or to repair. Therefore, by means of the present invention, hard-copy technical documents are less complex, clearer and easier to use. Handbooks and drawings don't need any more to be populated with cumbersome alphabetic and numeric cross references.

[0164] Courses and Textbooks

[0165] Extensive reading is easier to do on paper, but digital graphics illustrations are much more effective for some particular purposes. The two can be tied together by placing hyperlinked items in a textbook. These hyperlinked items can provide optical pointers to items on electronic figures, diagrams or schemes retrieved through the Web.

[0166] Clinical and Medical Applications

[0167] For example, as illustrated on FIG. 12, it should be possible to highlight on digital images of human anatomy (e.g., on a Computerized Tomography (CT) Scan), the location of organs or pathologies cited in the case history of a patient.

[0168] Being apparent that, apart from the herein described applications, there are many other possible applications of this invention, what has been described is merely illustrative of the application of the principles of the present invention. Other arrangements and methods can be implemented by those skilled in the art without departing from the spirit and the scope of the present invention.