Method, system, apparatus, and computer program product for controlling and browsing a virtual book

[0001] This application claims priority to U.S. provisional application 60/410,278 filed on Sep. 13, 2002. This application is related to U.S. Pat. No. 5,909,207; U.S. Pat. No. 6,064,384; U.S. Pat. No. 6,407,757; U.S. Pat. No. 6,340,980; U.S. patent application Ser. No. 09/617,043 filed Jul. 14, 2000; and allowed U.S. patent application Ser. No. 09/686,902 filed Oct. 12, 2000; and allowed U.S. patent application Ser. No. 09/686,965 filed Oct. 12, 2000; U.S. patent application Ser. No. 60/318,638 filed Sep. 13, 2001; U.S. patent application Ser. No. 10/029,718 filed Jan. 23, 2001; U.S. patent application Ser. No. 10/029,868 filed Dec. 13, 2001; U.S. patent application Ser. No. 10/142,996 filed May 13, 2002; U.S. patent application Ser. No. 10/142,858 filed May 13, 2002; U.S. patent application Ser. No. 10/143,005 filed May 13, 2002; U.S. patent application Ser. No. 60/365,186 filed Mar. 19, 2002; and U.S. patent application Ser. No. 60/367,765 filed Mar. 28, 2002 the entire contents of each of these applications being incorporated herein by reference.

[0002] 1. Field of the Invention

[0003] The present invention relates to a hardware and software system including a controller that can be operatively combined with several methods of display on a display screen under software and hardware control for the purpose of browsing through data stored in a memory device. This facilitates the browsing of the data stored in a memory device in such a way that it allows a rapid view of what contents are present and the organization of the material in the data, as well as rapid access to the items in the data involved. Specifically the present invention relates to controllers that have specialized functions for effecting various actions on the data displayed on the display screen in such a way that users can rapidly browse through the data involved.

[0004] 2. Description of the Prior Art

[0005] Currently, the method for viewing documents stored in a virtual is through a virtual monitor screen. For documents that are longer/larger than can be contained within one screen, a means is provided to scroll the document up and down or to jump to a particular point in the document (through the use of, for example, a “mouse” coupled with scroll bars at the edges of the document displayed on the screen or through the use of hypertext links).

[0006] However, most people do not find this a particularly convenient way to view a document, as compared with a book. That this is true is evidenced in the fact that despite the fact that the virtual has been in increasing use, there is not only no corresponding reduction in paper use, but an increase in paper quantity is required to print what is conveniently stored in a virtual that can be seemingly easily and flexibly manipulated for viewing. Sometimes the printed version of the information involved is necessary, like in the case of printing a picture on a sheet of paper to be pasted on some surface (e.g., a wall), or in the case of the convenience of having a relatively light, hardy paperback book for reading in almost any possible situation and location, but there are times when a reader is quite happy to sit at a desk in front of a desktop virtual or handle a notebook virtual away from the desk, and yet the reader would still prefer a printed version of the document in a hand-held format. This is especially true in the case of looking through manuals, including software manuals, to understand how to operate a device or software application. The irony is that virtual software packages usually come with thick and heavy manuals containing information which can easily be stored on a light-weight CD-ROM, and even if the manuals are sometimes dispensed in the form of a CD-ROM, more often than not people would print them out into a hard-copy format in order to facilitate reading.

[0007] What is involved when one interacts with the printed material in a book is a subtle and complicated process. To start with, material in a book is presented in a sequential order, with a continuity of material from page to page, and there is also a hierarchical structure in the material presented (as the material is organized into chapters, sections, subsections, etc.) because ideas in the material are related to each other in some kind of conceptual hierarchy. The human perceptual system inputs data in a sequential manner, and after a book is read from the beginning to the end in a sequential fashion, the brain then recreates the conceptual hierarchy after viewing the material involved. However, very often one does not read a book (or input the material involved) from the beginning to the end because (a) one wants to have an overview of the material present; (b) one is searching for something of interest to him/her; or (c) one is interested in reading only portions of the book (in the case of, say, reading the manual to understand how to operate something). In these cases, one browses through the subject book to find the material of unique interest to that reader.

[0008] Two basic things are achieved in the browsing process. First, the browser has a glimpse of what are the contents of the book document. Second, the browser has an idea of approximately where the items of interest are so that the browser can (a) return to look for them later when needed, and (b) have an understanding of the relationships between the material currently being viewed and other material (i.e., an understanding of the hierarchical structure involved). When browsing a book document, many finger-operations are required of the browser in order to flip through the pages and, together with the inherent sequential order imposed by the pages, very quickly allow the browser to have an understanding of the nature, location and organization of the material involved.

[0009] In the process of browsing through a book, one can perform the following operations:

• • [0010] (a) flip through the pages at varying speeds depending on the level of detail at which one wishes to view the material in the book;
  • [0011] (b) jump to the approximate location of the item of interest;
  • [0012] (c) change the direction of flipping (forward or backward) very rapidly because
    • [0013] (i) one would like to compare and contrast material on different pages,
    • [0014] (ii) after jumping to an approximate location of some items of interest one would like to find their exact locations, or
    • [0015] (iii) one is unsure of where the item of interest is and is in the process of searching for it; and
  • [0016] (d) mark the locations of some pages of interest that one may want to later return.

[0017] All these operations are performed very rapidly with the fingers interacting with the flipping pages and with minimal unnecessary movements of the fingers and hands. Interestingly, a book/magazine with soft and flexible pages is harder to handle because more finger and hand movements are needed to browse through it, while books with stiff pages can be browsed with almost no movement of the hand.

[0018] In currently available methods of browsing through documents stored in a virtual, e.g., the use of a mouse combined with scroll bars and buttons on the virtual screen, more movements of the hands are necessary to effect the various operations described above. Also, fine control of the hand or fingers (depending on whether the mouse uses hand movement to move the cursor on the screen or finger movement like in the case of a track ball) is necessary to position the cursor on the screen at the required places. The process is both lengthy and clumsy. The lengthiness of the process taxes the human short term memory s ability to remember items encountered in the recent past for the purpose of establishing the relationships between items and the clumsiness of the process creates distraction and interferes with the short term memory process, a well known effect in perceptual psychology.

[0019] Because material in a book is organized into pages, it also enhances the ability of the reader to better remember the location of various portions of the material involved. Also, unlike the process of scrolling through a document on a screen like what is normally done in a word-processor, wherein the contents become a blur and reading is impossible, when one moves through the material in a book through flipping, one is still able to read at least the approximate contents, if not the details. It is due to these features that a person browsing through a book can acquire a good understanding of its contents, the location of specific items and organization of the material.

[0020] It is because of the reasons set forth above, people still prefer to read a book in their hands, rather than a document image displayed on a virtual screen using currently available methods.

[0021] U.S. Pat. No. 5,467,102 (Kuno et al.) discloses a device for document viewing that consists of two display screens. One of the purposes of using two display screens is to allow the user to display two different pages from the document so that they can be read side-by-side (e.g., a diagram and its textual explanation). Another purpose is to allow a large picture to be displayed simultaneously on both pages. The Kuno et al. device allows users to change the speed of movement through the document through a pressure sensor—the more pressure applied, the faster the pages in the document are moved through. The Kuno et al. device also allows the document to be viewed in the forward or backward direction by pressing on a forward sensor area or a reverse sensor area respectively. One can also select a page to jump to by pressing on an icon displayed on the screen. However the Kuno et al. device still does not provide the same convenience as browsing through a book, primarily because when switching between the operations for different controls—the speed of movement through the document, the change of direction of viewing, and the jumping to different parts of the document—there are a lot more hand and finger movements than is the case in manipulating a physical book. Moreover, the Kuno et al. device is a specialized, relatively costly device with sensors and hardware built onto two display screens, whereas the present inventor recognizes that a lower cost and more practical device would be one that adds modularly to the existing virtual system.

[0022] Currently, there are also virtual mice that can eliminate the above-mentioned problem of positioning cursor on the virtual screen with a conventional mouse (i.e., fine control of the hand or fingers is needed). These mice allow the user to specify “hot locations” on the screen on which the cursor “homes onto” with less fine control than conventional mice. Furthermore, a subset of these mice can generate “vertical only” or “horizontal only” movement of the cursor so that the scrolling process requires less fine control of the muscle than is required with a conventional mouse. These mice eliminate some, but not all, of the problems associated with the conventional methods of virtual input as far as virtual-based document browsing is concerned.

[0023] In U.S. Pat. No. 5,417,575 (1995) McTaggart discloses an electronic book that comprises laminated sheets bound together in the form of a book. On each of these sheets, printed material is arranged on the top layer and below that layer is an electronic backdrop containing thin light-emitting diodes (LED's) and pressure sensitive switches affixed onto a backing sheet. The LED's generate visual signals that can be seen through the top layer for the purpose of highlighting parts of the printed material. The pressure switches, positioned under certain items in the printed material, are for the purpose of sensing the user's selection of those items. A speaker is also provided on the book to generate audio signals for explaining the text or giving the user audio feedback. Contact or photo-sensitive switches are also embedded in the pages to allow the electronic circuits to know which pages are currently being viewed, so that the appropriate audio and visual signals can be generated. Even though this apparatus is in a form that allows a person to handle it like handling a typical book, with visual and audio enhancements of the printed material as well as facilities that accept the user's feedback, it is basically a hard-wired device that is not reprogrammable and different hardware has to be configured for books with different contents. No provision is available for downloading document files from a virtual for display on the electronic book nor is the electronic book able to display any arbitrary document file. This device is hence not suitable for browsing through documents stored in a virtual.

[0024] U.S. Pat. No. 5,909,207 (Ho) discloses a virtual-based system that allows users to view and manipulate documents in the form of an electronic virtual book that addresses the above problems in browsing information in the electronic medium. This virtual book system simulates the paper book very closely and has two major components. One is the representation of the paper book on the virtual screen, complete with realistic flipping pages and thickness representation. The other is the use of a browsing device that allows

• • [0025] 1. rapid change of direction of flipping,
  • [0026] 2. rapid change of speed of flipping,
  • [0027] 3. selection and jumping to any desired page, and
  • [0028] 4. bookmarking of the pages.

[0029] The browsing device allows one to use the fingers to manipulate the pages in the book much in the same way as in the case of the paper book. Therefore, the virtual manipulation of the pages together with the book image on the screen constitutes a virtual book system. This virtual book system improves on the methods used in current electronic means of information browsing in a very significant way such that the ease of browsing information can approach that obtainable with a paper book. This allows users to be able to quickly browse through a large number of pages (perhaps thousands of pages) of information and obtain the idea of the structure and contents involved and hence a good overview of the entire collection of material in the “virtual book”. Other than being able to have a good idea of the structure and contents of the material involved, the process of browsing through the material presented in the form of a virtual book much like in the case of the paper book also facilitates subsequent searches for items in the material.

[0030] In U.S. Pat. No. 5,909,207 (Ho) the virtual book browsing device basically consists of a pressure and position sensor. The pressure the finger exerts on the pressure sensor controls the speed of flipping through the pages of the virtual book. The movement of the finger on the position sensor on the browsing device in the virtual book system corresponds to the movement of the finger on the thickness of the paper book to select a page to jump to. The combination of the pressure and position sensor therefore allows one to rapidly browse through the virtual book.

[0031] However, the method of presenting and browsing information described in U.S. Pat. No. 5,909,207 (Ho) can be improved further by putting together the various components of the system into an integrated system with simple, one-button operations for most of the desired functions. This will greatly improve the consumer use of the system.

[0032] Therefore, the inventor has identified there exists a need for a specialized system for presenting the virtual book system for a consumer-oriented market. In particular, there is a need for a specialized system for presenting the virtual book system that can be interfaced to a computer, a regular or widescreen television (TV), or a regular or widescreen high definition television (HDTV). There is also a need to interface a virtual book system with a CD-R, CD-RW, a DVD, and a similar optical/non-optical portable large volume storage disk or non-disk medium.

[0033] In view of the aforementioned short-comings of presently available schemes for providing consumers with a one-button, simple operations for effecting various desired actions on a virtual book system, such as controlling and browsing a virtual book, one objective of the present invention is to provide a memory and control device to provide one button operations for data transfer and display operations.

[0034] Another objective of the present invention is to provide a controller that works with the memory and control device for the purpose of effectively and rapidly control and browse a virtual book on a virtual book system.

[0035] Yet another objective of the invention is that the positioning and design of the controls for the operations performed by the fingers on the controller are such that they allow minimal movements of the fingers, thus maximizing the ease of browsing through the displayed virtual book. The dexterity of the human fingers is to be fully exploited for these controls.

[0036] Each of these objectives include the objective to display on a computer, a regular or widescreen television (TV), or a regular or widescreen high definition television (HDTV) and/or to interface with a CD-R, CD-RW, a DVD, and a similar optical/non-optical portable large volume storage disk or non-disk medium.

[0079] Referring now to the drawings, wherein like reference numerals designate identical or corresponding parts throughout the several views, and more particularly to FIG. 1A thereof which depicts one embodiment of the virtual book interface. Basically, the information is displayed on two pages as in an opened book. At this stage, a number of pages have been browsed through and collected in the “virtual book” 100. The amount of material before and after the current pages (the pages in view) is shown as varying thicknesses on the left and right sides (101 and 102), respectively (as well as the left and right bottom edges (103 and 104)), of the book. On the left and right thicknesses (101 and 102) are shown a number of bookmarks (105-107) that bookmark some of the pages that have been viewed earlier. There are three types of bookmarks on these thicknesses. They are page-bookmarks 105, annotation bookmarks 106, and finger bookmarks 107. On the bottom thicknesses (103 and 104) are shown a number of keyword bookmarks 108. An Internet URL (Universal Resource Locator) field 109 above the virtual book 100 allows URL addresses to be entered.

[0080] In order to allow the user to select any desired page, jump cursors 110 and 111 are provided on the left and right thickness (101 and 102). The user, using a mouse or other control, can move the left or right jump cursors (110 and 111 respectively) up and down the left or right thicknesses (101 and 102 respectively). The page number may appear on the tag of the jump cursor to display the page selected by the jump cursor. When the user has decided to jump to a certain page using the jump cursor to select that page, he/she can use the mouse or other input device to activate the jump.

[0081] FIG. 1B depicts the process by which a certain selected page in the virtual book 100 is jumped to. This jump may have been either activated by the selection of a bookmark (e.g. page-bookmark 105) or by the jump cursor (e.g., jump cursor 110 or 111). The thickness 112 between the page originally in view 113 and the page jumped to 114 is shown to move across the virtual screen in continuous animation, much like in the case of the physical book.

[0082] FIG. 1C depicts a process by which one page 120 in the virtual book 100 is being flipped. The page 120 can be shown to bend and move across the virtual screen to reveal the underlying page 121.

[0083] FIG. 1D depicts a process by which, because of increased speed of moving through the virtual book 100 by flipping the pages, more than one page (122-125) is shown to be moving across the virtual screen at the same time. The user can select to freeze all the pages in motion.

[0084] FIG. 1E depicts a special jump cursor mode in which the thickness of the page in the region of the jump cursor (110 or 111) is selected to enter a mode of fine adjustment. Due to the resolution of the screen, if the thickness of the page (101 or 102) is used to represent, say, a large number of pages while the thickness representation on the virtual screen may only have a limited number of pixels, the movement of the jump cursor (110 or 111) over the thickness (101 or 102 respectively) in pixel-incremented steps may not allow the user to select some of the pages. Therefore, a fine adjustment mode is necessary. One embodiment is to right click on the jump cursor (say jump cursor 111), and the thickness region around the jump cursor 111 is “expanded” and represented as a rectangle 150. The thickness represented in the rectangle 150 represents a subset of the pages represented by the full thickness 102, around the page currently pointed to by the jump cursor 111. When the jump cursor 111 is moved up and down the thickness within the rectangle 150, it can access the pages to a finer level than on the original thickness 102. Further “enlargements” of the thickness 102 is also possible that leads to even finer access of the pages on the thickness 102.

[0085] FIG. 2 depicts an embodiment of a system 200 targeted toward a general consumer market. The system 200 consists of a display screen 201, a memory and control device 202, and a controller 203. The display screen 201 can be a computer screen, a TV screen or any other kind of display screen. The device 200 can be a computer-based system or any other kind of system that generates on command a virtual book interface on the display screen 201 to display information in the virtual book format 204. The controller 203 can be a mouse, a keyboard, a custom (wireless or wire-connected) remote controller, or any other kind of input device that generates the necessary signals to the device 202 to generate the necessary images in the display screen (e.g., a TV or DVD remote controller).

[0086] FIG. 3A depicts the basic flipping mechanisms of a virtual book 1000 on a computer screen. A left page 1010 and a right page 1011 are stationary while three pages 1020, 1021, and 1022 are being flipped across from the right side to the left side simultaneously. The faster the movement through the book, the more pages are flipped simultaneously and vice versa. (At the minimum, one page can be flipping across.) The speed of movement through the virtual book 1000 is also proportional to the speed of movement of each flipping page while it flips from the right to the left side of the virtual book 1000 or vice versa. When the flipping pages 1020, 1021, and 1022 reach the left side of the virtual book 1000, they are collected there, forming part of the left side of the virtual book 1000. The thicknesses 1005 and 1006 on both sides of the virtual book change accordingly as a result of disappearance of pages from or accumulation of pages on the corresponding side.

[0087] FIG. 3B depicts the basic jumping mechanism as a result of either an activation of a jump cursor 1030 on the right side thickness 1006 or a bookmark 1035 on the right side thickness 1006. A collection of pages between the current right-hand page 1011 and the page 1045 to jump to is shown to flip across from the right side to the left side of the virtual book 1000. A thickness 1040 proportional to the number of pages in between the right-hand page 1011 and the page 1045 to jump to is shown moving across the virtual book 1000 accordingly.

[0088] FIGS. 4-11 depict more complex flipping actions of the pages as a result of receiving browsing control commands to be described in detail later.

[0089] FIG. 4A shows a number of pages (1110-1113, etc.) being flipped from the right side of a virtual book 1100 to the left side of the virtual book 1100. (Pages 1114 and 1115 are still resting on the right side of the virtual book 1100). Starting from this flipping state of all the pages (1110-1113, etc.) in FIG. 4A, FIG. 4B shows the flipping direction of all the pages (1110-1113, etc.) being reversed and changed in the midst of flipping. Starting from the flipping state of all the pages (1110-1113, etc.) in FIG. 4A, FIG. 4C shows the direction of the trailing page 1113 being reversed and changed in mid-flip while the other pages (1110-1112) continue to flip to the left side of the virtual book 1100.

[0090] Starting from the flipping state of all the four pages (1110-1113) in the virtual book 1100 in FIG. 4A and a few more pages (1114 and 1115) being launched in the same direction (from right to left) subsequently, FIG. 4D shows the direction of flipping of a group of trailing pages (1113, 1114, and 1115) being reversed and changed to going from left to right while the other pages (1110, 1111, and 1112) continue to flip from right to left. The region 1120 between the two sets of pages (1110-1112 on the one hand and 1113-1115 on the other) begin to “open up” and as the two sets of pages continue to flip further in opposite directions, in FIG. 4E it is shown that the region 1121 between them (pages 1110-1112 on the one hand and pages 1113-1115 on the other) opens up further.

[0091] Starting from the flipping state of all the pages (1110-1115) in the virtual book 1100 in FIG. 4E, in which some of the pages (1110-11112) are flipping from right to left and some of the pages (1113-1115) are flipping from left to right, when a certain browsing control command is given, as shown in FIG. 4F, the direction of flipping of the trailing page of the group of pages (1113-1115) on the right side—page 1113—is reversed and changed to flipping from right to left while pages 1114 and 1115 continue to flip from left to right and the region 1123 between the page 1113 and the page 1114 begins to open up.

[0092] Starting from the flipping state of all the pages (1110-1115) in the virtual book 1100 in FIG. 4E, in which some of the pages (1110-1112) are flipping from right to left and some of the pages (1113-1115) are flipping from left to right, when a certain browsing control command is given, as shown in FIG. 4G, the direction of flipping of a group of trailing pages (1113 and 1114) of the group of pages (1113-1115) on the right side are reversed and changed to flipping from right to left, thus creating a region 1124 that opens up gradually.

[0093] Starting from the flipping state of all the pages (1110-1115) in the virtual book 1100 in FIG. 4E, in which some of the pages (1110-1112) are flipping from right to left and some of the pages (1113-1115) are flipping from left to right, when a certain browsing control command is given, as shown in FIG. 4H, the direction of flipping of a group of trailing pages (1111 and 1112) of the group of pages (1110-1112) on the left side are reversed and changed to flipping from left to right, thus creating a region 1125 that opens up gradually.

[0094] FIG. 41 shows that starting from the flipping configuration of the virtual book 1100 in FIG. 4A., say, or any of the flipping configuration of the virtual book 1100 in FIGS. 4B-4H, all the pages are made to stop flipping and “freeze” in mid-flip.

[0095] After the pages enter this frozen state, they can be “unfrozen”. There are 4 situations after “unfreezing”: 1. pages are unfrozen and then individual pages continue to flip in the directions prior to freezing; 2. pages are unfrozen and all pages flip in the same direction; 3. pages are unfrozen and some pages flip in the direction right to left to rest on the left side and some flip in the direction left to right to rest on the right side of the book; 4. pages are unfrozen and subgroups of pages flip in different directions.

[0096] FIG. 5A shows firstly a collection of pages 1212 is being flipped from right to left as a result of, say, a jump operation due to either a bookmark or a jump cursor activation such as that depicted in FIG. 3B, and then that it is followed by a number of trailing single flipping pages 1213-1215 that are launched subsequently.

[0097] Starting from the flipping configuration of FIG. 5A, say, in which a collection of pages 1212 being flipped is followed by a number of flipping single pages (1213-1215), a further launching of a collection of pages 1216 is activated as depicted in FIG. 5B as a result of a jump operation due to either a bookmark or a jump cursor activation such as that depicted in FIG. 3B. Subsequent to this, the further launching of any combination of single and/or collection of flipping pages is possible. Subsequent changes of direction of flipping and other movement effects including the freezing in mid-flip of all or some subgroups of this combination of single and/or collection of flipping pages in ways similar to those described for single flipping pages in FIGS. 4B-4I are possible. That is, the collections of pages such as 1212 and 1216 would behave like the single pages in FIGS. 4B-41.

[0098] FIGS. 6A and 6B depict a hold-and-flip method of manipulating a flipping page. Starting from a configuration in which a page is initially at rest, say on the right-hand side of a virtual book 1300, such as page 1306 shown in FIG. 6A, a pointer 1310 picks up the page 1306 from its resting position on the right-hand side of the virtual book 1300, and then the page is flipped by the pointer 1310 as depicted in FIG. 6B either in the direction right to left or left to right. This way, both sides of page 1306 can be inspected repeatedly as many times as it is desired.

[0099] In another method of entering the “hold-and-flip” mode, FIG. 7 depicts a pointer 1410 picking up a flipping page 1404 in the midst of a flipping action, which might be a flipping page in one of those flipping configurations depicted in FIGS. 4A to 4H or a page in a frozen state such as depicted in FIG. 41. Thereafter, all the pages 1401-1403 to the left of it and all the pages 1405-1407 to the right of it fall away and end up resting on the left side and the right side of the virtual book 1400 respectively. This picked-up page can then be flipped about and inspected in the hold-and-flip mode.

[0100] FIG. 8 depicts the picking up of a collection of pages 1503 in a virtual book 1500 and the subsequent flipping-about action. This is the hold-and-flip method of manipulating a collection of flipping pages instead of just a single page. The collection of pages 1503 can be picked up from its resting state from, say, the right side of the virtual book 1500 or from the midst of a flipping action, say, in a configuration such as that depicted in FIG. 5B in which a number of collections of flipping pages are shown to be in flipping action.

[0101] FIG. 9 depicts the simultaneous hold-and-flip action applying to more than one single page (e.g., 1603 and 1604) and/or collection of pages (e.g., 1605). A number of pointers (1610-1612) are involved in the holding and flipping action.

[0102] FIGS. 10A and 10B depict a hold-and-collect method of manipulating pages in a virtual book 1700. A page 1705 in the virtual book 1700 is first picked up by a pointer 1710 (either when the page 1705 is at rest on the right or left side of the virtual book 1700 or when it is in the midst of some flipping action). And then, pages 1720-1725 to its left and/or right sides, whether they are initially in the midst of some flipping action or at rest on the left and/or right side of the virtual book 1700, are all “collected” onto the page 1705, resulting in a collection of pages 1706 as shown in FIG. 10B. This entire collection 1706 can then be flipped about as depicted in FIG. 10B.

[0103] FIG. 10C depicts a number of pages 1730-1733 being released from a collection of pages 1707 being held in mid-flip by a pointer 1710.

[0104] FIG. 11A depicts a hold-and-browse method of manipulating pages in a virtual book 1800. A single page and/or a collection of pages 1801 is first picked up by a pointer 1810. And then, this collection of pages 1801 and the right side thickness 1803 become a new “sub book” between which all kinds of browsing and flipping actions can take place for a number of pages (such as pages 1805-1808).

[0105] FIG. 11B depicts another hold-and-browse method of manipulating pages in a virtual book 1800 involving two collections (1801 and 1804) of pages. The two collections of pages 1801 and 1804 are being picked up by two pointers 1810 and 1811 respectively. These two collections of pages 1810 and 1811 then become a “sub book” between which all kinds of browsing and flipping actions can take place for a number of pages (such as pages 1805-1808). In general, any two pages and/or collection of pages can be made into a “sub book” between which the usual browsing and flipping action of a number of pages can take place.

[0106] FIG. 12 summarizes all the basic and complex flipping actions described above.

[0107] All of the complex flipping actions depicted in FIGS. 3-11 can be controlled by a mouse, one of the browsing controllers described in U.S. Pat. No. 5,909,207 (Ho), and one of these browsing controllers in conjunction with a mouse and/or other pointing and control methods. Specifically, the flipping actions can be controlled by a controller as depicted in FIG. 13.

[0108] The controller 3001 depicted in FIG. 13 combines several functions into a single controller. The remote controller 3001

• • [0109] 1. controls the virtual book's 204 flipping action;
  • [0110] 2. selects to display one or more menus to be displayed on the display screen 201;
  • [0111] 3. selects the items on the displayed menu displayed on the display screen 201, and
  • [0112] 4. effects certain actions such as the ejection of a DVD drive or the deletion of certain selected memory on the memory and control device 202.

[0113] The STANDBY button 3010 on the controller (computer) turns on or turns off the display screen 201 and/or some other circuitries in the memory and control device 202. The CLOSE button 3011 closes a menu or a virtual book 204 on the display screen 201, returns from a full-screen zoom (to be described below), or returns from thickness or bookmark selection mode (to be described below).

[0114] The ZOOM-LEFT and ZOOM-RIGHT buttons (3012 and 3013) enlarge the left hand or right hand page respectively to full screen mode. The UP and DOWN buttons (3014 and 3018) are for the upward and downward selection movement on a menu or on the bookmarks (105-107) displayed on the sides of the virtual book 204. (That is, in the absence of a mouse, the method to select, say, the various bookmarks would be first to select to enter the bookmark mode (e.g., by pressing button 3024 to be described below), and then use the UP and DOWN buttons (3014 and 3018) to highlight the various vertically placed bookmarks, and when a certain one is desired, the pressing of an “OK” button (3017, to be described below) would select it.)

[0115] The LEFT and RIGHT buttons (3015 and 3016) are for the leftward and rightward selection movement on a menu, on the bookmarks, on the thicknesses (101 and 102), or on the left and right pages. If no menu, bookmarks, thickness or other mode is entered, the LEFT and RIGHT buttons (3015 and 3016) are for the flipping of the pages to the left or right respectively. The OK button 3017 is for the confirmation of any selected option. The LEFT THICKNESS and RIGHT THICKNESS buttons (3019 and 3020) are for the selection of the left and right thicknesses (101 and 102) respectively on the virtual book (100 or 204). The BOOKMARK button 3021 is for selecting to enter the bookmark mode. In the bookmark mode, one can either select to add a bookmark to or remove a bookmark from a page or select to jump to a bookmarked page. The process is as follows. First the BOOKMARK button 3021 is pressed. The system then enters the bookmark mode. Then, suppose there are already a number of bookmarks (e.g., bookmarks 105-107 in FIG. 1A) present on the virtual book (100 or 204), the user can use the LEFT button 3015 to highlight indicate that he would like to select one of the bookmarks on the left thickness 101, and then use the UP and DOWN buttons (3014 and 3018) to select the particular bookmark that he desires. When the desired bookmark is selected, the pressing of the OK button 3017 will cause the virtual book 204 to flip to the bookmarked page, complete with thickness animation such as that shown in FIG. 1B.

[0116] The use of the LEFT and RIGHT THICKNESS buttons (3019 and 3020) are as follows. First, the user uses the buttons (3019 and 3020) to select the left or right thickness (101 and 102 respectively) on the virtual book (100 or 204). A jump cursor (110 or 111) will appear on the thickness indicating the position selected on the thickness. (The position selected on the thickness corresponds to a point in the virtual book 204.) To change the selected position on the thickness, the user uses the LEFT and RIGHT buttons to move the jump cursor (110 or 111 on virtual book 100) left or right until the desired position is reached. Then, with the pressing of the OK button 3017, the virtual book 204 will flip to the desired page, complete with thickness animation such as that shown in FIG. 1B.

[0117] The MENU button 3022, when pressed, displays a menu on the display screen 201 for the purpose of triggering a variety of actions. The AUTOFLIP button 3023, when pressed, causes the virtual book 204 to enter the auto-flipping mode in which the pages are automatically flipped without the need for the pressing of any flipping button. The FLIPTO button 3024 allows the user to select to flip to a number of desired pages. On pressing of the button 3024, a menu will appear on the display screen 201 and the user can select to flip to the front cover, back cover, or any desired page based on, say, the page number.

[0118] The COPY button 3025, when pressed, causes the contents of the memory on the memory and control device 202 to be copied onto another medium (such as a writable CD) inserted into the memory and control device 202. The EJECT button 3027, when pressed, causes the DVD, CD or other kinds of memory drive to be ejected from the memory and control device 202. The DELETE button 3026, when pressed, causes the memory in the memory and control device 202 to be deleted.

[0119] FIG. 14 shows the front face of an embodiment 4001 of the memory and control device 202. The memory and control device 4001 consists of a memory drive 4002 and a memory card slot 4003. The button 4004 is the power on/off button and the indicator light 4005 indicates whether the memory and control device 4001 has been turned on or off. The memory card slot 4003 allows a memory card, such as that removed from a digital camera after a certain number of pictures have been stored inside, to be inserted and read. The CD drive 4002 allows either a blank writable CD/DVD, a pre-recorded CD/DVD or other similar memory medium to be inserted to be written to or read from. The pressing of the COPY button 4006, say, can transfer all the data from the memory card in the memory card slot 4003 to the medium in the memory drive 4002. The indicator light 4007, when lit, indicates that the transfer is in progress. The DELETE button 4008, when pressed, causes the memory card in the memory slot 4003 to be deleted. The indicator light 4009, when lit, indicates that the deletion is in progress. The eject button 4010, when pressed, causes the memory drive 4002 to be ejected. The indicator light 4011, when lit, indicates that the ejection is in progress.

[0120] The memory drive 4002 and memory card slot 4003 described above are only particular embodiments. In general, these can be any kind of drive for reading and writing to any kind of memory medium and there can be more than two of these.

[0121] When the memory and control device 202 begins operation and before any memory medium is inserted, the display screen (5001, FIG. 15) is blank (other than showing, say, a frame 5002 as shown in FIG. 15, to indicate that the memory and control device 202 is turned on). When a memory card is inserted into the memory card slot 4003, if there is data on the card, say, placed into three separate folders/files, the screen 5001 will show three separate folders/files 5010, 5011, and 5012 with graphic/textual indication that they are from the medium in the memory card slot 4003. When a medium is inserted into the memory drive 4002, icons representing the folders/files in the medium (5020, 5021, 5022) are also shown on the display screen 5001.

[0122] With the folders/files on the screen, the user can use the UP/DOWN and LEFT/RIGHT buttons (3014, 3018, 3015 and 3016) to select the folder/file desired. When the desired folder/file is selected, pressing the OK button 3017 will cause a virtual book 204 to be displayed on the display screen 201 containing the contents of the selected folder/file.

[0123] The information displayed on the display screen 204 and controlled by the controller 203 can come from the memory medium within the memory and control device 202 or from the Internet, through either a wire or wireless connection, or from other devices.

[0124] More buttons can be added to the controller 203 to effect the other complex flipping actions such as those listed in FIG. 12 and depicted in FIGS. 3-11. These buttons can also be used to effect fine control of searching for the page to jump to by using a fine-control thickness region 150 as shown in FIG. 1E and described above.

[0125] The memory and control device 202 (FIG. 2) or 4001 (FIG. 14), other than deriving its data from a memory card inserted in its memory card slot 4003 (FIG. 14), can also derive the data to be shown on the screen 201 (FIG. 2) as a virtual book 204 from a CD or other memory media inserted into its memory drive 4002 (FIG. 14) or from a connection to the Internet, wireless or otherwise.

[0126] FIG. 16 depicts a system 1600 and a process in which a virtual book viewed on the screen can be transferred onto a CD or other kinds of memory medium 1605. In the process of viewing a virtual book 1604 on the screen 1601 using the memory and control device 1602, a button 1606 on the controller 1603 is pressed and a menu 1607 is popped up and a selection 1608 can be made to transfer the entire virtual book 1604 onto a CD or other media 1605. This process is different from that described above for transferring data directly onto the CD or other media 1605 using the COPY button 3025 (FIG. 13) or the COPY button 4006 (FIG. 14). In the above cases, the files are transferred as they are. In the process here depicted in FIG. 16, a virtual book 1604 is created and transferred onto the medium 1605 (CD or other media). What this means is, a recipient of the medium 1605 can insert the medium 1605 onto a computer, another memory and control device 1602, or other device that can read the medium 1605, and automatically the same virtual book 1604 will appear on the associated screen (e.g., 1601). In order to achieve this, one method is to transferred a viewer software along with the contents of the virtual book 1604 onto the medium 1605 so that it can re-create the virtual book 1604 on the associated screen (e.g., 1601) when the medium 1605 is read by whatever reading device.

[0127] Other than using a selection 1608 on a menu 1607 to activate the above virtual book transfer process, one can also use a specialized button 1609 on the controller 1603.

[0128] The controller 3001 depicted in FIG. 13, other than controlling the basic flipping actions depicted in FIG. 1 can also be used to control all the complex flipping actions depicted in FIGS. 3-12. This can be achieved by the buttons depicted in FIG. 13 on the controller 3001 or additional buttons added to the controller 3001. These buttons can be used by themselves or in combination with other buttons to launch these complex flipping actions (FIGS. 3-12).

[0129] Instead of using a controller (3001 in FIG. 13 or 203 in FIG. 2) to control all the basic flipping actions depicted in FIG. 1 and the complex flipping actions depicted in FIGS. 3-12 of a virtual book 204 displayed on a screen 201 (FIG. 2), a computer mouse or other control/input device can also be used.

[0130] FIG. 17-22 show one embodiment of a series of menus that appear on the screen 201 (FIG. 2) for effecting various actions on the system 200.

[0131] FIG. 17 depicts the menu 1701 when the system 200 is first started. The selections include View Memory Card 1702, View CD Contents 1703, Memory Card to CD Copy 1704 and Options 1705.

[0132] FIG. 18 shows the display on the screen 201 with a menu 1801 when View Memory Card 1702 in FIG. 17 is selected. It shows that there are three folders 1802, 1803, and 1804 on the memory card 4003 (FIG. 14) and the currently selected folder is 1802 CAN001. The Preview 1805 shows a part of the contents of the selected folder 1802 CAN001.

[0133] FIG. 19 shows the display on the screen 201 with a menu 1901 when View CD Contents 1703 in FIG. 17 is selected. It shows that there are three folders 1902, 1903, and 1904 on the memory drive (e.g., CD Drive) 4002 (FIG. 14) and the currently selected folder is 1902 London. The Preview 1905 shows a part of the contents of the selected folder 1902 London.

[0134] On the menu 1701 in FIG. 17, if the item Memory Card to CD Copy 1704 is selected, all the contents of the memory card 4003 will be copied into the CD (or other writable medium) in the memory drive 4002. This effect can also be achieved by pressing the COPY button 4006 on the memory and control device 4001 (FIG. 14). In general, the source of data to be copied onto a CD (or other writable medium) in the memory drive 4002 can come from either the memory card 4003 or other memory medium connected to the memory and control device 4001 through a USB or other connection or. The source of data can also come from the Internet through a connection to the Internet.

[0135] On the menu 1701 in FIG. 17, if the item Options 1705 is selected, the Options menu 2001 (FIG. 20) will appear in which one can select to set various parameters such as Sound Effects, Album Music, Album Theme, Memory Card, Compact Disc, and Autoplay Interval.

[0136] On the menu 1801 depicted in FIG. 18, if the current selected folder is, say, 1802 CAN001, and the OK button 3017 on the controller 3001 (FIG. 13) is pressed, a photo album/virtual book 2101 based on the contents of the folder 1802 CAN001 is created as shown in FIG. 21.

[0137] On the menu 1901 depicted in FIG. 19, if the current selected folder is, say, 1902 London, and the OK button 3017 on the controller 3001 (FIG. 13) is pressed, a photo album/virtual book 2101 based on the contents of the folder 1802 CAN001 is created as shown in FIG. 21.

[0138] As one is viewing the photo album/virtual book 2101 (FIG. 21), and the MENU button 3022 on the controller 3001 (FIG. 13) is pressed, a menu 2202 appears on the screen on top of the photo album/virtual book 2101 as depicted in FIG. 22. There are effects that can be applied to the photos or the contents on the left and right pages. The actions such as Delete, Rotate, and other editing functions can be applied to the left page contents using the Left Page selection 2203 and similarly for the contents on the right page using the Right Page selection 2204. There are other Album Options 2205 that can be selected including Sound Effects, Album Music, etc. The Burn Album to CD option 2206 in this menu 2202 allows the entire photo album/virtual book to be burned onto the medium (e.g., a Compact Disc, CD) in the memory drive 4002 (FIG. 14). When the Burn Album to CD command 2206 is activated, one method of the photo album/virtual book transfer to the memory medium (e.g., a CD) in the memory drive 4002 is such that should someone later insert the memory medium (e.g., the CD) into a memory drive either in a computer system, in another one of the system 200 (FIG. 2) or other systems configured to read from the memory medium, an entire photo album/virtual book will appear on the screen of any of these systems showing the same contents organized in the same manner as that shown on the screen 201 (i.e., the virtual book 2101). To achieve this, it normally entails the writing of a viewer software onto the memory medium together with the contents to be transferred onto the medium or CD. To make the viewing experience seamless, this viewer software can be made to auto-launch when the medium or CD is inserted into the memory drive of the recipient system.

[0139] When the user is on the menu 2001 (FIG. 20), he/she can elect to change the Album Music 2002 or Album Theme 2003. When any of these items is selected, they will be prompted to insert a memory medium into the memory card 4003 or the memory drive 4002 (FIG. 14) which contains either music or picture files. They can select one or more of these files to change the Album Music (the music that is played and accompanies the viewing of a photo album/virtual book 2101) or the Album Theme (which may include front cover, page, back cover, and other graphics on the photo album or virtual book).

[0140] When the user is using the controller 3001 (FIG. 13) and the button Thickness Right 3020 is pressed while viewing a virtual book 2101 (FIG. 23), a jump cursor 2302 appears on the right side thickness 2301 as shown in FIG. 23. Now, if the user presses the LEFT 3015 or RIGHT 3016 button (FIG. 13), the jump cursor will move to the left or right respectively to select a desired page. A number 2303 associated with the jump cursor 2302 may be displayed to show the page number of the selected page. The jump cursor may be later used for jumping to the selected/desired page. For example, when the jump cursor is positioned on a particular selected page and when the OK 3017 button (FIG. 13) is pressed, the virtual book 2101 will flipped to the selected page, with a representation of the pages skipped over in the form of an animation of a thick collection of pages moving across the screen as depicted in FIG. 1B, the thickness of the collection of pages being proportional to the number of pages skipped over. Correspondingly, a jump cursor can be displayed on the left thickness 2305 of the virtual book 2101 and controlled and used similarly.

[0141] All the control inputs in FIGS. 22, such as the one-button Bum Album to CD option 2206 on the menu 2202, all the actions on the left page 2203, on the right page 2204 and the other Album Options 2204 on the menu 2202, all the selections View Memory Card 1702, View CD Contents 1703, Memory Card to CD Copy 1704 and Options 1705 on menu 1701 (FIG. 17), all the selections on menus 1801, 1901, and 2001 in FIGS. 18-20, and all the basic and complex flipping actions listed in FIG. 12 can also be made available as specialized buttons on the memory and control device 202 or on the remote controller 203 (FIG. 2).

[0142] The memory media on the memory and control device 4001 (FIG. 14) such as the memory card 4003 and the memory drive 4002 that the system 200 (FIG. 2) write to or read from can also be resident in another physical device outside the memory control device 4001 and connected to the memory and control device 4001 through wired connection, wireless connections, or through the Internet.

[0143] Each of these embodiments may be configured to display the virtual book or comparable information on a computer, a regular or widescreen television (TV), or a regular or widescreen high definition television (HDTV) and/or to interface with a CD-R, CD-RW, a DVD, or a similar optical/non-optical portable large volume storage disk or non-disk medium.

[0144] While particular embodiments of the present invention have been illustrated and described, it will be appreciated that numerous changes and modifications will occur to those skilled in the art, and it is intended that the appended claims cover all those changes and modifications which fall within the spirit and scope of the present invention.