Graphic symbols and method and system for identification of same

[0002] This application is a Continuation-in-Part of application Ser. No. 08/738,813 filed Oct. 28, 1996, now abandoned which is hereby incorporated herein by reference.

[0003] The present invention relates to a method for note taking information from documents. In particular, the invention relates to graphics containing machine readable spatial symbology.

[0004] Non-electronic methods for taking notes from books, documents, and other printed materials are generally slow or unreliable. Students, lawyers, and other people dependent upon prior authority have limited resources by which they may select desired portions of a document and reproduce them quickly and accurately. Generally, this requires marking selected passages from printed documents with tabs or highlighters and reproducing them manually at a later time. For example, U.S. Pat. No. 3,958,816 to Remmey, III discloses notation related book markers wherein double sided tabs may be adhered to pages of books. Numbers are written in opposite directions on opposite sides of the tab so that the tab may be affixed to a left or right page of a book to mark the page, and the point on the page, to which it refers. Key cards have numbers corresponding to the tabs. Notes are hand written on lines on the cards and the cards are attached to the front leaf of a book.

[0005] Recent developments in the area of optical character recognition (OCR) scanning, such as that disclosed in U.S. Pat. No. 5,920,877 to Kolster, make it possible to acquire discrete text strings and organize those strings in a preselected format for later use. Such devices, however, depend upon the scanner's ability to recognize a character based on known OCR techniques, such as geometric OCR, which detects the printed character's shape. Variations in fonts make it difficult to achieve total accuracy with geometric OCR.

[0006] U.S. Pat. No. 5,875,261 to Fitzpatrick et al. discloses a method of enhancing spatial character recognition by combining it with color coded OCR. Color coded OCR is a technique involving attempts to recognize a character based on color embedded in the character. For example, red may denote the letter “A”, whereas pink denotes the letter “a” and yellow indicates the letter “Y”. Color coding, by itself, may be subject to color processing errors involving color intensity, color density, color shifts, and color scanner misalignment. Anything effecting the spectrum, such as saturation or changes in hue, could also effect the processing. Consequently, Fitzpatrick et al. mixes color coding with geometric OCR and utilizes a hypothesis testing technique wherein the geometric OCR is used to generate a null hypothesis that a particular character has been recognized, and an alternative hypothesis that the character has not been recognized. Color coded OCR is then employed to select either the null hypothesis or the alternative hypothesis. This method increases accuracy of the scanning device, however, it is time consuming and requires color printing.

[0007] Barcode recognition and processing methods, such as those disclosed in U.S. Pat. No. 4,992,650 to Somerville, U.S. Pat. No. 5,227,616 to Lee, U.S. Pat. No. 5,229,584 to Erickson, and U.S. Pat. No. 5,451,760 to Renvall, provide optical scanning techniques with both speed and accuracy. Barcodes can be used for automatic reproduction of strings of printed marks and characters, however, barcode detracts from a printed page to be read by humans.

[0008] FIG. 1 illustrates a system in accordance with one embodiment of the present invention;

[0009] FIG. 2(a) illustrates, in perspective view, an example of an optical scanning device which may be used in preferred embodiments of the present invention;

[0010] FIG. 2(b) illustrates, in cut away view, the optical scanning device of FIG. 2(a);

[0011] FIG. 3(a) illustrates a graphic symbol in accordance with an embodiment of the present invention wherein spatial symbology is embedded in a single area of graphic symbol;

[0012] FIG. 3(b) illustrates a graphic symbol in accordance with an embodiment of the present invention wherein morse code is embedded in the graphic symbol;

[0013] FIG. 3(c) illustrates a graphic symbol in accordance with an embodiment of the present invention wherein barcode is embedded in the graphic symbol;

[0014] FIG. 3(d) illustrates a graphic symbol in accordance with an embodiment of the present invention wherein colored spatial symbology is embedded in the graphic symbol;

[0015] FIG. 3(e) illustrates a graphic symbol in accordance with an embodiment of the present invention wherein spatial symbology is embedded in a plurality of areas of the graphic symbol;

[0016] FIG. 4 is a flow chart illustrating a method in accordance with an embodiment of the present invention wherein graphic symbols are indicated by spatial symbology;

[0017] FIG. 5 is a flow chart illustrating a method in accordance with an embodiment of the present invention wherein graphic symbols are identified using spatial symbology and geometric OCR techniques.

[0018] FIG. 6 is a flow chart illustrating a method for recording selected passages from written materials.

[0019] FIG. 7 is an illustration of a document in accordance with an embodiment of the present invention.

[0020] A system is provided for recording and reproducing graphic symbols. An embodiment of the system includes an optical scanning device and a plurality of graphic symbols. The graphic symbols include a spatial symbology embedded within the graphic symbol for detection by the optical scanning device. The embodiment also includes a processor in communication with the optical scanning device for identifying graphic symbols indicated by the spatial symbology. The graphic symbols may be alphanumeric symbols, icons, or pictures. The spatial symbology may be black, white, grey, or any color. Further, the spatial symbology may be the same color as the graphic symbol or a different color. The optical scanning device and the processor may be in electrical communication or wireless communication. The system may also include a microphone in communication with the processor which may be used to orally annotate notes taken with the system. The microphone may be incorporated into the optical scanning device.

[0021] A graphic symbol for use with an optical scanning device is also provided. The graphic symbol includes configurations which may be distinguished with a human eye and spatial symbology embedded within the configurations. The spatial symbology is detectable by the optical scanning device for storage and identification. In a preferred embodiment, configurations are alphanumeric. In another embodiment, the spatial symbology indicates the graphic symbol in which it is embedded. In a further embodiment, the spatial symbology indicates a graphic symbol different from the graphic symbol in which it is embedded.

[0022] The spatial symbology embedded within the configurations may include one dimensional barcode, two dimensional barcode, morse code or the like or a combination of codes. The spatial symbology may be confined to one area of the graphic symbol, or may be spread out to include a plurality of areas of the graphic symbol. The spatial symbology may be the same color as the configurations or a different color. Embodiments of the invention include documents bearing the graphic symbols. A document may be in physical or electronic form.

[0023] A method for recording graphic symbols is also provided. An embodiment of the method includes providing a plurality of graphic symbols, the graphic symbols including a spatial symbology embedded within the graphics. The graphic symbols are scanned with an optical scanning device and the spatial symbology is transmitted to a processor. The processor has a memory and is in communication with the optical scanning device. The graphic symbol is identified by the spatial symbology in the processor. In accordance with another embodiment of the invention, the graphic symbol is identified using geometric OCR techniques in addition to detection of the spatial symbology.

[0024] The graphic symbols indicated by the spatial symbology are identified in the processor and may be reproduced in electronic form with a computer monitor. Alternatively, or in addition, the optical scanning device is in communication with a printing device and the graphic symbol is reproduced in physical form by the printing device.

[0025] A computer program product for recording and reproducing graphic symbols is also provided. The computer program product comprises a computer usable medium having computer readable program code. The program code includes code for reading spatial symbology detected by the optical scanning device and code for identifying graphic symbols indicated by the spatial symbology. The computer program product may also include code for storing the graphic symbols in a file and program code for translating the spatial symbology detected by the optical scanning device into an image of the graphic symbols.

[0026] An additional aspect of the invention involves a method for recording selected passages from written materials is provided. A multiple page text containing spatial symbology embedded in a plurality of graphic symbols is provided. The graphic symbols corresponding to passages, words, or other text selected by a user is scanned with an optical scanning device and the scanning device produces a computer readable representation of the graphic symbols being scanned. The computer readable representation is then transmitted to a processor which interprets the read signals using the spatial symbology and a file containing the selected passages is created.

[0027] FIG. 1 illustrates a system in accordance with one embodiment of the present invention. A document (e.g., a book, a webpage, a legal brief, etc) 10 contains a plurality of graphic symbols 12. A graphic symbol may be a number, letter, picture, icon or any other mark or image contained in the data base of a processor. The graphic symbols 12 include spatial symbology 14 embedded in the graphic symbol which is scanned by an optical scanning device 16. For purposes of this disclosure, “spatial symbology” is defined as an arrangement of marks, symbols or images, without regard to color, configured to express a data message detectable and interpretable by a scanning device and microprocessor, such as but not limited to a pattern of wide and narrow spaces, dots, lines, or any layout of shapes and spaces. The optical scanning device 16 may be in wireless 13 communication (e.g., using radio frequency or infrared radiation) with a processor 18 which identifies the graphic symbols 12 indicated by the spatial symbology 14. The processor 18 interprets the scanner read signals and retrieves a corresponding graphic symbol from an accompanying digital file, such as a CD. In a preferred embodiment, the graphic symbols may be alphanumeric characters, each of which are associated with code in the file, such as an ASCII code. The processor 18 may store the graphic symbols for further manipulation or reproduce them physically or electronically.

[0028] In a most straight forward approach, the spatial symbology 14 represents and is interpreted as the graphic symbol in which it is embedded. In this way, improved character recognition may be accomplished. In an alternate approach, the spatial symbology 14 can be used to provide a hidden meaning. Spatial symbology embedded in the letters of a text may represent and be interpreted as a message unrelated to the text in which it is embedded. In this alternate approach, the spatial symbology and the graphic symbol in which it is embedded do not necessarily match.

[0029] FIGS. 2(a) and 2(b) illustrate an example of an optical scanning device which may be used in preferred embodiments of the present invention. The scanning device 20 is described in greater detail in U.S. Pat. No. 5,301,243 to Olschafskie et al. and U.S. Pat. No. 5,574,804 also to Olschafskie et al. both of which are incorporated herein by reference. It is shown scanning a document 22. The scanning device 20 includes a housing 24 adapted to be held in the manner of a pen and moved easily. A movement sensor 26 is mounted to the housing 24 for detecting movement of the scanner across the surface of the document 22. An optical system 28 includes a lens 21 and a diode array detector 23 (FIG. 2(b)). The scanning device 20 makes contact with or projects a beam of light or laser over, across, or along graphic symbols. The computer readable representations are forwarded to a microprocessor. Scanning device 20 produces a computer readable representation of the graphic symbols being scanned with sufficient resolution to detect spatial symbology embedded within the graphic symbol.

[0030] FIG. 3(a) illustrates a graphic symbol in accordance with an embodiment of the present invention wherein spatial symbology 30 is embedded in a confined area within the graphic symbol 32. As stated above, the spatial symbology 30 may be any arrangement of marks configured to express a data message detectable and interpretable by a scanning device and microprocessor such as, but not limited to, a printed pattern of narrow spaces.

[0031] FIG. 3(b) illustrates a graphic symbol in accordance with an embodiment of the present invention wherein morse code 34 is embedded in the graphic symbol 32. Morse code 34 includes any series of dashes and dots.

[0032] FIG. 3(c) illustrates a graphic symbol in accordance with an embodiment of the present invention wherein barcode 36 is embedded in the graphic symbol 32. The barcode 36 may be any printed pattern of narrow spaces aligned vertically, horizontally, or circularly with respect to the document. Alternatively, the bar code may be a two-dimensional array of spaces (e.g., a checkerboard or bull's eye configuration).

[0033] FIG. 3(d) illustrates a graphic symbol in accordance with an embodiment of the present invention wherein colored spatial symbology 38 are embedded in the graphic symbol 32. It should be noted that, in this embodiment, color is employed to distinguish the spatial symbology from other marks in the graphic symbol. Thus, the particular color used is not relevant. Indeed, “color” as used herein refers to black, white, and gray in addition to the full spectrum of colors. When color is used to make the spatial symbology, the scanning device must use its ability to discriminate between colors. Software will be included for use by the processor 18 to use appropriate thresholds to distinguish the spatial symbology marks from the graphic symbol.

[0034] FIG. 3(e) illustrates a graphic symbol in accordance with an embodiment of the invention wherein spatial symbology 31 is embedded in a plurality of areas of the graphic symbol 32. In this embodiment, the arrangement of spatial symbology within the graphic symbol 32, shown here by the dotted lines 39, supplies the information necessary to identify the graphic symbol in the processor.

[0035] Printing of the above-described graphic symbols can be accomplished by providing a new font in a printing utility program that incorporates the spatial symbology into the graphic symbol. A more preferred method of printing would be to provide a spatial symbology overlay that could be used with most or all fonts. A printing utility to accomplish the spatial symbology overlay would eliminate pixels of the underlying graphic symbol and substitute the corresponding spatial symbology. Such an overlay would work effectively with most or all fonts and the resulting printed characters would permit highly accurate character recognition based upon the spatial symbology.

[0036] FIG. 4 is a flow chart illustrating a method in accordance with an embodiment of the present invention wherein graphic symbols are recognized by embedded spatial symbology. Graphic symbols as described in FIGS. 3 (a)-(e), wherein spatial symbology is embedded in the graphic symbol, are provided on a document 41. The embedded spatial symbology is preferably of small size such that it is undetectable by the human eye but is detectable by an optical scanning device. The graphic symbols are scanned 42 with an optical scanning device which produces a computer readable representation of the graphic symbols being scanned 43. The computer readable representation is then transmitted to a processor 44 and the processor interprets the spatial symbology and retrieves a corresponding graphic symbol from an accompanying digital file on a hard drive, CD-Rom, or the like 45. Graphic symbols may be identified by a digital code, such as the ASCII code for alphanumeric characters. The identities of the selected graphic symbols may then be stored for later use or reproduced in physical or electronic form as 46.

[0037] FIG. 5 is a flow chart illustrating a method in accordance with an embodiment of the present invention wherein graphic symbols are identified using spatial symbology and spatial OCR techniques. Graphic symbols are provided wherein spatial symbology is embedded in the graphic symbol 51. The graphic symbols are scanned 52 with an optical scanning device which produces a computer readable representation of the graphic symbols being scanned 53. The computer readable representation is then transmitted to a processor 54. The processor interprets the scanner read signals using the spatial symbology and spatial OCR techniques. Any indeterminancy in one technique can be rectified by considering the results of the other technique. The processor retrieves the determined graphic symbol from an accompanying digital file on a hard drive, CD-Rom, or the like 55. The identities of the selected graphic symbols may then be stored for later use or reproduced in physical or electronic form 56.

[0038] FIG. 6 is a flow chart illustrating a method for recording selected passages from written materials. A multiple page text containing spatial symbology embedded in a plurality of graphic symbols is provided 61. The graphic symbols corresponding to passages, words, or other text selected by a user is scanned with an optical scanning device 62. The optical scanning device produces a computer readable representation of the graphic symbols being scanned 63. The computer readable representation is then transmitted to a processor 64. The processor interprets the read signals using the spatial symbology 65. A file containing the selected passages is created 66.

[0039] FIG. 7 is an illustration of a document 70 in accordance with an embodiment of the present invention. The document 70 may be in physical, printed form such as a book, page, card, etc. The document 70 may also be in electronic form such as a word processing document or a webpage. The document 70 includes graphic symbols 72 which may be distinguished and interpreted by a human eye and the graphic symbols 72 include spatial symbology 74 embedded in the graphic symbol 72 which is invisible to the human eye but detectable by an optical scanning device. The graphic symbols are typically alphanumeric and punctuation symbols.

[0040] The methods and apparatuses of the present invention provide a fast, reliable way to store, record, and reproduce written text. Students, and others relying on prior authority, may simply scan a document and have any part of it stored for later use or reproduced instantaneously in electronic or physical form. While scanning, oral annotations may be recorded through a microphone. Research and citations for thesis papers, essays, legal briefs, research papers, and other documents may be compiled quickly and efficiently. Other uses and advantages of the invention will be apparent to those skilled in the art.

[0041] It should be understood that various changes and modifications to the preferred embodiment described above will also be apparent to those skilled in the art. For example, the graphic symbols may be letters or symbols associated with any alphabet or even pictures or pictographs. Any of a variety of code reading algorithms may be employed alone or in combination with any of a variety of additional code reading or character recognition algorithms to identify a scanned graphic symbol. These and other modifications can be made without departing from the spirit and scope of the invention and without diminishing its attendant advantages. It is therefore intended that such changes and modifications be covered by the following claims.