Apparatus with a display magnification changing function of annotation

[0002] This invention relates to a data processing apparatus having an annotation display function that reduces or enlarges the display image of the annotation pasted on a text as a result of being inputted through the annotation input window and that superposes the display image with the changed magnification in a specified position on the display image on the text display screen.

[0003] With a conventional data processing apparatus with an annotation display function that displays the annotations pasted on text, in a case where the user carries out the operation of pasting an annotation in a specific position on text, the annotation being entered on a window screen (an annotation input window), when the user displays the text again, a mark (an annotation mark) 22 indicating the fact that the annotation has been pasted is displayed as shown in FIG. 22.

[0004] When the user wants to check the contents of the annotation pasted in the position in which the annotation mark 22 appears, the user has to click the mark 22 to display the window screen again and check the contents of the annotation.

[0005] As described above, with the conventional data processing apparatus with an annotation display function, when an annotation is pasted on text, only a mark indicating that the annotation has been pasted is displayed on the text display screen as shown in FIG. 22. The mark alone prevents the user from checking the contents of the annotation written in the text at a single glance. Thus, the user has to repeat troublesome operations: that is, the user has to take the trouble to click the mark to display the contents of the annotation again on the window screen, check the contents, and then erase the window screen again to read the text.

[0006] It is, accordingly, an object of the present invention to provide a data processing apparatus with an annotation display function that enables the user to check the contents of annotations while reading the text, by reducing or enlarging the display image of an annotation input window with a specified magnification in displaying annotations and by superposing the display image in a specified position on a text display image on a text display screen.

[0007] Another object of the present invention is to provide a data processing apparatus with an annotation display function that enables the user to check the contents of annotations easily even when a simple reduction of annotations makes the annotations illegible because annotation data items in the annotations overlap with each other, by displaying a reduced display image of the annotations from which the overlap of annotation data items has been removed.

[0008] Still another object of the present invention is to provide a data processing apparatus with an annotation display function that enables the user to check the contents of annotations easily even when a simple reduction of annotations makes the annotations illegible because the size of annotation data items in the annotations is small, by reducing or enlarging only the area from which blanks have been removed and thereby making the contents of the annotations larger with the entire display size remaining unchanged.

[0009] Still another object of the present invention is to provide a data processing apparatus with an annotation display function that enables the user to check the contents of annotations easily by sensing annotation data items whose contents would be difficult to check if the annotations were simply reduced and displayed with a specified magnification because the display size is small and by using the smallest magnification that prevents the display size after the magnification change from being smaller than a display limit value for the annotation data items, thereby displaying the contents of the annotations larger with the entire display size remaining unchanged.

[0010] A data processing apparatus with an annotation display function according to the first aspect of the present invention is characterized by comprising: annotation input means for taking in the individual data items constituting an annotation pasted in a specified annotation position (annotation writing position) on a text as a result of the annotation being inputted from an annotation input window and acquiring the input positions of the annotation data items and information on their size; annotation image magnification change means for changing the display magnification of a display image of the annotation on the basis of the input positions of the individual annotation data items in the annotation and information on their size acquired by the annotation input means and a specified magnification; and text and annotation image display means for putting and displaying the display image of the annotation whose magnification has been changed by the annotation image magnification change means in the specified annotation position on a text display image. The specified magnification may be prepared beforehand in the system (a system-specified magnification) or a user-specified magnification. When the user does not specify any magnification in particular, a default magnification may be used. When the user has specified a magnification, the magnification may be used in place of the default magnification.

[0011] With such a configuration, the annotation image magnification change means creates a reduced or enlarged image of the display image of the annotation on the basis of the input positions of the individual annotation data items in the annotation and information on their size acquired by the annotation input means. The reduced or enlarged display image of the annotation is put and displayed in the specified annotation writing position on a text display image by the text and annotation image display means. The size of the reduced or enlarged display image of the annotation is the size of the annotation input window at annotation setting reduced or enlarged with the specified magnification.

[0012] Because the reduced or enlarged display image of the annotation can be superposed on the display image of the text on the text display screen, the user can check the contents of the written annotation while reading the text and need not take the trouble to use another window only for the purpose of checking the contents of the annotation, differently from a conventional equivalent.

[0013] In a case where the size of the annotation display image on the text display screen is made changeable, for example, when the contents have been checked, the user can reduce the size of the annotation display image substantially to make a display as if the conventional annotation mark were displayed. By the user's operation, all of the annotation display image or only the user-specified annotation display image on the text display screen may be erased.

[0014] A data processing apparatus with an annotation display function according to the second aspect of the present invention is such that annotation data display overlap removing means for sensing an area where at least two annotation data items overlap with each other, determining only one data item to be displayed in the overlapping area, and acquiring specific information indicating the overlapping area and an annotation data item to be displayed in the area is added to the data processing apparatus with an annotation display function according to the first aspect. The data processing apparatus of the second aspect is characterized in that the annotation image magnification change means changes the display magnification of a display image of the annotation where the display image portion of the annotation data items except for the annotation data item to be displayed in the overlapping area has been subtracted from the overlapping area indicated by the specific information acquired by the annotation data display overlap removing means and the display image of the annotation whose magnification has been changed is put and displayed in the specified annotation position on a text display image.

[0015] The approach of sensing an area where at least two annotation data items overlap with each other and determining only one annotation data item to be displayed in the overlapping area may be the approach of displaying the earliest or latest of the relevant annotation data items or the approach of prioritizing the individual types of annotation data items and displaying the one with the highest level of priority. In this case, the annotation data items may be rearranged in the order in which they have been entered, or in the opposite order, or in ascending order or priority level, taking out the annotation data items (called the first annotation data items) from the beginning one, sensing an area where the annotation data items (called the second data items) after the first data items overlap with each other, and registering information on the area so as to correspond to the first annotation data items as information on an area not to be displayed in the first annotation data items in displaying the display image of the first annotation data items.

[0016] With such a configuration, a reduced or enlarged image from which the overlap of annotation data items has been removed is created and put in the annotation position on the text display image.

[0017] Therefore, even when a simple reduction of an annotation makes the contents of the annotation illegible because annotation data items in the annotation overlap with each other, the contents of the annotation can be checked easily because a reduced display image of the annotation from which the overlap of annotation data items has been removed.

[0018] A data processing apparatus with an annotation display function according to the third aspect of the present invention is such that annotation blank area separating means for sensing the smallest rectangular area including the area in which individual annotation data items have been inputted from an annotation input window is added to the data processing apparatus with an annotation display function of the first aspect. The data processing apparatus of the third aspect is characterized in that annotation image magnification change means changes the display magnification of a display image in the rectangular area to change the magnification of only the display image of the annotation data items to be displayed in the rectangular area acquired by the annotation blank area separating means, and the display image of the annotation whose magnification has been changed is put and displayed in the specified annotation position on a text display image.

[0019] With such a configuration, only the smallest rectangular area including the area in which the annotation data items have been inputted from the annotation data input window, or only the area from which the blank has been removed, is subjected to magnification change display. For example, only the smallest rectangular area is subjected to reduced display and the reduced image is superposed in the specified annotation position on the text display image. Therefore, even when a simple reduction of an annotation makes the contents of the annotation illegible because the size of annotation data items in the annotation is small, the contents of the annotation can be displayed larger, making the contents legible.

[0020] A data processing apparatus with an annotation display function according to the fourth aspect of the present invention is such that annotation data magnification adjusting means which checks to see if the display size with a specified magnification is smaller than the display limit value indicating the limit value of a display size the user can identify and which, if the display size is smaller than the display limit value, determines the smallest magnification necessary for making the display size of the display image of the annotation data item larger than the display limit value to be the magnification for the annotation data item and, if the display size is not smaller than the display limit value, determines the specified magnification to be the magnification for the annotation data item is added to the data processing apparatus with an annotation display function of the first aspect. The data processing apparatus of the fourth aspect is characterized in that annotation image magnification change means changes the display magnification of a display image of the annotation using the magnification for each annotation data item determined by the annotation data magnification adjusting means and the display image of the annotation whose magnification has been changed is superposed in the specified annotation position on a text display image.

[0021] With such a configuration, annotation data items whose contents would be illegible because their display size is small if they were simply reduced with a specified magnification, that is, annotation data items whose display size reduced with a specified magnification is smaller than the display limit value determined by the display capability (resolution) of the display unit and the type of data (whether the annotation is of text character or of line drawing), are sensed. The smallest magnification that prevents the display size after the magnification of the annotation data items have been changed from being smaller than the display limit value is applied to the annotation data items. This enables the contents of the annotation to be displayed larger with the entire display size remaining unchanged. As a result, the user can check the contents of the annotation easily.

[0022] Additional objects and advantages of the present invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the present invention.

[0023] The objects and advantages of the present invention may be realized and obtained by means of the instrumentalities and combinations particularly pointed out in the appended claims.

[0024] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate presently preferred embodiments of the present invention and, together with the general description given above and the detailed description of the preferred embodiments given below, serve to explain the principles of the present invention in which:

[0025] FIG. 1 is a block diagram of a schematic configuration of a data processing apparatus with an annotation display function according to a first embodiment of the present invention;

[0026] FIG. 2 is an example of displaying a text display screen and an annotation input window simultaneously;

[0027] FIG. 3 is a diagram to help explain the coordinate system on the display screen applied in the first embodiment;

[0028] FIG. 4 is a diagram to help explain the bounding box of annotation data on the display screen applied in the first embodiment;

[0029] FIG. 5 is a flowchart to help explain the operation of the annotation image magnification change section 105 and the text and annotation image display section 109 in the configuration of FIG. 1;

[0030] FIG. 6 is a flowchart to help explain the operation of the annotation image magnification change section 105 and the text and annotation image display section 109 in a case where an approach different from that of FIG. 5 is applied;

[0031] FIG. 7 shows an example of text and an annotation display image appearing one on top of the other in the configuration of FIG. 1;

[0032] FIG. 8 is a block diagram of a schematic configuration of a data processing apparatus with an annotation display function according to a second embodiment of the present invention;

[0033] FIG. 9 is a flowchart to help explain the operation of the annotation data display overlap removing section 120 of FIG. 8;

[0034] FIG. 10 is a flowchart to help explain the operation of the annotation image magnification change section 125 and the text and annotation image display section 129 in the configuration of FIG. 8;

[0035] FIG. 11 shows an example of displaying a text display screen and an annotation input window simultaneously;

[0036] FIG. 12 shows the display image of the annotation input window of FIG. 11 reduced by the approach applied in the first embodiment and superposed on text;

[0037] FIGS. 13A and 13B show the annotation display image from which the overlap of the annotation data items entered from the annotation input window of FIG. 11 has been removed by the annotation data display overlap removing section 120 and also shows the annotation display image reduced and displayed on the text display screen.

[0038] FIG. 14 is a block diagram of a schematic configuration of a data processing apparatus with an annotation display function according to a third embodiment of the present invention;

[0039] FIG. 15 is a flowchart to help explain the operation of the annotation blank area separating section 130, the annotation image magnification change section 135, and the text and annotation image display section 139 in the configuration of FIG. 14;

[0040] FIGS. 16A and 16B show examples of displaying an annotation only at the top left corner of the annotation input window and also shows the display image of the annotation input window reduced by the approach applied in the first embodiment and superposed on text;

[0041] FIGS. 17A to 17D show the area obtained by removing the blank from the annotation data inputted from the annotation input window of FIGS. 16A and 16B and the annotation display image obtained by reducing the area, together with the annotation display image reduced by the approach applied in the first embodiment;

[0042] FIG. 18 shows the reduced image of the area from which the blank has been removed in FIGS. 17A to 17D, on the text display screen;

[0043] FIG. 19 is a block diagram of a schematic configuration of a data processing apparatus with an annotation display function according to a fourth embodiment of the present invention;

[0044] FIG. 20 is a flowchart to help explain the operation of the annotation data magnification adjusting section 140 in the configuration of FIG. 19;

[0045] FIGS. 21A to 21C show examples of displaying a string of text characters of a small font at the top of the annotation input window, the display image of the annotation input window reduced by the approach applied in the first embodiment and superposed on text, and an example of displaying text with a larger font on the annotation display image by adjusting the magnification with the annotation data magnification adjusting section 140 of FIG. 19; and

[0046] FIG. 22 shows a conventional text display screen with an annotation mark indicating that an annotation has been pasted on text.

[0047] Hereinafter, referring to the accompanying drawings, embodiments of the present invention will be explained.

[0048] FIG. 1 is a block diagram of a schematic configuration of a data processing apparatus with an annotation display function according to a first embodiment of the present invention. In the first embodiment, for convenience of explanation, it is assumed that a preset annotation to be displayed is read from a text stored in a text file. A preset annotation to be displayed may be read from the text received via a channel from a network. Additionally, the annotation may be taken in to open the annotation input window on the text display screen, prompt the user to enter an annotation on the window with a coordinate input device, such as a tablet, reduce or enlarge the display image of the window with a specified magnification, and then display the window again. The same holds true in a second to a fourth embodiment explained later.

[0049] The “annotation” in this specification is an annotation appearing on a window (an annotation input window) different from the text window and linked to a specific position on a text. Namely, an annotation is different from an annotation that is the same as text data in terms of data expression.

[0050] In a text file 100, texts and annotations set in the texts, and information on the positions in which the annotations are written on the texts are stored. The annotation is set so as to correspond to the text by the user opening the annotation input window 20 on the text display screen 111 as shown in FIG. 2, entering the desired annotation data from the window 20 using the coordinate input device, and specifying the position in which the annotation is to be written. The annotation is composed of a string of coordinate values of the individual annotation data items inputted from the annotation input window 20 (when characters and graphs are entered by handwriting with the coordinate input device) or the input coordinates and code data (when characters are inputted from the keyboard), and information on the size of the annotation input window 20 at the end of the input. While the annotation input window 20 is of the fixed size when the user has opened the window on the screen, the window can be enlarged or reduced to the size determined by its bottom right end position (acting as a reference) and its top left end position, when the user moves the window's bottom right end with a pen.

[0051] The annotation input section 102 takes in the annotation set so as to correspond to the text stored in the text file 100, creates information on the positions of the individual annotation data items constituting the annotation (referred to as input annotation data items) on the annotation input window 20 and their size, and creates display annotation data made up of the information and the original annotation data.

[0052] On the basis of the display annotation data obtained from the annotation input section 102, a magnification of R preset in a magnification storage section 104, and the size of the annotation input window at the time of annotation input, the annotation image magnification change section 105 reduces or enlarges the display image (annotation display image) of the annotation input window on which the annotation has been written and creates an annotation display image with the changed magnification.

[0053] Finally, the text and annotation image display section 109 creates a display image by superposing the annotation display image with the changed magnification generated by the annotation image magnification change section 105 in the user-specified annotation writing position (annotation position) on the text display image 107 and displays it on the text display screen 111 of a display unit 110. The text display image 107 is created by a text image creating section (not shown) on the basis of the text stored in the text file 100.

[0054] With the present embodiment, the display image of the annotation can be superposed, lying on the text display image 107 on the text display screen 111. Therefore, differently from the conventional equivalent, the user can check the contents of the annotation without opening the window displaying the contents of the annotation, while reading the text.

[0055] The coordinate system of the display screen is determined as shown in FIG. 3. The top left end of the screen is determined to be the origin (0, 0). The distance to the right of the origin is taken as the x-coordinate and the distance below the origin is taken as the y-coordinate. The right direction of the x-coordinate axis is determined to be positive and the down direction of the y-coordinate axis is determined to be positive. Here, “annotation data item” means a partial data string constituting an annotation. In FIG. 4, “Search Required,” the line-drawing characters entered freehand, is one annotation data item D1, and “DEADLINE:97/1”, the text character string, is another annotation data item D2.

[0056] Methods of organizing the input data items into a single annotation data item include (1) a method of organizing the input data items in the order in which the data items have been inputted, (2) a method of the user specifying the way to organize the data items after the data input, and (3) a method of the system automatically organizing the data items on the basis of the arrangement of the data items on the screen. While, in the example of FIG. 4, the freehand characters “Search Required” is a set of annotation data items (D1), they may be organized into several separate annotation data items.

[0057] Take the annotation data item as shown in FIG. 4 as in example. In the example of FIG. 4, annotation data item D1 is displayed in the area whose top left end's coordinates are (x1, y1), whose top right end's coordinates are (x1+w1, y1), whose bottom left end's coordinates are (x1, y1+h1), and whose bottom right end's coordinates are (x1+w1, y1+h1). The top left end's coordinates (x1, y1) are called the “display position” of annotation data item D1, or just called “position.” A combination (x1, y1, w1, h1) of the coordinates (x1, y1) and the width (w1) and height (h1) of the annotation data item is called a bounding box of annotation data item D1.

[0058] The operation of the configuration of FIG. 1 will be described by reference to the flowchart of FIG. 5.

[0059] In general, more than one annotation can be put on text. To display the display image of more than one annotation on text, the process for displaying an annotation is repeated as many times as there are annotations. Steps 200 to 202, and step 207 are steps for the repetitive process. Steps 203 to 206 enclosed by a broken-line box indicate the process of displaying the display image of one annotation on text.

[0060] The annotation image magnification change section 105 converts the contents of the annotation input window 20 entered by the user from the window into a display image by existing image processing techniques (step 203). Namely, the annotation image magnification change section 105 creates a screen image (an annotation display image) at the time when the user has entered an annotation from the annotation input window 20 by image-developing all the annotation data items of the annotation.

[0061] Then, on the basis of the magnification R (a magnification once the original is determined to be 1) indicted by the magnification storage section 104, the annotation image magnification change section 105 reduces or enlarges the annotation display image to the size (Wk*R, Hk*R) by existing image processing techniques (steps 204 and 205).

[0062] The text and annotation image displaying section 109 superposes the annotation display image whose magnification has been changed at the annotation image magnification change section 105 on the text display image 107 in the annotation position (xk, yk) on the text display screen 111 specified by the user for the relevant annotation stored in the text file 100 (step 206).

[0063] If the total number of annotations is N, the above processes will be repeated N times from k=1 to k=N (steps 200 to 202, step 207).

[0064] In the explanation, the magnification (the magnification of the annotation display image) of the annotation input window has been changed at the annotation image magnification change section 105 by the first approach of imaging the entire annotation input window and then reducing or enlarging the whole image. The magnification may be changed by the second approach of reducing or enlarging each of the individual images constituting the annotation.

[0065] Hereinafter, the operation of the annotation image magnification change section 105 and the text and annotation image display section 109 by the second approach (corresponding to the processing routine enclosed by a broken-line box in the flowchart of FIG. 5) will be explained by reference to a flowchart in FIG. 6.

[0066] The annotation image magnification change section 105 determines the size of the display image whose magnification of the annotation input window displayed on the text display screen 111 to be (Wk*R, Hk*R) from the size (Wk, Hk) of the annotation input window 20 and magnification R (steps 210, 211).

[0067] Next, the annotation image magnification change section 105 converts the coordinates (xi, yi) of the i-th (i is in the range of 1 to M) annotation data item on the annotation input window 20 before the change of magnification into the coordinates (xi*R, yi*R) on the annotation window after the change of magnification (at steps 210, 211) on the basis of the size (wi, hi) of the display image of the annotation data item. The annotation image magnification change section also changes the size of the display image (annotation display image) of the annotation data item with a magnification of (wi*R, hi*R). Receiving this, the text and annotation image display section 109 displays the display image of the i-th annotation data item with a size of (wi*R, hi*R) at the coordinates (xi*R, yi*R) on the annotation input window with the changed magnification (steps 215, 216).

[0068] The above processes are repeated M times for each annotation data item, starting i=1 up to M (steps 212 to 214, step 217).

[0069] As described above, the annotation image magnification change section 105 creates an annotation display image by changing the magnification of the display image of the annotation input window 20 by one of the aforementioned approaches. The text and annotation image display section 109 superposes the annotation display image with the changed magnification on the annotation display image 107 on the text display screen 111. This enables the display image (the annotation display image with the changed magnification) 7 indicating the contents of the annotation to be displayed on the text display screen 111 as shown in FIG. 7, when the annotation has been entered as shown on the annotation input window 20, for example. FIG. 7 shows an example of the screen on which the annotation entered on the annotation input window 20 of FIG. 2 has been displayed with a magnification of 0.5 (R=0.5).

[0070] FIG. 8 is a block diagram of a schematic configuration of a data processing apparatus with an annotation display function according to a second embodiment of the present invention. In FIG. 8, the same parts as those in FIG. 1 are indicated by the same reference symbols.

[0071] The data processing apparatus of FIG. 8 differs from that of FIG. 1 in that an annotation data display overlap removing section 120 that senses an area where representations of two or more annotation data items overlap with each other is added and that an annotation image magnification change section 125 and a text and annotation image display section 129 that differ partially in processing function from the annotation image magnification change section 105 and text and annotation image display section 109 are provided instead of the latter.

[0072] Hereinafter, the operation of the configuration of FIG. 8 will de described by reference to flowcharts in FIGS. 9 and 10.

[0073] The annotation input section 102 reads the annotation set so as to correspond to a text stored in the text file 100, creates information on the positions and size of the individual input annotation data items constituting the annotation, and creates display annotation data made up of the information and the original annotation data.

[0074] When the annotation input section 102 has created an M number of display annotation data items, the annotation data display overlap removing section 120 rearranges the annotation data items in ascending order of priority given to the type of data beforehand (e.g., when text overlaps with a line drawing, the text has priority over the drawing) (step S220). Note data items having the same priority level may be arranged in the order in which they have been entered (earliest or latest) or any other order. The rearrangement may be made in the order in which, for example, the annotation data items have been entered (earliest or latest).

[0075] The annotation data display overlap removing section 120 sets the i-th annotation data item, starting with the beginning annotation data item (steps 221, 222, 229) and senses an area (a rectangular area) Aji=(xij, yij, wij, hij) where the i-th (i is in the range of 1 to M−1) annotation data item overlaps with the j-th (j is in the range of i+1 to M) annotation data item after the (i+1)-th one, on the basis of the positions and size of both of the annotation data items (steps 225, 226).

[0076] The overlap ping of the annotation data items can be judged through the following procedure. It is assumed that annotation data item A and annotation data item B have a bounding box of (x1, y1, w1, h1) and that of (x2, y2, w2, h2), respectively.

[0077] In this case, if none of the following four expressions hold, it is found that annotation data item A overlaps with annotation data item B in display area:

[0078] x1≧x2+w2

[0079] x1+w1≦x2

[0080] y1≧y2+h2

[0081] y1+h1≦y2

[0082] At this time, it is found that the overlapped area Aij is a rectangular area of (X1, Y1, X2-X1, Y2-Y1),

[0083] where X1=max (x1, x2)

[0084] X2=min (x1+w1, x2+w2)

[0085] Y1=max (y1, y2)

[0086] Y2=min (y1+h1, y2+h2)

[0087] min (a, b) is a function that returns the smaller of a and b, and

[0088] max (a, b) is a function that returns the larger of a and b.

[0089] When sensing area Aij where the j-th annotation data item overlaps with the i-th annotation data item, the annotation data display overlap removing section 120 changes the display image of annotations so that only (the display image of the relevant portion of) the j-th annotation data item (having a higher level of priority) may be selected from the overlapped annotation data items and be displayed (step 227). This process is realized by registering (information on) area Aij in an undisplay area list Li prepared for the i-th annotation data item. The information on area Aij in the list Li indicates that the image portion of area Aij is made undisplayed in displaying the display image of the i-th annotation data item.

[0090] The annotation data display overlap removing section 120 repeats the processes at steps 225 to 227, while changing the annotation data items to be checked for an overlap of display with the i-th annotation data item, starting with j=i+1 up to M (steps 224, 228). Specifically, the annotation data display overlap removing section 120 not only senses the display overlapping of the (i+1)-th or later annotation data item with the i-th annotation data item, but also changes the display image of annotations when finding the overlapping so that only the relevant (i+1)-th or later annotation data item may be displayed.

[0091] The annotation data display overlap removing section 120 repeats the series of processes, while changing the i-th annotation data item one by one, starting with i=1 up to M−1 (steps 222, 229).

[0092] In this way, the annotation data display overlap removing section 120 acquires information on area Aij where annotation data items overlap with each other and the annotation data item to be displayed in the area. In the present embodiment, instead of creating a combination of “overlapping area Aij=(xij, yij, wij, hij), annotation data item to be displayed,” the data “undisplay area list Li” indicating area Aij not to be displayed in the annotation data item is created. In this case, because all of the M-th annotation data item is displayed, area Aij is not registered in the undisplay area list Li (=LM) for the M-th annotation data item. Therefore, the undisplay area list Li (=LM) may not be prepared.

[0093] When the number of annotations is plural, the annotation data display overlap removing section 120 creates an undisplay area list Li of each annotation data item for all of the annotations.

[0094] When the annotation data display overlap removing section 120 has created an undisplay area list Li of each annotation data item for one annotation, the annotation image magnification change section 125 creates an annotation display image by reducing or enlarging the display image on the basis of the size of the annotation input window at the time when the annotation was entered, the display annotation data item created by the annotation input section 102, the magnification R indicated by the magnification storage section 104, and the undisplay area list Li of each annotation data item created by the annotation data display overlap removing section 120 (i.e., information on the area where annotation data items overlap with each other and an annotation data item to be displayed in the area).

[0095] Then, the text and annotation image display section 129 creates a display image where the annotation display image of each annotation data item with the changed magnification is superposed in the user-specified annotation position on the text display image 107, and displays it on the text output screen 111.

[0096] The operation of processing one annotation with the annotation image magnification change section 125 and the text and annotation image display section 129 will be explained by reference to a flowchart in FIG. 10.

[0097] The annotation image magnification change section 125 determines the size of the annotation display image with the changed magnification of the annotation input window displayed on the text display screen 111 to be (Wk*R, Hk*R) from the size (Wk, Hk) of the annotation input window 20 and, magnification R (steps 231, 232).

[0098] Next, the annotation image magnification change section 125 converts the coordinates (xi, yi) of the i-th (i is in the range of 1 to M) annotation data item on the annotation input window 20 before the change of magnification into the coordinates (xi*R, yi*R) on the annotation input window after the change of magnification (at steps 231, 232) on the basis of the size (wi, hi) of the display image of the annotation data item. The annotation image magnification change section also changes the size of the display image of the annotation data item to size (wi*R, hi*R) (steps 236, 237).

[0099] At this time, the annotation image magnification change section 125 checks to see if (information on) area Aij has been registered in the undisplay area list Li for the i-th annotation data item created by the annotation data display overlap removing section 120. If it has been registered, the annotation image magnification change section also changes the magnification of all of the registered areas Aij (with a width of wij and a height of hij) using a magnification of R. For the sake of convenience, area Aij with the changed magnification is called area Aij*R. Then, the annotation image magnification change section 125 creates an annotation display image by removing the portion of area Aij*R from the annotation display image with the size (wi*R, hi*R).

[0100] Then, the text and annotation image display section 129 displays the annotation display image whose magnification has been changed to the size (wi*R, hi*R) of the i-th annotation data item in such a manner that the portion of area Aij*R has been removed from the annotation display image, using the coordinates (xi*R, yi*R) on the annotation input window 20 whose magnification has been changed by the annotation image magnification change section 125 as a reference (step 238).

[0101] The above processes are repeated M times for each annotation data item, starting with i=1 up to M (steps 233, 234, 239).

[0102] Even when the annotations entered by the user have a portion where two or more data items overlap with each other, the repetition of the aforesaid processes can remove the overlap in providing a reduced display, for example. This enables the user to imagine the contents of the annotation more accurately from the display image of the annotation.

[0103] A concrete example of removing such an overlap will be described by reference to a case where a line drawing (a handwritten wavy line) is written over text (the text character string “NOVELTY PRESENT”) as shown in FIG. 11.

[0104] First, simply reducing the annotation inputted onto the annotation input window 20 of FIG. 11 gives an annotation display image 12 as shown in FIG. 12. Because the display character size of the test character string of the annotation on the screen of FIG. 11 is large, the contents of the annotation can be understood sufficiently, even if a line drawing is written over the annotation. When the annotation is reduced as shown in FIG. 12, however, the display character size of the text character string gets smaller to the extent that the line drawing overlaps with the text character string, making the characters illegible.

[0105] When an annotation data item having the positional relationship as shown in FIG. 11 has been given, the annotation data display overlap removing section 120 senses an area where the line drawing overlaps with text in display and makes a change so that only one annotation data item may be displayed in the overlapping area. Suppose that text is allowed to appear in the area where the text and line drawing overlap with each other in FIG. 11. When the magnification is not changed, an annotation display image will be displayed as shown in FIG. 13A.

[0106] When the magnification R is set at 0.5, the annotation image magnification change section 125 creates an annotation display image by reducing the annotation display image of FIG. 13A with a magnification of 0.5 on the basis of the display image of the annotation input window 20 of FIG. 11, information on the change of the display state of the overlapping area created by the annotation data display overlap removing section 120 (information indicating that the annotation data item forming the line drawing in the overlapping area is to be removed and made undisplayed), and a magnification of 0.5. Then, the annotation image magnification change section displays the resulting annotation display image in the user-specified annotation position as a reduced annotation display image 13 as shown in FIG. 13B.

[0107] FIG. 14 is a block diagram of a schematic configuration of a data processing apparatus with an annotation display function according to a third embodiment of the present invention. In FIG. 14, the same parts as those in FIG. 1 are indicated by the same reference symbols.

[0108] The data processing apparatus of FIG. 14 differs from that of FIG. 1 in that an annotation blank area separating section 130 is added which senses a bounding box for annotation data items on the annotation input window and separates a blank in which annotation data items are not included from the other area (the area in which one or more annotation data items are included) and in that an annotation image magnification change section 135 and a text and annotation image display section 139 that differ partially in processing function from the annotation image magnification change section 105 and text and annotation image display section 109 are provided instead of the latter.

[0109] In the configuration of FIG. 14, the annotation input section 102 reads the annotation set so as to correspond to a text stored in the text file 100, creates information on the positions and size of the individual input annotation data items constituting the annotation on the annotation input window 20, and creates display annotation data made up of the information and the original annotation data.

[0110] The annotation blank area separating section 130 senses a bounding box of annotation data items on the annotation input window on the basis of the positions and size of the display annotation data items created by the annotation input section 102 and creates information (annotation blank separation information) used to separate a blank indicating an area including no annotation data item from the other area (the area including one or more annotation data items). The annotation blank area separating section 130 performs this process on each of the annotation data items constituting one annotation.

[0111] The annotation image magnification change section 135 creates an annotation display image by reducing or enlarging the display image on the annotation input window on the basis of the display annotation data created by the annotation input section 102, the magnification R indicated by the magnification storage section 104, the size of the annotation input window at the time when the relevant annotation has been entered, and each piece of the annotation blank separation information for each annotation data item created by the annotation blank area separating section 130.

[0112] Then, the text and annotation image display section 139 creates a display image where the annotation display image with the changed magnification is superposed in the user-specified annotation position on the text display image 107, and displays it on the text output screen 111.

[0113] As described above, when an annotation display image whose magnification has been changed, for example, a reduced annotation display image, is created on the text display screen 111, reducing only the portion of the area where the annotation data actually exists enables the annotation data in the reduced annotation display image to be made larger, even when the size of the reduced annotation display image is the same as in the first embodiment. This enables the user to imagine the contents of the annotation more accurately from the display image of the annotation.

[0114] The most distinctive operation of the annotation blank area separating section 130 in the third embodiment will be described by reference to a flowchart in FIG. 15.

[0115] It is assumed that when an annotation data item has been written only in an area (about one-fourth of the annotation input window) at the top left end of the annotation input window (20), the display image of the annotation input window is reduced and displayed in the area where the size of the annotation input window is reduced with a magnification of 0.5.

[0116] In this case, when the display image of the annotation input window of FIG. 16A is simply reduced and displayed, a reduced annotation display image 16 as shown in FIG. 16B is shown. In the display of the annotation display image 16, the line-drawing character string (the handwritten character string “Search Required”) is crushed, making it difficult to imagine the contents of the annotation from the reduced image.

[0117] When receiving the annotation data (annotation data composed of the handwritten character string “INVESTIGATION NEEDED” and annotation data composed of the text character string “DEADLINE: 97/1”) indicating the annotation as shown in FIG. 16A from the annotation input section 102, the annotation blank area separating section 130 extracts the area (the smallest rectangular area) in which all of the annotation data items have been inputted on the annotation input window on the basis of the positions (xi, yi) of each data item on the annotation input window and the size (wi, hi) of each data item (step 240 to 248).

[0118] The area in which all of the annotation data items (two annotation data items in the example of FIG. 16A) can be calculated is described through the following procedure.

[0119] Suppose the coordinates of data items in the annotation of FIG. 16A are given as shown in FIG. 4. In the figure, two annotation data items D1 and D2 have been inputted. If annotation data items D1 and D2 have the bounding box (x1, y1, w1, h1) and the bounding box (x2, y2, w2, h2), the smallest rectangular area in which annotation data items D1, D2 have been inputted will be given as a rectangular box having the bounding box (X1, Y1, X2-X1, Y2-Y1) (step 247),

[0120] where X1=min (x1, x2)

[0121] X2=max (x1+w1, x2+w2)

[0122] Y1=min (y1, y2)

[0123] Y2=max (y1+h1, y2+h2)

[0124] min (a, b) is a function that returns the smaller of a and b, and

[0125] max (a, b) is a function that returns the larger of a and b.

[0126] Although there are only two annotation data items in FIG. 4 (FIG. 16A), when there are three or more annotation data items, the smallest rectangular area in which all of the annotation data items have been entered can be calculated by repeating the same process (step 247) between the input area (X1, Y1, X2-X1, Y2-Y1) obtained through the preceding procedure and the third annotation data item's bounding box (steps 245 to 248).

[0127] The annotation blank area separating section 130 extracts the smallest rectangular area in which all of the annotation data items have been inputted on the annotation input window (i.e., including all of the annotation data input areas). This triggers the annotation image magnification change section 135.

[0128] The annotation image magnification change section 135 finds the size of the annotation display image at the time when the annotation input window has been reduced with a magnification of R and reduces the display image of the area extracted by the annotation blank area separating section 130 with a magnification of R′ that allows the display image to fit into the size (steps 249 to 252).

[0129] If the size of the annotation input window is (Wk, Hk), the magnification indicated by the magnification storage section 104 is R, the area extracted by the annotation blank area separating section 130 is (Xb, Yb, Wb, Hb) (steps 249, 250), the magnification R′ can be calculated as “THE SMALLER OF (Wk*R)/Wb, (Hk*R)/Hb” (step 251).

[0130] On the basis of the size (wi, hi) of the display image of the annotation data item, the magnification R′, and the area (Xb, Yb, Wb, Hb) extracted by the annotation blank area separating section 130, the annotation image magnification change section 135 converts the coordinates (xi, yi) on the window before the change of magnification into the coordinates ((xi−Xb)*R′, (yi−Yb)*R′). The annotation image magnification change section changes the size of the display image of the annotation data to (wi*R′, hi*R′) (step 252).

[0131] Then, the text and annotation image display section 139 displays the display image of the relevant annotation data item with the size (wi*R′, hi*R′) in the position having the coordinates ((xi−Xb)*R′, (yi−Yb)*R1) whose magnification has been changed by the annotation image magnification change section 135 (step 253).

[0132] The processes at steps 252, 253 (like steps 237, 238 in FIG. 10) are repeated for all of the annotation data items inputted on the annotation input window. In the flowchart of FIG. 15, the steps indicating the repetition (corresponding to steps 235, 239 in FIG. 10) are omitted for the sake of drawing.

[0133] FIGS. 17A to 17D show concrete examples of the above operation. It is assumed that when the annotation data composed of the handwritten character string “Search Required” and the annotation data composed of the text character string “DEADLINE: 97/1” have been written in the area at the top left end of the annotation input window, the display image of the annotation input window is reduced and displayed in the area where the size of the annotation input window has been reduced with a magnification of 0.5.

[0134] In this case, the annotation blank area separating section 130 extracts from the annotation input window the smallest rectangular area in which the two annotation data items have been entered, that is, the rectangular area from which a blank area has been removed, as shown in FIG. 17B.

[0135] Then, such a magnification of R′ is calculated as enables the display image of the two annotation data items in the area to fit into the size (Wk*R, Hi*R) of the annotation display image on the annotation input window reduced with the magnification R (=0.5). Then, the display image of the annotation data item is reduced and displayed with the magnification R′.

[0136] The display image of the annotation data item reduced with the magnification R′ in the area of the size of the annotation input window reduced with the magnification R (=0.5) is as shown in FIG. 17D. In contrast, the display image of the annotation data item simply reduced with the original magnification R (=0.5) is as shown in FIG. 17C.

[0137] As described above, with the third embodiment, even when the size of the area in which the display image of the annotation data item reduced with the magnification R′ of FIG. 17D is displayed is the same as the size of the display image of the annotation data item reduced with the magnification R of FIG. 17C, the display image of FIG. 17D is easier to see than the display image of FIG. 17C, because the magnification R′ for the area extracted by the annotation blank area separating section 130 is larger than the original magnification R.

[0138] Adjusting the magnification as described above causes the annotation of FIG. 16A to be reduced to the annotation display image 18 as shown in FIG. 18. The reduced annotation display image 18 is shown in the user-specified annotation position on the text display screen 111.

[0139] With the third embodiment, by limiting the area of annotation data items to be displayed, the contents of the annotation can be displayed so that they may be larger and easier to see, even when the size of the reduced annotation display image itself is the same as in the first embodiment.

[0140] FIG. 19 is a block diagram of a schematic configuration of a data processing apparatus with an annotation display function according to a fourth embodiment of the present invention. In FIG. 19, the same parts as those in FIG. 1 are indicated by the same reference symbols.

[0141] The data processing apparatus of FIG. 19 differs from that of FIG. 1 in that there is additionally provided an annotation data magnification adjusting section 140 that, when the display magnification of an annotation data item given from the annotation input section 102 has been changed to a magnification of R, adjusts the magnification to a magnification of Ri that enables the annotation data item to be enlarged to a size not smaller than a display limit value, if the size of the display image after the change of magnification is smaller than the display limit value and a display limit value storage section 141 for storing the display limit value for each type of annotation data item, and in that an annotation image magnification change section 145 and a text and annotation image display section 149 that differ partially in processing function from the annotation image magnification change section 105 and text and annotation image display section 109 are provided instead of the latter.

[0142] The display limit value stored in the display limit value storage section 141 is determined as follows. For example, it is assumed that when Chinese character text is displayed on a display unit 110 displaying the text display screen 111, a Chinese character made up of horizontal and vertical lines is crushed if the character is displayed in eight dots or less. In this case, when an annotation data item includes Chinese character text, the display limit value is eight dots.

[0143] Similarly, in the case of line-drawing data, when the portion where two lines are drawn separately before reduction is reduced to the extent that the two lines are in contact with each other, it is difficult to distinguish one line from the other on the reduced screen. The size that more than a certain amount of the area in the bounding box of line drawing is painted over with drawing lines can be determined to the display limit value.

[0144] In the configuration of FIG. 19, the annotation input section 102 reads the annotation set so as to correspond to a text stored in the text file 100, creates information on the positions and size of the individual input annotation data items constituting the annotation on the annotation input window, and creates display annotation data made up of the information and the original annotation data.

[0145] The annotation data magnification adjusting section 140, when the display image of the annotation data item has been reduced or enlarged with a magnification of R (in this case, R<1 for reduction), checks to see if the size of the display image after the change of magnification is smaller than the display limit value, on the basis of the positions and size of the display annotation data items created by the annotation input section 102, the magnification R indicated by the magnification storage section 104, and the display limit value in the display limit value storage section 141 corresponding to the annotation data item.

[0146] If the size of the display image after the change of magnification is smaller than the display limit value, the annotation data magnification adjusting section 140 will adjust the magnification of the corresponding annotation data item to the smallest magnification Ri that enables the annotation data item to be enlarged to a size not smaller than the display limit value.

[0147] If the size of the display image after the change of magnification is larger than the display limit value, the annotation data magnification adjusting section 140 will determine the magnification R indicated by the magnification storage section 104 to be the magnification Ri for the corresponding annotation data item.

[0148] From this time on, the operation of the annotation image magnification change section 145 and that of the text and annotation image display section 149 are the same as the operation of the annotation image magnification change section 105 and that of the text and annotation image display section 109 in FIG. 1 except that the magnification is Ri set for each annotation data item, not R common to each annotation data item. Specifically, the annotation image magnification change section 145 converts the coordinates (xi, yi) of each annotation data item on the annotation input window before the change of magnification into the coordinates (xi*Ri, yi*Ri) on the annotation input window with the changed magnification R on the basis of the size (wi, hi) of the display image of the annotation data item and the magnification Ri. The annotation image magnification change section also changes the size of the display image of the annotation data item with a magnification of (wi*Ri, hi*Ri).

[0149] Receiving this, the text and annotation image display section 149 displays the display image of the corresponding annotation data item with a size of (wi*Ri, hi*Ri) at the coordinates (xi*Ri, yi*Ri).

[0150] As described above, when the reduced image of an annotation is displayed on text, an annotation data item difficult is to see if it is reduced to a size smaller than the display limit value is reduced and displayed with the smallest magnification Ri (Ri>R) that enables the annotation data item to be enlarged to a size not smaller than the display limit value, instead of reducing all of the annotation data items equally with a specified magnification of R. This prevents a reduced image whose size is smaller than the display limit value from being displayed and makes the characters and drawings legible even on a reduced screen, as compared with the case where all of the annotation data items are reduced equally.

[0151] The operation of the annotation data magnification adjusting section 140 will be described in detail by reference to a flowchart in FIG. 20.

[0152] Suppose the text character string “NOVELTY IS PRESENT, INVESTIGATION OF RELATED PATENTS IS NEEDED” is written in a small size at the top of the annotation input window and the handwritten character string “RECONSIDERATION” is written below the former string.

[0153] In this case, when the display image of the annotation input window of FIG. 21A is simply reduced and displayed, a reduced annotation display image 21bas shown in FIG. 21B is shown on the text display screen 111. On the reduced annotation display image 21bof FIG. 21B, the text is displayed in very small characters and the character display area is crushed, making it difficult to imagine what is written.

[0154] Then, on the basis of the display limit value T and magnification R corresponding to the type of the annotation data item, the position (xi, yi) of the annotation data item obtained from the annotation input section 102 and the size (wi, hi), the annotation data magnification adjusting section 140 checks to see if the size of the annotation data item on the annotation input window is smaller than the display limit value T when the annotation data item has been reduced with the magnification R (steps 264, 265).

[0155] When the size of the annotation data item is smaller than the display limit value T, the smallest magnification that enables the annotation data item to be not smaller than the display limit value T is found. The smallest magnification is determined to be the magnification Ri applied to the relevant annotation data item (step 266). When the size of the annotation data item is not smaller than the display limit value T, the magnification R (the magnification indicated by the magnification storage section 104) is determined to be the magnification Ri applied to the relevant annotation data item (step 267).

[0156] Whether or not the reduced image of an annotation data item is smaller than the display limit value T can be judged as follows. If the bounding box of an annotation data item is (xi, yi, wi, hi) and at least one of the following equations holds:

[0157] wi*R<T

[0158] hi*R<T

[0159] it can be judged that the reduced image of the annotation data item is smaller than the display limit value T.

[0160] The smallest magnification Ri that prevents the size of the annotation data item from being smaller than the display limit value T is calculated as the smaller of T/wi and T/hi.

[0161] The annotation data magnification adjusting section 140 repeats the above processes for all of the annotation data items on the input window (steps 261 to 263 and step 268).

[0162] In this way, the annotation data magnification adjusting section 140 changes the display form of the annotation data item and finally a reduced annotation display image 21cas shown in FIG. 21C is displayed. On the reduced annotation display image 21C of FIG. 21C, because the text is displayed in larger characters than those on the reduced annotation display image 21bof FIG. 21B with the specified magnification R, even if the annotation data item is reduced, what has been written is legible. Since the magnification Ri is set for each annotation data item, it is impossible to accurately display the positional relationship and size of annotation data items in the reduced annotation display image. For example, in FIG. 21C, only the partial character string “NOVELTY IS PRESENT, INVESTIGATION OF” (of the full string “NOVELTY IS PRESENT, INVESTIGATION OF RELATED PATENTS IS NEEDED”) is displayed on the reduced annotation display image 21c.

[0163] In the configuration of FIG. 1, the processing functions of the annotation input section 102, the annotation image magnification change section 105, and the text and annotation image display section 109 are realized by using a recording medium (e.g., a memory card, a floppy disk, or a CD-ROM) on which a program that causes a data processing apparatus, such as a computer, having a CPU to realize the processing functions, loading the recording medium into the data processing apparatus, and forcing the data processing apparatus to read and execute the program recorded on the recording medium. The program may be down-loaded from a network through a communication channel. The same is true for the processing functions of the annotation input section 102, the annotation data display overlap removing section 120, the annotation image magnification change section 125, and the text and annotation image display section 129 in FIG. 8, the processing functions of the annotation input section 102, the annotation blank area separating section 130, the annotation image magnification change section 135, and the text and annotation image display section 139 in FIG. 14, and the processing functions of the annotation input section 102, the annotation data magnification adjusting section 140, the annotation image magnification change section 145, and the text and annotation image display section 149 in FIG. 19.

[0164] As described in detail so far, with the present invention, because the display image of the annotation input window at the time of annotation input is reduced or enlarged with a specified magnification and the display image is superposed in a specified position on the text display image on the text display screen, the user can check the contents of the annotation, while reading the text.

[0165] Furthermore, with the present invention, because a reduced display image of the annotation from which the overlapping of annotation data items has been removed is displayed, the contents of the annotation can be checked easily, even if they are difficult to check as a result of simply reducing the annotation.

[0166] Moreover, with the present invention, because only the area from which blanks have been removed is to be reduced and displayed, the contents of the annotation can be displayed larger with the entire display size remaining unchanged, even when simply reducing the annotation would make the contents difficult to check. This makes it possible to check the contents of the annotation easily.

[0167] Still furthermore, with the present invention, since an annotation data item is sensed whose contents would be difficult to check if the annotation data item were simply reduced and displayed with a specified magnification because the display size of the annotation data is small, and the smallest magnification that prevents the display size of the annotation data item after the change of magnification from being smaller than the display limit value is applied, the contents of the annotation can be displayed larger with the entire display size remaining unchanged, making it possible to check the contents of the annotation easily.

[0168] Additional advantages and modifications will readily occur to those skilled in the art. Therefore, the present invention in its broader aspects is not limited to the specific details, representative devices, and illustrated examples shown and described herein. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.