Image processing apparatus and method and storage medium

[0002] 1. Field of the Invention

[0003] The invention relates to image processing apparatus and method which can search accumulated images.

[0004] 2. Related Background Art

[0005] FIG. 21 shows an example of an image format which has conventionally been used.

[0006] As shown in FIG. 21, an image file is divided into an image header section 211 and an image data section 212. Generally, information which is necessary when data is read out from the image file and additional information to explain the contents of an image are stored in the image header section 211. In the example of FIG. 21, information such as image format identifier showing an image format name, file size, width, height, and depth of the image, presence/absence of compression, information of a color palette, resolution, offset of the image data to a storing position, and the like is stored.

[0007] The image data section 212 is a portion where image data has sequentially been stored. As a typical example of such an image format, a BMP format of Microsoft Co., Ltd., a GIF format of Compuserve Co., Ltd., or the like is widely spread.

[0008] In case of preserving the image files into a database and searching and using it, hitherto, a key word and an explanation sentence of a language base have been made correspond to one image. That is, in an example of FIG. 5, key words of “Sky, Sea, Sun, Yacht, Male, Female” are allocated to one image. A data table showing a correspondence between the image and the key words are shown in FIG. 14. For example, when the user inputs the word “yacht” as a search word, this image is obtained as a result in which the key word coincides.

[0009] In the conventional example, even if a plurality of persons, objects, and regions of different characteristics exist in an image, only a process such that all of the allocated key words are handled as attribute information for one image can be performed. Therefore, it is used as a search result irrespective of the position in the image where the contents corresponding to the key word exist.

[0010] Consequently, for example, when “apple existing at the center” is given as a search word, among the key words corresponding to the image, all of the key words including “apple” are derived as a search result and an image in which “apple” is shown at the corner of the screen is also searched. Thus, it is impossible to perform the search according to the contents of the given search word.

[0011] When the image extracted as a result of the key word search is displayed, it is necessary to display the whole image irrespective of the position in the image where the contents corresponding to the key word exist. It takes very long time to display the image in dependence on an image size.

[0012] For one kind of image, attribute information can be handled only for one image. Even when displaying an image of low resolution, the result of the search on the basis of the attribute information of the original image is merely displayed at low resolution. There is a problem such that it takes time until a display from the search, a desired image is dropped out from the search, or the like.

[0013] In the conventional example, even in the case where one image has images of a plurality of resolutions, attribute information such as key word, image characteristic amount, or the like can be handled only as attribute information for one image.

[0014] According to an embodiment of the invention, there is provided an image processing apparatus for dividing image data into a plurality of areas and allocating attribute information to each of the divided areas.

[0015] FIG. 1 is a block diagram showing a whole construction of the invention;

[0016] FIG. 2 is a diagram showing an example of an image divided like a tile which is used in the invention;

[0017] FIG. 3 is a diagram showing a correspondence table between tile numbers and key words included in the tiles;

[0018] FIG. 4 is a diagram showing a part of the image divided into the tiles;

[0019] FIG. 5 is a diagram showing an image which is not divided into tiles and a key word;

[0020] FIG. 6 is a diagram showing an example of an image which is divided into tiles and in which objects exist at the center and the lower right position of a screen;

[0021] FIG. 7 is a diagram showing an example of an image which is divided into tiles and in which an object exists at the lower right position of the screen;

[0022] FIG. 8 is a diagram showing an example of an image which is divided into tiles and in which an object exists at the center of the screen;

[0023] FIG. 9 is a diagram showing a structure of a table in which image IDs and image file storing paths are made correspond to each other;

[0024] FIG. 10 is a diagram showing a structure of a table in which image IDs, key words of tiles, and image characteristic amounts are made correspond to each other;

[0025] FIG. 11 is a diagram showing a structure of a table in which key words, image IDs, and tile numbers are made correspond to each other;

[0026] FIG. 12 is a diagram showing a structure of a table in which position designate words in an image and tile numbers are made correspond to each other;

[0027] FIG. 13 is a diagram showing a structure of a table in which image characteristic words and image characteristics are made correspond to each other;

[0028] FIG. 14 is a diagram showing a structure of a correspondence table of image IDs and key words in a conventional example;

[0029] FIG. 15 is a diagram showing a structure of a FlashPix image object;

[0030] FIG. 16 is a diagram showing a structure of a FlashPix image view object;

[0031] FIG. 17 is a diagram showing layer images of different resolutions;

[0032] FIG. 18 is a diagram showing an image divided into tiles;

[0033] FIG. 19 is a diagram showing attribute information which is used in the FlashPix;

[0034] FIG. 20 is a diagram showing a structure of each layer data of the FlashPix;

[0035] FIG. 21 shows an example of a conventional image format;

[0036] FIG. 22 is a flowchart for explaining the operation of an embodiment;

[0037] FIG. 23 is a flowchart for explaining the operation of another embodiment;

[0038] FIG. 24 is a diagram showing an example of layer images which are divided into tiles and are used in the invention;

[0039] FIG. 25 is a flowchart for explaining the operation of the embodiment;

[0040] FIG. 26 is a flowchart for explaining the operation of the embodiment;

[0041] FIG. 27 is a diagram showing a correspondence table of layer numbers, tile numbers, key words included in the tiles, and image characteristics;

[0042] FIG. 28 is a diagram showing a structure of a table in which key words, image IDs, and tile numbers are made correspond to each other; and

[0043] FIG. 29 is a diagram showing a structure of a table in which image IDs, key words of tiles, image characteristic amounts and layer numbers are made correspond to each other.

[0044] The invention will now be described in detail hereinbelow with reference to the drawings.

[0045] FIG. 1 shows an example of a whole construction of the invention.

[0046] In the construction shown in the diagram, reference numeral 11 denotes a microprocessor (CPU) for executing an arithmetic operation to search image information, a logical discrimination, and the like and controlling each of component elements connected to an address bus AB, a control bus CB, and a data bus DB through these buses.

[0047] The address bus AB transfers an address signal to instruct a component element as a target of control of the CPU 11. The control bus CB transfers a control signal of each component element as a control target of the CPU 11 and supplies. The data bus DB performs a data transfer among component equipment.

[0048] Reference numeral 12 denotes a fixed read only memory (ROM). An image searching program to realize the image processing apparatus of the invention has been stored in the ROM 12.

[0049] Reference numeral 13 denotes a writable random access memory (RAM) in which one word consists of 16 bits. The RAM 13 is used to temporarily store various data from the component elements. Although not shown, an input search word storing buffer WBUF, a search target tile number storing buffer TBUF, a search target image characteristic storing buffer FBUF, and a search result output buffer OBUF are temporarily stored in the RAM 13, respectively.

[0050] Reference numeral 14 denotes an external memory (DISK) such as a hard disk or the like. In the memory 14, there are stored: a reference table in which image IDs and image file storing paths, which will be explained hereinlater in FIG. 9, are made correspond to each other; a reference table in which image IDs, key words of tiles, and image characteristic amounts, which will be explained hereinlater in FIG. 10, are made correspond to each other; a reference table in which key words, image IDs, and tile numbers, which will be explained hereinlater in FIG. 11, are made correspond to each other; a reference table in which position designate words and tile numbers, which will be explained hereinlater in FIG. 12, are made correspond to each other; a reference table in which image characteristic words and image characteristics, which will be explained hereinlater in FIG. 13, are made correspond to each other; an image data file stored in a structure which will be explained hereinlater in FIG. 15; a program which is executed in accordance with a flowchart, which will be explained hereinlater in FIG. 22; and a word dictionary DIC (although not shown) in which information for a word division and a discrimination of a word classification which is referred in step S2202 in FIG. 22 has been stored. The image data file or the like is preserved as necessary. The preserved data is called as necessary by an instruction by a keyboard.

[0051] Reference numeral 15 denotes a keyboard (KB) having alphabet keys, Hiragana keys, Katakana keys, character symbol input keys such as a period and the like, and various function keys such as search key to instruct a search, cursor moving key to instruct the movement of a cursor, and the like.

[0052] Reference numeral 16 denotes a video memory for display (VRAM) in which patterns of data to be displayed have been stored.

[0053] Reference numeral 17 denotes a CRT controller (CRTC) for displaying the contents stored in the displaying video memory VRAM 16 onto a display device (CRT) 18.

[0054] Reference numeral 18 denotes the display device (CRT) using a cathode ray tube or the like. A display pattern constructed by dots and a display of a cursor on the display device CRT 18 are controlled by the CRT controller 17.

[0055] Reference numeral 19 denotes a network controller (NIC) for connecting to a network such as an Ethernet or the like.

[0056] The image processing apparatus of the invention comprising the above component elements operates in accordance with various inputs from the keyboard KB 15 and various inputs which are supplied from the network controller 19 via the network. When the input from the keyboard KB 15 and the input from the network controller 19 are supplied to the image processing apparatus, an interruption signal is first sent to the microprocessor CPU 11. The microprocessor CPU 11 reads out various control signals stored in the external memory DISK 14 and executes various controls in accordance with the control signals.

[0057] FIG. 2 shows an example in which an image shown in FIG. 5 is divided into a plurality of tiles. In this example, the target image is divided into total 20 tiles comprising four tiles in the vertical direction and five tiles in the lateral direction. Numerals in the diagram show tile numbers which are sequentially numbered like 0, 1, 2, . . . from the upper left position of the image.

[0058] An example of an image format such that the image is divided into tiles as mentioned above is shown and will now be explained.

[0059] According to the FlashPix™ (FlashPix is a registered trademark of Eastman Kodak Co., Ltd. in the U.S.A.) file format, which will be explained hereinlater, image attribute information and image data stored in the image header section are further structured and stored into files. The structured image files are shown in FIGS. 15 and 16.

[0060] Each property and data in the file are accessed by “storage” and “stream” corresponding to “directory” and “file” of MS-DOS. In FIGS. 15 and 16, portions with shadows (1501 to 1505, 1601 to 1603) correspond to storages and portions without a shadow (1506 to 1513, 1604 to 1611) correspond to streams. The image data and image attribute information are stored in the stream portion. The image data is layered by different resolutions and an image at each resolution is called a “Subimage” and those Subimages are shown by Resolutions 0, 1, . . . , and n, respectively. For each resolution image, information necessary to read out the image is stored into a Subimage Header 1513 and the image data is stored into a Subimage data 1512.

[0061] The Property set denotes that the attribute information is classified and defined in accordance with the use object and the contents. There are a “Summary info. Property set” 1506, an “Image info. Property set” 1509, an “Image contents Property set” 1508, and an “Extension list Property set” 1510.

[0062] The “Summary info. Property set” 1506 is not peculiar to the FlashPix but is a Property set which is indispensable for the structured storage of Microsoft Co., Ltd. and stores the title of the file, title, author, thumbnail image, and the like.

[0063] The “Image contents Property set” 1508 indicates attributes describing a storing method of the image data (FIG. 19). In the attributes, the number of layers of the image data, a width and a height of an image of the maximum resolution, a width and a height of an image of each resolution, a construction of the color, or a definition of a quantization table * a Huffman table when the JPEG compression is used is described.

[0064] The “Image info. Property set” 1509 stores various information which can be used when an image is used, for example, information showing how to fetch the image or how the image can be used.

[0065] information (File Source) regarding a fetching method or forming method of digital data

[0066] information (Intellectual property) regarding a copyright

[0067] information (Content description) regarding the contents (person in the image, location, and the like) of the image

[0068] information (Camera information) regarding a camera used for photographing

[0069] information (Per Picture camera settings) of the setting (exposure, shutter speed, focal distance, the presence or absence of the use of a flash, etc.) of the camera at the time of photographing

[0070] information (Digital camera characterization) regarding the resolution or mosaic filter which is peculiar to a digital camera

[0071] information (Film description) such as name of a manufacturer of a film, product name, kind (negative/positive, color/black and white), etc.

[0072] information (Original document scan description) regarding a kind and a size in the case where an original is a book or a printed matter

[0073] in case of a scan image, information (Scan device) regarding software of a scanner used and a person who operated.

[0074] The “Extension list Property set” 1510 is an area which is used when adding information which is not included in the basic specification of the FlashPix.

[0075] A “FlashPix image view object” 1601 in FIG. 16 is an image file to store both of a viewing parameter and image data which are used when an image is displayed. The viewing parameter is a set of processing coefficients to store processes for the rotation, enlargement/reduction, movement, color conversion, and filtering of the image so as to be adapted when the image is displayed.

[0076] A “Source/Result FlashPix image object” (1602, 1603) is a substance of the FlashPix image data. The “Source FlashPix image object” 1602 is the inevitable image data. The “Result FlashPix image object” 1603 is the option. The “Source FlashPix image object” 1602 stores the original image data. The “Result FlashPix image object” 1603 stores the image as a result obtained by performing an image processing by using the viewing parameter.

[0077] A “Source/Result desc. Property set” (1608, 1609) is a Property set to identify the image data and stores an image ID, a Property set of the inhibition of the change, a final updating date and time, and the like.

[0078] A “Transform Property set” 1610 stores Affine transformation coefficients for rotation, enlargement/reduction, and movement, color conversion matrix, contrast adjustment value, and filtering coefficients.

[0079] A method of handling the image data will now be described.

[0080] FIG. 17 shows an example of an image file constructed by a plurality of images of different resolutions. In FIG. 17, an image of the maximum resolution is constructed by (columns×rows) of (X0×Y0). The next large image is constructed by (X0/2×Y0/2). After that, the image is sequentially reduced by ½ at a time with respect to both of the columns and the rows. The reducing process is repeated until the number of pixels in each of the column and the row is equal to or less than 64 pixels.

[0081] As a result of the execution of the layer processing as mentioned above, “the number of layers in one image file” is necessary as attribute information of the image and, for the image of each layer, the header information and the image data which were described in the item of the conventional image format are also necessary as attribute information. Information regarding the number of layers in one image file, a width and a height of the image of the maximum resolution, a width and a height of the image of each resolution, a color construction, a compressing method, or the like is described in the “Image contents Property set” in FIG. 19.

[0082] Further, the image of the layer of each resolution is divided into (64×64) tiles as shown in FIG. 18. Now, when the image is sequentially divided into (64×64) tiles from the upper left position of the image, there is a case where a blank occurs in parts of the tiles at the right edge and the lower edge in dependence on an image.

[0083] In this case, by repetitively inserting the rightmost image or the bottom image, respectively, (64×64) pixels are constructed. In the FlashPix™, the image in each tile is stored by a method of any one of the JPEG compression, single color, and non-compression. The JPEG compression is an image compressing method which was international standardized by ISO/IEC JTC1/SC29 and an explanation of this method itself is omitted here. The image data divided into the tiles as mentioned above is stored into the Subimage data stream 1512. All of the total number of tiles, a size of each tile, a start position of the data, and a compressing method have been stored in the Subimage Header (FIG. 20).

[0084] The “single color” is a method such that only in the case where one tile is constructed by the same color as a whole, the color of the tile is expressed by one (single) color without recording the values of the individual pixels. This method is particularly effective in an image formed by computer graphics.

[0085] When the image is divided into the tiles as mentioned above, the characteristic portion in the image which has hitherto been described by a key word is separated into each tile and is classified into the tiles including each key word and the tiles including no key word.

[0086] When the image of FIG. 2 is divided into tiles 0 to 19 and a key word included in each tile is described, they are as shown in FIG. 3. That is, “sky” is described in the tiles 0, 1, 2, 5, and 10. “sky” and “sun” are described in the tiles 3, 4, 8, and 9. “sky” and “male” are described in the tiles 6 and 11. “sky” and “female” are described in the tiles 7 and 12. “sky” and “yacht” are described in the tiles 13 and 14. “sea” is described in the tile 15. “sea” and “male” are described in the tile 16. “sea” and “female” are described in the tile 17. “sea” and “yacht” are described in the tiles 18 and 19.

[0087] FIG. 4 shows a partial image constructed by the tiles 13, 14, 18, and 19 of the image shown in FIG. 2.

[0088] FIGS. 6, 7, and 8 also show an example of an image which is divided into 20 tiles and stored in a manner similar to FIG. 2.

[0089] FIG. 9 shows a table in which the image IDs and image file storing paths are made correspond to each other in the invention. Reference numeral 91 denotes an image ID serving as an identification number which is individually allocated for each image file. Reference numeral 92 denotes a file path showing a storage destination in the external memory DISK 14 of the image file corresponding to the image ID. The file path corresponds to the directory and file of the MS-DOS.

[0090] FIG. 10 shows a table in which the image IDs, the key words of the tiles, and the image characteristics are made correspond to each other. Reference numeral 101 denotes an area to store the image IDs corresponding to the image IDs 91 in FIG. 9; 102 an area to store the tile number which each image ID has; and 103 an area in which the image characteristics drawn on the tile corresponding to the tile number 102 are expressed by a language and in which the key word is stored by a character code (for example, unicode). A plurality of key words can be also stored with respect to one tile number. Reference numeral 104 denotes an area to store the image characteristic of the tile corresponding to the tile number. In the invention, particularly, as for the image characteristics, color components in which an average of the pixel values of RGB constructing the tile is obtained and the number of colors is subtracted to 27 colors are used as representative colors of the tile. As a method of subtractive colors, a well-known method such as method of simply dividing each of the RGB pixel values, method using a reference table, or the like is used. In the diagram, the image whose image ID is equal to 1024 has 20 tiles, the key word of “sky” is allocated to the tile of the tile No. 0, and the color component of “11” is allocated as an image characteristic.

[0091] FIG. 11 is a table in which the key words, the image IDs, and the tile numbers are made correspond to each other. The table of FIG. 11 is a table in which the corresponding image IDs and tile numbers are collected every key word shown at 103 in FIG. 10. Reference numeral 111 denotes a key word; 112 an area to store the image ID corresponding to the key word of 111; and 113 an area to store the number of the tile having the key word 111 among the tiles corresponding to the image IDs.

[0092] FIG. 12 is a table in which the position designate words to designate the position of a search target object in an image and the tile numbers are made correspond to each other. Reference numeral 121 denotes an area to store the position designate word by a character code (for example, unicode). In the invention, as position designate words, ten words of “Top”, “Upper Left”, “Upper Right”, “Middle”, “Center”, “Left Middle”, “Right Middle”, “Bottom”, “Lower Left”, and “Lower Right” are used. Reference numeral 122 denotes an area to store the tile number shown by the position designate word according to the divided tile. For example, when “Upper Right” is designated as a position designate word, the tile numbers of 3, 4, 8, and 9 shown in FIG. 2 are specified as search target positions.

[0093] FIG. 13 is a table in which the image characteristic words and the image characteristics are made correspond to each other. In the invention, the representative color of the tile is used as an image characteristic and a language showing the color is used as an image characteristic word. A color name (coloring) is stored in an area 131 as a characteristic word by a character code (for example, unicode). The image characteristic amount shown by the coloring is stored into an area 132 by the color component. In the diagram shown, for example, the coloring having the color components 2, 3, and 4 indicates “red”.

[0094] FIG. 14 shows an example of a correspondence table of the image IDs and the key words which has conventionally been used. In order to store the image shown in FIG. 5 and the key word corresponding to the image, the image ID has been stored in an area 141 and the key word corresponding to the image ID has been stored in an area 142.

[0095] FIG. 22 is a flowchart for explaining the operation of the image searching apparatus of the invention and the operation will now be described hereinbelow with reference to FIG. 22.

[0096] In step S2201, a search word serving as a search target object is inputted. The inputting process of the search word is executed by storing a character code inputted from the keyboard KB 15 into the search word storing buffer WBUF in the RAM 13.

[0097] In step S2202, subsequently, the search word stored in the search word storing buffer WBUF is recognized as a word. The word is recognized by a well-known morpheme analyzing method by referring to the word dictionary DIC in the external memory DISK 14.

[0098] In step S2203, which one of the search target object, search designate position, and search target image characteristic the classification of word recognized in step S2202 indicates is discriminated.

[0099] The discrimination of the word classification can be performed by storing the classification of word in the word dictionary DIC of the external memory DISK 14. For example, if the search word is “red”, the word classification is the search target image characteristic. If it is “left”, the word classification is the search designate position. If the word classification is none of the search target image characteristic and the search designate position, it is the search target object. In accordance with the word classification determined in step S2203, the processing routine is branched to each process in step S2204, S2205, or S2207.

[0100] Step S2204 is a process in the case where it is determined that the search word is a search target object. The key word corresponding to the search word is searched. The search of the key word is performed with reference to the data table in FIG. 11 stored in the memory DISK 14. First, a check is made to see if the key word 111 which coincides with the search word exists. If it does not exist, the number of search tiles is regarded as 0 and step S2209 follows, thereby informing a fact that the tile including the search word does not exist.

[0101] When the key word which coincides with the search word is found, the corresponding image ID 112 and tile number 113 are extracted. When the search word coincides with a plurality of key words, the number of coincident key words, the image IDs, and tile numbers are stored into the output buffer OBUF. Step S2209 follows. In step S2209, the image data is displayed on the basis of the key word search result.

[0102] For example, when the user inputs a word of “yacht” as a search word, as a result in which the key word coincides, the tile numbers 13, 14, 18, and 19 of an image ID 1024 are obtained as a search result. Since the image has been divided and preserved, when displaying, the images of those four tiles can be first displayed. Thus, the image of FIG. 4 can be obtained.

[0103] In case of the image of FIG. 4, its area is ⅕ of that of the image of FIG. 2 and the time which is required to display is also reduced in accordance with it. When such an image exists on a network of a slow transmitting speed, the time which is required to transfer the image data can be also reduced as compared with that in case of transferring the whole image.

[0104] Step S2205 is a process in the case where it is determined that the search word includes the search designate position, and the designate position tile number is obtained. To obtain the designate position tile, the position designate word 121 which coincides with the search designate position word is checked with reference to the data table of FIG. 12 stored in the memory DISK 14. When they coincide, the corresponding tile numbers are extracted from the area 122 and stored into the search target tile number storing buffer TBUF which is temporarily stored in the RAM 13. When the coincident position designate word does not exist, an invalid value (for example, negative number) is stored into the search target tile number storing buffer TBUF.

[0105] Subsequently, in step S2206, the search target object designated by the search designate position is extracted from the input search word storing buffer WBUF. The key word which coincides with the search target object is searched by a method similar to the process in step S2204. When the coincident key word exists, the image IDs and the tile numbers are stored into the output buffer OBUF. A check is made to see if they coincide with those stored in the search target tile number storing buffer TBUF set in step S2205.

[0106] When the tile number coincides with the designate position, it is held as it is in the output buffer OBUF and the processing routine advances to step S2209. When the tile number does not coincide with the designate position, the tile number which does not coincide and the image ID are deleted from the output buffer OBUF and the number of key words is also reduced.

[0107] Any one of the processes in steps S2205 and S2206 can be executed first.

[0108] The tile number in which the key word coincides can be also derived from the tile numbers obtained in step S2205. With this method, there is no need to refer to the whole data table in FIG. 11. It is sufficient to merely refer to the table including the tile number obtained in step S2205. The search processing time can be reduced.

[0109] In step S2209, the image data is displayed on the basis of the key word search result.

[0110] The processes in steps S2205 and S2206 will now be described by using a specific example. They will be explained here by using an example of performing the process in step S2206 first. For example, when “apple at a lower right position” is given as a search word, as a result in which the search target object is “apple”, the following three images are obtained: the image in which the image ID is equal to 1563 and the tile numbers are equal to 7 and 12; the image in which the image ID is equal to 1564 and the tile numbers are equal to 13, 14, 18, and 19; and the image in which the image ID is equal to 1565 and the tile numbers are equal to 6, 7, 11, and 12. Further, since “lower right” is designated as a designate position, only the image data of the image ID 1564 having the tile numbers 13, 14, 18, and 19 corresponding to the lower right is obtained as a search result and only the image shown in FIG. 7 is displayed.

[0111] Step S2207 is a process in the case where it is determined that the search word includes the search target image characteristic word (“red”, “blue”, etc.). The designated image characteristic amount is obtained. When the image characteristic amount is obtained, the coloring 131 which coincides with the image characteristic word is checked with reference to the data table of FIG. 13 stored in the memory DISK 14. When there is the coincident coloring 131, the corresponding image characteristic is extracted from the memory 132 and stored into the search target image characteristic storing buffer FBUF which is temporarily stored in the RAM 13. When the coincident coloring does not exist, the invalid value (for example, negative number) is stored in the search target image characteristic storing buffer FBUF.

[0112] In step S2208, the search target object designated by the search designation image characteristic is extracted from the input search word storing buffer WBUF. The key word which coincides with the search target object is searched by a method similar to the process in step S2204. When the coincident key word exists, the image ID and the tile number stored in the output buffer OBUF are extracted. The image characteristic 104 corresponding to the image ID and the tile number is extracted with reference to the data table of FIG. 10 stored in the memory DISK 14. A check is made to see if the extracted image characteristic coincides with the search target image characteristic storing buffer FBUF set in step S2207. When the image characteristic coincides with the designated image characteristic, they are held as they are in the output buffer OBUF. The processing routine advances to step S2209. When the image characteristic does not coincide with the designated image characteristic, the tile number and the image ID which do not coincide are deleted from the output buffer OBUF and the number of key words is also reduced.

[0113] In step S2209, the image data is displayed on the basis of the key word search result.

[0114] When the number of coincident key words is equal to 0 by referring to the output buffer OBUF outputted by the process in any one of steps S2204, S2205, and S2207, a fact that the search result does not exist is displayed on the CRT 18.

[0115] In the other cases, the tile number corresponding to the head image ID stored in the output buffer OBUF is displayed. The image ID and the tile number are designated and the image data of the corresponding tile is sent from the image file in the memory DISK 14 and is displayed on the CRT 18.

[0116] For example, when “red apple” is given as a search word, as a result of that the search target object is “apple”, the following three images are obtained: the image in which the image ID is equal to 1563, the tile numbers are equal to 7 and 12, and the image characteristic is equal to 3; the image in which the image ID is equal to 1564, the tile numbers are equal to 13, 14, 18, and 19, and the image characteristic is equal to 15; and the image in which the image ID is equal to 1565, the tile numbers are equal to 6, 7, 11, and 12, and the image characteristic is equal to 15. Since “red” is designated as an image characteristic, only the image data of the image ID 1563 having the image characteristic 3 corresponding to “red” is derived as a search result and only the image shown in FIG. 6 is displayed.

[0117] Although the above embodiment has been described on the assumption that the number of tiles into which the image is divided is equal to 20, even if the number of tiles is set to an arbitrary number, processes can be also similarly executed. In this case, it is sufficient to proportionally allocate the number of tiles as the tile numbers in the correspondence table of the search position designate words and the tile numbers which are stored in the memory DISK 14.

[0118] Although the number of search position designate words has been set to 10, the invention is not limited to those words. It is also possible to use a plurality of position designate words such that the same designate position is shown and morphemes are different such as “upper” and “upward”. Likewise, as for the image characteristic words, words showing similar colors such as “red”, “red color”, and “pure red” can be also handled as image characteristic words.

[0119] Although the representative colors of the tiles as image characteristic amounts have been used as color components, the image characteristic is not limited to the color components. Similar effects are also derived even if various image characteristic amounts such as luminance distribution of the image, a rough shape pattern due to the outline extraction, image frequency component, and the like may be handled.

[0120] It will be obviously understood that the invention is also accomplished by a method whereby program codes of software to realize the function of the foregoing embodiment are recorded into a storage medium, the storage medium is supplied to a system or an apparatus, and a computer (or CPU or MPU) of the system or apparatus reads out and executes the program codes stored in the storage medium.

[0121] In this case, the program codes themselves read out from the storage medium realize the new function of the invention. The storage medium in which the program codes have been stored constructs the invention.

[0122] As a storage medium to supply the program codes, for example, it is possible to use a floppy disk, a hard disk, an optical disk, a magnetooptic disk, CD-ROM, a CD-R, a magnetic tape, a non-volatile memory card, an ROM, or the like.

[0123] The function of the embodiment mentioned above is realized by executing the program codes read out by the computer. Further, the OS or the like which operates on the computer executes a part or all of the actual processes on the basis of the instruction of the program codes and the function of the embodiment mentioned above can be also realized by the processes executed.

[0124] Moreover, the program codes read out from the storage medium are written into a memory provided for a function expansion board inserted in a computer or a function expanding unit connected to a computer and, after that, on the basis of an instruction of the program codes, a CPU or the like equipped in the function expansion board or function expanding unit executes a part or all of the actual processes. The function of the foregoing embodiment can be also realized by the processes executed.

[0125] The invention can be also obviously applied to a case where the program codes of the software to realize the function of the embodiment mentioned above are recorded in the storage medium, and the program is transmitted from the storage medium to a requester through a communication infrastructure such as a personal computer communication or the like.

[0126] As described above, by dividing the image data into a plurality of areas and allocating the attribute information every area, the image data can be searched in consideration of the location in the image where the target search object exists.

[0127] By dividing the image data into a plurality of areas and allocating the attribute information every area, a portion including the target object can be extracted at a high speed from the image data in which the target image is included.

[0128] There is also an effect such that the time which is required to display is also reduced in association with it.

[0129] In the case where such an image exists on the network of a slow transmitting speed, the time which is required to transfer the image data can be also reduced as compared with the case of transferring the whole image.

[0130] A whole construction is omitted here because it is substantially the same as that mentioned above.

[0131] In the embodiment, an image search in which attention is paid to a point as a feature of the FlashPix format such that images of a plurality of resolutions are provided will now be described.

[0132] If the image of FIG. 2 has layer images like a FlashPix format, it is as shown in FIG. 24. The image file of the FlashPix format has images of a plurality of resolutions. The image of the lowest resolution assumes a resolution 0 (layer No. 0) and the images are sequentially allocated to resolutions 1, 2, and 3. Resolution information as numerical value information showing the resolution will be referred to as image designation information (layer No.) hereinbelow. The image file is divided every layer of the resolution. The divided image data is divided into tiles of (64×64) pixels and a key word included in each tile is as described in FIG. 27. That is, in the layer No. 3 as image designation information, the following key words are allocated. “sky” is allocated to the tiles 0, 1, 2, 5, and 10. “sky” and “sun” are allocated to the tiles 3, 4, 8, and 9. “sky” and “male” are allocated to the tiles 6 and 11. “sky” and “female” are allocated to the tiles 7 and 12. “sky” and “yacht” are allocated to the tiles 13 and 14. “sea” is allocated to the tile 15. “sea” and “male” are allocated to the tile 16. “sea” and “female” are allocated to the tile 17. “sea” and “yacht” are allocated to the tiles 18 and 19. Similarly, in the layer No. 0, key words of “sky, sun, sea, male, female, yacht” are allocated to the tile 0.

[0133] FIG. 4 shows a partial image constructed by the tiles 13, 14, 18, and 19 of the image shown in FIG. 2.

[0134] FIGS. 6, 7, and 8 also show an example of the image divided into 20 tiles and stored in a manner similar to FIG. 2.

[0135] FIG. 9 shows a table in which the image IDs and the image file storing paths are made correspond to each other in the invention. Reference numeral 91 denotes the image ID which is the identification number which is individually allocated to each image file. Reference numeral 92 denotes the file path showing a storage destination in the external memory DISK 14 of the image file corresponding to the image ID. The file path 92 corresponds to the directory and file in the MS-DOS.

[0136] FIG. 29 shows a table in which the image IDs, the layer numbers as image designation information, the tile numbers as area designation information, the key words as attribute information, and the image characteristics are made correspond to each other. The area 101 stores the image ID corresponding to the area 91 in FIG. 9. The area 102 stores the tile number which each of the image IDs has. In the area 103, the image characteristic drawn on the tile corresponding to the tile number 102 is expressed by a language and a key word is stored by a character code (for example, unicode). A plurality of key words can be also stored with respect to one tile number. The area 104 stores the image characteristics of the tile corresponding to the tile number. As image characteristics, in the invention, color components in which the average of the pixel values of RGB constructing the tile is obtained and the number of colors is subtracted to 27 colors are used particularly as representative colors of the tile. As a method of subtractive colors, a well-known method such as method of simply dividing each of the RGB pixel values, method using a reference table, or the like is used. In the diagram, the image whose image ID is equal to 1024 has 20 tiles, the key word of “sky” is allocated to the tile of the tile No. 0, and the color component of “11” is allocated as an image characteristic. Reference numeral 1005 denotes a layer number. Although it is made correspond with respect to only the image data of the high resolution of the layer 3, it is obviously inserted into the correspondence table in FIG. 29 with regard to all of the layers (0, 1, 2, 3, . . . ).

[0137] FIG. 28 is a table in which the key words, the image IDs, the tile numbers, and the layer numbers are made correspond to each other. The table of FIG. 28 is a table in which the corresponding image IDs and tile numbers are collected every key word shown at 103 in FIG. 29. Reference numeral 1101 denotes a key word; 1102 an area to store the image ID corresponding to the key word of 1101; 1103 an area to store the number of the tile having the key word 1101 among the tiles corresponding to the image IDs; and 1104 an area to store the layer number corresponding to the image ID. Although only the layer 3 is stored here and any other layers are omitted, it is assumed that all of the layers (0, 1, 2, 3, . . . ) have been stored in the correspondence table.

[0138] FIG. 12 is a table in which the position designate words serving as one of the attribute information to designate the position of a search target object in an image and the tile numbers are made correspond to each other. The area 121 stores the position designate word by a character code (for example, unicode). In the embodiment, as position designate words, ten words of “Top”, “Upper Left”, “Upper Right”, “Middle”, “Center”, “Left Middle”, “Right Middle”, “Bottom”, “Lower Left”, and “Lower Right” are used. The area 122 stores the tile number shown by the position designate word according to the divided tile. For example, when “Upper Right” is designated as a position designate word, the tile numbers of 3, 4, 8, and 9 are specified as search targets. As mentioned above, the position designate word (position designation information) indicates the position corresponding to the image of the tile.

[0139] FIG. 13 is a table in which the image characteristic words and the image characteristics are made correspond to each other. In the invention, the representative color of the tile is used as an image characteristic and a language showing the color is used as an image characteristic word. A color name (coloring) is stored in the area 131 as a characteristic word by a character code (for example, unicode). The image characteristic amount shown by the coloring is stored into the area 132 by the color component. In the diagram shown, for example, the coloring having the color components 2, 3, and 4 indicates “red”.

[0140] In the embodiment, the representative color is divided into 27 colors as image characteristic amounts. In this instance, each of an R (Red) axis, a G (Green) axis, and a B (Blue) axis is divided into three equal parts and an RGB space is divided into 27 cells, thereby dividing the representative colors. When the representative color of the tile is determined, a candidate indicating to which one of the divided 27 equal cells the (64×64) pixels included in the tile correspond is voted. The cell in which the number of votes is the largest is set to the representative color. The divided 27 equal cells are shown by the numbers 0 to 26, respectively.

[0141] FIG. 14 shows an example of a correspondence table of the image IDs and the key words which has conventionally been used. In order to store the image shown in FIG. 5 and the key word corresponding to the image, the image ID has been stored in the area 141 and the key word corresponding to the image ID has been stored in the area 142.

[0142] Prior to performing the search processing of the image processing apparatus of the invention, a database is made, namely, the correspondence table as mentioned above is made. Although the database can be made any time, unless the correspondence table of the database is made, the search cannot be performed.

[0143] The user first selects image data to be inserted into the database. The image processing apparatus of the invention forms an image characteristic amount from the selected image data by the foregoing method.

[0144] The key word is subsequently extracted. Upon extraction and formation of the key word, for example, by analyzing a text of document data, the word dictionary DIC stored in the external memory (DISK) 14 is referred, the relevant word is extracted, and the key word is formed. Nouns which are generally used have been stored in the word dictionary DIC. In the image data, when the key word is formed, the user can also designate or a key word can be also allocated to an image in a certain region and stored. For example, when there is an image of “apple” in the image data, the “apple” is allocated as a key word to the tile in which the “apple” exists. “apple” is formed as a key word at a position corresponding to the image ID, tile number, and layer number in the correspondence table of the database.

[0145] As mentioned above, the key word and image characteristic amount as attribute information are extracted from the image data and formed.

[0146] FIG. 23 is a flowchart for explaining the operation of the image processing apparatus of the invention and the operation will now be described hereinbelow with reference to FIG. 23. The process of the operation is realized by controlling by the CPU 11 on the basis of the search program stored in the ROM 12.

[0147] In step S101, the layer number as image designation information to be searched is inputted. The layer number is inputted from the keyboard KB 15.

[0148] In step S102, the layer number inputted in step S101 is stored in the RAM 13.

[0149] In step S103, the search processing is executed in the layer designated by the layer number selected in step S102. The search processing in step S103 will be described in detail hereinlater with reference to FIG. 24.

[0150] When the search processing is finished in step S103, either one of the mode to finish the processing routine and the mode to perform the search in a new layer can be selected. In case of newly searching, the processing routine is returned to step S101.

[0151] FIG. 25 is a flowchart for explaining the operation of the image processing apparatus of the invention and the search processing operation will now be described hereinbelow with reference to FIG. 25. The process of this operation is realized by controlling by the CPU 11 on the basis of a searching program stored in the ROM 12.

[0152] A search word as a target of the search is inputted in step S201. The input of the search word is performed by storing a character code inputted from the keyboard KB 15 into the search word storing buffer WBUF in the RAM 13.

[0153] The search word stored in the search word storing buffer WBUF is recognized as a word in step S202. The recognition of the word is performed by the well-known morpheme analyzing method by referring to the word dictionary DIC in the external memory DISK 14.

[0154] Which one of the search target object and the search target image characteristic the classification of word recognized in step S202 indicates is discriminated in step S203. The discrimination of the word classification can be performed by storing the classifications of words in the word dictionary DIC in the memory DISK 14. For example, if the search word is “red” or “blue”, the word classification is the search target image characteristic. In the cases other than the color, the word classification is the search target object. The processing routine is branched to each process in step S204 or S205 in accordance with the word classification discriminated in step S203.

[0155] Step S204 is a process in the case where it is determined that the search word is the search target object. The key word corresponding to the search word is searched. The key word search is performed with reference to a data table of FIG. 28 stored in the memory DISK 14. First, a check is made to see if the key word 1101 which coincides with the search word exists in the search table (FIG. 28) corresponding to the layer selected in step S102 of the flowchart of FIG. 23. If the keyword 1101 exists, it is stored as a search result into the output buffer OBUF. If it does not exist, the number of search tiles is set to 0 and the processing routine advances to step S207.

[0156] Step S205 is a process when it is determined that the search word is the search target image characteristic word (“red”, “blue”, or the like). The designated image characteristic amount is obtained. When obtaining the image characteristic amount, the coloring 131 which coincides with the image characteristic word is checked with reference to the data table of FIG. 13 stored in the memory DISK 14. When the coloring 131 coincides, the corresponding image characteristic amount is extracted from the area 132 and stored into the search target image characteristic storing buffer FBUF which is temporarily stored in the RAM 13. When the coincident coloring does not exist, the invalid value (for example, negative number) is stored into the search target image characteristic storing buffer FBUF.

[0157] In step S206, the search target object which is designated by the search target image characteristic word is obtained by again classifying the word. For example, the word decorated by the search target image characteristic word “red” is searched, thereby obtaining the search target object of “apple”. In only the layer in which the obtained search target object (for example “apple”) is selected and designated by the layer number by a method similar to the process in step S204, the key word which coincides with the search target object is searched. As a search result, the image ID, layer number, and tile number are stored in the output buffer OBUF. When the key word is searched, the image ID, tile number, and layer number stored in the output buffer OBUF are extracted and the image characteristic 104 corresponding to the image ID, tile number, and layer number is extracted with reference to the data table of FIG. 29 stored in the memory DISK 14. A check is made to see if the extracted image characteristic coincides with the search target image characteristic storing buffer FBUF set in step S205. That is, in case of searching “red apple”, “apple” is searched by the key word search. The image characteristics (0 to 26) are obtained from the searched image ID with reference to the data table of FIG. 29. A check is made to see if they coincide with the search target image characteristic (“red”).

[0158] When the image characteristic obtained with reference to the data table coincides with the search target image characteristic, it is held as it is in the output buffer OBUF. The processing routine advances to step S207. When the image characteristic does not coincide with the designated image characteristic (“blue apple” or the like), the tile numbers, image IDs, and layer numbers which do not coincide are deleted from the output buffer OBUF. The number of key words is also reduced.

[0159] In step S207, on the basis of the key word search result and the image characteristic amount search result, the image data of the layer selected on the basis of the layer number as image designation information in the flowchart of FIG. 23 is displayed.

[0160] When the number of coincident key words is equal to 0 with reference to the output buffer OBUF outputted by the process in either one of steps S204 and S205, a fact that the search result does not exist is displayed on the CRT 18. When the number of key words is not equal to 0, the tile number and the layer number corresponding to the head image ID stored in the output buffer OBUF are displayed. The image ID, tile number, and layer number are designated. The image data corresponding to the tile number of the corresponding layer number is sent from the image file in the memory DISK 14 and displayed on the CRT 18.

[0161] For example, when the user designates the layer number as “3” and inputs the word of “yacht” as a search word, as a result in which the key word coincides, the tile numbers 13, 14, 18, and 19 of the layer No. 3 of the image ID 1024 are obtained as a search result. Since the image has been divided and preserved, when displaying, the images of those four tiles can be first displayed. Thus, the image of FIG. 4 is obtained.

[0162] In the image of FIG. 4, the area is ⅕ of that of the image of FIG. 2. The time which is required to display is also reduced in association with it. When such an image exists on a network of a slow transmitting speed, the time which is required to transfer the image data can be also reduced as compared with that in case of transferring the whole image.

[0163] For example, when “red apple” is given as a search word, as a result in which the search target object is “apple”, the following three images are obtained: the image in which the image ID is equal to 1563, the tile numbers are equal to 7 and 12, and the image characteristic is equal to 3; the image in which the image ID is equal to 1564, the tile numbers are equal to 13, 14, 18, and 19, and the image characteristic is equal to 15; and the image in which the image ID is equal to 1565, the tile numbers are equal to 6, 7, 11, and 12, and the image characteristic is equal to 15. Further, since “red” is designated as an image characteristic, only the image data of the image ID 1563 having the image characteristic 3 corresponding to “red” is obtained as a search result. Only the image of FIG. 6 is displayed.

[0164] For example, when the user designates the layer number as “0” and inputs a word of “yacht” as a search word, as a result in which the key word coincides, the tile No. 1 of the layer No. 0 of the image ID 1024 is obtained as a search result. When displaying, an image of 2303 locating at the resolution 0 in the format diagram of FIG. 24 is displayed.

[0165] In the above description, although only the tiles of the relevant image in the search have been displayed, the invention is not limited to them but the whole image including the relevant tiles can be also displayed.

[0166] FIG. 26 is a flowchart showing the operation for an approximating search per layer of the image processing apparatus of the invention. The search processing operation will now be described hereinbelow with reference to FIG. 26. The process of this operation can be realized by controlling by the CPU 11 on the basis of the search program stored in the ROM 12.

[0167] First, in step S301, a designation of an image to be approximately searched and a layer (resolution) is inputted. For example, it is now assumed that the image of FIG. 7 and the layer number “0” (low resolution) were selected by the user.

[0168] In step S302, the image characteristic amount of the image inputted in step S301 is obtained with reference to the data table of FIG. 29. For example, “24” as an image characteristic amount corresponding to the layer number “0” of 1564 as an image ID in FIG. 7 is obtained.

[0169] In step S303, an image having an image characteristic amount near the image characteristic amount as a search target object obtained in step S302 is searched from the data table of FIG. 29. The searched image is stored into the output buffer OBUF. A searching range here is set to only the image corresponding to the layer inputted in step S301. For example, FIG. 8 as an image having an image characteristic amount near “24” as an image characteristic amount obtained in step S302 is derived from the image group of the layer number “0”.

[0170] The extraction of the image having the near image characteristic is performed on the basis of a penalty matrix which is determined by the allocation of the image characteristic amounts. The penalty matrix has been stored in the memory DISK 14. As in the embodiment, the numbers 0 to 26 are allocated to the cells obtained by dividing the color space into 27 regions (cells obtained by dividing into three equal portions for the RGB axes), a difference (penalty) of the color of each number is decided by an allocating method, and it is stored into a table called a penalty matrix. For example, distances in the color spaces of the image characteristic amounts “1” and “10” have been stored in the penalty matrix (difference matrix). The distance in the color space becomes nearness of every image characteristic amount.

[0171] In step S304, the image data searched in step S303 is displayed. The image IDs and the layer numbers which were stored in the output buffer OBUF are displayed with reference to the output buffer OBUF which was searched and outputted by the process in step S303. The image ID and layer number are designated.

[0172] The image data corresponding to the corresponding layer number is sent from the image file in the memory DISK 14 and is sequentially displayed on the CRT 18 by the penalty matrix in accordance with the order from the near image characteristic amount. For example, FIGS. 8 and 6 are searched as images near FIG. 7. FIG. 8 is first displayed and FIG. 6 is subsequently displayed because of the order from the near image characteristic amount.

[0173] In step S305, a check is made to see if an instruction to perform the re-search has been inputted. If the instruction to perform the re-search is inputted, the processing routine is returned to step S301 and the re-search is executed. When an instruction indicating that the re-search is not performed is inputted, the search program is finished. In the re-search, steps S301 to S304 are repeated until the satisfactory search is executed. The process for re-searching is executed for only the image in which a target to be searched in step S303 has been searched in step S304. The search target is selected from the output buffer OBUF and only the portion in the new layer of the same image ID is used as a search target with reference to the data table in FIG. 29, so that the process can be easily performed.

[0174] By enabling the re-search in step S305 to be executed as mentioned above, there is an advantage such that a rough temporary (primary) search is performed at the low resolution and a detailed secondary search at a high resolution can be executed from the search result.

[0175] When the user is satisfied with the result of the temporary (primary) search, the user designates from the thumbnail display of the temporary search result by the keyboard KB 15, thereby obtaining the high resolution image data of the designated image through the memory DISK 14, RAM 13, or NIC 19 and displaying on the CRT 18.

[0176] In the image 2303 in FIG. 24, its area is equal to {fraction (1/64)} of that of the image of FIG. 2 and the time which is required to display is also extremely reduced. The image 2303 is optimum as a list of the search result because of the thumbnail display as well. For the user, the temporary search is executed in the layer 0 of the low resolution and the thumbnail display as a list of the result is derived. Therefore, an image which he wants to see in detail later can be also designated. Further, when the user wants to search finer, the secondary search can be performed in the layer 3 of the high resolution from the search result of the temporary search. In case of the key word search, there is a merit such that the tile corresponding to the result is displayed. In case of searching the image characteristic amount, as shown in FIG. 3, since the image characteristic amount differs due to the average value depending on the layer, an effect such that the rough approximation search is performed in the layer 0 and the detailed search can be executed in the layer 2 or 3 is also derived.

[0177] Although the case such as “red apple” in step S202 of obtaining the search word in FIG. 25 has been described, other search words such as “top is sky, bottom is sea” or “top is white, bottom is blue” can be also inputted. By this method, a search to discriminate whether the key words “sky” and “sea” exist in the tile corresponding to the position designate word mentioned in FIG. 12 or not is performed. In case of the image characteristic amount, by discriminating whether the image characteristic amount “26” of white exists in the top tiles “0, 1, 2, 3, 4” or not and whether the image characteristic amount “9, 10” of blue exists in the bottom tiles “15, 16, 17, 18, 19” or not, the image characteristic amount can be easily obtained.

[0178] However, in case of an image such as a photograph or the like, there is a case where the portrait and landscape are replaced in dependence on a method of taking a picture. In other words, in case of a portrait photograph, since the top becomes the right (or left), so long as a search processing as mentioned above, a case where the search result is not obtained is considered. According to the invention, however, since it is possible to search every resolution, even in case of the search as mentioned above such that a missing will occur if the image is searched at a high resolution, a rough search of a low resolution can be performed. Therefore, a search missing ought to be extremely reduced.

[0179] Although the program to realize the operation of the image processing apparatus of the invention described in FIGS. 23, 25, and 26 has been stored in the ROM 12 as mentioned above, the invention can be embodied even if such a program is inputted from the external memory 14 as a storage medium such as floppy disk, CD-ROM, or the like. Such a storage medium is also obviously incorporated in the scope of claims of the invention.

[0180] Although the invention has been described by using the image of the FlashPix format in the embodiment, naturally, the invention is not limited to it. Any other image formats can also realize the present invention so long as it is an image format having a plurality of images of different resolutions for a certain image or an image format in which certain image data is divided into a plurality of areas (tiles).

[0181] In the embodiment, although the search processing has been performed with regard to the image stored in the memory DISK 14, the invention can be also realized by searching images existing in an image server on the network through the NIC 19 and by downloading only the necessary images from the search result. In this case, in the process for displaying the search result on a tile unit basis, it is effective in terms of a point that the load of the network is reduced and the display can be performed at a high speed.

[0182] By realizing the image processing apparatus of the invention as in the embodiment, the image is layered to a plurality of images of different resolutions, and there is provided the means for allocating the layer numbers to the images of the respective layers, for making the attribute information showing the characteristics of the images correspond every image, for dividing the image of each layer into the continuous tiles, for allocating the tile numbers to the divided tiles, for making the key word showing the characteristic of the image correspond every tile, and for accumulating the correspondence relation between the tile numbers and the key words together with the image data. Therefore, from the image including a target object, a portion including the target object can be extracted at a high speed.

[0183] Since the search can be performed every resolution, the rough temporary search is performed in the image of a low resolution, the whole search result can be displayed like a thumbnail display, and further, the detailed secondary search can be executed at a high resolution. A situation such that the image to be actually searched is dropped out from the search processing is remarkably reduced. The desired image of the user can be searched and displayed at a high speed.

[0184] As mentioned above, since the storing means for storing the attribute information and the image designation information, respectively, for each image of the image file having a plurality of images is provided, a plurality of attribute information can be possessed for one kind of image.