Apparatus and method for evaluating web pages

[0001] 1. Field of the Invention

[0002] The present invention relates to an apparatus and a method for evaluating the information disseminated on Web pages of Internet.

[0003] 2. Description of the Related Art

[0004] A set of Web pages normally comprises a top page and a plurality of pages linked to the top page in a hierarchical structure, and each page includes dissemination information. It is said that there are two through four billions of Web pages in the world. To appropriately disseminate information in a domain by enterprises and persons having the same purpose, it is not satisfactory only to plan to increase the frequency of access to the Web pages or simply arrange the format. Furthermore, since a retrieving technology using a keyword is being widely spread, it is important to appropriately prepare dissemination contents on a Web page to be retrieved.

[0005] To attain this, it is necessary to evaluate in detail the contents of the information disseminated on a Web page, and to enhance the value and the completion level of the Web page by properly feeding back the evaluation result.

[0006] It is necessary to enhance the completion level of a Web page not only to disseminate information to a viewer, but also to effectively prepare the Web contents for supporting the development technology in the process of developing the technology of extracting information, summarizing information, etc. because the extraction and the summarization of information are based on each piece of target information disseminated on the Web page.

[0007] Several evaluation criteria of a Web page have already been suggested as described in the following electronic information mainly in the nations of English speaking people.

[0008] J. Alexander and M. A. Tate, Evaluating Web Resources, 1996.

[0009] http://www2.widener.edu/Wolfgram-Memorial-Library/webevaluation/webeval.htm

[0010] The evaluation criteria are obtained by adding the features of Web to evaluation items on common printed matter. For example, there is critical evaluation of a Web page using a check list and its effect with the current state of Web taken into account and a check list generated including new viewpoints. On the other hand, the value of a Web page can also be computed by the time required to open the Web page with a view to attraction of the attention by the largest possible number of viewers. Furthermore, an attempt to evaluate a Web page by requesting a large number of registered members to answer questionnaires has already been practically realized.

[0011] However, in the above mentioned conventional technology, the contents of the descriptions on a Web page are not explained, but formal evaluation and subjective evaluation are mainly described. Therefore, there is the problem that an evaluation value depends on the value judgments and current situation of an evaluator.

[0012] An object of the present invention is to provide an apparatus and a method for practically and objectively evaluating a Web page.

[0013] The Web page evaluation apparatus according to the present invention includes a storage device, a generation device, an evaluation device, and an output device.

[0014] The storage device stores the correspondence between a plurality of domains in Internet and a plurality of evaluation items. The generation device refers to the correspondence stored in the storage device, extracts an evaluation item corresponding to a specified domain in the plurality of evaluation items, and generates an evaluation set formed by the extracted evaluation items. The evaluation device computes the evaluation score on a Web page to be evaluated using an evaluation item contained in the generated evaluation set, and the output device outputs an evaluation result including the obtained evaluation score.

[0015] FIG. 1 shows the configuration of the evaluation apparatus according to the present invention;

[0016] FIG. 2 shows a flow of the process;

[0017] FIG. 3 shows the process of a learning unit;

[0018] FIG. 4 shows target data of a tag determining process;

[0019] FIG. 5 shows the data of a Web page;

[0020] FIG. 6 shows the data after a tag cleaning process;

[0021] FIG. 7 shows directory information;

[0022] FIG. 8 shows a view (1) of an evaluation set;

[0023] FIG. 9 shows a view (2) of an evaluation set;

[0024] FIG. 10 shows a view (3) of an evaluation set;

[0025] FIG. 11 shows a view (4) of an evaluation set;

[0026] FIG. 12 shows a view (5) of an evaluation set;

[0027] FIG. 13 shows a view (6) of an evaluation set;

[0028] FIG. 14 shows the data after a tag assigning process;

[0029] FIG. 15 shows the process of detailing a company profile;

[0030] FIG. 16 shows the process of detailing product information;

[0031] FIG. 17 is a view (1) showing the ontology of an item name;

[0032] FIG. 18 is a view (2) showing the ontology of an item name;

[0033] FIG. 19 is a view (3) showing the ontology of an item name;

[0034] FIG. 20 is a view (1) showing a refined and adjusted evaluation set group;

[0035] FIG. 21 is a view (2) showing a refined and adjusted evaluation set group;

[0036] FIG. 22 is a view (3) showing a refined and adjusted evaluation set group;

[0037] FIG. 23 is a view (4) showing a refined and adjusted evaluation set group;

[0038] FIG. 24 is a view (5) showing a refined and adjusted evaluation set group;

[0039] FIG. 25 is a view (6) showing a refined and adjusted evaluation set group;

[0040] FIG. 26 shows the process of a generation unit;

[0041] FIG. 27 is a view (1) showing a weight-assigned specified domain evaluation set;

[0042] FIG. 28 is a view (2) showing a weight-assigned specified domain evaluation set;

[0043] FIG. 29 is a view (3) showing a weight-assigned specified domain evaluation set;

[0044] FIG. 30 is a view (4) showing a weight-assigned specified domain evaluation set;

[0045] FIG. 31 is a view (5) showing a weight-assigned specified domain evaluation set;

[0046] FIG. 32 is a view (6) showing a weight-assigned specified domain evaluation set;

[0047] FIG. 33 shows the process of an evaluation unit;

[0048] FIG. 34 is a view (1) showing the evaluation of an information disseminating item;

[0049] FIG. 35 is a view (2) showing the evaluation of an information disseminating item;

[0050] FIG. 36 shows the evaluation of a detailed information item;

[0051] FIG. 37 shows the process of a result processing unit;

[0052] FIG. 38 shows the process of a construction support unit;

[0053] FIG. 39 shows the configuration of an information processing device; and

[0054] FIG. 40 shows storage media.

[0055] The embodiments of the present invention are described below in detail by referring to the attached drawings.

[0056] The Web page evaluation apparatus according to the present invention comprises a storage device, a generation device, an evaluation device, and an output device.

[0057] The storage device stores the correspondence between a plurality of domains in Internet and a plurality of evaluation items. The generation device refers to the correspondence stored in the storage device, extracts an evaluation item corresponding to a specified domain in the plurality of evaluation items, and generates an evaluation set formed by the extracted evaluation items. The evaluation device computes an evaluation score of a Web page to be evaluated using an evaluation item contained in the generated evaluation set, and the output device outputs an evaluation result containing the obtained evaluation score.

[0058] By storing in advance the correspondence between a domain and an evaluation item, when a user specifies an arbitrary domain, the generation device can automatically extract a practical evaluation item corresponding to the domain. Thus, a different evaluation set is generated depending on a specified domain, and the evaluation device computes a different evaluation score depending on the specified domain for the same Web page to be evaluated.

[0059] The user can obtain a practical evaluation result only by specifying a domain, and can obtain a different evaluation result by changing the domain. Any user can use the evaluation apparatus and obtain the same evaluation result if the same domain is specified. Therefore, a Web page to be evaluated can be evaluated practically and objectively.

[0060] The storage device corresponds to, for example, an external storage device 185 described later by referring to FIG. 39, and the correspondence between domains and evaluation items corresponds to, for example, a refined and adjusted evaluation set group 96 described later by referring to FIG. 3. The generation device and the evaluation device correspond to, for example, a generation unit 22 and an evaluation unit 23 described later by referring to FIG. 1. The output device corresponds to, for example, an output device 184 shown in FIG. 39.

[0061] When a Web page is generated, a tool and an application are provided from the viewpoints of the easiness on the generator side, the attractancy of the attention of users, the appropriate representation configuration, etc.

[0062] When a Web page is started up, the above mentioned viewpoints are required. However, since there are a number of similar Web pages and the number of Web pages increases ever day, these tools and applications are not enough to obtain a direct effect in generating and maintaining a Web page for dissemination of information. To attain this object, a supporting tool and application relating to the contents of the dissemination information are required.

[0063] To consider in detail the contents of the dissemination information, the following three functions are required.

[0064] (a) Arranging in detail evaluation items relating to the contents of a domain to be evaluated

[0065] (b) Determining a score of objective evaluation for the arranged evaluation items

[0066] (c) Analyzing an evaluation result from various viewpoints of a disseminator, a viewer, etc., and easily satisfying a request from them.

[0067] The domain-specified Web page evaluation apparatus according to the present embodiment conducts a consultation of a Web page depending on the evaluation set by arranging the contents of the disseminated information on the Web page as an evaluation set of the domain. The evaluation method is not a total evaluation (visibility, comprehensibility, etc.) of a Web page using the conventional check sheet, questionnaires, etc., but an evaluation method with the contents of each piece of disseminated information taken into account. Sequentially described below are the configuration and the operation of the Web page evaluation apparatus having the functions of (a) through (c).

[0068] FIG. 1 shows the configuration of the above mentioned evaluation apparatus. The evaluation apparatus shown in FIG. 1 is configured using an information processing device (computer), and comprises a learning unit 21, a generation unit 22, an evaluation unit 23, a result processing unit 24, and a construction support unit 25. Then, it has the function of evaluating a Web page on which information is disseminated using a specified domain evaluation set based on the disseminated contents and the function of improving the Web page based on the evaluation result.

[0069] The learning unit 21 learns an evaluation item from the Web page group in Internet. At this time, all items used in evaluation are extracted and arranged to generate an evaluation set group 26. The evaluation set is a set of evaluation items of each Web page, and formed by information disseminating items and detailed information items. These evaluation items are related to a domain in Internet in the evaluation set.

[0070] An information disseminating item is a basic item set in advance as the information to be disseminated when a Web page is generated. A detailed information item is an item obtained by analyzing contents information other than the information disseminating item. As an evaluation set, only information disseminating items can be used, or both information disseminating items and detailed information items can be used. As information disseminating items, for example, data provided with a tag in advance in a Web page is extracted.

[0071] Also as a detailed information item, unit information obtained by performing a natural language process on the information expressed by text, and extracting the expressed contents in detail is used. A natural language process can be, for example, a process of assigning a new tag using a pattern matching rule in the expression of text, and a process of extracting a morpheme determined to be noun from among the combinations of a morpheme extracted by the morphological analysis from the expression of text and its part of speech.

[0072] The generation unit 22 extracts an evaluation set in the user-specified domain from the evaluation set group 26, arranges the evaluation items, and generates a specified domain evaluation set 27. Upon receipt of a Web page to be evaluated, the evaluation unit 23 evaluates the contents using the specified domain evaluation set 27, and outputs an evaluation result. The result processing unit 24 presents an evaluation result to the user, and performs a process for improving the contents of the Web page based on the evaluation result. The construction support unit 25 supports the construction of a new Web page using the specified domain evaluation set 27.

[0073] With the above mentioned configuration, the user can flexibly specify a domain, and practically and objectively evaluate a Web page using an evaluation set optimized for a specified domain. Furthermore, the processes for presenting an evaluation result and improving a Web page can be smoothly performed. A user of the evaluation apparatus can be a disseminator and a viewer of the information.

[0074] FIG. 2 is a flow of the process performed by the evaluation apparatus shown in FIG. 1. First, the learning unit 21 extracts (31) information disseminating items from the Web page using the Web page cleaning technology, the morphological analysis technology, etc. when an evaluation item is learned from the Web page group, and registers them as an evaluation set in the database. Thus, the evaluation set group 26 of a Web page group is generated.

[0075] A mix-up occurs when an information disseminating item is selected and utilized if the extracted information disseminating items are collected at random. Therefore, the learning unit 21 arranges and sorts (32) the information disseminating items for each category contained in a system of a Web page group (a system of domains) according to directory information 33 for arranging the system, thereby generating an evaluation set.

[0076] However, sufficient evaluations with the contents taken into account cannot be conducted only using information disseminating items. Therefore, the learning unit 21 refines evaluation items so that the described contents can be considered in detail by utilizing the elements of the contents to be processed in extraction of information, summarization of information, etc., which are contents analyzing technology, as evaluation items, thereby generating a refined evaluation set group (34).

[0077] At this time, the learning unit 21 applies the contents analyzing technology such as the information extracting rule (for extracting information from text), etc. prepared in advance to the evaluation set group of information disseminating items, extracts a detailed information item, and extends the evaluation set group 26.

[0078] In the extracted information disseminating items and detailed information items, various expressions are used for the same item, and it is difficult to correctly evaluate data as is. Therefore, the learning unit 21 removes the fluctuation in expression (unification) using the ontology (dictionary of synonyms) 36 for each category of the system, and the evaluation items are appropriately arranged (35). For example, for the information disseminating items for each category arranged in the system, the item names are unified and standardized using the ontology 36 depending on the category, thereby unifying and standardizing the evaluation set group 26.

[0079] Then, the user selects some unnecessary categories from among a plurality of categories systematically arranged in advance, thereby specifying a domain. The generation unit 22 generates the specified domain evaluation set 27 from the evaluation set group 26 (41). Thus, a domain-dependent evaluation result can be obtained by limiting the domain to be evaluated based on the system arranged systematically in advance.

[0080] However, evaluation of the disseminated information of the domain cannot be sufficiently performed in some cases by using only the evaluation items extracted from an actual Web page. Therefore, necessary items are added according to the recent trend, new tendency, request, etc., and the extracted items are corrected by providing the respective functions (42).

[0081] Thus, the user can add necessary items (required and expected) to the specified domain evaluation set 27. The added items are extracted as new items from, for example, the trend of the entire industry of the domain, the questionnaires to the persons concerned, interested clients, etc.

[0082] There are also the cases in which the weight of evaluation of an evaluation item of the specified domain evaluation set 27 is to be changed for each category. In these cases, a function of assigning a weight in each category is provided (43). The weight is assigned by, for example, setting a computation function for each category, etc.

[0083] There can be the case in which a weight of evaluation is changed for each evaluation item depending on the evaluation viewpoint, etc. in the domain of the same category. In this case, a function of assigning a weight is provided for each evaluation item of the specified domain evaluation set 27 (44).

[0084] The viewpoint of assigning a weight is, for example, a providing enterprise side, a client side, an enterprise utilizing side, an information collecting side, etc. Depending on the viewpoint, the significance is determined for each item, and the weight is adjusted for each item. Furthermore, based on the ranking of each Web page, a weight can be assigned by the commonness of the items on the high ranking pages. The ranking in this case can be a significance ranking depending on the number of linked Web pages, the common popularity ranking, etc.

[0085] When the learning unit 21 and the generation unit 22 perform the above mentioned processes, the evaluation items in each domain to be evaluated can be practically arranged in detail with the contents of the disseminated information taken into account.

[0086] Then, the evaluation unit 23 evaluates the Web page to be evaluated by the evaluation item of the specified domain evaluation set 27 (51). Since it is necessary to output an evaluation result as quantitative information for a specified domain, an evaluation score is computed. There are some score computing methods, and a user can specify an arbitrary computing method.

[0087] In simple score computation, the existence of each information disseminating item of the specified domain evaluation set 27 is checked on a Web page to be evaluated, and the evaluation score can be simply computed from the number of existing items (52).

[0088] In the score computation including the URL (uniform resource locator) hierarchy, the evaluation score is computed with the depth of the hierarchy (link) from the top page taken into account for each page containing the information about each information disseminating item to incorporate the viewpoint of the user convenience into the evaluation score (53).

[0089] In the score computation in the order of items, an evaluation score is computed using the display order of the evaluation items of the specified domain evaluation set 27 on the Web page to be evaluated (54). The item display order can be obtained from the position of the item in the menu, the pop-up window, etc., and the evaluation score can be computed in the common order computed in the specified domain as a reference.

[0090] In the score computation of an information extraction result, the existence of each detailed information item of the specified domain evaluation set 27 is checked on the Web page to be evaluated, and the evaluation score is computed from the number of existing items (55). At this time, it is determined whether or not there is a corresponding detailed information item by actually analyzing the contents of the Web page to be evaluated.

[0091] In the score computation including the extension of the description of an extraction result, an evaluation score is computed for the page including the information about each detailed information item with the depth of the hierarchy from the top page and the extension of the description position in contents analysis taken into account to consider the viewpoint of user convenience for a detailed information item (56).

[0092] An objective score determination is performed on arranged evaluation items by the evaluation unit 23 performing a score computation as described above.

[0093] Then, the result processing unit 24 displays a computed evaluation score as an evaluation result on the screen (61).

[0094] When there is a fluctuation in expression in the same domain, a displayed evaluation result can be mistakenly understood by a user. Therefore, a function of performing a support (issuing an instruction) when an expression different from the common expression in the corresponding domain is used on each Web page is provided (62).

[0095] By the function, the result processing unit 24 presents advice of correcting an item name using the category-dependent ontology for the evaluation item for each category in the specified domain evaluation set 27. Thus, the user can correct the item name appropriately.

[0096] Furthermore, the result processing unit 24 can start up the interface for automatically making a correction instead of presenting advice to the user. In this case, at an instruction from the system, the fluctuation in expression in the same domain is actively corrected (63).

[0097] Although the evaluation result is quantitatively obtained in a specified domain by presenting an evaluation score, it is necessary to recognize what items and contents have problems when a user (disseminator) requests to take action for improvement of the obtained evaluation. Therefore, the result processing unit 24 presents the specific evaluation items and contents which have lowered the score for use in a Web page improving method (64)

[0098] Furthermore, if it is known how the evaluation result quantitatively changes by changing which item can be quantitatively changed as the improvement action 65 for the evaluation, a plan of the order of operations can be made after recognizing an improvement effect. Therefore, the result processing unit 24 predicts and presents the influence of the improvement result of each item on an evaluation score (66). For example, a new evaluation score is computed when some information disseminating items are actually added, and the value is presented.

[0099] The result processing unit 24 not only performs a simulation of the improvement of each item, but also actually starts up the input interface to allow a user to perform an improving operation (adding a necessary item, correcting the contents of disseminated contents, etc.) (67). In this case, an activated input interface automatically displays an input form for an improving operation.

[0100] In the evaluation of the contents of a Web page, the amount of the information to be disseminated which is contained (or missing) is presented as a main evaluation result. However, whether or not there is disseminated information unique to the Web page to be evaluated and not found on other Web pages is useful information to the manager of the Web page. Such information can be collected as a by-product during the evaluating process.

[0101] The result processing unit 24 presents an item other than the evaluation items of the specified domain evaluation set 27 as special information, and give advice that it corresponds to an odd item, a pulling power item, a wasteful item, etc. (68).

[0102] When the result processing unit 24 performs the above mentioned process, an evaluation result can be analyzed from various viewpoints such as the viewpoint of a disseminator, a viewer, etc., and the process can be easily performed at a request from them.

[0103] The construction support unit 25 supports a user when constructing a new Web page corresponding to a specified domain (71).

[0104] When a new Web page is generated, the support of adopting an item arrangement, an expression method, etc. is required, and a number of generation supporting tools are actually used. However, no support of specifying the contents of dissemination information, etc. in a specified domain is currently offered.

[0105] Using the construction support unit 25, suitable items and contents information can be presented to a user who desires to generate a new Web page in a specified domain using the information generated for evaluation of a Web page and disseminate information.

[0106] At this time, the user specifies a domain by selecting a category, and the construction support unit 25 generates basic meta-information about dissemination of information from the specified domain evaluation set 27, and presents it to the user (72). The basic meta-information can be a standard tag set, the DTD (document type definition) of an XML (extensible markup language), etc.

[0107] Furthermore, the construction support unit 25 not only presents the basic meta-information, but also actually starts up an input interface and allows the user to perform an information inputting operation for actual generation of a Web page according to the basic meta-information (73). The activated input interface automatically displays an input form for input of information according to the basic meta-information.

[0108] When the construction support unit 25 performs the above mentioned process, a disseminator can efficiently generate a Web page including the information corresponding to an evaluation item of a specified domain.

[0109] Then, a specific example of each process shown in FIG. 2 is explained by referring to FIGS. 3 through 38.

[0110] FIG. 3 shows the process of the learning unit 21. The learning unit 21 shown in FIG. 3 comprises a cleaning unit 81, an item extraction unit 82, a system arrangement unit 83, a system correspondence process unit 84, an information extraction unit 85, and an item fluctuation check unit 86, and outputs a refined and adjusted evaluation set group 96 using the input of a Web page group in Internet. The important function of the learning unit 21 is extracting evaluation items from a Web page and systematizing the extracted evaluation items.

[0111] First, the cleaning unit 81 performs Web cleaning to remove unnecessary tags and dust from the information on the Web page. By the Web cleaning, the part not to be processed as text such as a banner advertisement, moving picture/still image information, etc. displayed on the Web page can be removed.

[0112] Then, the item extraction unit 82 extracts tagged information as an information disseminating item 91 by performing a tag determining process, a tag cleaning process, a morphological process, a part-of-speech determining process, etc. on the cleaned information. The tag determining process is performed by extracting a tag of a menu item displayed on the Web page. For example, when the data as shown in FIG. 4 is given, the alt data used in the img tag, etc. is to be extracted. Furthermore, the tag cleaning process is to remove a tag from the given data.

[0113] FIG. 5 shows the description relating to a company profile contained in a Web page of a hypothetical enterprise. When such data is given, the tag cleaning removes a tag such as TABLE, TR, TD, B, etc. For example, when the tag cleaning is performed on a portion 101 of the data, the data shown in FIG. 6 is extracted.

[0114] The morphological process is performed by applying a morphological analysis, which is a common natural language process tool, to the portion of text information, and extracting a candidate for an evaluation item with a divided unit as a target. The part-of-speech determining process is performed by, for example, determining a part of speech of unit information divided in the morphological process, and extracting a specific part of speech as a candidate for an evaluation item. In this example, unit information that is recognized as a noun and appears N (for example, N3) times or more is extracted as a candidate.

[0115] The system arrangement unit 83 shown in FIG. 3 arranges in advance the system of the Web page group as the directory information 33. The directory information 33 is obtained by systematizing the categories of the Web page group, and is utilized in arranging the evaluation items of an evaluation set group 93 which is a learning result.

[0116] FIG. 7 shows an example of the directory information 33. The directory information has a hierarchical structure of 6 layers formed by categories 1 through 6, and each category has other categories at lower layers. For example, the industry of category 1 corresponds to the highest layer, and the company M of category 6, etc. corresponds to the lowest layer. Furthermore, the categories of the primary industry are agriculture, fishing, livestock industry, forestry, and hunting.

[0117] The system correspondence process unit 84 checks the part of the system of the directory information 33 corresponding to the extracted item, generates a systematized information disseminating item 92 as a combination of each item and a corresponding category, and stores it in the evaluation set group 93.

[0118] FIGS. 8 through 13 show an example of the evaluation set group 93 generated by extracting evaluation items (information disseminating items) from the Web pages actually through Internet. The extracted information disseminating items can be classified into a common item common to all Web pages, a specific item specific to each category, and an individual item specific to each Web page.

[0119] FIG. 8 shows the name, URL, and common items of the enterprises belonging to the category of a food manufacturer. FIG. 9 shows specific items and individual items of the enterprises. FIG. 10 shows the name, URL, and common items of the enterprises belonging to the category of a cosmetics manufacturer. FIG. 11 shows specific items and individual items of the enterprises. Furthermore, FIG. 12 shows the name, URL, and common items of the enterprises belonging to the category of an automobile manufacturer. FIG. 13 shows specific items and individual items of the enterprises.

[0120] In FIGS. 9, 11, and 13, the description order of specific items and individual items corresponds to the description order of the company names shown in the corresponding FIGS. 8, 10, and 12.

[0121] Then, the information extraction unit 85 extracts detailed information item 95 from the information on the Web page based on information extracting rule 94 stored in advance, and adds it to the information disseminating items of the evaluation set group 93. In this example, a detailed information item is extracted by assigning a new tag to the information on the Web page.

[0122] For example, the data as shown in FIG. 14 can be obtained when a tag is assigned based on the information extracting rule. In FIG. 14, tags of annunciator organization information, foundation information, element1, date of foundation, establishment information, date of establishment, uncertain organization supplementary information, other information, president information, capital information, and employee information are newly assigned. In this process, date of foundation, date of establishment, president information, capital information, employee information, etc. are extracted as detailed information items.

[0123] FIG. 15 shows an example of detailed information items obtained by describing in detail the information disseminating item of company profile. By applying the information extracting rule to the contents information disseminated in the item of company profile, detailed information items such as the company name, address, telephone number, representative (president), capital, foundation year, establishment year, number of employees, etc. are extracted.

[0124] FIG. 16 shows an example of detailed information items obtained by describing in detail the information disseminating item of product information. In this example, by applying the information extracting rule to the contents information disseminated in the item of product information, the detailed information items such as a product name, type information, product supplementary information, manufactured product information, manufacturing organization information, manufacturing organization supplementary information, etc. are extracted.

[0125] Thus, by describing in detail the information disseminating item in the information extracting process, the evaluation items of the evaluation set group 93 is refined.

[0126] Then, the item fluctuation check unit 86 performs a name unifying process using the ontology 36, and removes the fluctuation in item names. Thus, the item names of the evaluation set group 93 are unified into a representative item name for each category, and the standardization of item names can be realized.

[0127] FIGS. 17 through 19 show examples of the ontology 36. FIGS. 17, 18, and 19 show the representative names of the respective items belonging to the categories of the food manufacturer, cosmetics manufacturer, and automobile manufacturer, and the names of the corresponding prospects. The representative name can be, for example, selected using the number of occurrences of each item name as a reference.

[0128] FIGS. 20 through 25 show an example of the refined and adjusted evaluation set group 96 generated as a learning result of an evaluation set group. In these figures, the evaluation sets of respective categories of the food manufacturer, cosmetics manufacturer, and automobile manufacturer, and evaluation sets of respective enterprises are listed. However, in this example, only the information disseminating items are indicated for convenience. These items are classified into the representation of the common top menu, specific item by industry, and other menus.

[0129] FIG. 20 shows the names of the enterprises belonging to food manufacturer, the structure of the top page (top structure), and the representation of the common top menu. FIG. 21 shows the specific items by industry and other menus of the enterprises. FIG. 22 shows the names of the enterprises belonging to cosmetics manufacturer, the structure of the top page, and the representation of the common top menu. FIG. 23 shows the specific items by industry and other menus of the enterprises. FIG. 24 shows the names of the enterprises belonging to automobile manufacturer, the structure of the top page, and the representation of the common top menu. FIG. 25 shows the specific items by industry and other menus of the enterprises.

[0130] In FIGS. 21, 23, and 25, the description order of specific items by industry and other menus corresponds to the description order of the enterprise names in FIGS. 20, 22, and 24.

[0131] In FIGS. 20, 22, and 24, the representative item names such as new arrivals, release, etc. are added to each of the categories of food manufacturer, cosmetics manufacturer, and automobile manufacturer.

[0132] Furthermore, in FIG. 21, the representative item name such as plant tour, top text message, etc. is added. In FIG. 23, the representative item name such as cosmetology, fashion, etc. is added. In FIG. 25, the representative item name such as plant tour, recall, etc. is added.

[0133] Then, FIG. 26 shows the process performed by the generation unit 22. The generation unit 22 shown in FIG. 26 comprises a category selection unit 111, a partial system selection unit 112, an item addition unit 113, a category generation unit 114, a category selection unit 115, a weight assignment unit 116, an item selection unit 117, and a weight assignment unit 118, receives the refined and adjusted evaluation set group 96, and outputs a weight-assigned specified domain evaluation set 122.

[0134] To select a specified domain evaluation set 121 from the refined and adjusted evaluation set group 96, the functions of specifying a domain, and adding and correcting an item are important. When the user specifies a domain, each category of a systematized evaluation set can be selected, and a set of categories can be selected by specifying a category in a higher layer using the partial system selection unit 112. Each category corresponds to the URL of each Web page, and a set of categories correspond to a set of a plurality of Web pages.

[0135] When one or more categories contained in the lowest layer in the system are selected using the category selection unit 111, all selected categories are specified as a domain. If a higher layer category is specified by the partial system selection unit 112, then the specified category and the lower layer categories enclosed by it are specified as a domain. Therefore, using the category selection unit 111 and the partial system selection unit 112, each of the necessary categories can be picked up or a portion of a systematic structure can be selected, thereby specifying a domain.

[0136] Then, evaluation items corresponding to the selected categories are extracted from the refined and adjusted evaluation set group 96, and the specified domain evaluation set 121 is generated.

[0137] When the evaluation items extracted from the existing Web pages are not sufficient, the user adds an item to and modifies the evaluation items of the specified domain evaluation set 121 using the item addition unit 113 and the category generation unit 114. At this time, the item addition unit 113 adds the item input by the user to each category. Furthermore, the category generation unit 114 deletes a user-specified item from the items in the category, or changes the category name into a user-specified name.

[0138] To process each evaluation item of the specified domain evaluation set 121 with the significance level assigned to each evaluation item, the category weighting process and the item weighting process are important functions. In the category weighting process, the user selects a category to be weight-assigned using the category selection unit 115, and specifies the weight to be assigned using the weight assignment unit 116. In the item weighting process, the user selects an item to be weight-assigned using the item selection unit 117, and specifies a weight to be assigned using the weight assignment unit 118. Thus, a weight-assigned specified domain evaluation set 122 is generated.

[0139] FIGS. 27 through 32 show an example of the weight-assigned specified domain evaluation set 122. In this example, however, it is assumed that all categories of the evaluation set group shown in FIGS. 20 through 25 are specified as a domain.

[0140] In FIGS. 28, 30, and 32, the description order of specific items by industry and other menus corresponds to the description order of enterprise names shown in the corresponding FIGS. 27, 29, and 31.

[0141] In FIGS. 27, 29, and 31, the weight of a category is assigned for each category of an enterprise name, and the weight of an item is assigned for each item of representation of a common top menu. Furthermore, in FIGS. 28, 30, and 32, the weight of an item is assigned for each specific item by industry.

[0142] In the evaluation set group shown in FIGS. 20 through 25, when only a part of the categories are specified as a domain, only the evaluation items of the specified portion are extracted, and the specified domain evaluation set 121 and the weight-assigned specified domain evaluation set 122 are generated. For example, when food manufacturer is specified as a domain, a weight-assigned specified domain evaluation set 122 is configured by the evaluation items shown in FIGS. 27 and 28.

[0143] Then,. FIG. 33 shows the process of the evaluation unit 23. As a pre-process of the evaluation unit 23, an information disseminating item 131 is extracted from the Web page to be evaluated by the cleaning unit 81 and the item extraction unit 82 shown in FIG. 3, and a detailed information item 132 is extracted by the information extraction unit 85 from the Web page to be evaluated. Then, the item fluctuation check unit 86 unifies the names of the items, and arranged items to be evaluated for the Web page to be evaluated are prepared.

[0144] The evaluation unit 23 shown in FIG. 33 comprises an item check unit 141, a hierarchy check unit 142, an extraction contents check unit 143, an extension check unit 144, and an order check unit 145, receives the weight-assigned specified domain evaluation set 122, the information disseminating item 131, and the detailed information item 132, and outputs a Web page evaluation result 151.

[0145] The item check unit 141 and the hierarchy check unit 142 performs the process of an information disseminating item level, and the extraction contents check unit 143 and the extension check unit 144 performs the process of a detailed information item level. Furthermore, the order check unit 145 performs the process relating to both of these items.

[0146] The item check unit 141 checks the existence/non-existence of an information disseminating item, and performs a simple score computation. At this time, it is checked whether or not the contents information corresponding to each information disseminating item in the evaluation set 122 is contained in the item to be evaluated which is extracted from the Web page to be evaluated. The item check value of the existing item is set to 1, and the item check value of the non-existing item is set to 0. If there is a weight assigned to each item, then it is taken into account in computing the score of each item.

[0147] For example, as shown in FIGS. 27 through 32, the score is computed by multiplying the item check value indicating the existence/non-existence of an item by (weight of item/10) when 10 ranks of weights are assigned using the integers of 1through 10. The evaluation score is computed by the following equations.

[0148] FIGS. 34 and 35 show examples of item check values of information disseminating items when the Web page of each enterprise is evaluated using the food manufacturer as a specified domain. FIG. 34 shows the item check values of the representation of a common top menu on the Web page of each enterprise. FIG. 35 shows the item check values of the specific items by industry on the Web page.

[0149] The hierarchy check unit 142 checks the hierarchical position of an information disseminating item on the Web page to be evaluated by performing an URL hierarchical computation, and performs a score computation based on the result. At this time, the appearance layer number of the top page for the Web page to be evaluated is assumed to be 1, and the appearance layer number (depth) of each information disseminating item is computed. Then, the item check value is multiplied by the reciprocal of the appearance layer number to obtain a score of each item. In this case, the evaluation score is computed by the following equations.

[0150] The extraction contents check unit 143 checks the existence/non-existence of a detailed information item, and performs a score computation including an extraction result. At this time, it is checked whether or not the contents information corresponding to each detailed information item in the specified domain evaluation set 122 is contained in the item to be evaluated extracted from the Web page to be evaluated. Then, the evaluation score is computed using the algorithm of the simple score computation by the item check unit 141.

[0151] FIG. 36 shows examples of item check values of detailed information items on the Web page of each enterprise shown in FIG. 34. In this example, an item check value is indicated for each item of enterprise name, capital, etc.

[0152] The extension check unit 144 checks the extension of the position and description of a detailed information item before performing a score computation. At this time, on a Web page to be evaluated, the sequence of a plurality of detailed information items and the separation degree among the items are computed. The separation degree indicates how many other items exist between two items, and is set to 1 when no other items exist. Then, the item check value is multiplied by the reciprocal of the separation degree to compute the score of each item. In this case, the evaluation score is computed by the following equations.

[0153] For example, when a post code and an address are extracted as detailed information items, the evaluation score is high when these items are close to each other, and the evaluation score is low when they are separate from each other.

[0154] The order check unit 145 checks the display order of information disseminating items and detailed information items, and performs a score computation. At this time, it obtains the order (sequence of menu items, etc.) on the Web page to be evaluated of the evaluation items of the specified domain evaluation set 122, compares the order with the reference order, and computes the evaluation score. The reference order is, for example, computed in advance from the items of the Web page of a specified domain.

[0155] If there is a weight assigned to each category in the score computation of each item, then the score computation is performed with the weight taken into account. In this case, for example, the item check value*(weight of item/10) is multiplied by (weight of category/10), thereby computing the score of each item.

[0156] By the above mentioned score computation, the three types of values, that is, the evaluation score at an information disseminating item level, the evaluation score at a detailed information item level, and the evaluation score by an item order, can be computed. An averaging process depending on the number of items can be applied to the evaluation scores at the information disseminating item level and the detailed information item level.

[0157] FIG. 37 shows the process of the result processing unit 24. The result processing unit 24 shown in FIG. 37 comprises a disseminating item evaluation unit 161, an extracted contents evaluation unit 162, an item order evaluation unit 163, an item name evaluation unit 164, an improving process unit 165, an improvement result prediction unit 166, an improving operation support unit 167, and a special information process unit 168, presents an evaluation result to the user using the Web page evaluation result 151 as input, and supports the improving operation of the Web page.

[0158] The disseminating item evaluation unit 161 presents the evaluation score at the information disseminating item level, and the score of each information disseminating item. The extracted contents evaluation unit 162 presents the evaluation score at the detailed information item level, and the score of each detailed information item. The item order evaluation unit 163 presents the evaluation score by the item order.

[0159] Presenting not only the evaluation score but also the score of each item allows the user to easily recognize a low evaluation item, thereby taking improvement action. For example, if the item score is 0, the corresponding item does not exist on the Web page. Therefore, by adding the item to the Web page, the Web page is improved.

[0160] The item name evaluation unit 164 displays the fluctuation in expression of an item name on a Web page, obtains an appropriate representative item name using the ontology, and presents it as a modification candidate. Thus, the user can easily modify the item name to the presented representative item name. Furthermore, the improving process unit 165 starts up an interface for modifying the fluctuation in expression. The activated interface automatically modifies an item name to a representative item name.

[0161] The improvement result prediction unit 166 predicts an improvement result of an item contained in the evaluation result, and presents it. For example, if there is an information disseminating item whose item score is 0, the improvement result prediction unit 166 requests the evaluation unit 23 to perform the score computation with the above mentioned item virtually added, and presents the obtained evaluation result to the user. Furthermore, the improving operation support unit 167 starts up the input interface to support the improving operation. The activated input interface automatically displays an input form, and reflects the change input by the user on the Web page.

[0162] The special information process unit 168 performs an information difference computation, checks whether or not an item other than the evaluation item (difference item) is contained on the Web page. If a difference item is contained, then it is displayed as special information. Thus, the user can recognize that unique information not obtained in a specified domain has been disseminated.

[0163] FIG. 38 shows the process of the construction support unit 25. The construction support unit 25 shown in FIG. 38 comprises a meta-information extraction unit 171, a meta-information presentation unit 172, and an improving process unit 173, and supports the construction of a new Web page using the weight-assigned specified domain evaluation set 122 as input.

[0164] The meta-information extraction unit 171 generates meta-information (tag set, DTD, etc.) in a specified domain from the evaluation items of the specified domain evaluation set 122. The meta-information presentation unit 172 presents the extracted meta-information to the user.

[0165] The improving process unit 173 starts up an input interface, and the activated input interface automatically displays an input form according to the meta-information, and generates a Web page containing the information input by the user.

[0166] The evaluation device shown in FIG. 1 is configured, for example, using the information processing device (computer) as shown in FIG. 39. The information processing device shown in FIG. 39 comprises a CPU (central processing device) 181, memory 182, an input device 183, an output device 184, an external storage device 185, a medium drive device 186, and a network connection device 187. They are interconnected through a bus 188.

[0167] The memory 182 comprises, for example, ROM (read only memory), RAM (random access memory), etc and stores a program and data for use in the process. The CPU 181 performs a necessary process by executing the program using the memory 182. In this case, the learning unit 21, the generation unit 22, the evaluation unit 23, the result processing unit 24, and the construction support unit 25 shown in FIG. 1 correspond to the program stored in the memory 182.

[0168] The input device 183 can be, for example, a keyboard, a pointing device, a touch panel, etc., and are used in inputting an instruction and information from a user. The output device 184 can be, for example, a display, a printer, a speaker, etc., and are used in outputting an inquiry to a user, and a process result (evaluation result, etc.).

[0169] The external storage device 185 can be, for example, a magnetic disk device, an optical disk device, a magneto-optic disk device, a tape device, etc. The information processing device stores the above mentioned program and data in the external storage device 185, and loads to the memory 182 to use them as necessary. The external storage device 185 is also used as a database storing the directory information 33 and the ontology 36 shown in FIG. 2, and a database accumulating the refined and adjusted evaluation set group 96 shown in FIG. 2.

[0170] The medium drive device 186 drives a portable storage medium 189, and accesses the stored contents. The portable storage medium 189 can be an arbitrary computer-readable storage medium such as a memory card, a flexible disk, CD-ROM (compact disk read only memory), an optical disk, a magneto-optic disk, etc. A user stores the above mentioned program and data in the portable storage medium 189, and loads them to the memory 182 and uses them as necessary.

[0171] The network connection device 187 is connected to an arbitrary communications network such as a LAN (local area network), Internet, etc., and converts data during the communications. The information processing device receives the above mentioned program and data through the network connection device 187, loads them to the memory 182, and uses them as necessary.

[0172] FIG. 40 shows computer-readable storage media capable of providing a program and data for the information processing device shown in FIG. 39. The program and data stored in the portable storage medium 189 and a database 191 of a server 190 are loaded to the memory 182. The server 190 generates a propagation signal for propagating the program and data, and transmits them to the information processing device through an arbitrary transmission medium in a network. The CPU 181 executes the program using the data, and performs a necessary process.

[0173] The evaluation of a Web page has been a subjective or conventional evaluation of a viewer. However, according to the present invention, evaluation items for a specified domain realizes a practical and objective evaluation of a Web page.

[0174] Additionally, not only an item set in advance when a Web page is generated, but also an item obtained by a contents analysis can be used as an item to be evaluated. Therefore, a more detailed evaluation can be obtained. Furthermore, by freely selecting a specified domain depending on the URL and systematic structure, evaluation items can be flexibly set, and a weight can be assigned to evaluated contents in some methods, thereby obtaining an appropriate evaluation result.

[0175] Furthermore, various detailed information items accumulated for evaluation of a Web page can be used as basic data for construction of an ontology for each specified domain. The construction of an ontology requires advanced know-how of experts in each field. Therefore, no ontology has been easily generated. Additionally, a large cost is required to maintain the ontology by appropriately understanding the trend which changes with time. However, using the technology of the present invention, these basic operations can be easily realized.

[0176] When a new Web page is generated, the function of supporting a Web page construction process using a detailed information item extracted in a specified domain, not a browsing function or a conventional constructing support, can be realized. It is indispensable to provide the optimum Web page generation interface, but under the present situation with an increasing number of Web pages, a construction support for detailed information items is specifically demanded.