Dynamically generating expanded user messages in a computer system

[0002] The present invention relates the dynamic generation of expanded user messages in a computer system.

[0003] In order to assist developers of computer programs or service personnel to determine the cause of a problem or the reason for unexpected behaviour of a computer system, it is known to generate and store diagnostic information for subsequent analysis. Such diagnostic information typically consists of messages produced in response to events occurring within the computer system.

[0004] These events, which are defined as changes occurring within the system, usually trigger messages, which may report on successful events, or, more typically report to the end user that an unsuccessful event, that is, an error, has occurred. The messages contain information as to why the error has occurred and on how the error can be resolved. The messages can assist the developer of the code or a service organisation in problem analysis and may typically include components such as the following:

[0005] The messages may be written to log files by applications or other operating system programs. These log files provide a history of the events that have occurred, including when the events occurred and causes of the events. The files can be queried to obtain a visual output of the messages or may automatically display the messages to the end user. A typical log file may be the record of system messages to and responses from a systems operator.

[0006] Also known are “trace” files, which are usually produced when the sequence of events during execution of a computer program is tracked. The results of the tracing operation are stored in trace files, which may also be displayed for perusal. The term “log file” as henceforth referred to, also includes trace files.

[0007] However, as can be seen from the above message format, the diagnostic information displayed is often cryptic, unintuitive and may contain inadequate information as to possible cause and suggested solutions. Also, the interpretation or analysis of these messages in log files is potentially complex and prone to error.

[0008] U.S. Pat. No. 5,121,475, issued Jun. 9, 1992 to Garry Child et al, describes a method of dynamically generating more user-friendly messages by lookup in a message table of a pre-existing message matching an error log request and the modification of the pre-existing message by data included in the request. The source of the user-friendly message text, however, is not described.

[0009] Other approaches to the analysis and presentation of diagnostic information are also known in the art. For example, Abend-Aid XLS from Compuware Corporation provides diagnostic reports alongside source code listings.

[0010] Another source of information about the program itself, including a failing portion, is in the form of comments in the program source code. These may contain a great deal more detail than is typically available from error messages. Comments are available to both developers and service organisations as a tool for problem analysis. However, the inspection of comments has to occur manually, since the original source code must be viewed. This is laborious and requires a detailed understanding of the program source code.

[0011] Thus, there is a need for user messages to provide sufficient and accurate information to the end user, without requiring the program source code to be examined or displayed. There is also a need for the user messages to be displayed in a user-friendly format, which may be derived automatically.

[0012] Accordingly, the present invention provides a method for dynamically generating expanded user messages, in a computer system having a logging tool for creating a log file comprising log messages in response to events occurring during execution of an application program, said method comprising the steps of pre-analysing the application program source code to extract a plurality of types of program component including comments, creating a plurality of relationship tables between said different component types, reading an individual log message from said log file, determining said program components in said log message, reading comments associated with said program components from said relationship tables and combining said comments to produce said expanded user messages.

[0013] In other aspects, the present invention provides a system and computer program for generating expanded user messages in accordance with this method.

[0014] Specifically, the diagnostic process is improved since the existing comments present in the program source code, header files or message catalogs are used to supplement the original messages from the log files. This provides the end user with more meaningful information and may allow users to perform a greater proportion of problem determination, with a decreased amount of assistance from the service organisation responsible for the application or operating system code itself. The advantages are a reduced workload for service personnel and less time-consuming problem determination, since a detailed analysis of the program source code by the end user is not required.

[0015] In a further preferred aspect of the present invention, the program components described comprise functions, variables, conditions and comments. Preferably, pre-analysis of the program source code to create the plurality of relationship tables, includes textual parsing. Preferably, the comments obtained are parsed to form natural language text, which is easier for the end user to understand, hence, more rapid problem determination can occur. To obtain the full benefit of the invention, the comments written in the program source code should be structured in a pre-determined manner. In another preferred aspect of the invention, a mouse driven pointer, positioned over the log messages can select said log messages. Alternatively, the selection of further messages could be automated to increase efficiency.

[0016] The relationship tables comprise a function-comment table, a variable-comment table, a condition-comment table and a function-variable table. The relationship tables are stored in binary form and are hashed to enable the expanded user messages to be generated at a greater speed. However, an alternative embodiment could achieve the generation through use of relationship tables stored in a flat text form. The lookup of program components is envisaged to occur using a form of index searching, which is a simple and efficient technique.

[0017] In many cases, a log message arises due to an earlier failure or problem within the system. If the log message did not occur immediately after said failure or problem, such that the two log messages were sequential within the log file, the inter-relationship between the two log messages may not be readily apparent. In an extension of the present invention, further log messages in a log file can also be analysed, in order to determine relationships with the individual log message. Further log messages are read from the log file and its program component types are determined. Any inter-relationships between the individual log message and the further log message are determined and this process is recursively repeated until no further log messages can be selected from the log file. The comments associated with all the inter-related program components, derived from the plurality of relationship tables are sorted by order of execution. These comments are combined to produce expanded user messages.

[0018] With reference to determining inter-relationships, program components of the individual log message are determined by linking the log message to an identifiable function and set of variables. The program components of the individual log message are then compared recursively to a relationship table to extract all inter-related program components, until no further inter-relationships exist. The iterative process results in a set containing the original program component of the individual log message and the derived program component(s) from the individual log message. The program components are stored in sets by type.

[0019] The inter-relationship determination process occurs when a further log message is read. Once its program components have been determined, these are compared to those in the sets derived from the individual log message. An inter-relationship between said individual and said further log messages is established when there is an intersection between program components in the further log message and all the program component sets derived from the individual log message. An indication that an inter-relationship has been derived is displayed to the end user.

[0020] The extended invention is advantageous in that it allows multiple relationships to be displayed to the end user, therefore assisting in determining the cause and resolution for a given problem.

[0021] The present invention will now be described, by way of example only, with reference to preferred embodiments thereof as illustrated in the following drawings:

[0022] FIG. 1 is a block diagram of a computer system for generating expanded user messages, according to the present invention;

[0023] FIG. 2 shows in block diagram form, the operation of a pre-processor, to provide tables employed in the system of FIG. 1;

[0024] FIG. 3 is a flow diagram illustrating a method generating user messages for an individual log message from a log file.

[0025] FIG. 4 is a flow diagram illustrating an extension of the method of FIG. 3, to further log messages.

[0026] For a more complete understanding of the present invention, attention is directed to FIG. 1, in which the principal data structures and functional components of a computer system in which the invention is implemented are shown. Arrowed lines (1,2,3,4,5) illustrate the flow of data between components. In overview, log file viewer (12) permits selection of log message(s) in a log file (10). Comments are retrieved from a plurality of tables (15-18) corresponding to the program components of the selected log message(s) and are refined into a natural language output by a grammatical parser (13).

[0027] Firstly, a log message in log file (10) is selected, wherein the log message is in the format of full text. The message_identifier of the log message, in the message catalog is looked up in the logged message table (11). This determines the program components in said log message, such as functions and variables. Additionally, further inter-related program components can be determined and the associated comments read from other relationship tables.

[0028] These tables are generated by, a pre-processor (21), as shown in block diagram form in FIG. 2. The operation of pre-processor (21) is to extract information from the program source code, header files or message catalogs (20) and order this information into tabular form. The information extracted comprises of:

[0029] i) existing comments from the program source code, header files or message catalogs

[0030] ii) log calls which link a log message_identifier to a function or set of variables

[0031] iii) variable to function relationships;

[0032] The program source code files, header files and message catalogs (20) contain comments which conform to a pre-determined set of guidelines to promote usability in the comment text and allow easier manipulation by the log file viewer (12) for the purpose of creating natural language outputs. These guidelines should not require significant additional effort by developers as most comments are typically written in natural language. The guidelines will aid with inter-relationship determination, in that, rather than keeping comments short or abbreviated, the comments will contain more information and references to program components directly.

[0033] For example, the comment below would not be very useful to the present invention, as it does not contain enough detail:

[0034] However, the comment below would be more useful to both future developers and could also be used by the log file viewer (12):

[0035] Further guidelines would ensure that adjacent comment lines are regarded as being part of the same comment rather than separate comments. For example, the comment below would be treated as a single comment:

[0036] However, the following comments would be treated as two separate comments:

[0037] Once the pre-processor (21) has extracted information from the program source code, header files or message catalogs (20), these items are stored in a plurality of tables (15-18), which facilitate easy lookup of the items, namely:

[0038] i) a function-comment (15) table, which stores comments containing descriptive text about the role that is played by each function within the program source code, header files or message catalogs (20).

[0039] ii) a variable-comment table (16) which stores descriptive text about the role that is played by each variable within the program source code, header files or message catalogs (20).

[0040] iii) a condition-comment (17) table comprises comments describing the condition(s), which give rise to the log message in the logged message table (11). The table (17) stores the inter-relationships linking the condition to the description of the condition.

[0041] Extracting comments from the above tables (15-17) may expose other functions or variables, exemplified by lookup of function-function or variable-variable interactions in the function-variable table (18).

[0042] Referring once again to FIG. 1, thirdly, the resultant set of comments is combined by the log file viewer (12), to derive an expanded user message(s) linking the log messages together. These expanded user messages may then be further grammatically parsed by a parser (13) to produce natural language output which is displayed to the end user in either a window or a pop up menu (14).

[0043] FIG. 3 illustrates in flow diagram form, the process of interpreting and displaying an individual log message from a log file (10) according to a preferred embodiment of the present invention.

[0044] The viewer opens (30) the log file (10), an individual log message is selected (40) by a mouse-driven pointer and is read (50) by the log file viewer (12) from the logged message table (11). The table (11) contains the logging function and its associated parameters. Thus by using the log entry in the table (11), it is possible to link the log message written to the log file (10) to an identifiable function and sets of variables or conditions. Once the program components from the log message have been determined (60) and extracted (70), the comments associated with them are determined (80) from the plurality of relationship tables (15-18). Extracting comments from the relationship tables (15-18) may expose other functions or variables, exemplified by lookup of function-function or variable-variable interactions in the function-variable table (18).

[0045] Comments are read recursively until all outstanding reference to functions, variables and conditions have been resolved. For example, if the program component is determined (90) to be a function, the associated comments are read (100) from the function-comment table (15). This process is repeated for variables and conditions, in steps (110), (120), (130) and (140). If there are still more comments to be read (positive result to step (150)), these are determined (80), otherwise (negative result to step (150)), the resultant set of comments is combined (160) to derive an expanded user message(s) linking the log messages together. The expanded user message(s) may then be further grammatically parsed (170) to produce natural language output, which is displayed (180) to the end user.

[0046] As an example, consider a log message written to a log file (10) arising from the example code fragment in Appendix A:

[0047] The logged message table (11) contains the logging function of the log message above and its associated parameters, as described previously:

[0048] DISK_ERROR_—6 is an identifier that points to a message catalog (20). The message catalog (20) is a mechanism for separating program code from messages, which allows the messages to be translated into another language without requiring the program code to be edited. The corresponding message catalog entry, by way of example, is:

[0049] DISK_ERROR_—6 “function %s failed with params %s,%s, %s. Status is %s”

[0050] When the log_error function is run, the message catalog entry “function %s failed with params %s,%s, %s. Status is %s” is written to the log file (10). The values of the variables, such as scsi0, are inserted at the respective %s locations. When the entry “function %s failed with params %s,%s, %s. Status is %s” is found in the log file (10), the log file viewer (12) searches the message catalog (20) on the system for this entry. This will link the entry “function %s failed with params %s,%s, %s. Status is %s” to the identifier.

[0051] Thus, a relationship can be determined between the entry in the log file and the program components of the original log message, by looking up DISK_ERROR_—6 in the logged message table (11). In this example, looking through the message catalog would find DISK_ERROR_—6, which would then refer back to the specific log_error function.

[0052] Examples of the plurality of relationship tables (15-18) are shown in Appendix B.

[0053] Text associated with the condition log_error (LOGMSG2, connected_where, diskname, disk-adapter, rc, NULL); is read from the condition-comment table (17) and the following sentence is derived:

[0054] Comments associated with the variables giving rise to the log message are extracted from the variable-comment table (16), which returns the following information:

[0055] In addition to the above variables, the text taken from the condition-comment table (17) contains a variable name, disk_adapter. This is subsequently extracted from the variable-comment table (16) and returns the following information:

[0056] Comments associated with the function of the log message are extracted from the function-comment table (15), which gives the following information:

[0057] The function connected_where identifies the location of the disk in the system

[0058] Lookup in the function-variable table (18) shows that connected_where calls the log_error and parent_device functions. The function parent_device in turn calls the syscfg function. Thus it is known, that there is a relationship between the connected_where function and the syscfg function, which generates the disk_adapter variable:

[0059] Lookup returns:

[0060] Similarly parent_device, is looked up in the relationship tables (15-18), returning the information below:

[0061] Since no further functions, variables or conditions exist, the iteration completes and the following set of expanded user messages is provided:

[0062] Additionally, the name of the disk, hdisk3 and the adapter scsi0 are known from the log message. There is much redundancy in the above user messages, therefore after combining these expanded user messages and parsing them with the grammatical parser (13), output similar to the following is displayed to the end user in a window or pop up menu (14):

[0063] FIG. 4 illustrates in flow diagram form, the process of interpreting and displaying further log messages from a log file (10) according to an alternative embodiment of the present invention.

[0064] The viewer opens (200) the log file (10) and a mouse-driven pointer selects (210) the individual log message which is read (220). Lookup of the individual log message in the logged message table (11) provides the lines of code that generated the original log message and the entries in the table (11) also show (230) the function, variables and conditions in use.

[0065] In order to extract (240) the program components in use, a list of other related program components need to be determined. For example for a function (250), a recursive extraction is performed by look up (260) in the function-variable table to extract (270) a set of unique functions that bear some relationship to each other. The original function and the derived functions are added (280) to a function set. This process is repeated for each extracted function in turn until all inter-related functions are extracted (290). A similar extraction process is also applied to variables and the resulting sets may now be used to identify inter-relationships between the individual log message and any subsequently selected one, as illustrated in steps (300) to (340).

[0066] The mouse-driven pointer is subsequently held down and dragged in order to select (345) further log messages above or below said individual log message. A further log message is read (350), and lookup of the further log message occurs in the logged message table (11), which will provide the lines of code that generated the original log message. Additionally, the entries in the table (11) also determine (360) the functions, variables and conditions in use when each log message was written to the log file (10). The program components are extracted (370) from the tables and are compared (380, 390) to the sets of functions and variables derived from the individual log message, to determine whether a common linkage is shared, either through use of functions or variables.

[0067] An inter-relationship is established under the following conditions:

[0068] i) there is an intersection (380) between the function in the further log message and the function set derived from the individual log message and;

[0069] ii) there is an intersection (390) between the variable(s) in the further log message and the variable set(s) derived from the individual log message.

[0070] Only if the above two conditions are both satisfied, is an inter-relationship said to exist between the individual log message and that which the pointer has subsequently selected. An indication that an inter-relationship has been derived is displayed (400) to the end user such as, with a colour change of the log messages or enabling the text of the log messages to be displayed in a larger font. If the above two conditions are not satisfied, an inter-relationship does not exist between log messages.

[0071] The following step checks (410) whether further log messages in the log file are present and if so, these are read (350) and the process repeats itself. If the log file (10) does not contain further log messages, the recursive process completes.

[0072] Once a relationship between two or more log messages is established, the comments associated with these are determined (420) from various relationship tables.For function, variable or condition program components (steps 430, 450, 470) the function-comment table, the variable-comment table and the condition-comment table are read respectively, in steps (440), (460) and (480). If there are any more comments to be read, as determined in step (490), these are determined (420) until all comments have been extracted. Otherwise, the comments are sorted (500) by order of execution and the resultant set of comments is combined to derive an expanded user message(s) linking the log messages together (510). The expanded user message(s) may then be further grammatically parsed (520) to produce natural language output, which is displayed to the end user (530).

[0073] As an example, consider selecting two log messages written to a log file (10) arising from the example code fragment in Appendix A:

[0074] The logged message table (11) contains the logging function of the log messages above and the associated parameters, as described previously:

[0075] As previously described, looking up DISK_ERROR_—6 and DISK_OPERATION_ERROR in the logged message table (11), refers back to the specific log_error function. It is known that there is a relationship between the two log messages in that they share the same variable, diskname.

[0076] Comparison of the function and variables in the further log message against the function and variable sets of the individual log message could similarly determine more complex inter-relationships. With reference to the individual log message, log_error (LOGMSG4, open_disk, diskname, rc, NULL); was generated by the function open_disk, when it was run using with the variables diskname and rc. By lookup in the function-variable table (18) it is known that the open_disk function has some relationship to the functions activate_disk ( ) and open ( ). For each of these extracted functions, similar relationships may be derived. For example, activate_disk has relationships to the functions disk_type( ), connected_where( ) and open_adapter( ). This recursive process is repeated for each extracted function in turn until all inter-related functions are extracted and no further inter-relationships exist. This gives a set of unique functions that have some relationship to the open_disk function.

[0077] Similar recursive extraction is performed on the function-variable table (18) to extract inter-relationships to the variables, diskname and rc. For each variable, this results in a set of unique variables that bear some relationship to each other. These function and variable sets may now be used to identify inter-relationships between the individual log message and any subsequently selected one.

[0078] Each further log message in the log file (10) is read as the mouse-driven pointer selects it and the corresponding line of program code is identified by lookup in the logged message table (11). From this, the program components may be identified as before and are compared against those in the sets derived from the individual log message. If there is both an intersection between the function in the individual log message and the function set and the variable(s) in the individual log message and the variable set(s), then an inter-relationship exists between the individual log message and the further log message. If no intersection is derived or there exists solely an intersection between either functions or variables, a relationship between the log messages does not exist and no additional processing is performed.

[0079] Examples of the relationship tables (15-18) are shown in Appendix B.

[0080] Comment text for the two log messages is read from the condition-comment table (17):

[0081] Comments associated with the variables giving rise to the log message are extracted from the variable-comment table (16):

[0082] In addition to the above variables, the comment text taken from the condition-comment table (17) contains a variable name, disk_adapter. This is also extracted from the variable-comment table (16):

[0083] Comments associated with the functions giving rise to the log messages are extracted:

[0084] As previously described, connected_where generates the disk_adapter variable. Lookup returns:

[0085] Similarly parent_device, is looked up in the relationship tables (15-18), returning the information below:

[0086] Since no further functions, variables or conditions exist, the iteration completes and the following set of expanded user messages is provided:

[0087] Additionally, the name of the disk, hdisk3 is known from the log message. There is much redundancy in the above user messages, therefore after combining these user messages and parsing them with the grammatical parser (13), output similar to the following is.displayed to the end user in a window or pop up menu (14):

[0088] The use of the present invention offers many advantages. For example, since the log messages displayed to the end user are all derived from functions, variables and conditions within the program source code, header files or message catalogs, accurate information is provided. An extension of the present invention allows interpretation of further log messages, which is invaluable to end-users. By allowing analysis of further log messages, dependencies can be determined between log messages, which may assist with the debugging of more complex errors.

[0089] Furthermore, since the process of determining inter-relationships is finite as it is based upon a set number of tables, the search is not exhaustive such as in data mining tools. This allows the process of determining inter-relationships to be much quicker, resulting in a more rapid display of possible cause and solution to the end user. Additionally, in accordance with one aspect of the invention, the display to the end user of log messages occurs in natural language which may aid in increasing the level of understanding which the end user has of the events and may even increase the proportion of self-diagnosis. Since a greater amount of the analysis process can be shifted to the end user, this may even allow resources in the technical support areas to be freed up more quickly.

[0090] Given that many applications primarily use standard header files, once these have been structured to allow comments to be written according to defined guidelines, they become immediately useful to all subsequent applications which make use of the guidelines. The additional effort required in enabling a new application for use with the present invention is little more than that which is required in writing the application initially. Furthermore, modifying existing applications is simply a matter of creating comments and running the program source code through the pre-processor to generate the plurality of relationship tables.

[0091] A further advantage of the present invention is concerned with the aspect of keeping all program-specific content in the plurality of relationship tables, hence enabling the log file viewer to be a generic tool. This allows the present invention to support multiple applications and additionally, by ensuring that a pre-processor generates relationship tables to a consistent format, the log file viewer is capable of supporting applications written in different or multiple programming languages.