System and method for remote performance analysis and optimization of computer systems

[0001] The present invention relates to software. More particularly, the present invention relates to software for remote performance analysis and diagnosis of computer systems.

[0002] Computer programs, which are essentially the sets of instructions that control the operation of a computer to perform tasks, have grown increasingly complex and powerful. While early computer programs were limited to performing only basic mathematical calculations, current computer programs handle complex tasks such as voice and image recognition, predictive analysis and forecasting, multimedia presentation, and other tasks that are too numerous to mention.

[0003] However, one common characteristic of many computer programs is that the programs are typically limited to performing tasks in response to specific commands issued by an operator or user. A user therefore must often know the specific controls and commands required to perform specific tasks. As computer programs become more complex and feature rich, users are called upon to learn and understand more and more about the programs to take advantage of the improved functionality.

[0004] Software developers typically produce a software component in an iterative process from idea conception to prototyping, testing, performance analysis and through to production. The step in this process of analyzing and optimizing performance of a software component often relies on knowledge and skill outside the scope of a typical developer's everyday tasks. In addition, computer products change so often that it is difficult even for knowledgeable developers to keep up-to-date about all the products that they are using to produce their own work. For large software development teams, this general lack of skills can be negated by employing specialist team members for measuring and improving software performance. Computer systems have become so complex that the average software developer does not have the skills or time to perform this vital task in this stage of the development process.

[0005] A definite need exists for a system and method which delivers interactive, semi-automated, comprehensive and dynamic performance analysis tools that give individual developers the collected tuning knowledge for a wide variety of software and hardware products that they would not normally have access to.

[0006] A server has a memory and an analyzer. The memory stores a library of symptom descriptions, a library of corresponding diagnoses, a library of corresponding remedies, and a library of corresponding probes. The analyzer is coupled to the memory and has an identifier, a comparator, and a reiterater. The identifier identifies at least one symptom of an application to be probed based on an input. That input can either be a user input describing the symptoms of the application or symptoms previously already identified. The comparator compares the symptoms of the application with the library of symptom descriptions. The reiterator reiteravely operates the identifier on the comparator until the symptoms correspond with a diagnosis from the library of corresponding diagnoses.

[0007] The accompanying drawings, which are incorporated into and constitute a part of this specification, illustrate one or more embodiments of the present invention and, together with the detailed description, serve to explain the principles and implementations of the invention.

[0008] In the drawings:

[0009] FIG. 1A is a schematic diagram of a network computer system illustrating a process for remote performance analysis in accordance with one embodiment of the present invention.

[0010] FIG. 1B is a table illustrating a library of symptom descriptions, a corresponding library of diagnoses, a corresponding library of remedies, and a corresponding library of probe according to one embodiment of the present invention.

[0011] FIG. 2 is a flow diagram illustrating a server side operation of a process for remote performance analysis in accordance with one embodiment of the present invention.

[0012] FIG. 3 is a flow diagram illustrating a client side operation of a process for remote performance analysis in accordance with one embodiment of the present invention.

[0013] Embodiments of the present invention are described herein in the context of a system and method for remote performance analysis and optimization of computer systems. Those of ordinary skill in the art will realize that the following detailed description of the present invention is illustrative only and is not intended to be in any way limiting. Other embodiments of the present invention will readily suggest themselves to such skilled persons having the benefit of this disclosure. Reference will now be made in detail to implementations of the present invention as illustrated in the accompanying drawings. The same reference indicators will be used throughout the drawings and the following detailed description to refer to the same or like parts.

[0014] In the interest of clarity, not all of the routine features of the implementations described herein are shown and described. It will, of course, be appreciated that in the development of any such actual implementation, numerous implementation-specific decisions must be made in order to achieve the developer's specific goals, such as compliance with application- and business-related constraints, and that these specific goals will vary from one implementation to another and from one developer to another. Moreover, it will be appreciated that such a development effort might be complex and time-consuming, but would nevertheless be a routine undertaking of engineering for those of ordinary skill in the art having the benefit of this disclosure.

[0015] In accordance with the present invention, the components, process steps, and/or data structures may be implemented using various types of operating systems, computing platforms, computer programs, and/or general purpose machines.

[0016] A representative hardware environment 100 suitable for a user 102 is illustrated in FIG. 1A, where a networked computer system 104 communicates with another networked computer system 106 through a network 108. Network 108 represents any type of networked interconnection, including but not limited to local-area, wide-area, wireless, and public networks (e.g., the Internet). Preferably, the computer system 104 comprises a client system and the computer system 106 may comprises a server system.

[0017] In accordance with one embodiment, server system 106 comprises a memory 110 and analysis software 120, also referred to as an analyzer. Memory 110 stores a library of symptom descriptions 112, a corresponding library of diagnoses 114, a corresponding library of remedies 116, and a corresponding library of probes 118. The analysis software 120 comprises an identifier 122, a comparator 124, and a reiterator 126.

[0018] FIG. 1B illustrates an example of the libraries stored in memory 110. The example illustrates symptoms, diagnoses, corresponding remedies, corresponding probes in the context of excessive paging. In column 1, the diagnosis found was excessive paging. Remedies in column 2 include suggestions for monitoring or fixing memory leaks. If no memory leaks were found, the remedy would be a recommendation for a larger system. In column 3, symptoms of memory leaks include high scan rate, high system time, growing memory footprints. The corresponding probes are vmstatProbe and pmapProbe illustrated in column 4. Because memory leaks are generally hard to pinpoint, the library provides suggestions for remedial action in either case (if memory leak is found or not).

[0019] The following is an example of a code written as a korne-shell script for the probe vmstatProbe illustrated in FIG. 1B:

[0020] Vmstat 1 10 (echo :; sed '

[0021] 1 d

[0022] 3 d

[0023] s/{circumflex over ()}t*//g

[0024] s/tt*/:/g')

[0025] According to another embodiment, memory 110 stores information collected from external sources and systems organized in a codified set of, for example, environments and corresponding tests, causes and corresponding effects, situations and corresponding remedy rules. Memory 110 comprises a collection of information codified with an independent platform language, such as, for example, eXtensible Markup Language (XML). XML is a web-friendly markup language that provides platform-independent way of marking up structured data sets. For illustration purposes, the collection of information comprises, for example, rules relating to tests to be performed on a System Under Test (SUT) 128 on client system 104 with the user's hardware and software environment, rules relating to probable causes with corresponding effects measured by client system 104, rules relating to probable remedies with corresponding causes, rules relating to probable solution documentation with corresponding remedies. The previous rules are for illustration purposes and may further comprise other rules appropriate for performance analysis and diagnosis of the client system 104. FIG. 1B illustrates an example of rules relating to probable remedies with corresponding causes.

[0026] The system architecture prescribes that the implementation of memory 110 and analysis software 120 be independent of the platform of client system 104 so that the implementation of the system functionality may be independent from various technologies. In particular, the prescribed interface specification, based on XML documents, for communication between client system 104 and server system 106 where client system 104 remains stable in the face of functional changes on server system 106. One of ordinary skills in the art will recognize that other structural languages may be implemented as an alternative embodiment.

[0027] According to one embodiment, analysis software 120 accesses memory 110, communicates and controls client system 104 through network 108 for data collection, user interaction, symptom identification, and presentation of probable remedies. Analysis software 120 comprises a set of software programs, which uses the information in memory 110 to gather environmental factors from user 102 about the System under Test (SUT) 128. Analysis software 120 also commands client components (Probe(s) 130 and Agent 132) on the SUT 128 to run prescribed performance analysis programs and receives a collection of performance statistics from the prescribed tests. Analysis software 120 then analyzes the returned statistics for possible performance problems based on the codified rules in memory 110. Analysis software 120 optionally commands client components to run further tests in an attempt to narrow the scope of problem isolation and presents a set of probable diagnoses and their related remedy documentation from memory 110. This process may be reiterated until user 102 signifies satisfaction or dissatisfaction with the results or until a diagnosis from the memory 110 corresponds with the symptoms described by the collected data. Analysis software 120 further stores the user's satisfaction reply in memory 110 for further enhancement to the reasoning process, either through manual human intervention or automatically through an Analysis Wizard logic (not shown).

[0028] According to another embodiment, analysis software 120 comprises a software program that calculates probabilities based on the following inputs: environmental conditions on the SUT 128 (collected automatically or interactively by Agent 132), performance data collection from client components, and user feedback on accuracy of previous probability calculations. Analysis software 120 makes calculations by traversing a tree-structure of cause/effect nodes held in memory 110. The logic follows user 102 experience in that it has the following functions: taking input about SUT 128 and determining what probe(s) from the corresponding library of probes 118 to run, receiving output of the selected probe(s) 130 and identifying possible or probable performance deficiencies. Based on the given performance deficiencies, proper documentation and remedy is identified so that user 102 can read to understand and improve their client system 104 or application performance of SUT 128. The functionality of the analysis software 120 is further described in more detail below.

[0029] According to one embodiment, client system 104 comprises software modules: Probe(s) 130 and Agent 132, both interacting with user 102, automatically collecting performance data from client system 104, in particular SUT 128, transferring measured statistics to server system 106, and presenting results and remedy documentation to user 102. The probe(s) 130, also known as performance probes, are software programs that measure specific performance statistics on the SUT 128. Each probe 130 produces output in a standard data format (the Collection) to be used by other system components. Probe(s) 130 executes for a defined time period, until a specific observed event occurs, or until stopped by the user or another program.

[0030] Agent 132, also known as a Collector Agent, is a software program user 102 runs interactively. Agent 132 may be downloaded from server system 106 over network 108. Agent 132 then downloads and installs probe(s) 130 from the corresponding library of probes 118 if the needed probes are not available on client system 104. Probes 130 have knowledge of their version number that can be queried by Agent 132 to determine if the appropriate probe is installed on client system 104. Agent 132 also receives one or more Collection Descriptors 134 from server system 106 specifying what probe names and versions to execute. Agent 132 then executes selected probe(s) 130 automatically or with user interaction on the SUT 128. Agent 132 then filters and formats output statistics returned by probe(s) 130. Collection Descriptors 134 may define a subset of data that is output by probe(s) 130, in which case Agent 132 removes selected data or inserts calculated results to be returned to analysis software 120. Such data output may conform to the API standard defined for communication with server system 106. Agent 132 is also responsible for transferring formatted and raw (i.e. unfilter and unformatted) statistics data (Collection Document 136) to server system 106 over network 108.

[0031] According to another embodiment, for every data collection instance, Agent 132 downloads Collection Descriptors 134 (an XML document containing the details of what probe 130 to run and what information to be filtered from their output). The collected data may be categorized at a high level into one of the five following categories: static (system-wide), dynamic (system-wide), static (application specific), dynamic (application specific), and interactive (dialogue-driven data specified by the user).

[0032] Based on information contained in Collection Descriptors 134, Agent 132 runs the appropriate probe(s) 130 and post-processes the output data. According to one embodiment, before this post-processing, probe(s) 130 preferably generates a two-dimensional grid of output data addressable by cells (row and column) in a format understood by Agent 132this is also termed raw data. The collected data is organized as the Collection Document 136, which is an XML document containing the static configuration (only in the first collection) and a set of samples. Each sample is based on the output of probe(s) 130, propounded by attributes such as row-count, start time, and duration.

[0033] According to one embodiment, the SUT 128 comprises a software application to be probed and analyzed.

[0034] Turning now to FIG. 2, a flow chart illustrating a server side method for remote performance analysis according to a specific embodiment of the present invention is shown. User 102 begins interaction with server system 106, by loading through network 108, an authentication web page (not shown) in a web browser (not shown) on client system 104. Client system 104 comprises SUT 128 and a web browser (not shown). The authentication page (not shown) contains a form (not shown) for the user's login and password. After entering this information, user 102 submits the form to the server system 106 where the user's login is validated and his/her session commences. Server system 106 provides user 102 with a set of tools for managing measurement sessions through a project-based database. Thus, user 102 may utilize server system 106 for multiple SUTs over time. Results are saved and user 102 can interrupt the measurement session to be continued at a later time.

[0035] After login, user 102 is presented with another browser form, an initial page, that user 102 fills in a user input with information describing symptoms of the SUT 128. Such symptom description may include a description software and hardware on the SUT 128. The user input may also include symptoms describing problems on SUT 128. The information entered may also include, but is not limited to: application type (e.g. operating system binary or Java byte-code), identification of application (e.g. binary filename or Java class name), process ID of in-memory executable, locations of software components on disk, and duration of the user's workload. This information may be archived in memory 110. User 102 may also give the problem description a name that can be related to a particular project. At a first block 202, server system 106 receives the user input.

[0036] At 204, Analyzer 120 of server system 106 receives the user input and makes a decision as to what performance tests, if any, need to be carried out on the SUT 128. In particular, Analyzer 120 takes into account all information collected from user 102 and SUT 128 to identify the symptoms of the SUT 128. All collected information may include the initial user input, any additional user input, and data output of selected probes based on the user input(s). The selected probe(s) from the corresponding library of probes 118 are selected based on the collected information. A particular set of symptoms may prompt Analyzer 120 to further probe the SUT 128 for more information to narrow down the corresponding diagnosis and remedy. According to one embodiment, Analyzer 120 comprises Identifier 122, Comparator 124, and Reiterator 126. Identifier 122 identifies symptoms from the collected information on the SUT 128 from the collected information. Comparator 124 compares the collected symptoms of the SUT 128 with the library of symptom descriptions 112 to correspond a set of symptoms in the library 112 with the collected symptoms of the SUT 128. Reiterator 126 reiteratively operates the Identifier 122 on the Comparator 124.

[0037] In decision block 206, if Analyzer 120 determines that it needs to gather more information from SUT 128, it may present user 102 with more questions. If more information is needed from user 102, Analyzer 120 receives additional information in block 208. Upon receipt of the additional user input in block 208, analyzer 120 repeats the data analysis of block 204.

[0038] In decision block 210, Analyzer 120 needs more information probed on SUT 128, one or more probes may be further selected from the corresponding library of probes 118 in block 212 with another Collection Descriptor 134. After the additional probe(s) 130 are executed by Agent 132 on the SUT 128, Analyzer 120 collects output data from the additional probe(s) 130 in block 214. The additional output data is then analyzed in block 204.

[0039] According to one embodiment, client system 104 next downloads an applet, for example, a Java applet (Agent 132) that controls the test software (the selected probes 130). If the required measurement component is not present on client system 104, it is downloaded by Agent 132 from server system 106 and installed on SUT 128. If probes 130 are out-of-date, they are replaced with up-to-date versions from server system 120. Furthermore, Agent 132 may download a collection descriptor 134 containing details of what probes to run and what information to be filtered from their output.

[0040] According to another embodiment, Agent 132 executes selected probe(s) 130 on SUT 128 either to collect static information about the SUT 128 or to collect information about the SUT 128's run-time characteristics. When probes 130 are finished executing, Agent 128 automatically transfers the results as a collection of raw data to Analyzer 120 for analysis.

[0041] According to an alternative embodiment, in block 216, all the collected information from the user inputs and the executed probes may be archived in memory 110 for future reference.

[0042] Once analyzer 120 is able to substantially match the collected information with a set of symptoms from the library of symptoms 112 in memory 110, a corresponding diagnosis from the corresponding library of diagnoses 114 is generated in block 218. A list of diagnoses may detail Analyzer 120 assumptions of probable performance deficiencies and their causes, listed in order of probability.

[0043] According to an alternative embodiment, in block 220, the diagnosis generated in block 220 may be archived in memory 110 for future reference.

[0044] Once the corresponding diagnosis is generated in block 218, a list of remedies from the corresponding library of remedies 116 may be proposed to user 102 at block 222. The remedies may include relevant resource suggestions and feedback collection from user 102 such as technical articles, tuning tips, or code examples. The remedies may also include referring user 102 to another source on the Internet.

[0045] According to an alternative embodiment, user 102 may provide a feedback about the resulting diagnoses and remedies at block 224. If the user feedback is negative, another analysis may be performed at block 204. Therefore, the process may be reiterated until the user 102 signifies satisfaction or dissatisfaction with the results. Such user's satisfaction may be stored in the memory 110 for further enhancement to the reasoning process.

[0046] Turning now to FIG. 3, a flow chart illustrating a client-side operation of a process for remote performance analysis according to a specific embodiment of the present invention is shown. When data collection through selected probes is needed on the SUT 128 of client system 104, Collection Descriptor 134 is generated in block 212 of FIG. 2. In block 302, Collection Descriptor 134 is downloaded to client system 132 through network 108 in a first block 302. In block 304, Agent 132 reads and interprets Collection Descriptor 134 in block 304 to find out which probes it needs to launch. In block 306, Agent 134 launches the selected probe 130 specified in the Collection Descriptor 134. In block 308, after the selected probe 130 is executed on the SUT 128, Agent 132 collects and format raw data generated by the selected probe 130. In decision block 310, if Collection Descriptor 134 specifies more than one selected probe, Agent 132 reiterates blocks 306 and 308 with the remaining selected probes.

[0047] Once all selected probes have been executed, Agent 134 consolidates all the raw data generated by the selected probes into a Collection Document 136 in block 312. Agent 134 then uploads the Collection Document 136 through network 108 to Analyzer 106 of the server system 106 for further analysis.

[0048] While embodiments and applications of this invention have been shown and described, it would be apparent to those skilled in the art having the benefit of this disclosure that many more modifications than mentioned above are possible without departing from the inventive concepts herein. The invention, therefore, is not to be restricted except in the spirit of the appended claims.