Methods and system for browsing large text files

[0002] This patent is a Continuation-In-Part of U.S. patent application Ser. No. 09/567,861, filed May 9, 2000 Pat. No. 6,292,802 which is, in turn, a Divisional of Ser. No. 08/995,676 U.S. Pat. No. 6,055,538, issued Apr. 25, 2000 and filed on Dec. 22, 1997, both of which are hereby incorporated by reference.

[0003] 1. Field of the Invention

[0004] The present invention relates to systems for browsing documents and in particular to methods and systems for using a web browser to quickly search large collections of documents such as arbitrary text documents.

[0005] 2. Discussion of Related Art

[0006] It is common to use a computer to assist a user in browsing through large collections of documents. For example, patent attorneys and patent examiners frequently review large patent documents or collections of related patent or legal documents. Or, for example, computer programmers frequently browse large files of computer source language programs or collections of related source language programs. Computers are applied to assist in such situations to improve, in particular, the speed of searching for symbols or keywords in the collection of documents. Manually searching large collections of documents can be extremely cumbersome and unproductive.

[0007] Text editors or word processors on computer systems are known to allow such browsing by simple sequential paging or scrolling through the documents or by search capabilities to locate particular words or phrases. However, such known techniques typically do not use indexed searching techniques to locate desired search terms in the document(s). Indexed searches are those which use an index to rapidly locate occurrences of a particular symbol or keyword in the text. Rather, simple linear search techniques are most commonly utilized by known text editor or word processing techniques. Such simple linear search techniques are impractical when scaled up to very large collections of documents. Simple, non-indexed search techniques cannot provide adequate performance when used in very large collections of documents.

[0008] For example, a team of programmers may need to rapidly search for related terms or phrases in the collection of source code files which implement an operating system. One such operating system, by way of example, comprises over 14,000 directories including 70,000 files totaling over 40,000,000 lines of source code. Simple, non-indexed search techniques are inadequate for such large collections of files.

[0009] To aid in browsing applications for computer programmers, source code browser programs are often included in program development environments (e.g., in computer aided software engineering (CASE) toolsets). Source code browser programs are usually tightly coupled to the underlying program development package and therefore are only operable in conjunction with the corresponding tools. However, source code browsers do not in general provide browsing service for arbitrary text documents outside the context of the program development tools. Furthermore, they are often constrained by the underlying databases which control the operation of the program development toolset. The databases which contain design information regarding a software development “project” often cannot handle such large collections of files as noted above. Lastly, different source code browser programs each provide a unique user interface potentially forcing a user to learn a proprietary user interface in order to scan collections of documents.

[0010] It is a particular problem to display a large text document and quickly jump to a region of interest as represented, for example, by a line number. Present solutions for text browsing access files in a generally sequential manner from first line through last line. To display a region of interest that is not near the start of the file of text requires that the browser sequence through other lines of text at the start of the file to locate the region of interest to the requesting user. Requiring the text browser to sequence through all lines of text in the file before displaying the user's requested line slows the perceived responsiveness of the system.

[0011] It can be seen from the above discussion that a need exists for a text search capability that is efficient at searches of large collections of documents and in particular rapidly displays the user's region of interest in the text files.

[0012] The present invention solves the above and other problems, thereby advancing the state of the useful arts, by providing a system and associated methods for using a text browsing (viewing) client/server system to improve perceived performance of the system. More specifically, the present invention provides a client process that requests text files from a server process and presents the received text files in a manner intended to improve perceived performance by the user. In a first preferred embodiment, the client process receives requested text from the server process in blocks (also referred to herein as “chunks”) and parses the received chunks of text to identify line numbers of text received. The received, parsed text is stored in local cache memory associated with the client process and indexed to rapidly locate desired line numbers. In a second preferred embodiment, the server process is directed to return chunks of the text file in other than sequential order. In accordance with this second preferred embodiment, the server process returns chunks of the text file in other than sequential order. In particular, the client process provides the server process with one or more desired line numbers. The chunk or chunks containing the desired line numbers are returned first to the client process followed by other remaining chunks of the text file. As used herein the desired line numbers are referred to as “hot lines” and the chunks containing the hot lines are referred to as “hot chunks.” Other lines and chunks of the text file are referred to herein as “normal.”

[0013] A first aspect of the invention provides for a system for displaying large text files comprising: a display for presenting text to a user; a client process responsive to user requests to display selected portions of a text file on the display; and a cache memory associated with the client process for storing data representative of the text file, wherein the client process is operable to parse the text file to identify line numbers associated with the text file and wherein the client process is further operable to store indices in the cache memory identifying the line numbers and corresponding portions of the text file and wherein the client process is further operable to display the selected portions of the text file in accordance with the line numbers.

[0014] In another aspect of the invention, the system further comprises a server process responsive to requests from the client process to retrieve an identified text file and to return chunks of text from the identified text file to the client process. And the client process includes: a graphical user interface thread for interacting with a user of the system and for displaying the selected portions of the text file on the display; a fetcher thread operable substantially in parallel with the parser thread for receiving the chunks of text from the server process and for storing the chunks of text in the cache memory; and a parser thread operable substantially in parallel with the graphical user interface thread for parsing the text file to identify the line numbers. And further, requests from the client process to the server process includes at least one line number of interest to a user of the system, and the server process is operable to return chunks of text that include the at least one line number before other chunks of text that do not include the at least one line number.

[0015] Additional advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The advantages of the invention may be realized and attained by means of instrumentalities and combinations particularly pointed out in the appended claims and depicted in the figures as follows.

[0016] FIG. 1 is a block diagram of a system of the present invention for rapid browsing of large text files.

[0017] FIG. 2 is a flowchart describing operation of a GUI thread of the client process of the system of FIG. 1.

[0018] FIG. 3 is a flowchart describing operation of a fetcher thread of the client process of the system of FIG. 1.

[0019] FIG. 4 is a flowchart describing operation of a parser thread of the client process of the system of FIG. 1.

[0020] FIG. 5 is a flowchart describing operation of a server process of the system of FIG. 1.

[0021] While the invention is susceptible to various modifications and alternative forms, a specific embodiment thereof has been shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that it is not intended to limit the invention to the particular form disclosed, but on the contrary, the invention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined by the appended claims.

[0022] FIG. 1 is a block diagram of a system 1 of the present invention including a client process 100 operable to display text files on a user display 112. The client process 100 retrieves the text file to be displayed from any storage medium. In a preferred exemplary embodiment, client process 100 retrieves a desired text file using distributed computing techniques coupling the client process 100 to a server process 110 via a distributed computing communication medium 150. For example, client process 100 may be operable on a user's PC or workstation and server process 110 may be operable on a remote file server coupled to the user's PC via network communication medium 150. Those skilled in the art will readily recognize that client/server computing paradigms as depicted in FIG. 1 may be utilized in network distributed computing environments as well as other multiprocessing and multiprogramming environments.

[0023] Client process 100 preferably comprises a plurality of threads for performing various aspects of the desired function substantially in parallel. A GUI thread 102 is principally responsible for the overall coordination of retrieval of the requested text file and presentation of the retrieved text file to the user on display 112. Fetcher thread 104 is operable in the depicted client/server computing environment to interact with server process 110 to retrieve chunks of text of the requested text file in parallel with operation of GUI thread 102. Fetcher thread 104 preferably stores retrieved chunks of text of the requested text file in cache memory 108 associated with client process 100. Parser thread 106 is operable in parallel with GUI thread 102 and fetcher thread 104 to parse the retrieved chunks of text stored in cache memory 108 to identify line numbers associated with the text file. Parser thread 106 generates line number index data structures in cache memory 108 to permit rapid access to any portion of the text file in cache memory 108 utilizing line numbers as a standard index. In general, GUI thread 102 utilizes the line number index data structures to rapidly locate a specific line number of the text file requested by the user for presentation on display 112. In accordance with the structure of FIG. 1, the present invention allows for rapid browsing through a large text files in response to a user's request for presentation of a particular line number in the large text file.

[0024] FIG. 2 is a flowchart describing operation of GUI thread 102. As noted above, GUI thread 102 is responsible for overall control of the retrieval, parsing, and presentation of specified portions of a large text file on the user's display 112. GUI thread 102 is preferably designed as an event driven process as is well known in the art. The event driven GUI thread 102 commences processing with element 200 to await receipt of an event message indicating the need to process a particular event. Upon receipt of an event message indicating a user request for presentation of a specified text file, at a specified line number, element 202 is first operable to determine if the requested file is already resident in the cache memory associated with the client process. If so, processing continues with element 210 as discussed further herein below.

[0025] If the requested file is not presently in local cache memory (as indicated for example by presence of a corresponding parsed text object), element 204 is next operable to create a parsed text object in the local cache to serve as a repository for the text file data and associated metadata and indexing structures associated with parsing and presentation of the text file. Those skilled in the art will readily recognize a variety of data structures and program structures to represent such a parsed text object. Element 206 then starts the fetcher thread operable in parallel with the GUI thread to retrieve chunks of text from the text file as provided by the server process. The line number requested by the user is preferably supplied to the fetcher thread to identify an important section or portion of the text file. A line number identified as important by the user is referred to herein as a “hot line.” A chunk of text that contains a hot line is referred to herein as a “hot chunk.” Other chunks of text that do not contain any hot line are referred to herein as “normal chunks.” Element 208 then starts the parser thread operable in parallel with the GUI thread to parse retrieved chunks of text to identify and index line numbers contained in each chunk. Processing of the user generated event message requesting presentation of the text file is then completed and the thread awaits receipt of another event message is indicating the availability of parsed text in the local cache memory.

[0026] In response to receipt of an event message indicating the availability of new parsed text in the local cache memory, element 210 is first operable to get the parsed text object from the local cache memory. Element 212 then inspects the parsed text object to determine if the line number requested by the user (the hot line or lines) has been located in a return chunk of text. If so, element 216 is operable to display the portion of the text file corresponding to the requested line number and surrounding context from the text file to complete the screen display. If the requested line number is determined by element 212 to be not present in the chunks of text parsed in local cache, element 214 is operable to add the requested line number to the a set of hot lines field associated with the parsed test object. As discussed further herein below, the set of hot lines is used by the fetcher and parser threads to assure that the lines the requested by the user are processed as soon as possible in the processing to retrieve the large text file.

[0027] FIG. 3 is a flowchart describing operation of the fetcher thread 104 of FIG. 1. As noted above, fetcher thread 104 is operable in parallel with GUI thread 102 and parser thread 106. Fetcher thread 104 is primarily responsible for interacting with the server process or other storage medium to retrieve chunks of the requested text file. As used herein, a “chunk” refers to any convenient block size typically less than the size of the large text file. Typically, a chunk may be equal to a particular predefined block size standard to a particular operating system on which the invention is implemented. Those skilled in the art will readily recognize that a chunk may be any convenient size useful for a particular application of the present invention.

[0028] Element 300 is first operable to determine if all chunks of the file have been read by the fetcher process. If so, the fetcher process completes processing and awaits restarting by the GUI thread in response to a new user request for new file. If element 300 determines that additional chunks remain to be read from the server process (or other storage medium), element 302 is next operable to read the next chunk provided by the server process. Typically, element 302 will await receipt of the next chunk sent from the server process to the client process. In an alternative embodiment, element 302 may simply read a chunk of text from a local mass storage medium. Element 304 is then operable to determine whether the chunk just read contains a “hot line” (a line identified as of interest by the user). If not, element 306 is operable to add the chunk to a set of normal chunks field associated with the parsed text object and awaiting parsing by the parser thread. If element 304 determines that the chunk does contain a hot line, element 308 adds the chunk to a set of hot chunks associated with the parsed text object in local cache memory and awaiting processing by the parser thread. In both cases, processing continues by looping back to element 300 to determine if further chunks of the text file remain to be read.

[0029] FIG. 4 is a flowchart describing operation of parser thread 106. As noted above, parser thread 106 is operable substantially in parallel with GUI thread 102 and fetcher thread 104. Parser thread 106 is primarily responsible for parsing chunks of text received by fetcher thread 104 to identify line numbers and associated lines of text within the retrieved chunks of text of the text file. A list of line numbers found in the parsing is created as an line number index field associated with the parsed text object in local cache memory.

[0030] Element 400 is first operable to determine if more chunks of text from the text file remain to be processed by the parser thread. If not, the parser thread completes and awaits restarting by the GUI thread in response to a new user request for presentation of another text file or another portion of a text file. If element 400 determines that additional chunks of text remain to be processed, element 402 determines if a “hot chunk” is available in cache memory to the parsed. As noted above, hot chunks are those that contains a hot line number (a line number of interest to the user requesting the presentation of the text file). Also as noted above, the parsed text object includes a data structure or field identifying hot chunks as distinguished from normal chunks. If element 402 determines that no hot chunk is available for parsing in local cache memory, element 404 is operable to parse the next available normal chunk to identify line numbers and associated lines of text found in the normal chunks and to add any located line numbers to the line number index field associated with the parsed text object. If element 402 determines that a hot chunk is available for parsing, element 406 then parses the next available hot chunk to similarly identify lines of text and associated line numbers found in the hot chunk and to add the line numbers to be line number index field associated with the parsed text object. In both cases, element 408 is next operable to generate an event message to notify the GUI thread that additional line numbers have been located and indexed in the parsed text object in local cache memory. Processing then continues by looping back to element 400 until all chunks of text fetched by the fetcher thread for the text file have been processed by the parser thread.

[0031] As discussed above, the line number index field associated with the parsed text object serve as an index to permit the GUI thread to rapidly locate any selected portion of the large text file for presentation to the requesting user. The GUI thread then utilizes the line number index so generated to permit rapid access to any portion of a text file regardless of the size of the text file.

[0032] As described thus far, client process 100 of FIG. 1 may operate in conjunction with a standard sequential storage medium whereby chunks of text are retrieved sequentially from the start of the large text file through the end of the file. The client process as described above improves performance of presenting selected portions of a large text file to a requesting user by building and utilizing a line number index in local cache memory. Further performance improvements can be achieved by enhancing the retrieval of chunks of the text file to retrieve chunks in other than sequential order. In particular, the first chunk retrieved for a large text file is preferably that chunk which contains the requested line number identified by the user—the hot line number. In another preferred embodiment, client process 100 interacts with an enhanced server process to retrieve chunks of text from the text file in other than sequential order. In particular, the client process requests the server process to retrieve the file and identifies the selected portion of the file by providing a hot line number used by the server to select the first chunk to be retrieved and returned to the client process. FIG. 5 is a flowchart describing operation of such an enhanced server further improving performance of the system of the present invention for browsing of large text files.

[0033] Element 500 is operable to await receipt of a client process request for an identified text file with a supplied hot line number. Element 502 then returns a first chunk of text of the requested text file that contains the supplied hot line number—the hot chunk. Element 504 then preferably returns other chunks near the hot chunk. For example, the preceding and following chunks containing text lines adjacent the line in the hot chunk. Providing these neighboring chunks of text helps assure that portions of text near the hot line will be available for rapid display to the requesting user. Lastly, element 506 returns the remainder of chunks of text of the requested file. By returning chunks of text from the requested file in other than sequential order, the present invention with such an enhanced server process further improves perceived performance by the user for retrieval and presentation of a selected portion of an identified text file. The selected portion (the hot chunk) will be returned by the server to the client before other chunks of text of the text file. As the user browses the selected portion of the file, other chunks of text of the requested file will be received and parsed by parallel operating thread's of the client process.

[0034] Those skilled in the art will readily recognize a number of equivalent data structures and methods for implementing the system and methods of the present invention. In particular, those skilled in the art will recognize other functional decompositions of the multi-threaded client process described above. For example, a single threaded program structure may achieve the same function. Further, those skilled in the art will recognize a variety of equivalent data structures and object definitions for implementing the parsed text object and associated fields for representing the text file and line number indices of the present invention. Such design choices are well known to those skilled in the art.

[0035] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description is to be considered as exemplary and not restrictive in character, it being understood that only the preferred embodiment and minor variants thereof have been shown and described and that all changes and modifications that come within the spirit of the invention are desired to be protected.