Automatic Text Grade Level Analyzer for a Text Processing System

1. Technical Field

This invention relates to text processing systems and more particularly to methods in text processing systems for automatically analyzing the grade level of a text document.

2. Background Art

The development trend in the text processing system industry is to produce as many useful functions as possible in text processing systems to aid support personnel and principals in document preparation and to reduce the required repetitive tedious work. However, little, if any, of the development work has been directed to the reader or consumer of the text processing system output. Some effort has been made in the area of reviewing documents for sentence readability by studying sentence length, word length and number of punctuations. By formulating criteria based on these factors to which the text of a document can be compared, the author is given an assessment of the relative sentence structure lucidity. This does not address the word content of the document nor does it consider the capability of the document audience.

It has been discovered that improved text proofing for the purpose of reviewing word content against educational level of the intended audience can be achieved by coupling a specialized dictionary of words including grade level data to a text processing system for automated text review and recomposition to meet a desired grade level. The system includes means for setting a desired maximum grade level for a composition and reviewing the text of the composition against the grade level dictionary. The text words that exceed the set grade level are highlighted on the system display screen and a set of synonyms offered as substitutes for the highlighted word. The system operator can select a desired one of the synonyms for the highlighted word for automatic insertion into the composition, replacing the highlighted word.

FIG. 1 is a block diagram of an interactive text processing system embodying the present invention;

FIG. 2 is a functional diagram of the microprocessor shown in FIG. 1.;

FIG. 3 is a functional diagram illustrating the data flow path between portions of the memory and the microprocessor and the display refresh buffer;

FIG. 4 is a diagrammatic view of the display shown in FIG. 1;

FIG. 5 illustrates the presentation of text on the display screen of FIG. 4 in accordance with the present invention; and

FIG. 6 is an operational flow diagram for controlling the text processing system to perform the function of the present invention.

The invention will now be described as embodied in an interactive text processing system of the type shown in FIG. 1. As shown in FIG. 1, the text processing system illustrated therein comprises a keyboard 10, a microprocessor 11, a display refresh buffer 12, a display device 14, a printer 15, and an auxiliary direct access storage device 16 such as a disk or diskette drive. A clock 17, for keeping the various components of the system in synchronism, is also shown in FIG. 1 and is effectively coupled to each of the units.

Keyboard 10 comprises a normal set of graphic symbol keys such as letters, numbers, punctuation marks, and special character keys, plus text format or control keys including carriage return, tab, index, etc. In addition, the keyboard 10 includes a second set of control keys for issuing special control commands to the system. The control keys include cursor movement keys, keys for setting the keyboard 10 into a number of different modes, etc.

The keyboard 10 is connected to the microprocessor 11 by means of a bus 20. The microprocessor 11, as shown in FIG. 2, comprises an input port 21, an output port 22, a random access memory 23, and a process execution unit 24.

Functionally, memory unit 23 stores both instructions and data in specified sections which will be described in more detail later in the specification. Data is entered into memory 23 from the keyboard 10 as bytes of binary information through input port 21. As shown in FIG. 3, the section of RAM 23 which receives the keystroke data from the keyboard 10 is designated keystroke queue 26. Data to be displayed is transferred by a series of instructions from queue 26 to the text buffer section 27 and then to the display refresh buffer 12 through output port 22 of the microprocessor 11. This is achieved in a conventional way by the microprocessor 11 executing a series of move instructions.

The microprocessor 11 may be an IBM Series 1, INTEL model 8086, or any of the recognized functionally equivalent, currently available microprocessors.

The display refresh buffer 12 is shown as a separate buffer connected between the output port 22 and the display device 14. Buffer 12, in practice, is normally a part of the display device 14 and functions to control the generation of characters on the screen of the display device 14 by exercising on-off control of the beam as it traces a series of horizontal lines across the screen.

The output port 22 also supplies data stored in memory 23 to the printer 15 and disk storage unit 16, each of which may have their own internal buffers which are not shown. Commands to transfer data from the random access memory 23 to the printer 15 or storage unit 16 are sent to the microprocessor 11 by the operator from the keyboard 10.

Printer 15 may be any suitable printer known in the art. In most text processing systems, the printer 15 is basically a standard input/output terminal printer having a type ball element or a daisy-wheel print element.

Disk storage 16 may also be any suitable disk storage device which is capable of storing serial by byte data supplied to it at determined sector address locations, each of which are randomly addressable by the microprocessor 11 to retrieve the data. Spatially related data supplied from disk drive 16 is stored in the display data area 28 of the memory 23 in encoded form. Another section of memory 23 shown in FIG. 3 is the display format buffer area 29 which is involved in the handling of spatially related data in decoded form in accordance with the method of the present invention.

FIG. 4 is a schematic representative of the screen 40 of display device 14. As shown in FIG. 4, the screen 40 has, for example, the capability of displaying 25 lines of characters where each line consists of 80 character column positions. In practice, one character position consists of a matrix of dot positions or picture elements sometimes referred to as pels. A typical character matrix for a display of the type represented by device 14 would be a matrix of six wide by ten high pels, which has been designated by reference character 42 in FIG. 4. The interaction of the refresh buffer 12 and the display 14 is to convert the characters stored at a location in the buffer 12 to the corresponding character as formed in a 6.times.10 dot matrix at the equivalent location on the display screen 40. Display 14 generally is provided with its own set of electronics to achieve that conversion. The microprocessor 11 need only supply the address and load the buffer 12 with the appropriate characters.

The disk storage device 16 also is generally provided with its own set of electronics for converting a byte of data supplied from the display data area 28 of memory 23 through the output port 22 to a serial by bit stream of data to be recorded at a predetermined sector of the one addressed concentric recording track on the disk. Data from the device 16 is supplied to the microprocessor 11 serial by byte from the addressed sector and storage tracks when requested.

It will be understood that all of the above described functions and interactions involving the microprocessor 11 are achieved through suitable programs and data which are also stored in memory 23 shown as memory blocks 30, 31, and 32 of FIG. 2 and which are called into operation in response to data from the keyboard 10 or interrupt signals generated by the various components of the system shown in FIG. 1.

The preferred embodiment of the present invention comprises a set of instructions or programs for controlling the text processing system of FIG. 1 for reviewing the text data on the display screen 40 against the entries in a dictionary list stored in memory 31. The entries in the dictionary memory 31 include text words and an appended code indicating the grade level of the text words. The text words in the dictionary also include appended codes which point to synonysms for the text words in the memory 32. The synonyms are stored and accessed in accordance with the encoding disclosed in co-pending application Ser. No. 218,222, entitled "Binary Vocabulary Matrix Automatic Thesaurus", filed Dec. 19, 1980, by W. S. Rosenbaum, et al, and assigned to the same assignee as the present application and incorporated herein by reference. It is understood that no distinction need be made between dictionary words and synonyms. For example, all words can be stored together and include an appended grade level code and an appended synonym code which points to other synonyms for the word.

The grade level codes assigned to the text words in the dictionary list may be based either on the extensive testing of students or by the analysis of required text books by grade to determine at what grade level a given word has been sufficiently introduced into a student's lexicon to assume that it is known. There are several publications available which also provide grade level information. One such publication is "The Living Word Vocabulary, A National Vocabulary Inventory" by Edgar Dale and Joseph O'Rourke, World Book-Child Craft International, Inc., copyright 1981. This publication provides a list of words in a format similar to that shown in Table 1. The "score" column indicates a statistical percentage of students at a given grade level who will recognize the word in question and know its meaning. In setting up a grade level dictionary list, the score percentage required to place a word in a particular grade level may be arbitrarily set. Applicants have recognized that generally a score of 67% is sufficient when placing a word in a grade level category.

Referring now to FIG. 5, there is shown an example of data on the screen 40 of the display 14. The display has highlighted the words "abandon" and "abashed" at the upper part of the screen. In operation, the grade level analyzer function may be invoked in a text processing system through a conventional "menu" task selection function such as disclosed in U.S. Pat. No. 4,308,582, issued Dec. 29, 1981 to David A. Berger and assigned to the International Business Machines Corp. and incorporated herein by reference.

During the menu task selection function, the text processing system displays to the operator a menu of functions that the system can perform. The operator may then select one or a combination of such functions through operation of cursor control and select or enter keybuttons on the system keyboard 10. For example, functions to paginate, check spelling, hyphenate for formatting, and analyze the grade level of text words in a document can be invoked singularly or in any combination. As an alternative to menu selection, the text processing system could be provided with a separate keybutton to invoke each of the functions that the system is capable of performing.

During the menu setup procedure, the system prompts the operator to enter a number representing the grade level to which the input document is to compared. In the example shown in FIG. 5, the grade level 5 is shown at 42 on the command line 41 of the display 40. The two words "abandon" and "abashed" are merely shown as examples. In actual operation, a text document such as a letter, page of a text book, technical report, or other document would appear on the display screen 40. The grade level analyzer would then scan the document word-by-word comparing each word and the input grade level appearing on the command line to the contents of the grade level dictionary.

FIG. 6 shows a flow diagram for the operation of the grade level analyzer of the present invention. The function is entered at block 50 and a test is performed to determine whether the end of test has been reached at block 51. If the end of text has not been reached, then at block 52 the next word of text is read and compared at block 53 to the contents of the grade level dictionary stored in dictionary memory 31. If the word is not found in the dictionary memory 31, then at block 59 the system is instructed to display a message on message line 45 indicating that the word was not found in the dictionary and processing returns to block 51 to continue reading text. If the word was found in the dictionary then at block 54 the grade level of the word in the dictionary is compared to the input grade level on the command line 5. When the grade level of the word does not exceed the input grade level then processing returns to block 51 to read the next word of text. However if the grade level of the word does exceed the input grade level then at block 55 the system is instructed to highlight the word on the display screen 40. In block 56 the grade level analyzer retrieves the synonyms for the word from the synonym dictionary 32 and displays the list of synonyms along with the part of speech of the word on the display screen underneath the word, for example, as shown in block 43. As previously stated, the synonyms are stored and accessed in accordance with the technique disclosed in co-pending application Ser. No. 218,222, entitled, "Binary Vocabulary Matrix Automatic Thesaurus", filed Dec. 19, 1980. The accessed synonyms are also scanned at block 57 to determine if their grade levels exceed the input grade level. If the grade level of all of the available synonyms exceed the input grade level then at block 58 the system displays a message on message line 45 indicating that no lower grade level word is available and processing returns to block 51. However, if synonyms are available whose grade level meets the input criteria, then all the synonyms are displayed and the synonyms whose grade level exceeds the input grade level are displayed in highlighted form. Alternatively, only those synonyms that do not exceed the input grade level could be displayed. The display of synonyms is executed at block 60 with the highlighting of synonyms occurring in block 61. The operator may then use the keyboard cursor control keys to position the display cursor underneath a synonym which is to be substituted for the highlighted word in the document. If this occurs, and a keystroke to enter the selected synonym is received at block 62, then at block 63 the highlighted text word is replaced with the cursored synonym and processing returns to block 51 to continue. After the last word of the text has been processed, the routine will return to the machine operation which was occurring when the routine was entered at block 64.

A program listing for the grade level analyzer routine is shown in Table 2 in pseudo code. The application and use of pseudo code is described in "An Introduction to Programming", (A Structural Approach Using PL/1 and PL/C-7) Second Edition, by Richard Conway and David Gries, Winthrop Publishers, Cambridge, Mass., copyright 1975.

While the invention has been particularly shown and described with reference to a best mode, it will be understood by those skilled in the art that various changes in form and detail may be made without departing from the spirit and scope of the invention.