System for Underlying Spelling Recovery

This invention relates to grammar checking systems and more particularly to a system for determining the underlying spelling of a word regardless of its capitalization or lack thereof.

As discussed in U.S. Pat. No. 4,868,750 issued to Henry Kucera et al, a colloquial grammar checking system involves automated language analysis via a computer for receiving digitally encoded text composed in a natural language and using a stored dictionary of words and analysis and an analysis program to analyze the encoded text and to identify errors. In particular such a system is utilized in the Microsoft Word program for detecting grammar errors.

One of the most troublesome problems associated with such systems is extremely high error rate when the system suggests a proper usage. The reason for the unreasonably high error rate derives from the system's incorrect analysis of a sentence. Also assuming a correct analysis of a sentence the Microsoft system often suggests an incorrect word.

There is also a class of systems which attempt to analyze a sentence based on the probability that the entire sentence is correct. The largest problem with such systems is that they require storage and processing power beyond the capability of present PCs and related memories.

Other systems attempt to detect incorrect grammar by analyzing sentences based on a training corpus. However, system constraints preclude this type of system from being utilizable in personal computing environments due to the massive storage involved as well as high speed processing required.

By way of example, prior grammar checking systems routinely miss inserting indefinite articles such as "a" and "an", which is indeed a large problem for foreign speaking individuals when trying to translate into the natural language presented by the system.

Also of tremendous importance is the lack of ability to insert the appropriate article such as "the" or "a" when sentences are composed by those not familiar either with the grammar rules or with the colloquial usage of such articles. Moreover, common mistakes made by prior art grammar checking systems include no recognition of incorrect verb sequences in which multiple verbs are used. Although multiple verbs can be used properly in a sentence, most foreign speaking individuals routinely make mistakes such as "He has recognize that something exists." Here "has" is a verb and "recognize" is a verb. As can be seen there is an obvious misusage of multiple verbs.

Most importantly, problems occur in so-called determiners such that for instance the sentence "I have cigarette" obviously is missing the determiner "a". Likewise there are often missing determiners such as "some" or "a few". Thus a proper sentence could have read "I have a few cigarettes". Note that the same sentence could properly be constructed by putting the noun in plural form, e.g. "I have a few cigarettes"; or "I have cigarettes".

An even further typical grammar error not corrected by either spell checkers or prior grammar systems includes the failure to correct improper word inflection. For instance as to improper verb inflections, such systems rarely correct a sentence such as "I drived to the market."

The above problems become paramount when taken from the view of a non-native speaker unfamiliar both with the idiom and the rules of the language. Especially with English, the rules are not as straightforward as one would like, with the correct "grammar" often determined by idiom or rules which are not familiar to those native speakers utilizing the language.

It is therefore important to provide a grammar checking system which takes into account the most frequent errors made by non-native speakers of a particular nationality. Thus for instance there is a body of errors normally made by Japanese native speakers which are translated into English in ways which are predictable and therefore correctable. Likewise for instance for French or any of the Romance languages, there are certain characteristic errors made when translating into English which can be detected and corrected.

Syntax recognizing systems have in general been limited to operating on text having a small, well-defined vocabulary, or to operating on more general text but dealing with a limited range of syntactic features. Extensions of either vocabulary or syntactic range require increasingly complex structures and an increasing number of special recognition rules, which make a system too large or unwieldy for commercial implementation on commonly available computing systems.

Another popular system for detecting and correcting contextual errors in a text processing system is described U.S. Pat. No. 4,674,065 issued to Frederick B. Lang et al, in which a system for proofreading a document for word use validation and text processing is accomplished by coupling a specialized dictionary of sets of homophones and confusable words to sets of di-gram and n-gram conditions from which proper usage of the words can be statistically determined. As mentioned before, doing statistics on words as opposed to parts of speech requires an exceptionally large training corpus and high speed computation, making the system somewhat unwieldy for personal computing applications. Moreover, this system, while detecting confusable words in terms of like-sounding words, is not sufficient to provide correction for those words which are confused in general usage but which do not sound alike.

Finally, U.S. Pat. No. 4,830,521 is a patent relating to an electronic typewriter with a spell checking function and proper noun recognition. It will be appreciated that the problem with noun recognition revolves around a capitalization scenario which may or may not be accurate in the recognition of a proper noun. Most importantly this patent tests words only to find if they are the first word in a sentence to determine the function of the capitalization, whereas capitalization can obviously occur for words anywhere in the sentence.

By way of further background numerous patents attack the grammar problem first through the use of spelling correction. Such patents include U.S. Pat. Nos. 5,218,536; 5,215,388; 5,203,705; 5,161,245; 5,148,367; 4,995,740; 4,980,855; 4,915,546; 4,912,671; 4,903,206; 4,887,920; 4,887,212; 4,873,634; 4,862,408; 4,852,003; 4,842,428; 4,829,472; 4,799,191; 4,799,188; 4,797,855; and 4,689,768.

There are also a number of patents dealing with text analysis such as U.S. Pat. Nos.5,224,038; 5,220,503; 5,200,893; 5,164,899; 5,111,389; 5,029,085; 5,083,268; 5,068,789; 5,007,019; 4,994,966; 4,974,195; 4,958,285; 4,933,896; 4,914,590; 4,816,994; and 4,773,009. It will be appreciated that all of these patents relate to systems that cannot be practically implemented for the purpose of checking grammar to the levels required especially by those non-native speakers who are forced to provide written documents in a given natural language. It will also be appreciated that these patents relate to general systems which are not specifically directed to correcting grammar and English usage for non-native speakers.

Finally there exists a number of patents which relate to how efficiently one can encode a dictionary, these patents being U.S. Pat. Nos. 5,189,610; 5,060,154; 4,959,785; and 4,782,464. It will be appreciated that encoding a dictionary is but one step in formulating a system which can adequately check grammar.

Of particular importance of the correction of grammar is the ability of unambiguously determine what word exists in a sentence. Words which are capitalized may either be proper nouns, ordinary words at the beginning of the sentence, acronyms, or titles. In the past, spell cheking systems have not been able to reliably distinguishing the underlying spelling of a word, much less utilize the underlying spelling both for spell checking and for inflection checking.

In order to solve the spell checking and the inflection checking problems, in the Subject Invention, the underlying spelling is determined after the parts of speech of the input sentence have been accurately determined. Underlying spelling is determined based on the comparison of two sentences in which the word in question appears in both versions, capitalized and uncapitalized. The sentence with the higher trigram probability is the one selected, such that the word as used in the selected sentence is deemed to be correct. The trigram probability refers to the probability that a sequence of three part of speech tags assigned to a sequence of three words is correct.

In summary, in a grammar checking system which includes first tagging a sentence as to parts of speech, underlying spelling is recovered by removing the effects of capitilazation of a word so that appropriate inflection and or spelling can be suggested by the system. In order to determine the underlying spelling the system determines whether a noun is a proper noun through the utilization of a part of speech tagger and the utilization of part of speech trigram probabilities, with capitalized and uncapitalized versions of the word having different trigram probabilities. The system also establishes whether a word is an ordinary word as opposed to a proper or other intrinsically capitalized word. With the system further determining which of two interpretations of the word is the best one.

These and other features of the Subject Invention will be better understood taken into conjunction with the Detailed Description in conjunction with the Drawings of which:

FIG. 1 is a block diagram of a complete grammar checking system illustrating the various modules utilized for complete grammar checking;

FIG. 2A is a block diagram illustrating the correction of easily confused words utilizing the probability of part of speech sequences;

FIG. 2B is a block diagram illustrating the use of sentence length averaging for determination of a probabilistic threshold for the correction of easily confused words for the probability determination component of FIG. 2A;

FIG. 3 is a flow chart illustrating the correction of the definite articles "a" and "an" utilizing specialized tables of English exceptions based on the way that words are pronounced as opposed as the way that words are spelled to derive the proper usage of the article;

FIG. 4A is a block diagram illustrating the correction of incorrect auxiliary verb sequences through the utilization of a starting point and ending point detector to achieve a corrected sentence;

FIG. 4B is a block diagram of the construction of the starting point and ending point detectors of FIG. 4A utilizing a directed acyclic graph representing correct verb sequences;

FIG. 4C is a directed acyclic graph representing the set of correct auxiliary verb sequences of the English language;

FIG. 4D is a directed acyclic graph representing a finite state transducer for proposing corrections for an incorrect auxiliary-verb sequence;

FIG. 5 is a block diagram illustrating an improved spell checking system for non-native speakers in which a list of incorrect words corresponding to a non-real English word dictionary is computed based on normal rules of English word formation applied systematically to all English words;

FIG. 6 is a block diagram illustrating the process of correct detected incorrect words utilizing the incorrect word dictionary derived from the system of FIG. 5 as well as an English word dictionary in which root and morphological features are utilized in the analysis of the detected incorrect word with respect to a list of previously generated incorrect English words;

FIG. 7 is a flow chart for use in detecting and correcting the improper use of determiners in which noun phrases are identified by maximally matching a pattern that defines which sequences of part of speech tag constitute valid noun phrases including a check to ascertain a missing determiner, an extraneous determiner, or number disagreement;

FIG. 8 is a flow chart illustrating the determination for a noun phrase of whether it is a title, if it contains a determiner, if it is a mass title noun, a mass noun, or a part of an idiom to permit reporting a missing determiner;

FIG. 9 is a flow chart illustrating the checking of a noun phrase for an extraneous determiner through the determination of whether or not the head noun is a proper noun and then ascertaining whether or not the noun phrase contains a determiner thereby to report an extraneous determiner;

FIG. 10 is a flow chart illustrating the checking of a noun phrase for number disagreement in which the determination is accomplished by detecting whether the head noun is a proper noun, or if not the presence of a determiner and whether or not the determiner agrees in number with the head noun thereby to report disagreement;

FIG. 11 is a block diagram illustrating a system for the recognition of proper nouns and other intrinsically capitalized word to recover underlying spelling of the word, in which a preprocessing module is utilized to ascertain whether or not a noun is a proper noun utilizing a training corpus revised to uncapitalized words that are not proper nouns or are not intrinsically capitalized, with a trigram model trained on the revised corpus;

FIG. 12 is a flow chart illustrating the tagging of the training corpus of FIG. 11 which is utilized to obtain the next word/tag pair, to see if it is capitalized and if the word is the first word of a sentence or it follows an open quote or colon, also testing to see if the word has been tagged as a proper noun or title, or if it is an acronym or the pronoun "T" thereby to ascertain if the word is uncapitalized;

FIG. 13 is a flow chart illustrating the decision making process for determining whether the word is intrinsically capitalized as illustrated in FIG. 11 by analyzing whether the word is capitalized, if it is the first word of a sentence or follows an open quote or colon, if the word is an acronym, and if not the probability of the sentence with the word uncapitalized is determined to see if it exceeds the probability of the sentence with the word capitalized; and,

FIG. 14 is a block diagram illustrating dictionary access based on context in which both a part of speech tagger and a morphological analyzer is utilized to determine which entries in the dictionary correspond to the word as it is used in context, and which entries in the dictionary do not correspond to the word as it is used in context.

While the Subject Invention relates to the recovery of underlying spelling, what is now described is a total grammar checking system in which various modules rely on the part of speech probability.

Referring now to FIG. 1, especially for foreign language spelling individuals, it is important to provide instant grammar checking for inputted sentences which is both accurate and easily used even for those not particularly computer literate. In order to accomplish grammar checking, an input sentence 10, is entered by a keyboard 12 into the CPU 14 of a word processing system 16.

It is important for reliable grammar verification that the parts of speech of the input sentence be accurately determined. While prior grammar checking systems have utilized the input sentence directly, it is a feature of the subject invention that the input sentence be broken down into parts of speech so as to provide a part-of-speech sequence. This is accomplished by part-of-speech analyzer 20 which is available as an implementation of Kenneth Church's Stochastic Parts Program published as "A Stochastic Parts Program and Noun Phrase Parser for Unrestricted Text" in the Proceedings of the Second Conference on Applied Natural Language Processing, Austin Tex., 1988. The result of having derived parts of speech is a part of speech sequence such as "PRONOUN, VERB, DETERMINER, NOUN, VERB" for an input sentence "I heard this band play".

Having merely derived parts of speech does not reliably assure that the derived parts of speech reflect a proper sentence.

In order to analyze and construct proper sentences, it is important to ascertain the probability that the part of speech sequence corresponds to a correct word sequence. In order to derive the probability of a sequence of an input sentence, the output of analyzer or tagger 20 is coupled to a part of speech sequence probability determination unit 22. The output of this unit is utilized by various modules in the analysis of the input sentence 10.

The first module is a part of speech verification unit 24 which selects between a set of easily confused words or sentences based on the probability of the corresponding parts of speech sequence. Selection of the correct word or sentence is determined, in one embodiment, by the probability exceeding a predetermined threshold. The selection of the correct sentence is accomplished by a unit 26, the inputs to which are of the probabilities of the various sentences as well as the input sentence. As will be described here and after, unit 26 is provided with a list of easily confused words.

While easily confused sentences may be corrected in the above fashion, a further module 28 is utilized to determine the underlying spelling of a word. While conventional spell checkers utilize lookup tables for spelling verification, they do not take into account capitalization which can result in annoying indications of spelling errors. Moreover, those grammar checking systems which rely on proper spelling are often mislead by capitalized words either at the beginning of a word, sentence, or acronym.

In order to provide more reliable spell checking and grammar correction, an underlying spelling recovering unit 28 treats capitalized words as "confused" words. In so doing, the above technique is used to provide the probability of a capitalized word being in one category or another based on a training corpus such as Brown's corpus.

Thus while traditional language processing systems have recovered the underlying spelling of a word by imposing the restriction that a word be either an ordinary noun or proper noun but not both, the subject recovery unit utilizes context and probabilities to categorize each word. This is accomplished by analyzing the sentence with the word in capitalized and uncapitalized form to ascertain which one has the higher probability. Thereafter, the word analyzed for spelling is that form of the word in the higher probability sentence. Having recovered the most likely spelling, the output of the recovery unit 28 is coupled to a inflection checking and corresting system 30. This spelling corrector may be either the conventional spell check variety or one tuned for a particular foreign speaking individual.

As an additional module, an auxiliary-verb correction unit 32 also requires correct parts of speech derived from part of speech sequence probability unit 22. An auxiliary-verb correction problem exists when there are multiple verbs in a sentence some of which are improper. This can occur in complex auxiliary-verb sequences when incorrect tenses are utilized. For instance, the sentence "he would living" involves the two verb "would" and "living". One correct form of the sentence would be "he would live". Thus the tense of the verb "live" is required to be corrected.

In order to accomplish this, auxiliary-verb correction unit 32 detects any incorrect auxiliary-verb sequence and then proposes corrections. This is accomplished first by utilizing a directed acyclic graph which describes a finite set of verb sequences. It would be appreciated that prior to establishing correct verb sequences it is important to correctly identify correct parts of speech which is accomplished by unit 22 as noted above.

The output of auxiliary-verb correction unit 32 is coupled to a correct sentence selection unit 34 for suggesting appropriate alternative sentences.

An additional module utilizing parts of speech is a determiner correction unit 36. It is the purpose of this unit to correct for those words that determine the referent of a noun phrase. Examples of determiners are words such as "the", "a", and "some". There are three classes of errors detected and corrected by this unit, namely, missing determiners, extraneous determiners, and lack of agreement between the determiner and a noun.

Examples of a missing determiner is "John read book" in which "the" is left out. An example of an extraneous determiner is "John went to the New York" with "the" to be deleted. Lack of agreement is evident in the sentence "John read many book" where the noun "book" must be pluralized to agree with the determiner "many". In order to detect an improper determiner, parts of speech tags identified so as to be able to identify noun phrases. The system identifies noun phrases by maximally matching a regular expression that defines which sequences of part of speech tags constitute valid noun phrases.

The system then tests each norm phrase to see if it is missing a determiner. As part of this process, a head noun is first detected followed by determination of whether this head noun is a mass nouns, mass title nouns, idiom or is missing a determiner. The system then tests each noun phrase to see if it has an extraneous determiner. Finally the system test whether the determiner and head noun of the noun phrase agree in number. The result is either the insertion, deleting or replacement of a word as illustrated at 38.

In addition, module 42 corrects the usage of the indefinite articles "a" and "an" based on input sentence 10.

Finally the accuracy provided by the part of speech sequence is useful in a context-sensitive dictionary lookup 40. Typically a given word can have out of context many parts of speech, each one of them corresponding to sub entries in a dictionary. The context-sensitive dictionary lookup module 40 accesses a dictionary and selects the appropriate definitions based on the part of speech of the word obtained by the part-of-speech module 20. For example, the word "love" can be a noun or a verb, and the noun "love" has many different entries in a dictionary, as for the verb "love". Assuming that the input sentence is "She was my first love", the word "love" is identified as a noun by the part-of-speech module, and the context-sensitive dictionary lookup module only selects the entries of the dictionary for the noun "love" and those for the verb "love".

It will be appreciated that once the underlying spelling of a word has been recovered by module 28 not only can this underlying spelling be utilized for inflection correction by module 30, it can also be utilized in a conventional spelling system 44. Thus conventional spell checking system can be made to overlook acronyms during the spell checking process rather than presenting an incorrect array of suggestions.

a) Grammar Correction Based on Part of Speech Probabilities

In the past, several of the aforementioned grammar checking systems have attempted to correct English usage by correcting improper use of some troublesome words, especially those in those identical sounding words are spelled differently. For example: "too", "to" and "two"; "their", "they're" and "there". Other common mistakes revolve around whether a word should be one word or two words such as "maybe" and "may be". There are also words which do not sounds alike but that are often misused such as which and whose.

In the past, in order to ascertain proper usage, the grammaticality of a sentence was computed as the probability of this sentence to occur in English. Such statistical approach assigns high probability to grammatically correct sentences, and low probability to ungrammatical sentences. The statistical is obtained by training on a collection of English sentences, or a training corpus. The corpus defines correct usage. As a result, when a sentence is typed in to such a grammar checking system, the probability of the entire sentence correlating with the corpus is computed. It will be appreciated in order to entertain the entire English vocabulary about 60,000 words, a corpus of at several hundred trillion words must be used. Furthermore, a comparable number of probabilities must be stored on the computer. Thus the task of analyzing entire sentences is both computationally and storage intensive.

In order to establish correct usage in the Subject System, it is the probability of a sequence of parts of speech which is derived. For this purpose, one can consider that there are between 100 and 400 possible parts of speech depending how sophisticated the system is to be. This translates to a several million word training corpus as opposed to several hundred trillion. This type of analysis can be easily performed on standard computing platforms including the ones used for word processing.

Thus in the subject system, a sentence is first broken up into parts of speech. For instance, the sentence "I heard this band play" is analyzed as follows: PRONOUN, VERB, DETERMINER, NOUN, VERB. The probability of this part of speech sequence, is determined by comparing the sequence to the corpus. This is also not feasible unless one merely consider the so-called tri-grams. Tri-grams are triple of parts of speech which are adjacent in the input sentence. Analyzing three adjacent parts of speech is usually sufficient to establish correctness; and it the probability of these tri-grams which is utilized to establish that a particular sentence involves correct usage. Thus rather than checking the entire sentence, the probability of three adjacent parts of speech is computed from the training corpus.

Assuming two sentences, one which is confused with the other, it is possible with the above technique to determine which would be the correct usage. Since the above system can determine this with a low error rate, there are two benefits. The first benefit is obviously ascertaining which of the two sentences is correct. The second benefit, is that having established a correct sentence, its parts of speech can be used by other grammar checking modules for further processing.

Referring now to FIG. 2A, an input sentence S1 as indicated at 31, is coupled to a part-of-speech tagger 32 and also a candidate sentence S2 as illustrated in 34 which is provided with an input comprising a list of confused words 36. Tagger 32 brakes up sentence S1 into the most likely part of speech sequence T1 and its probability P1 as can be seen at 38. This is accomplished by an algorithm such as that described by Church in which the most likely part of speech sequence is obtained by computing the most likely product of probabilities of all possible overlapping triples of parts of speech. One algorithm for accomplishing this task is provided here in as Appendix A.

The words in the sentence S1 may be part of a list of easily confused words 36, in which case, all possible alternative sentences S2 to the sentence S1 are generated according to list 36. The output of sentence generator 34 is applied to tagger 32 to produce the most likely part-of-speech sequence T2 as shown at 40 and its probability P2, again by the algorithm of the Appendix A.

Having derived the probabilities P1 and P2 of the sentences S1 and S2 at 38 and 40, it is now important to determine which part of speech sequence is the most likely to be correct. In order to determine the appropriate sentence to be selected, and as shown at 42, P2 is compared to P1 and if P2-P1 is greater than some threshold e, than as illustrated in 44 sentence S2 is suggested. If P2-P1.rarw.e then no change is suggested as illustrated at 46.

For example, assuming the input sentence is "I want to here this band" where "here"is misused instead of the correct word "hear", one needs to compare the two sentences S1: "I want to here this band" and S2: "I want to hear this band".

In order to compare those two sentences, one can try to compare the overall probabilities of the sentences given some statistical model of English text. This approach, explored in an article by Eric Mays, Fred Damereau and Robert Mercer entitled "Context Based Spelling Correction" published in "Information Processing and Management", 27 (5) :517-422, 1991, is computationally extremely expensive and therefore impractical on standard computers when dealing with unrestricted text which requires vocabulary of more than 40,000 words. Being able to directly compute the sentence probabilities requires tremendous amounts of training data, e.g. a minimum of 400,000,000 training words, and tremendous amounts of storage space.

In contrast, the subject system as illustrated in FIG. 2A compares the probability of the most likely part-of-speech sequence for the given input sentence and the possible sentence with which it is likely to be confused. For example, instead of computing the probability of sentence "I want to here this band", the system derives the most likely part of speech sequence, e.g. "PRONOUN VERB TO ADVERB DETERMINER NOUN" for that sentence and computes the probability of this part of speech sequence for the input sentence. Similarly the system derives the most likely part of speech sequence for "I want to hear this band", e.g. "PRONOUN VERB TO VERB DETERMINER NOUN", and computes its probability for the related sentence. Then, the subject system decides between the usage of "here" and "hereby" comparing the probabilities.

Rather than comparing the above mentioned probabilities, in a preferred embodiment, the subject system compares the geometric average of these probabilities by taking into account their word lengths, i.e. by comparing the logarithm of P1 divided by the number of words in S1, and the logarithm of P2 divided by the number of words in S2. This is important in cases where a single word may be confused with a sequence of words such as "maybe" and "may be". Directly comparing the probabilities of the part of speech sequences would favor shorter sentences instead of longer sentences, an not necessarily correct result, since the statistical language model assigns lower probabilities to longer sentences. The above is illustrated in FIG. 2B.

The list of confused words 36 typically includes the following sets: to, too, two; I, me; its, it's; their, they're, there; whose, which; then, than; whose, who's; our, are; hear, here; past, passed; accept, except; advice, advise; lose, loose; write, right; your, you're; affect, effect and maybe, may be.

Note that the subject system is applicable to other confused words and other languages such as French, Italian and Spanish among others. Note that the method is general, in so far as part-of-speech tagging can be performed using the method described in Church, namely the trigram model.

In summary, the system of FIGS. 2A and 2B in addition to selecting more probably correct sentences is important in ascertaining other judgments about the grammaticality of sentences. The above provides a better and more reliable modality for breaking up sentences into parts of speech.

In order to correct sentences, it is first important to be able to break the sentence down into parts of speech. How accurately a grammar checker can operate depends critically on the accuracy of this break down. By providing more reliable part of speech generation, the end result for grammar checking can be made that much more reliable.

b) Correction of "a" vs. "an"

It will be appreciated that one of the most frequently occurring mistakes for foreign speaking individuals is the correct usage of the indefinite articles "a" and "an". The rules of English specify that the indefinite "a" should be used before words that are pronounced with an initial consonant and "an" should be used before words that are pronounced with an initial vowel. A naive and incorrect implementation of these rules of English test whether the first letter of the next word is a vowel or a consonant. Although it is the case that most words that are pronounced with an initial consonant (resp. vowel) are actually spelled with an initial consonant (resp. vowel), it is not always the case as in the following examples: an hour; a European. For example, the word "hour" has an initial consonant (h) but is pronounced with an initial sound corresponding to a vowel (e.g. ow). Similarly, the word "European" starts with an initial vowel (the letter "E") but is pronounced with an initial sound corresponding to a consonant (e.g. "ye").

Previous solutions to this problem consist in storing a dictionary of the pronunciation of all English words. These solutions are correct but require massive amount of storage for all words in the English language.

Rather than utilizing a dictionary lookup table for all words in the English language, the subject system applies simple rules when no exception to the rules is found. The exception to the rules are stored in two small tables corresponding respectively to the words not handled by the rules that start with a vowel but are initially pronounced with a consonant, and to the words not handled by the rules that start with a consonant but are initially pronounced with a vowel. The lookup tables for these words contain less than 300 words as opposed to a generalized dictionary based system of 60,000 words. The tables below list of the words for which there are English exceptions.

From the above it will be appreciated that a portion of the subject invention revolves around recognition that it is the initial sounds which are uttered when pronouncing a word that is important in determining the correct use of the indefinite article.

Having first established a limited list of exceptions, the following three rules are applied. The first rule applies when the word following the indefinite articles "a" or "an" starts with the characters "eu". In which case, the indefinite "a" should be used. The second rule applies when the word following the indefinite articles "a" or "an" starts with vowel character, "a", "e", "i", "o" or "u". In which case, the indefinite article "an" should be used. The third rule applies when the word following the indefinite articles "a" or "an" starts with consonant character. In which case, the indefinite article "a" should be used.

Referring to FIG. 3, each word w1 of an input sentence 300 and the word following it w2 are established by keeping track of the position of the current word i in the input sentence as determined by blocks 302, 304, 306. If the current word is not "a" or "an" as established by 308, the algorithm goes to the next word through blocks 404, 306. If the current word w1 is either "a" or "an", and the next word w2 is found in Table 1 as established by block 310 then the current word w1 must be corrected to "a" if needed as specified by 312. If the next word w2 is not found in Table 1 but is found in Table 2 as established by block 314, then, the current word w1 must be corrected to "an" if needed as specified by 316. Otherwise, if the next word starts with the letters "eu" as established by block 318, then, the current word w1 must be corrected to "a", if needed as specified by 320. Otherwise if the next word w2 starts with "a", "e", "i", "o" or "u" as established by block 322, then, the current word w1 must be corrected to "an" if needed as specified by 324. Otherwise, the current word w1 must be corrected to "a" if needed as specified by 326.

c) Correction of Incorrect Auxiliary-Verb Sequences

As mentioned here and before when non-native speakers try to write English text they often use an incorrect tense in a complex auxiliary-verb sequence. An example is "he has consider". Here the incorrect usage is the tense of the verb "consider". None of the current grammar checking systems check for auxiliary-verb sequences due to the apparent difficult in recognizing such sequences and also due to the fact that part of speech tags are usually not computed.

In the subject system, and referring now to FIG. 4, a sentence 410 is analyzed by a part of speech tagger 412. to derive the parts of speech of the sentence involve as illustrated at 414.

In order to detect the error, one has to detect both the ending point and the beginning point of the incorrect auxiliary-verb sequence. For instance, in the sentence "He has been consider this fact" it is important to detect the end of the error namely "consider" which is the fourth word in the sentence. All words after "consider" namely "this fact" do not affect the correctness of the auxiliary-verb sequence. Likewise, it is important to detect the starting point of the error namely "has" which is the second word in the sentence. All words before "has" are irrelevant to the determination of the correctness of the auxiliary-verb sequence.

Having generated the parts of speech of the sentence, an ending point detecting 416 is utilized to compute the end position of the incorrect auxiliary-verb sequence. In order to detect the end of the incorrect verb-sequence and as can be seen in FIG. 4B as indicated at 420, all correct part of speech sequences of all auxiliary-verb sequences are stored in a directed acyclic graph shown in FIG. 4C to be described hereinafter.

From the directed acyclic graph of all correct auxiliary-verb sequences, another directed acyclic graph corresponding to all possible incorrect auxiliary-verb sequences is generated at 422. Having the graph corresponding to 422, this graph will contain the incorrect auxiliary-verb sequence "have-3rd-person verb-infinitive". This corresponds to the incorrect auxiliary-verb sequence "has consider". In order to detect the ending point of the error, the graph is traversed from left to right until an end state is reached while the input string is read from left to right. Since the parts of speech correspond to words in the input sentence, when the input sentence parts of speech are read into the incorrect auxiliary-verb sequence graph, when the graph reaches a final state, this uniquely identifies the word at the end of the auxiliary-verb sequence in question. The identification of this word in term of its position in the sentence is then indicated by ending point detector 424.

Likewise, starting point detector 426 detects the word corresponding to the starting point of the auxiliary-verb sequence in question. This is accomplished by having detected the end point of the error and working backwards in the graph from right to left until one reaches the starting state of the graph. For instance, going from left to right, the system has identifies has as have-3rd-singular and consider as verb-infinitive. The system has detected that there is an error at this point and has identified the word "consider" as being the last word in the incorrect auxiliary-verb sequence. Then, moving backwards in the graph and in the input string, one goes past "consider" and past "has". This reaches the beginning of this particular graph and therefore identifies the word "has" as being the first word in the auxiliary-verb sequence.

Referring back to FIG. 4A, having determined the ending point of the auxiliary-verb sequence, the end position of this incorrect sequence is determined at 428 as the position of the last word in the incorrect sequence of the input sentence, likewise, the starting position of the incorrect sequence is determined at 430 as the position of the word starting the incorrect sequence as a number reflecting its position in the input sentence. As illustrated at 432, another directed acyclic graph illustrated in FIG. 4D specifies a set of possible correct sequences for each incorrect auxiliary-verb sequence. Unit 432 then runs through the incorrect auxiliary-verb sequence into the directed acyclic graph illustrated in FIG. 4D and outputs a set of possible correct auxiliary-sequences for view by the user as illustrated at 434.

Referring to FIG. 4C, a directed acyclic graph describing the set of correct auxiliary-verb sequences is constructed as follows for all possible auxiliary-verb sequences. As can be seen in FIG. 4C, at the left hand side of the graph from its starting point 440 are boxes 442 which contain all of the auxiliary verbs in the English language such as "be", "were", "was", "is", "am", "are", "been", "had", "have", "has", "could", "should", "might", "may", "can", "must", "would", "shall", "will", "do", "does" and "doesn't", "did". It will be appreciate that the words "be"-"been" are associated with node 444. In general a node specifies that the verbs that can follow those auxiliary verbs are the same. For instance, "is" can be followed by the word "being" as can the word "were" e.g. "were being". Thus the node 444 indicates that there is a set of auxiliary verbs for which following verbs can be the same. For instance node 446 associated with the set of words "had", "have" and "has" can be followed by the word "been". Similarly for node 448, the words "could"-"will" can be followed by the word "have". Also, these words can be followed by the word "do". Finally, node 450 specifies that the words "does", "do", "doesn't" can be followed by "have" but not by "do".

This way of graph English usage in fact assimilates all of the rules into a compact graphical representation so that correctness of incorrectness of auxiliary-verb sequences can be obtained.

As can be seen there exists boxes labeled "???" which follow the aforementioned nodes. For instance, box 452 It will be remembered that the input to this graph is a sequence of word followed by part of speech. This in essence tags the input with two variables. In order for the graph to remain compact the symbol "???" stands for anything not described at this node. Referring to node 454, box 456 indicates anything but "been" and "had" can go to node 458. Thus it can be seen that the utilization of a "???" box stands the ability to connect to the next node any symbol not described on the output of the state.

In addition to words, the input sentence also involves parts of speech. For instance, when the system analyzes the sequence "have considered", this graph is compared with the sequence "have have considered vbn" in which "vbn" stands for the past participle form. One start at the left hand side of the graph and finds the word "have" as illustrated at 460. From there, one moves to the right past node 446 to box 462 which as described above permits the passage of this word to node 454. From node 454 the possibilities are "been" at 464 or "had" at 466, neither of which match the input sentence. The other alternative is to go to box 456 which permits passage to the right to node 458 and then to the box 460 which specifies "vbn" standing for the past participle form. This permits the passage to node 470. The word considered is deemed to be acceptable because the analysis has passed through box 456 such that the sequence "have considered" is allowed to go to the end point 472 of the graph. Between the intermediate node 470 and end point 472 is a block 474 with the symbol<E>denoting an empty word. The use of <E>denoted box indicates that one can pass from one node to a following node without consideration of such things as a following word or a following part of speech.

For words which are not found in boxes 442, they can be analyzed by passing them through box 476 and node 480 to parts of speech box 482 and thence to node 484. Box 486 provides an arc to end point 472 if appropriate or passed node 484 through box 488 to node 490 and thence to part of speech box 492 or 494 prior to arriving at end point 472. Finally, node 484, if coupling the word with having passed to node 496 and box 498 to node 500. Box 502 passes node 484 via node 504 to part of speech box 506 and then to end point 472 if appropriate. If the word at 484 is to be coupled to both "having" and "been" it is passed to node 508 through box 510 to node 512 and thence through box 514 to node 516. Thereafter it is either part of speech 518 to end point 472 or box 520. Thus proper usage of the input word "having been" is determined as correct if it reaches end point 472 via the route previously noted above. If however the word "being" is to be added to this sequence the output of node 512 is passed to node 522 and box 524 to node 516.

In summary, the direct acyclic graph specifies all correct auxiliary-verb usages. Consequently, a similar graph can be constructed of all incorrect auxiliary-verb sequences. Thus having constructed a graph representing all correct usage, one instantly has a graph representing all incorrect usage. The compactness of this approach is exceptionally efficient in the analysis of sentences as can be seen from the instruction set of Appendix B.

Referring now to FIG. 4D a finite state transducer in the form of a directed acyclic graph is utilized for proposing corrections for incorrect auxiliary verb sequences as determined by the acyclic graph of incorrect verb sequences generated above. In order to propose appropriate corrections the auxiliary verbs are paired such that the left word in each pair is identified as being incorrect, and the right word is the correction. For instance, having identifies that the auxiliary verb sequence "will had" is incorrect the graph of FIG. 4D is utilized for specifying a correct sequence. Starting with an input node 530 one is permitted to go through box 532 with the left of this box is the same as the first word of the input. Having passed through nodes 534 and box 536 to arrive at node 538 the word now considered is the word "had". Box 540 indicates that "had" should be changed to "have" which fact is outputted to node 542 and thence through box 544 to end point 546. Having reaches end point 546 by this path the correct sequence suggested is "will have".

A more complicated case is one considering the incorrect sequence "would considered". The corresponding part of speech tags is "would would considered vbn". In this case one first reaches node 534 by having passed through box 550 denoting "would;would" and through box 536 to box 538. Here none of the boxes 540, 552, 554, 556, 558 or 560 apply. This is because none of these boxed have the word "consider" in. Note that via box 562 an appropriate and correct proposal via part of speech analysis box 564 is "would consider". This was arrived at because the graph detects that "considered" is a part tense of the word "consider" This box suggest the present tense be used and therefore suggest the word "consider". The analysis is denoted by "vbd:/vbd/vb". Note that vbd means past tense and vb means present tense. There are alternative nodes from node 538 which provide other correct changes to the input. For instance, the suggested sequence could "would have considered". Here box 566 specifies that the word "have" should be added. Box 568 specifies that the part of speech of "have", hv, should added also to the sets if tags. After proceeding through box 570 box 572 specifies that the past tense form should be transformed to the past participle form. In that case the word "considered" remains unchanged because it is both a past tense and a past participle. If the input word had been "knew" as opposed to "considered" then box 572 would have specified a change from "knew" which is the past tense to "known" which is the past participle.

The remainder of the graph of FIG. 4D is self explanatory to provide various suggested changes to incorrect verb sequences once having determined that they are incorrect. The program listing for this graphical sequences is presented in Appendix C.

d) Inflection Correction for Non-Native Speakers

As is common, spell checking systems typically detect a misspelled word through a dictionary lookup algorithm. While this is successful in detecting misspellings typically due to inadvertent key strokes or character transpositions, these systems are ineffective for other types of spelling errors. Most notably, spelling errors of non-native speakers or not usually inadvertent transpositions of letters in word, or inadvertent character insertion or omission, they are mainly due to grammar problems. For instance, taking the sentence "He drived his car yesterday", the error is not one of either inadvertence or lack of knowledge of a particular spelling, but rather an uncertainty as to the past tense of the verb "drive" in this case.

Typically, spell checkers suggest proper spellings based on the distance between the mistyped word and a word in the dictionary. The distance is typically based on the number of characters which would have to be replaced, inserted, transposed, or deleted. The result is oftentimes curious. For instance, while in the above example the correct suggestion would be the past tense of "drive", namely "drove", current spell checkers suggest "dried", and "dripped" amongst others. It is interesting to note that the correct word "drove" is not suggested. This is because current spell checking systems do not analyze detected spelling errors in terms of grammar.

Another example of the difficulty present systems have in the suggestion of proper spelling includes improper comparative adjectives. For instances a non-native speaker in selecting the comparative for "good" will oftentimes select gooder based on the usual rule for forming the comparative adjective. As a further example, a non-native speaker when wishing to form the plural of the noun "child" might select the word "childs" as opposed to "children" based again the usual pluralization rule involving in the addition of "s" to a singular noun.

To indicate the inability of current spell checkers to suggest appropriate words in the above example, a typical spell checkers suggest the following words, non of which are correct in context: "chills", "child's", "chill's", "child", "tildes". An even more inadequate suggestion by current spell checker, is the suggestion of how to properly spell "goodest" namely: "gooiest" and "goosed".

These types of errors not only are annoying to native-speaking individuals causing them to refuse to use the spell checking function, the level of frustration for non-native speaking individuals is even higher when forced to select amongst words unfamiliar in or out of context.

Referring now to FIG. 5 in the Subject Invention, it is important to identify typical examples of words which do not follow normal rules either as to pluralization, past tense, past participle, comparative formation, superlative formation. It is from this unique list of incorrect words generated on the basis of grammar that the subject system suggest more suitable replacement words. The subject spell checking system operates normally to detect misspellings by a dictionary lookup system. Thereafter, correct words are suggested based on both the compendium of typical incorrect words and root and morphology features as will be discussed below.

In FIG. 5, an English words corrector 600 includes an English words dictionary 602 and a list of incorrect English words 604 generated by comparing at 606 words from the English word dictionary 602 and a dictionary 608 generated by normal rules of English word formation. The result of the comparison is the above mentioned unique listing of troublesome words based not on spelling mistakes but rather on incorrect grammar.

Referring now to FIG. 6, in the process of actually correcting detected incorrect words, English words dictionary 602 is used along with the list 604 of incorrect English words previously generated as discussed in connection to FIG. 5. The detected incorrect word is available at 610, derived conventionally through dictionary lookup. Both the incorrect word which has been detected and the list of incorrect English words is applied to a unit 612 which determines the root of the incorrect word and its morphological features such as tense, number, comparative vs. superlative forms. For instance if the incorrect word "drived" the root form of this word is "drive" and its morphological feature is "past tense or past participle". The root and the morphological features are provided to a unit 614 which correlates the root and the morphological features with the corresponding English words in the English words dictionary 602 to provide a suggested corrected word thereby taking into account both rules of grammar and exceptions there too.

In essence, the system having derived the root and morphology based on typical incorrect usages is now capable of suggesting appropriate words correlated with these uncorrected usages. The system does provide a sophisticated lookup having identified problem words which are problems due to grammar as opposed to simple misspellings. The program listing describing the process is contained in Appendix D.

It will be appreciated that a part of speech tagger can be beneficial in improving the accuracy of the words suggested by the system. For instance, where a misspelled word could either be a past tense or a past participle. An example is from the above is the correction of "drived" which could lead to "drove" or "driven". Knowing the way in which the "incorrect" word is used in the sentence can result in a proper selection based on parts of speech.

e) Detecting and Correcting Improper Usage of Determiners

One of the more difficult problems for non-native speakers is the problem of determiner usage. Determiners are words such as "the", "a", and "some" that determine the referent of a noun phrase. There are three categories of errors involving determiners. The first is the missing determiner. For example, the sentence, "John read book" is missing a determiner for the noun phrase "book". A second class of determiner errors is the use of extraneous determiners. An example is "John went to the New York". Here the determiner "the" is improper and is to be deleted. The third class of determiner errors is the lack of agreement between a determiner and the associated noun. For instance, "John read many book" illustrates the lack of agreement in number between "many" and "book".

In order to detect the improper use of determiners, part of speech tags are utilized in the analysis. The part of speech tagger is described here and above in connection with FIGS. 2A, 4A, 11, 12 and 14. As an example of a tagged sentence, consider the sentence "John read long novel". Here the tag for "John" is "proper-noun"; the tag for "read" is "verb-past"; the tag for "long" is "adjective"; and the tag for "novel" is "singular-noun".

As illustrated in FIG. 7, the system identifies noun-phrases as illustrated in decision block 700, which identifies noun phrases in the sentence by maximally matching a pattern that defines which sequences of part of speech tags constitute valid noun-phrases. The pattern for noun-phrases is given by:

and the pattern for MODS is given by:

where DET, MOD, NOUN, and AND are defined as sets of part-of-speech tags for determiners, modifiers, nouns and coordinating conjunctions, respectively. The notation [X] means zero or one occurrences of the enclosed expression X. The notation (X)* means zero or more occurrences of the enclosed expression X. A plus superscript, as in X.sup.+, means one or more occurrences of the expression X.

The purpose of the above is for identifying noun phrases. For example, in the sentence given above, the noun phrases are "John", corresponding to the part of speech sequence "proper-noun", and "long novel", corresponding to the part of speech sequence "adjective singular-noun". The above uniquely identifies noun phrases by identifying the start of the noun phrase and its end, as can be seen by the program listed in Appendix E. It is of major importance that noun phrases be identified in order to check for either missing determiners, extraneous determiners, or lack of agreement in number for the constituents of the noun phrase.

Once a noun phrase is found, as illustrated at 702, the system tests whether the noun phrase is missing a determiner. The test looks at the entire noun-phrase, NP, and also looks at the head noun, NOUN.sub.head, which is the last word in the noun phrase. Head refers to the most important noun in the phrase and has been found to be the last word in most instances. The test for a missing determiner also looks at the determiner of the noun-phrase, DET, which either is the first word of the noun phrase or does not occur at all. If the head noun is a singular, non-proper noun, and DET is not present, as determined at 704 and 706 in FIG. 8, then the noun phrase is tested at 708 to see whether it is a title. A title is taken to be any capitalized phrase other than a proper noun; for instance, "The Atlanta Police Department" and "Grady Hospital" are titles. If the noun phrase is not found to be a title, then the head noun is tested to see whether it is a mass noun at 710. A mass noun is a noun that represents an unspecified quantity of a substance, for instance, "rice", "fish", or "carbon". It will be appreciated that mass nouns do not require determiners because they function effectively as plural nouns.

If the noun phrase is a title, then an analysis is done to ascertain whether the head noun is a mass title noun, as illustrated at 712. A mass title noun is analogous to a mass noun, but occurs in a title. For instance, in the sentence, "She attended Harvard University", the noun phrase "Harvard University" is a title, and "University" is a mass title noun. Note that "University" therefore appears in the sentence with no determiner. Observe also that mass title nouns are not the same as mass nouns. For instance, while "University" is a mass title noun, it is not a mass noun. This can be seen from the sentence, "She attended a fine university", where the noun "university" is given the determiner "a". It will thus be appreciated that no suggestions are made if it is determined that one has a mass title noun.

There is, however, a problem for idiomatic usage. As illustrated at 714, the noun phrase is analyzed to see if it is part of an idiom. This is done through lookup in an idiom dictionary. If the noun phrase is part of an idiom, again no suggestion is made. For example, in the sentence, "The event took place", no suggestion is made for the noun phrase "place", although it lacks a determiner, because it is part of the idiom "to take place".

For singular non-proper nouns which have no determiner, if the head noun is not either a mass noun or a mass title noun, and if the noun phrase is not part of an idiom, then the system suggests that there is a missing determiner, as illustrated at 716.

Referring now to FIG. 9, the system then checks the noun phrase 720 for an extraneous determiner. This is accomplished as follows. Whether or not the head noun is a proper noun is determined at 722 by introducing the noun phrase and ascertaining if a determiner is present as illustrated at 724. If the above conditions are met, it is determined that one has an extraneous determiner, as illustrated at 726. For example, "John went to the New York" would be indicated as having an extraneous determiner because the noun phrase "the New York" contains a head noun which is a proper noun and because there is a determiner, namely the word "the", in the noun phrase. Proper nouns are identified by the tagger which determines the existence of a proper noun based on probabilities and context.

Again referring back to FIG. 7, as illustrated at 730, the subject system then checks the noun phrase for number disagreement. How this is accomplished is illustrated in FIG. 10. The determination of number agreement is accomplished by introducing the noun phrase to a detector which determines whether the head noun in the noun phrase is a proper noun, as illustrated at 732. If it is, there can be no disagreement in number. This is because if a proper noun phrase contains a determiner, then it already will have been reported as an extraneous determiner error. Assuming that the head noun is not a proper noun, as illustrated at 734, the system determines whether or not the noun phrase contains a determiner. If not, there can be no problem of number disagreement.

As illustrated at 736, if there is a determiner, then the number of the determiner is checked against the number of the head noun, i.e., singular or plural. If they agree, then no error is signaled; whereas if they disagree, a suggestion is made to change the number of the head noun to agree with the number of the determiner. Thus for the sentence, "John read one books", it is suggested that the head noun "books" be changed to agree with the determiner, and is made singular. Likewise, for the sentence, "John read many book", the subject system suggests changing the head noun to plural to agree with the determiner. Alternatively, the system may be adapted to change the determiner as opposed to the head noun. However, this is a more unlikely course of action. The former yields better results because of the difficulty of ascertaining what the proper determiner should be. It is therefore assumed that the individual has properly entered the correct determiner as regards to number.

In summary, the subject system utilizes a number of techniques for detecting and correcting improper usage of determiners, through the utilization of a tagged sentence and the detection of noun phrases, head nouns, proper nouns, mass nouns, mass title nouns, and idioms. Critical to the proper determination of determiner misuse is the detection of noun phrases through the use of pattern matching described above in connection with FIG. 7.

f) Recognition of Proper Nouns and Other Intrinsically Capitalized Words

It is of some importance in the analysis of sentences to be able to recognize when a word is a proper noun, because it then behaves in a uniquely identifiable way as opposed to all other nouns. By having the ability to recognize not only proper nouns but also other intrinsically capitalized words, such as those that occur in titles, such as "Harvard University", sentences can be parsed and understood so that grammar can be analyzed.

A word may appear capitalized in an English sentence for two reasons. First, it is either a proper noun or other intrinsically capitalized word. Secondly, it occurs at the beginning of a sentence, or after certain punctuation, but would otherwise not be capitalized. As an example, considering the sentence, "Wells was an English novelist", it will be appreciated that "Wells" is capitalized because it is a proper noun. Considering the sentence, "Wells were dug to provide drinking water", "wells" is capitalized because it is the first word of the sentence.

Thus in the first sentence, a grammar-checking system must recognize that "Wells" is intrinsically capitalized and is therefore a proper noun. In the second sentence, the grammar-checking system must recognize that "wells" is not intrinsically capitalized and is therefore an ordinary plural noun.

In previous approaches to determining whether or not a noun is a proper noun, systems have applied relatively limited techniques to recognizing intrinsically capitalized words. One approach has been to assume that the first word of a sentence is never intrinsically capitalized. This fails as indicated by the first sentence and for any sentence that begins with a proper noun.

Another approach has been to classify every word as either a proper noun or an ordinary word, but not both. It will be apparent from the above two sentences that "Wells" can be both a proper noun and an ordinary word, causing this type of classification system to fail.

The obvious problem with failing to properly identify whether or not a word is a proper noun is that in dictionary lookup, the wrong definition will be retrieved. While in simple grammar checking, definitions are not required, sophisticated word-processing and grammar-checking systems which provide tutorial or informational data when determining proper usage require correct identification of proper nouns and other intrinsically capitalized words. Even when dictionary-lookup functions are not part of a grammar-checking system, recognition of proper nouns and other intrinsically capitalized words is important.

The importance of identifying whether a word is a proper noun or not affects the operation of the part-of-speech tagger which must accurately determine the part of speech of each word in a sentence through the use of trigram probabilities. Because the capitalized and uncapitalized versions of a word have different trigram probabilities, it is important for the tagger to know which version of the word is present in the sentence in order to apply the correct trigram probabilities. For example, the trigram probabilities for the proper noun "Wells" are different from the trigram probabilities for the ordinary noun "wells". Thus the tagger would have to realize that in the sentence, "Wells was an English novelist", the word "Wells" is a proper noun, and therefore it should apply the trigram probabilities for the capitalized version of "Wells".

In order to establish whether a word is an ordinary word, as opposed to a proper noun or other intrinsically capitalized word, the subject system determines which of the two interpretations of each word is the best one: the interpretation of the word as a proper noun, or the interpretation as an ordinary noun. It does this by generating two versions of the sentence, one assuming the noun is proper, the other assuming it is ordinary. It then compares the trigram probabilities of the two sentences. If the sentence assuming that the word is a proper noun has the higher probability, then the word is considered to be a proper noun. Otherwise the word is considered to be an ordinary noun.

Referring now to FIG. 11, in order to ascertain whether or not a noun is a proper noun, there are two steps to the decision-making process. The first step, as illustrated at 800, is a preprocessing step in which, as illustrated at 802, one starts with a tagged training corpus. This refers to a set of sentences in which the words of each sentence are annotated with their part-of-speech tags. Next, training corpus 802 is revised as illustrated at 804 to uncapitalize words that are not proper nouns, or, in general, are not intrinsically capitalized. A word is considered to be intrinsically capitalized if the word has been tagged as a proper noun or title, or if it is an acronym, or if it is the pronoun "I". Moreover, words are uncapitalized if and only if they occur at the beginning of a sentence, or after an open quote or colon.

More particularly, as illustrated in FIG. 12, the tagged training corpus 808 is analyzed at 810 to obtain the next word/tag pair, if any, from the corpus. If one is found, the word is analyzed at 812 to see if it is capitalized. If the word is capitalized, as illustrated at 814, it is ascertained if the word is the first word of a sentence or if it follows an open quote or a colon. If so, as illustrated at 816, the word is tested to see if it has been tagged as a proper noun or title, or if it is an acronym or the pronoun "I". If it is not, then the word is to be uncapitalized in the revised training corpus as shown at 818.

Referring back now to FIG. 11, the revised training corpus is analyzed at 820 to obtain a trigram probability model of the words. This provides a modified trigram model to eliminate errors associated with misidentifying a word as a proper noun when it is in fact an ordinary noun, or vice versa. After having preprocessed the tagged training corpus to eliminate errors, the trigram model is utilized at 822 in the decision-making for determining whether the word in question is intrinsically capitalized. This requires as an input a word in the sentence, with the output being the underlying spelling of the word.

As seen in FIG. 13, the decision-making process described at 822 to determine whether or not a word is intrinsically capitalized, starts with a word in the sentence, as illustrated at 850. This word is analyzed to determine if it is capitalized in that its initial letter is a capital letter. If not, as illustrated at 851, the interpretation of the word is that which is given literally by the sentence. That is, if it appears capitalized in the sentence, it is interpreted as a proper noun. If it appears uncapitalized in the sentence, it is interpreted as an ordinary word. Thus if the word is not capitalized, no special action is taken.

Now, assuming the word is capitalized, as can be seen at 854, it is determined if the word is the first word of a sentence or if it follows an open quote or colon. If not, no further action is taken. If so, as illustrated at 856, the word is processed further to ascertain if it is an acronym. An acronym is characterized by all of its alphabetic letters being capitalized or its existing in an acronym dictionary. If the word is determined to be an acronym, again there is no further processing.

If the word is not an acronym, then as illustrated at 858, the system calculates the probabilities of the two versions of the sentence, one with the word at issue treated as a proper noun, which is capitalized, and the other with the word at issue treated as an ordinary noun, which is uncapitalized, in accordance with the trigram model as illustrated at 859. The calculation is as described in accordance with the aforementioned part-of-speech tagger.

If, as illustrated at 860, the probability of the sentence with the word uncapitalized exceeds that of the sentence with the word capitalized, then the system returns the uncapitalized spelling of the word as the most probable underlying spelling, so that this spelling can be utilized for further grammar checking. Otherwise, as illustrated at 864, the system returns the capitalized spelling of the word as the most probable underlying spelling.

The algorithms associated with the FIGS. 11-13 block diagrams is presented hereinafter as Appendix F.

What will be appreciated is that by recovering the underlying spelling of the word, grammar-checking systems can be made more accurate and more useful. The recovery of the underlying spelling involves two steps in which the first step corrects the part-of-speech tags of the training corpus for errors which are induced through the mischaracterization of whether the words are proper nouns or not. Secondly, a series of analyses are performed to ascertain whether the capitalized or uncapitalized spelling of the word is more appropriate. This is accomplished through decision-making elements which decide if the word is intrinsically capitalized using the revised trigram probability model obtained in the preprocessing step.

g) Dictionary Access Based on Context

When writing text, non-native speakers rely on the availability of a monolingual or bilingual dictionary. A dictionary is one of the most useful sources of information about language that non-native speakers rely on. It will be appreciated that the use of a dictionary is not confined to the problem of grammar checking but is generally useful when writing text. It will also be appreciated that even native speakers heavily rely on the use of a dictionary or a thesaurus when composing text.

Accessing an dictionary entry is not as simple as it may appear because words out of context are very ambiguous, both in their syntactic behavior and in their meaning. It will be appreciated that a given word in a dictionary may have typically as many as twenty, thirty or even more entries. This large number of entries make the usage of a dictionary very time consuming.

For example, out of context the word "left" has many entries in an English dictionary: entries for the adjective "left" as in the sentence "His left arm"; entries for the adverb "left" as in the sentence "he moved left on entering the room"; entries for the noun "left" as "Make a left at the next corner"; and entries for the past tense of the verb "leave" as in the sentence "He left a minute ago". However, when the word "left" occurs in an English sentence, only one of this entries is relevant to the context. Currently, no dictionary provides the ability to access the correct entries of a word based on context.

In the subject system, the entries of a dictionary are selected and ranked based on the part of speech assigned to the given word in context. The entries corresponding to the word in context are first selected. The other entries not relevant to the current context are still available at the request of the user. The part of speech of the given word in context is disambiguated with the part of speech tagger described above.

By way of illustration, assuming the word "left" in the sentence "He left a minute ago", the part of speech tagger assigns the tag "verb past tense" for the word "left" in that sentence. For this case, the Subject System selects the entries for the verb "leave" corresponding to the usage of "left" in that context and then selects the entries for "left" not used in that context, in particular the ones for "left" as an adjective, as an adverb and as a noun.

Assuming the word "bases" in the sentence "It has several bases", the part of speech tagger assigns the two tag "noun plural" for the word "bases" in that sentence. It will be appreciated that out of context the word "bases" can be the plural of the noun "basis", the plural of the noun "base", as well as the third person of the verb "bases". For the context "It has several bases", the Subject System selects the entries for the nouns "base" and "basis" corresponding to the word "bases" in that context, and then selects the entries for "bases" not used in that context, in particular the ones for the verb "bases".

Referring now to FIG. 14, in order to select entries of a word occurring in a sentence 900 from a dictionary based on context, the word is analyzed by a morphological analyzer 910 which computes the set of pairs of root forms and parts-of-speech corresponding to the word independent of the context. As an example, for the word "left", the morphological analyzer will output the following set of pairs of root forms and parts-of-speech: ("left", "adjective"), ("left", "adverb"), ("left", "singular noun"), ("leave", "verb past tense"). Morphological analyzer 910 operates by looking up into a table indexed by all inflections of all words of English and whose entries are sets of pairs of root forms and parts-of-speech. The word is also analyzed by a part of speech tagger 930 in context in order to produce the unique Part of Speech Tag T 940 corresponding to the word in context. This is achieved by a part-of-speech tagger which is available as an implementation of Kenneth Church's Stochastic Parts Program described in "A Stochastic Parts Program and Noun Phrase Parser for Unrestricted Text" in the Proceedings of the Second Conference on Applied Natural Language Processing, Austin Tex., 1988.

For example, if the word is "left" in the context "He left a minute ago", the part of speech tagger outputs the part-of-speech tag "verb past tense". In order to separate the morphological roots that correspond to the context from the ones that do not correspond to the context, a unit 920 splits the set of pairs of roots and parts-of-speech 920 into two sets, a set 950 that corresponds to the part of speech tag 940, and the set 960 that do not correspond to the part-of-speech tag 940. In the previous example, the set of pairs of roots and parts-of-speech that correspond to the context is: "leave", "verb past tense". The set of pairs of roots and parts-of-speech that do not correspond to the context is: ("left", "adjective"), ("left", "adverb"), ("left", "singular noun"). In order to display the entries from the dictionary that correspond to the context, all the entries in a dictionary 970 that correspond to a root found in the set of pairs of roots and parts-of-speech that correspond to the context 950 are displayed at 980. In the above example, all entries for the verb "leave" will be displayed as entries relevant to the context. In order to display the entries from the dictionary that do not correspond to the context, all the entries in the dictionary 970 that correspond to a root found in the set of pairs of roots and parts-of-speech that do not correspond to the context 960 are displayed at 980. In the above example, all entries for the word "left" as an adjective, as an adverb and as a singular noun are displayed as entries not relevant to the context. A program listing for the above is available as Appendix G.

It will be appreciated that the ability of selecting entries from a dictionary based on context can be used for monolingual dictionaries as well as bilingual dictionaries, for native or non-native speakers. The subject system is able to select those entries relevant to the context therefore drastically reducing the number of entries that the user has to read.

Having above indicated several embodiments of the Subject Invention, it will occur to those skilled in the art that modifications and alternatives can be practiced within the spirit of the invention, It is accordingly intended to define the scope of the invention only as indicated in the following claims. ##SPC1##