System and method for incorporating dyslexia detection in handwriting pattern recognition systems

[0002] 1. Field of the Invention

[0003] The present invention relates to systems and methods for the detection of dyslexia, and more particularly to such systems and methods incorporated in existing handwriting pattern recognition systems

[0004] 2. Description Related to the Prior Art

[0005] Dyslexia is a specific learning difficulty that hinders the learning of literacy skills. Its etymology is based on the Greek: ‘difficulty with words’. It affects millions of people worldwide.

[0006] Dyslexia affects reading, spelling, writing, memory and concentration, and sometimes math, music, foreign languages and self-organization It is apparently neurologically based and tends to run in families. Other symbolic systems, such as mathematics and musical notation, can also be affected. However, there is controversy as to the exact definition of the syndrome of dyslexia and from contradictory theories that surround its etiology.

[0007] There seems to be a consensus as to the need for early detection and treatment. The symptoms can be difficult to detect, however, especially at an early age. In pre-schoolers these symptoms may include the following¹: THE DYSLEXIA INSTITUTE http://www.dyslexia-inst.org.uk

[0008] Later than expected learning to speak clearly.

[0009] Jumbles phrases, e.g. ‘cobbler's club’ for ‘toddler's club’,‘teddy-dare’ for ‘teddybear’.

[0010] Quick ‘thinker’ and ‘do-er’.

[0011] Use of substitute words or ‘near misses’.

[0012] Mislabeling e.g. lampshade for lamp post.

[0013] A lisp—‘duckth’ for ‘ducks’.

[0014] Inability to remember the label for known objects, e.g. colours.

[0015] Confused directional words, e.g. ‘up/down’ or ‘in/out’.

[0016] Excessive tripping, bumping and falling over nothing.

[0017] Enhanced creativity—often good at drawing—good sense of colour.

[0018] Obvious ‘good’ and ‘bad’ days for no apparent reason.

[0019] Aptitude for constructional or technical toys, e.g. bricks, puzzles, lego blocks, control box for TV and video, computer keyboards.

[0020] Enjoys being read to but shows no interest in letters or words.

[0021] Difficulty learning nursery rhymes.

[0022] Finds difficulty with rhyming words, e.g. ‘cat mat fat’.

[0023] Finds difficulty with odd-one-out, e.g. ‘cat mat pig fat’.

[0024] Did not crawl—was a ‘bottom shuffler’.

[0025] Difficulty with ‘sequence’. e.g. coloured bead sequence.

[0026] Attempts to diagnose dyslexia generally fall into one of two categories: (1) generalized behavioral diagnoses, as in the list above, and (2) physiological tests involving detection of psychological and/or neurophysiological symptoms such as abnormal EEG or evoked potential and/or erratic eye movements.

[0027] Many of the patents in this area are of the latter type. U.S. Pat. No. 4,889,422, for instance, takes this approach, as do many others.

[0028] The current invention, on the other hand, is based on the well-known tendencies of dyslexics to perceive characters in unique ways, both in the perception of the characters, and in attempts to reproduce those characters by drawing the characters as they see them. Among these typical dyslexic misperceptions is the tendency to produce mirror images of characters.

[0029] This invention is implemented by means of the various handwriting recognition systems which are well developed and in wide use at the present time. These systems all have one characteristic in common: they compare a person's handwriting, character by character, to a library of acceptable or model characters, and reject those characters which do not conform to any recognized character model in the library.

[0030] The present invention expands this technique to include not only recognizable legitimate characters, but also illegitimate characters, such as the mirror images of legitimate characters. When the user of one of these handwriting recognition systems generates a significant number of these distorted characters, the invention records these in the memory associated with the handwriting recognition system, and creates a statistical profile of the errors, creating a warning that the user may be displaying characteristics associated with dyslexia.

[0031] This invention has the great advantage that it does not require any advances in the basic technology in handwriting analysis: it merely requires the expansion of the available character sets to include the models of distorted characters typically generated by dyslexics.

[0032] The current invention can be incorporated into simple word games currently available on computer and INTERNET systems for young children of pre-school and early school age, thus providing the early warning so urgently needed.

[0033] It is a general objective of the current invention to incorporate a dyslexia detection system into an existing handwriting recognition system.

[0034] In accordance with one aspect of the invention, a method for detecting dyslexia in a handwriting pattern recognition system, which includes a database of legitimate characters includes the steps of including in the database a set of distorted characters, then presenting a character to be input by a student, and inputting characters by the student. The method proceeds with the detecting of a correspondence between each input character and one of the distorted character, then determining a pattern of inputting of distorted characters, and finally, reporting the pattern.

[0035] In accordance with a second aspect of the invention, the set of distorted characters includes mirror images of the legitimate characters.

[0036] In accordance with a third aspect of the invention, each distorted character is associated with a legitimate character, and the reporting is done by means of statistical reporting of the distorted characters input.

[0037] In accordance with a fourth aspect of the invention, the set of distorted characters includes both a subset of global distorted characters and one or more subsets of local distorted characters.

[0038] In accordance with a fifth aspect of the invention, the subsets of local distorted characters are collected by statistical sampling methods from individual users, and a subset of local distorted characters is associated with each individual user.

[0039] In accordance with a sixth aspect of the invention, each distorted character is associated with a true character, and the true character is reported together with the distorted character detected.

[0040] In accordance with a final aspect of the invention, the database of distorted characters, the means for presenting a word to be input by a student, the means for inputting words containing characters by a student, the means for detecting a correspondence between each input character and one of the distorted characters, the means for determining a pattern of inputting of distorted characters, the means for detecting a pattern of selection of distorted characters in the words input, and the means for reporting said pattern are accomplished by either personal computers, by personal digital assistants, through means of computer games, or by a device communicating over a data communications network with an INTERNET server.

[0041] These, and further features of the invention, may be better understood with reference to the accompanying specification and drawings depicting the preferred embodiment, in which:

[0042] FIG. 1 depicts a block diagram of the system.

[0043] FIG. 2 depicts a flow chart of the first preferred embodiment of the present invention.

[0044] FIG. 3 depicts a flow chart of the second preferred embodiment of the present invention.

[0045] FIG. 4 depicts input screen of a simple user interface for the present invention.

[0046] FIG. 5 depicts a successful result on the results screen of the simple user interface.

[0047] FIG. 6 depicts a reports screen summarizing the test results of a test series by a student.

[0048] FIG. 7 depicts a second reports screen summarizing the test results of a test series by a student.

[0049] FIG. 8 depicts an unsuccessful result on the results screen of the simple user interface.

[0050] FIG. 9 depicts the system used to implement the current invention.

[0051] In the following descriptions, the term “handwriting” will be used to refer to both cursive and printed forms, and the invention described herein operates with both forms.

[0052] The current system is designed to detect, quantify, and catalog the symptoms of dyslexia at the earliest age of the person so afflicted. The system detects an individual's inability to see, recognize, and to faithfully reproduce characters of the alphabet. The cataloging of the failed attempts to recreate the target character results in the identification of specific characters which an individual is unable to recognize and recreate, and this data is then used to report individualized patterns of perceptual dysfunction

[0053] The overriding principle in the current invention is to create a cyborg environment, made up of a man-machine system specifically designed and configured to allow man/machine interaction in order to provide character-specific visual evidence of a person's personal perceptual dysfunction.

[0054] In one of the basic implementations, as shown in FIG. 9, a system containing the current invention includes a computer 101, a monitor 102, and a printer 104. The computer software, including the handwriting analysis software, resides on the computer. A keyboard 105 provides a standard input to control the system. A sketch pad 106 is electronically connected to the computer, and provides the means for the user to supply handwritten input to the computer. The database 103 contain the libraries of characters described hereinafter may be contained within the computer 101, or may be external to the computer.

[0055] The present invention may be implemented by a number of electronics means. Besides the personal computer system shown in FIG. 9, they include the Personal Digital Assistant, such as the PalmPilot™², a new variety of NET appliances, or other devices specifically designed for teaching handwriting to children and illiterate adults. The invention may further be implemented as an interactive application running on the INTERNET, and accessed either by a personal computer, or by an electronic device specially manufactured to access the INTERNET, including cell phones

[0056] The system is shown in its most basic form in FIG. 1. The system requires a display 22, which shows a student the character or words to be emulated. These characters or words are input to the system by some device 10, which may be a pen input device to a PC. Such devices include pen/pad inputs sold by Aiptek, Inc., and by WACOM, Inc. Alternatively, the input may be accomplished by use of a stylus and a pressure-sensitive surface, as used in a variety of personal digital assistants, or PDAs, including the PalmPilot™ series.

[0057] Still referring to FIG. 1, a character recognition process 12 interprets the input by one of a number of well-developed character recognition means. In the prior art, the manually input character would be compared against a database 16 of legitimate characters. In the more sophisticated handwriting input programs, the comparison is contextual; that is, the characters are identified as possibly being one of several different legitimate candidate characters. Then, words using the identified characters are compared to a database 20 dictionary of legitimate words to determine which of the most likely candidates characters produce legitimate words.

[0058] In the current invention, however, the input characters are also compared to a database 18 of distorted characters. These distorted characters consists of the common variations that a dyslexic person produces when attempting to write certain characters. In addition, the database of distorted characters will also contain links to the legitimate characters that these distorted characters represent. Typical distortions of this nature are shown in FIG. 4.

[0059] In its simplest implementation, the process proceeds according to the steps shown in the flow chart of FIG. 2. The process starts by displaying a character to copy 30. In a PC-based system the display is done on the monitor. The student then inputs the character 32 shown on the screen, attempting to reproduce it correctly. The system next determines 34 whether the character input was the character displayed, within the limits of the handwriting analysis software of the prior art. If the exemplar input by the student is correct, the process continues by notifying the student that his input was correct, and then proceeding to display 30 a new character to copy.

[0060] If, however, the input character was not recognized as one of the legitimate character set, then the input character is then tested 36 to determine whether the input character was one of the distorted character set. If not, then the input is simply reported 38 as an error, and is added to an error data base as well for subsequent statistical reporting. However, if the character input is one of the distorted character set, a special entry is made 40 to the report database for subsequent output. It should be noted that all errors are recorded for subsequent analysis.

[0061] The process then tests 42 for completion, and will recycle to the beginning with a new display 30 of a character to copy until a termination 42 is finally indicated, at which time a report 44 is generated.

[0062] The final report will typically indicate how many characters were input, how many general errors occurred, and how many distorted characters were generated. The report should further indicate which legitimate characters resulted in the student's inputting which distorted characters

[0063] In the preferred embodiment, these reports will not be automatically output to the student. Rather, the reports will be stored internally in the computer, PDA, or other device, where they can be output later by those monitoring the student progress.

[0064] In a further variation of this preferred embodiment, the distorted characters will not be contained within a separate database, but will be generated on the fly by a separate process. For instance, a common type of distortion perceived, and generated by those suffering from dyslexia is a mirror image distortion.

[0065] Thus, still referring to FIG. 2, the test of the input character against a distorted set 36 is implemented by taking a character from the legitimate set, creating the mirror image by data processing means, and then comparing the input character to the just-calculated mirror image.

[0066] It should be noted that this embodiment, as well as the one described below, may be implemented by means of a computer game format.

[0067] A simplified user interface is shown in FIG. 4. The input screen 73 depicted in this figure may appear on a PC monitor, a PDA, or the like. The user is shown a character 70 to reproduce, in this case, the letter “A”. The user is then asked to reproduce this character, by drawing, using a pen-type device. The user then attempts to draw the character in the input box 72.

[0068] Referring now to FIG. 5, which shows a results screen 75 after the user has drawn an “A” in the input box 72. The system recognizes the drawn figure as acceptable, and rewards the user with a “star” in the result area 78, indicating that the input was correct.

[0069] Referring now to FIG. 8, a results screen 75 is shown after the user has been presented with the number “3” in box 70. In this case, the user has drawn an incorrect character, resembling the number “8”, in input box 72. As a result, the system produced a result 78 which signifies an incorrect input.

[0070] Referring now to FIG. 6, a report screen 83 summarizes the results of a test session. This screen contains a header 84, in which information about the user appears, including name, date of the exercise, grade level, and the test series used. This screen also shows a column 80 containing the characters presented for duplication. A second column 82 shows the results produced by the student after the first series. Although FIG. 6 shows the results of only one test series, additional series results may appear in columns to the left of column 82, which are blank in this figure.

[0071] Referring now to FIG. 7, a different report screen 85 summarizes the results input by the student in response to individual characters, in this case, to the character “B” presented for duplication. This screen contains a header 90 similar to the header of the report screen shown in FIG. 6. Still referring to FIG. 7, a row of characters 86 drawn by the student in successive attempts to reproduce the letter “B” appears. A summary 88 of these attempts appears beneath row 86.

[0072] In a second preferred embodiment the implementation of the current invention is based on a prior art method of character recognition in which a database of legitimate words is included in the system, in addition to a database of legitimate characters.

[0073] Referring now to FIG. 3, The user is presented with a display 50 of words to copy. He then inputs 52 the word presented, and a test 54 is made, using the handwriting pattern recognition system. If the students input is judged acceptable by the system, a new word is displayed 50 for the student to copy, and the process continues. If not, however, the system attempts to determine 56 if any of the characters input by the student are contained in the distorted character set. If they are not, the student's attempt is reported as an ordinary error 58, and control returns to the top of the process 50.

[0074] If, however, the characters contained appear as part of the distorted character set, then the corresponding legitimate characters are substituted 60 for the distorted characters. The word input by the student, with legitimate characters substituted for the distorted characters, is tested against the dictionary of acceptable words, and the result is stored as a variant revealing a misperception by the student, for inclusion in the final report.

[0075] At the end of each word input, a determination 64 is made as to whether any more test words will be presented to the student. If not, the report or reports 66 will be generated. If not, control returns to the beginning 50 of the process.

[0076] In a third embodiment of the present invention the handwriting analysis is performed in the normal way in accordance with the prior art, except that the characters considered to be errors, or not identifiable, are all stored for subsequent analysis. These unacceptable characters are stored, together with an indication of the character presented to the user which resulted in the unacceptable character.

[0077] At the end of the test session, the unacceptable characters are analyzed by the system software to determine if they contain characteristics associated with dyslexia, identifying specific patterns of perceptual dysfunction. The reports generated in this embodiment contain characterizations of the types of errors found. Referring to FIG. 7, the various errors are displayed in column 86. The results of the analysis appear in area 88. As seen in FIG. 7, a separate report may be generated for each different character presented to be written or printed by the student.

[0078] The reports will be output either on the screen of the testing device, or may be printed as hard copy for later analysis.

[0079] It has been observed that people afflicted with dyslexia will often distort characters in a common manner, as in the case of writing mirror images of characters. These types of distortions will be referred to herein as “global distortions” In addition, many dyslexics will write characters which are unique to that particular person only. The system may be used to detect such distortions in the character sets which are unique to a particular user. When a user makes a mistake by inputting a character which has repeated, consistent characteristics, the character presented for copying, or “true character”, together with the misperceived character are stored. These misperceived characters unique to this particular user, are called “local distortion”s. The characteristics of the local distortion are collected over a large number of samples and stored. Average characteristics of the sample are collected, and a typical “local distortion” is compiled. The characteristics of this “local distortion” are defined in the same way as is used for the “true character” in the particular character recognition methodology used by the system.

[0080] In this embodiment, the reports are segregated into separate reports which show global distortions, and those which show local distortions.

[0081] A database, or library of local distortions will be assembled for each user. Thus, when a particular user logs onto the system, the system will access not only the library of true characters and global distortions, but also the library of local distortions for the particular user who has logged onto the system. In this way, the system in personalized for particular users.

[0082] In another embodiment, the characters input by a user will be compared not only to the database of true characters, but also to the libraries of global and local distortions. The character drawn by the user may then be translated back into a “true character” as part of the detection process, and displayed for the user to see. This embodiment is useful in cases where the user is not only making an erroneous drawing of a character, but confusing the character to be copied with another character.

[0083] Because this invention is intended to detect dyslexia in young children, it is especially suitable for incorporation into computer or INTERNET games. Such games require the use of an electronic pad on which to draw characters which then appear on the screen of a computer or similar display. A typical game of this type may involve a simple mimicking of a letter shown on a computer screen, or an attempt to write down a word which is displayed on the screen in a setting where the user has a limited time to copy a sample using the users own handwriting, where the sample disappears after a short period of time. Such computer games, and similar handwriting development programs which uses computers are currently in use in some schools, and will be increasingly available for this purpose.

[0084] While the invention has been described with reference to specific embodiments, it will be apparent that improvements and modifications may be made within the purview of the invention without departing from the scope of the invention defined in the appended claims.