Method and apparatus for training a person to learn a cognitive/functional task

[0002] Traditionally, when one needs to obtain knowledge in a particular field or learn a skill, one can generally learn it from a teacher, through a video tape, or through a personal computer. For example, there are numerous video tapes and CD-ROMs for teaching people foreign language skills and other subjects, such as mathematics, etc. The results of learning through these means are generally acceptable when the skill does not involve performing a cognitive/functional task that requires a sequence of decisions and a corresponding sequence of actions. For example, to train a child or a mentally disabled person to cross a street under different traffic conditions, a video tape would not be effective since the person cannot take appropriate actions when a street under a particular traffic condition is presented to him in the form of a video image. Thus, if the video tape shows a street with a green pedestrian light, the trainee will not learn to decide upon a course of action such as is involved in crossing that street under that condition since the street is not physically present, and their decision/actions have no influence on the observed scenario. Therefore, the effectiveness of this type of training is discounted. On the other hand, to have a teacher to train each trainee in all actual environments would be too costly.

[0003] Therefore, there exists a need for an improved training tool that trains a trainee in a more effective manner and that allows the trainee to learn to make critical decision and then to safely practice appropriate actions in a simulated environment.

[0004] The present invention provides a method and apparatus for training a person to learn a cognitive/functional task that requires a sequence of decisions and a corresponding sequence of actions.

[0005] According to a preferred embodiment of the invention, a cognitive/functional task is decomposed into critical elements and non-critical elements. The critical elements have distinguishing features that are essential to the cognitive/functional task. The critical and non-critical elements are presented to a trainee in a predetermined sequence in a simulated environment through, for example, a virtual reality device. The critical elements may be presented separately and in combination, and then the critical and non-critical elements may be presented in combination.

[0006] According to a preferred embodiment of the invention, the trainee is trained to act in accordance with the critical and non-critical elements presented. Furthermore, a data processing system is used to monitor the actions taken by the trainee in the simulated environment in response to the presented critical and non-critical elements. Moreover, the learning receptivity by the trainee is measured. According to a preferred embodiment of the invention, the learning receptivity is measured by measuring fatigue and attention levels of and learning achievement by the trainee via a brain wave detection device. The detection device detects brain wave signals of the trainee. If the brain wave signals are in a particular stable state, it indicates that the person has a low level of fatigue (absent theta activity and/or stable theta). Another brain wave, alpha is monitored as well. When alpha wave is detected and/or presented to a greater degree, (i.e., there is an altered occurrence to a statistically likely threshold over the current detection level of the particular wave assessed), the person is not paying attention. If the brain wave signals are in low theta and alpha, it indicates that the person has a low level of fatigue and is paying attention. In alternative cases, the training should be temporarily terminated, the trainee should be cued in accord with the particular training level employed and training resumes as learning receptivity is again achieved. Learning achievement is determined by the application of a learning achievement algorithm to the brain wave signals continuously gathered.

[0007] According to a preferred embodiment of the invention, a training process may be defined into different levels of complexity for different learning stages and for different trainees. In each level, the critical and non-critical elements are presented in a particular manner. After a training process is completed, the trainee is tested to perform the complete cognitive/functional task in the simulated environment. If the trainee fails the test, the training process may be modified to adapt to the specific training needs of the trainee.

[0008] The present invention finds many applications in various fields where training people to perform cognitive/functional tasks is needed, such as in rehabilitating brain-damaged persons, and industrial training, etc.

[0009] Other objects and attainments together with a fuller understanding of the invention will become apparent and appreciated by referring to the following description and claims taken in conjunction with the accompanying drawings.

[0010] FIG. 1 shows a functional block diagram of a training system according to a preferred embodiment of the invention;

[0011] FIG. 2 shows a flowchart illustrating a training methodology according to a preferred embodiment of the invention; and

[0012] FIGS. 3A to 3C illustrate an example using 3D images in a simulated training environment for training a person to cross a street.

[0013] FIG. 1 shows a functional block diagram of a training system for training a trainee 101 in accordance with a preferred embodiment of the present invention. The training system comprises a data processing system 102 including a processor 103 and a memory 104, an input device 107, a speaker 108, a virtual reality device 109, a brain signal detection device 110, and a monitor 113. Input device 107, speaker 108 and virtual reality device 109 are connected data processing system 102 via input/output lines 106. It will be understood by persons of ordinary skill in the art that there are numerous other elements not shown in the figure, such as disk drives, mouses, keyboards, additional memory busses, etc., that can be connected to data processing system 102. Memory 104 includes training software 114 for teaching trainee 101 to learn a cognitive/functional task that requires a sequence of decisions and corresponding sequence of actions. Input device 107 can be a joystick that is used by trainee 101 to simulate actions he would take during a training session. Other input devices may also be used, such as a treadmill for use by the trainee to simulate the action of crossing a street. Virtual reality device 109 is worn by trainee 101. Virtual reality device 109 is a conventional device that displays three-dimensional (3D) images to simulate an actual environment in which the trainee is to be trained to perform a cognitive/functional task.

[0014] A cable 118 has one end attached to the head of trainee 101 and the other end connected to a brain wave detection device 110 which is connected to a monitor 113 and data processing system 102 via input/output lines 106. Detection device 110 detects the brain wave signals of the trainee to measure the learning receptivity and learning achievement of the trainee during a learning process. The learning receptivity is represented by the fatigue of and attention by the trainee. Learning achievement is measured by application of the BEAM learning achievement algorithm to standard EEG data, using commercially available EEG measurement equipment. The BEAM learning achievement algorithm is commercially available from B.E.A.M. Inc., Boston, Mass. In a preferred embodiment, an alpha-wave/theta-wave detector is used as detection device 100. Alpha-wave/theta-wave detectors are commercially available and are made by Neuro Cyber Netics, Encino, Calif. Monitor 113 is connected to the output of detection device 110 for displaying the brain wave signals of trainee 101.

[0015] FIG. 2 shows a flowchart illustrating a training methodology 200 for training a person to learn a skill of performing a cognitive/functional task in accordance with a preferred embodiment of the present invention. Methodology 200 is implemented through training program 114 executed by processor 103 in FIG. 1. In accordance with the methodology, a cognitive/functional task is decomposed into critical and non-critical elements (step 201). This step may be performed by training program 114 through commands input by a trainer. Critical elements are those components in a cognitive/functional task that have distinguishing features and are essential to performing the cognitive/functional task the distinguishing features having operative characteristics which demand a certain action on the part of the person. For example, in a process of training a child or a mentally disabled person to cross a street, the critical elements of performing this cognitive/functional task include traffic signs, traffic lights, and moving vehicles since they have distinguishing features and are essential to crossing a street. Trees and buildings are non-critical elements since they do not relate to crossing a street. In another example, in learning the skill of driving a car, the steering wheel, the brake pedal, and the gas pedal are critical elements; the seats and passengers are non-critical elements since they do not relate to the core skills of driving a car. Introduction of the non-critical stimuli elements would represent unnecessary distraction in the early learning period. In later learning periods, these elements are introduced, and the training focuses on the endurance of the core cognitive/functional competence in the presence of distraction.

[0016] Once the critical isolated elements of a cognitive/functional task have been mastered (step 202), the training continues, now introducing levels of complexity. Then, the training processes are defined in terms of complexity level (step 204). For example, a particular training process may be defined in three levels of complexity, e.g., low, medium and high. These three levels may be used at different learning stages and for different trainees. In the above example of training a person to cross a street, a low level of complexity in the simulation training environment may be one where only some of the critical elements, such as traffic signs are displayed in a street scene with no cars, busses or sound. The person will be trained to cross the street under these circumstances. A medium level of complexity may add missing features of the simulation training environment such as cars, changing traffic lights, distraction, noise, etc. The person will be trained to cross a street under these traffic conditions. At this level, cognitive demands are increased, and success will require important new skills. A high level of complexity is an important next step towards the real world. In the real world, cars don't always stop at lights and rules are sometimes broken. At a high level complexity, the trainee deals with rule violations, deliberate distractions, and interrupted action plans. Success here requires additional skills.

[0017] After the training process is defined, an appropriate level of complexity is selected for training a person (step 206). The selection is based on the background of the trainee. For people with good fundamental skills in a particular cognitive/functional task, a relatively high level of complexity may be selected for the initial training, once it has been demonstrated that the isolated critical features of the target cognitive/functional task are indeed mastered. Next, the critical and non-critical elements relating to the cognitive/functional task are displayed on the virtual reality device (step 206) in accordance with the selected training level. The critical and non-critical elements may be displayed separately and in combination in a predetermined manner. Then, the trainee will be trained by a trainer (not shown) to perform the cognitive/functional task through the use of the input device under various conditions (step 207).

[0018] In the above example of learning to cross a street, for instance, only traffic signs or lights are initially displayed in a street scene by the virtual reality device. Then moving vehicles are added to the street scene. Next, non-critical elements such as trees and buildings are also added to the street scene.

[0019] Under the various traffic conditions, the trainee will be trained to cross the street. During the training process, actions takes by the trainee via the input device are monitored through the data processing system (step 208). As the trainee completes the task, the time spent in each simulation is monitored and success, as measured by freedom from errors, is recorded. Simulated natural consequences are generated for errors. In the example of crossing a street, if the trainee is hit by a moving vehicle when he is crossing the street, a beep is generated indicating an error. Over individual and multiple sessions, cumulative errors are stored in the computer and error rates can be reviewed. Thus, actual performance evaluation of real life simulation activities is possible. By altering the level of complexity, monitoring errors and progressively challenging the trainee to reach for the highest level of independence, a trainer can know for sure if the training within a sophisticated training environment is effective. The ability of the training system to produce performance reports enhances a trainer's ability to demonstrate a trainee's progress. Additionally, cognitive/functional tasks are linked into combinations such as in a scenario of crossing a street to enter a store, and then to shop in the store. The training system presents the trainee with this complex scenario, and then monitors performance of the trainee in their recreation of this combination of cognitive/functional tasks.

[0020] In methodology 200, fatigue of and attention by the trainee are also detected (step 210). At this step, the learning receptivity of the trainee is measured by brain wave detection device 110. If the brain wave signals from the trainee are detected as being in low theta and low alpha, it indicates low fatigue and high attention. On the other hand, if the brain wave signals are detected otherwise, then it indicates poorer learning receptivity.

[0021] Based on the recorded performance results and the detected fatigue and attention levels (and learning achievement), the learning receptivity is measured (step 211). Then at step 214, training continues. If high fatigue and low attention are detected as compared to the levels at the beginning of the training session, this indicates that the trainee is probably too tired to learn the skill at this time and the program will cue the trainee to refocus (step 212). Then learning receptivity is again measured (step 213). If refocus occurs, training continues (step 214). If refocus does not occur, the system will exit. If, however, low fatigue and high attention are detected, the training will continue, irrespective of the performance results. Then, at step 215, whether or not the training process is completed is determined and demonstrated by both performance success and brain wave signals indicative of learning achievement. If not, the training process continues at step 206. Learning achievement is determined by the application of for example, the BEAM learning achievement algorithm to the brain wave signals continuously gathered. Other appropriate learning achievement algorithms can also be used.

[0022] According to the methodology of the invention, a person is first trained to achieve small goals. After a cognitive/functional goal is successfully achieved by the person, larger goals (cognitive/functional tasks in combination) are presented for training the person, until the ultimate goal is successfully achieved by the trainee. An ultimate goal is determined a priori by the trainer and may be such a combination of cognitive/functional tasks as is involved in leaving an apartment, driving to a store, shopping, returning to the apartment, and preparing a meal. Alternatively, assessing the repair needs of a non-functional engine of a vehicle, ordering and receiving parts and repairing the engine would be another such ultimate goal. Monitoring learning achievement becomes critical at this juncture. With both successful task accomplishment and learning achievement detected, the trainee has demonstrated learning of the desired skill which could be confirmed in the following test phase. If task successfully occurs without learning achievement detected, the likelihood of random accomplishment without true learning arises. The task would be repeated until such time as both successful performance and learning achievement are both expressed.

[0023] After the ultimate goal is successfully achieved by the trainee, i.e., after the trainee has been trained to perform combination of cognitive/functional tasks represented by the ultimate goals, a test is given to the trainee to analyze his ability to perform the task in the simulated environment (step 216). In the example of crossing a street described above, the trainee would be required to go to a grocery store that is several blocks away from his home and then come back. Precedent to the test, the trainee would be presented with the desired target performance by either the trainer and/or by the system. The analysis results are determined based on the error rates in the process of performing the task (step 218), the time to complete the task, and the degree of optimization realized by the trainee as defined by the relationship of their performance to the target performance that had been originally presented to them by the trainer and/or by the system. If the trainee performs well, the training is over, and the trainee has learned the skill to perform the cognitive/functional task. If, however, the results of the analysis is not satisfactory, the training process will be modified to adapt the trainee's individual needs (step 220). For example, in the example of crossing a street, if the trainee fails to stop when a red pedestrian light is on, he will be repeatedly trained on discriminating among the different colors of the light. He will also be trained to take the appropriate actions when the light is at each color. The training process will continue in a similar manner as above until the ultimate goal is achieved and the trainee's ability to perform the cognitive/functional task is demonstrated as measured by the successful performance of the cognitive/functional task.

[0024] FIGS. 3A-3C illustrate an example of training a person to cross a street according to a preferred embodiment of the invention. These figures illustrate 3D views that would be seen by a trainee from position 300 through the virtual reality device during a typical training process.

[0025] As shown in FIG. 3A, the 3D street scene is presented to a trainee through the virtual reality device. In the figure, the trainee is standing at position 300 and will be trained to cross a street along a cross walk 302. In this 3D street scene, selected elements are presented to the trainee. These include pedestrian light 304, traffic light 306, street sign 308, and bus stop sing 310. Non-critical elements, such as buildings, trees, etc. are not displayed. In a preferred embodiment of the invention, these signs and lights will be initially shown to the trainee individually and then in combination. Once the trainee is familiar with these signs and lights and appreciates their implied meanings, additional elements will be added to the scene.

[0026] In FIG. 3B, moving vehicles are added to the scene. Specifically, a moving car 312 and a moving bus 314 are added to the street scene. In addition, sound may be generated to simulate the actual noises generated by the moving vehicles. At this stage, the trainee will be trained to act in accordance with the traffic rules. For instance, when pedestrian light 304 is green, he may cross the street; but if pedestrian light 304 is red, he must stop walking. Then he will be asked to cross the street when there are moving vehicles, and when the pedestrian light 304 is at different colors. The results of his performance are recorded. Each time he successfully crosses the street, a score will be recorded. Each time he is hit by a moving vehicle, a beep is generated to simulate the natural consequence of an accident. The fatigue of and attention by the trainee are continuously detected. If the brain wave signals of the trainee are in a predefined state, the trainee is probably too tired and the training will be temporarily terminated. If the brain wave signals of the trainee are acceptable, the training may continue.

[0027] In FIG. 3C, in addition to the elements in FIG. 3B, other non-critical elements including buildings 316, 318, and trees 320, 322, 324 are presented to the trainee. FIG. 3C essentially represents a complete simulated street scene. The trainee is again asked to cross the street. He will be trained to discriminate among various critical and non-critical elements and to take actions accordingly. After the trainee successfully completes the training process, a test will be given to him to analyze his ability to cross a street in a simulated environment. For example, he will be instructed to go to a grocery store which is several blocks away from his home and come back and/or he will be shown this performed successfully. If he successfully passes the test, then he has learned the skill. However, if he fails, the training process may be modified to accommodate the trainee's specific needs. The new training process may be presented to him, until he successfully learns to perform the task.

[0028] The present invention is useful for training a person to perform a cognitive/functional task in a simulated environment. The present invention is particularly applicable for rehabilitating brain-damaged persons who, after certain accidents, do not remember how to do many cognitive/functional tasks. The present invention also finds application in various other fields, e.g., in industrial training, etc.

[0029] While the invention has been described in conjunction with specific embodiments, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art in light of the foregoing description. Accordingly, it is intended to embrace all such alternative, modifications and variations as fall within the spirit and scope of the appended claims and equivalents.