Apparatus and method for postural assessment while performing cognitive tasks

[0001] This Application claims the benefit of U.S. Provisional Patent Application Ser. No. 60/468,327, filed May 6, 2003.

[0002] The U.S. Government has a paid-up license in this invention and the right in limited circumstances to require the patent owner to license others on reasonable terms as provided for by the terms of Grant No. N00014-02-0808 of the Office of Naval Research.

[0003] 1. Field of the Invention

[0004] The present invention relates to an apparatus and method for analyzing postural changes to assess mental engagement during cognitive tasks.

[0005] 2. Description of the Related Art

[0006] Determining the degree of mental engagement in cognitive work is potentially critical for a wide range of tasks, including flying an airplane, operating a boat, air traffic control, operation of automobiles, and/or operation of unmanned robotic vehicles through virtual interfaces.

[0007] Present efforts towards analyzing cognitive engagement focus on identifying physiological measures from neural or physiological domains and attempting to correlate these measures to cognitive states. For example, EEG or pupillometry are highly susceptible to eye movement, blinking, and other sources of activity, in addition to the time between EEG recording and food intake. Accordingly, there is a need for an apparatus and method for measuring automatic motor behaviors to measure cognitive engagement.

[0008] Presently available means of measuring the position and orientation of sensors attached to various body parts, such as the head, arms, legs or torso, include magnetic positioning sensors such as those marketed by Ascension Technology Corp. under the trademark FLOCK OF BIRDS. The operation of such motion tracker is explained in U.S. Pat. Nos. 5,744,953 and 5,831,260. The algorithms for determining the position and orientation of the sensors with respect to a reference sensor are best explained in U.S. Pat. Nos. 4,849,692, 4,945,305, and 6,172,499.

[0009] Presently available position and orientation measuring means using ultrasonic positional sensors include those marketed by Logitech, Inc., and are described in U.S. Pat. No. 5,142,506. Other ultrasonic position measuring devices are disclosed in U.S. Pat. Nos. 5,450,752 and 5,591,914.

[0010] Although there are presently available means for monitoring the position and orientation of various body parts, there is a need for correlating these measurements with cognitive engagement so that they may be used as a measure of cognitive engagement during the performance of cognitive work. Additionally, there is a need to perform these measurements upon seated personnel during the performance of cognitive tasks.

[0011] The above-described needs and others are met by the present invention, which provides an apparatus and method for dynamic postural assessment of seated personnel.

[0012] The present invention provides a chair equipped with a plurality of sensors for determining changes in posture. For example, these sensors may include pressure sensors within the seat and back pad of the chair. Additionally, these sensors may include magnetic positioning sensors on the chair itself and worn by person seated on the chair, for example, on the head or torso. These positioning sensors may be, for example, magnetic or ultrasound positioning sensors. Additional sensors including motion sensors, possibly having accelerometers and/or rate sensors therein, may be mounted within the seat itself to monitor movement of the vehicle within which the seat is mounted. Lastly, the apparatus includes a computer for monitoring the signals collected by the sensors and the workload of the person seated in the chair, and to perform the postural assessment.

[0013] The method of postural assessment begins by recording the initial readings of the various pressure and/or position sensors. As the seated person performs cognitive tasks, the position of the head and/or torso, the position of the seat, and/or the pressure exerted on the seat and back pad, may all be measured. These measurements may then be compared with the workload of the individual at that point in time, along with the movement of any vehicle within which the individual may be seated, and analyzed to determine the level of cognitive engagement of the individual. An example of postural indicators of cognitive engagement include movement of the head towards a visual display corresponding to an increasing number of tasks presented by the visual display. Alternatively, an individual may leave his head relatively close to the visual display to maintain vigilance as the workload presented by the visual display decreases, until the individual is satisfied that the level of work is tapering off. The individual may also alter his posture in response to movements of the vehicle within which the chair is located, for example, compensating for the rhythmic movements of a boat or the angle of flight of an airplane or helicopter. Furthermore, the pressure exerted by the individual upon the chair will indicate the level of cognitive engagement, with a large pressure concentrated over a small area indicating a braced posture and high level of mental engagement, and a lower pressure distributed over a larger area being indicative of a relaxed posture and lowered cognitive engagement. A high standard deviation of the rate of change in pressure between various sensors at a given point in time is indicative of changes in posture that may indicate fidgeting (impatience), fatigue and a need to relieve personnel. As another alternative, the center of pressure applied by the right and left buttock may be determined, and the movements of these centers of pressure may be analyzed in the same manner as head movement, with forward movement of the centers of pressure indicating torso and head movement towards a visual display of a workload. Again, changes in pressure from one side to the other that deviate from the known patterns of the individual, or the movement characteristics of the vehicle in which the person is sitting, may indicate fatigue.

[0014] The present invention is therefore useful for a wide variety of individuals who perform tasks for which monitoring cognitive engagement is important. Examples include pilots, those working aboard a ship, vehicle drivers, air traffic controllers, and others who perform their work while seated. Upon detection of posture changes indicative of fatigue, additional stimulus may be provided to increase cognitive engagement, or personnel may be relieved if performance degrades below acceptable levels. Additionally, the present invention provides a means of assessing the balance function in medical patients who are incapable of standing for prolonged periods of testing.

[0015] It is therefore an object of the present invention to provide an apparatus for dynamic postural assessment of seated individuals during performance of cognitive tasks.

[0016] It is another object of the present invention to provide a method of measuring mental engagement of seated individuals during the performance of cognitive tasks.

[0017] It is a further object of the present invention to provide a means of dynamic measurement of postural changes of seated individuals, and correlating these postural changes with various levels of cognitive engagement.

[0018] It is another object of the present invention to provide a means for assessing the balance function in medical patients who are incapable of standing for prolonged periods of time.

[0019] It is a further object of the present invention to provide a determination of when to provide additional stimulus to personnel performing seated cognitive work in order to increase mental engagement.

[0020] It is another object of the present invention to provide a means for determining when seated personnel performing cognitive work need to be relieved due to fatigue or impatience.

[0021] These and other objects of the invention will become more apparent through the following description and drawings.

[0040] Like reference characters denote like elements throughout the drawings.

[0041] The present invention provides an apparatus and method for measuring postural changes of a seated individual during the performance of cognitive tasks, and correlating these postural changes with the individual's level of mental engagement with the tasks.

[0042] Referring to FIGS. 1–3, a postural assessment chair 10 is illustrated. The chair 10 includes a seat 12 and a back 14 each of which includes a pad 16, 18 having a plurality of pressure sensors therein. One preferred pressure sensitive pad is the FSA pad manufactured by Vista Medical. These pads 16, 18 each contain 256 sensors in a 16 by 16 pressure sensor array. These sensors typically operate at a rate of 4.5 Hz.

[0043] The chair 10 also may includes means for measuring the position and orientation of various body parts of the individual 20 seated within the chair 10, for example, the person's head 22, and/or chest 24. For this purpose, sensors marketed by Ascension Technology Corp. under the trademark FLOCK OF BIRDS may be used. A transmitter 26 is mounted on the chair 10, with a preferred location being near the top 28 of the back 14. A reference sensor 30 is mounted on the back 14. A head sensor 32 is worn by the person 20 sitting in the chair, possibly on headband 34, or alternatively within any helmet that may be worn by the individual performing the task for which measurement is desired. A chest sensor 36 is also worn by the person 20, possibly by being clipped to the person's shirt 38, or alternatively installed within a shoulder harness of a seat belt, if one is typically worn by the individual during performance of the task. Lastly, a 6 degree-of-freedom movement sensor having linear and angular accelerometers is mounted to the seat 12 of the chair 10. The operation of the transmitter 26, reference sensor 30, head sensor 32, and chest sensor 36 is well known to those skilled in the art of magnetic positioning and orientation sensors. Example methods of operation are described in U.S. Pat. Nos. 5,744,953 and 5,831,260, expressly incorporated herein by reference. Movement of the head sensor 32 and chest sensor 36 with respect to the reference sensor 30, may be detected as a combination of any of six different movements: forward and backward translation, side to side translation, upward and downward translation, rotation about a forward-backward axis, rotation about a side to side axis, and rotation about a vertical axis. The movement of each sensor may be calculated according to the signal received by the algorithms described in U.S. Pat. Nos. 4,849,692, 4,945,305, or 6,172,499, all of which are also expressly incorporated herein by reference.

[0044] Alternatively, an ultrasonic position sensor may be used. A preferred ultrasonic head tracker will have six degrees of freedom (both translation along and rotation around the x, y, and z axes) and may in some embodiments have a sampling rate of about 50 Hz. Such a sensor is presently offered by Logitech, Inc. of Fremont, Calif., on an exclusive marketing arrangement with VR Depot (www.vrdepot.com), and is disclosed in U.S. Pat. No. 5,142,506, which is expressly incorporated by reference. Referring to FIG. 18, a receiver 33, having one of the three microphones 25,27,29 at each apex of the triangle, may be worn on the head using a headband or any helmet typically worn while performing the task, and/or secured to the chest using a clip or any seatbelt typically worn during the task. An ultrasonic transmitter array 35, having one of the three speakers 37,39,41 at each apex of the triangle, may be mounted to a computer display or other fixed object in the workspace. The time required for the signal to travel from each speaker 37,39,41 to each microphone 25,27,29 can be used to calculate the position and orientation of the head and/or torso using well-known trigonometric principles.

[0045] FIGS. 6–7 explain the basic method of using the chair 10 to monitor the posture of the person 20, and thereby monitor his level of mental engagement. Referring to FIGS. 6A–6B, the operation of the head sensor and its correlation to cognitive engagement are illustrated. Initially, the starting head position is received and recorded at 40. At each time increment, the head position at that time is recorded at 42. Referring briefly to FIGS. 8 and 9, an expected head position for various tasks is illustrated. Typically, the head will move from its rest position towards a visual stimulus, represented here by the computer monitor 44 (FIG. 1), or the implements used to perform the work, represented here by the keyboard 46 and mouse 48. FIGS. 8 and 9 were generated during an exercise wherein a person 20 played a video game simulating a warship commander task (Warship Commander, Pacific Science and Engineering, San Diego, Calif.). The task included monitoring planes displayed on the monitor 44, and using the mouse 48 to interrogate each plane to determine if it was a friend or foe, warn each foe that crossed a defensive perimeter, also using the mouse, and lastly using the mouse to shoot down any planes identified as a foe and remaining within the defensive perimeter three seconds after the warning. FIG. 8 represents a task involving six planes, and FIG. 9 represents a task involving 24 planes. The actual head movement is illustrated by the curves 50, 52, respectively. Typically, the head 22 will move towards the monitor 44 more rapidly at first, and then less rapidly, until a comfortable head position is reached for either performing the tasks or maintaining vigilance. This head movement can be represented by fitting the actual head movement points to a curve using least squares methodology, described in Richard L. Burden and J. Douglas Faires Numerical Analysis, Fourth Edition, 427–49 (1989), expressly incorporated herein by reference. This step is performed at step 54 in FIG. 6A, by the computer 43 (FIGS. 1 and 18) with the resulting curves illustrated as 56 in FIG. 8 and 58 in FIG. 9.

[0046] The head movement may be further analyzed by looking at an exponential response to the head movement with respect to time, with exponential response=((head movement magnitude) X exp(−time/20))−(initial head position). This calculation is performed at step 60 within FIG. 6A, with the resulting curve illustrated in FIG. 10. This curve may be further broken down into two separate elements: the curve itself as represented in FIG. 11, and the error between the actual data point and the approximated curve as shown in FIG. 12. By calculating the amount of the error, or the root means square of the exponential response, at step 62 in FIG. 6A, and then dividing the magnitude by the root means square at step 64, the possible degree of error in the head movement is shown. If this error is too high as determined at step 66 in FIG. 6B, this may be an indication that either additional stimulus or relief may be needed for the person 20, as provided at step 68 in FIG. 6B.

[0047] An additional comparison can be made between the head movement and the amount of work remaining at the point in time when head movement is measured. The level of remaining work is recorded at step 70 in FIG. 6A. A linear regression may then be performed between the head position and number of queued work tasks at each time increment at step 72. This correlation is calculated over time windows (intervals) that are task-specific, typically in the range of 6–25 seconds. A correlation coefficient of this linear regression may be calculated at step 74 in FIG. 6B. Referring to FIGS. 13a–b and 14a–b, the relationship between head position and tasks on the computer monitor is illustrated. Within FIG. 13a, head position is represented by curve 76, and work remaining is illustrated by curve 78. Likewise, head position is represented by curve 80 in FIG. 14a, and work remaining is represented by curve 82. As can be seen from these figures, the head will move closer to the monitor 44 as increasing number of tasks are displayed on the monitor 44, until a comfortable head position is reached. The head will remain within this comfortable position, possibly with one adjustment at points 84, 86, after which time the head will settle into a comfortable position for maintaining vigilance even as work remaining decreases, until the person 20 is satisfied that no additional tasks remain, at which point the head will return to its at-rest position. Turning to the bottom portions of FIGS. 13 and 14, showing head position as a function of work remaining, the linear relationship between the two becomes apparent, with an increasing number of targets on the screen resulting in increasing forward motion of the head towards the screen. Therefore, a good fit between head position and work remaining at each time interval, determined at step 88 in FIG. 6B, indicates that the person 20 is mentally engaged in the task, and no intervention is needed, as shown in step 90. However, if there is not a good fit between head position and work remaining, a further inquiry needs to be made to determine if the person 20 has adopted a posture of vigilance, as indicated by a high magnitude of forward positioning of the head, determined at step 94. If the head position indicates that the person 20 is being vigilant, no intervention is needed. If not, then additional stimulus may be needed to prevent boredom, or the person 20 may need to be relieved.

[0048] FIG. 7 illustrates the method of determining the level of mental engagement using seat pressure. The expected pressure across the seat 12 and back 14 given a high cognitive workload is illustrated in FIG. 15. As this figure illustrates, there will be a relatively high pressure distributed over a smaller area of the seat and back. Additionally, the body will be braced, resulting in only small changes within the pressure distribution. Conversely, FIG. 16 illustrates the expected pressure distribution across the seat 12 and back 14 for a low cognitive workload. The more relaxed posture will result in lower pressure distributed across a wider are of the seat 12 and back 14. Furthermore, with a low cognitive workload, or high fatigue, one would expect more loosely constrained body linkages, or more frequent adjustments in posture, resulting in greater changes in the pressure distribution. Referring back to FIG. 7, the initial back and seat pressure at rest is first determined at step 94. At each time interval, the pressure at each back and seat sensor is read at step 96. The rate of change of pressure at that point in time is calculated at step 98 for each sensor. A method of calculating this rate of change is described in Burden, Numerical Analysis, at 146–63, expressly incorporated herein by reference. At this point, the standard deviation of the change in pressure at all sensors at the time increment in question may be calculated at step 100. It is this standard deviation that determines whether there is a likelihood that the person 20 is experiencing fatigue, which will be indicated by large changes in pressure, which will result in a large standard deviation. This comparison is made at step 102. For the above-described warship commander task, it has been found that a standard deviation of greater than 0.2 indicates that there is an 80% likelihood that the person 20 is fatigued or perceiving a reduced work load, indicating that intervention may be needed at step 104 in the form of either task modification to prevent boredom, shift of workload to other operators or relief by a new operator. Otherwise, no intervention is needed at step 106.

[0049] FIGS. 17A–17B explain an alternative method of using the chair 10 to monitor the posture of the person 20, and thereby monitor his level of mental engagement. FIGS. 17A–17B illustrate the use of the seat sensors to measure the movement of the center of pressure applied to the seat by the right and left buttocks and thighs, and the correlation to cognitive engagement. Initially, after reading the pressure at each sensor at step 108, the midline is established at step 110, and the starting center of pressure for the right and left buttock and thigh is read and recorded at 112,114, respectively. At each time increment, the position of the right and left center of pressure at that time is recorded at 116,118. Typically, each center of pressure will move forward, as the head moves from its rest position towards a visual stimulus. Referring briefly to FIG. 8, the head 22 will typically move towards the monitor 44 more rapidly at first, and then less rapidly, until a comfortable head position is reached for either performing the tasks or maintaining vigilance. This head movement will be matched by a movement in the center of pressure applied to the seat by the right and left buttocks. This movement of the centers of pressure can be represented by fitting the actual head movement points to a curve using least squares methodology, described in Richard L. Burden and J. Douglas Faires Numerical Analysis, Fourth Edition, 427–49 (1989), expressly incorporated herein by reference. This step is performed at steps 120,122 in FIG. 17A.

[0050] The movement of the right and left centers of pressure may be further analyzed by looking at an exponential response to this movement with respect to time, with exponential response=((center of pressure movement magnitude) X exp(−time/20))−(initial center of pressure). This calculation is performed at steps 124,126 within FIG. 17A. The resulting curve may be further broken down into two separate elements: the curve itself, and the error between the actual data point and the approximated curve. By calculating the amount of the error, or the root means square of the exponential response, at step 128, and then dividing the magnitude by the root means square at steps 130,131, the possible degree of error in the movement in the right and left center of pressure is determined. If this error is too high as determined at step 132 in FIG. 17B, this may be an indication that either additional stimulus or relief may be needed for the person 20, as provided at step 134 in FIG. 17B.

[0051] An additional comparison can be made between the movement of the right and left center of pressure and the number of tasks in the work queue at the point in time when these movements are measured. The level of remaining work is recorded at step 136 in FIG. 17A. A linear regression may then be performed between the right and left center of pressure and number of tasks in the work queue at each time increment at steps 138,140. A correlation coefficient of this linear regression may be calculated over a task-specific time window (typically 6–25 seconds) at step 142 in FIG. 17B. A good fit between the position of the right and left center of pressure and work remaining at each time interval indicates that the person 20 is mentally engaged in the task, and no intervention is needed, as shown in step 144. However, if there is not a good fit between the right and left center of pressure and work remaining, a further inquiry needs to be made to determine if the person 20 has adopted a posture typical of vigilance for that individual, or if the individual is reacting properly to movements of a vehicle within which the person is sitting, determined at step 146. If the position of the right and left centers of pressure indicate that the person 20 is shifting to the right and left appropriately, no intervention is needed. If not, then additional stimulus may be needed to prevent boredom, or the person 20 may need to be relieved.

[0052] The correlation between head position, seat position, and cognitive engagement was tested using a “warship commander” simulation wherein the test subject was presented with up to six, or up to twenty-four, “target” airplanes on a monitor, and was required to select individual planes, inquire about their identity, warn “foe” planes away, and destroy “foe” planes that get too close. The approach developed in the Warship Commander Task was then validated in a similar Aegis command simulation task. This task differed from the Warship Commander Task in several important ways: the appearance of new targets was uncued, the targets were embedded in a more complex display and the average latency for the appearance of new targets was longer (12 seconds versus 2 seconds). The operator was required to perform three sequential types of actions, (1) hook a target, (2) select a target and (3) identify a target, with a decision process at each step whether to continue with the sequence or hook a new target. These decisions were based upon a set of priority rules. The maximum values of the correlation coefficients head position and seat center of position with the number of targets were calculated over a 12 second window, and were designated the overall engagement gauge. The quantitative performance of the subject in the task (scores based upon correct decisions) was correlated highly with the proportion of the time that the overall engagement gauge exceeded 0.5 (meaning that 50% of the center of seat position or head position changes relative to the monitor were explained by the number of targets on the screen), which supports the role of the gauge as a predictor of cognitive engagement. When at least 50% of head or seat position shifts are explained by the number of targets on the monitor, the probability of the next action occurring within 2 seconds was found to be 0.616, the probability of action within 3 seconds was 0.766, the probability of action within 4 seconds was 0.860, and the probability of action within 5 seconds was 0.946. Therefore, this device and algorithm provide a high degree of predictability of an operator's activity within the ensuing few seconds. Therefore, the present invention provides a means of monitoring changes in posture that may be indicative the degree of cognitive engagement on the part of a seated person performing cognitive work. Furthermore, changes in posture may be compared to movement of the vehicle itself. For example, if the radar operator shown in FIG. 5, on the boat in FIG. 4, is fully cognitively engaged, his changes in posture will be such that he has predicted the movements of the boat. By comparing the movement sensor in the chair to the head and/or torso sensor, or pressure pad sensor, a determination can be made as to whether the radar operator is predicting or reacting to the motions of the boat. If he is reacting to the motions of the boat, fatigue is indicated. As another example, comparing the postural changes of a helicopter pilot to the movements of the helicopter itself may indicate that the pilot has lost his sense of the correct direction of “up”, and that therefore certain maneuvers of the helicopter should be prohibited. Other examples of cognitive work performed while seated for which there is a need to determine boredom and/or fatigue, and provide appropriate stimulus or relief, include airplane pilot, air traffic controller, vehicle driver, boat operator, etc. Lastly, by mounting the chair on a moveable platform, the postural assessment chair 10 may be used to test postural function in patients who are unable to stand for conventional testing.

[0053] While a specific embodiment of the invention has been described in detail, it will be appreciated by those skilled in the art that various modifications and alternatives to those details could be developed in light of the overall teachings of the disclosure. Accordingly, the particular arrangements disclosed are meant to be illustrative only and not limiting as to the scope of the invention which is to be given the full breadth of the appended claims and any and all equivalents thereof.