Method and apparatus for testing sleepiness

[0001] This application is a continuation of U.S. application Ser. No. 10/442,152, filed May 21, 2006, allowed, which claims the benefit of Australian Patent Application No. PS 2543, filed May 24, 2002, each incorporated herein by reference in its entirety.

[0002] The invention relates to a method and apparatus which allows an objective measure of sleepiness to be authenticated.

[0003] Excessive sleepiness may result from several causes one of which is Sleep Disordered Breathing (SDB). Excessive sleepiness may be particularly dangerous for people who have to operate equipment such as cars, trucks, buses or other machinery. There have been a number of fatal accidents recently where truck drivers have fallen asleep at the wheel. It has been subsequently established that the drivers suffered from sleep apnea. Fatigue in general has been identified as a cause of vehicle accidents. It has been suggested that sleepiness or wakefulness tests would be a requirement for drivers before they are allowed to start a long drive or periodically in order to renew their licence, particularly those who have been diagnosed with sleep apnea.

[0004] There are a number of tests to measure sleepiness such as the Epworth Sleepiness scale tests, Vigilance tests and Pupillometry.

[0005] The Epworth Sleepiness Scale test is based on a series of questions which are asked of the patient. In some forms, the questions may be posed by an administrator who is physically present while the patient answers. In other forms, the questions may be remotely posed, for example, for self assessment via regular mail, the internet or via an embedded device such as the Health Buddy manufactured by Health Hero.

[0006] Vigilance tests measure a patient's reaction time and cognitive alertness, which are generally understood to be indicative of a patient's sleep propensity. This type of test is typically administered using a personal computer (PC) and requires that the patient provide responses to displayed indicia via the input devices associated with the PC. The patient's reaction time in providing the response and/or the accuracy of the response to the displayed indicia are measured and stored to determine the patient's level of alertness. For example, the patient is shown a recognizable object on the PC display, and the patient's reaction time in identifying the object and the accuracy of the identification are measured. One form of vigilance test is to have the person perform a repetitive task, such as clicking a button with a finger upon a signal and to monitor the reaction time or accuracy with which the person clicks the button when signaled to do so.

[0007] One form of wakefulness test using a finger-based test is described in “Microsleep during a Simplified Maintenance of Wakefulness Test: A Validation Study of the OSLER Test” by Priest et al. in Am. J. Respir Crit Care Med Volume 163, pp 16190-1625, 2001.

[0008] One form of fingerprint identification unit is the “Sony Fingerprint Identification Unit” FIU-710.

[0009] Pupillometry is based on the observation that the pupils of sleepy individuals display measurable characteristics which are significantly different to the pupils of people who are not sleepy. During pupillometry recording in the dark, the pupils of behaviorally sleepy individuals oscillate widely in size. This phenomenon is named pupillary “fatigue waves”, and is not observed in people who are not sleepy. Low-frequency components are dominant in persons with excessive daytime sleepiness. The amplitude of slow oscillations (typically less than 0.5 Hz) can reach several millimeters. It has also been observed that the pupils of sleepy people became increasingly miotic (decreased in size) with ensuing sleepiness. In contrast the pupils of people who are behaviorally alert maintain a stable size. The phenomenon has been observed by Lowenstein and colleagues and reported in an article entitled “Pupillary movements during acute and chronic fatigue” published in Investigative Opthalmology, 2, 138-157 (1963).

[0010] In the article “Pupillographic Assessment of Sleepiness in Sleep-deprived Healthy Subjects” in the SLEEP journal, volume 32, no. 3, 1998 by Wilhelm et al., a method of providing sleepiness measures is described. The contents of this article are hereby incorporated by reference.

[0011] In this specification, the term “sleepiness test” is intended to include somnolence and wakefulness tests.

[0012] A potential difficulty with the use of these tests in certain applications is that of authenticating the person who is being tested. This may be critical in applications where the person may be required to operate potentially dangerous equipment, such as truck driving, where there may be an incentive for the person to work in spite of being tired. For example, if the Epworth Sleepiness Scale test were to be administered remotely, the person may ask another, presumably non-tired, non-sleepy or more wakeful person to take the test instead and there may be no way to verify who has taken the test. An automatic system for sleepiness testing may be capable of correctly identifying whether or not a person is sufficiently alert to drive, or to undertake a particular task such as operating machinery, however, the system may be defeated if a substitute person takes the sleepiness test.

[0013] Similarly sleepiness testing may be administered on a periodic basis to determine whether a user has been compliant with a medical treatment for their condition. A determination of compliance can be useful as part of management of their condition. A user's compliance with medical treatment may also be a precondition for the grant or maintenance of a permit such as a driver's licence. Therefore it would be desirable for a sleepiness test to be conducted on a periodic basis in a location convenient to the user and preferably without the need of attendance by a skilled operator. Ideally the sleepiness test will be capable of self administration by the user.

[0014] It is one aspect of the invention to overcome problems with prior art sleepiness tests.

[0015] One aspect of the present invention is to provide a method and apparatus whereby a sleepiness test is conducted on a person substantially simultaneously with an authentication technique on the person using the same or a similar physical feature of the person.

[0016] Another aspect of the invention is to provide a method and apparatus whereby a scan is performed on an eye of a person to calculate eye scan data and both an authentication step and a sleepiness measure are calculated on the eye scan data.

[0017] In another aspect of the invention, sleepiness is measured using pupillometry and authentication is performed using an iris scan of the same eye. In yet another aspect of the invention, vigilance is measured using a repetitive finger-based cognitive test, while finger-print scanning is performed on the same finger.

[0018] Another aspect of the invention is that the sleepiness and authentication measurements are performed on physically similar features to reduce the likelihood that the sleepiness test could be administered to one person while the authentication is performed on another person.

[0019] Another aspect of the invention provides that the sleepiness and authentication measurements are performed either simultaneously, or in rapid succession, for example, with micro- or milli-seconds between measurements.

[0024] FIG. 1 illustrates one embodiment of an apparatus 5 structured to conduct a sleepiness test and an authentication technique on a person. The apparatus 5 comprises a digital camera 10 connected to a computer 20 that is structured to substantially simultaneously conduct pupillometry for the sleepiness test and iris scan identification for the authentication technique. In one form, “substantially simultaneously” means within a time period of small enough magnitude which would make it physically impossible/difficult to substitute the tested physical feature between the authentication step and the sleepiness testing step.

[0025] In the illustrated embodiment, the computer 20 includes an authentication software module 30, a sleepiness software module 40, and a certification software module 50. The digital camera 10 may be a monochrome camera with a CCD with a resolution of 480×640 pixels to provide the scan. A low light level is used. The camera 10 scans at a frequency of approximately 30 f.p.s., hence images of the eye are taken within 1/30th of a second of one another. In one embodiment, the person to be tested is provided with the visual feedback on a separate monitor of the camera image and instructed to position themselves so as to present an eye 15 to the camera 10. The image is automatically checked to ensure that it is in focus. If it is in focus, then the position of the iris within the image data is automatically determined. The same frame of iris data is then sent to both the authentication software module 30 and the sleepiness software module 40.

[0026] Generally, light wavelengths in the visible spectrum (400 nm to 750 nm) and portions of the near-infrared spectrum up to 1100 nm are able to be detected by a CCD based image sensor. In one embodiment, the camera for taking images of the eye may be the ToUCam Pro PCVC740K manufactured by Philips Electronics. This camera uses a CCD imaging sensor. The camera is connected to a personal computer via a USB cable. The images from the camera are stored as bitmaps.

[0027] In the embodiment shown in FIG. 2, a camera 10 is mounted on a frame 12 which supports the camera and makes it wearable. A hood 14 is provided which projects from the camera and which serves to isolate the eye 15 from incident light. In this way the amount of light which falls on the eye can be controlled. If there is too much light falling on the eye, it may be difficult to monitor sleepiness. However, if there is too little light falling on the eye, the pupil may dilate to such a degree that it may be difficult to scan the iris of the eye for authentication.

[0028] The hood defines a chamber 16. The hood has a soft, comfortable edge 18 which in use is positioned against the face of the person 22. The soft edge 18 of the hood 14 may be constructed from a material such as silicone. The chamber includes lights 24 positioned to illuminate the eye 15 at a predetermined light level for the eye to be scanned, but not so bright as to cause the iris to be too small for authentication. The lights are arranged to minimize light reflections in the scanned image. The lights provide a diffuse light source so as to provide a soft, even illumination of the eye. In one embodiment, a diffuser 26 is positioned over the light.

[0029] In one embodiment, the lights may be infrared light emitting diodes, for example, the HE8807FL manufactured by Hitachi Ltd.

[0030] In one embodiment of the invention, scans are taken at different lighting levels. For example, a low level of illumination is used for a first series of scans and then a second, higher level of illumination is used for a second series of scans. In another embodiment, the high level of illumination is used before the low level of illumination.

[0031] When the hood 14 first covers the eye, it may take some time, for example 3 minutes, for the pupil to adapt to the low level of lighting which is provided by the lights. During this adaptation time, a series of scans of the eye are made to monitor the adaptation. In this way the apparatus can determine when the eye is ready to perform the authenticated sleepiness measurements.

[0032] FIG. 3 shows an eye 15. The iris 38 has been automatically detected and recognized within the image. In the authentication software module 30, each isolated iris pattern is then demodulated to extract its phase information using quadrature 2D Gabor wavelets, for example, as described in U.S. Pat. No. 5,291,560 (Daugman), the contents of which are hereby incorporated by reference. This technique provides a distinctive 256-byte code which may be used to authenticate the iris of the person.

[0033] As shown in FIG. 3, the inner 36 and outer 34 edges of the iris 38 are automatically detected and recognized within the image of the eye. In the sleepiness module 40, the size of the pupil is measured and recorded with respect to time. An index of sleepiness is derived from the size of the pupil and oscillations in pupil diameter. In a sleepy person, there are relatively large changes in pupil diameter. In one embodiment, the pupil diameter versus time data is transformed into a frequency versus power form via a Fast Fourier transform. Sleepy individuals exhibit peak powers at characteristic frequencies such as 1 Hz. A sleepiness index may be derived from the total power of signals in the region of 1 Hz.

[0034] FIG. 4 shows a series of four images of an eye. In the different images (a), (b), (c) & (d), the pupil has different sizes. The inner and outer edges of the iris are automatically detected and the rate of change of the size of the pupil is determined.

[0035] In one embodiment of the invention, the following algorithm may be performed:

• • [0036] Perform a first scan of the eye,
  • [0037] Identify features of the eye from the first scan,
  • [0038] Calculate a first code based-on the features identified from the first scan,
  • [0039] Measure the size of the pupil from the first scan;
  • [0040] Perform a second scan of the eye,
  • [0041] Identify features of the eye from the second scan,
  • [0042] Calculate a second code based on the features from the second scan,
  • [0043] Measure the size of the pupil from the second scan;
  • [0044] Compare the first and second codes to determine whether the eyes used in the first and second scans are the same, similar or different;
  • [0045] If the same eye has been confirmed to be used in both scans, calculate the rate of change of pupil size between the first and second scans.
  • [0046] The first and second scans may be interposed with other scans. A series of scans may be performed over a period of time, for example 10 minutes, in order to conduct accurate pupillometry measurements.

[0047] In one embodiment of the invention, the pupillometry and authentication steps may be performed in real time, that is, as soon as a scan is made of the eye, the images are sent to the software modules for calculations. In another embodiment of the invention, images of the eye may be time and date stamped and stored in a memory unit for later analysis.

[0048] In one embodiment of the invention, image analysis may be performed in the same location as the person being scanned. In another embodiment of the invention, the scanned images may be transmitted to a remote location and image analysis is conducted at the remote location.

[0049] In one embodiment of the invention, the results of the authentication process may be compared with a database. For example, iris scan or fingerprint scanning results may be compared to a database having names, addresses and other bibliographic information. In this way the identification of the person who is being scanned can be confirmed.

[0050] In another embodiment of the invention, sleepiness may be determined from blink rate. For example, as described in U.S. Pat. No. 5,570,698 or 6,097,295, the contents of both of which are hereby incorporated by reference.

[0051] In another embodiment of the invention, the sleepiness and authentication tests may be performed on a persons finger. Fingerprint scanning may be performed using a technique such as described in U.S. Pat. No. 6,270,011.

[0052] Once both the authentication test and sleepiness test have been performed, the results are sent to the certification software module 50. This module provides a signed certificate in physical or electronic form which certifies the result of the sleepiness test, matched to the authentication result. When provided in digital form, the data may be signed electronically using a technique such as public key cryptography as provided by the PGP programme by Philip Zimmerman. In one embodiment, the data may be emailed to a time stamping service such as that provided by the UK firm, IT Consult. In one embodiment, the camera may provide image data together with a serial number which identifies the imaging chip of the camera to the certification software modules. In one embodiment, the certificate may be sent to a database electronically via a network connection. In another embodiment, the authentication test may be cross checked with a database to add bibliographic information of the person performing the test to the certificate.

[0053] In one embodiment of the invention, an authentication step may be performed on the person at a first time, a measure of sleepiness is performed over a period of time and a test is performed to determine whether the person has moved from the test apparatus. For example, initially, the iris of the person is scanned and thereafter the sleepiness test is performed over time, no further iris scanning is performed during the sleepiness test, however, the eye being scanned is monitored to determine whether the eye moves away from the camera. If the eye moves away from the camera or there is a significant shift of the eye in the view of the camera, a new authentication step is performed on the iris of the person.

[0054] In one embodiment of the invention, the authentication and sleepiness tests may not be performed on exactly the same eye scan data, however, they may be performed on eye scan data which have been determined very close in time to one another. For example, a first scan may be conducted and an authentication step is performed on the data from the first scan. Soon after the first scan, a second scan is performed and the sleepiness test is performed on data from the second scan. This use of sequential scans may continue for the duration of the sleepiness test. The time delay between the first and second scans is short, for example less than a second, so as not to allow the sleepiness and authentication steps to be performed on different people.

[0055] In another embodiment of the invention, an additional authentication step may be used. For example, while sleepiness and authentication steps are being performed on an eye of a person, a fingerprint scan of the person may also be conducted. The results from both the iris scan and the fingerprint scan may be used to identify or characterize the person being tested.

[0056] Although the invention has been described with reference to a particular embodiment, it is to be understood that this embodiment is merely illustrative of the principles of the invention. Thus, it is to be understood that numerous modifications may be made in the illustrative embodiment of the invention and other arrangements may be devised without departing from the spirit and scope of the invention.