Surveillance System Having a Voice Verification Unit

This invention relates to a surveillance system. More particularly, this invention relates to a surveillance system which integrates voice identification with a passive monitoring mechanism.

Verifying compliance with the terms of a probation, parole, or a release is a difficult and onerous task for the authorities charged with that responsibility. In the past, periodic meetings and telephone conversations were usually the only mechanisms which could be utilized to locate and supervise individuals convicted of criminal acts who were not institutionally confined or incarcerated. However, recent technical innovations coupled with new criminal justice procedures have introduced alternative sentencing mechanisms. In particular, home confinement systems have been recently developed that permit a convicted individual to be restricted to his home, or a designated locality, to serve out a term of sentencing. Such systems have also been used to monitor the whereabouts of parolees and probatees whose freedom to travel has been restricted.

However, most of the presently available home confinement systems are based on radio frequency (RF) communications. In an exemplary RF system, an individual under surveillance carries an anklet or bracelet radio transmitter which transmits continuously to a centrally-located radio receiving station which monitors the presence of the individual at a designated locality. The major drawback with such a system is that it is subject to outside interference which can be caused by many different sources, for example, house obstructions; electrical fields produced by electric motors, fluorescent lights, and the like: environmental conditions: and external radio signals. Further, if power transmission is raised to overcome an interference, an RF system may cause radio interference with third party devices, such as televisions or telephones, and possibly violate federal communication regulations. Second, RF systems are passive monitoring systems which do not provide a positive identification of an individual under surveillance but instead rely on the security of the anklet/bracelet connection. In addition, presently available RF systems lack the capability to immediately retest the presence of an individual in the event the system cannot detect the individual's presence. Therefore, in the case of a system failure or interference, an RF system may set off a false alarm.

Active monitoring systems that carry out voice verification tests over standard telephone lines have also been developed. Although these systems may not experience the aforementioned problems of RF systems, they nonetheless are subject to various technical problems. For example, the standard telephone handset is designed to transmit only the necessary frequencies and amplitudes associated with intelligible voice transmission and not all the characteristics of speech needed for voice verification. In addition, the standard telephone handset inherently has a limited dynamic range that will introduce severe harmonic distortion outside its dynamic range and, thus, can cause false voice recognition. Further, ordinary telephone lines used in such systems will introduce additional distortion because of their limited dynamic range and noise that regularly appears on telephone lines (e.g., background hiss, ringing, cross talk, and echoes). Devices and techniques used to overcome these limitations can be costly to incorporate into an active monitoring system and may degrade a system's performance.

Briefly, the invention provides a surveillance system having a central station which stores data regarding the status of each individual under surveillance and a plurality of remote stations, each of which is located at a designated locality for a respective individual. The central station has an input to receive data and an output to transmit data and command signals to the remote stations. Each remote station is in the form of a voice verification unit having a first input to receive data and command signals from the central station, inputs to receive active and passive monitoring signals from a respective individual, test means for analyzing the active and passive monitoring signals according to the data and command signals received from the central station, and an output to transmit the analysis results from the test means to the central station. In addition, the surveillance system includes means for selectively connecting the central station with each of the remote stations.

Each remote station can include an electroacoustic transducer connected to a second input for receiving as an active signal a preselected series of spoken words by a respective individual. Each remote ,station can also include a receiver element connected to a third input for receiving as a passive signal a continuous signal transmitted from a respective individual indicating the individual's presence at a designated locality. The test means can also cooperate with a signalling element which emits a signal to prompt the respective individual to speak the preselected series of words into the electroacoustic transducer.

For a better understanding of the invention, reference is made to the following description of an exemplary embodiment thereof, and to the accompanying drawings, wherein:

FIG. 1 is a block diagram of a surveillance system constructed in accordance with the invention; and

FIG. 2 is a block diagram of a single voice verification unit connected to a central station of the surveillance system of FIG. 1.

Referring to FIG. 1, the surveillance system 10 comprises a central station 12, located at the appropriate supervising authority, such as the local probation department or police precinct, and a plurality of remote stations in the form of voice verification units (VVU) 13, each of which is located at a designated locality, or localities, for a respective individual under surveillance, such as an individual's home, a place of employment, or a rehabilitation clinic.

The central station 12 maintains and analyzes all relevant data for each individual under surveillance and initializes and retrieves information from each VVU 13. The central station 12 comprises a control computer 14 and a respective data storage unit 14a, 15a, such as a magnetic disk drive, and a respective output display device 14b, 15b, such as a printer, a CRT, a CRT-printer combination, etc. The control o computer 14 is connected to an input of the central station 12 to receive data from the VVU's 13 and to an output of the central station 12 to transmit data and command signals used to initialize the VVU's 13. The violations computer 15 is connected only to the input of the central station 12 to receive data, in the form of test analysis results, from the VVU's 13.

The central station 12 is connected to each remote VVU 13 via an ordinary telephone line 17 and the local telephone company central office 18. If desired or warranted by the application, each VVU 13 may have a dedicated telephone line 17 to the central station 12 or to each computer 14, 15 therein. Further, the system 10 is not restricted to local telephone areas as long distance telephone lines and switching stations may also be utilized. Each telephone line 17 connects, at a respective designated locality, to a telephone modem 19 which can be tied to or incorporated within the respective VVU 13. The modem 19 can be an off-the-shelf product having, for example, a 1200/300 baud rating and autoanswer capability.

Each VVU 13 has a first input connected to the modem 19 to receive data and command signals from the central station 12 and inputs to receive active and passive monitoring signals from an individual under surveillance. Each VVU 13 has test means (not shown) connected via a second input to an associated microphone 22, or other electroacoustic transducer, which receives active monitoring signals in the form of a preselected series of spoken words by the individual. The microphone 22 can be configured as required by the application, for example, as a handset similar to those found in pay telephone stations, as a free-standing speaker, as a handset-speaker combination, etc. The test means is also connected via a third input to passive surveillance elements, as described below, to receive the passive monitoring signals. The test means (not shown) of the VVU 13 analyzes the active and passive monitoring signals according to a stored algorithm and certain data and command signals received from the central station 12. The test means is also connected via an output to a means for prompting, such as a signalling element which emits a signal to prompt an individual to speak a preselected series of words into the microphone 22. The signalling element can be, for example, a sounder 23, comprising a piezoelectric element, which emits an audible signal to prompt an individual to speak. Each VVU 13 also has an output connected to the modem 19 to transmit the analysis results from the test means to the central station 12.

A VVU 13 provides various functions on an ongoing basis. Primarily, a VVU 13 performs a voice verification test on the individual who answers the prompt signals from the sounder 23 and records the results internally. Upon initialization or actuation by the central station 12 at the start of a monitoring period, the VVU 13 will establish a test schedule and conduct the voice verification test according to the schedule. The test schedule is randomly created for each monitoring period, e.g., daily, and is in compliance with the terms of the respective sentence, parole or probation. In addition, the VVU 13 monitors the condition of the telephone line 17 as well as the power line which supplies the operating power for the unit. The status of each utility line for the monitoring period is recorded along with the test results. Further, the VVU 13 can, if desired, monitor certain external events that require immediate notification to the central station 12. For example, the VVU 13 can be connected, either directly or via a radio frequency mechanism, to sensors at the points of entry/exit 24 (i.e., doors, windows, garages, etc.) on the premises of a designated locality in those applications requiring house arrest of an individual. In the event of an entry/exit 24 being used, the VVU 13 can automatically transmit that information to the central station 12. Further, the VVU 13 can monitor its own housing and automatically inform the central station 12 of any tampering with the housing. This can be implemented as indicated in FIG. 2 for example by way of a detecting means such as a tamper switch element 28 for monitoring the security of the VVU by transmitting data to the central station 12.

The system 10 also contains continuous passive surveillance elements which are integrated with the active voice verification elements. In particular, the system 10 also comprises a signal transmitter 26 which can be conveniently configured as an anklet or bracelet and which is securely attached to the individual under surveillance. The anklet/bracelet transmitter 26 continuously transmits a signal, such as a radio frequency signal, to a receiver 27 connected to the third input of the respective VVU 13 at the designated locality. Any interruption or disruption of the signal will cause the receiver 27 to actuate the VVU 13 to conduct an unscheduled voice verification test if the VVU is programmed to do so. In addition, the receiver 27 detects any tampering with the transmitter 26 and immediately informs the central station 12 via the VVU 13. The transmitter 26-receiver 27 combination can be an off-the-shelf product.

The central station 12 may also be connected to the modem 19 at the designated localities via at least one telecommunications station 29 as provided by the local telephone company or other telecommunications entity. The telecommunications station 29 represents an alternative telecommunications technology to eliminate the risk of a telephone line disruption or failure between the central station 12 and a remote VVU 13. For example, the technology may utilize cellular telephones, long distance radio frequency transmission, microwave transmission, etc. Depending on the technology and the application, the telecommunications station 29 may or may not be situated at a designated locality. Also, the modem 19 may not be necessary for the operation of the system 10 if the VVU 13 can communicate with the telecommunications station 29 without an intermediate device.

Referring to FIG. 2, each VVU 13 comprises four integrated circuit components: a voice verification board 32, a support board 33, a power supply board 34, and a microprocessor 37 with the voice verification board 32 and the support board 33 constituting the test means.

The voice verification board 32 acts as a means for conducting voice verification tests of an individual according to a test schedule and contains test element circuitry which implements speech recognition/verification functions to positively identify the individual under surveillance whose voice is used as an input signal to the circuitry. Exemplary circuitry is the K3 Speech Recognition integrated chip set manufactured by Nippon Electric Company, Ltd. of Tokyo, Japan. The Voice verification board 32 also contains certain input/output circuitry for the VVU 13. As indicated above, an input of the voice verification board 32 is connected to the associated microphone 22 for the VVU 13 and, thus, utilizes the voice, i.e., the transduced voice, of the individual under surveillance as an input signal. The transduced voice is analyzed according to the stored algorithm of the test element circuitry and the analysis results appear at an output of the voice verification board 32.

The support board 33 acts as a means for establishing a test schedule in response to a signal from the central station 12 and provides a range of ancillary functions for the VVU 13. For example, the support board 33 contains memory circuitry to store all VVU 13 activity, such as the various monitoring activities and the analysis results from the voice verification board 32 output, until the central station 12 requests the information and in the event of a disruption or failure with the modem 19 or communication channels between the VVU 13 and the central station 12. The capacity of the temporary storage is a function of the application and economics although a five day back-up is sufficient and cost-effective in most cases. The temporary storage also has a limited life so that if the contents are not removed within a prescribed time period then the contents can be written over by succeeding information. The support board 33 has additional memory circuitry that is used to contain code and instructions (e.g., EPROM code, etc.) and RAM necessary for the operation of the microprocessor 37. In addition, the support board 33 monitors the passive surveillance elements of the system 10, such as the transmitter 26-receiver 27 combination, the telephone line status, the power line status, the sensors at the points of entry/exit 24, and any tampering of the VVU 13 housing and of the transmitter 26. The support board 33 also contains the clock and calendar for the VVU 13. Note that the support board 33 can monitor additional sensors as required as well as key switches which can be used to issue local commands to the VVU 13.

The power supply board 34 provides regulated power to the components of the VVU 13 from a conventional power source, such as a standard convenience outlet 38. In the event of a power disruption or outage, the power supply board 34 connects an on-board, stand-by power source (not shown), such as a lithium battery, to operate the memory circuitry and the clock on the support board 33 and employs an off-board, stand-by power source 39, such as a standard 12 volt battery, to operate the remainder of the components of the VVU 13. The power requirements of the VVU 13 are sufficiently low that stand-by power can operate the VVU 13 for lengthy periods of time if necessary.

The microprocessor 37 controls the operation of the VVU 13. The microprocessor 37 comprises a mother board (not shown) which is the interface between the microprocessor 37 and the other components of the VVU 13, i.e., the voice verification board 32, the support board 33 and the power supply board 35. The microprocessor 37 also comprises a programmable logic element (not shown) that directs the overall operation of the VVU 13 in accordance with a software program

stored on the support board 33 and establishes a voice verification test schedule for the voice verification board 32 in accordance with data and command signals received from the central station 12. The microprocessor 37 further comprises input/output circuitry which controls the modem 19 to enable the VVU 13 to communicate with the central station 12.

A VVU 13 and the central station 12 communicate using different formats or information packets according to the information to be exchanged. An initialization packet is transmitted from the control computer 14 to the microprocessor 37 of each VVU 13 at the start of a monitoring period. The initialization packet can contain the phone number of the central station 12 used to report test violations or failures, a case number to identify a respective individual under surveillance, and the test schedule parameters which include the start and end times of a voice verification test and the number and interval of tests within the schedule. A data packet is transmitted back and forth between the control computer 14 and a microprocessor 37 at various times. The data packet contains date and time information as well as status information. A time packet is transmitted from the control computer 14 to a microprocessor 37 to synchronize the local clock and calendar; the packet only contains date and time information. A violation/service packet is transmitted by a VVU 13 to the violations computer 15 upon certain conditions, which depend on the application. For example, the packet can be transmitted upon the failure of a voice verification test by an individual, upon an unscheduled request by the central station 12, or upon a regularly scheduled request from the central station 12 along with other data. The violation/service packet contains the case number of the individual, time and date information, and status information. The status information in the data and violation/service packets can contain such information as the telephone line status (active/inactive), the power line status (active/inactive), the source of the test actuation (test schedule/radio frequency signal interruption), the test result, and test response (individual responded/individual did not respond).

In addition to the exchange of information packets, the control computer 14 also transmits commands to the microprocessor 37 of each VVU 13. As a safety feature, all commands require password verification prior to execution to prevent any tampering with a VVU 13. The current password is stored by the support board 33 and the verification of the password is performed by the microprocessor 37. The main commands of the control computer 14 direct a VVU 13 to a) receive an initialization packet using the modem protocol (in response, a VVU 13 generates a test schedule and begins processing voice verification tests); b) to transmit all voice verification records as data packets using the modem protocol; to transmit the violation/service packet; and d) to receive, or to transmit, all data including the initialization packet, schedule and verification records and to begin processing current password, performing local diagnostics, and receiving a time packet. Note that all information exchanged between a VVU and the central station 12, including data and commands, can be encrypted as a security measure.

Upon installation of a VVU 13 at a designated locality, the voice of an individual under surveillance must be enrolled with the VVU 13 so that the unit can recognize the voice during a subsequent voice verification test. If desired, an individual can be enrolled again at the start of each monitoring period or of selected periods. The enrollment process has two steps, level determination and training, which take into account four major factors affecting the verification: 1) the particular words chosen to test an individual, since people say words differently; 2) the number of times a word is trained; 3) the number of chances to say the word correctly; and 4) a security level value. Each of the above-mentioned factors are carefully chosen to maintain a low possibility of false verification and false rejection while allowing for the normal variation found in speech.

The first step of the enrollment process requires an individual to say a word a number of times into the microphone 22 until the test element circuitry on the voice verification board 32 detects a consistent speaking level. This level is stored in the support board 33 and used during future verification tests. The second step requires an individual to speak the words to be used during an actual voice verification test. As an example, three different words can be selected: A, B, and C. The VVU 13 can be programmed to train each of the words a number of times and, thus, allow for a greater variation in the individual's speaking manner. However, in such case, there is a trade off. If the number of times a word is trained is too large, the variation tolerance increases and the possibility of false verifications increases. Upon installation of a VVU 13, the number of times a word is trained should be greater than at the start of subsequent monitoring periods because of the unfamiliarity of an individual with the VVU 13. Once the individual's understands what is expected, the number of times a word is trained can be reduced for subsequent monitoring periods and, consequently, the possibility of false verification is reduced.

During the second step, an individual will speak the selected words in sequence two times the number of times a word is to be trained. The VVU 13 uses the first sequence of A, B, and C to create templates for the words which are stored in the support board 33 and used during a future voice verification test. The VVU 13 uses the second sequence of A, B, and C to check for proper recognition of the words just trained. This insures that the individual and the VVU 13 are communicating. In this example, if the words are to be trained four times, the result of the second step is the creation of a set of 12 templates, 4 templates for each of the three words.

During an actual voice verification test, the VVU 13 will prompt, via a signal, an individual to speak each of the words a number of times. The words are prompted in a random order to prevent the use of a simple tape recorder to cause a false verification. The number of times each word is to be spoken is determined by the number of tries plus one, the number of tries being a programmable parameter of the VVU 13. Increasing the number of tries allows the individual more chances to correctly speak the words for proper verification. However, as with the number of times each word is trained, increasing the number of tries also increases the chances of a false verification.

The additional time an individual must speak each word (the "plus one" mentioned above) is used by the VVU 13 to establish and maintain a dynamic set of templates. Hidden among the prompt signals used for Verification are prompt signals to train and create new templates. If an individual's identity is verified during a test, the VVU 13 replaces the oldest set of templates with the new templates. Establishing and maintaining a dynamic set of templates allows for normal voice variations over a period of time. For example, a cold could alter an individual's voice such that verification would be impossible. By establishing and maintaining a dynamic set of templates, the subtle changes in a voice at the onset of the cold, and as the cold subsides, would be incorporated by the VVU 13.

Finally, the last parameter affecting verification is the security level value which is arbitrarily established as having a base value, for example, 10. Increasing the value allows for a greater latitude in the way an individual speaks a particular word. Decreasing the value decreases the difference allowed between the previously trained word and the spoken word. Decreasing the value to zero would require an individual to speak the word exactly as it was spoken when trained.

In operation, after the installation of a VVU 13 and at the start of a monitoring period, the control computer 14 at the central station 12 places a telephone call to the VVU 13 at a designated locality for a respective individual under surveillance. The control computer 14 places the telephone call by using appropriate polling software although other means may be used depending on the application (e.g. an automatic calling system). The modem 19, with its auto-answer capability, answers the incoming telephone call received from the telephone line 17. Instantaneously, the control computer 14 at the central station 12 transmits a command signal to the VVU 13 (through the modem 19) to receive an initialization packet using the modem 19 protocol. The microprocessor 37 of the VVU 13 receives the command signal and performs a comparison of the password contained in the signal with the current password stored in the support board 33. Upon verification of the password, the microprocessor 37 executes the command and prepares for receipt of the packet.

The initialization packet is transmitted by the control computer 14 either with the command signal or immediately thereafter. Upon password verification, the packet is accepted by the microprocessor 37. In response to the packet, the microprocessor 37 randomly generates or establishes a voice verification test schedule using the parameters provided in the initialization packet, i.e., start time, test intervals, number of tests and end time. The control computer 14 also transmits a time packet to synchronize the clock and calendar on the support board 33. At the time of installation of a VVU 13, or when desired, the VVU 13 may be tested on site after the initialization process to insure that the regular testing routine can proceed.

At the start time of the test schedule, the voice verification board 32 will begin a voice verification test. The voice verification board 32 will prompt or summon the individual under surveillance to the VVU 13 by actuating the sounder 23 to emit a series of audible tones. The individual then speaks the test words into the microphone 22 in response to additional prompt signals actuated by the voice verification board 32. The voice verification board 32 uses the transduced voice from the microphone 22 as an input signal for the test element circuitry 20. The transduced voice is analyzed and compared, according to the stored algorithm, to the templates formed during the enrollment process. If the test is passed by the individual, the microprocessor 37 replaces the oldest set of templates which are stored in the support board 33 with the newly created templates from the hidden trainings. To eliminate false alarms, a second voice verification test is conducted automatically in the event the first test was failed by the individual or the individual did not respond. The VVU 13 will then conduct the next test, regardless of the current test results, according to the test schedule.

The individual under surveillance also carries the anklet/bracelet transmitter 26 which continuously emits a signal, such as a radio frequency signal, that is monitored by the receiver 27 connected to the support board 33 of the VVU 13. The receiver 27 will cause the microprocessor 37 to actuate the voice verification board 32 to conduct an unscheduled voice verification test whenever the signal is disrupted or interrupted for any length of time. This can occur if the individual leaves the designated locality and, thus, goes beyond the receiver's range, or if the transmitter 26-receiver 27 combination is tampered with or malfunctions. In addition, the voice verification board 32 conducts a second test in the event the unscheduled test was failed or the individual did not respond.

In addition to conductinq the voice verification tests, the VVU 13, and in particular, the support board 33, monitors the telephone line and the power line to the VVU 13. In the event of a power failure, the support board 33 alerts the power supply board 34 to switch from the convenience outlet 38 to stand-by power, i.e., the back-up batteries. In the event of a telephone line failure, the support board 33 alerts the microprocessor 37 to record the event for later retrieval. The failure of either utility will be recorded by the support board 33 for subsequent transmission to the central station 12 as part of the status information for a monitoring period. If desired, the support board 33 can also monitor sensors at the points of entry/exit 24. In the event of an entry or exit being used, e.g., an attempted door opening, the VVU 13 will . automatically transmit that information to the central station 12 for appropriate action as well as store the violation as part of the status information.

In the event of a failure or a non-response to a voice verification test, the microprocessor 37 automatically place a telephone call via the modem 19 to the central station 12 using the phone number received in the initialization packet. If a busy signal is reached at the central station 12, the VVU 13 will continue to call, using additional telephone numbers stored on the support board 33, until a connection is made. Upon connection, the microprocessor 37 transmits a violation/service packet to the central station 12. The input of the central station 12 identifies the packet, via code information contained therein, as containing violation information and routes the packet directly to the violations computer 15. The violations computer 15 compiles and analyzes the status information contained in the packet and routes the information to the associated data storage unit 15a (for a periodic report to be outputted later on by the associated output display device 15b) and to the associated output display device 15b (for an immediate report).

In addition, the results of all tests and monitored events are stored and maintained in a detailed log in the support board 33. The detailed log is normally transmitted to the central station 12 in response to a regularly scheduled request (i.e., a polling schedule) by the control computer 14, e.g daily. The control computer 14 can also command the microprocessor 37 to transfer the detailed log, or any verification records the support board 33 continues to store, at any time. The information packet received by the central station 12 is identified, via code information contained therein, as containing status information and is routed directly to the control computer 14. The information is compiled and analyzed by the control computer 14 which then routes the information to the associated data storage unit 14a and to the associated output display device 14b. Both periodic and immediate reports can then be generated as desired. The detailed log is not deleted from the support board 33 upon transmission but continues to be stored until overwritten by new information.

The embodiment described herein is merely illustrative of the principles of the present invention. Various modifications may be made thereto by persons ordinarily skilled in the art without departing from the scope or spirit of the invention.