Personal Computer-Based Home Security System

This invention relates generally to home security systems and, in particular, to a personal computer-based home security system.

Along with the rapid increase in processor speeds, memory size, and disk capacity in commonly available personal computers ("PCs"), the types and capabilities of standard input/output devices included in PCs have also begun to increase. In particular, PCs are currently routinely sold with a microphone and audio speakers along with the software and hardware components required to capture sound through the microphone and store the captured sound in data files on a magnetic disk. The PC user can purchase any number of software packages that allow the user to edit and play back the recorded sound through the audio speakers.

Electronic home security systems have been sold in the consumer market for many years. These home security systems normally include a variety of sensors, including photo detectors, motion detectors, and sound detectors, along with a microprocessor and driving programs that coordinate monitoring of the sensors that analyze data collected through monitoring of sensors to detect suspicious or uncharacteristic events, and that can effect certain remedial actions in response to detected events. These home security systems are often expensive, and require extensive installation procedures, particularly of the sensing devices.

The present invention provides a personal computer-based home security system, implemented as a software program, that runs on commercially available personal computers. In one embodiment, the personal computer-based home security system monitors an environment to detect and remedy unusual circumstances that occur in the environment. This personal computer-based home security system includes a monitoring routine that detects threshold events that indicates a change in the environment. When such a change has been detected, the personal computer-based home security system launches a close surveillance routine. The close surveillance routine closely monitors the environment to detect, characterize, and record events that occur in the environment. When the close surveillance routine detects sufficient events to determine that a suspicious set of circumstances has occurred in the environment, the personal computer-based home security system calls a remedy routine to diagnose the suspicious set of circumstances and initiate an appropriate remedial action consistent with the diagnosis.

FIG. 1 displays a simple schematic drawing of the internal components of a PC.

FIG. 2 displays a flow control diagram for a PC-based home security system.

FIG. 3 displays example event tables.

FIG. 4 displays example detected event tables.

FIG. 5 displays an example diagnoses table.

FIG. 6 displays an example remedy table.

The present invention provides a PC-based home security system. In one embodiment, the PC-based home security system monitors the environment within a home to detect threshold events that may warrant closer examination. Following detection of a threshold event, the home security system then conducts a close surveillance of the home environment to detect suspicious or uncharacteristic circumstances, diagnoses those circumstances where possible, and initiates remedial action in the case that the diagnosed circumstances are of a serious nature. The embodiment may be implemented on standard, commonly available PCs that already include a microphone or that include both a microphone and a video camera. The PC-based home security system of some embodiment of the present invention is thus an easy-to-install and inexpensive software program that runs on commonly available PCs.

FIG. 1 displays a simple schematic drawing of the internal components of a PC. A PC includes a CPU 101, memory 102, a hard disk nonvolatile data storage device 103, various input/output devices 104-105, and one or more internal buses 106 that enable the various components to exchange data. Software programs are executed by the CPU, which fetches and executes the instructions of the program stored in memory 102. Permanent copies of the software programs are stored on the hard disk 103 and transferred to memory prior to execution. Each separate hardware controller 104 and 105 interfaces with one of a variety of different types of input/output devices, including keyboard, mouse, a microphone, a video camera, audio speaker, a printer, a fax, and a modem. Under the direction of executing software programs, the input/output device controllers enable transfer of data from input devices over the internal bus to memory and transfer of data over the internal bus from memory to output devices. A software program can direct, for example, a microphone to record the sound environment of a PC and can direct storage of the data representing the recorded sound into memory and into permanent data files stored on the hard disk.

FIG. 2 displays a flow control diagram for one embodiment of the PC-based home security system. The PC-based home security system of FIG. 2 comprises one or more software programs instantiated as one or more corresponding executing processes within the PC. When the home security system is started, it begins to monitor, in step 201, input data from the environmental input devices attached to the PC, including a microphone or a microphone and a video camera. The PC-based home security system continues to monitor this input data in step 201 until it detects a threshold event or, alternatively, until it times out or is interrupted.

A threshold event is generally a discontinuity in the input data stream that rises above a certain threshold value. For example, for data input from a microphone, a threshold event might be an abrupt increase in the detected amplitude at a particular sound frequency or an increase in the average sound level. For a video camera, a threshold event might be the detection of movement against the normal background, a marked decrease in contrast, or a rapid change in overall brightness.

When the monitoring step either stops or is interrupted, the home security system determines, in step 202, whether the termination of monitoring represents an intentional interrupt command generated by the user or represents a time-out either based on the length of the monitoring period or based on the time of day. If such an expected or intentional termination is detected in step 202, the home security system program returns in step 203. If the monitoring step has not been intentionally terminated, then the home security system determines in step 204 whether the monitoring step has detected a threshold event. If no threshold event has been detected, control returns to the monitoring step 201. If, however, a threshold event has been detected, then control flows to the close surveillance step 205.

In the close surveillance step, the home security system closely monitors and records data input from the environmental input devices. The home security system continues to closely monitor this data either until the home security system determines that a suspicious set of circumstances that require remedial action has occurred, or until no further threshold events have been detected for a certain period of time. In step 206, the home security system determines, following the close surveillance step 205, whether suspicious circumstances requiring remedial action have occurred. If not, control returns to the monitoring step 201. If, however, suspicious circumstances have been detected, then, in step 207, the home security system diagnoses those circumstances, if possible, and takes the appropriate remedial action.

In one embodiment of the invention, remedial action generally involves connecting via the voice enabled FAX/modem included in the PC to an outside telephone number and transferring over the connection one or more of a number of stored messages, depending on the nature of the receiving party and on the diagnosis of the suspicious circumstances. For example, if a fax machine is called following detection of unusual sounds, then the home security system may send a fax-based message. If, on the other hand, the home security system elects to call a police station in response to diagnosing the presence of an intruder, then the home security system may broadcast through the modem a voice message stored as a voice data file on the hard disk of the PC that informs the police of the address of the house and a warning that an intruder is present. The activities conducted by the home security system in steps 201, 205, and 207 will be discussed in greater detail below.

In the monitoring step 201, the home security system essentially listens and watches through the microphone and video camera for significant changes to the normal background environment within the home. Many different criteria may be used to detect these changes. For example, in the case of sound data obtained through the microphone, a sharp rise in the overall sound level within the home above some threshold sound level value might be interpreted by the security system as a threshold event. Similarly, in the case of the video camera input, a rapid change from darkness to lightness within the home or the detection of a large object moving within the field of the video camera over a certain period of time may be considered by the home security system to be a threshold event. These threshold events are not immediately perceived by the home security system to be suspicious or uncharacteristic. They simply trigger increased surveillance by the home security system for a certain period of time in order to detect and record a number of events. In step 201, the input data may be temporarily recorded in a circular buffer so that the home security system can append the recorded data just prior to the threshold event to data recorded subsequently in the close surveillance step 205 in order to have an entire record of the time period just before the threshold event up until close surveillance is discontinued.

A more sophisticated approach that involves adaptation to the normal background environment of the house can be employed. The home security system can monitor the environment at a particular location for a period of time in order to characterize the normal environment with respect to the time of day. Using this more sophisticated monitoring approach, the home security system can detect threshold events that represent changes in the expected background environment at a given time of day.

It is preferable in the home security system to process the raw data input from the environmental input devices during close surveillance in order to enumerate, characterize, and time stamp various types of events. To that end, the home security system may include a database of different types of events along with corresponding data patterns that characterize those events. FIG. 3 displays two example event tables. These event tables include a table of sound events 301 and a table of video events 302. In the description of an event contained in each row of each table, a numeric key for the event, along with a character string representation of the event, is combined with a pattern characterizing the event and a severity formula by which the severity of the event can be calculated from the recorded data. For example, in the sound table 301, the first event has a key value 303 of "1", a character string representation 304 of "glass breaking," a data pattern stored in the file "gbFFT.dat" 305, and the severity formula "A" 306.

The key is a simple numeric designation of the event type. The data pattern stored in the file depends on the method by which the home security system processes input data in order to recognize patterns. In the case of sound data, for example, recorded data can be processed via a Fast Fourier Transform to provide the amplitudes at various discrete characteristic frequencies as a function of time. Thus, a recorded event can be processed using a Fast Fourier Transform to produce the pattern of amplitudes at characteristic frequencies for sample times within a time period, and that resulting pattern can be compared to stored patterns in the sound database in order to choose an event type that most closely corresponds to the recorded sound input.

The home security system may also store a severity formula for calculating from recorded input data a severity metric that corresponds to a perceived seriousness of the recorded event. For example, in the case of a glass breaking event, if it were possible for the microphone to detect the sound of glass breaking in a neighbor's house several hundred feet away from the house being monitored, then perhaps the severity formula would be a simple function of the overall amplitude or volume recorded during the glass breaking event, so that only events loud enough to have occurred within the house being monitored are designated as being serious. The example sound table 301 includes additional example events, including a footstep 307, the sound of a door being kicked open 308, and the sound of a light switch being clicked or turned on or off 309. In similar fashion, the video event table 302 includes a movement event 310, a contrast change event 311, and a dark-to-light event 312. Such tables would include a miscellaneous or catch-all type of event to represent events that cannot be characterized as belonging to one of the narrow, predetermined events such as glass breaking or footsteps. A default severity formula may be assigned to these unrecognized events that may be subsequently changed by a user of the security system. Several different types of unrecognized events may be included in event tables, and unrecognized events may be associated with generalized data patterns that would serve to distinguish one unrecognized event type from the other unrecognized event types.

In the close surveillance step 205, the home security system closely monitors and records the input data to detect events and to characterize and store the detected events into detected event tables. FIG. 4 displays example detected event tables. There is a sound event table 401 and a video event table 402. In both tables, events are classified according to a key for the event. The keys are defined in the sound and video event tables of FIG. 3. The classifications of events occur in columns 403 and columns 404 of the sound and video detected event tables. Along with each event detected by the home security system, the time that the event started and the time that the event ended, t.sub.start and t.sub.end, are stored along with the calculated severity of the event in columns 404, 405, and 406 of the sound detected event table and in columns 407, 408, and 409 of the video detected event table. The recorded data may be stored in ".AVI" and ".WAV" files or in specially formatted files on either the hard disk of the PC or on secondary non-volatile storage devices like floppy drives or zip drives.

In the close surveillance step 205, the home security system may employ more than one executing process. A single process can, for example, closely monitor the input data and detect the starting point for events by detecting abrupt discontinuities in the input data. This process can then store records in the detected event tables that include only the starting time for the event. A second process can then process the detected event tables by looking up the starting times stored by the first process, using the stored patterns in the sound event and video event tables shown in FIG. 3 to characterize or pattern match the recorded events with known events and to calculate a severity for each recorded event. In the example shown in FIG. 4, the surveillance system has detected and characterized seven different types of events. At a starting time of 0, the home security system detected the sound of breaking glass and stored an entry 410 in the detected event table to correspond to that event. The home security system next detected the sound of four footsteps, stored in the detected event table in entries 411-414. Next, the home security system simultaneously detected the sound of a light switch being clicked on 415, as well as a dark-to-light event 416 detected from input video data and stored in the video detected event table.

Thus, the close surveillance step 205 both records the input data as well as processes the data in order to characterize discrete events that occur during close surveillance. The close surveillance step may continue for some set period of time or until either sufficient evidence has been collected to characterize the accumulated events as being suspicious and requiring remedial action or until no further events have been detected for a prolonged period of time.

The close surveillance step 205, like the monitoring step 201, will generally make a threshold determination based on the events detected and stored in the detected events tables shown in FIG. 4. The surveillance step can determine the threshold of suspicion by first computing a computed events metric and then comparing that computed events metric to a threshold value for the metric. When the computed events metric exceeds the threshold value, then the close surveillance step would indicate that a suspicious set of circumstances has occurred.

The following three equations show three different types of computed events metrics that can be employed by the close surveillance step:

N.sub.s =number of detected sound events;

N.sub.v =number of detected video events;

S.sub.i =severity of detected sound event i;

S.sub.j =severity of detected sound event j;

.sigma..sub.ij =correlation between type of sound event i and type of sound event j;

.DELTA.T.sub.surv =length of the surveillance period;

.DELTA.T.sub.e =expected time lapse between sound event and video event; and

.DELTA.T.sub.ij =actual time lapse between sound event and video event.

The first computed events metric is simply the sum of the number of sound events and video events detected. Thus, using this simple metric, if more sound and video events have been detected than some threshold value, the close surveillance system indicates that a suspicious set of circumstances has occurred. The second computed events metric formula is the combined sum of the sum of the severities of the events detected for the sound input device and the sum of the severities of the detected video events. Thus, if this second computed events metric is used, the close surveillance step will perceive suspicious circumstances to have occurred when the accumulated severities of detected events exceeds some threshold value. Finally, a more sophisticated computed events metric, shown in equation (3), might take into account the accumulated severities for the detected events along with an additional term that correlates the different sound events with the different video events that have been detected. For this formula, a table of event-type correlations would be maintained by the home security system along with expected time lapses between pairs of events. For example, the expected time lapse between the click of a light switch and a dark-to-light video event would be essentially 0. On the other hand, the expected time lapse between the sound of breaking glass and the detection of movement might be something on the order of 2 or 3 minutes, if not longer. Even more sophisticated computed events metrics can be employed.

If the close surveillance step determines that the computed events metric exceeds a certain threshold, and therefore perceives that a set of suspicious circumstances has occurred in the house, then the remedy step 207 is called by the home security system. The step may employ diagnoses and remedies of various sophistications and complexities. In the preferred embodiment, the remedy step attempts to correlate events detected by the close surveillance step to determine a general diagnosis of the suspicious circumstances, and then makes one or more telephone calls depending on the resulting diagnosis.

FIG. 5 displays one embodiment of a stored diagnosis table that is used by the remedy step to diagnose the sequence of events that have occurred. There are a variety of different forms and underlying algorithms that can be employed for this diagnosis. In an example shown in FIG. 5, the diagnosis table 501 includes a diagnosis key 502 and an event sequence symbolic description 503 for each possible diagnosis. For example, the first entry indicates that a diagnosis with key 1 corresponds to detection of either an event of type 1 or type 3, where the event types are defined in the key column of the event tables of FIG. 3, followed by multiple events of type 2, followed by an event of type 103, followed by an event of type 4. With reference to the event tables of FIG. 3, this first diagnosis represents detection of breaking glass or the sound of a door being kicked in, followed by a number of footsteps, followed by detection of a dark-to-light event by the video camera along with detection of the sound of a light switch being clicked. This same type of diagnosis, type 1, may also result, as shown in entry 505 in table 501, from the detection of either the sound of breaking glass or the sound of a door being kicked in, followed by the detection of movement by the video camera. The diagnosis table also generally includes a miscellaneous or catch-all diagnosis type that describes circumstances that do not fit the more specific or narrowly defined diagnoses stored in the diagnosis table.

The remedy step thus compares the events logged in the detected event tables of FIG. 4 with the event sequences of various diagnoses listed in column 503 of diagnosis table 501 in order to match the accumulated events detected by the close surveillance step with one or more diagnoses for what has happened within the home. The remedy step then employs a remedy table to determine what action to take in response to the diagnosed circumstances.

FIG. 6 displays an example remedy table. The remedy table 601 includes columns for the character string representation of a diagnosis 602, for the key or type of the diagnosis 603 corresponding to one of the diagnosis keys stored in column 502 in table 501, for a telephone number 604, for a type of receiver 605, and for a message 606. Continuing with the example used above, the first row or entry of the remedy table 607 indicates that the diagnosis having a key value of 1 is described as "intruder," that the telephone number 392-4566 should be called by the home security system when an "intruder" diagnosis has been made by the home security system, that a voice-type message should be transmitted to this telephone number, and that the voice-type message is contained in the file "intrdr.wav." Thus, entry 607 in the diagnosis table indicates to the remedy step that if the sequence of events corresponding to a diagnosis type of "1" has been found in the detected event tables of FIG. 4, then the most likely diagnosis is that an intruder has entered the house, that the telephone number corresponding to the police station should be called, and that a previously recorded voice message that includes the address of the house and an indication that it is believed that an intruder has broken into the house will be played once either a human or an answering machine has answered the telephone. As shown in the remedy table of FIG. 6, a particular diagnosis may have more than one entry. For example, entry 608 specifies a different telephone number to be called in the case that an intruder has broken into the house and that a fax should be sent to the fax machine that answers the telephone at that number. Entries 609-612 contain the telephone numbers and messages to be transmitted to those telephone numbers in the event of detection by the home security system of different types of diagnosed circumstances, including a fire 609, vandalism 610 and 611, and the outbreak of a teenage party 612.

Although the present invention has been described in terms of the several embodiments, it is not intended that the invention be limited to these embodiments. Modification within the spirit of the invention will be apparent to those skilled in the art. For example, a wide variety of different computed events metrics might be used by the close surveillance step in order to make a threshold determination of suspiciousness. Such metrics might correlate events with the time of day that the events are detected. Different environmental input devices besides microphones and video cameras might be employed. Less expensive home security systems can be implemented on PCs having only a microphone, monitoring the environment entirely by means of audio data. Different types of databases with different data organizations can be used to store event characterizations, detected events, diagnoses, and remedial actions. Remedial actions other than phone calls can be undertaken, like, for instance, playing through the audio speakers a voice message to frighten intruders. The scope of the present invention is defined by the claims which follow.