System, apparatus, and method for detecting home anomalies

[0001] This application was prepared with financial support from the Saudi Arabian Cultural Mission, and in consideration therefore the present inventor has granted, the Kingdom of Saudi Arabia a non-exclusive right to practice the present invention.

[0002] The “background” description provided herein is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in this background section, as well as aspects of the description which may not otherwise qualify as prior art at the time of filing, are neither expressly or impliedly admitted as prior art against the present invention.

[0003] Keeping a home safe from intrusions, other physical attacks, and cyber-attacks is often a priority for homeowners. Homes are increasingly equipped with smart appliances and home controllers that connect to a network to allow a homeowner to remotely operate the smart appliances via a Smart Phone, tablet, or computer.

[0004] According to an exemplary embodiment, an anomaly detection system includes appliances connected to a network and circuitry configured to receive statuses of the appliances. Patterns of use are determined for the appliances including time periods in which the appliances are most likely to be used. The circuitry is also configured to determine that anomalous events have occurred when an amount of deviation from a normal baseline status for the appliances is greater than a predetermined threshold. Cyber-attack events are detected based on changes to network configuration settings of the appliances. Alerts are output to an external device based on the one or more anomalous events and/or cyber-attack events related to the appliances. The patterns of use are updated based on the status of the appliances and a response from the external device related to the anomalous events or cyber-attack events.

[0005] The foregoing general description of the illustrative embodiments and the following detailed description thereof are merely exemplary aspects of the teachings of this disclosure, and are not restrictive.

[0016] In the drawings, like reference numerals designate identical or corresponding parts throughout the several views. Further, as used herein, the words “a,” “an” and the like generally carry a meaning of “one or more,” unless stated otherwise. The drawings are generally drawn to scale unless specified otherwise or illustrating schematic structures or flowcharts.

[0017] Furthermore, the terms “approximately,” “approximate,” “about,” and similar terms generally refer to ranges that include the identified value within a margin of 20%, 10%, or preferably 5%, and any values therebetween.

[0018] Aspects of this disclosure are directed to a smart home anomaly detector that detects anomalous occurrences in a home by monitoring statuses of at least one smart appliance in the home. According to some implementations, a smart appliance is any type of electronic device that can be configured to connect to a wired or wireless network. Examples of smart appliances may include refrigerators, washing machines, dishwashers, media controllers, wireless routers, smoke detectors, home security systems, and the like that can be controlled by a user via an external device, such as a computer or mobile device.

[0019] The smart home anomaly detector can determine whether the current status of the smart appliances is normal or abnormal based on learned patterns of use. In certain embodiments, the smart home anomaly detector can notify a homeowner via a phone call, text message, email, or the like, that one or more appliances are in an abnormal status. The smart home anomaly detector can also notify emergency response personnel, such as a police and/or fire department that an emergency, such as a fire or burglary, may have occurred due to a change in status of the smart appliances in the home.

[0020] In addition, the smart home anomaly detector can detect occurrences of cyber-attacks related to the smart appliances, which may include instances of when network configuration settings for the smart appliances are compromised, and the smart appliances are used for malicious purposes, such as performing multiple data requests in order to overwhelm a computer system or network. In addition, the smart home anomaly detector may detect when the security settings of a smart appliance, which may include personal identification information, have been violated, which may be indicative of a cyber-attack.

[0021] FIG. 1A is an exemplary illustration of a smart home anomaly detection system 100, according to certain embodiments. A smart home anomaly detector (SHAD) 102 is connected to a server 106, database 108, computer 110, mobile device 112, smart appliance 114, and smart grid 116 via a network 104. The SHAD 102 can be an external device or can be embedded within a smart home controller or home appliance. The computer 110 acts as a client device that is connected to the server 106, the database 108, the mobile device 112, the smart appliance 114, the smart grid 116, and the SHAD 102 via the network 104. The server 106 represents one or more servers connected to the computer 110, the database 108, the mobile device 112, the smart appliance 114, the smart grid 116, and the SHAD 102 via the network 104. The database 108 represents one or more databases connected to the computer 110, the server 106, the mobile device 112, the smart appliance 114, the smart grid 116, and the SHAD 102 via the network 104.

[0022] The mobile device 112 represents one or more mobile devices connected to the computer 110, the server 106, the database 108, the smart appliance 114, the smart grid 116, and the SHAD 102 via the network 104. The network 104 represents one or more networks, such as the Internet, connecting the computer 110, the server 106, the database 108, the mobile device 112, the smart appliance 114, the smart grid 116, and the SHAD 102. In some embodiments, more than one SHAD 102 is included in the smart home anomaly detection system 100. As such, the terms referring to the one or more than one SHAD 102 can be used interchangeably.

[0023] The smart appliance 114 represents one or more smart appliances connected to the computer 110, the server 106, the database 108, the mobile device 112, the smart grid 116, and the SHAD 102 via the network 104. According to certain embodiments, the smart appliance 114 is configured to conduct two-way communications via the network 104. For example, the smart appliance 114, such as a washing machine, can receive a signal to commence a wash cycle from a user at an external device, such as the mobile device 112. The washing machine is also able to output a signal to the user at the mobile device 112 when the wash cycle has completed. In addition, the at least one smart appliance 114 can include home security devices, such as a smoke detectors and security systems that can be armed and/or disarmed remotely. The home security devices can also output a signal to alert the user, such as a homeowner, that an alarming condition exists.

[0024] The smart grid 116 represents one or more smart grids connected to the computer 110, the server 106, the database 108, the mobile device 112, the smart appliance 114, and the SHAD 102 via the network 104. According to certain embodiments, the smart grid 116 is an electrical grid that can predict and respond to the behaviors of electric power consumers. The smart grid 116 can control power distribution through a wide area network (WAN) that covers an area that may include one or more substations, capacitor banks, and voltage regulators. On a smaller level than the WAN, the neighborhood area network (NAN) controls power distribution to a community metering network, such as a in a residential neighborhood or one or more city blocks. The smart grid 116 can also monitor and control the power distribution to one or more electrical components in a home, such as the at least one smart appliance 114. Through communication with the at least one smart appliance 114 via the network 104, the smart grid 116 can determine patterns of how much power is consumed by the at least one smart appliance 114 and can tailor energy distribution to the home based on the power consumption patterns.

[0025] In certain embodiments, the SHAD 102 performs wireless or wired communications with at least one smart appliance 114. In some implementations, the at least one smart appliance 114 includes any appliance with circuitry that outputs a current status to the SHAD 102. For example, a refrigerator may be able to output a temperature and/or a status of a door being opened or closed. Examples of the at least one smart appliance 114 include home security systems, washing machines, kitchen appliances, HVAC systems, smoke detectors, and the like. The SHAD 102 and the at least one smart appliance 114 can communicate via wireless networks such as WI-FI, BLUETOOTH, cellular networks including EDGE, 3G and 4G wireless cellular systems, or any other wireless form of communication that is known. Other types of wireless networks used in smart homes with at least one smart appliance 114 can include ZIGBEE, Z-WAVE, HOMEPLUG, and the like.

[0026] As would be understood by one of ordinary skill in the art, based on the teachings herein, the mobile device 112 or any other external device could also be used in the same manner as the computer 110 to receive status updates regarding one or more anomalous conditions of the at least one smart appliance 114 from the SHAD 102. For example, when the SHAD 102 detects that the refrigerator door has been left open for a period of time that is greater than a predetermined threshold, the server 106 can output a text message or email to the mobile device 112 of the homeowner to alert the homeowner that an abnormal condition has occurred. In addition, the homeowner can access alerts from the SHAD 102 via an application on the mobile device 112. The SHAD 102 can also issue an alert to the mobile device 112 of the homeowner when one or more conditions exist that may indicate that the physical security and/or cyber security of the home has been violated. Details regarding the processes performed by the smart home anomaly detection system 100 are discussed further herein.

[0027] FIGS. 1B-1D are exemplary illustrations of configurations for the smart home anomaly detection system 100, according to certain embodiments. The computer 110, mobile device 112, server 106, and database 108 are not shown but can also be included in the embodiments illustrated by FIGS. 1B-1D. FIG. 1B illustrates a smart home anomaly detection system 100 where the SHAD 102 is a stand-alone external device that communicates with the at least one smart appliance 114 and the smart grid 116 via the network 104. In some implementations, the SHAD 102 includes a power source that is independent from other electrical loads in the home so that the SHAD 102 can detect losses of power to the at least one smart appliance 114. In addition, the SHAD 102 can have more than one power source to add power redundancy, which may include AC power, battery power, solar power, and the like.

[0028] FIG. 1C illustrates a smart home anomaly detection system 100 where the SHAD 102 is included in a smart home controller 118, such as the INSTEON home controller. In the embodiment illustrated by FIG. 1C, the SHAD 102 executes software processes for detecting home anomalies on the hardware of the smart home controller and also communicates with the at least one smart appliance and the smart grid 116 via the network 104. FIG. 1D illustrates a smart home anomaly detection system 100 where the SHAD 102 is included within the at least one smart appliance 114 as a built-in feature of the at least one smart appliance 114. For example, the at least one smart appliance 114 includes circuitry configured to execute the software processes of the SHAD 102 for detecting anomalous occurrences within the home.

[0029] FIG. 2 is an exemplary illustration of software architecture for a smart home anomaly detector 102, according to certain embodiments. Processing circuitry of the SHAD 102 continuously learns the status of the at least one smart appliance 114 in the home, detects a change in status of at least one smart appliance 114, determines whether the change in status of the at least one smart appliance 114 is normal and/or expected, issues alerts to the homeowner via the computer 110 or mobile device 112 if an activity related to one of the appliances is abnormal, and learns from the homeowner's response to the anomalous event. The processing circuitry of the SHAD 102 includes a detector module 202, a learner module 204, and an informer module 206.

[0030] The detector module 202 receives inputs from one or more sensors via network 104 to detect a change in status of the at least one smart appliance 114. For example, a refrigerator can have a temperature sensor and a door sensor to determine whether the door to the refrigerator has been opened or closed. If the temperature of the refrigerator increases above a predetermined range, such as 40° F.-45° F. and the door sensor indicates that the refrigerator has remained closed during the period of temperature increase, the detector module 202 may determine that an anomalous event has occurred, such as a loss of power to the refrigerator. The network 104 can include wired and wireless connections between the SHAD 102, the at least one smart appliance 114, and/or external devices, such as the computer 110 or mobile device 112. For example, the detector module 202 also receives data from the external devices which may include responses by the homeowner regarding anomalous events that have occurred.

[0031] In addition, the detector module 202 can also receive inputs from the smart grid 116 for the home that include power consumption, voltage, and current readings for the at least one smart appliance 114. For example, the processing circuitry of the detector module 202 can detect that a power failure has occurred with the at least one smart appliance 114 based on an immediate drop in voltage and/or current at the smart grid 116. In certain embodiments, the detector module 202 continuously monitors the one or more sensors to detect a change in the status of the smart appliances. In other implementations, the detector module 202 receives sensor values at a predetermined sampling rate based on the patterns of appliance status change determined by the learner module 204.

[0032] In some implementations, the processing circuitry of the detector module 202 is configured to detect instances of cyber-attack on the at least one smart appliance 114. For example, the detector module 202 can execute one or more software processes directed to detecting malicious use of the at least one smart appliance 114, such as a compromising of an IP address, network configuration settings, or security control settings, and other types of cyber hacking.

[0033] The detector module 202 compares the observed status for the appliance to a normal status of the appliance that has been determined by the learner module 204, as will be discussed further herein. Throughout the disclosure, the “normal” status of an appliance refers to an “expected” or “learned” status based on user preferences, learned patterns of appliance use, and the like. In addition, an “abnormal” or “anomalous” status of an appliance refers to an “unexpected” status that differs from one or more normal or learned statuses of the appliance. The normal or abnormal status of the at least one smart appliance 114 determined by the detector module is output to the learner module 204 and informer module 206. Details regarding processes performed by the detector module 202 are discussed further herein.

[0034] The learner module 204 employs machine learning techniques, such as artificial neural networks, support vector machines, the like, to determine patterns of use and patterns of status change of the at least one smart appliance 114 in the home. In certain embodiments, the learner module 204 determines a normal baseline status for the at least one smart appliance 114 based on time of day, day of the week, season, and the like. The normal baseline status based on patterns of use and historical data obtained by monitoring the statuses of the at least one smart appliance 114 and can include an average sensor value associated with the at least one smart appliance 114 based on time of day or an expected operational state. The learner module 204 also outputs the statuses of the at least one smart appliance 114 along with an associated normality or abnormality determination to appliance status logs that are saved in the database 108. According to certain embodiments, the appliance status logs are data structures that include cumulative appliance status data for a period of time. The processing circuitry of the SHAD 102 applies the machine learning techniques to the data in the appliance status logs to determine the normal baseline status for the at least one smart appliance 114. Details regarding processes performed by the learner module 204 are discussed further herein.

[0035] The informer module 206 receives alerts from the detector module 202 regarding one or more anomalous events with respect to the at least one smart appliance 114 and issues alerts to external devices of one or more interested parties, such as a homeowner, an emergency contact, or an emergency first responder. The alerts can be issued to the external devices, such as mobile devices 112, of the one or more interested parties via text message, voice call, email, or a notification via an application on the mobile device 112.

[0036] According to certain embodiments, the informer module 206 receives a response from the external device of an interested party, such as the homeowner, verifying whether the status of the smart appliance is actually anomalous. For example, based on the learned patterns of use of a washing machine, the processing circuitry of the detector module 202 may determine that the status of “running” for the washing machine at 12:00 PM on a Wednesday is anomalous, and the informer module 206 issues an alert to the mobile device 112 of the homeowner via the network 104 regarding the anomalous event of the running washing machine. If, in fact, the homeowner is using the washing machine at the time that the alert is issued, the homeowner can respond to the alert to indicate that the running washing machine is a normal event. The informer module 206 outputs the response from the interested party to the learner module 204 so that the historical data can be updated based on the response of the homeowner. Details regarding processes performed by the informer module 206 are discussed further herein.

[0037] FIG. 3 is an exemplary flowchart of a detector module process 300, according to certain embodiments. At step S302, the detector module 202 of the SHAD 102 receives a status of the at least one smart appliance 114. In some implementations, the received status is based on one or more sensors, settings, and/or operational states associated with the smart appliance 114. For example, an ambient room temperature can be received by the detector module 202 to determine the status of the HVAC system. In certain embodiments, the detector module 202 continuously monitors the one or more sensors to detect a change in the status of the smart appliances. In other implementations, the detector module 202 receives sensor values at a predetermined sampling rate based on the patterns of use determined by the learner module 204. For example, the predetermined sampling rate at which the detector module 202 receives the status of the at least one smart appliance 114 may be higher than the sampling rate at time periods during the day or week when the at least one smart appliance 114 is less likely to be used. The detector module 202 can also receive appliance setting values from the at least one smart appliance 114, such as on, off, standby, sleep, and the like, at the predetermined sampling rate. In addition, the detector module 202 also receives inputs from the smart grid 116 for the home that include power consumption, voltage, and current readings for the at least one smart appliance 114 at the predetermined sampling rate.

[0038] In some implementations, the processing circuitry of the detector module 202 is configured to detect instances of cyber-attack on the at least one smart appliance 114, which are considered by be anomalous activities. For example, the detector module 202 can execute one or more software processes directed to detecting malicious use of the at least one smart appliance 114, such as a compromising of an IP address or network security settings and other types of cyber-hacking.

[0039] At step S304, the detector module 202 determines an amount of deviation from the normal baseline status for the at least one smart appliance 114. In certain embodiments, the amount of deviation from the normal baseline status is assigned a value of “very low,” “low,” “medium,” or “high,” which may be determined based on the historical data saved in the appliance status logs by the learner module 204. In some implementations, the amount of deviation from the normal baseline status is based on a probability of occurrence of the smart appliance 114 status based on the historical data saved in the appliance status logs. For a received status with a probability of occurrence of approximately one, the amount of deviation from the normal baseline status is assigned a value of very low. For a received status with a probability of occurrence of approximately zero, the amount of deviation from the normal baseline status is assigned a value of high. In addition, for instances of suspected cyber-attack on the at least one smart appliance 114, the amount of deviation from the normal baseline status is assigned a value of high.

[0040] For example, the learner module 204 may determine that the washing machine is normally run between 5:00 PM and 10:00 PM on Sundays. If the detector module 202 receives the appliance status of “running” from the washing machine at 6:00 PM on a Sunday, then the amount of deviation from the normal baseline status is determined to be “very low.” Likewise, if the detector module 202 receives the operational state of “off,” “standby,” or “sleep” for the washing machine at 3:00 AM on a Friday, then the amount of deviation from the normal baseline status is also determined to be “very low.” In addition, if the washing machine is running at 4:00 PM on a Sunday, the detector module 202 may assign an amount of deviation of “low” to the status of the washing machine. However, if the washing machine is running at 10:00 AM on a Tuesday, the detector module 202 may assign an amount of deviation of “high” to the status of the washing machine.

[0041] At step S306, it is determined whether the status of the smart appliance 114 is normal or anomalous. Throughout the disclosure, normal and anomalous statuses of the at least one smart appliance 114 are also referred to as normal and anomalous events. In certain embodiments, the detector module 202 determines that the status of the smart appliance 114 is normal if the amount of deviation from the normal baseline status is less than a predetermined threshold, such as when the amount of deviation from the normal baseline status is very low or low. In addition, the detector module 202 may determine that the status of the smart appliance 114 is anomalous if the amount of deviation from the normal baseline status is greater than a predetermined threshold, such as a medium or high amount of deviation. If it is determined that the status of the smart appliance 114 is normal, resulting in a “yes” at step S306, then step S310 is performed. Otherwise, if it is determined that the status of the smart appliance 114 is anomalous, resulting in a “no” at step S306, then step S308 is performed.

[0042] In some implementations, when an anomalous event is detected by the detector module 202 for the at least one smart appliance 114, the anomalous event is categorized by type. For example, types of anomalous events can include appliance failures, attacks, and dangers to homeowner. Appliance failures include anomalous events that indicate that the at least one smart appliance 114 has lost power due to a smart grid 116 failure or due to failure of the smart appliance 114. Failures can be detect by sensing a loss of voltage and/or current supplied to the at least one smart appliance 116.

[0043] Attacks can include physical attacks where the physical security of the home has been violated. This may be detected by an alarming security system, lights or fans that are turned on while the homeowner is away from home, and the like. Attacks can also include cyber-attacks where the integrity of communications for the at least one smart appliance 114 is violated, such as through a compromising of network security settings for the at least one smart appliance. Anomalous events that indicate dangers to the homeowner can include lack of use of the at least one smart appliance 114 for hours, days, and weeks, which may indicate that the homeowner may away from home or may be injured and unable to call for help. In some implementations, the SHAD 102 can receive input from an external device regarding instances when the homeowner is scheduled to be away from home. For example, the homeowner can send an email or text message to indicate to the SHAD 102 that he or she is away from home.

[0044] At step S308, if it is determined at step S306 that the status of the smart appliance 114 is anomalous, then an alert is issued to the informer module 206. For example, if the detector module 202 receives a status that the washing machine is running at 3:00 AM on a Wednesday, then the status may be assigned a high amount of deviation from the normal baseline status for the washing machine, resulting in an anomalous event. The detector module 202 can output the status of the washing machine, normality determination, and an associated time of occurrence to the informer module 206. The informer module 206 alerts the homeowner that the anomalous event has occurred regarding the at least one smart appliance 114 via email, text message, voice call, and the like, as will be discussed further herein.

[0045] At step S310, if it is determined at step S306 that the status of the smart appliance 114 is normal, then the result is output to the learner module 204. For example, if the detector module 202 receives a status that the washing machine is running at 5:30 PM on a Sunday, then the status may be assigned an amount of deviation from the normal baseline status of very low for the washing machine, resulting in a normal event. The detector module 202 can output the status of the washing machine, normality determination, and an associated time of occurrence to the learner module 204.

[0046] FIG. 4A is an exemplary flowchart of a learner module process 400, according to certain embodiments. At step S402, the learner module 204 of the SHAD 102 records appliance data for the at least one smart appliance 114. The appliance data is based on sensor values and statuses output from the at least one smart appliance 114. In some implementations, the learner module 204 receives the appliance status from the at least one smart appliance based on received status is based on one or more sensors associated with the smart appliance 114. For example, the learner module 204 can receive oven temperature sensor values and record the temperature values with an associated time in the appliance status log for the oven, which is saved in the database 108 along with an associated time at which the status of the at least one smart appliance 114 was received. In addition, the learner module 204 can record operational state from the smart appliance, such as on, off, standby, sleep, and the like. In addition, the learner module 204 can also receive inputs from the smart grid 116 for the home that include power consumption, voltage, and current readings for the at least one smart appliance 114 that are also recorded in the appliance status logs. In certain embodiments, the learner module 204 continuously records the sensor values from the at least one smart appliance 114.

[0047] In other implementations, the learner module 204 records sensor values in the appliance status logs at a predetermined sampling rate based on patterns of appliance use. For example, the learner module 204 may determine one or more time periods during the day or week when the at least one smart appliance 114 is most likely to be used. The predetermined sampling rate with which the learner module 204 records the status of the at least one smart appliance 114 in the appliance status log may be higher than the sampling rate at time periods during the day or week when the at least one smart appliance 114 is less likely to be used. For example, the sampling rate with which the learner module 204 records the status of the oven may be higher at times of the day when the oven is more likely to be used, such as between 12:00 PM and 10:00 PM, than during time periods when the oven is less likely to be used, such as between 12:00 AM and 6:00 AM.

[0048] In addition, the learner module 204 receives the normality determinations for the at least one smart appliance 114 from the detector module 202 during steps S308 and S310 of the detector module process 300. In some implementations, the learner module 204 records the normality determination along with the associated appliance status and time of occurrence in the appliance status log. For example, the processing circuitry of the detector module 202 may make a normality determination of normal for an oven that has status of “bake” and a temperature sensor at 400° F. at 6:00 PM. The learner module 204 records the normality determination in the appliance status log for the oven along with the time, status, and temperature sensor value.

[0049] According to certain embodiments, the learner module 204 also receives one or more homeowner responses to alerts regarding anomalous statuses for the at least one smart appliance 114. As will be discussed further herein, the processing circuitry of the SHAD 102 issues alerts to the homeowner via text message, email, voice call, and the like when the detector module 202 determines that an anomalous condition exists with the at least one smart appliance 114. The response by the homeowner to the alert from the informer module 206 is received by the learner module 204 and saved in the appliance status log for the at least one smart appliance 114.

[0050] At step S404, the processing circuitry of the learner module 204 analyzes the data in the appliance status logs to determine one or more patterns of use for the at least one smart appliance 114, according to certain embodiments. The processing circuitry employs machine learning techniques, such as artificial neural networks, support vector machines, the like, to determine patterns of use and patterns of status change of the at least one smart appliance 114 in the home. As the SHAD 102 acquires more data regarding how the at least one smart appliance 114 is used, the learner module 204 adapts the patterns of use based on changes in the home environment or in the homeowner's habits. In some implementations, the processing circuitry of the detector module 202 may determine that the current status of the at least one smart appliance 114 is anomalous based on the patterns of use determined by the learner module 204.

[0051] However, the homeowner may report that the current status of the at least one smart appliance is actually not anomalous when informed by the SHAD 102. The learner module 204 uses the information input by the homeowner to modify the patterns of use. In addition, in some implementations, the learner module 204 can analyze the security settings, network communication speed, network configuration, and the like of the at least one smart appliance 114 to determine a likelihood of cyber-attack. For example, an immediate reduction in the network communication speed may indicate that cyber-attackers are using the at least one smart appliance 114 to overwhelm the network 104. In addition, the patterns of use of the at least one smart appliance can be translated into the normal baseline status for the at least one smart appliance 114 that is used by the detector module 202 to determine whether the current status of the at least one smart appliance 114 is normal or anomalous.

[0052] At step S406, the processing circuitry of the learner module 204 stores the result of the data analysis in the database 108. According to certain embodiments, the results of the data analysis include the patterns of use of the at least one smart appliance 114 determined at step S404. FIG. 4B is an exemplary illustration of data analysis results 450 that are stored in the database 108, according to certain embodiments. The data analysis results 450 include entries for the at least one smart appliance 114 as well as likelihoods of use on a daily basis. For example, the data analysis results 450 illustrated by FIG. 4B show patterns of use for the dishwasher on Mondays. In some aspects, the data analysis results 450 include one or more times scales, such as weeks, months, seasons, or years, so that patterns of use can be determined based on the one or more time scales.

[0053] For each hour of the day, the processing circuitry of the learner module 204 determines a probability of use of the dishwasher based on historical data of dishwasher use saved in the appliance status log for the dishwasher. The probability of use can also be used to determine the normal baseline status for the at least one smart appliance 114. For the example of the dishwasher data analysis results 450 in FIG. 4B, the hours of the day with a probability of use of 0.0 to 0.5, indicating a low likelihood of use, are assigned a normal baseline status of “not running.” The hours of the day with a probability of use of greater than 0.5 are assigned a normal baseline status of “running.” In addition, the normal baseline status can also include a confidence factor that indicates a quality of the data.

[0054] In some implementations, the normal baseline status for smart appliances with statuses dependent on temperature sensors, such as HVAC systems, refrigerators, freezers, ovens, and the like, the normal baseline status can include an average temperature for each hour of the day. The amount of deviation from the normal baseline status determined at step S304 of the detector module process 300 may be based on the normal baseline status as well as the probability of use. For example, the processing circuitry may use the equation, D=|x_i−m(X)| to determine the amount of deviation from the normal baseline status for the HVAC system based on temperature. D represents the deviation from the normal baseline status, x_i represents a sample temperature value, and m(X) is the mean value of temperature sensor values over a predetermined period of time. In some implementations, the mean of the temperature sensor values is determined to be the normal baseline status for the HVAC system.

[0055] FIG. 5 is an exemplary flowchart of an informer module process 500, according to certain embodiments. At step S502, the informer module 206 receives an alert from the detector module 202 that an anomalous event has occurred with respect to the at least one smart appliance 114. The alert received by the informer module 206 can include the affected smart appliance 114, time that the anomalous event occurred, and the sensors values and/or settings of the smart appliance 114 affected by the anomalous event.

[0056] At step S504, the processing circuitry of the informer module 206 prioritizes the anomalous event alerts issued by the detector module at step S308 of the detector module process 300. According to certain embodiments, the prioritization is based on a predetermined severity level for the anomalous events based on ensuring safety of the home, user preferences, and learned patterns of use of the at least one smart appliance 114. For example, in some implementations, an alarming smoke detector or an alarming home security system is assigned a higher priority over the anomalous event of a refrigerator temperature that is outside the designated range of 40° F. to 45° F. In addition, anomalous events that are classified as attacks or dangers to the homeowner are assigned a higher priority than failures of the at least one appliance 114.

[0057] At step S506, an anomalous event alert is issued to a primary user, such as a homeowner, via a text message, email, voice call, or the like, which is received at an external device, such as the mobile device 112 or computer 110. In some implementations, the primary user is notified of the anomalous event via one or more alert screens of an application on an external device, such as the mobile device 112.

[0058] FIG. 6 is an exemplary illustration of an anomalous event notification, according to certain embodiments. In FIG. 6, the anomalous event is issued to the primary user, such as the homeowner, via an application on the mobile device 112. The screen on the mobile device 112 informs the homeowner of the time that the anomalous event occurred, the at least one smart appliance 114 affected by the anomalous event, and a summary of the anomalous event. For example, the homeowner can be notified at the mobile device 112 that the detector module 202 has determined that the washing machine is running at a time that is unlikely based on the patterns of use determined by the learner module 204. The homeowner can also make a selection at the interface for the application in order to view additional information regarding the anomalous event. According to certain embodiments, the homeowner can respond to the anomalous event alert by making a selection on the screen of the mobile device 112 regarding whether the event is truly anomalous or not. In other implementations, the homeowner can reply to the text, email, or voice call issued by the informer module 206.

[0059] Referring back to FIG. 5, at step S508, a determination is made regarding whether the homeowner has responded to the anomalous event alert within a predetermined period of time. According to some implementations, a user can specify the predetermined period of time for the primary user, such as the homeowner, to respond to the anomalous event via an interface at the SHAD 102, mobile device 112, or computer 110. For example, predetermined period of time for responding to the anomalous event can be set to 1 minute, 5 minutes, 10 minutes, and any other time period. If the homeowner has responded to the anomalous event alert within the predetermined period of time, resulting in a “yes” at step S508, then step S518 is performed. Otherwise, if the homeowner has not responded to the anomalous event alert within the predetermined period of time, resulting in a “no” at step S508, then step S510 is performed.

[0060] At step S510, if it is determined at step S508 that the primary user has not responded to the anomalous event alert within the predetermined period of time, then the informer module 206 issues the anomalous event alert to a secondary user, such as an emergency contact indicated by the primary user at an interface at an interface at the SHAD 102 or other external device, such as the computer 110 or mobile device 112. According to certain embodiments, the anomalous event alert is issued to the emergency contact via text message, email, voice call, or the like, which is received at an external device, such as the mobile device 112 or computer 110. In some implementations, the emergency contact is notified of the anomalous event via one or more alert screens of an application on an external device, such as the mobile device 112 as described previously with respect to FIG. 6.

[0061] At step S512, a determination is made regarding whether the emergency contact has responded to the anomalous event alert within a predetermined period of time. According to some implementations, a user can specify the predetermined period of time for the secondary user, such as the emergency contact, to respond to the anomalous event via an interface at the SHAD 102, mobile device 112, or computer 110. For example, predetermined period of time for responding to the anomalous event can be set to 1 minute, 5 minutes, 10 minutes, and any other time period. If the emergency contact has responded to the anomalous event alert within the predetermined period of time, resulting in a “yes” at step S512, then step S518 is performed. Otherwise, if the homeowner has not responded to the anomalous event alert within the predetermined period of time, resulting in a “no” at step S512, then step S514 is performed.

[0062] At step S514, if it is determined at step S512 that the secondary user has not responded to the anomalous event alert within the predetermined period of time, then the informer module 206 issues the anomalous event alert to an emergency responder via emergency response system, such as a dispatch center for emergency responders that may include medical first responders, law enforcement, and the like. According to certain embodiments, the anomalous event alert is issued to the emergency response system via text message, email, voice call, or the like, which is received at an external device, such as the mobile device 112 or computer 110. In some implementations, the emergency response system is notified of the anomalous event via one or more alert screens of an application on an external device, such as the mobile device 112 as described previously with respect to FIG. 6.

[0063] According to some embodiments, the emergency responder notification of step S514 is performed if the anomalous event is related to a potential safety or security issue for the home. For example, anomalous events related to the home security system or smoke detector are reported to the emergency responder at step S514, but anomalous events such as an unexpectedly running washing machine or dishwasher may not be reported to the emergency responder.

[0064] At step S518, if it is determined at step S508 or step S512 that the homeowner or the emergency contact has responded to the anomalous event alert, resulting in a “yes” at step S508 or step S512, then it is determined whether the anomalous event was truly abnormal based on the response of the homeowner or emergency contact. In the example of FIG. 6, the processing circuitry of the informer module 206 determines whether the homeowner or emergency contact selected “yes” or “no” regarding whether the event is truly anomalous or not. If the homeowner or emergency contact confirms that the event is truly anomalous, then a “yes” is output to step S516. Otherwise, if the homeowner or emergency contact responds that the event is normal, as in not anomalous, then a “no” is output to step S516.

[0065] At step S516, the user response by the homeowner, emergency contact, or emergency responder to the anomalous event is output to the learner module 204. The user response to the anomalous event can be associated with appliance data recorded in the appliance status logs at step S402 in order to update the patterns of use for the at least one smart appliance 114.

[0066] The smart home anomaly detection system 100 allows homeowners to know and maintain a level of safety of their homes by monitoring changes in the statuses of the at least one smart appliance 114 that communicates with the SHAD 102 so that the at least one smart appliance 114 contributes to the overall integrity and security of the home. In addition, as the number of devices and appliances within a home that can communicate with the network 104 increases, the devices and appliances become increasingly vulnerable to cyber-attack. The SHAD 102 is able to detect compromises in network configurations and security settings of the at least one smart appliance 104 to minimize the effects of potential cyber-attacks.

[0067] A hardware description of the smart home anomaly detector 102 according to exemplary embodiments is described with reference to FIG. 7. In addition, the hardware described by FIG. 7 can also apply to the computer 110, the server 106, and the mobile device 112. Implementation of the processes of the smart home anomaly detection system 100 on the hardware described herein improves the speed and accuracy of the detection of anomalous events within a home. The SHAD 102 includes a CPU 700 with the processing circuitry that performs the processes described herein. The process data and instructions may be stored in memory 702. These processes and instructions may also be stored on a storage medium disk 704 such as a hard drive (HDD) or portable storage medium or may be stored remotely. Further, the claimed advancements are not limited by the form of the computer-readable media on which the instructions of the inventive process are stored. For example, the instructions may be stored on CDs, DVDs, in FLASH memory, RAM, ROM, PROM, EPROM, EEPROM, hard disk or any other information processing device with which the SHAD 102 communicates, such as the computer 110.

[0068] Further, the claimed advancements may be provided as a utility application, background daemon, or component of an operating system, or combination thereof, executing in conjunction with CPU 700 and an operating system such as Microsoft Windows 7, UNIX, Solaris, LINUX, Apple MAC-OS and other systems known to those skilled in the art.

[0069] CPU 700 may be a Xenon or Core processor from Intel of America or an Opteron processor from AMD of America, or may be other processor types that would be recognized by one of ordinary skill in the art. Alternatively, the CPU 700 may be implemented on an FPGA, ASIC, PLD or using discrete logic circuits, as one of ordinary skill in the art would recognize. Further, CPU 700 may be implemented as multiple processors cooperatively working in parallel to perform the instructions of the inventive processes described above.

[0070] The SHAD 102 in FIG. 7 also includes a network controller 706, such as an Intel Ethernet PRO network interface card from Intel Corporation of America, for interfacing with network 104. As can be appreciated, the network 104 can be a public network, such as the Internet, or a private network such as an LAN or WAN network, or any combination thereof and can also include PSTN or ISDN sub-networks. The network 104 can also be wired, such as an Ethernet network, or can be wireless such as a cellular network including EDGE, 3G and 4G wireless cellular systems. The wireless network can also be Wi-Fi, Bluetooth, or any other wireless form of communication that is known.

[0071] The SHAD 102 further includes a display controller 708, such as a NVIDIA GeForce GTX or Quadro graphics adaptor from NVIDIA Corporation of America for interfacing with display 710 of the SHAD 102 and the computer 110, such as a Hewlett Packard HPL2445w LCD monitor. A general purpose I/O interface 712 at the SHAD 102 interfaces with a keyboard and/or mouse 714 as well as a touch screen panel 716 on or separate from display 710. General purpose I/O interface 712 also connects to a variety of peripherals 718 including printers and scanners, such as an OfficeJet or DeskJet from Hewlett Packard.

[0072] A sound controller 720 is also provided in the SHAD 102, such as Sound Blaster X-Fi Titanium from Creative, to interface with speakers/microphone 722 thereby providing sounds and/or music.

[0073] The general purpose storage controller 724 connects the storage medium disk 704 with communication bus 726, which may be an ISA, EISA, VESA, PCI, or similar, for interconnecting all of the components of the SHAD 102. A description of the general features and functionality of the display 710, keyboard and/or mouse 714, as well as the display controller 708, storage controller 724, network controller 706, sound controller 720, and general purpose I/O interface 712 is omitted herein for brevity as these features are known.

[0074] In other alternate embodiments, processing features according to the present disclosure may be implemented and commercialized as hardware, a software solution, or a combination thereof. Moreover, instructions corresponding to the detector module process 300, learner module process 400, and informer module process 500 in accordance with the present disclosure could be stored in a thumb drive that hosts a secure process.

[0075] A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of this disclosure. For example, preferable results may be achieved if the steps of the disclosed techniques were performed in a different sequence, if components in the disclosed systems were combined in a different manner, or if the components were replaced or supplemented by other components. The functions, processes and algorithms described herein may be performed in hardware or software executed by hardware, including computer processors and/or programmable circuits configured to execute program code and/or computer instructions to execute the functions, processes and algorithms described herein. Additionally, an implementation may be performed on modules or hardware not identical to those described. Accordingly, other implementations are within the scope that may be claimed.