US 9,690,933 B1Grant
Framework for classifying an object as malicious with machine learning for deploying updated predictive models
Issue Date:2017-06-27
•28 Claims
•8 Drawing Sheets
Abstract
According to one embodiment, an apparatus comprises a detection engine and a classification engine. The detection engine is responsible for analyzing an object to determine if the object is malicious. The classification engine is configured to (i) receive results of the analysis of the object conducted by the detection engine and (ii) analyze, based at least in part on the results from the detection engine, whether the object is malicious in accordance with a predictive model. Responsive to the detection engine and the classification engine differing in determinations as to whether the object is malicious, information associated with at least a portion of the results of the analysis of the object by at least one of the detection engine and the classification engine is uploaded for determining whether an update of the predictive model is to occur. An update of the predictive model is subsequently received by the classification engine.
Metadata
Assignee
- FireEye, Inc.
Inventors
- Abhishek Singh
- Ali Mesdaq
- Anirban Das
- Varun Jain
Application Information
Application Number:US 14/579,896
Filing Date:2014-12-22
Priority Date:2014-12-22
Art Unit:2438
Classifications
IPC:
G06F21/56G06N5/02H04L29/06
Field of Search:
G06F 21/56G06F 21/552G06F 21/566H04L 63/1408H04L 63/1416H04L 63/1425
Patent Drawings (8 sheets)
Description
FIELD
[0001] Embodiments of the disclosure relate to the field of cyber security. More specifically, one embodiment of the disclosure relates to a system, apparatus and method for automatically updating a classification engine that analyzes an object and determines whether the object is to be classified as malicious.
GENERAL BACKGROUND
[0002] Over the last decade, network devices that access the Internet or other publicly accessible networks have been increasingly subjected to malicious attacks. These malicious attacks may simply involve the use of stolen credentials by an unauthorized person in efforts to illicitly gain access to information stored within a network device. However, other malicious attacks may be more complex.
[0003] In general, malicious attacks may be carried out via an exploit or malware. An exploit is information that attempts to take advantage of a vulnerability in computer software or systems by adversely influencing or attacking normal operations of a targeted computer.
[0004] For example, malicious attacks may involve malicious software that has been downloaded by the network device. In some situations, the victim is unaware that the malicious software has been downloaded and stored within her network device. In other situations, the victim is aware that the software has been downloaded, but is unaware of its malicious activity. After being stored on the victim's network device, malicious software may, by design, compromise the network device, for example, by employing an exploit to take advantage of a software vulnerability in the network device in order to harm or co-opt operation of the network device. For instance, the malicious software may (i) gain access to certain stored information and attempt to upload such information to a targeted Command and Control (CnC) server or (ii) establish connectivity between the network device to a remote computer in efforts to exfiltrate stored information.
[0005] New malicious software is released to the Internet regularly. The speed at which attackers revise the attacks of their malicious software through code modifications requires cyber security service providers to match this speed in revising detection capabilities for these threats. For a two-stage threat detection platform, which conducts both static and dynamic analysis of incoming data, a classification engine that classifies whether the data under analysis is “malicious” (e.g., a classification that identifies a certain likelihood that the data is malicious), needs to be regularly updated to remain effective.
[0006] The classification engine is responsible for classifying data as malicious or not based on whether such data includes one or more features that already have been determined to suggest maliciousness at an associated probability level. These features may include (i) a particular file size, (ii) presence of an attachment, (iii) format type (e.g., whether the file includes an executable, a portable document format “pdf” document, etc.), (iv) specific data patterns, (v) source of the file, and (vi) a structure of the file. Reliance on manually initiated updates for the classification engine tends to be problematic as these updates are not regularly provided, due to human error in some cases. A technique is needed that will automatically update the detection capabilities of the classification engine with a reduced update cycle time.
BRIEF DESCRIPTION OF THE DRAWINGS
[0007] Embodiments of the invention are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings, in which like references indicate similar elements and in which:
[0008] FIG. 1 is an exemplary block diagram of a sequence of operations for a threat detection platform.
[0009] FIG. 2 is an exemplary block diagram of a communication system deploying one or more threat detection platforms (TDPs) communicatively coupled to a management system.
[0010] FIG. 3 is an exemplary block diagram of logic associated with the TDP of FIG. 2 .
[0011] FIG. 4 is an exemplary block diagram of logic associated with the classification testing system of FIG. 2 .
[0012] FIGS. 5A-5C are general exemplary flowcharts that illustrates operation conducted by an electronic device for determining and conducting an update of a classification engine.
[0013] FIG. 6 is an exemplary embodiment of an endpoint device including a security agent that conducts the classification engine update scheme as described in FIG. 1 .
DETAILED DESCRIPTION
[0014] Various embodiments of the disclosure relate to a framework which, based upon the threats detected, periodically, aperiodically, or continuously updates a classification engine to better recognize the presence of malicious software and/or exploits (referred collectively referred to herein as “malware”) within analyzed objects. This framework features a training engine that, based on information associated with detected threats, automatically (and without user intervention) issues an alert when the classification engine requires updating. Once the alert is issued by the training engine, one or more parameters (e.g., particulars to an analyzed feature of the object, including score, weighting, string length, bit size, character type, etc.) within a predictive model are modified (e.g., added, deleted and/or changed). The “predictive model” is logic that controls the analysis conducted by a classification engine, and modification of the predictive model is conducted to reduce the number or rate of false negative events by the classification engine.
[0015] As described herein, the classification engine may be deployed for classifying an object as malicious or not malicious based on static analysis results, although inventive aspects set forth in the disclosure may also be employed to automatically update a classification engine operating on behaviors observed during run-time analysis or operating on both static analysis results and observed behaviors.
I. Detailed Summary
[0016] According to one embodiment of the disclosure, logic is provided to enhance operability of a classification engine by evaluating results in the analysis of an object by that classification engine operating in accordance with a current predictive model. This logic, referred to herein as a “training engine,” conducts this evaluation in order to determine whether the current predictive model needs updating to more accurately classify whether an object (e.g., any collection of data including a file, document, web page, etc.) is “malicious” (e.g., varying in degree from definitely malicious to a level where the object is suspected to be malicious, sometimes referred to as “suspicious”), or benign (non-malicious). Additionally, the training engine is configured to automatically generate an updated predictive model (hereinafter referred to as a “reference model”) based on actual feature(s) of the suspect object.
[0017] Herein, the type of triggering event that causes the feature(s) of the suspect object to be provided to the training engine may vary, depending on the desired involvement of the training engine in the modification of the classification engine. As an example, one triggering event may be based on identifying a disagreement as to whether an object under analysis is malicious between a detection engine and a classification engine within the same security appliance (herein referred to as a “threat detection platform”). The disagreement may be based on difference in the degree (or level) of detected “maliciousness” or “non-maliciousness” (e.g., malicious v. non-malicious, detected “malicious” levels differ, etc.). According to this embodiment of the disclosure, the detection engine and the classification engine correspond to the same analysis engine (e.g., static analysis engine or the dynamic analysis engine as illustrated in FIG. 2 ).
[0018] Another triggering event may be based on outputted results from a collective classification engine that receives results from one of more of the analysis engines (e.g., detection engines from the static analysis engine and/or the dynamic analysis engine). Herein, the collective classification engine generates a result that identifies, for reporting purposes, whether the object under analysis is malicious—the triggering event may occur when the result fails to accurately identify the level of maliciousness for that object. For example, the collective classification engine may determine a confidence score (X) that barely classifies the object as “malicious”, and the triggering event occurs when the confidence score (X) deviates from a predetermined confidence score (X+Y or X−Y) by a prescribed threshold (e.g., less than a maximum confidence score (X+Y) by the prescribed threshold (Z, Z<Y) or greater than a set confidence score (X−Y) by less than the prescribed threshold (Z, Z>Y)).
[0019] For instance, as an illustrative embodiment, the training engine may be deployed to receive features associated with a suspect object that is considered to be “malicious” by a detection engine. In this embodiment, deployed within a static analysis engine, the detection engine conducts static analysis such as exploit-specific checks, vulnerability-specific checks or rule-based checks or checks based upon emulation for example, to determine whether the analyzed object includes one or more features that suggest that the object is “malicious” (e.g., indicates that there exists at least a prescribed probability that the one or more features may be associated with a malicious attack).
[0020] Where the detection engine determines that the suspect object may include one or more malicious features, but the classification engine using the current predictive model for analysis classifies the object as non-malicious (e.g., assigns a confidence value that falls below a certain threshold), this denotes a potential false negative event. As a result, the threat detection platform transmits a control message to the training engine, where the control message may include (1) an identifier of the object (e.g., hash value of the object), (2) one or more suspect features of the object that can be used by the predictive model to classify the file as malicious, and/or (3) results from the preliminary classification of the suspect object by this platform-based classification engine (e.g., confidence values). Concurrently, where the disagreement between the detection engine and the classification engine occurs in the static analysis engine, the object and perhaps the results from the detection engine may be provided to a virtual execution environment within the threat detection platform for dynamic analysis.
[0021] Otherwise, when the object is determined to be malicious by both the detection engine and the (platform-based) classification engine associated with the static analysis engine, the object under analysis is provided to the virtual execution environment without initiating a triggering event.
[0022] Potentially located outside the enterprise and accessible by a cyber-security service provider, the training engine receives the identifier of the suspect object (e.g., hash value of the object) to determine if the object has been evaluated previously in accordance with the current predictive model. If not, the training engine may modify one or more parameters associated with the current predictive model to better detect those feature(s) associated with the suspect object that have been determined to be malicious by the detection engine as described above. This modification may involve altering parameters by changing certain values in a decision-tree analysis associated with the current predictive model, which produces an updated predictive (reference) model.
[0023] As an illustrative example, the modification of the current predictive model may include changing character string values associated with a name of the object, which may signify the object is malicious. These character string values may include, but are not limited or restricted to length or character types. Another modification may include increasing values (e.g., confidence scores) that are assigned to certain types of features to decrease the number (or rate) of false negative events, or may involve decreasing values assigned to certain types of features to reduce the number (or rate) of false positive events. Additionally, modification may further involve adding or deleting analytical operations from the decision-tree analysis of the current predictive model (e.g., adding/removing certain analysis, etc.).
[0024] Based at least in part on the results from the preliminary classification by the platform-based classification engine (e.g., confidence values), which over time identifies whether the current predictive model is ineffective or is becoming less effective in detecting malicious attacks, an alert is provided from the training engine to update a reference classification engine accessible by the cyber security service provider along with one or more platform-based classification engines with the reference model.
[0025] It is contemplated that the classification engines may be updated based on analysis and classification of objects conducted by other security appliances (for a more holistic view of malware features) and of forensic work by expert analysts and laboratories. As a result, the reference model is not modified based solely on objects uploaded by a single security appliance.
II. Terminology
[0026] In the following description, certain terminology is used to describe features of the invention. For example, in certain situations, both terms “engine,” “component” and “logic” are representative of hardware, firmware and/or software that is configured to perform one or more functions. As hardware, engine (or component/logic) may include circuitry having data processing or storage functionality. Examples of such circuitry may include, but is not limited or restricted to a microprocessor, one or more processor cores, a programmable gate array, a microcontroller, an application specific integrated circuit, wireless receiver, transmitter and/or transceiver circuitry, semiconductor memory, or combinatorial logic.
[0027] Engine (or component/logic) may be software in the form of one or more software modules, such as executable code in the form of an executable application, an application programming interface (API), a subroutine, a function, a procedure, an applet, a servlet, a routine, source code, object code, a shared library/dynamic load library, or one or more instructions. These software modules may be stored in any type of a suitable non-transitory storage medium, or transitory storage medium (e.g., electrical, optical, acoustical or other form of propagated signals such as carrier waves, infrared signals, or digital signals). Examples of non-transitory storage medium may include, but are not limited or restricted to a programmable circuit; a semiconductor memory; non-persistent storage such as volatile memory (e.g., any type of random access memory “RAM”); persistent data store such as non-volatile memory (e.g., read-only memory “ROM”, power-backed RAM, flash memory, phase-change memory, etc.), a solid-state drive, hard disk drive, an optical disc drive, or a portable memory device. As firmware, the executable code is stored in persistent storage.
[0028] The term “object” generally refers to a collection of data, whether in transit (e.g., over a network) or at rest (e.g., stored), often having a logical structure or organization that enables it to be classified for purposes of analysis. During analysis, for example, the object may exhibit a set of expected features and, during processing, a set of expected behaviors. The object may also exhibit a set of unexpected features and a set of unexpected behaviors that may evidence malware and potentially allow the object to be classified as malware.
[0029] The term “model” generally refers to logic that is used in classifying an object under analysis as malicious or not. One type of model includes a predictive model, which may be logic in the form of a decision-tree based analysis in which parameters associated with the analysis or certain decisions may be modified in order to “tune” the analysis to improve performance.
[0030] The term “transmission medium” is a physical or logical communication path between two or more electronic devices (e.g., any devices with data processing and network connectivity such as, for example, a security appliance, a server, a mainframe, a computer such as a desktop or laptop, netbook, tablet, firewall, smart phone, router, switch, bridge, etc.). For instance, the communication path may include wired and/or wireless segments, and/or shared memory locations. Examples of wired and/or wireless segments include electrical wiring, optical fiber, cable, bus trace, or a wireless channel using infrared, radio frequency (RF), or any other wired/wireless signaling mechanism.
[0031] The term “computerized” generally represents that any corresponding operations are conducted by hardware in combination with software and/or firmware.
[0032] Lastly, the terms “or” and “and/or” as used herein are to be interpreted as inclusive or meaning any one or any combination. Therefore, “A, B or C” or “A, B and/or C” mean “any of the following: A; B; C; A and B; A and C; B and C; A, B and C.” An exception to this definition will occur only when a combination of elements, functions, steps or acts are in some way inherently mutually exclusive.
[0033] The invention may be utilized for updating classification engines. As this invention is susceptible to embodiments of many different forms, it is intended that the present disclosure is to be considered as an example of the principles of the invention and not intended to limit the invention to the specific embodiments shown and described.
III. Classification Engine Update Scheme
[0034] Referring to FIG. 1 , an exemplary block diagram of a sequence of operations for threat detection within a system that is configured to detect malicious attacks is shown. The system 100 may broadly include a plurality of components, namely a processing engine 110, one or more analysis engines 115, a training engine 130, and a reference classification engine 140. According to this embodiment, the processing engine 110 and the analysis engine(s) 115 operate on a security appliance (referred to as “threat detection platform”) located within an enterprise while the training engine 130 and the reference classification engine 140 may operate as cloud computing services for example, may operate in a different enterprise, or may operate within the same enterprise as the processing engine 110 and analysis engine(s) 115.
[0035] Located in the threat detection platform 150, the processing engine 110 receives an object and converts that object into a format, as needed or appropriate, on which deep scanning by at least a first analysis engine 120 can be applied. This conversion and scanning may involve decompression of the object, decompilation of the object, extraction of specific data associated with the object, and/or emulation of the extracted data (like Javascript).
[0036] Located in the threat detection platform 150 as part of the first analysis engine 120 (e.g., static analysis engine), a first detection engine 122 is configured to conduct exploit-specific checks, vulnerability-specific checks and/or rule-based checks on the data associated with the object, as described below. Based on the results of the check(s), the first detection engine 122 may uncover one or more features that may indicate that the suspect object is malicious. These features may also be supplied to a first, platform-based classification engine 124 that determines, based on analysis of the feature(s) given its predictive model, whether the suspect object is malicious or not.
[0037] According to one embodiment of the disclosure, in response to a triggering event, such as where the object is determined to be malicious by the first detection engine 122, but is classified by the first classification engine 124 as non-malicious, information associated with the object may be provided to the training engine 130. It is contemplated that this triggering event may occur in response to this discrepancy or may be programmed for throttling the triggering event based on the degree of discrepancy, the rate or periodicity of discrepancies, or the number of times that the discrepancy occurs. When provided, this object information may include, but is not limited or restricted to (1) an identifier 126 of the object (e.g., hash value of the object such as a message digest produced in accordance with Message Direct “MD5” algorithm), (2) one or more features 127 of the suspect object that are associated with a malicious attack, and/or (3) results 128 of the preliminary classification by the first classification engine 124 (e.g., confidence values). Concurrently, the object and results in the preliminary analysis by the first analysis engine 120 are provided to a second analysis engine 160.
[0038] Of course, it is contemplated that, where the object is determined to be non-malicious (or malicious) by both the first detection engine 122 and the first classification engine 124, the object and results are merely provided to the second (dynamic) analysis engine 160. The training engine 130 is to receive information where the analysis results from the first detection engine 122 and the first classification engine 124 differ as to whether the object under analysis is malicious or not. However, according to another embodiment of the disclosure, this difference in analysis may occur when the suspect object is determined to be malicious by both the first detection engine 122 and the first classification engine 124, but the confidence score produced by the first classification engine 124 falls below a prescribed score threshold.
[0039] Located remotely from the threat detection platform 150, such as part of a cloud computing service or within a different enterprise network for example, the training engine 130 is configured to receive an identifier for the object in order to determine if the object has been previously analyzed in a potential update of the current predictive model. If not, the training engine 130 makes use of one or more features of the suspect object from the first detection engine 122 to determine if the object is malicious and determines which portions within the current predictive model need to be modified to better detect such features for classifying files as malicious.
[0040] There exist a few schemes that may be used by the training engine 130 to determine when an alert should be issued to update the current predictive model used by the reference classification engine 140 and/or platform-based classification engine(s) 124 and/or 164. For instance, the training engine 130 may rely on the received confidence score from the first classification engine 124 to determine accuracy in classifying a detected malicious object. In response to consistent false negative events by the classification engine 124 (e.g., the detected false negative events exceed a certain number or a prescribed rate) or a prescribed period of time has elapsed between updates, the training engine 130 may issue signaling, referred to as an “alert,” to update the reference classification engine 140 and/or platform-based classification engines 124 and/or 164 deployed at the customer. According to one embodiment, the reference model is generated as part of the cloud computing services or as part of services for an enterprise.
[0041] Similarly, the training engine 130 may issue an alert to update the reference classification engine 140 and subsequently platform-based classification engines 124/164 within the TDP(s) when an average confidence score for different objects that are detected as malicious by the first detection engine 122 (but are considered non-malicious by the first classification engine 124), as measured over a prescribed period of time, falls below a minimum detection threshold. Alternatively, the training engine 130 may issue an alert in response to a series of confidence scores for different objects that are detected as malicious by the first detection engine 122 (but are considered non-malicious by the platform-based classification engine 124) are consistently decreasing (over a prescribed period of time) by a certain amount (or percentage). As yet another alternative, the training engine 130 may prompt the reference classification engine 140 to analyze one or more features associated with a subsequent object under analysis using the reference model and determine whether the resultant confidence score exceeds the received confidence score by a prescribed amount (in value or in percentage).
[0042] Located as a cloud computing service or within a separate enterprise network or within the same enterprise network but at a different location as TDP 150 for this embodiment, the reference classification engine 140 receives the reference model (updated predictive model) produced by the training engine 130 or receives specific updates that are applied to the current reference model by the reference classification engine 140 to produce the reference model. According to one embodiment of the disclosure, the reference model is returned to the threat detection platform 150 to update the classification engine 124 that classifies analytic results associated with the first detection engine 122 of the static analysis engine 120. Additionally, or in the alternative, the reference model may be returned to the threat detection platform 150 to update a classification engine 164 associated with the second analysis engine 160, or any other platform-based classification engine, as described below in further detail.
[0043] Of course, it is contemplated that object classification associated with the static analysis results may be handled as cloud computing services without deployment of the first classification engine 124. For this embodiment, information associated with malicious objects would be uploaded to the reference (cloud-based) classification engine 140. The reference classification engine 140 would determine whether information needs to be provided to the training engine 130 for modification of the predictive model associated with the classification engine. The reference model also may be returned to another classification engine within the TDP 150 for subsequent classification of the suspect object by the second (dynamic) analysis engine 160.
IV. Exemplary System Architectures
[0044] Referring to FIG. 2 , an exemplary block diagram of a communication system 200 deploying one or more threat detection platforms (TDPs) 1501-150N (N≧1, N=3 for this embodiment) communicatively coupled to a management system 205 via a network 210 is shown. In general, the management system 205 is adapted to manage TDPs 1501-1503. While cloud computing services 230 (or services via a separate enterprise) may be responsible for updating predictive models utilized by one or more classification engines deployed within the TDP 1501, the management system 205 is responsible for updating software utilized by the TDPs 1501-1503. Such updating may be conducted automatically or conducted manually via uploads by an administrator. Also, such updating may be conducted freely among the TDPs 1501-1503 or subject to a subscription basis in which periodic or aperiodic updates of the predictive model are provided by a service including the classification testing system 235.
[0045] Herein, according to the embodiment illustrated in FIG. 2 , a first TDP 1501 is an electronic device that is adapted to analyze information associated with network traffic from at least one network device 212 for transmission over a communication network 214, such as communications with server. The communication network 214 may include a public network such as the Internet, in which case an optional firewall 216 (represented by dashed lines) may be interposed prior to accessing another network device 220. Alternatively, the communication network 214 may be a private network such as a wireless data telecommunication network, wide area network, a type of local area network (LAN), or a combination of networks.
[0046] As shown, the first TDP 1501 may be communicatively coupled with the communication network 214 via an interface 218. In general, the interface 218 operates as a data capturing device (sometimes referred to as a “tap” or “network tap”) that is configured to receive data propagating to/from the network device 220 within an enterprise network and provide at least some of this data to the first TDP 1501 or a duplicated copy of the data. Alternatively, although not shown in detail, the first TDP 1501 may be positioned behind the firewall 216 and at least partially in-line with network (client) device 220 so as to subject incoming traffic to analysis (e.g., through static analysis) and potentially block that which is classified as malware from reaching its destination.
[0047] According to an embodiment of the disclosure, the interface 218 may be further configured to capture metadata from network traffic associated with network device 220. According to one embodiment, the metadata may be used, at least in part, to determine protocols, application types and other information that may be used by logic within the first TDP 1501 to determine particular software profile(s). The software profile(s) are used for selecting and/or configuring one or more virtual machines 2851-285M (M≧1) within a run-time, virtual execution environment 162 of the dynamic analysis engine 160, as described below. These software profile(s) may be directed to different software or different versions of the same software application extracted from software image(s) fetched from storage device 270.
[0048] In some embodiments, interface 218 may provide connectivity to a server or any device with storage capability through a dedicated transmission medium such as a wireless channel, a wired cable or the like. Although not shown, interface 218 may be contained within the first TDP 1501. In other embodiments, the interface 218 can be integrated into an intermediary device in the communication path (e.g., firewall 216, router, switch or other networked electronic device, which in some embodiments may be equipped with SPAN ports) or can be a standalone component, such as an appropriate commercially available network tap.
[0049] As further shown in FIG. 2 , the first TDP 1501 comprises processing engine 110, the first (static) analysis engine 120, a scheduler 265, a storage device 270, a second (dynamic) analysis engine 160, reporting engine 290 with an optional user interface capability, and/or a collective platform-based classification engine 295. The first TDP 1501 operates in combination with the classification testing system 235, which includes the training engine 130, by uploading information that may assist the training engine 130 in determining whether logic within the classification engine 124, 164 and/or 295 requires updating. The logic may include a predictive model, namely decision-tree learning logic at least partially developed using machine learning techniques from prior analysis of labelled and unlabeled malware, benign objects and/or experiential knowledge from human analysts that determines whether feature(s) of the suspect object indicate that the object is malicious (e.g., associated with a particular malware).
[0050] The processing engine 110 of the first TDP 1501 is configured to receive an incoming object 240 (operation 1) and to convert the object 240 into a format that may be subsequently analyzed by the static analysis engine 120 to determine if the object 240 includes one or more features that are considered potentially associated with malicious activity. This conversion may involve decompression and/or decompilation of the object 240, extraction of specific data of the object 240, and/or emulation of the extracted data.
[0051] The static analysis engine 120 comprises a controller 250, a first data store 252, the detection engine 122, object transformation logic 258 and the first classification engine 124. The controller 250 is logic that controls operations conducted within the static analysis engine 120. These operations may include data storage within first data store 252; pattern matching of the incoming object 240 to determine whether the object 240 includes one or more malicious features; preliminary classification of the object 240; and/or cryptographic operations on the object 240, including one-way hash operations.
[0052] Herein, the static analysis engine 120 includes the detection engine 122 that includes one or more software modules that, when executed by controller 250, analyzes features for the incoming object 240 (e.g., a portion of network traffic, an uploaded file, etc.). As such, the detection engine 122 analyzes the object 240 through one or more pattern checking operations without execution of the object 240. Examples of the pattern checking operations may include signature matching 255 to conduct (a) exploit signature checks, which may be adapted to compare at least a portion of the object 240 with one or more pre-stored exploit signatures (pre-configured and predetermined attack patterns) from signature database (not shown), and/or (b) vulnerability signature checks that may be adapted to uncover deviations in messaging practices (e.g., non-compliance in communication protocols, message formats or ordering, and/or payload parameters including size). Other examples of these pattern checking operations may include (i) heuristics 256, which is based on rules or policies as applied to the object and may determine whether one or more portions of the object 240 is associated with an anomalous or suspicious feature (e.g., a particular URL associated with known exploits, or a particular source or destination address etc.) associated with known exploits; or (ii) determinative rule-based analysis 257 that may include blacklist or whitelist checking.
[0053] After operations are conducted by the detection engine 122 to uncover potentially malicious features in the object 240, the classification engine 124 determines whether this object 240 is “malicious,” namely whether certain features of the object 240 suggest an association with a malicious attack. According to one embodiment of the disclosure, in addition (or in the alternative) to being stored in the first data store 252, some or all of results produced by the detection engine 122 may be provided to the first classification engine 124, which is configured to determine a probability (or level of confidence) that the suspect object 240 is malware.
[0054] More specifically, the first classification engine 124 may be configured to determine a probability (or level of confidence) that the object 240 is associated with a malicious attack. This probability may be represented through a value (referred to as a “static analysis confidence score”) that may be used by the training engine 130 to identify the need for updating the predictive model utilized by the first classification engine 124. The static analysis confidence score may be determined based, at least in part, on (i) the particular pattern matches; (ii) heuristic or determinative analysis results; (iii) analyzed deviations in messaging practices set forth in applicable communication protocols (e.g., HTTP, TCP, etc.); (iv) analyzed compliance with certain message formats established for the protocol (e.g., out-of-order commands); and/or (v) analyzed header or payload parameters to determine compliance.
[0055] Furthermore, the static analysis engine 120 may route this suspect object 240 (or specific portions or features of the suspect object 240) to the dynamic analysis engine 160 for more in-depth analysis. Also, results 253 of the static analysis may be stored within the first data store 252. The static analysis results 253 may include (i) a static analysis confidence score (described above) and/or (ii) metadata associated with the object. The metadata may include (a) features associated with malware (e.g., matched signature patterns, certain heuristic or statistical information, etc.), and/or (b) other types of metadata associated with the object under analysis (e.g., name of malware or its family based on the detected exploit signature, anticipated malicious activity associated with this type of malware, etc.).
[0056] After analysis of the object, the static analysis engine 120 may route the suspect object 240 to the dynamic analysis engine 160, which is configured to provide more in-depth analysis by analyzing the suspect object in a VM-based operating environment. Although not shown, the suspect object 240 may be buffered by the first data store 252 or a second data store 282 until ready for processing by virtual execution environment 162. Of course, if the object 240 is not suspected of being part of a malicious attack, the static analysis engine 120 may denote that the object is benign, and thus, refrain from passing information associated with object 240 to the training engine 130. Instead, the object 240 is passed to the dynamic analysis engine 160 for subsequent analysis.
[0057] More specifically, after analysis of the features of the suspect object 240 has been completed (or after analysis of multiple suspect objects where certain features are buffered), the static analysis engine 120 may provide at least some or all of the features that are identified as being potentially associated with malware, to the training engine 130 for determination as to whether updating of the first classification engine 124 (and/or any other classification engine 140, 164 and/or 295) is necessary. According to one embodiment of the disclosure, the static analysis engine 120 provides (1) an identifier (e.g., hash value) of the suspect object 240, (2) one or more suspect features of the suspect object 240 and/or (3) some or all of the results 253 from the static analysis, which may include static analysis confidence value. The upload of this information is identified by operations 3-4.
[0058] Located outside an enterprise featuring the TDP 1501, the training engine 130 receives the identifier 126 of the object (e.g., hash value of the object) to determine if the object has been evaluated previously in accordance with the current predictive model. This may be accomplished by comparing a listing of identifiers maintained by the training engine 130, where each identifier represents an object whose features have already been evaluated in updating the current predictive model. If the suspect object 240 has been previously evaluated, the training engine 130 may disregard such features or further adjust parameters within the updated current predictive (reference) model given that there are repeated occurrences of this type of malicious object or the object now includes different features that have not been considered.
[0059] If the object has not been previously evaluated, the training engine 130 analyzes the results, and based on the analysis, may modify one or more parameters associated with the reference model to better detect the one or more features associated with the received object that have been determined to be malicious by the static analysis engine 120 as described above. Determined through analysis of the results 253 from the static analysis engine 120, this parameter modification may include changing certain values in the decision-tree analysis as provided by the current predictive model.
[0060] As an illustrative example, suppose that the filename analysis in the current predictive model is represented by the following in which a score of 80 out of a maximum 100 is applied if the name of the object is greater than 15 characters and does not begin with an alphanumeric character (A-Z or 0-9):
-
- [0061] if length_of_string (string)>15
- [0062] if first_char≠char ‘[A-Z]’
- [0063] if first_char≠‘[0-9]’
- [0064] score=80
The parameter modification may involve altering the length of the string parameter, checking for first character, etc. upon the training engine 130 determining that this feature (e.g., name length and/or character string types) has a higher correlation to maliciousness than previously assigned. In combination with or in the alternative, the parameter modification may involve removal of the first character “0” as suspect objects that have been received by the training engine 130 are now tending to include the “0” at the start of the object name.
- [0061] if length_of_string (string)>15
[0065] Based at least in part on the static analysis confidence values, which over time identifies whether the current predictive model remains ineffective or is becoming less effective, an alert is provided from the training engine 130 to update the reference classification engine 140, which is immediately accessible by the cyber security service provider. The reference classification engine 140 may be used to update platform-based classification engines (e.g., classification engine 124, 164 and/or 295) with the reference model as illustrated in operation 5. Of course, it is contemplated that the TDP 1501 may not feature any platform-based classification engines, and in this type of deployment, the reference classification engine 140 would only need to be updated.
[0066] Referring still to FIG. 2 , in one embodiment of the disclosure, the scheduler 265 may be adapted to configure one or more VMs 2851-285M based on metadata associated with a flow that includes the object under analysis. For instance, the VMs 2851-285M may be configured with software profiles corresponding to the software images stored within storage device 270. As an alternative embodiment, the VMs 2851-285M may be configured according to one or more software configurations that are being used by electronic devices connected to a particular enterprise network (e.g., network “client” device(s) 220) or prevalent types of software configurations (e.g., a Windows® 7 OS; a certain version of a particular web browser such as Internet Explorer®; Adobe® PDF™ reader application). As yet another alternative embodiment, the VMs 2851-285M may be configured to support concurrent (e.g. at least partially simultaneous in time) virtual execution of a variety of different software configurations in efforts to verify that the suspicious object is part of a malicious attack. Of course, it is contemplated that the VM configuration described above may be handled by logic other than the scheduler 265.
[0067] According to one embodiment of the disclosure, the dynamic analysis engine 160 is adapted to execute one or more VMs 2851-285M to simulate the receipt and execution of content associated with the object under analysis within a run-time environment as expected by the type of object. For instance, dynamic analysis engine 160 may optionally include processing logic 280 to emulate and provide anticipated signaling to the VM(s) 2851, . . . , and/or 285M during virtual processing.
[0068] For example, the processing logic 280 may be adapted to provide, and sometimes modify (e.g., modify IP address, etc.) packets associated with the suspect object 240 in order to control return signaling back to the virtual execution environment 162. Hence, the processing logic 280 may suppress (e.g., discard) the return network traffic so that the return network traffic is not transmitted to a network providing connectivity to the network (client) device 220.
[0069] Although not shown in FIG. 2 , monitoring logic within the virtual execution environment 162 may be configured to monitor behaviors of one or more VMs 2851, . . . , and/or 285M, such as VM 2851 configured to execute the suspect object 240. This monitoring is conducted to detect anomalous (e.g., unexpected, irregular, etc.) activity indicative of malware. When anomalous activity is detected, the monitoring logic operating with the second classification engine 164 to compute a dynamic analysis confidence score and provide VM-based results 288 (e.g., dynamic analysis confidence score, information associated with the detected anomalous behaviors, and/or other information associated with the detected malicious activity by the suspect object) to the platform-based classification engine 295.
[0070] It is noted that the second classification engine 164 may not be implemented within the dynamic analysis engine 160. Instead, the platform-based classification engine 295 receives the VM-based results 288 (without the dynamic analysis confidence score) and conducts a classification of the object based on the VM-based results and/or SA results (or static analysis confidence value). It is contemplated that the confidence score produced by the VM-based results 288 may be weighted differently than the static analysis confidence score.
[0071] In general, the collective platform-based classification engine 295 may be configured to receive the VM-based results 288. According to one embodiment of the disclosure, the classification engine 295 comprises prioritization logic 296 and score determination logic 297. The prioritization logic 296 may be configured to apply weighting to VM-based results 288 and/or static analysis-based results 260 from static analysis engine 120. According to one embodiment, these VM-based results 288 may include the dynamic analysis confidence score and/or the SA-based result 260 may include the static analysis confidence score.
[0072] The score determination logic 297 comprises one or more software modules that are used to determine a final probability as to whether the suspect object 240 is malicious, and the resultant score representative of this final probability may be included as part of results provided to the reporting engine 290 for reporting. The score determination logic 297 may rely on the predictive model (or updated predictive model provided as the reference model) to determine the score assigned to the object.
[0073] Herein, the reporting engine 290 generates reports (e.g., various types of signaling such as messages including text messages and email messages, display images, or other types of information over a wired or wireless communication path) to identify to a network administrator the presence of a detected suspect object in the received network traffic. The reports may include a detailed summary of at least the malware detected by the TDP 1501.
[0074] Although the illustrative embodiment describes the updating of the predictive model for the classification engine 124 within the static analysis engine, it is contemplated that the similar operations may be conducted for the classification engine 164 of the dynamic analysis engine 160 and/or the collective classification engine, where the results from the detection engines of the static analysis engine 140 and/or dynamic analysis engine 160 is in disagreement and the result of the classification engine 295.
V. Exemplary Logic Layout of TDP
[0075] Referring now to FIG. 3 , an exemplary block diagram of logic associated with TDP 1501 of FIG. 2 is shown. TDP 1501 comprises one or more processors 300 that are coupled to communication interface logic 310 via a first transmission medium 320. Communication interface logic 310 enables communications with other TDPs 1502-1503 and management system 205 of FIG. 2 . According to one embodiment of the disclosure, communication interface logic 310 may be implemented as a physical interface including one or more ports for wired connectors. Additionally, or in the alternative, communication interface logic 310 may be implemented with one or more radio units for supporting wireless communications with other electronic devices.
[0076] Processor(s) 300 is further coupled to persistent storage 340 via transmission medium 330. According to one embodiment of the disclosure, persistent storage 340 may include (a) static analysis engine 120, including the first detection engine 122 and the first classification engine 124; (b) the dynamic analysis engine 160 that comprises the second detection engine 162 that includes the virtual execution environment and the second classification engine 164; (c) the platform-based classification engine 295 including prioritization logic 296, score determination logic 297; (d) reporting engine 290; and (e) data stores 252 and 282. Of course, when implemented as hardware, one or more of these logic units could be implemented separately from each other. The engines contained within persistent storage 340 are executed by processor(s) and perform operations as described above.
VI. Exemplary Logic Layout of Classification Testing System
[0077] Referring now to FIG. 4 , an exemplary block diagram of logic associated with the classification testing system 235 of FIG. 2 is shown. The classification testing system 235 comprises a housing 405 that at least partially encases one or more processors 400, which are coupled to communication interface logic 410 via a first transmission medium 420. Communication interface logic 410 enables communications with one or more TDPs (e.g., TDP 1501) to receive information associated with the suspect object for adjustment of logic associated with classification engine 140 and/or classification engines deployed with the TDP 1501. According to one embodiment of the disclosure, communication interface logic 410 may be implemented as a physical interface including one or more ports for wired connectors. Additionally, or in the alternative, communication interface logic 410 may be implemented with one or more radio units for supporting wireless communications with other electronic devices.
[0078] Processor(s) 400 is further coupled to persistent storage 440 via transmission medium 430. According to one embodiment of the disclosure, persistent storage 440 may include (a) training engine 130, including object comparison logic 450, parameter modification logic 455 and model comparison logic 460; (b) the reference classification engine 140; (c) reference model update logic 465; and (d) data store 470. Of course, when implemented as hardware, one or more of these logic units could be implemented separately from each other.
[0079] Herein, executed by the processor(s) 400, the training engine 130 receives an identifier of the object and activates the object comparison logic 450 to determine if the object has been evaluated previously in accordance with the current predictive model. The object comparison logic 450 maintains a listing of identifiers that represent those objects for which features have been evaluated in an update of the current predictive model.
[0080] If the object is determined by the object comparison logic 450 to have been previously evaluated in generation of the reference model 475, the parameter modification logic 455 may disregard such features or further adjust parameters within a current predictive model 480 (e.g., decision-tree analysis) given that there are repeated occurrences of this type of malicious object. If the object has not been previously evaluated in generation of the reference model 475, the parameter modification logic 455 analyzes the static analysis results of the object, and based on these results, may modify one or more parameters associated with the current predictive model 480 in generation of the reference model 475 to better detect malicious objects with these types of features. Examples of parameter modifications may include changing certain values in the decision-tree analysis as provided by the current predictive model.
[0081] Upon the model comparison logic 460 determining that the current predictive model 480 remains ineffective in detecting malicious objects or is becoming less effective in detecting malicious objects, an alert is provided to the reference model update logic 465 to update the current predictive model 480 by substitution of the reference model 475 at the reference classification engine 140 (or provide the updates 485 as represented as an optional feature by dashed lines). Furthermore, reference classification engine 140 may propagate the reference model 480 to other classification engines, including classification engine 124 and 164 that are utilized by the analysis engines 120 and 160 as well as platform-based classification engine 295.
VII. Exemplary Threat Detection and Prevention Processes
[0082] Referring to FIG. 5A , a general exemplary flowchart is shown that illustrates operations conducted by an electronic device, such as a TDP or a portable device of FIG. 6 for example, for determining whether an update of the classification engine may be warranted. Upon receiving a suspect object, an analysis is conducted to determine whether the results produced by different engines in analyzing the same object for malicious features represent a false negative result, which suggests that a potential update of the classification engine is warranted (blocks 500 and 510). More specifically, this analysis may involve (i) a first analysis by a detection engine that determines whether an object under analysis is malicious and (ii) a second analysis by the classification engine that determines either (a) the suspect object is non-malicious or (b) the suspect object is malicious but the degree of malicious falls below a prescribed threshold. Based on the analysis associated with the object, if there are no potential false negative events in the analysis, no modification of the classification engine is needed.
[0083] If the analysis denotes a potential false negative event, information associated with the analysis of the object is provided to a training engine to determine whether the classification engine should be updated (block 520). As described above, this information may include, but is not limited or restricted to (1) the identifier of the object (e.g., hash value of the object such as a message digest produced in accordance with Message Direct “MD5” algorithm), (2) one or more suspect features of the object and/or (3) results of the preliminary classification by the classification engine (e.g., confidence values).
[0084] Thereafter, as shown in FIG. 5B , without user intervention and when an update of the classification engine is needed, the TDP automatically receives a reference model, namely an updated predictive model, which is used to modify the functionality of the classification engine (block 530). Upon receipt, the classification engine is updated with the reference model (block 540).
[0085] Referring now to FIG. 5C , a general exemplary flowchart is shown that illustrates a process for determining whether the update of the classification engine is warranted. First, a training engine receives information associated with the suspect object that is associated with a preliminary false negative event (block 550). Stated differently, the training engine receives one or more features of the object where a detection engine determined that the object includes malicious features, but the classification engine determined that the suspect object is non-malicious or is malicious but the degree of malicious falls below a prescribed threshold. As described above, this information may include, but is not limited or restricted to (1) an identifier of the object (e.g., hash value of the object such as a message digest produced in accordance with Message Direct “MD5” algorithm), (2) one or more suspect features of the object and/or (3) results of the preliminary classification by the classification engine (e.g., confidence values).
[0086] Thereafter, the training engine determines whether the suspect object is included in a training data set that is used to produce a current predictive model (block 555). This may involve a comparison of the provided identifier to a list of identifiers representing those objects that have been analyzed in determination of the current predictive model. If not, a secondary determination is made as to whether there exist certain features of the suspect object that were not present in any of the prior objects considered in generation of the current predictive model (block 560). If the objects include different features for analysis or the suspect object is not part of the training set data used to produce the current predictive model, the training data set is updated with the different features, along with different weighting and/or scores (block 565). Otherwise, the training data set is not updated.
[0087] Thereafter, the current predictive model is updated using the updated training data set to produce the reference model (block 570). Using at least part of the information received with the feature(s) of the suspect object, if the level of effectiveness of the current predictive model falls below a particular threshold (e.g., number of false negative events is now greater than a preset number, confidence values for detected malicious features falls below a set value, etc.), the classification engine is updated with the reference model (blocks 575, 580 and 585). Otherwise, the classification engine continues to utilize the current predictive model for classification of suspect objects as malicious or not (block 590).
[0088] It is contemplated that the classification engine update scheme, as described above, may also be conducted by a security agent 600 as shown in FIG. 6 . Herein, the security agent 600 is stored within a memory 610 encased within a housing 625 of an endpoint device 620. Upon execution by a processor 630, the security agent 600 activates a static analysis engine 635 to conduct a static analysis of at least a portion of information 640 received by a transceiver 650 of the endpoint device 620. As before, in response to the static analysis engine and the classification engine 660 reaching different determinations (or the same determinations with results that exceed a certain difference threshold), security agent 600 uploads the control message including the one or more malicious features to a training engine located as part of cloud computing services or residing within a particular enterprise network (not shown). Thereafter, the classification engine 660 may be automatically updated, without user intervention, depending on the findings by the training engine.
VIII. Second Embodiment for a Classification Engine Update Scheme
[0089] More specifically, according to another illustrative embodiment, malware may be discovered through a two-stage process in the threat detection platform 150, including the static analysis engine 120 and the dynamic analysis engine 160. Herein, the static analysis engine 120, upon determining that a suspect object is suspicious (e.g., exceeds a certain likelihood that the object is malicious), submits the suspect object for behavior analysis by processing this object in a run-time (virtual) environment. After behavioral analysis, the object may be classified as malicious or non-malicious.
[0090] If the initial static analysis determines that an object is not malicious, the object may be further analyzed through a secondary static analysis operable after the behavioral analysis, which extracts and analyzes relevant features of the object. The relevant features include those that may have been obfuscated during the initial static analysis but manifested themselves during execution (e.g., due to encryption or other encoding). Maintained within the threat detection platform 150 or located in the cloud, as shown, the classification engine 140 associated with the secondary static analysis may be configured to determine if the object is malicious by evaluating each feature and pattern of features received from the secondary static analysis engine.
[0091] As before, this classification engine 140 may use a decision-tree learning algorithm as a predictive model, where the decision-tree learning algorithm may be developed using machine learning techniques from prior analysis of labelled and unlabeled malware and benign objects and/or experiential knowledge from human analysts. Herein, the classification engine 140 computes a score associated with the features and pattern of features reflecting the probability that the object is malicious. Once the score for the features has been determined, the classification engine 140 may transmit that score to the dynamic analysis engine 160 to be used in the analysis of the object or may transmit information to the training engine 130 to modify the predictive model to account for malicious detection discrepancies, as described above.
[0092] According to another embodiment of the disclosure, where the object is determined to be malicious by the first detection engine 122 (e.g., static analysis engine) and is classified by the platform-based classification engine 140 as malicious by assigning a confidence score (e.g., a value representing a probability of the object being malicious), but the confidence score fails to exceed a prescribed threshold score, the above-described information associated with the object may also be provided to the training engine 130. Concurrently, the object and results by the static analysis engine 120 are provided to the dynamic engine 160 for analysis.
[0093] The training engine 130 is responsible for generating a new feature-specific predictive (reference) model from features and patterns of features identified through actual static analysis of the object, but for this embodiment, where a “current” predictive model used in the platform-based classification engine 140 determines that the object is malicious. For this embodiment, results produced by a test analysis conducted in accordance with the reference model, is compared to results produced by a test analysis conducted in accordance with the current predictive model. The comparison is conducted to assess whether the two results yield substantially different scores related to the probability that the object is malicious. If the difference in scores exceeds a threshold, the current predictive model may be modified to reflect the reference model, at least with respect to the features or pattern of features identified in the static analysis.
[0094] In arriving at the scores prescribed by the reference model, the following operations may be practiced. The current predictive model may yield, for example, a score of “80” for the object, where any score over “75” denotes that the object is to be classified by malicious. Since an object is classified as malicious, it may be deemed to deserve an overall score of “100”, and thus, the difference is determined to be “20” (100 minus 80). If the threshold for updating the predictive model is 15, for example, the predictive model associated with the classification engine requires updating with the reference model or modification of the current predictive model in accordance with the generated updates. The updates may be achieved by simply (1) decomposing that “100” score and (2) assigning component scores to each feature or pattern of features so as to yield a higher score for some than that associated with the feature (or pattern) by the current predictive model. For example, if the current predictive model associates a score of 30 with a particular feature (e.g., length of a string) identified in the object, but the reference model determines that that string is a stronger indicator of maliciousness, and accords this feature with a score of 40, the predictive model is modified by changing the associated score from 30 to 40 for future analysis of object. This approach is not dependent on the type of object analyzed, which may be an Office® document, PDF file, or JAR files, for example.
[0095] In the foregoing description, the invention is described with reference to specific exemplary embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention. For instance, training engine 130 may be referenced and/or provide updated predictive models only upon the classification of the object as a particular exploit (e.g., crimeware, malvetisement,) or a malicious object from a specific threat actor.
Claims
What is claimed is:
1. An apparatus comprising:
one or more processors; and
a non-transitory storage medium communicatively coupled to the one or more processors, the non-transitory medium includes a first analysis engine that comprises
a detection engine that analyzes an object to determine if the object is malicious, and
a classification engine that is configured to (i) receive results of the analysis of the object conducted by the detection engine, and (ii) analyze, based at least in part on the results from the detection engine, whether the object is determined to be malicious in accordance with a predictive model,
wherein, in response to the detection engine and the classification engine differing in a determination as to whether the object is to be classified as malicious, transmitting information associated with at least a portion of the results from the analysis of the object by at least one of the detection engine and the classification engine to a service remotely located from the apparatus and receiving an update of the predictive model for use by the classification engine, the update of the predictive model is conducted to reduce a number or a rate of at least false negative determinations by the classification engine using the predictive model, and
wherein, in response to the detection engine and the classification engine being consistent in the determination as to whether the object is to be classified as malicious or non-malicious, refraining by the first analysis engine from transmitting the information associated with at least the portion of the results to the service.
2. The apparatus of claim 1, wherein the classification engine further receives the update of the predictive model in response to one of the false negative determinations that includes the detection engine determining that the object is malicious while the classification engine determining that the object is non-malicious.
3. The apparatus of claim 1, wherein the classification engine further receives the update of the predictive model in response to (i) a confirmation that the apparatus subscribes to a service that automatically updates predictive models used by the classification engine and (ii) the detection engine determines that the object is malicious while the classification engine determines that the object is non-malicious.
4. The apparatus of claim 1, wherein the detection engine and the classification engine are part of a static analysis engine that performs pattern checking operations on the object including heuristic or determinative rule-based analysis.
5. The apparatus of claim 1, wherein the detection engine is part of a static analysis engine and the classification engine is part of a dynamic analysis engine, the dynamic analysis engine includes one or more virtual machines that process the object and are configured in accordance with one or more software configurations.
6. The apparatus of claim 1, wherein the uploaded information comprises (1) an identifier of the object, and (2) one or more features of the object that can be used by the predictive model to classify the object as malicious.
7. The apparatus of claim 6, wherein the uploaded information further comprises (3) results from a preliminary classification of the object by the classification engine.
8. The apparatus of claim 6, wherein the identifier of the object is a hash value of the object.
9. The apparatus of claim 6, wherein the uploaded information further comprises (3) the results from a preliminary classification of the object by the detection engine.
10. The apparatus of claim 1, wherein, in further response to the detection engine and the classification engine being consistent in the determination as to whether the object is to be classified as malicious or non-malicious, providing, by the first analysis engine, at least the object to a second analysis engine that includes at least a second classification engine, the second analysis engine to conduct an additional analysis of the object to classify whether the object is malicious or non-malicious.
11. The apparatus of claim 10, wherein, in further response to the detection engine and the classification engine being consistent in the determination as to whether the object is to be classified as malicious or non-malicious, providing by the first analysis engine at least the results of the analysis of the object conducted by the detection engine along with the object to the second analysis engine.
12. The apparatus of claim 10, wherein, in further response to the detection engine and the classification engine being consistent in the determination as to whether the object is to be classified as malicious or non-malicious, providing by the first analysis engine at least results of a preliminary classification of the object by the classification engine along with the object to the second analysis engine.
13. The apparatus of claim 1, wherein the service includes either (i) part of cloud computing services or (ii) part of services for an enterprise network.
14. A classification testing system comprising:
a hardware processor; and
a data store coupled to the processor, the data store comprises one or more engines that, upon execution by the hardware processor and in response to a prior analysis of an object by a classification engine, which is operating as part of an electronic device remotely located from the classification testing system, analyzing the object using a first predictive model to determine whether the object is associated with a malicious attack, and responsive to detecting a potential false negative event, modifying one or more parameters associated with the first predictive model to produce a second predictive model that is provided to the classification engine for now detecting whether the object is associated with a malicious attack.
15. The apparatus of claim 14, wherein the one or more engines to receive information associated with the prior analysis of the object, the information comprise (i) an identifier of the object and (ii) one or more features of the object that are determined as being associated with the malicious attack.
16. The apparatus of claim 15, wherein the one or more engines compare the identifier with stored identifiers to determine if the object has been previously analyzed in an update of the first predictive model.
17. The apparatus of claim 15, wherein the one or more engines modify the one or more parameters that are directed to a decision-tree analysis of the one or more features of the object.
18. The apparatus of claim 17, wherein the one or more engines modify a first parameter of the one or more parameters by changing a value corresponding to the first parameter in the decision-tree analysis associated with the first predictive model to produce the second predictive model.
19. The apparatus of claim 15, wherein the information received by the one or more engines further comprise (iii) results of a preliminary classification conducted by the classification engine that produced the false negative event, the results of the preliminary classification include a confidence score representing a probability that the object is associated with the malicious attack.
20. The apparatus of claim 15, wherein the information received by the one or more engines further comprise (iii) results of a preliminary classification conducted by a detection engine that detected the object as being associated with the malicious attack in contrast with a result of a preliminary classification conducted by the classification engine that includes the first predictive model for analysis of the object for malware.
21. The apparatus of claim 14, wherein the one or more engines that, upon execution by the hardware processor and in response to modifying the one or more parameters associated with the first predictive model to produce the second predictive model, output the second predictive model to the classification engine for updating the first predictive model that is currently being used by the classification engine.
22. An apparatus comprising:
one or more hardware processors; and
a memory coupled to the one or more processors, the memory comprises
a detection engine that, when executed by the one or more processors, analyzes an object to produce a first result, the first result identifies whether the object is malicious or non-malicious; and
a classification engine that, when executed by the one or more processors, analyzes one or more features of the object to produce a second result, the second result identifies whether the object is malicious or non-malicious in accordance with a predictive model,
wherein, in response to the first result of the detection engine and the second result of the classification engine differing in a determination as to whether the object is malicious, uploading information associated with at least a portion of results from the analysis of the object by at least one of the detection engine and the classification engine and, based on the uploaded information identifying that the predictive model is ineffective or is becoming ineffective, receiving an update of the predictive model to reduce a number or a rate of at least false negative determinations by the classification engine, and
wherein, in response to the first result of the detection engine and the second result of the classification engine being consistent in the determination as to whether the object is to be classified as malicious or non-malicious, refraining from uploading the information associated with at least the portion of results from the analysis of the object.
23. The apparatus of claim 22, wherein the classification engine to receive the update of the predictive model subsequent and in response to one of the false negative determinations that includes the detection engine determining that the object is malicious and the classification engine determining that the object is non-malicious.
24. The apparatus of claim 22, wherein the classification engine to receive the update of the predictive model subsequent and in response to (i) the apparatus subscribing to a service that automatically updates predictive models used by the classification engine and (ii) the detection engine determining that the object is malicious and the classification engine determining that the object is non-malicious.
25. The apparatus of claim 22, wherein the detection engine is part of a static analysis engine and the classification engine is part of a dynamic analysis engine, the dynamic analysis engine includes one or more virtual machines that process the object and are configured in accordance with one or more software configurations.
26. The apparatus of claim 22, wherein the uploaded information comprises (1) an identifier of the object, and (2) one or more features of the object that can be used by the predictive model to classify the object as malicious.
27. The apparatus of claim 26, wherein the uploaded information further comprises (3) the results from a preliminary classification of the object by the classification engine that include a confidence score that identifies a likelihood of the object being malicious.
28. An apparatus comprising:
one or more processors; and
a non-transitory storage medium communicatively coupled to the one or more processors, the non-transitory storage medium includes an analysis engine that comprises
a detection engine that analyzes an object to uncover one or more features of the object that are used to determine whether the object is malicious or non-malicious, and
a classification engine that is configured to (i) receive the features of the object conducted by the detection engine, and (ii) determine, based at least in part on the information from the detection engine and in accordance with values assigned to different parameters of a predictive model to identify features of the object that tend to be associated with malware, a confidence score associated with the object, the confidence score identified a probability of the object is malicious,
wherein, in response to the detection engine and the classification engine differing in a determination as to whether the object is to be classified as malicious, uploading information associated with at least a portion of the results from the analysis of the object by at least one of the detection engine and the classification engine for use in determining whether an update of values associated with one or more parameters of the predictive model so as to reduce a number or a rate of at least false negative determinations by the classification engine, and
wherein, in response to the detection engine and the classification engine being consistent in the determination as to whether the object is to be classified as malicious or non-malicious, refraining by the analysis engine from uploading the information associated with at least the portion of the results.
Patent Citations (520)
| Patent | Date | Inventor | Cited By |
|---|---|---|---|
| US4292580(A) | 1981-09-01 | Ott et al. | Applicant |
| US5175732(A) | 1992-12-01 | Hendel et al. | Applicant |
| US5440723(A) | 1995-08-01 | Arnold et al. | Applicant |
| US5490249(A) | 1996-02-01 | Miller | Applicant |
| US5657473(A) | 1997-08-01 | Killean et al. | Applicant |
| US5842002(A) | 1998-11-01 | Schnurer et al. | Applicant |
| US5978917(A) | 1999-11-01 | Chi | Applicant |
| US6088803(A) | 2000-07-01 | Tso et al. | Applicant |
| US6094677(A) | 2000-07-01 | Capek et al. | Applicant |
| US6108799(A) | 2000-08-01 | Boulay et al. | Applicant |
| US6118382(A) | 2000-09-01 | Hibbs et al. | Applicant |
| US6269330(B1) | 2001-07-01 | Cidon et al. | Applicant |
| US6272641(B1) | 2001-08-01 | Ji | Applicant |
| US6279113(B1) | 2001-08-01 | Vaidya | Applicant |
| US6298445(B1) | 2001-10-01 | Shostack et al. | Applicant |
| US6357008(B1) | 2002-03-01 | Nachenberg | Applicant |
| US6417774(B1) | 2002-07-01 | Hibbs et al. | Applicant |
| US6424627(B1) | 2002-07-01 | S.o slashed.rhaug et al. | Applicant |
| US6442696(B1) | 2002-08-01 | Wray et al. | Applicant |
| US6484315(B1) | 2002-11-01 | Ziese | Applicant |
| US6487666(B1) | 2002-11-01 | Shanklin et al. | Applicant |
| US6493756(B1) | 2002-12-01 | O'Brien et al. | Applicant |
| US6550012(B1) | 2003-04-01 | Villa et al. | Applicant |
| US6700497(B2) | 2004-03-01 | Hibbs et al. | Applicant |
| US6775657(B1) | 2004-08-01 | Baker | Applicant |
| US6831893(B1) | 2004-12-01 | Ben Nun et al. | Applicant |
| US6832367(B1) | 2004-12-01 | Choi et al. | Applicant |
| US6895550(B2) | 2005-05-01 | Kanchirayappa et al. | Applicant |
| US6898632(B2) | 2005-05-01 | Gordy et al. | Applicant |
| US6907396(B1) | 2005-06-01 | Muttik et al. | Applicant |
| US6941348(B2) | 2005-09-01 | Petry et al. | Applicant |
| US6971097(B1) | 2005-11-01 | Wallman | Applicant |
| US6981279(B1) | 2005-12-01 | Arnold et al. | Applicant |
| US6995665(B2) | 2006-02-01 | Appelt et al. | Applicant |
| US7007107(B1) | 2006-02-01 | Ivchenko et al. | Applicant |
| US7028179(B2) | 2006-04-01 | Anderson et al. | Applicant |
| US7043757(B2) | 2006-05-01 | Hoefelmeyer et al. | Applicant |
| US7069316(B1) | 2006-06-01 | Gryaznov | Applicant |
| US7080407(B1) | 2006-07-01 | Zhao et al. | Applicant |
| US7080408(B1) | 2006-07-01 | Pak et al. | Applicant |
| US7093002(B2) | 2006-08-01 | Wolff et al. | Applicant |
| US7093239(B1) | 2006-08-01 | van der Made | Applicant |
| US7096498(B2) | 2006-08-01 | Judge | Applicant |
| US7100201(B2) | 2006-08-01 | Izatt | Applicant |
| US7107617(B2) | 2006-09-01 | Hursey et al. | Applicant |
| US7159149(B2) | 2007-01-01 | Spiegel et al. | Applicant |
| US7213260(B2) | 2007-05-01 | Judge | Applicant |
| US7231667(B2) | 2007-06-01 | Jordan | Applicant |
| US7240364(B1) | 2007-07-01 | Branscomb et al. | Applicant |
| US7240368(B1) | 2007-07-01 | Roesch et al. | Applicant |
| US7243371(B1) | 2007-07-01 | Kasper et al. | Applicant |
| US7249175(B1) | 2007-07-01 | Donaldson | Applicant |
| US7287278(B2) | 2007-10-01 | Liang | Applicant |
| US7308716(B2) | 2007-12-01 | Danford et al. | Applicant |
| US7328453(B2) | 2008-02-01 | Merkle, Jr. et al. | Applicant |
| US7346486(B2) | 2008-03-01 | Ivancic et al. | Applicant |
| US7356736(B2) | 2008-04-01 | Natvig | Applicant |
| US7386888(B2) | 2008-06-01 | Liang et al. | Applicant |
| US7392542(B2) | 2008-06-01 | Bucher | Applicant |
| US7418729(B2) | 2008-08-01 | Szor | Applicant |
| US7428300(B1) | 2008-09-01 | Drew et al. | Applicant |
| US7441272(B2) | 2008-10-01 | Durham et al. | Applicant |
| US7448084(B1) | 2008-11-01 | Apap et al. | Applicant |
| US7458098(B2) | 2008-11-01 | Judge et al. | Applicant |
| US7464404(B2) | 2008-12-01 | Carpenter et al. | Applicant |
| US7464407(B2) | 2008-12-01 | Nakae et al. | Applicant |
| US7467408(B1) | 2008-12-01 | O'Toole, Jr. | Applicant |
| US7478428(B1) | 2009-01-01 | Thomlinson | Applicant |
| US7480773(B1) | 2009-01-01 | Reed | Applicant |
| US7487543(B2) | 2009-02-01 | Arnold et al. | Applicant |
| US7496960(B1) | 2009-02-01 | Chen et al. | Applicant |
| US7496961(B2) | 2009-02-01 | Zimmer et al. | Applicant |
| US7519990(B1) | 2009-04-01 | Xie | Applicant |
| US7523493(B2) | 2009-04-01 | Liang et al. | Applicant |
| US7530104(B1) | 2009-05-01 | Thrower et al. | Applicant |
| US7540025(B2) | 2009-05-01 | Tzadikario | Applicant |
| US7565550(B2) | 2009-07-01 | Liang et al. | Applicant |
| US7568233(B1) | 2009-07-01 | Szor et al. | Applicant |
| US7584455(B2) | 2009-09-01 | Ball | Applicant |
| US7603715(B2) | 2009-10-01 | Costa et al. | Applicant |
| US7607171(B1) | 2009-10-01 | Marsden et al. | Applicant |
| US7639714(B2) | 2009-12-01 | Stolfo et al. | Applicant |
| US7644441(B2) | 2010-01-01 | Schmid et al. | Applicant |
| US7657419(B2) | 2010-02-01 | van der Made | Applicant |
| US7676841(B2) | 2010-03-01 | Sobchuk et al. | Applicant |
| US7698548(B2) | 2010-04-01 | Shelest et al. | Applicant |
| US7707633(B2) | 2010-04-01 | Danford et al. | Applicant |
| US7712136(B2) | 2010-05-01 | Sprosts et al. | Applicant |
| US7730011(B1) | 2010-06-01 | Deninger et al. | Applicant |
| US7739740(B1) | 2010-06-01 | Nachenberg et al. | Applicant |
| US7779463(B2) | 2010-08-01 | Stolfo et al. | Applicant |
| US7784097(B1) | 2010-08-01 | Stolfo et al. | Applicant |
| US7832008(B1) | 2010-11-01 | Kraemer | Applicant |
| US7836502(B1) | 2010-11-01 | Zhao et al. | Applicant |
| US7849506(B1) | 2010-12-01 | Dansey et al. | Applicant |
| US7854007(B2) | 2010-12-01 | Sprosts et al. | Applicant |
| US7869073(B2) | 2011-01-01 | Oshima | Applicant |
| US7877803(B2) | 2011-01-01 | Enstone et al. | Applicant |
| US7904959(B2) | 2011-03-01 | Sidiroglou et al. | Applicant |
| US7908660(B2) | 2011-03-01 | Bahl | Applicant |
| US7930738(B1) | 2011-04-01 | Petersen | Applicant |
| US7937761(B1) | 2011-05-01 | Bennett | Applicant |
| US7949849(B2) | 2011-05-01 | Lowe et al. | Applicant |
| US7996556(B2) | 2011-08-01 | Raghavan et al. | Applicant |
| US7996836(B1) | 2011-08-01 | McCorkendale et al. | Applicant |
| US7996904(B1) | 2011-08-01 | Chiueh et al. | Applicant |
| US7996905(B2) | 2011-08-01 | Arnold et al. | Applicant |
| US8006305(B2) | 2011-08-01 | Aziz | Applicant |
| US8010667(B2) | 2011-08-01 | Zhang et al. | Applicant |
| US8020206(B2) | 2011-09-01 | Hubbard et al. | Applicant |
| US8028338(B1) | 2011-09-01 | Schneider et al. | Applicant |
| US8042184(B1) | 2011-10-01 | Batenin | Applicant |
| US8045094(B2) | 2011-10-01 | Teragawa | Applicant |
| US8045458(B2) | 2011-10-01 | Alperovitch et al. | Applicant |
| US8069484(B2) | 2011-11-01 | McMillan et al. | Applicant |
| US8087086(B1) | 2011-12-01 | Lai et al. | Applicant |
| US8171553(B2) | 2012-05-01 | Aziz et al. | Applicant |
| US8176049(B2) | 2012-05-01 | Deninger et al. | Applicant |
| US8176480(B1) | 2012-05-01 | Spertus | Applicant |
| US8201246(B1) | 2012-06-01 | Wu et al. | Applicant |
| US8204984(B1) | 2012-06-01 | Aziz et al. | Applicant |
| US8214905(B1) | 2012-07-01 | Doukhvalov et al. | Applicant |
| US8220055(B1) | 2012-07-01 | Kennedy | Applicant |
| US8225288(B2) | 2012-07-01 | Miller et al. | Applicant |
| US8225373(B2) | 2012-07-01 | Kraemer | Applicant |
| US8233882(B2) | 2012-07-01 | Rogel | Applicant |
| US8234640(B1) | 2012-07-01 | Fitzgerald et al. | Applicant |
| US8234709(B2) | 2012-07-01 | Viljoen et al. | Applicant |
| US8239944(B1) | 2012-08-01 | Nachenberg et al. | Applicant |
| US8260914(B1) | 2012-09-01 | Ranjan | Applicant |
| US8266091(B1) | 2012-09-01 | Gubin et al. | Applicant |
| US8286251(B2) | 2012-10-01 | Eker et al. | Applicant |
| US8291499(B2) | 2012-10-01 | Aziz et al. | Applicant |
| US8307435(B1) | 2012-11-01 | Mann et al. | Applicant |
| US8307443(B2) | 2012-11-01 | Wang et al. | Applicant |
| US8312545(B2) | 2012-11-01 | Tuvell et al. | Applicant |
| US8321936(B1) | 2012-11-01 | Green et al. | Applicant |
| US8321941(B2) | 2012-11-01 | Tuvell et al. | Applicant |
| US8332571(B1) | 2012-12-01 | Edwards, Sr. | Applicant |
| US8365286(B2) | 2013-01-01 | Poston | Applicant |
| US8365297(B1) | 2013-01-01 | Parshin et al. | Applicant |
| US8370938(B1) | 2013-02-01 | Daswani et al. | Applicant |
| US8370939(B2) | 2013-02-01 | Zaitsev et al. | Applicant |
| US8375444(B2) | 2013-02-01 | Aziz et al. | Applicant |
| US8381299(B2) | 2013-02-01 | Stolfo et al. | Applicant |
| US8402529(B1) | 2013-03-01 | Green et al. | Applicant |
| US8464340(B2) | 2013-06-01 | Ahn et al. | Applicant |
| US8479174(B2) | 2013-07-01 | Chiriac | Applicant |
| US8479276(B1) | 2013-07-01 | Vaystikh et al. | Applicant |
| US8479291(B1) | 2013-07-01 | Bodke | Applicant |
| US8510827(B1) | 2013-08-01 | Leake et al. | Applicant |
| US8510828(B1) | 2013-08-01 | Guo et al. | Applicant |
| US8510842(B2) | 2013-08-01 | Amit et al. | Applicant |
| US8516478(B1) | 2013-08-01 | Edwards et al. | Applicant |
| US8516590(B1) | 2013-08-01 | Ranadive et al. | Applicant |
| US8516593(B2) | 2013-08-01 | Aziz | Applicant |
| US8522348(B2) | 2013-08-01 | Chen et al. | Applicant |
| US8528086(B1) | 2013-09-01 | Aziz | Applicant |
| US8533824(B2) | 2013-09-01 | Hutton et al. | Applicant |
| US8539582(B1) | 2013-09-01 | Aziz et al. | Applicant |
| US8549638(B2) | 2013-10-01 | Aziz | Applicant |
| US8555391(B1) | 2013-10-01 | Demir et al. | Applicant |
| US8561177(B1) | 2013-10-01 | Aziz et al. | Applicant |
| US8566946(B1) | 2013-10-01 | Aziz et al. | Applicant |
| US8584094(B2) | 2013-11-01 | Dadhia et al. | Applicant |
| US8584234(B1) | 2013-11-01 | Sobel et al. | Applicant |
| US8584239(B2) | 2013-11-01 | Aziz et al. | Applicant |
| US8595834(B2) | 2013-11-01 | Xie et al. | Applicant |
| US8627476(B1) | 2014-01-01 | Satish et al. | Applicant |
| US8635696(B1) | 2014-01-01 | Aziz | Applicant |
| US8682054(B2) | 2014-03-01 | Xue et al. | Applicant |
| US8682812(B1) | 2014-03-01 | Ranjan | Applicant |
| US8689333(B2) | 2014-04-01 | Aziz | Applicant |
| US8695096(B1) | 2014-04-01 | Zhang | Applicant |
| US8713631(B1) | 2014-04-01 | Pavlyushchik | Applicant |
| US8713681(B2) | 2014-04-01 | Silberman et al. | Applicant |
| US8726392(B1) | 2014-05-01 | McCorkendale et al. | Applicant |
| US8739280(B2) | 2014-05-01 | Chess et al. | Applicant |
| US8776229(B1) | 2014-07-01 | Aziz | Applicant |
| US8782792(B1) | 2014-07-01 | Bodke | Applicant |
| US8789172(B2) | 2014-07-01 | Stolfo et al. | Applicant |
| US8789178(B2) | 2014-07-01 | Kejriwal et al. | Applicant |
| US8793787(B2) | 2014-07-01 | Ismael et al. | Applicant |
| US8805947(B1) | 2014-08-01 | Kuzkin et al. | Applicant |
| US8806647(B1) | 2014-08-01 | Daswani et al. | Applicant |
| US8832829(B2) | 2014-09-01 | Manni et al. | Applicant |
| US8850570(B1) | 2014-09-01 | Ramzan | Applicant |
| US8850571(B2) | 2014-09-01 | Staniford et al. | Applicant |
| US8881234(B2) | 2014-11-01 | Narasimhan et al. | Applicant |
| US8881282(B1) | 2014-11-01 | Aziz et al. | Applicant |
| US8898788(B1) | 2014-11-01 | Aziz et al. | Applicant |
| US8935779(B2) | 2015-01-01 | Manni et al. | Applicant |
| US8984638(B1) | 2015-03-01 | Aziz et al. | Applicant |
| US8990939(B2) | 2015-03-01 | Staniford et al. | Applicant |
| US8990944(B1) | 2015-03-01 | Singh et al. | Applicant |
| US8997219(B2) | 2015-03-01 | Staniford et al. | Applicant |
| US9009822(B1) | 2015-04-01 | Ismael et al. | Applicant |
| US9009823(B1) | 2015-04-01 | Ismael et al. | Applicant |
| US9027135(B1) | 2015-05-01 | Aziz | Applicant |
| US9071638(B1) | 2015-06-01 | Aziz et al. | Applicant |
| US9104867(B1) | 2015-08-01 | Thioux et al. | Applicant |
| US9106694(B2) | 2015-08-01 | Aziz et al. | Applicant |
| US9118715(B2) | 2015-08-01 | Staniford et al. | Applicant |
| US2001/0005889(A1) | 2001-06-01 | Albrecht | Applicant |
| US2001/0047326(A1) | 2001-11-01 | Broadbent et al. | Applicant |
| US2002/0018903(A1) | 2002-02-01 | Kokubo et al. | Applicant |
| US2002/0038430(A1) | 2002-03-01 | Edwards et al. | Applicant |
| US2002/0091819(A1) | 2002-07-01 | Melchione et al. | Applicant |
| US2002/0095607(A1) | 2002-07-01 | Lin-Hendel | Applicant |
| US2002/0116627(A1) | 2002-08-01 | Tarbotton et al. | Applicant |
| US2002/0144156(A1) | 2002-10-01 | Copeland | Applicant |
| US2002/0162015(A1) | 2002-10-01 | Tang | Applicant |
| US2002/0166063(A1) | 2002-11-01 | Lachman et al. | Applicant |
| US2002/0169952(A1) | 2002-11-01 | DiSanto et al. | Applicant |
| US2002/0184528(A1) | 2002-12-01 | Shevenell et al. | Applicant |
| US2002/0188887(A1) | 2002-12-01 | Largman et al. | Applicant |
| US2002/0194490(A1) | 2002-12-01 | Halperin et al. | Applicant |
| US2003/0074578(A1) | 2003-04-01 | Ford et al. | Applicant |
| US2003/0084318(A1) | 2003-05-01 | Schertz | Applicant |
| US2003/0101381(A1) | 2003-05-01 | Mateev et al. | Applicant |
| US2003/0115483(A1) | 2003-06-01 | Liang | Applicant |
| US2003/0188190(A1) | 2003-10-01 | Aaron et al. | Applicant |
| US2003/0191957(A1) | 2003-10-01 | Hypponen et al. | Applicant |
| US2003/0200460(A1) | 2003-10-01 | Morota et al. | Applicant |
| US2003/0212902(A1) | 2003-11-01 | van der Made | Applicant |
| US2003/0229801(A1) | 2003-12-01 | Kouznetsov et al. | Applicant |
| US2003/0237000(A1) | 2003-12-01 | Denton et al. | Applicant |
| US2004/0003323(A1) | 2004-01-01 | Bennett et al. | Applicant |
| US2004/0015712(A1) | 2004-01-01 | Szor | Applicant |
| US2004/0019832(A1) | 2004-01-01 | Arnold et al. | Applicant |
| US2004/0047356(A1) | 2004-03-01 | Bauer | Applicant |
| US2004/0083408(A1) | 2004-04-01 | Spiegel et al. | Applicant |
| US2004/0088581(A1) | 2004-05-01 | Brawn et al. | Applicant |
| US2004/0093513(A1) | 2004-05-01 | Cantrell et al. | Applicant |
| US2004/0111531(A1) | 2004-06-01 | Staniford et al. | Applicant |
| US2004/0117478(A1) | 2004-06-01 | Triulzi et al. | Applicant |
| US2004/0117624(A1) | 2004-06-01 | Brandt et al. | Applicant |
| US2004/0128355(A1) | 2004-07-01 | Chao et al. | Applicant |
| US2004/0165588(A1) | 2004-08-01 | Pandya | Applicant |
| US2004/0236963(A1) | 2004-11-01 | Danford et al. | Applicant |
| US2004/0243349(A1) | 2004-12-01 | Greifeneder et al. | Applicant |
| US2004/0249911(A1) | 2004-12-01 | Alkhatib et al. | Applicant |
| US2004/0255161(A1) | 2004-12-01 | Cavanaugh | Applicant |
| US2004/0268147(A1) | 2004-12-01 | Wiederin et al. | Applicant |
| US2005/0005159(A1) | 2005-01-01 | Oliphant | Applicant |
| US2005/0021740(A1) | 2005-01-01 | Bar et al. | Applicant |
| US2005/0033960(A1) | 2005-02-01 | Vialen et al. | Applicant |
| US2005/0033989(A1) | 2005-02-01 | Poletto et al. | Applicant |
| US2005/0050148(A1) | 2005-03-01 | Mohammadioun et al. | Applicant |
| US2005/0086523(A1) | 2005-04-01 | Zimmer et al. | Applicant |
| US2005/0091513(A1) | 2005-04-01 | Mitomo et al. | Applicant |
| US2005/0091533(A1) | 2005-04-01 | Omote et al. | Applicant |
| US2005/0091652(A1) | 2005-04-01 | Ross et al. | Applicant |
| US2005/0108562(A1) | 2005-05-01 | Khazan et al. | Applicant |
| US2005/0114663(A1) | 2005-05-01 | Cornell et al. | Applicant |
| US2005/0125195(A1) | 2005-06-01 | Brendel | Applicant |
| US2005/0149726(A1) | 2005-07-01 | Joshi et al. | Applicant |
| US2005/0157662(A1) | 2005-07-01 | Bingham et al. | Applicant |
| US2005/0183143(A1) | 2005-08-01 | Anderholm et al. | Applicant |
| US2005/0201297(A1) | 2005-09-01 | Peikari | Applicant |
| US2005/0210533(A1) | 2005-09-01 | Copeland et al. | Applicant |
| US2005/0238005(A1) | 2005-10-01 | Chen et al. | Applicant |
| US2005/0240781(A1) | 2005-10-01 | Gassoway | Applicant |
| US2005/0262562(A1) | 2005-11-01 | Gassoway | Applicant |
| US2005/0265331(A1) | 2005-12-01 | Stolfo | Applicant |
| US2005/0283839(A1) | 2005-12-01 | Cowburn | Applicant |
| US2006/0010495(A1) | 2006-01-01 | Cohen et al. | Applicant |
| US2006/0015416(A1) | 2006-01-01 | Hoffman et al. | Applicant |
| US2006/0015715(A1) | 2006-01-01 | Anderson | Applicant |
| US2006/0015747(A1) | 2006-01-01 | Van de Ven | Applicant |
| US2006/0021029(A1) | 2006-01-01 | Brickell et al. | Applicant |
| US2006/0021054(A1) | 2006-01-01 | Costa et al. | Applicant |
| US2006/0031476(A1) | 2006-02-01 | Mathes et al. | Applicant |
| US2006/0047665(A1) | 2006-03-01 | Neil | Applicant |
| US2006/0070130(A1) | 2006-03-01 | Costea et al. | Applicant |
| US2006/0075496(A1) | 2006-04-01 | Carpenter et al. | Applicant |
| US2006/0095968(A1) | 2006-05-01 | Portolani et al. | Applicant |
| US2006/0101516(A1) | 2006-05-01 | Sudaharan et al. | Applicant |
| US2006/0101517(A1) | 2006-05-01 | Banzhof et al. | Applicant |
| US2006/0117385(A1) | 2006-06-01 | Mester et al. | Applicant |
| US2006/0123477(A1) | 2006-06-01 | Raghavan et al. | Applicant |
| US2006/0129382(A1) | 2006-06-01 | Anand | Examiner |
| US2006/0143709(A1) | 2006-06-01 | Brooks et al. | Applicant |
| US2006/0150249(A1) | 2006-07-01 | Gassen et al. | Applicant |
| US2006/0161983(A1) | 2006-07-01 | Cothrell et al. | Applicant |
| US2006/0161987(A1) | 2006-07-01 | Levy-Yurista | Applicant |
| US2006/0161989(A1) | 2006-07-01 | Reshef et al. | Applicant |
| US2006/0164199(A1) | 2006-07-01 | Gilde et al. | Applicant |
| US2006/0173992(A1) | 2006-08-01 | Weber et al. | Applicant |
| US2006/0179147(A1) | 2006-08-01 | Tran et al. | Applicant |
| US2006/0184632(A1) | 2006-08-01 | Marino et al. | Applicant |
| US2006/0191010(A1) | 2006-08-01 | Benjamin | Applicant |
| US2006/0221956(A1) | 2006-10-01 | Narayan et al. | Applicant |
| US2006/0236393(A1) | 2006-10-01 | Kramer et al. | Applicant |
| US2006/0242709(A1) | 2006-10-01 | Seinfeld et al. | Applicant |
| US2006/0248519(A1) | 2006-11-01 | Jaeger et al. | Applicant |
| US2006/0248582(A1) | 2006-11-01 | Panjwani et al. | Applicant |
| US2006/0251104(A1) | 2006-11-01 | Koga | Applicant |
| US2006/0253906(A1) | 2006-11-01 | Rubin | Examiner |
| US2006/0288417(A1) | 2006-12-01 | Bookbinder et al. | Applicant |
| US2007/0006288(A1) | 2007-01-01 | Mayfield et al. | Applicant |
| US2007/0006313(A1) | 2007-01-01 | Porras et al. | Applicant |
| US2007/0011174(A1) | 2007-01-01 | Takaragi et al. | Applicant |
| US2007/0016951(A1) | 2007-01-01 | Piccard et al. | Applicant |
| US2007/0033645(A1) | 2007-02-01 | Jones | Applicant |
| US2007/0038943(A1) | 2007-02-01 | FitzGerald et al. | Applicant |
| US2007/0064689(A1) | 2007-03-01 | Shin et al. | Applicant |
| US2007/0074169(A1) | 2007-03-01 | Chess et al. | Applicant |
| US2007/0094730(A1) | 2007-04-01 | Bhikkaji et al. | Applicant |
| US2007/0101435(A1) | 2007-05-01 | Konanka et al. | Applicant |
| US2007/0128855(A1) | 2007-06-01 | Cho et al. | Applicant |
| US2007/0142030(A1) | 2007-06-01 | Sinha et al. | Applicant |
| US2007/0143827(A1) | 2007-06-01 | Nicodemus et al. | Applicant |
| US2007/0156895(A1) | 2007-07-01 | Vuong | Applicant |
| US2007/0157180(A1) | 2007-07-01 | Tillmann et al. | Applicant |
| US2007/0157306(A1) | 2007-07-01 | Elrod et al. | Applicant |
| US2007/0168988(A1) | 2007-07-01 | Eisner et al. | Applicant |
| US2007/0169195(A1) | 2007-07-01 | Anand | Examiner |
| US2007/0171824(A1) | 2007-07-01 | Ruello et al. | Applicant |
| US2007/0174915(A1) | 2007-07-01 | Gribble et al. | Applicant |
| US2007/0192500(A1) | 2007-08-01 | Lum | Applicant |
| US2007/0192858(A1) | 2007-08-01 | Lum | Applicant |
| US2007/0198275(A1) | 2007-08-01 | Malden et al. | Applicant |
| US2007/0208822(A1) | 2007-09-01 | Wang et al. | Applicant |
| US2007/0220607(A1) | 2007-09-01 | Sprosts et al. | Applicant |
| US2007/0240217(A1) | 2007-10-01 | Tuvell | Examiner |
| US2007/0240218(A1) | 2007-10-01 | Tuvell et al. | Applicant |
| US2007/0240219(A1) | 2007-10-01 | Tuvell et al. | Applicant |
| US2007/0240220(A1) | 2007-10-01 | Tuvell et al. | Applicant |
| US2007/0240222(A1) | 2007-10-01 | Tuvell et al. | Applicant |
| US2007/0250930(A1) | 2007-10-01 | Aziz et al. | Applicant |
| US2007/0256132(A2) | 2007-11-01 | Oliphant | Applicant |
| US2007/0271446(A1) | 2007-11-01 | Nakamura | Applicant |
| US2008/0005782(A1) | 2008-01-01 | Aziz | Applicant |
| US2008/0028463(A1) | 2008-01-01 | Dagon et al. | Applicant |
| US2008/0032556(A1) | 2008-02-01 | Schreier | Applicant |
| US2008/0040710(A1) | 2008-02-01 | Chiriac | Applicant |
| US2008/0046781(A1) | 2008-02-01 | Childs et al. | Applicant |
| US2008/0066179(A1) | 2008-03-01 | Liu | Applicant |
| US2008/0072326(A1) | 2008-03-01 | Danford et al. | Applicant |
| US2008/0077793(A1) | 2008-03-01 | Tan et al. | Applicant |
| US2008/0080518(A1) | 2008-04-01 | Hoeflin et al. | Applicant |
| US2008/0086720(A1) | 2008-04-01 | Lekel | Applicant |
| US2008/0098476(A1) | 2008-04-01 | Syversen | Applicant |
| US2008/0120722(A1) | 2008-05-01 | Sima et al. | Applicant |
| US2008/0134178(A1) | 2008-06-01 | Fitzgerald et al. | Applicant |
| US2008/0134334(A1) | 2008-06-01 | Kim et al. | Applicant |
| US2008/0141376(A1) | 2008-06-01 | Clausen et al. | Applicant |
| US2008/0184373(A1) | 2008-07-01 | Traut et al. | Applicant |
| US2008/0189787(A1) | 2008-08-01 | Arnold et al. | Applicant |
| US2008/0201778(A1) | 2008-08-01 | Guo et al. | Applicant |
| US2008/0209557(A1) | 2008-08-01 | Herley et al. | Applicant |
| US2008/0215742(A1) | 2008-09-01 | Goldszmidt et al. | Applicant |
| US2008/0222729(A1) | 2008-09-01 | Chen et al. | Applicant |
| US2008/0263665(A1) | 2008-10-01 | Ma et al. | Applicant |
| US2008/0295172(A1) | 2008-11-01 | Bohacek | Applicant |
| US2008/0301810(A1) | 2008-12-01 | Lehane et al. | Applicant |
| US2008/0307524(A1) | 2008-12-01 | Singh et al. | Applicant |
| US2008/0313738(A1) | 2008-12-01 | Enderby | Applicant |
| US2008/0320594(A1) | 2008-12-01 | Jiang | Applicant |
| US2009/0003317(A1) | 2009-01-01 | Kasralikar et al. | Applicant |
| US2009/0007100(A1) | 2009-01-01 | Field et al. | Applicant |
| US2009/0013408(A1) | 2009-01-01 | Schipka | Applicant |
| US2009/0031423(A1) | 2009-01-01 | Liu et al. | Applicant |
| US2009/0036111(A1) | 2009-02-01 | Danford et al. | Applicant |
| US2009/0037835(A1) | 2009-02-01 | Goldman | Applicant |
| US2009/0044024(A1) | 2009-02-01 | Oberheide et al. | Applicant |
| US2009/0044274(A1) | 2009-02-01 | Budko et al. | Applicant |
| US2009/0064332(A1) | 2009-03-01 | Porras et al. | Applicant |
| US2009/0077666(A1) | 2009-03-01 | Chen et al. | Applicant |
| US2009/0083369(A1) | 2009-03-01 | Marmor | Applicant |
| US2009/0083855(A1) | 2009-03-01 | Apap et al. | Applicant |
| US2009/0089879(A1) | 2009-04-01 | Wang et al. | Applicant |
| US2009/0094697(A1) | 2009-04-01 | Provos et al. | Applicant |
| US2009/0113425(A1) | 2009-04-01 | Ports et al. | Applicant |
| US2009/0125976(A1) | 2009-05-01 | Wassermann et al. | Applicant |
| US2009/0126015(A1) | 2009-05-01 | Monastyrsky et al. | Applicant |
| US2009/0126016(A1) | 2009-05-01 | Sobko et al. | Applicant |
| US2009/0133125(A1) | 2009-05-01 | Choi et al. | Applicant |
| US2009/0144823(A1) | 2009-06-01 | Lamastra et al. | Applicant |
| US2009/0158430(A1) | 2009-06-01 | Borders | Applicant |
| US2009/0172815(A1) | 2009-07-01 | Gu et al. | Applicant |
| US2009/0187992(A1) | 2009-07-01 | Poston | Applicant |
| US2009/0193293(A1) | 2009-07-01 | Stolfo | Examiner |
| US2009/0199296(A1) | 2009-08-01 | Xie et al. | Applicant |
| US2009/0228233(A1) | 2009-09-01 | Anderson et al. | Applicant |
| US2009/0241187(A1) | 2009-09-01 | Troyansky | Applicant |
| US2009/0241190(A1) | 2009-09-01 | Todd et al. | Applicant |
| US2009/0265692(A1) | 2009-10-01 | Godefroid et al. | Applicant |
| US2009/0271867(A1) | 2009-10-01 | Zhang | Applicant |
| US2009/0300415(A1) | 2009-12-01 | Zhang et al. | Applicant |
| US2009/0300761(A1) | 2009-12-01 | Park et al. | Applicant |
| US2009/0328185(A1) | 2009-12-01 | Berg et al. | Applicant |
| US2009/0328221(A1) | 2009-12-01 | Blumfield et al. | Applicant |
| US2010/0005146(A1) | 2010-01-01 | Drako et al. | Applicant |
| US2010/0011205(A1) | 2010-01-01 | McKenna | Applicant |
| US2010/0017546(A1) | 2010-01-01 | Poo et al. | Applicant |
| US2010/0031353(A1) | 2010-02-01 | Thomas et al. | Applicant |
| US2010/0037314(A1) | 2010-02-01 | Perdisci et al. | Applicant |
| US2010/0043073(A1) | 2010-02-01 | Kuwamura | Applicant |
| US2010/0054278(A1) | 2010-03-01 | Stolfo et al. | Applicant |
| US2010/0058474(A1) | 2010-03-01 | Hicks | Applicant |
| US2010/0064044(A1) | 2010-03-01 | Nonoyama | Applicant |
| US2010/0077481(A1) | 2010-03-01 | Polyakov et al. | Applicant |
| US2010/0083376(A1) | 2010-04-01 | Pereira et al. | Applicant |
| US2010/0115621(A1) | 2010-05-01 | Staniford et al. | Applicant |
| US2010/0132038(A1) | 2010-05-01 | Zaitsev | Applicant |
| US2010/0154056(A1) | 2010-06-01 | Smith et al. | Applicant |
| US2010/0180344(A1) | 2010-07-01 | Malyshev et al. | Applicant |
| US2010/0192223(A1) | 2010-07-01 | Ismael et al. | Applicant |
| US2010/0220863(A1) | 2010-09-01 | Dupaquis et al. | Applicant |
| US2010/0235831(A1) | 2010-09-01 | Dittmer | Applicant |
| US2010/0251104(A1) | 2010-09-01 | Massand | Applicant |
| US2010/0281102(A1) | 2010-11-01 | Chinta et al. | Applicant |
| US2010/0281541(A1) | 2010-11-01 | Stolfo et al. | Applicant |
| US2010/0281542(A1) | 2010-11-01 | Stolfo et al. | Applicant |
| US2010/0287260(A1) | 2010-11-01 | Peterson et al. | Applicant |
| US2010/0299754(A1) | 2010-11-01 | Amit et al. | Applicant |
| US2010/0306173(A1) | 2010-12-01 | Frank | Applicant |
| US2011/0004737(A1) | 2011-01-01 | Greenebaum | Applicant |
| US2011/0025504(A1) | 2011-02-01 | Lyon et al. | Applicant |
| US2011/0041179(A1) | 2011-02-01 | St hlberg | Applicant |
| US2011/0047594(A1) | 2011-02-01 | Mahaffey et al. | Applicant |
| US2011/0047620(A1) | 2011-02-01 | Mahaffey et al. | Applicant |
| US2011/0055907(A1) | 2011-03-01 | Narasimhan et al. | Applicant |
| US2011/0078794(A1) | 2011-03-01 | Manni et al. | Applicant |
| US2011/0093951(A1) | 2011-04-01 | Aziz | Applicant |
| US2011/0099620(A1) | 2011-04-01 | Stavrou et al. | Applicant |
| US2011/0099633(A1) | 2011-04-01 | Aziz | Applicant |
| US2011/0113231(A1) | 2011-05-01 | Kaminsky | Applicant |
| US2011/0145918(A1) | 2011-06-01 | Jung et al. | Applicant |
| US2011/0145920(A1) | 2011-06-01 | Mahaffey et al. | Applicant |
| US2011/0145934(A1) | 2011-06-01 | Abramovici et al. | Applicant |
| US2011/0167493(A1) | 2011-07-01 | Song et al. | Applicant |
| US2011/0167494(A1) | 2011-07-01 | Bowen et al. | Applicant |
| US2011/0173460(A1) | 2011-07-01 | Ito et al. | Applicant |
| US2011/0219449(A1) | 2011-09-01 | St. Neitzel et al. | Applicant |
| US2011/0219450(A1) | 2011-09-01 | McDougal et al. | Applicant |
| US2011/0225624(A1) | 2011-09-01 | Sawhney et al. | Applicant |
| US2011/0225655(A1) | 2011-09-01 | Niemela et al. | Applicant |
| US2011/0247072(A1) | 2011-10-01 | Staniford et al. | Applicant |
| US2011/0265182(A1) | 2011-10-01 | Peinado et al. | Applicant |
| US2011/0289582(A1) | 2011-11-01 | Kejriwal et al. | Applicant |
| US2011/0302587(A1) | 2011-12-01 | Nishikawa et al. | Applicant |
| US2011/0307954(A1) | 2011-12-01 | Melnik et al. | Applicant |
| US2011/0307955(A1) | 2011-12-01 | Kaplan et al. | Applicant |
| US2011/0307956(A1) | 2011-12-01 | Yermakov et al. | Applicant |
| US2011/0314546(A1) | 2011-12-01 | Aziz et al. | Applicant |
| US2012/0023593(A1) | 2012-01-01 | Puder et al. | Applicant |
| US2012/0054869(A1) | 2012-03-01 | Yen et al. | Applicant |
| US2012/0066698(A1) | 2012-03-01 | Yanoo | Applicant |
| US2012/0079596(A1) | 2012-03-01 | Thomas et al. | Applicant |
| US2012/0084859(A1) | 2012-04-01 | Radinsky et al. | Applicant |
| US2012/0110667(A1) | 2012-05-01 | Zubrilin et al. | Applicant |
| US2012/0117652(A1) | 2012-05-01 | Manni et al. | Applicant |
| US2012/0121154(A1) | 2012-05-01 | Xue et al. | Applicant |
| US2012/0124426(A1) | 2012-05-01 | Maybee et al. | Applicant |
| US2012/0167219(A1) | 2012-06-01 | Zaitsev | Examiner |
| US2012/0174186(A1) | 2012-07-01 | Aziz et al. | Applicant |
| US2012/0174196(A1) | 2012-07-01 | Bhogavilli et al. | Applicant |
| US2012/0174218(A1) | 2012-07-01 | McCoy et al. | Applicant |
| US2012/0198279(A1) | 2012-08-01 | Schroeder | Applicant |
| US2012/0210423(A1) | 2012-08-01 | Friedrichs et al. | Applicant |
| US2012/0222121(A1) | 2012-08-01 | Staniford et al. | Applicant |
| US2012/0255015(A1) | 2012-10-01 | Sahita et al. | Applicant |
| US2012/0255017(A1) | 2012-10-01 | Sallam | Applicant |
| US2012/0260342(A1) | 2012-10-01 | Dube et al. | Applicant |
| US2012/0266244(A1) | 2012-10-01 | Green et al. | Applicant |
| US2012/0278886(A1) | 2012-11-01 | Luna | Applicant |
| US2012/0297489(A1) | 2012-11-01 | Dequevy | Applicant |
| US2012/0330801(A1) | 2012-12-01 | McDougal et al. | Applicant |
| US2013/0014259(A1) | 2013-01-01 | Gribble et al. | Applicant |
| US2013/0036472(A1) | 2013-02-01 | Aziz | Applicant |
| US2013/0047257(A1) | 2013-02-01 | Aziz | Applicant |
| US2013/0074185(A1) | 2013-03-01 | McDougal et al. | Applicant |
| US2013/0086684(A1) | 2013-04-01 | Mohler | Applicant |
| US2013/0097699(A1) | 2013-04-01 | Balupari et al. | Applicant |
| US2013/0097706(A1) | 2013-04-01 | Titonis et al. | Applicant |
| US2013/0111587(A1) | 2013-05-01 | Goel et al. | Applicant |
| US2013/0117852(A1) | 2013-05-01 | Stute | Applicant |
| US2013/0117855(A1) | 2013-05-01 | Kim et al. | Applicant |
| US2013/0139264(A1) | 2013-05-01 | Brinkley et al. | Applicant |
| US2013/0160125(A1) | 2013-06-01 | Likhachev et al. | Applicant |
| US2013/0160127(A1) | 2013-06-01 | Jeong et al. | Applicant |
| US2013/0160130(A1) | 2013-06-01 | Mendelev et al. | Applicant |
| US2013/0160131(A1) | 2013-06-01 | Madou et al. | Applicant |
| US2013/0167236(A1) | 2013-06-01 | Sick | Applicant |
| US2013/0174214(A1) | 2013-07-01 | Duncan | Applicant |
| US2013/0185789(A1) | 2013-07-01 | Hagiwara et al. | Applicant |
| US2013/0185795(A1) | 2013-07-01 | Winn et al. | Applicant |
| US2013/0185798(A1) | 2013-07-01 | Saunders et al. | Applicant |
| US2013/0191915(A1) | 2013-07-01 | Antonakakis et al. | Applicant |
| US2013/0196649(A1) | 2013-08-01 | Paddon et al. | Applicant |
| US2013/0227691(A1) | 2013-08-01 | Aziz et al. | Applicant |
| US2013/0246370(A1) | 2013-09-01 | Bartram et al. | Applicant |
| US2013/0263260(A1) | 2013-10-01 | Mahaffey et al. | Applicant |
| US2013/0291109(A1) | 2013-10-01 | Staniford et al. | Applicant |
| US2013/0298243(A1) | 2013-11-01 | Kumar et al. | Applicant |
| US2014/0053260(A1) | 2014-02-01 | Gupta et al. | Applicant |
| US2014/0053261(A1) | 2014-02-01 | Gupta et al. | Applicant |
| US2014/0130158(A1) | 2014-05-01 | Wang et al. | Applicant |
| US2014/0137180(A1) | 2014-05-01 | Lukacs et al. | Applicant |
| US2014/0169762(A1) | 2014-06-01 | Ryu | Applicant |
| US2014/0179360(A1) | 2014-06-01 | Jackson et al. | Applicant |
| US2014/0328204(A1) | 2014-11-01 | Klotsche et al. | Applicant |
| US2014/0337836(A1) | 2014-11-01 | Ismael | Applicant |
| US2014/0351935(A1) | 2014-11-01 | Shao et al. | Applicant |
| US2015/0026810(A1) | 2015-01-01 | Friedrichs | Examiner |
| US2015/0096025(A1) | 2015-04-01 | Ismael | Applicant |
| GB2439806(A) | 2008-01-01 | Applicant | |
| GB2490431(A) | 2012-10-01 | Applicant | |
| WO223805(A2) | 2002-03-01 | Applicant | |
| WO206928 | 2003-11-01 | Applicant | |
| WO2007117636(A2) | 2007-10-01 | Applicant | |
| WO2008041950(A2) | 2008-04-01 | Applicant | |
| WO2011084431(A2) | 2011-07-01 | Applicant | |
| WO2011/112348(A1) | 2011-09-01 | Applicant | |
| WO2012/075336(A1) | 2012-06-01 | Applicant | |
| WO2012145066(A1) | 2012-10-01 | Applicant | |
| WO2013/067505(A1) | 2013-05-01 | Applicant |
Non-Patent Literature (81)
- Thomas H. Ptacek, and Timothy N. Newsham , “Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection”, Secure Networks, (“Ptacek”), (Jan. 1998).Applicant
- U.S. Appl. No. 11/717,475, filed Mar. 12, 2007 Final Office Action dated Feb. 27, 2013.Applicant
- U.S. Appl. No. 11/717,475, filed Mar. 12, 2007 Final Office Action dated Nov. 22, 2010.Applicant
- U.S. Appl. No. 11/717,475, filed Mar. 12, 2007 Non-Final Office Action dated Aug. 28, 2012.Applicant
- U.S. Appl. No. 11/717,475, filed Mar. 12, 2007 Non-Final Office Action dated May 6, 2010.Applicant
- U.S. Appl. No. 14/059,381, filed Oct. 21, 2013 Non-Final Office Action dated Oct. 29, 2014.Applicant
- Venezia, Paul, “NetDetector Captures Intrusions”, InfoWorld Issue 27, (“Venezia”), (Jul. 14, 2003).Applicant
- Whyte, et al., “DNS-Based Detection of Scanning Works in an Enterprise Network”, Proceedings of the 12th Annual Network and Distributed System Security Symposium, (Feb. 2005), 15 pages.Applicant
- Williamson, Mathew M., “Throttling Virses: Restricting Propagation to Defeat Malicious Mobile Code”, ACSAC Conference, Las Vegas, NV, USA, (Dec. 2002), pp. 1-9.Applicant
- Adobe Systems Incorporated, “PDF 32000-1:2008, Document management—Portable document format—Part1:PDF 1.7”, First Edition, Jul. 1, 2008, 756 pages.Applicant
- Apostolopoulos, George; hassapis, Constantinos; “V-eM: A cluster of Virtual Machines for Robust, Detailed, and High-Performance Network Emulation”, 14th IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, Sep. 11-14, 2006, pp. 117-126.Applicant
- Clark, John, Sylvian Leblanc,and Scott Knight. “Risks associated with usb hardware trojan devices used by insiders.” Systems Conference (SysCon), 2011 IEEE International. IEEE, 2011.Applicant
- FireEye Malware Analysis & Exchange Network, Malware Protection System, FireEye Inc., 2010.Applicant
- FireEye Malware Analysis, Modern Malware Forensics, FireEye Inc., 2010.Applicant
- FireEye v.6.0 Security Target, pp. 1-35, Version 1.1, FireEye Inc., May 2011.Applicant
- Gibler, Clint, et al. AndroidLeaks: automatically detecting potential privacy leaks in android applications on a large scale. Springer Berlin Heidelberg, 2012.Applicant
- Gregg Keizer: “Microsoft's HoneyMonkeys Show Patching Windows Works”, Aug. 8, 2005, XP055143386, Retrieved from the Internet: URL:https://web.archive.org/web/20121022220617/http://www.informationweek—.com/microsofts-honeymonkeys-show-patching-wi/167600716 [retrieved on Sep. 29, 2014].Applicant
- Heng Yin et al, Panorama: Capturing System-Wide Information Flow for Malware Detection and Analysis, Research Showcase @ CMU, Carnegie Mellon University, 2007.Applicant
- Idika et al., A-Survey-of-Malware-Detection-Techniques, Feb. 2, 2007, Department of Computer Science, Purdue University.Applicant
- Isohara, Takamasa, Keisuke Takemori, and Ayumu Kubota. “Kernel-based behavior analysis for android malware detection.” Computational intelligence and Security (CIS), 2011 Seventh International Conference on. IEEE, 2011.Applicant
- Kevin A Roundy et al: “Hybrid Analysis and Control of Malware”, Sep. 15, 2010, Recent Advances in Intrusion Detection, Springer Berlin Heidelberg, Berlin, Heidelberg, pp. 317-338, XP019150454 ISBN:978-3-642-15511-6.Applicant
- Leading Colleges Select FireEye to Stop Malware-Related Data Breaches, FireEye Inc., 2009.Applicant
- Li et al., A VMM-Based System Call Interposition Framework for Program Monitoring, Dec. 2010, IEEE 16th International Conference on Parallel and Distributed Systems, pp. 706-711.Applicant
- Lindorfer, Martina, Clemens Kolbitsch, and Paolo Milani Comparetti. “Detecting environment-sensitive malware.” Recent Advances in Intrusion Detection. Springer Berlin Heidelberg, 2011.Applicant
- Mori, Detecting Unknown Computer Viruses, 2004, Springer-Verlag Berlin Heidelberg.Applicant
- Oberheide et al., CloudAV.sub.—N-Version Antivirus in the Network Cloud, 17th USENIX Security Symposium USENIX Security '08 Jul. 28-Aug. 1, 2008 San Jose, CA.Applicant
- U.S. Pat. No. 8,171,553 filed Apr. 20, 2006, Inter Parties Review Decision dated Jul. 10, 2015.Applicant
- U.S. Pat. No. 8,291,499 filed Mar. 16, 2012, Inter Parties Review Decision dated Jul. 10, 2015.Applicant
- Wahid et al., Characterising the Evolution in Scanning Activity of Suspicious Hosts, Oct. 2009, Third International Conference on Network and System Security, pp. 344-350.Applicant
- Yuhei Kawakoya et al: “Memory behavior-based automatic malware unpacking in stealth debugging environment”, Malicious and Unwanted Software (Malware), 2010 5th International Conference on, IEEE, Piscataway, NJ, USA, Oct. 19, 2010, pp. 39-46, XP031833827, ISBN:978-1-4244-8-9353-1.Applicant
- Zhang et al., The Effects of Threading, Infection Time, and Multiple-Attacker Collaboration on Malware Propagation, Sep. 2009, IEEE 28th International Symposium on Reliable Distributed Systems, pp. 73-82.Applicant
- “Network Security: NetDetector—Network Intrusion Forensic System (NIFS) Whitepaper”, (“NetDetector Whitepaper”), (2003).Applicant
- “Packet”, Microsoft Computer Dictionary Microsoft Press, (Mar. 2002), 1 page.Applicant
- “When Virtual is Better Than Real”, IEEEXplore Digital Library, available at, http://ieeexplore.ieee.org/xpl/articleDetails.jso?reload=true&arnumber=990073, (Dec. 7, 2013).Applicant
- Abdullah, et al., Visualizing Network Data for Intrusion Detection, 2005 IEEE Workshop on Information Assurance and Security, pp. 100-108.Applicant
- Adetoye, Adedayo, et al., “Network Intrusion Detection & Response System”, (“Adetoye”) (Sep. 2003).Applicant
- AltaVista Advanced Search Results. “attack vector identifier” Http://www.altavista.com/web/results?Itag=ody&pg=aq&aqmode=aqa=Event+Orchestrator . . . , (Accessed on Sep. 15, 2009).Applicant
- AltaVista Advanced Search Results. “Event Orchestrator”. Http://www.altavista.com/web/results?Itag=ody&pg=aq&aqmode=aqa=Event+Orchesrator . . . , (Accessed on Sep. 3, 2009).Applicant
- Aura, Tuomas, “Scanning electronic documents for personally identifiable information”, Proceedings of the 5th ACM workshop on Privacy in electronic society. ACM, 2006.Applicant
- Aziz, Ashar, System and Method for Malware Containment, U.S. Appl. No. 14/620,060, filed Feb. 11, 2015, non-Final Office Action dated Apr. 3, 2015.Applicant
- Baecher, “The Nepenthes Platform: An Efficient Approach to collect Malware”, Springer-verlaq Berlin Heidelberg, (2006), pp. 165-184.Applicant
- Baldi, Mario; Risso, Fulvio; “A Framework for Rapid Development and Portable Execution of Packet-Handling Applications”, 5th IEEE International Symposium Processing and Information Technology, Dec. 21, 2005, pp. 233-238.Applicant
- Bayer, et al., “Dynamic Analysis of Malicious Code”, J Comput Virol, Springer-Verlag, France., (2006), pp. 67-77.Applicant
- Boubalos, Chris , “extracting syslog data out of raw pcap dumps, seclists.org, Honeypots mailing list archives”, available at http://seclists,org/honeypots/2003/q2/319 (“Boubalos”), (Jun. 5, 2003).Applicant
- Chaudet, C., et al., “Optimal Positioning of Active and Passive Monitoring Devices”, International Conference on Emerging Networking Experiments and Technologies, Proceedings of the 2005 ACM Conference on Emerging Network Experiment and Technology, CoNEXT '05, Toulousse, France, (Oct. 2005), pp. 71-82.Applicant
- Cisco “Intrusion Prevention for the Cisco ASA 5500-x Series” Data Sheet (2012).Applicant
- Cisco, Configuring the Catalyst Switched Port Analyzer (SPAN) (“Cisco”), (1992-2003).Applicant
- Cohen, M.I., “PyFlag—An advanced network forensic framework”, Digital investigation 5, ELSEVIER, (2008), pp. S112-S120.Applicant
- Costa, M., et al., “Vigilante: End-to-End Containment of Internet Worms”, SOSP '05 Association for Computing Machinery, Inc., Brighton U.K., (Oct. 23-26, 2005).Applicant
- Crandall, J.R., et al., “Minos:Control Data Attack Prevention Orthogonal to Memory Model”, 37th International Symposium on Microarchitecture, Portland, Oregon, (Dec. 2004).Applicant
- Deutsch, P., ““Zlib compressed data format specification version 3.3” RFC 1950, (1996)”.Applicant
- Distler, “Malware Analysis: An Introduction”, SANS Institute InfoSec Reading Room, SANS Institute, (2007).Applicant
- Dunlap, George W. , et al., “ReVirt: Enabling Intrusion Analysis through Virtual-Machine Logging and Replay”, Proceeding of the 5th Symposium on Operating Systems Design and Implementation, USENIX Association, (“Dunlap”), (Dec. 9, 2002).Applicant
- Excerpt regarding First Printing Date for Merike Kaeo, Designing Network Security (“Kaeo”), (2005).Applicant
- Filiol, Eric , et al., “Combinatorial Optimisation of Worm Propagation on an Unknown Network”, International Journal of Computer Science 2.2 (2007).Applicant
- Goel, et al., Reconstructing System State for Intrusion Analysis, Apr. 2008 SIGOPS Operating Systems Review vol. 42 Issue 3, pp. 21-28.Applicant
- Hjelmvik, Erik, “Passive Network Security Analysis with NetworkMiner”, (In)SECURE, Issue 18, (Oct. 2008), pp. 1-100.Applicant
- IEEE Xplore Digital Library Sear Results for “detection of unknown computer worms”. Http//ieeexplore.ieee.org/searchresult.jsp?SortField=Score&SortOrder=desc&ResultC . . . (Accessed on Aug. 28, 2009).Applicant
- Kaeo, Merike, “Designing Network Security”, (“Kaeo”), (Nov. 2003).Applicant
- Kim, H., et al., “Autograph: Toward Automated, Distributed Worm Signature Detection”, Proceedings of the 13th Usenix Security Symposium (Security 2004), San Diego, (Aug. 2004), pp. 271-286.Applicant
- King, Samuel T., et al., “Operating System Support for Virtual Machines”, (“King”), 2003.Applicant
- Krasnyansky, Max, et al., Universal TUN/TAP driver, available at https://www.kernel.org/doc/Documentation/networking/tuntap.txt (2002) (“Krasnyansky”).Applicant
- Kreibich, C., et al., “Honeycomb-Creating Intrusion Detection Signatures Using Honeypots”, 2nd Workshop on Hot Topics in Networks (HotNets-11), Boston, USA, (2003).Applicant
- Kristoff, J., “Botnets, Detection and Mitigation: DNS-Based Techniques”, NU Security Day, (2005), 23 pages.Applicant
- Liljenstam, Michael, et al., “Simulating Realistic Network Traffic for Worm Warning System Design and Testing”, Institute for Security Technology studies, Dartmouth College, (“Liljenstam”), (Oct. 27, 2003).Applicant
- Lok Kwong et al: “DroidScope: Seamlessly Reconstructing the OS and Dalvik Semantic Views for Dynamic Android Malware Analysis”, Aug. 10, 2012, XP055158513, Retrieved from the Internet: URL:https://www.usenix.org/system/files/conference/usenixsecurity12/sec12—final107.pdf [retrieved on Dec. 15, 2014].Applicant
- Marchette, David J., Computer Intrusion Detection and Network Monitoring: A Statistical (“Marchette”), (2001).Applicant
- Margolis, P.E., “Random House Webster's Computer & Internet Dictionary 3rd Edition”, ISBN 0375703519, (Dec. 1998).Applicant
- Moore, D., et al., “Internet Quarantine: Requirements for Containing Self-Propagating Code”, INFOCOM, vol. 3, (Mar. 30-Apr. 3, 2003), pp. 1901-1910.Applicant
- Morales, Jose A., et al., ““Analyzing and exploiting network behaviors of malware.””, Security and Privacy in Communication Networks. Springer Berlin Heidelberg, 2010. 20-34.Applicant
- Natvig, Kurt, “SANDBOXII: Internet”, Virus Bulletin Conference, (“Natvig”), (Sep. 2002).Applicant
- NetBIOS Working Group. Protocol Standard for a NetBIOS Service on a TCP/UDP transport: Concepts and Methods. STD 19, RFC 1001, Mar. 1987.Applicant
- Newsome, J., et al., “Dynamic Taint Analysis for Automatic Detection, Analysis, and Signature Generation of Exploits on Commodity Software”, In Proceedings of the 12th Annual Network and Distributed System Security, Symposium (NDSS '05), (Feb. 2005).Applicant
- Newsome, J., et al., “Polygraph: Automatically Generating Signatures for Polymorphic Worms”, In Proceedings of the IEEE Symposium on Security and Privacy, (May 2005).Applicant
- Nojiri, D. , et al., “Cooperation Response Strategies for Large Scale Attack Mitigation”, DARPA Information Survivability Conference and Exposition, vol. 1, (Apr. 22-24, 2003), pp. 293-302.Applicant
- Peter M. Chen, and Brian D. Noble, “When Virtual Is Better Than Real, Department of Electrical Engineering and Computer Science”, University of Michigan (“Chen”), 2001.Applicant
- Reiner Sailer, Enriquillo Valdez, Trent Jaeger, Roonald Perez, Leendert van Doorn, John Linwood Griffin, Stefan Berger., sHype: Secure Hypervisor Approach to Trusted Virtualized Systems (Feb. 2, 2005) (“Sailer”).Applicant
- Silicon Defense, “Worm Containment in the Internal Network”, (Mar. 2003), pp. 1-25.Applicant
- Singh, S., et al., “Automated Worm Fingerprinting”, Proceedings of the ACM/USENIX Symposium on Operating System Design and Implementation, San Francisco, California, (Dec. 2004).Applicant
- Spitzner, Lance, “Honeypots: Tracking Hackers”, (“Spizner”), (Sep. 17, 2002).Applicant
- The Sniffers's Guide to Raw Traffic available at: yuba.stanford.edu/˜casado/pcap/sectionl.html, (Jan. 6, 2014).Applicant