Leveraging behavior-based rules for malware family classification

[0001] Embodiments of the disclosure relate to the field of cyber-security, and more specifically, to the labeling of malware classes and the classification of malware into its relevant classes based on its malicious behavior.

[0002] Malicious software or “malware” has become a pervasive problem for corporations and individual users alike, as the functionality of most electronic devices is influenced by downloaded data. Normally, electronic devices communicate with trusted resources and download data that is free of malware. However, there are occasions where an electronic device communicates with a particular resource, even a trusted resource, but the electronic device receives downloaded data that contains malware. When loaded in an electronic device, malware may adversely affect its operability and may attempt to compromise a local network by attempting to propagate the malware from the electronic device to other electronic devices connected to the local network.

[0003] Given the increased presence of malware, the security vendors have been developing systems and tools to protect electronic devices by detecting a presence of malware within data received as input. Due to the continuous and growing volume of released malware, however, security vendors are faced with a substantial challenge in accurately classifying detected malware. Currently, malware is classified by mapping the received (input) data into pre-defined categories of malware (hereinafter “malware classes”). However, as samples may vary greatly from each other, especially due to increasingly sophisticated (and morphing) malware and inconformity with malware classification by security vendors, many conventional classification techniques are not designed to handle malware variants.

[0004] In fact, some classification techniques tend to experience a high level of false positives when attempting to cluster (group) different (but highly related) malware into malware classes. This high level of false positives may also be due, at least in part, to the presence of “white noise”; namely, the presence of benign (or non-pertinent) behaviors that have no influence in determining whether the malware should be classified as part of a certain class or not. A reduction of false positives by removal of the “white noise” would improve overall system reliability and speed of malware detection.

[0012] Various embodiments of the disclosure are directed to an electronic device, system and method for detecting and classifying malware. More specifically, one embodiment of the disclosure relates to logic that is configured to analyze and classify input data as part of a particular malware class based upon a detected sequence of rules triggered during processing of the input data. Herein, the input data is malicious as being associated with known malware or exhibiting behaviors that suggest that the input data includes malware (hereinafter referred to as a “malware sample”). The input data is processed and a particular behavior or a pattern of selected behaviors exhibited by the processed sample are monitored (hereinafter generally referred to as “monitored behaviors”). The monitored behaviors, which normally are a subset of the behaviors exhibited by the malware sample during processing, may be selected through the placement of interception points within one or more software components processing the input data, where the interception points allow information associated with these behaviors to be captured by monitoring logic.

[0013] More specifically, during processing of the malware sample, salient information observed during processing is received by a threat detection system. The threat detection system may be configured as part of cloud services, may include an active agent operating in run-time, or may include one or more virtual machines with at least some of the operations being conducted in a sandboxed environment. The salient information may include state information associated with the virtual machine. Additionally, the salient information may include information associated with the monitored behaviors. The monitored behaviors may include a communication-based anomaly or an execution-based anomaly that (a) unexpectedly alters the functionality of an electronic device (e.g., causes application software to conduct operations in an atypical manner such as an unexpected attempt to access a particular file, registry or process, or even an attempt to access the network); and/or (b) provides unwanted functionality which may be generally acceptable in another context.

[0014] According to one embodiment, at least some of the salient information associated with the monitored behaviors is maintained in a chronological order based on when the behaviors were monitored. This chronological sequence of information may be initially placed in a prescribed format (sometimes referred to as an “event summary”). A plurality of pre-defined rules associated with malicious behaviors (i.e., behaviors known or suspected to cause unwanted or unexpected activities to occur) are applied to the chronological sequence of monitored behaviors that are part of the event summary. This produces a first sequence of rules that is included as part of a rule sequence report (e.g., rules, metadata associated with behaviors, etc.). The first sequence of rules includes a subset of the pre-defined rules that are triggered in response to a presence of certain behaviors within the event summary and the chronological order of these triggered rules is consistent with the chronological order of the monitored behaviors.

[0015] Some or all of the first sequence of rules, sometimes referred to as a “rule aggregation sequence,” serve as a fingerprint of the malware sample. This rule aggregation sequence may be used to map the malware sample to a certain class (family type) of malware (hereinafter referred to as a “malware class”). According to this embodiment, the rule aggregation sequence is a subset of the first sequence of rules, as some rules in the first sequence of rules may have little significance (e.g., no or minimal influence) in classifying the malware sample. Stated differently, some of the triggered rules in the first sequence of rules may constitute “white noise” during classification. Removal of “white noise” rules enhances the overall accuracy in malware classification.

[0016] A malware sample is classified based on the degree of relatedness between the chronologically ordered rules forming the rule aggregation sequence and the ordered rules for one or more rule sequences that are associated with known malware (sometimes referred to as a “reference rule sequence”). Hence, the malware sample may be assigned to a particular malware class based on an ordering of rules that are present in both the rule aggregation sequence and the reference rule sequence. According to one embodiment of the disclosure, the malware sample associated with the rule aggregation sequence may be assigned to a particular malware class based on a certain number of the chronologically ordered rules of the rule aggregation sequence corresponding to an ordering of rules of the reference rule sequence. According to another embodiment of the disclosure, the malware sample associated with the rule aggregation sequence may be assigned to a particular malware class based on (1) a certain percentage of rules that are part of the rule aggregation sequence having the same ordering as rules that are part of the reference rule sequence or (2) a total number (or a maximum number) of these chronologically ordered rules of the rule aggregation sequence having the same ordering as rules associated with the reference rule sequence. A labeling scheme may be used for classifying the malware samples into labels to account for variations in labeling context by vendors or polymorphic malware.

[0017] Terminology

[0018] In the following description, certain terminology is used to describe features of the invention. For example, in certain situations, the terms “component” and “logic” are representative of hardware, firmware or software that is configured to perform one or more functions. As hardware, a component (or logic) may include circuitry having data processing or storage functionality. Examples of such circuitry may include, but are not limited or restricted to a hardware processor (e.g., microprocessor, one or more processor cores, a digital signal processor, a programmable gate array, a microcontroller, an application specific integrated circuit “ASIC”, etc.), a semiconductor memory, or combinatorial elements.

[0019] A component (or logic) may be software in the form of one or more software modules, such as executable code or an instance, an Application Programming Interface (API), a subroutine, a function, a procedure, an applet, a servlet, a routine, source code, object code, a shared library/dynamic link library, or even one or more instructions. These software modules may be stored in any type of a suitable non-transitory storage medium, or transitory storage medium (e.g., electrical, optical, acoustical or other form of propagated signals such as carrier waves, infrared signals, or digital signals). Examples of non-transitory storage medium may include, but are not limited or restricted to a programmable circuit; semiconductor memory; non-persistent storage such as volatile memory (e.g., any type of random access memory “RAM”); or persistent storage such as non-volatile memory (e.g., read-only memory “ROM”, power-backed RAM, flash memory, phase-change memory, etc.), a solid-state drive, hard disk drive, an optical disc drive, or a portable memory device. As firmware, the executable code may be stored in persistent storage.

[0020] The term “object” generally refers to a collection of data, whether in transit (e.g., over a network) or at rest (e.g., stored), often having a logical structure or organization that allows for an analysis of the object for malware. During analysis, for example, the object may exhibit certain expected characteristics (e.g., expected internal content such as bit patterns, data structures, etc.) and, during processing, conduct certain expected behaviors. The object may also exhibit unexpected characteristics or conduct a set of unexpected behaviors that may offer evidence of the presence of malware. The object or portions thereof (hereinafter referred to as a “malware sample”) may undergo more in-depth analysis to determine whether the malware sample as part of a malicious attack.

[0021] Examples of a malware sample may include data received as input or transmitted as output, which are associated with known malware or, when processed, exhibit behaviors that suggest the data is malware. The data may include related packets, where a “packet” broadly refers to as a series of bits or bytes having a prescribed format. According to one embodiment, the related packets may include network-based packets, frames, cells, addressable memory locations, or the like. The malware sample may correspond to a collection of data that may take the form of a number of packets carrying related payloads, e.g., a single webpage received over a network. Alternatively, the malware sample may be a file retrieved from a storage location over an interconnect.

[0022] As a self-contained element, the malware sample may be an executable (e.g., an application, program, segment of code, dynamically link library “dll”, etc.) or a non-executable. Examples of non-executables may include a document (e.g., a Portable Document Format “PDF” document, Microsoft® Office® document, Microsoft® Excel® spreadsheet, etc.), an electronic mail (email), downloaded web page, or the like.

[0023] The term “behavior” should be generally construed as an activity conducted by the malware sample when processed by a software component running on the electronic device (e.g., instance processing the malware sample within a virtual machine, instance processed outside a sandbox environment, etc.), where the activity has been selected for monitoring. The activity may cause an undesired action to occur, such as violating or complying with a predefined rule of operation.

[0024] The term “electronic device” should be generally construed as electronics with the data processing capability and a capability of connecting to any type of network, such as a public network (e.g., Internet), a private network (e.g., a wireless data telecommunication network, a local area network “LAN”, etc.), or a combination of networks. Examples of an electronic device may include, but are not limited or restricted to, the following: an endpoint device (e.g., a laptop, a smartphone, a tablet, a desktop computer, a netbook, a medical device, or any general-purpose or special-purpose, user-controlled electronic device configured to support virtualization); a server; a mainframe; a router; or a security appliance that includes any system or subsystem configured to perform functions associated with malware detection and may be communicatively coupled to a network to intercept data routed to or from an endpoint device.

[0025] The term “malware” may be broadly construed as information, in the form of software, data, or one or more commands, that are intended to cause an undesired behavior, where the behavior is deemed to be “undesired” based on customer-specific rules, manufacturer-based rules, and any other type of rules formulated by public opinion or a particular governmental or commercial entity. This undesired behavior may operate as an exploit that takes advantage of a vulnerability, which results in a communication-based anomaly or an execution-based anomaly described above. Normally, an exploit attack is an attempt to gain control of a targeted electronic device.

[0026] The term “interconnect” may be construed as a physical or logical communication path between two or more electronic devices. For instance, the communication path may include wired and/or wireless transmission mediums. Examples of wired and wireless transmission mediums may include electrical wiring, optical fiber, cable, bus trace, a radio unit that supports radio frequency (RF) signaling, or any other wired/wireless signal transfer mechanism.

[0027] The term “computerized” generally represents that any corresponding operations are conducted by hardware in combination with software or firmware.

[0028] Lastly, the terms “or” and “and/or” as used herein are to be interpreted as inclusive or meaning any one or any combination. Therefore, “A, B or C” or “A, B and/or C” mean “any of the following: A; B; C; A and B; A and C; B and C; A, B and C.” An exception to this definition will occur only when a combination of elements, functions, steps or acts are in some way inherently mutually exclusive.

[0029] General System Architecture

[0030] Referring now to FIG. 1, an exemplary block diagram of an electronic device 100 (e.g., security appliance, endpoint device, etc.) is shown, where the electronic device 100 is configured with a threat detection system that is adapted to detect and classify malware based on substantial similarities between a chronologically ordered sequence of rules associated with a malware sample under analysis and the ordered sequence of rules exhibited by known (classified) malware. Herein, the electronic device 100 comprises one or more hardware processors (referred to as “processor(s)”) 110, a memory 120, one or more network interfaces (referred to as “network interface(s)”) 130, and one or more network devices (referred to as “network device(s)”) 140 connected by a system interconnect 150, such as a bus. These components are at least partially encased in a housing 160, which is made entirely or partially of a rigid material (e.g., hardened plastic, metal, glass, composite, or any combination thereof) that protects these components from atmospheric conditions.

[0031] The processor(s) 110 is a multipurpose, programmable device that accepts digital data as input, processes the input data according to instructions stored in its system memory, and provides results as output. One example of a processor may include an Intel® x86 central processing unit (CPU) with an instruction set architecture. Alternatively, a processor may include another type of CPU, a digital signal processor (DSP), an Application Specific Integrated Circuit (ASIC), a field-programmable gate array (FPGA), or the like.

[0032] The network device(s) 140 may include various input/output (I/O) or peripheral devices, such as a keyboard, a keypad, a touch screen, or a mouse for example. The network interface 130 may include different types of mechanisms and/or electrical circuitry (e.g., one or more network ports, modem, network interface card “NIC”, etc.). The network interface 130 provides connectivity for the electronic device 100 to a network to thereby facilitate communications to other remotely located electronic devices. To that end, the network interface(s) 130 may be configured to transmit and/or receive messages using a variety of communication protocols including, inter alia, Transmission Control Protocol/Internet Protocol (TCP/IP), Hypertext Transfer Protocol (HTTP), or HTTP Secure (HTTPS).

[0033] The memory 120 operates as system memory that may include different storage device types such as semiconductor memory (e.g., any type or random access memory, any type programmable read-only memory such as flash memory or any type of electrically erasable programmable read-only memory “EEPROM”) or a storage device. Certain types of storage devices may include a solid state drive (SSD) or a hard disk drive (HDD). The memory 120 includes a plurality of locations that are addressable by the processor(s) 110 and the network interface(s) 130 for storing software components (including software applications) and data structures associated with such software components. Some of the stored software components, associated with a threat detection system described below, may include the following: a behavior reporting logic 170, a behavioral rule-matching logic 180, and/or a malware classification logic 190. It is contemplated, however, that some or all of logic 170, 180 and 190 may be hardware logic that performs the functionality described below.

[0034] According to one embodiment of the disclosure, the behavior reporting logic 170, when executed by the processor(s) 110, operates as dynamic analysis logic that processes a malware sample and monitoring logic that monitors one or more behaviors of the malware sample during such processing. For this embodiment, the behavior reporting logic 170 operates within one or more virtual machines (referred to as “VM(s)”) running in the electronic device 100. The one or more virtual machines may be configured with different applications and operating systems (OSes), the same applications and different OSes, different applications and the same OSes, the same OSes but different versions of the same application, or the like. The behaviors to be monitored may be selected through placement of interception points within one or more software components running in the VM(s), where the interception points allow information to be routed to monitoring logic that resides within or outside each corresponding VM. The interception points may be placed to detect a malicious behavior, such as an unauthorized (or unexpected) attempt to access a particular file, registry or process, or even an unauthorized (or unexpected) attempt to access a network. These malicious behaviors may be time-stamped (or assigned a sequence number) and gathered as a chronological sequence that partially forms an “event summary” described below.

[0035] The behavioral rule-matching logic 180 analyzes the chronological sequence of selectively monitored behaviors of the event summary using predefined-defined rules (referred to as “correlation rules”). Compliance or non-compliance of the correlation rules may denote that the behavior is anomalous (i.e., unexpected or unwanted). The selection of the correlation rules may be based, at least in part, on the analysis mode (e.g., what device is conducting the analysis such as a forensic analysis appliance or web-based analysis appliance), the type of sample under analysis (e.g., document, executable, etc.), and a type of behaviors that are being monitored.

[0036] More specifically, operating as correlation logic, the behavioral rule matching logic 180 applies the correlation rules to the chronological sequence of behaviors to produce an output, namely a first sequence of rules triggered by the presence of certain behaviors and having a chronological order consistent with the monitored behaviors. The first sequence of rules normally is a subset of the selected correlation rules. The triggering of each rule identifies a rule violation (e.g., at least one of the sequence of behaviors fails to comply with a rule where compliance denotes non-malicious (expected) behavior) or a rule compliance (e.g., at least one of the sequence of behaviors complies with a rule where compliance denotes potential malicious (anomalous) behavior). Hence, when executed by the processor(s) 110, the behavioral rule-matching logic 180 identifies triggered rules associated with anomalous behaviors and assigns weight values to each of these triggered rules. The assigned weight values (e.g., weighting 1-10) may be based on the likelihood that the corresponding behaviors are being caused by malware. Optionally, the behavioral rule-matching logic 180 may also generate a description of the anomalous behaviors in a form of an alphanumeric message for transmission to a selected administrator.

[0037] Once the analysis is completed and the first sequence of rules is determined, the behavioral rule-matching logic 180 selects only the rules within the first sequence of rules having weight values greater than or equal to a prescribed weight value (e.g., rules with “non-zero” weighting, rules with weight values greater than less than a prescribed weighting of “2”, etc.). Stated differently, the behavior rule matching logic 180 removes the triggered rules with weight values less than the prescribed weight value to concentrate subsequent classification analysis on those rules having a stronger likelihood of being associated with behaviors caused by malware and having greater significance in malware classification. This produces a sub-sequence of the first sequence of rules, which is referred to as the “rule aggregation sequence”. An output, including one or more the rule aggregation sequences along with their corresponding representation of the sample (e.g., hash value) and aggregate weight value, is provided to the classification logic 190.

[0038] The malware classification logic 190 may operate in a training mode or a testing mode. When operating in a training mode, the malware classification logic 190 is configured to generate a malware training dataset. The malware training dataset is a collection of malware samples each comprising (1) a representation of the sample (e.g., hash value), (2) type designation (e.g., file, executable, etc.), (3) the reference rule sequence, and (4) the class name or label of the malware. The malware training dataset serves as the reference ground truth to which everything from the malware test set is compared.

[0039] One classification scheme involves an analysis of the relatedness (similarity or difference) between (i) a filtered sequence of rules that are triggered during analysis of monitored behaviors encountered during processing of the malware sample (e.g. rule aggregation sequence) and (ii) at least one reference rule sequence. A degree of relatedness needed for classification of the malware sample to a particular malware class may be reached based on either (i) at least a certain percentage of rules (and their ordering) of the rule aggregation sequence are identical or substantially similar to the rules (and their ordering) of the reference rule sequence, or (ii) both sequences feature a sub-sequence of chronologically ordered rules that exceeds a predetermined number of rules. In response to determining that the rule aggregation sequence is related to a particular reference rule sequence, the sample is classified as part of the same malware class represented by the reference rule sequence.

[0040] In order to maintain uniform labeling for a malware sample that may be associated with different labels from different security vendors (e.g., anti-virus “AV” vendors), a universal label may be generated and assigned to that malware sample or cluster of malware samples. The universal label may be a combination of labels used by of the different security vendors to ensure consistent designation of malware classes.

[0041] When operating in testing mode, the classification logic 190 is configured to analyze malware samples that are part of the malware test set, namely a dataset of malware samples each comprising (1) a representation of the sample (e.g., hash value), (2) type designation (e.g., file, executable, etc.), and (3) the rule aggregation sequence. This dataset would be created from malware binaries which are confirmed as being associated with malicious behaviors as described above. The classification logic 190 iterative fetches information associated with the malware samples and performs a sequence comparison measurement, such as a Longest Common Subsequence (LCS)-based similarity measurement for example, to identify classes for malware samples associated with rule sequences that are similar to a reference rule sequence associated with one or more malware training datasets. In general, the classification involves an analysis of the degree of relatedness between (i) a sequence of rules (e.g., rule aggregation sequence) where the rules are triggered during analysis of monitored behaviors encountered during processing of the malware sample and (ii) reference rule sequences. When the degree of relatedness exceeds a threshold (e.g., certain percentage of rules that are part of the rule aggregation sequence having the same ordering as the rules that are part of a reference rule sequence, meeting or exceeding a prescribed number of these chronologically ordered rules of the rule aggregation sequence having the same ordering as the rules associated with the reference rule sequence, etc.), the malware sample may be assigned a particular malware class (family) that is identified in accordance with a labeling scheme that is dependent on whether or not the sample is already labeled.

[0042] In summary, according to one embodiment of the disclosure, the classification logic 190, operating in cooperation with the behavior reporting logic 170 and the behavioral rule-matching logic 180, is configured to classify malware samples based on the degree of relatedness between the chronologically ordered rules of the rule aggregation sequence for that malware sample and the ordered rules of a reference rule sequence.

[0043] The memory further comprises an operating system (OS) 195 that is resident in memory 120 and, when executed by the hardware processor(s) 110, functionally organizes the electronic device 100 by, inter alia, invoking operations that support operations by the behavior reporting logic 170, the behavioral rule-matching logic 180, and the classification logic 190. Examples of types of OSes may include, but are not limited or restricted to the following: (1) a version of a WINDOWS® series of operating system; (2) a version of a MAC OS® or an IOS® series of operating system; (3) a version of a LINUX® operating system; or (4) a versions of an ANDROID® operating system, among others. Although not shown, the memory 120 comprises one or more applications that are responsible for processing different sample types as well as virtual system resources. The applications may include different word processing applications (e.g., WORD®, etc.), PDF reader applications, one or more web browsers (e.g., EXPLORER®, CHROME®, etc.), or the like. The virtual system resources operate as a logical representation (and virtualization) of certain system resources such as a virtual CPU (vCPU), a virtual network interface controller (vNIC), or a virtual machine monitor (sometimes referred to as a “VMM” or a “hypervisor”).

[0044] Architecture of the Threat Detection System

[0045] As shown in FIG. 2, a first embodiment of the electronic device 100 employing a threat detection system 200 is shown. The threat detection system 200 operates within the electronic device 100 and is adapted to analyze incoming data (e.g., network traffic propagating over a network, input data from another type of transmission medium including a dedicated transmission medium, etc.). According to this illustrative embodiment, the threat detection system 200 may be communicatively coupled with a network through a communication interface 220, where the network may include a public network such as the Internet, a private network (e.g., a local area network “LAN”, wireless LAN, etc.), or a combination thereof. The communication interface 220 may operate as (or operate in cooperation with) a data capturing device that intercepts (or alternatively duplicates) at least a portion of the received data, namely an incoming object for analysis and, in some case, the metadata associated with the object (hereinafter generally referred to as an “object”). Alternatively, the communication interface 220 may be configured to receive files or other types of objects with different formats that are not provided over a network. For instance, as an illustrative example, the communication interface 220 may be a data capturing device that automatically (or on command) accesses data stored in a storage system or another type of interface, such as a port, for receiving samples manually provided via a suitable dedicated communication link or from storage media such as portable flash drives.

[0046] In some embodiments, as shown, the communication interface 220 may be contained within the electronic device 100 as part of the threat detection system 200. In other embodiments, the communication interface 220 can be integrated into an intermediary device in a communication path for network traffic (e.g., an optional firewall, router, switch or other networked electronic device) or can be a standalone component, such as an appropriate commercially available network tap.

[0047] For this illustrative embodiment, however, the communication interface 220 may be configured to capture the received data, namely an incoming object 210. Metadata associated with the object (captured with the object or subsequently generated) may be used, at least in part by logic within the threat detection system 200, to determine protocols, application types and other information which may be used to determine a particular software profile for use in virtual machine (VM) configuration and/or VM operation scheduling. For instance, one or more software profiles may be used for selecting and/or configuring one or more virtual machines, such as VM 245 operating as part of the dynamic analysis.

[0048] As further shown in FIG. 2, the threat detection system 200 includes some or all of the following components: communication interface 220, static analysis logic 230, dynamic analysis logic 240, correlation logic 270, classification logic 280, and/or reporting logic 290. Herein, according to this embodiment of the disclosure, the communication interface 220 receives the object 210 and converts that object 210 into a format, if needed or as appropriate, on which scanning may be conducted by the static analysis logic 230. This conversion may involve some form of de-obfuscation, such as decompression of at least a portion of the object 210 for example. It is contemplated that the communication interface 220 may conduct decompilation, disassembly or other de-obfuscation activities on the object 210 and/or extraction of specific data associated with the object 210. However, as shown below, the de-obfuscation and data extraction activities may be handled by logic within the static analysis logic 230.

[0049] Referring still to FIG. 2, the static analysis logic 230 is responsible for extracting and/or generating metadata contained within and/or associated with incoming data (e.g., formatted object 212) from the communication interface 220 (e.g., network traffic, downloaded data). As described above, the metadata may be subsequently used for configuring one or more VMs, such as VM 245 as shown, for conducting a dynamic analysis of a sample. The static analysis logic 230 analyzes characteristics of the formatted object 212. Such analysis may involve the performance of one or more checks on the formatted object 212 without its execution. Examples of the checks may include signature matching to conduct (a) exploit signature checks, which may be adapted to compare at least a portion of the formatted object 212 with one or more pre-stored exploit signatures (pre-configured and predetermined attack patterns) from signature database (not shown), and/or (b) vulnerability signature checks that may be adapted to uncover deviations in messaging practices (e.g., non-compliance in communication protocols, message formats or ordering, and/or payload parameters including size). Other examples of these checks may include (i) heuristics, which is based on rules or policies as applied to the formatted object 212 and may determine whether one or more portions of the formatted object 212 is associated with an anomalous or suspicious characteristic (e.g., a particular URL associated with known exploits, or a particular source or destination address etc.) associated with known exploits; or (ii) determinative rule-based analysis that may include blacklist or white list checking.

[0050] Upon static analysis of the characteristics of the formatted object 212, the static analysis logic 230 determines whether this object 212 is “suspicious,” namely whether the object 212 has characteristics that suggest it may be associated with a malicious attack. Thereafter, the static analysis logic 230 may pass some or all of the information pertaining to the object 212, referred to as a “malware sample” 215, to the dynamic analysis logic 240 for more in-depth analysis by the VM 245.

[0051] According to one embodiment of the disclosure, the dynamic analysis logic 240 features one or more VMs, where each VM generally simulates the processing of the sample 215 within a run-time environment. For instance, as shown, the VM 245 is configured to process the malware sample 215 and monitor selective behaviors of the malware sample during such virtual processing. Monitoring logic 250 may be positioned within the software components operating with the VM 245 or operating in conjunction with these software components to monitor for certain selected behaviors performed by the malware sample 215 during processing. Such monitoring may include capturing information associated with these monitored behaviors along with maintaining an ordering of the occurrences of these behaviors. According to one embodiment of the disclosure, the monitoring logic 250 may include file system monitoring logic 252, process monitoring logic 253, registry monitoring logic 254 and/or network access monitoring logic 255.

[0052] For instance, the monitoring logic 250 may be used to detect one or more malicious behaviors (e.g., anomalous behavior), such as unexpected attempts to access or modify a particular file, a particular process, or a particular registry. The occurrence of these behaviors also may trigger a gathering of salient information by the monitoring logic 250, including state information. Examples of the state information may include (1) information directed to software components running in the virtual machine or the malware sample 215 being processed (e.g., running application name, version number, file type, object type, etc.) and (2) information directed to one or more behaviors (e.g., path, detected behavior name, etc.).

[0053] According to one embodiment of the disclosure, temporal identification logic 257 may be located within the monitoring logic 250, where the temporal identification (temp_id) logic 257 assigns an identifier to the detected malicious behavior and/or the salient information associated with the detected malicious behavior, where the identifier may be used in the chronological sequencing (or ordering) of behaviors received by the monitoring logic 250. Examples of various types of identifiers may include a time-stamp that is based on a current time as measured by a real-time clock (RTC) 258 communicatively coupled via interconnect 259 to the file system monitoring logic 252, the process monitoring logic 253, the registry monitoring logic 254 and/or the network access monitoring logic 255. Alternatively, the identifier may include a sequence number as generated by a monotonic counter for example. Connectivity to a common time source (or a counting source such as a monotonic counter) ensures that the chronological ordering of the behaviors.

[0054] A communication port 256 may be configured to order the behaviors and/or their salient state information in a chronological sequence (or ordering) based on their identifiers. As described above, the various types of identifiers may include a monotonic sequence number, a time-stamp, or the like. The salient information associated with the behaviors along with their corresponding identifiers are routed to a behavior reporting component 260, which organizes the behaviors chronologically in a prescribed format with their other salient information (referred to as an “event summary 262”). It is contemplated that, in lieu of deployment of the temporal identification logic 257, the behavior reporting component 260 may be configured to assign the identifier for chronological ordering of the monitored behaviors. The event summary 262 may be stored in a data store 265 until provided to the correlation logic 270.

[0055] Referring still to FIG. 2, the correlation logic 270 comprises rule matching logic 272, which analyzes the chronological sequence of behaviors from the event summary 262 using correlation rules stored in rules database 274, where compliance or non-compliance of the correlation rules denotes that the behavior under analysis is anomalous (e.g., unexpected communication or execution). The type of correlation rules may vary depending on the type of analysis being conducted, as different types of appliances perform different types of analyses and the analysis may be directed to different types of malware samples (e.g., document, executable, etc.). The type of correlation rules may also vary based on the type of behaviors being monitored.

[0056] The rules matching logic 272 applies the correlation rules to the sequence of behaviors that are part of the event summary 262 to initially produce, at least in part, the first sequence of rules that are triggered for each malware sample (being part of the “rule sequence report”). The triggering of each rule denotes a rule violation (e.g., at least one of the sequence of behaviors failed to comply with a rule where compliance denotes expected, non-malicious behavior) or a rule compliance (e.g., at least one of the sequence of behaviors complied with a rule where compliance denotes malicious behavior). The rules matching logic 272 assigns weight values to each triggered rule of the first sequence of rules based on the probability of that the corresponding behavior is associated with malware.

[0057] Based on the first sequence of rules, the rule matching logic 272 produces an output, referred to as a rule aggregation output 276, which includes a subset of the triggered correlation rules forming the first sequence of rules (sometimes referred to as the “rule aggregation sequence”). More specifically, when executed by the processor(s) 110, the rules matching logic 272 produces the rule aggregation sequence by selecting only the rules assigned with weight values greater than or equal to a prescribed weight value (e.g., rules with “non-zero” weighting or weighting less than prescribed value). Hence, rules associated with benign behaviors (e.g., “zero” weight values) are removed from the first sequence of rules to produce the rule aggregation sequence. It is contemplated that the types and/or number of rules set forth in the rule aggregation sequence may be useful in subsequent determinations as to the maliciousness of a sample under analysis.

[0058] Additionally, the rule matching logic 272 may also generate a description of the malicious behavior associated with the output in a form of an alphanumeric message for reporting purposes as part of the rule aggregation output 276. Hence, the rule aggregation output 276, on a per malware sample basis, may comprise the rule aggregation sequence, an analysis type (described below), an aggregate weight value for the rules of the rule aggregation sequence, and a representation for the malware sample (e.g., hash value, assigned sample identifier, etc.). The rule aggregation output 276 is supplied to the classification logic 280.

[0059] In receipt of the rule aggregation output 276, the classification logic 280 comprises a training logic 281 and a testing logic 285. When the classification logic 280 is operating in a first (training) state, the training logic 281 receives the rule aggregation sequence, normally a plurality of triggered rules associated with one or more malicious behaviors detected by the dynamic analysis logic 240. Herein, the training logic 281 creates a malware training dataset 282 for storage within a first (training) database 283 by identifying unique behaviors represented by a unique sequence of correlation rules. The unique sequence of correlation rules, which is a portion of the rule aggregation sequence, may be included as part of the malware training dataset 282. The representation for the malware sample (e.g., hash value), type designation (e.g., file, executable, etc.), and/or the class name or label of the malware may also be included as part of the malware training dataset 282.

[0060] Additionally, a class label may be assigned to the malware testing dataset 282. The class label may be determined by querying a label database (not shown) that includes labels for all known malware classes, where the representation for the malware sample may operate as a look-up parameter. The label database may be off-site database that maintains an aggregation of malware class labels and signatures from security vendors. A response to the query to the label database may include one or more known malware class labels that are used by different security vendors to identify the dataset associated with the sample 215. The training logic 281 determines a class label for the malware training dataset 282 based on a label used by a majority of the security vendors in accordance with the label database.

[0061] When the classification logic 280 is operating in a second (testing) state, the testing logic 285 receives the rule aggregation output 276, which includes the rule aggregation sequence, which temporarily stores one or more malware test sets 286 in a second (testing) database 287. Herein, as described above, the testing logic 285 classifies each malware sample by conducting a sequence comparison measurement, such as a Longest Common Subsequence (LCS)-based similarity measurement for example, to identify classes of malware samples that exhibit similar rule sequences as the rule aggregation sequence and stored in a results database 288. An analysis of the degree of relatedness between the rule aggregation sequence associated with the particular malware sample and one or more reference rule sequences set forth in the malware training datasets is conducted to determine whether the malware sample can be classified. Where the degree of relatedness exceeds a certain threshold (e.g., a certain percentage of matching rules with similar ordering, certain number of these chronologically ordered rules of the rule aggregation sequence correspond to an ordering of the rules associated with the malware testing rule sequence, etc.), the malware sample may be mapped to represent that it is part of a particular malware class and potentially assigned a new class label as described herein.

[0062] The reporting logic 290 is configured to receive information 289 from the classification logic 285 and generate electrical (alert) signals 292, especially in response to a determination that the malware sample 215 is being now classified as malicious and part of a malware family. The electrical (alert) signals 292 may include various types of messages, which may include text messages and/or email messages, video or audio stream, or other types of information over a wired or wireless communication path. The reporting logic 290 features an optional user interface 294 (e.g., touch pad, keyed inputs, etc.) for customization as to the reporting configuration.

[0063] Referring now to FIG. 3, an exemplary embodiment of the correlation logic 270 of the threat detection system 200 of FIG. 2 is shown. Herein, the correlation logic 270 comprises the rule matching logic 272, which analyzes the sequence of behaviors from the event summary 262 using correlation rules stored in the rules database 274. The rules database 274 comprises one or more sets of correlation rules, where each set of correlation rules may correspond to an analysis type conducted by the electronic device. For instance, a first set of correlation rules 300 stored within the rules database 274 may be applicable to the electronic device 100 of FIG. 2 operating as a web-based security appliance (first analysis type) while a second set of correlation rules 310 stored within the rules database 274 may be applicable to the electronic device 100 of FIG. 2 operating as a forensic-analysis appliance (second analysis type).

[0064] The rule database 274 is modifiable, where the first set of correlation rules 300 may be updated by a network administrator with sufficient credentials or automatically updated on a periodic or aperiodic basis. For instance, the correlation rule updates may be conducted on a subscription-basis with daily, weekly or monthly updates. As an example, these updates may involve one or more correlation rules being added to the first set of correlation rules 300. Alternatively, one or more correlation rules may be deleted from the first set of correlation rules 300 or substituted for one or more rules currently part of the first set of correlation rules 300.

[0065] Based on the analysis of the sequence of behaviors from the event summary 262 using correlation rules stored in the rules database 274, the rules matching logic 272 generates the first sequence of rules 320 that is triggered for each malware sample. The first sequence of rules 320 undergoes format conversion by format converter logic 330 for placement into a format readable by the classification logic 280. Additionally, a weighting logic 340 may assign weight values to each rule of the first sequence of rules 320 as well as an aggregate weight value for the entire sequence of rules associated with a particular malware sample. The weight value (and aggregate weight value) represents the probability of the behavior that corresponds to the triggered rule or rules being associated with malware. The formatted sequence of rules (e.g., rule aggregation sequence of FIG. 2) and corresponding weight values are provided as part of the rule aggregation output 276, which is supplied from the correlation logic 270 to the classification logic 280.

[0066] General Operational Flows

[0067] Referring now to FIG. 4, an embodiment of an operational flow of the classification logic 280 of FIG. 2 operating as part of the threat detection system 200 is shown. Herein, the operational flow includes (1) rule sequence extraction; (2) rule sequence comparison; (3) grouping (clustering) of malware samples based on rule sequence relatedness; and (4) label assignment. More specifically, for each malware sample, the sequence of correlation rules that is part of the rule aggregation output 276 and associated with a particular malware sample (referred to as the “rule aggregation sequence”) is extracted for analysis with one or more sequences of rules associated with malware training datasets that are associated with known malware (hereinafter “reference rule sequence(s)”), as illustrated in items 400 and 410.

[0068] Thereafter, the rule aggregation sequence is analyzed using the reference rule sequence(s) as illustrated in block 420. This analysis may be conducted through a LCS-based similarity analysis or any other sequence analysis technique to identify particular sections of the rule aggregation sequence that exhibit similar or identical rule sequences as found in the reference rule sequence(s). For example, during such analysis, relatedness may be detected when a certain percentage of chronologically ordered rules of the rule aggregation sequence are determined to correspond to the ordered rules of the reference rule sequence(s). As another example, during such analysis, relatedness may be detected when a prescribed number of rules within the rule aggregation sequence occur in the same order as a prescribed number of rules within the reference rule sequence(s).

[0069] In response to determining the relatedness between the rule aggregation sequence associated with the particular malware sample and at least one of the reference rule sequences, the malware sample is classified in accordance with any of three classification schemes with label assignments as needed (items 430 and 440). The first classification scheme is where the malware sample is determined to be from the same malware class as the malware sample from a particular malware training dataset based on the LCS-based relatedness score. This is determined to be a positive observation and the malware sample is classified accordingly. If the rule aggregation sequence is related to the reference rule sequence, but the classes are different, then the labels associated with these classes are merged to form a “superclass” (e.g., new class label derived from multiple class (family) names) and details of the superclass is added to the malware training dataset. In the event that the relatedness between the rule aggregation sequence and the reference rule sequence does not satisfy a prescribed threshold of relatedness (i.e. low or no similarity), the observation is considered a false (negative) observation and the malware sample is not classified.

[0070] In accordance with a second classification scheme, a new unlabeled sample is tested in connection with a labeled malware training dataset. As described above, the rule aggregation sequence of the unlabeled sample is compared to one or more reference rule sequences to determine the relatedness between these sequences. Where the degree of relatedness exceeds a threshold level (e.g., a certain percentage of matching rules with similar ordering, a certain number of these chronologically ordered rules of the rule aggregation sequence is also found in the reference rule sequence of the particular malware training dataset, etc.), the malware sample is classified according to the malware training dataset class.

[0071] In accordance with a third classification scheme, all undetected, unlabeled malware samples from the first or second classification schemes are added to the undetected dataset. Thereafter, in order to identify and label classes, the clusters of malware samples that exhibit similarities with each other based on their rule-sequence behavior are clustered. This may be conducted in accordance with a LCS-based clustering operation as described above. For each cluster of malware samples within the undetected dataset, a random name is generated and assigned to that cluster. The cluster is subsequently validated, potentially off-line and manually by a malware analysis, and an associated label is assigned to that cluster. Thereafter, information associated with that cluster would be added as part of the malware training dataset in order to improve classification.

[0072] Referring now to FIG. 5, an exemplary embodiment of the operability of the threat detection system for generating malware training datasets is shown. Herein, malware samples from a malware repository 500 (e.g., malware samples generated with the electronic device 100 of FIG. 1, malware samples from other sources, etc.) undergo a dynamic analysis (items 510 and 520). The dynamic analysis may conduct virtual processing of the malware samples within one or more virtual machines that include monitoring logic for gathering information associated with those behaviors of the samples being monitored. Alternatively, the dynamic analysis may feature actual processing of the samples with a security agent capturing selected behaviors. Both of these analyses produce corresponding event summaries (item 530). Each of the event summaries includes a chronological sequence of detected behaviors presented in a prescribed format. The behaviors associated with each event summary are analyzed by applying correlation rules to the chronological sequence of detected behaviors to generate a sequence of rules that are triggered in response to a presence of certain behaviors. This sequence of triggered rules corresponds to the “first sequence of rules” described above. From the first sequence of rules, identifiers for the rules triggered for each malware sample are extracted and formatted into a rule aggregation sequence for subsequent classification operations (item 540).

[0073] According to one embodiment of the disclosure, the rule aggregation sequence includes a subset of rules within the first sequence of rules. The rule aggregation sequence is configured to exclude triggered rules that appear to have lesser significance in detecting whether a malware sample includes malware and classifying the malware sample (i.e., the triggered rules operating as “white noise” during classification).

[0074] Thereafter, the malware sample is classified based on an analysis of the rule aggregation sequence to a sequence of rules that are associated with known malware (item 550). Hence, the malware sample associated with the rule aggregation sequence may be assigned to a particular malware class (and/or clustered with other malware samples) based on an ordering of rules that are present in both the rule aggregation sequence and the sequence of rules associated with known malware. According to one embodiment of the disclosure, the malware sample associated with the rule aggregation sequence may be assigned to a particular malware class based on a certain number of the chronologically ordered rules within the rule aggregation sequence corresponding to an ordering of rules within the reference rule sequence that operates as a fingerprint for that particular malware class. According to another embodiment of the disclosure, the malware sample associated with the rule aggregation sequence may be assigned to a particular malware class based on a certain percentage, a total number or a maximum sequential number of these chronologically ordered rules within the rule aggregation sequence corresponding to an ordering of the rules associated with the reference rule sequence for that particular malware class. This assignment clusters malware samples associated with a similar sequence of rule violations (or rule compliance) together to form an updated (or new) malware class.

[0075] Additionally, the updated (or new) malware class may require re-labeling to formulate a universal label, where the malware samples have been labeled differently by different security vendors (items 560, 570 and 580). This may be determined through a scanning operation (e.g., multiple anti-virus “AV” scan), which is conducted by scanning logic (e.g., part of the classification logic 190) that reviews the label database for labels associated with known malware, perhaps from different security vendors, having a signature similar to the binary of the malware sample (item 560). If one or more labels are uncovered, the label used by a majority of the security vendors may be passed to the label assignment logic along with other labels that may be used for similar malware samples (item 570). The universal label construct may be a full or partial aggregation of these different labels (item 580). The resultant data, which may include a representation of the malware sample, the newly created (universal) label, and the rule sequence is stored as part of a malware training dataset (item 590).

[0076] Referring now to FIG. 6, an exemplary embodiment of the operability of the testing logic 285 of FIG. 2 within the classification logic of the threat detection system is shown. The testing logic 285 is responsible for classifying malware samples that are part of a stored malware test set. Herein, information pertaining to a malware sample from the malware test set is fetched and a comparison may be conducted through a LCS-based similarity (relatedness) analysis between particular sections of the rule aggregation sequence associated with the malware sample and a sequence of rules stored as part of the malware training dataset (referred to as “reference rule sequence”), as illustrated in items 600, 605 and 610. During this operation, a determination is made as to the degree of relatedness between the rule aggregation sequence for the malware sample and the reference rule sequence (item 615). As an example, the degree of relatedness may be based, at least in part, on the percentage of chronologically ordered rules within the rule aggregation sequence that are determined to correspond to the ordered rules of the reference rule sequence. As another example, the degree of relatedness may be based, at least in part, on the length of a sub-sequence of “x” rules within the rule aggregation sequence (x≥3) that correspond to the ordered “x” rules of the reference rule sequence.

[0077] Where the degree of relatedness does not exceed a threshold (e.g., less than a prescribed percentage or no series of “x” rules matching the ordered rules of the reference rule sequence), the malware sample is not classified and is added as part of the undetected dataset as an unlabeled malware sample (see item 655). This testing process repeats for another malware sample (item 620). However, where the degree of relatedness exceeds the threshold, a determination is made as to whether the malware sample features a label (item 625). If so, the classification performs a first classification scheme labeled “case 1” (item 630).

[0078] The first classification scheme involves a first determination as to whether the malware sample has already been labeled to be the same class as a particular known malware (item 635). If so, this result is determined to be a positive observation and the malware sample is classified as part of the same class as the malware sample from the particular malware training dataset (item 640). Thereafter, information associated with the newly classified malware sample (e.g., rule aggregation sequence and a representation for the sample such as a hash value, assigned sample identifier, etc.)) may be stored in results database (item 647). However, if the rule sequences are similar, but the label currently associated with the malware sample is different, then a new class label is formulated (e.g., class names are merged to form a “superclass”) and details of the new class label is added as part of the malware sample (item 645). Thereafter, information associated with the classified malware sample (rule aggregation sequence, a representation for the sample, etc.) may be added to the results database (item 647).

[0079] Where the degree of relatedness exceeds the threshold but the malware sample does not feature a label, the classification logic performs a second classification scheme labeled “case 2” (item 650). The second classification scheme classifies the malware sample to a class of the classified (known) malware sample.

[0080] Herein, as shown, all unlabeled malware samples are added to the undetected dataset and the classification logic performs a third classification scheme labeled “case 3” (item 655). Thereafter, in order to identify and label classes, the clusters of malware samples that exhibit similarities with each other based on their rule sequences are clustered (item 660). This may be conducted in accordance with a LCS-based clustering operation as described above. For each cluster of malware samples within the undetected dataset, a random name is generated and assigned to that cluster (item 665). The cluster is subsequently validated, potentially off-line and manually by a malware analyst or a through an automated process, and an associated label is assigned to that cluster (item 670). Thereafter, information associated with that cluster would be added as part of the malware training dataset in order to improve classification (item 675).

[0081] In the foregoing description, the invention is described with reference to specific exemplary embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention as set forth in the appended claims.