METHOD AND SYSTEM FOR DETECTING MALICIOUS FILES IN A NON-ISOLATED ENVIRONMENT

[0001] The present application is a continuation of International Patent Application No.: PCT/RU2020/000089, entitled “METHOD AND SYSTEM FOR DETECTING MALICIOUS FILES IN A NON-ISOLATED ENVIRONMENT,” filed on Feb. 25, 2020, the entirety of which is incorporated herein by reference.

[0002] The present technology relates broadly to the field of cybersecurity, and in particular, to a method and system for detecting malicious files in a non-isolated environment.

[0003] Certain prior art approaches have been proposed to address the problem of identifying malicious files before launching thereof.

[0004] United States Patent Application Publication No.: 2019/0005239-A1, published on Jan. 3, 2019, assigned to Samsung Electronics Co Ltd., entitled “ELECTRONIC DEVICE FOR ANALYZING MALICIOUS CODE AND METHOD THEREFOR”, discloses a method for analyzing a malicious code by an electronic device, the method comprising the steps of: receiving an executable file; before the received executable file is installed, analyzing the executable file so as to collect suspected malicious code data from the executable file; normalizing the collected suspected malicious code data and analyzing the same on the basis of a probability model algorithm, so as to make a determination on the suspected malicious code data; and outputting the result of the determination.

[0005] Russian Patent No.: 2,654,146-C1, issued on May 16, 2018, assigned to AO Kaspersky Lab, entitled “SYSTEM AND METHOD OF DETECTING MALICIOUS FILES ACCOMPANIED WITH USING THE STATIC ANALYSIS ELEMENTS”, discloses a system for acknowledging a file to be malicious, which comprises: a) means for resource extraction intended to: extract resources from the file analyzed; transmit the resources extracted to the rule formation means and rule search means; b) the rule formation means intended to: form at least one rule regulating functional relationship between the obtained resources (hereinafter the rule), while the rule is formed by creating artificial neural network from the resources obtained, where the nodes of the artificial neural network are the means of the obtained resource analysis and relationships between the nodes, formed during creation of the neural network, indicate the functional relationship between the resources obtained; transmit each formed rule to the rule comparison means; c) the rule search means intended to: searching for at least one rule in the malicious file resource base on the basis of the resources obtained; transmit each rule found to the rule comparison means; d) the rule comparison means intended to: compute a degree of commonality between the rules received from the rule formation means and the rule search means; transmit the degree of commonality computed to the decision means; e) the decision means intended to: acknowledge the file analyzed to be malicious if the degree of commonality obtained exceeds the threshold preset.

[0006] It is an object of the present technology to address at least some inconveniences associated with the prior art.

[0007] Developers of the present technology have appreciated that the malicious files, such as malicious executable files, can be more effectively detected by analyzing both binary and disassembled versions thereof, which allows receiving a greater amount of data indicative of maliciousness of the files, such as numerical and string parameters thereof. Also, the method and systems described herein are directed to determining file behavior abnormalities based on analyzing control-flow graphs and data flow graphs associated with potentially malicious executable files. Further, non-limiting embodiments of the present technology are directed to training, based on the so retrieved training data, an ensemble of classifiers to determine if a given executable file is malicious or not, which may allow for an improved accuracy of the malicious file detection. In at least some non-limiting embodiments of the present technology, each one of the ensemble of classifiers can be trained based on training data of a respective different data type.

[0008] More specifically, in accordance with a first broad aspect of the present technology, there is provided a computer-implementable method for detecting malicious files in non-isolated environment. The method is executable by a processor. The method comprises: during a training phase: acquiring, by the processor, a plurality of executable files including at least one malicious executable file and at least one non-malicious executable file; analyzing, by the processor, a given executable file, the analyzing comprising analyzing both a binary and disassembled form of the given executable file, to obtain: (i) data associated with the given executable file; (ii) a control-flow graph associated with the given executable file, and (iii) a data-flow graph associated with the given executable file; determining, by the processor, based on the data, parameters of the given executable file; determining, by the processor, the parameters of the given executable file as being indicative of one of a malicious executable file and a non-malicious executable file; generating, by the processor, based on the parameters, at least a first feature vector and a second feature vector; generating, by the processor, based on the control-flow graph, a third feature vector; generating, by the processor, based on the data-flow graph, a fourth feature vector, each one of the first feature vector, the second feature vector, the third feature vector, and the fourth feature vector being different from an other one thereof; and training, by the processor, an ensemble of classifiers to determine if a given in-use executable file is one of malicious and non-malicious, the training comprising: training a first classifier of the ensemble of classifiers based on the first feature vector; training a second classifiers of the ensemble of classifiers based on the second feature vector; training a third classifiers of the ensemble of classifiers based on the third feature vector; and training a fourth classifiers of the ensemble of classifiers based on the fourth feature vector; assigning, by the processor, to each one of the ensemble of classifiers, a respective decisive priority value, the respective decisive priority value being indicative of a respective weight assigned to a prediction outcome of a given classifier of the ensemble of classifiers, the respective weight having been determined based on prediction accuracy of the given classifier.

[0009] In some implementation of the method, the method further comprises during an in-use phase following the training phase: obtaining, by the processor, the given in-use executable file; generating, by the processor, a first in-use feature vector, a second in-use feature vector, a third in-use feature vector, and a fourth in-use feature vector associated with the given in-use executable file; feeding, by the processor, the first in-use feature vector, the second in-use feature vector, the third in-use feature vector, and the fourth in-use feature vector to the ensemble of classifiers to determine a prediction outcome indicative of whether the given in-use executable file is one of malicious and non-malicious; and generating a report including the prediction outcome for presentation thereof to a user.

[0010] In some implementation of the method, the first feature vector comprises numerical ones of the parameters of the given executable file, and the second vector comprises string ones of the parameters of the given executable file.

[0011] In some implementation of the method, obtaining the data associated with the given executable file comprises obtaining at least one of: metadata, an Import Table, an Export Table, and a Byte/Entropy Histogram, associated with the given executable file.

[0012] In some implementation of the method, the analyzing the binary form of the given executable file comprises analyzing at least one of: byte n-grams, data of fields of the given executable file, a file section entropy, metadata, and line length distribution histograms.

[0013] In some implementation of the method, the data of the fields of the given executable file comprises at least one of: a number of sections in the given executable file, a size of headers, a code size, an availability of digital signature, an availability of imported DLL and functions where these libraries have been used, names of exported functions, data from a resource catalog, and an assembler version of the given executable file.

[0014] In some implementation of the method, the file section entropy has been determined using a sliding window applied to the binary form of the given executable file.

[0015] In some implementation of the method, the metadata of the binary form of the given executable file comprises a number of sections in the file, sizes thereof, a presence of a checksum, a file size, a creation time, an operating system version, and an assembler version associated with the given executable file.

[0016] In some implementation of the method, analyzing the disassembled form of the given executable file comprises analyzing at least one of: characters, registers, operation codes, a system interface (Windows APIs) access rate, sections, and metadata, associated with the given executable file.

[0017] In some implementation of the method, the analyzing further comprises determining, in a code of the disassembled form of the given executable file, a frequency of occurrence of characters, including at least one of: “−”, “+” “*”, “[”, “[”, “?”, and “@”.

[0018] In some implementation of the method, the analyzing further comprises determining a ratio between a number of known registers and a number of unknown registers obtained from the disassembled form of the given executable file.

[0019] In some implementation of the method, the sections obtained from the disassembled form of the given executable file include at least one of: .text, .data, .bss, .rdata, .edata, .idata, .rsrc, .tls, and .reloc.

[0020] In some implementation of the method, the metadata extracted from the disassembled form of the given executable file comprises the file size and a number of code lines.

[0021] In some implementation of the method, the prediction outcome includes one of a binary prediction outcome and a probabilistic prediction outcome.

[0022] In some implementation of the method, the respective decisive priority of a given classifier in the ensemble of classifiers is determined based on one of: an accuracy associated with the given classifier, a completeness associated with the given classifier, and an F-measure associated with the given classifier.

[0023] In accordance with a second broad aspect of the present technology, there is provided a system for detecting malicious files in non-isolated environment. The system comprises a processor and a non-transitory computer-readable medium storing instructions. The processor upon executing the instructions, is configured to: during a training phase: acquire a plurality of executable files including at least one malicious executable file and at least one non-malicious executable file; analyze a given executable file, the analyzing comprising analyzing both a binary and disassembled form of the given executable file, to obtain: (i) data associated with the given executable file; (ii) a control-flow graph associated with the given executable file, and (iii) a data-flow graph associated with the given executable file; determine, based on the data, parameters of the given executable file; determine the parameters of the given executable file as being indicative of one of a malicious executable file and a non-malicious executable file; generate, based on the parameters, at least a first feature vector and a second feature vector; generate, based on the control-flow graph, a third feature vector; generate, based on the data-flow graph, a fourth feature vector, each one of the first feature vector, the second feature vector, the third feature vector, and the fourth feature vector being different from an other one thereof; and train an ensemble of classifiers to determine if a given in-use executable file is one of malicious and non-malicious, by: training a first classifier of the ensemble of classifiers based on the first feature vector; training a second classifiers of the ensemble of classifiers based on the second feature vector; training a third classifiers of the ensemble of classifiers based on the third feature vector; and training a fourth classifiers of the ensemble of classifiers based on the fourth feature vector; assign, to each one of the ensemble of classifiers, a respective decisive priority value, the respective decisive priority value being indicative of a respective weight assigned to a prediction outcome of a given classifier of the ensemble of classifiers, the respective weight having been determined based on prediction accuracy of the given classifier.

[0024] In some implementation of the system, during an in-use phase following the training phase, the processor is further configured to: obtain the given in-use executable file; generate a first in-use feature vector, a second in-use feature vector, a third in-use feature vector, and a fourth in-use feature vector associated with the given in-use executable file; feed the first in-use feature vector, the second in-use feature vector, the third in-use feature vector, and the fourth in-use feature vector to the ensemble of classifiers to determine a prediction outcome indicative of whether the given in-use executable file is one of malicious and non-malicious; and generate a report including the prediction outcome for presentation thereof to a user.

[0025] In accordance with a third broad aspect of the present technology, there is provided a computer-implementable method for detecting malicious files in non-isolated environment. The method is executable by a processor. The method comprises: during a training phase: acquiring, by the processor, a training set of data including a plurality of executable files, the plurality of executable files including at least one malicious executable file and at least one non-malicious executable file; and training, by the processor, based on the training set of data, an ensemble of classifiers to determine if a given in-use executable file is one of malicious and non-malicious, the training comprising: training a first classifier of the ensemble of classifiers based on a first plurality of feature vectors, a given one of the first plurality of feature vectors having been determined based on parameters associated with a respective one of the plurality of executable files; training a second classifiers of the ensemble of classifiers based on a second plurality of feature vectors, a given one of the second plurality of feature vectors having been determined based on the parameters associated with the respective one of the plurality of executable files; training a third classifiers of the ensemble of classifiers based on a third plurality of feature vectors, a given one of the third plurality of feature vectors having been determined based on a control-flow graph of the respective one of the plurality of executable files; and training a fourth classifiers of the ensemble of classifiers based on a fourth plurality of feature vectors, a given one of the fourth plurality of feature vectors having been determined based on a data-flow graph of the respective one of the plurality of executable files, each one of the given one of the first plurality of feature vectors, the given one of the second plurality of feature vectors, the given one of the third plurality of feature vectors, and the given one of the fourth plurality of feature vectors being different from an other one thereof; and assigning, by the processor, to each one of the ensemble of classifiers, a respective decisive priority value.

[0026] In some implementation of the method, the method further comprises during an in-use phase following the training phase: obtaining, by the processor, the given in-use executable file; generating, by the processor, a first in-use feature vector, a second in-use feature vector, a third in-use feature vector, and a fourth in-use feature vector associated with the given in-use executable file; feeding, by the processor, the first in-use feature vector, the second in-use feature vector, the third in-use feature vector, and the fourth in-use feature vector to the ensemble of classifiers to determine a prediction outcome indicative of whether the given in-use executable file is one of malicious and non-malicious; and generating a report including the prediction outcome for presentation thereof to a user.

[0027] In the context of the present specification, unless expressly provided otherwise, a computer system may refer, but is not limited, to an “electronic device”, an “operation system”, a “system”, a “computer-based system”, a “controller unit”, a “control device” and/or any combination thereof appropriate to the relevant task at hand.

[0028] In the context of the present specification, unless expressly provided otherwise, the expression “computer-readable medium” and “memory” are intended to include media of any nature and kind whatsoever, non-limiting examples of which include RAM, ROM, disks (CD-ROMs, DVDs, floppy disks, hard disk drives, etc.), USB keys, flash memory cards, solid state-drives, and tape drives.

[0029] In the context of the present specification, a “database” is any structured collection of data, irrespective of its particular structure, the database management software, or the computer hardware on which the data is stored, implemented or otherwise rendered available for use. A database may reside on the same hardware as the process that stores or makes use of the information stored in the database or it may reside on separate hardware, such as a dedicated server or plurality of servers.

[0030] In the context of the present specification, unless expressly provided otherwise, the words “first”, “second”, “third”, etc. have been used as adjectives only for the purpose of allowing for distinction between the nouns that they modify from one another, and not for the purpose of describing any particular relationship between those nouns.

[0037] The following detailed description is provided to enable anyone skilled in the art to implement and use the non-limiting embodiments of the present technology. Specific details are provided merely for descriptive purposes and to give insights into the present technology, and in no way as a limitation. However, it would be apparent to a person skilled in the art that some of these specific details may not be necessary to implement certain non-limiting embodiments of the present technology. The descriptions of specific implementations are only provided as representative examples. Various modifications of these embodiments may become apparent to the person skilled in the art; the general principles defined in this document may be applied to other non-limiting embodiments and implementations without departing from the scope of the present technology.

[0038] Certain non-limiting embodiments of the present technology are directed to methods and systems to detect and block malicious files before execution thereof to prevent causing harm to computer systems.

[0039] With reference to FIG. 1, there is depicted a system 10 configurable for detecting malicious files in a non-isolated environment, in accordance with certain non-limiting embodiments o the present technology. The system 10 may comprise at least one of: a data extraction module (S10) configured to extract data from at least one executable file, a computer device (S20) configured to implement the method for detecting malicious files in non-isolated environment, and an output module (S30).

[0040] According to certain non-limiting embodiments of the present technology, each one of the data extraction module S10, the computer device S20, and the output module S30 can be implemented as a hardware module of the system 10 including all or some components of a computing environment 500 described below with reference to FIG. 5. Further it should be expressly understood that at least some of the data extraction module S10, the computer device S20, and the output module S30 can be implemented as a single hardware module including the computing environment 500, as will be described below.

[0041] Withe reference to FIG. 2, there is depicted a flowchart diagram of a method 100 of detecting malicious files in a non-isolated environment, in accordance with certain non-limiting embodiments of the present technology. The method 100 can be executed by a processor 501 of the computing environment 500.

[0042] The method 100 commences at step 111 with the processor 501 being configured to acquire a plurality of executable files including at least one executable file known to be malicious and at least one executable file known to be non-malicious, or benign.

[0043] According to certain non-limiting embodiments of the present technology, the processor 501 can further be configured to use the plurality of executable files to generate, during a training phase, a training set of data for training a machine-learning algorithm (MLA) to determine whether an in-use executable file is malicious or not, as will be described in greater detail below.

[0044] It should be expressly understood that it is not limited how the MLA is implemented and in some non-limiting embodiments of the present technology can comprise an ensemble of classifiers, each of which comprises a forest of decision trees (such as a Random Forest), as an example. In another example, each classifier of the ensemble of classifiers can be implemented as an SVM-based classifier.

[0045] The method 100 thus proceeds to step 112.

Step 112: Analyzing, by the Processor, a Given Executable File, the Analyzing Comprising Analyzing Both a Binary and Disassembled Form of the Given Executable File, to Obtain: (I) Data Associated with the Given Executable File; (II) a Control-Flow Graph Associated with the Given Executable File, and (III) a Data-Flow Graph Associated with the Given Executable File;

[0046] Further, at step 112, in some non-limiting embodiments of the present technology, the processor 501 can be configured to analyze a given executable file, such as a binary and disassembled forms thereof. By doing so, the processor 501 can be configured to obtain data for generating the training set of data.

[0047] More specifically, in accordance with certain non-limiting embodiments of the present technology, for the given executable file, the processor 501 con be configured to obtain at least one of: (i) data associated with the given executable file; (ii) a control-flow graph associated with the given executable file, and (iii) a data-flow graph associated with the given executable file.

[0048] In the context of the present specification, the control-flow graph represents a variety of all possible ways of the given executable file being executed. Thus, use of the control-flow graph for static code analysis can be practical in finding unique ways of quick and optimal execution.

[0049] Further, in the context of the present specification, the data flow graph represents dependence of one variables from other ones taking into account their visibility and execution context, therefore the data flow analysis is directly related to the control flow analysis, since the latter provides the execution context.

[0050] Besides, it is also worth noting that the control-flow graph and the data flow graph can be obtained by constructing a finite-state automation for all possible variants of the sequence of system calls of the given executable file. Thus, the algorithm records cyclic and branching structures of the given executable file, enabling it to identify behavior variations and detect abnormalities. The ability to focus on a behavior of given executable file allows for shorter system training periods and a smaller number of false responses, and also lesser strain on the memory, such as a RAM 502 described below with reference to FIG. 5.

[0051] The method 100 hence advances to step 113.

[0052] Thus, at step 113, the processor 501 can be configured to determine, based on the data associated with the given executable file received at step 112, parameters of the given executable file.

[0053] Therefore, in some non-limiting embodiments of the present technology, the data can include at least one of: data from a PE-header associated with the given executable file, data from an Import Table associated with the given executable file, data from an Export Table associated with the given executable file, byte n-grams of a respective binary form of the given executable file, data of fields of the given executable file, file section entropy, metadata, string length distribution histograms, and the like.

[0054] More specifically, the processor 501 can be configured to determine, based on the obtained data associated with the given executable file, one or more of following parameters:

• • [0055] data from the Import Table to specify what external functions and from which modules are required for the given executable file to operate normally;
  • [0056] data from the Export Table specifying names and/or numbers of the exported functions with their position in the process virtual memory;
  • [0057] metadata from the given executable file header, for example, number of sections in the given executable file, creation time, processor type, operating system version, assembler version, total size of sections, checksum;
  • [0058] Byte/Entropy Histogram, modeling byte distribution in the given executable file, which can be generated by a sliding window of 1024 bytes moving in increments of 256 bytes in the respective binary form of the given executable file, as an example.

[0059] To determine byte n-grams, the processor 501 can be configured to extract a byte sequence of n bytes from the respective binary version of the given executable file, wherein each element of the byte sequence can have one of 256 values. Therefore, the processor 501 can be configured to obtain byte frequency histograms in the form of 256 length vectors, which represent frequency of occurrence of each value.

[0060] Further, the data of the fields of the given executable file can include, for example, without limitation, number of sections in the given executable file, size of headers, code size, availability (or otherwise presence) of a digital signature, names of imported DLL and functions where these libraries have been used, names of exported functions, data from a resource catalogue, an assembler version of the given executable file, a version of an operating system, processor type, total size of sections, checksum.

[0061] Further, based on the file section entropy, the processor 501 can be configured to determine, using a statistical analysis, whether a source (or a machine) code associated with the given executable file has been obfuscated or not. It is worth noting that malware is often transmitted in an encrypted form, accordingly a byte frequency is equalized thereby increasing the entropy.

[0062] More specifically, in some non-limiting embodiments of the present technology, the processor 501 can apply a method of sliding window to represent the respective binary version of the given executable file in a form of a set of entropy measures. Window entropy is calculated, and after calculation, for the purpose of this set characterization, the set of statistical measures is considered, for example, mathematical expectation, variance, quantile, percentile. Thus, for example, if the entropy is close to 1, i.e. occurrence of different characters is nearly the same, the code is determined as having been obfuscated and/or encrypted.

[0063] Further, to determine the string length histograms, the processor 501 can be configured to extract, from the respective binary form of the given executable file, all ASCII lines and determine their lengths.

[0064] Further, in some non-limiting embodiments of the present technology, the processor 501 can be configured to analyze a respective disassembled form of the given executable file.

[0065] By way of example, and not as limitation, the processor 501 can be configured to determine at least of the following parameters of the given executable file by analyzing the respective disassembled form thereof:

• • [0066] characters, where characters mean frequency of occurrence of at least following characters: +, *,], [, ?, @, since they are specific to the code which tries to bypass the detection system, for example, by making indirect calls or downloading a dynamic library;
  • [0067] registers, since the majority of register names characterize their purpose—therefore, the frequency of using unknown registers could be indicative of a harmfulness degree of malicious executable files. Besides, for further analysis such parameters are also introduced: number of known registers, number of unknown (renamed) registers, ratio between the known registers and the unknown ones, as an example;
  • [0068] operation codes, wherein, the codes more frequently used by the malware are detected beforehand, and frequency of their occurrence in the code itself is also predetermined. Since the list of all instructions is rather big, the most popular operation codes stored, by the processor 501, in a system internal database (such as that hosted by a storage 503 described below with reference to FIG. 5) are used for analysis;
  • [0069] a system interfaces (Windows APIs) access rate, wherein, it is also possible to use Windows APIs access rate for analysis, but since the number of such interfaces is too large, the most specific for the malware are selected and stored, by the processor 501, in the system internal database for the sake of improving the efficiency.
  • [0070] sections of the respective disassembled form of the given executable file (executable PE file), including, for example, at least one of: .text, .data, .bss, .rdata, .edata, .idata, .rsrc, .tls and .reloc.

[0071] For example, parameters of these sections, which are used in the analysis by the processor, are non-exhaustively listed in the Table 1.

TABLE 1
Parameter name	Meaning
bss_lines_number	number of lines in .bss section
data_lines_number	number of lines in .data section
edata_lines_number	number of lines in .edata section
idata_lines_number	number of lines in .idata section
rdata_lines_number	number of lines in .rdata section
rsrc_lines_number	number of lines in .rsrc section
text_lines_number	number of lines in .text section
tls_lines_number	number of lines in .tls section
reloc_lines_number	number of lines in .reloc section
known_sections_number	total number of sections
unknown_sections_number	total number of unknown sections
known_sections_pr	proportion of known sections of total
	number
unknown_sections_pr	proportion of unknown sections of total
	number
.text_pr	proportion of .text section of the whole
	file
.data_pr	proportion of .data section of the whole
	file
.bss_pr	proportion of .bss section of the whole
	file
.rdata_pr	proportion of .rdata section of the whole
	file
.edata_pr	proportion of .edata section of the whole
	file
.idata_pr	proportion of .idata section of the whole
	file
.rsrc_pr	proportion of .rsrc section of the whole
	file
.tls_pr	proportion of .tls section of the whole
	file
.reloc_pr	proportion of .reloc section of the whole
	file

[0072] Further, based on the respective disassembled form of the given executable file, the processor 501 can be configured to determine additional metadata thereof including a file size and a number of code lines of the given executable file.

[0073] The method 100 thus proceeds to step 114.

Step 114: Determining, by the Processor, the Parameters of the Given Executable File as being Indicative of One of a Malicious Executable File and a Non-Malicious Executable File

[0074] Further, at step 114, the processor 501 can be configured to determine which of the parameters of the given executable file determined at step 113 are specific to malicious executable files and/or, vice versa, to non-malicious ones. To that end, the processor 501 can be configured to apply a statistical method.

[0075] More specifically, for example, it is possible to examine the given executable file for presence therein of a checksum as a parameter. Further, the processor 501 can be configured to determine missing checksum as being specific to malware. In another example, if a proportion of unknown sections to a total number of the given executable file is close to 1, it can be indicative of much obfuscation in the code, and the processor 501 can thus be configured to determine this parameter as being specific to malware. In yet another example, a proportion of known sections to the total number of section being close to 0, that is, everything being more or less standard, can be specific to legitimate, non-malicious, software. In yet another example, very frequent calls of certain API can also be specific to malware.

[0076] The method 100 hence advances to step 115.

[0077] Further, having received the data associated of the given executable file and determined the parameters thereof, at step 115, the processor 501 can be configured to generate respective feature vectors for training the MLA to determine if the in-use executable file is malicious or not. To that end, first, the processor 501 can be configured to divide all the parameters associated with the given executable file, which have been determined at step 114, into numerical parameters and string parameters.

[0078] By way of example, in some non-limiting embodiments of the present technology, the numerical parameters can include, without limitation, a section entropy, size of each section, ratio between the number of known sections and unknown ones, proportion of .text sections of the given executable file, a number of lines in each section, a frequency of occurrence of code obfuscating characters (−, +, *,], [, ?, @), and others described above.

[0079] Further, the processor 501 can be configured to generate, based on the numerical parameters, a first feature vector for training a first classifier of the ensemble of classifiers.

[0080] More specifically, each cell of the first feature vector is predetermined for a respective numerical parameter. For example:

[0081] cell 1—known_sections_number=10;

[0082] cell 2—unknown_sections_pr=0.93 (proportion of unknown sections of total number);

[0083] cell 3—some_section_lines_number=505 (number of lines in a section);

[0084] n−1 cell—lines_number=5623 (number of code lines in the disassembled file);

[0085] n cell—file_size=10893 (file size in bytes).

[0086] As it can be appreciated, the processor 501 can further be configured to apply the steps described above with respect to the given executable file to other ones of the plurality of executable files and thus generate a first plurality of feature vectors, each of which includes respective numerical parameters associated with a respective one of the plurality of executable file, similarly to the first feature vector. It is worth noting that the vector length of each one of the first plurality of vectors depends on a number of numerical parameters identified, by the processor 501, for each one of the plurality of executable files, as described above.

[0087] Further, the processor 501 can be configured to train the first classifier of the ensemble of classifiers, based on the first plurality of feature vectors, to determine whether the in-use executable file is malicious or not. For example, the processor 501 can be configured to train the first classifier to generate a first prediction outcome indicative of maliciousness of the in-use executable file, such as 0 being indicative of a non-malicious executable file, and 1—malicious executable file.

[0088] Further, the string parameters can include, without limitation, names of exported functions, data from the resource catalogue, names of imported DLL and functions where these libraries have been used, and the like. Further, based on the string parameters, in some non-limiting embodiments of the present technology, the processor 501 can be configured to generate a second feature vector comprising Boolean values, each of which is indicative of whether a respective string parameter has been encountered by the processor 501 during the analysis, that is, whether it is present in the given executable file or not.

[0089] Thus, the processor 501 can be configured to generate the second feature vector having, for example, a following configuration:

• • [0090] each cell name being pre-associated with a respective string parameter of all possible string parameters; and
  • [0091] a given cell includes a value of one of True and False indicative of whether the respective parameter has been identified for the given executable file.

[0092] Additionally, in some non-limiting embodiments of the present technology, the processor 501 can be configured to apply, to the second feature vector an encoding algorithm to convert the contents of the second feature vector to Boolean values to reduce the required storage space in the storage 303.

[0093] Thus, the processor 501 can be configured to generate the second feature vector for training a second classifier of the ensemble of classifiers. As it can be appreciated, the processor 501 can further be configured to apply the steps described above with respect to the given executable file to other ones of the plurality of executable files and thus generate a second plurality of feature vectors, each of which is indicative of respective string parameters associated with a respective one of the plurality of executable file, similarly to the second feature vector.

[0094] Further, the processor 501 can be configured to train the second classifier of the ensemble of classifiers, based on the second plurality of feature vectors, to determine whether the in-use executable file is malicious or not. For example, the processor 501 can be configured to train the second classifier to generate a second prediction outcome indicative of maliciousness of the in-use executable file, such as 0 being indicative of a non-malicious executable file, and 1—malicious executable file.

[0095] The method 100 hence advances to step 116.

[0096] Further, after generating the first and second feature vectors, according to certain non-limiting embodiments of the present technology, the processor 501 can be configured to generate a third feature vector associated with the given executable file based on the control-flow graph obtained at step 112, as described above.

[0097] With reference to FIG. 3, there is depicted a flowchart diagram of the processor 501 generating feature vectors for executable files based on control-flow graphs associated therewith, in accordance with certain non-limiting embodiment of the present technology It should be noted that in the flowchart of FIG. 3, the training steps are depicted with a solid line, and in-use steps—with a dashed line.

[0098] As noted above, the control flow graph is a variety of all possible ways of the given executable file running, which are represented as a graph.

[0099] In some non-limiting embodiments of the present technology, the processor 501 can be configured to generate the control-flow graph. To that end, the processor can be configured to analyze the respective disassembled for of the given executable file, generated, for example, using IDA Pro. Also, if the given executable file has been packed by some packager, the processor 501 can be configured to unpack it using, for example, standard UPX packager of executable files.

[0100] Further, based on the respective disassembled form of the given executable file, the processor 501 can be configured to construct the control-flow graph for the given executable file. More specifically, in the control-flow graph, each node (point) of the graph corresponds to a respective basic block representative of a portion of the disassembled code that does not include either control transfer operation, nor points, where the control is transferred to from the other portions of the code. A first block instruction corresponds to a point in the control-flow graph to which the execution is transferred, and the last block instruction corresponds to an instruction to another block.

[0101] It should be noted that the control-flow graph thus constructed is an oriented graph. Directed arcs can be used in the control-flow graph to represent instructions for control transfer from one point to another. Also, in some non-limiting embodiments of the present technology, two special blocks can be added:

• • [0102] input block, through which control enters the graph; and
  • [0103] output block, which completes all the paths in this graph.

[0104] In specific non-limiting embodiments of the present technology, to generate, based on the respective disassembled form of the given executable file, the control-flow graph, the processor 501 can be configured to execute a script developed, for example, in Python, which uses standard APIs, e.g. IDA Pro to extract information about each block:

$\begin{array}{cc}\mathrm{ClassTreeNode}\left\{\begin{array}{c}\mathrm{BlockID}\left(\mathrm{Numeric}\right)\\ \mathrm{StartAddr}\left(\mathrm{Hex}\right)\\ \mathrm{EndAddr}\left(\mathrm{Hex}\right)\\ \mathrm{LeftChild}\left(\mathrm{Pointer}\right)\\ \mathrm{RightChild}\left(\mathrm{Pointer}\right)\end{array}\right\},& \left(1\right)\end{array}$

where BlockID—unique block ID, StartAddr—start address, EndAddr—end address, LeftChild—all assembler instructions comprised in the basic block, RightChild—descendant blocks (their unique block ID).

[0105] In some non-limiting embodiments of the present technology, when constructing the control-flow graph, the processor 501 can be configured to ignore cycles in the respective disassembled form as their consideration will result in infinitely long execution paths. Thus, based on the above data, the processor 501 can be configured to construct the control-flow graph associated with the given executable file.

[0106] Further, the processor 501 can be configured to represent the control-flow graph in a form of the operation code call sequence. To that end, the processor 501 can be configured to apply a modified algorithm of depth-first traversal to identify and reject cycles:

	Counter = 0
	Push StartNode on Stack
	While Stack!= Empty do
	Pop Stack
	CurrExecPath = CurrExecPath ∪ Basic BlockID
	Counter = Counter + 1
	If Exists(RightChild) and RightChild  CurrExecPath
	Push RightChild on Stack
	PathLen = PathLen + Counter
	endif
	If NotExists(LeftChild) and LeftChild ⊂ CurrExecPath
	TotalExecPath = TotalExecPath + CurrExecPath
	Delete_nodes_from CurrExecPath till_last_added RightChild
	Adjust_Counter
	Elseif Exists(LeftChild) and LeftChild  CurrExecPath
	Push LeftChild on Stack
	endif
	endwhile
	Return TotalExecPath

[0107] As a result, the processor 501 can be configured to generate a sequence of ID blocks thereby representing the control-flow graph in the form of the operation code sequence.

[0108] Further, in some non-limiting embodiments of the present technology, the processor 501 can be configured to go over each block in accordance with an order of its execution, extracting operation codes and rejecting their operands. Thus, the processor 501 can be configured to generate a profile of the given executable, which is represented by operation code sequence as illustrated below.

• • [0109] ‘xor’, ‘inc’, ‘nov’, ‘cnp’, ‘jz’, ‘nov’, ‘test’, ‘jnz’, ‘nov’, “push’, ‘push’, ‘call’, ‘pop’, ‘pop’, ‘pop’, ‘nov’, ‘test’, ‘jnz’, “push’, ‘push’, ‘call’, ‘cnp’, ‘jnz”, ‘nov’, ‘nov’, ‘nov’, ‘nov’, ‘cnp’, ‘call’, ‘pop’, ‘test’, ‘jnz’, ‘inc’, ‘jnp’, ‘call’, ‘push’, ‘push’, ‘call’, ‘xor’, ‘nov’, ‘nov’, ‘le ‘push’, ‘push’, ‘call’, ‘test’, ‘jnz’, ‘inc’, ‘nov’, ‘cnp’, ‘nov’, ‘test’, ‘jnz’, ‘nov’, ‘cnp’, ‘jnz’, ‘push’, ‘push’, ‘call’, ‘pop’, ‘pop’, ‘nov’, ‘test’, ‘jnz’, ‘push’, ‘push’, ‘call’, ‘cnp’ ‘jz’, ‘cnp’, ‘jnz’, ‘cnp’, ‘jz, ‘novzx’, ‘push’, ‘call, ‘pop’, ‘test’, ‘jnz’, ‘inc’, ‘nov’, ‘jnp’.

[0110] Further, in some non-limiting embodiments of the present technology, the processor 501 can be configured to analyze trigrams of the operation code.

• • [0111] ‘cnp jbe test’, ‘jbe test jz’, ‘test jz cnp’, ‘jz cnp jnz’, ‘cnp jnz nov’, ‘jnz nov test’, ‘nov test jz’, ‘test’, ‘test jnz push’, ‘jnz push pop’, ‘push pop push’, ‘pop push push’, ‘push push push’, ‘push push push’, ‘cnp jaz cap’, ‘jnz cnp jnz’, ‘cnp jnz nov’, ‘jnz nov nov’, ‘nov nov jnp’, ‘nov jnp call’, ‘jnp call rein’, ‘push push call’, ‘push call xor’, ‘call xor nov’, ‘xor nov nov’, ‘nov nov lea’, ‘nov lea push’, ‘lea push call’.

[0112] Each such trigram will subsequently be a parameter in our system. Further, the processor 501 can be configured to convert these trigram profile associated with the given executable file from text to vector representation, thereby generating a third feature vector for training a third classifier of the ensemble of classifiers.

[0113] Needless to say, that similarly to generating the first and second pluralities of feature vectors, the processor 501 can further be configured to apply the step 116, described above with respect to the given executable file, to other ones of the plurality of executable files to generate a third plurality of feature vectors.

[0114] Further, the processor 501 can be configured to organize the third plurality of feature vectors in a matrix where rows are representative of executables files of the plurality of executable files and columns are representative of trigrams of associated profiles of the executable files as described above. In some non-limiting embodiments, the processor 501 can be configured to include in the matrix only unique trigrams associated with respective ones of the plurality of executable files.

[0115] Further, in some non-limiting embodiments of the present technology, the processor 501 can be configured to reduce dimensionality of each one of the third plurality of feature vectors by applying a tf-idf algorithm, which is required for estimation of word importance in a context of the respective disassembled form of the given executable file (profile of one PE file) in the plurality of the executable files.

[0116] Thus, similarly to the training the first and second classifiers, the processor 501 can be configured to feed the third plurality of feature vectors to the third classifier to train it to generate a third prediction outcome indicative of the in-use executable file being malicious.

[0117] The method thus advances to step 117.

[0118] At step 117, according to certain non-limiting embodiments of the present technology, the processor 501 can be configured to generate, based on the data-flow graph associated with the given executable file, a fourth feature vector for training a fourth classifier of the ensemble of classifiers. To that end, in some non-limiting embodiments of the present technology, the processor 501 can be configured to apply a data flow analysis to the given executable file.

[0119] In the context of the present specification, data flow analysis is a method to collect information about possible set of values, calculated in different points of the given executable file, which identifies what data passes therethrough.

[0120] Besides, the method of the data flow analysis execution, for example, could include setting data-flow equations for each node of the control-flow graph described above and solving them by multiple calculation of output with respect to input and data conversions locally in each node until the system has stabilized.

[0121] To conduct the data flow analysis, in some non-limiting embodiments of the present technology, the processor 501 can be configured to solve the problem of points definition in the given executable file where the variable has been defined, wherein each assignment is a definition, initially and up to what point it has been used until it is overwritten or deleted.

[0122] Therefore, d variable reaches p point, if there is such a path from the point following immediately after d to p point, such that d is not overwritten along this path. Each value defined in the given executable file or obtained during entering the given executable file relocates with a variety of other variables between the blocks of the control-flow graph, as schematically depicted by FIGS. 4A and 4B, in accordance with certain non-limiting embodiments of the present technology.

[0123] Thus, in FIGS. 4A and 4B, B0-B2 denote program blocks, D (definition)—definition of new variable, x,y—variable values, p—block in which d disappears (i.e. the variable has been defined somewhere, then the variable passes from in to out through the blocks and then enters p block, where it is deleted), and f (transfer function of the block), that is, a function which converts input into output.

[0124] Then, let us define the general form of equation for each block:

out[b]=Gen[b]∪(in[b]−Kill[b])  (2)

where:
Gen[b]—set of values generated in the block;
In[b]—input set of variables in block b;
Kill[b]—set of killed variables in block b;
Out[b]—output set of values.
Where, in[b]=out[p1]∪out[p2]∪ . . . ∪out[pn], where b—each block, p1 . . . pn—all previous blocks leading to this one.

[0125] For obtaining all input-output data in the control-flow graph blocks, the processor 501 can be configured to apply a following algorithm:

// Boundary condition
out[Entry] = Ø
// Initialization for iterative algorithm
For each basic block B other than Entry
out[B] = Ø
// iterate
While (Changes to any out[ ] occur) {
For each basic block B other than Entry {
in[B] = ∪ (out[p]), for all predecessors p of B
out[B] = fB(in[B])   // out[B]=gen[B]∪(in[B]−kill[B])
}

[0126] Thus, after applying the depth-first traversal algorithm to delete cycles, the processor 501 can be configured to obtain a vector of block numbers in control-flow graph. Each block can be associated with two sets of variables (in and out). Thus, the processor 501 can be configured to obtain vectors of input and output sets of variables, wherein each cell comprises a set of input and output variables of each block, respectively.

[0127] Thus, for example, many types of malwares replicate themselves infecting other binary files or entering the code via other system processes. It often results in a nearly even distribution of data volumes of outbound edges of respective graph nodes of the control-flow graph.

[0128] Therefore, the processor 501 can be configured to generate a more informative vector from the obtained set of vectors, wherein each cell of the more informative vector which comprises a difference between capacities of the two sets: input and output variables of the respective block.

[0129] Then, using a number of statistical measures (standardized estimate, standard deviation, expected value, interquartile distance) with the obtained vector, and also calculating a window entropy of this vector and applying the same statistical measures to the obtained vector, the processor 501 can be configured to generate the fourth feature vector for training the fourth classifier of the ensemble of classifiers.

[0130] Similarly, the processor 501 can be configured to generate a fourth plurality of feature vectors by applying, mutatis mutandis, step 117 described above to each other one of the plurality of executable files.

[0131] The method 100 hence proceeds to step 118.

[0132] Thus, the processor 501 can be configured to feed each one of the first plurality of feature vector, the second plurality of feature vector, the third plurality of feature vector, and the fourth plurality of feature vectors to a respective one of the first classifier, the second classifier, the third classifier, and the fourth classifier of the ensemble of classifiers. By doing so, the processor 501 can be configured to train each one of the first classifier, the second classifier, the third classifier, and the fourth classifier of the ensemble to determine a respective prediction outcome of whether the in-use executable file is malicious or not.

[0133] In some non-limiting embodiments of the present technology, a given prediction outcome can be one of:

• • [0134] binary (malicious/non-malicious);
  • [0135] probabilistic (if probability is above 50%, the file is considered to be malicious).

[0136] In additional non-limiting embodiments of the present technology, the processor 501 can be configured to train a fifth classifier of the ensemble of classifiers based on n-grams of operation code associated with the given executable file.

[0137] The method 100 thus advances to step 119.

[0138] Further, at step 119, according to certain non-limiting embodiments of the present technology, the processor 501 can be configured to assign, to each one of the ensemble of classifiers, a respective decisive priority, which is indicative of a respective weight assigned to a respective prediction outcome of each one of the ensemble of classifiers. In some non-limiting embodiments of the present technology, the processor 501 can be configured to determine the respective weight based on a prediction accuracy of each one of the ensemble of classifiers for both classes of executable files, malicious and non-malicious.

[0139] Other metrics that can be used for determining the respective weights can include completeness and F-measure associated with each one of the ensemble of classifiers. More specifically, accuracy within the class denotes a proportion of executable files actually belonging to a given class relative to all the objects which a given classifier of the ensemble has identified as being associated with the given class; completeness denotes a proportion of all the executable files identified by the given classifier of the ensemble, which belong to the given class relative to all the executable files of the given class in the training set of data, and F-measure denotes a harmonic mean between accuracy and completeness, while it tends to zero, if accuracy or completeness tends to zero.

[0140] Thus, having trained the ensemble of classifier, in some non-limiting embodiments of the present technology, the processor 501 can further be configured to terminate the training phase of the method 100, and use, during an in-use phase, the ensemble of classifiers to determine whether the in-use executable file is malicious or not. To that end, the processor 501 can be configured to analyze the in-use executable file to generate a first in-use feature vector, a second in-use feature vector, a third in-use feature vector, and a fourth in-use feature vector associated therewith, as described above at step 111 to 116.

[0141] Further, the processor 501 can be configured to feed the first in-use feature vector, the second in-use feature vector, the third in-use feature vector, and the fourth in-use feature vector to the ensemble of classifiers to determine the prediction outcome of whether the in-use executable file is malicious or not.

[0142] As noted above, the prediction outcome can be binary (malicious/non-malicious) or probabilistic. In the latter case, the processor 501 can be configured to determine the in-use executable file as being malicious if the prediction outcome exceeds a predetermine probability threshold, such as 50%.

[0143] Further, in some non-limiting embodiments of the present technology, the processor 501 can be configured to generate a report including the prediction outcome associated with the in-use executable file for presentation thereof to a user.

[0144] The method 100 hence terminates.

[0145] With reference to FIG. 5, there is depicted an example functional diagram of the computing environment 500 configurable to implement certain non-limiting embodiments of the present technology including the method 100, described above.

[0146] In some non-limiting embodiments of the present technology, the computing environment 500 may include: the processor 501 comprising one or more central processing units (CPUs), at least one non-transitory computer-readable memory 502 (RAM), a storage 503, input/output interfaces 504, input/output means 505, data communication means 506.

[0147] According to some non-limiting embodiments of the present technology, the processor 501 may be configured to execute specific program instructions the computations as required for the computing environment 500 to function properly or to ensure the functioning of one or more of its components. The processor 501 may further be configured to execute specific machine-readable instructions stored in the at least one non-transitory computer-readable memory 502, for example, those causing the computing environment 500 to execute the method 100, as an example.

[0148] In some non-limiting embodiments of the present technology, the machine-readable instructions representative of software components of disclosed systems may be implemented using any programming language or scripts, such as C, C++, C#, Java, JavaScript, VBScript, Macromedia Cold Fusion, COBOL, Microsoft Active Server Pages, Assembly, Perl, PHP, AWK, Python, Visual Basic, SQL Stored Procedures, PL/SQL, any UNIX shell scrips or XML. Various algorithms are implemented with any combination of the data structures, objects, processes, procedures and other software elements.

[0149] The at least one non-transitory computer-readable memory 502 may be implemented as RAM and contains the necessary program logic to provide the requisite functionality.

[0150] The storage 503 may be implemented as at least one of an HDD drive, an SSD drive, a RAID array, a network storage, a flash memory, an optical drive (such as CD, DVD, MD, Blu-ray), etc. The storage 503 may be configured for long-term storage of various data, e.g., the aforementioned documents with user data sets, databases with the time intervals measured for each user, user IDs, etc.

[0151] The input/output interfaces 504 may comprise various interfaces, such as at least one of USB, RS232, RJ45, LPT, COM, HDMI, PS/2, Lightning, FireWire, etc.

[0152] The input/output means 505 may include at least one of a keyboard, a joystick, a (touchscreen) display, a projector, a touchpad, a mouse, a trackball, a stylus, speakers, a microphone, and the like. A communication link between each one of the input/output means 505 can be wired (for example, connecting the keyboard via a PS/2 or USB port on the chassis of the desktop PC) or wireless (for example, via a wireless link, e.g., radio link, to the base station which is directly connected to the PC, e.g., to a USB port).

[0153] The data communication means 506 may be selected based on a particular implementation of a network, to which the computing environment 500 can have access, and may comprise at least one of: an Ethernet card, a WLAN/Wi-Fi adapter, a Bluetooth adapter, a BLE adapter, an NFC adapter, an IrDa, a RFID adapter, a GSM modem, and the like. As such, the connectivity hardware 504 may be configured for wired and wireless data transmission, via one of a WAN, a PAN, a LAN, an Intranet, the Internet, a WLAN, a WMAN, or a GSM network, as an example.

[0154] These and other components of the computing device 500 may be linked together using a common data bus 510.

[0155] It should be expressly understood that not all technical effects mentioned herein need to be enjoyed in each and every embodiment of the present technology.

[0156] Modifications and improvements to the above-described implementations of the present technology may become apparent to those skilled in the art. The foregoing description is intended to provide certain examples of implementation of the non-limiting embodiments of the present technology rather than to be limiting. The scope of the present technology is therefore intended to be limited solely by the scope of the appended claims.