METHOD AND SYSTEM FOR DETECTING MALICIOUS DOMAIN NAMES AT AN UPPER DNS HIERARCHY

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 61/438,492, filed Feb. 1, 2011, which is incorporated by reference in its entirety.

[0002] FIGS. 1-3 illustrate details related to a system for detecting a malicious domain name, according to one embodiment.

[0003] FIG. 4 illustrates a method for detecting a malicious domain name, according to one embodiment.

[0004] FIG. 5 illustrates details of how domain name statistical information can be collected, according to one embodiment.

[0005] FIGS. 1-3 illustrates details related to a system for detecting a malicious domain name, according to one embodiment. (It should be noted that when the terms “a”, “an”, “said”, etc. are used throughout this application, they should be interpreted as “at least one” “the at least one”, etc.) In particular, a system can be provided that analyzes domain name system (DNS) query patterns at the upper DNS hierarchy (e.g., an authoritative name server (AUTH NS) level, a top-level domain name server (TLD NS) level, a root name server (NS) level) for the purpose of detecting new malicious domain names. In this way, in one embodiment, DNS operators can (e.g., automatically, using data available at their DNS servers) detect and remediate malicious domain names within their name space, helping to enable a rapid response. FIG. 1 illustrates various levels within a DNS hierarchy. As one moves higher up in the DNS hierarchy, global visibility of domain name resolutions can be increased. Thus, if DNS traffic is monitored (e.g., with a sensor) at an AUTH NS, all DNS queries originating from all recursive domain name system (RDNS) resolvers that attempt to resolve a domain name for which the AUTH NS has authority or is a point of delegation can be monitored. Even in the case where the NS server is anycasted (e.g., where the NS server sends data to the nearest host(s), which will then send the message to its nearest host(s), etc.), it can be assumed that the same physical authority governs all the anycasted servers. Therefore the global data gather can still be possible by the same physical authority. Similarly, if DNS traffic is monitored (e.g., with a sensor) at a TLD NS, all DNS queries originating from all RDNS resolvers and all AUTH NSs that attempt to resolve a domain name for which the TLD NS has authority or is a point of delegation can be monitored. Thus, in one embodiment, less sensors can be utilized, which can lessen: operational costs, privacy concerns, or difficulties related to establishing and maintaining relationships between different network operators, or any combination thereof. It should be noted that as one moves up higher in the DNS hierarchy, DNS caching can become stronger, and thus only a smaller subset of queries to domain names may be visible. This can be because, at the TLD level, the effect of DNS caching from recursive DNS servers worldwide can be analogous to the Time To Live (TTL) value for each domain name (e.g., the TTL for google.com can be shorter than the TTL for .com).

[0006] FIG. 2 illustrates an example of a domain name tree and a domain name resolution process. Section (a) of FIG. 2 illustrates how a domain name system can be structured like a tree. A domain name can identify a node in the tree. For example, the domain name F.D.B.A. can identify the path from the root “.” to a node F in the tree. The set of resource information associated with a particular domain name can be composed of resource records (RRs). The domain name related to a given RR is called the owner of that RR. The depth of a node in the tree can be referred to as a domain level. For example, A can be a TLD, B.A. can be a second-level domain (2LD), D.B.A. can be a third-level domain (3LD), etc.

[0007] The information related to the domain name space can be stored in a distributed domain name database. The domain name database can be partitioned by “cuts” made between adjacent nodes. After all cuts are made, each group of connected nodes can represent a separate zone. Each zone can have at least one node, and thus a domain name for which it is authoritative. For each zone, a node which is closer to the root than any other node in the zone can be identified. The name of this node can be used to identify the zone. For example, assume there is a zone cut in the path between nodes B and A, as shown in section (a) of FIG. 2. In this case, the zone to which the domain F.D.B.A. belongs can be identified with the domain name B.A. (The last “.”, which represents the root, is often omitted, such as when “corn” is used instead of “corn.”.) The RRs of the nodes of a given zone can be served by one or more AUTH NSs. A given AUTH NS can support one or more zones. AUTH NSs that have complete knowledge about a zone (e.g., storing the RRs for all the nodes related to the zone in question in its zone files) can be referred to as having authority over that zone. AUTH NSs can also delegate the authority over part of a (sub-) zone to another AUTH NS.

[0008] DNS queries can be initiated by a stub-resolver (e.g., a WINDOWS application) on a user's machine, which can rely on a RDNS resolver for obtaining a set of RRs owned by a given domain name. The RDNS resolver can be responsible for directly contacting the AUTH NSs on behalf of the stub-resolver, obtaining the requested information, and forwarding it back to the stub-resolver. The RDNS resolver can also be responsible for caching the obtained information for a certain period of time (e.g., called time to live (TLL)), so that if the same or another stub-resolver queries again for the same information within the TTL time window, the RDNS resolver will not need to re-contact the AUTH NS. This can help improve efficiency, for example, by downloading a website faster.

[0009] Section (b) of FIG. 2 depicts a query resolution process. In 1, the stub resolver requests a resolution for www.example.com. Assuming the RDNS resolver has no cached information whatsoever, it can start the iterative process. In 2, the RDNS resolver can request from a root NS a referral for www.example.com. The root NS (in 3) can provide back the delegation for www.example.com to the .com TLD NS. Following the same iteration (4 and 5), the .com TLD NS can refer the RDNS resolver to the AUTH NS for the zone example.com. In 6 and 7, the RDNS resolver can receive the DNS resolution for the domain name www.example.com. In 8, the RDNS resolver can provide back to the stub resolver the DNS resolution for the domain name www.example.com, which is 192.0.32.10.

[0010] FIG. 3 illustrates a system 300 for detecting a malicious domain name, according to one embodiment. The system 300 described in FIG. 3 can divide monitored data streams into epochs {E_i} (where i=1, . . . m). For example, an epoch can be one month, one week, several days, one day, one hour, or any other time period. (It should be noted that if root servers are used, in addition to or instead of AUTH NSs and/or TLD NSs, the epoch may need to be longer than if just AUTH NSs and/or TLD NSs are used.) At the end of each epoch, system 300 can summarize the DNS traffic related to a given domain name d by computing a number of statistical features, such as: the diversity of the IP addresses associated with the RDNS servers that queried domain name d, the relative volume of queries from the set of querying RDNS servers, or historic information related to the IP space pointed to by d, or any combination thereof. System 300 can comprise a detection application 305, AUTH NSs 335, TLD NS 345, or detection reports 330, or any combination thereof. The detection application 305 can comprise: feature computation module 320, learning module 315, statistical classifier module 310, or knowledge database 310, or any combination thereof.

[0011] System 300 can comprise a training mode and an operation mode. As discussed below in FIG. 4, in training mode, domain name statistical information can be collected from one or more TLD NSs and/or one or more AUTH NSs and the collected domain name statistical information can be utilized to determine if a domain name is malicious or benign (e.g., legitimate). In operation mode, an additional domain name can be determined to be malicious or benign depending on how close the additional domain name's statistical information is to a known malicious domain name's statistical information.

[0012] Referring back to FIG. 3, a knowledge database 310 can store information about known malicious and known benign domain names, and related resolved IPs, for which the monitored AUTH NSs and TLD NSs are authoritative or are a point of delegation. The knowledge database 310 can store summary information about query/response behavior of each domain name across m days. Each domain name in the knowledge database 310, and in turn each feature vector in a set of training vectors V_train can be associated with a malicious or benign label. The learning module 315 can take as input the set of training vectors V_train={νⁱ_d} (where i=1, . . . , m), where ∀d is in the knowledge base KB.

[0013] In operation mode, the feature computation module 320 and/or the statistical classifier module 310 can be utilized. The feature computation module 320 can comprise a function F (d, E_i)=νⁱ_d that can map the DNS traffic in the epoch E_i related to d into a feature vector νⁱ_d. The statistical classifier module 310 can utilize the feature vector to classify: the diversity of the IP addresses associated with the RDNS servers that queried the domain name d, the relative volume of queries from the set of querying RDNS servers, or historic information related to the IP space pointed to by d, or any combination thereof.

[0014] FIG. 4 illustrates a method 400 for detecting a malicious domain name, according to one embodiment. In 405, domain name statistical information can be collected from one or more TLD NSs, or one or more AUTH NSs, or one or more root servers, or any combination thereof. It should be noted that in some embodiments, only TLD NS(s) and/or AUTH NS(s) can be used. In 410, the collected domain name statistical information can be utilized to determine if a domain name is malicious or benign. In 415, an additional domain name can be determined to be malicious or benign depending on how close the additional domain name's statistical information is to a malicious domain name's statistical information.

[0015] For example, a stream of DNS traffic can be monitored and, at the end of an epoch, each domain d′∉KB (e.g., all unknown domains) extracted from the query/response streams can be mapped into a feature vector ν^j_d′. As discussed in more detail below, the feature vectors can be: a requester diversity vector, a requester profile vector, or a requester reputation vector, or any combination thereof. For example, assume the domain name d′ is “imddos.my03.com”. (This domain name (along with others) was detected and sinkholed in the middle of 2010 (http://www.damballa.com/IMDDOS/) as part of a botnet.) Based on the daily observations (in this case on the day May 21, 2010), the following three statistical vectors (which are described in more detail below) can be computed:

[0016] At this point, given a domain d′, the statistical classifier module 325 can gather the statistical information from any or all of these vectors (or any others), compare it to historical information in knowledge database 310, and assign a label and a confidence score c (I_d′,j), which can express whether the query/response patterns observed for d′ during epoch E_j resemble either malicious or benign behavior, and with what probability. In this example, the trained classifier can label the imddos.my03.com domain name as being malicious (i.e., black) with 0.8 confidence. The output of the classifier can be the following:

[i: ‘2010-05-21’, ii: ‘imddos.my03.com’, iii: ‘black’, iv: ‘black’, v: ‘unknown’, vi: 0.80000000000000004],
where (i) is the evaluation day, (ii) is the domain name to be evaluated against the trained classifiers (e.g., using the last five days of traffic), (iii) is the correct label for the domain name (or evaluation label—which can be mandatory for proper evaluation of the classification system) (e.g., black), (iv) is the output of the classification system (e.g., black—which can be what the classifier proposes as a classification label), (v) is the label we have for the domain name at the classification day (e.g., unknown label—at the classification date there can be a number of domain names for which we have no prior information; thus we need the classification system and its proposed prediction label for the domain names with unknown labels), and (vi) is the classification probability confidence threshold on the label (iv). An operator may alter the classification probability confidence threshold so he/she can tune the false positive and true positive designations accordingly. In the example, the threshold can be set to 0.5. Detection application 305 can gather a series of labels and confidence scores S (ν^j_d′)={l_d′,j,c (l_d′,j)} (where j=t, . . . , (t+m)) for m consecutive epochs (where t refers to a given starting epoch E_t). Detection application 305 can then compute the average confidence score= C_M=avg_j{c(l_d′,j)} for the malicious labels assigned to d′ by S across the m epochs, and an alarm can be raised if C_M is greater than a predefined threshold θ.

[0017] Utilizing the above method, DNS operators can monitor DNS traffic towards their AUTH NS(s) and TLD NS(s), and independently (e.g., without the need of data from other networks) detect malicious domain names within the DNS zones over which they have authority, and can thus quickly take action to “clean up” their zone files by removing the detected malicious domain names. It should be noted that removing a domain name from a zone file can effectively “convict” the domain name and prevent future successful resolution from anywhere on the Internet.

[0018] FIG. 5 illustrates details of how domain name statistical information can be collected (405 of FIG. 4), according to one embodiment. In 505, requester diversity information can be collected. In 510, requester profile information can be collected. In 515, reputation information can be collected.

[0019] For each DNS query q_j regarding a domain name d and the related DNS response r_j, the response can be translated into a tuple Q_j(d)=(T_j, R_j, d, IPs_j), where T_j can identify the epoch in which the query /response was observed, R_j can be the IP address of the machine that initiated the query q_j, d can be the queried domain, and IPs_j can be the set of resolved IP addresses as reported in the response r_j. It should be noted that since DNS queries and responses from the upper hierarchy are being monitored, in some cases, the response may be delegated to a name server which may not be monitored by system 300. In the case where the response does not carry the resolved IP address, the IPs set can be derived by leveraging a passive DNS database or by directly querying the delegated name server.

[0020] Given a domain name d and a series of tuples Q_j(d) (where j=1, . . . , m), measured during a certain epoch E_t (i.e., T_j=E_t, ∀j=1, . . . , m), the system 300 can extract the following groups of statistical features.

[0021] Requester Diversity. In one embodiment, the requester diversity information can help characterize whether the machines (e.g., RDNS servers) that query a given domain name are localized or are globally distributed. The requester diversity information can be helpful based on the fact that the distribution of machines on the Internet that query malicious domain names is on average different from the distribution of IP addresses that query legitimate domain names. For example, if an IP address with a certain border gateway protocol (BGP) prefix queries a malicious domain name, it is more likely that other IP addresses with the same BGP prefix will also look up the same malicious domain, compared to if the domain name was benign. For example, if one computer in a business' network is compromised, it is more likely that other computers in the business' network are also compromised. For example, given a domain d and a series of tuples {Q_j(d)} (where j=1, . . . , m), the series of requester IP addresses {R_j} (where j=1, . . . , m), can be mapped to: a BGP prefix, autonomous system (AS) numbers, and country codes (CCs) to which the IP addresses belong. Then, the distribution of occurrence frequencies of the obtained BGP prefixes (sometimes referred to as classless inter-domain routing (CIDR) prefixes), the AS numbers, and CCs can be computed. For each of these three distributions, the mean, standard deviation and variance can be computed. Also, the absolute number of distinct IP addresses (e.g., distinct values of {R_j} (where j=1, . . . , m), and the number of distinct BGP prefixes, AS numbers and CCs can be considered. Thus, in some embodiments, thirteen statistical features that summarize the diversity of the machines that query a particular domain names, as seen from an AUTH NS or TLD NS, can be obtained.

[0022] In the example of imddos.my03.com for the day of May, 21, 2010, the requester diversity vector can be:

cidr_div: 227
as_div: 82
cc_div: 23
rdns_cnt: 557
overall_cnt: 659
cc_avg: 28.652173913
cc_std: 62.0934842839
cc_var: 3855.60079051
asn_avg: 8.03658536585
asn_std: 18.1526076227
asn_var: 329.517163505
cidr_avg: 2.90308370044
cidr_std: 4.88354100521
cidr_var: 23.8489727496

[0023] In the above example, cidr_div can reflect the Classless Inter-Domain Routing (CIDR) diversity, which can be the number of different networks that the RDNS requests originated from during an epoch. The as_div can reflect the autonomous system (AS) diversity, which can be the number of autonomous systems that the RDNS requests originated from during the epoch. The cc_div can reflect the country code (CC) diversity, which can be the number of different country codes that the RDNS requests originated from during the epoch. The rdns_cnt can be the distinct IP addresses that resolved the particular domain name in the epoch. The overall_cnt can be the total number of IP addresses that resolved the particular domain name in the epoch. The cc_avg, cc_std, and cc_var can reflect the average, standard deviation, and variance of the frequency of unique IPs address per unique country codes that are observed in the epoch. The asn_avg, asn_std, and asn_var can reflect the average, standard deviation, and variance of the frequency of the unique IPs addresses per unique AS observed in the epoch. The cidr_avg, cidr_std, and cidr_var can reflect the average, standard deviation, and variance of the frequency of unique IPs per unique CIDRs observed in the epoch.

[0024] Requester Profile. In one embodiment, the requester profile information can help determine the level of popularity of the querying RDNS servers that query a domain name. The requester profile information can be helpful based on the fact that malicious domains can tend to be queried from requesters with a large number of Internet service provider (ISP) networks because, for example, ISP networks can offer little or no protection against malicious software propagation. In addition, the population of machines in ISP networks can be very large, and therefore the probability that a machine in the ISP network will become infected by malware can be very high. On the other hand, legitimate domains are often queried from both ISP networks and smaller organization networks, such as enterprise networks, which are usually better protected against malware and therefore tend to query fewer malicious domain names.

[0025] In one embodiment, a higher weight can be assigned to RDNS servers that serve a large client population because a large network can have a large number of infected machines. While it may not be possible to precisely estimate the population behind an RDNS server, because of the DNS cache effect, the population can be measured as follows: The DNS query/response stream for a large AUTH NS that has authority over a set of domains D can be monitored. Given an epoch E_t, all query tuples {Q_j(d)}, ∀j, d, seen during E_t can be considered. R can be the set of all distinct requester IP addresses in the query tuples. For each IP address R_k∉R, the number c_t,k of different domain names in D queried by R_k during E_t can be counted. The weight associated to the requester's IP address Rk can be defined as

$w_{t,k}=\frac{c_{t,k}}{\max \text{?}\text{?}}$ $\text{?}\text{indicates text missing or illegible when filed}$

[0026] Once the weights w_t,k have been defined, the requester profile features can be measured. {Q_i(d′)} (where i=1 . . . h) can be the set of query tuples related to d′ observed during an epoch E_t. In addition, R(d′) can be the set of all distinct requester IP addresses in {Q_i(d′)}. For each R_k∉R(d′), the count c_t,k can be computed as previously described. Then, given the set C_t(d′)={c_t,k}_k, the average, the biased and unbiased standard deviations, and the biased and unbiased variances of the values in C_t(d′) can be computed. (It should be noted that the biased and unbiased estimators of the standard deviation and variance can have different values when the cardinality |C_t(d′)| is small.)

[0027] In the example of imddos.my03.com for the day of May 21, 2010, the un-weighted requester profile vector can be:

[0028] The Cav can be the average count of unique domain names that a particular RDNS server requested in an epoch. Similarly, the Cstd, Cstdpop, Cstdsamp, and Cvar can be the standard deviation, the population standard deviation, the sample standard deviation and variance (respectively) of the frequency of the unique domain names that were looked up by each RDNS server in an epoch. In one embodiment, the proposed size of the epoch can be at least a week and the frequency count of unique domain names can be done based (at most, in some embodiments) on a daily granularity. It should be noted, however, that many other epochs and frequencies of counts can be utilized.

[0029] Similarly, for each R_k∉R(d′), the count c_t,k can be computed as previously described. Afterwards, each count can be multiplied by the weight w_t-n,k to obtain the set WC_t(d′)={c_t,k*w_i-n,k}_k of weighted counts. (Note that n is the number of epochs.) It should also be noted that the weights w_i,j can be computed based on historical data about the resolver's IP address collected n epochs (e.g., seven days) before the epoch E_t. Then, the average, the biased and unbiased standard deviation, and the biased and unbiased variance of the values in WC_t(d′) can be computed.

[0030] In the example of imddos.my03.com for the day of May 21, 2010, the weighted requester profile vector can be:

[0031] The weighted features can follow the same logic described above, with each feature being multiplied by the previous described weighting factor (WC_t(d)).

[0032] Requester Reputation. For the requester reputation information, in one embodiment, a dynamic domain name reputation system can be used. A dynamic domain name reputation system can use historic information about domain name resolutions to classify new domain names, for which little information (if any) is known. For example, it can be determined whether, and to what extent, a certain domain name d is related to domain names and IP addresses that have been historically recognized as either malicious or benign domain names. If d points to an IP address space that is known to host malicious activities, it is more likely that d itself is also involved in malicious activities. Alternatively, if d points to a well known, professionally run legitimate network, it is less likely that d is actually involved in malicious activity. A low reputation score can be assigned to new domain names that appear to share some similarities with known malicious domain names. Conversely, a high reputation score can be assigned to those domain names that share some similarities with legitimate, professionally administered domain names.

[0033] For example, given a domain name d and the set of query tuples {Q_j(d)} (where j=1, . . . , h), which can be obtained during an epoch E_t, the overall set of resolved IP addresses IPs(d, t)=∪_hj=1 IPs_j can be considered (where IPs_j can be an element of the tuple Q_j(d)). If BGP(d, t) and AS(d, t) are the set of distinct BGP prefixes and autonomous system (AS) numbers to which the IP addresses in IPs(d, t) belong, the following features can be computed:

• • [0034] Malware Evidence. Malware evidence can help determine whether a piece of malware can be linked with a particular network address space. Malware evidence can include the average number of known malicious domain names that in a certain epoch of time have pointed to each of the IP addresses in IPs(d, t), each of the BGP prefixes in BGP(d, t), or each of the AS numbers in AS(d, t), or any combination thereof, that are on a malware list.
  • [0035] SBL Evidence. Spamhouse Block List (SBL) evidence (e.g., see www.spamhaus.org/sbl) can help show if a particular address space has historically been linked with fraudulent activities (e.g., spamming, phishing). SBL evidence can also include the average number of known malicious domain names that in a certain epoch of time have pointed to each of the IP addresses in IPs(d, t), each of the BGP prefixes in BGP(d, t), or each of the AS numbers in AS(d, t), or any combination thereof, that are on a SBL list.
  • [0036] Whitelist Evidence. The number of IP addresses in IPs(d, I), the BGP prefixes in BGP(d, t), and the AS numbers in AS(d, t) that include IP addresses pointed to by domain names in all or part of certain white lists (e.g., DNSWL, ALEXA) can be determined.

[0037] For the example of imddos.my03.com for the day of May 21, 2010, the requester reputation vector can be:

botipcAV: 0.0
botcidrcAV: 0.0
botascAV: 16.0
sblipcAV: 0.0
sblcidrcAV: 0.0
sblascAV: 0.0
whipcAV: 0.0
whcidrcAV: 0.0
whascAV: 294.0

[0038] The botipcAV can be the average number of botnet IPs historically linked with this IP based on public evidence (e.g., Zeustracker https://zeustracker.abuse.ch/). The botcidrcAV and the botascAV can be the average number of botnet IPs historically linked with the CIDR and the AS that the IP address from the domain name points to. This count can also be obtained based on public evidence (e.g., Zeustracker https://zeustracker.abuse.ch/).

[0039] The sblipcAV can be the average number of SBL listings historically linked with this IP based on public evidence. The sblcidrcAV and the sblascAV can be can be the average number of SBL listings historically linked with the CIDR and the AS that the IP address from the domain name points to.

[0040] The whipcAV can be the average number of DNSWL (www.dnswl.org) listings historically linked with this IP based on public evidence. The whcidrcAV and the whascAV can be the average number of DNSWL (www.dnswl.org) listings historically linked with the CIDR and the AS that the IP address from the domain name points to.

[0041] While various embodiments of the present invention have been described above, it should be understood that they have been presented by way of example, and not limitation. It will be apparent to persons skilled in the relevant art(s) that various changes in the form and detail can be made therein without departing from the spirit and scope of the present invention. Thus, the invention should not be limited by any of the above-described exemplary embodiments.

[0042] In addition, it should be understood that the figures described above, which highlight the functionality and advantages of the present invention, are presented for example purposes only. The architecture of the present invention is sufficiently flexible and configurable, such that it may be utilized in ways other than that shown in the figures.

[0043] Further, the purpose of the Abstract of the Disclosure is to enable the U.S. Patent and Trademark Office and the public generally, and especially the scientists, engineers and practitioners in the art who are not familiar with patent or legal terms or phraseology, to determine quickly from cursory inspection the nature and essence of the technical disclosure of the application. The Abstract of the Disclosure is not intended to be limiting as to the scope of the present invention in any way.

[0044] It should also be noted that the terms “a”, “an”, “the”, “said”, etc. signify “at least one”, “the at least one”, “said at least one”, etc., in the specification, claims and drawings. In addition, the term “comprising”, etc. signifies “including, but not limited to”, etc. in the specification, claims and drawings.

[0045] Finally, it is the applicant's intent that only claims that include the express language “means for” or “step for” be interpreted under 35 U.S.C. 112, paragraph 6. Claims that do not expressly include the phrase “means for” or “step for” are not to be interpreted under 35 U.S.C. 112, paragraph 6.