Packet classification method through hierarchical rulebase partitioning

[0001] This application claims benefit under 35 U.S.C. § 119 from Korean Patent Application No. 2004-32944, filed on May 11, 2004, and No. 2004-59058, filed on Jul. 28, 2004, the disclosures of which are incorporated herein by reference.

[0002] 1. Field of the Invention

[0003] The present invention relates to a packet classification method, and more particularly to a packet classification method through hierarchical rulebase partitioning, which can reduce a search time required for packet classification by partitioning a rulebase into independent sub-rulebases through a plurality of hashing stages and classifying input packets only in a related sub-rulebase.

[0004] 2. Description of Related Art

[0005] Packet classification is one of the most fundamental building blocks in many networking functions such as Diff-Serv traffic conditioning, firewall, VPN, traffic accounting and billing, load-balancing, and policy-based routing. These functions need to track flows and give the same treatment to the packets in a flow. A rulebase stores classification rules, which define the flows and their corresponding treatments. Since a flow is defined by the header values of a packet, a classfier's duty is to examine the header and identify the corresponding flow.

[0006] Internet traffic is not only fast growing, but it is also diversifying both in applications and in protocols. New applications and protocols such as Internet telephony, security protocols, and peer-to-peer applications are being rapidly deployed in addition to the traditional Internet applications such as Web, ftp, and email. As a result, the rulebase size is rapidly increasing. In a recent study [T. Woo, “A Modular Approach to Packet Classification: Algorithms and Results”, In Proceedings of the IEEE INFOCOM '00. March 2000.], Woo argues that a rulebase with over a million entries is possible in future packet classification applications. From the classifier's viewpoint, this implies that for each packet the classifier must be able to find the matching rule with the highest priority amongst all the rules in the rulebase at the wire speed. Thus, there has been a renewal of interest in the scalability issue in the terms of the size of a rulebase. [(1) P. Gupta and N. McKeown, “Packet Classification on Multiple Fields”, In Proceedings of the ACM SIGCOM '99, Vol. 29, issue 4, August 1999. (2) F. Baboescu and G. Varghese, “Scalable Packet Classification”, In Proceedings of the ACM SIGCOM '01, Vol. 31, August 2001. (3) Flow Analysis of Passive Measurement Data, http://pma.nlanr.net/PMA/Datacube.html. (4) T. V. Lakshman and D. Stiladis, “High-speed Policy-based Packet Forwarding using Efficient Multi-dimensional Range Matching”, In Proceedings of the ACM SIGCOMM '98 Vol. 28, pp. 191-202, 1998. (5) V. Srinivasan, S. Suri, G. Varghese, and M. Valdvogel, “Fast and Scalable Layer Four Switching”, In Proceedings of the ACM SIGCOMM '98, Vol. 28, pp. 203-214, 1998. (6) V. Srinivasan, G. Varghese, and S. Suri, “Packet Classification Using Tuple Space Search”, In Proceedings of the ACM SIGCOMM '99, Vol. 29, pp. 135-146, August 1999. (7) M. M. Buddhikot, S. Suri, and M. Waldvogel, “Space Decomposition Techniques for Fast Layer-4 Switching”, In Proceedings of the IFIP Sixth International Workshop on Protocols for High Speed Networks. Vol. 66, No. 6, pp. 277-283, August 1999. (8) A. Feldmann and S. Muthukrishnan, “Tradeoffs for packet Classification”, In Gigabit Networking Workshop of the Proceedings of the IEEE INFOCOM '00. March 2000. (9) P. Gupta and N. Mckeown, “Packet Classification using Hierarchial Intelligent Cuttings”, In Proceedings of the Hot Interconnects VII, 1999.]

[0007] Most of existing works, however, mainly focus on relatively small classifiers, for example, with less than 20 K rules. To address this issue, we propose a new scalable packet classification algorithm that can scale well up to this size.

[0008] Also, the conventional packet classification has problem as follows. Given a rulebase,

[0009] $R=\sum _{i=1}^n{r}_i,$
which is a set of rules, a packet classifier needs to identify the rule that incoming packet matches to by looking up one or more fields of the packet header. Each rule is specified by the range of values in one or more fields of a packet header. Specifically, in d-dimensional packet classification, each rule r_i is defined over d fields. Formally, r_i is defined by a tuple (C_i, A_i) where C_i is called a classification space and, A_i is the associated action of rule r_i. The classification space is defined by the crossproduct,

[0010] $C_i=F_1\otimes F_2\otimes \dots F_d=\prod _{k=1}^d{F}_k^i,$
is a range of values the field k must take. A rule r_i match a packet p={b₁, b₂, . . . , b_d} if for

[0011] $\forall k,b_k\in F_k^i$
where b_k is a singleton. Multiple rules can match a packet. Thus, a classifier must identify the highest priority rule among all the matching rules. Intuitively, this requires the classifier to lookup the header fields of an incoming packet and to compare them against the rules in the rulebase one by one in order of decreasing priority. When n, for example the number of rules, is large of arrival rate λ of incoming packet is high, this is a time-consuming serial process, which will limit the speed of the classifier. Thus, the essence of the problem is to find a fast yet scalable classification function both in time and in space.

[0012] The present invention has been developed in order to solve the above drawbacks and other problems associated with the conventional arrangement. An aspect of the present invention is to provide a packet classification method through hierarchical rulebase partitioning which can improve the search efficiency by hierarchically partitioning a rulebase into independent sub-rulebases of a small size using an efficient hash key selection algorithm.

[0013] For this, the present invention proposes diverse hash key algorithms and maximum entropy hashing for selecting a hash key that maximizes the hash key algorithm as the most efficient algorithm.

[0014] Another aspect of the present invention is to provide a packet classification method which can be applied to a packet classifier having a large-sized rulebase that is difficult to be processed through the existing packet classification method, to keep pace with the tendency to increase the size of the rulebase.

[0015] According to an aspect of the present invention, there is provided a packet classification method for searching for a rule having a highest priority and matching among a plurality of rules included in a specified rulebase with respect to a packet input from an external network and processing the packet according to the searched rule. The method includes: preprocessing by partitioning the rulebase into a plurality of independent sub-rulebases based on a specified condition and producing a hash table on the basis of the sub-rulebases; and classifying the packet by searching the hash table using a hash key extracted from a packet header of the inputted packet and mapping the packet on the sub-rulebase corresponding to the packet.

[0016] The preprocessing may include a first partitioning of the rulebase into the plurality of sub-rulebases based on a specified protocol and a specified port number and producing the hash table on the basis of the sub-rulebases.

[0017] A specified application may be determined according to the protocol and the port number.

[0018] The preprocessing may also include a second partitioning of re-partitioning a specified sub-rulebase among the plurality of sub-rulebases using the hash key extracted by a specified method from departure and destination IP addresses, and producing the hash table on the basis of the sub-rulebases if the number of rules in the specified sub-rulebase exceeds a specified threshold value.

[0019] The specified method may be one among an MSB pattern, an exponential growing pattern, a mask distribution pattern and an entropy-maximizing pattern.

[0020] The MSB pattern method may select a specified number of bits from most significant bits of the destination IP address and the departure IP address.

[0021] The exponential growing pattern method may select a bit position corresponding to an exponential function of 2 from the destination IP address and the departure IP address.

[0022] The mask distribution pattern method may calculate accumulated values at respective bit positions bi by adding the number of bits which are not a dummy bit in all defined rules belonging to the rulebase, calculate the total accumulated value by adding the accumulated values from the most significant bit to the least significant bit, and then select the specified bit position whenever the accumulated value at the specified bit position becomes a multiple of a value obtained by dividing the total accumulated value by K.

[0023] The K may be a bit value of the hash key to be produced.

[0024] The entropy-maximizing pattern method may select a specified number of bits that maximize entropy from the destination IP address and the departure IP address.

[0025] According to another aspect of the present invention, there is provided a method of reducing a search time for packet classification of an input packet, including: hierarchally partitioning a rulebase into sub-rulebases by hashing bit fields selected from a classification field; extracting a hash key from a header of the input packet; and searching a hash table using the hash key to match the input packet to a corresponding sub-rulebase.

[0026] According to another aspect of the present invention, there is provided a packet classification method of searching for a rule having a highest priority in a rulebase with respect to an input packet and, including: partitioning the rulebase into a plurality of sub-rulebases based on a specified condition and producing a hash table on the basis of the sub-rulebases; extracting a hash key from a packet header of packet; and classifying the packet by searching the hash table using the hash key; and mapping the packet on the sub-rulebase corresponding to the packet.

[0027] Additional and/or other aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.

[0035] Reference will now be made in detail to an embodiment of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. The embodiment is described below in order to explain the present invention by referring to the figures.

[0036] In the following description, the same drawing reference numerals are used for the same elements even in different drawings. The matters set forth in the description such as detailed constructions and elements are examples provided to assist in a comprehensive understanding of the present invention. Thus, it is apparent that embodiments of the present invention can be carried out without those defined matters. Also, well-known functions or constructions are not described in detail.

[0037] 1. Proposed Algorithm

[0038] A packet classification method according to an embodiment of the present invention is based on the assumption that in a rulebase, only some of the rules will match a given packet. The packet classification method according to an embodiment of the present invention will now be explained with reference to Table 1, which shows a rulebase of a typical firewall. [Elizabeth D. Zwicky et al. “Building Internet Firewall”, 2^nd edition. O'Reilly, 2000].

[0039]

TABLE 1
		Departure	Destination	Departure	Destination
Rule	Protocol	Port	Port	IP	IP	Action	Purpose of Rule
R0	*	*	*	Inside	Inside	Deny	Protection against
							Spoofing Attacks
R1	TCP	1024~65535	80	Outside	Inside	Accept	HTTP Service
R2	TCP	1024~65535	23	Outside	Inside	Accept	Telnet Service
R3	TCP	1024~65535	21	Outside	Inside	Accept	FTP Service
D	*	*	*	*	*	Deny	Default Rule

[0040] In Table 1, the term ‘inside’ means a local network protected by a firewall, and ‘outside’ means a network separated from the inside network by the firewall.

[0041] The inside network performs various application services such as a hypertext transport protocol (HTTP), a terminal emulation (Telnet) and a file transfer protocol (FTP). First to third rules R1, R2 and R3 receive access requests for diverse application services, and the zeroth rule R0 protects the inside network from a spoofing attack. The spoofing attack is a technique of attacking other target systems by deceiving its own ID information. Meanwhile, D denotes a default rule for excluding all communications with the outside. In Table 1, the packet using a user datagram protocol (UDP) protocol can match only R0 or D. Thus, it is not required for R1 to R3 to match the packet.

[0042] FIG. 1 is a view explaining the concept of a packet classification method through hierarchical rulebase partitioning according to an embodiment of the present invention. Referring to FIG. 1, the packet classification method through hierarchical rulebase partitioning according to an embodiment of the present invention is performed through two stages of preprocessing and classification. The idea is to use divide-and conquer approach.

[0043] At the preprocessing stage, the original rulebase is hierarchically partitioned into independent sub-rulebases of a small size based on the values of classification fields where each rule is defined. The degree of partitioning depends on the density of the sub-rulebase in a classification space. The higher the density of the sub-rulebase is in the classification space, the more the partitioning is required. If all the sub-rulebases become small enough, the partitioning is interrupted.

[0044] Next, at the classification stage, a packet classifier looks up the same header fields of an incoming packet and identifies the sub-rulebase (s) over the size of the original rulebase (n) is small, then scalability issue can be overcome by a single memory look up to the hash table, which is constructed during the preprocessing phase. If a sub-rulebase is still large, then the sub-rulebase can be re-partitioned until the final sub-rulebase is small enough.

[0045] According to [H. Kim, J. Heo, L. Choi, and S. Kim, “Taming Large Classifiers with Rule Reference Locality”, In Proceedings of the ICOIN. Vol. 1 pp. 35-50, February 2003.], a rule

[0046] $r_1=\left(\prod _{k=1}^d{F}_k^1,A_1\right)$
overlaps with a rule

[0047] $r_2=\left(\prod _{k=1}^d{F}_k^2,A_2\right)\mathrm{if}\forall {\mathrm{kF}}_k^1\bigcap F_k^2\ne \Phi$

[0048] Intuitively, two rules overlap if there exists any instance of a packet that matches both rules. Since sub-rulebases differ at least in those bits that are selected as the hash key, a packet cannot match both sub-rulebase at the same time. Thus, the independence among sub-rulebases is guaranteed. Therefore, we need to look up only the relevant sub-rulebase after inspecting a packet on the same bit fields.

[0049] Hereinafter, the preprocessing stage and the classification stage will be explained in order.

[0050] 1.1 Preprocessing Stage: Rulebase Partitioning and Hash Table Construction

[0051] In preprocessing stage, we partition the original rulebase into many independent sub-rulebase. For example, the rules governing HTTP, FTP, and SMTP traffic can be partitioned into separate sub-rulebases. Then, by looking up the protocol field of an incoming packet, we only need to look up the sub-rulebase with the same protocol.

[0052] We can choose any of the bits in the classification fields as a hash key. If we select 8 bits, then we create a hash table with 2⁸=256 entries, each of which points to a sub-rulebase. Intuitively, two rules may overlap if they map to the same sub-rulebase while rules mapped to different sub-rulebases would never overlap, which implies that they are independent. Sub-rulebases larger than a threshold value, such as 16 rules, can be repartitioned with another hash key, which must be different from the first hash key. This hierarchial partitioning stops until all the sub-rulebases are small enough. However, our experimentation results show that two levels of partitioning are enough for a rulebase under 500 K rules.

[0053] Both the space and time complexity of our classification algorithm depend on the number of nodes and the depth of the partitioning hierarchy. To reduce the number of partitioning we need to partition a rulebase into sub-rulebase as evenly as possible so that the number of empty sub-rulebase is minimized and the number of rules in sub-rulebases must follow uniform distribution. This partitioning efficiency depends on the hash key selection algorithm.

[0054] 1.2 Classification Stage

[0055] After the rulebase is partitioned and the hash table is produced at the preprocessing stage, the packet classifier maps an input packet on a corresponding sub-rulebase, so that the search range of the rulebase can be narrowed. The packet classifier searches the hash table using a hash key extracted from the packet header.

[0056] FIG. 5, parts (a)-(c) are views illustrating a rulebase and hash table made using a hash key algorithm that extracts respective 8 MSBs from the departure and destination IP addresses.

[0057] Referring to FIG. 5A, it is assumed that R0 to R3 are arranged in the order of their priorities. If the packet does not match the four rules R0 to R3, the packet matches a rule D having the lowest priority.

[0058] In this embodiment of the present invention, 5-dimensional packet classification is assumed. The packet classification is performed using 104 bits in total, i.e., 8 bits from a protocol field of the packet header, 16 bits from a source port, 16 bits from the destination port, 32 bit from the departure IP address, and 32 bits from the destination IP address. Generally, the protocol includes 8 bits, and has a specified number according to its kind.

[0059] Accordingly, the kind of the protocol is confirmed through the number of the protocol. Actually, the number of protocols appearing in the rule is less than 5 to 20. The ports are classified into the departure port and the destination port. The respective port is composed of 16 bits, and has one fixed number or area. Particularly, one of the departure and the destination on the basis of the firewall refers to the service port having a fixed number, and the other refers to the client having a temporary part in the area of 1024 to 65535.

[0060] Referring to FIG. 5B, only the most important 8 bits (hereinafter referred to as ‘8MSBs’) are presented in the classification space, and indicate a specified header field such as the protocol field.

[0061] By partitioning the rulebase into 256 buckets using 8MSBs as shown in FIG. 5A, the hash table as shown in FIG. 5B is produced. In the hash table of FIG. 5B, the rules that belong to one sub-rulebase do not overlap the rules that belong to other sub-rulebases.

[0062] When the packet arrives, the packet classifier extracts 8MSBs from the packet header and uses the extracted 8MSBs as the index of the hash table. The hash key used for the packet classification is identical with the hash key used to partition the rulebase. If an entry of the hash table is not empty after the hashing is completed, the packet classification can be performed in the sub-rulebase that belongs to the entry. If the entry of the hash table is empty, the default rule D becomes the matching rule. FIG. 5C shows a hash table made using b3 and b5 as hash keys.

[0063] 1.3 Hash Key Selection

[0064] 1.3.1 The First-Level Partitioning

[0065] At the first level partitioning of the preprocessing stage, the rulebase is partitioned into a large number of independent sub-rulebases only based on the kind of protocol and a port number. For example, the rules that control HTTP, FTP and simple mail transfer protocol (SMTP) traffic can be partitioned into the independent sub-rulebases. Thereafter, by searching a header field of the inputted packet, only the sub-rulebases having the same protocol field as that of the inputted packet are searched.

[0066] At the preprocessing stage, a hash key of specified bits is selected in the classification space for the partitioning of the rulebase. At the preprocessing stage, the classification space is composed of 40 bits. Here, the 40 bits are extracted from an 8-bit protocol field, 16-bit source port and 16-bit destination port. That is, only the protocol number and the port number are considered as the hash key for partitioning the rulebase. This is because the protocol number and the port number can naturally classify the rules based on the Internet services which are controlled by the rules. For example, the HTTP service corresponds to the 6th protocol and 80th server port.

[0067] If the 8-bit hash key is selected, a hash table having 28 (=256) entries can be made. Here, the respective entries mean the sub-rulebases. Meanwhile, in order to reduce the size of the hash table, one subset is selected from the classification space as a hash key. In this case, it is preferable to select the hash key so that the size of the hash table does not become too large. For example, in the case of using the hash key of 17 bits, a hash table having 128 K entries is produced. Meanwhile, a memory space and the depth of the partitioned layer depending on the size of the hash key are in a trade-off relationship.

[0068] In the case of using the hash key of 17 bits, initial 6 bits are extracted from the protocol field using a key selection algorithm that maximizes the entropy. Since only two protocols, i.e., TCP and UDP, are needed in specifying the port number, additional 11 bits for the protocols are selected from the port number.

[0069] The port number uses one bit to select either of the departure and the destination, and it is preferable to select a server port having a high density. It is preferable that if the selected port is the server port, additional 10 LSBs (Least Significant Bits) are selected from the port field of the server port, while if the selected port is a client port characterized by an upper boundary and a lower boundary, additional 6 MSBs (Most Significant Bits) are selected from the port field of the client port.

[0070] Typically, the server port having a high density points out a specified port number in the range of 0 to 1023. By contrast, the client port having a low density uses a random port number in the range of 1024 to 65536. Accordingly, lower 10 bits are used for the server port, and upper 6 bits are used for the client port.

[0071] Specifically, the hash key is related to [protocol field], [direction bit] and [lower 10 bits of the server port or upper 6 bits of the client port]. The server port has a partitioning efficiency higher than that of the client port, and if the rule can specify the server port that belongs to either of the departure and destination ports, the server port is used irrespective of the direction. If the rule can specify the client port that belongs to the departure and destination ports, the rules that belong to the hash table of the departure and the destination are spread using 6 MSBs.

[0072] If two rules are mapped on the same sub-rulebase, the two rules may overlap with each other. However, if the two rules match different sub-rulebases, the two rules never overlap with each other. This means that the two rules are independent. In the case of the sub-rulebase having a size larger than a threshold value, it can be partitioned with a hash key different from the first hash key. Here, the threshold value means the number of rules per sub-rulebase.

[0073] This hierarchical partitioning continues until all the sub-rulebases become sufficiently small. However, in order to reduce space and time complexities, the frequency of partitioning of the rulebase should be reduced. For this, the rulebase should equally be partitioned into sub-rulebases as far as possible so that the number of empty rulebases is minimized and the rules belonging to the sub-rulebase are uniformly spread. The efficiency of the rule-base partitioning as described above depends on the hash key selection algorithms.

[0074] 1.3.2 The Second-Level Partioning

[0075] The second level partitioning is applied only to buckets larger than the threshold value after the first level partitioning is completed. Since the hash key at the second level partitioning is derived from the hash key at the first level partitioning, the rulebase is partitioned based on the departure IP address and the destination IP address at the second level partitioning. To limit the size of the hash table, a subset is selected from the classification space as a hash key. The IP address is composed of 64 bits including the departure and destination addresses, and the partitioning of the rulebase is performed through the selection of a proper number of bits among the 64 bits. Preferably, about 16 bits including the departure and the destination are selected to perform the partitioning. In this case, since the bits selected for the partitioning directly affects the final result, the present invention proposes four hash key selection algorithms as follows in order to find the optimum bit pattern.

[0076] 1) MSB Pattern

[0077] The hash key of 16 bits is produced by extracting 8 MSBs from the departure and destination IP addresses. This algorithm uses the concept that most prefix masks select several significant bits from the IP address field. This simple hash key algorithm is used as a reference for comparing the performances of other hash key algorithms. The time complexity of this hash key algorithm is 0(1), and this means that it is independent of the number of rules that belong to the rulebase.

[0078] 2) Exponential Growing Pattern (EXP)

[0079] This algorithm produces a hash key of 12 bits by selecting the bit position corresponding to an exponential function of 2, i.e., b1b2b4b8b16b32, from the departure and destination IP addresses.

[0080] That is, the bit position is selected so that the distance between the bits becomes by two times, starting from the MSB. This algorithm is based on the concept that as the bit position of the IP address field are lowered, the number of masked-out bits becomes larger. Preferably, in order to produce a hash key of 16 bits, two bits of b6b11 are added. The time complexity of this key selection algorithm is also 0(1).

[0081] 3) Mask Distribution Pattern (MASK)

[0082] This algorithm uses the concept that a dummy bit cannot provide any information. Accordingly, the respective bit bi in the classification space has information according to a reverse rate of the number of dummy bits.

[0083] This hash key searching process is as follows. At respective bit positions bi, bits, which are not the dummy bit, are added in all defined rules that belong to a rulebase, and the added values are accumulated from MSB to LSB. In order to produce a hash key of K bits, one bit is selected whenever the accumulated value at the bit position becomes a multiple of a value obtained by dividing the total accumulated value by K. If the total number of rules that belong to the rulebase is n, the time complexity of this algorithm is 0(kn). K used as an experimental result is 16.

[0084] 4) Entropy-Maximizing Pattern (Ent)

[0085] In this embodiment of the present invention, the concept of entropy is used in order to find a good hash key. The concept of entropy is used in information theory. [Robert B. Ash, “Information Theory”, Dover Publications, 1^st edition, November 1990]

[0086] A good hash key should uniformly distribute the rules that belong to the hash table so that the difference between the number of null entries and the object number in the respective sub-rulebase is minimized. As is known, if the possibilities of producing all entries are equal, the entropy becomes maximum. Accordingly, a good hash key that can uniformly partition the rulebase should be searched through an entropy calculation. Hereinafter, a maximum entropy calculation for searching for a proper hash key will be explained.

[0087] Using the notion of entropy, a hash key

[0088] $K\sigma =K\sigma -1{\oplus }_q,\mathrm{where}\oplus$
is the concatenation operator and q is the bit from the classification space that produces the maximum entropy. The algorithm starts by calculating the entropy for the hash key of length 1 and determines the bit position that produces the maximum entropy value. Then, the algorithm repeats this process for the hash key of length 2 and so on until the length of the hash key reaches σ or the entropy does not increase further. Based on this algorithm, a 16-bit hash key is created by selecting an 8-bit hash key from each IP address field. The time complexity this algorithm is

[0089] $O\left(n·\left[\frac{s}{2}\left(2w-s+1\right)\right]\right),$
where w is the length of classification space, s is the length of a hash key, and the n is the total number of rules in a rulebase.

[0090] 1.4 Adaptation of the Algorithm for Prefix Mask and Range Specification

[0091] Up to now, the packet classification and the accurate matching values have been assumed. However, The definition of a rule sometimes includes a prefix mask and fields having a specified range. Hereinafter, how the present embodiment of the present invention can process other field specification will be explained.

[0092] 1.4.1 Prefix Mask Field

[0093] This is generally used to specify the range of an IP address field. Table 2 shows an example of a rulebase having a prefix mask.

[0094]

	TABLE 2
		Field Description
	Rule	(b0b1b2b3b4b5b6b7)
	R0	0000 0000
	R1	0110 0000
	R2	1000 0000
	R3	1*10 0000

[0095] A dummy bit is indicated by *. Here, it should be considered that if a rule includes a specified field having a mask, it is required to increase the rule by a multiple of an entry that belongs to the hash table.

[0096] Table 3 shows two different hash tables that may be constructed for the rulebase as illustrated in Table 2.

[0097]

TABLE 3
Index	Hash Key b0b1	Hash Key b0b2
00	R0	R0
01	R1	R1
10	R2, R3	R2
11	R3	R3

[0098] If the hash key b0b1 is selected, R3 can be spread into two entries having indexes 10 and 11 because b1 is a dummy term in R3. Hereinafter, this phenomenon is described as a ‘rule spreading’. As the rules overlap due to the rule spreading, it may refer to the size of the hash table.

[0099] However, if the hash key b0b2 is selected, the rule spreading as shown in Table 3 can be avoided. In order to prevent the rule spreading as much as possible, the modification of the maximum entropy key selection algorithm is required. That is, when calculating the entropy of one bit, the algorithm should disregard the defined rule so as to specify the dummy condition for the selected bit. This is because the dummy bit cannot give any information to the system from the viewpoint of entropy.

[0100] 1.4.2 Range Field

[0101] This is generally used to specify a TCP or UDP port. As explained above, a server port generally indicates specified ports in the range of 0 to 1023. By contrast, a client port indicates optional port number in the range of 1024 to 65536. Here, the basic concept is to convert the range specification into a prefix mask specification.

[0102] The range of prefix change for dividing a certain range into a group of prefix masks can be used. For example, [1024, 65535] in the range of 16 bits may be grouped into 6 prefix masks such as 000001*, 00001*, 0001*, 001*, 01* and 1*. However, this method undesirably causes an excessive rule spreading in the algorithm.

[0103] In this embodiment of the present invention, a precision-directed grouping is proposed. It is preferable to accurately group a rule having a specified range by multiple values of the rule. For example, the rule having the range of [71, 74] can be divided into 4 rules having accurate values in the range of 71 to 74. However, a rule having a wide range such as [49152, 65535] can be divided into 16,384 rules. Fortunately, the TCP/UDP port having one range is generally preponderant. For example, even if the total number of prepared ports is very large such as [0, 49151], 80% of port numbers used for most rules is less than 3,999.

[0104] This means that different numbers of rules, which depend on the density range, can be grouped. Preferably, in a specified port range of 16 bits, 10 LSBs are used as the hash key in a high-density area of [0, 1023] as a single entry is produced with respect to ports less than 1024. By contrast, 6 MSBs are used as the hash key for a low-density area of [1024, 65535] as 63 entries each of which has 1024 ports are produced.

[0105] 2. Experimentation and Results

[0106] In this section, the performance of the proposed algorithms for 5-dimensional classification is demonstated. Since it is difficult to obtain large real-life classification rulebases, we synthesized large rulebase from real-life packet traces. The packet traces were collected from PUBNET for five to eight hours during three days. One-hour trace is nearly 70 million packets. The first trace is used to synthesize the rulebase and others are used as data for packet classification. To create a synthetic rulebase that resembles real-life rulebases, we carefully synthesized a rulebase that resembles real-life characteristics observed from real-life firewall applications. All of our experimentation was performed in 1.7 GHz Pentium IV system with 512 MB of memory running Linux.

[0107] 2.1 Rulebase Partitioning

[0108] 2.1.1 First Level Partitioning

[0109] FIG. 2 shows results of the first level partitioning. In FIG. 2, the average size and the maximum size of the sub-rulebases after the partitioning are shown. After the partitioning, the average size of the rulebase is actually reduced. The reduction rates of the rulebases in the rulebases having rules, the numbers of which are 10000 (10 K), 100000 (100 K) and 500000 (500 K), are 0.0028, 0.0014 and 0.0014, respectively. However, as can be seen from the maximum size of the rulebase in the drawing, the rules are not uniformly spread in the partitioned rulebase.

[0110] The largest sub-rulebase in all the rulebases used in the experiment has rules the number of which corresponds to about 24% of the rules of the original rulebase in all the rulebases tested. As might be expected, these rules are related to the HTTP service corresponding to the protocol 6 and the port number 80. In the rulebase having rulebases the numbers of which are 10000 (10 K), 100000 (100 K) and 500000 (500 K), the numbers of sub-rulebases that exceed the threshold value (which corresponds to 16 rules per sub-rulebase) are 141, 192 and 1114, respectively. With respect to these sub-rulebases, a second level partitioning is performed.

[0111] 2.1.2 Second Level Partitioning

[0112] FIGS. 3 and 4 show the results obtained by performing the second level partitioning using diverse hash key selection algorithms. FIG. 3 shows an average number of rules per sub-rulebase, and FIG. 4 shows the size of the largest rulebase. In the case of using the maximum entropy hash key algorithm, the second level partitioning reduces the rulebases having 10000 (10 K), 100000 (100 K) and 500000 (500 K) rules by the rates of 0.054, 0.054 and 0.052, respectively, in comparison to the first level partitioning.

[0113] After the first level partitioning and the second level partitioning are all completed, the rulebases having 10000 (10 K), 100000 (100 K) and 500000 (500 K) rules are partitioned into sub-rulebases having 1.6, 7.6 and 36.6 rules, respectively, on the average. This means that the rulebases are reduced by the rates of 0.00016, 0.000076 and 0.000073.

[0114] The second level partitioning is effective in reducing an enormous sub-rulebase that includes 24% of the original rulebase after the first level partitioning. After the second level partitioning is performed, the sizes of the largest rulebases can be reduced from the rulebases having 10000 (10 K), 100000 (100 K) and 500000 (500 K) rules to rulebases having 31, 258 and 1281 rules, respectively, using the maximum entropy hashing algorithm.

[0115] These rulebases correspond to 0.31%, 0.26% and 0.26% of the original rulebases. This means that in the case of the rulebase having 100 K rules, 256 rules should be confirmed with respect to one packet in the worst case. The second level partitioning is more efficient than the first level partitioning in reducing the size of the largest rulebase. The first level partitioning partitions the original rulebases according to the Internet service that controls the rules.

[0116] Accordingly, the largest sub-rulebases that control the HTTP service are grouped into a single sub-rulebase. By contrast, the second level partitioning can reduce the largest sub-rulebases by considering the distribution of the rules in the classification space using the maximum entropy hash key algorithm.

[0117] Referring to FIGS. 3 and 4, the efficiencies of the different hash key algorithms in the second level partitioning can be compared. As was expected, an optimum result of partitioning was derived through selection of the maximum entropy key. Particularly, this method can reduce the size of the largest sub-rulebase in comparison to other key selection algorithm. The maximum entropy selection algorithm is efficient as much as the factors of 2.38, 3.42 and 3.49 with respect to the rulebases having 10 K, 100 K and 500 K rules in comparison to the MSB pattern key algorithm. Comparative results of the average sizes of the sub-rulebases are illustrated with respect to all the key selection algorithms. However, the maximum entropy algorithm is most efficient with respect to rulebases having 100 K or more rules.

[0118] It is difficult to judge the expansion from the results of rulebase partitioning as shown in FIGS. 3 and 4 since the size of the rulebase is not linearly increased on a horizontal axis. However, as a result of experiment, it is proved that as the size of the rulebase is increased, the maximum size and average size of the sub-rulebase are linearly increased. This results from the two level partitioning. If needed, much more level partitioning may be performed to further reduce the size of the sub-rulebase. As described above, the amount of memory required for the algorithm according to an embodiment of the present invention was investigated considering the total number of rules, the total number of buckets and the increase rate of the total number of buckets that exceed the threshold value, and the algorithm appears a good scalability with respect to the total number of rules, the total number of buckets and the increase rate of the total number of buckets that exceed the threshold value.

[0119] 2.2 Classification Performance

[0120] To demonstrate the classification performance of our algorithm, real-life packet traces which collected from PUBNET are used. The size of packet trace is 202 million packets, which is big enough to obtain the confidence of our experiment. After the first and second-level partitioning, the worst-case search algorithm is assumed, for example, a simple linear search, to find the matching rule in the final sub-rulebase so that the pure effectiveness of the proposed partitioning algorithm can be showed.

[0121] FIG. 6 is a graph showing the classification performance from the viewpoint of the average number of rules referred to for searching the matching rule. Referring to FIG. 6, the result of packet classification using the maximum entropy key selection algorithm shows that the rulebase having 10000 (10 K), 100000 (100 K) and 500000 (500 K) rules has been reduced to the rulebase having 5.6, 42.64 and 207.02 rules.

[0122] This result is very encouraging in consideration of the fact that the most superior packet classification method in the prior art requires at lest 13th times referring of a memory for the packet classification in the rulebase having 5000 (=5 K) rules. According to an embodiment of the present invention, even in the case that a linear search, which is the worst for the final sub-rulebase, is performed, only two times referring of a memory is required to search the hash table, and only 5 to 6 rules should be referred to.

[0123] According to the above-described embodiment of the present invention, the size of the rulebase can actually be reduced through two-level partitioning. For example, in the rulebase having 100 K rules, the number of rules is 7.6 on the average, and 258 in the worst case. In the rulebase having 100000 (=100 K) rules, the packet classifier checks only 258 rules with respect to the input packet in the worst case.

[0124] According to the experimental data in the actual packet trace, the packet classifier checks only 4.2, 5.6, 8.4, 20.4, 42.6, 83.2 and 207 rules, on the average, in the rulebase having 5000 (5 K), 10000 (10 K), 20000 (20 K), 50000 (50 K), 100000 (100 K), 200000 (200 K) and 500000 (500 K) rules, respectively.

[0125] Even if the search is performed in the sub-rulebase in the worst case, the above-described embodiment of the present invention surpasses the RFC method, which is one of the optimum algorithms, in the case that the size of the rulebase is relatively small. Here, the RFC method requires 13th times referring of a memory with respect to the rulebase having a size of 500 (=5 K) rules. Also, as the size of the rulebase is increased, the above-described embodiment of the present invention has a peculiar scalability in space and time.

[0126] Although an embodiment of the present invention have been shown and described, the present invention is not limited to the described embodiment. Instead, it would be appreciated by those skilled in the art that changes may be made to the embodiment without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.