Computer network management

[0002] This invention relates to computer networks and, more particularly, to the management of computer networks.

[0003] Because of the explosion in the complexity of computer networks, computer network management has become critical. Network management is required to perform fault diagnosis, performance management, predict loads, plan for future traffic and the like. Indeed, automated tools for computer network management on such large-scale complex and heterogeneous networks are crucial to ensure that the networks remain healthy and available.

[0004] Known network management tools and methodologies are presently not capable of filtering information intelligently at the individual network elements. Furthermore, there is little support for event notification, which results in excessive network management traffic.

[0005] The present dominant standard for network management is the “Simple Network Management Protocol” (SNMP). SNMP and other known network management methodologies suffer from a number of deficiencies including the following:

[0006] Generate a High Volume of Management Traffic: The SNMP protocol supports retrieval of single objects stored at network elements but does not allow any sort of computation to be performed at the individual network elements. As a result, large volumes of data may need to be transferred to a network manager (station at which network management is being performed) and the network manager may filter most of the retrieved data.

[0007] No Support for Event Notification: Although there is primitive support for event notification in the form of traps in SNMP, it is not sufficiently expressive. Therefore, network management using SNMP is predominantly polling based, which results in the familiar problems of either missing an event (if the polling interval is long) or incurring a large overhead (if the polling interval is short). To perform effective and efficient network management, support for complex event detection and notification is required. For example, a network manager may want to be notified when the average error rate on all the interfaces of a switch exceeds ten percent.

[0008] Centralized processing: Network management has traditionally been performed in a centralized fashion primarily to ensure that the impact of adding network management to managed nodes is minimal. However, the central network manager could become a bottleneck as the network complexity increases.

[0009] Problems and limitations of prior known computer network management arrangements are addressed by incorporating database technology into individual network elements of the computer network. This, in turn, allows such enhanced network elements to filter management information intelligently and also to notify an associated network manager of the occurrence of complex events of interest. More specifically, the network elements are enhanced through use of database technology to process declarative queries and to support triggers.

[0010] Additionally, one or more auxiliary network managers, that perform as proxies for network elements that have not been enhanced with database technology, are employed to collect and integrate management information from one or more non-enhanced network elements. Consequently, the management information supplied to a network manager from the auxiliary network mangers could be significantly less than that collected from the network elements. Thus, the auxiliary network managers further reduce the network management traffic.

[0011] In a specific embodiment of the invention, support is embedded into the individual network elements for a declarative query language, one example being the structured query language (SQL). Support is also added for event notification to the individual network elements. One or more auxiliary network managers are employed that can answer declarative inquiries. Moreover, the management information base information stored in the individual network elements is modeled as relational tables that are queried.

[0012] FIG. 1 shows, in simplified block diagram for, details of a network in which an embodiment may be advantageously employed;

[0013] FIG. 2 illustrates a flow diagram showing steps in the query process employed in the network manager of FIG. 1;

[0014] FIG. 3 illustrates a flow diagram showing steps in the query process employed in an auxiliary network manager used in the network of FIG. 1;

[0015] FIG. 4 illustrates a flow diagram showing steps in the query process employed in an enhanced network element utilized in the network of FIG. 1;

[0016] FIG. 5 graphically illustrates a base table useful in describing an embodiment of the invention; and

[0017] FIG. 6 graphically illustrates another base table also useful in describing an embodiment of the invention.

[0018] FIG. 1 shows, in simplified block diagram for, details of a network in which an embodiment may be advantageously employed. Specifically, shown is network manager (NM) 101 that is a computer station at which network management is performed. NM 101 communicates via a communications medium 102 to, for example, auxiliary network manager (ANM) 103, ANM 104, sub-network 105 and sub-network 106. Communications medium 102 may be a local area network (LAN), wide area network (WAN), wireless link, telephone link, or the like. ANM 103 communicates, in this example, via sub-network 107 with network elements 109, 110 and 111. In this example, network element 109 is an enhanced network element that is described below. Network elements 110 and 111 are ordinary network elements including typical network element modules. Similarly, ANM 104 communicates via sub-network 108 with ordinary network elements 112 and 113. Sub-network 105 communicates with enhanced network element (ENE) 114 and sub-network 106 communicates with ordinary network element 115. It is noted that in this example, queries are supplied in a Structured Query Language (SQL).

[0019] It should be noted that a simple network management protocol (SNMP) has emerged as the current standard for network management in the internet. SNMP has two important components:

[0020] The notion of a Management Information Base (MIB) that is essentially a schema for storing data objects related to the activity of a network element. The schema is essentially a hierarchical database in that the entire data is organized as a tree.

[0021] A standard protocol for retrieving information stored in the MIBs. This standard allows network management processes to retrieve specific objects (using snmpget) in the MIB or to retrieve an entire subtree (using snmpwalk) rooted at a node.

[0022] FIG. 2 illustrates a flow diagram showing steps in the query process employed in the network manager 101 of FIG. 1. Typically, network manager 101 includes the following modules: a query receiver, a query parser, a query optimizer, a query execution plan (QEP) generator and a query execution engine (called evaluator in the implementation). In this example, in step 201 a user inputs a Structured Query Language (SGL) query that is received by the query receiver. Usually the SQL query will be parsed by the parser. That is, the query parser is typically a process that analyzes a statement, e.g., the query, and resolves it into a form that can be understood by network manager 101. In this example, the query parser is a SQL parser. Such parsers are known in the art. It is further noted that the query parser does not have schema meta-data, so it infers the schema of involved tables from the query itself. Another option (which is not employed in this example) is to make the parser MIB-knowledgeable, so that it can identify schema problems early, before query execution is actually carried out.

[0023] Then, step 202 causes NM 101 to determine the enhanced network elements (ENEs) and/or auxiliary network managers (ANMs) required to answer the supplied SQL query. It is noted that there may be a set on such ENEs including zero, one or more enhanced network elements and/or a set of ANMs including zero, one or more auxiliary network manager units. Thereafter, step 203 causes a query execution plan (QEP) to be computed, i.e., determined, for each of the determined ENEs and ANMs and, thereafter, sends, i.e., transmits, the QEPs to the determined associated ENEs and ANMs. Usually, this is realized by a query optimizer that takes in the supplied SQL query and outputs the needed Query Execution Plans (QEPs). One such example follows: public class QueryOptimize

[0024] A QEP is basically a relational algebraic tree (RAE), with the addition of two types of nodes: snmp_union and snmp_singleton. They both can have only one child. An snmp_union signals that its child should be sent to all ANMS, and the union of the returning results taken; while an snmp_agent means that its child needs only be sent to a single ANM.

[0025] Since the optimizer doesn't have statistical information about base tables (which are virtual and not materialized), it basically just pushes selection and projections down the algebraic tree, while bringing snmp_union and snmp_singleton nodes up the algebraic tree. It can also identify common subtrees. This optimization not only reduces computing strength of the query, but also reduces network traffic used to ship partial results back and forth between NM 101 and ANMs 103 and 104, and between ANMs 103 and 104.

[0026] A query execution engine, i.e., evaluator, is typically employed to execute the QEP and one example is as follows:

[0027] The evaluator is actually composed of two parts, i.e., a relational algebra engine (RAE) and SNMP wrapper. Since accessing SNMP data is potentially much more slower than accessing a true relational database on a local disk, the relational algebra engine should be made as parallel as possible. For example, relations involved in a multi-way join should be evaluated simultaneously, unless the result of evaluating one particular relation limits the number of ANMs to use to evaluate other relations (i.e. there is a join on snmp_agent attribute which shall impose a constraint on the possible values of that attribute-semi-join).

[0028] Then, step 204 causes the results to be obtained, i.e., transmitted, from the determined ENEs and ANMs and causes those results to be combined to yield the query result. Step 205 causes NM 101 to display the obtained query result to the user.

[0029] FIG. 3 illustrates a flow diagram showing steps in the query process employed in an auxiliary network manager (ANM) 103, 104 employed in the network 100 of FIG. 1.

[0030] An ANM 103, 104 typically includes a query execution engine (virtually the same as contained in NM), a SNMP wrapper (embedded in evaluator in the implementation) and a Java remote method invocation (RMI) interface. Specifically, step 301 obtains, i.e., receives, an associate QEP for the ANM 103, 104 from NM 101. Then, step 302 causes the translation of the QEP into a sequence SNMP calls to one or more associated network elements (NEs). In this example, a SNMP wrapper converts SQL queries or relational algebraic expressions into the series of SNMP calls. As is known, relational algebra is a simple language to express queries, such as, SQL queries, to a database. A relational algebra engine accepts relational algebra queries and executes them and returns the result.

[0031] The Java RMI interface of an ANM 103, 104 is as follows:

[0032] Relational algebraic expression (RAE) is the QEP.

[0033] Step 303 obtains results of the SNMP calls to the NEs and combines the obtained results to generate the result of the QEP. Then, step 304 returns the result of the QEP to NM 101.

[0034] FIG. 4 illustrates a flow diagram showing the steps in the query process employed in an enhanced network element (ENE) employed in the network 100 of FIG. 1. As is known, SNMP provides a simple “get” and “set” mechanism to get values of variables and to set them. The variables are defined in a MIB and every network element has an associated MIB. Thus, to retrieve information from a network element a sequence of SNMP calls may be used and, then correlate the results of the calls. It is noted that use of SQL queries makes it significantly easier to realize this for the user. Again, this requires that the SQL query be internally converted to the sequence of SNMP calls. Consequently, the user does not have to write any software code to realize this conversion from SQL to the SNMP calls. An enhanced network element (ENE) 108, 114 typically includes a query execution engine (virtually the same as contained in NM 101 and ANM 103, a SNMP wrapper (embedded in evaluator in the implementation) and a Java remote method invocation (RMI) interface. Specifically, step 401 obtains, i.e., receives, an associated QEP for the NE 108 or 114 from NM 101. Then, step 402causes the translation of the received QEP into a sequence of SNMP calls for this enhanced network element (ENE). In this example, the SNMP wrapper converts SQL queries or relational algebraic expressions into a series of SNMP requests. The Java RMI interface is essentially identical as that employed in ANM 103, 104 and described above. Step 403 evaluates the SNMP calls and collates the results of the. SNMP calls to obtain the QEP result. Then, in step 404 the QEP result is returned, i.e. supplied or otherwise transmitted, to NM 101.

[0035] The following is a relational data model for network management data.

[0036] All network management data as viewed by a network management (NM) station 101 (FIG. 1) over a specific network management domain—the set of SNMP agents manageable by the NM 101—are conceptually viewed as a relational database. The schema of the (conceptual) management database is described below.

[0037] First, it is felt best to explicitly distinguish four different types of identifiers used in SNMP. An SNMP identifier can be one of the following:

[0038] (a) a non-leaf ASN.1 object identifier (i.e., not denoting any type or instance), e.g., interfaces;

[0039] (b) an identifier denoting the single instance of a certain non-aggregate object type, e.g., interfaces.ifNumber.0 and interfaces.ifTable.ifEntry.ifType. 1;

[0040] (c) a leaf ASN. 1 object identifier denoting a non-aggregate data type, e.g., interfaces.ifNumber

[0041] (d) an identifier denoting an aggregate type, e.g. interfaces.ifTable, interfaces.ifEntry.

[0042] Identifiers of types (a) and (b) do not appear in our schema. Identifiers of type (d) denoting an entry of a table (e.g., interfaces.ifEntry) also does not appear in our schema.

[0043] (For simplicity, 0 attributes are of type string in the network 100. Any leaf node in ASN.1 object identifier tree defines a new data type, however it may be just a stereotyped ASN.1 syntax as defined in SMI or a subtype of such a stereotyped syntax.) Single-instance variables are:

[0044] For each type-c SNMP identifier <c>, we have the following base table (Table A, FIG. 5):

[0045] <c>(snmp_agent, value).

[0046] It is a collection of values of <c>. 0 on different SNMP agents, tagged with the IP address of those SNMP agents (snmp_agent attribute). For example, we can have: [interfaces. ifNumber](snmp_agent, value) and we can raise a query at a network management station such as:

[0047] SNMP tables are:

[0048] For each type-d SNMP identifier denoting a table <t>, we have the following base table:

[0049] <t>(snmp_agent, <cl>, <c2>, . . . ).

[0050] It is the union of individual SNMP tables of the SNMP agents in the domain, with the added attribute snmp_agent.

[0051] For example, we can have (Table B, FIG. 6): [interfaces.ifTable](snmp_agent, ifIndex, ifDescr, ifType, . . . , ifSpecific), and we can raise a query at a network management station such as:

[0052] Example queries are:

[0053] Systems information about all agents in the domain.

[0054] Number of interfaces of all agents in the domain.

[0055] All 100 Mbps interfaces.

[0056] Find the immediate NEXT HOPS of a given agent.

[0057] Find the immediate PREVIOUS HOPS of a given agent

[0058] When a user submits a query at a NM 101, the NM 101 receives the query, determine ENEs and ANMs required to answer the SQL query, usually parses the query, optimizes the query and generates a distributed query execution plan (QEP). The distributed QEP is then carried out on a distributed query execution engine. The distributed query execution engine involves the NM 101 and ANMs 103, 104 or network-enabled SNMP agents (which exposes an ANM interface). Basically, the execution engine at the NM 101 sends subqueries to involved ANMs 103, 104, gets back subqueries results, and recomposes the final query result. Note that multiple rounds between a NM 101 and ANMs 103, 104, and between ANMs 103 and 104 may be necessary to get a complex query answered.

[0059] Further, note that network 100 base tables are essentially horizontally partitioned among ANMs 103 and 104. Each ANM 103, 104 is responsible for a set of SNMP agents. Ideally, each SNMP agent becomes network 100 enabled, and works as an ANM for itself. Such SNMP agents are intelligent agents with the capability of carrying out relational queries. However, with legacy systems, network 100 will most likely still run on a many-snmp-agents-per-ANM basis.

[0060] The user interface of an ANM 103, 104 should enable administrators to configure the set of SNMP agents that ANM 103, 104 is responsible for. This function should preferably be able to be done dynamically. However, since there is no way to automatically locate SNMP agents, this configuration function has to be done manually.

[0061] In assigning SNMP agents to ANMs 103, 104, an administrator should be very careful to cover all SNMP agents of interest. Overlapping is allowed, and the network 100 will automatically pick one ANM among several ANMs representing a same SNMP agent. In addition, the administrator should assign an SNMP agent to the closest ANM to reduce total network traffic (and benefit from the network 100).

[0062] The configuration of a NM 101 could also be done manually, e.g., let the administrator compile a list of IP addresses of ANMs. A better option is to use a network plug-and-play system such as Jini to make the process both automatic and dynamic, i.e., when new ANMs 103, 104 come and go, the NM 101 automatically discovers them, and updates its list of ANMs.

[0063] The limitation of an automatic and dynamic configuration of ANMs 103, 104 is that the it is not easy for an administrator to control the set of SNMP agents in a network management domain. With Jini, it's possible to do lookup (for ANM services) using a certain policy, such as based on location. However, there are too many possible policies, and it is extremely difficult or impossible to implement all of them.

[0064] The current network 100 uses the following policy: an ANM will multicast lookup discovery requests to the standard Jini-specified IP address (224.0.1.85) and port (4160); the TTL can be set to limit the area of discovery: a value of one (1) will limit discovery to the local LAN segment, and a value below 64 (in the United States) will usually limit discovery in a company site. An ANM 103, 104 will register itself with all Jini lookup services it discovers. When a NM 101 starts up, a Jini lookup service discovery wizard will guide the user through the process of finding an available Jini lookup service. Generally a multicast discovery is sufficient. The NM 101 will form a network 100 management domain from all ANMs 103, 104 registered to that Jini lookup service (chosen by the user).

[0065] The arguments for such a policy are: a) it's simple and easy to understand; b) it's very automatic and (potentially) dynamic; c) most importantly, since it's easy to filter SNMP agents using SQL's WHERE clause, we want to include as many SNMP agents as possible in a network 100. However, under such a policy a user presently has less control over the forming of a network 100 management domain.

[0066] Another possibility is that the user specifies a list of SNMP agents in a network 100 management domain, and the NM 101 attempts to locate one ANM 103, 104 for each SNMP agent by matching the SNMP agents information in an ANM 103, 104 registers with the Jini lookup service.

[0067] The above-described embodiments are, of course, merely illustrative of the principles of the invention. Indeed, numerous other methods or apparatus may be devised by those skilled in the art without departing from the spirit and scope of the invention.