System and method for managing faults using a gateway

[0002] The information-communication industry is currently undergoing tremendous change from the constant evolution of new technologies and changing market conditions. As a result, network and information service providers such as telephone, cable, and wireless carriers, Internet Service Providers and utility companies all have the need to rapidly deploy new systems and system elements and alter their existing management systems to accommodate evolving business and network requirements. These requirements include the ability to integrate existing network equipment and systems with new elements and applications, customize existing systems and applications, and scale systems to accommodate growing networks and traffic volumes.

[0003] One area of management involves fault management. Fault alarm incidents (or messages) are routinely generated for the various components of a network to allow the service provider to monitor the operational state of the network. Fault management systems generally receive and process these alarm incidents in accordance with fault management objectives as defined by the service provider. Unfortunately, however, current fault management systems can be difficult to implement and modify as the network, along with management objectives, change over time. In addition, existing systems are inefficient with substantially all of the management processing occurring in a centralized management processor system.

[0004] Accordingly, what is needed is an improved fault management system and method.

[0005] The present invention provides an improved fault management system and method. In one embodiment, the system includes a gateway and a management processor system. The gateway is communicatively connected to a network for receiving alarm incidents from the network. The gateway has a rule engine for (1) selecting a control object from a set of control objects based on information from the alarm incident, and (2) processing the selected control object. The management processor system has a processor for processing configuration objects in response to the selected control object for implementing fault management objectives defined by at least one user.

[0006] The foregoing has outlined rather broadly the features and technical advantages of the present invention in order that the detailed description of the invention that follows may be better understood. Additional features and advantages of the invention will be described hereinafter which form the subject of the claims of the invention. It should be appreciated by those skilled in the art that the conception and specific embodiment disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present invention. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the invention as set forth in the appended claims.

[0007] For a more complete understanding of the present invention, and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:

[0008] FIG. 1 is a functional depiction of the conceptual TMN relationship between a management system and the managed network.

[0009] FIG. 2 shows a logical functional diagram of a TMN-based management system.

[0010] FIG. 3A diagrammatically shows the various TMN management layers.

[0011] FIG. 3B shows a table including the management layers of FIG. 3A with associated functional groups.

[0012] FIG. 4 depicts one embodiment of a management system for managing and providing network services.

[0013] FIG. 5A shows one embodiment of a management processor for implementing a management system.

[0014] FIG. 5B shows another embodiment of a management processor for implementing a management system.

[0015] FIG. 6 illustrates one embodiment of a class tree for object classes within the element, network, and services layers.

[0016] FIG. 7 shows one embodiment of a fault management system with a diagrammatic depiction of the processing flow of an incoming alarm incident.

[0017] FIG. 8A shows one embodiment of a fault management control object class.

[0018] FIG. 8B shows embodiment of a list of basic processing attributes for the control object class of FIG. 8A.

[0019] FIG. 8C shows embodiment of a list of advanced processing attributes for the control object class of FIG. 8A.

[0020] FIG. 8D shows one embodiment of lists of correlation, thresholding, escalation, and suppression sub-attributes for the advanced processing attributes of FIG. 8C.

[0021] A. Overview

[0022] One embodiment of the present invention operates in connection with an operations support systems (OSS) framework for managing networks and network services that are provided to customers. One example of an OSS framework is NetExpert™, which is available from Objective Systems Integrators of Folsom, Calif. This OSS framework is based on the standard Telecommunication Network Management (TMN) architecture promulgated by the International Telecommunications Union. FIG. 1 functionally depicts the conceptual TMN relationship between OSS 120 and the managed network 110, which includes network elements 115. FIG. 1 illustrates the overseeing nature of OSS 120. Network elements 115 correspond to the physical modules and systems (e.g., switches, termination points, databases, sub-networks) that are.“managed” by OSS 120. One of the aspects of the TMN paradigm is that it promotes inter-operability between different components, systems, and networks within the managed “network” 110, regardless of their particular configurations and protocols. The present invention not only implements these TMN objectives, but also, provides an OSS (or at least Network Management System (NMS) sub-systems) that have improved efficiency and scalability over existing management systems.

[0023] FIG. 2 shows a logical functional diagram of an OSS 120 consistent with the TMN standard. OSS 120 includes operational systems functions 122, mediation functions 124, adaptor functions 126, network element functions 128, and user interface functions 132. Mediation functions 124 communicatively link operational systems functions 122, adaptor functions 126, and network element functions 128 between one another. User interface functions 132 are linked to operational systems functions 122.

[0024] Operations systems functions 122 correspond to functions that manage the OSS. It performs various activities including obtaining management information such as acquiring alarm information from managed network elements, performing the required information processing activities on the network (e.g., correlating alarms, implementing service requests), and directing the managed elements to take appropriate action such as performing a test. Mediation functions 124 perform functions for mediating communications between the various functional blocks. It includes resources for storing, filtering, and translating data in order for information to be in a suitable form for a receiving functional block. For example, network element information may arrive at the NMS 120 packaged in various protocols such as X.25, Protocol 95, and BX.25. Adaptor functions 126 interface communications between the managed network 110 and third-party systems not necessarily managed by the NMS 120. Network element functions 128 correspond to the actual physical elements that make up the network 110. Incidents (or information packets) corresponding to the actual managed network elements are provided to the operations systems functions 122 via the mediation functions 124 in various manners. Some network elements (e.g., a switch) may generate and transmit its own incidents while others (e.g., a router or circuitpack) may be managed by an element manager, which generates and transmits the incidents for its managed elements. Finally, the user interface functions 132 provide to human users access to the operational Hi systems functions 122. Note that the adaptor, network element, and user interface functions are represented as being partially in and out of the OSS 120 because they are part of the system, but they also interface with the real physical world.

[0025] As shown in FIG. 3A with management layer triangle 140, the TMN standard abstracts four different layers for managing a network through an OSS framework. These layers include business management layer 142, service management layer 144, network management layer 146, and element management layer 148. Also fit into this conceptual depiction is a network element layer 152, which interfaces between the physical network and the NMS 120 through the element layer 148. Network element layer 152 corresponds to the network element fimctions 128 from FIG. 2. The element layer 148 is the lowest layer in the model; it corresponds to (abstracts) the most fundamental aspects of the network including its physical components and systems. The next layer upward is the network layer 146. It generally corresponds to (abstracts) the network's higher-level components and interconnections. The next upward layer is the service layer 144. This layer generally corresponds to (abstracts) the nature of the services (e.g., number and type, of ports, quality) provided to subscribers or clients of the network. (The term client is used broadly and covers any type of network client including clients within an intranet or subscribers to a network service provider such as a telephone, wireless, cable, or Internet service provider, for example.) Finally, the business layer 142 corresponds to (abstracts) business aspects of the network such as market performance, and profit and loss centers.

[0026] The TMN architecture also articulates general functional groups that are to be managed across the various network management layers. These functional groups include fault, configuration, accounting, performance, and security. FIG. 3B shows table 160, which includes management layer rows (business layer 162, service layer 164, network layer 166, and element layer 168) and functional group columns (fault 172, configuration 174, accounting 176, performance 178, and security 182). The depicted functional categories, which apply to each layer, are derived from the TMN architecture (M.3200).

[0027] B. Management System

[0028] FIG. 4 shows one embodiment of a management system 200 for performing one or more of the FCAPS (fault, configuration, accounting, performance, and security) functions across the various management layers. Management system 200 includes gateways 225 and management processor system 230, which includes management processors 230A, 230B, 230C, and 230D (for the element, network, service, and business layers, respectively). As shown in FIG. 4, service management processor 230C is communicatively interconnected between the network and business management processors 230B and 230D, respectively. Likewise, network management processor 230B is interconnected between the element management processor 230A and the service management processor 230C. The element management processor 230A is linked through gateway 225 to the managed network elements 210 for sending commands to and receiving incidents from the managed network elements. Thus, the element management processor 230A serves as the interface for the management processor system 230. Management processor system 230 is also connected to third party systems 220 through gateways 225.

[0029] Managed network elements (or network elements) 210 correspond to the various elements and associated element managers of the managed network. (This Managed network elements block 210 corresponds to the network elements functions 128 and network elements 115 from FIGS. 2 and 1, respectively.) Each managed element provides (either directly or through an element manager) an incident message that includes pertinent information about a particular element. For example, an incident for a switch could be an alarm incident that identifies the switch and indicates that some portion thereof has failed. With networks being composed of elements from different vendors, incidents can vary in form (e.g., different protocols such as X.25, HTP/IP, Ethernet, Serial Asynchronous (RS-232), TCP/IP, SNMP, CMIP) from one element to another. Accordingly, incident-receiving gateway(s) 225 include, among other things, a rule engine for (1) identifying, parsing, and normalizing incoming incidents, and (2) performing basic processing tasks.

[0030] Normalizing involves translating (or mapping) the incident into a suitable, consistent form (e.g., ASCII), which may be discernable within the entire management system 200. Identifying and parsing involves identifying the incident's source and associated management level so that it may be processed in the correct management processor (i.e., element 230A, network 230B, service 230C, or business 230D). Most (but not all) of the incidents received from network elements 210 will be element layer incidents because these incidents correspond to signals from the physical network “elements.” However, incidents are not exclusively generated within the physical network itself Management processors 230A, 230B, 230C may also generate incidents for higher level layers and pass them upward to the corresponding management processor. For example, a given customer may have contracted for premium accessibility. This would correspond to a service layer function or issue. A number of physical network or element layer elements responsible for providing this service may be impaired. However, the elements themselves are not capable (or even in a position) to communicate this “service” problem to the system. Thus, the element and/or network management processors may determine, from an aggregate of element and/or network alarm incidents, that this customer's premium service is impaired. An appropriate service alarm incident would then be generated and provided to the service management processor 230C by the element and/or network management processors 230A, 230B, respectively.

[0031] This gateway 225 is also capable of performing basic processing tasks. In one embodiment (as will be discussed in more detail below), configuration objects, which include both control and scenario objects, are initiated and executed for performing management functions. The gateway 225, with its processing capability, selects and at least partially processes an initial control object in response to a received incident. In this manner, processing is more efficiently distributed between the management processor system 230 and the gateway 225 rather than exclusively occurring in the management processor system 230, which may be implemented with a centralized server. Moreover, while FIG. 4 depicts a single incident-receiving gateway, multiple incident-receiving gateways may actually be implemented with a centralized management processor system 230. Basic control object processing is performed in the distributed gateways with advanced processing being performed in the management processor system 230. However, cross element manager or cross domain processing would typically not be carried out in the gateways. In these embodiments, enhanced efficiency resulting from distributed processing over the multiple gateways becomes even more significant.

[0032] The management processor system 230, which includes management processors 230A, 230B, 230C, and 230D, performs basic, as well as advanced processing tasks, for managing or implementing a given function (e.g., fault, configuration, accounting, performance, security) across the various management layers. The management processor system 230 may be implemented on one or more connected servers. The individual processors 230A, 230B, 230C, 230D (which will be discussed below) may be physically, as well as conceptually, distinct from one another. FIGS. 5A and 5B show management processors 230A (FIG. 5A) and 230B, 230C, 230D (FIG. 5B) with more particularity. These depictions are essentially the same except that in FIG. 5A, an incident-receiving gateway 225 is included since it is connected to and communicates directly with the element management processor 230A in-the depicted management processor system 200. In this embodiment, the other management processors (230B, 230C, and 230D) essentially function the same as the element management processor 230A except they communicate with adjacent management processors and not directly with the incident-receiving gateway 225. The element management processor 230A will be discussed with the same general principles applying as well to the other management processors.

[0033] Management processor 230A includes core processor 240, user interface. 245 and management information base (MIB) 250. The core processor 240 is connected to the gateway 225, MIB 250, user interface 245, and to the upwardly adjacent network management processor 230B (not shown). The user interface is also connected to the MIB 250. Note that with the other management processors 230B, 230C, 230D in the depicted embodiment, the core processor 240 would be connected to upper and lower adjacent management processors and not to the incident-receiving gateway 225. MIB 250 further includes a network model objects section 252, application rules section 254, and configuration objects section 256. MIB 250 also has network model objects editor 253 for editing the network model objects section 252 and a configuration objects editor 257 for editing the configuration objects section 256. Note that with the element management processor 230A, MIB 250 and user interface 245 are also directly connected to the gateway 225. This allows the gateway 225 to select and process control objects in response to receiving an incoming incident. It should also be noted that user interface 245 may actually comprise multiple user interfaces located at different sites. For example, with multiple gateways 225, a separate user interface 245 could be provided for each gateway 225, as well as for the management processor.

[0034] Core processor 240 includes a processing (or rule) engine for performing tasks including advanced processing of configuration objects (which include both control and scenario objects) that are initiated from one or more control objects selected and processed in the gateway 225. This processing is responsive to the incident's included parameters, as well as to the particular attributes associated with the selected and initiated control objects. Object processing in gateway 225 and core processor 240 carries out the predefined policies (or objectives) for managing a given function(s) as defined by a user.

[0035] User interface 245 may be any suitable device (or devices) such as a display terminal for providing users with interactive access to the management system 200 through gateway 225 and/or management processors 230A, 230B, 230C, 230D. As discussed above, the user interface may actually comprise a plurality of user interfaces depending upon the particular requirements of the managed network.

[0036] MIB 250 is an information base for storing objects and rules for managing the network tag in response to incoming incidents. In one embodiment, MIB 250 comprises Network elements section 252, application rules section 254, and configuration objects section 256.

[0037] The network model objects section 252 stores network model objects, which are objects that correspond to the managed elements of the network. (It should be noted that these managed elements can exist in any management layer and not simply the element layer.) These element objects contain attributes that reflect the state of the actual, physical element. Thus, the entirety of element objects within this section (for each of the management layers) model the network and enable the management system 200 to track and model the state of the managed network.(It should be recognized, however, that various embodiments of the present invention may not use or require complete or even partial network models.)

[0038] FIG. 6 shows one embodiment of a class tree 300 for network model object classes within the element, network, and service layers. Class tree 300 includes service layer object classes 310, network layer object classes 320, and element layer object classes 330. The network model objects model and track the state of the network, which is useful in various OSS applications such as configuration management. In addition, the network model objects maintain records for the managed elements and provide an interface model for conveying information (e.g., alarm status) through the user interface to a user.

[0039] The application rules section 254 comprises reusable (generally applicable) application logic that may be loaded and executed by gateway 225, as well as by core processor 240. These application rules provide gateway 225 and core processor(s) 240 with an operational framework for both initiating and processing configuration objects on the one hand and building, modifying, and using network model objects on the other hand. The MIBs in the various management processors 230A, 230B, 230C, and 230D have different configuration (control, scenario) and network model objects, but can use the same application rules.

[0040] The configuration objects section 256 includes the control and scenario objects for performing basic and advanced processing for managing the assigned function of the management system 200. Control objects include attributes and methods (or operators) for performing basic processing tasks. In turn, scenario objects include attributes and methods for performing advanced processing tasks. Scenario objects are invoked by control objects when such advanced processing is required and called upon by the control object.

[0041] The network model objects editor 253 edits the network model objects within the network model objects section 252. Depending upon the particular embodiment, it could include a work-station for manually augmenting or otherwise modifying this set of model objects. For example, a user could add an object for a router that is to be added to the network.

[0042] Configuration editor 257 is used for editing the configuration objects within the configuration objects section 256. Configuration objects are edited in order to change how incoming incidents are processed so as to effectuate the objectives or policies of the management system. With the use of configuration objects, these policies can be readily changed. Rather than having to modify a particular rule set or procedure for each possible incident, only high-level objects need to be changed depending upon the general applicability of the policy change. These changes affect the way in which the system responds to the incoming incident(s) associated with the configuration objects. This is but one benefit in using configuration objects for processing incidents.

[0043] In operation, an incoming incident is initially normalized at gateway 225. Once normalized, the incident causes a rule engine in the gateway 225 to search for the “closest” control object to initially process the control object. Once identified, this initial control object is processed by the gateway 225 to carry out basic processing tasks. Based on the parameters of the incident, this initial control object causes a chain of one or more control objects to be further processed in the gateway and then in the core processor 240. Scenario objects may be invoked by control objects in order for advanced processing tasks (e.g., correlation in responding to a fault incident). Essentially, the incoming incident sets off a chain of control/scenario (configuration) object processing that causes the management system 200 to react to the incident consistent with an overall management scheme. With the present invention, this “scheme” may readily be changed (or maintained) as the network and its parameters are modified. By simply creating, modifying or removing the control and scenario objects, a user is able to not only vary how the incident is processed, but also, what control object is selected as the initial control object for an incoming incident. In one embodiment, the gateway rule engine uses a combination of hierarchical and relational logic for selecting the “closest” control object. As those of skill in the art will recognize, other artificial intelligence based matching techniques may be used. The incoming incident is matched, from very specifically to very generally, against a set of possible control object name strings. For example, an incident could include information about a network element including its domain, its function, and its specific assigned identification code. The rule engine would look for a control object name with all of these parameters; however, if not found, it could then select one with two or even just one of the parameters. (Ties could be broken with predefined rules.)

[0044] With this general network management system in mind, a fault management system of the present invention will now be discussed.

[0045] C. Fault Management System

[0046] FIG. 7 shows a fault management system (FMS) 400 that implements one embodiment of management system 200. Among other things, the fault management system 400 performs alarm surveillance, alarm clearing, reporting, correlation, suppression, escalation, and thresholding at all of the management layers: business, service, network, and element.

[0047] FMS 400 generally includes gateway 425, fault management processor system 430, and display terminal interface 445 for displaying to a user fault alerts (via an Alert Display) and for receiving from the user appropriate command and configuration information. Topological representations of the network and its state may also be included. Gateway 425 is interconnected between management processor system 430 and the network elements 210 for receiving incoming fault incidents from network elements and element managers. Display terminal interface 445 is connected to both the gateway 425 and the management processor system 430. In one preferred embodiment, gateway 425 actually comprises a plurality of distributed gateways. Accordingly, the display terminal interface comprises several display terminals for servicing the fault management processor system 430.

[0048] FIG. 7 diagrammatically depicts the processing flow of an incoming alarm incident. In operation, the network element 210 sends a raw alarm incident to the gateway 425 where it is received at gateway interface 426. It is passed to a normalization rule engine at 427 where it is normalized; that is, identification and parse rules are applied in order to translate it from a vendor specific form into a form that is amenable for processing throughout the system. Based on data from this normalized incident, the gateway 425 then selects at 428 the “closest” control object 465 and processes it. In doing so, the rule engine selects the control object whose name most closely matches the normalized incident data. Once the control object is selected, the attributes in the control object determine how the alarm incident will be processed. In some cases, advanced processing in the management processing system 430 will be required. The processed control object may cause one or more other control objects 467 and/or any necessary scenario objects 469 (e.g., for performing correlation or suppression) to be processed. The management processing system 430 would then take appropriate action such as displaying alert information on the display terminal interface 445. In other cases, the basic processing in the gateway 425 may directly cause, for example, an alert message to be displayed on the Alert Display.

[0049] 1. Selecting the “Closest” Object

[0050] With the selection of the closest control object, the normalized alarm data is compared with a set of control object names to find the closest match. Values used to map the incoming alarm incident to the correct control object are extracted from the incoming alarm incident and placed into a form that is compatible with the naming scheme used for the control objects. In one embodiment, when comparing the formatted alarm against the control object list, the gateway rule engine searches from the very specific to the general. If no match is found, the default control object is used.

[0051] Control object names can be as specific as a particular type of alarm from a particular card in a particular network device, to as general as one alarm from any device. For example, fault management control could be specified at one or more of the following levels: alert name: AMO name, alert name:location, alert name:main element type, alert name:domain, alert name:parent equipment type, alert name:parent equipment id, alert name:equipment type, alert name:equipment id, alert name:any other alert attribute (such as severity). Although the current description of the invention uses the alert name as the primary key, this is not mandated, and a primary key such as equipment identifier, or type, can be used in its stead. As a simple example, consider the following list of control objects, which are named in the form and at the most particular level of alert name:AMO name. (Note that this list is ordered with increasing generality; a control object name does not necessarily have to include each component in the form.)

[0052] 1. FMgO1000:A

[0053] 2. FMgO2000:B

[0054] 3. FMgO1000

[0055] 4. FmgO2000

[0056] 5. FMgODefault

[0057] If an incident that contains an AMO name of A and an alarm name of 1000 is received, the first control object, which has both of these indicia is chosen. The same is true of a 2000:B alarm incident (the second control object would be selected). However, if an incident that contains an AMO name of A and an alarm name of 2000 is received, the fourth control object, to which has an alarm name of 2000 is chosen since there is no control object with the 2000:A combination. Similarly, if an incident that contains an alarm name of 3000 is received, the fifth (or default) control object is chosen because there is no control object that has an alarm name of 3000.

[0058] A combination of the above control object naming styles can be used, for example, alert name:domain:location:main element type could be implemented. In addition, if possible, the particular naming convention should ideally be specified in the element manager for that alarm stream.

[0059] Additional or exceptional configuration objects can be created when there is a need for the system to know the difference between control applied to one device set as opposed to another main type.

[0060] 2. Fault Management Control Objects

[0061] The control object contains attributes that dictate system response to an incoming alarm. Basic functionality such as alert generation, logging, forwarding is described by the control object itself. Initiation of advanced functions (e.g., Suppression, Escalation, Correlation, and Thresh-holding) are handled through scenario objects initiated by a control object. Finally, the alarm incident is processed and FMS 400 responds in accordance with the pre-defined policies of the system (e.g., display fault alert).

[0062] FIG. 8A shows one embodiment of a fault management control object class (or template) 500 having basic processing attribute variables 510 and advanced processing attribute variables 560. These attributes determine how an alarm incident will be processed. When creating specific control object instances, a user may select appropriate values for these attributes so that the incoming fault incidents are managed as desired by the user. Depending upon the particular advanced processing attribute values (as discussed below), scenario objects 700 may be initiated by a control object instance.

[0063] a. Basic Processing Attributes

[0064] With reference to FIG. 8B, the basic processing attribute 510 include Control Object Name 511, Control Object Group 513, Alarm Type 515, Problem Cause 517, Severity 519, Clear Type 521, E-mail List 523, Description 525, Detailed Desc. 1527, Detailed Desc. 2529, Generate Alert 531, Log Alert 533, Enable Behavior 535, Timeout 537, Manual Clear 539 Trouble Ticket 541, User Notes 543, Create Info. 545, and Update Info. 547.

[0065] The Control Object attribute 511 defines the name of a given control object instance. Control Object Group 513 defines the group to which the given control object instance belongs. The gateway 425 parse and identification rule engine has its own “element managers” that use the control objects belonging to the-groups they are assigned to find the correct behavior specification for the incoming alarm incidents. The Alarm Type attribute 515 contains the X.733/M.3100 alarm category for the alarm behavior being defined. It is related to the next field (Probable Cause 517). This value is forwarded across the normalization rule engine if forwarding is selected. The Probable Cause attribute 517 describes the likely cause-of this type of alarm based on the X.733/M.3100 probable cause related to the alarm type 515. It is related to the previous field (Alarm Type 515). This value is also forwarded across the normalization rule engine within gateway 425.

[0066] The Severity attribute 519 defines the severity level associated with a given control Object instance. Its possible values include critical, major, minor, warning, and indeterminate. It is used to overwrite the alarm severity value received from the Element Manager (or device); it invokes a user to select a severity from a pick list. EM means apply the severity received from the Element Manager. A value of AD indicates the severity defined in the Alert Definition should be used.

[0067] The Clear Type attribute 521 is used to control alert clearance behavior. Its possible values include ACK, MAN, RESET, and FULL with its default being RESET. ACK causes the alert to clear from the system on receipt of a clear indication from the equipment only if the alert was manually acknowledged. If unacknowledged, the alert severity will be set to indeterminate when the clear is received. MAN causes any clear messages received from the Element Manager or device to be ignored. These alerts should always be manually cleared. RESET causes the alert severity to be set to “indeterminate” when a clear for the alert is received. The indeterminate event must then be manually selected and cleared to remove it from the Alert Display. FULL causes the alert to fully clear from the Alert Display.

[0068] The Email List attribute 523 identifies the email addresses of the people to be automatically notified when the alert using the control object is generated. The default is blank; that is, nobody would be notified via email. This attribute may also reference a contact object. This contact object may list several possible ways of contacting the user.

[0069] The Description attribute 525 is used to overwrite the alarm description received from the Element Manager or device. If left empty, the EM description is used.

[0070] The Detailed Desc. 1 attribute 527 is used to overwrite the alarm description received from the Notification ID (NM). If left empty, the NID description is used. The NID is a unique tag used to distinguish this alert. This number is logged to the database if logging is turned on and can be used for reference.

[0071] The Detailed Desc. 2 attribute 529 is used to provide additional or supplementary alarm information. The rules may be written to fill this with information from the incoming alarm message if this attribute is left unassigned.

[0072] The Generate Alert attribute 531 is a boolean variable with a default of True. If its value is True, it causes the system to generate an alert when that control object is triggered. If False, an alert is not generated. This allows the system to log the alert or to perform advanced fault management functions on the alert without generating it.

[0073] The Log Alert attribute 533 is used to control alert logging behavior. Its possible values are 0, 1, and 2 with a default value of 2.0 means do not log the alert details when this control object triggers system response. 1 means log the alert details only if the alert is generated on the Alert Display. 2 means always log the alert details.

[0074] The Enable Behavior attribute 535 is a boolean variable with a default of True. If False, the system includes this control object when searching for a match against the incoming alarm incident. If False, the control object will not be part of the list of possible “closest” control objects. This allows a user to temporarily deactivate control objects, or to activate control objects for special occasions or events. If an unspecified control object (alert name only) is disabled, system default behavior is applied.

[0075] The Timeout attribute 537 defines a number of minutes. It is used as a timeout to set the number of minutes for the system to wait before clearing the alert. This is useful with alarms such as an open door to a piece of equipment that is not cleared by the equipment once the fault condition is resolved.

[0076] The Manual Clear attribute 539 is of boolean type with a default of True. If True, the alarm incident is permitted to be cleared manually. This is used to control manual clearance behavior. If this box is checked, manual clearance is permitted. Note that for correlation alerts, manual clear should be True only if the alert needs to be manually cleared in exceptional circumstances. If an alert is selected on the Alert Display to be manually cleared, and the value is False, the alert will remain on the alert display.

[0077] The Trouble Ticket attribute 541 is also of boolean type with a default of False. If True, the system automatically generates a trouble ticket when that control object is triggered.

[0078] The User Notes Attribute 543 includes documentation information about the control object. It is blank if nothing is entered by a user.

[0079] The Create Info. attribute 545 shows the time/date/user ID information for when the control object was created. Its values are automatically generated. Accordingly, value assignment from a user is not required.

[0080] The Updated attribute 547 is also automatically defined. It shows the time/date/user ID information for when the control object was last updated.

[0081] b. Advanced Processing Attributes

[0082] With reference to FIG. 8C, advanced processing attribute variables 560 include Alternate Event 561, Special Pre-Process 563, Special Post-Process 565, Correlation Scenario 570, Threshold Scenario 590, Escalation Scenario 610, and Suppression Scenario 630.

[0083] The Alternate Event attribute 561 includes the name of an alternate event if defined by a user; otherwise, this attribute remains empty. The use of an Alternate Event name means that, at run time, when an alarm incident matches the control object, the system will bypass (skip) default system functionality and instead uses the Alternative event instead. This can be configured for informational alarm incidents where the system can use the information carried within the alarm and update some system attributes. The Heartbeat message is an example for such alarms.

[0084] The Special Pre-Process attribute 563 either includes the name of an event or remains empty.

[0085] This entered event runs before system default functionality is triggered. It can be used to pre-set attributes that the system uses during run time. As an example this event can be used to read a text file or run a database command and pass the result to the system through system standard attributes.

[0086] The Special Post-Process attribute 565 also includes the name of an event or remains empty.

[0087] If entered, the event runs after the system processes the alarm incident. It can be used to write results to an external database or text file, or to fax alarm details. Writing system attributes to a text file for debug purposes is a good example on where this event can be used.

[0088] The following Correlation 570, Threshold 590, Escalation 610, and Suppression 630 Scenario attributes all use a pick list from which a user can select an existing scenario, or a configuration object editor that allows a user to create a new “custom” scenario.

[0089] The Correlation Scenario attribute 570 identifies a particular correlation scenario (if any) to be invoked when the control object is initiated. If desired, a user assigns a correlation scenario in this attribute. Such a scenario may be selected from a pre-existing group or created by the user.

[0090] When an incoming alarm incident completes a predefined alarm pattern, a new “correlation” alert is generated on an Alert Display Window at the display terminal interface 445. All alerts comprising the alert pattern are then temporarily removed from the Alert Display Window. While the correlation alert is active, subsequent alarms which match the correlation pattern of the correlation alert are not displayed.

[0091] Correlation alerts should not be manually cleared. When an “alarm clear” for one of the alerts comprising the underlying correlation alert pattern is received from the EM or device, the “correlation” alert is automatically cleared from the Alert Display Window, if the Auto Clear attribute is set to True. The remaining alerts that were temporarily removed from the Alert Display, or which were suppressed before being generated as a result of the correlation alert being present, are regenerated. If the Auto Clear attribute is set to False, the underlying alerts will not be regenerated when the correlation alert pattern is no longer active.

[0092] There are two standard correlation methods: pattern correlation (Oust previously described), and Circuit correlation. Circuit Correlation works for links and circuits, where, if one or both ends of the link are in alarm, this indicates that the link is down.

[0093] The main difference between circuit correlation and pattern correlation is that circuit correlation searches a pre-populated table to identify the circuit components (a-end, b-end and circuit name). An alert name for the other end can be specified. If no alert name is provided, the same alert name as the incoming alert will be used. The absent alert conditions cannot be applied to the correlation scenario, and only a single network component (the circuit) can be alarmed.

[0094] When an alarm is received on one end, and if AND is selected for the present conditions, the other end is checked for the same alarm, or the alarm specified in the correlation scenario. (The other end is identified through a lookup table entry). If AND was specified and the conditions of the circuit correlation alarm are met, both alerts are hidden and a new alert is generated to indicate that the link (or circuit) is in alarm. If OR was specified, then the correlation alert will be generated whether or not the other end alert has been received. In this case, if the other end alert is received after the correlation alert has triggered, this alert will not be displayed, but will be regenerated when the correlation condition is no longer active (and Auto Clear is True), and if this alert has not been cleared in the meantime.

[0095] With reference to FIG. 8D, the correlation scenario attribute includes a plurality of sub-attributes 571-583. These sub-attributes include Correlation Scenario 571, Correlation Event 572, Auto Clear Correlation 753, Correlation Time 574, Correlation Mos 575, Correlation Alert 576, Present MO List 577, Present Alert List 578, Absent MO List 579, Absent Alert List 580, User Notes 581, Create Info. 582, and Update Info 583.

[0096] The Correlation Scenario sub-attribute 571 identifies the name of the correlation scenario related to the control object includes the attribute. The Correlation Event attribute 572 identifies an event name for the correlation scenario.

[0097] The Correlation Time 574 defines the time (in seconds) for the system to wait before generating the correlation alert. If the pattern is no longer active after this period of time, correlation should not be initiated.

[0098] The Correlation Mos attribute 575 indicates a network component name. This attribute is used for identifying the network components against which the correlation alert must be generated. A symbol can be used to indicate that the location attribute value of the incoming alert should be substituted in the network component name. Note that any combination of symbols that represent attribute values can be used. A unix-like regular expression could also be used to specify which network components are to generate the correlation alert against.

[0099] The Correlation Alert attribute 576 identifies the name of the correlation alert to be generated when the correlation condition is satisfied.

[0100] The Present MO List attribute 577 is used to identify on which network components the alerts (fault conditions) must be present for the correlation to take effect. When naming the network component, a symbol can be used to indicate that the location attribute value of the incoming alert should be substituted in the network component name. Note that any combination of symbols that represent attribute values can be used. A unix-like regular expression may also be used to specify which network components should be present.

[0101] The Present Alert List attribute 578 is used to identify the alerts (fault conditions) that must be present for the correlation to take effect. Each present alert list entry should correspond to the network component name in the Present MO List.

[0102] The Absent MO List attribute 579 is used to identify which network components the alerts (fault conditions) must be absent from for the correlation to take effect. This list is related to the following field (Absent Alert List). When naming the network component, a symbol can be used to indicate that the location attribute value of the incoming alert should be substituted in the network component name. Note that any combination of symbols that represent attribute values can be used. A unix-like regular expression could also be used to specify which network components should be absent.

[0103] The Absent Alert List attribute 580 is used to identify the alerts (fault conditions) that are to be absent for the correlation to take effect. This list is related to the previous attributes (Absent MO List). Each absent alert list entry corresponds to the network component name in the Absent MO List.

[0104] The User Notes attribute 581 is used to store and document information about the correlation scenario.

[0105] The Create Info attribute 582 tracks and stores the time/date/user ID information for when this correlation scenario was created.

[0106] Likewise, the Update Info attribute 583 tracks and stores the time/date/user ID information for when this correlation scenario was last updated.

[0107] The Threshold Scenario attribute 590 identifies a particular threshold scenario (if any) to be invoked when the control object is initiated. If desired, a user assigns to the control object a threshold scenario in this attribute. Such a scenario may be selected from a pre-existing group or created by the user. If no threshold scenario is assigned, then a threshold scenario will not be initiated from the control object when it is processed.

[0108] Thresholding causes the generation of an alert to be delayed until specified conditions are met. These conditions can include the system receiving a specified number of alarms or alarm clears within a specified period of time.

[0109] Setting Threshold parameters allows alarm incidents that occur often to generate alerts on the Alert Display only when a certain number of alarms are received within a given time interval. In other words, when they reach a preset threshold. This is known as rate thresholding. Thresholding can also count the number of alert clears that are received in a given time before an alert is generated, this is fleeting Thresholding.

[0110] With thresholding, the number of occurrences of an alarm or alarm clear within a sliding time interval are recorded as they are received. If the required number of alarms are received, optionally for a specified severity level, within the required time interval, an alert is generated with the specified severity.

[0111] Different grades of thresholds for a given time interval can be set up. For example, 5 alarms in 10 minutes generates a minor alarm, 10 alarms in 10 minutes generates a major alarm, and 20 alarms in 10 minutes generates a critical alarm.

[0112] If additional alarms are received such that the threshold requirements for a higher severity level are satisfied, the severity of the alert is changed to reflect the new specified value. Clears will similarly decrement the count and severity level as a previous threshold is reached.

[0113] When a new thresholded alarm is received after the specified time frame, it sets the severity of the currently thresholded alert to indeterminate and a new threshold count starts.

[0114] There are two standard thresholding methods: rate thresholding and fleeting thresholding. Rate thresholding is handled by suppressing the number of alarms that are received until the threshold is reached for the alarm that is being thresholded, and then generating the threshold alert. This type of threshold can escalate or de-escalate because each clear received subtracts from the count of alarms. Different high-water marks can cause different severities to be applied to the alarm. Alarm clears are always recognized in threshold processing whether the Clear Type is set to manual or not. If an alert already exists on the Alert Display for the previous threshold—according to the internal threshold count—the severity level is adjusted as required, or if the threshold requirement is no longer satisfied for any severity level, the alert is removed from the Alert Display. Three ways for clearing a rate threshold include Clearing the threshold rate manually, automatically clearing when enough clears are received to bring the threshold level down to the lowest preset level, and automatically clearing when there are not enough alarms received during a preset interval of time and a new alarm of the specified type is received.

[0115] This latter method is handled by suppressing the alarms received until the required number of clears for the alarm that is being thresholded are received (that is, the alarm has come and gone a number of times within a specified period of time). Then the threshold alert is generated.

[0116] If Clear Type is set to manual, this does not prevent the clears from counting up. (Fleeting alarms rarely de-escalate.) So there is no logical inconsistency between setting Clear Type=MAN and configuring a control object for fleeting alarms.

[0117] Different high-water marks cause different severities to be applied to the alarm. This function counts the number of clears received. If a new (different) alert is generated as a result of a fleeting alarm condition, a manual clear is required because the Element Manager never sends a clear for this alarm.

[0118] Two ways to clear a fleeting threshold include clearing the threshold rate manually and clearing it when there are not enough alarms received during a preset interval of time and a new alarm of the specified type is received.

[0119] As shown in FIG. 8D, the Threshold Scenario attribute 590 includes a plurality of sub-attributes 591-600. These sub-attributes include Thresholding Scenario 591, Thresholding Event 592, Threshold Severity 593, MO Name 594, Threshold Alert 595, Count 596, Interval 597, User Notes 598, Create Info. 595, and Update Info. 600. Each of these sub-attributes (or attributes) will now be discussed.

[0120] The Thresholding Scenario attribute 591 identifies the name of the thresholding scenario (if any) for the control object.

[0121] The Threshold Event attribute 592 includes an event name for identifying the type of thresholding event.

[0122] The Thresholded Severity attribute 593 identifies a severity level (critical, major, minor, warning, indeterminate, or blank with a default value of blank. This attribute is used to specify a severity for the alert to trigger thresholding. If blank is specified, alerts of all severities will be thresholded. If a severity is specified, then only alerts with this severity will be thresholded.

[0123] The MO Name attribute 594 is used to identify the name of a different network component on which an alert is to be generated when the thresholding condition is satisfied. If no network component name is provided, thresholding generates on the incoming network component. When naming the network component, a symbol can be used to indicate that the location attribute value of the incoming alert should be substituted in the network component name. Note that any combination of attribute values can be used. A unix-like regular expression could also be used to specify which network components the alerts should be generated on.

[0124] The Threshold Alert attribute 595 identifies the name of a different alert to be generated when the thresholding condition is satisfied. This alert name should already be defined (e.g., by a user). If no alert name is provided, the threshold alert is generated on the control object definition alert name, that is, the incoming alarm name.

[0125] The Count attribute 596 has count values for each severity “box” (clear, warning, minor, major, and critical) with a default of blank in each box. The scenario uses the value in each severity box to set the number of alarms of that severity you want the system to wait before the threshold alert changes to the matching severity.

[0126] The Interval attribute 597 identifies the number of days, minutes, and hours, respectively, to indicate the time within which alarms must be received for a threshold alert to be generated. This attribute does not indicate a pause between when the threshold condition is met and when the Threshold alarm is to be generated. As such there is only one Wait Time value required and not a list.

[0127] The User Notes attribute 598 stores information about the thresholding scenario.

[0128] The Create Info 599 tracks and stores the time/date/user ID information for when this thresholding scenario was created.

[0129] The Update Info attribute 600 tracks and stores the time/date/user ID information for when this thresholding scenario was last updated.

[0130] The Escalation Scenario attribute 610 identifies a current escalation scenario (if any) to be initiated by the control object.

[0131] Escalation causes the alert severity or description of the alert to be changed, a new alert to be generated, or an e-mail to be sent if the alert is not acknowledged (or cleared) within a specified period of time. If the alert is acknowledged or a clear message is received during the wait time, escalation behavior is canceled.

[0132] The Escalation Scenario attribute 610 has a plurality of attributes (or sub-attributes) 611-623. These attributes include Escalation scenario 611, Escalation Event 612, Escalation Time 613, Email List 614, Escalated Severity 615, Escalated Description 616, Generated Mos 617, Generated Alerts 618, Generated Severities 619, Generated Description 620, User Notes 621, Create Info. 622, and Update Info. 623.

[0133] The Escalation Scenario attribute 611 identifies the name of the Escalation scenario (if any) for the given control object.

[0134] The Escalation Event attribute 612 identifies a method for escalating alarms.

[0135] The Escalation Time attribute 613 specifies the time in seconds within which alarms must be received for the escalation to be initiated.

[0136] The Email List attribute 614 stores an email addresses of the people to be automatically notified when the escalation criteria is met. In the future, a contact object may be referenced in its stead.

[0137] The Escalated Severity attribute 615 has possible severity values including blank, critical, major, minor, warning, indeterminate, and AD with a default of blank. This attribute is used to set the severity to apply to the alert when it's escalated. AD indicates the severity defined in the Alert Definition that is to be used. blank indicates the severity should not change.

[0138] The Escalated Description attribute 616 describes the new alert that is generated by the escalation function. Blank indicates that the description should not change.

[0139] The Generated MOs attribute 617 identifies a list of network components that should be put into the alarm state if the escalation condition remains valid after the wait period (if configured). When naming the network components, a symbol can be used to indicate that the location attribute value of the incoming alert should be substituted in the network component name. Note that any combination of symbols that represent attribute values can be used. One could also use a unix-like regular expression to specify which network components the alert(s) should be generated on.

[0140] The Generated Alerts 618 attribute stores the names of the new alerts generated on the network components identified above when the escalation condition is satisfied.

[0141] The Generated Severities attribute 619 has possible values that include critical, major, minor, warning, indeterminate, and AD with a default of blank. This attribute is used to view or edit the list of severities that should be applied to the alerts identified above if the escalation condition remains valid after the wait period. AD indicates the severity defined in the Alert Definition that is to be used. If this field is left blank, the severity of the incoming alarm is used.

[0142] The Generated Descriptions 620 is used to store the descriptions generated on the alert identified above when the escalation condition is satisfied. If this field is left blank, the description in the incoming alarm will be used.

[0143] The User Notes attribute 621 stores information about the escalation scenario.

[0144] The Create Info attribute 622 tracks and stores the time/date/user ID information for when this escalation scenario was created.

[0145] The Update Info attribute 623 tracks and stores the time/date/user ID information for when this escalation scenario was last updated.

[0146] The Suppression Scenario attribute 630 is used to identify (or assign) a current suppression scenario (if any) for the control object.

[0147] Suppression causes specified alerts to be permanently removed from the Alert Display window when a suppressor alarm is received.

[0148] While the suppressor alert is active, subsequent alarms that match the suppression pattern do not display on the Alert Display Window.

[0149] There are six typically-used suppression methods: Suppress Transient, Suppress Pattern, Suppress Containment, Suppress Branch, Suppress Equip Range, and Suppress And Gen.

[0150] The Suppress Transient method is used for alarms that come and go in a short period of time. The system generates an alert only if a clear is not received within a specified period of time. Otherwise, the system ignores the alarm.

[0151] The suppression of patterns method is used for alarms that have a suppressor suppressee relationship. The relationship is defined using lists of alarm names and network components (managed objects) identified in the suppressor control object. The suppressee control objects identify the suppressor.

[0152] The Suppress Containment method is based on containment. When this method is triggered by an incoming alarm, the system generates the received alert, and suppresses specified alerts on contained equipment.

[0153] The Suppress Branch method is based on containment. When this method is triggered by an incoming alarm, the system suppresses all alerts on the parent network component, and all contained network components, with the exception of the network component that triggered the method. This method can be used for power supply failure scenarios.

[0154] The Suppress Equip Range method is also equipment based and suppresses specified alerts names when they are received on a specified range of equipment.

[0155] The Suppress And Gen method is used when generation of a different alert for an alarm is desired. When this scenario is activated, the triggering alert is not generated, but the specified alert is generated in its place. This method can be used to clear alerts, when the offnormal condition and the return-to-normal condition messages use different alarm name identifiers.

[0156] Another type of suppression scheme is Do not generate alert. This is handled through an Alert Generation form. For example, the system could be configured not to generate 2567 alerts for CCSUs, or not to generate 2567 alerts for a specific device and its components (like an MSC).

[0157] The suppression attribute 630 includes a plurality of sub-attributes 631-645. These sub-attributes include Suppression Scenario 631, Suppression Event 632, Suppression Severity 633, Suppression Threshold 634, Suppressor Equipment Range 635, Suppressor MOs 636, Suppressor Alerts 637, Suppressed Equipment Range 638, Suppressed MOs 639, Suppressed Alarms 640, Suppressed Severities 641, Wait Time 642, User Notes 643, Create Info. 644, Update Info. 645.

[0158] The Suppression Scenario attribute 631 identifies the name of the Suppression scenario for the control object.

[0159] The Suppression Event attribute 632 identifies the method to use for suppressing alarms (or to generate a new alert).

[0160] The Suppression Severity attribute 633 specifies the severity level (e.g., critical, major, minor, warning, indeterminate) of an alert (generated from an alarm incident) that will activate this scenario. If the severity of the alert does not match the selected severity, no suppression will occur. Blank means any severity will activate the scenario.

[0161] The Suppression Threshold attribute 634 specifies the number of objects or the 25 percentage of objects which must be in alarm for the scenario to be activated. Count applies to range and pattern suppression methods. Percentage applies to the containment and branch suppression methods. This feature will change over time.

[0162] The Suppressor Equipment Range attribute 635 is used for the Suppress Equip Range method. It is used to identify the ID ranges of the equipment or network components to act as a suppressor. The range can be for specific IDs (e.g. [1,4,19,90]) or can be a regular expression ([0-9] or [100-200]).

[0163] The Suppressor MOs attribute 636 identifies the name of the network component(s) on which the alert must be active for related suppression to be initiated. When naming the network component, a symbol can be used to indicate that the location attribute value of the incoming alert should be substituted in the network component name. Note that any combination of symbols that represent attribute values can be used. A unix-like regular to expression could also be used to specify which network components the alert(s) should be generated on.

[0164] The Suppressor Alerts attribute 637 identifies the name of the alert(s) that must be active for related suppression to be initiated.

[0165] The Suppressed Equipment Range attribute 638 is used for the Suppress Equip Range method. It stores the ID ranges of the equipment to be suppressed. The range can be for specific IDs (1,4,19,90) or can be a regular expression (0-9) or (100-200).

[0166] The Suppressed MOs attribute 639 stores the network components(s) on which the alarms identified in the Suppressed Alarm List attribute must be suppressed if this alarm is received. A symbol can be used to indicate that the location attribute value of the incoming alert should be substituted in the network component name. Note that any combination of symbols that represent attribute values can be used. A unix-like regular expression could also be used to specify which network components the alert(s) should be generated on.

[0167] The Suppressed Alarms attribute 640 identifies the alarm(s) that must be suppressed if this alarm is received, and if these alarms are on the network components identified in the Suppressed MO List attribute. It is used for the Related suppression method, else it should be blank. There is an alarm name for each corresponding device (and severity) specified.

[0168] The Suppressed Severities attribute 641 identifies the severity level (critical, major, minor, warning, or indeterminate) of the alert to be suppressed. blank means suppress alerts of all severity. If a severity is specified, alerts with the specified severity are suppressed.

[0169] The Wait Time attribute 642 specifies the length of time for which the alarm must be active before the alert is generated. This attribute is ONLY used in conjunction with the Transient suppression method, and should otherwise be set to 0 (zero).

[0170] The User Notes attribute 643identifies information about the suppression scenario.

[0171] The Create Info attribute 644 tracks and stores the time/date/user ID information for when this suppression scenario was created.

[0172] The Update Info attribute 645 tracks and stores the time/date/user ID information for when this suppression scenario was last updated.

[0173] New advanced fault management methods will be identified and added to the invention over time. What is particularly relevant is that the invention allows customers to easily add create methods, and to apply these methods through the user interface associated with the invention. New methods are easy to create because they work within the structured framework or context of the invention. This greatly reduces the time required to test and verify changes to management procedures.

[0174] Although the present invention and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims. For example, a management processor system could include one or more management processors. Again, the management processors of the present invention represent processing functions that could actually be implemented in a variety of manners. Thus, a fault management system could include only one management processor system for implementing fault management on one or more management levels. Moreover, management processor systems may be implemented on any suitable hardware system that may include one or more servers or work stations.

[0175] Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, composition of matter, means, methods and steps described in the specification. As one of ordinary skill in the art will readily appreciate from the disclosure of the present invention, processes, machines, manufacture, compositions of matter, means, methods, or steps, presently existing or later to be developed that perform substantially the same function or achieve substantially the same result as the corresponding embodiments described herein may be utilized according to the present invention. Accordingly, the appended claims are intended to include within their scope such processes, machines, manufacture, compositions of matter, means, methods, or steps.