Method and system for memory management in a network processing system

[0001] 1. Technical Field

[0002] The present invention relates in general to computer networks, and in particular, to a network processor for managing network traffic. Specifically, the invention relates to memory management between a control point processor and network processors.

[0003] 2. Description of the Related Art

[0004] Computer networks have evolved to a point where switch routers are insufficient to handle complex network routing and filtering requirements. Network processors, such as the International Business Machines (IBM) Network Processor (NP) have been developed to fill this need. Utilizing a connection similar to that of a network switch, the NP provides a level of control and flexibility that surpasses that of the network switch. Parallel lookup operations can be performed on a frame, complex modifications can be performed on packets, and policies can be managed at complex levels. Multiple processors are used to achieve a very high frame processing throughput.

[0005] It is not desirable to have a high overhead operating system running on a network processor. Due to the requirements of high frame processing throughput, the NP support tasks need to be reduced to a minimum. However, there is a need for memory allocation for data structures and control instructions that control the network processor. In addition, the NP may execute local applications, for example complex filtering applications performing frame filtering and blocking. These NP-based applications require a memory allocation scheme that does not require constant communication with a central control point. In addition, it is disadvantageous to require direct mapping of address space between central control point applications or operating systems running on a central processor and data structures or programs running on NPs.

[0006] Therefore, it would be desirable to provide a method and system for managing memory in a network processing system so that access to the memory can be achieved without requiring high processing overhead and direct mapping of address spaces between a control point processor and one or more network processors.

[0007] The objective of managing memory in a network processing system without requiring direct memory mapping is achieved in a method and system for managing memory in a network processing system, which includes a control point processor and multiple network processors (NPs). A dynamic memory manager, having a control point processor (CP) component and a network processor component, is provided for registering applications for allowing those applications to request allocation of memory on the NPs by transmitting a request from the CP component to the NP component over a bus. In response to a request, the NP allocates physical memory and returns a token representing the allocation from the NP component to the CP component. The token is used to provide read/write and other access to the physical memory.

[0008] All objects, features, and advantages of the present invention will become apparent in the following detailed written description.

[0016] With reference to the figures, and in particular to FIG. 1, a network processing system in accordance with an embodiment of the present invention is depicted. A packet routing switch 12 is coupled to a group of network processors (NPs) 14A–14C. NPs 14A–14C provide media interfaces for connecting to networks having various protocols, such as 10 base-T or 100 base-T ethernet connections 16A–16E or other types of communication interfaces that are known in the art. Thus, NP 14A may route frames received at any of the media interfaces to port 0 on packet routing switch 12, and may perform address translations, respond to Quality-of-Service (QoS) requests, block packets, or perform any complex routing or frame processing function on the incoming frames. NP 14A also receives frames from port 0 on packet routing switch 12 via an Egress interface and may perform similar complex operations to route frames to the media interfaces. Control memory 15 provides storage for control software that drives the routing functionality and frame processing, and, specifically, the software managed tree (SMT) dynamic classifier of the present invention. Lookup ZBT SRAM 17 provides fast access memory for storing fixed match tree (FMT) hardware assisted matching trees and can be used to provide fast storage for SMTs or portions thereof. Control point central processing unit (CPU) 11 provides management of network processor operations and downloading of applications, classifier trees and other data. CPU 11 compiles and creates the SMTs for downloading. The SMTs are built by applications running on CPU 11, then downloaded to memory coupled to NPs 14A–14C.

[0017] Referring now to FIG. 2, a simplified block diagram of NP 14A is shown. Ingress Enqueue/Dequeue/Scheduling logic (EDS) 27 manages frame buffering and control for frames routed to switch fabric 22. Frames are received from the media interface connections by ingress physical MAC (Medium Access Control) multiplexer (PMM) 28, and are translated and processed by protocol processors 25. An ingress switch interface 23, provides a connection to switch fabric 22 and may connect to another NP or a packet routing switch 12. An egress switch interface 24 receives frame data from switch fabric 22 and the frame data is stored in egress data storage 31. Protocol processors 25 can then perform classifier searches to process the frames and route them through an egress PMM 29.

[0018] Referring now to FIG. 3, a detailed block diagram of NP 14A in accordance with a preferred embodiment of the invention is depicted. Memories 43 for storing frame data, classifier trees and applications data and instructions are coupled to the NP by control store arbiter 42, allowing core processing units 46 to share access to external and internal memory. Each core processing unit 46 contains dyadic protocol processing units (DPPU) 44A–44B and tree search engine 45. An instruction memory 47 is coupled to the core processing unit 46 for storing the picocode that drives the tree search engines 45 (TSEs). Power PC® processor core 41, provides management of the network processor unit 40. Software managed tree data and fixed match tree data may be downloaded into memories 43 to provide control for TSEs 45. Once a leaf in the tree is matched, it is loaded into internal registers in the network processor unit 40. The trees used with TSEs 45 are referred to as static or dynamic trees. Dynamic trees are so called because they may be updated incrementally and quickly to produce changes in the processing of frames. Static trees are not incrementally upgraded and require a reload of the tree each time the tree is modified. Static trees are useful for providing more complex matching such as applying a leaf to a range of IP addresses. Hardware classifier 48 pre-processes received frames to prepare for matching.

[0019] The present invention embodies a method and system for managing memory on NPs 14A–14C so that the applications on CPU 11 and applications running on NPs 14A–14C may allocate memory for use in storing control instructions, data, and frame classifier trees. This memory may be data storage for NP applications, data storage for CP applications, SMT storage for frame classifiers, etc.

[0020] Referring now to FIG. 4, the software modules that implement the method and system of the present invention are depicted. A dynamic memory manager includes two portions, namely, a control point memory manager (CPMM) 53 and an NP memory manager 64. The CP component of the dynamic memory manager resides within memory coupled to CPU 12, and the NP component resides within memory coupled to NP 14A. Within CPU 12, an application software 52 provides control of network processing performed by network processors 14A–14C. In order to allocate memory for data storage on NPs 14A–14C, application software 52 communicates with application programming interfaces (APIs) in CPMM 53. These APIs provide allocation and deallocation services for memory on NPs 14A–14C as well as read/write access services and request for memory status information such as the remaining free space. The APIs may also provide selection of fast or slow memory allocation and transfer operations from one memory to another within NPs 14A–14C. Unlike ordinary procedure calls and ordinary memory allocation schemes within operating systems such as AIX (manufactured by IBM Corporation of Armonk, N.Y.) or UNIX, the APIs in the dynamic memory manager are not synchronous.

[0021] In order to provide allocation services without delaying application execution or interrupting NP 14A–14C processing, the response to allocation and access requests is asynchronous, reporting results after the transaction has propagated to NPs 14A–14C. The dynamic memory manager also supports multicast memory allocation and access, in that multiple NPs may be specified in each request, for example a multicast API allows CPU 12 to allocate space and write data on a particular subset of NPs 14A–14C in a single set of operations: a single allocate call and a single write call. The dynamic memory manager notifies the CP application 52 via CPMM 53 when all of the responses have been received or when an error (including timeout due to no response from one or more NPs) has occurred.

[0022] In order to accomplish asynchronous operation and organize data that is owned by applications, a Register( ) and De-Register( ) API are provided by the dynamic memory manager. Applications register themselves with the DMM and a table is built to record the allocations requested by the application. When the application calls De-Register( ), the DMM waits for pending requests to timeout, then deletes all of the tokens held by the application.

[0023] The operations supported by the dynamic memory manager CP component (the part of the dynamic memory manger resident within CPU 12) include the Register( ) and De-Register( ) APIs, along with Read( ), Write( ), Resize( ), Allocate( ) and Deallocate( ) APIs. Additional inquiries are available to return the amount of free memory and information on availability of memories with different access speeds. For example, a CP application preparing to download a frame processing algorithm that will be used frequently may want to determine if enough free bytes are available in fast on-chip memory to hold the code implementing the algorithm. A CheckMemoryAvailability( ) API is provided this purpose.

[0024] Packets representing memory operations are propagated to a bus connecting CPU 12 with NPs 14A–14C. A device driver 54 provides packaging of the information required for the request, i.e. token and data for a write request, into stimuli for a bus interface 55. The bus that connects the NPs and the CPU 12 may be an Ethernet connection, an Fibre Channel connection or some other interface that allows bidirectional communications.

[0025] Within NP 14A, the bus packets are received by a bus interface 62 and decoded by guided cell handler 63, which then communicates the requests encapsulated within the packets to the NP component of the dynamic memory manager 64. The dynamic memory manager then may allocate or access memory 43 coupled to the NP. An SMT download facility 65 provides allocation for SMTs that are downloaded to NP 14A. Table initialization control 66 sets up the framework for the NP component 64 of the dynamic memory manager. The tables created track allocations and tokens that have been passed to applications and provide data for converting received token information to pointers so that physical locations within the NP can be accessed. Applications 68 executing within NP may also allocate and access memory using these mechanisms so that a common interface is provided between applications running on the CPU 12 and applications running on the NPs.

[0026] By using a single token to refer to a particular datum created by an application, multiple applications or multiple instances running on the various processors can refer to the same datum, even though it is located within different memories at unique addresses. For example a request to delete a particular record that was allocated on multiple processors could be broadcast to all processors. The deallocation would proceed on those processors for which the record exists and fail on those on which it did not exist. The results of a memory operation are returned to the application by a callback mechanism that is provided during application registration. The callback is made by the dynamic memory manager supplying the token and result for each of the targeted NPs. The callback routine in the application can then determine the results of the operations that were asynchronously queued earlier. The use of the token provides a mechanism that allows the application to identity which requests are completed and which are still pending.

[0027] The dynamic memory manager may also block requests depending on the status of pending requests associated with a given token. For example, the write operation should be blocked if an allocate request is pending, since the physical location has not been assigned and there is no target yet for the write data.

[0028] Referring to FIG. 5, a flowchart of a method for managing memory in a network processing system in accordance with an embodiment of the invention is depicted. An application registers with the dynamic memory manager (step 70) and the dynamic memory manager builds a table of references for the application (step 71). The application can then request allocation of memory (step 72) and that request is propagated to one or more network processors on the bus (step 73). Physical memory is allocated within the NP subsystem and a token is returned to refer to the allocation (step 74). The responses to the allocation request are returned from the NPs and collected by the dynamic memory manager (step 75). The application is then notified of the results of the allocation (step 76).

[0029] Referring now to FIG. 6, a flowchart of a method of downloading an SMT in accordance with a preferred embodiment of the invention is depicted. The SMT download facility 65 provides storage allocation for the SMTs that are downloaded and the multiple storage allocations used to maintain frame classifier operation while the new SMT is downloaded. A new SMT is built on CPU 11 (step 80) and the classifier timer is stopped if it is already running (step 81). This prevents applications running on CPU 11 or NPs 14A–14C from making any additions or changes to the SMT while the download of a new SMT proceeds. The new SMT is checked to determine if rule changes have occurred which require download of the new SMT (step 82) and if not, the process ends. If the SMT needs to be downloaded, it is generally first downloaded to slow storage (step 83), since fast storage is a limited resource and there may not be enough fast storage (either internal memory within NP 40 or external memory 43 such as ZBT SRAM 17).

[0030] The pointer that specifies the location of the active SMT for the NP frame processor TSE 45 is changed to point at the newly downloaded SMT (step 84). Then the old SMT is purged from fast memory (step 85). Another copy of the new SMT is downloaded to fast memory (step 86). Finally, the active classifier tree pointer is changed to point at the new SMT copy in fast memory (step 87), the SMT is purged from slow storage (step 88), and the classifier timer is restarted (step 89). This accomplishes an update of the SMT without halting operation of the TSE 45 while maximizing the size of available fast memory. If the SMT was not temporarily used from slow memory, then the fast memory would have to be larger to accommodate the old SMT and the new SMT, or the operation of TSE 45 would be interrupted while the download is being performed.

[0031] It is not necessary to perform the download twice to perform the method of the present invention. For instance, SMT download may occur in a network processing system where all memory is accessible at the same speed.

[0032] In this configuration, a single download is performed and the active pointer is switched to the new SMT location, since there would be no added benefit from recovering the location of the old SMT.

[0033] While the preferred embodiment of the invention has is illustrated with a configuration having a single CPU and multiple NPs, the present invention may be implemented in a system having multiple CPUs and a single NP, or other variants providing control of network routing and processing. It will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention.