NETWORK COMMUNICATION NODE COMPRISING A PLURALITY OF PROCESSORS FOR PROCESSING LAYERS OF COMMUNICATION AND ASSOCIATED NODE

[0001] This invention relates generally to data communications and more particularly to a method and apparatus for processing and rerouting the data packets in a data communication network

[0002] Data communication networks are commonly used to transport data between end users. A source device sends a group of data packets or a stream of data to a recipient device through a connection established in the data communication network. Nodes of the data communication network basically process the transport data of the packet streams at various level of granularity, that is, various traffic volume per aggregate and variable degrees of knowledge and effort required to handle the aggregate. For example, some VLAN low-layer aggregates have only local significance and very simple switching tables, while non-aggregated flows with end-to-end IP address pairs can need a huge routing table to find out what the network node should do. Conversely, an ideal network node would automatically reconfigure its resources so that all layers of networking, from highly power-efficient but wholesale optical wavelength switching to power-hungry but very fine-grained worldwide IP routing, receive exactly the required amount of hardware and power consumption, proportional to demand.

[0003] At the time being, new layers for next-generation networks should easily be integrated. Moreover, the packet grooming tasks that manage the multiplexing/demultiplexing of add and drop traffic with the transit traffic, at the optimal layer and pipe size, should operate at the highest efficiency by keeping as much traffic as possible in the lowest, most energy-efficient layers. The electronic implementations are nowadays mostly separate by layer. Parallel processors allow processing a huge routing talk by avoiding an overloading when the data load becomes very large.

[0004] But, the overall aggregation tasks are responsible for a large amount of the global consumption at the core network nodes. Conventional equipment often handles packets sequentially on specialized cards and shelves, one per layer of network, using power-hungry chip and module interconnections (backplane line drivers, serializer-deserializer, . . . ) To enhance the power efficiency, alternative solutions have to be proposed without compromising the layered organization of the network, however. A multi-layer control plane such as the Generalized Multi-Protocol Label Switching (GMPLS) is a standard integrating such new data plane solutions, across all the interfaces between equipments from different vendors.

[0005] It is useful now to develop a solution allowing reducing the energy consumption of the packet processing equipment.

[0006] One object of this invention is to improve the ability of the processing into the nodes of a data communication network and provide a reduction of the energy consumption in the packet processing equipment.

[0007] This is achieved by a method of processing of data received in a network communication node comprising a plurality of processors aimed at processing said data at a given layer of communication so as to route them towards a given destination, the method being characterized in that it comprises the steps of:

• • [0008] determination of at least one layer of communication for processing the received data,
  • [0009] selection of a given number of processors to process the data according to the determined layer of communication,
  • [0010] switching of the selected processors in an active mode to process the determined layer of communication applied to the received data,
  • [0011] switching of the selected processors in low consumption mode once processing carried out.

[0012] With this manner, the present invention proposes to design a better functional block, making its resources shareable and dynamically configurable on multiple layers. Such solution, among others, allows reducing the energy consumption by adapting the number of active processors in function to the layer of communication to process.

[0013] According to an aspect of the present invention, the selection of a given number of processors is performed periodically, the same processor being assigned to the processing of different layer of communication at different time. With this manner, a processor may be assigned according to the global need of the node at any time.

[0014] According to a further aspect of the present invention, the received data are associated with information identifying the at least one layer of communication to be processed by the said node. With this manner, the node easily identifies the layer of communication to process.

[0015] According to another aspect of the present invention, the received data are processed by several layers of communication; the step of selection comprises the selection of a first group of processors performing a first layer of communication and at least a second group of processors performing at least a second layer of communication. With this manner, the node configures its resources, i.e. the available processor according to the different layers of communication to perform. According to an additional aspect of the present invention, the numbers of processors in the group processing each layer are selected so that the time of data processing by groups of processors being appreciably the same one. With this manner, the number of used processors is optimized.

[0016] According to another aspect of the present invention, the processing at a given layer of communication comprises the aggregation of several data packets for sending together them towards the given destination. With this manner, several packets of data are processed together in the node and sent together to the next node.

[0017] According to a further aspect of the present invention, the processors are gathered in a plurality of clusters of processors located geographically near in the node, the processors selected for processing a given layer of communication being with priority selected in the same cluster. With this manner, the processing between the processors performing the same layer of communication is optimized.

[0018] The present invention also refers to a network communication node for processing data received comprising a plurality of processors aimed at processing said data at a given layer of communication so as to route them towards a given destination, the node being characterized in that it comprises:

• • [0019] means for determination of at least one layer of communication for processing the received data,
  • [0020] means for selecting of a given number of processors according to the determined layer of communication,
  • [0021] activation means for switching of the selected processors in an active mode to process the determined layer of communication applied to the received data, and switching of the selected processors in low consumption mode once processing carried out.

[0030] The embodiments of the present invention refer to the use of a plurality of processors equipping a node of communication such that the energy consumption power is globally reduced.

[0031] FIG. 1 illustrates a block diagram of a general data communication system 100 that includes a plurality of nodes 101-106. The nodes 101-106 are intercoupled in a manner that allows data communication amongst the nodes included in the system 100. In one example a connection may be established between endpoint A and endpoint B. Data being transmitted from endpoint A to endpoint B may traverse the network by starting at node 101, travelling through nodes 102 and 103, and finally arriving at node 106. The intermediate nodes 102-105 being intercoupled, the data may travel through numerous different ways. In the intermediate nodes, the data packets may be aggregated if they travel towards the same node. The communication between the nodes may be by wire (optical or electrical) or wireless.

[0032] In each intermediate node 102-105, the data packets are analyzed. According to the processing applied on the data, the data cross one or several logical layers above the physical layer. Above the optical layers that comprise the topology of fibers and the routes of wavelengths, there are different electronic layers of networking, such as Ethernet, Virtual Local Area Network (VLAN), MPLS, IP. . . . These layers are organized as shown in FIG. 2. Each layer has its own packet headers and its own view of the network neighborhood. The higher the layer, the more work per packet must generally be done in the interface electronics named “line cards”. In particular, packet headers are added and removed or modified in a succession of aggregation and de-aggregation processes between each neighboring layer.

[0033] In current equipments, the core routers for example, the packet forwarding process is performed independently at each layer, with at least a dedicated processing unit (card) per layer. Typically, an added or dropped packet flow traverses intermediate cards that correspond to the layer crossing jobs. It must be stressed here that the multiple interconnections between cards are a source of energy efficiency loss due to the need for driving copper tines, serializing parallel data, and protecting each data transfer by means of dedicated error correction techniques.

[0034] New network layers—corresponding to new packet headers—may be developed for next generations of networks; for example, the current Internet may become just a minor application on top of them. The hardware provides total flexibility, so that an arbitrary assignment of the current or future layers to processors can be made. Following power-saving design principles, many low-frequency low-voltage circuits advantageously replace few high-frequency processors.

[0035] FIG. 2 shows the different types of traffic that are encountered in an intermediate network node. The data may cross several sub-layer splitting such as:

• • [0036] pure optical transit traffic, which does not require any electronic processing (most power efficient processing) and thus remains at the lowest (physical) layer, noted “layer 1”,
  • [0037] add or drop traffic (from or to IP packet level), which requires to go through every layer and thus through all grooming stages (least power efficient case),
  • [0038] intermediate cases for which data processing requires electronic as, for example, at boundaries between two network domains.
  • [0039] switching of data from the IP level is the most costly process because it uses huge tables. On the other hand, transit traffic in low layers such as the one that is aggregated in a single wavelength almost requires no electronic effort at all, noted “layer 3”.

[0040] According to the present invention, the interface modules, i.e. the line cards, commonly use many universally programmable processors driven by scheduling resources that is assigned to any type of packet a programmable local path through assigned processors.

[0041] The FIG. 3 illustrates the traffic splitting among layers within an intermediate node 102-105. At the level of bottom wire (layer 1), the traffic aggregated on one wavelength passes through without any Optical/Electrical/Optical (O/E/O/) conversion. At the level of middle wire, an O/E/O/conversion is needed, the data are processed in the layer 2 line card. At the level of the top wire, a complete termination of aggregation up to the IP layer is performed, that requires full node functionalities.

[0042] According to the present invention, a certain number of processors are activated according to the layers crossed by the data. To make that, a generic pool of processing resources are shared to perform many different forwarding and cross-layer aggregation tasks: for example, processing at layer 2 and layer 3 use a single common processing platform while maintaining the logical layer separation.

[0043] The FIG. 4 illustrates the processing of aggregation through an intermediate node, in accordance with a particular embodiment of the present invention.

[0044] The input and output line-cards successively go through the four basic functions:

• • [0045] media access controlling
  • [0046] framing of the packets
  • [0047] classifying of the packets
  • [0048] managing of the queue of the traffic.

[0049] More the processing is performed in high level of layer, more its contribution to resource consumption is high. The most important consumption of resource is used by the upper networking layers, such as IP routing. The present invention proposes to design a better functional block, making its resources shareable and dynamically configurable on multiple layers.

[0050] The FIG. 5 illustrates the cross-layer parallel architecture for processing tasks, in accordance with a particular embodiment of the present invention.

[0051] The intermediate nodes 102-105 comprise a plurality of processors that are close together on a packet processing card. In a card, or module, three sets of objects cooperate:

• • [0052] the set of processors (also called processing units),
  • [0053] the set of tasks (each one associated with a networking layer or with a layer-crossing event),
  • [0054] the set of packet flows through the unit.

[0055] A packet on its way through the module can see a sequence of tasks and walk along a route to different processors. The switch matrix allows collecting data from different packets and put the data in a single packet. Then, this packet passes through the different steps of formatting and is placed in the flow.

[0056] At the level of an intermediate node, subsequent tasks can run on the same processor or on different ones that preferably are neighbors. In the simplest cases, designed to be the most frequent ones, the packet sees only one short task, one processor. The dynamic assignments of tasks to processors, and the assignments of packets to a route of pairs (task, processor), are the job of an optimizing task scheduler. The variable paths, or routes, of a packet in the module are a generalization of the well-known packet pipelines. Even if the design seemingly mixes all networking layers on the same chip, the tasks are designed to be logically independent per layer. For example, a task of layer N only touches the packet header for layer N; a task for the transition N+1 to N adds a piece of header for layer N; a task for the transition N to N+1 removes the header belonging to N. The software design neatly respects the layered organization of the network, while the hardware can do anything almost anywhere. This is the reason for the claimed power efficiency of the architecture, which eliminates long transmission lines and their serialization/deserialization between specialized boards.

[0057] At the left of the FIG. 5, a collection of layers that correspond to data tables and to processing tasks on the board is illustrated. Face to the collection of layers and corresponding tasks, the FIG. 5 shows a task scheduler that manages a pool of processors, for example 30 processors. At a given time instant, each processor is idle state or is assigned to a task and to the processing of a packet. The FIG. 5 shows a group of 4 processors assigned to the processing of MPLS layer, this group sends the data for processing at the level of IP layer to a group of 6 processors. The cross-layer Control Plane has informed the processor board of the actual aggregation/de-aggregation requirements and provided the forwarding tables for the layers involved. The task scheduler evaluates the amount of processing needed at each layer; it reserves the processing resources for each sub-layer and builds a local path inside the processor board so as to successively perform all aggregation tasks. The data remain inside the processor board until all tasks are achieved.

[0058] The FIG. 5 also shows the packet trajectory through the intermediate node. The first level strips off the Ethernet MAC header, the next level terminates a VLAN and corresponding headers to be stripped off. The third level knows that an MPLS header must be used, and label-switching decides on which tunnel the packet should now travel. Such a packet sees two layer crossing tasks on its way, plus one switching task in a specific layer. Although the packet leaves the chip on the FIG. 5, it might as well in subsequent on-chip stages be re-multiplexed into some other fat pipe, for example another 10 Gb-Ethernet, with or without VLANs. The active processors are drawn in grey in the FIG. 5. The inactive processors within the considered time frame are turned off (or set to a low power consumption idle mode) by the task scheduler (in white in the figure).

[0059] The FIG. 6 illustrates a flow chart of an example process for an exemplary embodiment at the intermediate node. The data packets are received by the intermediate node (step 6.1). Each incoming data packets is analyzed for determining which layer or inter-layer number (or task identifier) required for processing (step 6.2). Then, at the step 6.3, the task scheduler assigns for each task (i.e. used layer) the appropriate number of processors. The selected processors switch from an idle mode to an activation mode (step 6.4). When the task assigned to one or several processors is ended, the processors switch off and are placed in idle mode.

[0060] The task scheduler determines the number of processors assigned to certain tasks according to several constraints:

• • [0061] processors without a task assignment go to a power-saving sleep state,
  • [0062] the number of processors per task (or layer) is high enough, with respect to the stochastic process of traffic demand, so that no packet has to wait more than a minimal time (a few microseconds, depending on the Quality of Service—QoS) for a free processor
  • [0063] packets may move through a pipeline of more than one processor when they change layers. This is the case of add and drop traffic, corresponding to the aggregation of higher-layer input into a lower-layer pipe, or vice versa the case of the de-aggregation process.
  • [0064] the task-switching times of the processors, or the wake-up delay from sleep state, should be low: typically 10 microseconds.
  • [0065] the number of tasks switches and wake-ups per second should be below 1000 or so (to be defined, related to models of power efficiency)
  • [0066] packets of the same micro-flow, or alternatively the same pair of source and destination address, should never (or below a given probability) be processed out of sequence.

[0067] The statistics of traffic, for example its burstiness, are continuously monitored. The number of processors available for processing each layer is overprovisioned with a safety margin that avoids buffering. There are a number of buffers on small on-chip memories; the overhead (delay, power consumption) of external RAMs would be too high. Advantageously, the intermediate node contains more processors than memory buffers to the chip. If the traffic demand exceeds an acceptable toad level, compatible with Quality of Service (QoS), and if no sleeping resources are available, any new traffic is discarded and appropriate signaling is sent to the relevant neighbors in the network and to the control plane.

[0068] Take an example: at a particular time, the intermediate node illustrated in the FIG. 5 contains a pool of 30 processors which may be partitioned like this:

Assigned task	Number of processors	Time of processing per packet
layer 3 - IP	2	1 μs per packet
layer 2 - MPLS	6	0.3 μs per packet
layer 2 - VLAN	8	0.1 μs per packet
layer 2 - Ethernet	7	0.1 μs per packet
Idle mode	7	—

[0069] The above numbers of processors assigned to each groups is a picture at a very short time (typically, one millisecond), the next moment, the number of processors for each group may be totally changed, and a given processor is assigned to the processing of another layer. The above example corresponds to a node interface where most of the traffic can be passed on in a lower layer (layer 2—Ethernet) : processing there requires only 10% of the time (and energy) per packet, compared to the higher layer (layer 3). It is the job of the intelligent control plane to manage a good sharing of “lower-layer bypass”. With this manner, the node electronics touches the packets at higher layers only when this is absolutely required, to perform the task of add, drop, access aggregation and de-aggregation. In the example above, the intermediate node is not overloaded since it has 7 processors in idle mode.

[0070] Multiplexing gain in shared processors, like the one in shared data flows, can schematically be expressed as follows, at a given time instant, processing resource needs for overall aggregation tasks are so that:

$\underset{\mathrm{All}\_\mathrm{sublayers}}{\mathrm{Max}}\left(\mathrm{\Sigma processing\_needs}\right)\le \underset{\mathrm{All}\_\mathrm{sublayers}}{\Sigma }\left(\mathrm{Max}\left(\mathrm{processing\_needs}\right)\right)$

[0071] For each layer, the maximum number of processors is specified by the manufacturer for a given traffic at a given time instant. The number of processors of the pool is lower or equal than the sum of maximum numbers of processors specified for each layer for a given traffic at a given time instant. With the architecture disclosed for the invention, the number of processors may be optimized regarding the maximum numbers processing the layers for any time.

[0072] An enhanced control plane transmits GMPLS information identifying the type of processing task for each received packet flow. According to a simple mode of embodiment, the GMPLS information may be a binary flags field specifying the layers has to be processed by the intermediate node. Thank to this information, the task scheduler allocates resources for processing the identified layers at the level of intermediate node. With this manner, the task scheduler that shares the processors between the different layers is aware of which layers each data packet has to cross. According to a variant, the control plane may also supply the task scheduler with the required switching tables and packet handling rules. It can be coupled to a global on-line optimization software that maps the network's traffic-demand and physical resources onto power-efficient layered logical topologies. As far as possible, low-layer bypass paths are created and the task scheduler receives updated decision look-up tables from the control plane. Due to the top-down control, the same group of processors can handle packets on different cleanly separated layers.

[0073] According to a main aspect of the invention, the unused processors are turned off, so as to further decrease the power consumption. The task manager periodically scans the state of the processor by a traffic detector at each layer/sub-layer which allows the task scheduler to share the tasks between the processors, and switches the unused processor in an idle mode. Typically, the task manager scans the traffic 1000 times per second.

[0074] Packets can typically be handled as transit-traffic (input and output in the same layer), add-traffic (higher layer input, then multiplexing into a lower-layer output stream) or drop-traffic (low-layer aggregate is demultiplexed into higher layer subsets). The amount of saved power also depends on the quality of the above-mentioned optimizer software that creates and dimensions the low-layer paths in the first place. Such an optimization is not part of this description, but there are other efforts under way to integrate optical-path and packet-transport optimizers, and even to enhance the GMPLS control plane with relevant message types.

[0075] The task scheduler manages the analyzing of the input data and takes decision for the processing. The decisions are triggered by packet events: either incoming packets from a physical interface, or packets moving inside the chip along a “pipelined path”. The task manager performs the following tasks:

• • [0076] check the type of layer, i.e. the type of relevant packet header,
  • [0077] compute the hash value, then look up the processor assigned to this pair (layer, hash-value). With this manner, the tack scheduler can share the task on several processors,
  • [0078] if no processor is assigned, wake up a sleeping processor, and update the hash-value-to-core mapping,
  • [0079] if the assigned processor is busy but a buffer space is available, queue the packet for that processor,
  • [0080] if no processor and no buffer are available, discard packet and send alarms.

[0081] To simplify the processing of the task scheduler, the determination of the number of processors to activate is driven by lookup tables. The tower the layer, the simpler the table. For each received packet, the following task will be performed:

• • [0082] address the table using fields from the packet header,
  • [0083] retrieve action codes: how to modify the current header (push, pop, swap some fields for example), what next step for the packet (continue on-chip, transfer to specific card interface, shape, police, discard . . . ),
  • [0084] apply any chosen shaping/policing filters (example: leaky buckets),
  • [0085] update statistics counters.

[0086] Based on the decision taken by the task scheduler, the pool of processor receives and emits specialized messages. In particular, most table entries are fed from the control plane's “Forwarding Information Bases”. Here are the events sent by the task scheduler:

• • [0087] send alarms, answer queries about statistics per layer
  • [0088] create, modify, destroy layers and lookup table entries, whenever requested by the control plane
  • [0089] modify QoS related parameters (e.g. in filters) on request.

[0090] Moreover, time-out events are performed at the level of each processors of the processor core. For example, if a processor or a buffer space is unused for more than a certain time, they are put to steep and the mappings from hash values to those pieces of hardware are deleted.

[0091] The FIG. 7 illustrates a cross-layer parallel architecture according to an improvement of the present invention. Once the task scheduler determines the number of processors associated with a certain task, it jointly selects them according to a strategy of allowance of the processors based on their proximity (from a card architecture point of view) in order to maximize the processing efficiency. For example, if the tack scheduler needs five processors to process a task, it selects five geographically close processors within the pool. According to this improvement, the task scheduler defines a number of clusters of processors which are firstly intended to work together, of the fact for example of their geographical proximity within the node of communication. The proximity of the processors is advantageous in term of consumption of energy because the data flows on a short distance between the processors. For example, let us suppose that the pool of processors is divided into 10 clusters of 50 processors and, at a given time instant that in each cluster a certain number of processors are activated:

C1	C2	C3	C4	C5	C6	C7	C8	C9	C10
45	10	48	22	47	42	38	43	45	37

[0092] If the incoming of new data packets requires the activation of 8 processors, the task scheduler preferably activates 8 processors of the same cluster, by example: cluster 2,4,6,7 or 10. the cluster 6 will be selected in priority in order to occupy it to the maximum and thus to make it possible to leave the maximum of possibilities for the other clusters.

[0093] The FIG. 8 illustrates the packet processor management and task scheduling an intermediate node. Four processors are shown. They access and maintain common resources like program code, lookup tables, traffic monitoring results, etc. Incoming packets are pre-classified in a simple logic part that fixes the initial layer they belong to. Eventually packets are buffered, if no processor for the intended layer or layer-transition is available. Processors are dynamically assigned to packet tasks on one of the implemented layers, or to an idle power-saving state, if there is no need for all processors. According to an improvement, the program code of a processor is downloaded from a memory to be configured for the processing of a determined layer.

[0094] Packets are routed to a processor that handles their layer's forwarding job, with layer-specific program code and lookup data. Depending on configuration, a packet may change layers during processing and may be routed through more than one processor. For example, the packet enters on layer L1 through processor P3, which then hands it over on L2 to P4, and finally is routed to and processed in P1, using another variant of layer code L1. This corresponds to a moderately complex switching operation: de-multiplexing from a large L1 traffic aggregate, followed by re-multiplexing into another large aggregate. The task scheduler defines which processors are allocated to which layers or layer pairs (and eventually to which sub-program on the layer, if there are several branches).

[0095] For clarity, the interfaces to the control plane are not shown on FIG. 8. The task scheduler tells to the preclassifer the packets which initial processor is available to them. The tasks may be responsible for the main classification of the packets, header insertion/deletion/modification, according to the packet's layer, and the look-up of destination routes, forwarding classes, queues. The packet processor management may be completely reconfigurable for the probable case that new intermediate layers of networking are added.

[0096] Moreover, old Internet protocols may be downgraded merely to specialized legacy applications on top of them. Equipment can be more compact and power efficient through resource sharing and re-use, while the logic of neatly separated networking layers and minimal interfaces is strictly preserved.

[0097] Where a component (e.g. an assembly, device, memory, etc.) is referred to above, unless otherwise indicated, reference to that component (including a reference to a “means”) should be interpreted as a reference to any component which performs the function of the described component (i.e. is functionally equivalent to the described component), including components which are not structurally equivalent to the disclosed structure which performs the function in the illustrated exemplary embodiments of the invention.

[0098] Where a step in a method is referred to above, unless otherwise indicated, reference to that step should be interpreted as a reference to any step which achieves the same result as the step (i.e. is functionally equivalent to the described step), including steps which achieve a stated result in different ways from those disclosed in the illustrated exemplary embodiments of the invention. As will be apparent to those skilled in the art in the light of the foregoing disclosure, many alterations and modifications are possible in the practice of this invention.