Method and system for controlling data traffic in a network

[0001] 1. Field of the invention

[0002] The present invention relates to a method and a system for communication of data in a network. In particular, the present invention relates to a method and a system for controlling the communication of data via an optimum data path from a sending node to a receiving node in a network having a plurality of nodes being connected via data links, wherein the optimum data paths used for data communication in the network are defined as the data paths which lead to the minimum link costs.

[0003] 2. Background of the invention

[0004] For data communication in a network, the data to be communicated is routed through the network, i.e. the data or data packets thereof are directed via data links of the network from a data sending node to a data receiving node. In order to route data correctly via data links of a network and over nodes of the network connected by the data links routing strategies or so-called routing protocols representing routing strategies are employed. The most popular routing protocols are the Routing Information Protocol (RIP), the Enhanced Internet Gateway Routing Protocol (EIGRP), the Intermediate System-to-Intermediate System (IS-IS) and the Open Shortest Path First (OSPF) protocol. These protocols are used as so-called interior gateway protocols for routing of data in networks which are under a common network administration. Data networks (e.g. the INTERNET) do not have predefined data paths/connections between a sending node and a receiving node, in contrast to networks having fixed communication connections (e.g. telephone networks). Interior routing protocols are in general based on an approach wherein data communication from a sending node to a receiving node is not performed via predefined data links and nodes of the network. Instead, data or data packages are communicated from one node to another, wherein each routing decision which data link to the next node has to be chosen is made up by each node separately. Therefore, a node also referred to as router transmitting data to other nodes/routers does not know the complete data path or the data links forming the data path, respectively, for the communication of data from the sending node to the receiving node. Each node/router for itself make its own routing decision depending on its routing information to which next node/router the data or data packages thereof can be transmitted. As a consequence, the routing decision, i.e. the determination of the next node/router to which data is transmitted is made locally by the node/router communicating the respective data.

[0005] If the operation condition of each node/router within the network is known, it is possible to determine a data path and the data links forming the data path over which data or data packages thereof are communicated through the network. The operation condition of each node/router and, in particular, the routing decision of each node/router is determined via a cost function which determines the cost of the links. The cost function is protocol specific and can depend on the capacity of the links, the type and/or amount of data to be communicated, the utilization of data links to nodes to which data have to be transmitted, the (physical) delay and the data communication situation of the whole network. In practice the cost functions depend only on static and/or predefine variables like the capacity of the links and a delay value for the links which are once defined in the router database. Therefore if no errors in the network occur, the operating conditions of nodes/routers of a network do not vary during data communication, which makes it possible to determine the data path over which data will be communicated from the sending node to the receiving node.

[0006] A further fundamental principle of the interior routing protocols mentioned above is to always choose the shortest data path between a sending node and a receiving node. The routing protocols define a metric according to which the length of a data path is determined and information for the determination of the shortest data path are distributed over the network. For example, the Open Shortest Path First (OSPF) protocol commonly uses an additive metric based on time independent parameters such as the data communication capacity of data links (e.g. CISCO routers) or/and the delay for data communication over data links. In the OSPF-metric, the costs for each data link from a node/router to other nodes/routers connected thereto are inputted. Data communication parameters of a data link between a node/router to the next node/router can depend on the direction of data communication over the data link. Thus, the costs for data links may vary in relation to the data communication direction. The overall costs of a data path is obtained by a summation of all costs of the data links included by the data path.

[0007] The metric of the Enhanced Internet Gateway Routing Protocol (EIGRP) is more complicated than the metric of the OSPF-protocol but is mostly used in a form reduced to an additive metric.

[0008] The approach always to select the shortest data path implies a single-path routing, i.e. the routing between two nodes/routers is identical for all data flows. It is possible to define a multipath routing but for stability (e.g. loop protection) and protocol depended reasons (e.g. TCP performance degeneration in the case of reordered packets) it is rarely used. As a result of the fixed cost metric and the single path routing, congestions of data paths or single data links included of a data path may occur. This occurs for example if the amount of data is larger than the data communication capacity of single data links. Similar problems can occur if the data communication capacity of a node/router is not sufficient to communicate the desired amount of data, e.g. if the node/router is included by data paths of different sending nodes and/or different receiving nodes.

[0009] Furthermore, specific data paths and/or data links may be utilized very often, while other may be used only sometimes. This unequal distribution of data communication over the data links results in an unequal link utilization of the network data links. This unequal link utilization may also lead to congestions for different data links. Even in the case of high utilized links the network is not applicable for real time applications like IP telephony because of the nonlinear dependency between the load and the waiting time. Furthermore, such a routing of data in a network leads to an uneconomic operation of the network since data links, which are provided within the network requiring expensive hardware and software means, are not used.

[0010] The conventional approach to optimize routing of data in a network was to improve/enhance existing routing protocols or to develop new routing protocols. Since altered or new routing protocols must be established throughout the whole network, usually, altered or new hardware and software components are required. Therefore, this optimization is mainly limited due to organizational problems.

[0011] The object of the present invention is to solve the above described problems of the state of the art. In particular, the present invention is directed towards an optimization of routing of data in a network using existing routing protocols, namely without modifying or replacing routing protocols already used in a network and the respective hardware and software components of the network. Moreover, the present invention provides a method and a system for routing data in a network to achieve a homogeneous distribution of communicated/routed data or data traffic, respectively, throughout the network. The method is also applicable for traffic engineering in MPLS and routing in further (e.g. with optical/digital wrappers) optical networks.

[0012] To solve the above problem, the present invention provides a method for controlling data traffic in a network as defined in claim 1.

[0013] The method according to the invention is based on the approach to optimize the routing of data in a network on the basis of a linear optimization problem. The linear optimization problem according to the invention has to be defined such that data routed/communicated in a network also referred to as data traffic is uniformly/homogeneously distributed over the network. In terms of the invention, the expression uniform/homogeneous distribution of data traffic should not be understood as equal distribution of data traffic. An equal distribution of data traffic throughout the network would result in identical/equal data traffic for each data link connecting nodes of the network.

[0014] Although an equal distribution of data traffic can represent an optimum solution of the linear optimization problem according to the invention, the method according to the invention is not specifically applied to obtain an equal distribution of data traffic but is directed to an optimization of routing of data in a network wherein link utilizations of data links in the network may be different.

[0015] The method according to the invention is used to communicate data via an optimum data path from a sending node to a receiving node in a network having a plurality of nodes connected by data links. In the following, the term data path is used for data paths between the sending node and the receiving node, wherein data paths are formed by respective ones of the data links.

[0016] For the determination of an optimum data path from a sending node to a receiving node, all loop-free data paths from the sending node to the receiving node over respective data links and nodes are determined and data paths are selected such that all data communicated between two nodes are communicated via the same data path. Following, link costs for the selected data paths are determined and the data path having the minimal link costs is defined as the optimum data path. Then, the data are communicated from the sending node to the receiving node via the optimum data path.

[0017] Further, the selection of data paths can comprise determination of data communication capacities of the data paths and selecting data paths having data communication capacities sufficient to communicate the data from the sending node to the receiving node, and/or determination of physical delays of the data paths and selecting data paths having physical delays equal or smaller than a predetermined maximum physical delay.

[0018] The determination of the optimum data path from the sending node to the receiving node can be based on the definition of an equation system for a linear optimization problem and the solution of the equation system to define the optimum data path.

[0019] For the equation system, an objective function can be defined which includes a term being indicative of the link costs.

[0020] Also, it is possible to define an objective function including a term representing maximum link utilizations, or average link utilizations, or values indicative of combinations of the maximum link utilizations and/or the average link utilizations and/or the link costs.

[0021] Beside the objective function, a linear optimization problem requires the definition of constraints. According to the invention, cost constraints for the link costs of the selected data paths can be defined.

[0022] Moreover, transport constraints can be defined for flow conservation and/or together with the average link utilization part of the objective function for determination of loop-free data paths.

[0023] Further, routing constraints can be defined such that all data, which are to be communicated from the sending node to the receiving node, are communicated via the same data path. Such a definition of routing constraints allows to utilize existing routing protocols which select the shortest data path between nodes/routers and employ a single-path routing.

[0024] Additionally, capacity constraints can be defined for the determination of data paths having sufficient data communication capacities, preferable maximum data communication capacities being equal or larger than the amount of data to be communicated and/or data communication capacities being equal or smaller than a predefined maximum link utilization.

[0025] A further definition of physical delay constraints allows the determination of data paths having physical delays being equal or smaller as a predefined maximum physical delay. In terms of the invention, the physical delay represents the time required to communicate data between nodes of the network connected by respective data links.

[0026] In order to obtain an optimum data path, the equation system for the linear optimization problem is solved by a minimization of the object function regarding the constraints. Solving the equation system, all possible data paths from the sending node to the receiving node are determined and the data path having the minimal link costs is defined as optimum data path. Especially for complex networks including many nodes and connecting data links, it can be advantageous to perform the minimization of the objective function iteratively.

[0027] With respect to the value indicative of combinations of the average link utilization and/or the maximum link utilization and/or the link costs, a function can be used which represents the average link utilization and/or the maximum link utilization and/or the link costs weighted in relation to each other.

[0028] More particularly, the component of the objective function having a higher influence/importance for the determination of the optimum data path is weighted by a higher value compared to the weighing value for the other component of the objective function.

[0029] In order to minimize the number of selected data paths which are to be considered in the determination of the optimum data path, it is possible to exclude data paths having link costs exceeding a predefined link cost range. For example, the link cost range can be defined between 1 and a predefined upper threshold, e.g. a value four times larger compared to the number of routers of a network including the sending and receiving nodes.

[0030] With respect to the complexity for solving of the equation system, which can be very time consuming, it is possible to use the last solution of the equation system which has been determined within a predefined time interval. For example, this approach is preferred if, within the predefined time interval, the objective function is not feasible, no minimum can be determined for the objective function, or the objective function does not converge.

[0031] Depending on the network for which the method according to the invention is used, the number of variables for the definition of the equation system can be extremely high. In particular, the number of variables for the objective function is in the magnitude of N²M², wherein N represents the number of nodes/routers and M represents the number of data links. Determined by the routing constraints, the number of variables for the constraints according to the invention is in the magnitude of N³M². In order to reduce the number of variables and/or to reduce the number of data links to be considered for the solution of the equation system, preferable in the case of large networks, the method comprises a determination of the time for data communication from the sending node to the receiving node for all loop-free data paths. Further, data paths are identified as data paths to be considered for the solution of the equation system for which the smallest data communication time and/or a data communication time being smaller than a predefined maximum data communication time have been determined. Preferable, the maximum data communication time is predefined in relation to the smallest determined data communication time.

[0032] With respect to the identification of data paths being considered for the solution of the equation system, only data links included in possible data paths are used. Moreover, it is possible not to consider data links for the solution of the equation system which are not included in all possible data paths.

[0033] For some networks, predefined communication time limits representing the maximum time for data communication between nodes are employed. In order to consider such predefined communication time limits, it is possible to define the physical delay constraints on the basis of predefined communication time limits. Here, the physical delay constraints are not determined by the actual time required for the communication of data between nodes but by the predefined communication time limits for respective data links.

[0034] Likewise, characteristics of a network resulting from the band widths of the data links are considered by defining the physical delay constraints on the basis of the bandwidths of the data links. Instead of using the actual time for the communication of data over a specific data link, the bandwidths of the data links are transformed into respective physical delays.

[0035] As an alternative to the above equation system, it is possible to determine the link costs on the basis of a linear optimization problem. Here, it is preferred to obtain the link costs by computing an inequality system for the linear optimization problem. Accordingly, it is possible that a data path of the selected data paths having a sum of link costs being larger compared to sums of link costs for remaining data paths can be selected as the optimum data path.

[0036] It is preferred to define cost constraints for all of the selected data paths other than the optimum data path such that the link costs of the optimum data path are forced to be smaller than the link costs for all of the selected data paths other than the optimum data path.

[0037] In order to reduce the complexity of the inequality system, it is possible to define cost constraints for those of the selected data paths other than the optimum data path which start from a node neighboring the sending node of the optimum data path and end at the receiving node of the optimum data path, and which do not include further nodes of the optimum data path, while no cost constraints are defined for the remaining of the selected data paths other than the optimum data path.

[0038] Due to the fact that the amount of data to be communicated between a sending node and a receiving node may vary over the time and due to the fact that several sending and/or receiving nodes are communicating over the network, it is preferred to perform the determination of the optimum data path according to the invention continuously, at predefined times, or in predefined time intervals. Here, it is possible to determine the amount of the data to be communicated continuously, at predefined times, or in predefined time intervals, wherein different predefined times and/or different predefined time intervals can be used for the determination of the optimum data path and the determination of the amount of data to be communicated.

[0039] As an alternative, it is possible to determine the optimum data path for a predefined (e.g. maximal) amount of data to be communicated. Preferable, the predefined amount of data to be communicated is sensed or approximated.

[0040] Especially in the case of very large/complex networks, there is often a possibility to divide the network into at least two parts or partial networks for each of which the determinations of the optimum data paths are performed separately. Due to the shortest path principle the concatenated solution of the smaller networks is equal to the solution obtained by the optimization of one large network.

[0041] Moreover, it is possible to group at least a part of a network to a virtual node. Here, the determination of the optimum data path in the remaining portion of the network including the virtual node and the determination of the optimum data path in the grouped part of the network representing the virtual node are performed separately.

[0042] The present invention also provides a system for communication of data in a network via an optimum data path according to claim 26. The system comprises a network having nodes/routers connected by data links, wherein at least one of the nodes is a sending node and at least one of the nodes is a receiving node, and control means being connected to the network for controlling data communication in the network. In particular, the control means are adapted to carry out one of the methods according to the invention for communicating data via an optimum data path.

[0043] The control means can be a central control system or a component of a central control system (e.g. an INTERNET-server) controlling data communication of at least one node/router, preferable of all nodes/routers.

[0044] Further, the control means can comprise at least one control unit associated with one or several nodes/routers for controlling data communication thereof. In order to control data communication in an optimal manner and/or to control data communication of further nodes/routers, such a control unit should be in communication with the further nodes/routers, preferable with all nodes/routers to obtain information of data communication within the network.

[0045] Especially for large networks and/or network exhibiting a high amount of data communication, a combination of a central control system and at least one control unit can improve the identification of optimum data paths, because data communication control functions for single nodes and data communication control functions related to the entire network can be accordingly distributed to different components of the control means.

[0046] FIG. 1 illustrates the effects of a single-path routing on data paths in a network.

[0047] FIG. 2 illustrates the effects of the transport constraints on data paths in a network.

[0048] FIG. 3 illustrates the effects of the routing constraints according to the invention on data paths in a network.

[0049] FIG. 4 shows example networks with 6, 8 and 14 nodes/routers.

[0050] FIG. 5 illustrates the data routing according to the invention for the 6 node network for minimized average link utilization.

[0051] FIG. 6 illustrates the data routing according to the invention for the 6 node network for minimized maximum link utilization.

[0052] FIG. 7 illustrates the link costs for the 6 node network with minimized average link utilization according to the invention.

[0053] FIG. 8 illustrates the link costs for a 6 node network with minimized maximum link utilization according to the invention.

[0054] FIG. 9 shows a diagram for the changes of link utilizations according to the invention for the 8 node network.

[0055] FIG. 10 shows a diagram for the changes of link utilizations according to the invention for the 14 node network.

[0056] FIG. 11 shows a diagram for the link utilizations according to the invention for selected data links of the 14 node network.

[0057] FIG. 12 shows loops which are only avoided by the objective function and not by the transport constraints.

[0058] FIG. 13 shows an example for link cost algorithm according to the invention.

[0059] In general, nodes of a network are computer based systems/units(server) which can receive and transmit data via data links which are connected with respective nodes. The nodes can be operated as sending nodes from which data send into a network are originating. Further, the nodes can be operated as receiving nodes serving as a destination for receiving data communicated from a respective sending node. Moreover, the nodes can be operated as so-called routers which receive data from the sending node or from another node/router and transmit the received data to a further node/router or the receiving node.

[0060] Data links between nodes of the network are often referred to as interfaces since the data links do not only comprise lines for the physical transmission of electrical/optical signals representing data to be transmitted but also hardware and software components required for the transmission of data in a data network. Such hardware and software components include conventional electrical lines, data busses, buffer/memory units, processor based units, software programs for data conversions and implementation of data communication protocols (i.e. routing protocols), and the like.

[0061] In order to achieve the objective of the present invention, namely to minimize all link utilizations (average link utilizations), to minimize the maximum link utilization, and to keep the physical delays of the data links between nodes of a network in a certain range, it is necessary to describe formerly the network and the data traffic thereof, i.e. the amount and flow of data communicated in the network. For this formal description, three matrixes are used:

[0062] First, a network with N nodes is defined by a capacity matrix C having the size NN, wherein each entry of the capacity matrix C represents the link capacity c_ij for each pair of nodes i and j. In case no data link is existing between two nodes, the respective entry in the capacity matrix C is set to 0.

[0063] The second matrix is a traffic matrix F which comprises an entry f_ij for each pair of nodes i and j. Each entry f_ij represents the data traffic/data flow between two nodes i and j. If no data is exchanged between two nodes, the respective entry in the traffic matrix F is set to 0. Depending on the implementation of the method according to the invention, the entries f_ij of the traffic matrix F can represent static data flows or, e.g. if the method according to the invention is carried out continuously during the operation of a network, the entries f_ij can represent the actual data flows between nodes of the network.

[0064] Third, a matrix D is introduced which describes the physical delays of the data links. The physical delay of a data link identifies the time necessary to communicate data between two nodes over the respective data link. The entries d_ij of the matrix D represent the physical or approximated delay of a data link connecting two nodes i and j.

[0065] Based on these matrixes, the routing optimization according to the invention can be defined as a linear optimization problem. As known, a linear optimization problem is described by an equation system consisting of two parts, the objective function and the constraints. The constraints define a multi-dimensional solution space wherein an optimal solution of the objective function has to be determined. The optimal solution of the objective function is the solution within the solution space producing, in case of the invention, the minimal value for the objective function. To achieve an optimization of data routing in a network, different constraints limiting possible solutions of the objective function have to be fulfilled.

[0066] First, for data communicated from a sending node to a receiving node, data paths have to be determined over which data can be transmitted/routed. In case the sending node and the receiving node are not directly connected by a data link, possible data paths include not only the sending and receiving nodes but also further nodes and respective data links connecting the same. In order to avoid that different data transmitted from a sending node to a receiving node are routed/transmitted over different data paths or data links, respectively, possible routings/data paths were restricted to a so-called single path routing. The single-path routing constraints ensures that exactly one data path between two nodes/routers is defined for a possible solution of the objective function. The effect of such a single-path routing is illustrated in FIG. 1. The shortest data path, independent of the used routing protocol metric, between nodes/routers C and F is either over node/router D or over node/router E. As a result of the single-path routing, both data flows A-F and B-F are routed first over node/router C and then either over node/router D or over node/router E, but not one flow over node/router D, while the other is routed over node/router E.

[0067] Second, capacity constraints have to be fulfilled according to which the amount of data communicated over a specific data link does not exceed the capacity of the respective data link, i.e. the maximal amount of data which can be communicated over the respective data link. The capacity of a data link in a network is not limited to the physical transmission capacity of the different transmission components of the data link (i.e. electrical wires, busses, processors), since data to be transmitted or data packages thereof can be stored for example in intermediate memories of data links and/or of nodes/routers before transmission. Two approaches can be used to define capacity constraints in terms of a linear optimization problem. One of the approaches is path-oriented, while the other is flow-oriented with respect to data links. Here, the flow-oriented approach was chosen to present in detail because of it's easier understanding. After the description of the flow oriented approach the changes in the notation of the method for the path oriented approach are described.

[0068] In order to apply the method for an routing optimization according to the invention in combination with existing routing protocols, the routing algorithms specified by the used routing protocols has to be considered. As mentioned above, routing protocols currently in use, i.e. the OSPF-protocol and the EIGRP, are based on routing algorithms for the determination of the shortest data path for data to be routed from a sending node to a receiving node. These algorithms for the shortest data path routing imply that all data communicated between two nodes/routers are routed over the same data links.

[0069] The objective is to achieve a distribution of transmitted data or a data traffic distribution as homogeneous/uniform as possible throughout the entire network. In other words, all data links of the network should be utilized as equal as possible at a data communication level as low as possible. Since the flow-oriented approach is utilized, for each data flow between nodes/routers i and j having a data traffic entry f_ij0 in the data traffic matrix F and for each data link having a capacity entry c_ij>0 in the capacity matrix C, a Boolean variable x_ij^uv is introduced. This variable is set to 1 if a data flow uv from a sending node u to a receiving node v is routed over the data link ij between nodes i and j, otherwise this variable is set to 0. Further, a variable t is introduced defining an upper threshold for the link utilization of all data links.

[0070] Both the maximum link utilization and the average link utilization have to be minimized. Therefore, the objective function includes two additive parts both of which have to be minimized: $\begin{array}{cc}{a}_i*i+\underset{\mathrm{ij}}{}\underset{\mathrm{uv}}{}\frac{f_{\mathrm{uv}}{x}_{\mathrm{ij}}^{\mathrm{uv}}}{c_{\mathrm{ij}}},\mathrm{with}{c}_{\mathrm{ij}}>0& \left(1\right)\end{array}$

[0071] The left additive part of the objective function (1) represents the maximum link utilization, while the right additive part of the objective function (1) represents the average link utilization. The parameter at of the maximum link utilization part of the objective function (1) is used as a weighting factor for weighting the maximum link utilization part with respect to the average link utilization part of the objective function (1). The parameter at is chosen in regard to the influence/importance of a small/minimized maximum link utilization for the selection of an optimum data versus the influence/importance of a small/minimized average link utilization. For example, if the minimization of the maximum link utilization has a higher priority compared to the minimization of the average link utilization, the parameter at should be chosen to have a large value resulting in a reduction of the maximum link utilization by directing data traffic to less utilized data links. Comparable, the parameter at should be chosen to have a small value if the influence of the minimization of the average link utilization onto the selection of an optimum data path should be higher than the influence of the minimization of the maximum link utilization.

[0072] Transport Constraints

[0073] The transport constraints ensure that for each data flow, for example a data flow from a sending node u to a receiving node v, exactly one loop-free data path from the sending node u to the receiving node v is specified by the values of the variables x_ij^uv resulting from the solution of the objective function (1). Here, the transport constraints comprise four sub-constraints for each data flow:

[0074] 1. The data flow uv from the sending node u to the receiving node v being transmitted over node/router i is routed over only one data link leading out of node/router i: $\begin{array}{cc}\underset{j=1,c_{\mathrm{ij}}>0}{\overset{N}{}}{x}_{\mathrm{ij}}^{\mathrm{uv}}1,\mathrm{for}\mathrm{all}\mathrm{flows}\mathrm{uv}\mathrm{and}\mathrm{all}\mathrm{nodes}/\mathrm{routers}i& \left(2\right)\end{array}$

[0075] 2. The data flow uv from the sending node u to the receiving node v is routed via exactly one link ui from the sending node u to another node/router i: $\begin{array}{cc}\underset{j=1,c_{\mathrm{ui}}>0}{\overset{N}{}}{x}_{\mathrm{ui}}^{\mathrm{uv}}-\underset{i=1,c_{\mathrm{iu}}>0}{\overset{N}{}}{x}_{\mathrm{iu}}^{\mathrm{uv}}1,\mathrm{for}\mathrm{all}\mathrm{flows}\mathrm{uv}& \left(3\right)\end{array}$

[0076] 3. For all data flows uv from the sending node u to the receiving node v and for each router i except the sending and receiving nodes u and v, the number of data links leading into router i and the number of data links leading out of the router i used by data flows uv are equal: $\begin{array}{cc}\underset{j=1,c_{\mathrm{ij}}>0}{\overset{N}{}}{x}_{\mathrm{ij}}^{\mathrm{uv}}-\underset{j=1,c_{\mathrm{ji}}>0}{\overset{N}{}}{x}_{\mathrm{ji}}^{\mathrm{uv}}=0,\mathrm{for}\mathrm{all}\mathrm{flows}\mathrm{uv}\mathrm{and}\mathrm{all}\mathrm{nodes}\text{/}\mathrm{routers}i\left\{u,v\right\}& \left(4\right)\end{array}$

[0077] 4. The data flow uv from the sending node u to the receiving node is routed over exactly one data link leading into the receiving node v: $\begin{array}{cc}\underset{i=1,c_{\mathrm{iv}}>0}{\overset{N}{}}{x}_{\mathrm{iv}}^{\mathrm{uv}}-\underset{i=1,c_{\mathrm{vi}}>0}{\overset{N}{}}{x}_{\mathrm{vi}}^{\mathrm{uv}}1,\mathrm{for}\mathrm{all}\mathrm{flows}\mathrm{uv}& \left(5\right)\end{array}$

[0078] The effect of the transport constraints defines exactly one loop-free data path from the sending node u to the receiving node v is illustrated in FIG. 2. According to equation (3), only data link ua connecting the node u and the node a carries the data flow uv. One data link is leading into each of the nodes/routers a, b and c. In line with equation (4), one data link is leading out of each of the nodes/routers a, b and c for the data flow uv. Conditioned by equation (5), the data path for the data flow uv is directed over the data link cv between node/router c and the receiving node v and terminates in the receiving node v.

[0079] Nevertheless, it is still possible that a looped data path exists beside the chosen data path for the data flow uv since such loops are not avoided by the transport constraints (see FIG. 12). However, such loops are avoided by the minimization of the average link utilization in the objective function (1).

[0080] Capacity Constraints

[0081] The capacity constraints ensure that the amount of data or the data traffic over data links do not exceed the capacity of the respective data links. The capacity constraints comprise two sub-constraints for each data link ij between nodes i and j.

[0082] The first sub-constraints force the link utilization to be below a fixed level defined by a parameter a_c: $\begin{array}{cc}\underset{\mathrm{uv}}{}{x}_{\mathrm{ij}}^{\mathrm{uv}}{f}_{\mathrm{uv}}{a}_c{c}_{\mathrm{ij}},\mathrm{for}\mathrm{all}\mathrm{data}\mathrm{links}\mathrm{ij}\mathrm{with}{c}_{\mathrm{ij}}>0& \left(6\right)\end{array}$

[0083] As set out above, data links within a data network are able to handle for a short time interval data traffic being larger than the physical transmission capacity of the respective data link. Therefore, the parameter a_c is not restricted to values between 0 and 1. To set the parameter larger than 1 is also useful to identify links which are the first bottlenecks in case of congestion. For the method according to the invention, the parameter a_c is set to 1 as a default value.

[0084] The second sub-constraints given by equation (7) defines an upper threshold for the link utilizations. The variable t used in the objective function (1) defines an upper bound for the utilization of all links. While the parameter a_c is a fixed value, the variable t is to be minimized. The second sub-constraints force all link utilizations to have a value being lower than the value of the parameter t: $\begin{array}{cc}\underset{\mathrm{uv}}{}{x}_{\mathrm{ij}}^{\mathrm{uv}}{f}_{\mathrm{uv}}\frac{t}{100}{c}_{\mathrm{ij}},\mathrm{for}\mathrm{all}\mathrm{data}\mathrm{links}ij\mathrm{with}{c}_{\mathrm{ij}}>0& \left(7\right)\end{array}$

[0085] Routing Constraints

[0086] The routing constraints are the most important constraints since the routing constraints define only such combinations of data links or data paths, respectively, sufficiently to calculate the costs for data communication/routings obtained by a solution of the equation system for the linear optimization problem. Using the above-mentioned routing protocols, data are always communicated/routed over the shortest data path which is defined by a certain routing protocol metric for each data link. As a consequence, all data flows from node/router i to node/router j have to be communicated/routed over the same data link(s) between nodes/routers i and j.

[0087] These constraints are achieved by a recursive system of sub-constraints:

[0088] Conceiving an iterated or recursive application of equations (8) and (9), the routing between nodes/routers i and j is identical for all data flows through these nodes/routers. Therefore, the found solutions based on the routing constraints are corresponding with the implementation of the routing protocols.

[0089] FIG. 3 illustrates the effect of the routing constraints. Assuming that data flow ub is routed over nodes u, a and b, data flow ac is routed over nodes a, b and c and data flow cv is routed over nodes c, d and v, due to equations (8) and (9), data flow uv from the sending node u to the receiving node v has to be routed over nodes u, a, b, c, d and V.

[0090] For data flow uc, two further data paths exist either over node a or node b. The data path including node a not be used for the data flow uc, since this data path does not include node a which would be contradictory to equation (9). Moreover, the data path including node b can not be used for the data flow uc, since this data path does not include node b which would be contradictory to equation (8). In other words, the data path including node a and the data path including node b can not be used for the data flow uc, because these data paths do not fulfill the routing constraints to route all data flows, i.e. the data flows uc, ac and bv, over the same data links.

[0091] Physical Delay Constraints

[0092] On the basis of the objective function (1) in combination with the transport constraints, the capacity constraints and the routing constraints, a data routing can be achieved having the shortest path routing with an homogeneously, uniformly distributed data traffic. In order to obtain the goal to keep the physical delays for data communication in a network in a certain range, the physical delay constraints being described below have to be applied.

[0093] The range of possible physical delays is defined over a parameter a_r, while the lowest physical delay for a data flow is represented by a variable d_uv^min. If P is a set of all loop-free data paths from node u to node v and if d_p is the sum of all physical delays of the data links of a path p, the minimum physical delay d_uv^min is the delay of the path p from node u to node v with the smallest physical delay d_p:

[0094] The value of the minimum physical delay is obtained by computing all physical delays of all possible loop-free data paths from node u to node v. Introducing a parameter ar, an upper threshold a_rd_uv^min can be defined representing the maximum physical delay for each data flow: $\begin{array}{cc}\underset{\mathrm{ij},c_{\mathrm{ij}}>0}{}{x}_{\mathrm{ij}}^{\mathrm{uv}}{d}_{\mathrm{ij}}{a}_r{d}_{\mathrm{uv}}^{\min },\mathrm{for}\mathrm{all}\mathrm{data}\mathrm{flows}\mathrm{uv}& \left(11\right)\end{array}$

[0095] Reduction of the Complexity of the Equation System for the Linear Optimization Problem

[0096] For the solution of the objective function (1), the number of variables to be considered is in the magnitude of N²M², wherein N represents the number of nodes/routers and M represents the number of data links. Determined by the routing constraints, the number of constraints is in the magnitude of N³M². Especially for large networks including many nodes/routers and data links, the solution of the equation system for the linear optimization problem according to the invention can be very time consuming, complex, or even impossible. Therefore, a so-called pre-solver is used which reduces the number of variables and constraints by considering only relevant data paths for the solution of the equation system.

[0097] Relevant data paths are identified on the basis of the respective physical delay. For each data flow between two nodes/routers, all data paths being loop-free paths are determined. For each of these loop-free data paths, the physical delay is calculated, i.e. the sum of the physical delay of the data links forming the respective data path. Data paths having the smallest physical delay and having a delay lower than the upper physical delay threshold a_rd_uv^min are identified as relevant data paths. Further, all data links of the relevant data paths are checked to be included in all relevant data paths or in none of the relevant data paths. Data links included in all relevant data paths will be used for routing the respective data flow over such data links. Data links included in none of the relevant data paths are not considered by the solution of the equation system of the linear optimization problem such that the respective data flow is not routed via such data links. Data links included only by some of the relevant data paths are considered for the solution of the equation system and the utilization of such data links is determined by the solution of the equation system.

[0098] Sub-Optimal Routing

[0099] In some cases, for example for very complex networks, it might be advantageous not to calculate an routing using the entire equation system but to utilize an approach described below resulting in a sub-optimal (sub-optimal: get a solution which is near the optimum in a practicable time) routing. In contrast to the above described procedure, the maximum link utilization is not directly minimized. Here, the minimization of the maximum link utilization is achieved by an iterative solution of a linear optimization problem having a different, reduced objective function and different, reduced capacity constraints.

[0100] The component a_t*t of the objective function (1) for the minimization of the maximum link utilization is omitted. As a consequence, the sub-constraints for the capacity constraints represented by equation (7) forcing the utilization of different data links below the upper threshold for link utilizations is not necessary anymore. On the other hand, the other sub-constraints for the capacity constraints represented by equation (6) is defined more strictly. Instead of using the default value of 1 for the parameter a_c, the parameter a_c is iteratively minimized.

[0101] Starting from a value of 1 for the parameter a_c, the equation system including the reduced objective function is solved. The link utilizations of single data links are determined on the basis on the found solution of the reduced equation system. In the next step the parameter a_c is set to a value slightly smaller than the smallest determined link utilization. This procedure is repeated until the solution of the reduced equation system is converging. In case the equation system is not converging within a predetermined time interval or no solution of the equation system is existing, the routing determined by the last solution of the equation system is considered as sub-optimal solution of the linear optimization problem.

[0102] Path-Oriented Approach

[0103] As mention before it is possible to formulate the problem as a path-oriented approach. The method is equal to the flow oriented approach with the following changes:

[0104] Variable:

[0105] The Boolean variable p_k^uv is introduced. This variable is set to 1 if the data flow uv from the sending node u to a receiving node v is routed over the data path k, otherwise the variable is set to 0. k is a index which is equal to u, b, . . . , v where u, b, . . . , v are notes which are building the path.

[0106] Objective Function:

[0107] Similar to the flow oriented approach the objective function of the path oriented approach includes two additive parts both of which have to be minimized: $a_i*t+\frac{1}{L}\underset{\mathrm{ij}}{}\frac{1}{c_{\mathrm{ij}}}\underset{\mathrm{uv}}{}\underset{k,\mathrm{ij}k}{}{f}_{\mathrm{uv}}{p}_k^{\mathrm{uv}},\mathrm{with}{c}_{\mathrm{ij}}>0$

[0108] Transport Constraints:

[0109] The transport constraints ensure that one loop-free data path form the sending note u to a receiving node v exists. $\underset{k}{}{p}_k^{\mathrm{uv}}=1,$

[0110] for all k possible paths from u to v

[0111] Capacity Constraints:

[0112] The sum over the flows on each link must be smaller then the capacity of the link $\underset{\mathrm{uv}}{}\underset{k,\mathrm{ij}k}{}{p}_k^{\mathrm{uv}}{f}_{\mathrm{uv}}{a}_c{c}_{\mathrm{ij}},$

[0113] for all data links ij with cij>0

[0114] and also smaller than the maximum link utilization t $\underset{\mathrm{uv}}{}\underset{k,\mathrm{ij}k}{}{p}_k^{\mathrm{uv}}{f}_{\mathrm{uv}}\frac{t}{100}{c}_{\mathrm{ij}},$

[0115] for all data links ij with cij>0

[0116] Routing Constraints:

[0117] If p^uv define a path from u to v and p^kv define a path for k to v and both paths are routing over i, both paths are identical for the path i to v, thus

[0118] This routing constraints can be written as a recursive iteration over the index parameter k. E.g. a path from u to v is routed over a, b and c. Thus, k is defined as ku, a, b, c, v thus the flow from a to v must use the path a, b, c, v and also the flow from b to v must use the path b, c, v and so on. This recursive iteration is also true in the opposite direction of the index parameter k, e.g. ku, a, b, c, v then also the flow from u to c must use the path u, a, b, c and so on.

[0119] Specification of the Link Costs

[0120] In order to specify the data link costs for the routing determined by the equation system or the reduced equation system, the data link costs have to be determined such that the found routing and the optimized data path, respectively, corresponds with the shortest path routing. By the way of example, this can be achieved as follows:

[0121] 1. Additional Constraints:

[0122] To obtain the link costs as part of the solution of the first linear program, the following additional constraints have to be subjoined to the linear program: $L_k^{\mathrm{uv}}=\underset{\mathrm{ij}k}{}{I}_{\mathrm{ij}}$

[0123] The additional variable I_ij specifies the interface or link cost value of the link between i to j, whereas L_k^uv defines the overall cost of the k^th path from u to v.

[0124] This inequality guaranties that, if path s is chosen as the shortest path, the path s has the smallest overall cost value from u to v. If path s is chosen as the shortest path then p_s^uv1 and thus the inequality L_k^uvL_s^uv 0 has to be fulfilled. Otherwise the difference between L_k^uvL_s^uv must be greater than i_max.

[0125] The cost value of the link or interface should be between 1 and a value i_max. i_max determines the maximum cost value for a link or an interface. For example, i_max is set to the maximum value specified in the routing protocol (e.g. for OSPF i_max is 65535) for the link or interface cost value. To save calculation time it is also possible to choose a smaller value, for example on the basis of the following rule of thumb:

[0126] It is not necessary to modify the objective function. But, to minimize the cost values the additional term $a_g\underset{\mathrm{ij}}{}{I}_{\mathrm{ij}}$

[0127] can be added to the objective function. a_g defines the importance of this term in the objective function and may be choosen (much) smaller than 1.

[0128] 2. Equation System for a Linear Optimization Problem:

[0129] A further possibility is the specification of the link costs by an equation system for a linear optimization problem. In this case, the objective function is rather unimportant since the desired solution for the data link costs is obtained by computing an inequality system. A linear optimization problem is represented by an inequality system in combination with an objective function. Therefore, the determination of the interface costs can be calculated using an equation system for a linear optimization problem. As a result, a standard solver for linear optimization problems can be used for the determination of the data link costs.

[0130] The variable m_ij represents the cost of interface ij. The shortest path from u to v is the path with the least cost. The path cost is the sum of the cost values of the single interfaces. For the flow from u to v the cost m^uv is given by $m^{\mathrm{uv}}=\underset{\mathrm{ij}}{}{m}_{\mathrm{ij}}{x}_{\mathrm{ij}}^{\mathrm{uv}}$

[0131] where x_ij^uv describe the paths of the previous routing optimization. m^uv has to be the minimum cost of all possible paths from u to v. Therefore, for each loop free path p other than the optimized path p_opt^uv, a constraint has to be specified that forces the cost of p_opt^uv to be the smaller one: $\underset{\mathrm{ij}}{}{x}_{\mathrm{ij}}^{\mathrm{uv}}{m}_{\mathrm{ij}}<\underset{\mathrm{ij},{}_{\mathrm{ij}}\left(p\right)1}{}{m}_{\mathrm{ij}}$

[0132] To include all such constraints for all paths would lead to a very complex problem for larger networks. However, not all of these constraints are required to obtain a suitable solution. For a flow from u to v the mandatory constraints are identified by the following algorithm:

[0133] Consider each neighbor router i of u:

[0134] if p_opt^uv and p_opt^iv have no common node except v, add the constraint: $\begin{array}{cc}\underset{\mathrm{st}}{}{x}_{\mathrm{st}}^{\mathrm{uv}}{m}_{\mathrm{st}}<m_{\mathrm{ui}}+\underset{\mathrm{st}}{}{x}_{\mathrm{st}}^{\mathrm{iv}}{m}_{\mathrm{st}}& \left(12\right)\end{array}$

[0135] else add no constraint.

[0136] The algorithm is explained using the example of FIG. 13. Every path from u to v runs over a neighbor router of u. In the example the three neighbors of u are i, j, and additionally the first router of p_opt^uv. Three types of paths exist between a router n, which is adjacent to u, and v:

[0137] Optimize path p_opt^nv without a node in common with p_opt^uv, in the example (FIG. 13) the solid gray line between i and v.

[0138] Optimize path p_opt^nv with a node in common with p_opt^uv, in the example (FIG. 13) the solid gray line between j and v.

[0139] A non-optimal path from n to v, in the example the dashed gray line.

[0140] Following the above algorithm a constraint is necessary only for the path including p_opt^iv. The constraints concerning the other paths are already fulfilled by the constraints of other flows.

[0141] In the case of the path p_g, indicated by the dashed gray line between i and v, the constraint $\underset{\mathrm{ij}}{}{x}_{\mathrm{st}}^{\mathrm{iv}}{m}_{\mathrm{st}}<\underset{\mathrm{st},{}_{\mathrm{st}}\left(p_g\right)=1}{}{m}_{\mathrm{st}}$

[0142] already exists when flow form i to v is considered. This constraint together with equation 12 implies that the cost of the path using p_g is larger than the cost of p_opt^uv.

[0143] Considering the path between j and v, the part between k and v is common with p_opt^uv. According to the routing principles we know that the shortest path from u to k is routed along the black line. For this flow from u to k the constraint exists that the path along the black line p_opt^uk has to be shorter than the path over j and then along the gray line. But this constraint implies that p_opt^uv is shorter than the path over j and then along the gray line.

[0144] If the solution of such an inequality system shows that for all data paths except the desired/optimum data path the sum of the data link costs of the data paths is larger than the sum of the link costs of the desired/optimum data path, the desired/optimum data path is corresponding with the shortest data path.

[0145] To demonstrate the effect of the method according to the invention, the results of an optimized routing for example networks is described in the following. The used networks are shown in FIG. 4. The network on the right of FIG. 4 with only 6 nodes/routers was chosen because its size allows to illustrate the resulting paths. The networks having 8 and 14 nodes/routers were selected because of their complex structure which makes many different data paths possible. As example, the link capacity matrix C and the data flow matrix F for the network having 6 nodes/routers is shown in tables 1 and 2.

TABLE 1
Capacity matrix C for the network having 6 nodes/routers
	0	1	2	3	4	5
0	0	120	0	100	0	0
1	110	0	80	0	0	0
2	0	90	0	100	0	70
3	130	0	80	0	90	0
4	0	0	0	70	0	120
5	0	0	75	0	125	0

[0146]

TABLE 2
Flow matrix F for the network having 6 nodes/routers
	0	1	2	3	4	5
0	0	5	6	7	4	7
1	4	0	6	4	8	5
2	6	8	0	4	6	4
3	8	9	4	0	5	6
4	5	7	8	5	0	3
5	5	7	6	8	4	0

[0147] The physical delays of the different data links were set to 1, while the parameter ar was set to 3 in every optimization. Using the above described pre-solver, for example, the number of variables for the network having 14 nodes/routers was reduced from a value of 8800 to a value of 4714. Moreover, the number of constraints could be reduced to less than the half.

[0148] For all networks, optimizations with several parameter settings were performed. First the optimization was performed without an effective upper threshold for the maximum link utilization. This was achieved by omitting the constraints keeping the maximum link utilization below the value of the parameter t (as described in section Sub-optimal optimization) and by setting the parameter a_c to a value of 10. The resulting routing for the network having 6 nodes/routers is illustrated in FIG. 5. The data link having the highest utilization of 42.9% is the data link 4-3. The value for the minimized average link utilization was calculated to 22.4%.

[0149] Then, the parameter a_c was set to a value of 0.4 (40%) being slightly smaller than the calculated maximum link utilization of 42.9%. Iteratively reducing the parameter a_c, namely to values of 0.375 (37.5%) and 0.36 (36%), the optimum routing was found for the parameter a_c having the value of 0.36 which corresponds with the result of the default optimization.

[0150] In the default optimization, using the objective function (1), the parameter at was set to a value of 1000. The result of this optimization for the network having 6 nodes/routers is illustrated in FIG. 6. Again, the most utilized data link is the data link 4-3, however, its utilization was decreased to a value of 35.7%. As a compensation, the average link utilization was increased to a value of 22.70%.

[0151] As shown in FIGS. 5 and 6, the data flow 5-0 was taken away from data link 4-3 and routed over nodes/routers 2 and 1. Therefore, the data path for the data flow 2-0 was also routed over node/router 1. Further, data flow 2-4 was routed over node/router 5 instead over node/router 3, in order to reduce the link utilization of data link 3-4 from a value of 40% to a value of 35.7%.

[0152] On the basis of the above described determination of data link costs, the data link costs as small as possible were specified such that the shortest path routing is equivalent to the routing defined by the above optimization. FIGS. 7 and 8 illustrate the calculated data link costs wherein the respective data link costs and the respective physical delays are drawn in the gray boxes. As shown in FIG. 7, the overall data link costs for data flow 5-0 is 3. Other possible data paths for data flow 5-0 include nodes/routers 2 and 1 or nodes/routers 2 and 3. The overall physical delay for these paths are 5 and 4, respectively. The shortest path is the same as the desired path having a minimized average link utilization.

[0153] In FIG. 8, based on an optimization with respect to a minimized maximum link utilization, the physical delay for the data path of data flow 5-0 is 3, also. The physical delays of the other possible data paths over nodes/routers 4 and 3 or over nodes/routers 2 and 3 are 4 and 6, respectively. Again, the shortest data path is corresponding with the desired/optimum data path.

[0154] The optimizations of data routings for the networks having 8 and 14 nodes/routers was performed similarly. First, the average link utilization was minimized without restriction for the maximum link utilization. Then, the maximum link utilization was decreased iteratively. At the end, the optimization was performed to obtain the minimal possible maximum link utilization. However, the network having 14 nodes/routers proves to be complex to obtain an optimal solution. The maximum link utilization was reduced to 38.8%, while the minimal value for the maximum link utilization is not known. However, the minimal link utilization has to be greater than a value of 32%, since for this value the equation system for the linear optimization problem proved to be infeasible.

[0155] In FIGS. 9 and 10, the results for the routing optimizations with selected upper thresholds for the maximum link utilizations for the networks having 8 and 14 nodes/routers are shown. The maximum link utilization could be decreased, while the average link utilization almost remains unchanged. As represented by the bars identifying the difference between link utilizations of the most and the least utilized data links, the data traffic distribution becomes more homogeneous with a stricter upper threshold of the maximum link utilizations. In particular, the difference could be reduced to almost a quarter of the network having 8 nodes/routers (see FIG. 9). For the network having 14 nodes/routers, the optimization effect is less clear since in this network links are existing with an utilization of only about 1% independent of the selected upper threshold for the maximum link utilization (see FIG. 10).

[0156] Since the optimization results in a reduction of the highest value for the maximum link utilization, link utilization of specific links have to be increased as a compensation. As illustrated in FIG. 11, the link utilization of selected data links is shown for different upper thresholds for the maximum link utilization. The link utilization of data links initially highly utilized is decreased for a reduction of the upper threshold for the maximum link utilization. This applies for example for data links 9-4, 4-9 and 4-2. In consequence, the link utilization of data links 8-5, 5-8 and 3-1 is increased.

[0157] Measure the End-to-End matrix (e.g. SNMP, Accounting/Billing Software) and collect the data on a central system (INTERNET-server)

[0158] Build from the collected data the capacity, flow, delay matrix

[0159] Use the method to calculate the optimal routing parameter.

[0160] Configure the router with the optimal parameter over the router configure software.

[0161] For smaller networks it's possible to collect the needed information to build the matrixes with the routing protocol (e.g. change the routing protocol or use a extra protocol) and calculate the optimal routing parameters in every router. In this case no central unit is needed.