Quality of service based path selection for connection-oriented networks

[0002] This invention relates generally to the field of telecommunications and in particular to a more efficient method for selecting a transmission path in a connection-oriented network.

[0003] It is apparent that connection-oriented networks will play an increasing role in data networking. Connection-oriented networking offers important advantages over connectionless networks, including the advantage of providing Quality of Service (QoS) guarantees, which facilitate new classes of applications such as multimedia, voice over internet protocol, and other real-time services.

[0004] A connection-oriented network includes of a set of switches interconnected by point-to-point links or interfaces. The switches support two kinds of interfaces: user-network interfaces (UNI) and network-to-network or network-node interfaces (NNI). A UNI connects end-systems (hosts, routers, and so on) to a switch, while an NNI may be imprecisely defined as an interface connecting two switches together. More precisely, however, an NNI is any physical or logical link across which two switches exchange the NNI protocol. Various NNI protocols, such as the Private Network-Network Interface (PNNI) protocols designed for asynchronous-transfer-mode networks, include a routing protocol that exchanges metrics, such as available and maximum bandwidth, cell loss ratio (CLR), cell delay variation (CDV), and cell transfer delay (CTD), related to Quality of Service (QoS). Routes are then computed using the data collected by the routing protocol. Most commonly-used route determination algorithms (such as Dijkstra shortest path algorithm) use a single, cumulative metric such as link weightings (i.e., administrative weight, hereinafter AW) or costs that relate to the efficient use of the network.

[0005] For example, cell loss ratio (CLR) is one of the metrics, or measurements, that can used to determine whether a given network cell, consisting of a link or node or concatenation of links and nodes, can meet the requested quality of service (QoS). Cell delay variation (CDV) and cell transfer delay (CTD) are other such metrics or measurements. In contrast, administrative weight (AW) is a measurement of cost that relates to the efficient use of the network.

[0006] Other path selection algorithms, as the one proposed in the PNNI v1.0 standard, use a simple Generic Connection Admission Control (SGCAC) procedure to account for bandwidth. However, neither the SGCAC nor the complex GCAC proposed in PNNI v1.0 is effective in providing the QoS guarantees expected by many data networking customers.

[0007] In more detail, the problems in connection-oriented data networking arise out of the following typical existing situations.

[0008] As an example of routing and signaling in a connection-oriented network, the PNNI signaling protocol is used to set up asynchronous-transfer-mode switch connections between the source end-system and the destination end-system. The UNI signaling request is mapped into PNNI signaling at the ingress switch. The PNNI signaling is remapped back into UNI signaling at the egress switch. The PNNI signaling protocol is an extension of the UNI signaling and incorporates additional Information Elements (IEs) for PNNI related parameters such as the Designated Transit List (DTL). The PNNI routing protocol is used to disseminate topology, loading conditions and reachability data. For scalability reasons, the notion of hierarchical peer groups is created with Peer Group Leaders (PGLs) elected for each peer group. The PNNI routing protocol is used for hierarchical aggregation and propagation of information. Data about a peer group is aggregated by the peer group leader, which then distributes this data to all the nodes in the higher-level peer group in PNNI Topology State Packets (PTSPs). Thus, aggregated data is passed “up” the hierarchy by peer group leaders. Nodes in a peer group receiving data sent by one of their peer nodes at the higher levels of hierarchy distribute the received data “down” the hierarchy. This allows a switch to maintain a view of the “global” topology with reachability information for endpoints.

[0009] The PNNI routing protocol supports a large number of link and node state parameters that are advertised by the switches to indicate their current state at regular intervals which enables QoS-sensitive routing. This includes two types of parameters: attributes, such as Available Cell Rate, Cell Rate Margin, Variance Factor, used to determine whether a given network link or node can meet requested QoS; and metrics, such as maxCDV, CTD, CLR, AW, that are used to determine whether a given path, consisting of a set of concatenated links and nodes (with summed link metrics), can meet the requested QoS. The link metrics and attributes are per port per service category. Individual nodes (physical or logical) will need to determine and then advertise the values of these parameters for themselves.

[0010] As a result of these exchanges, a topology database is created, which has reachability information for all the network nodes and the available resources on the network links and nodes. All network nodes can obtain an estimate of the current state of the entire network. Unlike most current link state protocols, the PNNI routing protocol advertises not only link metrics but also nodal information.

[0011] When a switch receives a connection setup request, a number of steps are executed. Among these are Connection Admission Control (CAC), Generic Connection Admission Control (GCAC) and shortest path computation. CAC is used to determine if the requested connection can be admitted without violating QoS guarantees of established connections. CAC is a local switch function, and is, dependent on the architecture of the switch and local decisions on the strictness of QoS guarantees required by the switch. The other two algorithms, GCAC and shortest-path computation, are used by the ingress node receiving a connection setup request to determine a shortest-path hierarchical route for the connection while ensuring that all the links on the path have the available cell rate (bandwidth) requested for the connection. This form of routing, where the ingress node determines the shortest-path for the connection, is referred to as source routing. The PNNI v1.0 standard specifies two GCAC algorithms: simple GCAC and complex GCAC. Either of these algorithms can be used to determine if each link on a selected path has the requisite available cell rate (bandwidth) for the connection. The shortest path computation is essentially an execution of a shortest-path algorithm, such as Dijkstra's or Bellman-Ford's, with the link and node weights set to those of the metrics advertised in the received topology information.

[0012] Since PNNI networks are hierarchical, the shortest-path computed by the ingress node (after applying Dijkstra's algorithm and the GCAC available cell rate check algorithm) does not specify the exact set of switches through which the connection must be routed. Instead, it specifies a stack of Designated Transit Lists (DTLs), one identifying the physical switches in its peer group, and others identifying logical switches in higher-level peer groups. The DTLs are passed as parameters in the SETUP message. This eliminates the need for intermediate nodes (except border nodes of peer groups) to perform route selection. The border node of each peer group is involved in route selection since nodes outside a peer group P do not have a detailed picture of the topology and loading conditions of the peer group P. The border node through which the connection setup enters the peer group computes a DTL for its peer group and places this on the stack of DTLs as it propagates the PNNI SETUP message.

[0013] Once a path has been selected for the connection, each node on the path executes CAC (Connection Admission Control) to ascertain that it can admit the connection without violating guarantees for existing connections. Each node in the path needs to perform its own connection admission control because its own state may have changed since it last advertised its state with the network used for the GCAC at the ingress or border node.

[0014] If the connection admission control procedure is successful, each node then programs the switch fabric with channel identifier translation information and with information for user-plane algorithms, such as scheduling, UPC (Usage Parameter Control), traffic shaping, etc.

[0015] The signaling protocol is used to send a message from node to node to complete connection setup. Once a connection is set up, user data packets are forwarded through each node according to the channel identifier translation information. Since the route selection phase (executing shortest-path and GCAC algorithms) could consume a significant amount of time, route precomputation is suggested in Appendix H of the PNNI v1.0 specification. Precomputing routes could potentially speed up the route selection process. With precomputed paths, a connection setup request is handled by the ingress switch first determining the destination switch (logical node) through which to reach the called endpoint. If a precomputed route exists to the destination switch, the GCAC algorithm is applied to each link of the precomputed path to check for cell rate availability.

[0016] If this test passes, the route selection step is complete. If not, an “on-the-fly” route selection, consisting of both shortest-path and GCAC algorithm executions, is needed. Again, each node in the path needs to perform its own connection admission control because its own state may have changed since it last advertised its state within the network used for the GCAC at the ingress or border node.

[0017] The new services and reduced rates made possible by such peer groups of networks and nodes have been very attractive to users, but poorly-controlled delays and occasional failure of connectability have been drawbacks.

[0018] In the copending patent application of two of us with others, Ser. No. 09/188,023, filed Nov. 6, 1998, a dynamic programming method is disclosed for determining the length of a shortest path from a source node to any of v other nodes in the network with delay equal to an amount less than a specified time. A table is then built for such shortest-path routes for different specified delay times, so that a user may specify a maximum delay time as one of the requirements for quality of service. Because of the demands for computer time made by this method for large and complex networks, it is desirable to improve the method to be more efficient in use of computer time, particularly when the table must be supplemented by “on-the-fly” calculations.

[0019] According to the present invention, a more efficient delay-based path selection is provided in connection-oriented data networking. The connection-requesting user not only specifies a delay threshold T for the incoming request but also specifies a delay threshold tolerance ε within which the path delay should be satisfied. The method of the invention accesses stored values of the topology of the network and delay-related parameters for a set of links between different ones of the plurality of packet switches and provides to the user a delay-based shortest-path connection to one or more other users via at least one of said packet switches. The connection is provided by testing combinations of links by scaling the permissible delay threshold to a value τ, appropriately chosen to ensure the user-specified delay threshold is satisfied to within the user-specified tolerance, correspondingly scaling the, delay measure on each link, and then truncating all the scaled values to integers, and accumulating the scaled link delays for the connection using a dynamic programming algorithm. The method exploits the new user specifications and computes paths from one source to all destinations in a coamputationally efficient manner.

[0020] In a first non-iterative implementation of the present invention, τ is set to min (T, (n−1)/ε), where T and ε are respectively the delay threshold and the threshold tolerance specified by the user, and n is the number of nodes in the network. The dynamic programming algorithm computes Table L(v, τ) and Table P(v, τ), where L is the length of the path from a given node to node v and P encodes the shortest path from the given node to node v whose delay is no more than τ. The method then computes the delays for each link in the original data for the topology of the network for each of the paths P(v, τ) and store the delays in a table D(v). The algorithm provides outputs L(v, τ) and D(v) and terminates.

[0021] The time complexity of this algorithm is min(ET, E(n−1)/ξ), and thus can be considerably faster than the dynamic programming algorithm used without scaling.

[0022] In a second implementation of the invention, which is the iterative scaling algorithm, the time complexity is further reduced to 4ED/ξ, where D is the maximum hop length of the optimal path, and E is the number of links in the network. The method of this implementation first sets τ=τ₀(a small number <<T) and scales all of the link delays. The dynamic programming algorithm is then used to compute Table L(v,τ) and Table P(v, τ).

[0023] The method then computes the delays for each path P(v, τ) using unscaled original link delays and stores the delays in a table D(v). If the condition exists that D(v)>T(1+ε) and 2τ<n/ε, set τ=2τ, rescale all delay parameter values, truncate them to integers, and recompute Table L(v,τ) and Table P(v,τ), else copy L(v,τ) to L(v) and P(v,τ) to P(v) and terminate.

[0024] The table calculated during the last iteration of DSA (Dynamic Scaling Algorithm) contains the constrained shortest paths.

[0025] The method according to either implementation of the present invention has two compelling advantages. It generates paths from one source to multiple destinations, an attribute that is critical in designing route caching algorithms that improve the grade of service. Further, it greatly improves computational speed.

[0026] Further features and advantages of the present invention, as well as the structure and operation of various embodiments of the present invention are described in detail below with reference to the accompanying drawing.

[0027] The teachings of the present invention can be readily understood by considering the following detailed description in conjunction with the accompanying drawings, in which:

[0028] FIGS. 1 and 2 show separable parts of a flow diagram for the non-iterative scaling method according to the present invention;

[0029] FIG. 3 shows a flow diagram including selection between iterative and non-iterative scaling, with the steps for each, and also illustrates the truncating aspect of the invention.

[0030] FIGS. 4A and 4B show an additive-delay-based path selection algorithm;

[0031] FIG. 5 shows a flow chart for the precomputation of routing tables in a flat network using the algorithm of FIGS. 4A and 4B;

[0032] FIG. 6 shows a call setup procedure at a switch for a flat network using routing data created according to FIGS. 4A, 4B, and 5;

[0033] FIGS. 7A and 7B show an asymptotic-delay-based path selection algorithm;

[0034] FIG. 8 shows a flow chart for the precomputation of routing tables in a flat network using the algorithm of FIGS. 7A and 7B;

[0035] FIG. 9 shows a call setup procedure at a switch for a flat network using routing data created according to FIGS. 7A, 7B, and 8;

[0036] FIG. 10 shows a flow chart for the precomputation of routing tables in a hierarchical network using the algorithm of FIGS. 4A and 4B;

[0037] FIGS. 11A and 11B show a call setup procedure at a switch for a hierarchical network using routing data created according to FIGS. 7A, 7B, and 8;

[0038] FIG. 12 shows a flow chart for the precomputation of routing tables in a hierarchical network using the algorithm of FIGS. 7A and 7B;

[0039] FIGS. 13A and 13B show a call setup procedure at a switch for a hierarchical network using routing data created according to FIGS. 7A, 7B, and 12;

[0040] FIG. 14 shows a system diagram in block diagrammatic form for a hierarchical network using the PNNI telecommunications standard; and

[0041] FIG. 15 shows a general description of a hierarchical telecommunication network.

[0042] A preferred embodiment of the invention will now be described while referring to the figures, several of which may be simultaneously referred to during the course of the following description.

[0043] The following technical background will be helpful in understanding the terminology that is definitive of the preferred implementations of the present invention.

[0044] Shortest-path algorithms are applied with additive metrics, while GCAC is applied with a non-additive metric. The question of whether maxCTD (maximum Cell Transfer Delay) and peak-to-peak CDV(α) (the α quantile of the Cell Delay Variation) are additive metrics or not, and hence whether they should be included in the shortest-path determination phase or GCAC phase has not been addressed. Appendix H of the PNNI v1.0 standard specification indicates that delay could be used in the shortest-path computation phase implying that it is an additive metric. Nevertheless, if a weighted average of CTD and AW metrics is used, the solution may not meet the end-to-end CTD constraint. This would require a second step to test whether the “shortest” path meets the end-to-end CTD requirement.

[0045] Furthermore, CTD has been identified as being a non-additive metric. Simply adding the CTD contributions from the switches on a path leads to a very pessimistic estimate of the end-to-end CTD.

[0046] Some of the delay metrics that could be used in implementing the present invention include the following. Cell Transfer Delay (CTD) is defined as the elapsed time between a cell exit event at the source UNI and the corresponding cell entry event at the destination UNI for a particular connection. The Cell Transfer Delay is, therefore, the sum of the total inter-ATM node transmission delay and the total ATM node processing delay in the path. Two end-to-end delay parameter objectives are negotiated: Peak-to-Peak CTD and maxCTD. Peak-to-peak CTD (Cell Transfer Delay) is defined as the cc quantile of the CTD minus the fixed CTD. The term peak-to-peak refers to the difference between the best and the worst case of CTD, Where the best case equals to fixed delay and worst case equals to a value likely to be exceeded with probability no greater than (1−α).

[0047] In the preferred implementation of the invention described hereinafter, dynamic scaling is applied facilitate the use of delay-based route selection to help satisfy quality-of-service standards. Dynamic scaling and truncating of scaled values to integers to shorten calculations is supported by the systems and techniques now to be described.

[0048] FIGS. 1 and 2 show separable parts of a caching algorithm used in scaling according to the present invention. This algorithm deals with a problem of computing constrained shortest paths from one source to multiple destinations. The algorithm provides for off-line route cache computation and on-line route computation. The K paths in the route cache for each destination are indexed by 1, 2, 3, k, . . . K. Let RC_kbe the delay threshold of path k. The paths are ordered so that RC_k>RC_k+1, so that the paths are ordered in descending order of delay thresholds.

[0049] When a connection request arrives with the delay requirement of T and the traffic descriptors indicated as leaky bucket values, step S100 computes an effective bandwidth for the connection. This effective bandwidth is computed based on conservative assumptions to avoid the rejection of this call by the connection admission control of a downstream switch. Step S200 checks if T is <RC_K, the smallest cached delay. If ‘yes’ a cache miss results, and an on-line computation is needed. If ‘no’, a path is selected from the cache. In step S300, if T>=RC₁, the largest cached delay, then a running index p is set to 1 (step S350). Otherwise, in step S400 running index p is found such that RC_(p−1)>=T>=RC_p. In step S600, it is checked whether the path in cache position p has a larger bandwidth than the previously set required bandwidth. If ‘yes’, a cache hit results, the call is accepted, the connection is set up, and the counter for p is reset to zero (step S700). If ‘no’, a cache miss results, and, for either cache miss, the on-line calculation of FIG. 2 is required.

[0050] In FIG. 2, an initial decision must be made in step S800 whether the remaining time for calculation is so short that length scaling, without delay scaling, must be used, or whether the indicated delay scaling (here called time scaling) can be used. If so, step S800 includes the modification of the dynamic programming algorithm of FIG. 4A and 4B described below in connection with Equation 6. The most recent network topology information is used. If the result of the on-line calculation is that a path is found (‘yes’ in step S900), step S1000 invokes general connection admission control. Step S1100 determines whether the characteristics of the call meet the criterion according to the collected histogram for inclusion into the route cache. If a Route Cache Hit Ratio (CFR) is defined as the fraction of cache look ups that find a path to the desired destination and if the Cache Path Success Ratio (CPSR) is defined as the fraction of times the path in the route cache passes the bandwidth test, then the real-time budget for on-line computation is $\begin{array}{cc}\frac{1.0}{\mathrm{Call}\mathrm{rate}*\left(1.0-\mathrm{CHR}\right)*\left(1.0-\mathrm{CPSR}\right)}& \left(4\right)\end{array}$

[0051] If the delay scaling algorithm's complexity is less than the real-time budget, it is chosen because of its desirable property of computing paths to all destinations. Otherwise, the length scaling algorithm is used. It can also be used if the characteristics of the call are rare or there is a destination correlation.

[0052] Step S1100 gives a ‘yes’ answer if delay scaling is used for the on-line calculation and if the collection of just-calculated paths meet a pre-existing policy for updating. If ‘yes’, the route cache is updated (step S1200), the counter reset (step S1300), and the call accepted. Even if the cache is not to be updated, the counter is reset, and the call accepted.

[0053] With respect to steps S900 and S1000, if a path is not found or in the case that GCAC fails (not shown), step S950 updates the computation histogram (the p counter is reset), and the call is rejected.

[0054] FIG. 2 also shows steps S1400 and S1500 for updating the route cache off-line, based on changing network topology and current network state. The updating would be done so as not to interfere with on-line computations.

[0055] According to separate aspects of the present invention, both a method and an apparatus include such delay metrics in the route selection process, implemented with improved efficiency according to the present invention, in either cases implemented to satisfy FIG. 1 and FIG. 2.

[0056] The scaling and truncating aspect of the invention, as implicit in step S800 of FIG. 2, will now be further explained with reference to FIG. 3.

[0057] In FIG. 3, step S2100 selects iterative or non-iterative scaling, for example, based on prior experience with the type of connection requested.

[0058] If non-iterative scaling is selected, step S2200 sets τ to min (T, (n−1)/ε) and, in step 2300, the dynamic programming algorithm runs. In step 2400 it computes Table L(v, τ), and in step 2500 it computes Table P(v, τ), where L is the length of the path from a given node to node v and P encodes the shortest path from the given node to node v whose delay is no more than τ. In step 2800, the method then computes the delays for each link in the original data for the topology of the network for each of the paths P(v, τ) and store the delays in a table D(v). For the totaled values of the scaled, truncated, and accumulated delay parameter that satisfies the above condition, step 2600 copies L(v, τ) to output L(v) and step 2700 copies P(v, τ) to output P(v) and terminates.

[0059] In FIG. 3, in the iterative-scaling path, step 3200 sets τ=τ₀(a small number <<T). Step 3250 scales all link delays according to t_e=t_e×τ/T. The dynamic programming algorithm is invoked in step 3300. Step 3400 computes Table L(v,τ) and step 3500 computes Table P(v,τ).

[0060] The method then computes the delays for each link in the original data for the topology of the network for each of the paths P(v, τ). Step 3500 stores the delays in a table D(v).

[0061] If the condition exists that D(v)>T(1+ε) and 2τ<n/ε, as tested in step 3700, step 3750 sets τ=2τ and re-enters step 3250 to rescale all delay parameter values, truncates them to integers, and recomputes Table L(v,τ) and Table P(v,τ), else copy L(v,τ) to L(v) and P(v,τ) to P(v) and terminates.

[0062] The table calculated during the last iteration of DSA (Dynamic Scaling Algorithm) contains the constrained shortest paths.

[0063] A dynamic programming algorithm such as may be used in FIG. 3 will now be explained with reference to FIGS. 4A and 4B. This algorithm involves the additive delay accumulation method of the above-referenced copending patent application and is readily applied to the same parameters as the present invention. An overview is now given. A switch at an additional node to be connected in the path receives accumulated peak-to-peak CTD and adds its own contribution of the peak-to-peak CTD (α) to the accumulated peak-to-peak CTD. This approach is based on estimating the end-to-end CTD (α) as the sum of individual CTD (α) values along the path from source to destination. If there are N switches along the path and if we denote the (α) quantile of CTD in switch i by CTD_i(a), then the total accumulated CTD is: $\begin{array}{cc}\mathrm{CTDtotal}\left(\alpha \right)=\sum _{i=1}^N\mathrm{CTDi}\left(\alpha \right)& \left(1\right)\end{array}$

[0064] This simple method requires only one parameter, CTD, for its computation. The estimated CTD is always an upper bound of the actual CTD but it may be very conservative for connections that traverse many hops.

[0065] As described with reference to FIG. 4A through FIG. 13, this algorithm is not essential to the present invention, and could be replaced by another dynamic programming algorithm, but provides a useful tutorial for the reader.

[0066] In more detail, with reference to FIGS. 4A and 4B, the constrained shortest path problem is solved using dynamic programming procedures assuming a discretized and bounded domain for the CTD and performing an exhaustive search over it.

[0067] Define l_ijand t_ijto be the AW and the CTD of the link that connects nodes i and j, respectively. Let T be an upper bound (delay constraint up to T) on the end-to-end CTD of any path in the network and f_i(t) the length of a shortest path from node 1 to node i with CTD less or equal to t.

[0068] Step 11 in FIG. 4A includes the following steps: (See R. Hassin, Approximation Schemes for the Restricted Shortest Path Problem, Mathematics of Operations Research, Vol 17, No. 1, February, 1992, pp. 36-42.)

[0069] 1. Initialize f₁(t)=0, t=0, . . . , T

[0070] 2. Initialize f_j(0)=∞, j=2, . . . , N, where the ∞ implies a number large enought that a sufficiently exhaustive search of possible links can be made.

[0071] Steps 12-17 represent the sequencing of variables in the following command:

[0072] 3. Compute f_j(t)=min{f_j(t−1),min_k|tkj≦t{f_k(t−t_kj)+l_kj}}j=2, . . . ,N,t=1, . . . ,T

[0073] Decision circuit 19 determines whether there is a link from node k to node j. In the latter event, step 21 retrieves link delay across link kj and link length from a topology data base for the peer group of networks (See FIG. 14.). So long as the retrieved link delay is greater than the previously accumulated value t, decision circuit 22 allows the process to increment k (Step 26).

[0074] As seen in the lower part of FIG. 4A, index circuit 23 and decision circuit 24 allow the process to increment k if a function of the index and the linklength is not less than a prescribed value MIN. If the function is less that MIN, MIN is set equal to the function, at Step 25. Then, and also by repeat of steps 19-25 when k becomes greater than N (decision circuit 27), decision circuit 28 tests whether f(j)(t−1) is greater than MIN. If it is, then f(j)(t) is equal to MIN (Step 29); and node min k is in the path to node j (step 30). If not, then f(j)(t) is set equal to f(j)(t−1)(step 34).

[0075] If delay has not reached the maximum T, then step 31 of the process allows t to increment. If delay has reached T, step 32 tests whether j has, reached N. If not, j is incremented. If j has reached N, the process is stopped (Step 33).

[0076] The complexity of the above algorithm is O(n1T), with i and 1 the number of nodes and links in the network, respectively. Compare this with Dijkstra's algorithm, of O(n²). It is shown in the above-referenced copending patent application that the constrained shortest path algorithm is consistently slower than Dijkstra's algorithm and does not scale well for large and dense networks. The configuration of FIG. 14 uses it only for “precomputations”, where these considerations are not a problem, rather than “on-the-fly” computations.

[0077] The delay scaling method according to the present invention attempts to balance a tradeoff between scaling down the delay requirement and the inaccuracy introduced by truncating scaled delays to integers. Specifically, whenever the delay metric is scaled by multiplying the delay of each link by a factor of τ/T and then truncating to an integer, a shortest path of delay no greater than τ is equivalent to delay no greater than

[0078] T(1+(n−1)/τ)

[0079] obtained according to the method of the above-cited copending patent application. The introduction of the increment (n−1)/τ is due to the truncating to an integer for up to (n−1) links.

[0080] As set out in the above-cited application, the delay-scaling algorithm DSA (G, T, ε), defined below, finds an approximate solution to the constrained shortest path problem.

[0081] More specifically, it returns a tree rooted at node 1 with the following preperties:

[0082] 1. The length of the path from 1 to any other node v on this tree is no more than the length of the shortest path from 1 to v satisfying the delay constraint T.

[0083] 2. The delay of the path from 1 to v is no more than T (1+ε).

[0084] For a Graph G of two variables to be mutually optimized, that is, path length (or network cost) and connection delay, to be specified by a user, the following can be demonstrated.

[0085] Refer back to the above-description of FIG. 1. A τ-scaling G_τ of Graph G is obtained by multiplying the delay on each edge in G by τ/T and then truncating the new delay to an integer. The corresponding delay constraint for the new scaled graph is τ.

[0086] The algorithm DSA is implemented on a digital central processing unit as follows:

[0087] ALGORITHM DSA—ITERATIVE METHOD

[0088] Inputs

[0089] Graph G with edge weights l_eand t_e

[0090] Delay constraint T

[0091] Relative error ε

[0092] Auxiliary algorithm DAD, a dynamic programming algorithm based on delays, illustratively, the algorithm of the above-referenced copending patent application.

[0093] Outputs

[0094] Table L(v), 1≦v≦n; the L(v) entry being the length of the shortest path for 1 to v whose delay is no more than T(1+ε).

[0095] Table P(v), 1≦v≦n; the entry P(v) encoding the actual shortest path, from 1 to v whose delay is no more than T(1+ε) and whose length is given by L(v).

[0096] Steps

[0097] 1. Set τ=τ₀(a small number <<T).

[0098] 2. Call DAD (G_τ, τ) to compute Table L(v,τ) and Table P(v,τ).

[0099] 3. Compute the delays in the original graph G, for each of paths P(v, τ) and store the delays in D(v).

[0100] 4. If ∃vεG: such that D(v)>T(1+ε) and 2τ<n/ε, set τ=2τ and go to step 2 else copy L(v,τ) to L(v) and P(v,τ) to P(v) and terminate.

[0101] The table calculated during the last iteration of DSA contains the constrained shortest paths. It can be shown that the worst case complexity of the algorithm is 4ED/ε, where D is the maximum hop length of the optimal path, and E is the number of inks in the network. Clearly D<n. The worst case running time of this algorithm is independent of the delay/length values and, hence, this algorithm scales well as these values change. Further, a single invocation of DSA finds the constrained shortest path from the source to all destinations.

[0102] The following observations about DSA are significant:

[0103] The time complexity result has a value D, but the algorithm does not need to know this value.

[0104] As mentioned earlier, a slightly weaker time complexity result is 4En/ε, but in most real-life cases when the paths have a hop-count restriction (for example, 5), a tighter complexity result is order(s) of magnitude better than the weaker result. DSA has a monotonicity property that is useful in practice. The accuracy of the solution monotonically improves with successive iterations. DSA can be terminated in the middle and still produce reasonable answers. For instance, when there is a call processing time budget during call setup, the execution can be terminated when time runs out, and a reasonably accurate solution will still be obtained. This monotonicity property does exist in the method of the above-referenced copending application or in other prior art methods seeking constrained shortest path results.

[0105] DSA is strongly polynomial, i.e., the running time bound does not depend on the actual delay and the length values. In contrast, the method of the above-referenced copending application or the other prior art methods seeking constrained shortest path results are pseudo polynomial and should be considered less robust.

[0106] As shown in FIG. 5, the algorithm of FIGS. 4A and 4B is used in the precomputation of routing tables in a flat (non-hierarchical) network. The flow diagram of FIG. 5 basically implements precomputation of “shortest path” routes using the shortest path algorithm described above and stores the routes for different values of CTD (or t) in a routing table or cache memory (See FIG. 14). Store the path(s) in a cache of the form (Traffic descriptor, CTD, route).

[0107] In more detail, in FIG. 5, the precomputation of routing tables starts with the selection of one (e.g., k=1) of the numbers and sizes of bandwidths (BW(k)) and cell delay variation (CTD(k)) options available for interconnection in the network. Thus, step 41 sets k=1, and step 42 selects BW(k) and CTD(k).

[0108] Step 43 then prunes the search of the network so that only links having bandwidth greater than BW(k) are investigated. Step 44 applies the algorithm of FIGS. 4A and 4B to find the shortest paths f satisfying the CTD(k) requirement from the node in which this algorithm is being executed to all other nodes in the network. Step 45 writes the result into the routing table. As indicated in steps 46-48, this procedure is repeated for each available bandwidth and cell delay variation that may appear in a call setup request.

[0109] FIG. 6 illustrates the call setup procedure invoked by a call setup request (Step 51) in a flat network.

[0110] When the call setup request arrives at a switch (See FIG. 14), step 52 selects an optimal path from the table of precomputed routes that satisfies both the CTD and BW constraints in the call setup request.

[0111] Step 53 checks whether a path was found. If so, in step 54, call admission control and switch configuration are performed, and a connection setup request to the next switch in the path is generated. If not, an on-the-fly procedure, starting at step 55, is invoked to find an appropriate path. Step 55 prunes the search so that network links with insufficient bandwidth to satisfy the call setup request are not searched. Before the on-the-fly search is actually commenced, step 56 employs a decision circuit to decide whether the use of the constrained shortest path algorithm of FIGS. 4A and 4B is allowed. The answer is “Yes” if its use is judged to be not too expensive in terms of time and resources. Then step 57 uses the algorithm of FIGS. 4A and 4B in real time. Otherwise the answer is “No”, and step 58 performs a Dijkstra shortest path algorithm with cell delay variation as the cost function. Once a path is found, the procedure is ended (step 59).

[0112] The second delay-based path-selection algorithm of the above-cited prior application employs delay accumulation calculated by the so-called asymptotic method. This is an existing measure that has heretofore been used, for other purposes. It uses both mean and variance of transfer delay, and the actual delay variation in each switch, in order to compute the end-to-end CTD. The intuition behind this is the central limit theorem. Irrespective of the distributions of individual random variables are, the sum of the random variables is normally distributed as N gets large. The error is compensated by adding the maximum difference between the estimate and actual CTD in the switches along the path. The end-to-end CTD over N switches, assuming independent delays in the switches, is given by: $\begin{array}{cc}\mathrm{CTD}\mathrm{total}\left(\alpha \right)=\sum _{i=1}^N{\mu }_i+\sqrt{\sum _{i=1}^N{\sigma }_i^2}\times t\left(\alpha \right)+\underset{1\le i\le N}{\max }\{\mathrm{CTD}i\left(\alpha \right)-({\mu }_i& \left(2\right)\end{array}$

[0113] where, t(α) denotes the a quantile of standard normal distribution, μ_idenotes the mean delay in switch i, and σ_idenotes the standard deviation of delay in switch i . The quantity

[0114] d(α)={CTDi(α)−(μi+σi×t(α))}  (3)

[0115] is referred to as discrepancy, d(α). This method is also an upper bound of the actual CTD but the bound is much tighter when compared with the additive method. The asymptotic method requires each switch to report four parameters: mean queuing delay (μ), variance of queuing delay (σ), discrepancy parameter d(α), and fixed delay f.

[0116] In an implementation of this approach, as set forth in FIGS. 7A and 7B, asymptotic-type delay accumulation is set forth in a algorithm that may be appreciated to be a variation of that of FIGS. 4A and 4B. The significant difference in the algorithm of FIGS. 7A and 7B as compared to that of FIGS. 4A and 4B resides in the use of the above four parameters. All other components and/or steps are the same or essentially similar to those in FIGS. 4A and 4B.

[0117] To modify the dynamic programming procedure for the constrained shortest path algorithm, proceed as follows: Define l_ijand t_ijthe AW and the CTD of the link that connects nodes i and j, respectively. Let L be an upper bound on the end-to-end delay of any path in the network. Let also g_i(l), μ_i(l), σ_i²(l) and d_i(l) the CTD, the (cumulative) mean of the delay, the (cumulative) variance of the delay and the discrepancy of the delay of a shortest path from node 1 to node i with AW at most 1.

[0118] 1. Initialize g₁(1)=0, 1=0, . . . , L

[0119] 2. Initialize g_j(0)=∞, j=2, . . . , N

[0120] 3. Initialize μ_j(l), σ_i²(l)=d_j(l)=0, 1=0, . . . ,L, j=2, . . . N

[0121] 4. Compute g_j(l)=min{g_j(l−1),min_k|lkj≦l{g_k(l−1_kj){circle around (x)}t_kj}}j=2, . . . ,N, =l1, . . . , L

[0122] With g_k{circle around (x)}t_kjis

[0123] μ_k(l−l_kj)+μ_kj+[{square root over (σ_k²()}l−l_kj)+σ_kj²]×t(α)+max {d_k(l−l_kj),d_kj(α)}  (6)

[0124] defined as:

[0125] where μ_kj, σ_kj, and d_kj(α) denote the mean, the variance and the discrepancy of link that connects nodes k and j.

[0126] In more detail, step 61 in FIG. 7A includes initialization of variables according to the four steps outlined above. Again, the ∞ implies a number large enough that a sufficiently exhaustive search of possible links can be made.

[0127] Steps 62-67 represent the sequencing of variables in the calculation of step 4 set out above. Decision circuit 69 determines whether a link exists from node k to node j. In the latter event, step 71 retrieves link delay across link kj and link length from a topology data base for the peer group of networks (See FIG. 14.). So long as the retrieved link length is greater than the previously accumulated value l, decision circuit 72 allows the process to increment k (step 76).

[0128] As seen in the lower part of FIG. 7A, index circuit 73 and decision circuit 74 allow the process to increment k if a function of the index and the link delay is not less than a prescribed value MIN. If the function is less that MIN, MIN is set equal to the function, at Step 75 (FIG. 7B). Then, and also by repeat of steps 69-75 when k becomes greater than N (decision circuit 77), decision circuit 78 tests whether G(j)(l−1) is greater than MIN. If it is, then G(j)(l) is equal to MIN (Step 79); and node min k is in the path to node j (step 80). If not, then G(j)(l) is set equal to G(j)(l−1) (Step 84).

[0129] Step 81 tests whether j=N. If not, j is incremented. In step 82, a decision circuit tests whether delay or length has reached a maximum. If not, l is allowed to increment (step 63 in FIG. 4A). If either has occurred, the process is stopped (Step 83).

[0130] As shown in FIG. 8, the algorithm of FIGS. 7A and 7B is used, according to the present invention, in the precomputation of routing tables in a flat (non-hierarchical) network. The flow diagram of FIG. 8 basically implements precomputation of “shortest path” routes using the shortest path algorithm described above and stores the routes for different values of CTD (or t) in a routing table or cache memory (See FIG. 14). Store the path(s) in a cache of the form (Traffic descriptor, CTD, route).

[0131] In more detail, in FIG. 8, the precomputation of routing tables starts with the selection of one (e.g., k=1) of the numbers and sizes of bandwidths (BW(k)), cumulative cell delay variation (CTD(k)), and cumulative administrative weightj (AW(k)) options available for interconnection in the network. Thus, step 91 sets k=1, and step 92 selects BW(k), CTD(k), and AW(k).

[0132] Step 93 then prunes the search of the network so that only links having bandwidth greater than BW(k) are investigated. Step 94 applies the algorithm of FIGS. 7A and 7B to find the shortest paths f satisfying the CTD(k) and AW(k) requirements from the node in which this algorithm is being executed to all other nodes in the network. Step 95 writes the result into the routing table. As indicated in steps 96-98, this procedure is repeated for each available bandwidth, cell transit delay, and administrative weight that may appear in a call setup request.

[0133] FIG. 9 illustrates the call setup procedure invoked by a call setup request (Step 101) in a flat network.

[0134] When the call setup request arrives at a switch (See FIG. 14), step 102 selects an optimal path from the table of precomputed routes that satisfies both the CTD and BW constraints in the call setup request.

[0135] Step 103 checks whether a path was found. If so, in step 104, call admission control and switch configuration are performed, and a connection setup request to the next switch in the path is generated. If not, an on-the-fly procedure, starting at step 105, is invoked to find an appropriate path. Step 105 prunes the search so that network links with insufficient bandwidth to satisfy the call setup request are not searched. Before the on-the-fly search is actually commenced, step 106 employs a decision circuit to decide whether the use of the constrained shortest path algorithm of FIGS. 7A and 7B is allowed. The answer is “Yes” if its use is judged to be not too expensive in terms of time and resources. Then step 107 uses the algorithm of FIGS. 7A and 7B in real time. Otherwise the answer is “No”, and step 108 performs a Dijkstra shortest path algorithm with cell delay variation as the cost function. Once a path is found, the procedure is ended (step 109).

[0136] As shown in FIG. 10, the algorithm of FIGS. 4A and 4B is used in the precomputation of routing tables in a hierarchical network (see FIG. 14). The flow diagram of FIG. 10 basically implements precomputation of “shortest path” routes using the shortest path algorithm of FIGS. 4A and 4B and stores the routes for different values of CTD (or t) in a routing table or cache memory (See FIG. 14). Store the path(s) in a cache of the form (Traffic descriptor, CTD, route).

[0137] In more detail, in FIG. 10, the precomputation of routing tables starts with the selection of one (e.g., k=1) of the numbers and sizes of bandwidths (BW(k)) and cell delay variation (CTD(k)) options available for interconnection in the network. Thus, step 111 sets k=1, and step 112 selects BW(k) and CTD(k).

[0138] Step 113 sets n=1, representing the topmost level of the hierarchical network. Step 114 sets the maximum allowable amount T, according to the request, of the cumulative cell delay variation CTD(k).

[0139] Step 115 then prunes the search of the network so that only links having bandwidth greater than BW(k) are investigated. Step 116 applies the algorithm of FIGS. 4A and 4B to find the shortest paths f satisfying the CTD(k) requirement with constraint T from the node in which this algorithm is being executed to all other nodes in the peer group at level n. Step 117 sets T equal to the cell delay variation of the logical group node that is the ancestor (in level n, FIG. 15) of node I, which is the node handling the call setup request, at level n. Step 118 increments n to n+1; and step 119 uses a decision circuit to check whether n is less than or equal to the total number of levels (FIG. 15). If “yes”, the process loops back to step 115 to perform route precomputation at each level of the hierarchical network. If “no”, step 120 writes the computed routes into the routing database for BW(k) and CTD(k). As indicated in steps 121-123, this procedure is repeated for each available bandwidth and cell delay variation that may appear in a call setup request.

[0140] FIGS. 11A and 11B illustrate the call setup procedure invoked by a call setup request (Step 131) in a hierarchical network.

[0141] In FIG. 11A when the call setup request arrives at a switch (See FIG. 14), step 132 sets maximum cell delay variation and bandwidth equal to those specified in the call setup request. Step 133 sets n=1, representing the topmost level of the hierarchical network. Step 134 selects an optimal path through the peer group at level n from the table of precomputed routes that satisfies both the CTD and BW constraints in the call setup request.

[0142] Step 135 sets CTD equal to the CTD of the logical group node that is the ancestor of node I at level n. Step 136 increments n to n+1; and step 137 uses a decision circuit to test whether n is equal to the total number of levels. If “no”, the process returns to step 134. If “yes”, at step 138 a decision circuit checks whether a set of paths, one per level, was found. If so, in step 139, call admission control and switch configuration are performed, and a connection setup request to the next switch in the path is generated, the process being done, as indicated at 149A, at the current switch.

[0143] If a set of paths was not found, an on-the-fly procedure, starting at step 140 in FIG. 11B, is invoked to find an appropriate path. Step 141 resets n to 1. Step 142 prunes the search so that network links with insufficient bandwidth to satisfy the call setup request are not searched. Before the on-the-fly search is actually commenced, step 143 employs a decision circuit to decide whether the use of the constrained shortest path algorithm of FIGS. 4A and 4B is allowed. The answer is “Yes” if its use is judged to be not too expensive in terms of time and resources. Then step 144 uses the algorithm of FIG. 4A and 4B in real time. Otherwise the answer is “No”, and step 145 performs a Dijkstra shortest path algorithm with cell delay variation as the cost function. So that a set of paths through peer groups at all levels may be found, step 146 sets CTD equal to CTD of the logical group node that is the ancestor of node I at level n. Step 147 increments n to n+1. Step 148 uses a decision circuit to check whether n has reached the total number of levels. If not, the process is returned to step 142. Once a complete set of paths is found, the procedure is ended (step 149).

[0144] As shown in FIG. 12, the algorithm of FIGS. 4A and 4B is used, according to an asymptotic implementation of the present invention, in the precomputation of routing tables in a hierarchical network (see FIG. 15). The flow diagram of FIG. 12 basically implements precomputation of “shortest path” routes using the shortest path algorithm of FIGS. 4A and 4B and stores the routes for different values of CTD (or t) in a routing table or cache memory of FIG. 12. The table or data base is a cache of the form (Traffic descriptor, CTD, route).

[0145] In more detail, in FIG. 12, the precomputation of routing tables starts with the selection of one (e.g., k=1) of the numbers and sizes of bandwidths (BW(k)) and cell delay variation (CTD(k)) options available for interconnection in the network. Thus, step 151 sets k=1, and step 152 selects BW(k) and CTD(k).

[0146] Step 153 sets n=1, representing the topmost level of the hierarchical network. Step 154 sets the maximum allowable amount T, according to the request, of the cumulative cell delay variation CTD(k).

[0147] Step 155 then prunes the search of the network so that only links having bandwidth greater than BW(k) are investigated. Step 156 applies the algorithm of FIGS. 4A and 4B to find the shortest paths f satisfying the CTD(k) requirement with constraint T from the appropriate node I in which this algorithm is being executed to all other nodes in the peer group at level n. Step 157 sets T equal to the cell delay variation of the logical group node that is the ancestor (in level n, FIG. 12) of node I at level n. Step 158 increments n to n+1; and step 159 uses a decision circuit to check whether n is less than or equal to the total number of levels (FIG. 12). If “yes”, the process loops back to step 155. If “no”, step 160 writes the computed routes into the routing database for BW(k) and CTD(k). As indicated in steps 161-163, this procedure is repeated for each available bandwidth and cell delay variation that may appear in a call setup request.

[0148] FIGS. 13A and 13B illustrate the call setup procedure invoked by a call setup request (Step 171) in a hierarchical network.

[0149] In FIG. 13A when the call setup request arrives at a switch (See FIG. 14), step 172 sets maximum cell delay variation and bandwidth equal to those specified in the call setup request. Step 173 sets n=1, representing the topmost level of the hierarchical network. Step 174 selects an optimal path through the peer group at level n from the table of precomputed routes that satisfies both the CTD and BW constraints in the call setup request.

[0150] Step 175 sets CTD equal to the CTD of the logical group node that is the ancestor of node I at level n. Step 176 increments n to n+1; and step 177 uses a decision circuit to test whether n is equal to the total number of levels. If “no”, the process returns to step 174. If “yes”, at step 178 a decision circuit checks whether a set of paths, one per level, was found. If so, in step 179, call admission control and switch configuration are performed, and a connection setup request to the next switch in the path is generated, the process being done, as indicated at 189A, at the current switch.

[0151] If a set of paths was not found, an on-the-fly procedure, starting at step 180 in FIG. 13B, is invoked to find an appropriate path. Step 181 resets n to 1. Step 182 prunes the search so that network links with insufficient bandwidth to satisfy the call setup request are not searched. Before the on-the-fly search is actually commenced, step 183 employs a decision circuit to decide whether the use of the constrained shortest path algorithm of FIGS. 7A and 7B is allowed. The answer is “Yes” if its use is judged to be not too expensive in terms of time and resources. Then step 184 uses the algorithm of FIGS. 7A and 7B in real time. Otherwise the answer is “No”, and step 185 performs a Dijkstra shortest path algorithm with cell delay variation as the cost function. So that a set of paths through peer groups at all levels may be found, step 186 sets CTD equal to CTb of the logical group node that is the ancestor of node I at level n. Step 187 increments n to n+1. Step 188 uses a decision circuit to check whether n has reached the total number of levels. If not, the process is returned to step 182. Once a complete set of paths is found, the procedure is ended (step 189).

[0152] In all of the techniques described in FIGS. 4A-13, performance can be improved by the use of the present invention.

[0153] In FIG. 14, a switch module is shown block diagrammatically to point out the relationship between the typical switch hardware 192 and the various software or firmware and database modules 193-197, as employed according to the present invention.

[0154] In brief, the routing protocol 193, involving basic routing rules, accesses topology database 194 to set up numbering of nodes (and levels, if applicable), particularly with respect to network boundaries. Additional portions of database 194 are accessed and employed by precomputation module 195 (per FIGS. 5, 8, 10, and 12, each employing respective ones of FIGS. 4A, 4B, 7A and B) to commence the precomputation process.

[0155] The results are stored in routing database 196. Stored values in routing database 196 are fed to call processing module 197 (per FIGS. 6, 9, 11, and 13 ) upon request therefrom. Call processing module 197 appropriately activates switch hardware 192.

[0156] FIG. 15 illustrates the peer group clustering of nodes or switches in a hierarchical network. For purposes of illustration, asynchronous transfer mode switches are assumed to be at the nodes. With equal validity, other types of switches can, and are, assumed to be at the switches. Nodes at level n=M, which includes switches A.1.1, A.1.2, A.1.3, A.1.4, A.2.1, A.2.2, B.1.1, B.1.2, B.1.3, B.1.4, B.1.5, B.2.1, B.2.2, B.2.3, B.3.1, B.3.2, B.3.3, B.3.4, B.3.5, and B.3.6 represent physical switches. Nodes at higher levels, n=2 and n=1, are logical group nodes that represent peer groups.

[0157] The following description compares the DAD algorithm of the prior application, the length scaling algorithm of R. Hassin, and the DSA (delay scaling, multiple destinations) algorithm according to the present invention. Execution times were measured-on a 150 MHZ Silicon Graphics, Inc. computer using a UNIX® operating system. Performance gains were also compared. For all of our experiments the error tolerance parameter ε was set to 5%.

[0158] The basic standard for comparison is the performance of the classical Dijkstra unconstrained shortest path algorithm (DSP), which finds the shortest paths to all destinations without regard to delay. The gain of the other algorithms with respect to DSP can be defined as follows: Let S denote the set of nodes for which there exists a feasible path satisfying the delay constraint, D_minbut DSP found an infeasible path with delay D_i. Let ǵ be the gain factor. $\begin{array}{cc}\stackrel{.}{g}=\frac{{\Sigma }_{i\in S}\frac{\mathrm{Di}-D_{\min }}{D_{\min }}}{N_{\mathrm{delay}}-N_{\mathrm{dijkstra}}}& \left(1\right)\end{array}$

[0159] Where D_i, D are the path delays to destination node i, N_delaydenotes the number of destinations that satisfy the delay constraint for the constrained shortest path algorithm and N_dijkstradenotes the number of destinations, whose shortest paths using DSP happen to satisfy the delay constraint.

[0160] The results of these measures showed substantial cost and speed advantage for the scaling technique with minimal adverse impact on caller expectations.

[0161] While the invention has been described above in preferred implementations, various alternatives should be apparent from the above description or may become apparent from the practice of the invention. All implementations for delay-based shortest path selection will employ the basic advance of the invention, as set forth in the claims below or equivalents thereof