System and Method for Equalizing Delay in a Dynamic Packet Switching Network

1. Field of the Invention

The present invention relates generally to routing and transferring data and, in particular, to a system and method for equalizing delay in a dynamic packet switching network that routes and transfers data in addressed packets.

2. Description of the Related Art

Telecommunication networks exist that interconnect large numbers of user stations using telecommunication facilities. These networks utilize transmission systems, switching systems, and station equipment to transmit voice, video, and data between two points. The physical circuits between two points in the network are referred to as links, and the points of junction of the links are referred to as nodes. The user stations in data transmission networks may be telephones, terminals, printers, facsimile units, computers, and the like.

Packet switching networks were designed to provide a more efficient method of transferring data over networks. However, packet switching networks can also be used to transmit digitized voice. A network that uses packet switching as a means of transmitting data is commonly referred to as a packet switching data network (PSDN).

A packet is a discrete unit of a data message that is routed individually over a PSDN. Each packet contains control information that enables the message to be reassembled in proper sequence before it reaches its final destination. Packet switching is efficient because packets occupy the channel or path through the network only for the brief time they are in transit, in contrast to a circuit-switched message, which requires the use of the transmission line for the duration of the message. On completion of the data transmission, the channel or path is made available for transfer of other packets. The transmission lines through the PSDN are supplemented with computerized switches that control traffic routing and flow. A standard feature of packet switching is automatic error detection and correction of transmitted packets.

A conventional communications network using data packet switching is shown diagrammatically in FIG. 1. Users and other networks access this network through user access stations ("UAS"), shown, for example, in FIG. 1 as UAS.sub.1, UAS.sub.2, and UAS.sub.3. Other networks N.sub.1, N.sub.2 are considered to be like other users. User access stations send user data to the network and receive user data from the network through one or more switches S.sub.j. Paths are established through the network of switches S.sub.j in order to set up virtual communication channels between users on different user access stations. The transmissions delay for a data packet to go from one UAS to another UAS depends on the particular path chosen.

A typical transmission delay time through the network will be referred to as T. For example, T could be 20 milliseconds. The longest path chosen for a virtual connection through the network in a typical case has a transmission delay of 10 T. For example, for T=20 milliseconds, 10 T=200 milliseconds.

Data packet size need not be constant. Data packet size can be fixed, as it is in ATM networks. However, it must be no longer than a maximum length. The maximum packet length is such that the latency time to transmit the packet over any of the links k.sub.i or l.sub.y is smaller than T/10. The links k.sub.i are those connecting the user access stations to the switches, and the links l.sub.y are those connecting switches to other switches.

Some packet switching data networks are dynamic in a number of ways, for example, a network of packet switches on satellites in non-earth synchronous orbits. The communication links k.sub.i between the user access stations UAS and the switches S.sub.j are not permanent. Links k.sub.i are alive for a typical period of time of about 15,000T (for example, 5 minutes for T=20 milliseconds). When old links k.sub.i are removed, new ones are established, but the new links k.sub.i are almost always established on a new switch S.sub.j. For example, when link k.sub.2 (FIG. 1) is removed a new link between UAS.sub.1 and S.sub.2 may be established. There is always at least one link between a UAS and a network switch S.sub.j.

The links l.sub.y between switches S.sub.j are not permanent; they have typical lifetimes of about 30,000 T (for example, 10 minutes for T=20 milliseconds). However, the pattern of links must satisfy certain conditions. There are always enough links to allow any UAS to communicate to any other UAS. Some of the links k.sub.i or l.sub.y may be permanent or much longer lived than stated above.

In a typical system, there are control stations CS having communication links to the network switches S.sub.j. Their function is to control the network and to set up and tear down virtual connections. Any user access station UAS can always communicate with at least one control station CS. The location of the control stations CS are not germane to the present invention. It will be assumed that the network interconnection pattern (links k.sub.i and l.sub.y) is predictable and can be computed by the control stations CS for any future time. In practice, all that is needed is the ability to compute the pattern into the future for the duration of the longest virtual connection established at the present time. Malfunctions can affect the predictability, but there exist ways of handling malfunctions.

A virtual connection between two network users that lasts longer than the link lifetimes must take different paths through the network during the connection lifetime. It will be assumed that the dynamic interconnection pattern (links k.sub.i and l.sub.y) is such that a series of paths can be chosen for the duration of the virtual connection, and each path can be used for at least a time period 1,200 T (for example, 4 minutes for T=20 milliseconds). This is, of course, as in any other kind of network, assuming bandwidth availability. If there is not enough bandwidth available, the connection cannot be set up. Resources are reserved for the expected duration of the call. As shown in FIG. 2a, for each of a series of consecutive time intervals t.sub.1, t.sub.2, t.sub.3, . . . , t.sub.n there are corresponding paths P.sub.1, P.sub.2, P.sub.3, . . . P.sub.n, such that path P.sub.i is used during time interval t.sub.i for the virtual connection.

Referring to FIG. 2b, a first path P.sub.1, goes through switches S.sub.1, S.sub.3, S.sub.4, S.sub.5, S.sub.9, S.sub.10, S.sub.11, and S.sub.13, while a second path P.sub.2 goes through switches S.sub.2, S.sub.6, S.sub.7, S.sub.11, and S.sub.13. For the same virtual connection between UAS.sub.1 and UAS.sub.2, path P.sub.1 is used during time interval t.sub.1, and path P.sub.2 is used during time interval t.sub.2, and so forth.

Problems are caused by the changes in paths through the network used by one virtual connection. As shown in FIG. 3, a virtual connection is set up between user U.sub.1 on UAS.sub.1 and user U.sub.2 on UAS.sub.2. During the first time interval t.sub.1, path P.sub.A is used. During the second time interval t.sub.2, path P.sub.B is used. Path P.sub.A has a transmission delay time from UAS.sub.1 to UAS.sub.2 equal to t.sub.a, while path P.sub.B has a transmission delay time from UAS.sub.1 to UAS.sub.2 equal to t.sub.b. It should be noted that the transmission delay times t.sub.a and t.sub.b are not intervals during which paths P.sub.A and P.sub.B are used, that is given by the intervals t.sub.1 and t.sub.2.

It will first be assumed that t.sub.a <t.sub.b (for example, t.sub.a =20 milliseconds and t.sub.b =100 milliseconds), and that there is no buffering at UAS.sub.2. After the end of time interval t.sub.1, the virtual connection uses path P.sub.B instead of path P.sub.A. The first data packet traveling along path P.sub.B will arrive at UAS.sub.2 a period of time t.sub.b -t.sub.a later than it would have if it had gone on path P.sub.A. This will leave a silent gap of duration t.sub.b -t.sub.a in the data stream (for example, t.sub.b -t.sub.a =80 milliseconds) which is too large to be acceptable for many communication services. This problem is simple to correct, however changing from a longer path P.sub.B to a shorter path P.sub.A causes more difficult problems, as explained below.

It will now be assumed that during the first time interval t.sub.1, path P.sub.B is used, and during the second time interval t.sub.2, path P.sub.A is used (see FIGS. 4a and 4b). Again, t.sub.a is the transmission delay along path P.sub.A, and t.sub.b is the transmission delay along path P.sub.B, and t.sub.a <t.sub.b. There is no buffering at UAS.sub.2 or at UAS.sub.1.

Now assume that paths P.sub.A and P.sub.B have a common switch S.sub.C, as shown in FIG. 4a. At the end of the time interval t.sub.1, the connection will be changed from path P.sub.B to path P.sub.A. When a data packet first arrives at switch S.sub.C along path P.sub.A there are earlier data packets from the connection still traveling along path P.sub.B (there is also an interpacket spacing time, but this is negligible for the problem described). There will be a period of time equal to t.sub.b -t.sub.a during which switch S.sub.C receives packets from the connection at twice the normal rate. If the link from switch S.sub.C to UAS.sub.2 is fully (or close to fully) utilized, switch S.sub.C must buffer cells from this connection for a period much longer than t.sub.b -t.sub.a. If left uncorrected, this will cause delay and possibly increase cell loss probabilities for other virtual connections (cell=data packet).

If paths P.sub.A and P.sub.B do not have a common switch, as shown in FIG. 4b, UAS.sub.2 must maintain two links for a time interval t.sub.b -t.sub.a, during which time it gets cells at twice the normal rate for the connection. This will also result in an increased delay and increased cell loss probabilities for other virtual connections.

It is thus an object of the present invention to provide an improved system and method for routing and transferring data in addressed data packets, which overcomes the above-described problems in dynamic packet switching networks.

It is a further object of the present invention to provide a system and method for equalizing delay in a dynamic packet switching data network and eliminating packet rate doubling upon changing transmission paths.

Additional objects, advantages and novel features of the invention will be set forth in the description that follows, and will become apparent to those skilled in the art upon reading this description or practicing the invention. The objects and advantages of the invention may be realized and attained by the appended claims.

To achieve the foregoing and other objects and in accordance with the purpose of the present invention, as embodied and broadly described herein, the apparatus of this invention comprises a system for equalizing delay in a dynamic packet switching network, comprising a buffer means for buffering a packet transmission through the network for equalizing packet transmission delay and for eliminating packet rate doubling upon changing transmission paths through the network.

It is preferred that the buffer means comprises a first buffer means for buffering a packet transmission at a receiving user access station for equalizing packet delay through the network upon changing from one transmission path to another transmission path. The buffer means also preferably comprises a second buffer means for buffering a packet transmission at a transmitting user access station for eliminating packet rate doubling when a change is made from a longer transmission path to a shorter transmission path through the network. A means for controlling the second buffer means provides a first amount of buffering at the transmitting user access station immediately upon changing from a longer transmission path to a shorter transmission path through the network, and means for gradually shifting the first amount of buffering from the second buffer means to the first buffer means during a period of time following the change from a longer transmission path to a shorter transmission path.

In a further aspect of the present invention, in accordance with its objects and purposes, the apparatus hereof comprises a dynamic packet switching network, comprising first and second user access stations each having at least one buffer to delay packet transmission, and a network of switches and communication links interconnecting the first and second user access stations. A control station having communication links to the switches and user access stations provides means for setting up and changing transmission paths between the first and second user access stations, and means for controlling the buffers in the first and second user access stations for equalizing packet transmission delay through the network for different transmission paths.

It is also preferred that the control station have means for controlling the buffers in the first and second user access stations for eliminating packet rate doubling when a change is made from a longer transmission path to a shorter transmission path through the network. The means for controlling the buffers in the first and second user access stations comprises means for causing the buffer in the second user access station to delay packets received from the first user access station for a sufficient time to cause a total transmission delay time for each transmission path equal to a transmission delay time of a longest one of the transmission paths.

It is also preferred that the means for controlling the buffers in the first and second user access stations comprises means for causing the buffer in the first user access station to delay packets transmitted from the first user access station to the second user access station for eliminating packet rate doubling when a change is made from a longer transmission path to a shorter transmission path. The means for controlling the buffers also preferably comprises means for gradually shifting the buffering provided by the buffer in the first user access station to the buffer in the second user access station during a period of time following a change from the longer transmission path to the shorter transmission path.

In a further aspect of the present invention, in accordance with its objects and purposes, the method hereof comprises a method for equalizing delay in a dynamic packet network, comprising the steps of providing first and second user access stations each having at least one buffer to delay packet transmission, and a network of switches and communication links interconnecting the first and second user access stations, setting up and changing transmission paths between the first and second user access stations across the network of switches and communication links, and controlling the buffers in the first and second user access stations to equalize packet transmission delay through the network for all of the transmission paths.

It is also preferred that the method comprise the step of controlling the buffers in the first and second user access stations to eliminate packet rate doubling when a change is made from a longer transmission path to a shorter transmission path through the network. The method also includes the step of gradually shifting a buffering from the buffer in the first user access station to the buffer in the second user access station after changing from a first longer path through the network to a second shorter path through the network.

The present invention will become more clearly appreciated as the disclosure of the present invention is made with reference to the accompanying drawings, wherein:

FIG. 1 is a schematic diagram providing a general overview of a conventional dynamic packet switching data network.

FIG. 2a illustrates an allocation of resources for a series of consecutive time intervals for a conventional dynamic packet switching data network.

FIG. 2b is a schematic diagram depicting the paths used during two intervals (for example) in a conventional virtual connection.

FIG. 3 is a schematic diagram showing a virtual connection that uses different paths between two users of a conventional dynamic packet switching data network.

FIG. 4a is a schematic diagram showing two paths with a common switch used in a virtual connection between two users of a conventional dynamic packet switching data network.

FIG. 4b is a schematic diagram showing two paths with separate links into a receiving user access station of a conventional dynamic packet switching data network.

FIG. 5a is a schematic diagram of a preferred embodiment of the present invention in which a buffer system is used at each user access station to equalize delay in the network.

FIG. 5b is a schematic diagram of the present invention showing two paths with separate links into a receiving user access station.

FIG. 5c is a schematic diagram of the present invention showing the delay equalizing system of the present invention used to equalize delay across three transmission paths.

FIGS. 6a and 6b are flow charts of the process steps used by the present invention to equalize delay in a dynamic packet switching data network.

Reference will now be made in detail to a preferred embodiment of the invention, an example of which is illustrated in the accompanying drawings.

Referring to FIGS. 5a, 5b, 6a, and 6b, a system and method for equalizing data packet delay through a dynamic packet switching data network will be described. The system according to the present invention will also eliminate data packet rate doubling at a common switch along two paths used by one connection or data packet rate doubling at the receiving UAS.

As shown in FIGS. 5a and 5b, two buffers are used at each user access station UAS. A receive buffer RB.sub.n operates to delay packets so that delay is equalized to that of the longest paths used by the virtual connection. A transmit buffer TB.sub.n operates to eliminate data packet rate doubling when a change is made from one path to another shorter path. Buffer control RBC.sub.n and TBC.sub.n provide control their associated buffers RB.sub.n and TB.sub.n. A control station CS having communication links to the network switches S.sub.j and the user access stations UAS functions to control the buffers and the virtual connections of the network. The communication feeds to and from the buffers TB.sub.n and RB.sub.n are combined at cross connect elements CC.sub.n in each of the user access stations. The details of the buffers and their operation in the packet network are described below.

The change of a virtual connection from a path P.sub.A to a longer path P.sub.B will be described first. The paths can have a common switch S.sub.C before the receiving user access station UAS.sub.2 (FIG. 5a), or the paths can enter the receiving user access station UAS.sub.2 through two links without a common switch (FIG. 5b). The connection starts, for example, using path P.sub.A. Packets traveling along path P.sub.A undergo a transmission delay time t.sub.a from UAS.sub.1 to UAS.sub.2. When the virtual connection is moved from path P.sub.A to path P.sub.B, the packets traveling along path P.sub.B undergo a transmission delay time t.sub.b from UAS.sub.1 to UAS.sub.2.

In order to equalize the transmission delay times for paths P.sub.A and P.sub.B, packets traveling along path P.sub.A are delayed for a time t.sub.b -t.sub.a in buffer RB.sub.2. On the other hand, packets traveling along path P.sub.B are not delayed at RB.sub.2. After moving the virtual connection from path P.sub.A to path P.sub.B, the first packet for the connection traveling along path P.sub.B arrives at RB.sub.2 after the last packet traveling along path P.sub.A leaves the buffer RB.sub.2. There is no gap in the data packet stream, and transmission delay time is the same from U.sub.1 to U.sub.2 when paths P.sub.A or P.sub.B are used.

The change of a virtual connection from path P.sub.B to a shorter path P.sub.A will be described next. During the time the connection is using path P.sub.B, the transmission delay is t.sub.b, and the packets are not delayed at either TB.sub.1 or at RB.sub.2. When the change is made to the shorter path P.sub.A, the present invention avoids packet rate doubling by delaying packets for transmission along path P.sub.A for a time t.sub.b -t.sub.a in buffer TB.sub.1. In this way, the first packet traveling along path P.sub.A will arrive at S.sub.C (FIG. 5a) or at UAS.sub.2 (FIG. 5b) after the last packet traveling along path P.sub.B arrived at S.sub.C or at UAS.sub.2. This eliminates data packet rate doubling and equalizes the transmission delay times for paths P.sub.A and P.sub.B.

Referring to FIG. 5c, the present invention will next be described for the case where a change to a new path P.sub.C longer than P.sub.A is made. Assume the connection uses paths P.sub.B, P.sub.A, and P.sub.C (in that order) with transmission delays t.sub.b, t.sub.a, and t.sub.c, respectively. Also assume, as above, that t.sub.a <t.sub.b and that t.sub.a <t.sub.c <t.sub.b. Path P.sub.B is the path with longest delay of the three paths. When the change is made from path P.sub.B to the shorter path P.sub.A the procedure described above is used. As described, the packets are buffered and delayed for a time t.sub.b -t.sub.a in the transmit buffer TB.sub.1, but not in the receive buffer RB.sub.2. If the system simply waits until the change is made from path P.sub.A to path P.sub.C, there will be a gap of time t.sub.c -t.sub.a in the packet stream, and the total delay will be (t.sub.b -t.sub.a)+t.sub.c, which is larger than t.sub.b because t.sub.c -t.sub.a is positive.

The problem just described is avoided by shifting the buffering of packets for the virtual connection from TB.sub.1 to RB.sub.1 during the time that the path P.sub.A is being used. In one example of the present invention, it will be assumed that each path can be used for a time period of at least 12,000 T (e.g., 4 minutes for T=20 milliseconds), where T is the typical transmission delay through the network. It will also be assumed that there is a maximum transmission delay of 10 T (e.g., 200 milliseconds for T=20 milliseconds) for transmission across the longest path P.sub.B.

The buffering delays applied in order to equalize the total delay are equal to the transmission time differences along two paths. Buffering delays are therefore also bounded by a maximum time equal to 10 T. The average number of packets transmitted for the connection in a time 12,000 T (path P.sub.A minimum active time) is 1,200 times larger than the average number of packets transmitted in a time 10 T (maximum buffering delay). So the number of packets buffered at TB.sub.1 is 1/1,200 times the number of packets the virtual connection delivers during the time period 12,000 T, which is a lower bound to the period of time any one path is used, in particular, path P.sub.A.

Buffering can be shifted from buffer TB.sub.1 to RB.sub.2 by sending packets from TB.sub.1 to RB.sub.2 at a rate one per thousand larger than the average for the connection. This procedure will take time 1,000(t.sub.c -t.sub.a) which is smaller than 10,000 T and also smaller than the time path P.sub.A is used. During this time packets are delayed at RB.sub.2 for a time sufficient to make the total delay equal to t.sub.b. At the start of the procedure packets are delayed by t.sub.b -t.sub.a at TB.sub.1 and by zero at RB.sub.2. At the end of the procedure packets are delayed by zero at TB.sub.1 and by t.sub.b -t.sub.a at RB.sub.2.

During the time period 1,000(t.sub.b -t.sub.a) the delay in buffer TB.sub.1 changes linearly from t.sub.b -t.sub.a to zero, and in buffer RB.sub.2 the delay changes linearly from zero to t.sub.b -t.sub.a. At the end of this buffering shift, there is no delay at buffer TB.sub.1, and a delay of t.sub.b -t.sub.a at RB.sub.2 thus occurs near the end of the use of the path P.sub.A. When the change is made to the path P.sub.C, the transmission delay is t.sub.c, which is longer than t.sub.a. There will now be no gap in the packet stream out of UAS.sub.2 to user U.sub.2. The first packet traveling along path P.sub.C will arrive at the buffer RB.sub.2 a time t.sub.c -t.sub.a later than if it had gone along P.sub.A. During this time the connection to U.sub.2 is fed from the buffer RB.sub.2. The delay at buffer RB.sub.2 will be t.sub.b -t.sub.c for packets traveling along path P.sub.C. The total transmission delay including buffering is equal to t.sub.b, as mentioned above.

In order to accomplish the above delay equalizing procedure it is necessary to reserve 1/1,000 (0.1%) of the bandwidth in the network for buffer shifting.

Buffers TB.sub.2 and RB.sub.1 are used in an analogous way to equalize delay and eliminate gaps in the packet stream for the connection in the opposite direction. The requirement that a path must be used for at least a time 12,000 T need not apply to the last path used by the connection.

Referring to FIGS. 6a and 6b, the method for equalizing delay in a dynamic packet network according to the present invention will be further described as a series of process steps.

In FIG. 6a, the process steps for an initial start of the delay equalizing process are shown. The process starts by determining the path P.sub.max that will be used having the maximum time delay t.sub.max (e.g., P.sub.B and t.sub.b in the example above). If the duration of the data transmission is indeterminate, the system will use the upper bound for t.sub.max. A first path P.sub.s having a transmission delay t.sub.s is then set by the control system. If t.sub.s <t.sub.max, the system will delay packets by t.sub.max -t.sub.s in RB.sub.2 with no delay at TB.sub.1. If t.sub.s =t.sub.max, the system will not delay the packets at either TB.sub.1 or at RB.sub.2.

In FIG. 6b, the process steps for equalizing delay in the network upon switching paths are shown. When the time comes for a path change to be made, the buffering, if any, is at RB.sub.2. The current path is P.sub.C with transmission delay t.sub.c, and the new path is P.sub.N with transmission delay t.sub.c. If t.sub.n =t.sub.c, no buffering or delay changes are made by the control system. However, if t.sub.n >t.sub.c, a buffering delay of length t.sub.max -t.sub.n will be applied at RB.sub.2 to packets traveling along path P.sub.N. The buffering delay was t.sub.max -t.sub.c before the change and will be t.sub.max -t.sub.n after the change. If t.sub.n <t.sub.c, the control system will apply a buffering delay of length t.sub.c -t.sub.n in the buffer TB.sub.1. The system will then gradually shift the buffering from TB.sub.1 to RB.sub.2 using the procedure described above during the time path P.sub.N is used. At the end of the buffer shifting procedure all buffering delay will be at RB.sub.2 and will be equal to t.sub.max -t.sub.n. The total transmission delay is always t.sub.max.

The loop terminates during the use of the last path. If the process is in the middle of a buffer shifting procedure, there is no difficulty. The remaining packets in the network are delivered to U.sub.2 with a constant delay equal to t.sub.max.

The minimum length of time that any path (except the last) can be used can be reduced as desired. For example, if the minimum time a path must be usable is 1,200 T (instead of 12,000 T) then the buffering must be shifted from TB.sub.1 to RB.sub.2, when necessary, 10 times as fast. This implies sending one more packet per one hundred packets during the buffering shift process. This requires 1% of available bandwidth to be reserved for this purpose. This can be carried further if more bandwidth is made available for shifting.

It will be appreciated that the present invention is not limited to the exact construction or process steps that have been described above and illustrated in the accompanying drawings, and that various modifications and changes can be made without departing from the scope thereof. It is intended that the scope of the invention only be limited by the appended claims.