System and method for large file transfers in packet networks

[0002] 1. Field of Invention

[0003] This invention relates to information networks. More particularly, the invention relates to a system and method for improving large file transfers in packet networks while protecting service requirements of network users.

[0004] 2. Description of Prior Art

[0005] Packet switching networks using, for example, the Internet Protocols, transport large data objects in the form of packets in which the data to be transmitted is fragmented into individual packets of data, each packet having a unique identification (Id) and carrying its own destination address. In this manner, each packet, representing a fragment of the large data object can go by a different route to the destination. The packets may also arrive in a different order than the order in which they were shipped. The packet Id lets the data be re-assembled in proper sequence. A further description of packet switching is described in the text “Internetworking with TCP/IP”, Douglas Comer, Prentice Hall, Englewood Cliffs, N.J. Also see “Digital Telephony” by J. Bellamy, published by John Wiley & Sons, Inc., New York, New York, 1991, and described in Section 9.2.

[0006] Today, considerable difficulty exists in transferring large files (>10 MB) through packet switching network, such as the Internet or Intranet using E-mail or standard File Transfer Protocol (FTP). Many PC applications contain one or more components greater than 10 MB. WINZIP (a trademark of Nico Mak Computing Inc. of Bristol, Connecticut) files which are collections of files comprising a complete application are often this size. Digital video contents may be hundreds of megabytes or even gigabytes. While the senders and receivers of such large files recognize that their transfer will require significant time, they nonetheless desire a service which will guarantee delivery by a specific time. The transfer of large files lasting many minutes or even hours is susceptible to failure due to network congestions which last several seconds or minutes. Such failures are likely to necessitate the re-start of the entire file transfer. The transfer of smaller files is statistically less likely to encounter such problems and the penalty for having to re-start small transfer is relatively smaller. Large transfers can also cause problems in inter-network access points where intermediate packet re-ordering and re-assembly may occur because there may be insufficient local storage available. As the information transfer requirements continue to increase, network congestion and poor service is experienced. Applications requiring guaranteed bandwidth such as Internet telephony, and streaming audio and video, often perform poorly due to network congestion, which is exacerbated by large file transfers, as it is difficult to efficiently multiplex very large and very small packets on the same links while maintaining quality of service guarantees. Transferring a large file may consume all available bandwidth to the detriment of other users.

[0007] Network designers and standards groups are working towards mechanisms that will prevent the large user of bandwidth from affecting the service of others. These mechanisms include end-to-end bandwidth reservation, as well as maximum ingress bandwidth to the network. The mechanisms which prevent the transfer of large files from affecting other users exacerbate the difficulty users have in transferring large files or knowing how long their file transfer will take. Accordingly, a need exists in information networks, e.g., the Internet, to optimally transfer large files without difficulty while protecting the service requirements of other users in the network.

[0008] Demand for multimedia services, such as audio and video streaming and interactive collaborative services is creating the need for managing the quality of service in packet switched communication networks for the real-time flow of small packets of audio and video (and possible other kinds of data). This management will be performed by bandwidth managers on a link-by-link or global basis. The goal of the bandwidth manager is to ensure that requests by applications or users for guaranteed transmission performance (throughput, latency, packet loss rates, and so forth) can be met in concert with existing resource reservations. In effect, the bandwidth managers will allow users to get guarantees for the transmission of data at a priority higher than the traditional “best efforts” service; typically the user will pay a premium for such a service. The purpose of this invention is to exploit these emerging network capabilities to meet the needs of large file transfer.

[0009] Prior art related to large file transfers in computer networks includes the following:

[0010] U.S. Pat. No. 5,526,350 issued Jun. 11, 1996, discloses a telecommunications network having a plurality of switches for switching different types of traffic, such as audio data including voice data, fax and modem originated data, digital computer originated data and video data. A communication link connects a user's site to the network, and a bandwidth manager is provided to multiplex traffic of different types for transmission over the link. After transmission over the link, a complementary bandwidth manager is arranged to de-multiplex the traffic for application to the respective switches. The bandwidth manager is arranged to dynamically allocate bandwidth to the different types of traffic, thereby optimizing the available bandwidth provided by the communications link.

[0011] U.S. Pat. No. 5,680,400 issued Oct. 21, 1997, discloses a high speed data transfer mechanism for transferring files from a transmission host across a data link to a receiver host. Data is presented to a data splitter. The data splitter separates the data streams into N separate substreams by packaging data in packets which may be of different sizes. As data is packetized, each packet is sent and presented to a separate data transmitter. Data is sent to the array of transmitters in round-robin fashion such that the data is first presented to the first transmitter, then to the second transmitter, and so on until each transmitter has sent a packet, and all data packets have been sent to a transmitter. A receiving side then initializes the receivers as needed, or as many data receive substreams as are required, using as many receivers as are available. A substream reassembly unit reassembles the packets into a final output stream.

[0012] European Application 0762281 published Dec. 3, 1997, discloses a method of selectively obtaining formatted dump data from a remote software product using an agent representing the remote software product and a manager representing a local customer/vendor management station. A dump object is defined by the agent and exposed to the manager for modification. Modification of the dump object causes the agent to selectively create/store a formatted storage dump for one or more software product components. The manager retrieves the formatted dump data from the agent using a standard file transfer mechanism. Since the dump data is formatted by the agent, the problem of transmitting large amounts of data across a network is minimized or eliminated.

[0013] PCT Application WO 95/32573 published Nov. 30, 1995, discloses transferring a series of data files between two computers, each computer provided with a file transfer mechanism. Each file transfer mechanism controls the transfer of a series of files in accordance with a protocol. In this protocol, the computer receiving data files (the initiator) sends a poll message to the computer transmitting the data files (the responder). In response, the responder sends a reply message to the initiator containing a list of files which are available for transfer. From the list, the initiator selects the files it wishes to receive and then copies each file in turn. The initiator then sends an acknowledgement to the responder.

[0014] JP 8-307451 published Nov. 22, 1996, discloses data transmission systems employing large scale networks such as the Internet by determining optimum transfer demands by a client machine raising network utilization efficiency and reducing data access time.

[0015] U.S. Pat. No. 5,751,712 issued May 12, 1998 and filed Oct. 26, 1996 discloses a communication system administering control on bandwidth allocation to multimedia data. The bandwidth is partitioned into three bands dedicated to audio, video and data traffic in accordance with blocking probabilities associated with the respective media types. The value of each blocking probability is selected pursuant to the relative importance of the associated media type. With blocking probabilities in place , the dominance of a particular type of media traffic which may not be important would not cause blocking of traffic of other media types which may be relatively important.

[0016] None of the prior art discloses bandwidth managers and forwarding/receiving agents, which segment and reassemble large files, thereby enabling the file to be transmitted as multiple, smaller segments over one or more networks links using priority reservations, where available, as provided by the bandwidth manager thereby protecting the service demands of other network users during the large file transfer.

[0017] An object of the invention is an information network and method of operation which provides large file transfers without altering the service requirements of network users.

[0018] Another object is an information network, e.g., the Internet, having forwarding and receiving networks and multiple network attachments which optimize transmission of large files over the multiple networks.

[0019] Another object is a packet network and method of operation for large file transmissions managed by forwarding and receiving agents.

[0020] Another object is a packet network and method of operation for large file transmissions managed by forwarding and receiving agents on guaranteed bandwidth reservations provided by bandwidth managers.

[0021] Another object is a packet network and method of operation in which large file transmission are segmented and re-assembled for multiple network link transmissions by forwarding and receiving agents.

[0022] Another object is a packet switching network and method of operation having forwarding and receiving agents matching the segmentation of a file to a bandwidth reservation obtained from a network bandwidth manager.

[0023] Another object is a packet switching network and method of operation having forwarding and receiving agents optimizing the transmission of large files across sub-networks of a packet switching network divided into sub-networks.

[0024] These and other objects, features and advantages are achieved in a computer network, e.g. a packet network which may be a unitary or a segmented network with multiple paths and a bandwidth manager for each segment of the network. The bandwidth manager provides a sender coupled to the network with a guaranteed bandwidth reservation up to some maximum bandwidth, for example, 64 kbps or 512 kbps, for the transfer of any kind of data to a receiver over a single or multiple network paths. The manager will usually impose a maximum reservable bandwidth which is small compared to the total bandwidth of the network, so as to allow large numbers of such reservations to be handled simultaneously on behalf of many users. The use of bandwidth reservation enables the large file transfer service to guarantee that the file will be delivered by a specific time. By segmenting large files they may be allocated to multiple bandwidth reservations. The more immediate the required delivery time, the greater the number of reserved channels. Large files may therefore be transmitted over one such resource channel or in this invention over multiple such channels. Both the sender and receiver include file forwarding and receiving agents which interact with the bandwidth manager to obtain such guaranteed bandwidth reservations for intact file transfers or segmented file transfers over the network or network segments using single or multiple network paths. The agents manage the file transmission over the multiple reserved channels and perform segmentation and reassembly of transferred files. The agents also match the file intact or segmented to the associated bandwidth reservation. Where the sender and receiver are coupled to multiple overlaid networks, each of which connects the sender and receiver, the agents interacts with bandwidth managers of the different network to obtain guaranteed bandwidth reservations for intact file transfer or segmented file transfers over single or multiple paths in different networks. Forwarding and receiving agents optimize transmission of large files over multiple networks served by a host computer.

[0025] The invention will be further understood from a detailed description of a preferred embodiment taken in conjunction with an appended drawings, in which:

[0026] FIG. 1 is a representation of a packet network in which bandwidth managers are employed to manage prioritization of different classes of traffic for large file transfer in accordance with the principles of the present invention.

[0027] FIG. 2 is a representation of the packet network of FIG. 1 divided into several sub-networks with each sub-network incorporating a file forwarding and receiving agent to manage the transmission of large files.

[0028] FIG. 3A is a representation of a large file divided into multiple segments with each segment being available for transmission via a separate bandwidth reservation on the network or networks of FIG. 1.

[0029] FIG. 3B is a representation of a subsegmentation of a segment based on reservations obtained for intermediate subnets, as directed by agent pair responsible for that subnet in the network of FIG. 2.

[0030] FIG. 4 is a flow diagram for a file forwarding agent requesting a bandwidth reservation from a bandwidth manager.

[0031] FIG. 5 is a flow diagram for a file forwarding agent when a bandwidth reservation expires.

[0032] FIG. 6 is a flow diagram for a file forwarding agent requesting bandwidth reservation across multiple sub-networks.

[0033] In FIG. 1, an information network, e.g. a packet switch network 10 connects station A to station B through links 12, 14, 16 using bandwidth managers 20, 30, 40. Bandwidth managers are designed to regulate and assure the amount of traffic from each host up to some maximum rate; limit and report on the bandwidth utilization by a host running the Internet Protocols. For example, in a Local Area Network (LAN) environment, an Aponet Bandwidth Manager, Model 10, manufactured by Aponet, Inc. Wien, Austria supports 10 Mbps Ethernet bandwidth in monitoring, reporting, limiting and controlling features. Such bandwidth managers support large file transfers between stations A and B using guaranteed bandwidth reservation software, e.g., Resource ReSeRVation Protocol (P) (RSVP), issued by the Internet Engineering Task Force A (RSVP Working Group) as described in Request For Comments (RFC 2208). RSVP provides resource reservations for isochronous traffic over the network. RSVP is issued by routers to bandwidth managers to deliver quality of service requests to all nodes along the path(s) of the flow and to establish and maintain a state to provide a requested service.

[0034] In FIG. 2, sub-networks 1, 2, and 3 may be different networks, that is, station A may be connected to a private corporate network 1 while station B maybe connected to a different corporate network 5. Networks 1 and 3 may be connected to another network, such as the Internet. Each sub-network includes an agent 50 at the entrance point and an agent 51 at the exit point of the sub-network. The agents work in pairs. The agent 50 represents the sender and is designated a forwarding agent. The agent 51 represents the receiver and is designated a receiving agent. The forwarding/receiving agents segment the file and transmit or receive the segments, respectively, potentially in parallel over multiple paths in the same or different networks. The agents work synergistically with the bandwidth manager mechanism for bandwidth managers 20, 30, 40, using the RSVP reservation protocol, to make optimum use of the bandwidth for the sender or originator to transfer a file. The use of interim agents 54, 541 allows transmission of file segment to be further optimized by additional segmentation.

[0035] In FIG. 3A, a file 60 is divided into segments 1, 2, 3, . . . N, each segment having a guaranteed bandwidth obtained through an adjacent bandwidth manager. Each segment proceeds on the same or a different link in a network according to the reservation provided by the bandwidth manager.

[0036] For example, station A may request a reservation from a bandwidth manager to transmit 1 megabyte/30 seconds. The manager may deny this request as too resource consuming. The manager may then request two reservations over different links for 0.5 megabits/30 seconds. This request may be permissible for the network, allowing lesser usage on two routes traversing the network. Effectively, A is transmitting at 1 megabyte, however, the network has guaranteed lesser usage, and has potentially protected its ability to provide service for others.

[0037] Parallel transmission will result in segments of the file being received asynchronously and potentially out of order at the receiving station B. Returning to FIG. 2, the agent 51″ serving station B reassembles the file based on either information contained in the file segments or based on control information sent independently by agent 50″. Station B, of course, can receive only up to the capacity of its attachment to the network. Normal throttling mechanisms prevent station B from being overwhelmed.

[0038] The reservations obtained by A may each end at a different time. As the data transmission to each path loses its guarantee, the agent 50 again requests the maximum reservation it can obtain up to the capacity of the network. In this fashion, agent 50 adapts to the changing bandwidth guarantee that the network is able to offer.

[0039] If stations A and B have connections to multiple networks, agent 50 can further optimize the transmission. Where A is connected to “N” networks, for example, if A is connected to the first network having a bandwidth capacity C1, to a second network having a bandwidth capacity of C2, and further to the “Nth” network with bandwidth capacity C_n, then the agent 20 can seek reservations from each network up to the capacity of that attachment. Accordingly, the connection to multiple networks allows maximum transmission from station A.

[0040] If a network has been divided into sub-networks with multiple agent pairs, then each segment may be further optimized in successive sub-networks. As shown in FIG. 3B, if file segment L has been received from originator A at the location of agent 20, it may be necessary to further segment L for optimal transmission across sub-network L. That is, the transversal of sub-network L is managed for file segment L as if agent 50 were the originator and agent 54 were the receiver. Multiple parallel paths for sub-segments L₁, L₂, L_Netc. may be chosen through the sub-network L to optimize the traversal and arrange for the most rapid delivery of L at the agent 51.

[0041] Where networks do not have bandwidth management mechanisms, an agent will allocate to that path a file segment which it expects will be readily transmittable based on the capacity of the connection and policies related to past performance of the network in connection. Agents may be co-located with the originator and receiver or may reside elsewhere within the network. Agents may be provided as a service based on a network server for the users at the edge of the network. In this case, many agents for many users will be collated at the server.

[0042] FIG. 4 discloses a process for an agent located at a server and requesting a bandwidth reservation from a bandwidth manager for a data transmission. The process is entered at step 40 and proceeds to step 41 in which the agent Ax determines the maximum capacity of an attached device C, e.g. telephone, data terminal, etc. In a step 43, the agent determines the maximum bandwidth (M) to request based on the network and attachment. A test 46 is performed to determine if M is non-zero. A “no” condition ends the process. A “yes” condition initiates a step 47 to request M from the bandwidth manager. A test 48 is performed to determine whether the bandwidth manager provided a guaranteed reservation of M. A “no” condition initiates a step 49 to reduce the request to M′ and return to step 47. The process cycles until the agent receives a guaranteed reservation, whereupon a “yes” condition initiates a step 51 to segment the file M and initiate transmission of the file segments. In a step 53 the bandwidth manager reduces the available bandwidth by M; and in a step 55, reduces the available attachment capacity by M. A test 57 is performed to determine if more bandwidth is required. A “yes” condition returns the process to step 43. A “no” condition ends the process.

[0043] In FIG. 5, an agent located in a transmission link monitors the bandwidth reservations and identifies those or that reservation, e.g reservation “N” which expire in a step 61, after which in step 63 the agent increments the attachment capacity by N and enters step 57 in FIG. 4 to obtain more bandwidth using the process of FIG. 4.

[0044] Where stations A and B have connections to multiple networks, an agent can further optimize the transmission of large data files using the process shown in FIG. 6. The process is entered in a step 70, after which in step 72, the agent determines which networks are to be used for file transmissions. After network determination, the agent in step 74 alters the reservation in the network based on the suitability for the transfer; starting with the first network. In step 76, the agent performs the process of FIG. 4 for the first network. In a step 77, the process determines if the last network has been subject to the process of FIG. 4. A “yes” condition ends the process; a “no” condition initiates a step 78 to determine if more network capacity is needed. A “no” condition ends the process; a “yes” condition initiates a step 79 to go to the next network and perform the process of FIG. 4 after which the process ends in step 77.

[0045] Summarizing, a system and method of have been disclosed for large file transfers in packet networks, e.g., the Internet, without adversely affecting the service requirements of other network users. The system employs bandwidth managers in one or more different networks providing guaranteed bandwidth reservations for large segmented files by a forwarding agent and subsequent reassembly by a receiving agent for a receiving station. By segmenting the large files and using different networks to transmit the segments to the receiving agent for reassembly, the bandwidth capacity for the network is not substantially reduced to the point where the service requirements of other users are affected. Reservations and concurrent transmissions are arranged by the agents to the capacity of the network for the peripheral attachments to the sending station. The segmentation and reassembly maximizes a sender's transmission ability and allows the network to distribute the bandwidth usage over the network to prevent network congestion.

[0046] While various embodiments of the present invention have been described, it should be understood that they have been presented by way of example only and not limitation. The breadth and scope of the present invention should not be limited by any of the above-described exemplary embodiments but should be defined only in accordance with the following claims and their equivalents.