High level control of file transfer protocol with capability for repeated transfer attempts

[0002] 1. Field of the Invention

[0003] This invention relates to a file transfer mechanism for transferring a series of data files from one computer to another computer and also to a method of transferring a series of data files from one computer to another computer.

[0004] 2. Related Art

[0005] EP-A-O 413 074 discloses a method of managing the transfer of data files between a workstation and a host computer. The workstation is provided with an application program which manages the transfer of files between the workstation and the host computer.

[0006] When transferring a large series of files from one computer to another computer, it is desirable to do so in a manner which does not cause problems for the computer receiving the files and which enables the files to be transferred in a reliable manner.

[0007] According to a first aspect of this invention there is provided a computer having a mechanism for transferring a series of data files between said computer and another computer, said file transfer mechanism being arranged to transfer a series of files between said computer and another computer in accordance with a protocol for transferring a series of files from a first computer (the transmitting computer) to a second computer (the receiving computer) which includes the following sequence of operations: (i) the receiving computer sends a poll message to the transmitting computer asking for a list of files which are available for transfer; (ii) the transmitting computer sends a reply message to the receiving computer containing a list of files which are available for transfer; and (iii) the receiving computer selects files from the list received from the transmitting computer and attempts to copy each selected file.

[0008] The protocol used in the file transfer mechanism of this invention provides the advantages that the files are transferred at a time selected by the receiving computer and that only those files which have been selected by the receiving computer are transferred. Also, because the files are copied under the control of the receiving computer, the protocol provides the further advantage that the files may be copied in a reliable manner.

[0009] According to a second aspect of this invention, there is provided a computer having a mechanism for transferring a series of data files between said computer and another computer, said file transfer mechanism being arranged to transfer a series of between said computer and another computer in accordance with a protocol for transferring a series of files from a first computer (the transmitting computer to a second computer (the receiving computer) which includes the following sequence of operations: (i) the transmitting computer sends a message to the receiving computer stating that it has files available for transfer; (ii) the receiving computer sends a poll message to the transmitting computer asking for a list of files which are available transfer; (iii) the transmitting computer sends a reply message to the receiving computer containing a list of files which are available for transfer; and (iv) the receiving computer selects files from the list received from the transmitting computer and attempts to copy each selected file.

[0010] According to a third aspect of this invention there is a provided a computer having a mechanism for transferring a series of data files between said computer and another computer, said file transfer mechanism being arranged to transfer a series of files between said computer and another computer in accordance with a protocol for transferring a series of files from a first computer (the transmitting computer) to a second computer (the receiving computer) which includes the following sequence of operations: (i) the receiving computer sends a poll message to the transmitting computer asking for a list of files which are available for transfer; (ii) the transmitting computer sends a poll message to the receiving computer stating that there will be a delay in providing a list of files available for transfer; (iii) after a delay the transmitting computer sends a further message to the receiving computer stating that it has files available for transfer; (iv) the receiving computer sends a poll message to the transmitting computer asking for a list of files which are available for transfer; (v) the transmitting computer sends a reply message to the receiving computer containing a list of files which are available for transfer; and (vi) the receiving computer selects files from the list received from the transmitting computer and attempts to copy each selected file.

[0011] According to a fourth aspect of this invention there is provided a method of transferring a series of data files from a first computer (the transmitting computer) to a second computer (the receiving computer) including the steps of: (i) the receiving computer sending a poll message to the transmitting computer asking for a list of files which are available for transfer; (ii) the transmitting computer sending a reply message to the receiving computer containing a list of files which are available for transfer; and (iii) the receiving computer selecting files from the list received from the transmitting computer and attempting to copy each selected file.

[0012] According to the fifth aspect of this invention there is provided a method of transferring a series of data files from a first computer (the transmitting computer) to a second computer (the receiving computer) including the steps of: (i) the transmitting computer sending a message to the receiving compute stating that it has files available for transfer; (ii) the receiving computer sending a poll message to the transmitting computer asking for a list of files which are available for transfer; (iii) the transmitting computer sending a reply message to the receiving computer containing a list of files which are available for transfer; and (iv) the receiving computer selecting files from the list received from the transmitting computer and attempting to copy each selected file.

[0013] According to a sixth aspect of this invention, there is provided a method of transferring a series of data files from a first computer (the transmitting computer) to a second computer (the receiving computer) including the steps of: (i) the receiving computer sending a poll message to the transmitting computer asking for a list of files which are available for transfer; (ii) the transmitting computer sending a reply message to the receiving computer stating that there will be a delay in providing a list of files available for transfer; (iii) after a delay the transmitting computer sending a further message to the receiving computer stating that it has files available for transfer; (iv) the receiving computer sending a poll message to the transmitting computer asking for a list of files which are available for transfer; (v) the transmitting computer sending a reply message to the receiving computer containing a list of files which are available for transfer; and (vi) the receiving computer selecting files from the list received from the transmitting computer and attempting to copy each selected file.

[0014] This invention will now be described in more detail, by way of example, with reference to the drawings in which:

[0015] FIG. 1 is a block diagram of two computers between which a series of files may be transferred and each of which embodies this invention;

[0016] FIG. 2 is a block diagram showing the software components of one of the computers of FIG. 1;

[0017] FIG. 3 shows the individual parts of the file transfer mechanism component of the software and the relationship between these processes and the application programs component of the software;

[0018] FIG. 4 is a graph illustrating a basic service provided by a protocol used in the file transfer mechanism;

[0019] FIG. 5 is a graph showing an alarm service provided by the protocol used in the file transfer mechanism;

[0020] FIG. 6 is a graph illustrating a delayed service provided by the protocol used in the file transfer mechanism;

[0021] FIG. 7 is a graph showing a message service used in the protocol provided by the file transfer mechanism;

[0022] FIG. 8 is a table illustrating the relationship between the states of a finite state machine used in a computer receiving data files and associated events;

[0023] FIG. 9 is a table illustrating the relationship between the states of a finite state machine used in a computer transmitting data files and associated events;

[0024] FIG. 10 shows the hardware components of each of the computers shown in FIG. 1; and

[0025] FIG. 11 is a block diagram of a system for billing telecommunications calls.

[0026] Referring now to FIG. 1, there are shown two computers, namely, computer A and computer B between which data files may be transmitted via a communications link 10. As will be explained in more detail below, each of computers A and B is provided with a file transfer mechanism embodying the invention and which is used in transferring data files. Each of computers A and B uses a UNIX operating system. Each of computers A and B may also transmit data files to, and receive data files from, other computers, not shown.

[0027] The main hardware components of each of computers A and B are shown in FIG. 10. These hardware components comprise a central processing unit (CPU) 90, memory 91, a visual display unit 92, a keyboard 93 and an input/output port 94 connected together by a bus 95. The port 94 is used to connect the computer to the communications link 10 and may be used to connect the computer to other devices such as a printer. The computer software, including data, is stored in the memory 91.

[0028] The present invention is particularly suitable for transferring a series of data files from a computer which produces such files in very large numbers. Examples of such computers may be found in a system for billing telecommunications calls. Referring to FIG. 11, there is shown an example of a system for billing telecommunications calls. The system comprises a telecommunications switch (or exchange) 100, which is part of a network of switches, and three computers 101, 102, 103. The switch 100 generates files containing records of individual telecommunications calls. The files are transferred to computer 101 which performs preliminary processing operations on the files in preparation for transferring the files to computer 102. The computer 102 receives the call record files from computer 101 and also files containing data relating to the individual customers of the network from the computer 103. Using the data relating to individual calls contained in the call record files and the data relating to the individual network customers contained in the customer data files, the computer 102 prices the individual calls and produces files containing records of the priced calls. The priced call record files are transferred to the computer 103. The computer 103 uses the priced call records to produce bills for the individual customers. Each of the computers 101, 102 and 103 is provided with a file transfer mechanism embodying the invention. These file transfer mechanisms are used for transferring the call record files from computer 101 to computer 102, the priced call record files from computer 102 to computer 103, and the customer data files from computer 103 to computer 102. This invention will now be described further with reference to computers A and B shown in FIG. 1.

[0029] Referring now to FIG. 2, there are shown the main components of the software (in addition to the operating system) of each of the computers A and B. These components comprise an application programs component 20, a file transfer mechanism component 22, a file transfer protocol component 24 and a communication component 26. The application programs component 20 comprises one or more application programs and the nature of these programs will depend upon the operations performed by the computer. For example, where the computer forms part of a billing system, the application programs will perform the functions required of that computer within the billing system. The details of such application programs do not form part of the present invention. The file transfer mechanism component 22 is responsive for the transfer of a series of files between the computer and another computer and thus provides overall control of each transfer of a series of files. The file transfer protocol component 24 is responsible for ensuring each file is copied in a reliable manner. The file transfer mechanism component 22 will be described below in grater detail. In the present example, the file transfer protocol component 24 uses the File Transfer Protocol (ftp) available on a UNIX operating system. By-way of an alternative, the file transfer protocol component 24 may use the File Transfer and Access Management (FTAM) protocol of Open Systems Interconnection (OSI). The communication component 26 is responsible for controlling the transmission of individual bytes of data between the computer and another computer and, in the present example, the communication component 26 uses the two well known protocols known by the collective name TCP/IP and these two protocols are available on UNIX operating systems.

[0030] Referring now to FIG. 3, there are shown the parts of the file transfer mechanism 22 and the relationship of these parts with the application programs 20. The parts comprise a receiver 30, a dispatcher 32, a file store 34 and a calendar 36. The receiver 30 is responsible for receiving data files from another computer and it transfers received files to a database for use by one or more of the application programs 20. Data files generated by one or more of the application programs 20 are transferred to the file store 34 in preparation for transfer to one or more other computers. Where the computer transmits files to only one other computer, the data files are stored in a single queue. Where it transmits files to more than one computer, the data files are stored in an individual queue for each of the other computers. Thus, the queue or each queue of data files forms a series of files which are available for transfer to another computer.

[0031] The dispatcher 32 is responsible for transmitting files from the file store 34 to one or more other computers. The file transfer mechanism 22 provides the opportunity for data files to be transmitted to other computers in accordance with a pre-defined schedule. Where data files are transmitted in accordance with the schedule, the calendar 36 provides the necessary time keeping function.

[0032] The transfer mechanism 22 is responsible for controlling each transfer of a series of files between the computer in which it is located and another computer. Each transfer is controlled in accordance with a protocol which is illustrated in FIGS. 4 to 7. This protocol provides a basic service illustrated in FIG. 4, an alarm service illustrated in FIG. 5, a delayed service illustrated in FIG. 6 and a message service illustrated in FIG. 7. In each of FIGS. 4 to 7, the computer which is receiving data files is referred to as an “initiator” and a computer which is transmitting files is referred to as a “responder ”.

[0033] Referring now to FIG. 4, in the basic service, in order to commence each transfer of a series of files the initiator sends a poll message 40 to the responder. At the same time, the initiator starts a timer. The poll message 40 includes a unique identifier for this transfer and details of the data files which it wishes to receive. These details are expressed as parameters which define the type of files which it wishes to receive. The unique identifier may contain the name of the initiator and a serial number for this transfer. If the timer expires before a reply message is received from the responder, the initiator retransmits the poll message 40 to the responder. In the event that the initiator transmits the poll message a preset number of times without receiving a reply message from the responder it ceases to send poll messages and issues an alarm. If the responder is able to generate a list of files which are available for transfer after receiving a poll message 40, it sends a reply message 42. The reply message 42 contains the unique identifier for this transfer as specified in the poll message 40 and also the list of files. For each file, the list gives the name of the file, the size in bytes of the file and the date of creation of the file.

[0034] On receiving the reply message 42, the initiator selects those files from the list which it wishes to receive. It then requests the responder to copy each file to it in turn. In FIG. 4, the requests for individual files are shown by dotted lines 44 and the copying operation for each file is shown by a solid, arrow headed line 46. The files are copied using the File Transfer Protocol (ftp) provided on the UNIX operating system mentioned above. It is important to note that the responder does not push any file to the initiator without being specifically requested to do so. Consequently, there is no danger of the initiator receiving a file which it is not prepared to receive. If the initiator fails to copy a file, it proceeds by attempting to copy the next file which it has selected from the list. It continues in this manner until it has attempted to copy all of the files. It then makes a further attempt to copy those files which it has not received. It continues attempting to copy each file up to a preset number of times and, if it fails to copy any file, it raises an alarm.

[0035] After copying all the selected files, or attempting to do so up to the present number of times, the initiator sends a message 48 to the responder. This message contains for each selected file the name of the file and a flag representing the outcome of the attempt to copy it. After receiving the message 48, the responder erases the data files which have been successfully copied from its file store 34. In a modification, the responder transfers the files which have been copied successfully to a further file store, not shown, from which they are eventually erased.

[0036] Referring now to FIG. 5, at the commencement of the alarm service, the responder sends an alarm message 60 to the initiator to inform it that it wishes to transfer a series of files to it. The responder may issue the message 60 because a point has been reached in the agreed schedule at which files are to be transmitted, or because an application program has reached a point in its execution at which it wishes to transfer files to the initiator or because the queue of files in the responder for transfer to the initiator has reached a preset threshold. At the time that the responder sends the alarm message 60 to the initiator, it starts a timer. The alarm message 60 contains a unique identifier for the transfer together with an indication that files are available for transfer. If the timer expires before the responder receives a reply from the initiator, it retransmits the alarm message 60 and resets the timer. It retransmits the alarm message 60 upon each expiry of the timer up to a preset number of times and then raises an alarm.

[0037] If the initiator is able to receive files, in response to the alarm message 60, it sends a poll message 62 to the responder. The poll message 62 is identical to the poll message 40 described with reference to FIG. 4 except that it contains the unique identifier contained in the alarm message 60 and an acknowledgement that it has received this alarm message. After the responder has received the poll message 62, the alarm service continues in the manner described with reference to the basic service after the responder has received the poll message 40.

[0038] Thus, the transfer of messages and data files between the initiator and responder in the alarm service is identical to that in the basic service except that the alarm service includes the alarm message 60 and the poll message 62 uses the unique identifier contained in the alarm message 60 and acknowledges receipt of the alarm message.

[0039] The responder uses the delayed service when it requires some time to make files requested by the initiator available for transfer. For example, if the responder contains an application program which is capable of retrieving the requested files from an archive and transferring these to the data store of the associated transfer mechanism, there will be a delay in making these files available for transfer.

[0040] The delayed service commences in the same way as the basic service with the initiator sending a poll message 70 to the responder. The poll message 70 is identical to the poll message 40 of the basic service and, if necessary, is retransmitted up to the preset number of times.

[0041] After receiving the poll message 70, the responder sends an acknowledgement message 72. The acknowledgement message 72 includes the unique identifier of the poll message 70 and a statement that there will be a delay before files are available for transfer.

[0042] After a delay, when either files are available for transfer or the responder has found that it is not able to provide any files of the type requested in the poll message 70, it transmits an alarm message 74 to the initiator. The alarm message 74 contains the unique identifier of the poll message 70 and indicated either that files are available for transfer or that files cannot be made available for transfer. If the responder does not receive a reply to the alarm message 74, it retransmits the alarm message 74 in the same manner as described for the alarm message 60 of the alarm service.

[0043] In response to the alarm message 74, the initiator transmits a further poll message 76. The poll message 76 includes the unique identifier of the poll message 70 and also the details of the data files which it wishes to receive.

[0044] In response to the poll message 76, the responder transmits a reply message 42 which identical in format to the reply message 42 used in the basic service. The files are then transferred and acknowledged in the same manner as described with reference to the basic service.

[0045] Thus, the delayed service is identical to the basic service but additionally includes the poll message 70, the acknowledgement 72 and the alarm message 74. Also, the poll message 76 re-uses the unique identifier from the poll message 70 and acknowledges the alarm message 74.

[0046] In each of the basic service, the alarm service and the delayed service, the transfer of data files is controlled by the initiator. More specifically, in each service, the initiator receives a list of files in response to a poll message and thus there is not danger that the initiator will receive a list of files until it is ready to process the list. In each service, after receiving the list, the files are selected and copied under the control of the initiator and so there is no danger that the initiator will receive any file which is not ready to receive. The alarm service provides the further advantage that the responder can indicate that files are ready for transfer.

[0047] Although the basic service, alarm service and delayed service have each been described as forming part of the protocol used by the file transfer mechanism, it is to be appreciated that each of these services is also an individual protocol which could be used on its own or could be modified to include further messages.

[0048] The message service is used to transfer a comparatively small amount of data from the responder to the initiator. At the commencement of the message service, the initiator sends a request message 80 to the responder and at the same time it starts a timer. The request message 80 contains a unique identifier and also the details of the data required from the responder. If the timer expires before receiving a reply message from the responder, the initiator retransmits the request message 80 and restarts the timer. If no reply is received, it retransmits the request message 80 and restarts the timer up to a preset number of times.

[0049] If the responder is able to provide the data, it sends a reply message 82 to the initiator which contains the unique identifier of the request message 80 and the requested data.

[0050] For each individual transfer using one of the four services described above, both the initiator and responder create a finite state machine. Each finite state machine changes state after a given event has occurred. Each event is associated with an action process comprising a set of steps.

[0051] The events are set out in Table 1. In this table, the suffix “req” denotes the sending of a message and the suffix “ind” denotes the receipt of a message. Each finite state machine can assume six states and these six states are set out in Table 2. It should be noted that the meaning of each state differs between the initiator and the responder.

[0052] Note that some of the events are handled by the initiator and the others by the Responder.

[0053] The state tables for the initiator and responder are shown, respectively, in FIGS. 8 and 9. In each of these figures, the events are set out in the left hand column and the six states are set out across the top of the table. The numerals in the boxes indicate the new state following a particular event and the letters and numerals contained in brackets provide a reference to the action process associated with a particular event. Thus, in the initiator state table shown in FIG. 8, following an event POLLreq, the state changes from state 0 (IDLE) to state 1 (POLL). The event POLLreq is associated with an action process A0. The various action processes will now be described. For each action process, the series of steps differs between the initiator and the responder.

[0054] Action Process A0

[0055] In this action process, the initiator generates a new finite state machine and a unique identifier for the transfer, formats the poll message and sends it to the responder, starts a timer and then enters State 1. The responder generates a new finite state machine, passes the poll message to an application program in the responder and then enters State 1.

[0056] Action Process A4

[0057] In this action process, the initiator formats a poll message and sends it to the responder, starts a timer and then enters State 1. The responder passes a poll message to an application program and then enters State 1.

[0058] Action Process B1

[0059] In this action process, the initiator selects the data files which it wishes to receive from the list of data files supplied by the responder, determines the total size of the data files and checks that is has free space available and then copies each file in turn. If it fails to copy a file, it attempts to copy the next one and it repeats its attempt to copy each file up to a preset number of times. It then enters State 2 and prepares to send an acknowledgement message. For this action process, the responder formats a poll response message, sends it to the initiator and then enters State 2.

[0060] Action Process C1

[0061] In this action process, the initiator changes the state of its finite state machine for the transfer to State 3. In this action process, the responder formats a poll acknowledgment message, send it to the initiator and enters State 3.

[0062] Action Process D2

[0063] In this action process, the initiator formats an acknowledgement message, sends it to the responder, causes its finite state machine to enter State 0 and then deletes the finite state machine. The responder passes the acknowledgement message to the application program, causes its finite state machine to enter State 0 and then deletes the finite state machine.

[0064] Action Process E0

[0065] In this action process, the initiator creates a new finite state machine, stores the unique identifier received in the alarm message from the responder, passes the alarm message to the application program and then causes its finite state machine to enter State 4. The responder generates a new finite state machine and a unique identifier for this transfer, formats an alarm message and sends it to the initiator and then causes its finite state machine to enter State 4.

[0066] Action Process E3

[0067] In this action process, the initiator passes the alarm message to the application program and then causes the finite state machine for this transfer to enter State 4. The responder formats an alarm message, sends it to the initiator and causes its finite state machine for this transfer to enter State 4.

[0068] Action Process F0

[0069] In this action process, the initiator creates a new finite state machine and an unique identifier for this transfer, formats a message request and sends it to the responder and causes the finite state machine to enter State 5. the responder generates a new finite state machine, passes the request message to an application program and causes the finite state machine to enter State 5.

[0070] Action Process G5

[0071] In this action process, the initiator passes the response message to an application program, causes the finite state machine to enter State 0 and deletes the finite state machine. The responder formats a response message and sends it to the initiator, causes the finite state machine to enter State 0 and then deletes the finite state machine.

[0072] Action Process H1

[0073] This action process relates only to the initiator. The initiator increments a counter for counting the number of retries. If the counter has not yet reached the preset maximum value, it formats the poll message, sends it the responder and restarts the timer. If the reset counter has reached the preset maximum number, it provides an indication that there is an error, changes the state of the finite state machine to State 0 and then deletes the finite state machine.

[0074] Action Process H4

[0075] This action process relates only to the responder. In this action process, the responder increments a counter which counts the number of retries. If the counter has not yet reached a preset maximum value, it formats an alarm message and sends it to the initiator and then restarts the timer. If the counter has reached the preset maximum value, it provides an indication that an error has occurred, causes its finite state machine to enter State 0 and then deletes the finite state machine.

[0076] Action Process H5

[0077] This action process relates only to the initiator. In this action process, the initiator increments a counter for counting the number of retries. If the counter has not reached a preset maximum value, it formats a request message, sends it to the responder and then restarts the timer. If the counter has reached the preset maximum value, it provides an indication that an error has occurred, changes the state of the finite state machine to State 0 and then deletes the finite state machine.

[0078] Computers A and B together with other computers may form part of a system under the overall control of a central computer. Each computer in the system may send a log of each transfer to the central computer. If a queue of data files in a file transfer mechanism of one of the computers reaches an excessive level, the computer can send an indication of this to the central computer which can then take appropriate action to deal with this problem.