Data Subsystem Traffic Control Apparatus and Method

This application is related to one, or more, of the following U.S. patent application Ser. Nos. 670,682; 670,701 both filed on Nov. 13, 1984, Ser. Nos. 705,456; 705,457; 705,458; 705,459; 705,460; 705,461; 705,462; 705,463; 705,464 all filed on even date herewith. All of the above applications are assigned to the assignee hereof.

The present invention generally relates to an apparatus for controlling data traffic across a data subsystem and, in particular, relates to such a subsystem including a plurality of interconnected but independent means for autonomously controlling the flow of data traffic thereacross.

In the general field of modern communications systems, a data communication network is frequently formed by interconnecting a plurality of subsystems, or nodes. Conventionally, each subsystem, or node, is designed to support a prespecified capacity of data traffic. This capacity is normally dependent upon the subsystem hardware and the number of network connections throughout the network. Consequently, in the early stages of the design of a particular subsystem and/or network, one of the more critical issues to be resolved is the data flow capacity requirements of the end users. Without question, subsystems and networks can be designed to handle just about any prespecified data flow capacity. However, a major difficulty lies in the unanticipated increase in data handling requirements of the users after the network is installed.

Hence, when the data handling requirements of a network exceeds the anticipated, or design, levels thereof, a user must either commit to an expensive system redesign or suffer with reduced services.

Another common difficulty of modern subsystems is that, occasionally, during peak use by a small number of subscribers the remaining subscribers are either denied access to the network completely or are subjected to ergonomically unacceptable delays.

Considering the present growth in data transport technology in such fields as computers, microcomputers, electronic mail and so on, the demands upon data communication networks and the subsystems interconnected thereto has escalated far beyond anticipated levels. Such technological growth is anticipated to continue to outstrip the present day communication network designs.

In addition, many individual subscribers to such networks are requiring increased services. Further, numerous services are presently available to supply subscribers with such services as electronic mail, data bases of information, for example, in such fields as finance, law, business, chemistry, and numerous others. Consequently, more and more users are facing the difficulties discussed above.

In light of the above, it is clear that a growing need exists for a subsystem that is both autonomously adaptable to data traffic flow and inexpensively predesigned and adapted for modular expansion.

Accordingly, it is one object of the present invention to provide a subsystem apparatus that autonomously adapts to the data traffic flow thereacross and a method for use with such a subsystem.

This object is accomplished, at least in part, by a subsystem having a plurality of devices each autonomously regulating the data flow thereacross and wherein the network is provided with the means for automatically programming new modular devices as each is introduced into the subsystem.

Other objects and advantages will become apparent to those skilled in the art from the following detailed description read in conjunction with the appended claims and the drawings attached hereto.

FIGS. 1, 1A, 1B are block diagrams of a data communication network having a plurality of subsystems interconnected thereby and embodying the principles of the present invention;

FIG. 2 is a more detailed block diagram of the data transport controller shown in FIG. 1;

FIG. 3 is a pictorial of one implementation of the subsystem shown in FIG. 1;

FIG. 4 is a graphic illustration of total subsystem throughput as a function of processing time for a fixed number of peripheral interface devices and varying numbers of network interface devices;

FIG. 5 is a graphic illustration of delay characteristic versus offered packet traffic;

FIG. 6 is a functional block diagram of the mechanisms provided by each microcomputer of a subsystem having a multi-microcomputer architecture;

FIGS. 7A and 7B depict a state diagram and associated state table for a rate adaptation function;

FIGS. 8A-8C depict a state diagram and associated state tables for a traffic distribution function; and

FIGS. 9A-9F depict a state diagram and associated state tables for a congestion detection and control function.

A communication network, generally indicated at 10 in FIG. 1 and embodying the principles of the present invention, includes a plurality of substantially identical subsystems 12 interconnected to a networking medium 14 whereby communication is established between and among peripherals 16 interconnected to each subsystem 12. In one embodiment the networking medium 14 is a digital switching network such as the ITT SYSTEM 12 digital exchange.

In the preferred embodiment, each subsystem 12 includes one or more peripheral interface device 18, one or more network interface device 20, and a one or more gateway interface device 22. In addition, at least one of the subsystems 12 further includes a network control interface device 24 that, as more fully discussed below, interfaces with a system control means 26. The system control means 26 being operative to control a data bank 28 having stored therein, inter alia, all operative communication protocols for the communication network 10. Further, the peripheral interface devices 18, the network interface devices 20, the gateway interface devices 22 and, when present, the network control interface device 24 of each subsystem 12 are all interconnected by a multiple access masterless contention bus 30 having collision detection. The contention bus 30, hereinafter referred to as the intrasubsystem bus 30, thus establishes a communication path between and among all devices, 18, 20, 22 and 24, within each subsystem 12. Thus, the system control means 26 has access, via the network control interface device 24, to every subsystem 12 via the networking medium 14.

Each peripheral interface device 18 includes a data transport controller 32 that interfaces with the plurality of peripherals 16 via a plurality of universal synchronous/asynchronous receiver/transmitter (USART) devices 34. Each network interface device 20 includes a data transport controller 32 that interfaces with the networking medium 14 via a PCM interface device 36. Further, each gateway interface device 22 and network control interface device 24 also include such data transport controllers 32. Preferably, each subsystem 12 is of the type described and discussed in U.S. patent application Ser. No. 705,464 entitled "Communication Subsystem" filed on even date herewith and assigned to the assignee hereof. This application is deemed incorporated herein by reference. Further, each data transport controller 32, for each device 18, 20, 22 or 24, is substantially identical with every other such data transport controller 32 throughout the network 10. The primary difference among the data transport controllers 32, as more fully discussed below, is the traffic control tasks assigned thereto. Thus, each subsystem 12 has a fully distributed multi-microcomputer architecture.

Referring to FIG. 2, a more detailed block diagram of a data transport controller 32 and, as shown, includes a peripheral interface controller 38, a communication bus interface 40, a microcomputer 42 and a storage medium 44. In the preferred embodiment, the peripheral interface controller 38 is adapted to service a plurality of peripherals 16 operating according to the same comunication protocol. Such a peripheral interface controller 38 includes both a general peripheral control program and a device specific subprogram adapted to allow intercommunication to each of various types of peripherals 16 according to the individual operating requirements thereof. One particular peripheral interface controller 38 specifically adapted to provide such services is fully described and discussed in U.S. patent application Ser. No. 705,458 entitled "Device Interface Controller" filed on even date herewith and assigned to the assignee hereof. This application is deemed incorporated herein by reference.

In the preferred embodiment, the communication bus interface 40 interfaces with the masterless multiple access contention bus 30 with collision detection having a first bus 46, shown more clearly in FIG. 3, for sustaining data transmissions therealong and a second bus 48, distinct from the first bus 46, provided solely for collision detection.

Examples of such an interface 40 and bus 30 are described and discussed in U.S. patent application Ser. Nos. 670,682 and 670,701 entitled "Apparatus for Masterless Collision Detection" both filed on Nov. 13, 1984 and assigned to the assignee hereof. These applications are deemed incorporated herein by reference.

The microcomputer 42 preferably includes a microprocessor 50, a read-only-memory (ROM) 52, a random-access-memory (RAM) 54 and a means 56 for providing local services, such as high level protocol conversions that are relatively infrequently required, for example, for providing gateway functions. The microprocessor 50, the ROM 52, the RAM 54 and the means 56 are interconnected via the local bus 58 of the microcomputer 42. Preferably, the microcomputer 42 is directly connected to the peripheral interface controller 38 and the communication bus interface 40 only for the exchange of interrupt and channel attention signalling via lines, 60 and 62, respectively. Preferably, the microcomputer 42 is a high capacity mirrocomputer, such as the 80186 microprocessor device, manufactured and marketed by INTEL Corporation of Santa Clara, Calif. Such a microcomputer 42 includes sufficient capacity to execute the requisite single protocol conversion and to regulate the data traffic thereacross. An apparatus and method for providing a peripheral interface device 18 for each external protocol is described and discussed in U.S. patent application Ser. No. 705,461 entitled "Apparatus and Method for Executing Communication Protocol Conversions" filed on even date herewith and assigned to the assignee hereof. This application is deemed incorporated herein by reference. One data transport controller 32 particularly adapted for use in the present network 10 is described and discussed in U.S. patent application Ser. No. 705,457 entitled "Data Transport Control Apparatus" filed on even date herewith and assigned to the assignee hereof. This application is deemed incorporated herein by reference.

In one preferred implementation, shown in FIG. 3, a subsystem 12, designed according to the principles of the above-referenced U.S. patent application Ser. No. 705,464 entitled "Communication Subsystem" includes a printed circuit board mainframe 60, having means 62 for accepting a plurality of printed circuit boards therein. The printed circuit board mainframe 60 preferably includes the intrasubsystem bus 30 etched thereinto. Further, each peripheral interface device 18, each network interface device 20, each gateway interface device 22 and the network control interface device 24 is self-contained on separate printed circuit boards, 64, 66, 68 and 70, respectively. Hence, in this embodiment, the subsystem 12 is substantially completely self-contained and is, for all intents and purposes, prewired to be modularly expanded. Thus, any peripheral 16 operating under a previously unavailable communication protocol can be accommodated simply by the insertion of a printed circuit board carrying a new peripheral interface device 18 dedicated to only that protocol. Consequently, each subsystem 12 and the entire network 10 are easily expanded without reprogramming or redesigning the entire system. Naturally, depending upon the number of peripherals 16 having the same communication protocol to be connected, more than one peripheral interface device 18 can be provided for executing any specific communication protocol conversion. Thus, each subsystem 12 includes the means 62 for accepting additional peripheral interface devices 18 whereby additional peripheral devices can be added and interconnected to the intrasubsystem bus 30. Further, each subsystem 12, as discussed above, includes a means 62 for providing additional network interface devices 20 whereby the total traffic flow capacity through the subsystem 12 can be increased in a modular fashion.

The benefits derived from this implementation are graphically illustrated in FIG. 4 where the total subsystem throughput, in packets/second, is shown as a function of the proportion of processing time per packet between the network interface device 20 and the processing time per packet for the peripheral interface device 18 and the number of network interface devices 20 in that subsystem 12. The various delay characteristics associated with offered packet traffic for intrasubsystem transport, using the intrasubsystem bus 30, and the intersubsystem transport, using the networking medium 14 is depicted in FIG. 5.

In one preferred embodiment, the network 10 includes a uniform communication protocol for intrasubsytem data communication and for intersubsystem data communication via the network medium 14, the communication protocol being stored in the data bank 28 for downloading into the RAM 54 of the data transport controller 32 of any peripheral interface device 18 or any network interface device 20 introduced into the network 10.

In one embodiment, the system control means 26 can be a personal computer, or the functional equivalent thereof, having the data bank 28 stored, for example, on a floppy disk. The system control means 26 is adapted to monitor the status of each device, 18, 20 and 22 in each subsystem 12. The downloading to newly inserted devices is initiated according to a preprogrammed portion of the ROM 52 of every data transport controller 32 that, upon insertion into the means 62, alerts the system control means 26. The system control means 26 then downloads the appropriate software program into the RAM 54. The system control means 26, during the monitoring of the devices 18, 20 and 22, ensures that the newly inserted device 18, 20 or 22 is acknowledged by the remainder of that subsystem 12. Such programming for effecting a program downloading is within the skill of those familiar with the software arts.

A functional block diagram of the primary traffic control mechanisms provided is shown in FIG. 6. It will be understood that, due to the multi-microcomputer distributed nature of each subsystem 12, by the provision of a common data transport controller 32, each such controller 32 is provided with only those functions necessary to the operation thereof. For example, each peripheral interface device 18 includes those functions relating to intrasubsystem data transport and data transport with a peripheral 16 but not functions relating to the data traffic across the networking medium 14. However, because of the commonality and distribution of the data transport controllers 32, a uniform nested-layered communication protocol is provided. The provision of such a uniform network-wide communication protocol further enhances the autonomous traffic control by each device 18, 20 and 22, and, hence, each subsystem 12. Preferably, the communication protocol is based on the Open System Interconnection model of the International Standards Organization (hereinater OSI-ISO) and provides the first three level services thereof. As shown in FIG. 6, in addition to the normal services, designated at 72, each data transport controller 32 is provided with, rate adaptation 74, traffic distribution 76 and congestion detection and congestion control 78 functions.

The normal services take advantage of the, effectively, independent access to the storage medium 44 of each data transport controller 32 by the bus interface 40 and the peripheral interface controller 38 to dynamically assign the medium 44 on a demand basis. Thus, each channel transferring information via any storage medium 44 throughout each subsystem 12 is assigned memory according to its instantaneous need rather than on a fixed block basis. Hence, congestion is, in many cases, avoided where it would normally occur due to exceeding an assigned fixed block of memory during times when other preassigned fixed blocks of memory were empty or only partially full.

The rate adaptation function 74 is downloaded into every data transport controller 32 at each peripheral interface device 18 throughout the network 10. Preferably, the rate adaptation function 74 is implemented according to the state machine diagram shown in FIG. 7A based on the state table, shown in FIG. 7B and in accordance with the following definitions:

The rate adaptation function 74 is provided to equalize the data transfer rate between a fast data rate source and a slow data rate destination. Such a function more efficiently utilizes all commmunication links throughout the network 10.

Preferably, the traffic distribution function 76 is implemented according to the state machine diagram shown in FIG. 8A based on the state tables of FIGS. 8B and 8C and in accordance with the following dictionary:

The traffic distribution function 76, for example, across the network interface devices 20 on opposite sides of the networking medium 14 relies on status reporting between the data transport controllers 32 interconnecteed via the intrasubsystem bus 30. This reporting allows identification of the least loaded network interface device 20 in a subsystem 12. To distribute load uniformly among network interface devices 20 in the involved subsystem 12, a new virtual link is established between the least loaded network interface device 20 in each of the subsystems 12.

The congestion detection and control function 78 is downloaded into each data transport controller 32 and preferably is implemented according to the state diagram shown in FIG. 9A based on the state tables of FIGS. 9B-9F and in accordance with the following dictionary:

The following definitions describe the variables and procedures of the congestion detection and control which are used in the state machine transition tables.

In the preferred embodiment, the first congestion in the system is defined as the operational point of offered traffic to a data transport controller 32 wherein the available free buffer in the storage medium 44 thereof approaches complete utilization. lf this is allowed to continue with increasing offered traffic the performance of that data transport controller 32 would degrade as a result of lost packets due to unavailable memory and also the potential for deadlock would also occur. Therefore, congestion detection consists of monitoring the size of the free buffer in the storage medium 44 with respect to three thresholds. When the size of the buffer is below the first threshold the congestion control mechanism throttles new input traffic. When the buffer size falls below the second threshold then transit traffic is throttled. In the first case though input traffic was throttled; transit traffic was not affected for the reason of attempting to provide service to those packets that have already been partially serviced.

If the free buffer size falls below a third threshold then virtual circuits are torn down and packets may be discarded. For peripheral interface devices 18, input traffic is from the external peripherals 16 and transit traffic comes from the intrasubsystem bus 20. The network interface devices 16 have input traffic from the intrasubsystem bus 20 and transit traffic from the networking medium 14.

The above-described subsystem 12 is responsive to the growing need for the inexpensive expansion of subsystems 12 connected to a communications network 10, while avoiding a complete system redesign. Thus, the fact that, in any given subsystem 12, certain paths are more heavily used than others and the fact that it is not predeterminable which paths are to be the heaviest used and, additionally, the fact that the location of the heaviest used paths changes, does not affect the present subsystem 12. That is, the traffic flow across a subsystem 12 having, inter alia, a plurality of network interface devices 20 is both autonomously and automatically distributed by each network interface device 20 therein.

The communication system 10 as described herein is applicable to almost any type of network. Example of such applications can be found in U.S. patent application Ser. Nos. 705,462, 705,463 and 705,460, entitled "System For Providing Data Services To A Circuit Switched Exchange", "Data Communication Network" and "Full Service Voice/Data System" respectively, all filed on even date herewith and assigned to the assignee hereof. These applications are deemed incorporated herein by reference. The referenced application Ser. Nos. 705,462 entitled "System For Providing Data Services To A Circuit Switched Exchange" describes and discusses the provision of data services, via a data services adjunct, to subscribers of a circuit switched exchange. The data services adjunct therein is, essentially, comparable to the autonomous subsystem 12 having a multi-microcomputer architecture described herein. The referenced application Ser. No. 705,463 and 705,460 entitled "Data Communication Network" and "Full Service Voice/Data System" respectively describe and discuss a data only network and an integrated full service voice/data network respectively. Each of these networks can be designed to include autonomous subsystems embodying the principles of the present invention.

The present apparatus has been described with reference to a particular exemplary embodiment and is not intended to be limiting as other arrangements and configurations may be made without departing from the spirit and scope of the present invention. Consequently, the present invention is deemed limited only by the attached claims and the reasonable interpretation thereof.