Method and Apparatus for Arbitrating for a Bus to Enable Split Transaction Bus Protocols

The present invention relates to the transfer of information over a bus, and more particularly, to arbitration methods and apparatus for arbitrating for use of the bus among a plurality of masters and slaves coupled to the bus.

In personal computers, the utilization of an input/output (I/O) bus can be a limiting factor if this bus is not properly designed and implemented. Certain I/O buses, such as the industry standard architecture (ISA) bus, are adequate for single-user personal computers that do not require extensive video and direct access storage device (DASD) access. With the advent of multi-tasking operating systems and multi-processor (MP) hardware, however, the video and DASD subsystems take up the available I/O bus and memory subsystem bandwidths. This can severely limit the performance of the system as a whole. The attempts to solve this problem involve the use of new and faster I/O buses, such as the Microchannel and the EISA buses. Although partially successful, the demand for even faster buses has become apparent. These faster buses have been necessitated by the faster I/O devices (bus masters) that have become available. The previous buses became overwhelmed, especially in workstations and servers, where multiple masters (such as DASD, LAN, video adapters) commonly reside on the same I/O bus.

The personal computer industry has recently defined a bus standard known as the peripheral component interface (PCI) bus in an attempt to meet the current needs for a faster I/O bus. The PCI bus is well known and the specification of this bus is herein expressly incorporated by reference.

The PCI bus is a general purpose, standard bus, with high throughput. As currently specified, the PCI bus has a theoretical potential of transferring of up to 264 MB/sec. However, this performance is usually never attained since the slave device (typically memory) cannot access or store data at this high rate. Most memory subsystems are limited to a 40-80 MB/sec data transfer rate. The slave device is then forced to insert wait states between the address and data phases of the transfer. This can easily cut the actual bandwidth to 1/2 or 1/4 of the theoretical maximum bandwidth. It is therefore desirable to increase the usable bandwidth of the PCI bus towards its theoretical maximum.

In typical computer systems, there are commonly three phases to a data transfer:

(1) the arbitration phase;

(2) the address/command phase (hereinafter the "address phase"); and

(3) the actual data transfer phase.

The PCI bus is a "connected transaction bus" which is defined as a bus for which a data phase always immediately follows an address phase (wait states are not considered). In other words, the address phase and data phase are connected such that the data transferred during a data phase is always associated with the address placed on the bus immediately preceding the data phase. The PCI bus allows arbitration phases to overlap with the address and data phases, but does not allow address and data phases to overlap, nor does it allow data phases to occur out of order with respect to the address phase.

By contrast, a bus known as a "split transaction bus" typically has four phases to a data transfer:

(1) address arbitration phase;

(2) address/command phase;

(3) data arbitration phase; and

(4) data transfer phase.

In a split transaction bus the data phase does not have to immediately follow the associated address phase placed on the bus. It is not necessary that the data phases occur in the same order that the corresponding address phases occur. Therefore, it is possible for a master to execute multiple address phases before any data is returned. It is also possible to overlap a second master's address phase with a first master's data phase. The address arbitration in a split transaction bus is similar to that of a connected transaction bus. One difference, however, is that on the address phase, the master asserts an address/command signal and a transaction identification (ID) and then immediately releases the address bus. The master may then either wait until the slave returns the data or re-arbitrate for the address bus and execute additional address phases. The slave stores the address/command and the master identification issued during the address phase and then arbitrates for the data bus when it is ready to complete the transfer. When the slave is granted the data bus, it places the transaction ID on the bus at the same time it places the data on the data bus. The master then recognizes its transaction ID and will accept the data, thereby completing the transaction.

In contrast to a split transaction bus, a conventional PCI bus does not use a master that immediately releases the address bus upon the assertion of address/command and identification. Instead, on a conventional PCI bus, the signaling by a slave that it is not ready to complete the transaction causes a retry of the master (a "back-off/retry" feature). This causes the PCI master to re-arbitrate immediately in order to retry the transaction again. The master will normally keep requesting the bus and re-trying the same transaction until the slave is ready, making it more difficult for other masters to use the bus. This reduces the usable bandwidth of the PCI bus.

There is a need for an arbitration mechanism and arrangement that will allow a connected transaction bus, such as the PCI bus, to operate in a split transaction bus manner in order to increase the usable bandwidth. With respect to the PCI bus in particular, there is a need to allow split transactions without affecting the PCI architecture or requiring existing or future PCI masters to change.

These and other needs are met by the present invention which provides a system comprising a bus, a plurality of masters coupled to the bus, these masters including logic for requesting arbitration for the bus and for initiating a transaction upon being granted the bus, a plurality of slaves coupled to the bus, with at least one of the slaves including logic for indicating readiness of the slave to complete the transaction, and an arbiter. The arbiter includes logic for forcing a master off the bus when a slave from which the transaction has been requested is not ready to complete the transaction. Logic is also provided for preventing the master from re-arbitrating for the bus until the slave is ready to complete the transaction.

By preventing the master from re-arbitrating for the bus until the slave is ready to complete the transaction, the bus is freed for use by other masters. This directly leads to an increase in the usable bandwidth of the bus towards its theoretical maximum potential.

In certain embodiments of the invention, the slave will assert a first signal indicating that it is not ready to complete a requested transaction. The signal causes a mask bit (a mask indicator) corresponding to the requesting master to be set in a mask register.

Preferred embodiments of the invention use a central arbiter in which the mask register is located. Once this mask bit is set, the master can continue requesting the bus, but the central arbiter will ignore the request and not grant the bus to the masked master.

In order to allow the requesting master to actively arbitrate for the bus, a slave that is ready to complete the transaction will send a second signal, which includes a mask release signal, that causes the mask bit corresponding to the requesting master to be re-set. This allows the master to re-arbitrate for the use of the bus and complete the transaction.

Certain preferred embodiments of the present invention include means for temporarily assigning the highest arbitration priority to the master whose mask has been re-set until the master is granted the bus. Once granted, the master's priority reverts back to its original value. This has the advantage of assuring that the unmasked master will be the next device to be granted the bus, thereby completing the split transaction in a timely manner.

In order to avoid the use of a master identification bus, certain embodiments of the invention release all of the masters that have been masked upon the assertion of a second signal by any one of the slaves. If a master that has been unmasked is granted priority, but the slave for which the transaction is requested is still not ready to complete the transaction, this master is simply re-masked and the remaining unmasked masters re-arbitrate. This occurs until the master that requested the transaction that is now ready to be completed wins the arbitration for the bus. This method is a less efficient manner of executing the split transaction than the use of a master identification bus. However, this embodiment has the advantage of being less expensive and simpler to implement with respect to the hardware.

The foregoing and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.

FIG. 1 is a block diagram showing a system that uses the arbitration mechanism and method of the present invention, and includes a central arbitration device.

FIG. 2 is a block diagram similar to FIG. 1, but shows an embodiment that does not use a MASTER.sub.-- ID bus.

FIGS. 3, 3A, and 3B are a flow diagram illustrating an exemplary embodiment of the arbitration method of the present invention.

In the description which follows, the example of the connected transaction bus used is a PCI bus. However, other buses which are connected transaction buses having a back-off/retry feature can be used with the arbitration mechanism of the present invention.

A system arrangement is shown in FIG. 1 in which a multi-processor architecture is illustrated, although the invention is applicable to uni-processor applications. The multi-processor architecture illustrated in FIG. 1 is a symmetrical multi-processor architecture with shared coherent distributed memory. The exemplary architecture comprises a first cluster 10 and a second cluster 12 whose multi-processor buses are coupled by an inter-cluster link 14. Each multi-processor cluster 10, 12 has a plurality of CPU's 16, 18 coupled to the multi-processor bus 19 of that cluster. The multi-processor bus 19 is coupled to a memory controller 20. This memory controller 20 supplies data to the processors, such as CPU's 16, 18, as well as to I/O masters 26, 28. The I/O masters 26, 28 are coupled via a PCI bus 30 to the memory controller 20 and a PCI central arbiter 22.

In the architecture described in FIG. 1, latencies of data transfers can become large, and will affect both performance and available bus bandwidths. This becomes evident especially when a master on the PCI bus 30, such as master 26 of the first cluster 10, reads data located in memory located on cluster 12. The total latency for the reply data from memory could exceed three microseconds, for a transfer of a cache line. This would tie up the PCI bus for three microseconds for a 32-byte transfer, greatly limiting the available PCI bus bandwidth. The arbitration mechanism of the present invention, however, increases the overall bus bandwidth without affecting the PCI architecture and without requiring existing PCI masters to change.

The present invention provides for the communication of three sideband signals between the memory controller 20 and the PCI arbiter 22. These signals are the MASK.sub.-- MASTER; REL.sub.-- MASTER; and MASTER.sub.-- ID. The use of these signals by the arbitration mechanism of the present invention will be described in more detail below.

For ease of explanation, in the following operational example, a master 26 desires a transaction with the memory controller 20 of its own cluster, such as cluster 10. The master 26 arbitrates for the use of the PCI bus 30 with the master 28. Assume that the central arbiter 22 determines that the master 26 wins the arbitration and grants the master 26 the use of the PCI bus 30. The master 26 asserts address and control signals, as well as FRAME# to request the data from the memory controller 20. In the prior art arrangements, if the memory controller 20 is not ready to send the data at this time, it will insert wait states until the controller 20 is ready to complete the transaction. At the same time, if other masters such as master 28 wish to utilize the bus, they are unable to do so since the transaction between master 26 and the memory controller 20 is occupying the bus. This arbitration sequence occurs in the prior art since the PCI bus is normally a connected transaction bus in which the data phase for the transaction immediately follows the address phase for that transaction on the bus, with wait states not counting.

The apparatus and method of the present invention overcomes this limitation of the PCI bus 30 by allowing it to perform split transactions. To do so, the requesting master 26 is forced off the PCI bus 30 and prevented from re-arbitrating until the memory controller 20 signals that it is ready to complete the transaction. This increases the usable bandwidth since other masters such as master 28 can arbitrate for and make use of the PCI bus 30 during this time. The present invention makes use of two additional signals, the MASK.sub.-- MASTER signal and the REL.sub.-- MASTER signal that are provided as sideband signals on the PCI bus 30. A third signal, the MASTER.sub.-- ID signal, is used in certain embodiments to further increase bus efficiency by indicating which master is to be masked/released.

An embodiment of the method of the present invention is illustrated in flow diagram form in FIG. 3. In this example, the master 26 desires a transaction with the memory controller 20 which is acting as the slave for this transaction. In step 50, the master arbitrates for use of the PCI bus 30 with the other masters 28, etc. The central arbiter 22 determines whether a mask bit corresponding to the requesting master 26 is set in a SPLIT.sub.-- MASK register 23 in the arbiter 22. This is performed in step 52. If the mask bit is not set, the arbiter 22 allows the master 26 to arbitrate for the PCI bus 30. If the master 26 is not granted the bus (step 54), the process returns to step 50 where the master 26 will continue arbitrating for the use of the bus 30.

Assume now that the master has won the arbitration and has been granted the bus 30 by the arbiter 22. The winning master 26 asserts FRAME# and address and control signals to the memory controller 20 on the PCI bus 30. It is then determined whether the slave (memory controller 20) is ready to complete the transaction (step 58). If the memory controller 20 is ready, it will supply the requested data to the master 26 to complete the transaction. Normal bus arbitration (step 78) then follows.

If the memory controller 20 is not ready to complete the transaction, the memory controller 20 (acting as a slave) asserts the signal DEVSEL# along with the signals STOP#, MASK.sub.-- MASTER, and MASTER.sub.-- I.D. in step 60. The MASK.sub.-- MASTER and MASTER.sub.-- I.D. signals are sent to the arbiter 22. The assertion of the DEVSEL# and STOP# signals by the controller 20 on the PCI bus 30 causes the retry of the master 26. At the same time, the arbiter 22 sets the SPLIT.sub.-- MASK bit corresponding to master 26 in the SPLIT.sub.-- MASK register 23 of the arbiter 22 (step 62).

With its SPLIT.sub.-- MASK bit set in the SPLIT.sub.-- MASK register 23, the master 26 attempts to re-arbitrate for the bus 30 in step 64. It is then determined in step 66 whether the controller 20 (the slave) is ready to complete the transaction. If it is not, the SPLIT.sub.-- MASK bit is maintained (step 68) and the master 26 is denied the opportunity to re-arbitrate for the bus in step 70. Master 26 will then continue to attempt to re-arbitrate for the bus at step 64.

Steps 74, 80, 82 and 84 describe an embodiment of a priority determination that can be used in the present invention. In step 74, the arbiter 22 resets the SPLIT.sub.-- MASK bit and sets a TEMP.sub.-- MASK bit in a TEMP.sub.-- MASK register for all other requests. In step 80, the master 26 (now a priority master) attempts to re-arbitrate, and access is denied to other masters whose bits are set in the TEMP.sub.-- MASK register. The master is granted the bus in step 82, and the master's TEMP.sub.-- MASK bit is set in the TEMP.sub.-- MASK register. In step 84, it is determined whether all priority masters have been serviced. If not, the process returns to step 80. If all priority masters have been serviced, then the process returns to step 50.

Once the master 26 is granted the bus 30 (step 80), it asserts its address, control signals and FRAME# to the memory controller 20 (step 56). Since the memory controller 20 is ready to complete the transaction (step 58), the memory controller 20 will supply the requested data to the master 26 to complete the transaction (step 76) and normal bus arbitration will follow in step 78.

By the above-described method, the master 26 has been kept off the PCI bus 30 and prevented from re-arbitrating until the memory controller 20 is ready to complete the requested transaction. The other masters coupled to the bus 30 are able to arbitrate for and use the bus 30 during this time, thereby increasing the usable bandwidth of the bus 30.

It is desirable to execute the split transactions of the present invention efficiently. In order to do this, the slave needs to tell the arbiter 22 exactly which master should be unmasked in order to complete the data phase of the transaction. Additionally, it is preferable that the master be granted the bus as soon as possible (with high priority) since the slave may not be able to accept any more address phase transactions until it finishes the current data phase. In certain embodiments of the invention, a MASTER.sub.-- ID bus is used, this embodiment being more efficient, but more expensive in terms of module pin count and logic. During the address phase, the slave drives a unique ID on the MASTER.sub.-- ID bus, the ID being associated with the current address phase, while the slave is also asserting MASK.sub.-- MASTER and executing a retry. The arbiter 22 then stores this unique ID and associates it with the master, such as master 26, that is currently on the bus 30. When the slave is ready, it places the ID on the bus 30 at the same time that it activates REL.sub.-- MASTER. The arbiter 22 then resets only the SPLIT.sub.-- MASK bit in the SPLIT.sub.-- MASK register 23 that is associated with the MASTER.sub.-- ID and temporarily gives it the highest arbitration priority. This has the effect of ensuring that the unmasked master becomes the next device to be granted the use of the bus 30, thereby completing the split transaction in a timely manner. If multiple masters are released before they complete the data phase, the arbiter 22 keeps track of the order that the released IDs occur and gives each master priority on the bus 30 one-by-one. This is most easily done by allowing the SPLIT.sub.-- MASK bits to be reset one-by-one based on a predetermined algorithm. When the first master out of the queue initiates a frame, the next master is immediately released. This will cause the next master to complete arbitration during the current master's frame and therefore provide the most efficient use of the PCI bus 30.

In certain other embodiments of the invention, such as the exemplary embodiment of FIG. 2, the MASTER.sub.-- ID bus is not used. Since the arbiter 22 knows which master is currently granted the bus 30, it will mask this current master when a retry occurs upon the assertion of the MASK.sub.-- MASTER signal. Furthermore, since there is no MASTER.sub.-- ID, when the REL.sub.-- MASTER signal is activated, all of the masked masters are released and given priority over all of the masters which were previously unmasked. When only one master has been masked, this method achieves the same results as the other embodiments which use a MASTER.sub.-- ID bus. However, if multiple masters have been masked, the efficiency may go down since the master 26 may be granted the bus 30 even though the controller 20 may have data for master 28. In this case, the controller 20 will re-mask master 26 by activating MASK.sub.-- MASTER and executing a retry. Eventually, the master 28 will be granted the bus 30, complete its transaction and normal bus arbitration will occur.

In this other embodiment, creating priority for the recently unmasked masters can be performed in different manners, one method already being briefly described with respect to steps 74, 80-84 of FIG. 3. In this exemplary embodiment, a second set of masks (a TEMP.sub.-- MASK) is used to temporarily block all of the normal masters. During the clock cycle in which REL.sub.-- MASTER is received, a TEMP.sub.-- MASK bit is set corresponding to all SPLIT.sub.-- MASK bits which are currently reset. All of the SPLIT-MASK bits which are currently set are then reset. Several masters are released at a time, but this group of masters is given priority over all of the previously unmasked masters. The released priority masters then arbitrate based upon a normal priority. As each released master is granted the bus, its bit is set in the TEMP.sub.-- MASK register. Once all of the masters have either a SPLIT.sub.-- MASK or a TEMP.sub.-- MASK bit set, the TEMP.sub.-- MASK register is automatically reset, and the system resumes normal arbitration.

The embodiment of FIG. 2 also uses host bridges 21 between the multi-processor buses 19 and the PCI bus 30, the host bridges performing their normal functions. The memory controllers 20 are coupled in this embodiment to the multi-processor buses 19.

Although the present invention has been described with the use of a PCI bus, it is also applicable to other buses which have a back-off/retry feature that forces a master to leave a bus before a transaction is completed.

While the invention has been shown and described in detail, the foregoing description is in all aspects illustrative and not restrictive. It is therefore understood that numerous modifications and variations can be devised without departing from the scope of the invention.