Instruction Issuing Mechanism for Processors With Multiple Functional Units

This invention relates to computer architecture and specifically to an instruction issuing mechanism capable of detection of concurrencies in an instruction stream and issuing multiple instructions within a given machine cycle.

The emergence of VLSI technology has stimulated research into the use of execution structures employed by processors having multiple functional units. Such high performance processors are generally partitioned to two sections, an instruction unit (IU) and an execution unit (EU) such is illustrated in FIG. 1A. The IU and EU communicate with each other, with the IU fetching instructions from a memory and formulating and decoding those instructions. The IU is also employed to fetch operands if necessary. Additionally, the IU sends arithmetic/logic commands, that is, the decoded instructions together with a requisite operand to the EU. This invention relates specifically to an instruction issuing mechanism enhancing the throughput, that is, the number of instructions executed per unit of time of the EU.

Within the prior art, processors employing multiple functional units have been designed and implemented. Typical are the CRAY-1 and the IBM 360/91. Reference is made to, R.M. Russell, "The CRAY-1 Computer System" A.C.M. Communications, 21: 1, January, 1978, pp. 63-72, and to Srini et al, "Analysis of the CRAY-1S Architecture", A.C.M., 10th Symposium on Computer Architecture, June, 1983, pp. 194-206. Reference is also made to R. M. Tomasulo, "An Efficient Algorithm for Exploiting Multiple Arithmetic Units", IBM Journal, 11: 1, January, 1967, pp. 25-33, for description of the IBM 360/91 system.

In accordance with such EU structure employing multiple functional units as shown in FIG. 1B, a bank of registers is installed in the EU to act as a bridge between the fast functional units (F.U.) and a slow main memory. The functional units in the EU perform arithmetic/logic operations on various data types, it being noted that the units are not necessarily identical. For example, reservations stations and a common data bus can also be incorporated. (R. M. Tomasulo, supra). Thus, some of the functional units may be virtual. The registers supply operands to the functional units and receive results from them while at the same time loading from and writing into the main memory.

The IU loads sequences of instructions into an instruction stack from which instructions are issued to and then executed by the functional units. Within prior art systems employing multiple functional units, at most one instruction is issued from the instruction stack during every machine cycle. As a result, the instructions execution rate of such an EU structure cannot be greater than the inverse of the machine cycle time (generally expressed in seconds). Reference is made to R. M. Keller, "Look-Ahead Processors", Computing Surveys, 7: 4, December, 1975, pp. 175-195, and J. W. Bowra, et al, "The Modeling and Design of Multiple Function Units Processors", IEEE Transactions on Computers, C25: 3, March, 1976, pp. 2102-2210. Specific reference is also made to Srini et al, supra, which indicates that the CRAY-1S system while well balanced suffers from a major drawback. The authors specifically note that the instruction issuing mechanism is a major bottleneck in the CRAY-1S architecture.

Given the deficiencies in prior art computer systems employing multiple functional units, it is an object of the present invention to define an instruction issuing mechanism which is capable of detecting concurrencies in an instruction stream and issuing multiple instructions within a given machine cycle and which may be extended to modify the instruction stream.

It is a further object of the invention to define an arithmetic engine implemented in a VLSI environment that substantially enhances the throughput of such a processor.

Yet another object of this invention is to formulate and define an instruction issuing mechanism for arithmetic engines utilizing multiple functional units to achieve high instruction execution rates.

A further object of this invention is to define a dispatch stack component of the instruction issuing mechanism operating in a FIFO mode and detecting instructions that can be issued at each machine cycle.

Still another object of this invention is to define a precedent count memory component of the instruction issuing mechanism to assign alpha- and beta-values for each instruction being loaded into the dispatch stack and to assign general purpose registers to operands to enhance possible execution concurrencies.

These and other objects of the present invention are achieved by an instruction issuing mechanism which detects concurrencies and issues multiple instructions within a given machine cycle. This invention will be described in greater detail by referring the attached drawings and the description of the preferred embodiment that follows.

FIG. 1A is a schematic drawing of a black diagram of a processor partitioned into an instruction unit and an execution unit.

FIG. 1B is a schematic drawing of a block diagram of an arithmetic structure utilizing multiple functional units, illustrating the data flow section;

FIG. 2A is a schematic diagram showing the dispatch stack;

FIG. 2B is a schematic diagram illustrating the sequence of instructions deposited into the dispatch stack; and

FIG. 3 is a block diagram of an instruction issuing mechanism in accordance with the present invention.

Referring to FIGS. 1-3, the instruction format utilized by the present invention will be considered first. As noted herein, the processor is partitioned into two units, the instruction unit (IU) and the execution unit (EU). The IU fetches, formulates, decodes and then forwards arithmetic/logic instructions to the EU. A known format, employed in the CRAY-1 and other systems is as follows:

where,

D denotes the register receiving the result of the arithmetic/logic operation;

S1 specifies the register which provides the first of two operands, or the only operand called for;

S2 specifies the register which yields the second of the two operands required; and

OP denotes the arithmetic/logic operation to be performed. In this invention, we consider store register and load register as arithmetic/logic operations.

As a result of this format,

If the registers D and S1 are identical, the instruction takes the same form as instructions for the execution units in the IBM 360/91.

The dispatch stack configuration in accordance with the present invention is shown in FIG. 2A. The IU formulates and forwards sequences of 3-register arithmetic/logic instructions as defined herein to the EU. That is, the instruction format utilizes registers D, S1 and S2. These sequences are deposited into a dispatch stack (DS) shown in FIG. 2A. The dispatch stack operates in a first-in first-out (FIFO) fashion. Each stack cell is implemented with a register.

In accordance with known FIFO processing, instructions in the format specified by (1) are sent by the IU to the bottom cell of the DS. This is indicated by the instruction arrow in FIG. 2A. Instructions are then advanced upward in the stack as instructions in an instruction stream are loaded in the DS from the bottom. As a result, when the DS is full in a steady state, the top cell always contains the instruction in the head of the instruction stream. This can be illustrated as follows:

The IU will then fetch operands a.sub.0 - - - , a.sub.7 and deposit them into registers F0, - - - , F7, respectively. Additionally, the IU will deposit the following sequence of instructions into the DS.

where AD indicates an "ADD" operation. It is recognized that the instruction sequence set forth in (4) is merely representative of a number of evaluations for statement (3). The present invention detects and enables execution concurrencies which are inherent in a given statement.

In accordance with the present invention, the IU deposits the sequence of instructions set forth in (4) into an initially empty dispatch stack. Such is shown in FIG. 2B. An instruction tag field representing the left hand most column is added, it being noted that instruction IO occupies the top cell of the DS with subsequent instructions following. The second column contains the OP field, the third column S1, the fourth column S2 and the extreme right hand most colum D. The DS strives to issue arithmetic/logic instructions to the functional units as fast and as many as possible. Such is inherent to take advantage of the presence of multiple functional units in this arithmetic engine. The issuance of such instructions is therefore stopped when one or more of the following three conditions prevail:

(a) the lack of the requisite functional unit;

(b) the lack of the requisite interconnection paths to transmit operand and/or result; and/or

(c) data dependencies among instructions.

In known instruction issuing mechanisms, once an instruction is stopped for any of the three above cited reasons, the flow of subsequent instructions also stops. Thus, in existing instruction issuing mechanisms, the EU examines only the instruction at the head of the instruction stream. Consequently, at most only one instruction can be issued for each machine cycle. As noted, if the top instruction cannot be issued because of the existence of one of the three conditions as set forth above, the flow of the instruction stream is entirely stopped. This deficiency in the prior art tends to degrade the engine output as a consequence of under utilization of available resources, that is, the functional units and the interconnection paths. This deficiency can be illustrated by referring again to FIG. 2B.

With the reservation stations and a common data bus scheme (CDB) the IBM 360/91 utilizes a floating-point execution unit which will examine instruction IO and dispatch it to one of its three virtual adders. In the next machine cycle, the floating-point execution unit examines and dispatches instruction I1 and in the third machine cycle instruction I2 is examined and dispatched. In accordance with this system, for three machine cycles the multiplier will not receive any instruction. Another cause, due to data dependencies, which produces under utilization of functional units will be presented later. This defect is also true with the CRAY-1 system. This defect results in a wasteful under utilization of the functional units which are available in a contemporary EU structure. The situation becomes exacerabated.in the context of VLSI devices due to the continuing decline of hardware cost. That is, since at most one instruction is issued for each machine cycle the instruction execution rate is locked by much less than the inverse of the machine cycle time.

The present invention departs significantly from such known systems by providing the dispatch stack (DS) with capabilities to allow it to examine and issue one or more instructions for each machine cycle. Thus, in accordance with the present invention, the dispatch stack identifies and issues instructions that can be immediately executed with available functional units. The technique of determining data dependencies will now be discussed in the context of the present invention.

An instruction in an instruction stream can be immediately issued to an available functional unit, real or virtual, if it does not have any data dependencies with those preceding instructions which have not yet been completed. For example, referring to FIG. 2B, instruction I2 is data dependent upon instruction I0. This occurs since one of the source registers of I2, F0 is the destination register of I0. Stated differently, I2 utilizes the result of I0 as an operand and therefore it must wait for the completion of instruction I0.

Consequently, it can be generalized that an instruction is data dependent upon a preceding, uncompleted instruction if one of its source registers is the destination register of the latter.

Referring again to FIG. 2B, it can be shown that instruction I2 is data dependent upon instruction I0 in a second sense. Specifically, the destination register of I2, F0, is one of the source registers of I0. Thus, if instruction I2 is issued and completed before I0, I0 may mistakenly utilize the result of I2 as one of its operands. Consequently, it may also be generalized that an instruction is data dependent upon a preceding, uncompleted instruction if its destination register is a source register of the latter.

These two generalizations allow enrichment of the entries in the dispatch stack. The resultant fields are given below:

where .alpha.(Si) represents the number of times that a particular register Si is used as a destination register in preceding, uncompleted instructions;

.beta.(D) represents the number of times that register D is designated as a source register in preceding, uncompleted instructions; and

I.sup.2 represents the issue index field to be delineated herein.

The sequence of instructions entered into the dispatch stack DS, shown in FIG. 2B, is represented as follows:

As shown in CHART I, I0 is at the top of the stack. There is no preceding, uncompleted instruction. Consequently, .alpha.(F0)=.alpha.(F1)=.beta.(F0)=0. Instruction I1 has instruction I0 as a preceding, uncompleted instruction but, neither of the source registers associated with instruction I1, that is, registers F2 and F3 are used as the destination register by I0-.alpha.(F2)=.alpha.(F3)=0. Additionally, the destination register of instruction II, register F2 is not employed as a source register by I0-.beta.(F2)=0. One of the source registers of I2 is F0 which is used by I0 as the destination register. Thus, it can be established that .alpha.(F0)=1. The other source register of I2 is register F2 which is the destination register of instruction I1. Consequently, .alpha.(F2)=1. Other .alpha.- and .beta.-values can be similarly delineated from the sequence of instructions set forth herein above.

An execution structure having reservation stations in a common data bus, for example, found in the IBM system 360/91 will issue instruction I0 whose two .alpha.-fields and the .beta.-field are 0 indicating that there are no data discrepancies preceding uncompleted instructions. Instruction I0 can therefore be immediately executed. The same is also true relative to instruction I1. Instruction I2 is then issued but any of the following conditions will suffice to prevent it from being immediately executed;

(a) .alpha.(F0)=1 where F0 is used as a destination register by a preceding uncompleted instruction, that is an operand and not yet ready;

(b) .alpha.(F2)=1, same as above; and

(c) .beta.(F0)=1, where F0 is used as a source register by a preceding uncompleted instruction--the deposit of the result of instruction I2 may erase an operand needed by a preceding instruction.

Instruction I2 is assigned a reservation station corresponding to a virtual functional unit. Thus, this unit is wasted. To rectify this defect in accordance with the present invention, the Issue Index (I.sup.2) for an instruction is as follows:

In order to issue instructions and make the instruction resources used more efficiently, the dispatch stack is scanned from top to bottom. When an instruction with an I.sup.2 value of 0 is encountered the issuing mechanism reserves an appropriate functional unit if available and then issues the instruction to it. The implementation of this search and issue operation is in the form of a reservation circuit as shown functionally in FIG. 3.

Considering again CHART I representing the sequence of instructions with .alpha., .beta., and I.sup.2 fields when loaded into the DS. Assume now that there are four functional units which are capable of performing the "ADD" operation and these units are initially free. The search and issue mechanism identifies and issues instructions I0, I1, I3 and I4 concurrently to the four free functional units. This follows the policy rationale set forth herein. It is noted that if the common data bus scheme of the prior art is employed, instructions I0, I1, I2, I3 would have to be issued to the four free functional units. Instruction I2 due to data dependencies indicated in the chart occupies a functional unit without actually being computed. Thus, the unit could be advantageously employed to actually compute I4.

At the completion of an issued instruction, its destination register F0 is used as a "key" to content address the (S1) and the (S2) fields of those instructions which follow it in the dispatch stack and to decrement the appropriate .alpha.-values by 1.

Similarly its source registers are in turn used to content address the (D) fields of all subsequent instructions and decrement their values.

Illustrating the dispatch stack update process, is CHART II which follows showing that at the completion of the instruction I0 its destination register F0 is used to "content address" the S1 and S2 fields of all the instructions which follow I0 in the DS. The S1 fields of I2 and I6 match the F0 key and their corresponding .alpha.(S1) fields are decremented by 1. At the same time, the source registers of I0, that are F0 and F1, are used to content address the D fields of all instructions which follow I0 in the DS. The D field of I2 and I6 match the F0 key and their corresponding .beta.(D) fields are decremented by 1. Moreover, instruction I0 is removed from the DS and subsequent instructions are advanced, that is moved up. Those subsequent instructions and the instruction stream should be brought into occupy empty spaces at the bottom of DS thereby operating in a FIFO mode. Thus, after decrementation and shifting the following chart exists.

It will be recognized that similar and in some cases concurrent completions of I1, I3 and I4 will reduce the contents of the DS to that shown as follows:

Empty spaces ready for subsequent instructions.

The contents of the DS after decrementations and shifts initiated by the completion of Instructions I0, I1, I3 and I4 is therefore illustrated by CHART III.

Instructions I2 and I5 can now be issued and their completion will reduce the I.sup.2 value of I6 to 0.

Operating under the assumption that (1) the operands will be available at their designated registers and (2) adequate data paths are available to transmit operands and results, the issue and execution schedule of the sequence of instructions of FIG. 2B will then be:

First-I0, I1, I3, I4

Second-I2, I5

Third-I6

This schedule which produces the shortest computation time would not be detected and followed if a prior art common data bus scheme is employed.

The identification of data dependencies among instructions utilized in the example shown relative to FIG. 2B excludes the following case:

where

D.sub.a =D.sub.b and D.sub.a .noteq.S1.sub.i, D.sub.a .noteq.S2.sub.i for all i.

The instruction I.sub.a as shown is obviously superfluous since its result is not utilized or needed in subsequent instructions. This case should therefore be excluded from the system compiler software. Should this not be feasible then erroneous consequences will arise if instruction I.sub.b is completed before instruction I.sub.a. Nevertheless, this can be prevented by defining .beta.(D) as the number of times that register D is designated as a source register and/or destination registers in preceding uncompleted instructions.

At the completion of an issued instruction, its destination register is used as a "key" to content address the S1, the S2 and the D fields of those instructions which follow it in the dispatch stack. The destination register is also used to decrement the appropriate .alpha.- and .beta.-values by 1. Similarly, its source registers are used to content address the D fields of all subsequent instructions and decrement their .beta.-values. Thus, following this methodology instruction I.sub.b will not be issued until the completion of I.sub.a.

The precedence count memory (PCM) as shown in FIG. 3 will now be discussed. It has been set forth herein that the IU formulates sequences of 3-register instructions and loads them into the DS. This requires, for example, the assignment of appropriate registers to operands and results. It also requires the determination of .alpha.(S1), .alpha.(S2), and .beta.(D) for each instruction formulated. These two tasks can be facilitated with the introduction of the precedence count memory (PCM) shown in FIG. 3. The PCM is implemented with a rank of registers, each register corresponding to a general purpose register in the execution unit. Each register has an entry in the PCM. The .alpha.-field indicates the number of times that a specific register has been used as a destination register by instructions already in the DS. The .beta.-field denotes the number of times that a specific register has been used as a source register. For example, after seven instructions are loaded into the dispatch stack as shown in Chart I, the PCM will have the entries as depicted below in Chart IV.

Chart IV is therefore a "snapshot" of the PCM immediately after the DS has been loaded in the manner identified in Chart I.

When an instruction is removed from the DS upon completion, the .alpha.-value of its destination register and the .beta.-values of its source registers are each decremented by 1.

When a register is assigned to an instruction as a source register, its .alpha.-value in the PCM is used as .alpha.(S1) or a .alpha.(S2) and its .beta.-value is incremented by 1. When a register is appropriated to an instruction as its destination register, its present .beta.-value is used as the .beta.(D) field and its .alpha.-value is incremented by 1.

Thus, in accordance with the present invention a unique instruction issuing mechanism has been defined for execution structures with multiple functional units. This mechanism is capable of detecting concurrencies and then issuing multiple instructions within a given machine cycle. As a result, throughput of such processors is substantially enhanced. While the invention has been defined relative to a preferred embodiment herein, it is apparent that modifications may be practiced without departing from the essential scope of this invention.