Combined Multiprocessor Interrupt Controller and Interprocessor Communication Mechanism

This application is related to the following U.S. patent applications:

All of the related applications are assigned to the assignee of the present invention, and are hereby incorporated herein in their entirety by this reference thereto.

This application has an appendix. The appendix comprises the following documents:

Preliminary Chimaera Architectural Specification, Version 0.20, dated Feb. 16, 1993

Hydra Dell P/N 24002 Specification, Version 1.85, dated Apr. 14, 1992

Bifrost Specification, Version 1.2, dated May 8, 1992

Lethe Bus VHDL Model, dated Apr. 8, 1992

Bifrost-A VHDL Gorp, dated Apr. 17, 1992

Bifrost Hierarchy, Version 1.1, dated Apr. 20, 1992

Fifty (50) circuit diagrams depicting various portions of an actually designed embodiment of the present invention

The above listed documents of the appendix use the teaching disclosed herein setting forth particular details of an actual embodiment of the present invention, and are hereby incorporated by reference.

1. Field of the Invention

This invention relates to multiprocessor systems and, more particularly, to interrupt controllers and interprocessor communication mechanisms for use in such systems.

2. History of the Prior Art

The personal computer industry is a vibrant and growing field that continues to evolve as new innovations occur. The driving force behind this innovation has been the increasing demand for faster and more powerful personal computers. In order to meet this demand, computer designers have used various methods to increase the speed with which personal computers can process instructions. Historically, the personal computer has developed as a system utilizing a single microprocessor to handle all instruction execution. The microprocessor is the key working unit or "brains" of the personal computer, and its task is to handle all of the instructions that programs give it in the form of computer software.

One method that is being used to increase the speed of the personal computer is the incorporation of multiple microprocessors operating in parallel into a computer system. With the use of multiple processors, or multiprocessing, each microprocessor can be working on a different task at the same time. Systems that incorporate multiprocessing generally use standard microprocessors that operate off of a common bus and share a common memory. The use of multiprocessing has generally increased computer performance, but it has also introduced new design considerations that were not found in a single processor environment.

One consideration that arises in multiprocessing is how to allow the processors to communicate with each other. Communication between the processors in a multiprocessor system is generally necessary for the proper functionality of the system. One method that has been used for multiprocessor communication includes the use of input/output (I/O) registers which enable the processors to pass messages back and forth. The use of I/O registers, however, has conventionally required an address decode using a large number of address and control signals. Putting the registers on the local bus to minimize time spent by the processors on interprocessor communication leads to obstructive loading effects. Using other address lines, on the other hand, generally leads to increased pin count of connectors used with cards containing microprocessors.

Another consideration that arises in multiprocessing is how to control the operation of all of the processors. In some systems, a supervisory processor is used to control the operation of all other processors. However, this approach suffers insofar as it involves significant hardware and processing overhead.

An alternative approach is to eliminate the supervisory processor and have each processing unit capable of autonomous operation. Supervisory control is thereby accomplished within an operating system common to all of the processors. This approach requires a network of interprocessor communications such that the activities of each processor may be controlled by the operating system and synchronized, when required, with activities of other processors.

Prior art multiprocessor systems have only limited interprocessor communication capabilities. Most prior art systems employ a shared memory through which data may be exchanged by memory operations such as a read-modify-write sequence. Control functions may be effectuated in a similar manner by causing one processor to write to a control word location in shared memory, which location is subsequently read by another processor. Local copies of the shared memory space (or portions thereof) may be maintained by the individual processors. For example, the HEP processor of Denelcor utilizes a form of shared memory communications in which each shared memory location includes a lock bit for controlling access.

Shared memory multiprocessors typically communicate control information via messages sent through shared communication areas in memory. When one processor wishes to send a message to another, it first obtains exclusive access to a predetermined communication area. Exclusive access is obtained either by prior allocation of communication areas or by the use of locks provided by the operating system and implemented via indivisible operation provided by the processor's instruction set (e.g., test-and-set). Once the message has been written to the communication area, the sending processor uses a second mechanism to inform the receiving processor that data has been sent. In some cases, the message simply is left in the communication area, to be read by the receiving processor when it does a periodic poll of its communication areas. In other cases, the sending processor sends a message interrupt to the receiving processor via the bus interconnecting the processors. The receiving processor, upon recognizing the interrupt, processes the incoming message. Message and message interrupt sending typically are operating system functions.

Based upon the foregoing, those skilled in the art should understand and appreciate that the limited interprocessor communication capabilities of prior art multiprocessor systems present a multitude of problems. One problem is the fact that there is not a direct and efficient system that allows vectored interrupts to be routed to an arbitrary processor. A second, underlying problem is the fact that non-processor elements have not been well incorporated into interprocessor communication and multiprocessor interrupt control operations, to facilitate and enhance them. Heretofore, elements such as I/O bridges have been considered to be merely complicating factors with respect to communications and interrupt control in multiprocessing systems, rather than as presenting opportunities to speed those operations.

Accordingly, it is an object of the present invention to provide a scheme for efficiently allowing specific vectored interrupts to be routed to an arbitrary processor.

Another object of the present invention is to provide a scheme for enhancing and facilitating interprocessor communications and interrupt control in multiprocessing systems.

Yet another object of the present invention is to provide a mechanism whereby a non-processing element, an I/O bridge, is effectively elevated to processor status to enhance and facilitate interprocessor communications and interrupt control in a multiprocessing system.

According to the teachings of the present invention, in a computer system including a first processor and a second processor, the first processor having a programmable access controller capable of having a processor-associated vector therein and further capable of generating an interrupt request, a combined multiprocessor interrupt controller an interprocessor communication system includes a system bus, an input/output bridge element coupled to the system bus, and a system controller coupled to the system bus. Further according to the teachings of the present invention, the input/output bridge element includes circuitry for reviewing the interrupt request, circuitry for obtaining the processor-associated vector, and circuitry for packaging the processor-associated vector into an interprocessor communication message. Still further according to the teachings of the present invention, the system controller includes circuitry for receiving interprocessor communication message from the input/output bridge element, circuitry for decoding the interprocessor communication message, and circuitry for providing the interprocessor communication message to the associated processor.

Thus, embodiments of the present invention should be perceived to have speed and efficiency advantages with respect to interrupt control and interprocessor communications as compared to prior art systems.

The invention will be better understood and its numerous objects and advantages will become more apparent to those skilled in the art by reference to following detailed description of the invention, taken in conjunction with the accompanying drawings in which:

FIG. 1 is a high level block diagram of a personal computer system which includes an apparatus for connecting two EISA I/O channels to a system bus;

FIG. 2 illustrates a possible format for an interprocessor communication (IPC) message within the system depicted in FIG. 1;

FIG. 3 illustrates a possible format for an interrupt request within the system depicted in FIG. 1;

FIG. 4 illustrates a possible format for an action request within the system depicted in FIG. 1; and

FIG. 5 illustrates a possible format for a message transfer within the system depicted in FIG. 1.

Referring now to the drawings wherein like or similar elements are designated with identical reference numerals throughout the several views and, more particularly, to FIG. 1, there is shown a block diagram of a personal computer system 8 which includes two EISA I/O channels 10, 12 coupled to a system bus 14. One or more processors 16, 18 may also be connected to the system bus 14. An overall system controller 20 controls the prioritization and input of data from the two EISA I/O channels 10, 12.

Data from each EISA I/O channel 10, 12 passes through an I/O bridge 22, 24. The I/O bridges 22, 24 are subsystems which perform the functions of providing front-end interfaces directly to the system bus 14; providing back-end interfaces directly to the EISA I/O channels 10, 12; decoupling the EISA I/O channels 10, 12 from the system bus 14; and storing EISA I/O channel data in a cache memory until a burst of data can be sent onto the system bus. On a more detailed level, each bridge may comprise an application specific integrated circuit (ASIC) to handle address and control transfer, and two ASIC's to handle data transfer. The former element could be connected to the latter two elements by an internal bridge interface bus and it could provide virtually all of the functionality needed by them.

The system bus 14 includes a 64-bit data bus and a 32-bit address bus. A system bus clock 26 operates at a frequency of 331/3 Mhz. The system bus 14 supports all communications between processors, memories, and I/O channels in the computer system 8.

The computer system 8 further includes a first programmable interrupt controller (PIC) 28 for receiving a plurality of interrupt request signals (shown as IRQm) and for providing an interrupt signal INT1 to the system controller 20, where the PIC 28 is further coupled to the I/O bridge 22 for providing interrupt vector information. A second PIC 30 is also provided for receiving interrupt request signals (shown as IRQn) and providing an interrupt signal INT2 to the system controller 20 and further coupled to the I/O bridge 24 for providing interrupt vector data. The system controller 20 includes an interrupt circuit 20a, which receives the INT1, INT2 signals and provides corresponding IRQ signals to the I/O bridges 22, 24, respectively, for passing the interrupts from the PIC devices 28, 30. The interrupt circuit 20a further provides processor interrupt signals IRQ to the processor 16, 18, respectively, for interrupting the respective processor. The processors 16 and 18 provide interrupt acknowledge signals INTA1, INTA2 to the interrupt circuit 20a within the system controller 20 to acknowledge being interrupted. The system controller 20 further includes a bus interface circuit 20b for interfacing the system controller 20 to the system bus 14. In particular, the bus interface logic 20b requests access to the system bus 14 and allows the system controller 20 to read data from and write data to the system bus 14. The bus interface logic 20b is coupled to an interprocessor communications (IPC) decode circuit 20c for detecting IPC cycles on the system bus 14, for decoding IPC interrupts and messages as further described below and for providing data to the processors 16, 18.

The I/O bridges 22, 24 are implemented in a similar manner so that only details of the I/O bridge 22 are provided, it being understood that the I/O bridge 24 is implemented in a similar manner with similar components. The I/O bridge 22 includes an interrupt circuit 22a for detecting assertion of an IRQ signal and for receiving vector information from the PIC 28, where the interrupt circuit 22a is further coupled to an IPC circuit 22b for packaging IPC interrupts and messages, as further described below. To facilitate generating such IPC interrupts and messages, a lookup table 22c is provided which is preferably implemented with any type of memory as known to those skilled in the art, such as read-only memory (ROM) or the like. The lookup table 22c includes a table of processor numbers and vectors for identifying one of the processors 16, 18 for which a particular message or interrupt is intended. Further, the I/O bridge 22 includes a bus interface circuit 22d for interfacing with the system bus 14 for decoding and detecting cycles to and from the I/O bridge 22 and for reading from and writing data to the system bus 14.

The main processor used in the system 8 is, for example, a 662/3 Mhz Pentium microprocessor from Intel Corporation. Other processors may also be used. For instance, higher speed processors may be used if supported with synchronizers in the system controller 20. Likewise, slower speed i486 processors may also be used if supported with synchronizers in the system controller 20. Alternatively, 50 Mhz i486 processors may be run synchronously with only mild performance degradation.

The system 8 is optimized for uniprocessor performance, but provides good support for multiprocessor performance due to the following design characteristics:

(1) The system bus 14 runs at approximately 267 MBytes/second, thereby providing sufficient bandwidth to run two Pentium microprocessors;

(2) The computer system is designed to work with six nodes on the system bus; combinations of the following devices may be connected at the six nodes:

(a) Memory controllers (1 or 2);

(b) EISA I/O channel bridges (1 to 4);

(c) Processor cards (1 to 4); and

(d) Other masters/slaves;

(3) The system controller 20, together with an EISA bridge (e.g., bridge 22 or 24), acts as a multiprocessor interrupt controller (MIC) to coordinate and arbitrate interrupts on the system bus 14; and

(4) Interprocessor communications (IPC) messages are generated and transmitted over the system bus 14 between processors and other devices.

The present invention is solely concerned with multiprocessing systems, i.e., systems including multiple processors 16, 18.

In an embodiment of the present invention, IPC messages can be implemented as 16-bit I/O writes. A possible format of an IPC message is shown in FIG. 2 where it can be seen that the first two bits (designated by reference numeral 30) of the IPC message identify the IPC type; the next three bits (designated by reference numeral 32) identify the source of the IPC message; the next three bits (designated by reference number 34) identify the destination of the IPC message; and the last eight bits (designated by reference numeral 36) contain data which are defined by each specific type of IPC message.

The IPC type, identified by the first two bits 30, may be an interrupt request, action request, message transfer, or the like. Possible sources and destinations of IPC messages include processors and I/O channels.

The format of an interrupt request is shown in FIG. 3. In FIG. 3 it can be seen that the first two bits B`00` identify the IPC message as an interrupt request, and the last eight bits 36 provide an interrupt vector. This IPC message generates an interrupt request to the processor specified in the destination ID 34. When that processor (e.g., Processor 1 16) responds with an interrupt acknowledge, it (i.e., Processor 1 16) is given the vector provided in the vector field of the IPC message. This allows any processor to vector any other processor to any of 255 available interrupt vectors.

In an embodiment of the present invention it may be useful to have a certain destination ID to have a specific meaning. For example, B `00` to B `11` could be used with the low two bits indicating the destination processor. As another example, B `10` or some other special destination ID could be used to interrupt the processor with the lowest priority task executing. As yet another example, B `11` or some other special destination ID could be used to interrupt all processors.

Referring now to FIG. 4, there is shown an action request. It can be seen in FIG. 4 that in such a request the first two bits `01` identify the IPC message as an action request, and the last eight bits 36 provide an action to be performed. An action request provides gross control of system hardware. Actions which can be provided in embodiments of the present invention include the following:

Global: indicates that all nodes which implement IPCs should respond to this message, ignoring the destination ID 34.

Set Reset: asserts Reset to the node in the destination ID 34. Global messages to Set Reset are ignored.

Clear Reset: deasserts reset to the node in the destination ID 34. Global messages to Clear Reset are ignored.

Sync: asserts Sync to the node in the destination ID 34.

Set Flush: asserts Flush to the node in the destination ID 34.

Clear Flush: deasserts Flush to the node in the destination ID 34.

NMI: asserts a non-maskable interrupt to the node in the destination ID 34.

A possible format for a message transfer within an embodiment of the present invention is shown in FIG. 5. It can be seen in FIG. 5 that the first two bits `10` identify the IPC message as a message transfer, and the last eight bits 36 comprise the message data to be transferred. A message transfer allows for the delivery of a single byte of information from one processor to another processor in an overall system according to the teachings of the present invention. Reception of this message generates an interrupt with a specific vector. A receiving processor then reads the entire IPC message from the system controller 20 (see FIG. 1 ) and extracts the message data.

Within embodiments of the present invention, the system controller 20 integrates all of the unique functions of the system and acts as a central arbiter between devices competing for system access. Possible functions of the system controller 20 in an embodiment of the present invention include system bus arbitration; generation of interprocessor communications (IPC) messages; multiprocessor interrupt control (MIC), including programmable periodic interruption of each processor in the system; and floating point error interrupt support. Other possible functions of the system controller are error handling and reporting; communicating as a diagnostics controller port with a dedicated diagnostics processor; providing support for Halt, Shutdown, and Flush cycles; and providing cycle control for all system chipset register reads and writes.

The system controller 20 enables the input of data to the system bus 14 through the use of interrupts. In a preferred embodiment of the present invention, there are two methods of generating interrupts. One method offers optimum performance, and the other offers optimum compatibility for DOS. The interrupt mode is selected by programming an INTMODE bit in a CONFIG system configuration register.

When operating in the optimum performance mode, the system may operate in either a parallel or a serial mode. In either mode, the system controller 20 passes an interrupt from a programmable interrupt controller (e.g., an Intel 8259) to one of the I/O bridges 22, 24. The I/O bridge then uses the IPC interrupt request protocol. In addition to higher performance, the optimum performance mode also provides increased functionality by allowing intelligent interrupt redirection.

The parallel mode allows interrupts from both I/O channels to be processed by the system controller 20 concurrently. Serial mode forces the system controller 20 to process one interrupt at a time. For example, if both I/O channels request an interrupt, one interrupt is held while the other interrupt is serviced. The held interrupt remains on hold until the service routine for the serviced interrupt signals through an IPC message that the held interrupt may be serviced. When running in this optimum performance serial mode, it is required that all I/O interrupt service routines signal with the IPC message that enables the next interrupt.

With special reference to I/O channel 10 (or I/O channel "A") and associated I/O channel A bridge 22, the following is the sequence of events carried out during the optimum performance parallel mode:

1. An interrupt request is transmitted from a programmable interrupt controller to the system controller 20.

2. If the system controller 20 has completed processing the previous interrupt from I/O channel A 10, the system controller 20 activates an interrupt to the I/O channel A bridge 22. In parallel mode, the previous interrupt is considered to be completed when the CPU issues a second interrupt acknowledge to the interrupt from I/O channel A.

3. When the I/O channel A bridge 22 recognizes an interrupt request, it flushes its store queue, and then generates an interrupt acknowledge to read the vector from the programmable interrupt controller.

4. The I/O channel A bridge 22 then translates the vector from the programmable interrupt controller into a processor number and processor vector by performing a table lookup.

5. The processor number and vector is then packaged into an interprocessor communications (IPC) message.

6. The I/O channel A bridge 22 arbitrates for the system bus 14 and writes this IPC message to the system controller 20.

7. The system controller 20 decodes the IPC and interrupts the appropriate processor 16 or 18.

8. When the interrupted processor 16 or 18 responds with its second interrupt acknowledgment, the system controller 20 provides the vector specified for that processor in the IPC message, and is ready to process the next interrupt from I/O channel A.

Also with special reference to I/O channel A 10 and associated I/O channel A bridge 22, the following is the sequence of events carried out during the optimum performance serial mode:

1. An interrupt request runs from the programmable interrupt controller into the system controller 20.

2. If the system controller 20 has completed processing the previous interrupt, the system controller 20 activates an interrupt to I/O channel A bridge 22. In serial mode, the previous interrupt is considered completed when the interrupt service routine from the last interrupt serviced issues an end-of-interrupt IPC message. Note that in this mode only one interrupt from either I/O channel is allowed to be active at any one time.

3. When I/O channel A bridge 22 recognizes an interrupt request, it flushes its store queue and then generates the interrupt acknowledge to read the vector from the programmable interrupt controller.

4. The I/O channel A bridge 22 then translates the vector from the programmable interrupt controller into a processor number and processor vector by performing a table lookup.

5. The processor number and vector is then processed into an interprocessor communication (IPC) message.

6. The I/O channel A bridge 22 arbitrates for the system bus 14 and writes this IPC message to the system controller 20.

7. The system controller 22 decodes the IPC message and interrupts the appropriate processor 16 or 18.

8. When the interrupted processor 16 or 18 responds with its second interrupt acknowledge, the system controller 20 provides the vector specified for that processor in the IPC message.

9. When the processor generates the end-of-interrupt IPC, the system controller 20 has completed processing the interrupts.

The optimum performance mode creates a window between the time that the I/O channel bridge acknowledges the interrupt from the 8259 interrupt controller and the processor acknowledges the interrupt from the system controller 20. This window is not strictly DOS compatible and, therefore, under some circumstances the system may need to operate in the optimum compatibility mode. In the optimum compatibility mode, the system controller 20 passes interrupts directly from the 8259 to the CPU. The CPU then deals directly with the primary 8259 to handle the interrupt.

It should be noted that a system running in the full DOS compatibility mode will support only one processor. However, in an alternative embodiment, a boot processor runs DOS while the remaining processors run another operating system. The system controller 20 does not support two I/O channels when running in a uniprocessor, fully DOS-compatible mode.

The system controller 20 may also function as a multiprocessing interrupt controller (MIC) which allows specific vectored interrupts to be routed to an arbitrary processor. The implementation scheme requires the cooperation of the EISA I/O bridges 22 and 24 and the system controller 20. The following is the sequence of events carried out to route a specific vectored interrupt to a designated processor.

1. The interrupt request from the programmable interrupt controller in the ISP is transmitted to the I/O channel bridge.

2. When the I/O channel bridge sees an interrupt request, it flushes its store queue and generates an interrupt acknowledge to read the vector from the programmable interrupt controller.

3. The I/O channel bridge then translates the vector from the programmable interrupt controller into a processor number and processor vector by performing a table lookup.

4. The processor number and vector is then packaged into an interprocessor communication (IPC) message.

5. The I/O channel bridge arbitrates for the system bus 14 and writes this IPC message to the system controller 20.

6. The system controller 20 decodes the IPC message and interrupts the appropriate processor 16 or 18.

7. When the interrupted processor responds with an interrupt acknowledge, the system controller 20 provides the vector specified for that processor in the original IPC message.

The system bus 14 is equipped with a fully programmable system clock 26 which generates periodic interrupts and controls the various system time-outs. The system clock 26 is divided by a 13-bit value in a timer divisor register (TDR) to generate an intermediate clock signal. The intermediate clock also drives a second counter/divider stage which is used to generate the periodic interrupts. The divisor for this stage is set in a timer period register (TIP). To generate skewed periodic interrupts to each of the processors, interrupt timer count registers (TIC) are set to different values in the range of zero up to the value set in the TIP. These four-count compare registers generate their interrupt when the count in the timer matches the count in the register. Default values for all of these registers assume a 33-Mhz system bus clock and provide skewed 60 Hz processor interrupts. The second counter stage contains an 8-bit value which is available to the processors in the interrupt timer count register (TIC). This can be used for timing operations requiring finer granularity than is provided through the periodic interrupts.

The system controller 20 provides interrupt vectors to processors on interrupt acknowledge cycles which must be on bit zero. Therefore, all data communications with the system controller are through bits zero and one. There are six index registers, one for each processor, and one for each of the two possible I/O channels. The index registers always increment by words. Individual bytes are accessed via the appropriate enables.

Since the system controller 20 is designed for a multi-processor system, there are certain registers that are duplicated for each processor. In order to provide quick access to these registers without the requirement that a processor know its identity, the processor specific registers are multiplexed together via bus grant bits.

As noted above, the computer system is designed to work with six nodes on the system bus. The six system nodes may be masters, slaves, or both and some of these nodes may contain cache memory devices. As a result, data stored in main memory is not always valid. Correct data may reside in a system node's cache. Memory coherency is maintained by maintaining cache line state information in each node's cache. Further details are set forth in the related applications.

Although the foregoing is enough to enable those skilled in the art to practice the present invention, to facilitate ultimate use of the teachings herein an appendix setting forth complete details regarding an actually designed embodiment of the present invention is attached hereto.

Based upon the foregoing, those skilled in the art should understand and appreciate how the scheme according to the teachings of the present invention efficiently allows specific vectored interrupts to be routed to an arbitrary processor. The scheme according to the teachings of the present invention accomplishes this, in part, by effectively elevating an I/O bridge element to processor status with respect to its functionality in interrupt control and interprocessor communication operations. Thus, the present invention enhances and facilitates interprocessor communications and interrupt control in multiprocessing systems.

It is thus believed that the operation and construction of the present invention are apparent from the foregoing description. While the method, apparatus and system shown and described has been characterized as being preferred, it will be readily apparent that various changes and modifications could be made therein without departing from the spirit and scope of the, invention as defined in the following claims. ##SPC1##