Silent data corruption prevention due to instruction corruption by soft errors

[0002] 1. Field of the Invention

[0003] The present invention relates to the field of computer systems and more specifically to preventing data corruption due to soft errors.

[0004] 2. Description of the Related Art

[0005] Soft errors in data storage elements, such as memory cells and pipeline latches, occur when incident radiation charges or discharges the storage element, thereby changing its binary state. Soft errors are increasingly a concern with smaller scale fabrication processes as the size (hence, the capacitance) of the storage elements shrink, since incident radiation will have greater effect in causing the soft errors on such smaller scale storage elements. Previously, soft errors were statistically significant only for large and dense storage structures, such as cache memories. However, the smaller feature structures are now more prone to having soft error. Structures, such as pipeline latches (particularly, wide multi-bit datapath latches), are affected to a greater extent by soft errors, where probability of occurrence of such soft errors is much more significant.

[0006] A problem with soft errors is that they have a tendency to silently corrupt data in a program. The program continues to execute, since the system is unable to detect the corrupted data, and generates an incorrect result. In some instances the system may not be able to detect the corrupted result. For example, encoding of many instructions differ in only one single bit value. A compare-equal instruction and a compare-not-equal instruction may have be different by one bit value. A probability of for data corruption is much greater in this instance. Accordingly, this type of silent data corruption (SDC) is not desirable in critical applications, such as for commercial transaction server applications, where wrong results can have broad-reaching implications.

[0007] In a high frequency, highly-pipelined machine, the pipeline latches are susceptible to such soft errors. For example, instruction data latches make up a significant portion of the total number of latches on a chip that are susceptible to soft errors. These latches, if protected, can significantly reduce the total undetected error count of a high performance processor. Thus, it is desirable to optimize the machine design to reduce, or even prevent, incorrect results due to soft errors. Since it is virtually impossible to stop all soft errors from occurring, at the very minimum it is desirable that soft errors be detected when they occur, so at least the application can be terminated and any data corruption reported. A preferable option is to detect the error when it occurs and seamlessly continue execution of the application after correcting the error.

[0008] Past practices have used parity checking and error correcting code techniques to identify the occurrence of soft errors. Past practices have also used a hardware solution, such as functional redundancy, to check for soft errors. However, this later approach typically requires significant additional circuitry on-chip to provide the error detection. None have addressed soft error detection and seamless correction, when such soft errors are detected, to the extent provided by the practice of the present invention.

[0009] FIG. 1 is a block schematic diagram showing an embodiment of the present invention when parity checking is used along an instruction flow path to detect a soft error occurring on an instruction.

[0010] FIG. 2 is a block diagram showing an embodiment of a front-end parity checking performed to detect a soft error.

[0011] FIG. 3 is a block schematic diagram showing an embodiment of a back-end parity checking performed in pipeline stages to detect a soft error.

[0012] FIG. 4 is a block schematic diagram showing another embodiment of a back-end parity checking performed in pipeline stages to detect a soft error.

[0013] FIG. 5 is a block schematic diagram showing an exemplary computer system which implements the present invention to detect soft errors.

[0014] Referring to FIG. 1, a portion of a processor for fetching and issuing processor instructions is shown. An instruction processing apparatus or unit 10 is illustrated according to an instruction flow from fetching of the instruction to its retirement. The instruction processing apparatus is comprised of an instruction line buffer 11, instruction cache memory 12, instruction issue queue 13 and a plurality of pipeline stages 14. In the described embodiment, the processor containing the apparatus 10 is a high performance processor, such as a super-scalar processor, having a variety of on-chip circuitry for receiving, processing and outputting data. The processor includes other circuitry than that shown in FIG. 1 for fetching, issuing, decoding and executing instructions. The general concepts of processing instructions and executing them are known in the art. Accordingly, fill buffers, caches, issue queues and instruction pipeline stages known in the art can be readily adapted for those named units/stages of FIG. 1 and further adapted to practice the present invention.

[0015] The instruction cache memory (also simply stated as a “cache”) 12 is used to store frequently executed instructions. An instruction fill buffer 11, which may be part of the instruction cache 12, is associated with the caching of instructions. The fill buffer 11 is comprised of latches (or latch equivalents) to allow returned lines from memory or higher level caches to temporarily reside therein prior to being written into the instruction cache 12. Accordingly, an instruction line fill is coupled to the fill buffer 11, so that instructions from another source are returned to fill the buffer 11. As noted, the instructions may be returned from memory (either on-chip or off-chip) or, where multi-level cache structures exist, the instructions may be returned from a higher-level cache (either on-chip or off-chip).

[0016] In the particular embodiment, the fill buffer 11, once loaded, can be read in two different ways. The line can be read for writing into the instruction cache 12 (shown by path 15) or it can be read directly into the pipeline (shown by path 16). Generally, for new instructions, the selected line from the buffer 11 is written into the cache 12. In those instances where a line needs to be sent to the pipeline stages 14, path 16 is available.

[0017] In the practice of the invention, an instruction line fill to the buffer 11 also includes a parity bit 20 for the line stored in the buffer 11. The parity could be generated remotely and imported from memory or higher level cache, or it can be generated locally, as exemplified by a parity generation circuit 21. A typical method for parity generation is by performing an exclusive-OR (XOR) operation on the bits. The parity bit 20 is stored in the line buffer 11, respectively along with the instruction entry.

[0018] On reads from the fill buffer for updating the instruction cache 12, the corresponding parity bit is checked by a parity check circuit 22. This parity check detects if any single bit soft error occurred to the instruction when transitioning through the latches of the line buffer 11. Although possible, the probability of multi-bit soft errors in the buffer latches is statiscally low. If there is a parity check error, a parity error signal from the parity check circuit 22 causes the line not to be written into the cache and the entry is invalidated in the fill buffer.

[0019] When a parity error is indicated from the parity check circuit 22, causing an invalidation for the particular entry in the buffer 11, the entry can be rewritten (reloaded) into the buffer 11. However, an alternative embodiment is shown in FIG. 2. In this alternative technique, the manner in which a parity check from the check circuit 22 is handled will depend on the nature of the instruction being checked. A tag in the fill buffer entry will specify whether the particular fill is from an instruction demand-fetch or if the fill is from an instruction prefetch. A demand-fetch is a condition in which the processor is waiting to execute the instruction, but is not present in the instruction cache 12. A prefetch is a condition in which the processor may execute the instruction, but may not, due to the speculative nature of the instruction. A tag in the fill buffer will identify if the instruction is a demand-fetch instruction or if it is a speculative prefetch instruction.

[0020] In FIG. 2, a decision flow is shown in which block 30 identifies if a parity error has occurred from the parity check circuit 22. If no error is detected, the process proceeds normally allowing the instruction to be loaded into the cache 12. If an error is detected, then block 31 determines if the instruction entry is a prefetched instruction. If it is a prefetch, canceling the cache line fill and invalidating the fill buffer entry is sufficient to prevent data corruption in the latch from execution. The entry can be reloaded into the fill buffer 11. However, if it was a demand fetch, the processor is still waiting for the instruction to continue execution.

[0021] In this embodiment, a front-end replay circuit 32 is used to cause the instruction fetch to recur (replay) to the same instruction pointer address to replay the instruction. By this described technique, not only is the error detected, but the error is effectively corrected by replacing the erroneous instruction with an error-free version. Since many processors already implement instruction prefetch mechanisms (including, instruction replay), these devices can be readily adapted for the practice of the present invention with minimal incremental design overhead. Furthermore, although the replay is shown for use With only demand-fetch entries, the replay can be extended to prefetch entries as well, if desired.

[0022] Referring to FIG. 1 again, the parity 20 that is generated or loaded into the fill buffer 11 is stored along with the entry into the instruction cache 12 when the cache fill occurs. Typically, the instruction cache 12 is configured as an array. Then, on reads from the instruction cache 12, the entries are propagated to the instruction issue queue 13. The parity bit is also read along with the instruction into the queue 13.

[0023] Subsequently, the instructions are propagated through an instruction pipeline, comprised of pipeline stages 14. The actual number of such stages 14 will depend on the processor and/or the computer system, but generally, a number of stages is present for processing (including the decoding of instructions). It is appreciated that many modern processors have degrees of speculative execution built in them, including, but not limited to, branch misprediction and speculative execution of unsafe operations. These processors will utilize a pipeline structure, as well as a mechanism to flush the pipeline in case of an incorrect speculation and restart from a specific program location.

[0024] As shown in FIG. 1, the parity bit is propagated with the instruction for each entry through the queue 13 and to each pipeline stage 14. The present invention incorporates a number of parity check circuits 23 to check for parity at each of the pipeline stages 14. The parity check circuit 23 operates equivalently to the earlier described circuit 22 to determine the parity of the particular line and check that parity to the parity bit for that corresponding line. If a parity check fails, a fault condition (parity error) is signaled.

[0025] FIG. 1 shows two ways in which the parity fault can be detected. In one technique, the outputs of each parity check circuit 23 (shown as PF1-PFn) are used separately for fault detection. An embodiment of this first technique is described further below in reference to FIG. 3. In a second technique, a parity check circuit output is OR'ed with next stage parity check circuit output for stages 14. A series of latching circuits (latches or their equivalents) 25 are used to synchronize the parity check outputs through the pipeline. The OR'ing function is exemplified by OR gates 39 in FIG. 1. At the end of the pipeline stages 14, a single parity flag (PF) is generated. An embodiment of this second technique is described further below in reference to FIG. 4.

[0026] It is to be noted that in a typical high speed processor, the instruction queue read time is critical and generally does not provide sufficient time to trigger an instruction fetch replay without slowing the frequency of the processor. Typically, the issue queue 13 and the pipeline stages 14 utilize latches for propagating the instructions. The present invention uses the parity check at each of the pipeline stages (commencing at the output of the instruction entry from the issue queue, which is the input to the first stage of the pipeline) 14, to determine if a soft error occurred as the instruction propagates. When a soft error is detected, a fault condition is noted in way of a parity check fault.

[0027] Although a variety of techniques can be applied to handle a parity fault from the pipeline, two techniques described above are further illustrated in FIGS. 3 and 4. In FIG. 3, each fault is input to a back-end (pipeline) replay circuit 35a. The replay circuit 35ais capable of identifying in which stage the parity fault occurred and can respond accordingly.

[0028] In FIG. 4, each of the fault signals is OR'ed (as shown by OR gates 39) with the next stage fault signal to generate a parity flag (PF), so that the flag is also propagated down the stages. As noted above, a series of latches 25 are used for latching the parity check signals through the pipeline. A final flag output is coupled to a back-end replay circuit 35b, so that whenever a parity check fault is detected, the pipeline is flushed and replayed. The error flag can be factored into the flush logic to restore the processor back to the erroneous instruction.

[0029] As noted above, many modern processors already include replay mechanisms. These replay mechanisms can be readily adapted to provide the replay when a parity check fault is noted in the pipeline. Accordingly, by monitoring each stage of the instruction pipeline, any soft error which could corrupt an output generated by the instruction can be detected and corrected. Although it is desirable to have a parity check circuit for each stage of the pipeline, not every stage input need to be checked for parity. An embodiment of the invention implements select arbitrary stages for the parity checking, so that only some of the stages 14 are checked.

[0030] FIG. 5 illustrates one typical system implementation for the invention. A computer 40 is shown in which a processor 41 functions as a sole or one of a plurality of processors comprising the central processing unit (CPU) or units of the computer 40. Typically, the processor 41 is embodied in a single integrated circuit chip. The processor 41 includes an execution (processing) core 42, which has one or more execution units. A section of the processor is dedicated to include the instruction processing apparatus 10. The instruction processing unit 10 is shown coupled to the core 42. In this embodiment, the operating system (OS) and other programs are generally located in a memory.

[0031] The invention is practiced according to the description above to execute an instruction in the core 42. The memory can be located on-chip (as shown by on-chip memory 43) or off-chip (as shown by off-chip memory 44). Typically, the on chip memory can be a cache memory or part of the main memory (RAM). The off-chip memory is typically comprised of main memory (as well as off-chip cache, if present) and other memory devices, such as a disk storage medium. However, it is to be noted, that the invention can be configured in other ways to process the instructions for execution by the core 42.

[0032] The present invention provides an efficient technique to detect and correct corruption of instructions. The detection mechanism is situated at various levels or stages utilized for instruction processing. Thus, soft error detection is provided at the front-end where instructions are fetched and cached. Soft error detection is also provided at the back-end of the issue stage, which typically incorporates pipeline stages for decoding the instructions. The detection stages can be placed at all or only at some of the unit outputs to detect soft errors.

[0033] In the described embodiment, the soft error detection is provided by propagating a parity bit throughout the instruction processing units/stages, checking for parity at the output of those designated units/stages and providing some form of correction when a parity error is noted. Since replay mechanisms are typically present on modern processors, this replay mechanism is adapted to provide a replay where a fault is noted. In the particular embodiment described, a front-end replay and back-end replay circuits are used, however, a single replay circuit can be readily adapted for use to practice the present invention. The front-end replay allows for refetching of the instruction when errors are detected early on in the instruction cycle and if detected later in the pipeline, the back-end replay allows for a flush and prefetch of the pipeline.

[0034] An advantage of the replay is that such circuitry is already present in many processors, so additional chip space needed for practicing the invention is minimized. Also, the replay mechanism allows for corrections to be made seamlessly when parity errors are detected.

[0035] Furthermore, in the described embodiment, parity is used to detect corrupted instructions. However, other fault detection mechanisms (other than parity) can be employed as well. That is, the parity bit operates as an error detection bit, but other forms of error checking can be used. In that event, the parity check circuits, which checks for parity, would be replaced by other error detection circuits. Additionally, the invention is described in reference to instructions, since latches (which are targets of soft errors) are used considerably in latching instructions.

[0036] Thus, a technique for silent data corruption prevention due to instruction corruption by soft errors is described.