Processor and method of prefetching data based upon a detected stride

[0002] 1. Technical Field

[0003] The present invention relates in general to data processing and, in particular, to a processor and method for fetching data from memory. Still more particularly, the present invention relates to a processor and method for prefetching data from memory based upon a detected stride.

[0004] 2. Description of the Related Art

[0005] As processor clock frequencies have continued to increase, thereby permitting increased instruction throughput, memory latency has become a principal impediment to improved processor performance. Although instructions are commonly speculatively fetched from memory in order to ameliorate the effects of instruction access latency, in a conventional processor data is fetched from memory only after a load instruction has been executed to determine the address of the data to be fetched (i.e., only demand fetching is used).

[0006] The present invention includes a recognition that in order to reduce data access latency, it is desirable to intelligently prefetch data from memory.

[0007] It is therefore one object of the present invention to provide an improved method and system for data processing.

[0008] It is another object of the present invention to provide an improved processor and method for fetching data from memory.

[0009] It is yet another object of the present invention to provide a processor and method for prefetching data from memory based upon a detected stride.

[0010] The foregoing objects are achieved as is now described. According to the method of the present invention, a first difference between a first load address and a second load address is calculated. In addition, a determination is made whether a second difference between a third load address and the second load address is equal to the first difference. In response to a determination that the first difference and the second difference are equal, a fourth load address, which is generated by adding the third address and the second difference, is transmitted to the memory as a memory fetch address. In an embodiment of the data processing system including a processor having an associated cache, the fourth load address is transmitted to the memory only if the fourth load address is not resident in the cache or the target of an outstanding memory fetch request. The above as well as additional objects, features, and advantages of the present invention will become apparent in the following detailed written description.

[0011] The novel features believed characteristic of the invention are set forth in the appended claims. The invention itself however, as well as a preferred mode of use, further objects and advantages thereof, will best be understood by reference to the following detailed description of an illustrative embodiment when read in conjunction with the accompanying drawings, wherein:

[0012] FIG. 1 depicts an illustrative embodiment of a data processing system in which the present invention may advantageously be utilized;

[0013] FIG. 2 is a block diagram of an illustrative embodiment of data prefetch circuitry in accordance with the present invention; and

[0014] FIG. 3 is a logical flowchart illustrating an exemplary method for prefetching data in accordance with the present invention.

[0015] With reference now to the figures and in particular with reference to FIG. 1, there is depicted a block diagram of an illustrative embodiment of a processor, indicated generally at 10, for processing data and instructions in accordance with the invention recited in the appended claims. In the depicted illustrative embodiment, processor 10 comprises a single integrated circuit superscalar microprocessor. Accordingly, as discussed further below, processor 10 includes various execution units, registers, buffers, memories, and other functional units, which are all formed by integrated circuitry. Processor 10 preferably comprises one of the PowerPC™ line of microprocessors available from IBM Microelectronics, which operates according to reduced instruction set computing (RISC) techniques; however, those skilled in the art will appreciate from the following description that the present invention may alternatively be implemented in other suitable processors or memory devices.

[0016] As illustrated in FIG. 1, processor 10 is coupled to a processor bus 11 via a bus interface unit (BIU) 12 within processor 10. BIU 12 controls the transfer of data and instructions between processor 10 and devices coupled to processor 10, such as optional level two (L2) cache 39 and main memory 13. Processor 10, optional L2 cache 39, processor bus 11, and the devices coupled to processor bus 11 together form a data processing system.

[0017] BIU 12 is connected to instruction cache 14 and data cache 16 within processor 10. Upper level caches, such as instruction cache 14 and data cache 16, enable processor 10 to achieve relatively fast access time to a subset of data or instructions previously transferred from L2 cache 39 or main memory 13 to the caches, thus improving the speed of operation of the data processing system. Instruction cache 14 is further coupled to instruction flow unit 18, which fetches instructions from instruction cache 14. Branch instructions fetched from instruction cache 14 are processed by branch processing circuitry internal to instruction flow unit 18; sequential instructions, on the other hand, are temporarily buffered within instruction flow unit 18 until they are dispatched to the sequential execution circuitry of processor 10.

[0018] In the depicted illustrative embodiment, the sequential execution circuitry of processor 10 includes fixed-point unit (FXU) 22, load-store unit (LSU) 28, and floating-point unit (FPU) 30. Each of execution units 22, 28, and 30 typically executes one or more instructions of a particular type of sequential instructions during each processor cycle. For example, FXU 22 performs integer mathematical and logical operations such as addition, subtraction, ANDing, Oring, and XORing, utilizing integer source operands received from specified general purpose registers (GPRs) 32 or GPR rename buffers 33. Following the execution of an instruction, FXU 22 outputs the resulting data, if any, to GPR rename buffers 33, which provide temporary storage for the result data until the instruction is completed under the direction of completion unit 40 by transferring the result data from GPR rename buffers 33 to one or more of GPRs 32. FPU 30, on the other hand, typically performs arithmetic and logical operations, such as multiplication and division, on single and double-precision floating-point source operands received from floating-point registers (FPRs) 36 or FPR rename buffers 37. FPU 30 outputs data resulting from the execution of floating-point instructions to selected FPR rename buffers 37, which temporarily store the result data until the instructions are completed under the direction of completion unit 40 by transferring the result data from FPR rename buffers 37 to selected FPRs 36.

[0019] As its name implies, LSU 28 typically executes floating-point and fixed-point instructions that either load data from memory (i.e., either data cache 16, L2 cache 39, or main memory 13) into selected GPRs 32 or FPRs 36 or that store data from a selected one of GPRs 32, GPR rename buffers 33, FPRs 36, or FPR rename buffers 37 to memory. As discussed further below with respect to FIGS. 2 and 3, load addresses computed by LSU 28 are examined by data prefetching circuitry 50 in order to determine whether a repeated stride exists. If so, a load address can be generated by data prefetching circuitry 50 based on the detected stride, and the load address can be transmitted to memory to initiate data prefetching.

[0020] Processor 10 preferably employs both pipelining and out-of-order execution of instructions to further improve the performance of its superscalar architecture. Accordingly, instructions can be executed by FXU 22, LSU 28, and FPU 30 in any order as long as data dependencies are observed. In addition, instructions are processed by processor 10 at a sequence of pipeline stages, including, for example, fetch, decode/dispatch, execute, finish, and completion.

[0021] Referring now to FIG. 2, there is illustrated a more detailed block diagram of the data prefetching circuitry within processor 10 of FIG. 1. Data prefetching circuitry, indicated generally at reference numeral 50, includes an n-entry load address queue 52, subtractors 54 and 56, adder 58, and comparator 60. In addition, as indicated by dashed-line illustration, data prefetching circuitry 50 may optionally include loop detection circuitry, including n-entry instruction address queue 70, comparators 72, OR gate 73, and filter logic 74.

[0022] As depicted, each load address resulting from the execution of a load instruction by LSU 28 is passed to data cache 16 and is stored in the top entry (LAn-1) of load address queue 52. Load address queue 52 is preferably a first in, first out (FIFO) queue that shifts load addresses down in the queue in response to the addition of a new load address to the top of the queue. (The load address in entry LAO is discarded.) Entries LAn-1 and LAn-2 of load address queue 52 are coupled to subtractor 56 such that the load addresses stored in these entries form the inputs of subtractor 56. Similarly, entries LAn-2 and LAn-3 are coupled to subtractor 54 such that the load addresses stored within these entries provide the inputs of subtractor 54. Subtractors 54 and 56, which are each preferably implemented as an adder having an inverting input, calculate differences 80 and 82, which represent the relative difference between the load addresses in entries LAn-3 and LAn-2 and the difference between the load addresses in entries LAn-2 and LAn-1, respectively. Differences 80 and 82 form the inputs of comparator 60, which generates an equal signal if differences 80 and 82 are equal and otherwise outputs a not equal signal. The generation of an equal signal by comparator 60 indicates that a stride between load addresses has been detected.

[0023] Adder 58 is coupled to entry LAn-1 of load address queue 52 and to the output of subtractor 56 so that adder 58 receives the most recently received load address and difference 82 as inputs. Adder 58 sums these inputs to generate a load address 84 having a difference with the load address in entry LAn-1 equal to difference 82. Load address 84 is then sent to data cache 16. In an embodiment of data prefetching circuitry 50 in which filter logic 74 is omitted, cache controller 90 of data cache 16 receives the equal/not equal signal output by comparator 60 and utilizes this signal to determine whether or not load address 84 should be interpreted as a memory request address. Thus, in cases in which load address 84 is qualified by an equal signal, indicating that a stride has been detected, cache controller 90 handles load address 84 like the load addresses received from LSU 28. Conversely, when cache controller 90 receives a not equal signal, load address 84 is ignored.

[0024] In some data processing systems, for example, those primarily utilized to execute scientific applications characterized by looping code, it may be desirable to further qualify the equal signal generated by comparator 60. In one embodiment, such qualification can be achieved by employing additional subtractors to verify that the stride is observed over additional loads. In addition, data prefetching circuitry 50 can optionally be augmented to include loop detection circuitry, including instruction address queue 70, comparators 72, OR gate 73, and filter logic 74 as illustrated in FIG. 2. Instruction address queue 70 is a FIFO queue including n entries that each store at least a portion of an instruction address of a load instruction executed to generate the load address stored in the associated entry of load address queue 52. Each of the instruction address portions stored in entries IAO through IAn-2 of instruction address queue 70 forms a first input of a respective one of the n−1 comparators 72. The second input of each of the n−1 comparators 72 is the instruction address portion stored within entry IAn-1, i.e., the most recently received instruction address. Comparators 72 output n−1 signals indicating which, if any, of the instruction address portions stored in instruction address queue 70 match the instruction address portion stored within entry IAn-1. OR gate 73 receives each of the n−1 signals output by comparators 72 and performs a logical OR operation to generate a one-bit output that indicates whether the instruction address portion stored within entry IAn-1 matches any of the other instruction address portions stored within instruction address queue 70. The one-bit output of OR gate 73 is received as an input by filter logic 74, which may comprise, for example, a two-input AND gate. Based upon the logic states of the outputs of OR gate 73 and comparator 60, filter logic 74 outputs a valid signal to data cache 16 indicating whether load address 84 should be interpreted as a memory request address. In this manner, instruction address queue 70, comparators 72, OR gate 73, and filter logic 74 further qualify the output of comparator 60 by requiring that a program loop (i.e., a matching load instruction address) be detected before data prefetching is initiated.

[0025] As discussed above, if data cache 16 does not receive an equal signal from comparator 60 or a valid signal from filter logic 74, load address 84, if computed, is simply ignored. Alternatively, if data cache 16 receives both a load address 84 and an enable signal, data cache 16 handles load address 84 like any other load address received from LSU 28. That is, cache controller 90 determines whether data associated with the load address is resident in data cache array 92 or if the load address is the subject of an outstanding memory request. If either of these conditions is true, cache controller 90 discards load address 84. However, if load address 84 misses in data cache array 92 and is not the subject of an outstanding memory request, cache controller 90 forwards load address 84, which preferably comprises an effective address, to data memory management unit (DMMU) 94 for translation to a real (physical) address. This real address is then passed to BIU 12, which transmits the real address as a memory request address to main memory 13 and/or L2 cache 39.

[0026] With reference now to FIG. 3, there is depicted a logical flowchart of an exemplary method for prefetching data in accordance with the present invention. As illustrated, the process begins at block 100 and thereafter proceeds to block 102, which illustrates a determination of whether or not data prefetching circuitry 50 has received a next load address from LSU 28. If not, the process simply iterates at block 102 until a next load address is received from LSU 28. In response to receipt of the next load address from LSU 28, the load address is both transmitted to data cache 16 and stored in the top entry (LAn-1) of load address queue 52, as indicated at blocks 104 and 122, respectively.

[0027] As depicted at block 106, in response to receipt of the load address generated by LSU 28, cache controller 90 of data cache 16 determines whether or not data associated with the load address is resident in data cache array 92 or if the load address is the subject of an outstanding memory request (e.g., whether another data fetch will supply a cache line containing data associated with the load address). If either of these conditions is true, then data cache 90 performs no further processing on the load address, and the process returns to block 102. However, if both the conditions tested at block 106 are false, the process proceeds to block 108. Block 108 illustrates data cache 16 transmitting the load address to BIU 12, preferably after translating the load address into a real address. In response to receipt of the real address, BIU 12 transmits the real address to memory as a memory request address. The process then iterates at block 110 until data associated with the memory request address is returned to data cache 16 from memory, typically in the form of a cache line of data including the data associated with the load address. In response to receipt of the requested data, cache controller 90 stores the requested data within data cache array 92, as shown at block 112. In addition, if the load address utilized to obtain the memory request address was generated by LSU 28 (as opposed to data prefetching circuitry 50), selected data within the received cache line is passed to LSU 28 for storage within the GPRs 32 or FPRs 36 specified by the load instruction. Thereafter, the process returns to block 102.

[0028] Referring again to block 122, following the storage of the load address within entry LAn-1 of load address queue 52, the process proceeds to block 124. Block 124 depicts data prefetching circuitry 50 computing differences 80 and 82 and load address 84 and transmitting load address 84 to data cache 16 as described above with respect to FIG. 2. The process then proceeds to block 126, which depicts a determination of whether or not a stride between load addresses was detected. If not, the process passes to block 130, which depicts comparator 60 transmitting an invalid (i.e., not equal) signal to data cache 16. Thereafter, the process returns to block 102.

[0029] However, in response to a determination at block 126 that a stride between load addresses has been detected, the process passes either to block 132, if the optional loop detection circuitry is not implemented within data prefetching circuitry 50, or to block 128, if the optional loop detection circuitry is implemented. Block 128 represents a determination of whether or not the signal produced by OR gate 73 indicates that a loop has been detected. If not, the process passes to block 130, which has been described. If, on the other hand, a loop has been detected, the process proceeds to block 132, which illustrates either comparator 60 or filter logic 74 transmitting a valid (or equal) signal to data cache 16 to indicate that load address 84 should be treated as a valid load address. The process then passes to blocks 106-112, which depict data cache 16 processing load address 84 in the manner described above.

[0030] As has been described, the present invention provides a method and system for intelligently prefetching data based upon a detected stride between load addresses. In one implementation, data prefetching can be further qualified by loop detection. By prefetching data from lower level memory prior to receipt of a request for the data from a processor or upper level memory, data latency can be substantially reduced.

[0031] While the invention has been particularly shown and described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention. For example, although data prefetching circuitry 50 is illustrated in FIG. 2 as being distinct from LSU 28 and cache controller 90, it should be understood that in alternative embodiments data prefetching circuitry 50 can be incorporated within LSU 28 or within cache controller 90. Moreover, although the present invention has been described with respect to an illustrative embodiment in which a processor prefetches data from a lower level memory, it should be understood that data prefetching circuitry in accordance with the present invention is equally applicable to the prefetching of data by a lower level memory, such as L2 cache 39 prefetching data from main memory 13.