Energy-focused compiler-assisted branch prediction

[0001] This application claims the benefits of U.S. Provisional Application No. 60/541,875, filed on Feb. 04, 2004, and Confirmation No 3108, entitled: ENERGY-FOCUSED COMPILER-ASSISTED BRANCH PREDICTION, the contents of which are hereby incorporated by reference into this application as if set forth herein in full.

[0002] This invention relates generally to reducing energy and power consumption and improving performance in a microprocessor and, more particularly, to reducing energy and power consumption and improving performance by controlling branch prediction and branch prediction related activities in a processor. This is accomplished by the use of a combined hardware and static prediction mechanism assisted by compiler information inserted into the application code, that allows a branch target to be predicted energy efficiently and with the prediction of target addresses often preceding the fetch or decode of a branch.

[0003] Microprocessors (referred to herein simply as “processors”) consume energy/power during their operation. It is advantageous to reduce the amount of energy consumed, particularly in the case of devices that run off limited power supplies.

[0004] Various factors affect the amount of energy that a processor consumes. For example, the frequency at which the processor operates, the voltage level that powers the processor, as well as the load capacitances, affect processor energy consumption. Reducing the frequency of the processor or the voltage supply may decrease processor energy consumption, however, doing so may also adversely affect the performance of the processor.

[0005] Other techniques to reduce energy consumption, by for example reducing load capacitances, may include changes to processor architectures and processor circuits. Some other techniques rely on modifying the application itself, or any other system layer, to improve energy efficiency.

[0006] Branch prediction relates to activities that determine the target address of an instruction that changes the control-flow of a program at runtime such as a branch instruction.

[0007] As noted in the literature, branch prediction mechanisms related energy consumed is a significant fraction of a processor's energy consumption. In addition, branch prediction accuracy affects both performance and overall processor energy consumption.

[0008] Various factors affect the energy impact of branch prediction. In the following paragraphs, we provide an introduction into various branch prediction mechanisms.

[0009] There are different prediction approaches in place today to determine the target of a branch in time, i.e., to avoid the penalty of fetching from the wrong address in the case of a taken branch.

[0010] One such approach is static prediction that assumes that a branch behaves uniformly, i.e., it is either taken or not taken, during the entire execution of a program.

[0011] Static predictors require typically no hardware-table lookup; they need to find the direction of a branch. This can be easily accommodated in many conditional branch instructions.

[0012] For forward-pointing branches such a predictor would typically predict “not taken” as branch outcome, and for backward-pointing it would predict a “taken” target. This seemingly arbitrary rule is based on statistical branch outcomes in programs. For example, it is known that backward-pointing branches are taken 90% of the time and forward-pointing ones are not taken 56% of the time for integer programs such as in the SPEC benchmark suite.

[0013] If the prediction is that the branch will be taken, the target address would be calculated at runtime and the next instruction-group fetched from the target address, rather than sequentially. A correctly predicted branch provides, typically, an execution with no performance penalty. Moreover, in architectures such as ARM9 or ARM10 a branch can be folded (e.g., pulled or removed) from the pipeline resulting in what is called zero-cycle branches.

[0014] A part of the energy cost of a static prediction is related to pre-decoding instructions in a fetch-buffer, or prefetch buffer, to find out if a fetched instruction is a branch. Pre-decoding is required to determine changes in control-flow before a branch instruction enters the pipeline. In addition, checking the direction of a branch, i.e., to determine if a branch is forward- or backward-pointing, is necessary. The direction of a branch can be often determined by decoding the sign bit of the branch offset. Negative offsets represent backward-pointing branches while positive ones correspond to forward-pointing ones.

[0015] If a branch is predicted taken, the target address is typically calculated by performing an arithmetic operation. This is often an addition of the program counter and the constant offset decoded from the branch instruction itself.

[0016] Static prediction schemes are often limited to conditional branches where the offset is a constant. Branches that use registers to calculate the address, or indirect branches, cannot typically be predicted before the register content is available in the pipeline. Other complementary schemes such as a return stack would be used for branching related to procedure calls and returns.

[0017] While the energy consumed by static prediction is relatively low, such prediction is often not preferred or used, due to its low prediction accuracy. Static predictors rarely exceed 60%-65% prediction accuracy, although in some cases, e.g., for some branch instructions, they can do much better.

[0018] During a branch misprediction, there is a high performance and energy penalty. This is due to the extra unnecessary instruction fetches and decodes: a result of fetching from incorrect execution paths. The performance and energy penalty of a misprediction depends on the pipeline depth of the processor and especially the number of stages between the fetch and execute stages and the size of the prefetch buffers, that determine the extra fetches resulting from a mispredicted branch. Even in relatively short pipelines, the branch prediction penalty is significant, e.g., can be as high as 4-5 cycles in modern processors.

[0019] A variant of static prediction is based on compile-time static prediction. In such a scheme, a bit in each branch instruction is set by the compiler and determines the direction of a branch. As no hardware lookup is required in either case, the power cost of a runtime static predictor and a compile-time static predictor are similar.

[0020] Due to increasing performance requirements of emerging applications, e.g., emerging applications require both wireless and internet capabilities, processor vendors generally prefer dynamic branch predictors.

[0021] Dynamic predictors improve the accuracy of branch prediction, and therefore performance, for most applications.

[0022] Dynamic predictors are based on runtime mechanisms. Branch history tables or similar auxiliary hardware tables are used to predict the direction of a branch for each branch instruction occurrence. At runtime these tables are continuously updated to better reflect likely branch outcome.

[0023] A key difference between static and dynamic prediction is that the same branch instruction can be predicted differently depending on its execution context in the dynamic case. Branch target address caches are often used to store the target address, in addition to the prediction outcome information. This can speed up the process as the target address is readily available after the table lookup.

[0024] There are many different dynamic predictors available. For example, depending on whether only local branch-related information is used or if local branch information is combined with global information about the program context in which the branch occurs, at a given time, one can build various dynamic predictors. Many predictors are multi-level requiring lookup of multiple large hardware tables. These tables are implemented similar to caches, with search tags and data array segments. The search is done through an associative lookup mechanism.

[0025] Dynamic predictors are typically more accurate than the static predictors, as they capture the dynamic behavior of branches rather than predicting a branch outcome always taken or not taken. Therefore, they are almost always preferred in processor designs that issue multiple instructions per cycle, where the misprediction penalty due to branch misses is fairly large due to the high instruction fetch rates.

[0026] Predicting a branch with a dynamic predictor require looking up branch history tables to determine the prediction. The hardware complexity, table size, and associativity of these tables, as well as the lookup frequency, are key determining factors in the ultimate energy efficiency achieved.

[0027] A key challenge with branch prediction is, however, not only to determine a likely target address or predict a change of control, i.e., if a branch is taken, but to predict it before the branch is decoded in the pipeline. Otherwise, potentially unnecessary instructions are already fetched on the sequential path adding to the energy consumption and imposing a performance penalty.

[0028] A typical solution, often used in today's processors, is to predict branches in parallel with the instruction fetch. In this case, however, branch prediction is performed with almost every instruction fetch, regardless of whether a fetched instruction is a branch or not. This is very energy inefficient as many more table lookups are performed than actual branch instructions would require in a program. For example, assuming that there is one branch instruction per five regular (non-branch) instructions, this would roughly mean five times more branch table lookups and five times more energy consumed.

[0029] The compiler-architecture interaction based processor framework described herein addresses the foregoing need to reduce energy consumption due to branch predictions without impacting performance. In fact, the framework in its preferred embodiment improves branch prediction accuracy and performance and significantly reduces energy costs.

[0030] Some of the key factors that determine the energy efficiency of branch prediction mechanisms include: percentage of branches predicted, prediction accuracy, branch prediction related hardware table complexity and number of hardware tables, and frequency of lookups.

[0031] The framework, in one of its embodiments, uses static instructions to control branch predictions. Static instructions are control instructions containing compile time information such as hints or other information that might be used at runtime to control execution more efficiently. This is, however, not meant to be limiting. Other embodiments might use solutions that require no added instructions, e.g., static information could be fetched with each program block, such as a basic block, from a dedicated address space and possibly stored separately with dedicated hardware support.

[0032] Assuming an embodiment with static instructions, a preferred solution would support many other compiler-enabled energy optimizations encoded within the same instruction. This would amortize the cost of static information related fetch overhead across many energy reduction related techniques.

[0033] The framework, in one aspect, is based on making static information about a branch available at runtime some time before the branch is fetched. In one embodiment, every critical basic block that ends with a branch would have such information inserted at the beginning of the block. Whether a given block is critical or not is determined with static analysis and profiling.

[0034] In one embodiment, this static information is then decoded and pulled out from the prefetch buffer, not allowing the static control to enter the pipeline. A prefetch buffer is a buffer where many of the instructions are fetched before entering the processor pipeline.

[0035] The control information inserted can be used to control a branch at the end of the block by providing information about the branch's position, the mechanism used for prediction, such as static or different dynamic ones, and branch outcome if a branch is predicted statically. This is not intended to be limiting. Additional branch related information could be encoded in a similar manner.

[0036] When the static information suggests one of the dynamic predictions, a branch prediction can take place right after the static instruction is pre-decoded in a prefetch buffer. This way, a branch instruction target is often predicted before the branch even enters the fetch stage. This is due to the fact that static instructions are inserted some number of instructions ahead of a branch instruction, or typically at the beginning of a basic block.

[0037] As mentioned, static instructions are removed in the prefetch buffer to avoid causing pipeline bubbles. However, even if a static instruction would not be caught in the prefetch buffer, and would enter the pipeline, e.g., that can happen after a pipeline flush in some embodiments, the static instruction would be decoded in the decode stage: this would still allow time for the branch prediction to take place, as the branch would follow several instructions behind and there would still be time to predict the branch before entering the pipeline.

[0038] In one aspect, the framework allows branches to be predicted early and the predictions to occur sufficiently early to avoid any penalty due to missed or skipped branches entering the pipeline or predicted too late. Note that a skipped branch, even if predicted in or after the decode stage, would already have caused several unnecessary instruction fetches in case of a taken branch.

[0039] Because branch instructions are predicted with static instructions, there is no need to pre-decode or lookup dynamic tables, for each instruction fetch, in basic blocks that use static control for directing prediction. This reduces the frequency of table lookups and their energy costs typically by factor of five or so.

[0040] In addition, in a preferred embodiment, not all branch instructions would be predicted dynamically. This could reduce the size of the hardware prediction tables, further reducing the energy requirements of branch predictions.

[0041] There are many instructions that can be predicted very well statically and those instructions could instead be predicted by compile time static techniques. Many of the backward-pointing branches for example, would be predicted statically. Backward-pointing branches that are not predictable well statically, would be, however, predicted dynamically.

[0042] In addition, in many cases with static and profiling information, even some fraction of the forward-pointing branches could be deemed to have good static predictability, i.e., they would be always taken or not taken. Mapping these branches statically reduces the capacity requirements of hardware prediction tables and saves additional energy without performance impact.

[0043] In addition, there are branches that are not predictable, neither statically or dynamically. The framework, in one embodiment, would encode static information to stall the fetch and let these branches execute before additional fetches would occur.

[0044] The framework, in one aspect, could be utilized to perform optimizations that are not limited to energy reduction. Examples of optimizations performed may include, but are not limited to, energy, power, and performance. In general, in one aspect, the framework could be used to improve an application's performance or improve the application's energy-delay product or any other metric that is a combination of execution time and power consumption.

[0045] If the control bits refer to controlling several instructions during runtime, such as in a basic block, one control instruction inserted before the block can include control information for a variety of optimizations amortizing its added overhead across many optimizations. In one aspect, the performance overhead of control instructions could be avoided with early folding/removal in processor pipelines, accomplished with simple static instruction related pre-decoding logic added to prefetch buffers and instruction fetching buffers. In many predicated architectures, a similar early folding is used, for example, to achieve zero-cycle branches.

[0046] The invention can be used to save energy on any type of device that includes a processor. For example, the invention can be used to save energy on personal computers, devices containing embedded controllers, sensor networks, network appliances, and hand-held devices, cellular telephones, and/or emerging applications based on other device technologies.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although methods and materials similar or equivalent to those described herein can be used in practice or testing of the present invention, suitable methods and materials are described below. In addition, the materials, methods, and examples are illustrative only and not intended to be limiting.

[0048] Other features and advantages of the invention will become apparent from the following description, including the drawings.

[0049] FIG. 1 is a block diagram showing the relationship between a source-level compiler, instruction set architecture, and microarchitecture in a processor.

[0050] FIG. 2 is a block diagram of a static control instruction inserted in selected program blocks to control branch prediction mechanisms. It also shows other compiler-assisted optimizations that are similarly encoded with the same instructions.

[0051] In the embodiments described herein, a processor framework uses compile-time information to reduce or enable reduction of energy or power consumption, and/or improve performance in a processor.

[0052] Referring to FIG. 1, a compiler 10 is a software system that programs circuitry to translate applications from high-level programming languages (e.g., C, C++, Java) into machine specific sequences of instructions. An instruction set architecture (ISA) 12 is a set of rules that defines the encoding of operations into machine specific instructions. The ISA acts as the interface between compiler 10 and the microarchitecture (14). A computer program is a collection of machine level instructions that are executed to perform a desired functionality. Micro-architectural (or architectural) components 14 comprise primarily hardware and/or software techniques that are used during execution of the program. The actual machine can be a microprocessor or any other device that is capable of executing instructions that conform to the encoding defined in the ISA.

[0053] Compile-time refers to the time during which the program is translated from a high-level programming language into a machine-specific stream of instructions, and it is not part of execution or runtime. Runtime is the time it takes to execute translated machine instructions on the machine. Compilation is typically performed on a different host machine than execution and it is done on the source program.

[0054] In following sections we describe a preferred embodiment of the framework.

Overview of Compiler-Assisted Branch Control with Global Lookahead

[0055] The compiler-architecture interaction based processor embodiment described herein address the foregoing need to reduce energy consumption due to branch predictions, without impacting performance. In fact, in its preferred embodiment it improves branch prediction accuracy and performance and significantly reduces energy costs.

[0056] Some of the key factors that determine the energy efficiency of branch prediction mechanisms include: percentage of branches predicted, prediction accuracy, branch prediction related hardware table complexity and number of hardware tables, and frequency of lookups.

[0057] The framework, in this embodiment, uses static instructions to control branch predictions. This is, however, not meant to be limiting. Other embodiments might use solutions that require no added instructions, e.g., static information could be fetched with each program block, such as a basic block, from a dedicated address space and possibly stored separately with dedicated hardware support. Other embodiments would add information to the control-flow changing instructions.

[0058] An example of static instruction 20 is shown in FIG. 2.

[0059] Assuming an embodiment with static instructions, such as shown in FIG. 2, a preferred solution would support many other compiler-enabled energy optimizations encoded within the same instruction. This would amortize the cost of static information related fetch overhead across many energy reduction related techniques.

[0060] The embodiment, in one aspect, is based on making static information about a branch available at runtime some time before the branch is fetched. Critical basic blocks that end with a branch would have such information inserted at the beginning of the block. What is a critical block is determined with static analysis and profiling and as a function of other factors at compile time.

[0061] In this embodiment, this static information is then decoded and pulled in the prefetch buffer. A prefetch buffer is a buffer where instructions are fetched before entering the processor pipeline.

[0062] The control information inserted can be used to control a branch at the end of the block by providing information about the branch's position, the mechanism used for prediction, such as static or dynamic, branch outcome if a branch is predicted statically, and control to stall the pipeline.

Capturing Branches and Branch Predecoding

[0063] The fraction of branches captured is increased as branches are predicted at the time the static control instructions are pre-decoded in the prefetch buffer or decoded later in the pipeline. This allows prediction of more branches that is typically possible with the current state-of-the-art, where branches need to be caught before entering processor pipelines. This often improves performance as more branches are predicted, and fewer unnecessary fetches and decodes are performed as a results of less pipeline flushes.

[0064] No branch predecoding is required in the prefetch buffer, in any of the instructions or program blocks that are preceded by related static control information. Such instructions would encode the position of a branch relative to the current program counter. This reduces the energy overhead of branch predecoding that exists in many embedded processors today.

[0065] There are often branches with prediction patterns that would require more complex dynamic predictors such as based on global history information. In one embodiment a small such predictor can be added in the microarchitecture to complement several other simpler dynamic predictors, like the binomial predictor. At compile time, one would in that case select the right predictor or prediction mechanism for each control-flow changing instruction. All branches that would not need a sophisticated predictor or that are not critical during execution would simply not be directed to such a sophisticated predictor. Because the size of such global history based predictor in this framework could be kept to very small, for example limiting it to 16 branches rather than 1,000s of branches in current solutions, it would not affect the energy consumption significantly, but it could help improve the prediction accuracy overall that in fact would reduce overall processor energy consumption due to fewer fetched executed instructions.

Reducing the Frequency of Branch Prediction

[0066] Because branch instructions are predicted with static instructions, there is no need for looking up dynamic branch history tables in parallel with every instruction fetch, in basic blocks that use static control for directing prediction. This reduces the frequency of table lookups and their energy costs typically by factor of five or so.

[0067] Another factor that reduces the branch lookup frequency is because not all branches are directed by the control instruction to be predicted dynamically.

[0068] There are many branch instructions that can be predicted very well statically and those instructions are instead being predicted by compile time static techniques. Many of the backward-pointing branches, for example, would be predicted statically. Backward-pointing branches that are not predictable well statically, would be predicted dynamically by setting the branch control-related bits in the static control instruction accordingly. This combined approach achieves a good balance between reducing the energy cost of dynamic prediction without affecting prediction accuracy.

[0069] In addition, in many cases, by using static and profiling information, even some fraction of the forward-pointing branches could be deemed to have good static predictability, i.e., if they would be consistently taken or not taken at runtime.

[0070] In addition, there are branches that are not predictable, neither statically or dynamically. This embodiment would encode static information to stall the execution and let these branches execute before additional fetches would occur. This would remove the energy penalty due to the frequent misprediction of these branches.

Reducing the Size of Branch Prediction Tables

[0071] Mapping a large fraction of branches to be predicted statically reduces the capacity requirements of hardware prediction tables, used for the dynamic prediction cases, and saves additional energy without impacting performance. In this embodiment most of the backward-pointing branches and some of the forward-pointing branches are predicted fully statically. The size of the hardware tables could therefore be reduced to half for the same or better equivalent dynamic prediction accuracy. Difficult critical branches could be directed to small global predictors such as gshare.

[0072] This aspect significantly reduces energy consumed whenever a dynamic prediction lookup is performed as the branch lookup tables could be implemented more energy efficiently.

Prediction Accuracy

[0073] Prediction accuracy of branch predictions has a significant energy impact. The embodiment described here uses a small binomial predictor of 64 entry and a small gshare predictor of 16 entry, that captures global branch context information, for the dynamic part. Nevertheless, the framework works with any type of dynamic predictor or any number of predictors.

[0074] This embodiment often improves prediction accuracy while reducing energy consumption significantly, as compared to the prediction accuracy of a particular dynamic predictor used alone. As described before, this is accomplished by having more branches predicted; branches that are highly predictable statically are predicted statically; branches that require more sophisticated prediction would use such a predictor, branches that are not predictable dynamically or statically are controlled by stalling the fetch until the branch is resolved.

[0075] If energy reduction is not the primary concern, and the dynamic prediction tables could be made equal between the compiler-assisted prediction and the dynamic predictor when used alone, then, additional prediction accuracy improvement could result. This is because the dynamic prediction tables would be used for the branches that could be best predicted dynamically and less capacity and conflict misses in branch tables such as branch address caches would be incurred.

Compiler and Profiling Support

[0076] Compiler support is used to determine the position of the branches in the control-flow graph of the program relative to the static instruction's position. In addition, compiler support is used to analyze the outcome of branches. For backward-pointing branches static analysis is performed to determine a confidence in the percentage of time the branch is taken. Profiling information can be added to the compiler for any branch that is selected not to be predicted neither statically or with one of the dynamic mechanisms. Forward-pointing branches are selected for static prediction similarly based on profile information. At the code generation phase, a compiler inserts the static instructions and adds the corresponding control decided for branches controlled into the static control instruction.

Encoding

[0077] Control information is added per a sequence of instructions, such as a basic block that ends in a branch, such that the code dilution overhead of static control could be amortized across several other compiler-assisted energy optimizations. The sequence where optimization is applied can be determined with static analysis or other means such as profiling. A control instruction is typically inserted before the controlled instruction sequence but other solutions are also possible. Energy increase due to the extra fetches related to static control information can be often minimized with similar compiler-driven instruction memory energy optimizations. Its performance overhead is minimized with removing the control instructions before entering the pipeline. In addition, multiple-issue processors would help hide this overhead further.

[0078] Branch prediction control as well as other optimizations not described in this embodiment could be integrated into a processor-wide solution and controlled with one or more control instructions per sequence of instructions. This reduces the control energy overhead as the code-dilution overhead would be amortized across many different energy optimizations, and is the preferred embodiment of the framework.

[0079] In one aspect, the control bits could be added as data to an executable. This data can be accessed through a dedicated hardware mechanism that would associate branch targets to addresses where control bits are located in the data memory, for example through a hashing mechanism, to fetch the corresponding control bits.

[0080] Independent from how the static bits are added, a static decode unit would decode these bits and generate the control signals for the associated sequence of instructions. During execution, a sequencing mechanism would control both processor resources and select appropriate energy efficient access mechanisms corresponding to the encodings decoded from the compiler-generated control bits.

[0081] The control information can also be incorporated as regular extensions to the instruction set or as co-processor instructions.

[0082] Referring to FIG. 2, a static instruction is shown that is currently used to control execution in an implementation compatible with the instruction set of a leading embedded microprocessor. A Static Instruction (or BSI) is used to control a group of compiler selected instructions.

[0083] This BSI allows static information related to energy focused execution, in a sequence of instructions, to be exposed to the runtime architecture. The BSI instruction provides compiler information for chip-wide energy management, not only branch related. This BSI is used for fine-grained control. Additional instructions, could be used for more coarse-grained optimizations.

[0084] The BSI shown in FIG. 2 follows ARM's v5 coprocessor instruction format, specifically the LDC and STC instructions (see ARM Architecture Reference Manual-ARM DDI 0100E, pages: A4-28, A4-82).

[0085] Referring to FIG. 2, 30 encodes the type of prediction performed. The three bits encode eight different combinations possible that includes statically predicted backward-pointing branch predicted taken, statically predicted forward pointing predicted taken and not taken, stalling of fetching, various dynamic prediction, etc. The eight possibilities are often enough to encode a variety of scenarios, but it is not intended to be limiting. In other embodiments more or fewer bits could be used and other type of static information encoded.

[0086] The position of the branch instruction is encoded in bits 34. This tells the position of the branch relative to the BSI. Different embodiments could encode this information differently. These bits would indicate the occurrence of a branch and, together with the prediction type bits, allow dynamic prediction ahead of fetching the branch instruction. Bits 22,24,26,28 represent encodings of other optimizations that are compiler assisted in other processor domains. This BSI shows a preferred embodiment where the overhead of static control is amortized across different compiler-enabled energy optimizations. Another version of a BSI could encode short branches and allow removal of the branch instruction from the binary. Such an embodiment would remove the dilution overhead of adding static compiler information to a binary entirely as for each such instruction added a branch instruction would be removed.

[0087] The invention is not limited to the specific embodiments described herein. The invention is not limited to reducing energy consumption or improving performance. It could also be used, in other contexts where control flow prediction is necessary.

[0088] Other types of compiler analyses and/or architecture support may be used. The invention may be applied to control any appropriate component of a processor. The optimizations and/or analyses may be performed in a different order and combined with other techniques, in other embodiments.

[0089] There are different forms of energy consumption, for example such as dynamic or leakage power related. The invention is not limited to one source of energy consumption.

Other embodiments not described herein are also within the scope of the following claims.