Parallel processing apparatus

[0002] 1. Field of the Invention

[0003] The present invention relates to a parallel processing apparatus, and, more particularly, to a parallel processing apparatus which processes programs in parallel while generating and terminating a thread consisting of a plurality of instructions in a plurality of processors.

[0004] 2. Description of the Related Art

[0005] Today's typical computers are of a von Neumann-type whose built-in processor that plays the key role in each computer repeats a sequence of procedures of fetching a single instruction, decoding it, executing a process specified by that instruction, accessing the memory and writing the execution result back in the memory.

[0006] To improve the processing speed, the current computers each have a cache memory with a fast access speed provided between the main memory and the processor. The processor therefore mainly exchanges data with the cache memory. The operation of the processor of reading an instruction from the cache memory is called “instruction fetching”, and the operation of decoding an instruction is called “instruction decoding”, and the operation of writing the execution result back into the memory is called “write back”.

[0007] Pipelining is known as one of techniques that improve the processing speed of processors. The “pipelining” process is described in many books about computers, for example, “Computer Architecture” by Hennessy and Patterson. Pipelining is the technique that improves the processing performance by allowing a plurality of instructions, each of which performs only a part of the entire process, to be executed in an overlapped manner in one clock cycle.

[0008] FIG. 13 is a diagram for explaining a pipelining process.

[0009] An instruction is executed in separate pipeline stages called “instruction fetching (IF)”, “instruction decoding (ID)”, “instruction execution (EX)”, “memory access (MEM)” and “write back (WB)”. In cycle T1, an instruction at an address “1000” undergoes instruction fetching. In cycle T2, the instruction at the address “1000” undergoes instruction decoding and an instruction at an address “1004” undergoes instruction fetching at the same time. This technique of simultaneously executing a plurality of instructions in an overlapped manner is called “pipelining”. Registers placed between processes are called “pipeline registers” and a process unit for carrying out each process is called a “pipeline stage”. As is apparent from the above, pipelining speeds up the processing as a whole by executing instructions, described in a program, in parallel.

[0010] However, there occurs a circumstance where an instruction cannot be executed in the proper cycle due to a change in the program flow caused by a branching instruction. While a scheme of computing the address of the branching destination specified by the branching instruction at an early stage in the pipeline stages, such as the ID stage, is taken for faster processing, the branching destination cannot be determined for a conditional branching instruction until the condition is determined. For a conditional branching instruction, therefore, the cycle that stops pipelining is eliminated by carrying out a scheme of predicting whether or not its branching condition is satisfied by using history information (see pp. 302 to 307 in the aforementioned book entitled “Computer Architecture” by Hennessy and Patterson).

[0011] A “superscalar” system (“Superscalar” by Johnson) which improves the processing speed by providing a plurality of processing elements or processor elements in a single processor and simultaneously generating a plurality of instructions has already been put to a practical use. The superscalar system is ideally capable of executing instructions equal in number to the provided processor elements in one clock. It is however said that even if the number of processor elements should be increased limitlessly, instructions would not be smoothly executed due to a branching instruction and the actual performance would be restricted to about three to four times that of the case of using a single processor.

[0012] Another practical way of improving the processing speed is to perform parallel processing by using a plurality of processors. In a typical processor system which accomplishes parallel processing by using a plurality of processors, parallel processing is executed by carrying out communication among the processors to assign processes to the individual processors. A system which uses conventional processors accomplishes such communication by an interruption processing scheme that is from outside carried out externally each processor as an interrupt control on that processor.

[0013] In the interruption processing scheme, when an external unit interrupts a processor, a program to be executed in the processor is switched to an interruption program from a user program and the interruption process is then executed. When the interruption process is completed, the original user program is resumed. To switch the execution program in a processor, data which will be used again by the original user program, such as data in the program counter or register file, is saved in a memory device. The overhead that is need for such data saving for switching between programs is nonnegligibly large and an interruption process is generally takes time. A parallel processing system which uses interruption processing therefore suffers a large overhead in communications between processors, which is an impediment in enhancing the performance.

[0014] One solution to this problem is a so-called multi-thread architecture. This technique is disclosed in, for example, “A Multi-threaded Massively Parallel Architecture”, Proceedings of 19th International Symposium on Computer Architecture, R. S. Nikhil, G. M. Papadopuolos, and Arvind, pp. 156-167.

[0015] A “thread” is a sequence of instructions. A program consists of a plurality of threads. In a multi-thread architecture, thread-by-thread processes are assigned to a plurality of processors so that those processors can process threads in parallel. Therefore, the multi-thread architecture has a mechanism and an instruction for allowing a thread which is being executed on one processor to generate a new thread on another processor.

[0016] The generation of a new thread on another processor is called “to fork a thread” and an instruction to fork a thread is called a “fork instruction”. A fork instruction specifies to which processor element a thread should be forked and which thread to fork.

[0017] Control parallel processing has been proposed in, for example, “Proposition Of On Chip MUlti-Stream Control Architecture (MUSCAT)” by Torii et al., Joint Symposium Parallel Processing JSPP '97, pp. 229-236. The multi-stream control architecture analyzes the control flow of a program, predicts a path which is very likely to be executed soon, and speculatively executes the path before its execution is established. In this manner, the multi-stream control processes programs in parallel.

[0018] FIG. 14 is a diagram showing a model of multi-stream control.

[0019] A conventional sequence of instructions which are executed sequentially consists of threads A, B, and C. In the sequential execution, one processor processes the threads A, B, and C in order as shown in section (a) in FIG. 14. In the multi-stream control, by way of contrast, while a processor element (PE)#0 is processing the thread A, the thread B which is expected to be executed later is forked to and is processed by a processor element #1 as shown in section (b) in FIG. 14. The processor element #1 forks the thread C to a processor element #2. The speculative execution of threads which are expected to be executed later can ensure parallel processing of threads, thus improving the processing performance.

[0020] The aforementioned paper that has proposed the “MUSCAT” mentions that it is not always possible to predict, before execution, whether or not a thread is to be forked. It is also known that adequate parallel processing cannot be achieved merely by the established forking that involves threads whose forking has been established before execution. In this respect, the MUSCAT employs controlled speculation that analyzes a program at the time of compiling it and speculatively executes a thread which is highly likely to be executed before its execution is established. The fork instruction that is to be speculatively executed is called a “speculation fork instruction”. If the speculative execution in the multi-stream control has failed, however, the thread that has been speculatively executed must be canceled before actual execution. This means a wasteful operation of the processor elements, which undesirably leads to increased power consumption.

[0021] A thread which is executed by each processor element finishes a series of processes by its end instruction. When a thread is forked by a speculation fork instruction, the termination of the thread becomes effective in response to the end instruction. When a thread is not forked, however, it may be unnecessary to execute such an end instruction in some cases. To cope with this situation, the MUSCAT uses a conditioned end instruction so that executing an end instruction depends on whether or not that condition is met. As a plurality of threads are processed in parallel, however, a conditioned end instruction, which is to be executed after the condition is met, may be processed in the multi-stream control before an instruction to determine that condition is executed. In such a case, the conditioned end instruction should wait for the processing of the condition-determining instruction to end. If the termination is decided, fetching or the like of subsequent instructions which becomes wasteful is carried out until the condition is determined. This also results in increased power consumption.

[0022] Accordingly, it is an object of the present invention to provide a parallel processing apparatus which is used in a processor system that carries out parallel processing using a plurality of processors and which efficiently executes fork instructions for activating a plurality of processors, thereby reducing power consumption.

[0023] It is another object of this invention to provide a parallel processing apparatus capable of efficiently terminating a thread with respect to the aforementioned conditioned end instruction for the thread.

[0024] It is a further object of this invention to provide a parallel processing apparatus which efficiently accomplishes the execution of the aforementioned speculation fork instruction and thread-end-conditioned thread-end instruction in the form of a hardware unit.

[0025] To achieve the above objects, according to the first aspect of this invention, there is provided a parallel processing apparatus having processing means for generating (forking) a thread consisting of a plurality of instructions on an external unit,

[0026] the processing means including a predicting section for making a prediction of whether or not a fork condition of a fork-conditioned fork instruction is satisfied after fetching but before executing the instruction.

[0027] According to the second aspect of this invention, there is provided a parallel processing apparatus comprising processing means having means for issuing an externally forked thread,

[0028] the processing means including a predicting section for making a prediction of whether or not a thread-end condition of a thread-end-conditioned thread-end instruction for terminating the forked thread is satisfied after fetching but before executing the instruction.

[0029] According to the third aspect of this invention, there is provided a parallel processing apparatus comprising processing means for generating a thread consisting of a plurality of instructions on an external unit, the processing means including:

[0030] means for issuing an externally forked thread; and

[0031] a predicting section for predicting whether or not a fork condition of a fork-conditioned fork instruction is satisfied after fetching but before executing the fork instruction and whether or not a thread-end condition of a thread-end-conditioned thread-end instruction for terminating the forked thread is satisfied after fetching but before executing the thread-end instruction.

[0032] According to one modification of the parallel processing apparatuses of the first to third aspects, in addition to making the prediction, when an input instruction is a conditional branching instruction, the predicting section predicts whether or not the conditional branching instruction is satisfied.

[0033] In any one of the above-described parallel processing apparatuses, a plurality of the processing means may be provided.

[0034] In any one of the above-described parallel processing apparatuses, the predicting section may make the prediction using history information. In this case, it is preferable that the history information have a plurality of states according to probabilities of the prediction. In the latter case, the predicting section may predict the fork condition, the thread-end condition or the conditional branching instruction based on the states.

[0035] In the parallel processing apparatus according to the first aspect, it is preferable that the fork-conditioned fork instruction include information about the result of previous analysis of the probability of the fork condition, and the predicting section predicts whether or not the fork condition is satisfied in accordance with the probability.

[0036] In the parallel processing apparatus according to the second aspect, it is preferable that the thread-end-conditioned thread-end instruction include information about the result of previous analysis of the probability of the thread-end condition, and the predicting section predicts whether or not the thread-end condition is satisfied in accordance with the probability.

[0037] In the parallel processing apparatus according to the third aspect, it is preferable that the fork-conditioned fork instruction include information about results of previous analysis of the probability of the fork condition and a probability of the thread-end condition, and the predicting section predicts whether or not the fork condition and the thread-end condition are satisfied in accordance with the probabilities.

[0038] In the parallel processing apparatus according to the aforementioned second case, the processing means may further include memory means for storing the history information associated with at least two of the fork condition, the thread-end condition, and the conditional branching instruction.

[0039] In the parallel processing apparatus according to the modification, the processing means may further include address generating means for generating a top instruction address of a thread to be generated when the fork condition is satisfied and generating an instruction address of a branching target when the conditional branching instruction is satisfied.

[0040] According to a more specific example of the first aspect of this invention, there is provided a parallel processing apparatus comprising:

[0041] analysis means for analyzing an input instruction;

[0042] prediction means for, when the instruction analyzed by the analysis means is a fork-conditioned fork instruction, predicting whether or not a fork condition of the fork-conditioned fork instruction is satisfied after fetching but before executing the instruction and sending out a fork instruction in accordance with a result of the prediction; and

[0043] execution means for executing the instruction, deciding whether or not the prediction of the fork instruction is correct, and sending out an instruction to cancel a thread generated by the fork instruction, when the fork instruction has been sent out and the prediction is wrong.

[0044] According to a more specific example of the second aspect of this invention, there is provided a parallel processing apparatus comprising:

[0045] analysis means for analyzing an input instruction;

[0046] prediction means for, when the instruction analyzed by the analysis means is a thread-end-conditioned thread-end instruction for terminating a forked thread, predicting whether or not a thread-end condition of the thread-end-conditioned thread-end instruction is satisfied after fetching but before executing the instruction, and sending out a thread-end instruction in accordance with a result of the prediction; and

[0047] execution means for executing the instruction, deciding whether or not the prediction of the thread-end instruction is correct, and sending out an instruction to cancel stopping of a thread which has been stopped by the thread-end instruction, when the thread-end instruction has been sent out and the prediction is wrong.

[0048] In the parallel processing apparatus according to the specific examples of the first and second aspects of this invention, it is preferable that the prediction means should include memory means for storing history information and update means for updating the history information stored in the memory means;

[0049] the execution means informs the update means of a result of the decision; and

[0050] the update means updates the history information in accordance with the result of the decision.

[0051] FIG. 1 is a block diagram illustrating the structure of a parallel processing apparatus according to a first embodiment of this invention;

[0052] FIG. 2 is a block diagram showing the internal structure of fork-instruction predicting sections 14 and 24;

[0053] FIG. 3 is a status-transition chart for explaining an algorithm for predicting forking of a thread by using history information which is used in a predicting section 34;

[0054] FIG. 4 is a flowchart illustrating the general operation of the parallel processing apparatus according to the first embodiment of this invention;

[0055] FIG. 5 is a status-transition chart for explaining an algorithm for predicting forking of a thread by using history information which is used in the predicting section 34 when the thread is very likely to be forked;

[0056] FIG. 6 is a status-transition chart for explaining an algorithm for predicting forking of a thread by using history information which is used in the predicting section 34 when the thread is not likely to be forked;

[0057] FIG. 7 is a flowchart illustrating the general operation of the parallel processing apparatus when an instruction includes information about the result of previous analysis of the probability of a fork condition;

[0058] FIG. 8 is a block diagram illustrating the structure of a parallel processing apparatus according to a second embodiment of this invention;

[0059] FIG. 9 is a flowchart illustrating the general operation of the parallel processing apparatus according to the second embodiment of this invention;

[0060] FIG. 10 is a status-transition chart for explaining an algorithm for predicting the end of a thread by using history information which is used in a thread-end-instruction predicting section 42;

[0061] FIG. 11 is a block diagram illustrating the structure of a branching-etc. predicting section 60 included in a parallel processing apparatus according to a third embodiment of this invention;

[0062] FIG. 12 is a table for explaining history information in a history buffer 32;

[0063] FIG. 13 is a diagram for explaining a pipelining process; and

[0064] FIG. 14 is a diagram showing a model of multi-stream control.

[0065] Parallel processing apparatuses according to preferred embodiments of this invention will now be described in detail with reference to the accompanying drawings.

[0066] FIG. 1 is a block diagram illustrating the structure of a parallel processing apparatus according to the first embodiment of this invention.

[0067] As shown in FIG. 1, the parallel processing apparatus according to the first embodiment of this invention has two sets of processor elements (PE) 10 and 20 which are connected together by a bus B. The processor element 10 comprises a program counter register (hereinafter called “PC register”) 11, an instruction cache 12, an instruction decoding section 13, a fork-instruction predicting section 14, and an instruction executing section 15. The processor element 20 likewise comprises a program counter register 21, an instruction cache 22, an instruction decoding section 23, a fork-instruction predicting section 24, and an instruction executing section 25.

[0068] Each of the PC registers 11 and 21 holds the address of an instruction to be processed. The instruction caches 12 and 22 respectively cache instructions output from the PC registers 11 and 21. The instruction decoding sections 13 and 23 read and decode the instructions cached in the respective instruction caches 12 and 22. Each of the fork-instruction predicting sections 14 and 24 predicts whether or not a speculation fork instruction (fork-conditioned fork instruction) forks a thread in accordance with the history information of a fork instruction which is held in the predicting section. The instruction executing sections 15 and 25 execute the instructions that have been decoded by the respective instruction decoding sections 13 and 23. Each of the instruction executing sections 15 and 25 also determines if the prediction of the speculation fork instruction is correct.

[0069] The fork-instruction predicting sections 14 and 24 will now be discussed in detail.

[0070] FIG. 2 is a block diagram showing the internal structure of the fork-instruction predicting sections 14 and 24.

[0071] As shown in FIG. 2, each fork-instruction predicting section 14 or 24 comprises an address calculator 30, a history buffer 32, a predicting section 34 and a history-information update section 36.

[0072] The address calculator 30 computes the address of the forking destination from an instruction output from the associated instruction decoding section 13 or 23. The history buffer 32 stores history information indicating whether or not a speculation fork instruction has forked a thread. The predicting section 34 predicts whether or not a thread should be forked, based on the history information stored in the history buffer 32. The history-information update section 36 receives the result of a decision from the associated instruction executing section 15 or 25 in FIG. 1 and updates the history information stored in the history buffer 32 based on the decision result.

[0073] A description will now be given of the operation of the fork-instruction predicting section 14 or 24 having the above-described structure.

[0074] First, when the instruction decoding section 13 or 23 outputs a decoded instruction, the address calculator 30 calculates the address of the forking destination and sends it to the predicting section 34. When the instruction decoding section 13 or 23 outputs an instruction address, the history buffer 32 outputs history information according to this instruction address and provides the predicting section 34 and the history-information update section 36 with the history information. Based on the input history information, the predicting section 34 predicts whether or not the thread is to be forked.

[0075] When the predicting section 34 predicts that the thread is to be forked, the address of the forking destination output from the address calculator 30 is sent on a signal line 14aor 24aas the forking-destination address. When receiving the result of a decision on a fork prediction from the associated instruction executing section 15 or 25, the history-information update section 36 updates the retained history information and writes the updated history information in the history buffer 32 via a signal line 36a.

[0076] An explanation will now be given of an algorithm for predicting forking of a thread by using history information which is used in the predicting section 34 in FIG. 2.

[0077] FIG. 3 is a status-transition chart for explaining an algorithm for predicting forking of a thread by using history information which is used in the predicting section 34.

[0078] The history information stored in the history buffer 32 can represent four states. The four states are:

[0079] highly unlikely forking (St 1)

[0080] unlikely forking (St 2)

[0081] likely forking (St 3)

[0082] highly likely forking (St 4)

[0083] When the history information indicates either “highly unlikely forking” (St 1) or “unlikely forking” (St 2), the predicting section 34 predicts that a thread will not be forked. When the history information indicates either “likely forking” (St 3) or “highly likely forking” (St 4), the predicting section 34 predicts that a thread will be forked.

[0084] The state of the history information changes in accordance with the result of a decision on prediction as follows.

[0085] In the case where the history information specifies “highly unlikely forking” (St 1), when the prediction is a success, the state does not change and remains as “highly unlikely forking” (St 1). When prediction is a failure, the state changes to “unlikely forking” (St 2).

[0086] In the case where the history information specifies “unlikely forking” (St 2), when the prediction is a success, the state changes to “highly unlikely forking” (St 1). When the prediction ends in failure, the state changes to “likely forking” (St 3).

[0087] In the case where the history information specifies “likely forking” (St 3), when the prediction is a success, the state changes to “highly likely forking” (St 4). When the prediction is unsuccessful, the state changes to “unlikely forking” (St 2).

[0088] In the case where the history information specifies “highly likely forking” (St 4), when the prediction is successful, the state does not change and stays as “highly likely forking” (St 4). When the prediction ends in failure, the state changes to “likely forking” (St 3).

[0089] Using the above algorithm, the predicting section 34 predicts whether or not a thread will be forked based on the input history information.

[0090] The general operation of the parallel processing apparatus with the above-described structure according to the first embodiment of this invention will be discussed below.

[0091] FIG. 4 is a flowchart illustrating the general operation of the parallel processing apparatus according to the first embodiment of this invention.

[0092] In FIG. 4, IF, ID, EX, MEM and WB are pipeline stages, respectively indicating an instruction fetching stage, an instruction decoding stage, an instruction executing stage, a memory access stage, and a write-back stage.

[0093] In the IF stage, the processor element 10 sends an address stored in the PC register 11 to the instruction cache 12 (step S10). Then, the instruction decoding section 13 fetches from the instruction cache 12 an instruction specified by the address sent to the instruction cache 12 from the PC register 11 (step S12).

[0094] In the next ID stage, the instruction decoding section 13 decodes the instruction fetched in step S12 and determines a process to be executed. The instruction decoding section 13 also determines if the decoded instruction is a speculation fork instruction (step S14). When the fetched instruction is a speculation fork instruction, the instruction decoding section 13 informs the fork-instruction predicting section 14 to that effect. When the decoded instruction is not a speculation fork instruction, on the other hand, this instruction is executed in step S32.

[0095] When informed of the instruction being a speculation fork instruction, the fork-instruction predicting section 14 predicts whether or not the speculation fork instruction will fork a thread in accordance with the history information of the fork instruction that is held in the predicting section 14 (step S16).

[0096] When the decision result is “YES”, i.e., when it is predicted that the speculation fork instruction will fork a thread, the fork-instruction predicting section 14 sends the address of the forking destination to the PC register 21 of the processor element 20 via the signal line 14aand the bus B (step S18). This process generates a new thread for the processor element 20 which in turn starts processing an instruction. The processing by the processor element 10 proceeds to step S20.

[0097] When the decision result in step S16 is “NO”, i.e., when it is predicted that the speculation fork instruction will not fork a thread, the fork-instruction predicting section 14 does not generate a thread on the processor element 20 and proceeds to step S26.

[0098] In the EX stage, the instruction executing section 15 executes the instruction decoded by the instruction decoding section 13. The instruction executing section 15 also determines if the prediction of the speculation fork instruction is correct (steps S20 and S26).

[0099] When the decision result in step S20 is a “success”, it means that the prediction that “the thread will be forked” has been successful. In this case, the execution of the instruction continues and the instruction executing section 15 informs the fork-instruction predicting section 14 of the “successful prediction”. The fork-instruction predicting section 14 updates the history information of the speculation fork instruction. Further, the processor element 20 is informed of the establishment of the forked thread via a signal line 15b(step S22).

[0100] When the decision result in step S20 is a “failure”, it means that the prediction that “the thread will be forked” has ended in failure and the processor element 20 is informed of that failure via the signal line 15band the forked thread is canceled. The instruction executing section 15 informs the fork-instruction predicting section 14 of the “prediction failure”. When informed of the failure, the fork-instruction predicting section 14 updates the history information of the speculation fork instruction (step S24).

[0101] When the above processing is completed, the processor element 10 continues executing the instruction.

[0102] When the decision result in step S26 is a “success”, it means that the prediction that “the thread will not be forked” has been successful and the instruction executing section 15 informs the fork-instruction predicting section 14 of the “successful prediction”. Then, the fork-instruction predicting section 14 updates the history information of the speculation fork instruction (step S28).

[0103] When the decision result in step S26 is a “failure”, it means that the prediction that “the thread will not be forked” has ended in failure and the instruction executing section 15 sends the address of the forking destination to the PC register 21 of the processor element 20 via the signal line 15band the bus B. As a result, a new thread is generated on the processor element 20 which in turns starts processing an instruction. The instruction executing section 15 informs the fork-instruction predicting section 14 of the “prediction failure”, and the fork-instruction predicting section 14 updates the history information of the speculation fork instruction (step S30).

[0104] According to this embodiment, as described above, because the predicting section 34 predicts forking of a thread based on the history information, it is possible to suppress the generation of unnecessary threads by efficiently executing the generation of a new thread and the termination of a thread. This leads to a reduction in power consumption.

[0105] An explanation will now be given of an algorithm for predicting forking of a thread by using history information which is used in the predicting section 34 in FIG. 2 when there is a high probability that the thread will be forked.

[0106] FIG. 5 is a status-transition chart for explaining an algorithm for predicting forking of a thread by using history information which is used in the predicting section 34 when the thread is very likely to be forked.

[0107] The history information stored in the history buffer 32 represents four states as per the case that has been explained with reference to FIG. 3. The following are the four states for a speculation fork instruction with a high probability of forking.

[0108] possible non-forking (St 11)

[0109] likely forking (St 12)

[0110] highly likely forking (St 13)

[0111] most likely forking (St 14)

[0112] When the history information indicates “most likely forking” (St 14), “highly likely forking” (St 13) or “likely forking” (St 12), the predicting section 34 predicts that a thread will be forked. Only when the history information indicates “possible non-forking” (St 11), the predicting section 34 predicts that a thread will not be forked.

[0113] The state of the history information changes in accordance with the result of a decision on prediction as follows.

[0114] In the case where the history information specifies “possible non-forking” (St 11), when the prediction is a success, the state does not change and remains as “possible non-forking” (St 11). When prediction is a failure, the state changes to “likely forking” (St 12).

[0115] In the case where the history information specifies “likely forking” (St 12), when the prediction is a failure, the state changes to “possible non-forking” (St 11). When the prediction is a success, the state changes to “highly likely forking” (St 13).

[0116] In the case where the history information specifies “highly likely forking” (St 13), when the prediction is successful, the state changes to “most likely forking” (St 14). When the prediction is unsuccessful, the state changes to “likely forking” (St 12).

[0117] In the case where the history information specifies “most likely forking” (St 14), when the prediction is successful, the state does not change and stays as “most likely forking” (St 14). When the prediction ends in failure, the state changes to “highly likely forking” (St 13).

[0118] Using the above algorithm, the predicting section 34 predicts whether or not to fork a thread based on the input history information.

[0119] In short, when a speculation fork instruction is determined as having a high probability of forking a thread in the analysis that has been carried out during compiling, the algorithm shown in FIG. 5 includes information about that probability in the instruction. At the time of predicting such a speculation fork instruction having a high probability of forking a thread, the number of states in which it is predicted that the “thread will be forked” is made greater than, and different from, the number of states in which it is predicted that the “thread will not be forked”.

[0120] A description will now be given of an algorithm for predicting forking of a thread by using history information which is used in the predicting section 34 in FIG. 2 when there is a low probability that the thread will be forked.

[0121] FIG. 6 is a status-transition chart for explaining an algorithm for predicting forking of a thread by using history information which is used in the predicting section 34 when the thread is not likely to be forked.

[0122] The history information stored in the history buffer 32 represents four states as per the case that has been explained with reference to FIG. 3. The following are the four states for a speculation fork instruction with a high probability of forking.

[0123] most unlikely forking (St 21)

[0124] highly unlikely forking (St 22)

[0125] unlikely forking (St 23)

[0126] possible forking (St 24)

[0127] When the history information indicates “most unlikely forking” (St 21), “highly unlikely forking” (St 22) or “unlikely forking” (St 23), the predicting section 34 predicts that a thread will not be forked. Only when the history information indicates “possible forking” (St 24), the predicting section 34 predicts that a thread will be forked.

[0128] The state of the history information changes in accordance with the result of a decision on prediction as follows.

[0129] In the case where the history information specifies “most unlikely forking” (St 21), when the prediction is a success, the state does not change and remains as “most unlikely forking” (St 21). When prediction is a failure, the state changes to “highly unlikely forking” (St 22).

[0130] In the case where the history information specifies “highly unlikely forking” (St 22), when the prediction ends in failure, the state changes to “unlikely forking” (St 23). When the prediction is a success, the state changes to “most unlikely forking” (St 21).

[0131] In the case where the history information specifies “unlikely forking” (St 23), when the prediction is successful, the state changes to “highly unlikely forking” (St 22). When the prediction is unsuccessful, the state changes to “possible forking” (St 24).

[0132] In the case where the history information specifies “possible forking” (St 24), when the prediction is successful, the state does not change and stays as “possible forking” (St 24). When the prediction ends in failure, the state changes to “unlikely forking” (St 23).

[0133] Using the above algorithm, the predicting section 34 predicts whether or not to fork a thread based on the input history information.

[0134] When a speculation fork instruction is determined as having a low probability of forking a thread in the analysis that has been carried out before actual execution of the instruction, information about that probability is included in the instruction. At the time of predicting such a speculation fork instruction having a low probability of forking a thread, the number of states in which it is predicted that the “thread will be forked” is made smaller than, and different from, the number of states in which it is predicted that the “thread will not be forked”.

[0135] As is apparent from the above, for a speculation fork instruction which is likely to be predicted as “will fork a thread” in the analysis that is carried out before actual execution of the instruction, information about that probability is included in the instruction. At the time of making a prediction, the number of states in history information which indicate possible forking is made different from the number of states which indicate that the thread of interest will not be forked. This scheme can permit the effective use of an analysis which is carried out before actual execution of an instruction.

[0136] This difference in the number of states can improve the probability of prediction, thus resulting in efficient generation of a new thread and efficient termination of a thread. This makes it possible to suppress the generation of unnecessary threads, which leads to a reduction in power consumption.

[0137] Although the foregoing description has discussed the case where prediction is implemented based on history information, information about the probability of forking a thread may be included in advance in an instruction when that instruction is compiled. The operation in this case will now be discussed.

[0138] FIG. 7 is a flowchart illustrating the general operation of the parallel processing apparatus when an instruction includes information about the results of previous analysis of the probability of a fork condition.

[0139] When the operation is initiated, the processor element 10 sends an address stored in the PC register 11 to the instruction cache 12 in the IF stage (step S10). Then, the instruction decoding section 13 fetches from the instruction cache 12 an instruction specified by the address sent to the instruction cache 12 from the PC register 11 (step S12).

[0140] In the next ID stage, the instruction decoding section 13 decodes the instruction fetched in step S12 and determines a process to be executed. The instruction decoding section 13 also determines if the decoded instruction is a speculation fork instruction (step S14). When the fetched instruction is a speculation fork instruction, the instruction decoding section 13 informs the fork-instruction predicting section 14 to that effect. When the decoded instruction is not a speculation fork instruction, on the other hand, this instruction is executed in step S32.

[0141] When informed of the instruction being a speculation fork instruction, the fork-instruction predicting section 14 predicts whether or not the probability that a thread will be forked, based on information about the results of previous analysis of the probability of a fork condition included in the instruction (step S15).

[0142] When it is predicted that the probability of forking a thread is “high”, the fork-instruction predicting section 14 sends the address of the forking destination to the PC register 21 of the processor element 20 via the signal line 14aand the bus B (step S18). This process generates a new thread on the processor element 20 which in turn starts processing an instruction. The processing by the processor element 10 proceeds to step S20.

[0143] When it is predicted in step S15 that the probability of forking a thread is “low”, the fork-instruction predicting section 14 does not generate a thread on the processor element 20 and proceeds to step S26.

[0144] In the EX stage, the instruction executing section 15 executes the instruction decoded by the instruction decoding section 13. The instruction executing section 15 also determines if the prediction of the speculation fork instruction is correct (steps S20 and S26).

[0145] When the decision result in step S20 is a “success”, it means that the prediction that “the thread will be forked” has been successful. In this case, the execution of the instruction continues and the processor element 20 is informed of the establishment of the forked thread via the signal line 15b(step S23).

[0146] When the decision result in step S20 is a “failure”, it means that the prediction that “the thread will be forked” has ended in failure and the processor element 20 is informed of that failure via the signal line 15band the forked thread is canceled (step S25).

[0147] When the above processing is completed, the processor element 10 continues executing the instruction.

[0148] When the decision result in step S26 is a “success”, it means that the prediction that “the thread will not be forked” has been successful, in which case forking a thread will not take place (step S29).

[0149] When the decision result in step S26 is a “failure”, it means that the prediction that “the thread will not be forked” has ended in failure and the instruction executing section 15 sends the address of the forking destination to the PC register 21 of the processor element 20 via the signal line 15band the bus B. As a result, a new thread is generated on the processor element 20 which in turns starts processing an instruction (step S31).

[0150] According to this embodiment, as discussed above, when information about the probability of forking a thread is included in an instruction, the fork-instruction predicting section 14 predicts forking of a thread based on the probability information, so that the generation of unnecessary threads can be suppressed by efficiently executing the generation of a new thread and the termination of a thread. This leads to a reduction in power consumption.

[0151] A parallel processing apparatus according to the second embodiment of this invention will be discussed below in detail with reference to the accompanying drawings.

[0152] FIG. 8 is a block diagram illustrating the structure of the parallel processing apparatus according to the second embodiment of this invention.

[0153] As shown in FIG. 8, the parallel processing apparatus according to the second embodiment of this invention has two sets of processor elements (PE) 40 and 50 which are connected together by a bus B. The processor element 40 comprises a program counter register (hereinafter called “PC register”) 11, an instruction cache 12, an instruction decoding section 13, a thread-end-instruction predicting section 42, and an instruction executing section 15. The processor element 50 likewise comprises a program counter register 21, an instruction cache 22, an instruction decoding section 23, a thread-end-instruction predicting section 52, and an instruction executing section 25.

[0154] Those of the components, excluding the thread-endinstruction predicting sections 42 and 52, are the same as the above-described corresponding components of the first embodiment that have the same reference numerals. In accordance with history information of a thread-end instruction retained in the thread-end-instruction predicting section 42 or 52, that predicting section 42 or 52 predicts whether or not a thread-end-conditioned thread-end instruction terminates a thread.

[0155] The general operation of the parallel processing apparatus with the above-described structure according to the second embodiment of this invention will be discussed below.

[0156] FIG. 9 is a flowchart illustrating the general operation of the parallel processing apparatus according to the second embodiment of this invention.

[0157] In FIG. 9, IF, ID, EX, MEM and WB are pipeline stages and respectively indicate the instruction fetching stage, the instruction decoding stage, the instruction executing stage, the memory access stage, and the write-back stage.

[0158] In the IF stage, the processor element 40 sends an address stored in the PC register 11 to the instruction cache 12 (step S50). Then, the instruction decoding section 13 fetches from the instruction cache 12 an instruction specified by the address sent to the instruction cache 12 from the PC register 11 (step S52).

[0159] In the next ID stage, the instruction decoding section 13 decodes the instruction fetched in step S12 and determines a process to be executed. The instruction decoding section 13 also determines if the decoded instruction is a thread-end-conditioned thread-end instruction (step S54). When the fetched instruction is a thread-end-conditioned thread-end instruction, the instruction decoding section 13 informs the thread-endinstruction predicting section 42 to that effect. When the decoded instruction is not a thread-end-conditioned thread-end instruction, on the other hand, this instruction is executed in step S72.

[0160] When informed of the instruction being a thread-end-conditioned thread-end instruction, the thread-end-instruction predicting section 42 predicts whether or not the thread-end-conditioned thread-end instruction will terminate a thread in accordance with the history information of the fork instruction that is held in the predicting section 42 (step S56).

[0161] When the decision result is “YES”, i.e., when it is predicted that the thread-end-conditioned thread-end instruction will end a thread, the thread-end-instruction predicting section 42 sends information indicating the termination of the thread to the instruction executing section 15 (step S58).

[0162] When the decision result in step S56 is “NO”, i.e., when it is predicted that the thread will be terminated, the thread-endinstruction predicting section 42 does not send information indicating the termination of the thread and proceeds to step S66.

[0163] In the EX stage, the instruction executing section 15 executes the instruction decoded by the instruction decoding section 13. The instruction executing section 15 also determines if the prediction of the thread-end-conditioned thread-end instruction is correct (steps S60 and S66).

[0164] When the decision result in step S60 is a “success”, it means that the prediction that “the thread will be terminated” has been successful. In this case, the instruction executing section 15 informs the thread-end-instruction predicting section 42 of the “successful prediction”. Further, the thread-endinstruction predicting section 42 updates the history information of the thread-end-conditioned thread-end instruction (step S62).

[0165] When the decision result in step S60 is a “failure”, it means that the prediction that “the thread will be terminated” has ended in failure. In this case, stopping the fetching of subsequent instructions is released and the instruction executing section 15 sends the address of a subsequent instruction to the PC register 11 via the bus B. The instruction executing section 15 informs the thread-end-instruction predicting section 42 of the “prediction failure”. When informed of the failure, the thread-end-instruction predicting section 42 updates the history information of the thread-end-conditioned thread-end instruction (step S64).

[0166] When the above processing is completed, the processor element 40 continues executing the instruction.

[0167] When the decision result in step S66 is a “success”, it means that the prediction that “the thread will not be terminated” has been successful and the instruction executing section 15 informs the thread-end-instruction predicting section 42 of the “successful prediction”. Then, the thread-endinstruction predicting section 42 updates the history information of the thread-end-conditioned thread-end instruction (step S68).

[0168] When the decision result in step S66 is a “failure”, it means that the prediction that “the thread will not be terminated” has ended in failure and the fetching subsequent instructions is stopped. The instruction executing section 15 informs the thread-end-instruction predicting section 42 of the “prediction failure” and the thread-end-instruction predicting section 42 updates the history information of the thread-end-conditioned thread-end instruction (step S60).

[0169] According to this embodiment, as described above, the provision of the thread-end-instruction predicting sections 42 and 52 which predict the execution condition of a thread-end-conditioned thread-end instruction using the history information can achieve efficient termination of a thread and eventually leads to a reduction in power consumption.

[0170] An explanation will now be given of an algorithm for predicting the termination of a thread by using history information which is used in the thread-end-instruction predicting sections 42 and 52 in FIG. 8.

[0171] FIG. 10 is a status-transition chart for explaining the algorithm for predicting the end of a thread by using history information which is used in the thread-end-instruction predicting section 42.

[0172] The history information stored in the history buffer which is provided in the thread-end-instruction predicting section 42 can represent four states. The four states are:

[0173] highly unlikely termination (St 31) unlikely termination (St 32) likely termination (St 33) highly likely termination (St 34)

[0174] When the history information indicates either “highly unlikely termination” (St 31) or “unlikely termination” (St 32), the predicting section 34 predicts that a thread will not be terminated. When the history information indicates either “likely termination” (St 33) or “highly likely termination” (St 34), the predicting section 34 predicts that a thread will be terminated.

[0175] The state of the history information changes in accordance with the result of a decision on prediction as follows.

[0176] In the case where the history information specifies “highly unlikely termination” (St 31), when the prediction is a success, the state does not change and remains as “highly unlikely termination” (St 31). When prediction is a failure, the state changes to “unlikely termination” (St 32).

[0177] In the case where the history information specifies “unlikely termination” (St 32), when the prediction is a success, the state changes to “highly unlikely termination” (St 31). When the prediction ends in failure, the state changes to “likely termination” (St 33).

[0178] In the case where the history information specifies “likely termination” (St 33), when the prediction is successful, the state changes to “highly likely termination” (St 34). When the prediction is unsuccessful, the state changes to “unlikely termination” (St 32).

[0179] In the case where the history information specifies “highly likely termination” (St 34), when the prediction is successful, the state does not change and stays as “highly likely termination” (St 34). When the prediction ends in failure, the state changes to “likely termination” (St 33).

[0180] Using the above algorithm, the predicting section 34 predicts whether or not to end a thread based on the input history information.

[0181] The foregoing description of the first embodiment has mainly discussed the case where a thread is to be generated (forked) and the foregoing description of the second embodiment has mainly discussed the case where a thread generated by forking is to be terminated. It is however preferable that a parallel processing apparatus should have the combined structure of the first embodiment and the second embodiment in order to achieve the objects of this invention. That is, it is preferable that the parallel processing apparatus of this invention is equipped with means that predicts both the forking of a thread and the termination of a thread.

[0182] According to those embodiments, information about the probability of terminating a thread may be included in advance in an instruction at the time of compiling the instruction, so that when this information is included in an instruction, the thread-end-instruction predicting section 42 predicts the termination of a thread based on that probability information.

[0183] A parallel processing apparatus according to the third embodiment of this invention will specifically be discussed below.

[0184] The parallel processing apparatus according to the third embodiment of this invention fundamentally has the same structure as the combined structure of the first and second embodiments. The third embodiment differs from the first embodiment in the operation of the fork-instruction predicting section 14 or 24 shown in FIG. 1, particularly the operation of the predicting section 34 shown in FIG. 2. According to this embodiment, in the parallel processing apparatus, a branching-etc. predicting section 60 is provided in place of the fork-instruction predicting section 14 or 24 of the first embodiment, so that when any one of a conditional branching instruction, a speculation fork instruction, and a thread-end-conditioned thread-end instruction is input, the generation or termination of a thread is carried out based on the history information.

[0185] FIG. 11 is a block diagram illustrating the structure of the branching-etc. predicting section 60 included in the parallel processing apparatus according to the third embodiment of this invention.

[0186] The branching-etc. predicting section 60 comprises an address calculator 30, a history buffer 32, a predicting section 62, and a history-information update section 36. The address calculator 30, the history buffer 32 and the history-information update section 36 are the same as those of the first and second embodiments which have already been discussed.

[0187] With this structure, when a conditional branching instruction, a speculation fork instruction, or a thread-end-conditioned thread-end instruction is confirmed in the instruction decoding stage, the instruction is input to the address calculator 30 and its instruction address is input to the history buffer 32. When the instruction address and the instruction are input to the branching-etc. predicting section 60, history information is output from the history buffer 32 in accordance with each instruction address. When the input instruction is a conditional branching instruction or a speculation fork instruction, the address calculator 30 computes the instruction address of the branching destination or the forking destination, depending on the type of the instruction.

[0188] The history information output from the history buffer 32 is input to the history-information update section 36 and the predicting section 62. In the case of a conditional branching instruction or speculation fork instruction, the predicting section 62 sends out the address computed by the address calculator 30.

[0189] When the input instruction is a thread-end-conditioned thread-end instruction, the predicting section 62 sends out a thread-end signal according to the history information.

[0190] It is to be noted that the predicting section 62 performs predicting in the same manner as done in the first and second embodiments.

[0191] When the result of a decision from the instruction executing section (not shown) is input to the history-information update section 36, the history-information update section 36 updates the history information of the predicted instruction and writes back the updated history information in the history buffer 32.

[0192] As described above, this embodiment can integrate a series of predictions about a conditional branching instruction, a speculation fork instruction, and a thread-end-conditioned thread-end instruction.

[0193] FIG. 12 is a table for explaining the history information in the history buffer 32. The history information indicates four states using 2-bit signals. Specifically, the four states are distinguished from one another by “00”, “01”, “10” and “11”.

[0194] The history information of a conditional branching instruction, a speculation fork instruction, a speculation fork instruction including analysis information before execution, a thread-end-conditioned thread-end instruction, and a thread-end-conditioned thread-end instruction including analysis information before execution is indicated by the four states.

[0195] Analyzing whether the instruction input to the branching-etc. predicting section is one of the instructions shown in FIG. 12 can allow the entries in the history buffer 32 to be shared.

[0196] According to this embodiment as in the other embodiments, information about the probability of forking a thread, information about the probability of terminating a thread, or information about the probability of a conditional branching instruction may be included in advance in an instruction when that instruction is compiled, so that when this information is included in an instruction, the forking of a thread, the termination of a thread, or a conditional branching instruction is predicted based on that probability information.

[0197] In short, as is apparent from the foregoing description, this invention has the following advantages.

[0198] In a processor system that carries out parallel processing using a plurality of processors, fork instructions for activating a plurality of processors are efficiently executed, thus making is possible a reduction of power consumption.

[0199] It is also possible to efficiently terminate a thread with respect to a thread-end-conditioned thread-end instruction.

[0200] Further, it is possible to provide a hardware unit which can carry out a series of predictions about a conditional branching instruction, a speculation fork instruction, and a thread-end-conditioned thread-end instruction.