Method for performing dynamic optimization of computer code

[0002] The present invention relates to the field of computer software optimization. In particular, the present invention relates to a method of dynamically optimizing compiled Java programs using the technique of path profiling.

[0003] Most computer programs exist as compiled object code. The compiled object code is designed to operate only on one particular operating system running on one particular computer processor architecture. To distribute a program for several different computer architectures the original source code must be compiled into object code for each different computer processor architecture and each different operating system.

[0004] To create a highly portable programming environment, Sun Microsystems, Inc. introduced Java. Java is an object oriented programming language that is compiled from a well-defined source code into Java byte codes. The Java byte codes are designed to execute on a “Java Virtual Machine.” A Java Virtual Machine is not an actual hardware platform. Instead, a Java Virtual Machine is a low level software emulator that can be implemented on many different computer processor architectures and under many different operating systems. Thus, a single compiled Java program can be distributed to any computer processor architecture and operating system that has a Java Virtual Machine implementation available.

[0005] Early Java Virtual Machines were implemented as interpreters. In a Java interpreter, each Java byte code is examined and a corresponding operation is performed in the Java Virtual Machine. Although Java byte code interpreters achieve the goal of having a portable programming environment, Java interpreters suffer from slow performance.

[0006] To improve the performance of Java Virtual machines, Java compilers were created. Java compilers translate the Java byte codes into native computer instructions for the particular computer architecture that the Java Virtual Machine is running on. Then, the computer processor runs the Java program by directly running the compiled native code. Such compiled Java programs execute much faster than Java programs that are executed using a Java interpreter.

[0007] In many applications Java programs are distributed in real time across a computer network. For example, a user may download a Java program from a server into a local client computer system. The user then typically wishes to immediately execute the downloaded Java program. To provide an immediate response to the user, a Java byte code compiler immediately compiles the Java program into native processor code. This technique is known as “Just-In-Time ” compiling. Since the Just-In-Time compilation must be performed very quickly, the Just-In-Time compilation is usually not well optimized. Thus, although a Just-In-Time compiled Java program executes faster than an interpreted Java program, such Just-In-Time compiled Java programs can be improved further.

[0008] The present invention introduces a method of optimizing computer code. The method of the present invention first compiles source code into an object code. While compiling, the method introduces instrumentation code into the object code that performs path profiling. The path profiling instrumentation code counts the number of times each execution path is executed to determine a “hot” execution path. The hot execution path is then dynamically optimized.

[0009] The objects, features and advantages of the present invention will be apparent to one skilled in the art, in view of the following detailed description in which:

[0010] FIG. 1 illustrates the list of steps in downloading and running a Java program in a Java Virtual machine with an interpreter.

[0011] FIG. 2 illustrates a flow diagram that list the steps of downloading, compiling, and running a Java program in a Java Virtual Machine that compiles the code before execution.

[0012] FIG. 3 illustrates a block diagram of a Java Virtual Machine on a client computer system.

[0013] FIG. 4 list the steps of downloading, compiling, running, and optimizing a Java program according to the teachings of the present invention.

[0014] FIG. 5 illustrates example source pseudocode that has been divided into code regions.

[0015] FIG. 6aillustrates a listing of example code divided into basic blocks of code.

[0016] FIG. 6billustrates a graphical path diagram of the possible data paths through basic blocks of code from FIG. 6a.

[0017] FIG. 7 graphically illustrates a section of code that has been compiled and then later optimized once a hot execution path has been selected.

[0018] A method and apparatus for dynamically optimizing compiled computer programs is disclosed. In the following description, for purposes of explanation, specific nomenclature is set forth to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that these specific details are not required in order to practice the present invention. For example, the present invention has been described with reference to Java object-oriented programming language. However, the same techniques and teachings can easily be applied to other types of compiled object code.

[0019] Java programs (also known as Java class files) are often distributed in a computer network environment. For example, an Internet World-Wide-Web (WWW) site may store a Java class file consisting of Java byte codes. To run such a Java class file in a computer network environment, a user first accesses a computer server to download the desired Java class file into a client computer system where the Java class file is then executed.

[0020] An example of running a Java class file in a computer network environment is provided with reference to FIG. 1 and FIG. 3. FIG. 1 illustrates a set of steps that describe how a Java class file can be run by a user that has a client computer system with a Java Virtual machine interpreter. FIG. 3 illustrates, in block diagram form, the elements of a client computer system equipped to interpret Java class files.

[0021] To run a Java class file in a computer network environment, a user first accesses a computer server and downloads the desired Java class file into a client computer system as stated in step 110 of FIG. 1. FIG. 3 illustrates a block diagram of a client computer system that can interpret a Java class file. In client computer system 300, the computer hardware 310 is controlled by an operating system 320. The client computer system 300 includes a Java Virtual Machine 330 implementation for running Java class files. The Java Virtual Machine 330 running on client computer system 300 relies on services from the operating system 320 and the underlying computer hardware 310. To interpret Java class files the Java Virtual machine 330 uses a Java interpreter 331.

[0022] After the Java class file has been downloaded, then the Java class file passed to the Java Virtual machine 330 in the client computer system 300. The Java Virtual machine 330 first verifies the Java class file at step 130. The verification of the Java class file ensures that the program will not cause security violations nor will it harm computer system resources.

[0023] After the Java program has been verified, the Java interpreter 331 of the Java Virtual Machine 330 begins interpreting the Java bytes codes in order to execute the Java application as stated in step 150. The Java interpreter 331 interprets the Java byte codes one by one. Since the Java interpreter 331 interprets Java byte codes that are not native processor code, such Java interpreting tends to be slow. To improve performance of Java programs, Java “Just-In-Time” compilation was created.

[0024] FIGS. 2 and 3 will be used to describe how a Java class filed can be compiled and then executed. FIG. 2 illustrates the steps performed in downloading, compiling, and executing a Java program. Initially, at step 210, the user accesses a Java class file on a computer network and downloads the Java class file into the local client computer system 300. The Java Virtual Machine 330 then verifies the Java class file at step 220.

[0025] After verification of the Java class file, a Just-In-Time Java compiler 333 compiles the Java class file at step 230. Specifically, the Just-In-Time Java compiler 333 compiles the Java byte codes into native code for the processor of client computer system 300. Referring to FIG. 3, when a Java class file has been processed by the Just-In-Time Java compiler 333, the Just-In-Time Java compiler 333 creates compiled Java code 340. The compiled Java code 340 is then directly executed by the computer hardware 310 at step 240. To maintain the state of the Java Virtual machine 330, the compiled Java code 340 makes calls into the Java Virtual machine 330.

[0026] The present invention introduces a new method of executing Java class files. The present invention combines techniques of Just-In-Time compilation of Java class files and path profiling optimization. Specifically, each Java class file is first compiled as necessary before execution. Then, as the compiled Java program executes, an execution path profile is built to determine which execution paths are executed most often. When a particular execution path is determined to be executed very often, that particular execution path is then dynamically optimized. The present invention will be described with reference to FIGS. 4, 5, 6a,6b,and 7.

[0027] FIG. 4 illustrates an overall flow diagram of the Java class file execution and optimization method of the present invention. Initially, at step 410, a Java class file is downloaded into a local client computer system. The Java Virtual Machine of the client computer system then verifies the Java class file at step 420 to ensure that the Java class file is safe to execute.

[0028] Code Regions

[0029] Next, the present invention divides the Java class file into code regions at step 430. The present invention defines a code region as:

[0030] 1) A method body of code; or

[0031] 2) A program loop.

[0032] This definition is recursive since each loop within a method body is a region and each loop within a loop is a region.

[0033] FIG. 5 illustrates an example of pseudocode that has been divided into code regions. Referring to FIG. 5, the main method body is a first region of code labeled region 1. The main body region begins with instructions A, B, and C. After instructions A, B, and C there is a first program loop. The program loop includes instructions D, E, an “if” condition including instructions F and G, and a final instruction H. This program loop within the main method body comprises a second region labeled as region 2. After region 2, there are two more instructions—instruction I and J—that are part of region 1. After instruction J there is another program loop. The program loop after instruction J is a third region of code defined as region 3. Within region 3 there is yet another program loop. That program loop defines yet another region of code known as region 4. After the program loop of region 3 there are four more instructions R, S, T, and U and then the first region of code ends.

[0034] Referring back to FIG. 4, after a Java class file has been divided into code regions, the Java Virtual Machine then begins to execute the code. Before executing a region of code, the code region needs to be compiled first. Thus, at step 440, when a region of code begins to execute the method of the present invention first determines if the code region has yet been compiled. If the code region has not yet been compiled, then the method proceeds to step 450 where that region of code is compiled into native processor code along with some instrumentation that will be described later. Only that particular region of code that will be executing is compiled into native code. Any sub-regions of code that are within the code region to be executed are not compiled yet. Instead, hooks are left in the compiled code to jump to those code regions that have not yet been compiled. Thus, returning to FIG. 4 after the region of code has been compiled then that region of code is then executed at step 460.

[0035] When the region of code is executing and encounters a jump to a sub-region of code, the method proceeds back to step 440. At step 440, the present invention again determines if this region of code has been compiled yet. If the region has not been compiled yet, then region of code will be compiled as stated in step 450. Otherwise, if that region of code has been compiled then that region of code begins executing at step 460.

[0036] A description of code region execution is best provided by an example. An example of code region execution will be provided with reference to FIGS. 4 and 5. Referring back to step 430 of FIG. 4, each Java program is divided into code regions. FIG. 5 provides an example of code divided into code regions where a section of pseudocode has been divided into four regions. When the code of FIG. 5 begins execution, region 1 is initially selected for execution.

[0037] The first step of executing the code in FIG. 5 is to determine if this code region has already been compiled. At the initial execution, the code is not yet compiled thus region 1 must be compiled. Referring to step 450, the Java code is compiled into native code with additional instrumentation. Specifically, instructions A, B, C, I, J, and instructions R, S, T, U of region 1 are compiled into native code. Furthermore, hooks are placed into the compiled code of region 1 such that region 2 and region 3 will be executed after instructions C and J respectively. Furthermore, the compiled code will also include path profiling instrumentation hooks that will be described later.

[0038] When region 1 begins executing, compiled instructions A, B, and C will execute and then a hook to region 2 will be encountered. At this point region 2 is entered as a sub-region. Referring back to FIG. 4, the method of the present invention first determines if this code region is compiled. Since region 2 has not yet been compiled, region 2 needs to be compiled into native code with instrumentation. Thus, all the instructions within the loop of region 2 are compiled into native code. Region 2 is then executed. After region 2 has completed execution, the execution path returns back to region 1 which then continues execution. The execution of the code regions continues along in this manner.

[0039] For each region of code that is defined the present invention performs path profiling. Path profiling consists of determining all the possible execution paths through a region of code and then determining which execution paths are executed. When a particular execution path in a code region is determined to be executed very often (a “hot” path), then that particular execution path will be optimized such that the execution of that region of code will be optimized as long the execution flow follows the optimized execution path.

[0040] Basic Blocks and Execution Paths

[0041] An example of path profiling will be given with reference to FIGS. 6aand 6b.FIG. 6aillustrates a region of pseudocode. The region of pseudocode depicted in FIG. 6acontains no loop. Accordingly, the region of code in FIG. 6aconsists of a single code region that contains no sub-regions. (This example contains no sub-regions in order to simplify the example.)

[0042] To perform path profiling, each region of code is divided into basic blocks of code. A basic block of code is defined as a set of instructions in between branch instructions. Referring again to FIG. 6a,the code has been divided into six basic blocks labeled A through F. The first basic block of code (basic block A) consists of instructions 1, 2, 3, and the conditional “if” branch instruction following instruction 3. The next basic block of code (basic block B) consists of instructions 4, 5, and 6 that are performed if the conditional branch of basic block A is true. The next basic block of code (basic block C) comprises instructions 7, 8, and 9 that are executed if the condition in the conditional branch of basic block A is false. The next basic block (basic block D) comprises instructions 10, 11, 12, and a second conditional “if” branch instruction. The next basic block of instructions (basic block E) comprises instructions 13, 14, and 15 that are executed if the second conditional branch is true. Finally the last basic block of instructions (basic block F) comprises instructions 16, 17, and 18 that are always executed at the end of the code region.

[0043] The six basic blocks of code (basic blocks A, B, C, D, E, and F) from the code region in FIG. 6aare illustrated graphically in FIG. 6b.As illustrated in FIG. 6bthere is a first basic block A and then there are basic blocks B and C which may be executed depending on the first condition branch. After basic blocks B and C there is a basic block code D which is always executed. The end of basic block D has a branch instruction. Basic block E is executed after by basic block D only if the second condition is true. Finally, basic block of code F is always executed at the end of the code region. Depending on the conditional branching of a code region, several different execution paths may be followed in each code region. In the example code of FIG. 6a(depicted graphically as FIG. 6b), there are four possible different paths that may be executed. A first path that may be executed would be basic blocks A, B, D, E, F. A second path that may be executed would be basic blocks A, C, D, E, F. A third path that may be executed would be basic blocks A, B, D, F. And the final path that may be executed would be basic blocks A, C, D, F. Each possible execution path is labeled 1 through 4 respectively.

[0044] Execution Path Counters

[0045] When each code region is compiled by the Just-In-Time compiler 333 of the present invention, the Just-In-Time compiler 333 introduces instrumentation code after each basic block of code that counts and determines which basic block is being executed. At the end of the code region, a path profile counter determines which of the possible execution paths was followed. Thus, after each region of code is executed the present invention will know which execution path was followed.

[0046] Referring back to FIG. 4 after each region of code is executed at step 460 the method then determines whether or not the Java program has completed execution at step 465. If the program has completed execution then the program is completely done. If not, then an execution path counter associated with the followed execution path is incremented at step 470. The counter associated with the taken execution path is compared with a threshold value at step 480. If the execution path counter exceeds the threshold value, then that followed execution path is optimized at step 490. The optimized execution path contains no branches such that the optimized execution path will execute very quickly. Other optimization techniques that may be used in the optimization of the selected path include loop unrolling, dead code removal, common subexpression elimination, optimized register allocation, optimized instruction scheduling, copy propagation, constant propagation, and constant folding. After optimization, program execution then resumes at step 440.

[0047] Path Profiling Example

[0048] An example of how an execution path profile is maintained and used to optimize program code will be provided with reference to FIGS. 4, 6a,6b,and 7. Referring to step 460, each region of code is executed after the region of code has been compiled. After the execution of a code region, the method of the present invention determines at step 465 whether the Java class file has completed execution. If the program has completed execution, then program execution ends. Otherwise, a path counter associated with the taken execution path of the code region is incremented at step 470.

[0049] For example, referring to FIG. 6b,if the region of code follows the execution path of basic blocks A, B, D, E, and F then counter 1 associated with the first possible execution path will be incremented. Referring back to FIG. 4 the execution path counter is then compared with a threshold value at step 480. If the execution path counter does not exceed the threshold, then the Java program continues execution at the next region of code by returning to step 440.

[0050] However, if the execution path counter does exceed the threshold value then the method of the present invention proceeds to step 490 where an optimizing compiler generates an optimized set of code for that particular execution path. Specifically, the basic blocks of instructions of the execution path are compiled in that order. The optimized execution path code will execute very quickly provided that the optimized “hot” path is followed. However, there are hooks in the optimized execution path such that if the optimized path is not followed, then the program will be able to branch back to the original compiled code region.

[0051] FIG. 7 graphically illustrates the original compiled code 710 of the code region of FIGS. 6aand 6b.The original compiled code 710 is preceded by a “path selector” 705 that selects an execution path to follow for the code. The original compiled code 710 includes several different possible paths that may be executed. Specifically, the original compiled code 710 down the center path matches the diagram of FIG. 6b.

[0052] After the original compiled code 710 has executed a sufficient number times such that execution path 1 has exceed the threshold value, the present invention will create the first optimized path 720 as illustrated in FIG. 7. The first optimized path 720 comprises the basic blocks compiled in the order of A, B, D, E, F. Between the basic block there are hooks such that if the code does not exactly follow the optimized path, then the code will branch back to the original compiled code 710. For example, if after executing basic block A along the first optimized path 720 the code determines that it should proceed to basic block C then the code will branch back to basic block C in the original compiled code 710 as illustrated in FIG. 7.

[0053] As the region of code continues to execute, a different execution path may exceed the threshold value. For example, referring to FIG. 6b,path 4 along basic blocks A, C, D, F may exceed the threshold value. In such a case, the method of the present invention will create a second optimized path along basic blocks A, C, D, and F. This second optimized path 730 is illustrated in FIG. 7. When this particular region of code is executed, the path selector 705 will now select the second optimized path 730 as the execution path to execute. Again, if the code does not exactly follow the second optimized path 730 then the code will branch back to the original compiled code 710. To prevent interruption of the executing Java class file, the optimization of “hot” paths may be performed by a second thread that runs concurrently with the execution of the Java class file.

[0054] The foregoing has described a method for performing dynamic optimization of computer code. It is contemplated that changes and modifications may be made by one of ordinary skill in the art, to the materials and arrangements of elements of the present invention without departing from the scope of the invention.