Programming system and thread synchronization mechanisms for the development of selectively sequential and multithreaded computer programs

[0002] The present application claims priority under 35 U.S.C. 119(e) from provisional application No. 60/112,817 filed Dec. 17, 1998.

[0003] Many computer programs are computationally intensive, meaning that they require large amounts of computing power. As a consequence, these programs may execute more slowly than the computer user desires, even on the fastest computers. One way of increasing the execution speed of a computationally intensive computer program is to divide the program into multiple units, or loci, of concurrent execution. These units of execution are known as “threads”. A program with multiple threads of execution is known as a “multithreaded program”. A program with only a single thread of execution is known as a “sequential program”. The threads that make up a multithreaded program may be executed concurrently on multiple computer processors, allowing many operations in the program to be carried out simultaneously, thereby speeding up program execution.

[0004] In a multithreaded program, the program or operating system must control the access of threads to data objects in the program, in order to the prevent the multiple threads from concurrently accessing the same data object in an undesirable manner. If multiple threads modify the same data object concurrently, or read and modify the same data object concurrently, the resulting state of the program is extremely difficult to determine. Developing a multithreaded program is significantly more difficult than developing a sequential program because of the problems of (1) expressing the division of a program into multiple threads and (2) structuring and controlling the access of those threads to data objects.

[0005] The present application teaches a structured thread (“Sthread”) system with thread synchronization and production mechanisms.

[0006] Another aspect produces multithreadable code. The multithreadable code can be annotated using information indicative of its multithreadability. The multithreadable code constructs are code constructs that can be executed in a multithreadable manner, or equivalently in a sequential manner. Multithreadable code constructs may be expressed by annotating sequential code constructs to indicate that their multithreaded execution is equivalent to sequential execution. The multithreadable code can be used along with multithreaded code. Specific instances of a multithreadable constructs: a multithreadable block, and a multithreadable for loop, and are disclosed.

[0007] The second aspect of the system is the integration of multithreadable code constructs with traditional explicitly multithreaded code constructs. Explicitly multithreaded code constructs must always be executed in a multithreaded manner. Examples of explicitly multithreaded code constructs include multithreaded block constructs, multithreaded for loop constructs, and library-based thread creation functions. Multithreadable code constructs and explicitly multithreaded code constructs may be intermingled within a program as required, with well-defined meaning.

[0008] According to a first aspect, a special counter called an “s-counter”, is used as a thread synchronization mechanism.

[0009] Special “s-Flags” can also be used for thread synchronization, and flag synchronization is also described herein.

[0010] Yet another aspect is the implementation of the programming system within an existing compiler environment using a special pre-processing system.

[0011] The embodiments of the invention describe additional details, including the following:

[0012] The s-counter synchronizing the access of threads to shared data objects. The mechanisms use “monotonic” synchronization objects, with operations that can be constrained to only move the value of the object in one direction. Monotonic synchronization objects can be used to synchronize the access of threads to shared data objects in multithreadable code constructs in a manner that guarantees the equivalence of sequential and multithreaded execution. Specific instances of monotonic synchronization objects are disclosed, namely a form of counter called an “s-counter” and a form of flag called an “s-flag”. The s-counter is a particularly powerful thread synchronization mechanism in many contexts, with its use in multithreadable code constructs being one example.

[0013] The application describes implementation of the multithreaded programming system within an existing program development and compilation environment using a special source-to-source preprocessing system and high-level thread library. This allows the system to be transparently and seamlessly integrated with existing programming systems such as Microsoft Visual Studio for the Microsoft Windows family of operating systems, the GNU program development tools for Linux and other versions of the Unix operating system, or on any version of the Java programming language, for example.

[0014] These and other aspects will now be described in detail with respect to the accompanying drawings, wherein:

[0015] FIG. 1 is a process flowchart showing a prior art method for compiling multithreaded code;

[0016] FIG. 2 shows a computer system and its thread allocation system;

[0017] FIG. 3 shows a flowchart of defining an s-counter; and

[0018] FIG. 4 shows a flowchart of Sthreads execution of a program.

[0019] FIG. 1 is a process flowchart showing a prior art method for compiling multithreaded code. Source code text 300 including multithreaded code constructs is processed by a conventional compiler 302. The compiler communicates with a linker 304 which links pre-existing routines from a library 306 with the output of the complier to create an executable module 308.

[0020] Existing operating systems, including the WIN32 API, often provide a general purpose thread library which may allow carrying out defined tasks like these For example, a first thread may be defined for operating the CD ROM, and another for the modem.

[0021] Windows NT WIN32 thread creation is unstructured. A thread is created by passing a function pointer and an argument pointer to a CREATETHREADcall. The new thread then executes the given function with the given argument. There is no specific relationship between the created thread and the creating thread: the two threads are effectively asynchronous. One thread for example, can arbitrarily suspend, resume or terminate the execution of another thread.

[0022] This is not a problem for unrelated tasks like CD/modem tasks noted above. However, when two parts of a program are to be executed as threads, the synchronization operations are often complex and error prone. Unpredictable interactions among the multiple threads can induce problems including race conditions, and deadlock. Effectively, the user is left with the daunting task of using these thread libraries in a way that does not cause this problem.

[0023] The present application discloses a specific embodiment operating using the Windows NT™ system. It should be understood, however, that this system is portable across many platforms and that the same concept described herein can be used in those systems, including Linux, and any other operating system.

[0024] While a process has its own address space, a thread is often simply a program counter and stack pointer. A process may have many threads but all the threads share the same address space.

[0025] FIGS. 2 and 3 show this operation in a computer system. FIG. 2 shows a computer system, with four processors 200, 202, 204, 206. The processors can be in a multiple processor system as shown. The pool 199 of processors is associated with an operating system 210, a user interface 215, auxiliary hardware 220 (e.g. memory, chipsets, etc), a display 225 and other computer components.

[0026] The operating system 210 includes multiple threads 212, 214, and others. Each thread is resident on the stack within the heap. The threads are associated with processors, which execute the threads. FIG. 2 shows the pool of threads on the left and the pool of processors on the right. Each of the dynamically-created threads are peers. Of course there can be many more threads than processors. The operating system controls the threads to dynamically switch between the processors.

[0027] The present application defines an entirely new way of creating, synchronizing, and handling the synchronization among threads. The system uses a new way of compiling code based on multithreadable code, either alone, or in conjunction with multithreaded code.

[0028] The operating system or programming language has a higher level system that uses special constructs called “equivalency annotations”. A lower level function call based system is used with special objects. Those special objects can synchronize among the threads in a way that prevents the objects within the threads from having ambiguous states.

[0029] Many of these systems are based on the concept of equivalence annotations. Equivalence annotations can take many forms—pragmas, special keywords, special kinds of comments, special characters, textual modifications (such as boldfacing, underlining, or italics), and others. They could be part of the program text, or in a separate file i.e., an extra file that contains nothing but the annotations. The pragma can have meaning to a compiler. Pragmas often form a specified syntax, but usually convey nonessential information that is intended to help the compiler to optimize the program.

[0030] The present embodiment uses these pragmas as special equivalence annotations. Pragmas are convenient for annotations since many programming language already provide pragmas for other purposes. While a pragma is described as being used as the preferred annotation of the present application, the program can certainly be annotated in other manners. For example, Java does not support pragmas, so a special kind of comment line could be used. The equivalence annotations described throughout this specification should be understood to be interpretable in this way.

[0031] The MULTITHREADABLEequivalence annotation can be a pragma when embodied in the C programming language. This indicates that a block or loop can be executed in a multithreaded manner. This means that there is no timing dependent nondeterminacy, and the system can execute the instructions into a multithreaded system.

[0032] The MULTITHREADEDequivalence annotation means that a block or loop must be executed in a multithreaded manner. The multithreaded execution need not be equivalent to sequential execution. Lock synchronization can be used to introduce nondeterminacy if desired.

[0033] The equivalence annotation becomes part of the operating system. Special, monotonically increasing and otherwise constrained S-COUNTERS, and similarly constrained S-FLAGSare operated to synchronize the access of threads to shared memory in order to prevent unwanted interference.

[0034] A special SYNCHRONIZATION COUNTER, or S-COUNTERis defined as an object with three basic attributes. The S-COUNTERhas a non-negative integer value. The object only allows an INCREMENToperation and a CHECKoperation. An initial value of the S-COUNTERobject is set to zero. An INCREMENTfunction automatically increases the value of the counter by a specified amount. The CHECKoperation suspends the calling thread until the value of the counter becomes greater than or equal to a specified level.

[0035] The multi-threaded programming system has a higher level notation includes annotation objects in the program code. Using the example of the c language, this can be described as “multi-threaded c”.

[0036] A lower level structured thread library is described as “Sthreads”. The special annotation objects are transformed into special Sthread calls by the Sthreads annotation objects pre-processor.

[0037] The multithreaded model uses the thread synchronization/annotation objects disclosed above to synchronize among the threads.

[0038] Threads can be created in different ways. A first thread creation construct is the multithreaded block. This is indicated by the MULTITHREADEDkeyword placed immediately before an ordinary C block:

[0039] This notation specifies that the statements of the block should be executed as asynchronous threads. This is a conventional way of referring to these threads. For example, the operating system could create threads to read from CD, and threads to read from tape. The threads are executed and proceed concurrently. They all share the same address space as the parent program. Execution does not continue past the multithreaded block until all the threads have individually terminated. It is typically illegal for the program to contain any kind of jump between the individual statements of the block, from inside the block to outside the block, or from outside the block to inside the block.

[0040] A second thread creation construct is the multithreaded for-loop, indicated by the MULTITHREADEDkeyword placed immediately before an ordinary for-loop:

[0041] MULTITHREADED

[0042] FOR(I=expression; Icomparison expression; I=I+expression)

[0043] statement

[0044] This notation specifies that the iterations of the loop should be executed as asynchronous threads. The threads all share the same address space as the parent program. Each thread, however, has a local copy of the loop control-variable with a different value from the iteration range. The iteration scheme can restrict to a single control-variable and expressions that are not modified within the loop body. Execution does not continue past the multithreaded for-loop until all the threads have individually terminated. It is illegal for the program to contain any kind of jump from inside the loop to outside the loop or from outside the loop to inside the loop. In essence, a multithreaded for-loop is a quantified form of multithreaded block.

[0045] Multithreaded and ordinary blocks and for-loops can be arbitrarily nested.

[0046] Traditional approaches have often been categorized as either being explicitly multithreaded or implicitly multithreaded. With explicit multithreading, the programmer expresses exactly how the operations of the program are executed by threads. Implicit multithreading is carried out when the programmer writes an ordinary sequential program. The programming system, e.g. the compiler, determines how the operations can be executed by separate threads.

[0047] The present application goes beyond the multithreaded concepts described above into a concept of multithreadable code constructs. The multithreadable construct can be executed according to a specified sequential operational semantics. The most common operational semantics would be executed sequentially. An alternative, however, allows the multithreadable code construct to be operated according to multithreaded operational semantics.

[0048] Rules are defined that constrain the multithreaded execution such that its result is equivalent to sequential execution.

[0049] As disclosed herein, the multithreadable code construct is formed of:

[0050] i) a syntactic description of the form of the construct,

[0051] (ii) a sequential operational semantics, that, when executed, defines how to execute the construct sequentially,

[0052] (iii) a multithreaded operational semantics, defining how to execute the construct by a set of threads, and

[0053] (iv) a set of implicit or explicit programming rules that are sufficient to ensure the equivalence of sequential and multithreaded execution of the construct.

[0054] The MULTITHREADABLEpragma becomes an assertion by the programmer that the BLOCKor FORloop can be executed in a multithreaded manner without changing the results of the program. The MULTITHREADABLEpragma can be applied to blocks and for loops in which the statements or iterations are independent of each other. The multithreaded execution is equivalent to sequential execution in this case. It is not a directive that the BLOCKor FOR LOOPmust be executed in a multithreaded manner.

[0055] As a simple example, consider the following program to sum the elements of a two-dimensional array:

[0056] Multithreaded execution of the FORloop is equivalent to sequential execution because the iterations all modify different ROWSUM[I] and Jvariables. The arguments following the pragma indicate that multithreaded execution should assign iterations to NUMTHREADSdifferent threads using a blocked mapping. There is a rich set of options that control the mapping of iterations to threads.

[0057] Therefore, the Multithreaded C preprocessor has two modes: sequential mode in which the MULTITHREADABLEpragma is ignored, and multithreaded mode in which the MULTITHREADABLEpragma is transformed into Sthreads calls. Programs can be developed, tested, and debugged in sequential mode, then executed in multithreaded mode for performance. In addition, performance analysis and tuning can often be performed in sequential mode.

[0058] Determinacy of results is an important consequence of the equivalence of multithreaded and sequential execution. If sequential execution is deterministic (which is usually the case), multithreaded execution must also be deterministic. Determinacy is usually desirable, since program development and debugging can be difficult when different runs produce different results. In other multithreaded programming systems, determinacy is difficult to ensure. For example, locks, semaphores, and many-to-one message passing almost always introduce race conditions and hence nondeterminacy. However, nondeterminacy is important for efficiency in some algorithms, e.g., branch-and-bound algorithms.

[0059] Multithreaded and multithreadable code constructs are integrated in this system. The programming system incorporates both explicitly multithreaded constructs which must be executed according to the multithreaded semantics, along with multithreadable constructs which may selectively executed according to their sequential or multithreaded semantics. The multithreaded constructs are generally used to express multithreaded algorithms that have no sequential equivalent. This can include controlling different hardware that haw no integration with one another, or controlling simultaneous different windows in a graphical user interface.

[0060] Multithreadable constructs are used to express the opportunity to use multithreading to speed up the execution of a computationally-intensive algorithm, by using multiple threads on multiple processors.

[0061] By using both multithreaded and multithreadable constructs, the operating system can use one thread to control each window with multithreaded constructs, and the output to each window, within a window, is computed faster with the multiple threads using multithreadable constructs.

[0062] As described above, the synchronization can be carried out by an entry S-COUNTER, or an S-FLAG. Each are defined to have certain constraints.

[0063] An S-COUNTER, defined in the context of the C programming language, is diagrammed in FIG. 3. It can be defined as a type definition and a set of interface functions. The counters are encapsulated as a class in an object-oriented language such as C++ or Java. The definition of the fundamental programming interface for S-COUNTERSis as follows:

[0064] typedef counter type definition Counter;

[0065] int InitializeCounter(Counter *c);

[0066] /* Initializes value(c) to zero. */

[0067] /* Must be called only once, before all other operations on counter c. */

[0068] int FinalizeCounter(Counter *c);

[0069] /* Must be called only once, after all other operations on c. */

[0070] int CheckCounter(Counter *c, unsigned int level);

[0071] /* Suspends until value(c) greaterorequal level. */

[0072] int IncrementCounter(Counter *c, unsigned int amount);

[0073] /* Increases value(c) by amount. */

[0074] An S-COUNTERobject c implicitly has a nonnegative integer attribute value(c), which can only be accessed through the interface functions. The INITIALIZEfunction at 300 initializes value(c) to zero or some initial value.

[0075] Importantly, the counter is monotonic, as illustrated in 310. No decrement function is defined. Its value monotonically increases.

[0076] Any attempt to CHECKthe counter, shown as step 320, suspends the calling thread at 325. This prevents a condition which can catch or miss some action occurring during the check operation. Each S-COUNTERmaintains a dynamic list of thread suspension queues 330, with one queue for each value on which at least one Check operation is suspended.

[0077] CHECKcompares value(c) to level and suspends until value(c) becomes greater than or equal to level. This is generically shown as AWAKE in step 340. Increment at 310 atomically increases value(c) by amount, thereby reawakening all Check operations suspended on values less than or equal to the new value(c).

[0078] All the functions can return an error code. Possible error conditions include invalid arguments, operations on an uninitialized counter, and counter overflow.

[0079] The type definition described above is carefully selected to remove the possibility of race conditions occurring on counter synchronization. There is no DECREASEoperation. Therefore, the value of an S-COUNTERis monotonically increasing. There is no possibility of a CHECKoperation missing an INCREMENToperation since check suspends the thread. There is no PROBEor nonblocking CHECKoperation. It is recognized by the inventor that any instantaneous value may depend on the relative timing of the individual threads. Therefore, no operation can be based on the instantaneous value of an S-COUNTER.

[0080] A RESEToperation can also be used to efficiently reuse counters between different phases of a program.

[0081] Alternatively, the old counters can be deleted and recreated as new counters. RESETsimply resets value(c) back to zero. However, to avoid the possibility of race conditions, RESETmust not be called concurrently with any other operation on the same counter. RESETends the process, and is not intended as a means of synchronization between threads.

[0082] Another thread synchronization object is a special flag, called an S-FLAG. S-FLAGS, like S-COUNTERS, have restricted allowed operations within the multithreadable code concept. S-FLAGSsupport SETand CHECKoperations. Initially, an S-FLAGis not set. A SEToperation on an S-FLAGatomically sets the flag. A CHECKoperation on a flag suspends until the flag is set. Once an S-FLAGis set, it remains set.

[0083] Flags and counters are provided to provide deterministic synchronization within multithreadable constructs, as previously described.

[0084] In summary of the above, an S-COUNTERobject has the following operations (expressed in the C programming language):

[0085] Initialize (Counter *c)

[0086] Finalize(Counter *c)

[0087] Increment(Counter *c, unsigned int amount)

[0088] Check(Counter *c, unsigned int value)

[0089] Reset(Counter *c)

[0090] The Initialize operation initializes the Counter object and sets its value to zero. The Finalize operation destroys the Counter object. An Increment operation increases the value of the Counter object by amount. A Check operation suspends the calling thread until the value of the Counter object is at least value. A Reset operation resets the value of the Counter object to zero.

[0091] In the following simple example, a “producer thread” produces items and writes them to a buffer. A group of one or more concurrently executed “consumer threads” each independently reads the items from the buffer. The key synchronization issue is to prevent the consumer threads from reading items from the buffer that have not yet been written by the producer thread. The following program fragment gives implementations of the producer thread and consumer threads (in the C programming language) using a counter for synchronization.

[0092] After writing a block of items to the buffer, the producer thread increments the S-COUNTER. Before reading a block of items from the buffer, a consumer thread checks the counter. If the next block of items has not yet been written to the buffer, the consumer thread suspends until enough items have been written. The program does not require that the producer and consumer threads all use the same blockSize values.

[0093] The monotonicity of counters helps guarantee deterministic synchronization and the equivalence of multithreaded and sequential execution.

[0094] If shared variables are guarded against concurrent operations, a program that uses only counter synchronization can produce deterministic results on all executions. Moreover, if sequential execution of the program (i.e., execution ignoring the MULTITHREADEDkeyword) does not deadlock, multithreaded execution is guaranteed not to deadlock and to produce the same results as sequential execution. These properties are extremely useful in the testing and debugging of multithreaded programs.

[0095] Even in the absence of concurrent operations on shared variables, traditional synchronization mechanisms can introduce nondeterminacy into a program through timing dependent race conditions between threads. For example, consider the following program that uses a lock:

[0096] Even though there are no concurrent operations on x, the resulting value of x is nondeterministic because of the race condition on the order in which the two threads acquire the lock. In contrast, because counters are monotonic, once a synchronization condition is enabled it remains enabled, and there is no possibility of a race condition to catch or miss a particular counter value. For example, consider the following program that uses a counter:

[0097] The resulting value of x is deterministic, because the CHECKCOUNTERoperations will succeed in the same order in all executions, therefore the operations on x will occur in the same order. Moreover, since sequential execution does not deadlock, multithreaded execution cannot deadlock and will always produce the same results as sequential execution.

[0098] Programs that use only counter synchronization can still be erroneously nondeterministic if they do not guard against concurrent access to shared variables. For example, consider the following program using a counter:

[0099] The result of the program is nondeterministic because of the possibility of concurrent execution of the operations on x. The nondeterminacy is caused by concurrent access to a shared variable, not by a synchronization race condition.

[0100] As a simple example, consider the following program to sum the elements of a two-dimensional array:

[0101] Without the counter operations, multithreaded execution of the for loop would not be equivalent to sequential execution, because the iterations all modify the same *sum variable. However, the counter operations ensure that multithreaded execution is equivalent to sequential execution. In sequential execution, the iterations are executed in increasing order and the STHREADCOUNTERCHECKoperations succeed without suspending. In multithreaded execution, the counter operations ensure that the operations on *sum occur atomically and in the same order as in sequential execution. Iteration i suspends at the STHREADCOUNTERCHECKoperation until iteration i−1 has executed the STHREADCOUNTERINCREMENToperation.

[0102] Conditions are carved out to prevent concurrent access to shared variables using counters. Essentially, each pair of operations on a shared variable must be separated by a transitive chain of counter operations. If these conditions can be shown to hold in any one execution of the program, they must hold in all executions of the program. Therefore, if sequential execution satisfies the conditions, multithreaded execution is also guaranteed to satisfy the conditions, hence produce the same results as sequential execution. This result forms the basis of a powerful methodology for developing multithreaded programs using sequential reasoning, testing, and debugging techniques.

[0103] All the programs using counters so far discussed satisfy the conditions on shared variables, therefore are guaranteed to be deterministic. In addition, the program examples described herein have equivalent multithreaded and sequential execution. The cost of increased determinacy is decreased concurrency. Synchronization using counters provides an effective means of controlling this tradeoff between determinacy and concurrency.

[0104] Counters can also be used as a stronger form of lock synchronization, providing sequential ordering in addition to mutual exclusion on a critical section. With the traditional implementation of mutual exclusion using a pair of lock operations, the order in which threads enter the critical section is nondeterministic. This is desirable in terms of maximizing concurrency, but is undesirable in terms of reasoning, testing, and debugging, and simply might not satisfy the desired program specification. Replacing the pair of lock operations with a pair of counter operations can guarantee deterministic results, at the cost of decreased opportunities for concurrency.

[0105] Consider the computation of a result object formed by accumulating a series of independent subresults that are computed concurrently. For example, the result object could be a linked list and the accumulate operation could be an append, or the result object could be a summation and the accumulate operation could be an addition. Mutual exclusion is required to prevent interference between multiple concurrent accumulate operations on the result object.

[0106] The following program implements the computation with one thread computing each subresult, and a pair of lock operations to provide mutual exclusion:

[0107] Only one thread can hold RESULTLOCKat any given time, thereby ensuring mutual exclusion of the ACCUMULATEoperations. However, if the accumulate operation is not associative and determinacy of results is desired, some other mechanism is required to ensure sequential (or at least deterministic) ordering, in addition to mutual exclusion. For example, neither appending an item to a linked list, nor floating point addition are associative operations. With both these examples, the above program may produce different results on repeated executions.

[0108] The following program implements the computation with the pair of lock operations replaced with a pair of counter operations, to provide both mutual exclusion and sequential ordering:

[0109] As with the lock program, only one ACCUMULATEoperation can execute at any given time. However, the accumulate operations are now additionally constrained to execute in sequential order. RESULTCOUNT[I]=Iindicates that thread I−1 has completed its accumulate operation. The counter program has greater determinacy at the cost of less concurrency. With the lock program, an ACCUMULATEoperation can execute concurrently with compute operations in all other threads with the counter program, an ACCUMULATEoperation can only execute concurrently with compute operations in higher numbered threads.

[0110] The optimal tradeoff between determinacy and concurrency has to be made on a case by case basis. Counters are a powerful mechanism for providing sequential ordering on top of mutual exclusion in the many cases where determinacy is important and the performance consequences of less concurrency are not great.

[0111] The Sthreads Interface

[0112] The code produced according to the present application can be expressed using the Multithreaded C pragma notation. As described previously, there is a direct correspondence between the pragma notation for thread creation and the Sthreads library functions that support thread creation. As a simple example, the following is a program implemented using Sthreads:

[0113] Although this program is syntactically more complicated than the Multithreaded C version, it is considerably less complicated than the same program expressed using Windows NT threads. The mechanics of creating threads, assigning iterations to threads, and waiting for thread termination is handled within the Sthreads library call.

[0114] The Sthreads multithreaded programming system is implemented as a transparent add-on to an existing program development system, e.g., a compiler or interpreter, or other program development environment. The notation and implementation allows multithreaded and multithreadable code constructs to be directly translated into a high-level structured thread library. This translation is implemented as a preprocessor that can be transparently called prior to the standard compilation phase in an existing program development system.

[0115] For example, when integrated with the Microsoft Visual C++ programming system, the standard CL (Compiler-Linker) is replaced by a special Sthreads tool that calls the Sthreads preprocessor on program files, then calls the standard (renamed) Visual C++ CL.

[0116] Integration of Sthreads with existing programming systems allows programmers new flexibility without adopting new programming systems to use the power of multithreading. They can use their standard editor, debugger, compiler, etc., and simply add Sthreads to the system. It also means that Sthreads piggybacks on the quality of code generation and error analysis of the underlying development system.

[0117] Preprocessing had previously been used for many kinds of program “source-to-source” transformations. Sthreads in contrast, implements a full-fledged, sophisticated multithreaded programming system by using a preprocessing integrated with a standard program development environment.

[0118] One implementation has been created in the ANSI C language, thereby defining a “Multithreaded C” language. A structured thread library (Sthreads) is called by the languages. In both Multithreaded C and Sthreads, thread creation constructs are multithreaded variants of sequential “block” (i.e., sections of program code distinctly defined by conventional program constructs) and “for loop” constructs. In the Multithreaded C implementation, these constructs are supported as pragma annotations to a sequential program. With Sthreads, exactly the same constructs are supported as library calls. At both levels, synchronization objects and operations are supported as Sthreads library calls. In this embodiment, the Sthreads library for Windows NT can be implemented as a very thin layer on top of the Win32 thread API. As a consequence, there is essentially no performance overhead associated with using Sthreads or Multithreadable C, as compared to using Win32 threads directly.

[0119] Multithreaded C is implemented as a portable source-to-source preprocessor that directly transforms annotated blocks and for loops into equivalent calls to the Sthreads library. The programmer has the option of either using the pragma annotations and preprocessor or making Sthreads calls directly. The Sthreads library and Multithreaded C preprocessor are integrated with Microsoft Developer Studio Visual C++. Building a project preferably automatically invokes the preprocessor where necessary and links with the Sthreads library.

[0120] Multithreadable blocks and for loops are implemented as a sequence of CREATETHREADcalls followed by a WAITFORSINGLEOBJECTcall on an event. Terminating threads perform an INTERLOCKEDDECREMENTcall on a counter, and the last thread to terminate performs a SETEVENTcall on the event. Flags are implemented directly as Win32 events. Counters are implemented as linked lists of Win32 events, with one event for every value on which some thread is waiting. Locks are implemented directly as Win32 critical sections. Barriers are implemented as a pair of Win32 events and a Win32 critical section.

[0121] An important issue of the multithreading operation comes about when considering multiple processors. The hardware and operating systems of modern technology allow for multiprocessor systems. Current operation in multiprocessor systems, however, have often simply operated on one but not the other processor. By multithreading in this way, the different threads can actually be executed on the different processors.

[0122] In operation, when a multithreading indicator (such as a “compile as multithreaded” flag/button/environment-variable) is set, both multithreadable and multithreaded blocks/loops are compiled to multithreaded code. When the multithreading indicator is not set, the multithreaded blocks/loops are compiled to multithreaded code and the multithreadable blocks/loops are compiled into ordinary sequential code. This allows a programmer to mix constructs that only have a multithreaded meaning (e.g., real-time control and systems programming uses of threads) with constructs that can be compiled into threads for multiprocessor performance or compiled into equivalent sequential code when developing and debugging.

[0123] The invention allows a program to run as fast as a sequential program on one processor, but significantly faster on multiprocessors, without recompilation, relinking, or reconfiguration. The invention thus allows a program to adapt dynamically to changing resources. Use of monotonic flags and monotonic counters makes embodiments of the invention reliable and timely.

[0124] The mapping of Statements/Iterations onto threads is relatively simple. One thread is used for each statement/chunk, or for a small number of statements/chunks.

[0125] A Typical for loop may have thousands or millions of iterations. The Overhead associated with assigning units of work to threads is significant. The present application defines assigning the iterations in contiguous “chunks”. Significant unit of work should be performed by each chunk.

[0126] For example:

[0127] #PRAGMA MULTITHREADABLE CHUNKSIZE(1000), MAPPING(BLOCKED(T))

[0128] FOR(I=0; I<N; I++)

[0129] A[I]=B[I];

[0130] Interaction of Chunksize and Mapping is described in the following example:

[0131] #PRAGMA MULTITHREADABLE CHUNKSIZE(2), MAPPING(BLOCKED(4))

[0132] FOR(I=50; I>=10; I=I−2)

[0133] DOSOMETHING(I)

[0134] The Complete Sthreads Library includes a number of statements:

[0135] Processor Management: STHREADSGETNUMPROCESSORSPRESENT, STHREADSSETNUMPROCESSORSUSED, STHREADSGETPROCESSORSPRESENT, STHREADSSETPROCESSORSUSED.

[0136] Thread Creation: STHREADSBLOCK, STHREADSREGULARFORLOOP.

[0137] Thread Scheduling: STHREADSGETCURRENTPRIORITY, STHREADSSETCURRENTPRIORITY.

[0138] Flags: STHREADSFLAGINITIALIZE, STHREADSFLAGFINALIZE, STHREADSFLAGSET, STHREADSFLAGCHECK, STHREADSFLAGRESET.

[0139] Counters: STHREADSCOUNTERINITIALIZE, STHREADSCOUNTERFINALIZE, STHREADSCOUNTERINCREMENT, STHREADSCOUNTERCHECK, STHREADSCOUNTERRESET.

[0140] Locks: STHREADSLOCKINITIALIZE, STHREADSLOCKFINALIZE, STHREADSLOCKACQUIRE, STHREADSLOCKRELEASE.

[0141] Barriers: STHREADSBARRIERINITIALIZE, STHREADSBARRIERFINALIZE, STHREADSBARRIERPASS, STHREADSBARRIERRESET.

[0142] Examples of computer program code implementing each of these constructs are set forth in the appendix.

[0143] FIG. 4 is a process flowchart showing a method for compiling multithreadable code in accordance with one embodiment of the invention. The computer program source code text 400 includes annotations defining multithreadable code constructs (and, optionally, multithreaded code constructs) and any necessary processor management, thread creation, and synchronization constructs (such as monotonic flags and counters). If a multithreading indicator is set 401, the source code text 400 is processed by a pre-processor 402 that parses the source into an expanded computer program text. The expanded computer program text includes inserted calls to an Sthreads library 406 to invoke multithreaded program operations wherever a source code annotation called for multithreadable, functionality. A conventional compiler 406 then communicates with a linker 410 which links pre-existing routines from the Sthreads library 406 with the output of the complier to create an executable module 412.

[0144] If the multithreading indicator is not set 401, the original computer program source code text 400 is compiled and is linked in conventional fashion, with each section of multithreadable code constructs compiled as sequentially executing code. Annotations not recognized by the compiler 408 are ignored.

[0145] A convenient implementation shortcut that permits ready use of conventional compilers and linkers is to rename a pre-existing compiler-linker executable file to a new name, and assign the old name of the compiler-linker executable file to the pre-processor. The pre-processor then can call the compiler-linker executable file when needed by invoking the new name.

[0146] Synchronization Using Locks

[0147] Locks are provided to express nondeterministic synchronization, usually mutual exclusion, within multithreaded BLOCKSand FORloops. Sthread locks support the usual ACQUIREand RELEASEoperations. The order in which concurrent ACQUIREoperations succeed is nondeterministic. Therefore, there is very little use for locks within multithreadable blocks and FORloops. As a simple example, consider the following program to sum the elements of a two-dimensional array:

[0148] Like the flag operations in the program, the lock operations in this program ensure that the operations on *SUMoccur atomically. However, unlike the flag operations, the lock operations do not ensure that the operations on *SUMoccur in the same order as in sequential execution, or even in the same order each time the program is executed. Therefore, since floating-point addition is not associative, the program may produce different results each time it is executed. However, because execution order is less restricted, this program allows more concurrency than the program described above. This is an example of the commonly-occurring tradeoff between determinacy and efficiency.

[0149] Synchronization Using Barriers

[0150] S-thread barriers are provided to express collective synchronization of a group of threads in cases when thread termination and recreation is too expensive. The barriers described herein support the usual PASSoperation. All the threads in a group must enter the PASSoperation before all the threads in the group are allowed to leave the Pass operation. In current systems, the cost of N threads executing a PASSoperation is less than the cost of creating and terminating N threads. Therefore, a typical use of barriers is to replace a sequence of multithreadable loops with a single multithreaded loop containing a sequence of barrier PASSoperations. However, with modern lightweight thread systems such as Windows NT, we are discovering that barriers are required for efficiency in very few circumstances.

[0151] A number of examples are described herein.

[0152] Trivial Example: Independent Iterations

[0153] A more difficult example is shown in the following.

[0154] As can be seen, iterations cannot be executed as separate threads because of nondeterminacy in the top. However, the counters allow determinacy between the system therefore enabling the system to be multithreaded.

[0155] Single-Writer Multiple-Reader Broadcast

[0156] Counters can be used to provide elegant, flexible, and efficient dataflow synchronization between a single writer and multiple readers of a sequence of items written to an array. In this synchronization pattern, reading an item does not remove it from the sequence—each reader independently reads the entire shared array. Because a counter has multiple thread suspension queues, a single counter object can be used to synchronize the writer thread and any number of completely independent reader threads, with each thread potentially having a different granularity of synchronization. The writer thread incrementing the counter broadcasts the availability of data to the entire set of reader threads.

[0157] The following program demonstrates the single-write multiple-reader broadcast pattern with synchronization on every item:

[0158] One WRITERthread and an arbitrary number of Reader threads are executed concurrently, with communication through the shared data array, and synchronization through the DATACOUNTcounter. At any point, some Reader threads may be suspended in their CHECKCOUNTERoperation, waiting for the writer thread to increment DATACOUNT, while other Reader threads may be reading data items that have previously been written. The READERthreads execute independently of each other and do not synchronize their actions in any manner. The synchronization pattern is strictly a one-to-many broadcast from the WRITERthread to the READERthreads.

[0159] Synchronization on every item that is written and read may be too expensive if the time taken to generate and use an item is too small. The single-reader multiple-writer broadcast pattern can be generalized to allow the writer and each reader thread to synchronize on a block of items instead of on individual items. The following program adds an individual granularity of blocked synchronization to the writer and each reader thread:

[0160] The WRITERand READERthreads now increment and check the DATACOUNTcounter in multiples of BLOCKSIZEand write and read the data array in blocks of items. There is no requirement that BLOCKSIZEbe the same in all threads. Different threads can be passed different BLOCKSIZEbased on their individual performance characteristics and requirements. This pattern is now extremely flexible and easily adaptable with regard to practical performance tuning.

[0161] The single-writer multiple-reader broadcast pattern is a dataflow synchronization pattern that occurs in many diverse applications of threads to multiprocessing. For example, in the Paraffins Problem, an array of molecules of a certain size can be generated by one thread and concurrently read by other threads that in turn generate arrays of larger molecules. The pattern is very different from, for instance, the multiple-writers multiple-readers bounded-buffer problem, which is elegantly solved using semaphores. Just as counters are not well suited to implementing bounded buffers, semaphores and other traditional synchronization mechanisms are not well suited to implementing the single-writer multiple-reader broadcast pattern.

[0162] Another Example Application: Aircraft Route Optimization

[0163] The Aircraft Route Optimization Problem is part of the U.S. Air Force Rome Laboratory C3I Parallel Benchmark Suite. For this application, we achieved better performance using Sthreads on a quad-processor Pentium Pro system running Windows NT than the best reported results for message-passing programs running on expensive Cray and SGI supercomputers with up to 64 processors The flexibility of shared-memory, lightweight multithreading, and sequential development methods allowed us to develop a much more sophisticated and efficient algorithm than would be possible on a message-passing supercomputer.

[0164] The C3I Parallel Benchmark Suite

[0165] The U.S. Air Force Rome Laboratory C3I Parallel Benchmark Suite consists of eight problems chosen to represent the essential elements of real C3I (Command, Control, Communication, and Intelligence) applications. Each problem consists of the following:

[0166] A problem description giving the inputs and required outputs.

[0167] An efficient sequential program (written in C) to solve the problem.

[0168] The benchmark input data.

[0169] A correctness test for the benchmark output data.

[0170] For some of the problems, a parallel message-passing program is also provided. Rome Laboratory maintains a publicly accessible database of reported performance results.

[0171] The C3I Parallel Benchmark Suite provides a good framework for evaluating our structured multithreaded programming system. The problems are computationally intensive and involve a variety of complex algorithms and data structures. The sequential program provides us with a good starting point and a fair basis for performance comparison. The performance database allows us to compare our results with those of other researchers. For these reasons, we are developing multithreaded solutions to several of the C3I Parallel Benchmark Suite problems.

[0172] The task in the Aircraft Route Optimization Problem is to find the lowest-risk path for an aircraft from an origin point to a set of destination points in the airspace over an uneven terrain. The risk associated with each transition in the airspace is determined by its proximity to a set of threats. The problem involves realistic constraints on aircraft speed and maneuverability. The aircraft is also constrained to fly above the underlying terrain and beneath a given ceiling altitude.

[0173] The problem is essentially the single-source, multiple-destination shortest path problem with a large, sparsely connected graph. The airspace for the benchmark is 100 km by 100 km in area and 10 km in altitude, discretized at 1 km intervals. The 100,000 positions in space correspond to 2,600,000 nodes in the graph, since each position can be reached from 26 different directions. Because of aircraft speed and maneuverability constraints, each node is connected to only nine or ten geographically adjacent nodes. Therefore, the graph consists of approximately 2.6 million nodes and 26 million edges.

[0174] The sequential algorithm to solve the Aircraft Route Optimization Problem is based on a queue of nodes. Initially the queue is empty except for the origin node. At each step, one node is removed from the queue. Valid transitions from this source node to all adjacent destination nodes are considered. For each destination node, if the path to the node via the source node is shorter than the current shortest path to the node, the path to the node is updated and the node added to the queue. The algorithm continues until the queue is empty, at which stage the shortest paths to all reachable nodes have been computed.

[0175] The queue is ordered on path length so that shorter paths are expanded before longer paths. This has a significant effect on performance. Without ordering, longer paths are expanded, then discarded when shorter paths to the same points are expanded later in the computation. However, whether the queue is ordered, partially ordered, or unordered does not affect the results of the algorithm.

[0176] The most straightforward approach to obtaining parallelism in the Aircraft Route Optimization Problem is to geographically partition the airspace into blocks, with one thread (or process) responsible for each block. Each thread runs the sequential algorithm on its own block using its own local queue and periodically exchanges boundary values with neighboring blocks. This approach is particularly appealing on distributed-memory, message-passing platforms, because memory can be permanently distributed according to the blocking pattern. If the threads execute a reasonably large number of iterations between boundary exchanges, good load balance can be achieved.

[0177] The problem with this algorithm is that, as the number of blocks/threads is increased the total amount of computation also increases. Therefore, any speedup is based on an increasingly inefficient underlying algorithm. At any time, the local queues in most blocks contain paths that are too long and are irrelevant to the actual shortest paths. The processors are kept busy performing computation that is later discarded. At any given time, it is only productive to work on an irregular and unpredictable subset of the graph. However, irregular and adaptive blocking schemes do not solve the problem, since there is usually equal work available in all blocks. The issue is the distinction between productive and unproductive work.

[0178] Our solution is to statically partition the airspace into a large number of blocks and to use a much smaller number of threads. A measure of the average path length is maintained with each local queue. At each step, the blocks with local queues containing the shortest paths are assigned to the threads. Therefore, the subset of blocks that are active and the assignment of blocks to threads change dynamically throughout program execution. This algorithm takes advantage of the symmetric multiprocessing model, in which all threads can access the entire memory space with uniform cost. It also takes advantage of the lightweight multithreading model to achieve good load balance, since the workload within each thread at each step is highly variable.

[0179] The ability to develop, test, and debug using sequential methods was crucial in the development of this sophisticated multithreaded algorithm. The entire program was tested and debugged in sequential mode before multithreaded execution was attempted. In particular, development of the complex boundary exchange and queue update algorithms would have been considerably more difficult in multithreaded mode.

[0180] The ability to analyze and tune performance using sequential methods was also very important. Good performance depended on exposing enough parallelism without significantly increasing the total amount of computation. We determined efficient values for the number of blocks, the number of threads, and the number of iterations between boundary exchanges by measuring computation times and operation counts of the multithreaded program in running in sequential mode. This detailed analysis would have been very difficult to perform in multithreaded mode. We avoided memory contention in multithreaded mode by avoiding cache misses in sequential mode. The analysis of memory access patterns in sequential mode is much simpler than in multithreaded mode.

[0181] All Pairs Shortest Paths Example

[0182] This example describes the algorithmic and performance advantages of counter synchronization. In the example, a counter is used as a less restrictive, and consequently more efficient, replacement for a barrier. The example program is a multithreaded solution to the all-pairs shortest-path problem using the Floyd-Warshall algorithm. Using traditional synchronization mechanisms, this problem can be solved using one barrier or, more efficiently, an array of condition variables. We show how the efficient solution can be implemented using a single counter instead of an array of condition variables. We give timing measurements comparing the performance of the barrier, condition variable, and counter algorithms.

[0183] The all-pairs shortest-path problem takes as input the edge-weight matrix of a weighted directed graph, and returns the matrix of shortest-length paths between all pairs of vertices in the graph. The graph is required to have no cycles of negative length, and the weight of the edge from a vertex to itself is required to be zero.

[0184] The following program solves the all-pairs shortest-path problem using the sequential Floyd-Warshall algorithm:

[0185] Initially, PATH[I] [J] is assigned EDGE[I] [J], for all Iand J. (For brevity, we use a notational shorthand for array assignment.) After the kth iteration, PATH[I] [J] is the shortest path from vertex Ito vertex Jwith intermediate vertices only in vertices 0 to k. Therefore, after N iterations, PATH[I] [J] is the shortest path from vertex i to vertex j with no restrictions on the intermediate vertices.

[0186] The following program solves the all-pairs shortest path problem using a multithreaded version of the Floyd-Warshall algorithm, with a barrier for thread synchronization:

[0187] The multithreaded outer loop creates NUMTHREADSthreads. Each thread executes the N iterations of the Floyd-Warshall algorithm on a subset of the rows of the path matrix. To keep the iterations synchronized, the threads pass through an N-way barrier at the end of each iteration. There are no sharing violations on the concurrent accesses to PATHacross the threads, because the algorithm will never assign to PATH[I] [K] or PATH[K] [J] during iteration k.

[0188] The barrier algorithm successfully divides the work among an arbitrary number of threads. However, in requiring that all threads complete each iteration before any thread begins the next iteration, the algorithm does not express the full opportunities for concurrency inherent in the data dependencies. As a consequence, the program is less than optimally efficient. N-way synchronization at the barrier is a bottleneck that creates delays on entry and exit, and processor load imbalance can occur if all threads do not reach the barrier simultaneously.

[0189] A More Efficient Multithreaded Solution Using Condition Variable Synchronization

[0190] The following program solves the all-pairs shortest path problem using a more efficient multithreaded version of the Floyd-Warshall algorithm, with an array of N condition variables for thread synchronization:

[0191] As with the barrier algorithm, each thread executes the N iterations of the Floyd-Warshall algorithm on a subset of the rows of the PATHmatrix. However, each thread can individually continue with its next iteration as soon as the necessary data is available, instead of waiting for the previous iteration to complete in all the other threads. Condition variable KDONE[K] is set when row k of the PATHmatrix has been computed in iteration K−1. Each thread waits on KDONE[K] before executing iteration k. To avoid sharing violations, row k of the PATH matrix computed in iteration K−1 is stored in KROW[K].

[0192] The condition variable algorithm avoids the inefficiencies associated with barrier synchronization. Threads synchronize individually, rather than in an N-way bottleneck, and faster threads can execute many iterations ahead of slower threads. Potentially, the N threads can be executing in up to N different iterations. One extra cost of this algorithm is the storage for the KROWmatrix. However, the most significant extra cost is allocation of N condition variables.

[0193] The following program solves the all-pairs shortest path problem using the efficient multithreaded version of the Floyd-Warshall algorithm, with a single counter for thread synchronization in place of N condition variables:

[0194] Operations on N different values of the single counter replace operations on N different elements of the array of condition variables. The algorithm has the same performance advantages over the barrier algorithm, without the cost of statically allocating and maintaining N synchronization objects. Internally, the counter may create synchronization objects for the distinct counter values on which threads are suspended. However, in practice, the number of these objects in existence at any given time is likely to be a small fraction of N.

[0195] Three Synchronization Patterns Example

[0196] Three examples of practical synchronization patterns are described that can be expressed more elegantly (and often more efficiently) using counters than with traditional synchronization mechanisms. For each of these synchronization patterns, a small example program is provided to demonstrate the pattern and a description of the importance of the pattern to real problems. This is far from an exhaustive list of patterns to which counters can usefully be applied. Counters are equally applicable to many other situations, particularly dataflow style synchronization patterns arising in the application of threads to multiprocessing.

[0197] Counters can often be used to replace traditional barrier synchronization with a less restrictive form of “ragged” barrier. With a ragged barrier, each thread waits at the barrier point only until its own individual data dependencies have been satisfied, instead of until the data dependencies of all threads have been satisfied, as with a traditional barrier. We have already given one example of this pattern in Section 0, with the multithreaded Floyd-Warshall algorithm to solve the all-pairs shortest-path problem. In this section, we give another more straightforward example, based on boundary exchange in a time-stepped simulation.

[0198] Consider a time-stepped simulation of a one-dimensional object subdivided into N cells. The state of internal cell i at time t is a function of the states of cells i−1, i, and i+1 at time t−1. The states of the leftmost and rightmost cells remain constant over time. An example is simulation of heat transfer along a metal rod. Similar boundary exchange requirements occur in most multithreaded simulations of physical systems in one or more dimensions. These requirements are traditionally satisfied using barrier synchronization.

[0199] The following program implements the simulation using one thread for each cell, with traditional barrier synchronization between threads before cell state exchanges and updates at each time step:

[0200] All threads synchronize at the barrier twice every time step: once before exchanging cell states, and again before updating cell states. However, complete barrier synchronization between all threads is unnecessarily restrictive. The conditions for safely exchanging and updating the cell states involve dependencies between pairs of neighboring cells, not across all cells. As a consequence of using barriers, the performance of the program is potentially subject to synchronization bottleneck and load imbalance problems.

[0201] The following program implements the same simulation using an array of counters to provide ragged barrier synchronization between threads:

[0202] As with the traditional barrier algorithm, the threads synchronize every time step before exchanging cell states, and again before updating cell states. However, the synchronization is between pairs of neighboring threads via an array of counters. C[I]=2*T−1 indicates that thread Ihas finished reading both neighboring cell states in time step T, and C[I]=2*Tindicates that thread Ihas completed time step T. Pairwise synchronization removes the synchronization bottleneck of a traditional barrier and reduces load imbalance by allowing some threads to execute ahead of other threads. The barrier could be made even more ragged using separate counters to synchronize with left and right neighbors.

[0203] The major cost in the implementation of ragged barriers using counters is the need for N counter objects instead of one barrier object. However, the number of counters needed is proportional to the number of threads, not to the problem size. This cost is unlikely to be a practical problem on modern computer systems.

[0204] The present application can be used in multithreaded programming system, with any single or multiprocessor computers. Example multithreaded programming systems include Windows NT, UNIX/Pthreads and Java.

[0205] Other examples than those discussed above can of course be used. While the three examples discussed above are computationally intensive, other computationally intensive systems include volume rendering, terrain masking, threat analysis, protein folding, and molecular dynamics simulation.

[0206] As can be seen from the above, the system of the present application is highly advantageous and produces significant advantages.

[0207] Although only a few embodiments have been disclosed in detail above, other modifications are possible, and would understood by those having ordinary skill in the art reading the application. For example, although this application has only described certain operating systems which capable of handling multiple threads, it should be understood that other operating systems could be provided. A non-exhaustive list of operating systems includes Windows NT, Windows 2000, Java, UNIX, Linux or any other type system.

[0208] All such modifications are intended to be encompassed within the following claims, in which: