User Control of Multiple Memory Heaps

The present invention relates to improved memory management in a computing environment by permitting the dynamic allocation of multiple memory heaps as required during program execution, and providing the user with direct control over all memory managed or used by each heap.

The heap is that area in memory that is generally allocated at the commencement of program execution for dynamically constructing data objects used for executing the application. Traditionally, the heap remains static during program execution, and is destroyed at the completion of the application.

While early programming systems allocated only one heap during program execution, the intermingling of 16-bit and 32-bit code within applications has resulted in more recent operating systems, such as IBM's OS/2.TM. operating system, providing support for allocating two heaps. Generally, these two heaps are referred to as the regular heap for receiving only 32-bit coded data objects, and tiled heap that is usable either for 16 or 32-bit code.

The so-called regular heap is based on flat or linear pointers (addresses) referenced to a zero-based address. The heap itself, then, can span a very large segment of memory (up to four gigabytes in size) with pointers based on the same flat or zero address base.

Tiled memory, refers to the fact that the pointers are tiled- their base linear addresses are set to be multiples of 64K because of a limitation in 16-bit code precluding the referencing of data objects spanning 64K boundaries.

A further type of heap in recent development is a shared memory heap (whether in regular or tiled format) that would permit data objects to be accessed directly by several applications at once, or by multiple instances of a single application. Clearly, the shared memory concept reduces processing time by avoiding construction of the same data objects in multiple applications and by simplifying the problems of synchronization and coherence where one application modifies the data.

Multiple heaps (that is, even more than two), have been provided in very recent products such as "SmartHeap" of MicroQuill Software Publishing. The value of any double or multiple heap system includes better data locality, the ability to free one heap in a single efficient operation, while retaining data in other heaps, and less contention in multithreaded applications if each thread has its own heap, and this has been accomplished in the prior art through system calls controlled by the runtime library as described below.

As illustrated schematically in FIG. 1, an application call to allocate/deallocate additional heap memory (tiled or regular) is issued to the runtime library and is processed by a memory manager located in the library by issuing system calls to the operating system to acquire or release memory for the heap. Under the traditional approach, allocation of additional heap memory is controlled entirely by the runtime library, not by the user.

In "A List Box Replacement"--Benge, Mark A. and Smith, Matt, OS/2 Developer, Jan./Feb. 1994, pages 66 to 70, a modification of the traditional memory management system is described that is claimed to permit the user somewhat more flexibility in allocating memory from different heaps and to guarantee minimal memory configurations when required by the user. A new HeapAlloc call initiates the heap by requesting a block of memory using a DosAllocMem system call. The address returned by-DosAllocMem is the starting chunk of memory for the heap and is invariably used as the heap handle. When this value is passed to HeapMalloc, HeapCalloc, HeapRealloc, and HeapFree routines, each routine will correctly select the starting chunk and determine from which memory chain to carve or free the memory. To discard the heap, a new HeapRelease call is used that releases all the memory of the heap, (not individual memory blocks) back to the system.

Attempts have been made to deal with memory management without having to resort to expensive system calls at each instance. For example, in U.S. Pat. No. 5,339,411 Heaton, a method is disclosed in which the memory is allocated into a number of memory blocks of varying size at the time the memory space is initialized (usually during initialization of the application program). Then, in response to a memory allocation request during execution of the application, a routine is initiated that scans the stored block constants to locate a memory block of the appropriate size to meet the specifications of the allocation request. A mechanism is also provided that permits encroachment into a second adjacent memory block where no one memory block is large enough to meet the specifications of the memory allocation request.

Other references, such as U.S. Pat. Nos. 5,088,036--Ellis, et. al. and 5,136,706--Courts propose methods for more efficient memory management through dividing the available memory space into regions with specific attributes so that only active objects are located in active regions, to reduce paging and other accessing operations.

Improved garbage collection, such as in the methods proposed in U.S. Pat. Nos. 4,907,151--Bartlett and 5,109,336--Guenther, et al., is another way to maximize memory management efficiency in a predefined memory space.

In addition to the foregoing, U.S. Pat. No. 5,317,706--Pechter provides a hardware solution to increasing available virtual memory spacing in a processing environment through the addition of extended memory circuits. The extended memory is logically located or mapped to a portion of the original virtual address space which has been partitioned into a reduced virtual address space and an extended real memory address space.

However, none of the prior art methods proposed in the patent literature improve upon the traditional method for increasing heap memory allocation during program execution described above, that is through the routine allocation of additional memory instituted through system calls.

In addition, nothing in the prior art suggests a memory management system that actually permits the user to control the heap or type of heap in which the objects are to be constructed and controls the creation (allocation) and destruction (deallocation) of different heaps and different heap types.

It is therefore an object of the present invention to provide user control of multiple heaps in an operating system.

It is also an object of this invention to provide better performance of allocation and freeing operations within a heap, and to provide the user with the ability to free a single heap in one efficient operation while retaining data allocated in other heaps.

A further object is to provide less contention in multithreaded applications, by providing the user with the ability to allocate to each executing thread its own dedicated heap.

Accordingly, the present invention provides a mechanism for user heap management during program execution in an operating system having means for allocating dynamic memory. The mechanism consists of controls located in the executing program for directing heap allocation requests, preferably on data storage media adapted to be executed on a general purpose computer.

Preferably, in one aspect, the control consists of means for locating a block of heap memory of a suitable size for fulfilling the allocation request along with means for storing information in a portion of the heap memory defining parameters of the heap memory.

Preferably, in another aspect, the control consists of means for setting a default heap definition in the runtime library.

The present invention also provides a process for managing heap allocation from a user application executing in an operating system having means for extending heap memory. The process includes the steps of issuing a callback function from the runtime library to the user application for a heap extension of at least the minimum size, determining if a block of memory of at least the minimum size is available, and removing and returning to the user application an object of at least the minimum size from the block of heap memory.

Preferably, the process also includes the steps of initiating a call to the operating system from the user application for heap memory extension on receiving a null determination of available heap memory and inserting heap control data into a block of heap memory allocated by the call to the operating system.

Alternatively, the invention provides a process for directing heap memory use from a user application executing in an operating system having runtime library and at least secondary library. The process includes the steps of issuing a function call from the user application to the runtime library to define a default heap in the runtime library, the issuing of a function call from the user application to the secondary library for construction of a data object on the default heap, and finally issuing a heap allocation request from the secondary library to the runtime library for construction of the data object on the default heap.

Preferably, the above steps may be repeated for every thread in a multithreaded user application.

Embodiments of the invention will now be described in detail in association with accompanying drawings, in which:

FIG. 1 is a simplified block diagram illustrating the processing requests for heap memory allocation from an executing application, according to the method of the prior art;

FIG. 2 is a simplified block diagram illustrating the processing of requests for heap memory allocation from an executing application, according to one aspect of the present invention;

FIG. 3 is a flow diagram illustrating the use of the .sub.-- ucreate function to create a new heap;

FIG. 4 is a flow diagram illustrating the use of the .sub.-- uopen function to open an existing heap for use;

FIG. 5 is a flow diagram illustrating the use of the user exhausted routine during operation of a memory allocation, according to one aspect of the present invention;

FIG. 6 is a flow diagram illustrating the use of the .sub.-- udestroy function to collapse the entire heap, according to one aspect of the present invention;

FIG. 7, on the same page as FIG. 4, is a schematic diagram showing the partitioning of memory allocated in the heap according to the present invention; and

FIG. 8 is a schematic diagram illustrating the setting of a default heap, according to another aspect of the present invention.

Unlike the prior art process illustrated in FIG. 1, in the present invention, heap expansion or contraction requests from the runtime library are made to the application, which can then issue corresponding requests to the operating system as illustrated schematically in FIG. 2. There is no longer a direct communication for this purpose between the runtime library and the operating system, with the result that the user has full control over heap memory allocation.

This is accomplished by locating the heap control data, that include the heap type (tiled, shared, etc.) and semaphores for multithreading in memory supplied by the application, rather than in the runtime library as in the prior art. In the preferred embodiment, the heap control data includes "sanity marks" which are known to those skilled in the art and which are data of a predetermined value, such value being selected with the expectation that it would be unlikely to be found in a memory area chosen at random. In the preferred embodiment, the presence of the sanity marks in the expected location within the control data area of a purported heap is considered evidence that the purported heap is a valid heap. Then, for heap memory allocation, the runtime library must make callbacks to the application rather than system calls to the operating system to grow or shrink the pool of available memory on a heap.

The application thereby controls requests to create, free, allocate, release, exhaust and destroy heaps as illustrated by the event edges in FIG. 2.

Growable and shrinkable heaps are implemented by two callback functions, according to the preferred embodiment of the invention. One function, exhausted, is called during an allocation request that cannot be satisfied by available free heap memory and allows the heap creater to optionally grow the heap with memory of the appropriate type. The second function, release, is called when the heap is being destroyed and allows the heap creator to release all blocks of memory provided to that heap. The heap may also be grown by the user function .sub.-- uaddmem described below.

The user provides an initial starting block for the heap routines to use. If a particular pool cannot satisfy an allocation request, then an exhausted routine provided by the user will be called specifying the minimal amount of memory that the heap requires to satisfy the user allocation. This process will continue until the heap is destroyed and the user release routine is called for every block of memory that was inserted into the heap by the user.

The initial block of memory of the desired type is supplied by the user using the .sub.-- ucreate function, to be described in detail below. In addition to this, the following functions are defined in the runtime library in the preferred embodiment of the invention in order to implement callbacks from the runtime library:

As illustrated in FIG. 3 and in the C code fragment above, in a preferred embodiment, the invention provides .sub.-- ucreate which creates a user heap. The function operates similarly whether or not another heap already exists. In the first step (block 1) the application initiates the .sub.-- ucreate call and defines an initial chunk of memory needed, size, flags, exhausted function and release function. The .sub.-- ucreate function returns the handle heap.sub.-- t identifying the heap. The parameters 5 passed by the user include pool, which is the memory address of the initial block available for use, of the size init.sub.-- sz bytes, which must be at least .sub.-- HEAP.sub.-- MIN.sub.-- SIZE (block 2). If the initial size init.sub.-- sz of the block is smaller than HEAP.sub.-- MIN.sub.-- SIZE, the system returns a NULL (block 3). If the initial size is at least equal to .sub.-- HEAP.sub.-- MIN.sub.-- SIZE bytes, the runtime will mark the first .sub.-- HEAP.sub.-- MIN.sub.-- SIZE bytes as a reserved area for the heap which will be used to store information relative to the heap as described below with respect to the heap structure (block 4). NULL will also be returned if any other errors occur on the call.

Sanity markers are placed at the start and end of the reserved area (block 5). A pointer to the release function is assigned with the value passed (block 6), and a pointer to the exhausted function is assigned with the value passed (block 7). A pointer to the list of all the chunks of memory inserted into the heap is cleared (block 8), all these pointers in blocks 6, 7 and 8 being in the reserved area. A flag in the reserved area is set if the heap will be using shared memory, and is cleared if the heap will not be using shared memory (blocks 9, 10, and 11).

A structure containing all the necessary information for the heap management routines is initialized (block 12). This structure may contain the start of the free object list, a pointer to the used list, the minimum and maximum addresses inserted into the heap, and statistics indicating the amount of used memory. This is equivalent to placing all the global variables specific to the single heap implementation in a structure within this reserved area.

In the case of a multithreaded environment, a semaphore specific to this particular heap is created (block 13 and 14) via a system call, and the structure for it is also stored with the reserved area.

The remaining pool that was passed in, less the area marked as reserved, is inserted into the heap used to manage the memory (block 15 and 16). The operation of insertion onto the heap is specific to the base memory management algorithm used.

A pointer to the start of the reserved area is returned to the user as a heap handle to the newly created heap (block 17). The user will reference this value as a heap handle in future heap specific calls.

Invoid* (*exhausted) (size.sub.-- t *sz, int *clean), exhausted is a pointer to a callback function that will be issued by the runtime library if it has insufficient memory available in the heap to satisfy an allocation request from the application. The sz points to an unsigned integer variable initialized to the minimal amount of memory that this routine must provide back to the heap routines. The returned value is a pointer to a chunk of memory supplied by the application to extend the heap, or is a null if the application declines to extend the heap. The user has the option to return a larger block than requested by modifying the size variable, *sz.

The clean parameter will point to an integer variable that must be set by the exhausted function to the value .sub.-- BLOCK.sub.-- CLEAN to indicate that the block is all zeros or to |.sub.-- BLOCK.sub.-- CLEAN to indicate otherwise.

void (*release) (void *block, size.sub.-- t sz), the block will point to a memory block that was provided by the user by the exhausted routine or .sub.-- uaddmem.

This callback routine will be called from .sub.-- udestroy for a user created (or assigned) memory pool that requires blocks to be returned back to the operating system.

This function allows the current process to use the heap, and must be called in all processes where the heap will be accessed implicitly and explicitly. If successful, 0 is returned.

In operation, the .sub.-- uopen function called by the user application (FIG. 4, block 20) accepts a heap handle that points to the heap internal structure. The heap internal structure sanity marks are checked within the reserved area and an error reported if they are not the predetermined values (blocks 22, 24).

Knowing that a valid internal heap structure exists, the system specific structure for the semaphore is passed on to the operating system to grant access to this semaphore to the current process that initiated the call (block 26).

The return code received by the operating system is then returned to the user indicating the opening of the semaphore was performed successfully (block 28).

int .sub.-- uclose(Heap.sub.-- t heap)

where: heap is a valid heap handle identifying a heap that will be closed for usage.

This function notifies the runtime that the executing application will no longer require the usage of this heap. If successful, 0 is returned.

The operation of this routine although opposite in purpose to the operation of the .sub.-- uopen function, follows the same steps as those illustrated in FIG. 4. When called by the application (block 20), .sub.-- uclose will accept a heap handle that points to the heap internal structure. The heap internal structure sanity marks will be checked within the reserved area and an error reported if it is not a predetermined value (block 22, 24).

Knowing that a valid internal heap structure exists, the system specific structure for the semaphore will be passed on to the operating system to remove access of this semaphore by the current process that initiated the call (block 26).

The return code received from the operating system is then returned to the user indicating if the opening of the semaphore was performed successfully (block 28).

These functions will behave differently from the prior art single-heap versions malloc and calloc respectively in that allocations will be made from the supplied heap. NULL will be returned if the request cannot be satisfied by the exhausted function.

.sub.-- umalloc allocates memory from the supplied heap area of the size specified, as illustrated in FIG. 5. The routine is called by the application, and accepts a heap handle that points to the heap internal structure (block 30). The heap internal structure sanity marks will be checked within the reserved area and an error reported if they are not predetermined values (blocks 34, 36).

The heap is serialized by methods known to those skilled in the art, for example by locking a system semaphore stored in the heap reserved area (block 38).

Using this heap handle, a call to the memory allocation algorithm is made for a request of the provided size (block 38). If there is not enough memory available in the specified heap to satisfy the request (block 40), then the user's exhausted callback function (specified in .sub.-- ucreate) is called with the size required (block 42). If the callback returns a NULL indicating that it could not satisfy the request, then NULL is returned back to the original caller (block 44, 46). Otherwise, the new object inserted is added to the linked list maintained within the heap reserved area for inserted user objects.

The new block returned by the callback routine is inserted into the heap (block 48). The object of the required size is removed from the heap (block 50) and the size and handle are stored in the first eight bytes of the object (block 52). The user part of the object is returned to the application (block 54).

In order to recognize the heap where an object was allocated from during deallocation, an internal area within the object returned must be set to mark the size and heap handle used as illustrated in FIG. 7. The portions of the heap allocated to size 70 and heap handle 72 making up the internal information of the allocated object, are returned by regular.sub.-- umalloc function to the user. The remaining sz bytes constitutes the user area 74 in the object. The address returned to the user is the address of the user area 74 within the object.

The heap is deserialized by methods known to those skilled in the art, for example by unlocking a system semaphore stored in the heap reserved area.

The .sub.-- ucalloc function is similar to that of .sub.-- umalloc except the memory area is cleared with the null byte on return.

The routines will determine the heap used during the initial allocation, and then perform the given operation based on that heap. If NULL is passed to realloc, then the default heap will be used for the allocation. The return values and expected arguments will be the same as in free and realloc calls known to those skilled in the art.

This user function can be used to extend a heap by adding a chunk of memory to it. The parameter block here points to the chunk of memory that the application is providing to extend the heap.

int .sub.-- udestroy(Heap.sub.-- t heap, int force)

where: heap is a handle identifying a valid heap that will be destroyed.

.sub.-- FORCE forces destruction if allocated objects still exist

This function collapses the entire heap specified by calling the release function specified in .sub.-- ucreate for every chunk of memory supplied by the user via .sub.-- uaddmem, or via the exhausted callback function. Usage of the heap after .sub.-- udestroy results in undefined behaviour. If successful, 0 is returned.

The operation of this routine is illustrated in FIG. 6. The .sub.-- udestroy function call issued by the application (block 61) accepts a heap handle that points to the heap internal structure. The heap internal structure sanity marks are checked within the reserved area (block 62) and an error reported if it is not a predetermined value (block 63).

Access to the heap is serialized by locking a system semaphore stored in the heap reserved area (block 64).

Next, the linked list of user-inserted extensions of the reserved heap area is scanned (block 65). Each of the extensions was inserted into the linked list following a call to the user's exhausted routine when an allocation request could not be satisfied with the contents of the heap at that time.

For each of these extensions within the linked list, the user's release routine is called so they can properly return the memory extensions from wherever they obtained it (block 66). The call to the callback function returns the memory pointer originally provided as well as the size.

Access to the heap is deserialized by unlocking a system semaphore stored in the heap reserved area (block 67), and the semaphore specific to this heap is returned back to the operating system (block 68).

The remaining original chunk of memory supplied by the user during the .sub.-- ucreate, containing heap control data, is cleared to prevent further allocations from this heap (block 69).

The following example illustrates how the user heap routines are used to implement a separate allocation pool. In this example, a user heap is created with the minimum startup size. The example then allocates an object of 1000 bytes using the .sub.-- umalloc function which causes the user get.sub.-- fn to be called since the heap is empty at this point. The get.sub.-- fn is called with a length of 1000 bytes plus some overhead which it modifies to 65536 since it is a much more practical increment to get from the operating system.

The get.sub.-- fn callback routine rounds up the requested length to a multiple of the system page size and attempts to obtain a block of memory of this size directly from the operating system.

To collapse the heap, a call to .sub.-- udestroy is made which would call the user's free.sub.-- fn release function for every chunk that was provided, and then clear the original starting chunk provided during heap creation.

The following free.sub.-- fn callback routine will be called when the memory routines deem that a block of memory that was provided earlier is no longer needed and can be returned back to the operating system.

In the above examples, GetMemFromSystem and ReturnMemToSystem indicate standard system calls to obtain and release memory.

For compiler and runtime library writers that wish to implement a multiple heap memory manager on top of a single heap implementation, the goal is to convert a single heap implementation into a multiple heap version that can process all memory types seamlessly. Therefore, the base memory allocator must be modified as follows:

Create a heap structure to store all the global variables related to the memory routines. This information will be stored in the heap reserved area.

Declare a heap structure internally to contain the following;

Sanity value

Base memory allocator information

A function pointer to the exhausted routine

A function pointer to the release routine

A flag denoting the type of memory it is handling. This is only necessary if the base memory allocator needs to process different types of memory differently. For example, tiled memory is required in a segmented architecture to ensure a block of memory does not cross a segment boundary.

A pointer to maintain a linked list of blocks of memory that the user's callback routine returned during an allocation request.

Modify the routine that acquired memory from the operating system to use the exhausted callback routine specific for the heap call.

Modify the routine that returned memory to the operating system on heap destruction or termination to use the release callback routine specific for the heap call.

If the environment supports threads, then support should be added to synchronize code at the heap level rather than the function level. This will allow calls in separate threads to the same heap to execute concurrently.

Modify the base memory allocator to encode the heap handle used for the allocation within the returned object.

Modify the free and realloc routines to determine the heap used for the object to be released and perform the necessary operation on that specific heap only.

Define the following macros:

.sub.-- BLOCK.sub.-- CLEAN--Specify if inserted memory into the heap is all zeroes.

.sub.-- HEAP.sub.-- MIN.sub.-- SIZE--Minimum memory size required to create a heap. This value must be at least equal to the size of the heap structure previously declared in the runtime library.

A further aspect of the present invention is illustrated in the flow diagram of FIG. 8. Multiple dynamic memory heaps give users flexibility in managing their storage use, and allow them to selectively allocate data in different types of memory. For example, an application may use one private heap of regular memory, and a second heap built with shared memory that is accessible to other applications and processes.

However, a severe problem with allocations occurs in vendor supplied object code only (OCO) libraries that were not constructed with multiple heaps in mind, since the user is not in a position to make the source level modifications necessary to use multiple heaps.

Accordingly, this aspect of the invention is directed to providing a mechanism whereby users can specify a heap to temporarily replace the standard default heap defined in the runtime library. This specified default heap will be used for all allocation requests that do not explicitly specify a heap, and allows for transparent migration from single to multiple heap strategies, particularly for using shared memory heaps in unmodified OCO libraries.

As illustrated in FIG. 8, the application 80 first calls down to the runtime library 82 to set the default 84 to the required heap (88a, 88b or 88c).

The application 80 then calls the vendor library 86, which in turn issues a malloc call that will use the default 84 set previously set by the application.

New routines are written with the old names of the allocation routines that determine which heap is considered the default and call the multiple heap version with it as an argument.

This routine must be called within each thread that the default allocation routines are to be implicitly acquired from a private heap. Internally, all that is done is change an internal variable within the runtime library that denotes the current default heap to the value passed in by the user. The return value is the previous default heap prior to replacement.

This allocation routine will acquire storage size sz from the current active default heap. The active default heap is set previously within the application by the user making a .sub.-- udefault call. If such call was not made then the default runtime heap that the library determines will be used.

This function is similar to that of malloc except the memory area is cleared with the null byte on return.

This function reallocates the object pointed to by the p parameter, such that it will have size newsize. The object is reallocated in the same heap, except if pointer p is null, it will be reallocated in the current default heap.

In a multithreaded environment, the default heap handle set by the .sub.-- udefault call can be set on a per thread basis, and would be stored by the runtime library separately for each thread.

The invention can be used with user libraries that are built to use only one heap, by defining the secondary library function with redefines the default shared memory heap prior to the calling of the secondary library function. Following the completion of the secondary library function, the default heap is redefined to the previous heap, and thus the secondary library function does not interfere with the operations performed with any other thread. An example of such function is below:

The library function and all functions it calls will use shared memory heap to satisfy all non-heap-specific allocation requests in a way that it does not interfere with later operations performed in the current thread.

The foregoing invention has been described in association with specific embodiments. However, modifications to those skilled in the art are intended to be covered by the appended claims.