Method and Apparatus for Transporting Interface Definition Language-Defined Data Structures Between Heterogeneous Systems

The following related U.S. applications are hereby incorporated by reference: U.S. application Ser. No. 08/680,270 entitled "Method and Apparatus for Describing an Interface Definition Language-Defined Interface, Operation, and Data Type" by A. Schofield, filed Jul. 11, 1996; U.S. application Ser. No. 08/678,681 entitled "Method and Apparatus Using Parameterized Vectors For Converting Interface Definition Language-Defined Data Structures into a Transport and Platform Independent Format" by A. Schofield, filed Jul. 11, 1996; U.S. application Ser. No. 08/678,298 entitled "Data Structure Representing An Interface Definition Language Source File" by A. Schofield, filed Jul. 11, 1996; U.S. application Ser. No. 08/678,295 entitled "Method and Apparatus for Performing Distributed Object Calls" by A. Schofield filed Jul. 11, 1996; U.S. application Ser. No. 08/680,202 entitled "Method and Apparatus for Asynchronously Calling and Implementing Objects" by A. Schofield, filed Jul. 11, 1996; U.S. application Ser. No. 08/680,266 entitled "Method and Apparatus for Performing Distributed Object Calls using Proxies and Memory Allocation" by A. Schofield filed Jul. 11, 1996.

1. Field of the Invention

The present invention relates to a method and apparatus for transporting data structures described in the Object Management Group's Interface Definition Language between heterogeneous platforms. More particularly, the present invention utilizes functions for removing the alignment from data structures and storing the data structures in a predetermined format for transport to a file or across heterogeneous platforms.

2. Background

Distributed object computing combines the concepts of distributed computing and object-oriented computing. Distributed computing consists of two or more pieces of software sharing information with each other. These two pieces of software could be running on the same computer or on different computers connected to a common network. Most distributed computing is based on a client/server mode. With the client/server model, two major types of software are utilized: client software, which requests the information or service, and server software, which provides the information or service.

Object-oriented computing is based upon the object model where pieces of code called "objects"--often abstracted from real objects in the real world--own data (called "attributes" in object-oriented programming parlance) and provide services through methods (also known as "operations" or "member functions"). The data and methods contained in an object may be "public" or "private." Public data may be altered by any other object. Most data, however, is private and accessible only to methods owned by the object. Typically, the methods operate on the private data contained in the object.

A collection of similar objects make up an interface (or "class" in C++ parlance). An interface specifies the methods and types of data contained in all objects of the interface. Objects are then created ("instantiated") based upon that interface. Each object contains data specific to that object. Each specific object is identified within a distributed object system by a unique identifier called an object reference.

In a distributed object system, a client sends a request (or "object call") containing an indication of the operation for the server to perform, the object reference, and a mechanism to return "exception information" (unexpected occurrences) about the success or failure of a request. The server receives the request and, if possible, carries out the request and returns the appropriate exception information. An object request broker ("ORB") provides a communication hub for all objects in the system passing the request to the server and returning the reply to the client.

On the client side, the ORB handles requests for the invocation of a method and the related selection of servers and methods. When an application sends a request to the ORB for a method to be performed on an object, the ORB validates the arguments contained in the request against the interface and dispatches the request to the server, starting it if necessary. On the server side, the ORB receives such requests, unmarshals the arguments, sets up the context state as needed, invokes the method dispatcher, marshals the output arguments, and returns the results to the client, thereby completing the object invocation.

Both client and server must have information about the available objects and methods that can be performed. Through the hiding of private data ("encapsulation" in object-oriented parlance), the client does not need to know how the request will be carried out by the server. Nevertheless, both client and server must have access to common interface definitions to enable communication therebetween. Currently, the standard language for distributed object computing is the Object Management Group's ("OMG") Interface Definition Language ("IDL").

A distributed object system developer defines the system's available interfaces in IDL. An interface includes one or more operations that can be performed on objects of that interface. Each operation may receive one or more parameters. Each parameter is of a particular IDL data type.

IDL includes several data types. Integers are represented through long and short, signed and unsigned integer data types. OMG IDL floating point types are float and double. The float type represents IEEE single-precision floating point numbers while the double type represents the IEEE double-precision floating point numbers. OMG IDL defines a char data type consisting of 8-bit quantities. The boolean data type is used to represent true/false values. The "any" type permits the specification of values that can express any OMG IDL type. Complex types, such as structures, unions, and templates are also represented.

IDL is designed to be used in distributed object systems implementing OMG's Common Object Request Broker Architecture ("CORBA"). In a typical CORBA system, interface definitions are written in an IDL-defined source file (also known as a "translation unit"). The source file is compiled by an IDL compiler that maps the source file to a specific programming language. The IDL compiler generates programming-language-specific files, including client stub files, header files, and server skeleton files. Client stub files are then compiled and linked into client applications and are used to make requests. Header files are linked into client and server applications and are used to define data types. Server skeleton files are linked into server applications and are used to map client operations on objects (requests) to methods in a server implementation.

When object calls are made, data structures are transported from one computer system to another (client-to-server and server-to-client). Such object calls may occur between identical systems, but are likely to be made across heterogeneous platforms using different operating systems, programming languages, and compilers. Once IDL source files have been compiled and mapped to a particular programming language, each independent compiler vendor will align data structures on the stack in a particular manner. Accordingly, both the client and the server systems may align data structures differently. Moreover, both the client systems may align parameters within a structure differently. If the client and server application do not understand each other's method of alignment, the transported data structure will become garbled and an error will occur.

In addition, the hardware utilized by the client and server may be different. For example, one computer may include a CPU that requires so-called "BIGendian" integer representation where the most significant byte is listed first. The other computer may include a CPU that requires "LITTLEendian" representation where the least significant byte is listed first. Data structures cannot be passed effectively between these two machines without reciprocal knowledge of each system.

Moreover, if clients and servers are required to have detailed knowledge of each other, a primary goal of object-oriented computing--encapsulation--is lost. Both the client and the server applications must write detailed alignment functions to ensure compatibility during object calls. This extra coding work makes distributed object computing inefficient.

Accordingly, there is a need for a method for converting IDL-defined data structures into a platform-independent format, such that converted data types can be transported effectively across heterogeneous systems.

In addition, there is a need for a method that reduces the amount of code and execution time required by client and server applications to implement object calls.

The present invention is directed to a method that satisfies the need to convert IDL-defined data structures into a platform-independent format, such that converted data structures can be transported across a network. The need for reducing the amount of code and execution time required by client and server applications is also satisfied. Specifically, both systems have a description of a data structure defined in ASCII string format called the Compact IDL Notation ("CIN"). The CIN is converted to a "prepared CIN" format containing additional information about the offset and size of the data structure. The prepared CIN is used to extract data structure and store the data into an output buffer based upon the size and offset of the data structure. The data is densely packed into the output buffer without any alignment padding fields. The output buffer is transferred to an input buffer of the second system. The second computer extracts the data from its input buffer. The data is then converted to the format of the second system and realigned in a data structure based upon the prepared CIN.

The method is performed through the use of calls to implementation libraries made by the client and server systems. The Implementation Libraries include generic functions located in a run-time library. By calling generic functions, the client and server applications can easily make object calls without specific knowledge of each other and without requiring additional code.

A more complete understanding of the conversion method will be afforded to those skilled in the art, as well as a realization of additional advantages and objects thereof, by a consideration of the following detailed description of the preferred embodiment. Reference will be made to the appended sheets of drawings which will first be described briefly.

FIG. 1 is a diagram of a distributed computing environment using the method or apparatus of the present invention.

FIGS. 2A & 2B is a diagram of the Common Execution Environment infrastructure.

FIG. 3 is a diagram of conventional IDL source file compilation and linking.

FIG. 4 is a diagram of IDL source file compilation and linking utilizing the method of the present invention.

FIGS. 5(a) and 5(b) show an IDL-defined data structure, generated CIN description, and generated array of op.sub.-- tag structures.

FIG. 6 is a flow chart describing a first preferred embodiment of the client side of the method of the present invention.

FIG. 7 is a flow chart describing a first preferred embodiment of the server side of the method of the present invention.

Reference will now be made in detail to the preferred embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.

As illustrated in FIG. 1, the method of the present invention is designed for use in a distributed (client/server) computing environment 10. The client and server systems are connected by network connections 12, such as internet connections or the connections of a local area network. The server computer 11 communicates over a bus of I/O channel 20 with an associated storage subsystem 13. The server system 11 includes a CPU 15 and a memory 17 for storing current state information about program execution. A portion of the memory 17 is dedicated to storing the states and variables associated with each function of the program which is currently executing on the client computer. The client computer 21 similarly includes a CPU 27 and associated memory 23, and an input device 29, such as a keyboard or a mouse and a display device 33, such as a video display terminal ("VDT"). The client CPU communicates over a bus or I/O channel 40 with a disk storage subsystem 33 and via I/O channel 41 with the keyboard 29, VDT 33 and mouse 31. Both computers are capable of reading various types of media, including floppy disks and CD-ROMs.

The client/server model as shown in FIG. 1 is merely demonstrative of a typical client/server system. Within the context of the present invention, the "client" is an application that requests services while the "server" is an application that implements the requested service. Indeed, both the client and server application may reside on the same computer and within a common capsule, as discussed below. The client and server application may also reside on separate computers using different operating systems.

The method and apparatus of the present invention may be utilized within any distributed computing environment. In a preferred embodiment, the Common Execution Environment ("CEE"), which is a component of the TANDEM Message Switching Facility ("MSF") Architecture is utilized. The CEE activates and deactivates objects and is used to pass messages between client and server applications loaded in CEE capsules. The CEE may be stored in the memory of a single machine. The CEE and client and server applications may, however, be loaded on multiple machines across a network as shown in FIG. 1. The client-side CEE 75 is stored in the client memory 27. The server-side CEE 80 is stored in server memory 17.

The CEE uses a "capsule" infrastructure. A capsule encapsulates memory space and execution stream. A capsule may be implemented differently on different systems depending upon the operating system. For instance, on certain systems, a capsule may be implemented as a process. On other systems, the capsule may be implemented as a thread. Moreover, client and server applications may be configured within different capsules contained on different machines as shown in FIG. 1. Alternatively, the different capsules may be configured as shown in FIG. 2. FIG. 2a shows a client application 77 loaded in a single capsule 81 and a server application 87 may be loaded in a separate capsule 85. Both capsules, however, are stored on the same machine 21. Both the client and server applications may also be loaded within a single capsule 81 on the same machine 21 as shown in FIG. 2b. As stated above, the method of the present invention will be described with reference to the multiple capsule, multiple machine case. Accordingly, the client 12 and server machine 11 include a client-side CEE 75 and a server-side CEE 85 loaded in their respective memories.

FIG. 3 shows a CEE capsule 70 contained, for example, in a client computer memory 27 (not shown) that includes the CEE 75 and certain of the core CEE components and implementations of objects contained within Implementation Libraries 71. The Implementation Libraries 71 include the client application 79 (or the server application in the case of the server capsule) and client stubs 77 (or server stubs) generated from the IDL specification of the object's interface, as described below. The Implementation Libraries 71 and the CEE 75 interact through the down-calling of dynamically-accessible routines supplied by the CEE and the up-calling of routines contained in the Implementation Library. The CEE 75 can also receive object calls 82 from other capsules within the same machine and requests 84 from other CEE's. The client-side CEE 75 and the server-side CEE 85 may communicate using any known networking protocol. The client and server CEE's includes numerous libraries of routines that can be down-called from client and server applications. The Presentation Conversion Utilities ("PCU") 89 is a library of routines utilized in the method of the present invention.

Objects implemented in a CEE capsule may be configured or dynamic. Configured objects have their implementation details stored in a repository (such as the MSF Warehouse 85) or in initialization scripts. Given a request for a specific object reference, the CEE 75 starts the appropriate capsule based on this configuration data. The capsule uses the configuration data to determine which implementation library to load and which object initialization routine to call. The object initialization routine then creates the object. Dynamic objects are created and destroyed dynamically within the same capsule. Dynamic objects lack repository-stored or scripted configuration information.

The following paragraphs describe a system-level view of how the Implementation Libraries interact with the CEE 75. The CEE 75 implements requests to activate and deactivate objects within a capsule. In addition, the CEE facilitates inter-capsule object calls 82 as well as requests from other CEE's 84, as discussed above. Object activation requests arise when an object call from a client or server application must be satisfied. To activate an object, the CEE 75 loads the appropriate Implementation Library (if not already loaded) containing the object's methods and then calls a configured object initialization routine. The initialization routine specifies which interface the Implementation Libraries support and registers the entry points of the object's methods to be called by the CEE at a later time.

When the client and server systems start, both the client-side and server-side CEE's run their own initialization. This initialization tells client and server CEE's where to locate the various Implementation Libraries. Once located by the CEE, the initialization routines in the client and server applications are called. The initialization routines contained in the client and server applications must first carry out any required application-specific initialization. Next, both the client and server initialization routines call a stub function which, in turn, down-calls a CEE function (contained in a dynamic library as stated above) called CEE.sub.-- INTERFACE.sub.-- CREATE to specify the object's interface. An interface may be specified for each object. The interface description is normally generated from an IDL description of the interface, as discussed below. CEE.sub.-- INTERFACE.sub.-- CREATE creates an interface and returns an "interface handle" to the newly created interface. The handle is a unique identifier that specifies the interface. The server application initialization routine then uses the interface handle to down-call CEE.sub.-- IMPLEMENTATION.sub.-- CREATE. CEE.sub.-- IMPLEMENTATION.sub.-- CREATE creates an implementation description that can be used by one or more objects. CEE.sub.-- IMPLEMENTATION.sub.-- CREATE returns an "implementation handle" that is a unique identifier specifying the implementation for each operation in the interface. Finally, the server application initialization routine uses the implementation handle to call a stub function which down-calls CEE.sub.-- SET.sub.-- METHOD. CEE.sub.-- SET.sub.-- METHOD specifies the actual addresses of specific method routines of the implementation as contained in the server application. The CEE then has sufficient information to connect object calls in the client application to specific methods in the server application.

FIG. 4 shows how IDL source files are compiled and linked into client and server applications that will utilize the method and apparatus of the present invention. First, an IDL source file 101 is prepared containing IDL interface definitions. An IDL compiler 103 compiles the source file 101. The IDL compiler 103 parses the code 101 to produce an intermediate Pickled IDL file ("PIF") file 105 for storage of the original source file. A code generator 111 then parses the PIF file. The generation of a PIF file is described in U.S. application Ser. No. 08/678,298 entitled "Data Structure Representing An Interface Definition Language Source File" by A. Schofield, filed Jul. 11, 1996. Preferably, the IDL compiler and code generator are combined to generate code. The code generator 111 generates files in the language of the client and server applications. If the client and server applications are in different languages, different code generators are used. Alternatively, the code generator 111 and IDL compiler 103 may be combined in a single application to produce language-specific code. The code generator 111 produces a client stub file 77 containing client stub functions and a server stub file 87 containing definitions for object implementations. The client stub file 77 and the server stub file 87 are compiled by programming language-specific compilers 121, 123 to produce compiled client stub object code and compiled server stub object code. Similarly, a client application 79 and a server application 89 are compiled by programming-language-specific compilers to produce compiled client application object code and compiled server application object code. The client application 79 and the server application 89 also include a header file 119 generated by the code generator 111. The header file 119 contains common definitions and declarations. Finally, a language compiler 121 links the client application object code and the client stub object code to produce an implementation library 71. Similarly, a second language compiler 123 links the server application object code server stub object code to produce another implementation library 81.

In addition, the header file 119, the client stub file 115, and the server stub file 117 include a compact version of each IDL-defined data structure termed Compact IDL Notation ("CIN"). CIN is an ASCII (or other character-based) representation of an IDL data structure utilizing a special notation. The CIN descriptor is contained in the header file 119 which is included by both the client application 123 and the server application 123. The descriptor is also contained in the client and server stub files as well. Since both the client and server applications 121, 123 have access to the CIN, the generic functionality provided by the PCU library 130 can be used by heterogeneous communicants. The creation of CIN by the code generator is described in detail in U.S. application Ser. No. 08/680,270 entitled "Method and Apparatus for Describing an Interface Definition Language-Defined Interface, Operation, and Data Type" by A. Schofield, filed Jul. 11, 1996.

Now, the method of the present invention will be described. The method and apparatus of the present invention is implemented by using a group of generic functions or routines 130 that are available to the client and server applications at run-time. These routines are intended to be used in conjunction with data structures originally described in IDL. The principal routines are PCU.sub.-- PREPARE, PCU.sub.-- PACK, and PCU.sub.-- UNPACK.

An IDL source file may contain numerous type definitions for various data structures. When the source file is compiled and linked into client and server applications, the data structures are used by the client and server applications to perform object calls. A client application may request that an operation be performed on an object using data contained in a particular data structure. The data included in this data structure will be transported to the server application during the object call. The server application must align the data within the data structure properly in order to effectively implement the operation on an object.

The method of the present invention facilitates the transporting of IDL-defined data structures across heterogeneous systems. FIG. 5a shows a sample data structure 501 written in IDL. The structure, MyStruct, as written in IDL, includes three components: a char data type component, a long data type component, and a boolean data type component. This type definition is contained in an IDL source file 101, for example, along with interface definitions.

A code generator parses the IDL source file and produces a header file containing a CIN description 502. The CIN descriptor contains a series of ASCII characters that succinctly describes the structure without using identifiers (such as the name of the structure). In this example, the b3 characters identify the data structure as an IDL struct type containing three elements. The C indicates an IDL char type. The F character identifies an IDL long type and the B character identifies a boolean data type.

PCU.sub.-- PREPARE converts the CIN description of a data type into a "prepared CIN" form which is more convenient to use at run-time than the CIN description. Prior to utilizing the routines PCU.sub.-- PACK and PCU.sub.-- UNPACK, the CIN description of each data structure contained in the header file 119, as generated by the code generator 111, must be "prepared". PCU.sub.-- PREPARE is called once by both the client and the server. Since the call is a relatively expensive one, a single call to PCU.sub.-- PREPARE during initialization saves valuable system resources. The call is preferably made during an initialization routine of the client and server application. PCU.sub.-- PREPARE is defined in C as follows:

In this function, cinbuf is a pointer to the buffer containing the CIN description of the data structure. The parameter cinlen is the size of the CIN. PCU.sub.-- PREPARE returns a "prepared CIN" that will be stored in the address pointed to by prepbuf. To specify a maximum length for prepbuf, prepbufmaxlen may be set to a particular value. The function also returns prepbuf.sub.-- len, which specifies the size of the prepared CIN contained in prepbuf. A value of NULL may be passed as this parameter if this value is not required. The actual number of bytes that were read from cinbuf is returned in the parameter *cin.sub.-- used. NULL may also be used as this parameter if the value of *cin.sub.-- used is not required.

PCU.sub.-- PREPARE is used to create a prepared CIN, which is a table of op.sub.-- tag data structures that describes the data type, offset, size, and alignment of the CIN-described data structure. PCU.sub.-- PREPARE creates these op.sub.-- tag structures by initially creating a ctx data structure used to pass context to and from each internal function in PCU.sub.-- PREPARE. Using a ctx structure is preferred over passing individual parameters to the various internal functions. The ctx structure is defined in C as follows:

The fields of the ctx structure are as follows. The op pointer points to the current operation in the prepared CIN. This pointer is incremented as PCU.sub.-- PREPARE analyzes each CIN item (as described below). The op.sub.-- end pointer points to the last possible operation in the prepared CIN plus one. The cinptr pointer points to the next byte to be read from the entire CIN string being prepared. The cinend pointer points to the last byte plus one of the CIN string being prepared. The offset, align, and size fields of this ctx structure are output parameters of process.sub.-- cin.sub.-- item (described below) that specify the offset, required alignment, and size of the processed field in the CIN-described data structure. A running count of the number of unbounded sequences and strings encountered in the processed CIN string is contained in the nr.sub.-- unbounded. A running count of the fields using the IDL "any" data type is contained in the nr.sub.-- anys field. The field prev.sub.-- branch points to a union branch operation previously processed. A list of This field is used to build a list of branch operations whose head is contained in the main union operation. The union operation is pointed to by main.sub.-- union.

Once the ctx structure has been created, PCU.sub.-- PREPARE calls PROCESS.sub.-- CIN.sub.-- ITEM for each character in the CIN string, TAKE.sub.-- LONG for each signed long integer in the CIN string, and TAKE.sub.-- ULONG for each unsigned long integer in the CIN string. PROCESS.sub.-- CIN.sub.-- ITEM processes a single item in the CIN string. The ctx structure is passed to PROCESS.sub.-- CIN.sub.-- ITEM. PROCESS.sub.-- CIN.sub.-- ITEM can be implemented in many ways. Preferably, the function uses a C-language "switch" statement containing a "case" for each possible character in a CIN string. In addition, a case statement may be used to recursively call itself to handle complex structures such as a sequence of struct types or a union of unions.

TAKE.sub.-- LONG and TAKE.sub.-- ULONG are used in conjunction with particular data types that are followed by numerals (number of array dimensions, etc . . . ). TAKE.sub.-- LONG extracts a signed long integer from the CIN buffer and returns the value to PCU.sub.-- PREPARE. TAKE.sub.-- ULONG extracts an unsigned long integer from the CIN buffer and returns the value to PCU.sub.-- PREPARE. These values are used by PCU.sub.-- PREPARE to create the table of op.sub.-- tag data structures.

For each call, PROCESS.sub.-- CIN.sub.-- ITEM modifies the ctx data structure. First, PROCESS.sub.-- CIN.sub.-- ITEM increments the op pointer to ensure that the other fields of the structure correspond to the proper CIN item. In addition, the size, align, and offset fields of the ctx structure are changed. The alignment for each data type is determined based upon the following alignment rules. Base data types are aligned to their size. Thus, a short data type has two-byte alignment, a long has a four-byte alignment, etc . . . . Struct types and union types have the same alignment as the contained field with the highest alignment requirement. Nevertheless, struct and union types, preferably, have an alignment requirement of at least two bytes. Finally, struct and union types are preferably padded to a multiple of their alignment requirement.

When each call to PROCESS.sub.-- CIN.sub.-- ITEM returns, PCU.sub.-- PREPARE creates an op.sub.-- tag data structure based upon the modified ctx structure. An array of these op.sub.-- tag structures is then stored in the prepared CIN buffer, prepbuf, after calling PCU.sub.-- PREPARE. The op.sub.-- tag structure is a linear structure that can easily be manipulated by other functions. The structure, op.sub.-- tag, is defined as follows:

The type parameter indicates the IDL data type of the data structure. The type.sub.-- def type definition is an enumeration of all of the possible data types. The offset parameter is the offset of the component data structure from the start of the containing structure or union if the data structure is part of a structure or union. The alignment required by the data type (1, 2, 4, or 8 bytes) is specified by the align parameter. The size parameter indicates the size of the data structure in bytes including rounding. The nr.sub.-- elements parameter is used for different purposes. For an array, the parameter indicates the total number of elements for all dimensions. For sequences, the parameter indicates the maximum number of occurrences. For strings, the parameter specifies the maximum size excluding zero termination. For structures, it indicates the number of primary fields in the structure. For unions, it indicates the number of fields in the union. The branch.sub.-- label and is.sub.-- default.sub.-- branch parameters are for union branches only. The branch.sub.-- label parameter contains the case label value that was specified in the IDL specification of the union, while the is.sub.-- default.sub.-- branch parameter is true if the entry describes the default union branch. The is.sub.-- simple parameter is a boolean value that is true if the data structure is of an IDL base data type and is false if the data structure is a compound type. The *next.sub.-- branch parameter is used for unions and union branches and points to the address of the next branch entry belonging to the union. In the case of a union entry, the parameter points to the first branch. For the last branch, the parameter contains the value NULL. The *union.sub.-- end parameter points to the address of the next entry following the conclusion of the final branch. The *sequence.sub.-- end parameter, used for sequences only, points to the address of the next entry following the sequenced type. The *default.sub.-- branch parameter points to the address of the default branch entry. This is used if none of the branches in the branch list matched the union discriminator. If there is no default, the value of default.sub.-- branch is NULL. The reserve.sub.-- XXX parameter allows fields to be added to the op.sub.-- tag structure without causing errors in existing programs that erroneously assume the size of the prepared CIN.

FIG. 5b shows the generated array of op.sub.-- tag structures for the CIN string 502. The first structure 520 specifies the type, offset, size, alignment, and number of members for the MyStruct structure. The next three op.sub.-- tag structures 530, 540, 550 contain the type, offset, size, and alignment for each field in the MyStruct structure. This array of structures is stored in a buffer, prepbuf, that will be used by PCU.sub.-- PACK and PCU.sub.-- UNPACK to send structured data across a file or to a network.

Once the data structure has been "prepared" and the array of op.sub.-- tag structures is stored in prepbuf, various messages stored in that data structure can be packed into a buffer and transported using PCU.sub.-- PACK. PCU.sub.-- PACK is used to copy a structured data type into an output buffer during transport to a file or across the network. PCU.sub.-- PACK supports all IDL constructs including unions, unbounded sequences/strings and "any" types.

PCU.sub.-- PACK stores the components of a structured data type into an output buffer based upon a specified format. PCU.sub.-- PACK is defined in C as follows:

The first three parameters specify how data is to be packed into the output buffer. These parameters may be caller-defined functions for performing the conversion as provided by the caller. The first parameter, dst.sub.-- integer.sub.-- fmt, specifies the format to be used for short, long and long data types in the output buffer. Examples of possible values for this format are PCU.sub.-- INTEGER.sub.-- BIGENDIAN which specifies an integer representation where the byte significance decreases with increasing address or PCU.sub.-- INTEGER.sub.-- LITTLEENDIAN which specifies an integer representation where the byte significance increases with increasing address. The parameter dst.sub.-- real.sub.-- fmt specifies the format to be used for float and double data types in the output buffer. Sample values for this parameter are PCU.sub.-- REAL.sub.-- IEEE which specifies a floating point number representation using the standard IEEE format or a vendor-specific value, such as PCU.sub.-- REAL.sub.-- T16 which specifies a floating point number representation using the TANDEM T16 format, for example. The third parameter, dst.sub.-- char.sub.-- fmt specifies the format to be used for char and string types in the output buffer. One possible value for this parameter is a character representation using the ISO Latin-1 format, a super-set of ASCII. Anther possible value is EBCDIC, which permits compatibility with IBM.TM. hosts.

The *prepbuf parameter, as stated above, is a pointer to the address containing the prepared CIN description as returned by PCU.sub.-- PREPARE. The *inbuf parameter is a pointer to the address of the structured data to be stored into the output buffer. The *outbuf parameter is a pointer to the address of the output buffer that receives the actual packed data. The maximum number of bytes that can be accommodated by outbuf is contained in the outbuf.sub.-- max len parameter. The number of bytes actually written to outbuf is returned by the outbuf.sub.-- len parameter. A value of NULL may be passed as this parameter if the number of bytes is not needed. If PCU.sub.-- SHORTOUTBUF is returned by the function, then the outbuf.sub.-- len parameter gets the required outbuf size.

Accordingly, to dynamically allocate memory for the output buffer, the client application can call PCU.sub.-- PACK twice. On the first call, outbuf.sub.-- max.sub.-- len is set to zero. PCU.sub.-- PACK will then return PCU.sub.-- SHORTBUF and outbuf.sub.-- len will contain the required output buffer size. The correct amount of memory for the output buffer can then be allocated prior to calling PCU.sub.-- PACK for a second time.

PCU.sub.-- PACK initially creates a ctx structure. This ctx structure provides a similar function as the context structured used by PROCESS.sub.-- CIN.sub.-- ITEM. The structure allows large amounts of context to be shared between PCU.sub.-- PACK and the lower-level routines that are called by PCU.sub.-- PACK.sub.-- ENGINE. This ctx structure is used by the underlying functions to PCU.sub.-- PACK and is defined as follows:

The requested destination format as specified in the call to PCU.sub.-- PACK are passed to the ctx structure. These three fields are needed in case PCU.sub.-- PACK must be called recursively to handle an IDL "any" type. The outptr pointer points to the next byte to be written into the output buffer. Even if the output buffer is full, the pointer continues to be updated. This allows the correct size to be returned to the caller in case of overflow. The caller can then adjust the size of the output buffer. The pointer outbuf.sub.-- end points to the last byte plus one in the output buffer.

PCU.sub.-- PACK calls an internal function, PCU.sub.-- PACK.sub.-- ENGINE. PCU.sub.-- PACK.sub.-- ENGINE receives pointers to lower-level functions that perform the actual packing of data into the output buffer. PCU.sub.-- PACK also receives a pointer to prepbuf, a pointer to the data to be packed (contained in inbuf), and a pointer to the ctx structure created by PCU.sub.-- PACK. PCU.sub.-- PACK.sub.-- ENGINE goes element-by-element through the prepbuf buffer and calls the appropriate lower-level function for the element based upon the type of the element (as specified by the type contained in the op.sub.-- tag structure) and based upon the dst.sub.-- XXX.sub.-- fmt parameter to PCU.sub.-- PACK. PCU.sub.-- PACK.sub.-- ENGINE provides the address to the input buffer containing the structured data, the data type (via a CIN character), the ctx structure address, and the size of the data in the input buffer to pack into the output buffer (as specified by the size field of the op.sub.-- tag structure).

PCU.sub.-- PACK.sub.-- ENGINE calls the appropriate lower-level function based upon the type of data contained in the op.sub.-- tag data structure and the conversion specified on the call to PCU.sub.-- PACK. The lower-level functions are known, lower-level functions that pack data either transparently or perform some specified conversion (BIGendian to LITTLEendian, e.g.). Each caller-supplied function takes data from the input buffer and places it into an output buffer. The number of bytes to be taken from the input buffer is specified by the size parameter provided to the function from PCU.sub.-- PACK.sub.-- ENGINE. Once the data has been placed in the output buffer, the lower-level function modifies the outptr parameter of the ctx structure to point to the byte following the last byte written to the output buffer.

PCU.sub.-- PACK.sub.-- ENGINE uses the various lower-level functions to store data in the output buffer as follows. The structured data types in the input buffer are stored densely (byte-aligned) in the output buffer in the same order as they were originally defined in IDL. The contents of any padding fields inserted by the code generator to achieve correct alignment are discarded. Similarly, the functions do not place default values in those fields.

Base type data structures are stored in the output buffer in the representation specified by the dst.sub.-- XXX.sub.-- fmt parameters on the call to PCU.sub.-- PACK. Typically, these parameters are set to the packer's native format without any conversion. Thus, the server application (the unpacker) would perform the actual conversion. The routines utilized in the present invention, however, permit the packer to perform a conversion of the data structures as well.

The representation of shorts, unsigned shorts, longs, unsigned longs, long longs, and unsigned long longs are specified in the dst.sub.-- real.sub.-- fmt parameter to PCU.sub.-- PACK. This parameter specifies the format for representing floating point numbers. The alignment of floats and doubles are specified by the dst.sub.-- real.sub.-- fmt parameter. This parameter corresponds to the format for representing integers. The representation of chars are specified by the dst.sub.-- char.sub.-- fmt parameter. The dst.sub.-- char.sub.-- fmt parameter specifies a format for representing characters. Booleans and octets are not realigned. The "any" type is stored as an unsigned long specifying the length of the CIN description (whose alignment is based upon the dst.sub.-- integer.sub.-- fmt parameter), a CIN string describing the type (an unconverted ASCII string), and the data itself (stored based upon these conversion rules).

Compound types such as arrays and unions are also realigned. Arrays are stored with no padding between elements. Sequences are stored as unsigned long integers indicating the number of occurrences followed by that number of occurrences. Any padding between occurrences is removed. The format of the long integers depends upon the dst.sub.-- integer.sub.-- fmt parameter as stated above. A string is stored as an unsigned long indicating the length of the string followed by that particular number of characters stored as chars. The format of the chars is determined by the dst.sub.-- char.sub.-- fmt parameter. Structures are stored field by field without padding. Unions are stored as a long followed by the active union branch.

On the receiving end, the server application must extract the unstructured data type and its appendages from the buffer that was packed using PCU.sub.-- PACK. PCU.sub.-- UNPACK then places this unstructured data into a data structure based upon the prepared CIN for the data structure. PCU.sub.-- UNPACK is defined as follows:

The first three parameters correspond to the first three parameters of PCU.sub.-- UNPACK. These parameters specify the format of data types as stored in the input buffer. These parameters are preferably identical to their PCU.sub.-- PACK counterparts. The *prepbuf parameter is a pointer to the address containing the prepared CIN description as returned by PCU.sub.-- PREPARE. The address of the input buffer is pointed to by *inbuf. The length of inbuf is specified by inbuf.sub.-- len. The address of the output buffer is pointed to by *outbuf. The maximum number of bytes that can be accommodated by outbuf is specified by outbuf.sub.-- max.sub.-- len. The parameter *outbuf.sub.-- len obtains the number of bytes actually written to outbuf. The number of bytes read from the input buffer is specified by *inbuf.sub.-- used. If the number of written bytes or the number of read bytes are not needed, NULL may be passed as the value for these parameters.

PCU.sub.-- UNPACK creates a ctx structure that is used to pass context around to the internal functions of PCU.sub.-- PACK. This ctx structure is used by the underlying functions to PCU.sub.-- UNPACK and is defined as follows:

The first three parameters are the formats passed to PCU.sub.-- UNPACK. These parameters are needed by the internal functions in case PCU.sub.-- PACK is called recursively to handle an IDL "any" type. The inptr pointer points to the next byte to be read from the input buffer. The inbuf.sub.-- end pointer points to the last byte plus one in the input buffer.

After creating the context structure, PCU.sub.-- UNPACK calls PCU.sub.-- UNPACK.sub.-- ENGINE which provides the functionality for PCU.sub.-- UNPACK. PCU.sub.-- UNPACK.sub.-- ENGINE receives pointers to caller-supplied function for extracting data from the input buffer (the output buffer provided by PCU.sub.-- PACK) and placing it in an output buffer. The prepared CIN buffer is also provided as a parameter. PCU.sub.-- UNPACK.sub.-- ENGINE goes element-by-element through the prepared CIN and stores the data into a data structure as specified by the offset and size fields of the op.sub.-- tag structures.

For each element in the prepared CIN buffer, PCU.sub.-- UNPACK.sub.-- ENGINE calls the appropriate lower-level user-specified function to perform the unpacking and converting. PCU.sub.-- PACK.sub.-- ENGINE passes the data type and size of the data to be read from the input buffer along with the address of the output buffer to write the data. PCU.sub.-- UNPACK.sub.-- ENGINE also passes the ctx data structure to each of the functions. Each caller-specified function then extracts the data from the input buffer and places the data into a data structure. The number of bytes to be written from the input buffer to the output buffer is determined by the size parameter. For compound types, PCU.sub.-- UNPACK.sub.-- ENGINE provides additional parameters to the caller-supplied functions. If the data structure is an array, the number of elements in the array is provided. If the data structure is a sequence, PCU.sub.-- UNPACK.sub.-- ENGINE provides the maximum number of elements in the sequence along with the actual number of elements. If the data structure is a string, the maximum size and actual size of the string are provided to the caller-supplied functions. If the compound type is a struct data type, the number of members of the structure are provided.

Now, with reference to FIGS. 6 and 7, the method of the present invention will be described. FIG. 6 is a flow chart of the client side of the method of the present invention. Prior to performing the method of the present invention, as stated above, compact descriptions of data structures are created by the code generator 111 and are included in the client and server stubs. This description can be created using the method described in Application No. XXX. The client and server stubs are compiled and linked into the client and server applications. Once the client stubs have been linked into the client application, in a first step 601, the client application creates a prepared CIN description by calling the function PCU.sub.-- PREPARE. PCU.sub.-- PREPARE takes the CIN description of the data structure and, in step 603, converts the CIN to an array of op.sub.-- tag data structures by calling PROCESS.sub.-- CIN.sub.-- ITEM for each element of the CIN description. Each structure contains information regarding the type, offset, alignment, and size of the CIN-described data structure. A table of these structures are then stored in a memory buffer called prepbuf, in step 605.

In step 607, the client application calls PCU.sub.-- PACK which packs the data structure by copying the data into an output buffer based upon the size as specified by the size field of each op.sub.-- tag. PCU.sub.-- PACK removes any alignment padding fields from the data structure and places the data structure, in step 609, into an output buffer. Once the data structure has been packed into the output buffer, the data is transported in step 611. The data structure may be transported across a wire to a server application or transported to a file, such as a disk file. If another request involving the same data structure is made, this request is packed and the client application repeats steps 605-611 for the new request. The CIN description of the data structure need not be "prepared" again.

FIG. 7 shows the server side of the method of the present invention. The server application, in step 701, calls PCU.sub.-- PREPARE to obtain a prepared description of the CIN. The "prepare" step is similar to step 601 described above. The server then calls PCU.sub.-- UNPACK in step 703 to extract a structured data type and all of its appendages from the buffer that was packed using PCU.sub.-- PACK. In step 705, the structure is unpacked based upon the parameters passed to the function (the same parameters passed to the PCU.sub.-- PACK function). While extracting the data structure, the structure is realigned in step 707 from the format specified in the input buffer of the server to the native alignment of the server. If another request arrives at the server, the server can call PCU.sub.-- UNPACK to unpack the request without preparing the data structure.

Having thus described a preferred embodiment of a method and apparatus for converting IDL-defined data structures to and from a format convenient for transport, it should be apparent to those skilled in the art that certain advantages of the within system have been achieved. It should also be appreciated that various modifications, adaptations, and alternative embodiments thereof may be made within the scope and spirit of the present invention. For example, IDL-defined data structures have been illustrated, but it should be apparent that the inventive concepts described above would be equally applicable to hand-written data structures if the structures follow the same alignment rules as the code generators. The invention is further defined by the following claims.