Method, system and program products for modifying globally stored tables of a client-server environment

[0002] This application contains subject matter which is related to the subject matter of the following applications, each of which is assigned to the same assignee as this application and filed on the same day as this application. Each of the below listed applications is hereby incorporated herein by reference in its entirety:

[0003] “Method, System And Program Products For Efficiently Locking Resources Of A Global Data Repository,” Uceda-Sosa et al., Ser. No. 09/583,797, filed May 31, 2000;

[0004] “Method, System And Program Products For Concurrent Write Access To A Global Data Repository,” Uceda-Sosa et al., Ser. No. 09/584,523, filed May 31, 2000; and

[0005] “Method, System And Program Products For Concurrently Accessing A Global Data Repository By Multithreaded Clients,” Uceda-Sosa et al., Ser. No. 09/584,609, filed May 31, 2000.

[0006] This invention relates, in general, to a distributed computing environment, and in particular, to managing the accessing and modifying of data located in a global repository of the distributed computing environment.

[0007] Distributed systems are highly-available, scalable systems that are utilized in various situations, including those situations that require a high-throughput of work or continuous or nearly continuous availability of the system.

[0008] Typically, clients of a distributed system desire access to and modification of data residing in a global repository accessible by one or more servers of the system. In order to prevent corruption of the data, techniques are needed to manage the access and modification of the data.

[0009] One previous technique includes locally caching the data, so that clients have access to the data. This often results in complex and error prone data caching validation techniques, which can hinder performance of the repository, if the data access does not follow data locality patterns.

[0010] In a further example, the data is always accessed at the server and brought over the network. This allows the user to take advantage of the search engine of the server, but there is a high performance price to be paid.

[0011] Based on the foregoing, a need still exists for a capability that facilitates efficient access and modification of data stored in remote repositories.

[0012] The shortcomings of the prior art are overcome and additional advantages are provided through the provision of a method of modifying stored tables of a client-server environment. The method includes, for instance, providing, in local memory of a node of the client-server environment, a local table comprising at least a portion of a global table concurrently accessible by a plurality of users of the client-server environment, wherein the local table is manipulatable via database functions, as if it was a global table; modifying the local table; and atomically merging at least a portion of the modified local table with the global table, wherein the merging is performed without user locking of the global table.

[0013] System and computer program products corresponding to the above-summarized methods are also described and claimed herein.

[0014] Advantageously, data access and modification is facilitated by an efficient and highly functional capability of the present invention.

[0015] Additional features and advantages are realized through the techniques of the present invention. Other embodiments and aspects of the invention are described in detail herein and are considered a part of the claimed invention.

[0016] The subject matter which is regarded as the invention is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other objects, features, and advantages of the invention are apparent from the following detailed description taken in conjunction with the accompanying drawings in which:

[0017] FIG. 1 depicts one example of a computing environment incorporating and using aspects of the present invention;

[0018] FIG. 2 depicts one embodiment of a computing environment having a plurality of clusters, used in accordance with an aspect of the present invention;

[0019] FIG. 3 depicts one embodiment of a computing environment in which a system registry component is located on only a portion of the nodes of the environment, in accordance with an aspect of the present invention;

[0020] FIG. 4adepicts one embodiment of a data repository, in accordance with an aspect of the present invention;

[0021] FIG. 4bdepicts one embodiment of a table of the data repository of FIG. 4a, in accordance with an aspect of the present invention;

[0022] FIG. 5 depicts one embodiment of the logic used to select and modify a table, in accordance with an aspect of the present invention;

[0023] FIG. 6 depicts one example of a result table, in accordance with an aspect of the present invention; and

[0024] FIG. 7 depicts one embodiment of the logic used to merge one table with another table, in accordance with an aspect of the present invention.

[0025] FIG. 8 depicts one embodiment of the logic used to merge a working copy of the target table with the target table.

[0026] FIG. 9 depicts one embodiment of the logic associated with deleting a column in the target table.

[0027] FIG. 10 depicts embodiment of the logic associated with deleting a row in the target table.

[0028] FIG. 11 depicts one embodiment of the logic associated with adding a new column to the target table.

[0029] In accordance with one or more aspects of the present invention, a user application can explicitly and efficiently access and modify data stored in a global repository. Data extracted from the global repository is cached locally, and then manipulated using the repository search engine with full transaction capabilities. This provides a simple, efficient and highly functional facility for accessing and modifying data (e.g., remote data) of a distributed computing environment.

[0030] One example of a distributed computing environment incorporating and using aspects of the present invention is depicted in FIG. 1 and described herein. A distributed computing environment 100 includes, for instance, a plurality of frames 102, coupled to one another via a plurality of LAN gates 104. Frames 102 and LAN gates 104 are described in detail herein.

[0031] In one example, distributed computing environment 100 includes eight (8) frames, each of which includes a plurality of processing nodes 106. In one instance, each frame includes sixteen (16) processing nodes (each having one or more processors). Each processing node is, for instance, a RISC/6000 computer running AIX, a UNIX based operating system. Each processing node within a frame is coupled to the other processing nodes of the frame, via, for example, an internal LAN connection. Additionally each frame is coupled to the other frames via LAN gates 104.

[0032] As examples, each LAN gate 104 includes either a RISC/6000 computer, any computer network connection to the LAN, or a network router. However, these are only examples. It would be apparent to those skilled in the relevant art that there are other types of LAN gates, and that other mechanisms can also be used to couple the frames to one another.

[0033] The distributed computing environment of FIG. 1 is only one example. It is possible to have more or less than eight frames, or more or less than sixteen nodes per frame. Further, the processing nodes do not have to be RISC/6000 computers running AIX. Some or all of the processing nodes can include different types of computers and/or different operating systems. For example, this invention can be employed with LINUX and/or Windows operating systems. Further, a heterogeneous environment can include and utilize aspects of the invention in which one or more of the nodes and/or operating systems of the environment are distinct from other nodes or operating systems of the environment. The nodes of such a heterogeneous environment interoperate, in that they collaborate and share resources with each other. All of these variations are considered a part of the claimed invention.

[0034] A distributed computing environment, which has the capability of sharing resources is termed a cluster. In particular, a computing environment can include one or more clusters. For example, as shown in FIG. 2, a computing environment 200 includes two clusters: Cluster A 202 and Cluster B 204. Each cluster includes one or more nodes 206, which share resources and collaborate with each other in performing system tasks. Each node includes an individual copy of the operating system.

[0035] Each cluster is managed by a cluster architecture, which includes a plurality of components (e.g., a management component, a registry component, a liveness component, a group services component and a resource management component. Each component executes one or more corresponding processes on one or more nodes of a cluster. If the component executes a plurality of processes on a plurality of nodes, then the architecture is distributed across those nodes. One example of a cluster architecture and the management of that architecture is described in detail in a U.S. patent application, entitled “Method, System And Program Products For Managing A Clustered Computing Environment,” Novaes et al., Ser. No. 09/583,677, filed May 31, 2000, which is hereby incorporated herein by reference in its entirely.

[0036] The nodes of a cluster are capable of sharing resources and data with one another. In one embodiment, in order to facilitate the sharing of data, a system registry component 300 (FIG. 3) is used, which is highly available and provides access to one or more global data repositories 302. In one example, system registry 300 is replicated among a plurality of nodes 304, as system registry processes 306. In one embodiment, however, the system registry is replicated on fewer than all of the nodes of a cluster. As shown in FIG. 3, Nodes 1, 2 and 3 include a system registry process 306, while Nodes 4-7 do not. Nodes 4-7 are thus, considered registry client nodes of a client-server environment, while Nodes 1-3 are considered registry server nodes of the environment, since the registry function is executed on those nodes.

[0037] In one example, the data repository is a relational repository, in which the data is stored in one or more data structures (e.g., relational tables 400 (FIG. 4a)), that can be concurrently accessed in the client-server environment. Each table includes one or more rows 402 (FIG. 4b) and one or more columns 404. Typically, a column has a name and a fixed type, as well as a default value that is assigned to the column by the user when it is created. In one example, at least one of the columns is a distinguished column that acts as a primary key column. This column has no default value. Further, in accordance with an aspect of the present invention, each table has a reserved column 406, called a RowChangeCounter, that is updated at the time a row is modified. In one example, the RowChangeCounter is set equal to the value of a global counter 408, referred to as a TableChangeCounter, which is coupled to a given table and incremented each time a modification takes place in the table.

[0038] To facilitate access of the global data repository, in accordance with an aspect of the present invention, a user application executing on a client node of the client-server environment links to a Repository Client Library stored on or coupled to the client node. Further, both the clients and servers share a search and modification engine (an internal library stored on or coupled to the client and server nodes) that implements a complete set of data access services (e.g., retrieve, write, etc.). This allows a local table, including a local table that has been extracted from the server, to be manipulated via database functions, as if it was a global table.

[0039] One embodiment of the logic used to access and modify a table of the global data repository is described with reference to FIG. 5. Initially, when a user application running on, for instance, a registry client node, wishes to make a modification to a table in the data repository, the user issues an application programming interface (API) function in order to extract a new table (referred to herein as a source table or ResultTable) from a given server table, STEP 500.

[0040] In one example, the API function is stored in the Client Repository Library, and is called using the following pseudocode:

[0041] ResultTablePTR extract (InputTableName, ProjectionCriteria, SelectionCriteria, SelectionParameters[]),

[0042] in which the parameter, InputTableName identifies the server table from which data is to be extracted; and the ProjectionCriteria selects the columns to be part of the new result table. In one example, the RowChangeCounter column of the input server table and the TableChangeCounter column are also (at least implicitly) a part of the result table.

[0043] The SelectionCriteria is a string that allows the selection of any combination of rows of the input table; and the SelectionParameters is an array of variables utilized with the SectionCriteria, as described herein. In one example, the SelectionCriteria implements full boolean logic with regular expressions and parameter substitution. A scheme of the BNF (Backus-Naur Form) grammar used in one implementation of the SelectionCriteria is listed below:

[0044] <selectionCriteria>:=<andExp>|<andExp> OR

[0045] <selectionCriteria>

[0046] <andExp>:=<notExp>|<notExp> AND <andExp>

[0047] <notExp>:=<parExp>|NOT <parExp>

[0048] <parExp>:=<predicate>|(<predicate>)

[0049] <predicate>:=<columnName>==<expression>

[0050] <predicate>:=<columnName>!=<expression>

[0051] <predicate>:=<columnName><<expression>

[0052] <predicate>:=<columnName>><expression>

[0053] <predicate>:=<columnName><=<expression>

[0054] <predicate>:=<columnName>>=<expression>

[0055] <predicate>:=<columnName>?=<regexExp>

[0056] <expression>:=<constantLiteral>|% <paramVariable>

[0057] Where <constantLiteral> represents a string that can be converted to the type of the column, <regexExp> is a regular expression, and <paramVariable> is an uppercase letter that denotes the type of the column, and which works as a placeholder for the corresponding SeletionParameters array.

[0058] The extracted ResultTable is placed in the Repository Client Library, and it is unnamed. Further selects on this table can be performed from the client search engine in order to further refine the searches. The ResultTable can be manipulated and modified by adding/deleting rows and columns and/or by modifying existing rows and columns using the same primitives as the database, STEP 502.

[0059] One example of a ResultTable is described with reference to FIG. 6. In one example, a ResultTable 600 includes a plurality of columns and at least one row. For example, ResultTable 600 includes a RowChangeCounter column 602, and a second distinguished column, called the SourceRowChangeCounter 604. The SourceRowChangeCounter reflects the RowChangeCounter of the target table (i.e., the table from which data was extracted (e.g., the global table)). When the ResultTable is freshly extracted, the values of its RowChangeCounter and its SourceRowChangeCounter are the same. However, since the SourceRowChangeCounter column is read only, its values do not change. The ResultTable also has associated therewith a TableChangeCounter 606. Further, the RowChangeCounter of the ResultTable behaves, as described above.

[0060] Returning to FIG. 5, subsequent to modifying the extracted ResultTable, the user calls an API function (stored, for instance, in the Client Repository Library) to merge the modified table with the global table (a.k.a., the target table), STEP 504. As one example, the merge function has the following format:

[0061] ReturnCode Merge (localTableptr, TargetTableName, TableChangeCounter, RowChangeCounter),

[0062] in which the parameter, localTablePtr refers to the ResultTable to be merged; and TargetTableName indicates the table stored in the global storage, which is to be merged with the table identified by localTablePtr. The TableChangeCounter and RowChangeCounter parameters of this function call are optional. They indicate a desired version of a table that should be updated. For example, if the TableChangeCounter parameter is five, the target table is modified only if its TableChangeCounter is less than or equal to five. The same is true for the RowChangeCounter. Only rows with counters less than or equal to the RowChangeCounter parameter are modified. The SourceRowChangeCounter of the local table (i.e., the ResultTable) is passed to the server that is to perform the merge, but it is not merged with the target table.

[0063] The merge function allows any table (local or remote) to be merged with any other table. In one example, the merge operations are carried out in a working copy of the target table, while the target table is locked by the server (e.g., by mutexes or any other server synchronization mechanism). In accordance with the principles of the present invention, the merge function is performed without having the user perform any locking of the global table. That is, the locking is transparent to the user. If any of the operations fail, the working copy is discarded. If all the operations succeed, the working copy of the target table becomes the current version of the table. The TableChangeCounter of the target table is only updated once.

[0064] One embodiment of the logic used to merge the working copy of the target table (e.g., the global table) with the target table is described in detail with reference to FIG. 8. A working copy of the target table is obtained, STEP 800. One or more operation is then performed on the working copy to reflect one or more changes made to, in this example, a local table, STEP 802. If the one or more operations are performed successfully, the target table is replaced with the working copy, STEP 804. In response to replacing the target table, one or more counters associated there with is updated, STEP 806.

[0065] One embodiment of the logic used to merge the local table with the server table, such that the server table includes the user's modifications is described in detail with reference to FIG. 7. Although in this example, a local table (i.e., the source table) is being merged with a global table (i.e., the target table), this is only one example. Any table can be merged with any other table, local or remote. Further, the steps of this technique are performed in an order which provides consistent semantics and lightweight synchronization mechanisms for concurrent access. The first error encountered cancels the operation, as described below.

[0066] Referring to FIG. 7, initially, the names and types of the columns that were deleted in the source table (which are kept track of by the Client Repository Library) are shipped back to the server as part of the metadata of the table, STEP 700.

[0067] One embodiment of the logic associated with deleting a column is described with respect to FIG. 9. A counter associated with the column in the source table is compared with anther counter, STEP 900. The column in the target table is deleted if a predefined relationship exists between the counter associated with the column in the source table and the other column, STEP 902.

[0068] For example, any larget table column that has the same type and name is deleted, as long as the RowChangeCounter column of the target table has values no higher than those in the SourceRowChangeCounter column passed to the server. This is a synchronization check in case the table was altered by other users. Note that it is possible that a column is created and deleted and recreated again several times. In this case, only the last deletion is kept in the list.

[0069] Next, new columns that were created in the source table are handled, STEP 702. The new columns are added to the larget table. While checking for existing columns with the same name, (since not all the columns may have been extracted or another user may have added other columns to the global table in the meantime).

[0070] One embodiment of the logie for adding the new columns is shown in FIG. 11. First it is determined whether a column of the same name and type exists as that in the source table exists in the target table, STEP 1100. If the column dose not already exist in the target table, it is added, STEP 1102. If a column with the same name, but different type exists in the target table, an error is returned and the merge operation ends.

[0071] One embodiment of the logie associated with deleting a row is described with respect to FIG. 10. A counter associated with the row in the source table is compared with another counter, STEP 1000. The row in the target table is deleted if a predefined relationship exists between the counter associated with the row in the source table and the other row, STEP 1002.

[0072] Each row that exists in the source table, but not in the target table, is added to the target table, STEP 706. Default values are used for the fields of columns that exist in the target table, but not in the source table.

[0073] The remaining rows in the source table are scanned, comparing each field with the corresponding ones in the target table, STEP 708. When a difference is encountered, the value of the field in the source table replaces the value in the target table.

[0074] The values of the TableChangeCounter and RowChangeCounter in the source table are ignored, STEP 710. In the target table, the TableChangeCounter is incremented by 1, if there are any changes to be reflected. The RowChangeCounter field for any rows added or modified is set equal to the TableChangeCounter of the target table.

[0075] Described in detail above is a technique that allows a user application to explicitly and efficiently cache data locally and to make use of the repository search engine with full transactional capabilities. Further, parameters are provided with the merge function such that concurrent modifications to the target (e.g., global) table are allowed. Thus, a maximum level of concurrency is achieved, without employing typical database semantics.

[0076] While a clustered environment is provided as one example, the invention is not limited to such an environment. Further, one or more aspects of the present invention are applicable to homogeneous systems, as well as heterogeneous systems. As one example, capabilities are provided to facilitate the interoperability of the systems of a heterogeneous environment.

[0077] Aspects of the invention are applicable to client-client environments, as well as client-server environments. For example, local tables can be merged with other local tables. Likewise, global tables can be merged with other global tables.

[0078] Further, while in the examples described herein each data repository is remote from the nodes, this is only one example. In other examples, one or more of the repositories can be local to one or more nodes. Further, each repository can be remote or local to users of the repositories.

[0079] The present invention can be included in an article of manufacture (e.g., one or more computer program products) having, for instance, computer usable media. The media has embodied therein, for instance, computer readable program code means for providing and facilitating the capabilities of the present invention. The article of manufacture can be included as a part of a computer system or sold separately.

[0080] Additionally, at least one program storage device readable by a machine, tangibly embodying at least one program of instructions executable by the machine to perform the capabilities of the present invention can be provided.

[0081] The flow diagrams depicted herein are just examples. There may be many variations to these diagrams or the steps (or operations) described therein without departing from the spirit of the invention. For instance, the steps may be performed in a differing order, or steps may be added, deleted or modified. All of these variations are considered a part of the claimed invention.

[0082] Although preferred embodiments have been depicted and described in detail herein, it will be apparent to those skilled in the relevant art that various modifications, additions, substitutions and the like can be made without departing from the spirit of the invention and these are therefore considered to be within the scope of the invention as defined in the following claims.