Software Redevelopment System

1. Field of the Invention

The preferred embodiment of the present invention is directed to software redevelopment and is more particularly directed to a system and process for conducting large scale changes to highly connected and interrelated software systems.

2. Description of the Related Art

Large software systems have become a critical component in the day-to-day operations of many businesses. These software systems have data structures and executable programs which number in the thousands, hundreds of thousands, or even millions. Each individual program, contained in one of these software systems, has referential dependencies to other files, and likewise each data file has its own set of referential dependencies. This creates an enormous system of interrelated components for which considerable time, effort, expense, and inconvenience is required for incorporation of a change. Furthermore, a significant change can cause interruptions in user accessibility and service, resulting in damaging effects to business operations.

One example is the system glitches caused by the year 2000. During the development of software systems, programs were originally written to think only in 20th-century terms, and to use two digits to store dates. Therefore, the year 2000 would be seen as 1900 by the programs as written, resulting in critical computation errors. These errors create a rolling wave of data processing malfunctions which can quickly overwhelm a computer system, ultimately impairing a company's operations. Therefore, the programs and associated data tables must be expanded to 4 digits.

Currently, there are some software redevelopment tools commercially available. However, a total redevelopment system which offers a complete technical solution and which performs a desired software change in a manner which is transparent to day-to-day business operations, and thereafter continues to provide an integrated component for changes which arise in the future, has not been developed.

Therefore, it is the objective of the present invention to provide a redevelopment system which offers a complete technical solution for conducting large scale changes to highly connected and interrelated software systems, and thereafter continue to provide an integrated component for changes which arise in the future. A systematic approach encompassing the entire change process is required, with the complete solution providing the elements and relationships necessary to define a complete project plan, including a basis for managing the project (i.e. impact analysis, target date variance, and project completeness).

It is a further objective of the present invention to provide a redevelopment system that balances: 1) the need to keep critical systems operational during the change process (i.e. a process which is essentially transparent to the day-to-day operations of a business, 2) maintaining reasonable costs per unit rates, and 3) the ability to complete the task in as timely a fashion as possible, with enhancements outside the redevelopment project experiencing no delays.

An apparatus and method for implementing a predetermined change in a plurality of software objects throughout a software library is provided. The apparatus consists of a redevelopment database generated from the software library, the redevelopment database having relational information and resource information of the plurality of software objects. A partition is used for creating a plurality of clusters having the plurality of software objects, with the partitioner creating the plurality of clusters based upon the relational information. An assignor is used for allocating each of the plurality of clusters created by the partitioner to a resource, with the assignor allocating each of the plurality of clusters based upon the resource information, ultimately producing a plurality of resource allocated modules. A software change too is used in order to assist in the implementation of the predetermined change in each of the resource allocated modules, producing a changed resource allocated module. Finally, a configuration management system is used to receive the changed resource allocated module once the predetermined change has been completed. The configuration management system updates the software library with the changed resource allocated module, providing implementation of the predetermined change into the software library.

FIG. 1 is a system overview of the redevelopment system of the preferred embodiment of the present invention.

FIG. 2 is a high level flow chart of the functional flow of the preferred embodiment of the present invention.

FIG. 3 is a system overview for loading of the redevelopment database of the preferred embodiment of the present invention.

FIG. 4 is an illustration of some of the basic tasks associated with the propagation of relational information.

FIG. 5 is a flowchart of the functional flow associated with the identification and isolation of the elements requiring change of the preferred embodiment of the present invention.

FIG. 6 is an system overview of the synonym finder of the preferred embodiment of the present invention.

FIG. 7 is a flowchart of the functional flow of the synonym finder of the preferred embodiment of the present invention.

FIG. 8 is a flowchart of the functional flow associated with the partitioning of the preferred embodiment of the present invention.

FIG. 9 is a system overview of the partitioning system used in the preferred embodiment of the present invention.

FIG. 10 is an illustration of the four input files used by the partitioning system of the preferred embodiment of the present invention.

FIG. 11 is an illustration of the graphical user interface of the partitioning system used by the preferred embodiment of the present invention.

FIG. 12 is a flowchart depicting the functional flow of the two stage partitioning process performed by the partitioning system used by the preferred embodiment of the present invention.

FIG. 13 is a flowchart depicting the functional flow of the first stage of the partitioning process which is performed by the partitioning system used by the preferred embodiment of the present invention.

FIG. 14 is a flowchart depicting the functional flow of the second stage of the partitioning process which is performed by the partitioning system used by the preferred embodiment of the present invention.

FIG. 15 is an illustration of the logical design for implementation of the Partitioner application used by the preferred embodiment of the present invention.

FIG. 16 is a flowchart of the process associated with the changing of elements of the preferred embodiment of the present invention.

FIG. 17 is a system overview of the data name finder used by the preferred embodiment of the present invention.

FIG. 18 is a flowchart of the functional flow of the data name finder used by the preferred embodiment of the present invention.

FIG. 19 is a system overview of the truncation detector used by the preferred embodiment of the present invention.

FIG. 20 is a flowchart of the functional flow of the truncation detector used by the preferred embodiment of the present invention.

FIG. 21 is a system flow diagram depicting one mode of operation for adjusting sizes of data being exchanged between a computer program and a database.

FIG. 22 is an entity relationship diagram depicting the interrelationships of the data size adjuster and the datum transfer direction determinator.

FIG. 23 is a flowchart depicting the functional flow for determining whether to compress or expand the size of data.

FIG. 24 is a flowchart depicting the functional flow for when segments exist.

FIG. 25 is a flowchart depicting the functional flow for compressing data sizes.

FIG. 26 is a flowchart depicting the functional flow for expanding data sizes.

FIG. 27 is an illustration of the interaction between the redevelopment system and the DCCS (change management system).

FIG. 1 presents the redevelopment system 50 of the preferred embodiment of the present invention, and a large scale software system 52 utilized in the day-to-day operations of a business. In this detailed description, the large scale software system 52 is a management information system of a large automotive manufacturer, with the software system 52 containing a software library 54 having data and executable programs necessary for continued automobile production and support.

This software system 52 contains a large number of interrelated programs (approximately 100,000), some of which use eight digit part numbers employed throughout the company to uniquely identify all vehicle parts. As the business has grown and expanded, it was determined that it was necessary to expand the part number field size from eight to ten digits. Prior to the present invention, this field expansion, or a similar software change, would involve considerable time, effort, expense, risk, inconvenience, and potential for error. However, with the creation of the redevelopment system 50, this change can now be accomplished in a manner which is transparent to day-to-day business operations, and also reduce the time necessary to complete the system modifications, ultimately reducing costs associated with such a large scale task.

The software library 54 contains the executable software systems running on Information System (I/S) data centers and the associated data of these executable software systems. In the preferred embodiment of the present invention, a redevelopment database 56 is generated from the software library 54. The redevelopment database 56 contains a collection of tables 58 which store and relate information about the executable systems and data of the software library 54. The tables 58 hold information concerning software programs, and data files including, but not limited to Procedure Control Systems (PROCS), Job Control Language (JCL) elements, systems, Data Base 2 (DB2) tables, Information Management System (IMS) databases, Virtual Storage Access Management files (VSAM), sequential files, copylibs, Data Definition (DD) statements, and Data Set Names (DSNs) of the entire software library 54 portfolio.

Due to the immense size of the library 54 and the resulting redevelopment database 56 data pool, it is preferable to generate an element portfolio 60 which contains an information subset of the universe of elements that will potentially require a change (i.e. only that which will require a part number field expansion). This has an exponential effect in reducing complexity, change effort, and most importantly, the time frame for project completion.

The generation of the element portfolio 60 is assisted with a synonym finder 62 which allows the redevelopment database 56 to be queried for data elements that "look like" a given data name or names, or which appear to be synonymous with a name.

The data elements of the element portfolio 60 are flagged in the redevelopment database 56. From these flagged data elements, a flat file 68 is generated which is used by a partitioner 64 to develop an optimal set of work unit partitions or clusters 66. The partitioner 64 essentially takes the executables and data files having the data elements contained in the flat file 68, and divides them into a set of minimally overlapping groups based upon the connections and references of each executable or data file. The partitioner therefore allows changes to one cluster with minimal changes required to other clusters or work units, providing the ability to divide a large and complex task into manageable parts. It also reduces the amount of work necessary to complete the task in that each executable or file is modified a minimum number of times.

The clusters 66 are assigned, using assigner 69, according to resource information 68 contained in the tables 58 of the redevelopment database 56. In the part number field expansion project, the clusters 66 were assigned to a functional group which used the majority of the software or was identified as the "owner" of the majority of software contained in a cluster. The resource allocated modules 70 are then individually modified with an assortment of software change mechanisms 72.

The software change mechanisms 72 which were used to aid in performing the alterations to the resource allocated modules 70 are as follows: a data name finder 74, a truncation detector 76, and IMS bridge 78. The data name finder 74 locates lines of a executable code which require the desired modification. The truncation detector 76 examines compiler output to ensure all fields have been properly modified, and the IMS bridge 78 provides a means by which a software application can store and retrieve data from an IMS database despite the application and database having different formats for the data.

After one of the resource allocated modules 70 has been properly modified with the desired change (i.e. part number fields have been expanded and associated alterations have been performed, and unit and partition subsystem testing has been completed), a completed allocated module 79 is presented to a configuration management system or Data Center Control System (DCCS) 80 of the software system 52. The DCCS 80 then replaces/updates the software library 54 in the same manner as day-to-day changes or additions are incorporated.

It should be noted that concurrent with the predetermined change, which is being implemented by the redevelopment system 50, day-to-day operational changes and additions 82 are being implemented. These changes and additions 82, and the changes requested by the redevelopment system 50, result in an update of the redevelopment database 56. This is accomplished by directly linking the redevelopment database 56 to the DCCS 80. This integration allows the redevelopment database 56 to remain current with the changes and additions 82 and the predetermined changes incorporated to date (i.e. the completed part number field expansion changes).

FIG. 2 presents a high level flowchart of the functional flow of the preferred embodiment of the present invention. This change methodology for a highly connected and interrelated software system can be broken down into five basic steps: 1) collecting element information on the entire software portfolio 100, 2) identifying and isolating the elements requiring change 102, 3) partitioning into clusters 104, 4) changing the elements 106, and 5) implementing each partition 108. It is important to remember that this process is being performed concurrently with the day-to-day business operations, however the changes implemented by this process are conducted in such a manner as they are transparent to these day-to-day operations.

The step of collecting element information on the entire software portfolio 102 was implemented utilizing as a baseline a commercially available software package developed by the IBM Corporation that was enhanced to meet the requirements of this invention. An explanation of this software package can be found in IBM Redevelopment Directory, Version 3, Release 1, Modification 0. The redevelopment directory software package provides multiple database tables which store and relate information about the executable systems and the corresponding data.

As shown in FIG. 3, the redevelopment database 120 must be loaded with the specific system information 122 contained in the DCCS 124. The DCCS information is placed into the baseline tables 126 and user created tables 128 of the redevelopment database 120. The index report for the baseline tables 126 is provided in Appendix 1, and the user created data tables 128 are provided in Appendix 2. The two tables (126, 128) are loaded using standard IBM supplied load utilities 130.

Once the redevelopment database has been loaded with the required information, analysis tools of the redevelopment software package are utilized to propagate relational information which will be subsequently used in the partitioning, assignment, and executable and data modifications. This propagation of relational information is performed using automated analysis tools of the redevelopment directory software package. FIG. 4 presents some of the basic tasks associated with the propagation of the relational information.

In general, the redevelopment directory contains information about elements (program and data file variables) and how these elements are referentially related . The overall design is such that a query can be entered at any point in the Redevelopment Directory on a specific element, and the relationships can be identified to the higher and/or lower levels as needed. For example, if the entry point is a program name, the lower level queries would identify the names of any called modules or compile included members. The higher level queries would include the names of what procedure(s) the program is executed in, which jobs call the procedure, and the name of the personnel/group responsible for the job.

Although entity relationships were included in the baseline IBM software package, additional relationships were added to the present invention. The relationships were added in order to provide the ability to fully identify change impacts. The relationships added are as follows:

SITE/SCHEDULE

The Site/Schedule relationship established a production job execution schedule for a given site. The site was identified as a production host operating system. This allowed queries to determine if a change was made to a specific element, what site would be affected.

SCHEDULE/JOB

The Schedule/Job relationship established the production schedule for a given job. This information could be used to take job names that were to be changed and identify the affected production schedules. A review of these schedules would assist in determining the most opportune time to implement a job change.

JOB/PROC

The Job/Proc relationship established which procedures where used by a given production job. This information could be used to identify which jobs must be taken off line when a procedure was to be changed. In addition, after a procedure was identified, the job names could be used to query which production job schedules were affected.

PROC/FILE

The Proc/File relationship provided identification of files accessed in a procedure. This information could be used to identify which procedures were affected with a specific file change. In this way, changes to a file characteristic (i.e. file length) that required a change to the procedure that defined the file could be quickly identified and subsequently used to locate procedures affected and requiring modifications.

PROC/PROGRAM

The Proc/Program relationship was used to build the relational information between procedures and programs. The relationships were used to identify which procedures were affected when a given program name was changed. This provided a means by which affected procedures could be used to identify jobs and thereby allow identification of affects to production schedules.

PROGRAM/FILE

The Program/File relationship identifies which file is used by a program. This established which of the programs must be considered when a given file is changed or which of the files are affected when a program is changed.

PROGRAM/COPYBOOK

The Program/Copybook relationship established a link between programs and copybooks. In this way, identification of affects due to copybook changes could be accomplished.

PROGRAM/SCREEN

The Program/Screen relationship identified links between programs and on-line screen definitions. This provided a quick means to locate screen displays which would be affected by a program or field change. This also provided assistance in identifying screens which would need to be taken off line while changes were being incorporated, allowing notification to those who used the display.

DB2 TABLE/DB2

The DB2 Table/DB2 relationship established which of the DB2 views were referenced by a DB2 table. This information provided identification of the potential DB2 view requiring changes.

PROGRAM/DB2

The Program/DB2 relationship established which DB2 views were referenced by a program. These were utilized to identify the DB2 view which would require changes if modifications to a program were made.

COPYBOOK/FIELD ELEMENTS

The Copybook/Field elements relationship allowed queries than could be executed based on a field name, and a copybook using this field name could be located. This assisted in identifying which of the copybooks required changing when a field was identified as needing to be modified.

Following the propagation of relational information, the components of the system are mapped. Mapping codes are provided to each of the components of the system which will serve as a starting point during the partitioning, and will also be used to relate components to the responsible group which is to perform the modifications. Furthermore, call statements and other control transfers found in compilation units are mapped to the specific source compilation units that receive a call.

As previously indicated in FIG. 2, once the entire software portfolio (the universal portfolio) has been created 100, the elements requiring change are identified and isolated 102. As shown in FIG. 5, the process steps associated with identification and isolation of the elements requiring change 92 consists of four substeps: 1) determining aliases for the element to be changed 160, i.e. candidate sub-fields and aliases for part number, 2) searching the redevelopment database for all elements that meet the candidate criteria 162, 3) flagging the elements requiring change or examination 166, and 4) extracting the elements that meet the specified criteria 164.

The step of determining aliases for the element to be changed 160 and the step of searching the redevelopment database for all elements that meet the candidate criteria 162, are iterative. These two steps are conducted using the synonym finder 180, the system overview of which is shown in FIG. 6.

Referring to FIG. 6, a candidate list 182 is developed based upon user knowledge of the system. The candidate list 182 also includes wild cards. This candidate list is provided as input to the synonym finder 180 which searches the redevelopment database 184 and generates an output of data elements 186 which are synonymous with the elements contained in the candidate list 182. The output of data elements 186 is reviewed, and elements which are known not to be a synonym for the desired change (i.e. part number in this description) are placed in the exclusion list 188. For example, identification of an element belonging to a banking system or a human resource system would not have a part number, therefore an element which was identified and belonged to one of these groups would be placed in the exclusion list 188. The exclusion list 188, which is also provided as input to the synonym finder 180, prohibits these elements from being identified as a synonym element.

FIG. 7 shows a flowchart containing the functional flow of the synonym finder. Initially, the redevelopment database is searched for elements which meet the criteria of the input list 190. The elements which match the input list criteria are then compared to the exclusion list 192. The elements which are contained in the exclusion list are not placed in the output file 194, while those elements which are not contained in the exclusion list are placed in the output file 196.

The output file is then examined to determine if the elements requiring change have been identified 198. This examination includes reviewing programs having objects identified by the synonym finder to determine if an object requiring change is present (i.e. presence of part number fields). If the identified program does not contain a part number, the input list is examined to determine why the program was selected as an executable having a part number. Adjustments are then made to the input list to correct the over inclusion 200 or an entry is made in the exclusion list 202, and the process is repeated. For example, suppose P#R#T is in the input list, with # used as a wild card. The synonym finder is run and finds a program with the variable name PRINT, which is not a part number. Therefore, PRINT can be added to the exclusion list, or the form of the P#R#T inquiry in the input list can be changed. Once the elements requiring change have been identified, the process is complete 204.

Referring to FIG. 5, once the aliases for the element to be changed have been determined 160, and the redevelopment database has been searched for all elements that meet the candidate criteria 162, the change element portfolio is generated from the redevelopment database with the elements which meet the specified criteria of the synonym finder. The elements and relationships identified by the synonym finder are flagged as part number related 166 and are extracted 164 using programs which access the redevelopment database using Structured Query Language (SQL) statements. The element portfolio contains executable program name, data file names, referential information for each of these executable programs and data files, and resource information.

Once the elements requiring change have been identified and isolated, the partitioning into clusters or units of work is commenced. As shown in FIG. 8, this partitioning 210 can be broken into four substeps: 1) collecting all executed interdependencies between project elements 212, 2) calculating partition boundaries 214, 3) determining ownership and assigning the responsibilities 216, and 4) developing a work plan 218.

The step of collecting all executed interdependencies between project elements involves generating flat files that contain file and program referential information and resource information. These flat files were extracted from the redevelopment database using SQL inquiries. These SQL inquiries create four files which are then used by the partitioner to calculate partition boundaries (i.e., units of work).

The partitioning system 230 is shown in FIG. 9. The partitioning system 230 is comprised of: input data 232 consisting of four input files (234, 236, 238, 240), and the partitioner 242, which receives user provided input parameters 244 obtained through a graphical user interface 246 (GUI), and produces an output list 248 of optimally clustered executables and files.

Referring to FIG. 10, the first input 240 to the partitioner 242 is the ELEMENTS.PID file which is a list of Program Identifications and the partition, if any, to which the program has currently been assigned. It should be noted that if a program does not currently belong to a partition, a partition value of "-1" is used to indicate the un-associated status. The second input file 238 is the ELEMENTS.PID file which provides the same information as the ELEMENTS.PID file except that it is a list of File identifications and the partition, if any, to which the file has currently been assigned. The data format (250, 252) of these two files (238, 240) which were used are also shown in FIG. 10.

The third input 236 and fourth input 234 to the partitioner 242 are the Program to File interfaces contained in the INTERFAC.PID file and INTERFAC.FID file. These two files list each program a file has references to, and each file a program has references to, respectively. The data format (254, 256) of these two files (236, 234) which were used are also shown in FIG. 10.

The last set of inputs used by the partitioner 242 are provided by the user through the GUI 246, which was implemented using Microsoft's Visual Basic. As shown in FIG. 11, the GUI 246 has four input groups: input/output identification 258, partition parameters 260, minimum size criteria 262, and minimum to maximum size criteria 264.

The input/output identification 258 allows the user to enter the name of the four input files to be used by the partitioner. This includes Programs 266, which is the input file containing the Program names and partitions (if assigned), Files 268, which is the input file containing the Names and partitions (if assigned), Interfaces By Prog 270, which is the name of the file containing the Program/File connections ordered by Program name, and Interfaces By File 272, which is the name of the file containing the Program/File connections ordered by File name. Furthermore, the partitioner generated file which contains a list of the Programs and Files with the assigned partitions, may be designated by the user with Partitions 274.

The partition parameters group 260 provides the user with the ability to set Number 276, which is the number of partitions which are to generated, Start 278, which is the partition number from which to begin growing a partition, Minimum Size 280, which is the minimum number of objects to be contained in a single partition, and Maximum Size 282, which is the maximum number of objects to be contained in a single partition. The partition parameters group 260 also provides three buttons, Total 284, Programs 286, and Files 288, which allow the user to indicate if partition size is to be calculated based upon the number of both programs and files, the number of programs only, or the number of files only.

The Minimum Size group 262 allows the user to specify parameters used in the first stage of partitioning, which is to be subsequently described. The Percent Factor 290 and Count Factor 292 provide a means of expressing a tradeoff between the number of connections and the percentage of its total connections an object has in partitions. A high Percent Factor 290 and low Count Factor 292 will result in objects with a low number of connection being favored, while a low Percent Factor 290 and high Count Factor 292 will result in objects with a high number of connections being favored.

The Add Number 294 and Delete Number 296 allow the user to dictate the number of objects to be added in each cycle used to grow a minimum size partition of the first stage, and the number of objects to be deleted during the same first stage cycle. The difference between the Add Number 294 and Delete Number 296 controls the incremental increase of the partition during each stage 1 cycle. The magnitude of the Add Number 294 and Delete Number 296 controls the amount of "look ahead" the partitioning system performs in growing the partition. As an example, values of 10/5 versus 100/95 for Add Number 294/Delete Number 296 gives the same incremental increase of 5. The latter value performs a search which is increased by a magnitude, potentially providing a better solution. However, this potentially better solution comes with a higher computational cost.

The three user selected parameters of the Minimum to Maximum Size Criteria group 264, provides user control of the second stage of the partitioning, subsequently discussed. The Minimum Percentage to add from Pool 298 sets the threshold for the minimum percentage of connections an object must have after stage 1 has been completed in order to be included in the partition being generated. The higher the threshold, the lower the number of objects that will be added to the partition. A lower threshold is given for Minimum Percent Internal From Pool 300. Objects with a percentage of its connections in a partition will be added only if it is connected to an object already in the partition that has Minimum Percentage Internal in Partition 154 of its connections in the pool. This allows the user to "finish off" highly connected objects already in the partition by bringing in their remaining connections. The higher the thresholds, the lesser number of objects that will be added tao the partition.

Once the necessary information has been provided, the partitioner can begin to optimally cluster the pool of data center executables and files. The user initiates partition generation with the Grow button 304, and the Status Window 306 displays the current partition, partition size, and the stage of the partitioning system during execution.

The partitioner utilizes a two stage approach which is presented in FIG. 12. The application which implemented these two steps was developed using Microsoft's visual basic. The first stage 308 entails growing a partition to a minimum size. This is done in an iterative manner by adding a user specified number of objects (Add Number 294 of FIG. 11) from the pool, to the partition currently being generated 310. After the specified number is added, a user specified number of objects (Delete Number 296 of FIG. 11), which is less than the number of objects added, is deleted from the partition and returned to the pool 312. The objects added or deleted during this stage are selected based upon a calculated "score" to be subsequently described.

Because the number to be added is greater than the number to be deleted, each iteration results in a net increase in the partition size. After each addition and deletion, the partition size is measured 314 (based upon user entered Minimum Size 132 of FIG. 11) and once the minimum size is reached, the second stage 316 is commenced.

FIG. 13 provides additional details of the functional flow of the first stage of the partitioning process. Initially, a program or file is selected as the kernel object upon which to grow a partition 318. This initial selection is based upon a Score, which is calculated using equation (2). The Score has

two components. The first component weighs the importance of the number of connections that the object has in a partition, and the second component normalizes the score with the fraction of the number of connections in a partition in relation to the objects total number of connections of the object.

In the event that there are multiple programs or files with the highest Score, the program or file with the greatest number of connections is selected as the kernel object. In addition, if no partitions have been created, all programs and files will have a score of zero, therefore the program or file with the greatest number of connections will be used as the kernel.

Once the kernel object has been chosen, the score for all objects, which are connected to the kernel object (i.e. having a reference to or from the kernel object) are re-Scored 320 using equation (2). The user specified number of objects with the highest score are then added to the current partition being generated 322, and the Score for each object connected to an added object is recalculated using equation (2) 324. After this is accomplished, a user specified number of objects are deleted from the partition 326. The objects deleted are those with the lowest score. The objects connected to the deleted objects are then re-scored using equation (2) 328, and the deleted objects are then returned to the general pool 330.

After this has been accomplished, the partition size is measured to determine if the minimum size for a partition has been reached 332. If the minimum size has not been reached, the first stage process is repeated, starting with the addition of the specified number of objects with the highest score 322. If the minimum partition size has been reached, stage two is initiated 334.

Referring to FIG. 14, the second stage of the partitioning entails growing the partition of the first stage until a maximum partition size is reached. This is done by attempting to add all objects having a percentage of their connections in a stage 1 partition or a certain fraction threshold of their connections to the stage 1 generated partition 336, as it is referred (set by the user selection of Min Pct Internals to Add from Pool 298 of FIG. 11). If this results in a partition which is greater than or equal to the maximum desired size of a partition 338 (set by Maximum Size 282 of FIG. 11), the second stage is complete 342. If the maximum size of the partition is not reached, and all objects with a certain fraction threshold of their connections are included in the first stage partition, objects that have at least a certain lower minimum percentage threshold of their connections in partitions are added to the partition under development if connected to an object that has at least a certain higher minimum percentage of their connections in a partition already created 340. (Note that the certain lower minimum percentage is specified by the user in Min Pct Internals from Pool 300, and the certain higher minimum percentage is specified by Connected to Min Pct Internals in Partition 302, both of FIG. 11) Once this has been accomplished, the second stage is complete 342.

Referring to FIG. 12, after the second stage is complete, the determination is made as to whether the user selected number of partitions has been generated with the addition of the partition created during the current iteration 344. If the correct number of partitions has been generated, the partitioning is complete, and the application ends 208. If more partitions are required, stage 1 is re-initiated, beginning with the addition of a user specified number of objects to the new partition being generated 310.

FIG. 15 presents the logical design for implementing the partitioner application. The design is centered around a double linked 348 list of objects that are to be partitioned 350. The list objects 350 is ordered based upon the score of the objects, as calculated using equation (2). The higher the score, the closer an object is to the start position (352, 354) of the partition 356 and pool 358. Furthermore, by maintaining the start position (354) of the pool 358 and end position of the partition 356, the boundary 360 between the partition 356 and pool 358 is created.

An array of connections 362 is defined for each object. Each object has an array of pointers to all other objects that are referenced by the object. In this way, as an object is added or deleted from the partition, the array of connections 362 can be used to directly update the scores of the connected objects. Furthermore, an object with an updated score can be easily moved up or down the collection through the use of the double linked list, with most objects moved in small incremental steps, thereby minimizing the cost of a linear search in order to reorder the object in the double linked list.

The following partition object data type was defined in Visual Basic 3.0.

This allows an array of partitions to be created, with indexes in the Partition Object array used as pointers. The "Next" and "Previous" fields are pointers that provide the double linked list capability. "Total" is the total number of connections of an object, with the number of connections that an object has contained in a partition, specified by "Internal." The "ScoreFactor" based on the user provided parameters is maintained, along with the current "Score" of the object. The current partition to which an object is assigned is stored, and "Connection" provides an index into the Connection array that points to the set of references of an object.

A limitation of Visual Basic 3.0 is that a single dimension in an array limited to 32K. This limitation places the requirement to have a multi-dimensional array when a data object, having a 32K size, may exist. The partition objects array of the preferred embodiment supports nearly 100,000 Partition objects. In addition, a subroutine can be defined (`pntrListConvert`) that converts the `long` index into two `integer` indexes for the two dimension array using integer division and modulus functions. The declaration used are as follows:

For illustrative purposes, Appendix 3 presents an example of how an object (referred in the source code as an element) is moved through an array using the "Next" and "Previous" pointers. Movement forward in the List is based on the score of the object. The higher the score, the closer to the front of the list an object will be located. The "Previous" pointers of the objects in the list are used to find the correct position for the given object. Once found, the affected object pointers are updated. Conversion to the two dimension array is also shown in the illustrative example of appendix 3.

Once the partition boundary or work units have been calculated, the responsibility for each cluster or work unit is assigned to a responsible group. This was accomplished by importing the partitioner output, including associated resource information into an Access Database program. Using the database, partitions were assigned to a particular information system group or person. This assignment was based upon the number of programs and data files "owned" by a person or group in a partition. The group with ownership of the majority of data files and executables within a work unit was given the task to change the cluster. It should be noted that this assignment can be based upon a wide variety of resource information.

Once the partitioning and assignments are completed, the elements contained within the partitioned clusters are changed by the responsible group.

As shown in FIG. 16, changing the elements 96 consists of modifying the code 210, unit testing 212, and partition subsystem testing 214. In order to assist individuals in modifying the code 210, a data name finder was utilized to identify the locations of the elements requiring a change.

FIG. 17 illustrate a system overview of the data name finder. The data name finder 370 uses the program source code 372 contained in the DCCS 374 as an input. This tool is used with an editor 376 to search a program for variable names contained n the synonym list 378 generated by the synonym finder, or variables which receive a variable name, collated in the internal synonym during data name finder execution.

FIG. 18 shows the functional flow of the data name finder. Initially, all lines of the source code are marked as "undisplayed 382." In conjunction with an editor, each program line is searched for variable names contained in the synonym list and internal synonym list 384. When a line that contains a match to synonym finder item is located, the line is marked as "displayed" 386. Furthermore, each line is searched for a variable reassignment 388. If a line reassigns a found data name to another variable, the other variable is added to an internal synonym list 390. In this way, internal working variables of an element are also identified as potentially requiring change. For example, if part number in a program uses the content of a variable WORKING-STORAGE-MISC, lines with this variable will be marked as "displayed" so that an individual performing the changes to a program will investigate the potential need to change this variable due to the part number change. The process is repeated until each line of code is checked 392, Ultimately an output which lists all variables are marked as "displayed" and their location (i.e. line numbers) in the program is produced (see FIG. 17, data name finder output 394).

Individuals were also assisted with code modifications with the truncation detector. FIG. 19 provides a system overview of the truncation detector. The truncation detector 400 uses compiler output 402 as an input. It also utilizes the synonym list 404 to identify the variables which were to have been changed, and as with the Data Name Finder, it also uses an internal synonym list 406 which identifies variables not in the synonym list 404. The truncation detector 400 verifies the format of each variable against a specified format (which in the case is that the part number field should be at least 10 characters in length) and generates an output error file 408 containing variables that do not meet the format criteria prior to compilation of a program.

FIG. 20 shows the functional flow for the truncation detector. The first step consists of selecting a synonym from the synonym list, or if that list has been exhausted, the internal synonym list 410. The compiler output is then searched for the selected synonym 412, and if the variable is present, a check is made to see if the synonym variables contents are moved to a new variable 414. If this is found, the new variable is added to the internal synonym list 416. The format of the originally selected synonym is then verified to ensure the length is greater than or equal to the size required by the change 418. If the size is not greater than or equal to the size required by the change, the synonym variable is written to the error file 420. If all the synonyms in the synonym list and internal synonym list have been verified, the truncation detector ceases execution 422, otherwise, the above described process is repeated, starting with the selection of a synonym in the synonym list or internal synonym list 410.

The last tool used to assist in the changing the code, is the IMS bridge. The IMS Bridge facilitates the conversion of computer programs which access the hierarchical database IMS DL/I database (an IBM product). Specifically, the part number field of converted computer programs have a larger data size (i.e., ten positions for a part number datum) than what the IMS database had for its part number field (i.e., eight positions for a part number datum), the IMS Bridge is used to allow modification of the program without changing the IMS DL/I database.

FIG. 21 provides an example of one operational sequence of the IMS Bridge. The calls from a computer program 430 to an IMS database 434 are intercepted. The intercepting routine at function 438 compresses the Part Number field(s) to eight positions and passes to the IMS database 434 the current segment size on a Replace or Insert. The IMS Bridge expands at function 442 the Part Number field(s) and segment size, when the computer program 430 is accessing data from the IMS database 434.

FIG. 22 depicts the interrelationship between the IMS Bridge and its environment. A computer program 430 uses a datum 460 (e.g., a part number field) with a first size 464, such as ten positions. A data store 468 (e.g., a hierarchical IMS database 70) uses the datum 460 but with a second size 472, such as eight positions.

A datum transfer direction determinator 476 determines the direction by which the data will be exchanged. The direction indicates whether a datum is being transferred to the computer program 430 or to the data store 468. A database operation determinator 480 is used to examine the database operation being used to effect the exchange of data between the computer program 430 and the data store 468. A list 484 provides an example of the type of database operation which the database operation determinator 480 can use to determine the direction of the exchange. For example a retrieval database operation indicates that the direction for the exchange of data is from the data store 468 to the computer program 430.

The data size adjuster 488 adjusts the size of the datum 460 which is being exchanged between the computer program 430 and the data store 468 so that a datum 460 with an adjusted size 489 is produced. The adjustment of the size of the datum is based upon the determined datum transfer direction and upon a predetermined data size adjustment value 492. The data size adjustment value 492 indicates how much the data size should be expanded or compressed. For example, the data size adjustment value 492 would be two if the computer program 430 has size of the datum at ten and the data store 468 has the size of the datum at eight. Therefore, a compression of a datum from ten places to eight places deletes the last two positions (i.e., ending positions) of a predetermined number of the datum, and an expansion of a datum from eight places to ten places involves "padding" the datum with two blank spaces (each of EBCDIC value of 40 decimal) at the end of the datum.

The data size adjuster 488 is disposed between the computer program 430 and the data store 468 so that the datum is effectively intercepted before the datum has been received by the data store 468 or is effectively intercepted before the computer program 430 receives the requested datum depending upon the datum transfer direction.

An internal segment table 496 and an internal key table 500 store the data size adjustment value 492 for each particular datum type. The structure of the internal segment table 496 and the internal key table 500 is discussed in greater detail below.

Within the IMS environment, the IMS Bridge uses four called computer routines beginning with `CBLTCONV`. The `CBLTCONV` routine replaces the `CBLTDLI` routine. The `CBLTDLI` routine has the function of performing database operations against an IMS database from a COBOL or assembler program. The `CBLTCONV` routine has the function of performing the same database operations as the CBLTDLI routine while expanding/compressing I/O areas and compressing SSAs when found in internal tables.

The IMS Bridge allows the computer program conversion and expansion of Part Number fields independent to the database conversion. Any IMS on-line or batch program that converts its I/O area to accept expanded Part Number field(s), has the normal call to IMS changed from `CBLTDLI` to `CBLTCONV`, whenever it is attempting to Get, Replace, Insert to, or Delete data from the `converted` database. The I/O area refers to the segment(s) or records retrieved or to be inserted or updated from/to an IMS database. The parms passed to `CBLTDLI` are the same parms currently passed to `CBLTDLI`. The term "parms" refers to any items following a call (i.e. database operation, db pcb, I/O area, and SSAs). The call to `CBLTCONV` could be dynamic.

Program `CBLTCONV` is an assembler program that counts the number of parms passed from the computer program. It also calls a Cobol program `CONVCBL` dynamically.

Program `CONVCBL` is a Cobol program that makes the call to IMS using `CBLTDLI` before or after the I/O area and SSAs passed have been expanded or compressed. The term "SSA" refers to the keys or indexes passed to the IMS database to retrieve or update segment(s). Program `CONVCBL` calls `CONVSIZE`.

Program `CONVSIZE` is an assembler program that expands or compresses the I/O areas and SSAs passed to it.

Therefore, the IMS Bridge consists of 4 called programs:

CBLTCONV (Assembler-called statically or dynamically)

CONVCBL (Cobol-called dynamically)

CBLTDLI (Standard IMS call, static)

CONVSIZE (Assembler-called dynamically)

Within the IMS environment, Cobol programs which access IMS databases use a "call" routine "CBLTDLI" supplied by IBM. This call routine requires information to be passed from the program to IMS to complete the desired database operation.

An example of a Call routine is as follows:

The above "call" is replaced with:

The `GU` term above refers to `get unique`, which indicates to the IMS to get a specific segment(s). The term `db-pcb` above refers to the application view of the database structure.

Referring to FIG. 23, the start block 550 indicates that the first block to be processed is block 554. At block 554, the number of items being passed from the application program are calculated. In other words, a CBLTDLI call typically requires a minimum of three pieces of information (db operation, db-pcb, I/O area) and up to fifteen SSAs. Depending on the database operation and the number of SSAs passed, there may be more than one segment in the I/O area.

If Decision block 558 determines that SSAs exist, then processing continues at continuation block C 562. The existence of SSAs refers to the possibility that the key needs to be compressed. If Decision block 558 determines that SSAs do not exist, then processing continues at decision block 566.

If decision block 566 determines that the database operation is an insert, replace or delete operation, then block 572 is performed. Block 572 performs compression of the I/O area and then the CBLTDLI routine is called at block 573 before terminating at end block 750.

If decision block 566 determines that the database operation is not an insert, replace or delete operation, then decision block 576 is performed. If decision block 576 determines that the database operation is a retrieval, then the CBLTDLI routine is called at block 577 and the I/O are is expanded at block 580 before terminating at end block 750. If decision block 576 determines that the database operation is not a retrieval, then processing terminates at end block 750.

Referring to FIG. 24, continuation block C 562 indicates that block 600 is to be processed wherein the SSAs are parsed. At block 604, the SSAs are matched to the internal table to see if a match was found in the table. If decision block 608 finds a match, then the I/O area is compressed at block 612, else decision block 616 is immediately processed.

If Decision block 616 determines that there are more SSAs, then processing continues at block 600. If Decision block 616 determines that there are no more SSAs, then processing continues at decision block 617. For each match found in the internal table as determined at decision block 617, processing continues at decision block 620; else processing continues at block 629 wherein the CBLTDLI routine is called before terminating at end block 750.

If decision block 620 determines that the database operation is an insert, replace or delete, then processing continues at continuation block A 624. If decision block 620 determines that the database operation is not an insert, replace or delete, then processing continues at decision block 628. If decision block 628 determines that the database operation is a retrieval, then processing continues at continuation block B 632. If decision block 628 determines that the database operation is not a retrieval, then processing continues at block 629.

Referring to FIG. 25, continuation block A 624 indicates that block 691 is to be processed wherein the internal segment table is searched. If a match is found at decision block 693, then the I/O area is compressed at block 700 and decision block 704 is processed next. If a match is not found at decision block 693, then decision block 704 is processed.

If decision block 704 determines that there are more segments, then processing continues at block 691, else processing continues at block 705. Block 705 calls the CBLTDLI routine and then processing is terminated at end block 750.

Referring to FIG. 26, continuation block B 632 indicates that block 711 is to be processed wherein the CBLTDLI routine is called. Next, block 713 searches the internal segment table. If a match is found at decision block 715, then the I/O area is expanded at block 720 and decision block 724 is processed next. If a match is not found at decision block 715, then decision block 724 is processed. If decision block 724 determines that there are more segments, then processing continues at block 713, else processing terminates at end block 750.

The following processing is performed for compressing the I/O area from the computer program: using information from a prior retrieval operation, internal segment table and the SSAs passed, determine the size of the I/O area, the positions and lengths of the segment to be compressed; calls are then made to an assembler routine CONVSIZE to do the actual compression; a call is made for each segment in the I/O area. If the I/O area has more than one segment, the size of the I/O area as well as positions in the segment to be compressed are adjusted after each call to CONVSIZE.

After the SSA's and the I/O area are compressed, the normal CBLTDLI call is made. Control is returned to the application program (IMS returns a status).

The following processing is performed for expanding the I/O area from the computer: after SSAs are compressed, the normal CBLTDLI call are used to fill the I/O area; if the call was successful, the information from the internal segment table is used and the SSAs are passed, then the size of the I/O area is determined as well as the positions and lengths of the segments to be expanded.

Calls are then made to an assembler routine CONVSIZE to do the actual expansion. The number of calls depends on the number of segments returned from IMS. The expanded I/O area is passed to the application program. If the call was unsuccessful, return control to the application program, as well as IMS return status.

The following processing is performed to parse the SSAs as issued from the computer program: determine if the SSA has a valid, type, length or boolean operation present.

The following processing is performed for compressing the SSAs: Determine where in key, field is to be compressed; a call is made to assembler routine CONVSIZE to compress SSAs.

Internal Segment Table

The internal segment table is used to expand or compress the I/O area passed to program CONVCBL depending on the function call passed. It contains the database name, segment name, segment size, the ending position of the part number and the size to expand or compress. Currently the table allows for 25 occurrences of part numbers per segment. Since some calls to IMS may include path calls it is necessary to enter all the segments within a database and their size though the segment may not contain a part number. The program allows for five rows for the same segments. In other words 5.times.5 or 25 occurrences. The table below reflects five rows allowing 25 positions for expansion or compression. An example of an internal segment table is as follows:

In the above example:

Since part number occurred twice on the segment (a maximum of 25 part number occurrences are allowed), the two ending positions of part number and their corresponding change length were entered. If part number did not occur in the segment, but the segment could be used in a path call enter the database name, segment name and the segment size. The remainder of the fields should be zeroes.

New entries to the table requires the occurs statement to be modified. Entries in the table do not have to be in any particular order. A find all on x(45) will determine how many occurrences this table should have. If an error is made the Cobol compiler will catch the error.

Internal Key Table

The internal key table contains the database name, segment name, the key name, the ending position of the part number and the size to compress. Only the keys that contain part number are entered into this table. An example of an internal key table is as follows:

In the above example:

Entries need to be made to this table only if the key to the segment or the alternate key contain a part number. If a key is concatenated (with 2 or more part numbers or a part number and something else like a date), the position is entered where the part number(s) ends (a maximum of 25 part number occurrences are allowed). A find all on x(49) will determine how many occurrences this table should have. If an error is made the Cobol compiler detects the error.

The following is an example of an SSA:

In this example, FKPRTNMBR is the key to the SFPARTRT segment in the OCPTSDBD database. The length of the part number defined to IMS is 8 (application program will pass a 10 position part number) and the size is to be truncated 2 positions.

Once changes in a partition have been implemented, unit, string, and subsystem testing is conducted. The unit testing is generally conducted by the individual making the actual code modifications of a single object. It involves simulating program input and output, allowing verification that the program internals are correctly functioning. The string testing involved testing interdependencies between highly related programs. If one program writes to a file, and a program reads from the same file, a string test ensures both programs are using the proper format. Subsystem testing is string testing with the use of actual data used during system operation. This verifies that the functions work as defined by the customers and that the system operates on all permutations of data. Lastly, in the case of large partitions, multiple inter-related systems were testing in conjunction. This is a production test, performed on the day the programs are incorporated for actual production, and ensures that programs function to support existing processes. If a failure occurs at this level of test, the entire partition is backed out of production.

The last step in the redevelopment process entails implementing each changed partition. This occurs after a programmer knows which production elements are going to modified, and a DCCS project is created which includes all the specific elements to be changed.

FIG. 27 shows the program and data flow between the DCCS 800 and the Redevelopment data base 802. Prior to implementing a change to a particular program (or data file), a copy of the program 804 to be modified is obtained from the DCCS 800, and "signed out" by the programmer. This prohibits the existence of multiple copies being changed by different groups or programmers. After the changes are made and successfully tested, the modified program 806 is signed back into the DCCS 800. After all the programs and data files of a given partition are signed back into the DCCS 800, the partition owner establishes a production date for implementation.

When the DCCS project implementation date matches the current date, elements are transferred to the software library 808. In most cases, the current executing programs and data files are moved to a history file 810 that may be used to back the changed objects out if a problem is discovered. The status of production implementation is recorded in the DCCS 800 so that the owning programmer of the project and production operation personnel can track the project status.

As the DCCs 800 records production implementation, the indication of a production transfer is used to communicate to the redevelopment database 802 that a change has been made. The redevelopment database 802 retrieves the relational information concerning the elements which have changed, and updates the redevelopment tables 812 to reflect the changes. This update is executed nightly to keep the redevelopment data base as current as possible.

From the foregoing, it will be seen that the present invention provides a complete system and process for conducting large scale modifications to large software systems. The system is capable of being integrated with a wide variety of information systems. Therefore, while the invention is illustration and described in conjunction with a certain information system and a specific software change, it will be understood that the principles of the invention can be adapted to virtually any Information system and software change. Accordingly, it should be understood that the present invention is capable of certain modification and change without departing from the spirit of the invention as set forth in the appended claims. ##SPC1##