Managing unvirtualized data pages in real storage

[0002] 1. Field of the Invention

[0003] This invention relates generally to virtual memory storage systems for mini and mainframe computing systems, and particularly, to a novel system for managing storage of unvirtualized data pages in real storage.

[0004] 2. Discussion of the Prior Art

[0005] Virtual memory management schemes enable a computer system to present a large address space, larger than the amount of real storage on the system. A virtual memory management scheme consists of a mapping of an address space which provide a means of translating a virtual address of a page into a real address of a frame. A frame is the basic unit by which real storage is managed in a computer system. For example, in z/Architecture a frame consists of 4096 bytes.

[0006] Since real storage is a limited resource, when the operating system determines that it has become scarce it may try to write the data on some real frames to auxiliary storage, also called a direct access storage device (DASD) which is cheaper and more plentiful, but much slower to access. This operation is known as paging and involves initiating output device I/O to auxiliary storage. The operating system makes paging decisions at a global level and a local level. At the global level, the operating system determines which address spaces will suffer a potential performance penalty of having their data paged out to auxiliary storage. At the local level, the operating system determines which frames within an address space are good candidates to be paged to auxiliary storage. In general, frames which are least recently used are better candidates to be paged to auxiliary storage since they have a lower likelihood of being referenced in the near future. Computer systems, including z/Architecture, provide facilities to approximate least recently used (LRU) order. Part of the facility is the Referenced bit which is associated with each frame of storage in z/Architecture. The Referenced bit is turned on by the hardware whenever a frame of storage is referenced or changed. A separate bit called the Changed bit is turned on by the hardware whenever a frame is changed. z/Architecture also provides an instruction called Reset Reference Bit Extended (herein referred to a RRBE) which sets the Referenced bit to zero and sets a condition code which provides an indication of whether the frame was changed and/or referenced. The OS/390 operating system maintains a data structure, known as the Page Frame Table Entry (or PFTE) which includes data corresponding to a single frame. The aggregate of PFTEs is referred to as the PFT or Page Frame Table. PFTEs corresponding to frames used for a particular address space are placed on a queue. Periodically, this queue of PFTEs is traversed and the RRBE instruction is used to determine whether the frame is referenced; PFTEs corresponding to frames that have been referenced are moved to the back of the queue. This traversal of the PFTE queue is known as Unreferenced Interval Count Update (or UIC Update).

[0007] The act of writing real frames in use by an address space to auxiliary storage to replenish the number of available frames is referred to as stealing. When an address space is selected for stealing, steal candidates are selected by traversing the PFTE queue from the front to the back, in least recently used (LRU) order.

[0008] To support paging input/output (I/O) to/from auxiliary storage, virtual storage memory systems implement a facility known as Virtual Input/Output (VIO), which enables system-managed, temporary datasets to reside on virtual devices in external page storage. VIO is designed to reduce the need for the processor to transfer data between DASD and a real storage device and be processed using paging I/O. In operation, VIO's implement a scheme that partitions a dataset in 4K-byte pages, for example. By implementing VIO, system performance is improved as much of the overhead and time required to allocate a device and to move data physically between main storage and an I/O device is eliminated. As known, VIO is supported by services provided by: 1) an Auxiliary Storage Manager (ASM) which dynamically allocates page-size physical blocks to the VIO dataset, treats each VIO dataset as a logical group (LG) of 4096-byte pages and assigns each dataset page within a logical group an identifier known as a Logical Page Identifier (LPID); and, 2) a Real Storage Manager (RSM) which manipulates page table entries (PTE), external page table entries (XPTE), and page frame table entries (PFTE), and in response to requests made to connect (input) or disconnect (output) VIO dataset pages to or from a VIO buffer which is used by VIO to simulate I/O, by moving data between the VIO buffer and the user's buffer. The LPID is an 8 byte long word comprised of a first 4 byte word LGID (Logical Group ID) assigned by ASM to a Vio Dataset and, a second 4 byte word RPN (Relative Page Number) which is assigned by a Virtual block processor (VBP) to each page of the VIO Dataset.

[0009] As known, the following RSM requests are supported: ASSIGN which is an operation for generally assigning a Vio Dataset Page to virtual storage (hereinafter, a “Vio Window Page”) includes the following variations: an ASSIGN NULL which is a process requested by the VIO facility when a virtual page is required and which assigns the Vio Window Page first reference data (e.g., a page of zeroes); and, ASSIGN NORMAL which is an operation for connecting a VIO dataset page to the VIO buffer, i.e., a Vio Window page 20 gets its data from the Vio dataset page 25, as shown in FIG. 1; MOVE-OUT-NULL (Disconnect) a process for disconnecting the pages in the VIO buffer from virtual storage, i.e., the Vio Window Page is made first reference and the association between the Vio Window Page and the Vio Dataset Page no longer exists; MOVE-OUT (Non-Disconnect) a process for copying the data from the Vio Window Page to the Vio Dataset Page and putting the Vio Dataset Page on Auxiliary storage. The association between the Vio Window Page and the Vio Dataset Page remains intact; MOVE-OUT (Disconnect) is a process for causing a page-out to occur unless the page was stolen, i.e., data is copied from the Vio Window Page to the Vio Dataset Page. The Vio Window Page is made first reference and the association between the Vio Window Page and the Vio Dataset Page no longer exists.

[0010] Certain data such as dataset data that reside in external paging storage is known as “unvirtualized” and in virtual storage memory systems such as the IBM OS/390, requires I/O operations to access this data.

[0011] It would be highly desirable to provide a virtual memory storage system with the ability to keep such unvirtualized data, i.e., VIO datasets, in real storage in order to help reduce the I/O needed to reference this data.

[0012] It is thus an object of the present invention to provide a virtual memory storage system with the ability to keep unvirtualized data in real storage in order to help reduce the I/O needed in order to reference this data.

[0013] When data needs to be written out to a dataset residing in external paging space, no output I/O needs to be performed if the dataset data is kept in real storage. All that is necessary is a copy operation for copying the data from the real frame for the window page to a real frame that represents the dataset page. Additionally, when there is need to retrieve the dataset data into a virtual window, if the dataset page resides in real then there is no need for input I/O. Copying the data from the frame containing the dataset page to the frame that backs the virtual window is all that is needed.

[0014] According to the principles of the invention, there is provided in a computing system implementing a virtual memory system having real memory storage frames for storing virtual pages of data and an auxiliary storage system comprising auxiliary storage slots for storing copies of corresponding virtual pages, wherein I/O mechanisms are employed for effectuating transfer of data between the auxiliary and real storage, a system and method for managing storage of unvirtualized dataset pages destined for auxiliary storage, the method comprising the steps of: moving data from a virtual window in an address space to a virtual input/output (VIO) dataset allocated for storage in a frame in real memory; maintaining a pageable frame queue associated with the address space that owns the VIO dataset; generating a page frame associated with the VIO dataset page and queuing the page frame to the pageable frame queue; and, indicating the frame as including unvirtualized data and one of: marking the page frame for the VIO dataset page as unchanged if there is another copy of the VIO dataset page resides on the Auxiliary storage, or marking the frame as changed if the VIO dataset page is the only copy of the VIO dataset page, whereby the unvirtualized data is efficiently accessible without having to employ I/O mechanisms.

[0015] Advantageously, the system and method of the invention may be used to allow faster access to VIO data set data by allowing that data to be kept in real storage. Thus, when the system is not constrained for real storage VIO dataset pages that are to be written out to the external paging space are kept in real storage.

[0016] Further features, aspects and advantages of the apparatus and methods of the present invention will become better understood with regard to the following description, appended claims, and accompanying drawings where:

[0017] FIG. 1 is a block diagram depicting the operations 10 for moving VIO dataset data into real storage.

[0018] FIG. 2 depicts conceptually the Pageable Frame Queue (PFQ) 45 for the address space 15 that now owns the Vio dataset in Real storage.

[0019] FIG. 3 is a schematic diagram an ASM control sub-system 75 in the IBM OS/390 system.

[0020] This data management system of the invention enables unvirtualized data (such us dataset data that reside in external paging storage) to be kept in real storage. The ability to keep unvirtualized data in real storage allows for faster access to this data, by avoiding the need for I/O. When this data needs to be brought into a virtual storage window for processing, the data is copied from the real storage frame that represents the dataset page into the corresponding frame that backs the virtual window page. The real storage frames that represent unvirtualized data can be stolen according to the frame replacement algorithm implemented by the operating system just like frames that back virtual pages. Additionally, these frames are eligible to be swapped if they are part of the working set of the address space that owns the dataset that these frames represent.

[0021] With reference to FIG. 1, the operations 10 for moving VIO dataset data into real storage are now described. It should be understood that, when the OS/390 system is not constrained for real storage, VIO dataset pages that are to be written out to the external paging space may now be kept in real storage. That is, the Vio Dataset Page 25 may reside in Real storage or Auxiliary storage as directed by a system resource manager (SRM). Particularly, as shown in FIG. 1, a Vio Dataset Page 25 is located on AUX storage 30. An ASSIGN request 26 with the preload option is then issued and, in response, I/O is performed to get the data from the Vio Dataset Page on AUX to the Vio Window Page 20 residing in address space 15. It is understood that data in the Vio Window page 20 is only referenced. Then, a MOVEOUT Disconnect request 27 is issued. According to the invention, the SRM (not shown) directs that the Vio Dataset Page should go to Real storage 40. The ASM (not shown) is queried for the current location of the Vio Dataset Page 25 and ASM returns a LSID indicating that the VIO Dataset Page is on AUX storage 30. A frame is then obtained for the Vio Dataset Page and the window data from the Vio Window is copied to the Vio Dataset Page 25′ now in Real storage 40. Accordingly, as will be described with reference to FIG. 2, a Page Frame Table Entry 50 is initialized for the Vio dataset page. It is understood that PFTE's are created for each dataset page existing in Real storage. Additionally, the frame in real storage 40 that represents the Vio dataset page 25′ is marked as follows: Changed—if there does not exist another copy of the Vio Dataset Page (PFTALSID=0) on external paging space, or Unchanged—if there is exists another copy of the Vio Dataset Page on AUX (PFTALSID is non-zero). Afterwards, the PFTE is then queued on the pageable frame queue (PFQ) of the address space that owns the Vio dataset. The page frame table entry (PFTE) for the frame that represents the VIO dataset page is initialized to indicate that this is a frame that includes unvirtualized data. In the example depicted in FIG. 1, the frame 25′ is NOT marked changed since there exists a copy of the data on AUX storage 30.

[0022] FIG. 2 depicts conceptually the Pageable Frame Queue (PFQ) 45 for the address space 15 that now owns the Vio dataset in Real storage. As shown in FIG. 2, the PFQ 45 now includes a PFTE entry 50 corresponding to the Vio dataset 25′. According to the invention, enhancements are made to the PFTE 50 such that these frames may be distinguished from normal PFTE frames 51 that back virtual data and processed accordingly by functions such as steal and swap. These enhancements include initializing the PFTE entry 50 to include the following fields: a Pftviodp field 52 which indicates that this data is a Vio Dataset PFTE; a Pftlpidp field 54 which is a pointer for pointing to the LPID Block 60 for the Vio Dataset Page; and, a Pftalsid field 56 which is the logical slot identification (LSID) for the auxiliary storage copy of the Vio Dataset Page 25. If no auxiliary storage copy exists, the Pftalsid field 56 is set to zero.

[0023] As further shown in FIG. 2, LPID is stored in a Vio LPID Block 60 which is queued on a double threaded LPID Block queue 70 anchored in the RSM Address Space Block (RAB) 15 viofqptr/viobqptr are respective forward and back pointers for referencing other blocks 60 in the LPID Block queue 70. Further, besides being pointed from the PFTE 50 for a Vio dataset page, it is understood that the LPID Block 60 may still be pointed to by the external page table entry (PTEX) (not shown) for a Vio Window Page for the case when the Window Page contains the same data as the VIO dataset page, i.e., the data in the window has not changed since it was loaded from the VIO dataset page. Vio LPID block is obtained when the following occur: on an ASSIGN request when an LPID block is needed for a Vio Window Page; and, for a MOVEOUT disconnect request when an LPID block is needed for a Vio Dataset Page that resides in Real storage.

[0024] As unvirtualized data is now represented by a frame on a frame queue according to the invention, ASM and RSM are influenced as follows:

[0025] RSM is affected in the manner that any operation that implements processing of the PFQ of an address space requires that Vio dataset PFTEs are treated differently from the PFTEs that back virtual data. The following functions are also heavily affected: Vio Processing, Page Fault, Page Load, Steal (UIC, Pref, Misc.), Swap and Recovery.

[0026] The ASM is able to keep track of the current location of the VIO dataset page by keeping a table indexed by the Logical Page ID for the VIO dataset page. More specifically, as shown in FIG. 3, in the IBM OS/390 system architecture, the ASM control sub-system 75 includes one Logical Group Vector Table (LGVT) 80 that is created during ASM initialization time. When it is created at initialization time 125 LGVTE entries 82 are allocated (e.g., the 124 entries are used, the first header entry is not used), with each entry including a pointer 83 to a Logical Group Entry (LGE) table 85. When the ASM is called to get a LGID during a VIO dataset open time, the LGVT is expanded if all the 124 entries initially allocated are used. Expansion of the LGVT involves obtaining enough storage for the size of the current LGVT plus 128 extra LGVT entries. The old LGVT is copied into the new storage and the old storage is freed. At LGID assign time the LGVTE for the VIO dataset is also initialized. It is understood that the LGID serves as an index into the appropriate LGVTE entry.

[0027] A Logical Group Entry (LGE) 85 is further provided per Logical Group (VIO dataset). Storage is obtained for the LGE and the LGE is initialized when VBP calls ASM to assign a LGID for the VIO dataset. An ASM Page Correspondence Table (ASPCT) 90 is further provided per Logical Group (VIO dataset). Storage is obtained for the ASPCT which includes 256 Logical Page Map Entry (LPME) entries 94 during LGID assign time. The ASPCT header 92 is also initialized at this time. If more LPME entries are needed, than the initial 256 the ASPCT expansion happens when RSM calls ASM to inform ASM of the new location of the VIO Dataset page. The LPME 92 includes the location of the VIO Dataset Page. If the location of the VIO Dataset Page is in a real storage frame, then the LPME, e.g., LPME entry 95, includes a pointer to the PFTE for the VIO Dataset Page. If the VIO Dataset Page is on AUX, then the LPME, e.g., LPME entry 96, includes the Logical Slot ID (LSID) for the VIO Dataset Page. It should be understood that the RPN 98 serves as an index into the appropriate LPME entry. Every time the Real Storage Manager transfers the VIO dataset page either to Aux storage or another frame in Real storage (FIG. 1), ASM is informed of the new location of the VIO dataset page.

[0028] If the system needs to steal the VIO dataset frame, then if the PFTE contains an alternate LSID the ASM is called indicating that the VIO dataset page now resides on AUX storage as indicated by the alternate LSID, and no I/O is needed. If there is no alternate LSID in the PFTE 50 for the VIO dataset page then output I/O is performed and ASM is informed of the new location of the VIO dataset page.

[0029] When the VIO dataset page occupies a real storage frame that is now needed by the system for use by other functions (such as when the VIO dataset page occupies a frame that needs to be taken offline by the system), a VIO dataset frame may be exchanged with a new frame in real, in this case, the ASM is called in order to record the new location in real storage for the VIO dataset data. That is, a VIO dataset page is exchanged with another frame in real, when the VIO dataset page occupies a type of frame that the system needs. In this case, the system exchanges the VIO dataset page with another type of frame of which the system has plenty and takes the old VIO dataset page frame and frees it up for system use. In the example when the system is taking a particular range of real storage frames offline and the VIO dataset page happens to occupy one of these frames, the system exchanges the VIO dataset page frame with another frame in the system that is not being taken offline and returns the old VIO dataset page frame to the system so it may be taken offline. When a Vio dataset page frame is exchanged with another frame, the Vio dataset page data is copied to the new frame, and ASM is informed of the new location in real storage for the VIO dataset data (ASM updates the PFTE pointer in its tables to now point to the new frame for the Vio dataset page).

[0030] The same approach is used for the page load function for VIO window pages. During swap out of an address space VIO dataset pages may be swapped along with the working set for that address space. During swap in, ASM is called for VIO dataset pages so the new location in Real storage for the VIO data may be recorded.

[0031] While the invention has been particularly shown and described with respect to illustrative and preformed embodiments thereof, it will be understood by those skilled in the art that the foregoing and other changes in form and details may be made therein without departing from the spirit and scope of the invention which should be limited only by the scope of the appended claims.