Disk Caching System for Selectively Providing Interval Caching or Segment Caching of Vided Data

a. Field of the Invention

The present invention relates to a movie on demand system of the type wherein multiple clients are serviced by video streams delivered from a central video server.

b. Related Art

In distributed video servers, there may be a large number of local (front-end) nodes, each serving a small number of viewers. To reduce movement of data from the remote (back-end) nodes to the front-end nodes, video files may be cached at the front-end nodes. With a smaller number of viewers per node, a relatively large amount of buffer is required to obtain significant hit ratios. Due to the large price differential between the price of memory and disks, it is preferable to use disks for buffering.

Interval caching is an effective buffering method for video-on-demand servers. However, interval caching to disk may require use of disk bandwidth for writes, reducing the number of streams that can be supported from the disk. Static caching of hot movies on disk does not require any bandwidth but is not as effective as interval caching.

In light of the above, in accordance with the present invention there is provided a system and method for caching sequential data streams in a cache storage device. For each information stream, a determination is made as to whether its data blocks should be discarded from cache as they are read by a consuming process. Responsive to a determination that the data blocks of a stream should be discarded from the cache as they are read by the consuming process, the data blocks associated with that stream are cached in accordance with an interval caching algorithm. Alternatively, responsive to a determination that the data blocks of a stream should not be discarded from the cache storage device as they are read by the consuming process, the data blocks of that stream are cached in accordance with a segment caching algorithm. The present invention allows the cache manager to efficiently provide requested video data to consecutive streams without requiring that the video data be retained after reading by the latter of said consecutive streams.

FIG. 1 is a block diagram of a video-on-demand system according to an embodiment of the present invention;

FIG. 2 is a block diagram of the front-end cache manager of FIG. 1;

FIG. 3 shows the data structures used in the operation of the present cache management method;

FIGS. 4A-4D show the handling, by the cache manager, of a new request for a stream made by a client;

FIGS. 5A-5C show the handling, by the cache manager, of a read request made by a client;

FIGS. 6A-6B show the procedure used by the cache manager for allocating space to a segment; and,

FIG. 7 is shows the processing, by the cache manager, of a stop request from a client.

FIG. 1 is a block diagram of a video-on-demand system according to an embodiment of the present invention. It is assumed that clients 10 make requests from front-end nodes 35 via a communication network 20. Clients can submit start, stop and read requests. The front-end nodes 35 are connected to back-end nodes 40 that contain permanent copies of the videos, by way of a switching network 50. The back-end nodes have disks 55 that are used to store the permanent copies of the videos. The videos are assumed to be made up of segments. The front-end nodes have a front-end disk group 60 that can be used for caching. This disk group is managed by a front-end cache manager 38 (also referred to as the cache manager). Caching conserves server resources since the cached data does not have to be read from the back-end disks, reducing both the needed back-end disk bandwidth and the switching network capacity.

The front-end nodes can be embodied as workstations (e.g. IBM RISC SYSTEM 6000) each having a first network adapter for interconnection with the communication network 20 and a second network adapter for interconnection to the fast switching network 50. The back-end nodes can be also be embodied as workstations or as larger mainframe servers (e.g. an IBM ES/9000) each having a network adapter for interconnection to the switching network 50. The communication network can be embodied as a conventional broadband network. The switching network 50 can be embodied as either a conventional broadband network or as a cluster switch. The back-end disks 55 and front-end disks 60 can be embodied as either stand alone disks or as a striped group of disks such as a RAID-5 array. The front-end cache manager 38 is preferably embodied as program code and data structures instantiated in the memory of each of the front-end nodes.

FIG. 2 is a block diagram of the cache manager 38. The front-end disk group or cache memory 520 is divided into three parts. The segment caching area 530 consists of the blocks belonging to video segments that are or will be completely cached. The interval caching area 540 consists of blocks belonging to video segments that are partially cached. The free pool 545 consists of blocks that are free; i.e. do not contain data from any video. Data requests 570 are received by the cache management program 550 from users. The needed data are retrieved either from the back-end nodes 40 using a remote data stream 510 or a cache data stream 525 from the cache. After transmitting the data via a data stream 580 the cache management program may chose to insert the blocks into the cache, if the data was obtained remotely; or to retain or discard the blocks if they were retrieved from the cache.

FIG. 3 shows the data structures used in the operation of the present cache management method. For managing the free pool 545, each front-end node has a free pool counter 102 that keeps track of the number of unallocated blocks in the free pool. This is initially set to equal the size of the cache memory 520. There is also a free pointer 104 that points to a list of free block entries 106 containing the address of the free block 108 and a pointer to the next free block entry 109. The used BW 110 is the total used bandwidth of the front-end disk group. The threshold 115 is used to determine whether to cache entire segments or fragments of segments. It is set to a pre-determined value close to but less than the maximum bandwidth of the front-end disk group (e.g. 80% of the maximum bandwidth).

The interval list 120 contains a list of the intervals formed by pairs of consecutive streams viewing the same video. The cache manager may decide (as described below) to serve the later stream of the pair from the cache by retaining the blocks read by the earlier stream. The list consists of interval entries 122. The preceding stream 124 of an interval entry 122 is the identifier of the earlier stream of the pair while the following stream 126 is the later stream of the pair. The interval size 128 is the number of blocks read by the preceding stream that need to be retained for use by the following stream and is equal to the distance between the two streams while reading the video file. The allocated field 129 is the number of cache blocks allocated to the interval and the next interval field 130 is a pointer to the next interval entry 122. The interval list 120 is sorted on the interval size 128.

The segment table 130 contains an entry for each video segment consisting of the segment id 131, the segment size 133 which specifies the number of cache blocks required by the segment, the cache status field 132 which specifies whether or not the segment is cached, and statistics 134 such as the inter-reference time which is used to decide whether to retain the segment in the cache. The cache status field 132 may be TRUE if the segment is cached and FALSE otherwise. The stream table 200 contains an entry for each stream. The entry consists of the stream id 205, the stream state 210, the cache access flag 211 of the stream, the video id 215 containing the id of the video being watched by the stream, and the position 220 which contains the position of the stream in the video. The stream state 210 may be RETAIN, indicating that blocks read by the stream are to be retained or DISCARD, indicating the blocks are to be discarded. The cache access flag 211 may be TRUE or FALSE to indicate that the stream is accessing (writing or reading) or not accessing the cache, respectively.

The handling of a new request R is shown in FIGS. 4A-4D. In step 302, the cache manager checks the stream table 200 to see if there is a preceding stream P for the same video. If there is no preceding stream, the cache manager exits in step 304. If there is a preceding stream, then, using the interval caching algorithm, it creates a new interval entry 122 for the interval (P,R) and computes the interval size 128 of the interval (P,R). The interval is inserted into the interval list 120. Next, in step 308, the cache manager compares I with the size of the largest cached interval (Q,S). If I is greater than or equal to the size of (Q,S), the cache manager proceeds to step 340. Otherwise, in step 312, the cache manager deallocates cache from the interval (Q,S) by setting the allocated field 129 in the interval entry to 0, incrementing the free space counter 102 by the size of (Q,S), setting the state of Q to DISCARD and adjusting the used bandwidth 110. In step 318 it allocates cache to (P,R) by setting the allocated field 129 to the interval size, decrementing the free space counter 102 by the interval size, setting the state 210 of P is set to RETAIN, and incrementing the used bandwidth of the cache 110. In step 322, the used bandwidth 110 is compared to the threshold 115. If the used bandwidth 110 is greater or equal to the threshold 115, the cache manager exits in step 324. Otherwise, the cache manager checks in step 326 if there is sufficient cache in the free pool 545 and the segment caching area 530 to cache the interval (Q, S). If there is not sufficient cache, the cache manager exits in step 328. Otherwise, in step 334, the cache manager allocates cache from the free pool 545 and the segment caching area 530 to the interval (Q,S) and exits in step 336.

The cache manager executes step 340 from step 308 if I, the size of interval (P,R) is greater than or equal to the size of the largest cached interval (Q,S). In step 340, the cache manager checks if the used bandwidth 110 is greater than the threshold 115. If it is greater, the cache manager exits in step 342. Otherwise, the cache manager checks if there is enough space in the free pool 545 and the segment caching area 530 to cache the interval (P,R). If sufficient cache is not available, the cache manager exits in step 346. Otherwise, in step 352, the cache manager allocates cache from the free pool 545 and segment caching area 530 to the interval (P,R) by decrementing the free pool counter 110, incrementing the allocated field 129, setting the state of P to retain and updating the used bandwidth 110. Next, the cache manager exits in step 356.

The handling of a read request is shown in FIGS. 5A-5C. In step 402, the cache manager checks if the requested block is in the cache. If it is not in the cache, in step 404 the block is retrieved from the remote server. Otherwise, the block is retrieved from the cache. In both cases, the block is transmitted to the client in step 408. In step 410, the cache manager checks if the state 210 of the stream is RETAIN. If it is, segment caching is used, the block is retained in the cache and the cache manager exits in step 416. Otherwise, with interval caching invoked, in step 420 the cache manager checks to see if the block is the first block of the segment. If not, the cache manager proceeds to step 468. If it is the initial block, the cache manager checks in step 421 if the segment has already been allocated cache storage. If so, it exits in step 423. Otherwise, in step 422 the cache manager executes the allocate space for segment procedure of FIG. 6A-6B to attempt to allocate space for caching the segment. In step 424, the cache manager checks to see if the attempt to allocate space was successful. If not, the cache manager exits in step 426. Otherwise, the block is retained in the cache in step 428 and the cache manager exits in step 430.

Step 468 is executed if the check for the start of the segment in step 420 is unsuccessful. The cache manager checks the segment cache status field 132 in the segment table 130 for the segment to determine if it has any allocated cache. If not, the block is discarded in step 470 and the cache manger exits in step 478. Otherwise, the block is retained in step 472 and the cache manager exits in step 476.

FIGS. 6A-6B show the procedure used by the cache manager for allocating space to a segment. In step 604, the cache manager checks if enough space is available in the free pool 545 for caching the segment by comparing the free pool counter 102 with the size of the segment. If enough space is available, in step 606 the cache manager decrements the free space counter 102 by the size of the segment, sets the segment cache status field 132 to TRUE and exits in step 609. Otherwise, in step 610 it compares the used bandwidth 110 with the threshold 115. If the used bandwidth 110 is less than or equal to the threshold 115, the cache manager executes step 620. Otherwise, it executes step 612 where it checks if sufficient space is available in the free pools and the interval caching area 540 to cache the segment. The space available in the free pool is found from the free pool counter and the space available in the interval caching area 540 is found by summing up the allocated fields 129 of the entries in the interval list. If sufficient space is available, the cache manager releases space from the largest intervals by setting the allocated fields 129 of the corresponding intervals to 0, setting the state 210 of the preceding streams to DISCARD and decrementing the used bandwidth 110. It then executes step 606.

If the test in step 612 fails, the cache manager checks if sufficient space to cache the segment is available in the free pool 545, the interval caching area 540 and colder, less recently accessed segments. If not, the cache manager exits in step 617. Otherwise, the cache manager executes step 618 where space is released as described above from all intervals, and from the colder segments. The cache manager then executes step 606.

If the test in step 610 fails, the cache manager checks if sufficient space to cache the segment is available in the free pool 545 and colder segments. If not, the cache manager exits in step 624. Otherwise, the cache manager releases cache from colder segments in step 622 by setting the segment cache status flag 132 of the corresponding segments to FALSE and incrementing the free space counter. It then executes step 606.

FIG. 7 is a block diagram of the processing of a stop request. In step 804, the cache manager checks the access flag 211 of the stream to decide if the stream was accessing the cache. If not, the cache manager exits in step 808. Otherwise, the used bandwidth 110 is decremented. In step 816, the cache manager checks if the stream is the following stream of an interval. If not the cache manager exits in step 820. Otherwise, the cache manager increments the free pool counter 110 by the size of the interval and deletes the interval entry 122 from the interval list 120. In step 826, the cache manager exits.

Now that the invention has been described by way of the preferred embodiment, various modifications and improvements will occur to those of skill in the art. Thus, it should be understood that the preferred embodiment has been provided as an example and not as a limitation. The scope of the invention is defined by the appended claims.