Bandwidth optimization cache

[0003] 1. Field of the Invention

[0004] The present invention relates to bus bandwidth utilization. More particularly, the present invention relates to optimizing bus bandwidth utilization in a high-latency, bursty bus environment by using a prefetch cache to hold data for multiple small-sized data request.

[0005] 2. Background

[0006] In a high bus transaction environment where read accesses range from single word to multiple-word burst accesses it becomes important to optimize memory bus bandwidth utilization. Each read access requires some form of handshaking between a memory bus interface and a device sought to be accessed that are attached to the memory bus, such as a memory controller servicing a memory store. Each read access incurs a bandwidth overhead cost due to handshaking because handshaking consumes bus bandwidth that would have been otherwise available for transmitting access requests from other clients or additional data. Thus, single word accesses are more inefficient than multiple-word burst accesses because less data are transferred for a given bandwidth consumed.

[0007] Inefficient single word accesses take away bus time from other clients who may access the bus more efficiently such as clients that can perform large data access per request-grant handshake, further decreasing the memory bus interface's ability to provide optimum utilization of available memory bus bandwidth.

[0008] Moreover, in an environment where a memory controller servicing the data access through a memory bus needs to service high latency devices, such as hard drives, so that data may be transferred to another location through a high speed serial bus, such as a fibre channel bus, it becomes even more important that data accesses through the memory bus are performed efficiently.

[0009] Accordingly, it would be desirable to handle memory bus access requests of various sizes and without restricting clients to only performing large data fetches, while maintaining optimum utilization of the memory bus bandwidth.

[0010] The present invention optimizes bus bandwidth utilization in an environment where bus accesses range in size from single word to multi-word burst accesses by prefetching and caching additional words when responding to a single word access request so that if the word is not found in cache, the single word request is converted into a multi-word fetch request. The additional words in the fetch request are taken from the same line in memory in which the single word resides and stored in cache, increasing the chances that the next single word access by the client results in a cache hit. This optimizes the utilization of the bus bandwidth because the number of fetches through the memory bus is reduced through prefetching and when prefetching does occur, additional words besides the word requested are fetched.

[0011] In a preferred embodiment of the present invention, the method includes receiving a read request from a client; checking the contents of a cache to determine whether the cache contains information sought in the read request and returning the information from the cache if a cache hit results, If a cache miss results, a bus transaction is initiated by fetching a block of memory containing the information from a memory store, sending the information to the client, and caching the additional remaining information included with the fetched block of memory.

[0012] The bus transaction may further include synchronizing clock signals and handshake signals driving the cache with the clock signals and handshake signals driving to the bus and the memory store. Moreover, the method may further include maintaining cache consistency in the cache, while synchronization is being performed.

[0013] The method may be implemented using a bus interface having a cache, and which is responsive to a read request received from the client. The bus interface is coupled to the client and to a bus, the cache returning information from the cache if a cache hit otherwise the bus interface fetching a block of memory containing the information from a memory store through the bus, sending the information to the client, caching remaining information within the fetched block of memory that was not requested in the read request, if a cache miss results in response to the read request.

[0014] The implementation of the method may further include a synchronization devices for synchronizing clock signals and handshake signals driving the cache with clock signals and handshake signals driving the bus and the memory store. Moreover, the implementation may include a means for maintaining cache consistency in the cache, while the synchronization means is synchronizing the cache with the bus and memory store.

[0015] FIG. 1 is a block diagram depicting an apparatus and system that optimizes bandwidth utilization in a variable fetch data size capable memory bus in accordance with a preferred embodiment of the present invention.

[0016] FIG. 2A is a block diagram of devices for maintaining control signal synchronization between a memory bus interface and a memory bus in accordance with a preferred embodiment of the present invention.

[0017] FIG. 2B is a block diagram of devices for maintaining data signal synchronization between a bus interface and a memory bus in accordance with a preferred embodiment of the present invention.

[0018] FIG. 3 is a process diagram outlining a method of optimizing memory bus bandwidth utilization in accordance with a preferred embodiment of the present invention.

[0019] FIG. 4 is timing diagram of a various signals generated by the system employing a memory bus interface in accordance with a preferred embodiment of the present invention.

[0020] In the following description, a preferred embodiment of the invention is described with regard to preferred process steps and data structures. Those skilled in the art would recognize after perusal of this application that embodiments of the invention can be implemented using one or more general purpose processors operating under program control, or special purpose processors adapted to particular process steps and data structures, and that implementation of the process steps and data structures described herein would not require undue experimentation or further invention.

[0021] Apparatus and System Architecture

[0022] FIG. 1 is a block diagram depicting an apparatus and system that optimizes bandwidth utilization in a variable fetch size capable memory bus in accordance with a preferred embodiment of the present invention.

[0023] The apparatus includes memory bus interface 10 that uses a cache 12 to reduce the number of small-sized fetches through memory bus 14 that result from small-sized access requests 16 made by client 18. Memory bus interface 10 responds to small-sized access requests 16 by determining whether cache 12 can provide the information requested and if so (i.e., a cache hit occurs), provides the information from cache 12 to client 18.

[0024] If instead a cache miss results, memory bus interface 10 performs a bus transaction that includes fetching from memory store 20, which is serviced by memory controller 22, the information requested plus additional information that belongs on the same line of memory in memory store 20 as the information requested. Upon reception of the fetched information through memory bus 14, the information requested is received from memory bus 14 and sent directly to client 18 without caching. The additional information within the fetched block of information is stored in cache 12. Due to spatial locality, the next information requested by client 18 is likely to be in cache 12.

[0025] In accordance with a presently preferred embodiment of the present invention, a LSI-P-3544 small-sized fetch is limited to fetches that retrieve a single word of information. Similarly, a small-sized access request is defined as a request for a single word of information that corresponds to a given address. A large-size fetch is limited to fetches that retrieves information through memory bus 14 through a multi-word burst access. Also, cache 12 is a 16 words×32 bits fully-associative cache that is content addressable.

[0026] As shown in FIGS. 1-2B, in an alternative embodiment of the present invention, memory bus interface 10 may further include synchronization devices 28 that synchronizes the clock signals and handshake signals driving the devices within interface domain 24, which includes cache 12, with the clock signals and handshake signals driving devices within bus domain 26, which includes memory bus 14, memory controller 22, memory store 20, and the like. Flip-flops are used as the synchronization devices 28 for synchronizing the clock signals and handshake signals that drive devices in interface domain 24 with the clock signals and handshake signals that the devices in bus domain 26, as shown in FIG. 2A-2B. Synchronization using flip-flops is not meant to limit the invention in any way. Other synchronization implementations may also be used in lieu of the approach shown in FIGS. 2A-2B, as known by one reasonably skilled in the art.

[0027] FIG. 3 is timing diagram of a bus transaction in accordance with a preferred embodiment of the present invention.

[0028] Referring to FIGS. 1 and 3, a bus transaction is initiated after a request-grant handshake is completed between memory bus interface 10 and memory controller 22, or an equivalent device coupled to memory bus 14. Request 30 is generated by memory bus interface when requesting a fetch from memory store 20, or equivalent device, through memory controller 22. When ready and after some latency 32 due to synchronization, as discussed above with reference to FIG. 2A, memory controller responds with grant signal 34. Note that latency 32 can also include latency introduced by another client accessing memory store 32 during the same time the fetch request occurs.

[0029] The assertion of grant signal 34 results in a ready signal 36 being generated by memory controller 22, or equivalent device. Ready signal 36 is then synchronized as discussed above with reference to FIG. 2B, resulting in some latency 38 due to synchronization between interface domain 24 and bus domain 26 and memory latency. Latency 32 and latency 38 define the bandwidth overhead incurred for every bus transaction performed on memory bus 14. After synchronization is completed, a write enable signal 40 is generated resulting in the transfer of information signals 42 from memory store 20 to cache 12 via memory bus 14.

[0030] In another alternative embodiment of the present invention, memory bus interface 10 also includes a means for maintaining cache consistency in the cache, while synchronization devices 28 are synchronizing the devices within interface domain 24 and bus domain 26. For example, a cache consistency protocol may be followed that checks the current contents of the cache for consistency with the expected data and invalidates old data in the cache that has been replaced by any new data received.

[0031] Returning to FIG. 1, memory bus interface 10 is shown integrated with a system 44 for transferring information through memory bus 14, such as data and control signals, between high latency memory devices 46, such as devices including hard drives that may be buffered (via memory store 20) or unbuffered, and a high speed serial bus 48, such as a fibre channel bus. System 40 also includes a processor 50 running under program control that is treated by memory controller 22 as another requesting client. Memory bus interface not only services small-sized access requests from client 18, but may also service large-sized information requests from clients making multi-word burst access requests, such as transmission buffer 22 and receiving buffer 24 that are managed through DMA 26 for transferring information between memory bus 14 and high speed serial bus 48, such as a fibre channel bus. The fetches resulting from the multi-word burst access requests are not prefetched or cached by memory bus interface 10.

[0032] Method of Operation

[0033] FIG. 4 is a process diagram outlining a method of optimizing memory bus bandwidth utilization in accordance with a preferred embodiment of the present invention.

[0034] At step 60, memory bus interface waits for a small-sized read request from a client In accordance with a preferred embodiment of the present invention, the small-sized read request includes a single word address.

[0035] At step 62, if a small-sized read request is received, step 64 is performed. Otherwise step 60 is performed.

[0036] At step 64, the contents of a cache is checked to determine whether the cache contains the information requested.

[0037] At step 66, if the information requested is in cache 12 (a cache hit occurred), step 68 is performed. Otherwise step 70 is performed.

[0038] At step 68, the information is retrieved from the cache and forwarded to the client and memory bus interface 10 waits for another small-sized read request by performing step 60.

[0039] If at step 66, the information requested is not in cache 12 (a cache miss occurred), step 70 is performed.

[0040] At step 70, memory bus interface 10 initiates a bus transaction by fetching a block of memory containing the information requested from memory store 20, sends the information to client 18. Memory bus interface 10 also stores the remaining information within the fetched block of memory that was not requested in cache 12. In accordance with a preferred embodiment of the present invention, the block of memory fetched is eight words long including the information requested.

[0041] In an alternative embodiment of the present invention, step 70 may further include a step 70a. At step 70a, synchronizing devices 28 synchronize the clock signals and handshake signals that drive the devices within interface domain 24 with the clock signals and handshake signals that drive the devices within bus domain 26.

[0042] In another alternative embodiment of the present invention, step 72 may also be included. At step 72, cache consistency is maintained in cache 12, while synchronizing the devices in interface domain 24 and bus domain 26. This further optimizes bandwidth utilization of memory bus 14 because step 72 is performed at a time where memory bus interface 12 would otherwise be precluded from receiving data.

[0043] While embodiments and applications of this invention have been shown and described, it would be apparent to those skilled in the art that many more modifications than mentioned above are possible without departing from the inventive concepts herein. The invention therefore, is not to be restricted except in the spirit of the appended claims.