Snoop filter line replacement for reduction of back invalidates in multi-node architectures

[0001] This is a continuation of U.S. patent application Ser. No. 09/606,848, filed on Jun. 28, 2000, which is incorporated herein by reference.

[0002] The present invention pertains to snoop filters used in multi-processor systems. More particularly, the present invention relates to a technique for replacing cache line entries in a snoop filter so as to reduce back invalidates in a multi-node architecture.

[0003] A snoop filter is a device used to reduce bus traffic in certain computer systems, particularly multiple-processor (multi-processor) systems. In a multi-processor system, the snoop filter generally forms part of an interface between two or more nodes. Each node contains one or more processors, a node controller, and memory, including one or more levels of cache memory associated with each processor. The snoop filter is essentially a specialized cache for tracking cache coherency state information relating to the cache memories of the processors. The snoop filter keeps track of the coherency state of each cache line of each of the processors. The state information is used by the snoop filter to decide which bus transactions received from the various nodes need to be passed on to other nodes in the system. The snoop filter filters unnecessary bus transactions by preventing them from reaching those nodes for which they are not needed. Hence, a snoop filter can have a dramatic positive impact on the overall system performance by reducing bus traffic.

[0004] FIG. 1 shows an example of a four-node, eight-processor system 1. The four nodes 2 (node0, node1, node2, and node3) are coupled to each other through a multi-node interface 10, which includes a snoop filter 5 and a switch 9. The switch 9 controls the routing of communications traffic between nodes 2. Each node 2 includes two processors 3 coupled on a local bus 8. The two processors 3 of each node 2 are also coupled to a Random Access Memory (RAM) 4 of that node through a node controller 6. The node controller 6 of each node 2 is coupled to the snoop filter 5. In addition, associated with each processor 3 is a cache memory (or cache) 7. The caches 7 may be located within their respective processors 3, or they may be separate from but coupled to their respective processors 3 (e.g., off-chip or outside the processor but on the same chip).

[0005] Now consider a simple example of how the snoop filter 5 conserves bus bandwidth for the four-node system. Assume that a particular cache line, address A, is present only in one node of the system, node0. If a processor 3 in node3 wants to write to this cache line, the request first comes to the snoop filter 5, and the snoop filter 5 will send an invalidation request only to node0, since that cache line is only resident in node0. The request is filtered, i.e., not forwarded to the two remaining nodes, node1 or node2, eliminating unnecessary transactions on the local buses 8 of node1 and node2.

[0006] Because the cache lines of each processor 3 have an entry in the snoop filter 5, the snoop filter 5 will send a back invalidate when it is forced to replace a snoop filter entry that corresponds to a valid entry in some processor's cache 7. A back invalidate is simply a signal to one or more nodes 2 to invalidate, in their caches 7, the line that has been replaced in the snoop filter. Continuing the example above, if the entry corresponding to cache line address A is replaced, a back invalidate signal is sent to node3 before the snoop filter entry is allocated to a new cache line. When a node 2 receives a back invalidate signal, the node 2 marks that cache line as invalid. If another access is made to the cache line that was invalidated, a miss will occur and the accessing processor 3 will be forced to send a bus request to re-read the line.

[0007] In this way, back invalidates from a snoop filter increase bus traffic and can cause cache misses. Known snoop filter replacement methods do not address this problem adequately. A conventional method for choosing a replacement line in a snoop filter is to use a temporal-based replacement algorithm such as Least Recently Used (LRU). In LRU, the oldest entry is chosen for replacement based on the premise that if the line has not been accessed recently, it is unlikely to be accessed in the near future. Other temporal-based algorithms, such as Pseudo-LRU (PRLU) or First In First Out (FIFO), work in a similar manner but are less expensive to implement. However, known temporal-based algorithms suffer from the same limitation, i.e., the lack of temporal information available to the snoop filter. This lack of temporal information stems from the fact that the snoop filter is only updated on cache misses. The snoop filter is unaware of cache hits and thus receives only a fraction of the temporal information available in the cache. Perhaps more importantly, the temporal correlation between the access streams from different processors in a multi-node architecture is weak at best.

[0008] The present invention is illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements and in which:

[0009] FIG. 1 is a block diagram of a multi-processor system that includes a snoop filter;

[0010] FIG. 2 is a block diagram of a snoop filter;

[0011] FIG. 3 shows the format of a snoop filter entry;

[0012] FIG. 4 illustrates the manner of selecting a snoop filter entry for replacement, using the Reduce Back Invalidates (RBI) technique, for a four-way associative snoop filter;

[0013] FIG. 5 shows a hardware implementation of the RBI technique for a four-way associative snoop filter; and

[0014] FIG. 6 shows an example of applying the RBI technique in a four-way, four-node system.

[0015] A method and apparatus for replacing entries in a snoop filter are described. Note that in this description, references to one embodiment or an embodiment0 mean that the feature being referred to is included in at least one embodiment of the present invention. Further, separate references to one embodiment in this description do not necessarily refer to the same embodiment; however, neither are such embodiments mutually exclusive, unless so stated and except as will be readily apparent to those skilled in the art.

[0016] As described in greater detail below, a snoop filter in a multi-processor system is configured to reduce the number of back invalidates that would otherwise be required, thereby reducing bus traffic and cache misses. The replacement technique is therefore referred to hereinafter as the Reduce Back Invalidates (RBI) technique. The snoop filter maintains a plurality of entries, each representing a cache line that may be owned by one or more nodes. When replacement of one of the entries is required, the snoop filter selects for replacement the entry representing the cache line owned by the fewest nodes, as determined from a presence vector in each of the entries. A temporal based algorithm may be used to refine the selection if more than one cache line is owned by the fewest number of nodes. That is, a greater or fewer number of nodes and/or processors-per-node may be used.

[0017] A snoop filter employing the RBI replacement technique can be used in a multi-processor system such as shown in FIG. 1. Note the processors 3 may be conventional general-purpose, programmable microprocessors. Alternatively, the processors 3 may be any of various other types of processing device, such as programmable Digital Signal Processors (DSPs) or Application Specific Integrated Circuits (ASICs). The caches 7 referred to herein may be assumed to be off-chip (e.g., L3) caches. In other embodiments, however, the caches 7 may be on-chip (e.g., L1 or L2) caches. In addition, note that a multi-processor system that incorporates the RBI replacement technique does not have to be a four-node system, nor is a node required to have two processors for use with the RBI technique.

[0018] FIG. 2 is a block diagram of a snoop filter 20 that may employ the RBI replacement technique. The snoop filter 20 includes a control circuit 21 and memory 23 coupled to the control circuit 21. Memory 23 may be, for example, Content-Addressable Memory (CAM). Memory 23 stores a number of cache line entries 22, each representing a cache line owned by (resident in the caches of) zero or more of the nodes in the system. Note that ownership of a cache line, as the term is used herein, does not necessarily imply exclusive ownership by one node; that is, a cache line may be owned by multiple nodes. Similarly, an invalid cache line is considered to be owned by zero nodes. The control circuit 21 is configured to control the filtering of bus transactions based on the entries 22, as well as to control transmission of invalidates and back invalidates and replacement of snoop filter entries 22 when required.

[0019] FIG. 3 shows the format of each snoop filter entry. The snoop filter includes essentially a directory of which nodes have a copy of a particular cache line. In one embodiment, the snoop filter operation is based on the Modified, Exclusive, Shared and Invalid (MESI) coherency protocol. As shown in FIG. 3, each entry includes a tag field 31, an exclusive bit 32, and a presence vector 33. The tag field 31 is an address tag that identifies a particular cache line. The presence vector 33 indicates in which nodes a particular cache line is present. In one embodiment, the presence vector includes one bit for each of the nodes, and the value of each bit indicates whether the cache line is owned by the represented node. The exclusive bit 32 indicates if the cache line is owned exclusively by a single node. In addition, the snoop filter may include LRU information for each snoop filter set.

[0020] Referring back to FIG. 1, consider again the example of writing to a cache. Assume again that a particular cache line, address A, is present only in node0, and that the presence vector for that cache line entry in the snoop filter is, therefore, 0001. If a processor in node3 now wants to write to this cache line, the snoop filter will send an invalidation request only to node0, since the presence vector indicates that cache line A is only resident in that node. The request is not forwarded to node1 or node2. After the transaction has completed, the presence vector is updated to 1000, for example, to indicate that only node3 has the line cached.

[0021] As noted above, temporal based snoop filter line replacement techniques, such as LRU or PLRU, have proven to be less than optimal in terms of reducing bus traffic and cache misses. Rather than simply choosing the oldest entry for replacement, a more optimal replacement technique is to choose the entry that is owned by the least number of nodes, or in other words, to choose the entry that has the least number of bits set in the presence vector. This approach, the RBI approach, reduces the number of back invalidates that must be sent back to the processors when the entry is replaced. This has at least two positive impacts on system performance. The first is reduction in bus bandwidth consumption. By reducing the number of back invalidates, the bus traffic due to back invalidates is reduced and bus bandwidth is conserved. The second is reduction in cache miss rate. By reducing the number of back invalidates, the number of cache lines that are invalidated due to back invalidates is reduced. This results in a lower cache miss rate, which also has the secondary effect of further reducing the required bus bandwidth.

[0022] To improve the selection of an entry for replacement, an additional selection stage that reduces back invalidates is added to the snoop filter replacement selection hardware (in control circuit 21) to improve system performance. FIG. 4 illustrates the selection of an entry to be replaced for a four-way associative snoop filter. The nth set 42 is depicted, which contains four ways (entries or lines), i.e., way0, way1, way2, and way3. An RBI Way Selection stage 43 is placed before a temporal-based way selection stage 44. The RBI Way Selection stage 43 contains the additional hardware and/or software required to implement the RBI replacement line selection method. The RBI Way selection stage 43 may comprise programmable logic and/or hardwired-circuitry or a combination thereof, which may embody combinational logic and/or sequential logic.

[0023] When an entry in the snoop filter must be replaced, the index from the new address 41 (i.e., the address that will be used to replace the entry in the snoop filter) is used to select the set number within the snoop filter. If the snoop filter is associative, one of the entries within the set must be chosen for replacement. In the RBI method, the entry or entries with the least number of presence vector bits that are set (in the selected set) are selected for replacement by stage 43. If a tie occurs between multiple entries, then in one embodiment, a simple temporal-based selection algorithm is applied by stage 44 to pick a single line. Because temporal information is so weak in the snoop filter, a less-expensive algorithm, such as a random or FIFO selection, can even be used to break ties. Alternatively, any of various other techniques may be used to handle a tie between multiple entries, such as the techniques described below.

[0024] For the four-way associative snoop filter of FIG. 4, the RBI selection stage 43 has an output signal for each of the four entries in the set, i.e., signals W0 to W3. The output signal for a particular entry is active if the line has a presence vector with the fewest bits set. If VX represents the presence vector for a given way, WayX, and VX is the number of presence vector bits that are set for WayX, then the state of signal W0 may be determined according to the following pseudo-code (where represents not equal to):

[0025] If (V00) OR

[0026] ((V01) AND (V10) AND (V20) AND (V30)) OR

[0027] ((V02) AND (V12) AND (V22) AND (V32)) OR

[0028] ((V03) AND (V13) AND (V23) AND (V33)) OR

[0029] ((V04) AND (V14) AND (V24) AND (V34)))

[0030] Then W0On

[0031] Else W0Off

[0032] The manner of determining the state of signals W1 to W3 is similar, as will be readily apparent to those skilled in the art. Using the information on potential candidates for replacement that is provided by the W0 to W3 output, the temporal-based selection stage 44 breaks ties among multiple lines and provides a single entry number for replacement.

[0033] FIG. 5 shows an implementation of this technique, where V0i (1i4) represents the presence bit of Way 0). In FIG. 5, the Programmable Logic Array (PLA) block 51 is a simple, 2-stage combinational circuit which implements the logic to generate the seven outputs as a function of the four inputs corresponding to the presence vector for each way. The outputs W1, W2, and W3 are not shown, but they can be generated in the same manner as W0. Using the information on potential candidates for replacement that is provided by the W0 to W3 outputs, the second stage of selection, i.e., the temporal-based selection stage, breaks ties among multiple lines and provides a single line number for replacement. Note that while FIG. 5 shows a combinational logic implementation, the RBI technique can alternatively be implemented using sequential logic.

[0034] Assuming a four-way snoop filter and a four-node multiprocessor system (four nodes requires four bits for each presence vector, i.e. one bit to track the state of the line for each node), FIG. 6 shows an example of how the RBI replacement technique works. Assuming the presence vectors 51 are in the state shown in FIG. 6, the RBI selection stage 43 would turn on signals W1 and W3, because way1 and way3 each have only one presence vector bit set, and there are no other presence vectors with fewer bits set. Thus, way1 and way3 are chosen as potential replacement candidates, since replacement of one of these two lines would require the least number of back invalidates to be issued to the system. The temporal-based selection stage 44 then chooses between way1 and way3 using any number of conventional temporal-based selection techniques, such as LRU or PLRU. In this example, the temporal-based selection stage 44 chooses to replace the third line.

[0035] Without the RBI selection stage 43, any of the four different entries may have been chosen for replacement. For example, if way2 was the oldest entry and was chosen for replacement by a temporal-based selection method, back invalidate transactions would need to be sent to all four nodes of the system (since its presence vector has all four bits set). This would result in four times as many bus transactions and cache line invalidations as choosing way1 or way3, which require only one back invalidate for replacement.

[0036] As noted above, the RBI technique is not restricted to use in a four-node system. For example, the RBI selection stage 43 can easily be implemented for more than four nodes. Hence in a four-way, eight-node system, the state of signal W0 may be determined according to the following pseudo-code:

[0037] If ((V00) OR

[0038] ((V01) AND (V10) AND (V20) AND (V30)) OR

[0039] ((V02) AND (V12) AND 93 V22) AND (V32)) OR

[0040] ((V03) AND (V13) AND (V23) AND (V33)) OR

[0041] ((V04) AND (V14) AND (V24) AND (V34)) OR

[0042] ((V05) AND (V15) AND (V25) AND (V35)) OR

[0043] ((V06) AND (V16) AND (V26) AND (V36)) OR

[0044] ((V07) AND (V17) AND (V27) AND (V37)) OR

[0045] ((V08) AND (V18) AND (V28) AND (V38)))

[0046] Then W0On

[0047] Else W0Off

[0048] The manner of determining the state of the remaining signals WX is similar. It will be apparent that the benefit achieved by using the RBI technique increases as the number of processors in the system increases and/or the degree of sharing within the application increases.

[0049] The basic RBI technique relies on the information contained in the presence vectors to replace entries in the snoop filter. Numerous variations of this technique, especially in combination with the coherence state of the particular cache line, are possible. Four variations are now described. As indicated above, these variations may also be used to handle the case of a tie between multiple cache line entries (in terms of their node ownership).

[0050] In the first variation, in the case in which there is a tie with only a single presence bit set, the RBI technique will not choose any entry which has its cache line in the Modified state (i.e., Exclusive is set). With this variation, a possible costly writeback to the memory is avoided.

[0051] Another variation arises when there is a tie with one or more presence bits set: the RBI algorithm first attempts to back invalidate a line whose home node is closest to the processor where the line currently exists. This variation reduces the memory latency when the processor requests the line that had been back invalidated.

[0052] Both of the above variations may require extra hardware to incorporate the extra logic. A variation which requires less hardware is next considered. With a large number of nodes, for example, implementation of the RBI illustrated in FIGS. 5 and 6 could be expensive. In such a case, a less expensive, approximate form of the selection code can be used with a negligible performance loss. In the eight-node example, the logic could be significantly simplified by combining adjacent ranges of values of VX as follows (again, for determining the state of signal W0):

[0053] If ((V00) OR

[0054] ((V01 or 2) AND (V10) AND (V20) AND (V30)) OR

[0055] ((V03 or 4) AND (V13) AND (V23) AND (V33)) OR

[0056] ((V05 or 6) AND (V15) AND (V25) AND (V35)) OR

[0057] ((V07 or 8) AND (V17) AND (V27) AND (V37)))

[0058] Then W0On

[0059] Else W0Off

[0060] Again, the manner of determining the state of the remaining signals WX is similar. The RBI selection stage 43 can also be implemented for more or less than four-way associativity by simply changing the number of terms. For example, the following pseudo-code shows the manner of determining signal W0 for a five-way, four-node system (the V4 terms have been added):

[0061] If ((V00) OR

[0062] ((V01) AND (V10) AND (V20) AND (V30) AND (V40)) OR

[0063] ((V02) AND (V12) AND (V22) AND (V32) AND (V42)) OR

[0064] ((V03) AND (V13) AND (V23) AND (V33) AND (V43)) OR

[0065] ((V04) AND (V14) AND (V24) AND (V34) AND (V44)))

[0066] Then W0On

[0067] Else W0Off

[0068] The manner of determining the state of the remaining signals WX is similar.

[0069] Thus, a method and apparatus for replacing entries in a snoop filter have been described. Although the present invention has been described with reference to specific exemplary embodiments, it will be evident that various modifications and changes may be made to these embodiments without departing from the broader spirit and scope of the invention as set forth in the claims. Accordingly, the specification and drawings are to be regarded in an illustrative sense rather than a restrictive sense.