Three channel cache-coherency socket protocol

[0001] This application claims priority under 35 USC 119 from U.S. Provisional Application Ser. No. 61/551,917 filed on Oct. 26, 2011, titled A THREE-CHANNEL CACHE-COHERENCY SOCKET PROTOCOL by inventor Jean-Jacques Lecler, which is incorporated herein.

[0002] This disclosure is related generally to the field of semiconductor technology and more specifically to systems on chip with cache coherent agents

[0003] In a memory-mapped system initiators such as CPUs, GPUs, or other agents operate by reading and writing bytes of data in a target memory. Some agents are highly penalized by the latency to actually get access to their data. A usual approach is then for them to use a local-transient copy of a subset of the data, named a cache. As soon as a cache is present in a system, the system must ensure that no data is written in the memory target while some henceforth obsolete data still resides in that cache. The mechanism to ensure this property is called cache-coherency.

[0004] In a system where the various logic components may be designed by different teams, potentially in different companies, it is important to formalize the way cache-coherency information is exchanged. On a system on chip, the following protocols include Cache-Coherency: Open Core Protocol (OCP) 3.0, promoted by OCP International Partnership, and Advanced Microcontroller Bus Architecture (AMBA) Advanced eXtensible Interface (AXI) Coherency Extensions (ACE), promoted by ARM. Between integrated circuits, other protocols have been proposed: FSB (Front Side Bus, promoted by Intel), QPI (Intel Quick Path, promoted by Intel), and HyperTransport (promoted by the HyperTransport Consortium).

[0005] Protocols in general and Cache-Coherency protocols in particular have direct or indirect consequences on the logic blocks implementing them. Among other aspects are:

[0006] 1) the amount of logic gates required to fulfill the protocol requirements

[0007] 2) the complexity of this logic, impeding the maximum reachable frequency

[0008] 3) the ability to cope with delays induced by the spreading of the various agents across the silicon die.

[0009] 4) the architectural performance of the system.

[0010] 5) the risk of design bugs

[0011] The present invention is a simplified approach to implementing coherency. It implements a fully cache coherent socket interface using just three channels. It reduces the amount of logic required. The logic is simpler and therefore can run faster. The invention is scalable for large and physically complex systems. It eliminates complex dependencies and thereby improves architectural performance. The invention also is simpler and thereby reduces the risk of design bugs. Overall, the disclosed invention is a coherency protocol that is better suited to modern network-on-chip implementations of connectivity within systems-on-chip.

[0018] Referring now to FIG. 1, a system 10 is shown with an initiator 12 connected to a target 14 through a socket 16 to convey cache-coherency information. Information is conveyed between the initiator 12 and the target 14 through channels. A channel is a unidirectional interface for information transfer in which the sender may send information to the receiver and the receiver is able to stall the sender from sending information.

[0019] The socket 16 comprises several channels, some for which the initiator 12 acts as the sender and the target 14 as the receiver, and some for which the target 14 acts as the sender and the initiator 12 as the receiver. A usual implementation of a channel comprises:

[0020] a Data signal from the sender to the receiver, the signal carrying a word of information;

[0021] a Valid signal form the sender to the receiver, signifying the desire of the sender to convey a word of information; and

[0022] a Ready signal from the receiver to the sender, signifying the willingness of the receiver to accept the a word of information.

[0023] Both the sender and the receiver agree that a word of information is transferred when both Valid and Ready signals are asserted. Others implementation of channels are possible such as implementation in which the receiver indicates to the sender, through credits, an ability to accept information or implementation in which multiple channels are multiplexed on the same physical wires.

[0024] In accordance with one aspect of the present invention, the protocol uses 3 channels. The AXI protocol with ACE extensions uses eight channels (named AR, R, AW, W, B, AC, CR, and CD) and two degenerate channels (RACK, WACK) where the receiver must unconditionally accept the information provided by the sender. The OCP 3.0 protocol uses four channels.

[0025] Referring now to FIG. 2, the disclosed protocol includes three channels. Two channels 20 and 22, named (alpha) α and gamma (γ), are oriented to transfer information from the initiator 12 to the target 14. One channel 24, named beta (β), is oriented to transfer information from the target 14 to the initiator 12. Information passed through the channels is usually framed in packets, the process of which is known as packetization. Packets consist of one or more words of information.

[0026] As in many protocols, the initiator 12 or the target 14 may demand its counter-part to execute some operation, and the counter-part later acknowledges that the operation has actually been executed. Such an exchange is known as a transaction. A transaction consists in a request packet flowing one direction, and a response packet flowing in the other direction. A protocol which traffic is solely consists of transactions offers interesting properties. For example, it is always possible to determine at low-cost if anything is “in flight”, by counting the difference between the number of request packets and the number of response packets seen since the beginning of the world.

[0027] Referring now to FIG. 3, a transaction 30, called a command, includes a request packet sent through channel α20 from initiator to target, and a response packet sent through channel β24 from target to initiator. Transactions wherein the request goes from the initiator 12 to the target 14 (and hence the response from the target 14 to the initiator 12) are called commands.

[0028] Referring now to FIG. 4, a transaction 40, called an intervention, is shown. The request flows from the target 14 to the initiator 12 through channel β24. The response flows from the initiator 12 to the target 14 through channel γ22.

[0029] Referring now to FIG. 5, a transaction 50 is shown. First, a command request from initiator 12 to target 14 is sent through channel α20. Next, in accordance with one aspect of the present invention, a packet is sent from the target 14 to the initiator 12 on the channel β24. The packet comprises at the same time a command response and an intervention request. Finally, an intervention response packet is sent from initiator 12 to target 14 on channel γ22. Such a coupled command and intervention is called a self-intervention.

[0030] In a Cache-Coherent system, the address space is usually divided at a given granularity called a line. The line size is usually a power of 2 successive bytes of the address space. Whereas socket interfaces in general might require support for transaction splitting, transaction ordering, and partial response ordering, none of these are essential to the implementation of systems with caches as initiators. An optimized socket interface, according to the present invention, has simplified logic by virtue of a lack of support for requirements of non-coherent transactions. Embodiments of the invention avoid support of one or more of splitting logic, ordering logic, and partial response ordering logic.

[0031] In accordance with the teachings of the present invention, using a smaller number (just three) of channels has several benefits and advantages over existing standard protocols. Since every channel has an independent flow control, a certain amount a logic and data-buffering must be provided for each channel. This cost is minimized by having fewer channels. Furthermore, packets in a given channel are naturally ordered, whereas packets between channels are susceptible to having logic race conditions. Preventing such race conditions requires greater complexity in higher layer protocols. This is avoided by the ordering ensured by a shared command response and intervention request channel.

[0032] For example, in most cache-coherency protocols the initiator may gain ownership of a line and the target may request that the initiator release this ownership. On ACE, the ownership is transferred through the “R” channel, and released through the “AC” channel. This necessitates a particular rule in the protocol, restricting targets from requesting the release of a line until the target is certain that ownership of the line has reached the initiator; hence the existence of the “RACK” pseudo-channel. As a consequence, an ACE target has to maintain contexts of ownership currently being given to the initiator and refrain from requesting corresponding releases. To operate at full throughput, this table must be able to contain a number of contexts proportional to the round-trip time from the target to the initiator (through R) and back to the target (through RACK). In certain circumstances it may be desirable to introduce a clock domain crossing between the initiator and the target, lengthening this round-trip. Any intervention request on the “AC” channel must first lookup this context table. In some implementation, this necessary lookup may cost a full additional delay cycle.

[0033] According to the OCP 3.0 protocol, ownership is transferred through the command response channel and released through a different intervention request channel. In order to avoid the race, OCP uses self-interventions, where any command emitted by an initiator is actually looped back to itself through the target. Thereby the initiator is able to detect the relative ordering of interventions coming from the rest of the system and its own self-interventions. According to the present invention, the traffic overhead created by OCP 3.0 self-interventions is avoided.

[0034] In OCP 3.0, every command generates a self-intervention for ordering purposes. In the present invention, the relative ordering of command responses and intervention requests make most self-interventions unnecessary. However some self-intervention, as shown on FIG. 5, have interesting properties. For instance, on ACE, an initiator may yield back locally modified (usually called “dirty) data to the target through two different mechanisms, depending whether the transfer is initiated by the initiator or by the target. When initiated by itself, the initiator requests a “WriteBack” command on the AW channel, and later receives a response on the “B” channel. When initiated by the target, the initiator receives an intervention request through the “AC” channel, and responds through the “CR” channel. As the target initiated eviction must exist in a cache coherent system, the present invention reuses this necessary mechanism for initiator initiated evictions. The self-intervention makes this simple: the initiator sends a command requests which demands the target to evict the line from the initiator. The command response from the target is also an intervention request, demanding the eviction of the line.

[0035] In order to avoid the possibility of deadlock, many complex rules that require complex, slow, and mistake prone logic are required to be implemented by designers of logic on the target or the initiator side of socket interfaces with many channel protocols. In the present invention, the information flows through α, β, γ, in that order. With the simple rules that the initiator is not allowed to prevent forward progress on β as a result of flow control on α, and that the target is not allowed to prevent forward progress on γ as a result of flow control on β, the risk of transport-level deadlock is avoided. Furthermore the rules are simple enough to be verified by formal property-checkers.

[0036] It is also desirable in some systems to cascade cache coherent agents. This could be useful to, for example, interconnect multiple on-chip switch fabrics. To do so requires an agent that has an initiator socket interface on one side and a target socket interface on the other side. Such an agent might comprise an internal cache. A cascaded cache coherent system is impractical with the complex ordering rules of conventional protocols. For example, to preserve order rules, a snoop hits following a snoop miss must wait until the snoop miss response is returned. This would be time consuming and detrimental to system performance. The present invention makes practical implementations of cascaded cache coherent sockets.

[0037] Referring now to FIG. 6, a system of four DSP processors 60 are connected to switch fabric 62, which is also connected to RAM 64 and internal cache 66. Internal cache 66 is connected to a second switch fabric 68 which is connected to a system of two CPUs 70 and manages downstream backing store access. Any two connected elements can perform command requests, command response, intervention requests, and intervention responses using the three channel cache coherency protocol.

[0038] In modern systems, the addresses used by the actual initiators are usually not the ones of the physical memory. The addresses used by the initiators are called virtual addresses (VA), the addresses used by the memory are called physical addresses (PA). In a cache-coherent system, the virtual to physical address translation must occur upstream of the cache hierarchy. This is done by a cache of most recent translations called Translation Look-aside Buffer (TLB). When the TLB encounters an access whose translation is unknown, it must request for the translation. Computing the actual virtual to physical translation of a given address requires a number of successive accesses in the memory, in a data structure called the “page table”. As the piece of logic doing that translation walks through the page table, it is called the sometimes called a “Walker”. The shorter each step, the faster the walker operates, thus the walker tends to be located downstream of the cache hierarchy, close to the memory. As TLBs and Walker are distant, they have to communicate. The exchange is sometimes called an Address Translation Service (ATS).

[0039] ATS comprises exchanges where the TLB requests address translation from the walker, and other exchanges where the walker requests the invalidation of some previously established translations. The present invention is able to efficiently transport Address Translation Services, using the commands for translation requests and the intervention for invalidation requests.

[0040] In accordance with various aspects of the present invention, at least one of the describe components, such as the initiator or the target, is an article of manufacture. Examples of the article of manufacture include: a server, a mainframe computer, a mobile telephone, a personal digital assistant, a personal computer, a laptop, a set-top box, an MP3 player, an email enabled device, a tablet computer, a web enabled device having one or more processors, or other special purpose computer (e.g., a Central Processing Unit, a Graphical Processing Unit, or a microprocessor) that is configured to execute an algorithm (e.g., a computer readable program or software) to receive data, transmit data, store data, or performing methods. By way of example, the initiator and/or the target are each a part of a computing device that includes a processor that executes computer readable program code encoded on a non-transitory computer readable medium to perform one or more steps.

[0041] It is to be understood that this invention is not limited to particular embodiments or aspects described, as such may vary. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting, since the scope of the present invention will be limited only by the appended claims.

[0042] Where a range of values is provided, such as the number of channels or the number of chips or the number of modules, it is understood that each intervening value, between the upper and lower limit of that range and any other stated or intervening value in that stated range, is encompassed within the invention. The upper and lower limits of these smaller ranges may independently be included in the smaller ranges and are also encompassed within the invention, subject to any specifically excluded limit in the stated range. Where the stated range includes one or both of the limits, ranges excluding either or both of those included limits are also included in the invention.

[0043] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Any methods and materials similar or equivalent to those described herein can also be used in the practice or testing of the present invention.

[0044] All publications and patents cited in this specification are herein incorporated by reference as if each individual publication or patent were specifically and individually indicated to be incorporated by reference and are incorporated herein by reference to disclose and describe the methods and/or materials in connection with which the publications are cited. The citation of any publication is for its disclosure prior to the filing date and should not be construed as an admission that the present invention is not entitled to antedate such publication by virtue of prior invention. Further, the dates of publication provided may be different from the actual publication dates which may need to be independently confirmed.

[0045] It is noted that, as used herein and in the appended claims, the singular forms “a”, “an”, and “the” include plural referents unless the context clearly dictates otherwise. It is further noted that the claims may be drafted to exclude any optional element. As such, this statement is intended to serve as antecedent basis for use of such exclusive terminology as “solely,” “only” and the like in connection with the recitation of claim elements, or use of a “negative” limitation.

[0046] As will be apparent to those of skill in the art upon reading this disclosure, each of the individual embodiments described and illustrated herein has discrete components and features which may be readily separated from or combined with the features of any of the other several embodiments without departing from the scope or spirit of the present invention. Any recited method can be carried out in the order of events recited or in any other order which is logically possible.

[0047] Although the foregoing invention has been described in some detail by way of illustration and example for purposes of clarity of understanding, it is readily apparent to those of ordinary skill in the art in light of the teachings of this invention that certain changes and modifications may be made thereto without departing from the spirit or scope of the appended claims.

[0048] Accordingly, the preceding merely illustrates the principles of the invention. It will be appreciated that those skilled in the art will be able to devise various arrangements which, although not explicitly described or shown herein, embody the principles of the invention and are included within its spirit and scope. Furthermore, all examples and conditional language recited herein are principally intended to aid the reader in understanding the principles of the invention and the concepts contributed by the inventors to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, aspects, and embodiments of the invention as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents and equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure. The scope of the present invention, therefore, is not intended to be limited to the exemplary embodiments shown and described herein. Rather, the scope and spirit of present invention is embodied by the appended claims.