System and method for time multiplexing of multi-domain transactions

[0001] 1. Field of the Invention

[0002] This invention relates generally to the field of computer systems and, more particularly, to communication between domains in a computer system.

[0003] 2. Description of the Related Art

[0004] Modern computer systems may include many clients, such as microprocessors or device controllers, that perform many functions. These clients may share resources in the computer system and a group of clients may be connected to a resource over the same physical link. The clients or a central arbitrator may arbitrate transactions on the physical link to allow each client to communicate with a resource in turn.

[0005] Depending on the needs of a computer system, clients may be grouped together to operate as a domain. Generally speaking, a domain may comprise a group of clients that are configured to operate as if they were a single system. A client within a domain may communicate with other clients in its domain by conveying transactions to and receiving transactions from the other clients. These transactions may be conveyed on the same physical links as those from other clients that may belong to other domains.

[0006] At times, a client in a first domain may generate a failing transaction and may convey the failing transaction along a physical link that is shared with a client or clients in a second domain. Since the clients in the second domain are connected to the physical link, they may receive the failing transaction from the client in the first domain and may become corrupted. Once the clients in the second domain become corrupted, they could potentially corrupt clients in other domains such that the failing transaction propagates through the system and causes the system to crash. It would be desirable to keep a failing transaction in one domain from corrupting other domains. Accordingly, a system and apparatus for preventing a failing transaction from one domain from corrupting another domain is needed.

[0007] The problems outlined above are in large part solved by the use the system and apparatus described herein. Generally speaking, a system and apparatus for time multiplexing of multi-domain transactions is provided. A computer system may include multiple domains of clients where the domains share common physical links. The bandwidth on a physical link may be divided between the domains that use that link such that a domain may only convey and receive transactions specific to that domain during the time period allocated to it on that physical link. A counter may be used to partition the bandwidth of a physical link between the domains that use that physical link.

[0008] In one embodiment, a system may include a first domain of clients and a second domain of clients coupled to a central repeating level. The central repeating level may be configured to receive and convey transactions that correspond to the first domain of clients during a first time period and receive and convey transactions that correspond to the second domain of clients during a second time period. The central repeating level may include a counter such that the first time period may correspond to a first plurality of values of the counter and the second time period may correspond to a second plurality of values of the counter. The central repeating level may be coupled to additional central repeating levels that may be coupled to additional domains.

[0009] A central repeating level may include a receive queue, a transmit queue, a counter, and a control unit. The control unit may be configured to permit transactions that correspond to a first plurality of clients to be received into the receive queue and conveyed from the transmit queue in response to a first set of values of said counter. The control unit may also be configured to permit transactions that correspond to a second plurality of clients to be received into the receive queue and conveyed from the transmit queue in response to a second set of values of said counter. The control unit may be configured to inhibit transactions that correspond to the first plurality of clients from being received into the receive queue and conveyed from the transmit queue in response to the second set of values of the counter. The control unit may also be configured to inhibit transactions that correspond to the second plurality of clients from being received into the receive queue and conveyed from the transmit queue in response to the first set of values of the counter. The control unit may also be configured to select the first set of values of the counter and the second set of values of the counter according to a programmable input.

[0017] While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the drawings and detailed description thereto are not intended to limit the invention to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present invention as defined by the appended claims.

[0018] Turning now to FIG. 1, a block diagram illustrating one embodiment of a system for time multiplexing of multi-domain transactions. Other embodiments are possible and contemplated. FIG. 1 depicts central repeating level (CRL) 110, CRL 112, CRL 114, CRL 116, CRL 120, CRL 122, CRL 130. FIG. 1 also depicts client 102a, client 102b, client 104a, client 104b, client 106a, client 106b, client 108a, and client 108b. CRLs 110,112,114,116,120, 122, and 130 can be referred to individually as a ‘CRL’ or collectively as ‘CRLs’. Similarly, clients 102a, 102b, 104a, 104b, 106a, 106b, 108a, and 108b can be referred to individually as a ‘client’ or collectively as ‘clients’. FIG. 1 highlights one possible interconnection among the CRLs and clients. Other interconnections are possible and contemplated.

[0019] In the system of FIG. 1, the CRLs may be configured to convey transactions between the clients. The CRLs may also be configured to convey transactions between themselves. For example, client 102b can communicate with client 104b by conveying a transaction to CRL 112. CRL 112 can convey the transaction to CRL 120 which, in turn, can convey the transaction to CRL 130. CRL 130 can then convey the transaction to CRL 122, CRL 122, in turn, can convey the transaction to CRL 114 and CRL 114 can convey the transaction to client 104b. The other clients and CRLs in FIG. 1 can communicate in a similar manner. These transactions may include data transfer transactions, coherency transactions, and other operations where communication between the CRLs and/or clients occurs. The transactions may be generated by the CRLs and/or clients according to a communications protocol.

[0020] The clients of FIG. 1 can be configured to operate as one or more domains. Broadly speaking, a domain may be defined as a group of clients that are configured to operate as if they comprise a single system. Clients in a domain may share the same memory and may communicate with other domains using agents

[0021] Clients from different domains may convey transtions along the same physical links. Accordingly, the bandwidth on physical links that convey transactions from multiple domains may be divided between these domains. The bandwidth may be divided into portions of time or time periods such that each time period corresponds to a different domain. Transactions that correspond to a given domain may only be conveyed on physical links that handle transactions from multiple domains during the time period for that domain.

[0022] As an example, refering to FIG. 1, the clients may be configured as domains as follows:

[0023]

	DOMAIN	CLIENTS
	Domain A	102a & 102b
	Domain B	104a & 104b
	Domain C	106a & 106b
	Domain D	108a & 108b

[0024] In this example, it may be seen that clients from different domains may share the same physical links between the CRLs. For example, clients 102a and 102b of domain A and clients 104a and 104b from domain B may convey tsactions across the physical link between CRL 110 and CRL 120. For client 102a to communicate with client 102b, client 102a may convey a transaction to CRL 110. The transaction may then be routed from CRL 110 to CRL 120, from CRL 120 to CRL 112, and from CRL 112 to client 102b. Similarly, client 104a may communicate with client 104b, by conveying a transaction to CRL 110. The transaction may then be routed from CRL 110 to CRL 120, from CRL 120 to CRL 130, from CRL 130 to CRL 122, from CRL 122 to CRL 114, and from CRL 114 to client 104b. Thus, clients from domain A and domain B may convey transactions across the physical link between CRL 110 and CRL 120.

[0025] Since clients from domain A and domain B may convey transactions across the physical link between CRL 110 and CRL 120, the bandwidth across this physical link may be divided between domain A and domain B. Domain A and domain B may each be granted certain time periods in which they may convey transactions between CRL 110 and CRL 120. In this manner, the chances of a failing transaction from one domain corrupting the other domain may be minimized.

[0026] For example, if CRL 110 conveys a failing transaction from client 102a (domain A) to CRL 120, CRL 110 may only convey this failing transaction during the time period allocated to domain A. CRL 110 may not convey the failing transaction from domain A during the time period allocated to domain B. Accordingly, the chances that the failing transaction from domain A from corrupting domain B may be minimized.

[0027] One will appreciate that clients from different domains may also share the physical links between CRL 120 and CRL 112, CRL 120 and CRL 130, CRL 130 and CRL 122, CRL 122 and CRL 114, and CRL 122 and CRL 116 in this example. The bandwidth across these physical links may be divided between the domains that share them in a manner similar to that discussed above. One will also appreciate that in other examples, other numbers of clients may be included in each domain, other numbers of domains may share a physical link, and the conmections between the clients and the CRLs may be made in other manners.

[0028] Turning now to FIG. 2, a block diagram illustrating one embodiment of a central repeating level is shown. Other embodiments are possible and contemplated. CRL 200 includes receive queue 230 and transmit queue 240 coupled to physical links 232, 234, and 236. CRL 200 further includes counter 210 coupled to control logic 220, and control logic 220 coupled to receive queue 230 and transmit queue 240. Receive queue 230 is also coupled to transmit queue 240. Counter 210 may receive clock input 212 and control logic 220 may receive programmable input 252 as indicated.

[0029] CRL 200 may receive and convey transactions corresponding to multiple domains on physical links 232, 234, and 236. CRL 200 may be configured to divide the bandwidth across each physical link between the domains that share a given physical link as discussed above. Accordingly, CRL 200 may be configured to permit transactions from a given domain to be received and conveyed across a given physical link only during certain time periods. CRL 200 may be configured to prevent or inhibit transactions from other domains from being conveyed and received across the physical link during these time periods. CRL 200 may be configured to store transactions it receives in receive queue 230 and store transactions to be conveyed in transmit queue 240.

[0030] Control logic 220 may be used to divide the bandwidth across physical links 232, 234 and 236 according to the domains that share each physical link. Control logic 220 may be configured to assign a time period to each domain that shares a physical link. Control logic 220 may assign the same or different size time periods to each domain according to certain parameters. For example, control logic 220 may assign larger time periods to domains that have larger numbers of clients sharing a particular physical link and smaller time periods to domains that have smaller numbers of clients sharing that physical link. Control logic 220 may also assign time periods based on other factors such as the overall size or relative importance of a domain and may actively monitor the traffic on a given physical link to dynamically assign and re-assign time periods according to the volume of traffic from each domain. In this context, the volume of traffic for a domain may measured by the number of transactions generated by the clients of that domain over a period of time. Control logic 220 may further assign time periods based on information received from programmable input 252. Using programmable input 252, a user may cause control logic 220 to assign time periods based on one or more input variables. These input variables may specify the size of time periods or time slices.

[0031] Control logic 220 may cause time periods of time slices to be assigned to domains as illustrated in FIG. 3. FIG. 3 depicts a time line with a plurality of time slices 301a, 301b, 301c, and 301(n) where n may be any integer. As seen in the expanded view of time slice 301b, control logic 220 may assign time periods in each time slice 301a, 301b, 301c, and 301(n), wherein n may be any integer, to the domains that share a physical link. As shown, control logic 220 may assign time period 320a to domain A, 320b to domain B, and 320(n) to domain (N) where N may be the Nth domain. As noted above, each time period may be the same size or may differ in size depending on any number of factors.

[0032] Control logic 220 may also be configured to cause time periods for a particular domain to overlap between different physical links. Control logic 220 may further be configured to cause a transaction to be stored in receive queue 230 or transmit queue 240 until a time period corresponding to the domain of the transaction becomes available on a physical link. The ability to overlap time periods may be particularly useful when CRL 200 receives a transaction and needs to route the transaction to another CRL. For example, CRL 200 may receive a transaction corresponding to domain A from a first CRL or client on physical link 232 and may need to convey the transaction to a second CRL on physical link 234. This characteristic may be seen in the example shown in FIG. 4.

[0033] In FIG. 4, a diagram illustrating portions of time slices that correspond to different physical links is shown. For example, time slice portion 410 may correspond to physical link 232 and time slice portion 420 to physical link 234. As shown in this example, time slice portion 410 includes time period 412 assigned to domain A and time period 414 assigned to domain B. Similarly, time slice portion 420 includes time period 422 assigned to domain A, time period 424 assigned to domain B, and time period 426 assigned to domain C. It can be seen that the shaded region 432 identifies an overlap of time for domain A between time period 412 on a first physical link and time period 422 on a second physical link. Accordingly, control unit 220 may be configured to cause a transaction corresponding to domain A to be both received and conveyed during the time period identified by the shaded region 432. It can also be seen that the shaded region 434 identifies an overlap of time for domain B between time period 414 on a first physical link and time period 424 on a second physical link. Accordingly, control unit 220 may be configured to cause a transaction corresponding to domain B to be both received and conveyed during the time period identified by the shaded region 434.

[0034] The embodiment shown in FIG. 2 also includes counter 210. Control unit 220 may be configured to assign time periods to domains for a physical link based on values of counter 210. Counter 210 may be configured to increment or decrement in response to a clock signal received on clock input 212. The clock signal may be generated by system clock (not shown) or may be generated in CRL 200.

[0035] Control unit 220 may assign time periods according to a particular value or values of counter 210. For example, control unit 220 may assign a first time period to a first domain where the first time period corresponds to a first plurality of values of counter 210 and a second time period to a second domain where the second time period corresponds to a second plurality of values of counter 210. Control unit 220 may be configured to detect a particular value of counter 210 to detect the start or end of a certain time period. As shown in FIG. 5a and FIG. 5b, control unit 220 may need to detect only a certain bit or bits to detect the start or end of a certain time period.

[0036] FIG. 5a illustrates a first example of counter values that can be used for time multiplexing of multi-domain transactions. Counter values 502 and 504 each illustrate the values each bit of counter 210 at different times. As illustrated by FIG. 5a, control 220 may be configured to assign time slices based on a single bit of counter 210. Counter value 502 includes a bit with a value of zero and counter value 504 includes the same bit with a value of one. Control unit 220 may be configured to assign the time period where the bit is zero to domain A and the time period where the bit is one to domain B as illustrated. It can be noted that the X's in the other bit positions of the counter indicate “don't care” values that can be either zero or one during these respective time periods. The portion of counter values 502 and 504 that includes “. . . ” indicates that any number of other bits may be included in counter values 502 and 504.

[0037] FIG. 5b illustrates a second example of counter values that can be used for time multiplexing of multi-domain transactions. Counter values 512, 514, 516 and 518 each illustrate the values each bit of counter 210 at different times. As illustrated by FIG. 5b, control 220 may be configured to assign time slices based on multiple bits of counter 210. Counter value 512 includes two bits with values of zero. Counter value 514 includes the same bits with values of zero and one, respectively, counter value 516 includes the same bits with values of one and zero, respectively, and counter value 518 includes the same bits with values of one. Control unit 220 may be configured to assign the time period where the bits are zero to domain A, the time period where the bits are zero and one, respectively, to domain B, the time period where the bits are one and zero, respectively, to domain C, and the time period where the bits one to domain D as illustrated. It can be noted that the X's in the other bit positions of the counter indicate “don't care” values that can be either zero or one during these respective time periods. The portion of counter values 512, 514, 516 and 518 that includes “. . . ” indicates that any number of other bits may be included in counter values 512, 514, 516 and 518.

[0038] FIGS. 5a and 5b are shown by way of example. It can be noted that control unit 220 may be configured to assign time periods that correspond to counter values in numerous other manners. Further, counter 210 may include any suitable number of bits according to a particular embodiment.

[0039] Although the embodiments above have been described in considerable detail, other versions are possible. Numerous variations and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. It is intended that the following claims be interpreted to embrace all such variations and modifications.