Memory devices with buffered command address bus

[0001] This invention relates to memory modules having multiple memory devices. More particularly, this invention relates to memory modules having multiple memory devices with reduced signal loading at the devices' interface.

[0002] Computer systems are often designed to provide flexible memory resources (e.g., 512 megabytes, 1.2 gigabytes, etc.), and these memory resources can be designed with different combinations of memory modules, memory devices, or both. For example, the memory capacity of a computer system can be made up of one or more DRAM (dynamic random access memory) based memory modules. If a system's memory comprises two memory modules of equal size, each module represents half the system's memory capacity. One memory module can represent the lower half of the computer system's memory, and the other can represent the upper half. Alternatively, every other n-bytes of data can be stored in one memory module, while the remaining n-bytes of data can be stored in the other memory module.

[0003] Each memory module can include more than one rank of memory devices. Each rank of memory devices can include one or more memory devices that store data for a range of addresses in the computer system's memory and is individually selectable with a chip select signal. In addition, each DRAM-based memory device can be organized into internal banks and rows which have to be activated (or opened) before read or write accesses can occur. Typically, only one row (or page) in a particular internal bank of a DRAM-based memory device can be open at one time. However, more than one internal bank of such a memory device can be open at one time. Before another row in the same internal bank can be opened, the row currently open has to be closed. Banks and rows of DRAM-based memory devices are activated by commands received from a memory controller over a command address bus.

[0004] Inefficiencies in DRAM-based memory devices arise when successive accesses are made to different rows in the same internal bank. Such accesses cause the currently open row to be closed before a different row can be opened, thus delaying the access. One way to minimize these inefficiencies is to close a row as soon as it has been accessed, rather than leaving it open. Then, no unnecessary delay (i.e., row de-activation time) is incurred when a subsequent access is to another row. However, closing rows immediately after access can still result in an unnecessary delay should the next access be to the same row (because now that same row has to be re-opened).

[0005] More than one rank of memory devices also can be open at the same time. Thus, having as many ranks of memory as possible constitute the memory resources of a computer system is advantageous because many different internal banks and rows on different memory devices (or memory modules) can be open concurrently. Accordingly, memory modules made up of multiple ranks, each of which can be accessed independently, are commonly used. Examples of memory modules that include multiple ranks are double-sided DIMMs (dual in-line memory modules), which include at least one rank of memory devices on each side of the DIMM.

[0006] Typically, the interface between memory modules and the memory controller includes a command bus, an address bus, a data bus, and other control signal lines. Because multi-rank memory modules include memory devices connected in parallel, a potentially large number of memory devices use the same signals, which can result in electrical overloading of the interface between the memory controller and the memory modules. Moreover, each rank of memory devices is individually selected with a corresponding chip select signal. Each chip select signal is also used to establish a window of time during which signals on the command bus and the address bus (hereinafter referred to as the command address bus) are valid. As the number of ranks of memory devices in a computer system increases, more time may be needed to ensure that signal degradation of chip select signals and signals on the command address bus does not affect operation of the memory modules.

[0007] One approach to alleviating overloaded interface connections between parallel-connected memory devices is to provide duplicate signals from the memory controller to each device. Although this alleviates overloading of connections, duplication is not an economical solution for the large number of connections between a memory controller and each memory module because the memory controller will require more external pins.

[0008] Another approach to alleviating overloaded memory interface connections is to buffer the command address bus between the memory controller and the memory modules. However, buffering on the memory module is expensive and requires an extra clock of latency.

[0009] Yet another approach to minimizing the effects of an overloaded command address bus (and to maximizing the number of memory modules connectable to a memory controller) is to present the address to the memory two clock cycles in a row, with the chip select signal inactive for the first cycle and active for the second cycle. This provides an extra clock cycle of setup time to the memory devices. The chip select signal still runs in the same manner (i.e., active for only one clock cycle), but the chip select signal is only driven to one module (or rank), so loading is not a problem. This solution is called 2T addressing. However, 2T addressing diminishes command address bus bandwidth because for every command issued to the memory devices, an extra clock cycle is required. Therefore, for memory devices that can accept signals from a command address bus at maximum speed, 2T addressing reduces memory bandwidth.

[0010] In view of the foregoing, it would be desirable to be able to increase the number of memory modules and the number of ranks of memory devices connectable to a memory controller by alleviating the overloading of the command address bus while limiting any decrease in command address bus bandwidth.

[0011] It is an object of this invention to increase the number of memory modules and the number of ranks of memory devices connectable to a memory controller by alleviating the overloading of the command address bus while limiting any decrease in command address bus bandwidth.

[0012] In accordance with the invention, switches are provided between a memory controller and the memory modules. The switches isolate the command address bus from the memory modules. The command address bus is coupled to the memory controller and the switches. The memory controller also provides preferably multiple copies of chip select signals that can be used to enable/disable the switches. The switches can be advantageously implemented either on the memory modules or elsewhere between the memory modules and memory controller. The signal propagation delay through the switches is insignificant compared to the extra clock cycle required by buffering or 2T addressing.

[0013] The above and other objects and advantages of the invention will be apparent upon consideration of the following detailed description, taken in conjunction with the accompanying drawings, in which like reference characters refer to like parts throughout, and in which:

[0014] FIG. 1 is a block diagram of a typical computer system that includes memory modules;

[0015] FIG. 2 is a block diagram of a typical SDRAM (synchronous dynamic random access memory) device;

[0016] FIG. 3 is a timing diagram of signals between a memory controller and memory modules;

[0017] FIG. 4 is block diagram of a memory controller coupled to memory modules according to the invention;

[0018] FIG. 5 is a schematic representation of circuitry used to generate switch control signals according to the invention;

[0019] FIG. 6 is a timing diagram of signals between a memory controller, switches, and memory modules according to the invention;

[0020] FIG. 7 is a timing diagram of signals between a memory controller and memory modules;

[0021] FIG. 8 is a timing diagram of signals also between a memory controller and memory modules;

[0022] FIG. 9 is a timing diagram of signals also between a memory controller and memory modules; and

[0023] FIG. 10 is a block diagram of a computer system incorporating the invention.

[0024] SDRAM-based memory modules (including DDR (double data rate) SDRAM-based devices) and other types of memory modules that have multiple ranks advantageously can increase memory access throughput by having many activated internal banks and rows in each rank. Memory access schemes are known and can be developed that take advantage of this parallel activation of more than one rank of memory devices.

[0025] However, because a high number of ranks of memory devices can result in signal degradation on the command is address bus shared between these ranks, the command address bus is advantageously connected to switches between the memory controller and the memory modules in accordance with the invention. The switches electrically isolate the command address bus from the ranks of memory devices. Each rank can be individually selected using a corresponding chip select signal, which also can be used to control the switches. Electrical overloading of the command address bus is reduced because the command address bus is preferably driven to one rank of memory devices at a time.

[0026] FIG. 1 illustrates a typical computer system 100.

[0027] Computer system 100 includes CPU (central processing unit) 102 coupled to DIMM 114 via memory controller 106. Memory controller 106 can be implemented on an integrated circuit (e.g., a chipset) that includes other functions or can be implemented on the same integrated circuit as CPU 102. DIMM 114 has eight SDRAM devices, each of which are organized to store 16-bit data (note that the eight SDRAM devices are merely illustrative; computer systems may incorporate memory modules having other numbers and types of memory devices (such as, e.g., double data rate SDRAMs)). Each SDRAM memory device (e.g., memory devices 108, 110, and 112) is connected in parallel to memory controller 106 via a respective 16-bit data bus. Typically, all SDRAM memory devices are selected simultaneously for reading and writing by memory controller 106. If all eight memory devices are accessed at the same time, 128 bits of data can be written or read at the same time by CPU 102. This group of eight memory devices constitutes one rank of memory devices. Another rank of memory devices can be coupled in parallel, and selected simultaneously by memory controller 106. One way of adding an additional rank of memory devices is to have one rank on one side of DIMM 114 and another rank on the other side of DIMM 114. Command address bus 116 is coupled to DIMM 114 and is distributed to each memory device on the DIMM.

[0028] FIG. 2 shows a typical dual-bank SDRAM device. An SDRAM device can be organized into multiple memory arrays or internal banks, each having circuitry to address its corresponding memory storage. For example, the first internal bank of an SDRAM device (e.g., bank 212) includes memory array 206 which is organized into rows and columns of memory cells. To access the memory cells in a particular row and column of memory array 206, the row address is provided to bank 212 and decoded by row decoder 202, followed by a column address and subsequent decoding by column decoder 204. On read accesses, data is provided by memory array 206 to data input/output (I/O) registers/buffers 214 through sense amplifiers 208. On write accesses, data is provided to memory array 206 by I/O registers/buffers 214 through sense amplifiers 208. Each bank of an SDRAM device preferably includes at least the circuitry shown for bank 212, and is thus accessible in this manner independent of other internal banks. Each internal bank of an SDRAM device is connected to the same portion of the data bus from the memory controller (for a 16-bit SDRAM device, the banks in the device are connected to the same 16-bit portion of the data bus). This is illustrated in FIG. 2 by the parallel connection (via busses 216 and 218) of both banks to data I/O registers/buffers 214. The memory controller can be configured to advantageously coordinate bank selection. For example, another bank can be made ready for access (e.g., bank activation, row selection, column selection) while the memory controller is still writing or reading data to or from the other bank.

[0029] FIG. 3 illustrates typical command address bus timing for a DRAM-based DIMM. Clock signal CLK 302 synchronizes chip select signal CS 306 and command address bus 304. When chip select signal CS 306 is low, signals on command address bus 304 are considered valid by the DIMM.

[0030] If the memory controller is operated in 2T addressing mode, chip select signal CS 306 is low for one cycle (e.g., T₀), while the command address bus signals are held valid for two clock cycles. By keeping the command address bus signals valid for two clock cycles, the setup time of the command address bus signals relative to the rising edge of the clock signal is increased. The chip select signal is synchronized to the same clock signal as the command address bus signals.

[0031] However, the chip select signal does not have to be held valid for two clock cycles because each chip select signal drives only one rank of memory devices, and thus has a smaller load than the command address bus signals. Thus, the command address bus is given time to settle before the chip select signal indicates that a command is being issued on the command address bus. As shown in timing diagram 300, DIMMs operating in 2T addressing mode may slow down memory accesses because at least one extra clock cycle is used every time a command is sent to the DIMM. Moreover, a typical access in a typical DRAM-based DIMM may require the issuance of more than one command. For example, the memory controller activates (or opens) an internal bank and row before the row can be accessed, and the memory controller may be required to close the row after each access. In accordance with the invention, the memory controller can be advantageously operated in 1T addressing mode (i.e., command address bus signals are held active for preferably one clock cycle to indicate a valid command).

[0032] FIG. 3 also shows data bus 308, which provides data from the selected rank of memory devices. For example, as shown in FIG. 3, rank 1 of the memory devices can respond by providing data on data bus 308 after time interval T₁ from the beginning of a bank open command issued to rank 1. Time interval T₁ is determined by the amount of time needed by the memory devices to respond to bank open and bank read commands. Time interval T₁ is also affected by pending commands to other banks in the same rank or other ranks, because the same data bus can be shared between several ranks of memory devices and other types of memory devices.

[0033] FIG. 3 shows a memory access to rank 1 with a burst length of two (i.e., the memory controller has requested two words of data). After rank 1 has provided the data requested by the memory controller, data bus 308 is idle (e.g., tristated). The idle time of data bus 308 depends on the following: the amount of time needed to issue a command to a memory device, the response time of the memory devices, and the burst length of data requested from rank 1. Because the memory devices are operated in 2T mode, the component of time interval T₁ needed to issue a command is increased. In some computer systems, if the burst length of data requested from the memory devices is long, the effect of operating the memory devices in 2T mode is masked because the data bus is kept busy outputting long bursts of data. However, for other computer systems, the data bus remains idle for significant periods of time because the command address bus bandwidth is degraded by the operation of the memory devices in 2T mode. Although FIG. 3 illustrates a read access (the data is provided by the memory devices, and not by the controller), the data bus is potentially idle for long periods of time after a write access. A difference between a write access and a read access is the latency of the data with respect to the command address bus. For a write access, the data can be provided on the address bus at substantially the same time that the corresponding write command is issued, but a subsequent write access to another rank is delayed by the time required to open a bank and to issue a command, which is greater in 2T mode.

[0034] FIG. 4 shows memory configuration 400 in accordance with the invention. Memory configuration 400 includes memory controller 402 coupled to four bus switches 404, 406, 408, and 410. The signal quality of command address bus 412 is improved with the use of these switches because they isolate the command address bus from the memory modules. One type of switch suitable for use in accordance with the invention comprises an N-channel FET (field effect transistor) with low resistance when turned on (i.e., conducting or in the closed state) and high impedance when turned off (i.e., non-conducting or in the open state). However, any type of switch that exhibits low propagation delay and that does not substantially degrade signal quality is suitable for use with the invention.

[0035] Switches 404, 406, 408, and 410 isolate portions of the command address bus (i.e., buses 412, 414, 416, 418, and 420) from memory controller 402. Advantageously, command address bus 412 is loaded with only one of DIMMs 422, 424, 426, and 428 at a same time because only one of switches 404, 406, 408, and 410 is preferably turned on at a time. Switches 404, 406, 408, and 410 are controlled by chip select signal CS1, chip select signal CS2, chip select signal CS3, and chip select signal CS4, respectively, each of which is used to select one rank of memory devices. The chip select signals are not shared among ranks, thus it is not necessary for these signals to be isolated using the same scheme as the command address bus. The chip select signals can be used to drive switches 404, 406, 408, and 410 and DIMMs 422, 424, 426, and 428. Alternatively, the memory controller can provide extra copies of the chip select signals such that one copy is used to control the switches, and the other copy is used to control the DIMMs.

[0036] Using switches to isolate the command address bus from each rank of memory devices can introduce a delay between the chip select signal for each rank and signals on the command address bus. But this delay is insignificant compared to the loss of command address bus bandwidth that occurs with 2T addressing. Furthermore, this delay can be compensated for by sending the command address bus signals early or by delaying any strobe or clock signal indicating valid data received from the memory modules.

[0037] FIG. 4 also illustrates the logical placement of the switches with respect to the memory controller and the DIMMs. Typically the memory controller is mounted on a motherboard, and the DIMMs are implemented as printed circuit boards that are inserted into sockets on the motherboard. The switches can either be placed on the motherboard or on the DIMM circuit boards.

[0038] If DIMMs having more than one rank of memory devices are used, each rank of memory devices still shares the same command address bus if the switches used to isolate the command bus are not located on the DIMMs. When switches are used to couple the command address bus to the DIMMs, the command address bus to a particular DIMM (e.g., bus 414, 416, 418, or 420) is driven only when a rank of memory devices on that DIMM is being selected by its corresponding chip select signal. Thus, when more than one rank of memory devices is present on a DIMM (e.g., double-sided DIMMs), the command address bus has to be driven when either one of the ranks of memory devices is selected by its corresponding chip select signal. In an alternative embodiment, if the switches are implemented on the DIMMs and there is more than one rank of memory devices on a DIMM, the DIMM can include switches to isolate the command address bus for each rank of memory devices, thus eliminating the requirement that the command address bus be shared between two ranks of memory devices.

[0039] FIG. 5 shows one embodiment of the invention in which the chip select signals can be logically combined to control the switches, such that if there is more than one rank of memory devices on a DIMM, the command address bus is driven to the DIMM when any one of the ranks on the DIMM is selected. As shown in FIG. 5, AND gate 502 can be used to combine two chip select signals CS1 and CS2. The output of AND gate 502 can be used to control switch 504 to provide the command address bus to memory module 506, which has one rank of memory devices controlled by CS1 and another rank of memory devices controlled by CS2.

[0040] In an embodiment of the invention, the memory controller provides a switch select signal to drive the switch for each memory module, wherein the signal is configured to enable (i.e., close) the switch when either one of the ranks on the memory modules has been selected. This allows the controller to turn the switches on or off independent of the chip select signals, which is advantageous if the switches are noisy or require settling time when activated or de-activated. The timing of the switch select signal can be based on the chip select signals, but does not have to be exactly the same. For example, timing diagram 600 of FIG. 6 shows clock signal CLK 602 (which synchronizes command address bus 604), chip select signal CS 606, and switch select signal 608. Chip select signal CS 606 and switch select signal 608 control the switch for a first rank of memory devices. Because the memory controller has information about memory accesses, the memory controller can output a switch select signal that maintains switch correspondence to a particular rank of enabled memory devices, as long as the memory controller is not accessing another rank of memory devices. For example, as command address bus 604 shows, there are two successive accesses to a first rank of memory devices with no intervening accesses to another rank. Instead of having the switch select signal timing based exclusively on chip select timing, and thus cause the switches corresponding to the first rank to be de-selected during time interval T₀, switch select signal 608 is preferably active for the entire period that the memory controller is exclusively accessing one rank of memory devices.

[0041] Other embodiments of the invention can include circuitry to combine more than two chip select signals if more than two ranks are present on each memory module. Or, a memory controller can be configured to provide switch control signals that indicate the presence of more than two ranks on one memory module.

[0042] When switches for one rank of memory devices are turned off and another set of switches are turned on, settling time may be required between turning off the first set of switches and turning on the second set of switches. If no settling time is provided, erroneous signaling of commands may result.

[0043] Timing diagram 700 of FIG. 7 illustrates how erroneous signaling of commands could result from providing no settling time between accesses to a first rank of memory devices and accesses to a second rank of memory devices. When a command to a first rank is immediately followed by the issuance of a command to a second rank, command address bus 704 transitions from issuing a command to the first rank to issuing a command to the second rank. There may be a time interval, T₁, during which command address bus 704 is invalid because the set of switches driving the first rank is being turned off at approximately the same time that another set of switches driving the second rank is turned on. Therefore, a command to a second rank may not be issued immediately after a command to a first rank. The memory controller preferably waits a clock cycle before issuing commands to a second rank of memory devices after issuing commands to a first rank of memory devices. Although an extra clock cycle can be required between accesses to different ranks, this clock cycle does not significantly impact overall memory throughput on read accesses, because data bus 708 requires a clock cycle (a turnaround cycle) between two subsequent read accesses from two different ranks to avoid data from one rank being mistaken for data from another rank. As illustrated in FIG. 7, because data bus 708 is shared between more than one rank of memory devices, during time interval T₂, data could be driven by both ranks of devices and thus cause invalid data to be present on data bus 708. Thus, even if at T₁ it were possible to avoid inserting a clock cycle of delay between accesses to two different ranks, it would not be possible to accept data from two different ranks without an intervening turnaround cycle on read accesses.

[0044] Timing diagram 800 of FIG. 8 illustrates how the command address bus can be controlled to wait at least a clock cycle in between successive accesses to different ranks of memory devices. As shown in FIG. 8, an open (bank) command is followed by a read command to a first rank of memory devices on command address bus 804. These commands do not require an intervening turnaround clock cycle because the same set of switches is driven when commands are issued to one rank of memory devices. Turnaround clock cycle 812 is provided between the commands issued to the first rank and a subsequent read command to a second rank of devices. Because successive read accesses to different ranks of devices require an additional clock cycle of delay between accesses, the insertion of intervening clock cycle 812 on command address bus 804 does not cause a loss of memory bandwidth. FIG. 8 also illustrates a data output rate of twice the clock speed. Correspondingly, only a half turnaround clock cycle 814 can be inserted between read accesses from two different ranks of data. Although command address bus 804 in FIG. 8 is shown as having a clock cycle of turnaround time, a half clock cycle may suffice for command address bus 804 to settle in some embodiments of the invention. Although FIGS. 7 and 8 illustrate read accesses from ranks of devices, settling time may have to be provided between write accesses to different ranks of memory devices in the same manner illustrated for read accesses from different ranks. However, no corresponding turnaround time is required on the data bus between write accesses to two different ranks of memory devices.

[0045] In view of the turnaround time required to settle the command address bus between accesses to different ranks of memory devices, it can be advantageous to schedule together as many commands to one rank of memory devices as possible, without intervening commands to a different rank. One limitation on scheduling commands is the delay required between a bank activation command and bank read/write command. While the bank read/write command can be issued on the command address bus immediately after a bank activation command, the memory device cannot respond to a read/write command if such a command is issued before it is ready. In DRAM devices, the time required between the activation and read/write commands is the t_RCD (RAS-to-CAS delay) parameter. One way to do this is by using posted CAS accesses, which are available for at least DDRII (Double Data Rate II) DRAM devices. In posted CAS accesses, the read/write command to a particular bank (in a rank) can be delayed at the memory device internally after the receipt of the read/write command with the use of a buffer inside the memory device that holds the read/write commands until the memory device is ready to respond.

[0046] FIG. 9 illustrates the use of posted CAS accesses. Timing diagram 900 illustrates the timing of signals on the command address bus and chip select signals with and without the use of posted accesses. As shown in timing diagram 900, bank activate command to activate a bank from a first rank of devices is followed by a read command to that bank at T₁, as illustrated by command address bus signals 906. Time interval T₂ between the activate command and the read command has to be greater than the t_RcD parameter for the memory devices that are accessed. Timing diagram 900 of FIG. 9 illustrates a memory device that has a t_RCD parameter of at least three clock cycles (T₂). An activate command to a bank in a second rank of devices can be issued in time interval T₂ using command address bus 906 after sufficient time has elapsed to allow the command address bus to settle (turnaround cycles 916). At T₁, a read command for the bank in the first rank that has been activated can be issued on command address bus 906. At T₃, a read command for the bank in the second rank that has been activated can be issued on command address bus 906. Chip select signal CS 904 is active for the corresponding time intervals that commands to the first rank of memory devices are issued and chip select CS 908 is active for the corresponding time intervals that commands to the second rank of memory devices are issued and chip select. As shown by timing diagram 900 of FIG. 9, two activate and two read commands to two ranks of memory devices take at least seven clock cycles without the use of posted CAS accesses.

[0047] Timing diagram 900 of FIG. 9 also illustrates posted CAS accesses on the command address bus. By using posted CAS accesses, a read command to a first rank of memory devices is issued on command address bus 912 some time before T₁, i.e., before time interval t_RCD has elapsed. The number of cycles that a read command is issued on the command address bus in advance of when a memory device posts or sends the read command internally can be determined by how the memory device is programmed with respect to its posted CAS access feature. As shown by command address bus 912, the read command to a first rank of memory devices can be issued three cycles in advance of when the memory device can accept the read command internally. Command address bus 912 also illustrates the issuance of a read command to a second rank of memory devices in advance in advance of when the memory device can accept the read command internally. Thus, when memory devices are configured to use posted CAS access, the read command and the activate command to a specific rank of memory devices are advantageously scheduled without intervening commands to a different rank of devices. As shown by timing diagram 900 of FIG. 9, two activate and two read commands to two ranks of memory devices take at least five clock cycles with the use of posted CAS accesses, which is two cycles less than a similar sequence of commands without the use of posted CAS accesses. Although FIG. 9 illustrates the timing of the command address bus with respect to activate and read commands, the timing of the command address bus is substantially identical for write accesses.

[0048] In accordance with the invention, the number of ranks used in a computer system can be increased by using switches to limit the loading of the command address bus seen by the memory controller. The switches advantageously limit loading of the command address bus by effectively driving command address bus signals only to memory devices of a selected rank. Embodiments of the invention are not limited to SDRAM-based devices, and may include other types of memory devices that are configured to share a single command address bus, wherein overloading of the command address bus can be alleviated by using switches in accordance with the invention.

[0049] FIG. 10 shows computer system 1000 that includes a memory configuration in accordance with the invention. The components of computer system 1000 are merely illustrative and should in no way limit the number and type of components that can be used in a computer system in accordance with the invention. The components of computer system 1000 can be incorporated in one or more integrated devices, and some components may be incorporated in the same integrated devices. Moreover, other computer system components not shown can also be included. Computer system 1000 includes one or more CPUs 1002 and 1006, which are coupled to memory controller/chipset 1004 and cache 1008. Memory controller/chipset 1004 can provide memory interface 1022 such that memory modules 1010 and boot memory 1012 can be accessed by CPU 1002 and 1006 through memory controller/chipset 1004. Either memory controller 1004 or memory modules 1010 include switches for isolating the command address bus in accordance with the invention. Memory controller/chipset 1004 can also provide PCI (peripheral component interconnect) bus 1024, to which network controller 1014, bridge 1016, and video controller 1018 can be coupled. Bridge 1016 can be used to couple I/O devices 1020 (e.g., a modem circuit) to CPU 1002 and 1006 by providing connectivity between PCI bus 1024 and I/O bus 1026.

[0050] Thus it is seen that the number of memory modules and the number of ranks of memory devices connectable to a memory controller can be increased by providing switches that isolate the command address bus from the memory modules. One skilled in the art will appreciate that the invention can be practiced by other than the described embodiments, which are presented for purposes of illustration and not of limitation, and the invention is limited only by the claims which follow.