Progressive CPU sleep state duty cycle to limit peak power of multiple computers on shared power distribution unit

[0001] None.

[0002] Not applicable.

[0003] Not applicable.

[0004] 1. Field of the Invention

[0005] The present invention relates to power management of a computer system and, more particularly, utilizing a power distribution unit to control the amount of power consumed by a network of computers.

[0006] 2. Description of the Related Art

[0007] As computers get smaller, the computer's power density per unit of volume increases. That is, while the computer's size may decrease, the power demand of the computer generally remains the same. In some applications, a plurality of computer servers in a network are mounted onto a rack. The servers and other electronic components on the rack are fed by a common power source.

[0008] Generally, power management of the servers on the rack is performed in a somewhat inefficient and expensive manner. In calculating the power demands, first, the maximum power rating of individual servers is determined. Next, the amount of power required for each of the racks containing the servers is calculated. Last, a power source provides enough current for the maximum or peak current rating for all the components, including the servers, disk drives, cooling fans, etc., in the entire rack. This method ensures that if all components were operating at their peak power at the same time, the system would not overload the wirings or circuit breakers.

[0009] For safety reasons, building facility engineers design for worst case scenarios. For example, in an Internet Service Providers (ISP) or Application Service Providers (ASP) environment, the servers are seldom operating at their peak, and if so, rarely at the same time. Therefore, while the above described system may be reliable, such a system is inefficient and expensive for certain applications.

[0010] Briefly, a power management system for distributing power to a plurality of computers is employed. In one embodiment, the power management system monitors the power consumption of the various computers and regulates the overall power demand by momentarily putting microprocessors of the computers into a lower power state. As the overall power demand exceeds a preset limit, the power management system applies a variable increasing power down duty cycle to the microprocessors whereby the overall power demand of the computers decreases to an acceptable level.

[0019] FIG. 1 is a block diagram of an embodiment of a computer system C in accordance with the present invention. The computer system C typically includes various computing devices, such as servers 112, 114 and 116. For illustrative purposes, three (3) servers are shown in FIG. 1. However, one skilled in the art would recognize that there could be more than three (3) servers in a system powered by a common power source. Each server generally includes a microprocessor and a current monitoring device. For example, the server 112 includes a microprocessor 120 and a current monitoring device 126. The microprocessor 120 can be any type of general central processing unit (CPU), such as an Intel Pentium-type microprocessor. The servers 112, 114 and 116 are powered by a power distribution unit 100. Power P from a facility power drop is provided to the power distribution unit 100 via power lines 104.

[0020] In this embodiment, the power distribution unit 100 includes a power management controller 102 and a current monitoring device 134. The power management controller 102 includes the necessary hardware and software for ascertaining power demands of various servers coupled to it, and power P supplied from the facility power drop. The power P from the power distribution unit 100 is provided to the servers 112, 114 and 116 via power lines 106, 108 and 110, respectively.

[0021] The current measuring devices 126, 128 and 130, measure local current demand for their respective servers. Power P provided by the facility power drop to the power distribution unit 130 is monitored by the current measuring device 134.

[0022] The power data management controller 102 receives current demand information of the servers 112, 114 and 116 via a line 132. The power management controller 102 is also coupled to the microprocessors of the microprocessors of the various servers via a data bus 118. The data bus 118 can carry microprocessor commands; such as state requests. Although not shown, state requests can be encoded in-band over power lines 106, 108 and 110 and interpreted by the server's microprocessor before its power filtering circuit. Thus, a separate data bus line 118 would not be needed.

[0023] Generally, the total power consumed by a server is directly related to the utilization of its microprocessor. Normally, there is a base power level required to power the servers, fans and drives, such as a floppy drive or a CD-ROM drive. For microprocessors that include CMOS logic, power consumption of the microprocessor increases proportionally as the processor's logical state transitions increase. Thus, a microprocessor with fewer state transactions, translates to less power demand. The server's memory, disk and interface ports are also directly related to the microprocessor's activity. For example, a request may come over an Ethernet network. This request generates microprocessor activity, memory activity and possibly disk drive activity. Thus, power demand of the server increases during these requests. Slowing down the microprocessor may slow down the frequency of memory and I/O requests. Generally, the maximum power for a server is reached only when all the electronic devices in the computer are operating at their peak and for the rack when all components in the rack are operating at their peak.

[0024] The power distribution unit, described above, can be coupled to any type of electronic components. In addition, the power management controller could be located outside of the power distribution unit or be in many locations and networked together.

[0025] FIG. 2 is a chart illustrating the power demands of a plurality of computer servers. As mentioned previously, a microprocessor's utilization is directly related to the microprocessor's current demand. For example, in the computer system C with three (3) servers coupled to a shared power source, lines 202, 204 and 206 represent microprocessor utilization and current demands for respective servers 112, 114 and 116. The figure shows over time varying current demands of the three servers. Line 200 represents the cumulative current demands of servers 112, 114 and 116. Typically, in certain applications, such as ISP or ASP, the microprocessor's utilization for the servers is usually less than 25% on average.

[0026] Since the server's total power is typically related to the microprocessor activity, the power demand requirement for each server can be reduced by regulating the microprocessor's activity. The microprocessor's power demand can be regulated by temporarily putting the microprocessor into a lower power state, such as a sleep state for a short period of time.

[0027] FIG. 3 is a state table illustrating the lower power sleep states of an Intel Pentium III microprocessor. The sleep states of the Intel Pentium III microprocessor are described in Intel's Pentium III Processor with 512 KB L2 Cache at 113 GHz Datasheet, June, 2001 (herein incorporated in its entirety, by reference). Typically, when the microprocessor is operating, the microprocessor is in a normal state 300. When a STPCLK# signal is asserted, the microprocessor enters into a sleep state, such as a stop grant state 302. When the microprocessor is in the stop grant state 302, the microprocessor requires less power. When the microprocessor does not receive the STPCLK# signal, the microprocessor is in the normal state 300.

[0028] When the microprocessor is in the stop grant state 302, the bus clock (BCLK) continues to run. Snoops and interrupts are also allowed in the stop grant state 302. Thus, the microprocessor can operate in a lower power state, yet can respond to snoops and interrupts. Although, the Intel Pentium processor has even deeper sleep states, these deeper sleep states do not allow snoops or interrupts. Therefore, when placing a microprocessor in a lower power state, this lower power state should preferably be able to respond to snoops and interrupts. Snoops and interrupts are typically generated by asynchronous external events like keyboard, Ethernet network or disk drive responses.

[0029] Other techniques may be employed to momentarily reduce power consumption of a computer. Such techniques may not necessarily involve microprocessor commands.

[0030] FIG. 4 is a graph illustrating the cumulative current demand of the servers versus the microprocessor's power down duty cycle. The microprocessor power down duty cycle is a ratio of the time the microprocessor is in a sleep state versus the time the microprocessor is in the normal state. Thus, for example, if the microprocessor power down duty cycle is 10%, the microprocessor is in a sleep state 10% of a period of time. Generally, placing the microprocessor momentarily in a lower power state will not adversely affect the performance of the microprocessor. In certain applications, it is rare to have all the servers operating at their peak at any instant in time. Thus, a power down period may likely occur during an idle microprocessor period and should have little effect on overall performance.

[0031] FIG. 4 also includes a current minimum I_MIN and a current maximum I_MAX. When the current demand for the cumulative servers is below I_MIN, the microprocessors power down duty cycle is 0%. Thus, the microprocessors can operate in their normal state. Once the cumulative current demand for the servers exceeds I_MIN, the microprocessor's power down duty cycle is greater than 0%. FIG. 4 illustrates a linear increasing power down duty cycle, however one skilled in the art to recognize that the increasing duty cycle may be any variable function in time. When the cumulative current demand for the servers exceeds I_MAX, the microprocessor's power down duty cycle approaches a power down duty cycle maximum. For example, the Duty Cycle_MAX is 90% should the cumulative current demand for all of the servers exceed I_MAX.

[0032] FIGS. 5A and 5B are timing diagrams of a power down duty cycle of 10% and 70%. A STPCLK# signal is asserted in a low logical state (# sign) and not asserted in a high logical state. In FIG. 5A, the power down duty cycle for each server is shown. For example, the power down duty cycle over time for server A is represented by line 500. If the cumulative current demand for all of the servers is 155.5 amps (I_10%), the Server A microprocessor's power down duty cycle is 10%. (See FIG. 4). Therefore, with a 10% power down duty cycle, the power management controller 102 would send a STPCLK# signal to the microprocessor 112 for a duration of 10% of a time period. The timing diagram of FIG. 5A shows that a STPCLK# is asserted 508 at T₁ and deasserted 510 at T₂. During the time when the STPCLK# is asserted at T₁, the power management controller 102 does not apply a STPCLK# signal to either server B or C, as shown by 512 and 520.

[0033] After the STPCLK# signal is deasserted 510 at T₂, the power management controller 102 sends the STPCLK# signal to server B, as shown at 514. At this same time T₂, the STPCLK# signal is not sent to the microprocessor 124 of server C. Therefore, at any one time, only one of the server's microprocessor is in a sleep state, when the power down duty cycle is 10%.

[0034] Next, if the cumulative current demand for all of the servers is 188.8 amps (I_70%), a 70% power down duty cycle is generally applied to the servers (see FIG. 4). FIG. 5B is a timing diagram of a power down duty cycle of 70%. A STPCLK# signal is asserted by the power control unit 102 to the various microprocessors in a staggered state. For example, for Server A, the STPCLK# signal is asserted 552 at T₁₀ and deasserted 544 at T₁₃. The time between T₁₀ and T₁₃ represents a power down duty cycle of 70%. During this time, Server B is also in the stop grant state 558. The STPCLK# signal is deasserted 560 for Server B at T₁₂ and once again is asserted 562 at T₁₂.

[0035] Should the cumulative current demands for all of the servers approach I_MAX, a power down duty cycle maximum Duty Cycle_MAX is applied to the servers. In one embodiment, the Duty Cycle_MAX is a 90% power down duty cycle. The maximum power down duty cycle should be around 90% to prevent starvation of microprocessor time and allow it to run critical server and O/S operations. The power down cycling to each server is also staggered to prevent a power surge when the microprocessor comes out of its sleep state. When the microprocessor is in its sleep or lower power state, all other server activity generally decreases as well. This decrease in bus, memory, disk, network and other I/O references typically decreases the total power consumed by the individual server. After some time lag, due to the power circuit capacitance, the server's current demand for each server will typically decrease. The I_MAX must be high enough to allow the servers to perform at some adequate performance level, while low enough to provide power savings. Empirical data set can be ascertained from the servers to provide the system designer with adequate matrics to set the high current level mark. Thus, particular threshold values can be determined based on the specific application so as to minimize performance degradation yet maximize power savings. These threshold values can then be programmed into the power management controller 102.

[0036] FIG. 6 is a graph illustrating the microprocessor's utilization and cumulative power demand with a down power duty cycle is applied. Lines 602, 604 and 606 represents the current demands for Servers A, B and C. Line 600 is the cumulative current demands for Server A, Server B and Server C. As shown in the figure, there are times when the cumulative current demand 600 may exceed I_MAX. That is, during the periods when the multiple servers are operating at their peak, the microprocessor's applications may experience a small slow down time and perhaps some performance degradation. For example, if I_MAX is 200 amps, the total current draw of the three (3) servers is limited to this maximum current. Consequently, as shown in FIG. 5, a cumulative current demand is clipped at 200 amps, thus a down duty cycle time of 90% is applied. Server performance may be affected, however, the performance degradation may be small. This small downtime and possible performance degradation is outweighed by the lower current requirement. A lower current requirement thus translates into lower cooling requirements for the computer and a lower maximum current rating.

[0037] FIG. 7 is a flow diagram illustrating the steps taken according to an embodiment of the present invention. The method starts at step 700. At step 702, the power management controller monitors current demands for all computers. The current demands for each computer is summed to ascertain a total current demand at step 704. At step 706, the total current demand is greater than the minimum threshold but less than the maximum threshold, a duty cycle is calculated at step 708. If the total current demand is not greater than the minimum threshold and not less than the maximum threshold, the method proceeds to step 710. At step 710, if the total current demand is greater than the maximum threshold, the duty cycle equals a Duty Cycle _MAX if at step 710, the total current demand is not greater than the maximum threshold the method ends at step 718. At step 714, a STPCLK# is applied to a plurality of computers for a duration equal to the duty cycle calculated in step 708 and step 712, in a predetermined order. After the step 714 is executed, the method ends at step 718.

[0038] As indicated above, other techniques may be used to lower power consumption of a plurality of computers without departing from the spirit of the invention. These other techniques may not necessarily involve microprocessor commands.

[0039] The foregoing disclosure, a description on the invention are illustrative and explanatory thereof, and various changes in the size, shape, materials, components, circuit elements, wiring connections and context, as well as in the detail of the illustrated diagram and construction and method of operation may be made without departing from the spirit of the invention.