Cooling using complimentary tapered plenums

[0001] This invention was made with government support under Department of Energy Contract No. B517552. The government has certain rights in this invention.

[0002] This invention relates to the cooling of equipment that produces heat, and in particular to the cooling of electronic equipment. More particularly it is related to apparatus and methods used to cool racks of electronic equipment, and still more particularly to racks of electronic equipment of the type used in a data storage center.

[0003] For air-cooled, heat-producing electronic equipment requiring two or more racks arranged in rows on a raised floor, such as in a large computer installation, the cooling system may cause air to flow across the electronics in any of the three coordinate directions. Let X be the direction along a row, Y be perpendicular to the rows, and Z be upward. For systems in which the air flows across the electronics in the X direction, air-flow plenums typically must exist between each pair of racks.

[0004] FIG. 1 illustrates a prior-art arrangement of such plenums, 22A, 24A, 22B, 24B, interspersed between a row of heat-producing racks 20A, 20B, 20C. Although the figure depicts just three racks, in general the row may contain an arbitrary number of such racks. In the prior-art, hot plenums 22A, 22B and cold plenums 24A, 24B alternate along the row, as shown in FIG. 1, such that racks 20A and 20B share the cold-air supply provided by cold plenum 24A, while racks 20B and 20C share the hot-air exhaust provided by hot plenum 22B. This sharing scheme continues along the row as suggested by the air-flow arrows at the extreme right and left of the figure. As shown in FIG. 1, the hot plenums 22A and 22B are closed at their bottoms, while the cold plenums 24A and 24B are closed at their tops. As shown by the arrows 26A, 26B, 26C and 26D, air flows upward from the raised floor 28, through holes in floor tiles, into the cold plenums 24A and 24B, and is drawn into the adjoining racks 20A, 20B and 20C as indicated by the arrows 27. After flowing across the electronics in a rack (generally in the form of circuit cards, as more fully described with respect to FIG. 3A), the air emerges into a hot plenum 22A or 22B, as shown by the arrows 29, where it is joined by air emerging likewise from the rack on the opposite side of the hot plenum. This air flows upward to the exhaust, generally to be re-circulated after cooling.

[0005] The present invention is based on a discovery of a problem with this prior-art scheme; namely, that it does not efficiently use the space occupied by the plenums, because the constant width of the plenums is not matched to the variable volumetric flow rate of air that the various horizontal cross sections of the plenums must carry.

[0006] Specifically, the volumetric flow rate of air in each cold plenum is greatest at the bottom, because the bottom cross section must carry the full complement of air to feed the entire pair of adjoining racks. As distance z from the floor 28 increases, the volumetric flow rate in the cold plenum gradually diminishes as air flows into the racks. Finally, near the top of the cold plenum, the volumetric flow rate is nearly zero, inasmuch as nearly all the air has flowed into the racks.

[0007] This inefficiency of the prior art is depicted graphically in FIG. 1 by the upwardly decreasing thickness of the arrows 26A, 26B, 26C and 26D in the cold plenums 24A and 24B. The arrows' thickness decreases to emphasize that a cold plenum carries a lower and lower volumetric flow rate of air as z increases, inasmuch as its air is drawn into the adjoining racks as indicated by arrows 27. If the distribution of electronics is uniform top to bottom, the diminution of air flow in the cold plenums is roughly linear in z. In particular, the upward air flow is zero at the top of the cold plenum. In spite of this variable flow situation, the cross-sectional area of the prior-art cold plenum is the same top to bottom. Thus space is wasted near the top of the cold plenum.

[0008] Likewise, the inefficiency of the prior art is depicted in FIG. 1 by the upwardly increasing thickness of the arrows 32A, 32B, 32C and 32D in the hot plenums 22A and 22B. The arrows' thickness increases to emphasize that a hot plenum carries a higher and higher flow rate of air as z increases, inasmuch as it collects air along the entire height of the adjoining racks as indicated by arrows 29. Finally, at the top, the volumetric flow rate in the hot plenums 22A and 22B is a maximum, because the top cross section must carry the accumulated complement of air that has emerged from the entire height of the adjoining racks. In spite of this variable flow situation, the cross sectional area of the prior-art hot plenum is the same top to bottom. Thus space is wasted near the bottom of the hot plenum.

[0009] It is an object of the invention to provide an arrangement for the flow of coolant that efficiently and effectively cools heat-producing devices.

[0010] It is another object of the invention to provide an arrangement for the flow of coolant that efficiently and effectively cools racks of electronic equipment.

[0011] It is a further object of the invention to provide an arrangement for the flow of coolant which is efficient in the utilization of valuable space.

[0012] Thus, the invention is directed toward an apparatus that uses a fluid cooling medium to cool a plurality of heat-producing devices arranged in a row along a generalized coordinate direction, with a space between each adjacent pair of devices. The apparatus comprises, in each space, a partition that defines a boundary between a first plenum and a second plenum, the first plenum being used to carry cooling medium across an entrance and thence into a first heat-producing device located on a first side of the partition facing the first-plenum, and the second plenum being used to carry cooling medium away from a second heat-producing device located on a second side of the partition facing the second-plenum and thence across an exit, wherein the partition is disposed so that the first plenum becomes smaller in cross-sectional area as distance increases from the entrance, and the second plenum becomes larger in cross sectional area as distance decreases toward the exit.

[0013] The heat-producing devices may be disposed in parallel to one another, with a first plenum and a second plenum disposed between each adjacent pair of heat-producing devices, and the first plenum and the second plenum being complimentary in size and shape so as to occupy the space between adjacent heat-producing devices.

[0014] The partition is angled with respect to the two adjacent heat-producing devices so as to extend from a position that is, at the entrance, further from the first heat-producing device than it is from the second heat-producing device, and is, at the exit, further from the second heat-producing device than it is from the first heat-producing device.

[0015] The heat-producing devices may be collections of equipment disposed in racks, and in particular may be electronic equipment disposed in racks. The apparatus may further comprise a floor for supporting the racks, the floor having a space beneath it for conducting the cooling medium to the first plenums. Floor tiles with openings may be disposed so that the cooling medium flows through the openings from the space below the floor into the first plenums.

[0016] The apparatus may further comprise a fluid-moving apparatus for facilitating flow of the cooling medium. The fluid-moving apparatus may circulate the cooling medium from the second plenums to the space below the floor. The fluid-moving apparatus may comprise a plurality of fans. The fans may be disposed in planar arrays adjacent to the heat-producing devices.

[0017] The fluid-moving apparatus may comprise apparatus that circulates the cooling medium from the second plenums to the space below the floor, and a plurality of fans the fans disposed in planar arrays adjacent to the heat-producing devices. The apparatus may further comprise cooling apparatus for cooling the medium after it leaves the second plenums and prior to returning it to the space beneath the floor.

[0018] The partition may be substantially planar or it may be curved, and in particular, it may be curved to be convex toward the first plenum. The partition may have at least one bend along a line so as to define a plurality of planar regions. The at least one bend in the partition may reduce the cross section of the first plenum and increases cross section of the second plenum.

[0019] The partition may have a bend that results in a first region having a first angle θ₁ with respect to a plane of the heat-producing device through which cooling medium enters the heat producing device, and a second region having a second angle θ₂ with respect to the plane, where θ₁ is greater than θ₂, and the first region is smaller in size than the second region.

[0020] The second plenum may be larger in volume than the first plenum.

[0021] The apparatus of the invention may be combined with at least one row of heat-producing devices in a room, and at least one fluid-moving apparatus for facilitating flow of the cooling medium.

[0022] The invention is also directed to an arrangement for a room, comprising spaced parallel units of heat-producing equipment, and between the units of equipment, a partition defining a boundary between a first plenum for conveying a cooling medium to enter a first piece of the equipment and a second plenum for conveying the cooling medium from a second piece of the equipment, wherein the partition is disposed so that the first plenum becomes smaller in cross sectional area as distance increases from where cooling fluid enters the first plenum, and so that the second plenum becomes larger in cross sectional area as distance decreases to where the fluid leaves the second plenum.

[0023] The invention is further directed to a method for configuring apparatus for cooling rows of equipment with a cooling medium, comprising placing a partition so as to define a boundary between a first plenum for carrying the. cooling medium to enter the rows of equipment and a second plenum for carrying the cooling fluid after it has cooled the equipment, wherein the partition is placed so that the first plenum becomes smaller in cross sectional area as distance increases from where cooling fluid enters the first plenum, and so that the second plenum becomes larger in cross sectional area as distance decreases to where the fluid leaves the second plenum.

[0035] Variations described for the present invention can be realized in any combination desirable for each particular application. Thus particular limitations, and/or embodiment enhancements described herein, which may have particular advantages to the particular application need not be used for all applications. Also, it should be realized that not all limitations need be implemented in methods, systems and/or apparatus including one or more concepts of the present invention.

[0036] Referring to FIG. 2, the inter-rack spaces 30A, 30B, 30C, 30D (configured in the prior art of FIG. 1 as alternating hot and cold plenums 22A, 24A, 22B, and 24B) are instead r configured, in accordance with the invention, by dividing each inter-rack space, via a sloping partition, into a complimentary pair of tapered hot and cold plenums. Thus the inter-rack space 30A is divided by sloping partition 40A into the cold plenum 42A and hot plenum 44A; the inter-rack space 30B is divided by sloping partition 40B into cold plenum 42B and hot plenum 44B; the inter-rack space 30C is divided by sloping partition 40C into cold plenum 42C and hot plenum 44C; and the inter-rack space 30D is divided by sloping partition 40D into cold plenum 42D and hot plenum 44D. Thus racks 20A, 20B, 20C receive cold air from cold plenums 42B, 42C, 42D respectively, and exhaust hot air into hot plenums 44A, 44B, 44C respectively.

[0037] This novel arrangement of plenums exploits the observation, described earlier, that volumetric flow rate in the cold plenum decreases with distance z from the raised floor shown generally as 28, whereas volumetric flow rate in the hot plenum increases with z. Floor 28 has tiles 45, above a space 46, for conducting the cooling medium, space 46 being defined between tiles 45 and a floor 47. That is, the hot and cold plenums are naturally complementary; at a height where the volumetric flow rate in minimum in one, it is maximum in the other, and vice versa. By segregating the hot and cold plenums, the prior art fails to exploit this complementarity. In contrast, the present invention deliberately exploits it. By integrating a hot and cold plenum between each pair of racks, and separating the hot plenum from the cold by a sloping partition, the width (i.e. the cross sectional area) of each plenum is roughly matched, as a function of z, to the local volumetric flow rate. Moreover, because the partition walls 40A, 40B, 40C, 40D are sloped, the air flowing through each rack turns corners that are less sharp than in the prior-art scheme. For both reasons, it is therefore possible with this invention, in a fixed amount of plenum space, to incur less pressure drop and thereby to carry more air (for a given arrangement of fans or blowers), leading to enhanced cooling and lower temperatures of the electronics.

[0038] Thermal insulation of the sloping partition may or may not be required depending on the particular application. As a guideline, it is helpful to compute the heat-transfer rate Q across the sloping partition as a fraction of the total heat-transfer rate Q_TOTAL through the rack. If this fraction Q/Q_TOTAL is small, then the loss through the wall may be relatively negligible and insulation may not be needed. Let k, ρ, c, v, α, and Pr be respectively the thermal conductivity, density, specific heat, kinematic viscosity, thermal diffusivity, and Prandtl number of air. Let U be the free-stream velocity at mid-height on the partition wall, L be the length of the partition wall in the flow (Z) direction, A_WALL be the wetted area of the partition wall, V be the volumetric flow rate of air through the rack, ΔT be the air-temperature rise through the rack, and h be the film-transfer coefficient for convective heat transfer at the wall. Then,
Q=hA_WALLΔT  (1.1)
Q_TOTAL=ρcVΔT,  (1.2)
where an appropriate expression for h is that for the average convective heat transfer on a flat plate,

[0039] $\begin{array}{cc}h=0.664k{\hspace{0.17em}}^3\sqrt{\mathrm{Pr}}\sqrt{\frac{U}{\mathrm{vL}}},& \left(1.3\right)\end{array}$
as given by Eckert and Drake, Heat and Mass Transfer, McGraw Hill, 1959, which is hereby incorporated by reference in its entirety. Writing
V=U A_PLENUM  (1.4)
where A_PLENUM is the cross-sectional area of the hot plenum at mid-height, and using the definitions of Prandtl number,

[0040] $\begin{array}{cc}\mathrm{Pr}\equiv \frac{v}{\alpha }=\frac{\rho cv}{k},& \left(1.5\right)\end{array}$
and Reynolds number based on L,

[0041] $\begin{array}{cc}{\mathrm{Re}}_L\equiv \frac{\mathrm{UL}}{v},& \left(1.6\right)\end{array}$
it is found that substitution of eqs. (1.3) through (1.6) into eqs. (1.1) and (1.2) yields

[0042] $\begin{array}{cc}\frac{Q}{Q_{\mathrm{TOTAL}}}=0.664{\mathrm{Pr}}^{-2/3}{\mathrm{Re}}_L^{-1/2}\left(\frac{A_{\mathrm{WALL}}}{A_{\mathrm{PLENUM}}}\right).& \left(1.7\right)\end{array}$
The Prandtl number of air is about Pr=0.7. For a typical rack, the area ratio in equation (1.7) is

[0043] $\begin{array}{cc}\left(\frac{A_{\mathrm{WALL}}}{A_{\mathrm{PLENUM}}}\right)\sim 10& \left(1.8\right)\end{array}$
and the length of the sloping partition is
L˜2 m.  (1.9)
A typical value of velocity is
U˜5 m/s.  (1.10)
Substituting these values into eq. (1.7) yields

[0044] $\begin{array}{cc}\frac{Q}{Q_{\mathrm{TOTAL}}}=0.015& \left(1.11\right)\end{array}$
That is, in such a typical case, only 1.5 percent of the heat flux Q_TOTAL crosses through the partition wall, so in such a case, it may be permissible to leave the partition wall un-insulated, depending on the tradeoff of performance and cost.

[0045] Inasmuch as the invention provides optimal, space-efficient cooling of electronics in raised-floor installations, it is important for several reasons:

[0046] A. Keeping electronic chips cool is extremely important. According to a common rule of thumb used for processing chips, a chip's lifetime doubles and its performance (speed) increases 2–3 percent for every 10 degrees Celsius it is run below its maximum (spec) temperature.

[0047] B. The cost of floor space in such an installation is high; air-flow plenums may occupy a considerable fraction of the total and thus represent a considerable cost.

[0048] C. Inasmuch as raised floor tiles are always 24 inch (61 cm) square, it is advantageous to package the racks on a center-to-center distance that is a “round number”, such as 24 inches (61 cm) or 36 inches (91.4 cm), because then each rack (or every other rack) is synchronized with the floor tiles, such that only one type (or two types) of raised-floor opening holes must be designed for entrance to the cold plenums.

[0049] D. Under a fixed-space constraint such as that described in (C.), space-efficient plenums may make the difference between success and failure for a design where a large amount of electronics must be packaged in the fixed space and yet remain cool. In the prior-art method, any attempt to meet such a fixed-space requirement simply by shrinking the width of the plenums is ineffective, because the air flow is thereby constricted. This leads to higher temperatures and therefore shorter life and, compromised performance of the electronic components.

[0050] E. Space-efficient plenums allow racks to be closer together, which may lead to improved signal integrity on interconnecting cables.

[0051] Referring to FIG. 3A and FIG. 3B, the invention may be modeled in a thermal prototype comprising a rack 50 of electronics of the type used in a super-computer system. Rack 50 is flanked on the left side by a prototype hot plenum 52, having a size defined by the position and angle of the sloping hot-side partition 54. Likewise, rack 50 is flanked on the right side by a prototype cold plenum 56, having a size defined by the position and angle of the sloping cold-side partition 58. Prototype. rack 50 is thus analogous to rack 20B in FIG. 2, and the hot-side and cold-side partitions 54 and 58 are analogous, respectively, to the sloping partitions 40B and 40C in FIG. 2, such that the prototype system is one “unit cell” of the row of racks in FIG. 2.

[0052] During operation of the prototype system, heat is generated by several hundred mock-up, heat-sinked chips (not shown) located on circuit cards 57, each of which lies in a horizontal plane inside rack 50. A representative sample of fifty-one of these mock-up chips are instrumented with thermocouples (not shown) to measure the chips' case temperatures.

[0053] Circulation of air, along paths such as 60, is provided by a planar array of fans 62, which form a continuous wall, and thus provide cooling by forced convection. These fans may be axial fans of the type manufactured by Delta Electronics or ebm Industries. Other air-moving devices may be used in addition to or instead of the wall of fans. For example, forced convection may be enhanced by an additional wall of fans upstream of the circuit cards 57, or alternatively may be provided by one or more blowers located either at the bottom of the cold plenum 56 or at the top of the hot plenum 52. The advantage of the wall of fans 62 vis-à-vis blowers is that the pressure rise provided by the fans is distributed across the entire plane at the left in FIG. 3A of the stack of circuit cards 57, thereby allowing flow balancing among the stack of circuit cards to be accomplished simply and flexibly by adjustment of fan speeds. In contrast, the pressure rise provided by a single blower is concentrated at one place, thereby requiring, for the avoidance of uneven flow through the various circuit cards, mechanical balancing mechanisms requiring tedious (and perhaps sensitive) adjustment.

[0054] In practice, forced-air circulation may also be enhanced by a floor-mounted modular air-cooling unit of the type described by C. D. Patel et al. in Computational Fluid Dynamics Modeling of High Compute Density Data Centers to assure System Inlet Air Specifications, Proceedings of IPACK2001-15622 (2001). Reference is made to FIG. 1 of this paper. However, it will be understood that any of a number of air-moving devices may be utilized with the invention.

[0055] In order to test different sizes and shapes of the plenums, the left-to-right position of the top end 63 of the hot-side partition 54 is adjusted in discrete increments by attachment via screws 66 (FIG. 3B) to any one of a series of tapped holes 68. As shown in FIG. 3A, it happens to be attached at tapped hole 70. Similarly, the left-to-right position of the bottom end 72 of the hot-side partition 54 is adjusted, independently of the top, by attachment via screws 74 (FIG. 3B) to any one of a series of tapped holes 76; in FIG. 3A, it happens to be attached to hole 78.

[0056] The situation for the cold-side partition 58 is analogous: the left-to-right position of its top end 80 is adjusted in discrete increments by attachment via screws 82 (FIG. 3B) to any one of a series of tapped holes 84; in FIG. 3A, it happens to be attached to tapped hole 86. Similarly, the left-to-right position of the bottom end 88 of the cold-side partition 58 is adjusted, independently of the top, by attachment via screws; 90 (FIG. 3B) to any one of a series of tapped holes 92; in FIG. 3A, it happens to be attached to hole 94.

[0057] Thus there are four independent adjustments at the four series of holes 68, 76, 84, 92, thereby allowing simulation of the non-tapered plenums used in the prior art (FIG. 1), as well as the tapered plenums in accordance with the invention (FIG. 2).

[0058] The four adjustments represent four degrees of freedom in the arrangement of FIG. 3A and FIG. 3B. Among these four degrees of freedom, two constraints are imposed. First, the two walls must be parallel to create a consistent geometry for the several racks in a row, such as shown in FIG. 2. Second, a fixed pitch between racks is selected. To obtain the benefits A. to E. outlined above, a rack pitch of 36 inches (91.4 cm) is selected.

[0059] Thus, there are two degrees of freedom left, which may be represented by the following two parameters:

• [0060] θ=Plenum wall angle with respect to vertical, as shown in FIG. 3A.
• [0061] β=Fraction of plenum space devoted to the hot side (as measured from left and right edges of circuit cards to plenum walls at a point Z midway from lowest circuit card to highest). That is, referring to FIG. 3A,

[0062] $\beta \equiv \frac{x_H}{x_H+x_C}.$

[0063] With this parameterization, the prior art is confined to θ=0, whereas this invention permits θ to be greater than or equal to 0. It may be shown mathematically that, because of geometrical limitations wherein the bottom of the hot-side partition 54 abuts the lower-left edge of the rack 50, and/or the top of the cold-side partition 58 abuts the upper-right edge of the rack 50, only a portion of the θ-β space may actually be attained, as shown by the triangle SUV in FIG. 4. Within the confines of this triangle, the points indicated by the lettered dots A through S are tested experimentally. Thus the prior art (θ=0) is limited to cases A through E, whereas the invention permits any of the cases from A to S.

[0064] To evaluate performance, for each case, the system is run (typically for an hour or so) with a constant cold-air-inlet temperature of 20° C., until thermal equilibrium is reached. At equilibrium, the temperatures of the fifty-one mock-up chips previously mentioned (whose temperatures are monitored by thermocouples) are recorded, and statistics are computed such as the average temperature T_AVE, the maximum temperature T_MAX, and the standard deviation of temperature over the fifty-one-chip ensemble. For cases A through S, T_AVE and T_MAX. are shown in graphical form in FIGS. 5 and 6 respectively; the statistics are also shown in tabular form in Table 1 below.

[0065]

TABLE 1
					Std.
Case	θ (degrees)	β	TAVE (° C.)	TMAX (° C.)	Dev. (° C.)
A	0.00	0.3914	64.07	81.84	10.46
B	0.00	0.4951	57.99	69.25	5.92
C	0.00	0.5989	57.66	67.36	3.93
D	0.00	0.7026	62.82	72.48	4.66
E	0.00	0.7545	73.56	84.45	5.72
F	4.68	0.3777	58.49	77.39	10.30
G	4.68	0.5074	52.44	62.84	5.93
H	4.68	0.5592	50.94	62.78	5.99
I	4.68	0.6111	51.11	63.81	5.94
J	4.68	0.6630	51.43	61.06	4.81
K	4.68	0.7149	52.66	59.79	4.32
L	4.68	0.7408	52.84	61.31	4.59
M	5.95	0.4235	54.23	70.94	8.63
N	5.53	0.7194	50.52	58.47	4.32
O	7.64	0.4845	50.31	64.00	6.49
P	7.22	0.6767	47.55	54.65	3.94
Q	9.31	0.5455	47.89	59.57	5.15
R	8.89	0.6340	46.68	53.98	3.84
S	10.55	0.5913	46.47	56.25	4.34

[0066] The best case possible with the prior art (θ=0) is case C, whereas the best case possible in accordance with the invention is either Case R or Case S, depending on whether average chip temperature T_AVE or maximum chip temperature T_MAX is more important. The statistics for these cases, extracted from Table 1, are in Table 2.

[0067]

TABLE 2
Case	TAVE (° C.)	TMAX (° C.)	Comment
C	57.66	67.36	Best case possible
			with prior art
R	46.68	53.98	Best case with invention
			(lowest TMAX)
S	46.47	56.25	Best case with invention
			(lowest TAVE)

[0068] Thus, using this invention, the average chip for Case R is roughly 11° C. cooler, and the hottest chip 13° C. cooler, than that achievable with the best prior-art case (Case C). According to the rule of thumb previously stated, the average chip lifetime is thus more than doubled by use of the invention.

Alternative Embodiments

[0069] Certain variations on the basic scheme shown in FIG. 2 are within the scope of this invention. For example, while the sloped partitions between the hot and cold plenums have been shown as planar, they may instead have a curved profile as illustrated by partitions 100 in FIG. 7, where the partition is convex toward the cold plenum.

[0070] FIG. 8 illustrates an alternative in which the sloping partitions are each bent along a line at 102, so as to form relatively small lower rectangular regions 104 at a fairly large angle θ₁, and relatively larger rectangular regions 106 at a relatively smaller angle θ₂. The bend in FIG. 8, like the curve in FIG. 7, causes the partition to be convex toward the cold plenum. Yet another alternative, a straightforward extension of FIG. 8, is to bend the sloped partition at a plurality of lines analogous to the line shown as 102 in FIG. 8.

[0071] The arrangements shown in FIGS. 7 and 8, in which the sloped partitions 54 and 58 shown in FIG. 3 would be convex leftward rather than straight, may be advantageous when circuit cards 57 exist close to the bottom of rack 50, because the leftward convexity of partition 54 would prevent undue restriction of the exhaust flow from the lowest fans 62.

[0072] The potential advantage to be gained by curving or bending the sloping partitions may be bounded by measuring, in the prototype apparatus described in FIGS. 3A and 3B, the limiting cases when one or both of the sloping partitions 54 and 58 are removed altogether. These limiting cases represent unrealistic situations where plenum space is unconstrained, air flow through the rack is thereby maximized, and electronic chip temperatures are thereby minimized.

[0073] The results, shown in FIG. 9, indicate that the average chip temperature in the most ideal case (Case L₃, where both sloping partitions 54 and 58 are removed) is about 7° C. lower than the best case (Case S shown in FIG. 5) obtained with sloping partitions that are planar. Thus, for the prototype embodiment, curving or bending the sloping partitions may potentially obtain some fraction of this 7° C. temperature drop, but it cannot obtain more.

[0074] While the present invention has been illustrated in the context for cooling electronic equipment, and in particular, electronic equipment placed in racks in a data storage center, it will be understood that the principles of the present invention may be utilized to cool many different types of electronic equipment, and many types of equipment in general. While the preferred cooling medium discussed herein is air, it will be appreciated that in accordance with the principles of the invention, a variety of fluids may be used in applications where entry by human being is not as important, including other gases and non-corrosive and non-conducting liquids such as cooling oils. In such cases, provisions are made for containing the fluid by appropriate fluid-tight enclosures.

[0075] It is noted that the foregoing has outlined some of the more pertinent objects and embodiments of the present invention. The concepts of this invention may be used for many applications. Thus, although the description is made for particular arrangements and methods, the intent and concept of the invention is suitable and applicable to other arrangements and applications. For example, although the heat-producing devices are described herein as racks arranged in a rectilinear row, the principles of the invention also apply to any heat-producing equipment cooled by a fluid, and to cases where the equipment is laid out along a generalized coordinate, which includes curvilinear as well as rectilinear coordinates. For example, racks of electronic equipment may be placed in a circle so that the spaces between the racks are larger at the circumferential perimeter than elsewhere. Further, the circuit cards and the racks in which they are placed, when viewed from the top, may be shaped substantially as sectors of a circle, with spaces between the racks also being shaped substantially as sectors. Thus the invention may be applied to these geometries, and to many other geometries.

[0076] It will be clear to those skilled in the art that other modifications to the disclosed embodiments can be effected without departing from the spirit and scope of the invention. The described embodiments ought to be construed to be merely illustrative of some of the more prominent features and applications of the invention. Other beneficial results can be realized by applying the disclosed invention in a different manner or modifying the invention in ways known to those familiar with the art. Thus, it should be understood that the embodiments have been provided as an example and not as a limitation. The scope of the invention is defined by the appended claims.