This website uses cookies primarily for visitor analytics. Certain pages will ask you to fill in contact details to receive additional information. On these pages you have the option of having the site log your details for future visits. Indicating you want the site to remember your details will place a cookie on your device. To view our full cookie policy, please click here. You can also view it at any time by going to our Contact Us page.

Cool Water

15 March 2007

CoolTherm water-cooled installation in UK financial institution

Water cooling for large data centres continues to be a realistic option because as Dr Peter Koch explains, this well established technology not only provides for 100 per cent server optimisation but also offers major reductions in carbon footprint and facility costs

PROVIDING A RELIABLE, SECURE AND TECHNICALLY EFFICIENT data centre is a perquisite for any data facility. However, rapidly changing technology and significant increases in energy and environmental demands have created major new pressures on data centres to adapt new methods to maintain operating viability.

The increasingly significant role of server cabinets, particularly in larger data centres has necessitated new enclosure development that not only satisfies the thermal complexities of the exponential rise in server power but can also provide dramatic savings in overall facility operating costs with 40 per cent plus reductions reported.

The thermal demands of a traditionally air-cooled high-density large data centre, particularly when using the latest high power blade servers, is such that conventional CRAC (Computer Room Air Conditioning) technology can no longer satisfy server-cooling requirements with a viable, server to cabinet ratio. The power demands of the latest generation of blade servers create cabinet heatloads that cannot be met by conventional means. This occurs through the combined effect of computer room physics, enclosure runs and CRAC limitations, which cannot fully function unless the cabinets are ludicrously under-utilised.

For large data centres the basic calculation adopted by industry for CRAC reliant operation, allows 3 sq m total floor space per cabinet and surrounding area and 1.2 to 1.5kW of heat load per cabinet. This computes to 3-4kW per cabinet. Although it is possible to achieve circa 8kW from a CRAC reliant cabinet if very carefully located within the overall computer room, it is apparent that the latest blade servers which currently take 5kW from a 9U server, would accept just one 9U server per 47U high (2.1m) server cabinet, within a typical large CRAC reliant facility. The wasted space and very poor economics being selfevident.

£millions of savings per annum for a single facility can be achieved by employing the latest generation of ‘water-cooled’ thermally managed server cabinets which also provide the added benefit of meeting higher environmental standards by reducing all-important carbon foot-print.

A 500 cabinet, conventional air-cooled CRAC reliant cabinet system could readily reduce to a mere 100 CoolTherm cabinets or less. This would achieve a far better, controlled operating environment, in 80 per cent less space and providing significant built-in future proofing to accept higher loads as demand evolves.

Conventional air-cooled, CRAC reliant cabinets will continue to satisfy a significant portion of overall demand. For example, Knurr has 25,000 of its Miracel conventional air-cooled 19” cabinets currently active within the City of London alone.

A new improved approach to large data centre server cabinet cooling has become essential. The thermal demands of smaller data centre networks, even with relatively high heat load and high data transfer, presents a less demanding thermal control issue, or at least is more manageable compared to larger high quantity cabinet installations. A single or limited cabinet run for example of 5-20 cabinets, can be adequately cooled, even under high heat loads, if the scale of the room, cabinet layout and CRAC capability are adjusted accordingly. Such flexibility is not viable in a large data centre with hundreds of cabinets, especially if efficiently populated with 100 per cent server optimization of high-power blades.

Water-cooled server cabinet technology is fast becoming the new standard for larger, high data transfer organisations, either as a total solution or often combined with conventional server cabinets. Water for computer cooling is not new. By the late 1980’s, 90 per cent of mainframe computers were water-cooled as the heat dissipation of bipolar chips reached their limits.

Knurr (part of Emerson Network Power) has been producing enclosures for the IT/electronics/electrical/ transportation and general industry sectors for some 60 years. A pioneer of server cabinet cooling technology, it is the first enclosure company to develop commercially available air-to-water cooled cabinets by installing more than 100 CoolTherm, 37U x10kW capacity server cabinets in 2003, as part of the massive Grid computing (LCG) project - the world’s largest international scientific grid. Subsequent research and development has lead to the current range of CoolTherm cabinets providing up to 45kW of cooling capacity per single 19”cabinet, the highest capacity on the market.

The exponential growth in network server power demands and the parallel demand for additional cooling whilst being a relatively recent phenomena, has seen water-cooled take up rapidly grow. Knurr now have a significant number of CoolTherm data centre installations in operation within the UK and about 1,500 CoolTherm cabinet systems in operation worldwide.

An independent TCO (Total Cost of Ownership) study of Knurr CoolTherm cabinets, showed an 80 per cent plus reduction in floor space and cabinet quantities and their attendant costs, and 40 per cent overall facility energy and operating cost savings which for larger sites runs to millions per annum.

Water as a readily available medium, offers a 3,500 better thermal efficiency than air. Installation is very straight forward, requiring no specialised tools. Once the pipe runs are installed below a conventional computer room suspended floor, its very easy to expand the network by simply adding more cabinets and connecting into to the flow and return with flexible plastic lines. Each CoolTherm cabinet has a specially developed high-efficiency Vshaped heat exchanger that sits in the base of the cabinet, well away from any electronics, in a closed-loop. Totally controllable, each cabinet is fully sealed from room ambient air, making its operation independent of normal room temperature and without reliance on a CRAC system. Under a 35kW server load for example and with a cold-side air inlet temperature of 25ºC, the installed servers outlet pre-cooled air temperature is 50ºC. Nominal cold-water inlet temperature to the CoolTherm heat-exchanger being 12ºC with return water at 22ºC.

The cabinets’ chilled water supply can be taken from the normally available supply within larger buildings and controlled via the building services, or Knurr’s CoolTrans. This controls water flow rate and temperature and monitors and adjusts the specific thermal needs at any one time of up to five CoolTherm cabinets.

Although other thermal mediums have recently started to be offered - CO2 for example - water is considered by far the most economic and totally safe approach. Water cooled systems operate at low pressure <3bar, whilst CO2 for example is close to 50bar and is dangerous to humans, with a statutory airborne limit of 5,000ppm. It also requires a chiller high-pressure plant and a pressure resistant steel tubing and distribution system which is both expensive and complex to expand at a later date.

With the continuous demand for operating efficiency and the significant savings, as well technical functionality that water-cooled server cabinets provide, they will increasingly become a standard element in data centre design.

● Dr Peter Koch is Knurr’s Chief Technology Officer, Senior Vice President, Products, Design & Quality.

Print this page | E-mail this page