These cooling limitations have data centers exploring alternative cooling technologies, such as liquid-to-liquid cooling systems. Compared to air cooling, liquid cooling systems rely on the superior thermal properties of liquids to remove far more heat from IT equipment.
Data centers have considerable interest in direct-to-chip liquid cooling technology. Direct-to-chip (D2C) dissipates heat by circulating liquid coolant through a cold plate mounted directly on high-heat components such as the coolant distribution unit (CDU), which regulates the flow and temperature of the coolant by transferring the absorbed heat to a water loop or an external heat exchanger.
To better understand the depth of testing required to commission a CDU into service, a research team led by Pardeep Shahi, a senior mechanical engineer with NVIDIA in Santa Clara, Calif., and mechanical engineering faculty at the University of Texas-Arlington, took a deeper look at the key parameters that impact CDU performance.
Experimental setup
The team used a closed-loop liquid cooling system consists of two row manifolds, eight rack manifolds, and 32 cooling loops equipped on each of the 32 thermal test vehicles. The eight racks are liquid-cooled, each with a rack density of 25 kW. Fluid flow is distributed from the CDU to the racks via row manifolds. “Temperature, flow, and pressure sensors are integrated into the system to measure and cross-verify data with the CDU,” Shahi said.
The CDU consists of a heat exchanger, circulating pumps, coolant monitoring sensors, and a programmable logic controller that receives inputs and feedback from all coolant monitoring devices. “The major function of the CDU is to circulate the fluid, dissipate heat, and regulate the coolant’s supply temperature at a specific flow rate within the system,” he continued. The CDU can also be used to regulate the temperature of the refrigerant supply.
Testing the CDU
Shahi and his team tested the CDU in a variety of modes to evaluate its functionality and verify its proper operation. These tests included sensor calibration to measure the temperature across the supply and return sides. T-type thermocouples were utilized and calibrated, along with uncertainty analysis to evaluate the accuracy and reliability of temperature, pressure, and flow sensors. Testing also included CDU characterization to verify the function of the CDU at different modes, filters, and mesh sizes to prove stability.
Once stability was established, the researchers ran additional tests on the CDU to see if it qualified for the data center. The team looked for constant differential pressure to determine if the CDU could maintain a given set pressure drop by regulating the pump speed and constant flow rate mode. That would determine if the CDU could maintain a given flow rate by adjusting the pump speed to verify the setpoint flow rate. Additional tests included filter characterization by measuring the flow rate across the system and pressure drop across filters and a pump failover study designed to replicate a failure scenario by focusing on one of the CDU pumps experiencing a malfunction and observing the subsequent response from the remaining operational pump.
Other experiments included a leak detection system inside the CDU, thermal stability at low load and high load conditions to maintain a given setpoint temperature, and verifying the cooling capacity curve.
Commissioning the CDU
Shahi noted that accrediting liquid-to-liquid in-row coolant distribution systems is crucial to ensure the reliable and efficient functioning of high-power data centers. “These advanced cooling technologies enhance data center performance, energy efficiency, and environmental sustainability in the current era of demanding computer workloads,” Shahi said. “They do this by effectively addressing thermal management-related challenges.”
The successful implementation and optimal performance of liquid cooling systems within data centers are contingent upon the commissioning process for CDUs. From pre-installation inspections to experimental testing, this exhaustive process examines each facet of CDU functionality and infrastructure integration within the data center.
This data is critical for commissioning a CDU and ensuring the CDU can be integrated into the current data center infrastructure without disruptions.
“From the design and implementation of liquid cooling systems to their ongoing maintenance and optimization, every aspect of these technologies warrants careful consideration and analysis,” Shahi said. “Moreover, as liquid cooling continues to gain traction within the data center industry, it is essential to explore the latest advancements and innovations in the field. From novel cooling architectures to cutting-edge cooling fluids, the landscape of liquid cooling technologies is constantly evolving, presenting new opportunities and challenges for data center operators.”
Mark Crawford is a technology writer in Corrales, N.M.
