Understanding the Impact of Data Center Liquid Cooling on Energy and Performance of Machine Learning and Artificial Intelligence Workloads

Abstract Traditionally, Data Centers (DC) have used air cooling for IT equipment, but as Graphics Processing Units (GPUs) evolve, they demand more power and sophisticated cooling. Aiming for efficiency, Direct Liquid Cooling (DLC) emerges as a promising solution. We evaluated the effectiveness of DLC versus traditional air cooling on a Microsoft G50 GPU server performing AI/ML tasks. The results indicated that DLC greatly enhances GPU performance, increases efficiency by 2.7% in Gflops/s, cuts power usage by 12%, reduces execution times by up to 6.22%, and lowers chip temperatures by 20° compared to air cooling. Our research develops an overall performance metric that considers data center, hardware, and chip levels, concluding that DLC is extremely beneficial for AI workloads, increasing energy savings and balancing performance with power requirements.