GPU cluster in cooling data center

GPU cluster in a cooled data center reveals the infrastructure behind training artificial intelligence models.

About this subject

GPU clusters are the backbone of modern deep learning. Unlike traditional CPUs, GPUs have thousands of specialized cores for parallelism, ideal for processing large datasets in tasks like image recognition and natural language processing. In data centers, these clusters are organized in racks, interconnected by high-speed networks such as InfiniBand or 100 Gbps Ethernet to minimize latency during distributed training.

Cooling is a critical aspect. A single GPU can consume 300, 400 watts, and racks with dozens generate intense heat. Modern data centers use direct-to-chip liquid cooling or dielectric immersion systems, which are more efficient than traditional air conditioning. Google, for example, uses recycled water cooling in its data centers, achieving a PUE (Power Usage Effectiveness) close to 1.1.

The cluster in the image likely contains GPUs like NVIDIA A100 or H100, widely used in public clouds (AWS, Azure, Google Cloud) and supercomputers. Training models such as GPT-4 or Stable Diffusion may require thousands of GPUs running for weeks. The energy consumed rivals that of a small town, sparking debates about AI sustainability.

Fun fact: the world's largest GPU cluster, Frontier at Oak Ridge National Laboratory, has over 37,000 AMD GPUs and achieves 1.1 exaflops of performance. In Brazil, the Santos Dumont supercomputer at LNCC uses NVIDIA GPUs for research in materials science and computational biology.

Frequently Asked Questions

Why are GPUs used for AI instead of CPUs?

GPUs have thousands of smaller cores than CPUs, enabling massive parallel processing essential for the matrix calculations in deep learning. CPUs are better for sequential tasks.

How much energy does a GPU cluster consume?

A rack with 8 A100 GPUs can consume about 3, 4 kW. Large clusters with thousands of GPUs can draw tens of megawatts, equivalent to the consumption of a small town.

How does liquid cooling work in data centers?

Direct-to-chip liquid cooling uses water or dielectric fluid circulating through tubes to the GPUs, absorbing heat more efficiently than air. Immersion systems submerge components entirely in non-conductive liquid.