Key Takeaways
- Triple-digit temperatures are putting data center cooling systems to the test across the U.S.
- AI workloads are generating far more heat than traditional web hosting infrastructure was designed to handle.
- Technologies like liquid cooling, free cooling, and AI-powered controls help data centers stay online, but even those solutions have their limits.
Parts of the east coast is forecasted to hit more than 100°F this week, with overnight temperatures barely dipping below 80. More than 100 million Americans are under major or extreme heat alerts this week.
Meanwhile, millions of servers are running 24/7 inside of data centers across the country. How are they not overheating?
The answer depends on what those servers are doing. Traditional data centers use about 5 to 10 kW per rack, which is enough for standard air conditioning to keep cool. AI, though? That’s a completely different animal.
AI servers are now requiring 40 to more than 100 kW per rack, with some peaking beyond 120 kW. It’s like sticking a window unit in a ballroom in the middle of July: It just isn’t enough.
How Data Centers Are Keeping AI Cool
Data centers have been trying out all types of new cooling methods the past couple of years, especially as grid strain gets heavier and utility costs are being spread out to local communities.
| Cooling Method | How It Works | AI Ready | Pros | Cons |
|---|---|---|---|---|
| Rear-Door Heat Exchanger | Chilled water cools hot air exiting the back of the rack. | Limited for highest-density AI | Easy retrofit; familiar technology | Can struggle with next-generation AI racks |
| Direct-to-Chip | Liquid circulates through cold plates mounted directly on CPUs/GPUs. | Yes | Highly efficient; targets hottest components | Requires liquid plumbing; air still cools some components |
| Immersion Cooling | Entire servers operate submerged in dielectric fluid. | Yes | Excellent heat removal; reduced dust and server fans | Higher deployment cost; specialized maintenance |
| Predictive Cooling Control | AI software predicts heat loads and automatically adjusts cooling equipment. | Yes | Improves efficiency; lowers cooling energy use | Depends on existing cooling infrastructure |
As for whether they’re enough — or cost-effective — is up for debate. Here’s what some data center operators are doing.
Rear-Door Heat Exchangers
A rear-door heat exchanger works a lot like a car radiator: It’s mounted to the back of a server rack so that chilled water can pull heat out of the hot air before it leaves the cabinet. Since it can be added to existing racks, it’s an easy and popular retrofit option.
It’s also not enough for most modern-day AI workloads. A single NVIDIA B200 or B300 server draws more than 10 kW on its own, which is close to what an entire traditional rack used to need. Operators are already designing cooling infrastructure for racks approaching 250 kW as next-generation AI hardware rolls out.
Direct-to-Chip
Instead of cooling the entire server, this method cools the processor directly. A cold plate is mounted on the chip and carries it away with circulating content. Traditional air cooling still handles about 20-30% of the system.
Meta is also known for using direct-to-chip cooling. The company has also developed some hybrid approaches, like air-assisted liquid cooling (AALC), to retrofit older facilities, but its newest AI-optimized data centers primarily rely on closed-loop, direct-to-chip cooling.
Immersion Cooling
Immersion cooling does what it sounds like: Upon installation, servers sit in a bath of nonconductive liquid that constantly pulls heat away from their components. It’s like taking a cold shower after a hard workout…except it’s submerged 24/7.
A handful of hyperscalers, including Microsoft, have deployed immersive cooling to handle especially dense AI workloads. While it doesn’t eliminate maintenance completely, it does reduce dust buildup, eliminates the need for server fans, and uses a dielectric fluid that can last for years.
Predictive Cooling Control
One of the newest approaches uses AI to manage cooling systems for AI data centers and servers themselves.
By predicting where heat will build up, AI can automatically adjust cooling equipment before temperatures get too high. A recent study found that this kind of predictive optimization actually reduces cooling energy by 30%. Not bad at all.
Google, for example, developed an AI system, DeepMind, that predicts heat loads and automatically adjusts pumps, fans, and cooling equipment. Doing so has reduced Google’s energy usage by 40% in its data centers.
Even Modern-Day Cooling Has Its Limits
Every solution comes with a little asterisk. You know, the one that basically says “conditions apply.”
Liquid cooling is incredibly effective, but it also isn’t cheap. Liquid cooling requires new plumbing, monitoring systems, and electrical upgrades. It’s a major cost that not every operator can justify, which is why it’s often left to hyperscalers or operators who can afford to renovate their existing facilities.
But even then, cost isn’t always the biggest hurdle. Many older buildings where data centers have existed for decades simply don’t have the electrical capacity to power AI racks.
Traditional vs. AI Server Rack Power Requirements
Traditional enterprise racks typically draw 5-15 kW, while today's AI training racks often require 80-150 kW. Some next-generation designs are targeting 500 kW per rack.
- Min kW
- Max kW
Per TechRadar's findings, modern AI racks can use 80 to 150 kW per rack, compared to 5 to 15 kW for many legacy enterprise racks. Some future designs are even going to target 500 kW per rack! Upgrading utility services can take years, especially as power grids are already struggling to keep up with growing demand.
For example, Northern Virginia — the world’s largest data center market — is experiencing delays because utilities don’t have enough transmission capacity.
The U.S. grid in general is expected to add 445 GW of new capacity by 2030, which is basically the same thing as building another grid the size of the nation's largest regional power system. Amid regulations, NIMBYs, and actual connection and funding, the million-dollar question is: How will the U.S. build enough power to keep pace with AI?
Unfortunately, there is no single answer, no single breakthrough technology. Instead, we’re looking at a massive infrastructure buildout. And yeah, that’s going to take a lot of time.
