> In the right geography — somewhere with reliably cool outdoor air
Aaahh, there’s the catch.
“Save resources on cooling by building your data centre somewhere cold, and pollute the surrounding environment by dumping your waste heat wholesale into that!”
Good job Nvidia, I almost thought we had something good there.
show comments
amluto
This opens up an interesting synergy: district heating. 45C is low but not unworkable for a district heating loop, and a data center might be able to make a nice pitch to a community if the data center offers to provide heat to a district heating system for free. This brings the value to the local community of a nearby datacenter up from near zero to potentially a few million dollars per year.
Summer is still an issue, but fun solutions are possible. With the right geology, I think it’s possible to heat an underground volume in the summer and recapture (some of) that heat in the winter. In many, many climates, annual heating costs are far higher than cooling costs, at least if people aren’t stupid with skylights. [0]
[0] As a back-of-the-envelope heuristic, heating or cooling load due to conduction and air exchange is proportional to the difference between indoor and outdoor temperature. Outdoor temperatures of -10F to 30F are not unusual in the winter and are 40-80F away from an indoor temp of 70F. But outdoor temperatures in these climates rarely exceed 95F and are mostly lower in the summer, so that’s 15-25F of cooling. And heat pumps are more efficient at smaller temperature differences.
Radiative heating is an entirely different story.
show comments
kingofmen
Points for effort, but this will do literal nothing to appease the opposition, since the "water use" thing is a myth anyway. It's probably good, it sounds like it will be more efficient and efficiency saves resources for other uses, but politically it's completely useless.
show comments
why_at
Maybe I'm being dumb, but I don't understand what the innovation is here.
I get that they're using liquid coolant at higher than usual temperatures, but why couldn't they do that before? Most of the comparison in the article is for air cooled datacenters but what about other liquid cooled ones?
Surely in all the previous datacenters that have been designed there has been someone doing the math and determining what temperature things need to run at, how much energy it will use, how much heat it all will produce, etc.
edit: just saw this:
>Previous liquid-cooled servers were hybrid: GPUs and CPUs got cold plates, but the rest of the system stayed air-cooled, with finned heat sinks designed to shed heat into moving air. In a fully liquid-cooled server, the cooling for these components needed to be completely redesigned to use liquid.
show comments
pjdesno
Water use isn't a myth.
Our data center is mostly air-cooled, with 100F hot aisles and chilled water going to heat exchangers in every third rack position, although some of our newer GPU racks use chilled water directly. For most of the year that means we don't need chillers - the warm water return is sufficiently hotter than outdoor temp that heat "flows downhill" with a bit of pumping.
But we use an evaporative cooler to carry that heat away outside, because it's more efficient. If you look up the heat of vaporization of water, that means we evaporate about 10,000 gallons of water per day per megawatt of power dissipated. We're on canal off a large river, and we don't dissipate all that many megawatts, so I believe the water use isn't significant.
Using air as the working fluid to draw heat from your machines has a limitation - humans have to breathe that air whenever they work on the equipment, so the temperature is limited. Once the exterior temperature nears your hot aisle air temp, you either need active A/C to create a heat source hotter than ambient, or an evaporative cooler to lower the "effective" ambient temp to the dew point.
Liquid cooling lets you run your working fluid a lot hotter without killing folks like me who go into the data center, although honestly 45C sounds like an incremental improvement over the 100F places like ours are already running. (although to be fair, the warm water return from the heat exchangers is no doubt somewhat less than 100F) It also lets you run your "cold" side a lot hotter - if you "chill" your water down to 100F (38C) on a hot day, it's still cold enough to carry away a lot of heat at 45C.
(I'm skipping over the fact that there are multiple heat exchanger loops involved - e.g. any system with an evaporative cooler needs a heat exchanger to keep leaves and bird shit outside the building where it belongs)
show comments
kayo_20211030
> In favorable climates, NVIDIA’s 45-degree liquid-cooling architecture ....
What's a favorable climate, apart from, obviously, Greenland? The piece is a little light on details on the correlation between outside temperatures and efficiency & cost. It'd be nice to see even a broad-strokes discussion of that.
show comments
pjdesno
Historically, cooling alone has accounted for up to 40% of a data center’s electricity consumption
The key word here is "historically". Modern data centers typically have a PUE lower than 1.2, with the .2 including not only energy spent on cooling but power distribution losses as well.
randallsquared
It was already near zero compared to various other uses. It's always depressing to me to see a lot of effort put in to "solve a problem" (with subsequent fanfare) which is only a PR or image problem in the first place.
show comments
metabagel
The NASA Ames Research Center Modular Supercomputing Facility is highly efficient, both in terms of electricity and water use. The facility isn't air conditioned. The chips are water cooled, and the inlet water temperature is pretty high I believe - I think it's 90 degrees Fahrenheit.
If anyone else is disappointed by the terrible AI slop article: It's about a fully liquid cooled data center design.
The usual way to cool servers is with air and heatsinks attached to the hot hardware, similar to how your desktop computer or laptop works. As the hardware gets denser and more powerful, you need bigger and bigger heatsinks and cooler air blown over them. At some point you can't make the heatsinks bigger because of space constraints, and you can't blow the air faster (because of noise and efficiency), so you need cooler air. That's when you start running chillers that evaporate water to cool your intake air. This is the huge water consumption that we'd like to avoid.
The next step is, obviously, liquid cooling. Again, this is similar to your fancy gaming desktop. You can dump a lot of heat to a liquid medium through a small heat exchanger inside, where you're space constrained, and you can run the liquid through a gigantic heat exchanger outside, despite the temp delta between your coolant and outside air being pretty small.
This article is about a system that's FULLY liquid cooled — CPUs, GPUs, memory, networking, the whole thing. That's the actual cool part (pun unintended). On top of that, their solution is optimized to be able to run the coolant quite warm — this obviously limits the heat flux at the hardware side, but it allows you to run the outside heat exchangers "dry", i.e. without wasting any water for its latent heat.
maerF0x0
> and the same liquid can be recirculated in a closed loop so no new water is consumed to cool the chips
That doesn't mean a datacenter doesn't have a very high _initial_ need for water, or to replace _some_ amount of leakage, replacement etc (agreed it will be way lower than say a swamp cooler). For example, they could be using millions of gallons as a sort of "ballast" to keep the water temperature very stable in the short run.
This whole projects depends on the whole stack of components to be able to tolerate 45c from memory, drives, the lighting of the building, the humans ...
show comments
t0mpr1c3
with efficient heat exchange you could get the coolant up to mash temperature (65C) and run a combined data center/brewery
sabareesh
I am pretty much doing the same but running the coolant at 40 deg C instead of 45 as my pumps are rated for 45 C max temp. Here is bit more about my setup https://sabareesh.com/posts/blackwell-waterblock/
amoshebb
Why only 45? And why water cooling?
It strikes me that building everything around room temperature or slightly chilled air is a strange choice. This is already 290K-300K or so, and now this is suggesting that things run fine at 320K or 330K?
I've wondered why we couldn't just design everything to operate around 200°C and just use free-cooling by pumping ambient air through. Why don't data centres look more like chicken barns? Do things melt? Are there more errors of some other type at high temperatures?
show comments
johnsocks
After reading this I don't understand how this is a breakthrough. Looks to be the same closed cooling loop that is already in place in most commercial and industrial cooling applications. The article mentions being able to place a radiator outside in appropriate climates to reject the stored heat in the water/glycol loop. So everywhere thats not in the north pole would still need a condensing loop. Am I missing something?
vladar107
There's already a city in Finland getting free heating from a Microsoft data center. The problem isn't technical. It's that data centers and district heating grids rarely end up next to each other by accident.
show comments
Dead_Lemon
Increasing the cooling setpoint of the system isn't exactly a new concept, nor is data centre scale water cooling.
Computers can run hot and not throttle.
Water is better at moving heat than air.
Temperature delta above ambient increase the amount of heat that can be removed, with less mechanical assistance.
How exactly is any of this news.
VorpalWay
I never looked into this, but why would a datacenter consume water for cooling in the first place? Sure, they use some. But just like you fill up the cooling loop in a car, once it is there it just circulates between the heat source and radiators and/or heat exchangers, with perhaps some minimal top off needed (since flexible tubing isn't 100% water proof).
Or are they for some unfathomable reason using evaporative cooling in data centers?
show comments
prng2021
“That unlocks something beyond energy savings: the possibility of eliminating water consumption entirely.”
They’ve made this claim numerous times in the article and I really don’t understand it. The building has tons of water being recirculated through it. That water came from somewhere in the surrounding natural world. How is that 0 water consumption?
show comments
rglover
Semi-related. I think a better solution to this DC problem (and a nice revenue stream for vendors) is to figure out something like this [1][2] at scale. Basically, instead of saying "we're going to subsidize our DC build outs by increasing your energy costs," say "if we can install a small compute node that receives workloads in your house (make it a kit so it's easy), you get the heat for 'free.'"
I'm ignorant to the specifics of how this might work (and whether it'd even be feasible at a cost/logistics level), but it feels far less invasive and far more sustainable as an armchair thought. Curious if anyone has worked on something like this and what sort of gotchas you've found.
>Previous liquid-cooled servers were hybrid: GPUs and CPUs got cold plates, but the rest of the system stayed air-cooled, with finned heat sinks designed to shed heat into moving air. In a fully liquid-cooled server, the cooling for these components needed to be completely redesigned to use liquid.
Does that mean the whole server board is running in liquid?
If true, how do they do maintenance? Replacing parts must be extremely difficult.
show comments
eptcyka
What's the reason we can't use heat pumps here to turn that waste heat back into electricity?
show comments
nialse
Heat exchange is used instead of refrigerating the coolant. Makes sense. How do they manage the indoor climate for the humans working there though? Eventually everything will be at 45C in the building, will it not?
show comments
eqvinox
On one hand: great!
On the other hand: the heat has to go somewhere. So… where? Datacenters already create a warm microclimate in their vicinity, is that getting even worse?
show comments
fwipsy
My understanding is that datacenter water consumption is not really that big of a deal compared to other industries, and it's mostly inflated by people who are looking for a reason to criticize AI/datacenters. If Nvidia is hoping to placate those people, I predict it will not work.
show comments
uberex
Weird I was daydreaming about why isn't this done the other day (in the context of desert datacentres running on solar anf battery). Glad to see it is a thing.
mips_avatar
My GPUs at home produce 60°C water. I just wish I had a good use for them.
billjive
“Near” zero??
Leslie Groves, Oppenheimer
blondie9x
In a way does this study try to detract from the public backlash against datacenters?
Is this NVIDIA lawyer trying to influence the public perception of datacenter for his company?
sourcegrift
I'm worried about the carbon footprint and other impacts of the higher temperature
htrp
wasn't this announced at gtc in march?
emsign
Greenwashing, that's all. This is not going to be the standard it's just a light house project to calm some nerves to keep on going.
show comments
m3kw9
This is what PC heat sinks uses. Someone could have thought of that
qsxfthnkp2322
Claude write good.
Nvidia has so much money and they can’t afford to pay a human for a day of their time to write a blog post?
show comments
mchusma
This is also the type of thing that makes space based data centers more viable. I was previously more skeptical on the concept but have come around.
I do think ground based centers will have better economics when they can be built though, and this addresses noise and water complaints which are the big 2 regional complaints.
It seems like lots of bottlenecks are getting solved quickly, except for maybe memory.
> In the right geography — somewhere with reliably cool outdoor air
Aaahh, there’s the catch. “Save resources on cooling by building your data centre somewhere cold, and pollute the surrounding environment by dumping your waste heat wholesale into that!”
Good job Nvidia, I almost thought we had something good there.
This opens up an interesting synergy: district heating. 45C is low but not unworkable for a district heating loop, and a data center might be able to make a nice pitch to a community if the data center offers to provide heat to a district heating system for free. This brings the value to the local community of a nearby datacenter up from near zero to potentially a few million dollars per year.
Summer is still an issue, but fun solutions are possible. With the right geology, I think it’s possible to heat an underground volume in the summer and recapture (some of) that heat in the winter. In many, many climates, annual heating costs are far higher than cooling costs, at least if people aren’t stupid with skylights. [0]
[0] As a back-of-the-envelope heuristic, heating or cooling load due to conduction and air exchange is proportional to the difference between indoor and outdoor temperature. Outdoor temperatures of -10F to 30F are not unusual in the winter and are 40-80F away from an indoor temp of 70F. But outdoor temperatures in these climates rarely exceed 95F and are mostly lower in the summer, so that’s 15-25F of cooling. And heat pumps are more efficient at smaller temperature differences.
Radiative heating is an entirely different story.
Points for effort, but this will do literal nothing to appease the opposition, since the "water use" thing is a myth anyway. It's probably good, it sounds like it will be more efficient and efficiency saves resources for other uses, but politically it's completely useless.
Maybe I'm being dumb, but I don't understand what the innovation is here.
I get that they're using liquid coolant at higher than usual temperatures, but why couldn't they do that before? Most of the comparison in the article is for air cooled datacenters but what about other liquid cooled ones?
Surely in all the previous datacenters that have been designed there has been someone doing the math and determining what temperature things need to run at, how much energy it will use, how much heat it all will produce, etc.
edit: just saw this:
>Previous liquid-cooled servers were hybrid: GPUs and CPUs got cold plates, but the rest of the system stayed air-cooled, with finned heat sinks designed to shed heat into moving air. In a fully liquid-cooled server, the cooling for these components needed to be completely redesigned to use liquid.
Water use isn't a myth.
Our data center is mostly air-cooled, with 100F hot aisles and chilled water going to heat exchangers in every third rack position, although some of our newer GPU racks use chilled water directly. For most of the year that means we don't need chillers - the warm water return is sufficiently hotter than outdoor temp that heat "flows downhill" with a bit of pumping.
But we use an evaporative cooler to carry that heat away outside, because it's more efficient. If you look up the heat of vaporization of water, that means we evaporate about 10,000 gallons of water per day per megawatt of power dissipated. We're on canal off a large river, and we don't dissipate all that many megawatts, so I believe the water use isn't significant.
Using air as the working fluid to draw heat from your machines has a limitation - humans have to breathe that air whenever they work on the equipment, so the temperature is limited. Once the exterior temperature nears your hot aisle air temp, you either need active A/C to create a heat source hotter than ambient, or an evaporative cooler to lower the "effective" ambient temp to the dew point.
Liquid cooling lets you run your working fluid a lot hotter without killing folks like me who go into the data center, although honestly 45C sounds like an incremental improvement over the 100F places like ours are already running. (although to be fair, the warm water return from the heat exchangers is no doubt somewhat less than 100F) It also lets you run your "cold" side a lot hotter - if you "chill" your water down to 100F (38C) on a hot day, it's still cold enough to carry away a lot of heat at 45C.
(I'm skipping over the fact that there are multiple heat exchanger loops involved - e.g. any system with an evaporative cooler needs a heat exchanger to keep leaves and bird shit outside the building where it belongs)
> In favorable climates, NVIDIA’s 45-degree liquid-cooling architecture ....
What's a favorable climate, apart from, obviously, Greenland? The piece is a little light on details on the correlation between outside temperatures and efficiency & cost. It'd be nice to see even a broad-strokes discussion of that.
It was already near zero compared to various other uses. It's always depressing to me to see a lot of effort put in to "solve a problem" (with subsequent fanfare) which is only a PR or image problem in the first place.
The NASA Ames Research Center Modular Supercomputing Facility is highly efficient, both in terms of electricity and water use. The facility isn't air conditioned. The chips are water cooled, and the inlet water temperature is pretty high I believe - I think it's 90 degrees Fahrenheit.
https://www.nasa.gov/centers-and-facilities/ames/doing-more-...
https://www.nas.nasa.gov/assets/nas/pdf/ModularSupercomputin...
If anyone else is disappointed by the terrible AI slop article: It's about a fully liquid cooled data center design.
The usual way to cool servers is with air and heatsinks attached to the hot hardware, similar to how your desktop computer or laptop works. As the hardware gets denser and more powerful, you need bigger and bigger heatsinks and cooler air blown over them. At some point you can't make the heatsinks bigger because of space constraints, and you can't blow the air faster (because of noise and efficiency), so you need cooler air. That's when you start running chillers that evaporate water to cool your intake air. This is the huge water consumption that we'd like to avoid.
The next step is, obviously, liquid cooling. Again, this is similar to your fancy gaming desktop. You can dump a lot of heat to a liquid medium through a small heat exchanger inside, where you're space constrained, and you can run the liquid through a gigantic heat exchanger outside, despite the temp delta between your coolant and outside air being pretty small.
This article is about a system that's FULLY liquid cooled — CPUs, GPUs, memory, networking, the whole thing. That's the actual cool part (pun unintended). On top of that, their solution is optimized to be able to run the coolant quite warm — this obviously limits the heat flux at the hardware side, but it allows you to run the outside heat exchangers "dry", i.e. without wasting any water for its latent heat.
> and the same liquid can be recirculated in a closed loop so no new water is consumed to cool the chips
That doesn't mean a datacenter doesn't have a very high _initial_ need for water, or to replace _some_ amount of leakage, replacement etc (agreed it will be way lower than say a swamp cooler). For example, they could be using millions of gallons as a sort of "ballast" to keep the water temperature very stable in the short run.
This whole projects depends on the whole stack of components to be able to tolerate 45c from memory, drives, the lighting of the building, the humans ...
with efficient heat exchange you could get the coolant up to mash temperature (65C) and run a combined data center/brewery
I am pretty much doing the same but running the coolant at 40 deg C instead of 45 as my pumps are rated for 45 C max temp. Here is bit more about my setup https://sabareesh.com/posts/blackwell-waterblock/
Why only 45? And why water cooling?
It strikes me that building everything around room temperature or slightly chilled air is a strange choice. This is already 290K-300K or so, and now this is suggesting that things run fine at 320K or 330K?
I've wondered why we couldn't just design everything to operate around 200°C and just use free-cooling by pumping ambient air through. Why don't data centres look more like chicken barns? Do things melt? Are there more errors of some other type at high temperatures?
After reading this I don't understand how this is a breakthrough. Looks to be the same closed cooling loop that is already in place in most commercial and industrial cooling applications. The article mentions being able to place a radiator outside in appropriate climates to reject the stored heat in the water/glycol loop. So everywhere thats not in the north pole would still need a condensing loop. Am I missing something?
There's already a city in Finland getting free heating from a Microsoft data center. The problem isn't technical. It's that data centers and district heating grids rarely end up next to each other by accident.
Increasing the cooling setpoint of the system isn't exactly a new concept, nor is data centre scale water cooling. Computers can run hot and not throttle. Water is better at moving heat than air. Temperature delta above ambient increase the amount of heat that can be removed, with less mechanical assistance. How exactly is any of this news.
I never looked into this, but why would a datacenter consume water for cooling in the first place? Sure, they use some. But just like you fill up the cooling loop in a car, once it is there it just circulates between the heat source and radiators and/or heat exchangers, with perhaps some minimal top off needed (since flexible tubing isn't 100% water proof).
Or are they for some unfathomable reason using evaporative cooling in data centers?
“That unlocks something beyond energy savings: the possibility of eliminating water consumption entirely.”
They’ve made this claim numerous times in the article and I really don’t understand it. The building has tons of water being recirculated through it. That water came from somewhere in the surrounding natural world. How is that 0 water consumption?
Semi-related. I think a better solution to this DC problem (and a nice revenue stream for vendors) is to figure out something like this [1][2] at scale. Basically, instead of saying "we're going to subsidize our DC build outs by increasing your energy costs," say "if we can install a small compute node that receives workloads in your house (make it a kit so it's easy), you get the heat for 'free.'"
I'm ignorant to the specifics of how this might work (and whether it'd even be feasible at a cost/logistics level), but it feels far less invasive and far more sustainable as an armchair thought. Curious if anyone has worked on something like this and what sort of gotchas you've found.
[1] https://www.heata.co/
[2] https://thermify.cloud/how-it-works/
>Previous liquid-cooled servers were hybrid: GPUs and CPUs got cold plates, but the rest of the system stayed air-cooled, with finned heat sinks designed to shed heat into moving air. In a fully liquid-cooled server, the cooling for these components needed to be completely redesigned to use liquid.
Does that mean the whole server board is running in liquid? If true, how do they do maintenance? Replacing parts must be extremely difficult.
What's the reason we can't use heat pumps here to turn that waste heat back into electricity?
Heat exchange is used instead of refrigerating the coolant. Makes sense. How do they manage the indoor climate for the humans working there though? Eventually everything will be at 45C in the building, will it not?
On one hand: great!
On the other hand: the heat has to go somewhere. So… where? Datacenters already create a warm microclimate in their vicinity, is that getting even worse?
My understanding is that datacenter water consumption is not really that big of a deal compared to other industries, and it's mostly inflated by people who are looking for a reason to criticize AI/datacenters. If Nvidia is hoping to placate those people, I predict it will not work.
Weird I was daydreaming about why isn't this done the other day (in the context of desert datacentres running on solar anf battery). Glad to see it is a thing.
My GPUs at home produce 60°C water. I just wish I had a good use for them.
“Near” zero??
Leslie Groves, Oppenheimer
In a way does this study try to detract from the public backlash against datacenters?
Look at the author of the blog.
https://blogs.nvidia.com/blog/author/joshparker/
Is this NVIDIA lawyer trying to influence the public perception of datacenter for his company?
I'm worried about the carbon footprint and other impacts of the higher temperature
wasn't this announced at gtc in march?
Greenwashing, that's all. This is not going to be the standard it's just a light house project to calm some nerves to keep on going.
This is what PC heat sinks uses. Someone could have thought of that
Claude write good.
Nvidia has so much money and they can’t afford to pay a human for a day of their time to write a blog post?
This is also the type of thing that makes space based data centers more viable. I was previously more skeptical on the concept but have come around.
I do think ground based centers will have better economics when they can be built though, and this addresses noise and water complaints which are the big 2 regional complaints.
It seems like lots of bottlenecks are getting solved quickly, except for maybe memory.