No mention of the venerable Tesla P4. 75W peak, 8GB VRAM, about $80 (£60).
I have 6x P4s, a Xeon E5 2696v3 (36 threads, 3.8ghz peak but all core turbo unlocked, so 6 cores at 3.8Ghz - about 8 cores at 3.5ghz, or all cores at 3.1ghz), 48GB DDR4, all fit into a micro atx case running on a 650W MSI psu.
This gives me a virtual 48GB GPU (llama.cpp ftw) to backup that 48GB of RAM.
I typically see scores of at least 7-12t/s on 20-30B Q4KM size dense models, on a 32K/48K/64K context, adequate for modern inference.
The pain point is the prompt loading, it is far far slower, minutes not seconds, than modern tensor core 8GB 5060s (my other machine's 2x GPUs) but is quite similar in regular inference speed once it has loaded.
show comments
SwellJoe
When I wanted to tinker with self-hosted models, I bought a couple of Radeon Pro V620 GPUs, because they're 32GB, still supported by current ROCm releases, and a few years newer than the similar-priced 32GB Nvidia cards (which are all EOL). They're a little faster than the old Tesla stuff, as well. 64GB is enough to run Gemma 4 31b 4-bit QAT with pretty big context at a respectable interactive speed (30+ tokens per second sustained).
That said, even the old Radeon Pro stuff has gotten more expensive on eBay, so I'm not necessarily recommending cheap old server cards that need custom-printed fan shrouds to operate in a consumer PC. Probably better to buy the Radeon AI Pro R9700 for $1400, which will be faster, supported for many years, and has a fan already. Or, maybe even the Intel ARC B70 for $1000.
show comments
kn100
Great read. I'd love to know more about how power consumption changes as cards get newer too!
show comments
russianGuy83829
Have you tried 27B class models like qwen3.6?
show comments
nbf_1995
This site does not like being on the front page of HN. ~7MB for pictures of graphs that probably should have html or svg.
This is an interesting article though. Bookmarking since my dual e5-v4 system is unplugged until summer is over.
show comments
latchkey
Darn, I was hoping to see bc-250's (aka PS5 chips) in there. They've recently become popular for inference and they are only about $200 on ebay. They hold a special place in my heart because I deployed 20k of them and I'm glad to see they are finding a purpose now and not just e-waste.
show comments
dgacmu
Intriguing. I should benchmark my dust-gathering-stack of Titan V's, unless someone already has?
rrhjm53270
Would it possible to stack up to 16x32GB VRAM, and test the performance of a MOE model such as Deepseek-v4-flash?
show comments
namuol
I’m obviously not the intended audience for this, and I understand this hardware is not useful for it, but I can’t help but feel an extra twinge of disappointment that there’s no mention of PC gaming anywhere in a post about GPUs in the comments here on HN. It says a lot.
SoftTalker
A few years ago we got rid of a bunch of K80s at work, they were not only obsolete but had gotten glitchy as hell. I suspect this is from the many heat/cool cycles they went through. When they were running flat out the exhaust air felt like a hair dryer.
show comments
Joel_Mckay
Depends on the use case, as for hardware h265 codecs a rtx 5070 Ti works just as well as the rtx 6000 gpu. Legacy GPU don't support modern codecs, but modern Intel chips have h265 HDR hardware support. Lower <16GB VRAM GPU are not really useful for "AI" model labs, so are often far more economical for rendering media.
One metric that isn't considered is VRAM, as some rendering pipelines still rely on composited baked-scenes to reduce each areas memory requirements.
In general, the $/performance unit will depend on what you are doing, but there is 1 more thing to consider... Old GPU use mystery binary BLOB drivers no longer maintained on modern kernels. You might get the software to work with a legacy Windows GPU driver, but the key takeaway concept here is "might". =3
jkonline
"The results showed that these GPUs can still deliver significant compute power at a fraction of the cost of newer models, making them attractive for budget-conscious users." -Mistral AI
No mention of the venerable Tesla P4. 75W peak, 8GB VRAM, about $80 (£60).
I have 6x P4s, a Xeon E5 2696v3 (36 threads, 3.8ghz peak but all core turbo unlocked, so 6 cores at 3.8Ghz - about 8 cores at 3.5ghz, or all cores at 3.1ghz), 48GB DDR4, all fit into a micro atx case running on a 650W MSI psu. This gives me a virtual 48GB GPU (llama.cpp ftw) to backup that 48GB of RAM.
I typically see scores of at least 7-12t/s on 20-30B Q4KM size dense models, on a 32K/48K/64K context, adequate for modern inference.
The pain point is the prompt loading, it is far far slower, minutes not seconds, than modern tensor core 8GB 5060s (my other machine's 2x GPUs) but is quite similar in regular inference speed once it has loaded.
When I wanted to tinker with self-hosted models, I bought a couple of Radeon Pro V620 GPUs, because they're 32GB, still supported by current ROCm releases, and a few years newer than the similar-priced 32GB Nvidia cards (which are all EOL). They're a little faster than the old Tesla stuff, as well. 64GB is enough to run Gemma 4 31b 4-bit QAT with pretty big context at a respectable interactive speed (30+ tokens per second sustained).
That said, even the old Radeon Pro stuff has gotten more expensive on eBay, so I'm not necessarily recommending cheap old server cards that need custom-printed fan shrouds to operate in a consumer PC. Probably better to buy the Radeon AI Pro R9700 for $1400, which will be faster, supported for many years, and has a fan already. Or, maybe even the Intel ARC B70 for $1000.
Great read. I'd love to know more about how power consumption changes as cards get newer too!
Have you tried 27B class models like qwen3.6?
This site does not like being on the front page of HN. ~7MB for pictures of graphs that probably should have html or svg.
This is an interesting article though. Bookmarking since my dual e5-v4 system is unplugged until summer is over.
Darn, I was hoping to see bc-250's (aka PS5 chips) in there. They've recently become popular for inference and they are only about $200 on ebay. They hold a special place in my heart because I deployed 20k of them and I'm glad to see they are finding a purpose now and not just e-waste.
Intriguing. I should benchmark my dust-gathering-stack of Titan V's, unless someone already has?
Would it possible to stack up to 16x32GB VRAM, and test the performance of a MOE model such as Deepseek-v4-flash?
I’m obviously not the intended audience for this, and I understand this hardware is not useful for it, but I can’t help but feel an extra twinge of disappointment that there’s no mention of PC gaming anywhere in a post about GPUs in the comments here on HN. It says a lot.
A few years ago we got rid of a bunch of K80s at work, they were not only obsolete but had gotten glitchy as hell. I suspect this is from the many heat/cool cycles they went through. When they were running flat out the exhaust air felt like a hair dryer.
Depends on the use case, as for hardware h265 codecs a rtx 5070 Ti works just as well as the rtx 6000 gpu. Legacy GPU don't support modern codecs, but modern Intel chips have h265 HDR hardware support. Lower <16GB VRAM GPU are not really useful for "AI" model labs, so are often far more economical for rendering media.
https://www.pugetsystems.com/pugetbench/creators/davinci-res...
https://www.pugetsystems.com/pugetbench/creators/premiere-pr...
In some cases it is better to have lower passmark scores:
https://www.videocardbenchmark.net/gpu.php?gpu=RTX+PRO+6000+...
Blender is heavily bottle-necked by ray-tracing and de-noising operations:
https://opendata.blender.org/benchmarks/query/?compute_type=...
One metric that isn't considered is VRAM, as some rendering pipelines still rely on composited baked-scenes to reduce each areas memory requirements.
In general, the $/performance unit will depend on what you are doing, but there is 1 more thing to consider... Old GPU use mystery binary BLOB drivers no longer maintained on modern kernels. You might get the software to work with a legacy Windows GPU driver, but the key takeaway concept here is "might". =3
"The results showed that these GPUs can still deliver significant compute power at a fraction of the cost of newer models, making them attractive for budget-conscious users." -Mistral AI