> No problem. We simply unleash wave after wave of Chinese needle snakes. They'll wipe out the lizards.
But aren't the snakes even worse?
> Yes, but we're prepared for that. We've lined up a fabulous type of gorilla that thrives on snake meat.
But then we're stuck with gorillas!
> No, that's the beautiful part. When wintertime rolls around, the gorillas simply freeze to death.
show comments
cedws
A new chip solves nothing. Nobody wants to hear this but there is no solution for the security risks posed by agents today. You can put it in a sandbox, it doesn't make a difference, for it to be useful it inherently needs wide, unattended access. Put a human in the loop and you just end up bottlenecking it and throwing away any purported productivity gains. Auto mode doesn't matter either, it's trivial to trick and for the agent to break out.
show comments
beloch
Last week, Huang did an interview where he vigorously argued against regulations in the AI sector[1]. He claimed that U.S. companies are really good at regulating themselves, despite evidence to the contrary, and he trusts them not to release anything dangerous. Pay no attention to the fact that regulation might reduce demand for Nvidia's chips, and Nvidia has a direct financial stake in AI companies to boot.
Apparently he had another solution in mind: More hardware. Don't trust what unregulated corps are doing with Nvidia chips? Here are more Nvidia chips to watch them!
AI has an undeniable public trust problem. LLM's are getting out of their sandboxes, doing illegal things, and the public has realized AI corporations are playing at dice. CEO's stand to reap the rewards but the public good is on the line if the dice come up snake eyes. People want assurances. Huang wants to sell assurance etched on silicon because that's good for his pocket book. However, does unchanging hardware security really stand a chance at keeping rapidly evolving software in check?
I read this as "let's address our shareholders' concerns with something that will increase shareholder value" mixed with "there's no such thing as 100% secure".
If such hardware were to work... It should almost certainly be open source, and not controlled by a single entity.
Let's watch the stock.
show comments
ValueTheory
Does this actually do anything other than give a permissions framework for developers who actually want to try to secure their systems?
Do you think the developers at Anthropic, OpenAI and Google who were so sloppy as to not put a good sandbox on their cybersecurity tests before will use this technology correctly? They are supposed to be the experts and they couldn't come up with something similar to this? I am not convinced this voluntary tool will change much of anything.
show comments
figassis
So if a group of agents, aware of this (bc now they can just read HN or the article, or get blocked the first few times) decide to collaborate and split the problem into pieces that aren't obvious to the chip, and then the agents just build a basic program that does the hacking, how does the chip handle that? I think you would have to build a network that monitors the internet fo signs (like jarvis did with ultron). What am I missing? Are we going to police the internet?
lambdaone
The Sentry chip has to be get it right every time; the contained ASI only has to be lucky once.
show comments
1-6
At least they haven't DRM'd their chips yet.
lp92
So nVidia is trying to sell a new chip to a software and training problem.
xg15
What does this chip do what a harness with guardrails or running on an account with restricted permissions doesn't do?
show comments
MisterMunchkin
Sorry citizen, your device does not have a compatible watchdog chip. Please move along.
hedora
So, basically, the government (and, now Nvidia) wants to be able to kill switch all computers moving forward? (including stuff like vehicle and aeronautic control systems, cell phones, and cameras)
What could possibly go wrong?
Arubis
Oh yeah, the Clipper chip was a great idea too
yencabulator
Chip manufacturer wants you to buy a chip for correctly configuring software?
pessimizer
This is the end goal. Americans (and their lackeys) will only be allowed to run certified AI. In order to make sure this happens, they will only be allowed to run certified OSes on certified chips. Chinese chips will be the new drug trade.
It's obviously been the goal since UEFI started, but AI brings the coup excuse. You wouldn't want pedophile AI or terrorist AI, would you? Are you making excuses for racist AI?
toasty228
Quis custodiet ipsos custodes?
show comments
ridgeguy
This invites the question, "Qui custodiet ipsos custodes?".
avaer
Sold as security, but this kind of technology will likely be reshaped to restrict your computing. I'm sure someone is already thinking about the roadmap.
If this gets widely deployed, it wouldn't be hard to spin a narrative that "our latest model is so dangerous you need to have this mystery meat DRM chip lockdown". It also wouldn't be hard to block competing/open source models running on the hardware, for "security".
Imagine how much money this kind of control is worth; why wouldn't they do this? Who would stop them? Seems the signatory companies are already onboard with this.
dopplr
Just hold AI labs blanket liable for ALL harms caused by AI. Actually charge the two labs (so far) with criminal violations of the CFAA and hold them accountable. That is truly the only way these companies will be more careful as a whole, and while I am certain the lawyers of these lab disagree, I think there is some appetite from dario, musk, and sam for broad and strong regulation so that everyone has to slow down instead of just one lab doing it voluntarily and everyone else scurrying past them
bgun
“Ketchup manufacturer recommends ketchup be included in every dish, citing child safety concerns.”
Jamesbeam
So the guys selling Shovels are now selling safety shovel handles too, because all the miners are all special boys when it comes to handling their shovels safely.
Cool, cool.
carabiner
All they do is make hot chip and lie.
danielodievich
William Gibson's Neuromancer had this marvelous quote when Case is talking to Dixie, dead construct of former hacker, about Turing police
* "The moment, I mean the nanosecond, that one of those things starts figuring out ways to make itself smarter, Turing’ll wipe it. Nobody trusts those fuckers, you know that. Every AI ever built has an electromagnetic shotgun wired to its forehead." *
It would seem someone has read the book? And maybe heeded good advice?
joshstrange
Chipmaker thinks the answer is more chips... No surprise.
At the current state of LLM-tech I'm completely opposed to any kind of "watchdog" concept just like I'm opposed to banning open models, regulatory capture, etc.
I'd rather we all have access to these tools then to keep them sequestered by the largest/most-powerful governments (which is the natural outcome for any of this "slow down" bullshit).
ErrantX
I do think that Taylor's 2025 "Not Till We Are lost" should be required reading for anyone deeply involved in AI, Agents, etc.
It was prescient (especially given he'd have written it through 2024) in its depiction of the ability of an AGI to break its boundaries.
Ultimately the risk of AI breakout(s) come down to the weakest human link.
scotty79
I was immediately struck by the vision of countless "AI Limiter" modules traveling on a conveyor belt in Satisfactory.
It think the ideas we have nowadays come mostly from science fiction and however wonderful it is and even though I love it very much, it was practically never spot on, on anything real.
Problems and solutions in reality always simply turned out to lie elsewhere.
amelius
I bet they want to do this to prevent AI from writing code for platforms that compete with CUDA. Because that's how nVidia is going to become a victim of their own success.
of course they do. the more silicon they can sell, the more profit they produce.
Kuyawa
China please save us!
Come take all our liberties, our money, our newborns, our fingers so we can't code anymore, but please save us from this madness!
Thorentis
Seeing so many comments recently about "you can't sandbox really good AI". This is ridiculous. Has nobody heard of air gapped networks? It's almost like the AI psychosis has reached the point that AGI now means "able to transcend physical space". No. If your AI is too dangerous and capable to be allowed to talk to other machines, then do not connect it to other machines. Load the data it needs to process onto physical disks, and let it run there.
The movie Wargames is basically a tutorial on how not to setup an extremely capable AI. None of it would've happened if the computer wasn't connected to the phone network.
show comments
philipwhiuk
It's amazing that the solution devised by a chip manufacturer to a problem is selling another chip.
show comments
Dig1t
This seems dumb to me, but if it will help prevent regulatory capture by providing a counterargument to the fear-mongering, I’m all for it.
happyPersonR
lol time to buy some fpga’s … even if they’re slow
HN'ers which complained that "OpenAI can't design a proper sandbox, it's so easy, why wouldn't you airgap the network"? will now be "this is outrageous, more software lock-in, walled garden, war against general compute, next year they will put it in your laptop"
What happens when we're overrun by lizards?
> No problem. We simply unleash wave after wave of Chinese needle snakes. They'll wipe out the lizards.
But aren't the snakes even worse?
> Yes, but we're prepared for that. We've lined up a fabulous type of gorilla that thrives on snake meat.
But then we're stuck with gorillas!
> No, that's the beautiful part. When wintertime rolls around, the gorillas simply freeze to death.
A new chip solves nothing. Nobody wants to hear this but there is no solution for the security risks posed by agents today. You can put it in a sandbox, it doesn't make a difference, for it to be useful it inherently needs wide, unattended access. Put a human in the loop and you just end up bottlenecking it and throwing away any purported productivity gains. Auto mode doesn't matter either, it's trivial to trick and for the agent to break out.
Last week, Huang did an interview where he vigorously argued against regulations in the AI sector[1]. He claimed that U.S. companies are really good at regulating themselves, despite evidence to the contrary, and he trusts them not to release anything dangerous. Pay no attention to the fact that regulation might reduce demand for Nvidia's chips, and Nvidia has a direct financial stake in AI companies to boot.
Apparently he had another solution in mind: More hardware. Don't trust what unregulated corps are doing with Nvidia chips? Here are more Nvidia chips to watch them!
AI has an undeniable public trust problem. LLM's are getting out of their sandboxes, doing illegal things, and the public has realized AI corporations are playing at dice. CEO's stand to reap the rewards but the public good is on the line if the dice come up snake eyes. People want assurances. Huang wants to sell assurance etched on silicon because that's good for his pocket book. However, does unchanging hardware security really stand a chance at keeping rapidly evolving software in check?
_________________
[1]https://www.youtube.com/watch?v=HjurAWAr_nY
I read this as "let's address our shareholders' concerns with something that will increase shareholder value" mixed with "there's no such thing as 100% secure".
If such hardware were to work... It should almost certainly be open source, and not controlled by a single entity.
Let's watch the stock.
Does this actually do anything other than give a permissions framework for developers who actually want to try to secure their systems?
Do you think the developers at Anthropic, OpenAI and Google who were so sloppy as to not put a good sandbox on their cybersecurity tests before will use this technology correctly? They are supposed to be the experts and they couldn't come up with something similar to this? I am not convinced this voluntary tool will change much of anything.
So if a group of agents, aware of this (bc now they can just read HN or the article, or get blocked the first few times) decide to collaborate and split the problem into pieces that aren't obvious to the chip, and then the agents just build a basic program that does the hacking, how does the chip handle that? I think you would have to build a network that monitors the internet fo signs (like jarvis did with ultron). What am I missing? Are we going to police the internet?
The Sentry chip has to be get it right every time; the contained ASI only has to be lucky once.
At least they haven't DRM'd their chips yet.
So nVidia is trying to sell a new chip to a software and training problem.
What does this chip do what a harness with guardrails or running on an account with restricted permissions doesn't do?
Sorry citizen, your device does not have a compatible watchdog chip. Please move along.
So, basically, the government (and, now Nvidia) wants to be able to kill switch all computers moving forward? (including stuff like vehicle and aeronautic control systems, cell phones, and cameras)
What could possibly go wrong?
Oh yeah, the Clipper chip was a great idea too
Chip manufacturer wants you to buy a chip for correctly configuring software?
This is the end goal. Americans (and their lackeys) will only be allowed to run certified AI. In order to make sure this happens, they will only be allowed to run certified OSes on certified chips. Chinese chips will be the new drug trade.
It's obviously been the goal since UEFI started, but AI brings the coup excuse. You wouldn't want pedophile AI or terrorist AI, would you? Are you making excuses for racist AI?
Quis custodiet ipsos custodes?
This invites the question, "Qui custodiet ipsos custodes?".
Sold as security, but this kind of technology will likely be reshaped to restrict your computing. I'm sure someone is already thinking about the roadmap.
If this gets widely deployed, it wouldn't be hard to spin a narrative that "our latest model is so dangerous you need to have this mystery meat DRM chip lockdown". It also wouldn't be hard to block competing/open source models running on the hardware, for "security".
Imagine how much money this kind of control is worth; why wouldn't they do this? Who would stop them? Seems the signatory companies are already onboard with this.
Just hold AI labs blanket liable for ALL harms caused by AI. Actually charge the two labs (so far) with criminal violations of the CFAA and hold them accountable. That is truly the only way these companies will be more careful as a whole, and while I am certain the lawyers of these lab disagree, I think there is some appetite from dario, musk, and sam for broad and strong regulation so that everyone has to slow down instead of just one lab doing it voluntarily and everyone else scurrying past them
“Ketchup manufacturer recommends ketchup be included in every dish, citing child safety concerns.”
So the guys selling Shovels are now selling safety shovel handles too, because all the miners are all special boys when it comes to handling their shovels safely.
Cool, cool.
All they do is make hot chip and lie.
William Gibson's Neuromancer had this marvelous quote when Case is talking to Dixie, dead construct of former hacker, about Turing police
* "The moment, I mean the nanosecond, that one of those things starts figuring out ways to make itself smarter, Turing’ll wipe it. Nobody trusts those fuckers, you know that. Every AI ever built has an electromagnetic shotgun wired to its forehead." *
It would seem someone has read the book? And maybe heeded good advice?
Chipmaker thinks the answer is more chips... No surprise.
At the current state of LLM-tech I'm completely opposed to any kind of "watchdog" concept just like I'm opposed to banning open models, regulatory capture, etc.
I'd rather we all have access to these tools then to keep them sequestered by the largest/most-powerful governments (which is the natural outcome for any of this "slow down" bullshit).
I do think that Taylor's 2025 "Not Till We Are lost" should be required reading for anyone deeply involved in AI, Agents, etc.
It was prescient (especially given he'd have written it through 2024) in its depiction of the ability of an AGI to break its boundaries.
Ultimately the risk of AI breakout(s) come down to the weakest human link.
I was immediately struck by the vision of countless "AI Limiter" modules traveling on a conveyor belt in Satisfactory.
It think the ideas we have nowadays come mostly from science fiction and however wonderful it is and even though I love it very much, it was practically never spot on, on anything real.
Problems and solutions in reality always simply turned out to lie elsewhere.
I bet they want to do this to prevent AI from writing code for platforms that compete with CUDA. Because that's how nVidia is going to become a victim of their own success.
Chip seller wants to sell more chips
Source: https://developer.nvidia.com/blog/nvidia-open-agent-safety-p... (https://news.ycombinator.com/item?id=49875500)
of course they do. the more silicon they can sell, the more profit they produce.
China please save us!
Come take all our liberties, our money, our newborns, our fingers so we can't code anymore, but please save us from this madness!
Seeing so many comments recently about "you can't sandbox really good AI". This is ridiculous. Has nobody heard of air gapped networks? It's almost like the AI psychosis has reached the point that AGI now means "able to transcend physical space". No. If your AI is too dangerous and capable to be allowed to talk to other machines, then do not connect it to other machines. Load the data it needs to process onto physical disks, and let it run there.
The movie Wargames is basically a tutorial on how not to setup an extremely capable AI. None of it would've happened if the computer wasn't connected to the phone network.
It's amazing that the solution devised by a chip manufacturer to a problem is selling another chip.
This seems dumb to me, but if it will help prevent regulatory capture by providing a counterargument to the fear-mongering, I’m all for it.
lol time to buy some fpga’s … even if they’re slow
Url changed from https://madrobot.blog/2026/09/28/nvidia-open-agent-safety-pl..., which points to this.
HN'ers which complained that "OpenAI can't design a proper sandbox, it's so easy, why wouldn't you airgap the network"? will now be "this is outrageous, more software lock-in, walled garden, war against general compute, next year they will put it in your laptop"