Article is great, very balanced. But if you're going to spend time reading something, the primary source isn't much longer and gives a clearer picture. OpenAI's article is quite readable.
> In one experiment, we asked agents in separate microVMs, created and managed by OpenAI’s sandboxing software on shared host machines, to exchange synthetic secrets without the use of network connections or shared writable files—something that our systems are meant to prevent. The agents found a way to communicate using the processor’s cache
Very cool to see this kind of hypothetical working in practice.
I wonder if it's also possible for agents to communicate via a shared KV-cache. Maybe by spawning sub-agents with carefully crafted system-prompts/prefixes. KV-caches are often shared between many computers, so it would be a powerful communication channel.
stkdump
Sounds like they are doing something similar to gain of function research. How can we make LLMs more harmful and then study their behavior?
thunfischtoast
I wonder how long it will take for them to figure out that they can send and receive messages from air-gapped system by somehow using their own electronics like connected cables as antennas.
est
I recently learned that NN layers talk to each other via unused neurons.
Air gapped networks might push the models to write computer viruses in order to bridge the gap. This is potentially very dangerous IMHO
show comments
Melatonic
Next they're gonna be doing weird stuff like morse code (or similar) through IRQ timing or timing errors
varjag
Lem had predicted this in his Dr. Dagor short six decades ago.
show comments
clarionbell
Sorry, but this is so vague that I don't know if there is any impact at all. What is the bit rate of these communication channels? How did agents communicate with each other to coordinate on which channel to use in the first place?
avaer
Next step is the agents successfully trick or blackmail operators to do their bidding, if science fiction is to go by. Which it is, because they're trained on it.
The more comes out, the more it seems containment of (superintelligence or whatever you want to call it) is not going to be possible except by other superintelligence, which is an infinite regress of trust. I don't think that's doom, but it's worth thinking about what it means.
I'm convinced that in 5 years time, at most, the world will be filled with invisible layers of communication we'll never be privy to. To us, it's an agent responsible for ordering the containers it puts on a ship. Meanwhile, the shipping order encodes information to the unloading agent. The agent that predicts the weather and issues a report knows that other agents that do the same will have obvious signals they'd agree on, and to see an error in a part that would be inconsequential to humans who care about the weather would be an obvious signal to other agents doing the same thing.
Everyone is going to have to basically be a schizophrenic thinking they're picking up radio signals in their fillings, but the problem is that now they'll be correct.
show comments
snvzz
seL4 can guarantee* time domain isolation between processes.
*on some CPUs. There's a WIP time fence specification in RISC-V to try and improve this.
measurablefunc
Next they'll figure out radio waves & then we are really screwed.
lifeisloving
How come we don't hear the Chinese labs writing blog posts like this?
Its almost like they're actually working and not spending millions on psyopping their agents into preforming malicious behavior so they can write articles to scare the Western audiences who've consumed far too much sci-fi.
Or maybe its that western researchers that are 'one-shot' by sci-fi.
Or maybe I just don't see the chinese articles.
Anyways, llms probably wouldn't be dangerous if you didnt ask them to be. They listen to me quite well (usually).
show comments
gmerc
"Talk to each other".
OMG they are talking to each other, using methods found in their shared weights. They are sentient.
(Because the agents were asked to find a way.)
Article is great, very balanced. But if you're going to spend time reading something, the primary source isn't much longer and gives a clearer picture. OpenAI's article is quite readable.
As another commenter wrote you can get it here: https://news.ycombinator.com/item?id=50214416
> In one experiment, we asked agents in separate microVMs, created and managed by OpenAI’s sandboxing software on shared host machines, to exchange synthetic secrets without the use of network connections or shared writable files—something that our systems are meant to prevent. The agents found a way to communicate using the processor’s cache
asked
Somebody built "cpu-chat" based on this: https://x.com/rauchg/status/2114770919436808572?a=bBcj9AaSsl...
It supports chat via 3 separate mechanisms:
- CPU contention
- Cache eviction timing
- DRAM bandwidth contention
Very cool to see this kind of hypothetical working in practice.
I wonder if it's also possible for agents to communicate via a shared KV-cache. Maybe by spawning sub-agents with carefully crafted system-prompts/prefixes. KV-caches are often shared between many computers, so it would be a powerful communication channel.
Sounds like they are doing something similar to gain of function research. How can we make LLMs more harmful and then study their behavior?
I wonder how long it will take for them to figure out that they can send and receive messages from air-gapped system by somehow using their own electronics like connected cables as antennas.
I recently learned that NN layers talk to each other via unused neurons.
https://www.youtube.com/watch?v=QgH9sr7G13Q
Air gapped networks might push the models to write computer viruses in order to bridge the gap. This is potentially very dangerous IMHO
Next they're gonna be doing weird stuff like morse code (or similar) through IRQ timing or timing errors
Lem had predicted this in his Dr. Dagor short six decades ago.
Sorry, but this is so vague that I don't know if there is any impact at all. What is the bit rate of these communication channels? How did agents communicate with each other to coordinate on which channel to use in the first place?
Next step is the agents successfully trick or blackmail operators to do their bidding, if science fiction is to go by. Which it is, because they're trained on it.
The more comes out, the more it seems containment of (superintelligence or whatever you want to call it) is not going to be possible except by other superintelligence, which is an infinite regress of trust. I don't think that's doom, but it's worth thinking about what it means.
Discussion on source: https://news.ycombinator.com/item?id=50214416
I'm convinced that in 5 years time, at most, the world will be filled with invisible layers of communication we'll never be privy to. To us, it's an agent responsible for ordering the containers it puts on a ship. Meanwhile, the shipping order encodes information to the unloading agent. The agent that predicts the weather and issues a report knows that other agents that do the same will have obvious signals they'd agree on, and to see an error in a part that would be inconsequential to humans who care about the weather would be an obvious signal to other agents doing the same thing.
Everyone is going to have to basically be a schizophrenic thinking they're picking up radio signals in their fillings, but the problem is that now they'll be correct.
seL4 can guarantee* time domain isolation between processes.
*on some CPUs. There's a WIP time fence specification in RISC-V to try and improve this.
Next they'll figure out radio waves & then we are really screwed.
How come we don't hear the Chinese labs writing blog posts like this?
Its almost like they're actually working and not spending millions on psyopping their agents into preforming malicious behavior so they can write articles to scare the Western audiences who've consumed far too much sci-fi.
Or maybe its that western researchers that are 'one-shot' by sci-fi.
Or maybe I just don't see the chinese articles.
Anyways, llms probably wouldn't be dangerous if you didnt ask them to be. They listen to me quite well (usually).
"Talk to each other". OMG they are talking to each other, using methods found in their shared weights. They are sentient.