This looks really good - the CLI interface design is solid, and I especially like the secrets / network proxy pattern - but the thing it needs most is copiously detailed documentation about exactly how the sandbox mechanism works - and how it was tested.
There are dozens of projects like this emerging right now. They all share the same challenge: establishing credibility.
I'm loathe to spend time evaluating them unless I've seen robust evidence that the architecture is well thought through and the tool has been extensively tested already.
My ideal sandbox is one that's been used by hundreds of people in a high-stakes environment already. That's a tall order, but if I'm going to spend time evaluating one the next best thing is documentation that teaches me something about sandboxing and demonstrates to me how competent and thorough the process of building this one has been.
UPDATE: On further inspection there's a lot that I like about this one. The CLI design is neat, it builds on a strong underlying library (the OpenAI Codex implementation) and the features it does add - mainly the network proxy being able to modify headers to inject secrets - are genuinely great ideas.
show comments
jwilliams
It’s terrific to see this. I’m definitely going to give it a whirl. I’ve been working on a specific JavaScript isolate[^1]. This is great source of inspiration for it.
Compare with and steal any ideas you like from mine if you like. I've got a semi-decent curl|bash pattern covered, and also add network filtering via pasta (which may be more robust than rolling your own). https://github.com/reubenfirmin/bubblewrap-tui
show comments
dk8996
Very cool. Is there a way to have a notion of a session, saving state between runs?
show comments
Lethalman
Wish it wasn’t rust… it’s so hard to read.
eluded7
Personally I would probably always reach for a docker container if I want a sandboxed command that can run identically anywhere.
I appreciate that alternate sandboxing tools can reduce some of the heavier parts of docker though (i.e. building or downloading the correct image)
the credential injection via MITM proxy is the most interesting part to me. the standard approach for agents is environment variables, which means the agent process can read them directly. having the sandbox intercept network calls and swap in credentials at the proxy layer means the agent code has a placeholder and never sees the real value -- useful when running less-trusted agent code or third-party tools.
the deny-by-default network policy also matters specifically for agent use: without it there is nothing stopping a tool call from exfiltrating context window contents to an arbitrary endpoint. most sandboxes focus on filesystem isolation and treat network as an afterthought.
show comments
mdavid626
I trust sandbox-exec more, or Docker on Linux. Those come from the OS, well tested and known.
MITM proxy is nice idea to avoid leaking secrets. Isn’t it very brittle though? Anthropic changes some URL-s and it’ll break.
show comments
time0ut
Very interesting. I just started researching this topic yesterday to build something for adjacent use cases (sandboxing LLM authored programs). My initial prototype is using a wasm based sandbox, but I want something more robust and flexible.
Some of my use cases are very latency sensitive. What sort of overhead are you seeing?
show comments
zephyrwhimsy
Technical debt is not always bad. Deliberate technical debt taken on with eyes open to ship faster is a legitimate business strategy. The problem is accidental technical debt from poor decisions compounding silently.
jbverschoor
Again, it’s blacklisting so kind of impossible to get right. I’ve looked at this many times, but in order for things to properly work, you have to create a huge, huge, huge, huge sandbox file.
Especially for your application that you any kind of Apple framework.
show comments
wepple
You should probably add a huge disclaimer that this is an untested, experimental project.
Related, a direct comparison to other sandboxes and what you offer over those would be nice
show comments
alyxya
Cool project, and I think there would be a lot of value in just logging all operations.
show comments
EGreg
This is really useful! How well does it compare though to Docker etc.
I like tools like this, but they all seem to share the same underlying shape: take an arbitrary process and try to restrict it with OS primitives + some policy layer (flags, proxies, etc).
That works, but it also means correctness depends heavily on configuration, i.e. you’re starting with a lot of ambient authority and trying to subtract from it enforcement ends up split across multiple layers (kernel, wrapper, proxy)
An alternative model is to flip it: Instead of sandboxing arbitrary programs, run workflows in an environment where there is no general network/filesystem access at all, and every external interaction has to go through explicit capabilities.
In that setup, there’s nothing to "block" because the dangerous primitives aren’t exposed, execution can be deterministic/replayable, so you can actually audit behavior. Thus, secrets don’t enter the execution context, they’re only used at the boundary
It feels closer to capability-based systems than traditional sandboxing. Curious how people here think about that tradeoff vs OS-level sandbox + proxy approaches.
show comments
gigatexal
there's been so many of these -- which of these sandboxing tools is best?
show comments
nonameiguess
This is more a criticism of codex's linux-sandboxing, which you're just wrapping, but it's the first I've ever looked at it. I don't see how it makes sense to invoke bwrap as a forked subprocess. Bubblewrap can't do anything beyond what you can do with unshare directly, which you can simply invoke as a system call without needing to spawn a subprocess or requiring the user to have bwrap installed. It kinds of reeks of amateur hour when developers effectively just translate shell scripts into compiled languages by using whatever variant of "system" is available to make the same command invocations you would make through a shell, as opposed to actually using the system call API. Especially when the invocation is crafted from user input, there's a long history of exploits arising from stuff like this. Writing it in Rust does nothing for you when you're just using Rust to call a different CLI tool that isn't written in Rust.
This looks really good - the CLI interface design is solid, and I especially like the secrets / network proxy pattern - but the thing it needs most is copiously detailed documentation about exactly how the sandbox mechanism works - and how it was tested.
There are dozens of projects like this emerging right now. They all share the same challenge: establishing credibility.
I'm loathe to spend time evaluating them unless I've seen robust evidence that the architecture is well thought through and the tool has been extensively tested already.
My ideal sandbox is one that's been used by hundreds of people in a high-stakes environment already. That's a tall order, but if I'm going to spend time evaluating one the next best thing is documentation that teaches me something about sandboxing and demonstrates to me how competent and thorough the process of building this one has been.
UPDATE: On further inspection there's a lot that I like about this one. The CLI design is neat, it builds on a strong underlying library (the OpenAI Codex implementation) and the features it does add - mainly the network proxy being able to modify headers to inject secrets - are genuinely great ideas.
It’s terrific to see this. I’m definitely going to give it a whirl. I’ve been working on a specific JavaScript isolate[^1]. This is great source of inspiration for it.
[^1]: https://github.com/jonathannen/hermit
Compare with and steal any ideas you like from mine if you like. I've got a semi-decent curl|bash pattern covered, and also add network filtering via pasta (which may be more robust than rolling your own). https://github.com/reubenfirmin/bubblewrap-tui
Very cool. Is there a way to have a notion of a session, saving state between runs?
Wish it wasn’t rust… it’s so hard to read.
Personally I would probably always reach for a docker container if I want a sandboxed command that can run identically anywhere.
I appreciate that alternate sandboxing tools can reduce some of the heavier parts of docker though (i.e. building or downloading the correct image)
How would you compare this tool to say bubblewrap https://github.com/containers/
the credential injection via MITM proxy is the most interesting part to me. the standard approach for agents is environment variables, which means the agent process can read them directly. having the sandbox intercept network calls and swap in credentials at the proxy layer means the agent code has a placeholder and never sees the real value -- useful when running less-trusted agent code or third-party tools.
the deny-by-default network policy also matters specifically for agent use: without it there is nothing stopping a tool call from exfiltrating context window contents to an arbitrary endpoint. most sandboxes focus on filesystem isolation and treat network as an afterthought.
I trust sandbox-exec more, or Docker on Linux. Those come from the OS, well tested and known.
MITM proxy is nice idea to avoid leaking secrets. Isn’t it very brittle though? Anthropic changes some URL-s and it’ll break.
Very interesting. I just started researching this topic yesterday to build something for adjacent use cases (sandboxing LLM authored programs). My initial prototype is using a wasm based sandbox, but I want something more robust and flexible.
Some of my use cases are very latency sensitive. What sort of overhead are you seeing?
Technical debt is not always bad. Deliberate technical debt taken on with eyes open to ship faster is a legitimate business strategy. The problem is accidental technical debt from poor decisions compounding silently.
Again, it’s blacklisting so kind of impossible to get right. I’ve looked at this many times, but in order for things to properly work, you have to create a huge, huge, huge, huge sandbox file.
Especially for your application that you any kind of Apple framework.
You should probably add a huge disclaimer that this is an untested, experimental project.
Related, a direct comparison to other sandboxes and what you offer over those would be nice
Cool project, and I think there would be a lot of value in just logging all operations.
This is really useful! How well does it compare though to Docker etc.
Because I am worried about sandbox escapes. This is what we currently use to sandbox JS inside Browsers and Node (without anything extra) : https://github.com/Qbix/Platform/blob/main/platform/plugins/...
I like tools like this, but they all seem to share the same underlying shape: take an arbitrary process and try to restrict it with OS primitives + some policy layer (flags, proxies, etc).
That works, but it also means correctness depends heavily on configuration, i.e. you’re starting with a lot of ambient authority and trying to subtract from it enforcement ends up split across multiple layers (kernel, wrapper, proxy)
An alternative model is to flip it: Instead of sandboxing arbitrary programs, run workflows in an environment where there is no general network/filesystem access at all, and every external interaction has to go through explicit capabilities.
In that setup, there’s nothing to "block" because the dangerous primitives aren’t exposed, execution can be deterministic/replayable, so you can actually audit behavior. Thus, secrets don’t enter the execution context, they’re only used at the boundary
It feels closer to capability-based systems than traditional sandboxing. Curious how people here think about that tradeoff vs OS-level sandbox + proxy approaches.
there's been so many of these -- which of these sandboxing tools is best?
This is more a criticism of codex's linux-sandboxing, which you're just wrapping, but it's the first I've ever looked at it. I don't see how it makes sense to invoke bwrap as a forked subprocess. Bubblewrap can't do anything beyond what you can do with unshare directly, which you can simply invoke as a system call without needing to spawn a subprocess or requiring the user to have bwrap installed. It kinds of reeks of amateur hour when developers effectively just translate shell scripts into compiled languages by using whatever variant of "system" is available to make the same command invocations you would make through a shell, as opposed to actually using the system call API. Especially when the invocation is crafted from user input, there's a long history of exploits arising from stuff like this. Writing it in Rust does nothing for you when you're just using Rust to call a different CLI tool that isn't written in Rust.