Ex-Apple engineer here. I really like your implementation. A few years ago I built a similar tool to help me automate the testing of some of my native macOS apps. Being able to run multiple UI automation tests simultaneously was the big win in my case.
My only criticism is enabling telemetry by default. I'm a fan of having people opt-in.
show comments
krackers
Nice! Thanks for the technical writeup, ~2 weeks from me wondering how it's implemented [1] to being able to play with a replicated version!
This is one of the coolest hacks I've seen recently. Having done some much less involved MacOS hacking, I can't help but wonder if we may finally see momentum behind some flavor of agent-friendly Linux/Android if Apple doesn't give us more ways to let agents interact with our machines.
show comments
pimlottc
What is specific about this for using with agents? As opposed to offering it as a general automation library for any use?
show comments
j-conn
Incredible! I’m interested in doing something similar on windows, have you looked into that at all? Apparently codex computer use plans to support this on windows in the future. Were you able to see how codex was doing it, or the inspiration was just “they’ve shown it’s possible”?
show comments
alsetmusic
I tried out their Loom vm software a couple of months back. Worked well, fwiw. I'm not using it anymore because I decided to just give agents direct (supervised) access to my devices.
show comments
BenFranklin100
Being new to the idea of using agents to run programs on one’s computer, could someone provide several use cases?
show comments
davey2wavey
Its looking great.
The audit trail question is interesting and I haven't seen it come up much. When an agent clicks through an ERP or edits a file, you've got logs, but how do you explain the "why" behind each decision to, say, a compliance team?
Curious if that's something you're thinking about or if it's too early.
Ex-Apple engineer here. I really like your implementation. A few years ago I built a similar tool to help me automate the testing of some of my native macOS apps. Being able to run multiple UI automation tests simultaneously was the big win in my case.
My only criticism is enabling telemetry by default. I'm a fan of having people opt-in.
Nice! Thanks for the technical writeup, ~2 weeks from me wondering how it's implemented [1] to being able to play with a replicated version!
[1] https://news.ycombinator.com/item?id=47799128
This is one of the coolest hacks I've seen recently. Having done some much less involved MacOS hacking, I can't help but wonder if we may finally see momentum behind some flavor of agent-friendly Linux/Android if Apple doesn't give us more ways to let agents interact with our machines.
What is specific about this for using with agents? As opposed to offering it as a general automation library for any use?
Incredible! I’m interested in doing something similar on windows, have you looked into that at all? Apparently codex computer use plans to support this on windows in the future. Were you able to see how codex was doing it, or the inspiration was just “they’ve shown it’s possible”?
I tried out their Loom vm software a couple of months back. Worked well, fwiw. I'm not using it anymore because I decided to just give agents direct (supervised) access to my devices.
Being new to the idea of using agents to run programs on one’s computer, could someone provide several use cases?
Its looking great.
The audit trail question is interesting and I haven't seen it come up much. When an agent clicks through an ERP or edits a file, you've got logs, but how do you explain the "why" behind each decision to, say, a compliance team?
Curious if that's something you're thinking about or if it's too early.