i'm not sure why people expect agents to one shot everything to perfection with just a prompt.
There's a reason why we talk about software development lifecycle, design, architecture, testing ... It's because it's been the most reliable way to build and ship software. We shouldn't expect discard this and expect agents to perform well outside of this.
I'm treating LLM agents as junior devs who happen to have vast knowledge of software engineering. As their team leader i make them go through planning, implementation, bug sweeping cycles using strict workflows. And it works quite well, i've been working on several large projects (1M+ LOC java,typescript,c/c++) and by any measure the projects are healthy. Sure the code isn't that beautiful, sure i'd have written things differently but it's pretty good nonetheless.
Shameless plug here: i've been also working on https://kodfactory.com, the code factory i've built to work on these large projects with workflows, reviews, etc ... I'm cleaning things up to open source it later.
show comments
supriyo-biswas
I feel this, yes.
In effect, I’ve always wanted a pair programmer agent, not a zero to one programming agent. Unfortunately models these days are mostly of the latter kind and it has caused a major disruption in the way I work. I’d much rather appreciate a small model making fast and specific edits that I ask if it, rather than ingesting 20 files to make changes, and then starting to write tests, etc.
show comments
dzhar11
This article somewhat reflects my experience with autonomous agentic coding. I've run several experiments with similar results: the agent burns through all my tokens while making very little progress, or produces something unacceptable.
So I'd rather micromanage the process step by step. It takes more of my time, but the result is much, much closer to what I actually wanted.
alehlopeh
I tried, but I’m not sure I understand. The vibe tax is caused by the model trying to one-shot everything and doing so requires unnecessary tests? How are vibe coders training the model over months? Do you mean their sessions and preferences are being fed back into the RL?
show comments
jumploops
I've found that LLMs make throwaway software better than I ever did.
They handle edge cases, catch bugs, and write tests that I'd never write.
Even if, however, this leads to the average piece of software improving, this one-shot complexity has the same issues as any large project. The more code, the longer it takes to steer the ship.
This "rising tide lifts all boats" mentality will make exceptional software even rarer than it is today.
Excited for the Roller Coaster Tycoons of tomorrow[0].
I’ve never had an agent fail to write the actual implementation. Has it done so badly, yes, but not nothing but tests. This sounds to me like a rare case that doesn’t generalize.
If the general idea is that these agents write too many tests, sure I guess? ‘Too many tests’ doesn’t sound like a failure case of engineering to me; typically software has had too few tests. Also, a lot of the power of these agents is their ability to self-verify and correct, which the test loop is a part of.
Nobody is making you pay this supposed tax. Just tell it not to write tests.
ad_fontes
I feel like I'm living in a parallel universe when I read these types of posts.
My agents have never created code that is straight-up garbage and I have never flushed a week's worth of tokens down the toilet. I just can't identify with all the constant complaints about AI-assisted coding.
And my biggest project isn't some hello world app. It's a self-hosted, privacy-focused personal financial management application that I intend to open source. It's about 126k LOC against 240k LOC of regression tests and 30k LOC of CI/CD pipeline. I'm doing 24x7 mutation testing on a dedicated box against the accounting engine and temporal systems. I even have specialized agents doing audits against Regulation Z (US banking law) criteria so the app models the required behavior of banks.
Most of my complaints about everything are nits, like the overly verbose and dense way LLMs communicate with me. Or their predisposition to add, add, and add more stuff when proper engineering practices are more often about subtraction (but I've built mitigation guardrails against a lot of that).
show comments
nippoo
You can absolutely prompt agents not to write tests, or not to write extraneous asserts, or whatever, and I find that generally quite useful for the kind of code I write. I don't think it's "months of users training it", it's more that a lot of people do want a one-shot agent, and having a good test set really helps that.
freepiai
This really resonated for me. It's like the smarter the model gets, somehow the more tokens get burned? Same failure mode whether you’re on Claude, Codex, or Cursor: the harness will spend the whole pool if you let it. I'm building my own Harness on top of pi that is add supported (www.freepi.ai) mostly because pi is so much more efficient with tokens. (That said, it tens to be slower and vastly more verbose with information I don't need to know). But yeah, since I'm trying to offer free ad supported inference the vibe tax would kill the business model. I've even been thinking about installing the 'caveman' skill to reign in token costs.
itishappy
I feel like these are two competing goals:
* the dev wants to describe an app in natural language then fall asleep while an AI works on it
* the dev wishes that the same AI would write less comprehensive tests
What exactly is a vibe coder to this dev?
hmokiguess
Needs more info, has a good storyline but I am left trying to understand the overall pattern and trend implied there.
robomc
This is a confusing description of a real thing. They're clearly biasing the models more and more towards long horizon end-to-end software development, which leads to impressive "claude, build an X make no mistakes" demos, but is mainly an annoyance for expert users doing real work.
(If you give claude an inch these days it'll just steamroll through a whole program of work without checking what it should be doing - a kind of overenthusiastic pull towards the first draft that is often detrimental and definitely wastes tokens, and even for very basic tasks it's using many more tokens than it should because it's doing this full belt and braces thing for everything, just in case you're an idiot).
But also... it's something you can easily reign in if you want to.
esafak
Create a spec and have a dumb model execute it. Problem solved.
show comments
pgt
Pre-October 2025, maybe yes. But now? Couldn't disagree more. There is no insight in this post.
i'm not sure why people expect agents to one shot everything to perfection with just a prompt.
There's a reason why we talk about software development lifecycle, design, architecture, testing ... It's because it's been the most reliable way to build and ship software. We shouldn't expect discard this and expect agents to perform well outside of this.
I'm treating LLM agents as junior devs who happen to have vast knowledge of software engineering. As their team leader i make them go through planning, implementation, bug sweeping cycles using strict workflows. And it works quite well, i've been working on several large projects (1M+ LOC java,typescript,c/c++) and by any measure the projects are healthy. Sure the code isn't that beautiful, sure i'd have written things differently but it's pretty good nonetheless.
Shameless plug here: i've been also working on https://kodfactory.com, the code factory i've built to work on these large projects with workflows, reviews, etc ... I'm cleaning things up to open source it later.
I feel this, yes.
In effect, I’ve always wanted a pair programmer agent, not a zero to one programming agent. Unfortunately models these days are mostly of the latter kind and it has caused a major disruption in the way I work. I’d much rather appreciate a small model making fast and specific edits that I ask if it, rather than ingesting 20 files to make changes, and then starting to write tests, etc.
This article somewhat reflects my experience with autonomous agentic coding. I've run several experiments with similar results: the agent burns through all my tokens while making very little progress, or produces something unacceptable.
So I'd rather micromanage the process step by step. It takes more of my time, but the result is much, much closer to what I actually wanted.
I tried, but I’m not sure I understand. The vibe tax is caused by the model trying to one-shot everything and doing so requires unnecessary tests? How are vibe coders training the model over months? Do you mean their sessions and preferences are being fed back into the RL?
I've found that LLMs make throwaway software better than I ever did.
They handle edge cases, catch bugs, and write tests that I'd never write.
Even if, however, this leads to the average piece of software improving, this one-shot complexity has the same issues as any large project. The more code, the longer it takes to steer the ship.
This "rising tide lifts all boats" mentality will make exceptional software even rarer than it is today.
Excited for the Roller Coaster Tycoons of tomorrow[0].
[0]https://en.wikipedia.org/wiki/RollerCoaster_Tycoon_(video_ga...
I’ve never had an agent fail to write the actual implementation. Has it done so badly, yes, but not nothing but tests. This sounds to me like a rare case that doesn’t generalize.
If the general idea is that these agents write too many tests, sure I guess? ‘Too many tests’ doesn’t sound like a failure case of engineering to me; typically software has had too few tests. Also, a lot of the power of these agents is their ability to self-verify and correct, which the test loop is a part of.
Nobody is making you pay this supposed tax. Just tell it not to write tests.
I feel like I'm living in a parallel universe when I read these types of posts.
My agents have never created code that is straight-up garbage and I have never flushed a week's worth of tokens down the toilet. I just can't identify with all the constant complaints about AI-assisted coding.
And my biggest project isn't some hello world app. It's a self-hosted, privacy-focused personal financial management application that I intend to open source. It's about 126k LOC against 240k LOC of regression tests and 30k LOC of CI/CD pipeline. I'm doing 24x7 mutation testing on a dedicated box against the accounting engine and temporal systems. I even have specialized agents doing audits against Regulation Z (US banking law) criteria so the app models the required behavior of banks.
Most of my complaints about everything are nits, like the overly verbose and dense way LLMs communicate with me. Or their predisposition to add, add, and add more stuff when proper engineering practices are more often about subtraction (but I've built mitigation guardrails against a lot of that).
You can absolutely prompt agents not to write tests, or not to write extraneous asserts, or whatever, and I find that generally quite useful for the kind of code I write. I don't think it's "months of users training it", it's more that a lot of people do want a one-shot agent, and having a good test set really helps that.
This really resonated for me. It's like the smarter the model gets, somehow the more tokens get burned? Same failure mode whether you’re on Claude, Codex, or Cursor: the harness will spend the whole pool if you let it. I'm building my own Harness on top of pi that is add supported (www.freepi.ai) mostly because pi is so much more efficient with tokens. (That said, it tens to be slower and vastly more verbose with information I don't need to know). But yeah, since I'm trying to offer free ad supported inference the vibe tax would kill the business model. I've even been thinking about installing the 'caveman' skill to reign in token costs.
I feel like these are two competing goals:
* the dev wants to describe an app in natural language then fall asleep while an AI works on it
* the dev wishes that the same AI would write less comprehensive tests
What exactly is a vibe coder to this dev?
Needs more info, has a good storyline but I am left trying to understand the overall pattern and trend implied there.
This is a confusing description of a real thing. They're clearly biasing the models more and more towards long horizon end-to-end software development, which leads to impressive "claude, build an X make no mistakes" demos, but is mainly an annoyance for expert users doing real work.
(If you give claude an inch these days it'll just steamroll through a whole program of work without checking what it should be doing - a kind of overenthusiastic pull towards the first draft that is often detrimental and definitely wastes tokens, and even for very basic tasks it's using many more tokens than it should because it's doing this full belt and braces thing for everything, just in case you're an idiot).
But also... it's something you can easily reign in if you want to.
Create a spec and have a dumb model execute it. Problem solved.
Pre-October 2025, maybe yes. But now? Couldn't disagree more. There is no insight in this post.