Only three metrics went up: LOC, token spend and presumably release speed/frequency.
What's the markup (profit/COGS)? is it up?
Customer satisfaction? Is NPS moving one way or another?
show comments
nater5000
This post is funny to me, because I feel it falls well short of what a proper, forward-thinking "AI-native" engineering org would actually look like. That is, the author is presenting this as kind of a radical departure from normal engineering, but I just see it as normal engineering with more AI. I don't see that as AI-native.
To me, true "AI-native" engineering will be one where questions regarding how humans will do things like review code are irrelevant. You explicitly throw away any concept of typical software engineering and, instead, differ to the AI. Your job, as a human, is solely to do what the AI can't physically do. That's not the same as closing gaps between what AI is currently capable of doing and what you'd need it to be capable of doing. It's literally enforcing that anything an AI is able to do is treated as correct in itself, and you, as a human, only support it by being a human.
If that sounds outlandish, that's because it is. We don't have such capable AI systems yet. It's possible we never will. But that's beyond the point. The reality is trying to make an "AI-native" engineering org by just layering AI on top of existing processes and systems is futile. If your expectation is that AI will continue to improve in capabilities at an accelerating rate, then these stop-gaps are pointless. You might as well just ignore AI altogether until it is able to completely take over, because that's the inevitable goal, right?
If you're willing to buy that will be possible, then my guess is that it will occur in the form of small scale startups and actual mavericks rather than someone trying to run an existing business who will simply be too risk adverse to go all in when the time comes. Some guy in his basement is more likely to leverage AI "properly" to build an actual "AI-native" engineering org because that person won't have much to lose.
show comments
simonw
It this point, a whole lot of teams have been leaning into AI-first, not-entirely-reviewed code since around March this year.
That means we are already 6-8 months in, and not far from having a full year of experience of what that actually looks like, albeit weirdly influenced by the increase in model capabilities during that period.
I'm looking forward to those retrospectives. Did this turn out to work OK for the teams doing it? What were the biggest failings and/or surprises? With hindsight, did this produce good quality software?
show comments
lubujackson
Working at an "AI native" company and these questions and stats are far too hand-wavvy for me.
What works:
- Empowering engineers. Giving engineers more trust, ownership and product decision-making leads to more engaged engineers vs. micromanaged tickets. Engineers who are treated like owners tend to notice cross-cutting problems and push to get them cleaned up.
- Hire for diversity. Not race or gender, but diversity of thought. Different ages, backgrounds, approaches to AI, focus on frontend/backend, etc. Race/gender diversity tends to come along for free. This is a huge edge because every LLM is spinning up the same next token prediction, so approaching problems from different perspectives gives more robust coverage across your codebase.
- Flat org structure, low ego. Interns and 20 year vets work side-by-side and teach each other things - hire for openmindedness.
- Communicate about product. What should it do? How do we want a feature to work, and how to handle certain edge cases? If the product is well understood, the code follows. In-person/hybrid is valuable.
- Architecture matters more than ever, but it is easier than ever to fix. Listen when engineers mention tech debt. Notice when a simple change touches 27 files. Senior engineers shine for this, because they tend to have a better nose for code smell.
- Reviewing still matters. Anthropic may throw tokens at reviewing, but it is a great way to share ambient product/code knowledge across the team. 200 engineers throwing code against the wall is chaos and leads to YOLO codebases. You can move fast AND keep a rough mental model of the codebase intact.
show comments
fg137
I was just looking at some user-facing error messages Claude generated for me which contained unnecessary/irrelevant technical details while failed to provide clear instructions or fix.
Earlier, I reviewed code that does way more than specified scope, changes public API and introduces new behavior. I had to almost redo the whole thing.
I thought to myself, no way I am going to ship the code as written without actually reading through it (or as some people put it, only looking at the "intent"). I'll get more complaints from users while having absolutely no idea what's going on with the code or actual behavior.
It's a puzzle to me how these orgs can pull it off.
show comments
mirmor23
Spent few minutes reading the infomercial -- regretted it. Engineering principles form the bedrock first, and AI-tools are made to fit in them, not the other way around as this infomercial propounds. Its linkedin vibe is a big turnoff too.
It describes a team spending 30x payroll on tokens - $3,000,000 in token spend per month - building a multiplayer game where incidents and bug reports from users are passed directly to Fable 5/5.1 agents who then act on them and deploy the changes to production.
"We consider it an outage if we don't ship every hour"
It's worth reading because it's either a complete disaster or a glimpse of the future, or quite possibly both.
show comments
erminpour
Maybe it's just me, but "How much we ship per engineer" doesn't seem like the correct metric to track.
If that's the benchmark, developers are incentivized to produce over-engineered and bloated code.
show comments
liorgrossman
I see a lot of builders (myself included) falling into the trap of focusing on the wrong things. Development velocity doesn't matter per se, what matters are things like growth in profit or traction. It's really easy to conflate automation/systems/process with actual business results. Movement ≠ Progress.
zmmmmm
> None of this was free. Through the summer, much of our engineering effort went into tests and tooling, and customer-facing changes per engineer stayed between 17 and 30 a month
This is the reality I think for a lot of orgs and is absorbing a lot of the productivity gains. The gap between vibe coding a green field app from nothing which is inherently "AI native" and implementing AI tooling into complex orgs with large amounts of legacy infrastructure, integrations, ingrained practices etc is enormous. For us, it has been a big effort to enable agentic coding over a significant footprint of our codebase. And all this is happening while the plane keeps flying, with near zero tolerance for defects in critical systems. There is a lot more to it than you will understand if you just look at twitter or Hacker News.
t43562
I remember optimizing a huge build that took days. We optimised it to only hours on a cluster - it took years to get this right. We realised at the end of this that the packaging system (handled by another team) was taking about 2x longer than the build because there was one particularly huge zipfile that sat there compressing for ever and ever and ever while the rest of the cluster was idle.
As in this story we blew enormous effort optimizing something that was very difficult and because of our blinkers (and our management's blinkers) we missed that it was only able to achieve a limited result. We didn't look at the whole system. In the end it was never fixed because......siloed thinking...or something.
amelius
> What it cost us
> None of this was free. Through the summer, much of our engineering effort went into tests and tooling, and customer-facing changes per engineer stayed between 17 and 30 a month.
What it cost you is that you helped the AI labs become better engineers. At a certain point the Big Flip will happen, and the AI labs will stop their offerings and instead start competing with the engineers.
show comments
tty456
This guy knows nothing. A few years experience plus a Claude account...
Is there any way to read any of this if I don't have a twitter account?
[deleted]
jbvlkt
For the argument lets say that this approach works and results in high quality apps. But will be users happy that app is updated 3 times a week and all updates add major changes? Because code generation is so cheap and fast it means we should even more thing about what to build to not overwhelm our users and actually solve their problems.
sellmesoap
In my best impression of the slapchop guy:
First you take your Blendtech and "will it blend" your engineers, then upload the result to the cloud inside what is called an S3 bucket (don't forget to set access permissions on your buckets kids!) to be digested by the clankers, sit back and profit!
Dark humor aside I think disconnecting a bunch of problem solvers from their computing equipment could really enhance the world, putting that analytical power into the world in person could move a lot of cogs, rather then marshalling JSON.
fleetml
Nice work. How do you handle swapping in a new model version, hot reload or restart? And what happens to a batch that's mid-run when it changes?
linuxhansl
I know many (big) places that are already back-paddling.
From token leadership boards (more = better) to now saving on AI cost, because the benefit was less than expected. As if this wasn't obvious. Sigh.
Sure MS, Meta, etc, are still laying off engineers to have money for data centers. But I am 99.9% sure that they will thoroughly regret that move in less than 2 years.
skeeter2020
We shipped more code!
We merged 5x faster!
We spent more tokens!
>> Every time we sped up one step, work piled up at the next... we started going through the entire process of shipping software, one step at a time...
I feel like I'm being gaslit here. You can become CTO with this lack of business fundamentals, and a basic misunderstanding of systems engineering?
show comments
MisterMunchkin
Can we ban twitter links? I have to use a hack to even read it and it’s always just slop. If you can’t be bothered to write an actual article, why should we bother to read it.
worik
Here we are again, thirty years after I was taught the "waterfall" paradigm at business school.
When asked about "customer satisfaction" we were told, awkwardly, that became "political"
The paradigm did not survive its encounter with reality, and I do not think this one will either
The "only" 175% increase in productivity is very impressive, and seems realistic to me, when you remove the chores of writing code (thank you to coding agents)
What you build is still very dependant on what you need, and you do not know that until you build it
It is a paradox only for the "waterfall" paradigm. We rejected it, in the 1990s for a reason
Not a single mention of value or revenue.
Only three metrics went up: LOC, token spend and presumably release speed/frequency.
What's the markup (profit/COGS)? is it up?
Customer satisfaction? Is NPS moving one way or another?
This post is funny to me, because I feel it falls well short of what a proper, forward-thinking "AI-native" engineering org would actually look like. That is, the author is presenting this as kind of a radical departure from normal engineering, but I just see it as normal engineering with more AI. I don't see that as AI-native.
To me, true "AI-native" engineering will be one where questions regarding how humans will do things like review code are irrelevant. You explicitly throw away any concept of typical software engineering and, instead, differ to the AI. Your job, as a human, is solely to do what the AI can't physically do. That's not the same as closing gaps between what AI is currently capable of doing and what you'd need it to be capable of doing. It's literally enforcing that anything an AI is able to do is treated as correct in itself, and you, as a human, only support it by being a human.
If that sounds outlandish, that's because it is. We don't have such capable AI systems yet. It's possible we never will. But that's beyond the point. The reality is trying to make an "AI-native" engineering org by just layering AI on top of existing processes and systems is futile. If your expectation is that AI will continue to improve in capabilities at an accelerating rate, then these stop-gaps are pointless. You might as well just ignore AI altogether until it is able to completely take over, because that's the inevitable goal, right?
If you're willing to buy that will be possible, then my guess is that it will occur in the form of small scale startups and actual mavericks rather than someone trying to run an existing business who will simply be too risk adverse to go all in when the time comes. Some guy in his basement is more likely to leverage AI "properly" to build an actual "AI-native" engineering org because that person won't have much to lose.
It this point, a whole lot of teams have been leaning into AI-first, not-entirely-reviewed code since around March this year.
That means we are already 6-8 months in, and not far from having a full year of experience of what that actually looks like, albeit weirdly influenced by the increase in model capabilities during that period.
I'm looking forward to those retrospectives. Did this turn out to work OK for the teams doing it? What were the biggest failings and/or surprises? With hindsight, did this produce good quality software?
Working at an "AI native" company and these questions and stats are far too hand-wavvy for me.
What works: - Empowering engineers. Giving engineers more trust, ownership and product decision-making leads to more engaged engineers vs. micromanaged tickets. Engineers who are treated like owners tend to notice cross-cutting problems and push to get them cleaned up. - Hire for diversity. Not race or gender, but diversity of thought. Different ages, backgrounds, approaches to AI, focus on frontend/backend, etc. Race/gender diversity tends to come along for free. This is a huge edge because every LLM is spinning up the same next token prediction, so approaching problems from different perspectives gives more robust coverage across your codebase. - Flat org structure, low ego. Interns and 20 year vets work side-by-side and teach each other things - hire for openmindedness. - Communicate about product. What should it do? How do we want a feature to work, and how to handle certain edge cases? If the product is well understood, the code follows. In-person/hybrid is valuable. - Architecture matters more than ever, but it is easier than ever to fix. Listen when engineers mention tech debt. Notice when a simple change touches 27 files. Senior engineers shine for this, because they tend to have a better nose for code smell. - Reviewing still matters. Anthropic may throw tokens at reviewing, but it is a great way to share ambient product/code knowledge across the team. 200 engineers throwing code against the wall is chaos and leads to YOLO codebases. You can move fast AND keep a rough mental model of the codebase intact.
I was just looking at some user-facing error messages Claude generated for me which contained unnecessary/irrelevant technical details while failed to provide clear instructions or fix.
Earlier, I reviewed code that does way more than specified scope, changes public API and introduces new behavior. I had to almost redo the whole thing.
I thought to myself, no way I am going to ship the code as written without actually reading through it (or as some people put it, only looking at the "intent"). I'll get more complaints from users while having absolutely no idea what's going on with the code or actual behavior.
It's a puzzle to me how these orgs can pull it off.
Spent few minutes reading the infomercial -- regretted it. Engineering principles form the bedrock first, and AI-tools are made to fit in them, not the other way around as this infomercial propounds. Its linkedin vibe is a big turnoff too.
Related: this article was very unilkely make it to the Hacker News homepage thanks to the clickbait headline, but I recommend reading it anyway: https://twitter.com/jsnnsa/status/2115623382721126826
It describes a team spending 30x payroll on tokens - $3,000,000 in token spend per month - building a multiplayer game where incidents and bug reports from users are passed directly to Fable 5/5.1 agents who then act on them and deploy the changes to production.
"We consider it an outage if we don't ship every hour"
It's worth reading because it's either a complete disaster or a glimpse of the future, or quite possibly both.
Maybe it's just me, but "How much we ship per engineer" doesn't seem like the correct metric to track.
If that's the benchmark, developers are incentivized to produce over-engineered and bloated code.
I see a lot of builders (myself included) falling into the trap of focusing on the wrong things. Development velocity doesn't matter per se, what matters are things like growth in profit or traction. It's really easy to conflate automation/systems/process with actual business results. Movement ≠ Progress.
> None of this was free. Through the summer, much of our engineering effort went into tests and tooling, and customer-facing changes per engineer stayed between 17 and 30 a month
This is the reality I think for a lot of orgs and is absorbing a lot of the productivity gains. The gap between vibe coding a green field app from nothing which is inherently "AI native" and implementing AI tooling into complex orgs with large amounts of legacy infrastructure, integrations, ingrained practices etc is enormous. For us, it has been a big effort to enable agentic coding over a significant footprint of our codebase. And all this is happening while the plane keeps flying, with near zero tolerance for defects in critical systems. There is a lot more to it than you will understand if you just look at twitter or Hacker News.
I remember optimizing a huge build that took days. We optimised it to only hours on a cluster - it took years to get this right. We realised at the end of this that the packaging system (handled by another team) was taking about 2x longer than the build because there was one particularly huge zipfile that sat there compressing for ever and ever and ever while the rest of the cluster was idle.
As in this story we blew enormous effort optimizing something that was very difficult and because of our blinkers (and our management's blinkers) we missed that it was only able to achieve a limited result. We didn't look at the whole system. In the end it was never fixed because......siloed thinking...or something.
> What it cost us
> None of this was free. Through the summer, much of our engineering effort went into tests and tooling, and customer-facing changes per engineer stayed between 17 and 30 a month.
What it cost you is that you helped the AI labs become better engineers. At a certain point the Big Flip will happen, and the AI labs will stop their offerings and instead start competing with the engineers.
This guy knows nothing. A few years experience plus a Claude account...
https://xxcancel.com/sgondala2/status/2115832070274723875
Is there any way to read any of this if I don't have a twitter account?
For the argument lets say that this approach works and results in high quality apps. But will be users happy that app is updated 3 times a week and all updates add major changes? Because code generation is so cheap and fast it means we should even more thing about what to build to not overwhelm our users and actually solve their problems.
In my best impression of the slapchop guy:
First you take your Blendtech and "will it blend" your engineers, then upload the result to the cloud inside what is called an S3 bucket (don't forget to set access permissions on your buckets kids!) to be digested by the clankers, sit back and profit!
Dark humor aside I think disconnecting a bunch of problem solvers from their computing equipment could really enhance the world, putting that analytical power into the world in person could move a lot of cogs, rather then marshalling JSON.
Nice work. How do you handle swapping in a new model version, hot reload or restart? And what happens to a batch that's mid-run when it changes?
I know many (big) places that are already back-paddling. From token leadership boards (more = better) to now saving on AI cost, because the benefit was less than expected. As if this wasn't obvious. Sigh.
Sure MS, Meta, etc, are still laying off engineers to have money for data centers. But I am 99.9% sure that they will thoroughly regret that move in less than 2 years.
We shipped more code! We merged 5x faster! We spent more tokens!
>> Every time we sped up one step, work piled up at the next... we started going through the entire process of shipping software, one step at a time...
I feel like I'm being gaslit here. You can become CTO with this lack of business fundamentals, and a basic misunderstanding of systems engineering?
Can we ban twitter links? I have to use a hack to even read it and it’s always just slop. If you can’t be bothered to write an actual article, why should we bother to read it.
Here we are again, thirty years after I was taught the "waterfall" paradigm at business school.
When asked about "customer satisfaction" we were told, awkwardly, that became "political"
The paradigm did not survive its encounter with reality, and I do not think this one will either
The "only" 175% increase in productivity is very impressive, and seems realistic to me, when you remove the chores of writing code (thank you to coding agents)
What you build is still very dependant on what you need, and you do not know that until you build it
It is a paradox only for the "waterfall" paradigm. We rejected it, in the 1990s for a reason
It did not work
why would you want to do that?
Slop.
[flagged]
[flagged]