I never took tokenmaxxing to be about improving productivity directly; mundane feature work that comes out of it is just a side effect. I always saw it as a race between these big tech companies to get a generational advantage by being the one to discover the way of the future, with respect to harnessing AI to actually and truly automate software development.
EDIT: whoa, I used "way of the future" as a reference to Howard Hughes in "The Aviator", not this Way of the Future religious organization thing I just stumbled on; no intended reference there.
show comments
rowanseymour
Now feels like a very good time to be a small team of experienced developers who can largely work on stuff by themselves and not a corporation of hundreds of developers of varying abilities all now trying to show how much code they can generate and how many tokens they can burn.
show comments
jimkleiber
I wish they would spend some of that on the help/support function within their apps. Whenever I have a problem on Uber, it feels like a never-ending maze to figure out how to get any support, and I consider myself versed in navigating unseemly UI, I can only imagine how much it might frustrate people who struggle navigating apps.
Any idea why their help function seems to impenetrable and if AI might help with it?
louiereederson
Anthropic's annualized run rate is >$40b according to outside reporting. AWS hit that by Q4 2019. There were still debates on public cloud vs on prem at that time, but by late 2019 public cloud had facilitated the creation or adoption of entire categories of software within SaaS and PaaS, not to mention consumer internet businesses like Uber and Airbnb. The net impact of AI coding tools is far more ambiguous in comparison.
The profitability comparison is fraught but worth noting that by then AWS was already extremely profitable.
show comments
ex-aws-dude
What has been the end result of all the tokens companies are burning?
Where does it show up in quarterly results?
I can’t see how it’s sustainable just based on “this feels more productive”
show comments
cdrnsf
This doesn't account for the fact that review is still a bottleneck, engineers understand much less of the code they're shipping and there'll likely be tech debt they'll have to unwind in the future.
heathrow83829
Didn't google say that AI had increased their company's productivity by 10%? if that's the case, then how can they justify spending 50% to 100% of wages on it?
juancn
There's also the issue that in any large-ish org, code production is hardly ever the bottleneck.
show comments
dwa3592
I am not sure how uber is operating internally around the use of tokens but if they actually shipped features faster than before then it is still a win. if they learn that users don't want these features or want a different version of it; they have learned this new knowledge faster than they would have if they manually coded those features, which means in principle you should be able to iterate faster. but this will collide with creative ceiling that humans exhibit in a span of time and on top of that uber is prioritizing spending money on tokens over humans which seems like a mistake. you need humans for creativity.
I still get picked up by an Uber the same way. As an end user, nothing has changed for me.
So I wonder what the heck were all those billions of AI tokens burnt on that they extinguished it in just 4 months into the year?
show comments
deaton
Goodhart's law strikes again. Stop giving your engineers token-burning quotas or they'll burn tokens.
show comments
expedition32
You don't need justification to spend other people's money!
Nobody's going to jail.
DesiLurker
I actually think Chinese models have already popped the bubble, we just dont see it yet. the only way to justify AI IPO market cap is basically if they get to hold most sw industry code hostage and then token-flate budgets to collect AI tax. short of that AI expense would very quickly mean revert to model + some margin. this means the moat for AI 'trillion club' is gone. In fact AI virtually guarantees that there is no execution-moat left anywhere, definitely not in code or that engineer with knowledge about that obscure mechanism. without the moat most of the sw ecossytem's margins would shrink (as they should).
Ironically enough the only moat left would be what you can buy from Washington.
chollida1
I do find it to be true that with coding agents the famous quote from Jurassic Park goes through my head multiple time a day
"our scientists were so preoccupied with whether they could, they never stopped to ask if they should.
I've now come to the realization that if I'm having an llm work constantly all day writing code for me i'm probably doing something wrong as I'm no longer focusing on the core issue itself.
I may be in a minority here in that I write code to augment my self and not to ship to others so I can tell very quickly if I'm just gold platting something or if i'm actually delivering real value to my trading or risk management.
sbmthakur
Affordable inference will be around longer if more Big tech companies cap their AI sending.
Mistletoe
It feels like maybe the wheels are starting to fall off the AI hype train. I expect complete collapse once people start figuring out that the numbers on all this don’t make sense. I’m looking for investment portfolios that will weather that storm. If you are reading this and have a similar curiosity, this is a great place to start.
hot take: token spend can be used a honey pot, especially when compared to what you deliver. spend accordingly!
lenerdenator
My concern here is that they'll mix two things:
1) workforce reduction
2) AI spend (reduce tokenmaxing)
They'll expect fewer people to do more with even less, while "more" is continuously increasing.
When I say "more", I mean that the deluge that engineering teams deal with comes from two sources:
1) the business side of companies - marketing, sales, solutions teams, etc.
2) outside actors, mainly security threats
The first source can now move to generate work for engineering faster than ever. They expect the nerds to do what they're told and get the features out now. The more features, the better the product, right? The saving grace here is that they're bound by the same management concerns that engineering has. There's only so much money that they themselves can throw at generating more work for engineering teams, and that might also come under scrutiny from management, so that acts as a brake.
The second source has no such brake, especially not with security threats. Either there's good money to be made by holding company data hostage, or there's an endless supply of resources (read: nation-state resources) dedicated to the effort to attack the company's digital assets. And of course, they're using AI to enable this, just without the "but what about the shareholders!?" handwringing.
If you aren't very, very careful with your token cutting, you're going to put yourself at a disadvantage against that second group.
I never took tokenmaxxing to be about improving productivity directly; mundane feature work that comes out of it is just a side effect. I always saw it as a race between these big tech companies to get a generational advantage by being the one to discover the way of the future, with respect to harnessing AI to actually and truly automate software development.
EDIT: whoa, I used "way of the future" as a reference to Howard Hughes in "The Aviator", not this Way of the Future religious organization thing I just stumbled on; no intended reference there.
Now feels like a very good time to be a small team of experienced developers who can largely work on stuff by themselves and not a corporation of hundreds of developers of varying abilities all now trying to show how much code they can generate and how many tokens they can burn.
I wish they would spend some of that on the help/support function within their apps. Whenever I have a problem on Uber, it feels like a never-ending maze to figure out how to get any support, and I consider myself versed in navigating unseemly UI, I can only imagine how much it might frustrate people who struggle navigating apps.
Any idea why their help function seems to impenetrable and if AI might help with it?
Anthropic's annualized run rate is >$40b according to outside reporting. AWS hit that by Q4 2019. There were still debates on public cloud vs on prem at that time, but by late 2019 public cloud had facilitated the creation or adoption of entire categories of software within SaaS and PaaS, not to mention consumer internet businesses like Uber and Airbnb. The net impact of AI coding tools is far more ambiguous in comparison.
The profitability comparison is fraught but worth noting that by then AWS was already extremely profitable.
What has been the end result of all the tokens companies are burning?
Where does it show up in quarterly results?
I can’t see how it’s sustainable just based on “this feels more productive”
This doesn't account for the fact that review is still a bottleneck, engineers understand much less of the code they're shipping and there'll likely be tech debt they'll have to unwind in the future.
Didn't google say that AI had increased their company's productivity by 10%? if that's the case, then how can they justify spending 50% to 100% of wages on it?
There's also the issue that in any large-ish org, code production is hardly ever the bottleneck.
I am not sure how uber is operating internally around the use of tokens but if they actually shipped features faster than before then it is still a win. if they learn that users don't want these features or want a different version of it; they have learned this new knowledge faster than they would have if they manually coded those features, which means in principle you should be able to iterate faster. but this will collide with creative ceiling that humans exhibit in a span of time and on top of that uber is prioritizing spending money on tokens over humans which seems like a mistake. you need humans for creativity.
Dupe: https://news.ycombinator.com/item?id=48268871
I still get picked up by an Uber the same way. As an end user, nothing has changed for me.
So I wonder what the heck were all those billions of AI tokens burnt on that they extinguished it in just 4 months into the year?
Goodhart's law strikes again. Stop giving your engineers token-burning quotas or they'll burn tokens.
You don't need justification to spend other people's money!
Nobody's going to jail.
I actually think Chinese models have already popped the bubble, we just dont see it yet. the only way to justify AI IPO market cap is basically if they get to hold most sw industry code hostage and then token-flate budgets to collect AI tax. short of that AI expense would very quickly mean revert to model + some margin. this means the moat for AI 'trillion club' is gone. In fact AI virtually guarantees that there is no execution-moat left anywhere, definitely not in code or that engineer with knowledge about that obscure mechanism. without the moat most of the sw ecossytem's margins would shrink (as they should).
Ironically enough the only moat left would be what you can buy from Washington.
I do find it to be true that with coding agents the famous quote from Jurassic Park goes through my head multiple time a day
"our scientists were so preoccupied with whether they could, they never stopped to ask if they should.
I've now come to the realization that if I'm having an llm work constantly all day writing code for me i'm probably doing something wrong as I'm no longer focusing on the core issue itself.
I may be in a minority here in that I write code to augment my self and not to ship to others so I can tell very quickly if I'm just gold platting something or if i'm actually delivering real value to my trading or risk management.
Affordable inference will be around longer if more Big tech companies cap their AI sending.
It feels like maybe the wheels are starting to fall off the AI hype train. I expect complete collapse once people start figuring out that the numbers on all this don’t make sense. I’m looking for investment portfolios that will weather that storm. If you are reading this and have a similar curiosity, this is a great place to start.
https://portfoliocharts.com/2021/12/16/three-secret-ingredie...
hot take: token spend can be used a honey pot, especially when compared to what you deliver. spend accordingly!
My concern here is that they'll mix two things:
1) workforce reduction
2) AI spend (reduce tokenmaxing)
They'll expect fewer people to do more with even less, while "more" is continuously increasing.
When I say "more", I mean that the deluge that engineering teams deal with comes from two sources:
1) the business side of companies - marketing, sales, solutions teams, etc.
2) outside actors, mainly security threats
The first source can now move to generate work for engineering faster than ever. They expect the nerds to do what they're told and get the features out now. The more features, the better the product, right? The saving grace here is that they're bound by the same management concerns that engineering has. There's only so much money that they themselves can throw at generating more work for engineering teams, and that might also come under scrutiny from management, so that acts as a brake.
The second source has no such brake, especially not with security threats. Either there's good money to be made by holding company data hostage, or there's an endless supply of resources (read: nation-state resources) dedicated to the effort to attack the company's digital assets. And of course, they're using AI to enable this, just without the "but what about the shareholders!?" handwringing.
If you aren't very, very careful with your token cutting, you're going to put yourself at a disadvantage against that second group.