AWS / GCP / Azure aren't for individuals or small businesses. They won't tell you this anywhere, and they won't stop you from signing up - but they simply do not care one iota about users with anything less than $100k billing per month.
They treat big account owners like kings, they fly them out to Formula 1 events, they get 3 day workshops in swanky retreats, because a few k spent on this equals maybe millions of dollars.
If they respond to a small business quicker they don't get anything from it. They collect a bill that if it went missing they wouldn't notice.
I am not saying this is right - but people running small businesses on these platforms are operating under false pretenses.
tedivm
> AWS stomped on open source projects - despite the clear desire of projects like Elasticsearch, Redis, and MongoDB not to be cloned and monetized, AWS pushed ahead with OpenSearch, Valkey, and DocumentDB anyway, capturing the hosted-service money after those communities and companies had built the markets; the result was a wave of defensive licenses like SSPL, Elastic License, RSAL, and other source-available models designed less to stop ordinary users than to stop AWS from stripping open-source infrastructure for parts, owning the customer relationship.
This is completely backwards, at least with OpenSearch and Valkey. AWS didn't create the forks until after the upstream projects changed their license, so it's really weird to say that the forks "resulted" in the license changes when those forks where a response to the license changes. With Valkey in particular it was members of the former redis core development team that created Valkey.
show comments
Galanwe
These arguments against AWS are boring. 99% of the negative comments are along the line of "so i have a dead simple product, I dont know anything about AWS, I logged in and it was super complicated and it seemed pricey".
Well guess what, if you have a CRUD website and 100 users you're just not the target. Move on.
Some days ago I wanted to sketch a 3D model of my TV remote. I opened blender and what a mess of complicated windows and panes. I closed it immediatly. Do I think Blender is an over complicated mess? No, I just think I'm not the target. And I'm not offended to be too noob to use it.
show comments
raffraffraff
I always smile at posts like this. They're right and wrong at the same time. Systems should be "as simple as possible, but no simpler". And thinking that you can gloss over the detail is just going to create more hassle later on.
IAM is just complex. I can't think of any implementation of "users, groups, roles, policies, identity providers, oidc" that is truly simple.
I'm reminded of a guy I worked with, who fought against Kubernetes adoption because it was "too complex", only to slowly reinvent Kubernetes badly, adhoc, out of vault, consul, systemd, nomad, iscsi, ansible, jenkins, puppet, bash, spit, glue... making lots of mistakes along the way. You think you don't need to implement some feature until you do.
Another thing I'll say about AWS (having been the sole infra guy at a few startups) is that it's well within most people's abilities to learn it. And you can usually avoid the shitty stuff. You think lambdas stuck? Don't use them! You could use EKS, ECS or bare EC2.
show comments
sudosteph
I'm surprised by the author's hate towards DynamoDB. It's probably one of my favorite AWS Services. Great availability and no operational overhead. Cost was pretty minimal too each time I've used it, but you do need to spend some time architecting your data model up front, and that requires reading service docs and understanding it.
show comments
tailscaler2026
Anyone considering leaving AWS and thinking they'll transfer all their data for free [1], I've got news for you: It's a lie.
AWS takes as long as possible (for me it was a month) to respond to the initial DTO request,
then require you to submit a multi-page form answering a barrage of questions about why you're leaving, where you're going to, what services you used, and estimated data egress. A week or so later, if they approve the request, you're not allowed to begin DTO until 60 days after the approval.
By the time you can egress your data for "free", you've been stuck on AWS for 3-4 months since you first made the decision to leave.
I've transitioned between cloud services and self-hosting a few times:
1. Vercel Phase
My first project used Vercel. Since my project was Next.js, the experience was decent. But as my project gained some users, I found that even for projects under 100 users, I needed to pay $20 per month. Since my service didn't require high performance, this cost felt steep.
2. Self-host Phase (Hetzner + Coolify)
Later, I started setting up my own server with Hetzner and deploying with Coolify. Since Coolify is open-source and free, I only had to cover the cost of a VPS (even $5 a month was sufficient). I could deploy PostgreSQL instances and run a web server on it.
But later I discovered that even this way, I still had to spend a lot of effort maintaining PostgreSQL and Redis. Even though they were containerized with Docker, managing them was still troublesome. I needed to pass various system and environment variables between services, which was very tedious.
3. Cloudflare Phase
So later I switched to Cloudflare. With Cloudflare Workers, I can deploy fullstack applications and use D1 Database and Cloudflare KV to replace Redis. These features can be called directly within the Worker without needing to pass environment variables.
Plus, the local development experience is excellent and the pricing is very reasonable, so I've been using Cloudflare's entire suite ever since.
show comments
jfengel
I don't work in that area, so I only touch AWS once in a while for personal fun projects.
And every time it's a nightmare. I'm just banging out a server for my experimental card game, not setting up an new financial institution. Everything looks as if I'm preparing to scale to infinity tomorrow, with a staff of a thousand and a budget backed by VCs.
Fortunately there's Netlify and similar, who put a gloss on it so that I don't have to boil the ocean. I figure that one of these days I might actually be forced to learn IAM and VPNs and God only knows what else. Meantime, every time I touch it my eyes bug out.
show comments
aljgz
Years ago, I joined a company, took over a dev team and was asked to launch the product in 3 months.
They were using AWS, so I logged in the account to add a few more machines. Right there, in front of my eyes, were the signs of an adversarial, abusive relationship.
The UI to fire up a new machine did not show me the price. I had to look up the price in another table that did not have the specs.
I had to have the two tables open, cross check the specs and price.
If I had learned one thing from my past life was that if you see the signs of an abusive relationship, you have the option to walk out, and you don't, all that follows is your own fault.
Created a DigitalOcean account, moved everything over. Set up our CI/CDs to deploy there, and spent the next two months on the product, launching one month earlier than promised.
Some years before that I saw a video online where a person digs a hole near a river and puts a pipe connecting the river and the hole. The fishes push themselves hard in the pipe to get to their trap. Choosing the path of least resistance, and never backing off from a mistake: recipes to end up like those fishes. The video left a big impression on me.
show comments
mattbillenstein
I tend to use a few services on each cloud as possible so it's easy to switch between them; spinning up an Ubuntu VM that's identical on nearly every cloud is a superpower.
And, so if you keep it simple like this, it's not too complex and the costs are knowable - mostly VM hours and S3 for most of what I run.
But, the thing I've become increasingly disappointed with is simply the performance. The cpus are _slow_ - being forced to use EBS for a lot of things is _slow_ as hell; and starting/hydrating new VM volumes is super duper slow (have fun paying for fast launch).
So, for what you pay vs what you get, it's a huge difference, albeit very convenient.
Increasingly, I think about like racking stuff - like run most of your workload on dedicated hardware somewhere close to an AWS region and then burst into the cloud as needed and just use s3 in that region. Reduced cost, better performance for what matters, and you just pay for hands-on in the datacenter. Send them servers and just manage it all remotely.
exabrial
We invested in colocation 2022-2024 for non-prod (log aggregation, Gitlab, warehouse databases, analytics loads, etc). Didn't know what kind of savings we accidentally set ourselves up for. Investing 3 months DO and AWS Bills permanently cut our spending, and since then has never seen an increase. If these systems go offline, it's an inconvenience but not a show stopper.
We intentionally engineer prod so it doesn't rely on any system in the colo (so nothing like 'store our config in git and the apps pull it on startup' type party tricks).
With memory prices right now it's harder to recommend expanding colocation but it's something every company needs to do (eventually). Not every system you have has equal production value.
continueops_com
Had a similar one. They switched off Lambda and SNS because of a potential credential leak — none had actually leaked — and I was without service for 48 hours. Same flavour as the post: the provider's heuristic was probably right to fire, but you only find out which of your things were load-bearing once they're gone.
fxtentacle
"Maybe one day they will get around to unsuspending my account." is increasingly how support feels with all big cloud companies.
xmcp123
Something that has always bothered me an outsized amount is Elasticache.
I will bite the bullet and pay for RDS because it adds a lot of value - scalability, a reasonably optimized config, backups I don’t have to worry about.
But Elasticache is exploitatively priced with almost no value add.
It is slower, less optimized, less stable, and only supports one DB compared to a vanilla redis install with zero configuration.
There are some scalability improvements, but it’s extremely rare they’re even required because vanilla redis so wildly outperforms elasticache on a similar instance.
show comments
h1fra
To this day I still don't understand why people love AWS. It's overly complex, full of dark patterns, and not even that good compared to alternatives.
rembal
+1 on the IAM over engineering, though to AWS credit, I suspect it was evolved rather than design, and that's what you get when evolution has to maintain some level of backward compatibility (think humans still having to be able to lay eggs).
Another thing that happens occasionally for saas companies is AWS creating a copy of their product in a bit sus way - but it's not a technical problem, it's a business model problem.
show comments
djinn
AWS has been systematically hollowed out of technical staff since 2023. Either through mass layoffs or via 2 cycles of performance improvement plans. Often I find most skilled peers in presales or support are not with AWS whilst the ones with most ambiguous work history have been retained at promoted.
Use AWS at your own risk, Paul Vixie is not there to save you.
thomas_witt
It's hard to take the author seriously technically competent when reading "DynamoDB what a hot pile of garbage. I tried it and ended up with a $75USD bill by the end of the day."
Clearly there are many people - including me - who built highly scalable, available and near maintenance-free systems using DynamoDB for a ridiculously low cost.
I have no idea how you can actually burn more than $5 in development for DDB. If you don't make the effort to explore what a technology is built for and/or clearly didn't understand it, maybe you should holding back ranting about it. Unless you want to look like a fool.
Same goes for IAM. It's complex but still easily understandable to get the basics. Creating e.g. a rule where you can only read from a DynamoDB table but not delete entries or the whole table takes you under 10 clicks.
anymouse123456
Like OP, I was an AWS booster for many years (also a Heroku lover), but fell out of love about 10 years ago for the same reasons.
- It felt like far too much complexity just to do simple things.
- The obvious attempts to trap customers with slightly incompatible, higher level services felt gross
- The inability to run AWS trash on a dev machine had a MASSIVE hit on productivity
- Pricing didn't fall as fast as I felt it should (an obviously debatable position that reasonable, smart folks disagree with)
In my current company, we've been running basic SMB/tech startup functions on-prem (ACK! THE HORROR!) from ~6 basic computers (4 game machines and 2 nucs) for a few years now.
We just reconstituted the entire infra working part-time over about 2 weeks using Claude code and ansible.
It really doesn't make sense in this world to pay tens of thousands of dollars to rent a level of computation that can be purchased and managed for a tiny fraction of that money.
We're also seeing massive dividends paying out with this architecture because we have self-hosted gitea, along with a local workstation for our agents to run in, and now our agents have all of the context without us relying on Github or ingress/egress fees at all.
[edited for formatting only]
show comments
simonebrunozzi
Hmm... I was very early AWS, and it might even be "the guy from the US" that spoke at the first AWS Melbourne event was me, back in 2010.
I agree with few of the things that have annoyed Andrew Stuart, and brought him to leave. I disagree with a few. Let's pick one: DynamoDB was brilliant. I even knew one of the key engineers behind it, Stefano Stefani, as brilliant as he was hilariously funny as a person. It solved large scale problems beautifully, much better than SimpleDB or a combination of that and S3 would be able to do.
But I really disagree with one thing:
> And recently I went back to AWS. WHAT?!?!? WHY? You might ask. To get some research done. Do a few tests, get in and out.
I would never trust a person doing this, and would never hire him/her ever.
zmmmmm
I am curious the effect AI will have on these cloud offerings.
On the one hand, they bust through a bunch of the pain points of setting it up and configuring it. Especially if you are trying to do it using something like Terraform etc. So they make it more accessible.
But on the other hand, they equally reduce the pain of building all the premium part of the offering yourself. Why do I need AWS ECS / ALB / autoscaling etc etc services if I can get all that configured on bare metal just as easy now?
So in a different scenario all the lock in and premium services wither away and it all reduces to commodity compute - in some sense, where it should never have left. Initially experienced joy as the bitter battles I fought with Terraform became smooth prompts I issued to have Claude deal with all my problems. Life has gotten much better. But I'm now definitely moving into frustration because it's clear that AWS is mostly a middleman causing friction now across a whole set of infra that I could be managing directly. So I'm paying for the privilege of all this frustration. Why?
I don't know at the moment which way this will go, but I'm quite curious about it.
mavsman
This title sounds like an employment experience post: "returned to" and "left" both pretty strongly insinuate joining AWS and leaving employment, not simply using it as a customer.
imrozim
Setting up AWS for my startup felt like a full time job gave up and just used a supabse. AWS complexity is real even for similar stuff.
sqircles
The billing footguns are a major pain point for anyone that doesn't have the capital to just dump faith paired with a credit card into. This of course is not limited to AWS...
regularfry
Can report that exiting from Lambda to something more sane (like, say, a django task or api endpoint or something) is now pronounced "hey copilot, look in that directory and implement precisely the same functionality over here". Or thereabouts. A whole lot of things suddenly look a lot less locked in.
gchamonlive
> AWS Lambda - yeah I really bought the sell on this - "its scalable!!!!", and I ignored the slow startup times, the MASSIVE development complexity.
I don't know... Maybe I spent too much time studying how to tame AWS using IaC and gitops reproducible deployments, but AWS lambda seemed to me the most impressively simple and inexpensive product. Once I did an complete project, from end to end, designing the architecture and flow of multiple lambdas communicating with each other through SQS queues to search, extract and load info from geotiff files from S3 into a PostgreSQL database, and it was really straightforward.
If you leverage docker images for deployment and separate the interface for treating lambda requests from the core logic, it doesn't have much space for surprises.
If the author went with the cliché that lambda scalability can harm your budget, it wouldn't be original, but at least it would have been plausible, but complex? I don't know, maybe someone could present the case with more deails for why it's so complex.
amluto
My current favorites on AWS, in no particular order:
1. IAM and policies. I’m not convinced that anyone knows how IAM rules and policy rules interact. There’s a flow chart that appears to be incomplete. There is not obviously a complete enough spec that one could, say, write a test suite to confirm that the actual behavior follows the spec. LLMs, of course, don’t know either because the training data does not exist.
2. Utter nonsense pricing. The cost of listing an S3 bucket goes up by an order of magnitude if you set the default storage class to archive despite this having nothing whatsoever to do with the operation in question. (But GCS adds two orders of magnitude for the same offense.) Conclusion: NEVER EVER set your default storage class to an archive tier.
3. Boto. It’s an Unbelievable Piece Of Crap. It’s not a library at all — it’s a meta-library that generates itself at runtime because someone had fun doing that and because Python didn’t stop them. Python type checkers, of course, just give up. And Boto is, um, a community project that AWS claims not to care about. Which is, of course, why its maintainers refused to fix an interop bug with GCS (I fully documented the entire bug for them, and the fix would have been the removal of a bit of pointless code).
4. Egress pricing. And the way it multiplies if you use any advanced VPC features. Why on Earth is it cheaper to sent an object to S3 from my own machine than to send the same object to the same endpoint from within a different AWS region nearby?
5. Authentication. It’s so bad that they invented Identity Center to try to unsuck it. But if you use Identity Center you get logged out even while actively using the console, and you get a helpful link to the WRONG PLACE to sign back in. Because of course core AWS isn’t even aware that Identity Center exists.
I don’t even use AWS very much. I’m sure I would fall in love with more of it if I did.
finaard
> My business email system still does not work.
This is always the weird things in those rants. He's complaining that after 4 days his mails are offline.
Now I'm doing a mix of physical servers in rented rackspace, and rented servers - but even there I can have billing mixups where they deactivate servers for no good reason. And to get email working again the limiting factor would be the DNS TTL - new servers would be online somewhere else within hours of it going down. (And yes, I tested that just last year - one hoster threatened cutoff due to non-payment on a paid invoice, which prompted me to move the mail server just in case while getting this resolved).
show comments
bbbflgllglhlld
These fake open source rugpull companies deserve what Bezos did to them.
They made hosting their software hard, intentionally.
For example, prohibiting more than one node/replica, and being hostile to PRs/features that they consider their ”commercial offering”.
But the worst thing I’ve seen, for many software, is probably the hostility towards people who want to automate the software, for example putting the software in a container (10 years ago), then they refused to give support even if you had a valid paid contract.
aetherspawn
The one thing that’s good about Microsoft is the support. You can get someone on the phone in less than an hour and they can actually fix the thing.
That’s why it’s so far been hard to go past Outlook Plan 1 for (big scale) email hosting.
Completely agree about AWS, and we use Cloudflare now, but the jury is kind of out on whether CF is largely going the same way.
show comments
joefourier
> Cloud computing was an absolutely mind blowing revolution - suddenly your startup could run its own computer systems in minutes without need to install and run your own systems in a data center. This was an absolute game changer, and I really drank the AWS Kool Aid down to every last drop then I licked out the cup. I was all in on AWS in a big way.
Am I the only one who remembers that VPSes and dedicated hosting services were a thing before AWS came around? Yes you had to pay for a month at a time and scaling wasn’t as instant, but it wasn’t like the only option before cloud computing was having to drive to the datacentre and install your own server.
show comments
wg0
I think one big decision AWS could have taken earlier is that of declarative medium for cloud resources. Cloud formation is not human friendly as JSON or YAML. Problem with terraform has been that it had to keep track of the state separately which AWS already had (like what resources have been provisioned against a particular account number) in their databases and further more, I NEVER liked HCL, it never made sense to me.
Otherwise, some things that are good about AWS are as under:
1. IAM is I think good, logical and granular enough.
2. Separation of compute and storage in EC2 is very good.
3. S3 is amazing.
4. SQS is heavily underrated.
5. RDS is expensive but too good. I do not know how to go about 1 TB+ database size with daily backups without RDS. Similar ZFS setup with file system snapshots is complicated.
Not good things about AWS:
1. Super expensive. About 10 times. With zero support.
2. Current geopolitical environment would suggest getting off AWS if you are not a US company. The fascist idiots at the helm of affairs have lower IQ than the big void's average temperature in outer space.
EDIT: Typo + Formatting
morpheuskafka
> I am reminded why I left AWS and how I need to finish the job, get off AWS Workmail, move my domains from Route53 and never return.
Well, besides for the fact that the author's got suspended for no reason, WorkMail is being shut down March 2027 anyway. I recommend checking out Purelymail for a budget, batteries included option. Another option is to run your own server but have it use something like AWS SES to send externally, avoiding the IP reputation issue.
torginus
Imo lambdas are super cool, and the best way to have a no-headache fast-iteration time deployment service.
What most people realize, that you don't have to go microservice or fragment your code to a billion little repos, you could take a standard webserver, and move it to lambda, as long as you don't expect requests to be able to share on-server state.
show comments
dzonga
the A.I (LLM) merchants will tell you - that AI is now writing software (agentic coding they call it ) - yet one they can't bill you properly or have a broken billing mechanism.
their dashboards are trash & don't work - Google Cloud, AWS Console, Google Ads, Meta Ad manager
I won't even mention the hyped up LLM vendors.
but here we r - people being laid off due to A.I - money being funneled into Gigawatt datacenters
show comments
psanford
> If you're using AWS Lambda then you have to work to keep convincing yourself this is better than your own web servers. Keep convincing yourself that using AWS Lambda is not a horrible mistake.
lol ok. I have ~50 lambdas running in my personal aws account. Some of them are webservers running behind an api gateway or using a lambda function url to expose them to the internet. Some are running on a schedule, some are triggered from s3 events. The cost to run these for me is less than the cost of the cheapest vps (my total requests per month stay under the free tier limit). There is also zero maintenance I need to do for these functions (ok, this year I did have to find-replace al2 to al.2023 in my terraform config). I don't have to worry about making sure the os is patched for the latest vulnerabilities. And I don't have to worry about the specific hardware my code is running on at any time. Doing maintenance for old projects sucks. It is great to have servers I deployed years ago continue to chug along without me needing to think about it.
Now, all of my lambdas are written in Go and I suspect if I was using one of the manged runtime libraries I would find the language upgrades to be quite annoying. Go also helps quite a lot with cold start times.
Then again maybe I have just drank the koolaid. In my quest to use lambdas for as much as I can as cheaply as I can, I made a library[0] to use sqlite on top of s3 (not just readonly). It uses the sqlite session extension plus s3 compare-and-swap to allow you to write updates safely to s3, even if you have concurrent writers.
There was a time when AWS was truly innovative, but it’s long since transformed into Amazon’s cash cow and is behaving like such.
Innovation has ground to a halt of mostly just meh “hey us too” launches. Pricing and design patterns feel increasingly focused on locking you in. AWS folks tell me internally they talk a lot about making sure things are “sticky” with customers. The best engineering talent no longer wants to work there and it shows, especially in places like AI where AWS has just released wave after wave of discombobulated nonsense.
As a core “rent-a-server” concept with a few add on services there’s still a lot of utility, but AWS is gradually becoming a boring baseline utility with a ton of distracting half baked stuff jammed on top. Most companies I talk to are no longer focused on single cloud and increasingly are bringing a lot of workloads back on prem or in colos. Not everything, but for a lot of stuff that just makes more sense and is a heck of a lot cheaper.
The chips business in Annapurna is probably the most interesting thing and that plays to its strength of the boring low level infrastructure stuff. Nearly everything AWS tries to do beyond chips and rent-a-server plays is a hot mess.
AWS isn’t going away, but its future looks a lot less exciting and inspiring than the story that got us to this point.
andai
At last my quest to find the stooge has come to a bitter end!
I saw some 192 core instances on Vultr, but I haven't tried them yet. What are you doing with all them cores?
I often fantasized about spinning up hundreds of nodes for various projects that needed number crunching. Then realized "wait I can just rent one big box for an hour" haha. It's really cool that we can do that now.
show comments
mchl-mumo
I looked up DigitalOcean and it looks like a good alternative. What downsides should I be aware of, from those who have made the switch from AWS?
VerifiedReports
I left because of their shitty or nonexistent documentation AND absurd complexity.
After wrestling with their garbage for weeks, we started over and built a VPS from scratch. Development and deployment proceeded without a hitch after that. The only vestige remaining was S3.
I'm in the midst of a new project now, and I'm not even considering Amazon, even for S3 this time. I'm going to use an S3-compatible layer just in case, but I don't want to give Amazon a dime anymore.
Canada
And because we have fallen for the convenience we have lost a lot of alternatives.
The old rack/cage way was less convenient, but it came with a respect for our anatomy to run our stuff that I really miss.
show comments
nuker
> Complexity! Complexity!!
This guy needs supabase or heroku or similar.
recursive-call
I was in a team that used aws once for their quantum computers. we had $100 of api credits. while still trying to get the code to work, we somehow used all of them, and then it didn’t even alert us that we were out of credits and just spent an additional $100 that we didn’t have. I would not touch this system again with a 10 foot pole…
faangguyindia
Why do people even bother with cloud?
I’ve a couple of apps doing a few million a day. I am using Hetzner and before that used DigitalOcean. Mind you, for close to a decade.
People are unnecessarily complicating stuff, and these clouds can go very expensive very quickly.
Recently, I came across a company and they were spending $20k a month on GCP. I am like, are you kidding me, $20K for the kind of stuff you do??? It seems you do not understand how CPU, RAM and Disk work to plaster such "autoscaling hyper solutions" burning money in cloud.
I moved their stuff out of the GCP managed solution and ended up with a $200-400 per month bill. The CEO can still not believe how it's even possible.
I suggested them move to Dedicated servers but they didn't want it, they said they must show they are on Hyperscaling cloud.
OK fine, we'll stay in Hyperscaler but not use any of their service other than VMs.
They racked up a ton of bills by using cloud monitoring, Datastore, and autoscalers (with no proper tuning), Kubernetes.
I replaced all of it with Prometheus, Grafana, Loki, and most stuff from Datastore to Postgres and Mongo with replicas. I added Redis.
I implemented a custom scaler where you can scale off of app metrics, not by just using a random peg on CPU.
I implement hot data reload by packing the data updates in gzip file, uploading to GCS and pulling from autoscaled units. Moved the stuff to Spot VMs.
The complexity of stuff in cloud is high for nothing.
show comments
geoffbp
Slightly different but related topic - for people who work with people vibe coding, what is the easiest way to allow that for non tech users (and reducing risk)? AWS or something like vercel? Coolify?
show comments
nicman23
i mostly used the ec2 instances and the workflow to make a simple gpu instance from the terminal (terraform) is terrible in contrast to ie gcp.
from the UI it is even worse
also how it is so slowwwwwww
rglover
You can accomplish a lot by just having a basic knowledge of Linux sysadmin. I was clueless and then learned some systemd-and-curl-fu. Will never forget the "holy sh*t, this is deceptively simple" moment. A bit more research and I found that beyond convenience and specialty APIs, you really just don't need a lot of this stuff to run a healthy system (since reducing absolute cloud dependence, my reliability has gone through the roof).
show comments
hand2note
Almost everything is true about Azure as well, especially obscure pricing and complexity in absolutely everything.
dakiol
When are we gonna start hearing the same stories about Anthropic/Openai/etc? The whole AI thing kinda smells like the early days of AWS: everyone was getting onboarded, but later realized they'd built up a pretty big dependency that's not easy to shake
cantalopes
sorry to hijack this but since we're talking the AWS egress exorbitant prices, could someone recommend a reliable production-tested s3 service that will not ruin me financially? i have been with hetzner and while their vps server are great, they are absolutely terrible at s3. random downtimes, incomplete uploads/downloads, capacity issues, terrible api key management, etc - but the failure rate is really high
mt_
The well architected frameworks tells you to have separate accounts, your fault that you "tested" in a production environment.
https://imgur.com/a/Smal9fL
eluded7
I'd tend to agree with the author. If forced to choose a cloud platform though (and that often is the case) then AWS is probably the best of the bunch in terms of reliability. Have heard and experienced some real horror stories with Azure & GCP by comparison.
show comments
hhh
IAM is my favorite part of AWS.
sbinnee
I also tried. Only service I use is s3 for personal backup. I pay around 15 cents per month.
alde
The set of core services on AWS remains amazing: EC2, S3, IAM, EKS, Route53, RDS etc.
AWS IAM is extremely well designed when you compare it with the spaghetti monster IAM systems of other clouds.
Every time I try the new cool thing supposed to replace these services on some other provider - I understand how mature and polished the AWS ones are.
With that said, the rest 90% of AWS services like WorkMail, Cognito, API Gateway, are absolute hot garbage which no good meaning AWS expert will touch with a 10 meter stick.
show comments
stuaxo
The whole model of slicing the app into so many pieces (proprietary wrappers it seems like) to make more opportunities for billing is terrible.
znpy
> Of course I do not pay for premium support, so I have to wait the 24 hours that they said it would take them to reply. It's 3 days and AWS support has not replied.
The writing has been on the wall for a few years now, and this is particularly evident to those thar have worked at AWS: Amazon is in its day-2 era.
Amazon being in its day-2 era means that most of what has been written in the past twenty years about Amazon is bot valid anymore.
“Customer obsession” is literally their first leadership principle, and stellar support was their defining characteristic.
show comments
stiray
Amazon is bad at their blocking of accounts. They blocked mine for no reason (want me to call some USA phone number which I wont) a few years back and I was writing down everything that I would buy at their store, but I bought it elsewhere.
They have lost 3785.90 euros in sales due to their idiotic anti-user war.
Not to mention of all bad reputation that I gave them.
cynicalsecurity
Preach, brother.
maptime
How lambda is as bad as it is I have no clue. Not a lover of azure, but azure functions is such a nicer experience
dangoodmanUT
GCP would be perfect if they didn't have a history of randomly dropping quotas on startups, causing them downtime
show comments
mlhpdx
I’m not sure how someone can be an “AWS Fanboy”, drink in all the promise, and think IAM is evil. As far as I can tell it is the one glorious thing that separates AWS from others. IAM is the core that makes it sane.
show comments
stevepotter
I was such a fan of it that I ended up working there for 4 years. Now I avoid it and encourage others to do the same.
AWS used to have a nifty tool called "policy analyzer" or something that monitored for permissions used by a role so you could scope it down. The other day I had the need for it and when I went to use it, found out they charge something like $9/resource. So I would pay $45/month for metadata monitoring on just 5 things? Nuts. If they knew how to build truly delightful products, they would make something like a role that starts with broad permissions and automatically scopes itself down after some point. And it would be free or at least really cheap.
DDB is hardly a database. The only reason I can think of to use it is for massive amounts of data whose schema and query patterns are guaranteed to almost never change, which is very rare. Need to sort data on a field? Then you have to create a 'secondary index', which is a copy of the table that they charge you for and that is not strongly consistent. Schema change? Good luck with that. And don't you dare ask to use a nice ORM library. But hey it's serverless.
Here's a good one: you stop an EC2 instance and its volume keeps running and you pay for that. If you detach the volume, you still pay. There is no way to 'archive' an instance. And the only way I found out about that was I got hit with a big bill for those volumes with the charge labeled 'EC2 - Other' lol. Not very 'customer obsessed' to me.
My gripes are clearly not important to them because this is old stuff. So all I can do is go somewhere else, which is fine with me
show comments
high_byte
similar thing happened to me.
I'm not a heavy aws user but wanted to setup some s3 buckets few days ago but my account was suspended for the same reason
but unlike OP I just accepted this fate and moved away from aws :)
aeagentic
But is there a better one with same IaaC and API completeness?
_wire_
I love you baby, I need you! I'd never cheat on you! Come back!
Hey good lookin'
show comments
bironran
GCP has it's own share of issues.
...
I was writing a long vent about GCP but the mix of issues we had, just in the last few weeks, was too identifying and I don't want to sour an already tremulous relationship, as much as I'd like to spill it all here.
Let's just sum it up with resource crunch and degraded services because, apparently, when one customer signs a $200B deal [1] all the "just a few $10M" get thrown to the wayside.
AWS is also affected. Time to go to Azure? I never thought I'd say those words.
... skills issue. LOL. Good luck to making these tests easier on GCP / Azure. Or try to set up equivalent HW server without pulling your hair. Complexity is large part of why we, IT people, are paid those money.
(Should've created *new AWS account* for those tests. AWS's automated security algorithms almost certainly flagged this abnormal behavior as a "suspected security breach" to protect the (long time dormant) user from potentially devastating unauthorized charges)
tonymet
Every single complaint has a simple fix: “just use EC2”
show comments
raverbashing
> IAM - the hideously complex auth and access rules system - this was invented by Lucifer sitting on his burning throne in the ninth level of Hell as the worst possible torment for those who have been sent below for using AWS.
Perfect explanation - no notes
I don't think I remember anything so over-engineered and confusing in recent times (probably SELinux now that I think of it).
And I understand - we kinda need the complexity for what they intend to do but they do need a Come To Jesus moment here to make the Insane Asylum Machine make a bit more sense for mortals
2nd most annoying thing? Boto3 lib, where conventions don't matter and Pythonic is just a suggestion and the thing works more like a REST wrapper than anything else over a not-great API (please why tell me there's an S3 API and an S3Obj API)
atemerev
As if there's any alternative. Azure? That mess of everything smashed on top of each other that looks like it was vibecoded in a few month by hundreds of people at once, except that they looked like this from the very beginning where there was no AI? The one that makes you fill docx forms to enable quotas for some services? Or Google Cloud, which _looks_ like it might be simpler, but it has permissions for permissions to enable permissions, and endless micromanagement? I am trying things, but I always return to AWS :(
calmbonsai
The AWS UI should be, effectively, read-only for any infrastructure aside from setting up some initial roles and perms to manage all of it through an IAC system.
Put more bluntly, if you're using the AWS Console to spin-up/spin-down service instances you're doing it wrong.
show comments
fafa09
Interesting take on the migration.
xrd
There is one fortunate result that will come from the SaaSpocalypse combining with Mythos (color me skeptical but let's assume it is as powerful as Anthropic tells CIOs).
If anyone can clone any SaaS, then there will be millions of SaaS that offer all the features you need.
How will you choose?
AWS and Microsoft (and all the big clouds) make it easy for their customers to get hacked, and Mythos makes it more likely the cadence will only intensify.
But, if I vibe code a hosting service which is pure rust and doesn't use any external libraries and never open sources my code, my attack surface is much smaller and I only have three customers anyway.
Hackers are lazy and will go for the pond where the most fish live. AWS will always have a lot of marks and a lot of holes.
AWS will be expensive because you are paying the tax they have to add to fend off the hordes. It'll be an intelligent choice to avoid working with Rome and find a little village in Bergen.
lowbloodsugar
When it comes to email: “Don’t shit where you eat” is the closest analogy I can think of. Have your email somewhere else, it on any service that might decide to lock your account for any reason. Have your domain ownership somewhere else. And have already, or have a plan already, to move that somewhere if for some reason your email provider gets pissed at you.
h4kunamata
AWS AIM is hot garbage, GCP might not be the coolest kid of the block but its AIM rocks.
AWS CLI??? Holy guacamole, what a mess. AWS CLI looks what is now the digital identification to get the basics done.
While GCP CLI is like "sure, here"!
show comments
fnord77
Are the other two big providers any different?
fHr
>works for AWS
>quits the BS
>needs more money
>sells himself as a meat puppeteer once again to AWS
>big bs corpo is still the same surprisedpikatchu.jpeg
ok
tootie
This belies the fact that AWS is so far in the lead in cloud market share and even host so much of Anthropic's business. If you dabble, it's confusing. If you're an enterprise with a lot of expertise then it's indispensable.
MagicMoonlight
These complaints are very weak.
Lambda is incredibly simple to use, it just runs a function for you.
Not sure how you could burn so much with dynamodb. It’s serverless and incredibly cheap. Must have been doing something insane like a huge dataset where you scan through it over and over.
Being salty that Gary couldn’t sell enough of his paid service and AWS is competing with it isn’t a meaningful complaint. I want something in AWS, not on Gary’s servers.
show comments
waterTanuki
anyone building with AWS or any cloud provider should be setting up a forked pipeline for their data production: one goes to the cloud provider DB used for production, the other is a local-on prem DB you regularly backup and always have on hand should you need to leave the cloud.
AIorNot
Yup -like the honesty
Why don't cloud services have reviews like amazon products.- I’m so tired of enterprise sales speak in corporate docs- just cut the BS and do straight talk
If you haven’t used a service you shouldn’t have to search reddit for dev experience on it
fafa09
same pain here
themafia
> Somewhere in the depths of AWS some sort of security alarm had been triggered probably by the fact that my mostly dormant account suddenly started doing stuff with an expensive computer.
You mean the alarm that shows up with a notification bell in your console? Why not just post that?
> I am dreading having to "request quota" to be allowed to do that.
Why? It works fine. I've done it several times.
> IAM - the hideously complex auth and access rules system - this was invented by Lucifer sitting on his burning throne in the ninth level of Hell as the worst possible torment for those who have been sent below for using AWS.
It's literally a JSON policy document.
> - once I noticed the complexity of IAM I could not unsee the complexity everywhere in AWS.
All our policy actions are scripted at this point. You specify what functions the lambda calls and it builds the policy for you, sends it to IAM, and attaches it to the lambda.
Everytime I see someone complain about AWS I'm left wondering "did you read _any_ of the documentation? If you just want a linux server then run that but if you want out of the hassle of managing one then you need to learn just a _handful_ of new tricks."
If half the effort of complaining about AWS was spent reading documentation then most of these articles would never be published.
AWS / GCP / Azure aren't for individuals or small businesses. They won't tell you this anywhere, and they won't stop you from signing up - but they simply do not care one iota about users with anything less than $100k billing per month.
They treat big account owners like kings, they fly them out to Formula 1 events, they get 3 day workshops in swanky retreats, because a few k spent on this equals maybe millions of dollars.
If they respond to a small business quicker they don't get anything from it. They collect a bill that if it went missing they wouldn't notice.
I am not saying this is right - but people running small businesses on these platforms are operating under false pretenses.
> AWS stomped on open source projects - despite the clear desire of projects like Elasticsearch, Redis, and MongoDB not to be cloned and monetized, AWS pushed ahead with OpenSearch, Valkey, and DocumentDB anyway, capturing the hosted-service money after those communities and companies had built the markets; the result was a wave of defensive licenses like SSPL, Elastic License, RSAL, and other source-available models designed less to stop ordinary users than to stop AWS from stripping open-source infrastructure for parts, owning the customer relationship.
This is completely backwards, at least with OpenSearch and Valkey. AWS didn't create the forks until after the upstream projects changed their license, so it's really weird to say that the forks "resulted" in the license changes when those forks where a response to the license changes. With Valkey in particular it was members of the former redis core development team that created Valkey.
These arguments against AWS are boring. 99% of the negative comments are along the line of "so i have a dead simple product, I dont know anything about AWS, I logged in and it was super complicated and it seemed pricey".
Well guess what, if you have a CRUD website and 100 users you're just not the target. Move on.
Some days ago I wanted to sketch a 3D model of my TV remote. I opened blender and what a mess of complicated windows and panes. I closed it immediatly. Do I think Blender is an over complicated mess? No, I just think I'm not the target. And I'm not offended to be too noob to use it.
I always smile at posts like this. They're right and wrong at the same time. Systems should be "as simple as possible, but no simpler". And thinking that you can gloss over the detail is just going to create more hassle later on.
IAM is just complex. I can't think of any implementation of "users, groups, roles, policies, identity providers, oidc" that is truly simple.
I'm reminded of a guy I worked with, who fought against Kubernetes adoption because it was "too complex", only to slowly reinvent Kubernetes badly, adhoc, out of vault, consul, systemd, nomad, iscsi, ansible, jenkins, puppet, bash, spit, glue... making lots of mistakes along the way. You think you don't need to implement some feature until you do.
Another thing I'll say about AWS (having been the sole infra guy at a few startups) is that it's well within most people's abilities to learn it. And you can usually avoid the shitty stuff. You think lambdas stuck? Don't use them! You could use EKS, ECS or bare EC2.
I'm surprised by the author's hate towards DynamoDB. It's probably one of my favorite AWS Services. Great availability and no operational overhead. Cost was pretty minimal too each time I've used it, but you do need to spend some time architecting your data model up front, and that requires reading service docs and understanding it.
Anyone considering leaving AWS and thinking they'll transfer all their data for free [1], I've got news for you: It's a lie.
AWS takes as long as possible (for me it was a month) to respond to the initial DTO request, then require you to submit a multi-page form answering a barrage of questions about why you're leaving, where you're going to, what services you used, and estimated data egress. A week or so later, if they approve the request, you're not allowed to begin DTO until 60 days after the approval.
By the time you can egress your data for "free", you've been stuck on AWS for 3-4 months since you first made the decision to leave.
[1] https://aws.amazon.com/blogs/aws/free-data-transfer-out-to-i...
I've transitioned between cloud services and self-hosting a few times:
1. Vercel Phase My first project used Vercel. Since my project was Next.js, the experience was decent. But as my project gained some users, I found that even for projects under 100 users, I needed to pay $20 per month. Since my service didn't require high performance, this cost felt steep.
2. Self-host Phase (Hetzner + Coolify) Later, I started setting up my own server with Hetzner and deploying with Coolify. Since Coolify is open-source and free, I only had to cover the cost of a VPS (even $5 a month was sufficient). I could deploy PostgreSQL instances and run a web server on it. But later I discovered that even this way, I still had to spend a lot of effort maintaining PostgreSQL and Redis. Even though they were containerized with Docker, managing them was still troublesome. I needed to pass various system and environment variables between services, which was very tedious.
3. Cloudflare Phase So later I switched to Cloudflare. With Cloudflare Workers, I can deploy fullstack applications and use D1 Database and Cloudflare KV to replace Redis. These features can be called directly within the Worker without needing to pass environment variables.
Plus, the local development experience is excellent and the pricing is very reasonable, so I've been using Cloudflare's entire suite ever since.
I don't work in that area, so I only touch AWS once in a while for personal fun projects.
And every time it's a nightmare. I'm just banging out a server for my experimental card game, not setting up an new financial institution. Everything looks as if I'm preparing to scale to infinity tomorrow, with a staff of a thousand and a budget backed by VCs.
Fortunately there's Netlify and similar, who put a gloss on it so that I don't have to boil the ocean. I figure that one of these days I might actually be forced to learn IAM and VPNs and God only knows what else. Meantime, every time I touch it my eyes bug out.
Years ago, I joined a company, took over a dev team and was asked to launch the product in 3 months.
They were using AWS, so I logged in the account to add a few more machines. Right there, in front of my eyes, were the signs of an adversarial, abusive relationship.
The UI to fire up a new machine did not show me the price. I had to look up the price in another table that did not have the specs.
I had to have the two tables open, cross check the specs and price.
If I had learned one thing from my past life was that if you see the signs of an abusive relationship, you have the option to walk out, and you don't, all that follows is your own fault.
Created a DigitalOcean account, moved everything over. Set up our CI/CDs to deploy there, and spent the next two months on the product, launching one month earlier than promised.
Some years before that I saw a video online where a person digs a hole near a river and puts a pipe connecting the river and the hole. The fishes push themselves hard in the pipe to get to their trap. Choosing the path of least resistance, and never backing off from a mistake: recipes to end up like those fishes. The video left a big impression on me.
I tend to use a few services on each cloud as possible so it's easy to switch between them; spinning up an Ubuntu VM that's identical on nearly every cloud is a superpower.
And, so if you keep it simple like this, it's not too complex and the costs are knowable - mostly VM hours and S3 for most of what I run.
But, the thing I've become increasingly disappointed with is simply the performance. The cpus are _slow_ - being forced to use EBS for a lot of things is _slow_ as hell; and starting/hydrating new VM volumes is super duper slow (have fun paying for fast launch).
So, for what you pay vs what you get, it's a huge difference, albeit very convenient.
Increasingly, I think about like racking stuff - like run most of your workload on dedicated hardware somewhere close to an AWS region and then burst into the cloud as needed and just use s3 in that region. Reduced cost, better performance for what matters, and you just pay for hands-on in the datacenter. Send them servers and just manage it all remotely.
We invested in colocation 2022-2024 for non-prod (log aggregation, Gitlab, warehouse databases, analytics loads, etc). Didn't know what kind of savings we accidentally set ourselves up for. Investing 3 months DO and AWS Bills permanently cut our spending, and since then has never seen an increase. If these systems go offline, it's an inconvenience but not a show stopper.
We intentionally engineer prod so it doesn't rely on any system in the colo (so nothing like 'store our config in git and the apps pull it on startup' type party tricks).
With memory prices right now it's harder to recommend expanding colocation but it's something every company needs to do (eventually). Not every system you have has equal production value.
Had a similar one. They switched off Lambda and SNS because of a potential credential leak — none had actually leaked — and I was without service for 48 hours. Same flavour as the post: the provider's heuristic was probably right to fire, but you only find out which of your things were load-bearing once they're gone.
"Maybe one day they will get around to unsuspending my account." is increasingly how support feels with all big cloud companies.
Something that has always bothered me an outsized amount is Elasticache.
I will bite the bullet and pay for RDS because it adds a lot of value - scalability, a reasonably optimized config, backups I don’t have to worry about.
But Elasticache is exploitatively priced with almost no value add.
It is slower, less optimized, less stable, and only supports one DB compared to a vanilla redis install with zero configuration.
There are some scalability improvements, but it’s extremely rare they’re even required because vanilla redis so wildly outperforms elasticache on a similar instance.
To this day I still don't understand why people love AWS. It's overly complex, full of dark patterns, and not even that good compared to alternatives.
+1 on the IAM over engineering, though to AWS credit, I suspect it was evolved rather than design, and that's what you get when evolution has to maintain some level of backward compatibility (think humans still having to be able to lay eggs). Another thing that happens occasionally for saas companies is AWS creating a copy of their product in a bit sus way - but it's not a technical problem, it's a business model problem.
AWS has been systematically hollowed out of technical staff since 2023. Either through mass layoffs or via 2 cycles of performance improvement plans. Often I find most skilled peers in presales or support are not with AWS whilst the ones with most ambiguous work history have been retained at promoted.
Use AWS at your own risk, Paul Vixie is not there to save you.
It's hard to take the author seriously technically competent when reading "DynamoDB what a hot pile of garbage. I tried it and ended up with a $75USD bill by the end of the day."
Clearly there are many people - including me - who built highly scalable, available and near maintenance-free systems using DynamoDB for a ridiculously low cost.
I have no idea how you can actually burn more than $5 in development for DDB. If you don't make the effort to explore what a technology is built for and/or clearly didn't understand it, maybe you should holding back ranting about it. Unless you want to look like a fool.
Same goes for IAM. It's complex but still easily understandable to get the basics. Creating e.g. a rule where you can only read from a DynamoDB table but not delete entries or the whole table takes you under 10 clicks.
Like OP, I was an AWS booster for many years (also a Heroku lover), but fell out of love about 10 years ago for the same reasons.
- It felt like far too much complexity just to do simple things.
- The obvious attempts to trap customers with slightly incompatible, higher level services felt gross
- The inability to run AWS trash on a dev machine had a MASSIVE hit on productivity
- Pricing didn't fall as fast as I felt it should (an obviously debatable position that reasonable, smart folks disagree with)
In my current company, we've been running basic SMB/tech startup functions on-prem (ACK! THE HORROR!) from ~6 basic computers (4 game machines and 2 nucs) for a few years now.
We just reconstituted the entire infra working part-time over about 2 weeks using Claude code and ansible.
It really doesn't make sense in this world to pay tens of thousands of dollars to rent a level of computation that can be purchased and managed for a tiny fraction of that money.
We're also seeing massive dividends paying out with this architecture because we have self-hosted gitea, along with a local workstation for our agents to run in, and now our agents have all of the context without us relying on Github or ingress/egress fees at all.
[edited for formatting only]
Hmm... I was very early AWS, and it might even be "the guy from the US" that spoke at the first AWS Melbourne event was me, back in 2010.
I agree with few of the things that have annoyed Andrew Stuart, and brought him to leave. I disagree with a few. Let's pick one: DynamoDB was brilliant. I even knew one of the key engineers behind it, Stefano Stefani, as brilliant as he was hilariously funny as a person. It solved large scale problems beautifully, much better than SimpleDB or a combination of that and S3 would be able to do.
But I really disagree with one thing:
> And recently I went back to AWS. WHAT?!?!? WHY? You might ask. To get some research done. Do a few tests, get in and out.
I would never trust a person doing this, and would never hire him/her ever.
I am curious the effect AI will have on these cloud offerings.
On the one hand, they bust through a bunch of the pain points of setting it up and configuring it. Especially if you are trying to do it using something like Terraform etc. So they make it more accessible.
But on the other hand, they equally reduce the pain of building all the premium part of the offering yourself. Why do I need AWS ECS / ALB / autoscaling etc etc services if I can get all that configured on bare metal just as easy now?
So in a different scenario all the lock in and premium services wither away and it all reduces to commodity compute - in some sense, where it should never have left. Initially experienced joy as the bitter battles I fought with Terraform became smooth prompts I issued to have Claude deal with all my problems. Life has gotten much better. But I'm now definitely moving into frustration because it's clear that AWS is mostly a middleman causing friction now across a whole set of infra that I could be managing directly. So I'm paying for the privilege of all this frustration. Why?
I don't know at the moment which way this will go, but I'm quite curious about it.
This title sounds like an employment experience post: "returned to" and "left" both pretty strongly insinuate joining AWS and leaving employment, not simply using it as a customer.
Setting up AWS for my startup felt like a full time job gave up and just used a supabse. AWS complexity is real even for similar stuff.
The billing footguns are a major pain point for anyone that doesn't have the capital to just dump faith paired with a credit card into. This of course is not limited to AWS...
Can report that exiting from Lambda to something more sane (like, say, a django task or api endpoint or something) is now pronounced "hey copilot, look in that directory and implement precisely the same functionality over here". Or thereabouts. A whole lot of things suddenly look a lot less locked in.
> AWS Lambda - yeah I really bought the sell on this - "its scalable!!!!", and I ignored the slow startup times, the MASSIVE development complexity.
I don't know... Maybe I spent too much time studying how to tame AWS using IaC and gitops reproducible deployments, but AWS lambda seemed to me the most impressively simple and inexpensive product. Once I did an complete project, from end to end, designing the architecture and flow of multiple lambdas communicating with each other through SQS queues to search, extract and load info from geotiff files from S3 into a PostgreSQL database, and it was really straightforward.
If you leverage docker images for deployment and separate the interface for treating lambda requests from the core logic, it doesn't have much space for surprises.
If the author went with the cliché that lambda scalability can harm your budget, it wouldn't be original, but at least it would have been plausible, but complex? I don't know, maybe someone could present the case with more deails for why it's so complex.
My current favorites on AWS, in no particular order:
1. IAM and policies. I’m not convinced that anyone knows how IAM rules and policy rules interact. There’s a flow chart that appears to be incomplete. There is not obviously a complete enough spec that one could, say, write a test suite to confirm that the actual behavior follows the spec. LLMs, of course, don’t know either because the training data does not exist.
2. Utter nonsense pricing. The cost of listing an S3 bucket goes up by an order of magnitude if you set the default storage class to archive despite this having nothing whatsoever to do with the operation in question. (But GCS adds two orders of magnitude for the same offense.) Conclusion: NEVER EVER set your default storage class to an archive tier.
3. Boto. It’s an Unbelievable Piece Of Crap. It’s not a library at all — it’s a meta-library that generates itself at runtime because someone had fun doing that and because Python didn’t stop them. Python type checkers, of course, just give up. And Boto is, um, a community project that AWS claims not to care about. Which is, of course, why its maintainers refused to fix an interop bug with GCS (I fully documented the entire bug for them, and the fix would have been the removal of a bit of pointless code).
4. Egress pricing. And the way it multiplies if you use any advanced VPC features. Why on Earth is it cheaper to sent an object to S3 from my own machine than to send the same object to the same endpoint from within a different AWS region nearby?
5. Authentication. It’s so bad that they invented Identity Center to try to unsuck it. But if you use Identity Center you get logged out even while actively using the console, and you get a helpful link to the WRONG PLACE to sign back in. Because of course core AWS isn’t even aware that Identity Center exists.
I don’t even use AWS very much. I’m sure I would fall in love with more of it if I did.
> My business email system still does not work.
This is always the weird things in those rants. He's complaining that after 4 days his mails are offline.
Now I'm doing a mix of physical servers in rented rackspace, and rented servers - but even there I can have billing mixups where they deactivate servers for no good reason. And to get email working again the limiting factor would be the DNS TTL - new servers would be online somewhere else within hours of it going down. (And yes, I tested that just last year - one hoster threatened cutoff due to non-payment on a paid invoice, which prompted me to move the mail server just in case while getting this resolved).
These fake open source rugpull companies deserve what Bezos did to them.
They made hosting their software hard, intentionally.
For example, prohibiting more than one node/replica, and being hostile to PRs/features that they consider their ”commercial offering”.
But the worst thing I’ve seen, for many software, is probably the hostility towards people who want to automate the software, for example putting the software in a container (10 years ago), then they refused to give support even if you had a valid paid contract.
The one thing that’s good about Microsoft is the support. You can get someone on the phone in less than an hour and they can actually fix the thing.
That’s why it’s so far been hard to go past Outlook Plan 1 for (big scale) email hosting.
Completely agree about AWS, and we use Cloudflare now, but the jury is kind of out on whether CF is largely going the same way.
> Cloud computing was an absolutely mind blowing revolution - suddenly your startup could run its own computer systems in minutes without need to install and run your own systems in a data center. This was an absolute game changer, and I really drank the AWS Kool Aid down to every last drop then I licked out the cup. I was all in on AWS in a big way.
Am I the only one who remembers that VPSes and dedicated hosting services were a thing before AWS came around? Yes you had to pay for a month at a time and scaling wasn’t as instant, but it wasn’t like the only option before cloud computing was having to drive to the datacentre and install your own server.
I think one big decision AWS could have taken earlier is that of declarative medium for cloud resources. Cloud formation is not human friendly as JSON or YAML. Problem with terraform has been that it had to keep track of the state separately which AWS already had (like what resources have been provisioned against a particular account number) in their databases and further more, I NEVER liked HCL, it never made sense to me.
Otherwise, some things that are good about AWS are as under:
1. IAM is I think good, logical and granular enough.
2. Separation of compute and storage in EC2 is very good.
3. S3 is amazing.
4. SQS is heavily underrated.
5. RDS is expensive but too good. I do not know how to go about 1 TB+ database size with daily backups without RDS. Similar ZFS setup with file system snapshots is complicated.
Not good things about AWS:
1. Super expensive. About 10 times. With zero support.
2. Current geopolitical environment would suggest getting off AWS if you are not a US company. The fascist idiots at the helm of affairs have lower IQ than the big void's average temperature in outer space.
EDIT: Typo + Formatting
> I am reminded why I left AWS and how I need to finish the job, get off AWS Workmail, move my domains from Route53 and never return.
Well, besides for the fact that the author's got suspended for no reason, WorkMail is being shut down March 2027 anyway. I recommend checking out Purelymail for a budget, batteries included option. Another option is to run your own server but have it use something like AWS SES to send externally, avoiding the IP reputation issue.
Imo lambdas are super cool, and the best way to have a no-headache fast-iteration time deployment service.
What most people realize, that you don't have to go microservice or fragment your code to a billion little repos, you could take a standard webserver, and move it to lambda, as long as you don't expect requests to be able to share on-server state.
the A.I (LLM) merchants will tell you - that AI is now writing software (agentic coding they call it ) - yet one they can't bill you properly or have a broken billing mechanism.
their dashboards are trash & don't work - Google Cloud, AWS Console, Google Ads, Meta Ad manager
I won't even mention the hyped up LLM vendors.
but here we r - people being laid off due to A.I - money being funneled into Gigawatt datacenters
> If you're using AWS Lambda then you have to work to keep convincing yourself this is better than your own web servers. Keep convincing yourself that using AWS Lambda is not a horrible mistake.
lol ok. I have ~50 lambdas running in my personal aws account. Some of them are webservers running behind an api gateway or using a lambda function url to expose them to the internet. Some are running on a schedule, some are triggered from s3 events. The cost to run these for me is less than the cost of the cheapest vps (my total requests per month stay under the free tier limit). There is also zero maintenance I need to do for these functions (ok, this year I did have to find-replace al2 to al.2023 in my terraform config). I don't have to worry about making sure the os is patched for the latest vulnerabilities. And I don't have to worry about the specific hardware my code is running on at any time. Doing maintenance for old projects sucks. It is great to have servers I deployed years ago continue to chug along without me needing to think about it.
Now, all of my lambdas are written in Go and I suspect if I was using one of the manged runtime libraries I would find the language upgrades to be quite annoying. Go also helps quite a lot with cold start times.
Then again maybe I have just drank the koolaid. In my quest to use lambdas for as much as I can as cheaply as I can, I made a library[0] to use sqlite on top of s3 (not just readonly). It uses the sqlite session extension plus s3 compare-and-swap to allow you to write updates safely to s3, even if you have concurrent writers.
[0]: https://github.com/psanford/s3db
There was a time when AWS was truly innovative, but it’s long since transformed into Amazon’s cash cow and is behaving like such.
Innovation has ground to a halt of mostly just meh “hey us too” launches. Pricing and design patterns feel increasingly focused on locking you in. AWS folks tell me internally they talk a lot about making sure things are “sticky” with customers. The best engineering talent no longer wants to work there and it shows, especially in places like AI where AWS has just released wave after wave of discombobulated nonsense.
As a core “rent-a-server” concept with a few add on services there’s still a lot of utility, but AWS is gradually becoming a boring baseline utility with a ton of distracting half baked stuff jammed on top. Most companies I talk to are no longer focused on single cloud and increasingly are bringing a lot of workloads back on prem or in colos. Not everything, but for a lot of stuff that just makes more sense and is a heck of a lot cheaper.
The chips business in Annapurna is probably the most interesting thing and that plays to its strength of the boring low level infrastructure stuff. Nearly everything AWS tries to do beyond chips and rent-a-server plays is a hot mess.
AWS isn’t going away, but its future looks a lot less exciting and inspiring than the story that got us to this point.
At last my quest to find the stooge has come to a bitter end!
I saw some 192 core instances on Vultr, but I haven't tried them yet. What are you doing with all them cores?
I often fantasized about spinning up hundreds of nodes for various projects that needed number crunching. Then realized "wait I can just rent one big box for an hour" haha. It's really cool that we can do that now.
I looked up DigitalOcean and it looks like a good alternative. What downsides should I be aware of, from those who have made the switch from AWS?
I left because of their shitty or nonexistent documentation AND absurd complexity.
After wrestling with their garbage for weeks, we started over and built a VPS from scratch. Development and deployment proceeded without a hitch after that. The only vestige remaining was S3.
I'm in the midst of a new project now, and I'm not even considering Amazon, even for S3 this time. I'm going to use an S3-compatible layer just in case, but I don't want to give Amazon a dime anymore.
And because we have fallen for the convenience we have lost a lot of alternatives.
The old rack/cage way was less convenient, but it came with a respect for our anatomy to run our stuff that I really miss.
> Complexity! Complexity!!
This guy needs supabase or heroku or similar.
I was in a team that used aws once for their quantum computers. we had $100 of api credits. while still trying to get the code to work, we somehow used all of them, and then it didn’t even alert us that we were out of credits and just spent an additional $100 that we didn’t have. I would not touch this system again with a 10 foot pole…
Why do people even bother with cloud?
I’ve a couple of apps doing a few million a day. I am using Hetzner and before that used DigitalOcean. Mind you, for close to a decade.
People are unnecessarily complicating stuff, and these clouds can go very expensive very quickly.
Recently, I came across a company and they were spending $20k a month on GCP. I am like, are you kidding me, $20K for the kind of stuff you do??? It seems you do not understand how CPU, RAM and Disk work to plaster such "autoscaling hyper solutions" burning money in cloud.
I moved their stuff out of the GCP managed solution and ended up with a $200-400 per month bill. The CEO can still not believe how it's even possible.
I suggested them move to Dedicated servers but they didn't want it, they said they must show they are on Hyperscaling cloud.
OK fine, we'll stay in Hyperscaler but not use any of their service other than VMs.
They racked up a ton of bills by using cloud monitoring, Datastore, and autoscalers (with no proper tuning), Kubernetes.
I replaced all of it with Prometheus, Grafana, Loki, and most stuff from Datastore to Postgres and Mongo with replicas. I added Redis.
I implemented a custom scaler where you can scale off of app metrics, not by just using a random peg on CPU.
I implement hot data reload by packing the data updates in gzip file, uploading to GCS and pulling from autoscaled units. Moved the stuff to Spot VMs.
The complexity of stuff in cloud is high for nothing.
Slightly different but related topic - for people who work with people vibe coding, what is the easiest way to allow that for non tech users (and reducing risk)? AWS or something like vercel? Coolify?
i mostly used the ec2 instances and the workflow to make a simple gpu instance from the terminal (terraform) is terrible in contrast to ie gcp.
from the UI it is even worse
also how it is so slowwwwwww
You can accomplish a lot by just having a basic knowledge of Linux sysadmin. I was clueless and then learned some systemd-and-curl-fu. Will never forget the "holy sh*t, this is deceptively simple" moment. A bit more research and I found that beyond convenience and specialty APIs, you really just don't need a lot of this stuff to run a healthy system (since reducing absolute cloud dependence, my reliability has gone through the roof).
Almost everything is true about Azure as well, especially obscure pricing and complexity in absolutely everything.
When are we gonna start hearing the same stories about Anthropic/Openai/etc? The whole AI thing kinda smells like the early days of AWS: everyone was getting onboarded, but later realized they'd built up a pretty big dependency that's not easy to shake
sorry to hijack this but since we're talking the AWS egress exorbitant prices, could someone recommend a reliable production-tested s3 service that will not ruin me financially? i have been with hetzner and while their vps server are great, they are absolutely terrible at s3. random downtimes, incomplete uploads/downloads, capacity issues, terrible api key management, etc - but the failure rate is really high
The well architected frameworks tells you to have separate accounts, your fault that you "tested" in a production environment. https://imgur.com/a/Smal9fL
I'd tend to agree with the author. If forced to choose a cloud platform though (and that often is the case) then AWS is probably the best of the bunch in terms of reliability. Have heard and experienced some real horror stories with Azure & GCP by comparison.
IAM is my favorite part of AWS.
I also tried. Only service I use is s3 for personal backup. I pay around 15 cents per month.
The set of core services on AWS remains amazing: EC2, S3, IAM, EKS, Route53, RDS etc.
AWS IAM is extremely well designed when you compare it with the spaghetti monster IAM systems of other clouds.
Every time I try the new cool thing supposed to replace these services on some other provider - I understand how mature and polished the AWS ones are.
With that said, the rest 90% of AWS services like WorkMail, Cognito, API Gateway, are absolute hot garbage which no good meaning AWS expert will touch with a 10 meter stick.
The whole model of slicing the app into so many pieces (proprietary wrappers it seems like) to make more opportunities for billing is terrible.
> Of course I do not pay for premium support, so I have to wait the 24 hours that they said it would take them to reply. It's 3 days and AWS support has not replied.
The writing has been on the wall for a few years now, and this is particularly evident to those thar have worked at AWS: Amazon is in its day-2 era.
Amazon being in its day-2 era means that most of what has been written in the past twenty years about Amazon is bot valid anymore.
“Customer obsession” is literally their first leadership principle, and stellar support was their defining characteristic.
Amazon is bad at their blocking of accounts. They blocked mine for no reason (want me to call some USA phone number which I wont) a few years back and I was writing down everything that I would buy at their store, but I bought it elsewhere.
They have lost 3785.90 euros in sales due to their idiotic anti-user war.
Not to mention of all bad reputation that I gave them.
Preach, brother.
How lambda is as bad as it is I have no clue. Not a lover of azure, but azure functions is such a nicer experience
GCP would be perfect if they didn't have a history of randomly dropping quotas on startups, causing them downtime
I’m not sure how someone can be an “AWS Fanboy”, drink in all the promise, and think IAM is evil. As far as I can tell it is the one glorious thing that separates AWS from others. IAM is the core that makes it sane.
I was such a fan of it that I ended up working there for 4 years. Now I avoid it and encourage others to do the same.
AWS used to have a nifty tool called "policy analyzer" or something that monitored for permissions used by a role so you could scope it down. The other day I had the need for it and when I went to use it, found out they charge something like $9/resource. So I would pay $45/month for metadata monitoring on just 5 things? Nuts. If they knew how to build truly delightful products, they would make something like a role that starts with broad permissions and automatically scopes itself down after some point. And it would be free or at least really cheap.
DDB is hardly a database. The only reason I can think of to use it is for massive amounts of data whose schema and query patterns are guaranteed to almost never change, which is very rare. Need to sort data on a field? Then you have to create a 'secondary index', which is a copy of the table that they charge you for and that is not strongly consistent. Schema change? Good luck with that. And don't you dare ask to use a nice ORM library. But hey it's serverless.
Here's a good one: you stop an EC2 instance and its volume keeps running and you pay for that. If you detach the volume, you still pay. There is no way to 'archive' an instance. And the only way I found out about that was I got hit with a big bill for those volumes with the charge labeled 'EC2 - Other' lol. Not very 'customer obsessed' to me.
My gripes are clearly not important to them because this is old stuff. So all I can do is go somewhere else, which is fine with me
similar thing happened to me. I'm not a heavy aws user but wanted to setup some s3 buckets few days ago but my account was suspended for the same reason
but unlike OP I just accepted this fate and moved away from aws :)
But is there a better one with same IaaC and API completeness?
I love you baby, I need you! I'd never cheat on you! Come back!
Hey good lookin'
GCP has it's own share of issues.
...
I was writing a long vent about GCP but the mix of issues we had, just in the last few weeks, was too identifying and I don't want to sour an already tremulous relationship, as much as I'd like to spill it all here.
Let's just sum it up with resource crunch and degraded services because, apparently, when one customer signs a $200B deal [1] all the "just a few $10M" get thrown to the wayside.
AWS is also affected. Time to go to Azure? I never thought I'd say those words.
[1] https://www.engadget.com/2165585/anthropic-reportedly-agrees...
https://world.hey.com/dhh/merchants-of-complexity-4851301b
... skills issue. LOL. Good luck to making these tests easier on GCP / Azure. Or try to set up equivalent HW server without pulling your hair. Complexity is large part of why we, IT people, are paid those money. (Should've created *new AWS account* for those tests. AWS's automated security algorithms almost certainly flagged this abnormal behavior as a "suspected security breach" to protect the (long time dormant) user from potentially devastating unauthorized charges)
Every single complaint has a simple fix: “just use EC2”
> IAM - the hideously complex auth and access rules system - this was invented by Lucifer sitting on his burning throne in the ninth level of Hell as the worst possible torment for those who have been sent below for using AWS.
Perfect explanation - no notes
I don't think I remember anything so over-engineered and confusing in recent times (probably SELinux now that I think of it).
And I understand - we kinda need the complexity for what they intend to do but they do need a Come To Jesus moment here to make the Insane Asylum Machine make a bit more sense for mortals
2nd most annoying thing? Boto3 lib, where conventions don't matter and Pythonic is just a suggestion and the thing works more like a REST wrapper than anything else over a not-great API (please why tell me there's an S3 API and an S3Obj API)
As if there's any alternative. Azure? That mess of everything smashed on top of each other that looks like it was vibecoded in a few month by hundreds of people at once, except that they looked like this from the very beginning where there was no AI? The one that makes you fill docx forms to enable quotas for some services? Or Google Cloud, which _looks_ like it might be simpler, but it has permissions for permissions to enable permissions, and endless micromanagement? I am trying things, but I always return to AWS :(
The AWS UI should be, effectively, read-only for any infrastructure aside from setting up some initial roles and perms to manage all of it through an IAC system.
Put more bluntly, if you're using the AWS Console to spin-up/spin-down service instances you're doing it wrong.
Interesting take on the migration.
There is one fortunate result that will come from the SaaSpocalypse combining with Mythos (color me skeptical but let's assume it is as powerful as Anthropic tells CIOs).
If anyone can clone any SaaS, then there will be millions of SaaS that offer all the features you need.
How will you choose?
AWS and Microsoft (and all the big clouds) make it easy for their customers to get hacked, and Mythos makes it more likely the cadence will only intensify.
But, if I vibe code a hosting service which is pure rust and doesn't use any external libraries and never open sources my code, my attack surface is much smaller and I only have three customers anyway.
Hackers are lazy and will go for the pond where the most fish live. AWS will always have a lot of marks and a lot of holes.
AWS will be expensive because you are paying the tax they have to add to fend off the hordes. It'll be an intelligent choice to avoid working with Rome and find a little village in Bergen.
When it comes to email: “Don’t shit where you eat” is the closest analogy I can think of. Have your email somewhere else, it on any service that might decide to lock your account for any reason. Have your domain ownership somewhere else. And have already, or have a plan already, to move that somewhere if for some reason your email provider gets pissed at you.
AWS AIM is hot garbage, GCP might not be the coolest kid of the block but its AIM rocks.
AWS CLI??? Holy guacamole, what a mess. AWS CLI looks what is now the digital identification to get the basics done.
While GCP CLI is like "sure, here"!
Are the other two big providers any different?
>works for AWS >quits the BS >needs more money >sells himself as a meat puppeteer once again to AWS >big bs corpo is still the same surprisedpikatchu.jpeg ok
This belies the fact that AWS is so far in the lead in cloud market share and even host so much of Anthropic's business. If you dabble, it's confusing. If you're an enterprise with a lot of expertise then it's indispensable.
These complaints are very weak.
Lambda is incredibly simple to use, it just runs a function for you.
Not sure how you could burn so much with dynamodb. It’s serverless and incredibly cheap. Must have been doing something insane like a huge dataset where you scan through it over and over.
Being salty that Gary couldn’t sell enough of his paid service and AWS is competing with it isn’t a meaningful complaint. I want something in AWS, not on Gary’s servers.
anyone building with AWS or any cloud provider should be setting up a forked pipeline for their data production: one goes to the cloud provider DB used for production, the other is a local-on prem DB you regularly backup and always have on hand should you need to leave the cloud.
Yup -like the honesty
Why don't cloud services have reviews like amazon products.- I’m so tired of enterprise sales speak in corporate docs- just cut the BS and do straight talk
If you haven’t used a service you shouldn’t have to search reddit for dev experience on it
same pain here
> Somewhere in the depths of AWS some sort of security alarm had been triggered probably by the fact that my mostly dormant account suddenly started doing stuff with an expensive computer.
You mean the alarm that shows up with a notification bell in your console? Why not just post that?
> I am dreading having to "request quota" to be allowed to do that.
Why? It works fine. I've done it several times.
> IAM - the hideously complex auth and access rules system - this was invented by Lucifer sitting on his burning throne in the ninth level of Hell as the worst possible torment for those who have been sent below for using AWS.
It's literally a JSON policy document.
> - once I noticed the complexity of IAM I could not unsee the complexity everywhere in AWS.
All our policy actions are scripted at this point. You specify what functions the lambda calls and it builds the policy for you, sends it to IAM, and attaches it to the lambda.
Everytime I see someone complain about AWS I'm left wondering "did you read _any_ of the documentation? If you just want a linux server then run that but if you want out of the hassle of managing one then you need to learn just a _handful_ of new tricks."
If half the effort of complaining about AWS was spent reading documentation then most of these articles would never be published.