I love Postgres, and I agree with the general sentiment. But I read the (growing) genre of "use Postgres for everything" articles and they imply a difficulty in running other software that I just don't see.
I'm thinking of Redis in particular. If you're using it as incredibly fast but not critical storage, it's trivial to set up and it ~never crashes or requires maintenance. It creates no headaches, and in exchange gives me a k/v store that I can thrash without worrying about performance (I know it's fast), downstream impact (am I slowing down critical-path SQL queries), etc. Especially in the age of LLMs, which I've found to be great at devops-type tasks, I feel slightly less compelled to simplify my stack.
show comments
CodesInChaos
I'd love the ability to mark a table as "read committed" to prevent long running transactions from keeping old versions of a tuple alive, or even "read uncommitted" to enable in-place updates. Or perhaps instead of downgrading isolation, those serializable/snapshot transactions could simply fail when reading a value from such a table that was modified after they started.
An example of a table that would benefit from this would be rate-limits / concurrency-limits, which are commonly implemented using Redis instead of Postgres.
show comments
ComputerGuru
I am very much in camp “minimize your dependencies and take the utmost advantage of what you already have” but I also am in camp “your database is the most important bottleneck component.”
This has two implications:
1) make sure if you use Postgres for anything beyond core rdbms functionality that there is no dependency between the two, so you can rip out the additional functionality and move to a different platform when you end up needing to reduce the load on your db server
2) if using Postgres for non-essentials complicates your db backup workflow, risks the data integrity, makes it difficult to maintain or upgrade your Postgres instance (eg you have to wait months or years for compatibility with newer Postgres versions), or loads relatively shoddy or unstable code into the beating heart of your application, then you should either use a different Postgres server/install/container for these ancillary services or bite the bullet and introduce an alternative dependency, depending on which makes more sense.
JsonDemWitOster
I'll do you one better: do you really need Postgres and all these extensions when you already have a filesystem?
> only after pushing Postgres to its limits, documenting why it was insufficient, and accepting the operational cost of the alternative
I love Postgres as a DB but, really, this is ridiculous. No doubt these extensions can do the job well-enough but you might as well invest in learning the right tool for the problem from the start, when the stakes are still pretty low. Why wait until, ahem, Postgres is pushed to the limit before you spin up a Redis cluster?
You don't get free opcost by using Postgres for everything. Arguably if you end up with a monolith of a database, you are paying a higher opcost (imagine if too much caching can affect all CRUD ops in your platform). Or you can manage a cluster of PG instances but that's no less complex---each plugin still comes with its own opcost!
No Silver Bullet, No Free Lunch, and all that. If your problem domain really warrants something outside of relational storage, you're gonna pay that complexity cost one way or another. You can't escape it by shoehorning everything in Postgres, fantastic as a DB as it is.
show comments
wuliwong
For context, I am mainly talking about my personal projects, ideas I am trying out, MVPs/prototypes for potential new businesses, etc.
I have started just using Postgres to back queues. It is simpler (although I have spun up new apps with Redis so many times it is only a small improvement) but more importantly it is cheaper. I really do try a lot of stuff out, so completely removing a infrastructure piece is a nice money saver. Again, not massive but it's cheaper.
The downsides of doing this in the prototype/MVP context are minimal. At the scale of prototype and MVPs, I certainly don't see any difference in performance.
I did note in the graphic it listed Kafka but in the lower table graphic I did not see any Postgres replacement for Kafka. If I am at the scale where I really want Kafka, it is probably for performance and I just can't believe there's anyway Postgres could provide that.
So, I love the flexibility and all-in-one abilities of Postgres for prototyping and making MVPs. But at my job, nobody is proposing exclusively using Postgres for persistence.
armdave
Articles like these have been insanely effective in biasing LLMs (especially OpenAI) toward Postgres. There areas where Postgres beats MySQL, and vice versa - but ChatGPT/Codex overwhelmingly recommend "default to Postgres."
show comments
tudorg
As small piece of feedback, I think it would be really good if the tools list would include license information, and make it easy to filter for open source tools/licenses. IMHO source-available extensions break part of what is attractive to the Postgres ecosystem, which is that you can move from one provider to another without vendor lock-in.
stickfigure
Fun fact for people whose scaling plan anticipates moving from Postgres to CRDB:
SELECT FOR UPDATE SKIP LOCKED works great for turning Postgres into a job queue. It does not work the same way on CRDB and you will likely have to rewrite those queries or use an external job queue.
pavel_lishin
Are the various plugins easier to set up, configure, troubleshoot, etc. than the other software options? If I'm paged awake at 2am, will I have an easier time figuring out pgmq than I will RabbitMQ?
show comments
bitbasher
I was on the Postgres train for 10+ years. Then I fell in love with Sqlite and have been using it for everything.
h1fra
I have yet to have used UNLOGGED table in production, but I really want to try at some point. Anybody actually replaced a large Redis instance with this?
ubercore
I love postgres, but the complexity of using it for everything starts to get pretty high, compared to more tailor-suited tools. We should probably use it for _more_, in general, but the cost of "everything in postgres" is generally higher than I see acknowledged in articles like these.
show comments
devin
> This isn't about dogma. Sometimes you genuinely need specialized infrastructure. But the bar should be high: only after pushing Postgres to its limits, documenting why it was insufficient, and accepting the operational cost of the alternative.
I've seen a few "Use Postgres for Everything!" posts lately. It seems to be fashionable. It reminds me of the Choose Boring Technology[1] thing from 2018 or so, but more specific to a database.
I think the ideas of "don't add unnecessary dependencies" and "ruthlessly evaluate tradeoffs" and "prefer simplicity" and so on are general and have very little to with postgres, so when I see things like "All you need is X" I roll my eyes a little, because these decisions are highly dependent on your use case, and taken as blanket advice it is generally _bad_ advice even if the underlying rationale is sound.
ETA: I am going through their list and so much of this means that you are going to manage your own PG cluster and not take advantage of Aurora or RDS, which means you're already committing to a major tradeoff if you want to use a lot of these custom extensions.
simonbarker87
Couple that with a framework with decent server side rendered template support and htmx for network based interactions and you’ve got 90% of the <acronym_of_the_moment> stack with 10% of the complexity.
DonsDiscountGas
I suspect a lot of this is resume driven design.
valentynkit
The thing that usually pushes you off Postgres is rarely raw throughput, it's two workloads that want opposite tuning on the same box.
show comments
polycancel
> Document store -> FerretDB
Love FerretDB, but it doesn't really replace MongoDB's GridFS which is main reason why most people who are really using Mongo now day. Anyone knows a good replacement for GridFS?
The reason for separate systems is that some of them go down. And by "go down", that includes you wanting to deploy a new version. Or restore from backup as the article suggested.
show comments
bijowo1676
lets take it one step further: Do you really need Postgres when you already have SQLite?
show comments
DarkCrusader2
Given the unanimous consensus on HN that Postgres is all you will ever need. I will present 2 resources which I came across recently which changed my perspective on this blind reverence for Postgres for everything.
First is this Oxide and Friends episode [1] where Bryan and gang explains war stories related to operating Postgres during their Joyent days and why they went with Cockroach DB for Oxide.
Second is this amazing blog from brandur which explains several issues with using Postgres as high throughput queue and some mitigations.
Online forums like Hacker news can be a bit echo chamber-y. It is always good to ensure that the people you are taking advice from are solving the same problem as you.
... and unlike all the new databases, Postgres has a decent license. Everybody else is so afraid of being co-opted by AWS that they won't let you run them the way the way you want.
show comments
chestervonwinch
what about for blob storage too?
otabdeveloper4
Postgres sucks. It does a little bit of everything, but badly and with much manual intervention.
I love Postgres, and I agree with the general sentiment. But I read the (growing) genre of "use Postgres for everything" articles and they imply a difficulty in running other software that I just don't see.
I'm thinking of Redis in particular. If you're using it as incredibly fast but not critical storage, it's trivial to set up and it ~never crashes or requires maintenance. It creates no headaches, and in exchange gives me a k/v store that I can thrash without worrying about performance (I know it's fast), downstream impact (am I slowing down critical-path SQL queries), etc. Especially in the age of LLMs, which I've found to be great at devops-type tasks, I feel slightly less compelled to simplify my stack.
I'd love the ability to mark a table as "read committed" to prevent long running transactions from keeping old versions of a tuple alive, or even "read uncommitted" to enable in-place updates. Or perhaps instead of downgrading isolation, those serializable/snapshot transactions could simply fail when reading a value from such a table that was modified after they started.
An example of a table that would benefit from this would be rate-limits / concurrency-limits, which are commonly implemented using Redis instead of Postgres.
I am very much in camp “minimize your dependencies and take the utmost advantage of what you already have” but I also am in camp “your database is the most important bottleneck component.”
This has two implications:
1) make sure if you use Postgres for anything beyond core rdbms functionality that there is no dependency between the two, so you can rip out the additional functionality and move to a different platform when you end up needing to reduce the load on your db server
2) if using Postgres for non-essentials complicates your db backup workflow, risks the data integrity, makes it difficult to maintain or upgrade your Postgres instance (eg you have to wait months or years for compatibility with newer Postgres versions), or loads relatively shoddy or unstable code into the beating heart of your application, then you should either use a different Postgres server/install/container for these ancillary services or bite the bullet and introduce an alternative dependency, depending on which makes more sense.
I'll do you one better: do you really need Postgres and all these extensions when you already have a filesystem?
> only after pushing Postgres to its limits, documenting why it was insufficient, and accepting the operational cost of the alternative
I love Postgres as a DB but, really, this is ridiculous. No doubt these extensions can do the job well-enough but you might as well invest in learning the right tool for the problem from the start, when the stakes are still pretty low. Why wait until, ahem, Postgres is pushed to the limit before you spin up a Redis cluster?
You don't get free opcost by using Postgres for everything. Arguably if you end up with a monolith of a database, you are paying a higher opcost (imagine if too much caching can affect all CRUD ops in your platform). Or you can manage a cluster of PG instances but that's no less complex---each plugin still comes with its own opcost!
No Silver Bullet, No Free Lunch, and all that. If your problem domain really warrants something outside of relational storage, you're gonna pay that complexity cost one way or another. You can't escape it by shoehorning everything in Postgres, fantastic as a DB as it is.
For context, I am mainly talking about my personal projects, ideas I am trying out, MVPs/prototypes for potential new businesses, etc.
I have started just using Postgres to back queues. It is simpler (although I have spun up new apps with Redis so many times it is only a small improvement) but more importantly it is cheaper. I really do try a lot of stuff out, so completely removing a infrastructure piece is a nice money saver. Again, not massive but it's cheaper.
The downsides of doing this in the prototype/MVP context are minimal. At the scale of prototype and MVPs, I certainly don't see any difference in performance.
I did note in the graphic it listed Kafka but in the lower table graphic I did not see any Postgres replacement for Kafka. If I am at the scale where I really want Kafka, it is probably for performance and I just can't believe there's anyway Postgres could provide that.
So, I love the flexibility and all-in-one abilities of Postgres for prototyping and making MVPs. But at my job, nobody is proposing exclusively using Postgres for persistence.
Articles like these have been insanely effective in biasing LLMs (especially OpenAI) toward Postgres. There areas where Postgres beats MySQL, and vice versa - but ChatGPT/Codex overwhelmingly recommend "default to Postgres."
As small piece of feedback, I think it would be really good if the tools list would include license information, and make it easy to filter for open source tools/licenses. IMHO source-available extensions break part of what is attractive to the Postgres ecosystem, which is that you can move from one provider to another without vendor lock-in.
Fun fact for people whose scaling plan anticipates moving from Postgres to CRDB:
SELECT FOR UPDATE SKIP LOCKED works great for turning Postgres into a job queue. It does not work the same way on CRDB and you will likely have to rewrite those queries or use an external job queue.
Are the various plugins easier to set up, configure, troubleshoot, etc. than the other software options? If I'm paged awake at 2am, will I have an easier time figuring out pgmq than I will RabbitMQ?
I was on the Postgres train for 10+ years. Then I fell in love with Sqlite and have been using it for everything.
I have yet to have used UNLOGGED table in production, but I really want to try at some point. Anybody actually replaced a large Redis instance with this?
I love postgres, but the complexity of using it for everything starts to get pretty high, compared to more tailor-suited tools. We should probably use it for _more_, in general, but the cost of "everything in postgres" is generally higher than I see acknowledged in articles like these.
> This isn't about dogma. Sometimes you genuinely need specialized infrastructure. But the bar should be high: only after pushing Postgres to its limits, documenting why it was insufficient, and accepting the operational cost of the alternative.
I've seen a few "Use Postgres for Everything!" posts lately. It seems to be fashionable. It reminds me of the Choose Boring Technology[1] thing from 2018 or so, but more specific to a database.
I think the ideas of "don't add unnecessary dependencies" and "ruthlessly evaluate tradeoffs" and "prefer simplicity" and so on are general and have very little to with postgres, so when I see things like "All you need is X" I roll my eyes a little, because these decisions are highly dependent on your use case, and taken as blanket advice it is generally _bad_ advice even if the underlying rationale is sound.
[1]: https://mcfunley.com/choose-boring-technology
ETA: I am going through their list and so much of this means that you are going to manage your own PG cluster and not take advantage of Aurora or RDS, which means you're already committing to a major tradeoff if you want to use a lot of these custom extensions.
Couple that with a framework with decent server side rendered template support and htmx for network based interactions and you’ve got 90% of the <acronym_of_the_moment> stack with 10% of the complexity.
I suspect a lot of this is resume driven design.
The thing that usually pushes you off Postgres is rarely raw throughput, it's two workloads that want opposite tuning on the same box.
> Document store -> FerretDB
Love FerretDB, but it doesn't really replace MongoDB's GridFS which is main reason why most people who are really using Mongo now day. Anyone knows a good replacement for GridFS?
agree:
https://github.com/seanwevans/pg_os
https://github.com/seanwevans/pg_git
https://github.com/seanwevans/pg_gpt2
etc...
Reminds me Kai Lentit's ffmpeg video
https://youtu.be/9kaIXkImCAM
The reason for separate systems is that some of them go down. And by "go down", that includes you wanting to deploy a new version. Or restore from backup as the article suggested.
lets take it one step further: Do you really need Postgres when you already have SQLite?
Given the unanimous consensus on HN that Postgres is all you will ever need. I will present 2 resources which I came across recently which changed my perspective on this blind reverence for Postgres for everything.
First is this Oxide and Friends episode [1] where Bryan and gang explains war stories related to operating Postgres during their Joyent days and why they went with Cockroach DB for Oxide.
Second is this amazing blog from brandur which explains several issues with using Postgres as high throughput queue and some mitigations.
Online forums like Hacker news can be a bit echo chamber-y. It is always good to ensure that the people you are taking advice from are solving the same problem as you.
[1] https://oxide-and-friends.transistor.fm/episodes/whither-coc... [2] https://brandur.org/postgres-queues
... and unlike all the new databases, Postgres has a decent license. Everybody else is so afraid of being co-opted by AWS that they won't let you run them the way the way you want.
what about for blob storage too?
Postgres sucks. It does a little bit of everything, but badly and with much manual intervention.