Does anyone run Postgres without PgBouncer?

118 points76 comments4 days ago
conradludgate

(I work on the postgres proxy layer at Neon)

PgBouncer is entirely optional and it's not always the right choice. If you have a classical app (non serverless) and you can maintain a connection pool from your app, then I recommend avoiding pgbouncer.

The benefits of pgbouncer mostly come from irregular client connections (too many, too much churn). If you don't have that problem, go direct to postgres.

I'm exploring replacing pgbouncer with an alternative (maybe home grown) at the moment. Mostly for multi-tenancy and HA reasons. Pgbouncer has been good for us, but it's limited in how we can deploy it in a multi-tenant environment.

show comments
petcat

This question will get more interesting responses if it was qualified as:

"Does anyone run Postgres without PgBouncer for non-trivial workloads?"

Because, as we can see from the comments so far, lots of people are going to say you don't need it for your blog that gets 10 hits a month.

I've personally never heard of anyone not using PGbouncer, or some connection pooling proxy, for reasonably concurrent workloads. PG's process-per-connection architecture almost requires it. Otherwise even a small connection storm will wreak havoc on your server.

show comments
solatic

Article title is pure click-bait. PgBouncer adds complexity. If you need it, you need it. If you don't need it, then you added complexity for nothing.

> since neither IBM nor Oracle is a service that any self-respecting person not part of an enterprise sales cycle would actually use

Author needs a serious ego check. There are legitimate engineering reasons to pick IBM Cloud (like if you need to support Z mainframes, which you will sometimes need if you sell to those enterprise folk) as well as Oracle Cloud (they built datacenters in cities that are not served by other cloud providers and can thus offer the lowest latency). These reasons may not be common, but they're certainly legitimate.

show comments
matsemann

Python: absolutely necessary due to the amount of processes and various deployments to run an application once it grows.

Java: never felt the need even on quite big apps. As it's much easier to share a connection pool locally, it's not as many single connections across the whole app.

show comments
fabian2k

If you have a connection pooler in your application, and the DB is only used for this application, you don't need an external pool like PgBouncer. That is probably a pretty common scenario, and typical web frameworks include a connection pool anyway.

An additional complication is that more aggressive pooling methods have side effects that you must know and prevent in your application. They're not safe to use out of the box.

The need for a connection pool is a side effect of the heavy process-based PostgreSQL connections. And I would suspect that this will change at some point in the not so near future, so that users don't have to think about this part this much.

show comments
mbreese

I’d argue this is not the right question. Obviously people use Postgres without PgBouncer. If you include non-production in the mix (CI/CD), most connections probably avoid it.

But because PgBouncer is so common, the question should probably be, why isn’t connection pooling part of Postgres out of the box? I think this might be the more interesting question.

theandrewbailey

Yes. Sometimes PostgreSQL is overkill for a low traffic, simple application. PgBouncer would be even more overkill and add unnecessary complexity.

giovannibonetti

I see many comments comparing PgBouncer with application connection poolers without addressing the conceptual difference between them. Here it is:

1. Most application connection poolers follow a first-in-first-out (FIFO) algorithm, which is simple enough to implement and is enough to make sure the application always has a connection available to connect to the database. It optimizes low latency, and works great from the point of view from the application. The problem is that it has few mechanisms to remove redundant connections, since the application is constantly keeping them all "warm".

2. PgBouncer and very few external poolers follow the inverse idea – last-in-first-out (LIFO), and they optimize for reducing the number of connections that reach Postgres, thus improving its throughput. The idea might seem crazy at first – the last connection used is the first one to be picked up again – but this algorithm automatically removes excess connections, which will get cold and get closed.

When starting a new application, option (1) is enough, but as it scales up enough, at some time it is recommended to use (2), since having hundreds of open connections to Postgres is bad for performance if you can use PgBouncer or similar to cut it by 90%. Postgres' process-per-connection design works much better when there are fewer connections reaching it.

grsmvg

If you don’t use serverless but instead a few (vertically scaling) servers, and your ORM / query builder supports pooling (all node libraries I’ve used have a pooler)…

Having a setup with just a simple docker deploy, running a monolith, not using pgbouncer so you can use LISTEN/NOTIFY to implement your own job queue:

https://www.dbos.dev/blog/postgres-listen-notify-scalability

This gives me warm fuzzy feelings, also making me relatively cloud-agnostic in the process, even though devops is not my strong point.

Most projects I do don’t need something more complex or vendor locked-in than this.

saisrirampur

Totally agree. Postgres core should include connection pooling by default, and PgBouncer is probably the natural path to get there.

With PgBouncer addressing one of its biggest historical pain points - prepared statement support in transaction mode, in our managed Postgres offering, we’re increasingly seeing customers use the PgBouncer connection string by default for mose use-cases without running into any hiccups. That wouldn’t necessarily have been the case a few years ago.

PgBouncer is also battle-tested, widely validated, and offers a (surprising) level of configurability. You could also run a peered setup and make it multi-threaded, which is something I didn’t expect when I first came to know about it. https://news.ycombinator.com/item?id=48872874

KronisLV

> since neither IBM nor Oracle is a service that any self-respecting person not part of an enterprise sales cycle would actually use

Heh, snarky.

I guess (almost) everyone uses PgBouncer because they want to use a setup that will need the scaling needs of most users from the get-go, to avoid wasting support time.

Personally, I run without it due to a fairly small scale - I don't need more than about 128-256 connections max and for me a dedicated connection pooler (other than what's sometimes used app-side) would just add complexity.

At the same time, one could totally reasonably make the argument that if almost everyone uses it, then it SHOULD quite possibly be a built in feature, instead of a separate component - such a tighter integration would most likely bring the overall complexity down.

chuckadams

I guarantee you the vast majority of RDS instances aren't using RDS Proxy. For one, it's rather expensive.

henvic

For sure. I mean: many – if not most – people?

In my case, with Go, I have always relied on http://github.com/jackc/pgx pool, which works quite well, especially with the binary protocol.

hoppp

Yes, I don't always need it. Depends on the service architecture.

arjie

I wouldn't do it without pgbouncer. Asking for trouble when one day connections exceeds. It's just that you start with "oh I'll manage the pool from my app" and then you're stuck with either putting things into the app or tuning the pool for the other side-programs you need from the app.

1over137

Yes. I've never even heard of PgBouncer.

show comments
ComputerGuru

Clickbait post, already made the rounds on in other sites and was mercilessly torn to shreds. The answer is that millions do in production. PgBouncer is only needed for stateless backends, and even then, only under specific circumstances.

achanda358

A whole bunch of Cloudflare clients run Postgres (self hosted) without pgbouncer (they use hyperdrive).

georgewfraser

What pgbouncer does is indeed core functionality. Compare Postgres to MySQL and sql server, where analogous standalone connection pools are rarely used. The fundamental reason pgbouncer needs to exist is Postgres’ utterly retrograde design. Other examples: xid wraparound, conflict with recovery, lack of undo space.

recroad

I use the Ecto connection pool when using Elixir/Phoenix.

fires10

I currently run without pgbouncer, but I have at most 200 connections. 50 of them are just replication. I didn't want the complexity.

alexthedigger

Yes

131hn

I’m curius about pgcat, is there anyone here using it ?

show comments
barelysapient

Yes.

eqvinox

I mean… my self hosted Postgres with its ca 15 active connections certainly doesn't use PgBouncer, and it doesn't need to. But that was presumably not the intended scope of the question?

Then again, people forget you can just run your own Postgres (or anything really).

throw1234567891

The answer is: yes.

zzzeek

of course, the vast, vast majority of PostgreSQL users outside of managed cloud hosting are not using pgbouncer. pgbouncer introduces complexities into the database conversation (transaction-level pooling interacting with the prepared statement cache is a long recurring nightmare for us at sqlalchemy) that often not worth the complexity for small local installations.

this article seems to be talking about commercial cloud managed PG services, which yes, those absolutely need to support connection pooling and of course they're going to use pgbouncer.

nilamo

I use postgres. I've never heard of pgbouncer. So the answer is an easy Yes.

Saved you a click.

ParadisoShlee

pgdog.

root-parent

Are you kidding me? Have you heard about Data Direct?

https://docs.progress.com/bundle/datadirect-postgresql-odbc-...

show comments
lazyc97

Yes, except for serverless backend, you normally don't need it.

fenestella

The overhead of process-per-connection in Postgres is still the biggest hurdle for mobile-heavy workloads where you might have thousands of intermittent clients. Brandur is right that modern hardware makes the context switching less of a bottleneck, but the memory pressure from large work_mem settings on each backend process remains a real risk. I'm curious if anyone here has successfully moved to a pure built-in connection pool in v14+ without seeing a regression in latency during traffic spikes.

show comments
saadyousfi

Transaction mode has one real footgun: SET statements are session-scoped, but in transaction mode your session gets reassigned between transactions. Easy to miss until you're chasing a mysterious search_path bug at 2am.

psycopg3 prepares statements by default now, which breaks in transaction mode unless you explicitly opt out.

For apps with a persistent server process and an in-app pool the external bouncer is mostly ceremony. The math changes with serverless: no persistent process means no persistent pool, so a dedicated pooler starts pulling its weight.

CodeWithLeo

We run asyncpg directly against Neon Postgres without PgBouncer — works well for low-to-medium concurrency. Neon's serverless connection pooling handles a lot of what PgBouncer would, so the overhead didn't seem worth it for our use case.