A couple preliminary findings if you are considering adopting this extension to replace your previous pgvector's HNSW indexing.
- pg_vectorsearch uses a quantized IVF index called PRISM. PRISM caps at 1968 dimensions for vectors. If your previous HNSW index has more dimensions than that, you're SOL (at least for now until they fix that).
- pg_vectorsearch requires Postgres version 18 or above. If you run PG versions below that you need to migrate your database first. Many legacy PG production environment are way dated on PG 14 or 15, migration is prohibitively high cost to just try this out.
- pg_vectorsearch shipped yesterday and it's obviously alpha. Hasn't been battle tested in production environments outside of Timescale's. Also since it's so new, it isn't on Azure's allowed-extension list or any other Hyperscale's managed PG instances' allowed extension list. It takes time to graduate from alpha and mature. So if you run managaed PG instances from hyperscaler's cloud, you're SOL (at least for now until this matures into those ecosystems) or you have to migrate to a PG 18 self-hosted.
show comments
mfreed
[Tiger Data CTO here]
A lot of great questions, feedback, and already some PR contributions today about pg_vectorsearch. I've written a short technical deep dive into the architecture of its PRISM index to answer some, thought I'd share:
I'm currently running VectorChord and I love it - with a fast backing disk, it's cut my postgres cost in half. Excited to see competition in low-RAM vector index tools. RAMageddon seems to be creating incentives to find more ways to take advantage of how fast modern nvme drives are!
show comments
trnvll
Can’t wait to try this. Have a project where speed matters and still want to have a nice and simple DX with everything in postgres
Rapzid
Does it support filtering though?
Edit: post filtering only
I don't understand how people are efficiently implementing tenant/user/permission scopes with these indexes.
show comments
arunmu
Didn't these guys also write pgvectorscale which seems to be unmaintained now ?
Why a new index instead of adding onto or improving pgvectorscale?
show comments
ilumanty
It would be so awesome if SQLite supported anything along these lines.
show comments
potomushto
Interesting, I see no mention of support for bit precision and hammer distance operator which supposed to be much faster and storage efficient
alexpadula
Dark side of the PRISM! The image on the repo 100% inspired by Dark Side of The Moon by Pink Floyd.
show comments
swe_dima
Sadly if cloud providers don't offer it on managed postgres then not many people can use it...
> TLDR: We are launching pg_vectorsearch, a new open-source PostgreSQL extension that makes vector search dramatically faster and more cost-efficient. Its new PRISM index uses partitioning and compression instead of HNSW's graph-based approach, delivering faster queries with a fraction of the memory and significantly faster index builds. At 100 million vectors, PRISM achieved 6 ms query latency on a machine with 4× less memory than HNSW, with index builds over 20× faster.
show comments
jakobedding
How does this compare to pgvectorscale?
show comments
esafak
Shouldn't you lead with your benchmarks if you're fast? qps and precision/recall, please.
A couple preliminary findings if you are considering adopting this extension to replace your previous pgvector's HNSW indexing.
- pg_vectorsearch uses a quantized IVF index called PRISM. PRISM caps at 1968 dimensions for vectors. If your previous HNSW index has more dimensions than that, you're SOL (at least for now until they fix that).
- pg_vectorsearch requires Postgres version 18 or above. If you run PG versions below that you need to migrate your database first. Many legacy PG production environment are way dated on PG 14 or 15, migration is prohibitively high cost to just try this out.
- pg_vectorsearch shipped yesterday and it's obviously alpha. Hasn't been battle tested in production environments outside of Timescale's. Also since it's so new, it isn't on Azure's allowed-extension list or any other Hyperscale's managed PG instances' allowed extension list. It takes time to graduate from alpha and mature. So if you run managaed PG instances from hyperscaler's cloud, you're SOL (at least for now until this matures into those ecosystems) or you have to migrate to a PG 18 self-hosted.
[Tiger Data CTO here]
A lot of great questions, feedback, and already some PR contributions today about pg_vectorsearch. I've written a short technical deep dive into the architecture of its PRISM index to answer some, thought I'd share:
https://www.linkedin.com/pulse/inside-pgvectorsearch-archite...
I'm currently running VectorChord and I love it - with a fast backing disk, it's cut my postgres cost in half. Excited to see competition in low-RAM vector index tools. RAMageddon seems to be creating incentives to find more ways to take advantage of how fast modern nvme drives are!
Can’t wait to try this. Have a project where speed matters and still want to have a nice and simple DX with everything in postgres
Does it support filtering though?
Edit: post filtering only
I don't understand how people are efficiently implementing tenant/user/permission scopes with these indexes.
Didn't these guys also write pgvectorscale which seems to be unmaintained now ? Why a new index instead of adding onto or improving pgvectorscale?
It would be so awesome if SQLite supported anything along these lines.
Interesting, I see no mention of support for bit precision and hammer distance operator which supposed to be much faster and storage efficient
Dark side of the PRISM! The image on the repo 100% inspired by Dark Side of The Moon by Pink Floyd.
Sadly if cloud providers don't offer it on managed postgres then not many people can use it...
I'm curious how it compares to `pg_search` https://www.paradedb.com/blog/introducing-search
How does it compare to Roar graph? https://github.com/matchyc/RoarGraph
from the accompanying blog post: https://www.tigerdata.com/blog/announcing-pg_vectorsearch-sc...
> TLDR: We are launching pg_vectorsearch, a new open-source PostgreSQL extension that makes vector search dramatically faster and more cost-efficient. Its new PRISM index uses partitioning and compression instead of HNSW's graph-based approach, delivering faster queries with a fraction of the memory and significantly faster index builds. At 100 million vectors, PRISM achieved 6 ms query latency on a machine with 4× less memory than HNSW, with index builds over 20× faster.
How does this compare to pgvectorscale?
Shouldn't you lead with your benchmarks if you're fast? qps and precision/recall, please.