It’s 3 AM. Your Kafka consumer has fallen behind. Elasticsearch doesn’t reflect what’s in your database. And somewhere in a Debezium connector config is a setting that nobody on your team fully understands. You fix it, write a post-mortem, and move on.
Running Elasticsearch alongside Postgres has always been a trade-off — better relevance scoring in exchange for a three-system architecture and a permanent operational headache. In 2026, BM25 relevance scoring is available natively in Postgres, which removes the technical justification for keeping a separate cluster around. This article is about the search-specific side of that — part of the broader case for Postgres as the default AI database. We’ll cover what BM25 and hybrid search in Postgres actually look like, how RRF ties them together, and when migrating from Elasticsearch is worth the effort.
Why do teams still run Elasticsearch in 2026, and what has changed?
Elasticsearch earned its place because Postgres’s native full-text search produced poor relevance results. ts_rank counts term occurrences — no document length normalisation, no understanding of how rare a term is. Elasticsearch used BM25 via Lucene and consistently ranked results better. So teams added it to their stacks.
The problem is everything that came with it. Elasticsearch’s JVM runtime means garbage collection is a permanent operational concern. Pauses over 30 seconds cause the cluster to assume a node is dead and start redistributing data. Then there’s the sync pipeline — Postgres → Kafka/Debezium → Elasticsearch — three systems, three things that can fail, and data that goes stale the moment a consumer falls behind.
What’s changed is that all three pieces of Elasticsearch’s hybrid search — BM25, vector search, and RRF — are now available natively in Postgres via extensions. pg_search from ParadeDB and pg_textsearch from TigerData both bring BM25 to Postgres. pgvector and pgvectorscale cover vector search. And pgai eliminates the embedding sync pipeline entirely. For most teams, that changes the conversation.
What is BM25 and why does it produce better results than Postgres full-text search?
BM25 — formally Okapi BM25 — is the probabilistic ranking algorithm at the core of Elasticsearch’s relevance scoring. It combines three signals that ts_rank simply doesn’t have: term frequency with diminishing returns, inverse document frequency (IDF), and document length normalisation.
Here’s what that means in practice. ts_rank scores a 10,000-word document that mentions “authentication” ten times higher than a focused 100-word document that mentions it five times. The shorter document is probably more relevant — it’s specifically about authentication. BM25 adjusts for this. Shorter, focused documents are preferred over long ones that mention terms incidentally. IDF handles the rest: if “database” appears in 80% of your documents, it’s noise; if “authentication” appears in 5%, it’s a signal.
That’s precisely why Elasticsearch was adopted in the first place. Benchmarks on a 10-million-row dataset show pg_search (BM25) outperforming native Postgres FTS by 20–1000x on query latency, while matching Elasticsearch’s relevance quality.
How do you add BM25 to Postgres with pg_search or pg_textsearch?
There are two extensions worth evaluating. pg_search from ParadeDB creates BM25 indexes using CREATE INDEX ... USING bm25 syntax and queries with the @@@ operator — production-ready, available on Neon, V2 API out in late 2025. pg_textsearch from TigerData uses to_bm25query() syntax and fits naturally into teams already on the TimescaleDB ecosystem with pgvectorscale or pgai.
Both implement the same Okapi BM25 algorithm. The choice comes down to ecosystem fit: Neon or greenfield deployments lean toward pg_search; TimescaleDB shops lean toward pg_textsearch.
Installation is straightforward. Add the extension, create a BM25 index on the target text column, update your existing ts_rank queries. No data migration — existing data is indexed in place. One backup strategy, one security model, one monitoring setup. With Elasticsearch, you have to replicate your tenant isolation logic into the search engine as a separate concern. With Postgres BM25, row-level security applies automatically. One less place to get it wrong.
The vector component that pairs with BM25 in hybrid search is covered in depth in pgvectorscale for vector search.
How does Reciprocal Rank Fusion merge keyword and semantic search results?
BM25 keyword search is precise for exact-match queries — error codes, product SKUs, specific terminology. Vector semantic search handles intent and paraphrase — “why is my database slow” returns relevant performance docs even if none of them contain those exact words. Neither alone covers the full range of queries users actually type. That’s the problem RRF solves.
Reciprocal Rank Fusion scores each document as 1/(k+rank) from each retrieval system, where k is typically 60 and rank is the document’s position in that system’s results. Scores are summed across systems. Documents in both result lists get the additive benefit. Nothing is discarded.
What makes RRF practically useful is that it’s scale-independent. BM25 scores are log-probability ratios; vector cosine similarities are bounded between -1 and 1. Normalising between those two scales is awkward and error-prone. RRF sidesteps the problem entirely by working on ranks rather than raw scores.
The whole thing is expressible in pure SQL using CTEs. Two CTEs run the BM25 and vector searches; a third scores and merges using the RRF formula; a final SELECT returns the ranked list. No extension required for RRF itself — roughly 20–30 lines of SQL, wrappable in a reusable function.
What does temporal hybrid search add, and when do you need it?
Standard hybrid search ranks by relevance alone. Relevance has no awareness of time. A 2019 configuration guide can outrank the 2024 version if its content better matches the query — the old document was comprehensive, so its embedding is strong and its keyword density is high. That’s the problem.
TigerData documented the failure mode directly. Query: “How to enable logging in NovaCLI”. Both BM25 and vector search return the deprecated v1.0 doc. Standard hybrid search amplifies the wrong consensus. Only temporal hybrid search returns the correct v3.1 doc. In RAG pipelines, stale context propagates to the response — confidently wrong answers, from outdated docs that happened to be better written.
Temporal hybrid search adds a time-range pre-filter to the BM25 and vector CTEs before the RRF merge. Documents outside the recency window never enter the ranking pipeline at all. No additional extension required — it’s a WHERE clause on the existing CTEs. When you need it: RAG applications with frequently updated knowledge bases, agent memory systems, any search-powered AI feature where freshness is a correctness requirement. When you don’t: static corpora where time is irrelevant to user intent. For more on how this plays out in agent workflows, see temporal hybrid search in AI agents.
What does an Elasticsearch migration to Postgres actually involve?
The migration follows four phases, and the sequence matters.
First, add BM25 to your existing Postgres database. Install pg_search or pg_textsearch, create BM25 indexes on the text columns currently being synced to Elasticsearch. Existing data is indexed in place. No downtime, no data migration. Your Elasticsearch cluster stays running.
Second, run searches in parallel. Route queries to both systems simultaneously and log both result sets until you have enough data to evaluate relevance quality.
Third, validate relevance. Compare result sets using NDCG or spot-checking. For most document corpora — knowledge bases, support docs, product catalogues — BM25 parity with Elasticsearch is achievable. Faceted search is the known gap.
Fourth, cut over and decommission. Switch to Postgres, validate, then retire the Elasticsearch cluster and the Kafka/Debezium sync pipeline. Infrastructure, failure modes, three-system complexity — all gone.
A SaaS team that replaced Elasticsearch with Postgres put it plainly: “Simply put, it’s just one less thing to worry about.” Their dataset was around 100k documents; simple queries ran in 5–10ms. That’s typical for SaaS-scale workloads. For larger clusters, this is not a weekend project — decommissioning the sync pipeline requires confidence that nothing else depends on the CDC stream. The business case is laid out in full in the Elasticsearch consolidation ROI breakdown.
When should you keep Elasticsearch?
Three scenarios still favour retaining Elasticsearch.
The first is faceted search at e-commerce scale. Filter-heavy patterns — category, price range, rating, availability — are a known maturity gap in current Postgres BM25 extensions. pg_textsearch doesn’t currently support faceted search. Evaluate this carefully before you commit to migration.
The second is very high search query volume. If search QPS is causing read/write contention on the same Postgres instance handling transactional writes, a dedicated search cluster may still be worth it.
The third is deep tooling investment. Custom Kibana dashboards, Elasticsearch alerting, Elastic APM — the migration cost includes replacing all of that. For teams where that stability is worth more than the consolidation gains, that’s a perfectly legitimate call.
For most teams running RAG pipelines, semantic search, or moderate-scale full-text search, Postgres with BM25, pgvector, and RRF matches Elasticsearch quality and removes the overhead entirely. That’s the case for database consolidation in concrete terms — one database, one backup strategy, one security model, one thing to run.
Frequently Asked Questions
Can Postgres replace Elasticsearch for search in my application?
For most applications — RAG pipelines, semantic search, moderate-scale full-text search — yes. Postgres with BM25 via pg_search or pg_textsearch, combined with pgvector and RRF, matches Elasticsearch relevance quality while eliminating the sync pipeline and JVM overhead. Exceptions: very high search QPS requiring a dedicated cluster, and complex faceted search.
What is pg_search and how does it differ from pg_textsearch?
pg_search is ParadeDB’s BM25 extension for Postgres, using @@@ operator syntax and available on Neon. pg_textsearch is TigerData’s BM25 extension using to_bm25query() syntax, better suited to teams already in the TimescaleDB ecosystem. Both implement Okapi BM25 and produce comparable relevance quality.
What is RRF and why is k=60 the default?
Reciprocal Rank Fusion scores each document as 1/(k+rank) summed across result lists. k=60 is an empirical default from the original RRF research paper. It’s a tuning parameter — lowering k increases the weight of top ranks; raising it flattens the distribution.
Do I need Kafka and Debezium if I migrate to Postgres for search?
No. When search runs inside the same Postgres database as your primary data, writes are immediately searchable within the same ACID transaction. The sync pipeline that kept Elasticsearch updated is no longer needed.
Is Postgres BM25 search good enough for a 100,000-document dataset?
Yes. Benchmarks on a 10-million-row dataset show pg_search matching Elasticsearch relevance quality with comparable query latency. A SaaS platform with roughly 100k documents saw simple queries running at 5–10ms — fast enough for production without a dedicated search cluster.
What happens to my Elasticsearch mappings when I migrate?
You re-create the equivalent configuration as BM25 index definitions in Postgres — conceptual remapping, not a data migration. The analysis pipeline (tokenisation, stemming) is configured at index creation time.
How does temporal hybrid search prevent stale RAG results?
It adds a time-range pre-filter to the BM25 and vector CTEs before the RRF merge, limiting the candidate document pool to recently published content. Relevance ranking operates over recent data only, preventing outdated documents from surfacing in AI-generated responses.
Can I run hybrid search (BM25 + vector) in Postgres without any extensions?
No. Native Postgres full-text search doesn’t support BM25 scoring — you need either pg_search or pg_textsearch. pgvector is needed for vector search. RRF itself requires no extension and is implemented in pure SQL using CTEs.
How long does an Elasticsearch migration to Postgres realistically take?
For a small-to-medium dataset with a well-instrumented team, the four phases typically run 4–8 weeks. Simple keyword search applications cut over quickly; applications with complex relevance tuning or faceted search require longer validation periods.
Does Postgres BM25 support the same language analysers as Elasticsearch?
Modern Postgres BM25 extensions support the standard tokenisation and stemming analysers that cover most use cases. Elasticsearch has a broader ecosystem of custom analysers. Teams with highly customised analysis pipelines should evaluate their specific configuration before committing.
What does the RRF SQL pattern actually look like in practice?
Three SQL CTEs: one for BM25 results, one for vector results, and a third that joins both rank lists, applies the 1/(k+rank) formula, and orders by combined score. The complete pattern fits in roughly 20–30 lines of SQL.
