Building a Hybrid RAG Pipeline with Reciprocal Rank Fusion

Why one retrieval strategy is never enough

Retrieval-augmented generation stands or falls with what the retriever hands the model. Ask a knowledge base a question, and the answer is only as good as the chunks that precede it - and enterprise questions come in two shapes that no single search strategy handles well.

Semantic questions - "how does billing handle proration?", "what did we decide about vendor lock-in?" - are about meaning. Vector search shines here: it embeds the query and the document chunks into the same space and retrieves the nearest neighbors, so a chunk written in completely different words can still surface. But pure vector search is unreliable for the other shape of question: exact identifiers. Product codes, error strings, regulation numbers, people's names - embeddings blur them, and the nearest neighbor to an obscure error code is often some other, unrelated code.

Keyword questions are the mirror image. PostgreSQL full-text search nails exact tokens and rare terms, but it misses paraphrases entirely: ask for "staff augmentation" and it will not find the document that says "contractor resourcing".

Real corporate documents attract both kinds of questions, often within the same session. Simmetric Chat therefore refuses to choose. Every retrieval runs two legs in parallel - vector similarity and PostgreSQL full-text search - and fuses the results with Reciprocal Rank Fusion (RRF) before anything reaches the model. This article walks through that pipeline: the two legs, the fusion math, the ingestion path that feeds them, and the citation layer that makes answers auditable.

Two legs, one query

When a chat needs knowledge, the ReAct agent invokes its rag_search skill, and hybrid search starts two independent retrievals at the same time:

• The vector leg leaves the server and calls the collector, the service that owns the vector store. The collector embeds the query with the same embedding model used at ingestion time and returns the nearest chunks from the workspace's collection. The vector store itself is pluggable - LanceDB by default (file-based, local, air-gap friendly), or Qdrant, pgvector, or Chroma for deployments that prefer a server-side index.
• The full-text leg never leaves the server. It queries PostgreSQL directly, matching the query against a concatenated full-text vector column on the chunk table, backed by a GIN index and ranked by lexical relevance. Tokenization is locale-aware across seven languages, so Italian and German documents are searchable with their own word rules rather than English approximations.

Collections are per-workspace, which is what keeps knowledge siloed: a chat in workspace A retrieves from workspace A's chunks and nothing else. Running both legs on every query costs one extra round trip and buys coverage neither leg has alone.

The failure mode is deliberately redundant too. A guard compares the embedding model configured today with the one the workspace's documents were embedded with. On a mismatch - after a model change, for instance - the vector leg is skipped entirely and retrieval degrades to full-text only, rather than serving similarity scores from incomparable embedding spaces. The chat keeps working, and the operator gets an explicit re-index path in the admin panel instead of a silent quality regression.

Reciprocal Rank Fusion, concretely

The two legs return ranked lists measured on incommensurable scales: cosine similarity on one side, lexical relevance on the other. Averaging the raw scores would mean calibrating those scales against each other - brittle, and different for every embedding model and every corpus. Simmetric Chat fuses ranks instead of scores, using Reciprocal Rank Fusion:

fused(chunk) = sum over legs of 1 / (k + rank),  with k = 60

Every chunk accumulates one contribution per leg in which it appears, so a chunk ranked first on both legs scores roughly twice a chunk ranked first on only one. The constant k dampens the influence of the very top ranks - with k = 60, the difference between rank one and rank five on a single leg is a small delta, and consensus across legs is what actually moves a chunk up the merged list.

Two engineering details make the fusion production-grade rather than textbook:

• A source set per chunk records whether it was found by the vector leg, the full-text leg, or both. A chunk retrieved by both legs is the strongest corroboration the pipeline can produce, and the downstream formatting preserves that signal instead of flattening it.
• A deterministic tiebreaker - fused score descending, then document identifier, then chunk position - guarantees the same query returns the same order on every replica. When someone asks "why did it cite this paragraph?", the answer has to be reproducible; nondeterministic fusion would make the question unanswerable.

From documents to searchable chunks

Fusion is only half the pipeline; the other half happens at ingestion, when an uploaded document becomes chunks that both legs can search. The collector parses PDF, Word, Excel, PowerPoint, Markdown, CSV files and YouTube transcripts, splits the text with a recursive character splitter (1,000 characters per chunk, 200 characters of overlap so sentences crossing a boundary are not lost), embeds each chunk, and writes the vectors to the workspace's collection.

Crucially, the chunk text also lands in PostgreSQL, where the server maintains the full-text search vector that the FTS leg queries. The collector never touches the relational database - a hard service boundary that keeps the parse-and-embed pipeline swappable and leaves the search index owned by the server that queries it.

Version 0.22 made retrieval precise in a second dimension: metadata filtering. The agent can now scope a rag_search call to document types (PDF, Markdown, TXT, CSV, DOCX, XLSX) and to ingest-date windows, which fixes the classic "2023 documents answering 2025 questions" failure. Filters thread through both legs - as true pre-filters on stamped ingest metadata in the vector stores, and as parameterized SQL predicates on the full-text side. A server-side metadata backstop then re-checks every fused result against the authoritative documents table, so filter correctness does not depend on each vector provider implementing predicates perfectly. Legacy vectors ingested before the metadata stamping simply drop out while a filter is active, and regain filterability through an admin-triggered re-embed.

Answers that show their work

Retrieval quality is only half the story; the other half is honesty about what the answer is grounded on. Citations flow through the same streaming channel as tokens: citation badges render inline in the response, each carrying the source document, the chunk, and a relevance score, and each linking to the source so a reader can verify the claim in one click.

Freshness discipline lives here too. After the fused results are deduplicated, a grounding filter caps citations per document at the top two by relevance score and drops chunk citations whose text shares no meaningful lexical overlap with the final answer. The gate is deliberately conservative - wiki pages, archive results, and very short answers are always kept - but it eliminates the failure mode where a reply says "no further information" while displaying "Sources (5)" drawn from a single marginally-related document. The "no supporting evidence" disclaimer is judged against the same filtered set, so the disclaimer and the source list always describe the same reality.

Choosing and tuning the stack

The pipeline is deliberately configuration-driven, and the defaults are the privacy-first ones:

• Vector store: LanceDB by default - file-based, stored locally, no external service required, fully air-gap compatible. Qdrant, pgvector, and Chroma are one environment change away when a deployment outgrows it; the provider is selected at boot.
• Embeddings: local transformers (384 dimensions, entirely offline) by default, or OpenAI embeddings when a deployment accepts the outbound call. Switching the embedding model requires a re-index, which the admin panel drives; the mismatch guard keeps the system safe in the interim.
• Reranking: optional. A reranker can over-fetch and reorder the fused results when answer quality matters more than latency.
• Fusion constant: k = 60 ships as the default; it is a single constant in the hybrid search service if your ranking distributions call for different damping.

The full data-flow walkthrough - the server-collector boundary, the storage layout behind each vector provider, and the streaming citation events - is documented in the architecture guide.

Read the full guide in the docs →