Hybrid Search

Intent

Combine keyword search and vector search in a single query, so you miss neither synonyms nor exact terms. It is for search and RAG systems where users mix natural language with codes, acronyms, and proper names.

Problem

Pedro runs search for an online auto parts store. Vector search did well on “brake pads for a compact car”: it found pads compatible with several models. But it failed badly on “pad 4D0698151”: a part number’s vector carries no meaning, and the search returned other brands’ pads, none with that code.

The old keyword search got the code right and the natural language wrong. Each method fails where the other succeeds, and customers write both ways, often in the same sentence: “front pad 4D0698151 for A4”.

The same pattern shows up outside e-commerce. In the internal chat, “MV when WFH” needs lexical search for the acronyms and vector search for the remote-work paraphrase. On a Brazilian electronic invoice (NF-e), the 44-digit access key is only found by exact match. An embedding approximates meaning; it does not memorize identifiers.

The question this pattern answers: how does one search get both the meaning of a sentence and the exact term right?

Solution

Run both searches over the same set of documents and combine the results:

• Lexical search (BM25 or equivalent): scores by term occurrence. Gets codes, acronyms, names, and rare words right.
• Vector search: scores by closeness in meaning. Gets synonyms, paraphrases, and poorly phrased questions right.

The most common combination is rank fusion (Reciprocal Rank Fusion, Cormack et al., 2009): each document earns points for its position in each list, and the sum decides the final order. It does not require calibrating different score scales. Exact filters (brand, category, stock) apply to both searches.

Structure

flowchart LR
  Q(["1. Query"]):::io --> L["2a. Lexical search"]:::tool
  Q --> V["2b. Vector search"]:::tool
  I[("Index with text and vectors")]:::store --> L
  I --> V
  L --> F{"3. Rank fusion"}
  V --> F
  F --> R["4. Combined top N"]:::tool
  R --> OUT(["5. Results or context for the model"]):::io

1. The query arrives as the user wrote it.
2. Both searches run in parallel, with the same filters.
3. Fusion combines the lists by each document’s position.
4. The top N combined results move on.
5. They become the search results or a RAG’s context.

Worked example

Query: “front pad 4D0698151 for A4”. After normalizing the code (4d0698151), the two lists of 50 look like this at the top:

lexical                         vector
1. sku-4D0698151-a  (code)      1. sku-other-pads-a4          (meaning: pad + A4)
2. sku-4D0698151-b  (code)      2. sku-front-pad-compact
3. install-manual-4D06          3. sku-4D0698151-a

RRF with constant 60. The document at rank 0 gets 1/60, at rank 2 gets 1/62. sku-4D0698151-a appears on both lists, sums the two terms, and rises to first place combined. The vector’s “similar” pads, without the code, sit lower.

The mixed sentence stops being a tie between two bad systems. Each contributes what it knows, and fusion promotes whoever showed up on both.

Pseudocode

function hybrid_search(query, filters, k=10):
    parallel {                                                       # 2
        lexical = index.bm25(query, filters, top=50)
        vector  = index.knn(embed(query), filters, top=50)
    }
    # 3. Reciprocal Rank Fusion
    scores = {}
    for rank, doc in enumerate(lexical): scores[doc] += 1 / (60 + rank)
    for rank, doc in enumerate(vector):  scores[doc] += 1 / (60 + rank)
    ranked = sort(scores, desc)
    trace.span("hybrid", lexical_top=lexical[:3], vector_top=vector[:3])
    return ranked[:k]                                                # 4

Real-world scenarios

• E-commerce semantic search. “Running shoes for rainy days” calls for vectors; “SKU 88213” calls for lexical. Hybrid search handles both in the same search box.
• Internal knowledge base chat. Internal acronyms and system names are found by lexical search; vague questions, by vector search.
• Legal assistant. Statute and article numbers need exact matches; legal concepts, closeness in meaning.

Applicability

• Users mix natural language with codes, acronyms, and names.
• Vector search alone misses exact terms, or lexical search alone misses paraphrases.
• The domain has its own vocabulary that embedding models do not know well.

When not to use

• Queries are always by identifier. Use an exact lookup in the database.
• The material is small and fits in the context. See RAG for Small Data.
• Vector search already performs in the eval, with no failures on exact terms. One search is simpler.

Hybrid search is the base of RAG when the vocabulary mixes both worlds. It does not solve fine ordering: if the right result sits at position 20 of the combined list, the next step is Reranking. If the query arrives vague, the previous step is Query Rewriting.

How to implement

1. Build a search eval with real queries and the documents that should appear, including queries with codes and with paraphrases.
2. Index the same content for both searches, with the same identifiers and metadata.
3. Run both in parallel, with identical filters.
4. Combine with rank fusion; adjust with weights later if needed.
5. Measure recall@k for each search alone and for the combination.
6. Add Reranking if the top results still need precision.

Normalize identifiers before indexing and before searching. 4D0-698-151, 4D0698151, and 4d0 698 151 need to become the same token. Without that, lexical search misses the case it was supposed to hit.

Common mistakes

• Different filters. Lexical search respects category = pad and vector search does not. The combined result leaks a product from another category. The filter goes into both, or into fusion afterwards, never into only one.
• Adding raw scores. BM25 and cosine do not live on the same scale. RRF uses position so you do not have to calibrate that on day one.
• An eval with only natural sentences. You will conclude that vectors are enough. Include codes, customs numbers, invoice keys, internal acronyms.
• Two contents, two ids. If the same product has a different id in the text index and in the vector index, fusion never adds the points.

Variations

• Weighted fusion. One weight per list, calibrated in the eval. Useful when one query type dominates.
• Routing by query type. Queries that look like codes go only to lexical search. Cheaper, but it needs a reliable rule.
• Learned sparse vectors. Models that produce vectors with per-term weights, combining both ideas in one index.

Cost, latency, and failure modes

How to evaluate

• Recall@k for lexical, vector, and hybrid, split by query type (code, natural, mixed).
• MRR: the position of the first correct result.
• Queries with no relevant result, by type.

{ "query": "front pad 4D0698151 for A4", "relevant": ["sku-4D0698151-a", "sku-4D0698151-b"], "type": "mixed" }

If hybrid does not beat lexical on type=code nor vector on type=natural, fusion is broken (ids, filters, or normalization), not “a domain that does not combine”.

In production

• A span with each list’s top positions and the final list.
• A dashboard of searches with no click or no result, by query type.
• A weekly sample of queries with poor results, which become eval cases.

The span with both lists is what lets you see, on a bad question, whether lexical, vector, or fusion failed.

Security

Permission filters go into both searches, before fusion. Filtering afterwards, on the combined list, is late: ranking already mixed documents the user cannot see, and a bug in post-filtering leaks a passage.

Indexed content is still untrusted data at RAG time. Hybrid search only changes who enters the list, not the fact that a passage can carry an instruction. See Prompt injection in practice.

Pros and cons

Code

Both tabs run lexical and vector search in parallel and combine them with Reciprocal Rank Fusion.

import numpy as np
from rank_bm25 import BM25Okapi
from sentence_transformers import SentenceTransformer

embedder = SentenceTransformer("sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2")


def normalize(text: str) -> list[str]:
    return text.lower().replace("-", "").split()  # "4D0-698-151" and "4d0698151" become equal


class HybridIndex:
    def __init__(self, docs: list[dict]):
        self.docs = docs
        self.bm25 = BM25Okapi([normalize(d["text"]) for d in docs])
        self.vectors = embedder.encode([d["text"] for d in docs], normalize_embeddings=True)

    def search(self, query: str, k: int = 10, pool: int = 50, c: int = 60) -> list[dict]:
        lexical = np.argsort(-self.bm25.get_scores(normalize(query)))[:pool]  # 2a
        q = embedder.encode(query, normalize_embeddings=True)
        vector = np.argsort(-(self.vectors @ q))[:pool]  # 2b
        scores: dict[int, float] = {}
        for ranking in (lexical, vector):  # 3. Reciprocal Rank Fusion
            for rank, i in enumerate(ranking):
                scores[int(i)] = scores.get(int(i), 0.0) + 1.0 / (c + rank)
        best = sorted(scores, key=scores.get, reverse=True)[:k]
        return [self.docs[i] for i in best]  # 4

import { Pool } from "pg";

const db = new Pool();

// 2 and 3 in one query: both searches as CTEs and rank fusion in SQL
export async function hybridSearch(query: string, queryEmbedding: number[], category: string, k = 10) {
  const { rows } = await db.query(
    `WITH lexical AS (
       SELECT id, ROW_NUMBER() OVER (ORDER BY ts_rank_cd(tsv, q) DESC) AS r
       FROM products, plainto_tsquery('english', $1) q
       WHERE tsv @@ q AND category = $3
       ORDER BY r LIMIT 50
     ),
     vector AS (
       SELECT id, ROW_NUMBER() OVER (ORDER BY embedding <=> $2::vector) AS r
       FROM products WHERE category = $3
       ORDER BY embedding <=> $2::vector LIMIT 50
     )
     SELECT p.id, p.title, p.sku,
            COALESCE(1.0 / (60 + l.r), 0) + COALESCE(1.0 / (60 + v.r), 0) AS score
     FROM products p
     LEFT JOIN lexical l ON l.id = p.id
     LEFT JOIN vector v ON v.id = p.id
     WHERE l.id IS NOT NULL OR v.id IS NOT NULL
     ORDER BY score DESC LIMIT $4`,
    [query, JSON.stringify(queryEmbedding), category, k],
  );
  return rows; // 4
}

Check your understanding