HyDE: Precise Zero-Shot Dense Retrieval without Relevance Labels

Dense retrieval works by encoding both queries and documents into vectors in a shared embedding space, then finding documents whose vectors are closest to the query vector. The similarity is measured by the inner product:

The fundamental difficulty is this: queries and documents are structurally different. A query like "how long does wisdom tooth removal take?" is short, interrogative, and lacks context. A relevant document passage is long, declarative, and packed with details: "Wisdom tooth extraction typically takes 30 to 60 minutes depending on the complexity of the case..."

Training two encoders to map these structurally different inputs into a shared space where their inner product captures relevance requires massive amounts of labeled query-document pairs — datasets like MS-MARCO with hundreds of thousands of human judgments. Without such labels, the system has no gradient signal for what "relevant" means.

Unsupervised approaches like Contriever learn document-document similarity through contrastive learning, but they still struggle when the input is a query rather than a document — the gap between the question form and the passage form remains.

HyDE's key insight is deceptively simple: instead of trying to bridge the gap between queries and documents, transform the query into a document first, then search in document-only space.

The system works in two steps. First, an instruction-following language model (like InstructGPT) receives the query along with a task-specific instruction such as "Write a passage that answers the question" and generates a hypothetical document. This document is fictional — it may contain factual errors and hallucinations — but it captures the pattern of what a relevant answer looks like: the right vocabulary, the right structure, the right level of detail.

Second, an unsupervised contrastive encoder (like Contriever) encodes this hypothetical document into a vector. This vector now lives in the same space as all the real documents in the corpus. The search becomes document-to-document matching — a problem that contrastive learning handles well without any relevance labels.

The formal setup begins with a key design choice: use a single encoder for everything. Instead of separate query and document encoders, HyDE uses only the document encoder from an unsupervised contrastive model. Denote this encoder $f$:

Now consider an instruction-following language model $g$. Given a query $q$ and a task-specific instruction $\text{INST}$ (such as "write a passage to answer the question"), $g$ generates a hypothetical document:

The hypothetical document is then encoded by $f$ to produce the query vector. To reduce variance, HyDE samples $N$ hypothetical documents and averages their embeddings:

The inner product is then computed between $\hat{v}_q$ and all document vectors $\{f(d) \mid d \in D\}$. The most similar real documents are returned. The encoder $f$ acts as a lossy compressor: it maps the hypothetical document into a dense vector, discarding specific (possibly hallucinated) details and retaining only the semantic neighborhood. This is what grounds the retrieval — the vector identifies the right region of embedding space, even when the generated text is factually wrong.

Think of it as a postal system: the hypothetical document is an envelope with the right zip code but wrong street address. The encoder extracts the zip code (the semantic neighborhood) and delivers you to the right area, where the real documents live.

A critical design feature of HyDE is how it adapts to different retrieval tasks without any training. The same underlying models (InstructGPT + Contriever) serve web search, scientific fact-checking, financial QA, and multilingual retrieval — all by changing only the text instruction given to the language model.

For web search: "Please write a passage to answer the question." For scientific claims: "Please write a scientific paper passage to support/refute the claim." For financial QA: "Please write a financial article passage to answer the question." For Korean retrieval: "Please write a passage in Korean to answer the question in detail."

This is the power of instruction-following models: the same frozen model can generate documents in the right style for each task simply by reading a different instruction. No fine-tuning, no task-specific training data, no architectural changes.

HyDE was evaluated across 11 query sets covering web search (TREC DL19, DL20), low-resource retrieval (6 BEIR datasets), and multilingual retrieval (Mr.TyDi in Swahili, Korean, Japanese, and Bengali).

On web search, HyDE improved Contriever's nDCG@10 from 44.5 to 61.3 on DL19 — a 38% relative improvement. It matched or exceeded the performance of ContrieverFT, a version of Contriever fine-tuned on MS-MARCO's hundreds of thousands of labeled pairs.

On low-resource tasks, HyDE outperformed BM25 (the classical lexical baseline) on 5 out of 6 datasets, and outperformed supervised models like DPR and ANCE that were fine-tuned on MS-MARCO.

On multilingual retrieval, HyDE improved mContriever across all four languages, even though GPT-3's generation quality varies across languages. HyDE outperformed fine-tuned models like mDPR and mBERT, though it trailed the fully fine-tuned mContrieverFT.

The authors tested HyDE with three instruction-following language models of different sizes: FLAN-T5 (11B parameters), Cohere (52B), and InstructGPT (175B). The results revealed a clear scaling trend: larger language models produced better hypothetical documents, which led to better retrieval.

On DL19, nDCG@10 climbed from 48.9 (FLAN-T5) to 53.8 (Cohere) to 61.3 (InstructGPT). Even the smallest model improved over the Contriever baseline (44.5), confirming that the HyDE approach works across scales — but benefits significantly from stronger generation.

This has a practical implication: as language models continue to improve, HyDE will automatically improve too, without any retraining of the retrieval system.

At first glance, it seems paradoxical: the language model generates text that is factually unreliable, yet the retrieval results are precise. The resolution lies in the encoder's dense bottleneck.

When the encoder maps a hypothetical document to a vector, it compresses a variable-length text into a fixed-size vector (typically 768 dimensions). This compression is lossy by design — it cannot preserve every detail. What it does preserve is the overall semantic neighborhood: the topic, the domain, the type of content.

Specific factual claims — "wisdom tooth removal takes exactly 47 minutes" — are details that get averaged out in the embedding. But the general semantic signal — "this is about dental procedures and surgery duration" — is robustly captured. The retrieval finds documents in the right neighborhood, regardless of the exact (hallucinated) numbers.

This is why HyDE works: the bottleneck acts as a filter between hallucination and retrieval. It keeps the signal (semantic relevance) and discards the noise (specific fabrications).

The authors close the paper with a provocative reflection. Traditional retrieval systems model relevance as a numerical score — a learned function that maps query-document pairs to numbers. HyDE replaces this with language generation: the concept of relevance is captured by a language model producing an example of what a relevant document looks like.

This raises a deep question: is numerical relevance scoring just a statistical artifact of language understanding? If a strong enough language model can generate text that captures relevance patterns, and a simple encoder can ground that text in a corpus, do we need to learn relevance scores at all?

The authors do not claim to have the answer, but the results suggest that as language models grow stronger, the need for explicit relevance modeling may diminish. The boundary between information retrieval and natural language understanding is blurring.

HyDE's most lasting contribution is not a specific model but a conceptual reframing. It showed that retrieval does not require explicitly modeling relevance scores. Instead, a language model that understands relevance implicitly — through its ability to generate relevant-looking text — can be combined with simple unsupervised matching to achieve state-of-the-art results. This insight has reshaped how we think about the interface between language generation and information retrieval.