Dense Retrieval

Dense retrieval is an information-retrieval method that encodes queries and documents into dense vector embeddings and ranks results by vector similarity rather than lexical term overlap. A learned bi-encoder maps text into a shared semantic space so that conceptually related items are close even without shared keywords. It underpins semantic search and retrieval-augmented generation, often paired with approximate nearest-neighbour indexes for scale.

Semantic Classification

Content

Compositional Relationships (Components)

SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:hasPart ir:BiEncoderQueryEncoder))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:hasPart ir:BiEncoderPassageEncoder))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:hasPart ir:DenseVectorIndex))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:hasPart ir:ApproximateNearestNeighbourIndex))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:hasPart ir:ContrastiveLearningObjective))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:hasPart ir:HardNegativeMiningStrategy))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:hasPart ir:OfflineIndexingPipeline))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:hasPart ir:OnlineQueryProcessor))

Dependency Relationships

SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:requires ir:TransformerArchitecture))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:requires ir:ContrastiveLearning))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:requires ir:VectorDatabase))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:requires ir:ApproximateNearestNeighbourSearch))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:dependsOn ir:EmbeddingModel))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:dependsOn ir:HNSWIndex))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:dependsOn ir:LabelledRelevanceData))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:dependsOn ir:GPUComputeInfrastructure))

Capability Relationships

SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:enables ir:RetrievalAugmentedGeneration))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:enables ir:SemanticSearch))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:enables ir:OpenDomainQuestionAnswering))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:enables ir:CrossModalRetrieval))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:enables ir:AgenticRAGRetrieval))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:enables ir:ZeroShotSemanticMatching))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:enables ir:HybridRetrievalPipeline))

Implementation Relationships

SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:implements ir:BiEncoderArchitecture))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:implements ir:ContrastiveLearningObjective))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:implements ir:MaximumInnerProductSearch))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:implements ir:OfflineIndexBuildAndOnlineQuery))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:implements ir:HardNegativeMiningLoop))

Reduction Relationships

SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:reducesTo ir:EmbeddingSearch))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:reducesTo ir:VectorSimilaritySearch))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:reducesTo ir:NeuralTextEncoding))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:reducesTo ir:ApproximateNearestNeighbourSearch))
SubClassOf(ir:DenseRetrievalSystem
  ObjectSomeValuesFrom(ir:reducesTo ir:SemanticSimilarityMatching))

About

Dense retrieval emerged as a transformative approach to Information Retrieval in the years following the widespread adoption of pre-trained transformer models. Prior to 2020, the dominant paradigm for first-stage retrieval remained the sparse lexical methods descended from TF-IDF and BM25, which score documents by exact or morphologically normalised term overlap. Despite the remarkable capabilities of BERT and similar models for natural language understanding tasks, adapting them to large-scale retrieval was non-trivial: naive application of cross-encoder scoring over query-document pairs requires forward-passing through a transformer for every (query, candidate document) pair at retrieval time, which is computationally infeasible for corpora of millions to billions of documents. The bi-encoder architecture — separately encoding queries and documents into a shared vector space, then retrieving by vector similarity using a pre-built approximate nearest-neighbour index — decoupled the expensive encoding computation from the retrieval operation, making neural retrieval tractable at corpus scale.

The landmark demonstration of dense retrieval’s superiority over lexical methods came with Dense Passage Retrieval (DPR, Karpukhin et al., EMNLP 2020), which showed that a dual-BERT-encoder trained with in-batch negatives on NaturalQuestions substantially outperformed BM25 on open-domain question answering benchmarks, achieving 78.4% Top-20 retrieval accuracy on NaturalQuestions versus 59.1% for BM25. This result validated dense retrieval as a genuinely superior approach for semantic retrieval tasks and catalysed an explosion of follow-on research. Key advances included ANCE (Xiong et al., ICLR 2021), which proposed asynchronous updates of the hard negative index from the current model checkpoint to avoid training on stale negatives — a fundamental flaw in static hard-negative strategies; the Retrieval-Augmented Generation framework (Lewis et al., NeurIPS 2020), which demonstrated that combining dense retrieval with a generative language model produced superior open-domain question answering; and ColBERT (Khattab and Zaharia, SIGIR 2020), which introduced late interaction scoring to bridge the quality gap between bi-encoder dense retrieval and the more powerful but computationally expensive cross-encoder reranking paradigm.

By 2024–2026, dense retrieval had transitioned from a research prototype into production-grade infrastructure deployed across enterprise Semantic Search, Retrieval-Augmented Generation for Large Language Models, and Agentic RAG architectures. The field’s primary evaluation frameworks — BEIR (18 heterogeneous zero-shot retrieval datasets), MTEB (Massive Text Embedding Benchmark covering 56+ tasks), and MS MARCO passage ranking — drove rapid model improvements. Instruction-tuned embedding models such as E5-large-instruct, E5-Mistral, and NVIDIA’s NV-Embed-v2 demonstrated that prompting an LLM backbone for embedding generation without explicit contrastive training could surpass specialist bi-encoders on diverse retrieval tasks. Unified models such as BGE-M3 (BAAI, 2024) integrated dense bi-encoder retrieval, lexical sparse retrieval, and ColBERT-style multi-vector late interaction into a single model checkpoint supporting 100+ languages with 8,192-token input context, representing the convergence of previously separate retrieval paradigms. The MTEB English leaderboard in mid-2026 is led by NV-Embed-v2 (72.31 average across 56 English tasks) and Qwen3-Embedding-8B (70.58 on multilingual MTEB), reflecting the shift toward large-parameter embedding models that leverage pre-training from frontier LLM families.

Formal Architecture and Training Procedure

Bi-Encoder Architecture: The canonical dense retrieval system consists of two components: an offline indexing pipeline and an online query processing pipeline.

• Query Encoder E_Q: maps query text q to a d-dimensional dense vector vq = E_Q(q) ∈ ℝ^d. At inference time, E_Q is a transformer encoder (e.g., BERT-base, 768 dimensions) that produces the [CLS] token representation or mean-pooled token representations as the query vector.

• Passage Encoder E_P: maps passage text p to a d-dimensional dense vector vp = E_P(p) ∈ ℝ^d. In the original DPR formulation, E_Q and E_P are separate BERT models. In later models (e.g., BGE, E5), a single encoder handles both queries and passages, conditioned on an instruction prefix.

• Similarity Function: retrieval rank is determined by sim(vq, vp) = vq · vp (dot product) or cosine(vq, vp) = vq · vp / (||vq|| ||vp||). Dot-product similarity is standard for trained bi-encoders; cosine similarity is preferred for embedding models used across diverse tasks.

• Offline Index: all passages in the corpus are encoded by E_P, producing vectors {vp1, vp2, …, vpN}. These are stored in an Approximate Nearest Neighbour Search index (FAISS IVF-flat, HNSW Index, ScaNN) that supports sub-linear-time k-nearest-neighbour queries.

• Online Retrieval: given query q, compute vq = E_Q(q), query the ANN index to retrieve top-k passages by sim(vq, vpi), return corresponding passages ranked by similarity.

  Training Objective (Contrastive Learning): Dense retrieval models are trained with contrastive objectives over (query, positive passage, negative passages) triplets. The InfoNCE / in-batch negative loss for a batch of B query-passage pairs is:

  L = -1/B Σᵢ log [ exp(sim(vqi, vpi+) / τ) / (exp(sim(vqi, vpi+) / τ) + Σⱼ≠ᵢ exp(sim(vqi, vpj) / τ)) ]

  where τ is a temperature hyperparameter, vpi+ is the positive passage embedding, and {vpj}ⱼ≠ᵢ are in-batch negatives (other passages in the same training batch). The temperature τ (typically 0.01–0.1) controls the sharpness of the distribution; lower temperatures enforce tighter clustering of positive pairs.

  Hard Negative Mining Strategies:

• Random Negatives: passages sampled uniformly from the corpus. Simple but often too easy — the model quickly saturates on trivial negatives.

• In-Batch Negatives: other positive passages in the same training mini-batch serve as negatives. Efficient (no extra encoding needed) but limited by batch size — large batches (256–4096) with gradient checkpointing are standard.

• BM25 Hard Negatives: passages retrieved by BM25 for each query that are lexically similar but not relevant. Forces the model to distinguish semantic from lexical relevance. Used in DPR training.

• ANCE (Dynamic Hard Negatives): an asynchronous refreshing strategy (Xiong et al., 2021) that periodically re-encodes the entire corpus with the current model checkpoint and retrieves the top-k passages as hard negatives for the next training phase. Requires maintaining a continuously updated ANN index during training but produces significantly better models than static hard negatives.

• Knowledge Distillation Negatives: teacher cross-encoder scores over (query, passage) pairs are used as soft labels for distillation training of the bi-encoder student (RocketQAv2, AR2, DRAGON+). Achieves near cross-encoder quality with bi-encoder inference efficiency.

Retrieval Architectures and Model Families

Dense retrieval has diversified into several distinct architectural families, each with different quality-efficiency trade-offs:

Bi-Encoder (Single-Vector Dense Retrieval) The foundational architecture introduced by DPR. Each query and passage maps to a single fixed-dimensional vector; retrieval is maximum inner-product search. Representative models: DPR (Karpukhin et al., 2020), ANCE (Xiong et al., 2021), E5 family (Wang et al., 2022–2024), BGE (BAAI, 2023–2024), Contriever (Izacard et al., 2022), GTR (Ni et al., 2022). Inference cost: one transformer forward pass per query at retrieval time; passage encoding offline. Storage: one d-dimensional vector per passage (e.g., 768 × 4 bytes = 3KB per passage, 3GB for 1M passages). Query latency: 2–10ms for ANN search over 100M passages.

Late Interaction (Multi-Vector Dense Retrieval) ColBERT (Khattab and Zaharia, 2020) computes a matrix of token-level embeddings for both query and passage, scoring similarity as the sum of maximum inner products between query tokens and all passage tokens (MaxSim operator). Retains fine-grained token-level alignment similar to cross-encoder attention while preserving offline passage pre-computation. ColBERTv2 (Santhanam et al., 2022) adds residual compression to reduce storage by ~10×, with PLAID engine achieving 2.5–7× GPU speedup. Storage cost: ~600KB per 200-token document vs 3KB for bi-encoder — roughly 200× higher, addressable with quantisation. Production deployment via RAGatouille Python library. Jina-ColBERT-v2 (2024) extends ColBERT to multilingual and long-context retrieval.

Instruction-Tuned LLM Embeddings LLM-scale embedding models (7B–8B parameters) that leverage instruction following to produce task-conditioned embeddings without explicit contrastive training. E5-Mistral-7B-Instruct (Wang et al., 2023) prepends a task instruction to the query, producing embeddings outperforming specialist bi-encoders across BEIR and MTEB. NV-Embed-v2 (NVIDIA, 2024) achieves MTEB English score of 72.31 — the open-weight English leader as of mid-2026 — using a 7.85B-parameter LLM backbone with a latent attention layer for pooling. Qwen3-Embedding-8B (Alibaba, 2025–2026) leads MTEB multilingual with 70.58 score, supporting 100+ languages and 32K input context with Q4 quantisation (5GB memory footprint).

Unified Sparse-Dense-Multi-Vector Models BGE-M3 (BAAI, 2024) represents the convergence of dense bi-encoder retrieval, learned sparse retrieval (SPLADE-style term weights), and ColBERT-style multi-vector late interaction in a single 0.6B-parameter model supporting 100+ languages with 8,192-token context under MIT/Apache 2.0 licence. In production RAG pipelines as of 2026, BGE-M3 + BGE-reranker-v2 is the dominant open-weight baseline, combining all three retrieval signals from one checkpoint and eliminating the need for separate BM25 and dense retrieval systems for moderate-scale deployments.

Generative Retrieval An emerging paradigm in which a sequence-to-sequence model (T5, GPT) directly generates document identifiers (DocIDs) or passage identifiers for a given query, bypassing the ANN index entirely. DSI (Tay et al., 2022) and NCI (Wang et al., 2022) demonstrated this approach at modest scale; scaling challenges remain significant for corpora exceeding millions of documents. Represents a long-term alternative to vector-index-based retrieval that may reshape the field post-2026.

Use Cases / Major Families

Retrieval-Augmented Generation (RAG) Dense retrieval is the dominant retrieval mechanism in Retrieval-Augmented Generation pipelines that ground Large Language Models in external knowledge, preventing hallucination by supplying relevant passages as context. The standard 2025–2026 production RAG architecture deploys dense retrieval as the primary semantic retrieval stage, optionally combined with BM25 in Hybrid Retrieval via Reciprocal Rank Fusion (RRF) to cover both semantic paraphrase and exact keyword matching, followed by Cross-Encoder Reranking to reorder the fused candidate set before LLM generation. Enterprise RAG systems at companies including Cohere, Microsoft (Azure AI Search), and Google (Vertex AI Search) all employ dense retrieval as their core semantic component.

Open-Domain Question Answering Dense retrieval was originally motivated by and evaluated on open-domain QA — answering natural language questions using a large unstructured text corpus (typically Wikipedia) without pre-specified candidate documents. DPR established the retriever-reader architecture (retriever provides top-k passages; reader generates the answer from retrieved context) that became the standard for open-domain QA. Subsequent systems including FiD (Fusion-in-Decoder, Izacard and Grave, 2021) and Atlas (Izacard et al., 2022) achieved state-of-the-art results on NaturalQuestions and TriviaQA by combining dense retrieval with sequence-to-sequence generation.

Enterprise Semantic Search Organisations deploy dense retrieval to surface semantically relevant internal documents, knowledge base articles, customer support tickets, and product specifications in response to natural language queries. Unlike BM25 keyword search, dense retrieval correctly handles queries phrased differently from document language — “onboarding procedure for new employees” retrieves documents discussing “staff induction process” — reducing search friction in enterprise knowledge management. Platforms including Elasticsearch (with knn retriever), Weaviate, Pinecone, Qdrant, and pgvector all implement dense retrieval as a core primitive, with hybrid BM25+dense as the recommended production configuration.

Agentic Tool Retrieval and Multi-Hop Reasoning In Agentic RAG architectures, LLM agents use dense retrieval as a named tool invoked via function-calling to fetch relevant passages for multi-hop reasoning tasks. The agent formulates semantic queries expressing the information need, dense retrieval returns top-k passages, and the agent synthesises a chain-of-thought reasoning trace before issuing follow-up queries. Dense retrieval’s ability to match paraphrastic and inferential queries makes it superior to BM25 for the diverse, conversational queries that agents generate during complex task execution.

Cross-Modal and Multimodal Retrieval The bi-encoder architecture naturally extends to multimodal settings by training query encoders on text and passage encoders on image, audio, or code tokens with contrastive alignment objectives. CLIP (Radford et al., 2021) demonstrated text-to-image dense retrieval; subsequent models including BLIP-2, ImageBind, and Nomic-Embed-Vision extend dense retrieval to cross-modal search for media retrieval, medical imaging, and code search. Cross Modal Retrieval systems built on dense retrieval enable text-to-image, image-to-text, and audio-to-text retrieval across enterprise media archives.

Code Search and Developer Tools Dense retrieval models trained on code-natural language pairs enable semantic code search — finding relevant code snippets from natural language descriptions — outperforming keyword-based approaches that require exact API and function name matching. GitHub Copilot’s context retrieval, DeepMind’s AlphaCode search, and Amazon CodeWhisperer use dense retrieval to surface relevant code examples as suggestions. Models such as UniXcoder and CodeBERT serve as the bi-encoder backbone for code-specific dense retrieval.

Academic Context

The intellectual origins of dense retrieval trace to latent semantic analysis (Deerwester et al., 1990), which first demonstrated that representing documents and queries in a low-dimensional latent semantic space improved retrieval over pure term matching. The transition from bag-of-words LSA to neural dense representations passed through distributed word embeddings (word2vec, Mikolov et al., 2013; GloVe, Pennington et al., 2014), which captured semantic word relationships in dense vector spaces. Sentence-level embeddings suitable for retrieval emerged with InferSent (Conneau et al., 2017), Universal Sentence Encoder (Cer et al., 2018), and SBERT (Reimers and Gurevych, 2019), which showed that bi-encoder architectures fine-tuned on natural language inference datasets produced semantically meaningful sentence vectors.

The foundational breakthrough was DPR (Karpukhin et al., EMNLP 2020), which established that transformer bi-encoders trained with in-batch negatives and hard BM25 negatives could substantially outperform BM25 on open-domain QA retrieval — a result that triggered the modern dense retrieval research agenda. REALM (Guu et al., ICML 2020) simultaneously demonstrated gradient-through-retrieval end-to-end training by differentiating through an approximate retrieval step. The RAG model (Lewis et al., NeurIPS 2020) showed that DPR-retrieved passages combined with sequence-to-sequence generation produced state-of-the-art open-domain QA, establishing the retriever-reader-generator paradigm that defines modern LLM grounding.

Key theoretical contributions include the ANCE paper (Xiong et al., ICLR 2021) establishing the importance of dynamic hard negative mining; the ColBERT family (Khattab et al., 2020–2022) developing late interaction as a quality-preserving alternative to cross-encoders; and the BEIR benchmark (Thakur et al., NeurIPS 2021) demonstrating that dense retrieval generalises poorly across domains in zero-shot evaluation, motivating the instruction-tuning and contrastive pre-training research that produced E5, BGE, and NV-Embed. The MTEB benchmark (Muennighoff et al., EMNLP 2023) extended evaluation to 56 tasks across 8 embedding use cases, providing the comprehensive evaluation framework that now drives model development.

Current Landscape (2026)

The dense retrieval landscape in 2026 is characterised by three converging trends: the dominance of large-parameter LLM-backbone embedding models, the prevalence of unified dense-sparse-multi-vector models, and the standardisation of hybrid dense-sparse retrieval as the production default.

MTEB Leaderboard Leadership (June 2026): NVIDIA NV-Embed-v2 (7.85B parameters, 4096 dimensions) leads the MTEB English leaderboard with an average score of 72.31 across 56 tasks. Qwen3-Embedding-8B (Alibaba, 2025) holds the MTEB multilingual leaderboard top spot with 70.58, supporting 100+ languages with 32K context and strong cross-lingual performance. Both models leverage LLM pre-training backbones (Mistral-7B-Instruct and Qwen3-8B respectively), demonstrating that instruction-following generalisation from LLM training transfers to high-quality embedding tasks. For open-weight cost-effective deployment, BGE-M3 (0.6B parameters, MIT licence) remains the production standard for multilingual RAG, handling dense, sparse, and multi-vector retrieval in a single checkpoint.

Self-Hosted Production Costs: A self-hosted Qwen3-Embedding-8B with Q4 quantisation on an A100 80GB PCIe (at approximately $1.04/hour)achievessustainedthroughputofapproximately60,000tokens/second,costingroughly$0.0097 per million tokens at 50% utilisation — competitive with commercial API embedding costs from Cohere (embed-v4) and OpenAI (text-embedding-3-large) while offering full data control.

Research Frontiers (2025–2026): The Revela framework (arXiv:2506.16552, 2026) proposes training dense retrievers via language modelling objectives rather than contrastive loss, claiming improved sample efficiency and domain generalisation. Work on “LLM-based retrievers” (arXiv:2604.03676) examines whether large generative models used as retrievers (generating DocIDs or passage identifiers) justify their computational overhead versus smaller bi-encoders — preliminary results suggest bi-encoders retain efficiency advantages for most production use cases. The RAR-b benchmark (arXiv:2404.06347) introduces reasoning-augmented retrieval tasks that require multi-step inference to determine relevance, pushing beyond semantic matching towards logical retrieval.

Hybrid Retrieval as Production Standard: By 2026, hybrid BM25 + dense retrieval via Reciprocal Rank Fusion is the production default across major vector database platforms (Elasticsearch 8.x native rrf retriever, Weaviate hybrid mode, Qdrant hybrid search, pgvector with tsquery integration). Enterprise deployments consistently report 10–15 point nDCG@10 improvements from hybrid retrieval over dense-only, and 20–30% reductions in LLM hallucination rates versus dense-only RAG, driving the adoption of Hybrid Retrieval as the minimum viable retrieval stack.

UK Context

Dense retrieval research has significant UK institutional presence, with leading groups at the Universities of Glasgow, Edinburgh, and Cambridge, and strong industrial activity from DeepMind and UK-based AI startups.

The University of Glasgow’s Information Retrieval group — led by Iadh Ounis, Craig Macdonald, and Nicola Tonellotto — has been a foundational contributor to neural information retrieval research. Glasgow develops and maintains PyTerrier (pyterrier.readthedocs.io), the primary Python research platform for Information Retrieval experimentation, which supports dense retrieval models including DPR, ColBERTv2, and SPLADE alongside BM25 baselines within a unified pipeline API. Glasgow’s research on dense retrieval includes dense retrieval with progressive distillation training (US Patent 12,111,837), zero-shot dense retrieval for low-resource languages, and learned sparse retrieval building on SPLADE. The group publishes consistently at SIGIR, ECIR, and CIKM.

The University of Edinburgh’s School of Informatics, particularly the Language Technology group (including collaborators of Dr. Jeff Dalton, a leading figure in conversational and agentic information retrieval), conducts research on dense retrieval for conversational search, multi-turn QA, and grounded generation. Edinburgh benefits from a Google DeepMind PhD scholarship programme supporting research at the intersection of dense retrieval, LLM grounding, and agentic systems.

UCL Computer Science’s Department of Computer Science teaches dense retrieval as part of COMP0084 Information Retrieval and Data Mining, covering bi-encoder architectures, contrastive learning, and RAG pipeline construction. UCL also contributes to BEIR and MTEB benchmark development through collaborations with Hugging Face researchers (Nils Reimers, a British-trained researcher, led SBERT and MTEB development).

UK industrial AI companies active in dense retrieval include Jina AI (with significant UK engineering presence, developing Jina-ColBERT-v2 and Jina-Embeddings-v3), Hugging Face UK operations, and DeepMind (Alphabet/Google) in London, where researchers contribute to retrieval-augmented systems including AlphaCode and Gemini’s grounding infrastructure. Northern English technology hubs — particularly Manchester’s AI ecosystem including Peak AI and the broader MediaCityUK digital cluster — deploy dense retrieval in operational knowledge management and media search systems. Sheffield’s Advanced Manufacturing Research Centre uses semantic document retrieval for engineering specification and regulatory compliance search, where dense retrieval’s semantic matching capability outperforms BM25 for natural language process queries against technical document libraries.

Future Directions (2026–2030)

Dense retrieval in the 2026–2030 period will evolve across several interconnected dimensions: increasing model parameter counts and LLM backbone scale, improved multi-task and instruction-conditioned embedding, multimodal extension, and tighter integration with generative language model pipelines.

Scaling Laws for Embedding Models: The transition from 110M-parameter (BERT-base) to 7B–8B-parameter (Mistral, Qwen3) embedding backbones has consistently improved MTEB scores, suggesting that scaling laws analogous to those governing generative LLMs also apply to embedding models. Research through 2028 will investigate whether 70B+ parameter embedding models provide further systematic gains or whether quality plateaus emerge for retrieval-specific tasks.

Retrieval-Augmented Reasoning: Beyond factual retrieval, future dense retrieval systems will be evaluated on tasks requiring multi-hop inference over retrieved passages — finding evidence chains across multiple documents to answer complex compositional questions. RAR-b and HotpotQA benchmarks motivate retriever architectures that can iteratively retrieve evidence guided by partial reasoning traces from an LLM, blurring the boundary between retrieval and reasoning.

Long-Context and Full-Document Retrieval: As LLM context windows expand to 1M+ tokens, the boundary between retrieval and in-context storage shifts: for small-to-medium corpora, fitting all documents in-context may become feasible, reducing the role of retrieval to corpus-scale (millions-to-billions of documents). Dense retrieval will remain essential at corpus scale but its role in small-collection RAG may diminish as context windows grow.

Personalised and Contextual Dense Retrieval: Production retrieval systems require per-user and per-session personalisation — surfacing documents relevant to a specific user’s role, history, and current context. Contextual bi-encoders that condition query encoding on user context vectors (derived from session history or user profiles) are an active research area, enabling personalised Semantic Search without per-user index sharding.

Unified Retrieval and Reranking: ColBERT-style late interaction already blurs the boundary between bi-encoder retrieval and cross-encoder reranking. Research through 2030 will likely produce “single-model” retrieval systems that simultaneously perform first-stage candidate selection and final relevance scoring within one forward pass, eliminating the multi-stage pipeline overhead while retaining scalability through progressive token pruning and hierarchical scoring.

Benchmark Datasets and Evaluation Metrics

Dense retrieval is evaluated on a small number of canonical benchmarks that have shaped model development over the past five years. Understanding these benchmarks is essential for interpreting model comparisons and selecting the right model for a given deployment context.

MS MARCO Passage Ranking (Bajaj et al., NeurIPS 2016): The largest and most widely used training and evaluation resource for dense retrieval, derived from Bing search engine logs. The MS MARCO passage ranking task contains approximately 530,000 training queries paired with 8.8 million candidate passages extracted from web pages, with sparse human relevance annotations (on average one relevant passage per query). The standard evaluation metric is MRR@10 (Mean Reciprocal Rank at 10): the mean across queries of the reciprocal of the rank position of the first relevant passage. DPR fine-tuned on MS MARCO achieves MRR@10 ≈ 0.37 on the development set; BGE-M3 achieves approximately 0.42; instruction-tuned models such as E5-Mistral-7B reach 0.44+. MS MARCO Document Ranking is a complementary task using full documents rather than passages, evaluated on MAP (Mean Average Precision).

TREC Deep Learning Track (2019–2023): An annual NIST evaluation using MS MARCO queries with multi-level relevance judgements (0=not relevant, 1=related, 2=highly relevant, 3=perfectly relevant) on a subset of queries, enabling nDCG@10 evaluation. TREC-DL 2019 passage ranking: BM25 achieves nDCG@10 ≈ 0.50; DPR ≈ 0.62; ColBERTv2 ≈ 0.73; cross-encoder rerankers ≈ 0.78. These scores establish the quality ladder from sparse lexical retrieval through bi-encoder dense retrieval to late interaction and cross-encoder reranking.

BEIR: Heterogeneous Zero-Shot Benchmark (Thakur et al., NeurIPS 2021): The most important benchmark for assessing dense retrieval generalisation, covering 18 datasets across 9 domains (biomedical, financial, legal, argument retrieval, question answering, fact checking, Wikipedia, news, and tweet retrieval). BEIR is zero-shot — models are evaluated on BEIR datasets without any domain-specific training data. The critical finding was that DPR-style bi-encoders trained on MS MARCO fail to generalise: DPR averages nDCG@10 of 0.348 across BEIR (below BM25’s 0.428). This exposed the over-fitting problem of early dense retrieval and drove the research agenda toward instruction-tuned and contrastively pre-trained models (Contriever, GTR, E5) that generalise without domain-specific fine-tuning. By 2024–2026, BEIR average nDCG@10 for the best open-weight models (BGE-M3, E5-Mistral, NV-Embed-v2) reaches 0.52–0.56, representing a 25% improvement over BM25.

MTEB: Massive Text Embedding Benchmark (Muennighoff et al., EACL 2023): The comprehensive evaluation framework for all text embedding tasks, covering 56 English tasks across 8 categories: retrieval (15 datasets including BEIR), clustering, pair classification, reranking, semantic textual similarity (STS), summarisation, and classification. The MTEB English leaderboard (huggingface.co/spaces/mteb/leaderboard) is the industry-standard ranking for embedding models. As of mid-2026, NV-Embed-v2 (NVIDIA, 7.85B params) leads with 72.31 average; Qwen3-Embedding-8B leads the multilingual MTEB with 70.58. MTEB has driven rapid model improvement by providing a single authoritative multi-task score that aggregates across diverse embedding applications.

NaturalQuestions and TriviaQA (Open-Domain QA Retrieval): The original DPR evaluation benchmarks for open-domain question answering retrieval, measuring Top-20 retrieval accuracy (proportion of queries for which the correct answer passage appears in the top-20 retrieved). DPR achieves 78.4% Top-20 on NaturalQuestions versus BM25’s 59.1%, establishing dense retrieval’s semantic advantage for factoid QA. FiD+DPR achieves 51.4% exact match on NaturalQuestions, establishing the retriever-reader paradigm’s state of the art.

Key Evaluation Metrics:

• MRR@k (Mean Reciprocal Rank): Mean of 1/rank for the first relevant result in the top-k; used for MS MARCO where typically one passage is annotated relevant per query.
• nDCG@k (Normalised Discounted Cumulative Gain): Weighted ranking quality metric accounting for graded relevance and positional discounting; standard for TREC-DL and BEIR.
• Recall@k: Proportion of queries for which at least one relevant document appears in the top-k results; critical for first-stage retrieval where a reranker operates on the top-k candidate set.
• Top-N Accuracy: Proportion of queries for which the answer-containing passage appears in the top-N; used for open-domain QA evaluation.

Computational Complexity and Scalability

Dense retrieval’s computational profile differs fundamentally from sparse lexical retrieval in ways that shape architecture choices at every scale, from laptop-scale prototyping to enterprise production deployment serving billions of documents.

Encoding Complexity: At indexing time, every document in the corpus must be processed by the passage encoder to produce its dense vector. For a BERT-base bi-encoder (110M parameters, 768-dimensional output), encoding a 128-token passage requires approximately 4.7 × 10^9 FLOPs on a single A100 GPU, achieving throughputs of 2,000–5,000 passages/second (batch size 512). For a 100M-document corpus at 200 tokens/passage, full corpus encoding requires approximately 5–25 GPU-hours on a single A100 — a one-time offline cost amortised over all subsequent queries. For LLM-scale embedding models (7B parameters), encoding throughput drops to approximately 200–500 passages/second, requiring 50–250 GPU-hours for the same corpus — making index construction a non-trivial infrastructure operation for very large models.

Index Storage: Dense vectors for a 100M-document corpus at 768 dimensions in float32 require 768 × 4 × 10^8 = 307 GB of raw vector storage, compared to 1–5 GB for a BM25 inverted index over the same corpus. Product quantisation (PQ) with 64 bytes per vector reduces this to approximately 6 GB (50× compression) at 1–5% quality loss; int8 quantisation reduces to 76 GB with minimal quality loss; binary quantisation reduces to 9.6 GB with 3–10% quality loss. HNSW Index adds a proximity graph overlay requiring 1.1–1.5× the raw vector storage for navigation links, totalling 330–460 GB for a 100M-document HNSW index without quantisation.

Query Latency: At query time, the query encoder produces a single vector (one transformer forward pass, ~5–10ms for BERT-base on GPU, ~50ms on CPU), then an HNSW approximate nearest-neighbour search returns the top-k results in 2–15ms depending on corpus size and index parameters. Total end-to-end latency for dense retrieval is 7–25ms on GPU (encoder + ANN search), compared to 1–10ms for BM25 — a 3–5× latency overhead that motivates the parallel rather than sequential execution of BM25 and dense retrieval stages in hybrid pipelines.

Comparison with BM25: Dense retrieval requires 20–100× more storage than BM25 inverted indexes (before quantisation), 3–5× higher query latency, and substantial GPU infrastructure for both corpus encoding and (in some architectures) query encoding. BM25 requires no GPU and can run on commodity CPU infrastructure. However, dense retrieval’s semantic quality advantage — 25–30% higher nDCG@10 for in-domain retrieval — justifies the infrastructure cost for applications where retrieval quality directly impacts downstream task performance (RAG hallucination rates, QA accuracy, user satisfaction).

Approximate Nearest Neighbour (ANN) Index Types:

• FAISS IVF-flat: Inverted file (IVF) index that partitions vectors into nlist Voronoi cells; query searches nprobe cells. Good for exact nearest-neighbour with probe budget control. Faiss (Facebook AI Similarity Search) is the most widely used ANN library, supporting GPU-accelerated search.
• HNSW Index: Hierarchical Navigable Small World graph, supporting logarithmic-time approximate search with high recall (99%+). Default index in Weaviate, Qdrant, and Elasticsearch knn; requires 1.5× vector size for navigation graph storage.
• ScaNN (Google): Anisotropic vector quantisation optimised for maximum inner product search, achieving state-of-the-art recall-latency trade-offs for large embedding models.
• DiskANN: Microsoft’s disk-resident ANN index enabling billion-scale dense retrieval with SSD-backed storage, enabling trillion-parameter knowledge bases at reduced DRAM cost.

Comparative Analysis: Dense Retrieval vs Competing Paradigms

Dense Retrieval vs BM25: BM25 requires zero training data and domain-specific fine-tuning, achieving BEIR zero-shot average nDCG@10 of 0.428 versus DPR’s 0.348. Dense retrieval surpasses BM25 on in-domain benchmarks where training data is available (MS MARCO MRR@10: 0.37 vs 0.19) and on semantic matching tasks (synonym handling, paraphrase retrieval, inferential queries). BM25 maintains advantages for exact-match queries (product identifiers, proper nouns, technical codes), requires no GPU infrastructure, and is three to five orders of magnitude cheaper to operate per query on CPU. Production deployments use both in hybrid architectures rather than choosing between them.

Dense Retrieval vs Cross-Encoder Reranking: Cross-encoders process (query, document) pairs jointly through a transformer with full bidirectional attention, producing relevance scores that capture fine-grained semantic interactions invisible to the bi-encoder’s independently computed vectors. Cross-encoders consistently outperform bi-encoders by 10–15 nDCG@10 points (MS MARCO: MRR@10 ≈ 0.44 for bi-encoder vs 0.39 for cross-encoder on MS MARCO — reversal here, cross-encoders rerank the top-100 from bi-encoders and achieve 0.43). However, cross-encoders cannot build offline indexes and must compute a forward pass per (query, candidate) pair, making them computationally infeasible for first-stage retrieval over millions of documents. The standard pipeline is dense bi-encoder retrieval (first stage, top-100–1000 candidates) followed by cross-encoder reranking (second stage, top-k candidates).

Dense Retrieval vs ColBERT (Late Interaction): ColBERT retains per-token embeddings for both query and passage, computing relevance as the sum of maximum inner products between query token embeddings and all passage token embeddings (MaxSim). This late interaction paradigm achieves 5–8 nDCG@10 points above standard bi-encoder dense retrieval on MS MARCO and BEIR (ColBERTv2 BEIR average: 0.496 vs bi-encoder 0.450) while retaining offline passage pre-computation. The storage cost is 100–200× higher than bi-encoders (one vector per token vs one vector per passage), partially offset by residual quantisation in ColBERTv2. PLAID engine achieves production-scale latency. For high-stakes retrieval where quality is paramount, ColBERT is the preferred architecture; for large-scale corpora where storage cost is a primary constraint, bi-encoder dense retrieval is preferred.

Dense Retrieval vs SPLADE (Learned Sparse Retrieval): SPLADE trains a BERT encoder with a log-saturation activation to produce sparse vocabulary-wide term weight vectors that can be stored in and retrieved from an inverted index — combining the semantic quality of dense retrieval with the computational efficiency of BM25’s inverted-index query processing. SPLADE++ achieves BEIR average nDCG@10 of 0.504, substantially outperforming bi-encoder dense retrieval (0.450) while being 10–50× faster at query time due to inverted-index inference. SPLADE represents a convergence of the dense and sparse retrieval paradigms. BGE-M3 in its sparse retrieval mode produces SPLADE-like representations and can be used in place of separate SPLADE models.

Dense Retrieval vs Generative Retrieval (DocID Generation): Generative retrieval models (DSI, NCI) directly generate document identifiers from queries using a sequence-to-sequence decoder, bypassing the ANN index entirely. While conceptually elegant and highly memory-efficient (no explicit index), generative retrieval struggles to generalise to unseen documents and degrades significantly at corpus sizes beyond a few million documents. As of 2026, bi-encoder dense retrieval remains the standard for corpus-scale retrieval; generative retrieval is an active research direction with unclear production scalability.

Limitations and Failure Modes

Dense retrieval has systematic failure modes that motivate the hybrid architectures that dominate production deployments in 2026.

Domain Transfer and Out-of-Distribution Generalisation: The central limitation of early dense retrieval was poor zero-shot domain transfer. DPR trained on MS MARCO fails to generalise to biomedical, legal, or scientific retrieval because the semantic embedding space learned from web search queries and Wikipedia passages does not cover the terminology and query styles of specialist domains. The BEIR benchmark exposed this failure comprehensively. Instruction-tuned models (E5-Mistral, NV-Embed) and contrastive pre-training (Contriever) substantially mitigate but do not fully eliminate the domain gap: BGE-M3 still underperforms specialised models on highly technical domains (biomedical literature search, legal case retrieval) that are far from its pre-training distribution.

Exact-Match and Identifier Retrieval: Dense retrieval’s semantic generalisation comes at the cost of exact-match precision. A query for “AWS EC2 t3.medium” or “ICD-10 code J45.20” must retrieve the exact document mentioning these identifiers. Dense encoders trained on natural language text may embed these highly specific tokens in regions of the embedding space shared with semantically related but non-identical concepts, causing incorrect retrievals. BM25 handles exact-match queries with perfect precision through inverted-index term matching; hybrid BM25+dense retrieval is the standard remedy.

Computational Cost at Scale: For corpora exceeding billions of documents (Common Crawl web archives, enterprise document repositories, social media streams), the GPU infrastructure required for corpus encoding (for large embedding models) and ANN index storage (hundreds of GB to TB scale) creates significant operational costs. Quantisation (PQ, int8, binary) reduces storage at a quality cost; DiskANN enables SSD-resident billion-scale indexes at reduced DRAM requirements.

Negative Sampling Collapse and False Negatives: Dense retrieval training with in-batch negatives is susceptible to false negative contamination — passages that are genuinely relevant to the training query but are treated as negatives because they are not the designated positive. At large batch sizes, the probability of including a true positive passage as a false negative increases, destabilising training gradients. The ANCE approach partially mitigates this by re-mining hard negatives from the current model’s top retrievals, but false negatives from un-annotated relevant passages remain a training challenge, particularly for corpora with high document redundancy.

Catastrophic Forgetting in Fine-Tuned Models: Bi-encoders fine-tuned aggressively on a specific domain (e.g., MS MARCO) suffer catastrophic forgetting of their pre-training generalisations, performing poorly on out-of-domain queries. Continual learning strategies (elastic weight consolidation, task-incremental training, replay buffers) for dense retrieval models are an active research area enabling domain-specific fine-tuning without sacrificing broad generalisation.

Implementation Reference and Production Patterns

Dense retrieval is supported by a mature ecosystem of libraries and platforms enabling deployment from laptop-scale prototyping to enterprise-scale production.

FAISS (Facebook AI Similarity Search): The foundational ANN library (github.com/facebookresearch/faiss), available in Python and C++, supporting IVF, HNSW, PQ, and GPU-accelerated indexes. FAISS is the standard ANN backend for DPR and most research bi-encoder implementations. For 100M-document corpora, FAISS IVF256,Flat with nprobe=64 achieves 95%+ top-20 recall at ~20ms latency on CPU.

Sentence Transformers (Hugging Face): The sentence-transformers Python library (sbert.net) provides the standard API for bi-encoder dense retrieval in Python, wrapping Hugging Face Transformers models with a passage-encoding, index-building, and retrieval interface. SentenceTransformer('BAAI/bge-m3').encode(passages) produces embeddings ready for FAISS or vector database storage. Supports batch GPU encoding, ONNX export for production serving, and integration with LangChain and LlamaIndex.

LangChain and LlamaIndex RAG Pipelines: Both LangChain (OpenAIEmbeddings, HuggingFaceEmbeddings, various VectorStore integrations) and LlamaIndex (VectorStoreIndex) wrap dense retrieval in RAG pipeline abstractions, connecting embedding models to vector databases and LLM generation chains. These frameworks make dense retrieval the default retrieval mechanism in Agentic RAG applications.

Weaviate, Qdrant, Pinecone, pgvector: Production vector databases providing hosted or self-hosted dense retrieval infrastructure with REST/gRPC APIs, multi-tenancy, filtered vector search, real-time index updates, and horizontal scaling. Weaviate and Qdrant additionally support hybrid BM25+dense retrieval as a native query primitive.

PyTerrier Dense Retrieval: The University of Glasgow’s PyTerrier library (terrier-org/pyterrier) supports dense retrieval experiments via integration with FAISS, OpenNIR dense models, and ColBERT, within a unified pipeline API that enables direct comparison against BM25 baselines and cross-encoder rerankers on standard TREC and BEIR evaluations.

Haystack: Deepset’s Haystack framework provides DensePassageRetriever, EmbeddingRetriever, and ColBERTRetriever pipeline nodes that wrap DPR, SBERT, and ColBERT within a composable retrieval-and-reading pipeline, with backends including Elasticsearch, Weaviate, and Qdrant.

Historical Timeline

The development of dense retrieval spans three decades of neural representation learning applied to information retrieval, from early statistical language models through the transformer revolution to the LLM-scale embedding models that define the 2026 landscape.

• 1988–2000: Distributional Semantics and Latent Semantic Analysis. Deerwester et al. (1990) introduce Latent Semantic Analysis (LSA), representing documents as low-dimensional vectors via singular value decomposition of a TF-IDF weighted term-document matrix. LSA demonstrates that dense vector representations capture semantic co-occurrence patterns enabling cross-vocabulary matching, establishing the conceptual foundation for dense retrieval. Hofmann’s Probabilistic LSA (1999) provides a probabilistic generative model for co-occurrence. These methods achieve modest retrieval improvements but suffer from high-dimensional input matrices and computational expense.
• 2013–2015: Neural Word Embeddings. Mikolov et al. introduce word2vec (2013), learning distributed word representations from large corpora via skip-gram and CBOW objectives with negative sampling. GloVe (Pennington et al., 2014) provides global co-occurrence-based word embeddings. Both establish that neural training over text corpora produces semantically organised dense vector spaces where geometric operations (analogies: king − man + woman ≈ queen) reflect semantic structure. These word-level embeddings are not directly applicable to retrieval but establish the paradigm of learning semantic representations from raw text.
• 2017–2019: Sentence Embeddings for Semantic Similarity. InferSent (Conneau et al., 2017) trains sentence encoders on natural language inference data, producing sentence embeddings suitable for semantic textual similarity tasks. The Universal Sentence Encoder (Cer et al., 2018) extends this to multi-task training on diverse sentence-level tasks. Sentence-BERT (Reimers and Gurevych, 2019) fine-tunes BERT with siamese and triplet network objectives on NLI and semantic textual similarity datasets, producing the first bi-encoder embeddings with strong semantic sentence similarity performance. SBERT enables bi-encoder semantic search at scale and is the direct predecessor of modern dense retrieval.
• 2020: The Dense Retrieval Breakthrough. Four landmark papers in 2020 establish dense retrieval as a viable paradigm for large-scale information retrieval. DPR (Karpukhin et al., EMNLP 2020) demonstrates bi-encoder dense retrieval outperforms BM25 on open-domain QA. REALM (Guu et al., ICML 2020) enables gradient-through-retrieval end-to-end training. RAG (Lewis et al., NeurIPS 2020) integrates DPR with sequence-to-sequence generation for knowledge-intensive tasks. ColBERT (Khattab and Zaharia, SIGIR 2020) introduces late interaction as a quality-preserving efficiency bridge between bi-encoders and cross-encoders.
• 2021: Scaling and Hard Negative Mining. ANCE (Xiong et al., ICLR 2021) establishes dynamic hard negative mining via periodic ANN index refresh, significantly improving bi-encoder quality. RocketQA (Qu et al., NAACL 2021) introduces cross-encoder denoising and data augmentation for DPR training. SPLADE (Formal et al., SIGIR 2021) introduces learned sparse retrieval that combines inverted-index efficiency with neural semantic quality. The BEIR benchmark (Thakur et al., NeurIPS 2021) exposes dense retrieval’s domain generalisation failure, redirecting the field toward zero-shot capable models.
• 2022–2023: Instruction Tuning and Zero-Shot Generalisation. Contriever (Izacard et al., TMLR 2022) demonstrates unsupervised contrastive pre-training producing generalisable dense retrievers without labelled relevance data. E5 (Wang et al., 2022) introduces weakly-supervised contrastive pre-training at scale. GTR (Ni et al., EMNLP 2022) achieves strong zero-shot generalisation through large-scale bi-encoder training. ColBERTv2 (Santhanam et al., NAACL 2022) adds residual compression and distillation to ColBERT, enabling production-scale late interaction retrieval. E5-Mistral-7B-Instruct (Wang et al., 2023) demonstrates that LLM-backbone embedding models with instruction prompting surpass specialist bi-encoders across MTEB. DRAGON+ (Lin et al., EMNLP 2023) achieves strong generalisation through diverse augmentation.
• 2024: Unified Models and LLM Backbones. BGE-M3 (BAAI, 2024) unifies dense, sparse (SPLADE-like), and multi-vector (ColBERT-like) retrieval in a single multilingual model with 8K context, becoming the production default for multilingual RAG. NV-Embed-v2 (NVIDIA, 2024) achieves MTEB English score of 72.31 using a 7.85B-parameter LLM backbone, topping the English leaderboard. Jina-ColBERT-v2 (Jina AI, 2024) extends late interaction to multilingual and long-context retrieval.
• 2025–2026: LLM-Native Embedding Dominance. Qwen3-Embedding-8B (Alibaba, 2025) reaches 70.58 MTEB multilingual score, becoming the multilingual leaderboard leader. Self-hosted deployment of LLM-scale embedding models becomes commercially viable at sub-$0.01/million-token costs. Hybrid BM25+dense retrieval becomes the production default across all major vector database platforms. Research frontiers shift to retrieval-augmented reasoning (RAR-b), generative retrieval scaling, and personalised contextual bi-encoders.

Research & Literature

1. Karpukhin, V., Oğuz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W. (2020). Dense Passage Retrieval for Open-Domain Question Answering. EMNLP 2020. arXiv:2004.04906.
2. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., … & Kiela, D. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020. arXiv:2005.11401.
3. Xiong, L., Xiong, C., Li, Y., Tang, K. F., Liu, J., Bennett, P., … & Overwijk, A. (2021). Approximate Nearest Neighbor Negative Contrastive Estimation for Dense Text Retrieval. ICLR 2021. arXiv:2007.00808.
4. Khattab, O., & Zaharia, M. (2020). ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. SIGIR 2020. arXiv:2004.12832.
5. Santhanam, K., Khattab, O., Saad-Falcon, J., Potts, C., & Zaharia, M. (2022). ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction. NAACL 2022. arXiv:2112.01488.
6. Thakur, N., Reimers, N., Rücklé, A., Srivastava, A., & Gurevych, I. (2021). BEIR: A Heterogeneous Benchmark for Zero-Shot Evaluation of Information Retrieval Models. NeurIPS 2021 Datasets Track. arXiv:2104.08663.
7. Muennighoff, N., Tazi, N., Magne, L., & Reimers, N. (2023). MTEB: Massive Text Embedding Benchmark. EACL 2023. arXiv:2210.07316.
8. Wang, L., Yang, N., Huang, X., Jiao, B., Yang, L., Jiang, D., … & Wei, F. (2022). Text Embeddings by Weakly-Supervised Contrastive Pre-training (E5). arXiv:2212.03533.
9. Wang, L., Yang, N., Huang, X., Yang, L., Majumder, R., & Wei, F. (2023). Improving Text Embeddings with Large Language Models (E5-Mistral). arXiv:2401.00368.
10. BAAI. (2024). BGE-M3: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation. arXiv:2402.03216.
11. Lee, C., Dai, Z., Zhao, S., Li, X., Gao, M., Chen, J., … & Mao, J. (2024). NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models. arXiv:2405.17428.
12. Alibaba Qwen Team. (2025–2026). Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models. arXiv:2506.05176.
13. Guu, K., Lee, K., Tung, Z., Pasupat, P., & Chang, M. (2020). REALM: Retrieval-Augmented Language Model Pre-Training. ICML 2020. arXiv:2002.08909.
14. Izacard, G., & Grave, E. (2021). Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering. EACL 2021. arXiv:2007.01282.
15. Izacard, G., Lewis, P., Lomeli, M., Hosseini, L., Petroni, F., Schick, T., … & Grave, E. (2022). Atlas: Few-Shot Learning with Retrieval Augmented Language Models. JMLR 2023. arXiv:2208.03299.
16. Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. EMNLP 2019. arXiv:1908.10084.
17. Formal, T., Piwowarski, B., & Clinchant, S. (2021). SPLADE: Sparse Lexical and Expansion Model for First Stage Retrieval. SIGIR 2021. arXiv:2107.05720.
18. Izacard, G., Caron, M., Hosseini, L., Riedel, S., Bojanowski, P., Joulin, A., & Grave, E. (2022). Unsupervised Dense Information Retrieval with Contrastive Learning (Contriever). TMLR 2022. arXiv:2112.09118.
19. Ni, J., Ábrego, G. H., Constant, N., Ma, J., Hall, K. B., Cer, D., & Yang, Y. (2022). Large Dual Encoders Are Generalizable Retrievers (GTR). EMNLP 2022. arXiv:2112.07899.
20. Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by Latent Semantic Analysis. Journal of the American Society for Information Science, 41(6), 391–407.
21. Qu, Y., Ding, Y., Liu, J., Liu, K., Ren, R., Zhao, W. X., … & Chen, H. (2021). RocketQA: An Optimised Training Approach to Dense Passage Retrieval for Open-Domain Question Answering. NAACL 2021. arXiv:2010.08191.
22. Khattab, O., Potts, C., & Zaharia, M. (2021). Relevance-guided Supervision for OpenQA with ColBERT. TACL 2021. arXiv:2007.00814.
23. Lin, S.-C., Asai, A., Li, M., Oguz, B., Lin, J., Mehdad, Y., … & Chen, W.-T. (2023). How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrieval. EMNLP 2023 Findings. arXiv:2302.07452.
24. Ding, Y., Ma, Z., Yang, Y., Ma, J., Tang, J., Zhao, H., & Liu, J. (2023). RetroMAE-v2: Duplex Masked Auto-Encoder for Pre-Training Retrieval-Oriented Language Models. ACL 2023. arXiv:2211.08769.
25. Jina AI. (2024). Jina-ColBERT-v2: A Multilingual Late Interaction Retriever for Embedding and Reranking. Jina AI Blog. https://jina.ai/news/jina-colbert-v2-multilingual-late-interaction-retriever-for-embedding-and-reranking/
26. Are LLM-Based Retrievers Worth Their Cost? An Empirical Study of Efficiency, Robustness, and Reasoning Overhead. (2026). arXiv:2604.03676.
27. Revela: Dense Retriever Learning via Language Modeling. (2026). arXiv:2506.16552.
28. From BM25 to Corrective RAG: Benchmarking Retrieval Strategies for Text-and-Table Documents. (2026). arXiv:2604.01733.

Provenance