Embedding Search

Embedding search is a retrieval paradigm in which queries and documents are encoded into dense vector representations in a shared semantic space, and similarity—typically measured by cosine distance or dot product—is used to rank and retrieve the most relevant items. Unlike keyword-based search, embedding search captures semantic relatedness, enabling matches on meaning rather than exact lexical overlap.

Semantic Classification

Content

Compositional Relationships (Components)

SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:hasPart ai:EmbeddingModel))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:hasPart ai:VectorIndex))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:hasPart ai:QueryEncoder))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:hasPart ai:DocumentEncoder))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:hasPart ai:ApproximateNearestNeighbourIndex))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:hasPart ai:VectorStore))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:hasPart ai:RetrievalPipeline))

Dependency Relationships

SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:requires ai:VectorEmbedding))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:requires ai:VectorDatabase))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:requires ai:CosineSimilarity))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:requires ai:TrainingData))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:requires ai:LossFunction))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:dependsOn ai:TransformerArchitecture))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:dependsOn ai:ContrastiveLearning))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:dependsOn ai:SentenceEmbeddings))

Capability Relationships

SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:enables ai:RetrievalAugmentedGeneration))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:enables ai:EnterpriseSearch))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:enables ai:DocumentRetrieval))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:enables ai:QuestionAnswering))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:enables ai:RecommendationSystem))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:enables ai:CrossModalRetrieval))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:supports ai:BiomedicalInformationRetrieval))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:supports ai:LegalDocumentRetrieval))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:supports ai:CodeSearch))

Implementation Relationships

SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:implements ai:BiEncoderArchitecture))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:implements ai:DenseRetrieval))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:implements ai:ApproximateNearestNeighbour))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:implements ai:ReciprocalRankFusion))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:implements ai:HNSWIndex))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:uses ai:CosineSimilarity))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:uses ai:InnerProduct))

Reduction Relationships

SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:reducesTo ai:SemanticSearch))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:reduces ai:VocabularyMismatch))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:reduces ai:NullResultRate))
SubClassOf(ai:EmbeddingSearch
  ObjectSomeValuesFrom(ai:reduces ai:QueryReformulationBurden))

About

Embedding search emerged from the confluence of two strands of machine learning research: distributional semantics and scalable nearest-neighbour retrieval. The distributional hypothesis — that words appearing in similar contexts have similar meanings — was given algorithmic form by Bengio et al.’s neural language model (2003) and then operationalised at scale by word2vec (Mikolov et al., 2013) and GloVe (Pennington et al., 2014), which produced static word vectors that captured analogy structure and enabled early semantic similarity tasks. However, these word-level vectors were unsuitable for passage-level retrieval because they could not represent sentence or paragraph meaning in a single fixed-length vector that supported fast similarity lookup.

The transition to passage-level embedding search was catalysed by two innovations. First, the Universal Sentence Encoder (Cer et al., 2018) and Sentence-BERT (Reimers and Gurevych, 2019) fine-tuned transformer models to produce single fixed-length sentence or passage embeddings that could be compared directly by cosine similarity, enabling semantic textual similarity at scale. Sentence-BERT’s siamese network architecture — two BERT encoders sharing weights, trained on natural language inference triplets — demonstrated that meaningful sentence-level embeddings could be produced in 10–500× less inference time than cross-encoder models. Second, Dense Passage Retrieval (Karpukhin et al., ACL 2020) showed that a dual-encoder BERT architecture, trained with in-batch negative sampling on open-domain question-answering data, could outperform BM25 keyword search by 9–21 percentage points in top-20 retrieval accuracy on Natural Questions and TriviaQA, definitively establishing that dense embedding search was a viable production alternative to sparse retrieval for knowledge-intensive tasks.

The scalability challenge was addressed in parallel by approximate nearest-neighbour (ANN) libraries. FAISS (Johnson et al., 2019), developed at Facebook AI Research, introduced GPU-accelerated inverted file indexes with product quantisation (IVF-PQ) that could search one billion 128-dimensional vectors in under 50ms. The Hierarchical Navigable Small World (HNSW) algorithm (Malkov and Yashunin, 2018) provided a graph-based index with O(log N) expected query time and recall@100 above 97% at standard settings, becoming the default index structure in purpose-built vector databases. ScaNN (Guo et al., 2020) from Google introduced anisotropic quantisation that prioritised direction over magnitude, improving recall for cosine-similarity workloads. These libraries made billion-scale embedding search feasible on commodity hardware, enabling the subsequent explosion in production deployment.

Historical Development and Formal Foundations

Embedding search operates within the framework of metric space retrieval. Given a corpus D = {d₁, d₂, …, dN} and a neural encoder fθ: text → ℝ^k mapping each document to a k-dimensional vector, and a separate query encoder fφ: text → ℝ^k, the retrieval problem is to find the top-m documents by score sim(q, dᵢ) = fφ(Q) · fθ(dᵢ) / (|fφ(Q)| |fθ(dᵢ)|) — cosine similarity — or sim(Q, dᵢ) = fφ(Q) · fθ(dᵢ) — inner product — where the dot-product form supports maximum inner product search (MIPS) algorithms.

Offline indexing pre-computes {fθ(dᵢ)}_{i=1}^N and stores them in an ANN index. Online retrieval requires only fφ(Q) — one encoder forward pass taking 10–30ms on GPU — followed by an ANN query returning top-m vectors in 1–10ms. This asymmetry — expensive offline pre-computation, cheap online retrieval — is the key architectural insight that makes embedding search viable at production scale. Documents are indexed once; queries arrive continuously at low latency.

Training the encoders uses contrastive objectives. The Multiple Negative Ranking Loss (MNRL) maximises the similarity of a positive (query, passage) pair while using all other passages in the batch as negatives: L = -log(e^{sim(q,p+)/τ} / ∑_{j} e^{sim(q,pj)/τ}), where τ is a temperature hyperparameter (0.05 is common) and the sum runs over one positive and B-1 in-batch negatives. Hard negative mining — selecting difficult negatives using BM25 or a previous retriever checkpoint — substantially improves retrieval quality above random or in-batch negatives alone.

Matryoshka Representation Learning (MRL, Kusupati et al., 2022) extends the training objective to simultaneously optimise a family of nested sub-vectors: L_MRL = ∑_{m∈M} λ_m L(f_{1:m}(x), y), where M is a set of dimension checkpoints (e.g., {64, 128, 256, 512, 1024, 2048}) and f_{1:m} denotes truncation to the first m dimensions. This produces a single model whose output can be truncated at any checkpoint with graceful quality degradation, enabling adaptive storage-accuracy trade-offs without recomputation. By 2026 MRL is standard across all major embedding providers including OpenAI text-embedding-3, Google Gemini Embedding 2, Voyage 4, Cohere Embed v4, and Jina v5.

Components and Architecture

A production embedding search system typically consists of the following components:

• Embedding Model (Encoder): A transformer-based encoder (BERT, RoBERTa, E5, BGE, or instruction-tuned variants) fine-tuned for semantic similarity, producing vectors of 384–4096 dimensions. Modern instruction-tuned models (E5-mistral-7b-instruct, Qwen3-Embedding-8B) prepend a task-description prompt at query time to condition the representation for the retrieval objective.

• Document Ingestion Pipeline: Documents are chunked into segments of 128–512 tokens (with 20–50 token overlap), each chunk is embedded by the document encoder, and vectors are stored in a vector database alongside the raw text and metadata.

• Vector Index: HNSW, IVF-PQ, or hybrid structures within the vector database (Qdrant, Weaviate, Pinecone, pgvector, Milvus, FAISS) provide sub-100ms ANN lookup. HNSW uses M=16–64 bidirectional edges per layer and efSearch=64–256 at query time, trading memory for recall.

• Query Encoding: The user query is embedded by the same encoder model (query-side). Instruction-tuned models accept a task prefix such as “Represent this sentence for searching relevant passages: {query}” (E5 convention) to signal the retrieval intent.

• ANN Retrieval Stage: Top-k (typically k=50–200) candidate documents are retrieved by ANN lookup in 1–10ms using cosine similarity or inner product.

• Reranking Stage (Optional): A cross-encoder (e.g., Cohere Rerank, BGE-reranker) scores each of the top-k candidates by jointly attending over the query and the candidate document, producing a more accurate relevance score at the cost of higher latency (50–500ms for k=100).

• Hybrid Fusion (Optional): BM25 scores and dense vector scores are fused using Reciprocal Rank Fusion (RRF): score(d) = ∑_k 1/(60 + rank_k(d)), yielding a rank-normalised fusion that consistently outperforms either sparse or dense alone by 2–8% nDCG@10 on BEIR.

• Metadata Filtering: Pre-filters restrict ANN search to documents satisfying structured predicates (date range, source, category, access tier) before or during the vector lookup, implemented as filtered HNSW or post-filtering.

  Embedding Model Families

  Embedding search models are organised into several practical families:

• General-Purpose Text Encoders: SBERT, E5, BGE (BAAI General Embedding), Nomic-Embed-Text. Fine-tuned on heterogeneous retrieval, STS, and classification datasets. E5 and BGE dominate open-source MTEB Retrieval leaderboards.

• Instruction-Tuned Encoders: E5-mistral-7b-instruct, Qwen3-Embedding-8B, GTE-Qwen2-7B-Instruct, SFR-Embedding-2_R. Larger (7B+ parameters), use decoder backbones, accept task-description prompts. Qwen3-Embedding-8B achieves MTEB multilingual score of 70.6, topping the leaderboard as of June 2026.

• API-Hosted Proprietary Models: OpenAI text-embedding-3-large (1536/3072 dim, MTEB 64.6), Cohere Embed v4 (multimodal, supports text and images natively), Voyage AI voyage-3-large (MTEB ~70 text retrieval), Google Gemini Embedding 001 (MTEB English 68.32). All support MRL dimension truncation.

• Multimodal Embedding Models: CLIP (OpenAI, 2021), Gemini Embedding 2 (Google, March 2026, 5-modality text/image/video/audio/PDF, 3072 dim), Voyage MM-3.5 (text+image+video, MRL support), Jina CLIP v2 (text+image). These align multiple modalities in a single vector space enabling cross-modal retrieval.

• Specialised Domain Encoders: BioBERT, SciBERT, LegalBERT, CodeBERT — domain-adapted encoders for biomedical, scientific, legal, and code retrieval tasks where general-purpose models underperform due to domain vocabulary distribution shift.

• ColBERT Late-Interaction Models: ColBERTv2, PLAID index. Rather than a single vector per document, ColBERT stores per-token embeddings and scores MaxSim(Q,D)=∑_i max_j qᵢ·dⱼ, achieving higher accuracy at the cost of increased index storage. PLAID compression reduces storage to 24–32 bytes per token.

  Use Cases

  Embedding search underpins a diverse range of production applications:

• RAG Pipelines: The primary use case as of 2026. An embedding search retrieval stage finds semantically relevant chunks from a knowledge base to prepend to the prompt of a large language model, grounding generation in external facts.

• Enterprise Document Search: Employees query HR policies, technical documentation, and internal wikis using natural language; embedding search surfaces relevant sections regardless of vocabulary differences between author and querier.

• E-Commerce Product Discovery: Search queries such as “comfortable shoes for standing all day” retrieve relevant products even without keyword overlap with product descriptions.

• Biomedical Literature Search: Clinicians and researchers query PubMed, clinical trial registries, and drug databases using clinical terminology that differs from the language used in publications.

• Legal and Regulatory Research: Case law retrieval, contract review, and regulatory compliance search benefit from embedding search because legal terminology is highly specialised and different courts may use different phrasings for equivalent concepts.

• Code Search: CodeBERT and StarCoder embeddings enable natural-language queries over code repositories, returning relevant functions, modules, or API usages even without identifier overlap.

• Recommendation Systems: User interaction histories and item descriptions are embedded in a shared space, with embedding search used to retrieve personalisable candidates at low latency from catalogues of millions or billions of items.

• Duplicate and Near-Duplicate Detection: Documents, news articles, customer support tickets, and forum posts are embedded and clustered; near-duplicate pairs are identified by embedding proximity rather than token-overlap heuristics.

• Multimodal Search: Cross-modal retrieval allows a text query to retrieve images, videos, or audio clips, and vice versa, via a shared multimodal embedding space — used in Google Lens, Bing Visual Search, and media archive platforms.

• Customer Support: Support tickets are embedded and matched against resolved-issue databases or FAQ collections to surface relevant answers and reduce escalation rates.

  Academic Context

  The research foundations of embedding search span distributional semantics, information retrieval, and deep learning. The distributional hypothesis (Harris, 1954) provides the theoretical basis for the approach. Key milestones include:

• Bengio et al. (2003) — Neural probabilistic language model: the first demonstration that neural networks could learn distributed word representations.

• Mikolov et al. (2013) — word2vec skip-gram and CBOW: efficient shallow networks producing distributed word vectors at scale; analogy tasks demonstrated algebraic structure.

• Pennington et al. (2014) — GloVe: global vectors from co-occurrence statistics, combining global corpus statistics with local context windows.

• Kiros et al. (2015) — Skip-Thought Vectors: extending word2vec’s skip-gram idea to sentence-level prediction.

• Devlin et al. (2018) — BERT: bidirectional encoder representations from transformers; the foundation for contextual embedding extraction.

• Reimers and Gurevych (2019) — Sentence-BERT: siamese BERT fine-tuned on NLI, producing sentence-level embeddings suitable for fast semantic similarity retrieval.

• Karpukhin et al. (2020) — Dense Passage Retrieval (DPR): dual-encoder fine-tuned with in-batch negatives outperforms BM25 on open-domain QA benchmarks.

• Johnson et al. (2019) — FAISS: billion-scale ANN library from Facebook AI Research.

• Malkov and Yashunin (2018) — HNSW: hierarchical navigable small-world graph achieving O(log N) ANN search.

• Khattab and Zaharia (2020) — ColBERT: late interaction model storing per-token embeddings, enabling high-accuracy retrieval with structured PLAID indexing.

• Radford et al. (2021) — CLIP: contrastive language-image pre-training aligning text and image embeddings; the canonical multimodal embedding model.

• Thakur et al. (2021) — BEIR: heterogeneous zero-shot retrieval benchmark establishing consistent cross-domain evaluation.

• Muennighoff et al. (2022) — MTEB: massive text embedding benchmark with 56 tasks across 8 categories.

• Kusupati et al. (2022) — Matryoshka Representation Learning: nested embeddings enabling adaptive dimensionality truncation.

• Wang et al. (2022) — E5: text embedding by weakly supervised contrastive pre-training, achieving competitive MTEB scores without task-specific labelled data.

• Guo et al. (2020) — ScaNN: scalable nearest neighbours with anisotropic quantisation from Google Research.

• Izacard et al. (2022) — Contriever: unsupervised dense retrieval via momentum contrastive learning.

  Current Landscape (2026)

  As of June 2026, the embedding search landscape is characterised by rapid model improvement, convergence on hybrid retrieval architectures, and maturation of vector database infrastructure:

  Model Quality: Qwen3-Embedding-8B from Alibaba holds the top position on the MTEB multilingual leaderboard with a score of 70.6 as of June 2025, surpassing all proprietary API models including OpenAI text-embedding-3-large (64.6) and Google Gemini Embedding 001 (68.32 on the English leaderboard). The competitive frontier has shifted from BERT-base bi-encoders to 7–8B parameter instruction-tuned decoder-based encoders that accept task descriptions at query time.

  Matryoshka Representation Learning: MRL is now standard across all major embedding providers. Google’s Gemini Embedding 2 Preview (released March 2026) supports five modalities (text, image, video, audio, PDF) natively with MRL-enabled 3072-dimensional output and 100+ language support. Voyage MM-3.5 and Jina v5 similarly support MRL truncation for multimodal vectors with less than 1% accuracy loss from truncation.

  Hybrid Retrieval as Default: Production systems have largely converged on hybrid retrieval (dense vector search + BM25 sparse) as the default architecture. BM25 retains its advantages on exact-match queries for product codes, person names, and rare entities that may fall outside the embedding model’s training distribution, while dense retrieval handles semantic variation. Reciprocal Rank Fusion with k=60 is the standard fusion method, with Elasticsearch 8.x, OpenSearch, and Vespa all providing native hybrid retrieval combining dense and sparse search in a single query path.

  Vector Database Infrastructure: Qdrant (Rust-based, Series B funding March 2026) leads for production workloads requiring low-latency HNSW with excellent predicate-filtered search. Weaviate provides native hybrid search with GraphQL-based schema. pgvector is the default for PostgreSQL deployments with corpora under ~10M vectors. Milvus and Pinecone serve extreme-scale deployments with billions of vectors. All production vector databases now support multi-tenancy, real-time index updates, and payload-filtered ANN search.

  Long-Context Models: Embedding models supporting 8k–128k token context windows (LongEmbed models, GTE-Qwen2-7B-instruct at 131k tokens) have improved retrieval for lengthy legal documents, technical manuals, and scientific papers that do not chunk cleanly at the passage level.

  Agentic Retrieval: Embedding search is increasingly deployed within agentic frameworks where LLM-based agents decide when to invoke retrieval, what queries to construct, and how to synthesise retrieved context across multiple rounds of search. LangChain, LlamaIndex, and Haystack provide orchestration primitives; Self-RAG and Corrective RAG (CRAG) frameworks allow models to evaluate and filter retrieved context autonomously.

  UK Context

  The UK has contributed substantially to the foundational research underlying embedding search. The University of Edinburgh’s Institute for Language, Cognition and Computation (ILCC) — home to EdinburghNLP, one of the largest NLP research groups in the world with 14 core faculty as of 2026 — has produced influential work in machine translation, neural language models, and information retrieval. Edinburgh’s UKRI-funded Centre for Doctoral Training in Responsible and Trustworthy Natural Language Processing (CDT, studentship intake September 2026) reflects the UK’s investment in advancing and governing NLP including retrieval technologies. UCL NLP (90 High Holborn, London) organises the South England NLP Meetup and contributes to retrieval and representation learning research.

  In industry, UK financial services firms — concentrated in London, with significant presence in Manchester, Leeds, and Edinburgh — deploy embedding search for regulatory compliance document retrieval, anti-money-laundering entity matching, and analyst research tools. Manchester’s startup ecosystem has grown substantially, attracting £1.5 billion in funding in 2025 (a 47% year-over-year increase), with AI and search technology companies among the beneficiaries. NHS England’s digital transformation programme includes semantic search over clinical guidelines, drug formularies, and electronic health records to support clinical decision support; embedding search approaches are central to these deployments because clinical terminology mismatch between lay queries and technical documentation is acute.

  Sheffield’s Natural Language Processing group at the University of Sheffield (GATE project, Professor Hamish Cunningham’s team) has long-standing work in information extraction and retrieval for biomedical applications, directly relevant to embedding-based document retrieval. Arm Holdings (Cambridge) and Graphcore (Bristol) develop custom silicon that accelerates both the transformer inference required for query encoding and the matrix operations underlying ANN index construction, contributing UK-origin hardware to the embedding search stack.

  Future Directions (2026–2030)

  Several trajectories are reshaping embedding search:

• Sparse Autoencoder Embeddings: Interpretable sparse embeddings produced by sparse autoencoders (SAEs) applied to LLM activations are emerging as an alternative to dense vector representations, enabling concept-level retrieval and offering GDPR-compatible transparency about what an embedding encodes.

• Universal Multimodal Embeddings: Models like Gemini Embedding 2 and successors will align text, image, video, audio, graph, and structured data in a single vector space, enabling unified retrieval across all modalities without modality-specific indexes.

• Semantic Caching: Embedding similarity is used to detect near-duplicate queries and return cached results, reducing redundant LLM inference calls in agentic systems. This requires embedding search to be fast enough to serve as a caching layer for LLM calls themselves.

• Privacy-Preserving Embeddings: Concept-aware perturbation methods (e.g., Concept-Aware Privacy Mechanisms, arxiv 2602.07090) protect against embedding inversion attacks while preserving retrieval utility, driven by GDPR enforcement and EU AI Act requirements effective August 2026.

• Streaming and Incremental Indexing: Real-time index updates at millions-per-second insert rates (using delta indexes and background compaction) will enable embedding search over live data streams without indexing downtime windows.

• Long-Context Native Encoders: Models with context windows matching or exceeding document lengths (128k+ tokens) reduce the need for chunking strategies, enabling whole-document embeddings that capture global coherence rather than passage-local context.

• Benchmark Evolution: MTEB v2 (2026) introduces cross-lingual, long-document, and multimodal retrieval tasks alongside the original English text-only suite, driving models to optimise for a broader capability profile.

  Research and Literature

  1. Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C. (2003). A Neural Probabilistic Language Model. JMLR, 3, 1137–1155.
  2. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J. (2013). Distributed Representations of Words and Phrases and their Compositionality. NeurIPS 2013.
  3. Pennington, J., Socher, R., Manning, C. (2014). GloVe: Global Vectors for Word Representation. EMNLP 2014.
  4. Kiros, R., Zhu, Y., Salakhutdinov, R., et al. (2015). Skip-Thought Vectors. NeurIPS 2015.
  5. Devlin, J., Chang, M.W., Lee, K., Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL 2019.
  6. Reimers, N., Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. EMNLP 2019.
  7. Johnson, J., Douze, M., Jégou, H. (2019). Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3), 535–547.
  8. Malkov, Y.A., Yashunin, D.A. (2018). Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs. IEEE TPAMI, 42(4), 824–836.
  9. Karpukhin, V., Oguz, B., Min, S., et al. (2020). Dense Passage Retrieval for Open-Domain Question Answering. EMNLP 2020.
  10. Khattab, O., Zaharia, M. (2020). ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. SIGIR 2020.
  11. Guo, R., Sun, P., Lindgren, E., et al. (2020). Accelerating Large-Scale Inference with Anisotropic Vector Quantization. ICML 2020.
  12. Radford, A., Kim, J.W., Hallacy, C., et al. (2021). Learning Transferable Visual Models From Natural Language Supervision (CLIP). ICML 2021.
  13. Thakur, N., Reimers, N., Rücklé, A., Srivastava, A., Gurevych, I. (2021). BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. NeurIPS 2021 Datasets and Benchmarks.
  14. Muennighoff, N., Tazi, N., Magne, L., Reimers, N. (2022). MTEB: Massive Text Embedding Benchmark. EACL 2023.
  15. Kusupati, A., Bhatt, G., Rege, A., et al. (2022). Matryoshka Representation Learning. NeurIPS 2022.
  16. Wang, L., Yang, N., Huang, X., et al. (2022). Text Embeddings by Weakly-Supervised Contrastive Pre-training. arXiv:2212.03533.
  17. Izacard, G., Caron, M., Hosseini, L., et al. (2022). Unsupervised Dense Information Retrieval with Contrastive Learning (Contriever). Transactions on Machine Learning Research.
  18. Santhanam, K., Khattab, O., Saad-Falcon, J., Potts, C., Zaharia, M. (2022). ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction. NAACL 2022.
  19. Zhang, P., et al. (2023). BGE M3-Embedding: Multi-Lingual, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation. arXiv:2309.07597.
  20. Yang, H., et al. (2024). Qwen3-Embedding: Multilingual Instruction-tuned Text Embeddings. Technical Report, Alibaba DAMO Academy.
  21. Günther, M., et al. (2023). Jina Embeddings: A Novel Set of High-Performance Sentence Embedding Models. arXiv:2307.11224.
  22. Lee, C., et al. (2024). Gecko: Versatile Text Embeddings Distilled from Large Language Models. arXiv:2403.20327 (Google DeepMind).
  23. Rau, D., et al. (2024). BERGEN: A Benchmarking Library for Retrieval-Augmented Generation. arXiv:2407.01102.
  24. Shi, W., et al. (2024). In Defense of RAG in the Era of Long-Context Language Models. arXiv:2409.01666.
  25. Concept-Aware Privacy Mechanisms for Defending Embedding Inversion Attacks. (2026). arXiv:2602.07090.
  26. Filtered Approximate Nearest Neighbor Search in Vector Databases: System Design and Performance Analysis. (2026). arXiv:2602.11443.
  27. Benchmarking Filtered Approximate Nearest Neighbor Search Algorithms on Transformer-based Embedding Vectors. (2025). arXiv:2507.21989.
  28. LongEmbed: Extending Embedding Models for Long Context Retrieval. (2024). arXiv:2404.12096.

Provenance