Embeddings

Embeddings are dense, low-dimensional vector representations of discrete objects—words, sentences, images, code, graphs, or arbitrary entities—learned by neural networks such that geometric relationships in the vector space correspond to semantic or functional relationships between the original objects. The core property is that semantically similar inputs map to nearby vectors, enabling tasks like similarity search, clustering, and retrieval to be performed as efficient geometric operations. Embeddings are the foundational representation layer of modern deep learning, underpinning language models, recommendation systems, search engines, and retrieval-augmented generation pipelines. They transform high-dimensional sparse inputs into compact continuous representations that downstream neural architectures—particularly attention-based transformers—can process and reason over.

Semantic Classification

Content

Compositional Relationships (Components)

SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:hasPart ai:TokenEmbedding))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:hasPart ai:SoftPromptEmbedding))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:hasPart ai:EmbeddingModel))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:hasPart ai:PositionalEncoding))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:hasPart ai:KnowledgeGraphEmbedding))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:hasPart ai:MultimodalEmbedding))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:hasPart ai:SentenceEmbedding))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:hasPart ai:CodeEmbedding))

Dependency Relationships

SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:requires ai:NeuralNetwork))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:requires ai:TrainingData))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:requires ai:LossFunction))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:dependsOn ai:Backpropagation))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:dependsOn ai:TransformerArchitecture))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:dependsOn ai:ContrastiveLearning))

Capability Relationships

SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:enables ai:SemanticSearch))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:enables ai:RetrievalAugmentedGeneration))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:enables ai:VectorDatabase))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:enables ai:RecommendationSystem))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:enables ai:AnomalyDetection))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:enables ai:DenseRetrieval))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:enables ai:CrossModalRetrieval))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:enables ai:ZeroShotClassification))

Implementation Relationships

SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:implements ai:DistributionalHypothesis))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:implements ai:ContrastiveLearning))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:implements ai:MatryoshkaRepresentationLearning))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:implements ai:SelfSupervisedLearning))

Support Relationships

SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:supports ai:LargeLanguageModel))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:supports ai:NaturalLanguageProcessing))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:supports ai:ComputerVision))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:supports ai:KnowledgeGraph))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:supports ai:MultimodalLearning))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:supports ai:ComputationalBiology))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:supports ai:RetrievalAugmentedGeneration))

Reduction Relationships

SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:reducesTo ai:LatentVector))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:reducesTo ai:DimensionalityReduction))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:reducesTo ai:RepresentationLearning))
SubClassOf(ai:Embeddings
  ObjectSomeValuesFrom(ai:reducesTo ai:VectorSpace))

About

Embeddings solve the fundamental problem of making discrete symbolic objects tractable for continuous neural computation. A vocabulary of fifty thousand words cannot be fed directly into a network expecting continuous-valued inputs; one-hot vectors are technically valid but astronomically sparse and convey no relational structure whatsoever — the vector for “king” is no closer to the vector for “queen” than it is to the vector for “bicycle”. Embeddings compress such inputs into compact latent vectors — typically 64 to 4096 dimensions — where proximity in the vector space encodes semantic relatedness, enabling the entire apparatus of continuous optimisation, gradient-based learning, and geometric reasoning to apply to symbolic objects. The transformation from discrete symbol to continuous vector is the single most consequential representational choice in modern machine learning: almost every production AI system — language model, image generator, recommendation engine, biological structure predictor — depends critically on the quality of its embedding layer, yet this layer is often invisible to end users.

The theoretical grounding for embedding methods draws from multiple disciplines. Information theory motivates the compression aspect: an embedding should preserve the information relevant to downstream tasks while discarding task-irrelevant variation. Manifold learning motivates the geometric aspect: high-dimensional data (images, text, molecular structures) tends to concentrate on low-dimensional manifolds embedded in the ambient space, and embeddings aim to discover and represent these intrinsic structures. Functional analysis motivates the approximation aspect: given sufficient model capacity and training data, neural embedding functions can approximate any continuous function mapping input objects to target geometric relationships. Statistical learning theory provides generalisation guarantees: embeddings trained on sufficiently diverse corpora transfer to novel inputs because they capture statistical regularities at the level of latent factors rather than surface features.

The practical impact of high-quality embeddings is difficult to overstate. Before contextual embeddings, natural language processing systems required extensive feature engineering — hand-crafted n-gram features, parse trees, co-reference chains — for each specific task. BERT-derived embeddings reduced this engineering burden to near zero: a single pre-trained encoder, fine-tuned with a small task-specific head, achieved state-of-the-art performance across question answering, named entity recognition, sentiment analysis, and textual entailment simultaneously. The analogous impact in computer vision came through CLIP-style joint embeddings, which enabled zero-shot image classification by comparing image embeddings to text description embeddings without any task-specific training. The extension of these principles to protein sequences (ESM-2), molecular graphs (ChemBERTa), code (CodeBERT, StarCoder), and time series (TimesFM, Chronos) demonstrates the generality of the embedding paradigm across scientific domains.

The key insight operationalised by Word2Vec in 2013 is that training a shallow network to predict word context from surrounding words produces vectors with genuinely meaningful geometry: semantic analogies can be solved via vector arithmetic (king − man + woman ≈ queen), and clusters in the space correspond to syntactic categories, named-entity types, and thematic groups. This distributional hypothesis — that words appearing in similar contexts share meanings — had long precedent in structural linguistics (Firth 1957: “You shall know a word by the company it keeps”) but embeddings operationalised it at industrial scale. The 2013–2018 period saw rapid proliferation of static embedding families: GloVe (global co-occurrence statistics), FastText (subword units enabling morphologically-rich language coverage), and domain-specific variants for biomedical text, legal corpora, and code.

The modern era, inaugurated by BERT (Devlin et al., 2019) and advanced through subsequent bidirectional encoder families, introduces contextual embeddings: each token receives a representation conditioned on its full surrounding context, computed through multi-head self-attention layers. This resolves lexical ambiguity that static embeddings could not address — the token “bank” in “river bank” and “bank account” receives distinct context-dependent vectors. Sentence-BERT (Reimers & Gurevych, 2019) adapted this for practical retrieval by producing fixed-size sentence vectors comparable by cosine similarity, reducing pairwise comparison time for ten thousand sentences from 65 hours to under five seconds. As of 2026, the leading embedding models — Qwen3-Embedding-8B, Gemini Embedding 2, Cohere Embed v4 — are instruction-tuned LLM-scale architectures supporting multimodal inputs across text, image, video, audio, and PDF documents in shared 3,072-dimensional vector spaces, representing a qualitative expansion from unimodal text-only systems.

Training Paradigms and Objectives

The diversity of modern embedding applications is matched by a corresponding diversity of training objectives. Each objective encodes different assumptions about what geometric structure the embedding space should have, and selecting the appropriate objective for a deployment context is as important as selecting the model architecture.

Predictive objectives — the original paradigm, exemplified by Word2Vec and GloVe, trains embeddings to predict linguistic context. Skip-gram maximises the log-probability of observing surrounding words given a centre word; CBOW does the reverse. These objectives produce embeddings where co-occurrence statistics are reflected in vector geometry, but they provide no explicit signal about sentence-level or document-level similarity. Masked language modelling (BERT, RoBERTa) extends this by masking random tokens and training a full encoder-decoder stack to reconstruct them, producing contextual embeddings as a byproduct of the masking task.

Contrastive objectives — the dominant paradigm for dedicated embedding models, Contrastive Learning trains models to bring positive pairs together and push negative pairs apart in the embedding space. The choice of positive pair construction defines what kind of similarity the resulting embeddings encode. For sentence similarity, positive pairs are paraphrases (natural language inference entailment pairs, translation equivalents, or pairs generated by synonym substitution). For image-text embeddings (CLIP, ALIGN), positive pairs are naturally paired images and captions. For code-comment embeddings (CodeBERT), positive pairs are function bodies and their documentation strings. The InfoNCE loss is the dominant contrastive objective: it treats in-batch negatives as noise samples and maximises mutual information between positive pair representations.

Distillation objectives — many production embedding systems use a cross-encoder teacher (a model that processes query-document pairs jointly, achieving high precision but prohibitive inference latency) to distil knowledge into a bi-encoder student that processes query and document independently. The student learns to approximate the teacher’s similarity scores while maintaining the speed advantages of independent encoding. ColBERT and its successors use late interaction — producing per-token embeddings and computing similarity via maximum inner product search over token pairs — as an intermediate between bi-encoder speed and cross-encoder quality.

Generative objectives — GPT-style autoregressive language models produce contextual token representations as a byproduct of next-token prediction. These representations can serve as embeddings by pooling or extracting the last-token representation (for decoder-only architectures, the last token attends to all preceding tokens and thus summarises the full input). This approach, used by models like E5-Mistral-7B and GTE-Qwen, leverages the semantic richness of large autoregressive language models for retrieval without dedicated contrastive training.

Multi-task objectives — the highest-performing embedding models in 2025–2026 combine multiple objectives simultaneously or sequentially: initial pre-training with a generative objective, followed by contrastive fine-tuning on task-specific paired data, optionally supplemented by hard negative mining (using other embedding models to retrieve near-miss negatives that are semantically close but not actually relevant, providing harder training signal than random negatives). Instruction tuning adds a natural language task description to the input at training time, enabling a single model to switch between retrieval, classification, semantic similarity, and re-ranking tasks via instruction prefix.

Challenges and Failure Modes

Despite their ubiquity, embedding systems exhibit several systematic failure modes in deployment. Out-of-distribution (OOD) inputs — queries or documents from domains significantly different from the training distribution — may receive embedding vectors that are geometrically misleading, appearing near unrelated content due to superficial lexical overlap. A model trained on general web text may produce poor embeddings for highly specialised scientific or legal vocabulary not well represented in training data.

Semantic granularity mismatch — different levels of semantic abstraction within a corpus (e.g., article-level, section-level, and sentence-level content) require different embedding granularities for optimal retrieval. A model that effectively captures document-level themes may fail to retrieve the specific paragraph that answers a precise factual question. This motivates hybrid architectures that embed content at multiple granularities and select the appropriate level based on query type.

Cascading failure in RAG pipelines — embedding quality problems in Retrieval-Augmented Generation compound: a poor embedding model that retrieves marginally relevant documents provides incorrect grounding to the Large Language Model, which may generate confidently wrong answers. The retrieval quality is a ceiling on the generation quality, and embedding failures are often invisible to end users who see fluent but incorrect outputs. Rigorous evaluation of retrieval recall at the embedding stage — separate from end-to-end generation quality — is essential for diagnosing and improving RAG pipeline performance.

Embedding space drift — in production systems where new content is continuously added to the index, model updates require re-embedding all existing content to maintain a geometrically consistent index. This is computationally expensive for large indices and creates operational challenges: during the re-embedding transition, a query may compare an updated query embedding against outdated document embeddings, producing inconsistent retrieval. Hot-swap re-indexing strategies and version-controlled embedding spaces are active areas of production engineering for embedding infrastructure.

Scaling Laws and Efficiency

Embedding models exhibit consistent scaling behaviour: larger models, more training data, and longer training consistently improve MTEB performance, following power-law relationships similar to those documented for generative language models. The Qwen3-Embedding family demonstrates that 8-billion-parameter models can achieve MTEB scores (70.6) substantially above 1-billion-parameter models trained on identical data, suggesting significant headroom from further scaling.

However, the cost of embedding inference at production scale creates strong incentives for efficiency. A query-serving system embedding one billion queries per day at one millisecond per query per CPU core requires approximately 11,500 CPU cores — prohibitive for all but the largest organisations. This has driven investment in four complementary efficiency techniques. First, knowledge distillation from large teacher models to small student models (often 100M–400M parameters) with modest quality loss. Second, quantisation of embedding vectors from float32 to int8 or binary, reducing storage by 4x to 32x. Third, approximate nearest neighbour (ANN) indexing using HNSW or IVF-PQ indices, which reduce search time from O(N) to O(log N) or O(N/n_lists) at the cost of small recall reductions. Fourth, Matryoshka Representation Learning, enabling deployments to select embedding dimensions adaptively — using 128-dimensional vectors for coarse first-pass retrieval over billions of candidates and 1024-dimensional vectors for precise ranking of the top-1000.

Components / Architecture

Token Embedding layers — the first component in nearly every Large Language Model. A learned lookup table maps vocabulary indices to dense vectors, initialised randomly and trained end-to-end with the model. Vocabulary sizes range from 30,000 (original BERT) to 200,000+ (multilingual frontier models). Token embeddings encode lexical identity but not position or context.

Positional Encoding — since Transformer Architecture attention is permutation-invariant, positional information must be injected explicitly. Sinusoidal encodings (original Transformer) are fixed mathematical functions of position; learned absolute positional embeddings are trained parameters; Rotary Position Embedding (RoPE) and ALiBi encode relative position directly within attention weights, enabling length generalisation beyond training context windows.

Sentence and document embedding models — dedicated architectures (SBERT, E5, GTE, BGE, NV-Embed) trained specifically to produce high-quality fixed-size sentence or passage embeddings for retrieval. Training objectives include contrastive InfoNCE loss over positive/negative pairs, multiple-negatives ranking loss over in-batch negatives, and knowledge distillation from cross-encoder teacher models.

Soft Prompt Embedding — trainable embedding vectors prepended to inputs during prompt tuning; a parameter-efficient fine-tuning technique that modifies only a small set of embedding parameters rather than the full model, enabling task-specific adaptation at a fraction of full fine-tuning cost.

Multimodal joint embeddings — CLIP, ALIGN, Florence, and Cohere Embed v4 train vision and language encoders together using contrastive objectives over paired data (image-caption pairs, document-image pairs), producing a shared geometric space where text queries and image documents are directly comparable by cosine similarity.

Matryoshka Representation Learning (MRL) — a training technique introduced by Kusupati et al. (2022) that ensures the first N dimensions of an embedding are independently meaningful for varying N, permitting dynamic dimension truncation without retraining. This enables tiered retrieval pipelines that use small (128-dim) vectors for coarse candidate retrieval and larger (1024-dim) vectors for precise re-ranking, trading accuracy for speed adaptively.

Binary and scalar quantisation — embedding vectors range from 64-dimensional (efficient lightweight models) to 4096-dimensional. Binary quantisation (1 bit per dimension) compresses 99% of storage while retaining approximately 95% of retrieval quality, enabling billion-scale vector indices to fit in commodity RAM.

Instruction-Tuned and Task-Aware Embedding

A significant recent development is instruction-tuned embedding: rather than producing a single general-purpose embedding for each input, the model conditions its output on a natural language description of the target task. For example, an input document might be embedded with the instruction “Represent this passage for retrieval” when building a document index, and with “Represent this query for finding similar documents” when embedding a search query. This asymmetric encoding — different instruction prefixes for queries and documents — consistently improves retrieval performance by encoding task intent into the embedding geometry.

The E5-instruct (Mistral backbone), NV-Embed-v2 (NVIDIA, 7B Mistral backbone), and BGE-M3 (BAAI, supporting retrieval, reranking, and dense-sparse-colbert multi-functionality simultaneously) families pioneered this approach. The underlying mechanism is that instruction tuning causes the model to project inputs into a semantics-preserving subspace appropriate for the task, effectively implementing task-conditioned dimensionality reduction at the model level rather than as a post-processing step.

Instruction-tuned embeddings are particularly valuable in heterogeneous retrieval scenarios where a single corpus contains diverse content types — product descriptions, customer reviews, technical specifications, and news articles — each requiring different notions of similarity. Rather than training separate embedding models for each content type, a single instruction-tuned model can be directed to embed each type with an appropriate instruction prefix, unifying the index while preserving type-specific retrieval semantics.

Cross-Lingual and Multilingual Embedding

The extension of embedding methods to multilingual settings addresses one of the most practically significant limitations of early embedding research: models trained on English data produce representations incompatible with those trained on other languages, preventing cross-lingual retrieval and knowledge transfer. Multilingual embedding models — trained on parallel corpora (aligned translation pairs), multilingual masked language modelling objectives, or both — produce a shared vector space in which semantically equivalent texts in different languages map to nearby vectors.

XLM-R (Conneau et al., 2020, Facebook AI) demonstrated that a single model trained on 100 languages achieves competitive performance on cross-lingual NLI, NER, and QA benchmarks relative to monolingual models, establishing that shared multilingual representations are feasible without significant performance degradation on any individual language. LaBSE (Language-agnostic BERT Sentence Embedding, Feng et al., 2022, Google) specifically optimised sentence-level cross-lingual embeddings using additive margin softmax loss on 100 languages, achieving state-of-the-art bitext mining (finding translation pairs in large parallel corpora). SONAR (Sentence-level mOdality and laNguAge-univeRsal embeddings, Meta AI, 2023) extends to 200 languages and speech modality.

In the UK context, multilingual embeddings are particularly relevant for NHS clinical text processing (communities speaking South Asian languages, Welsh, and other languages), GCHQ signals intelligence processing of multilingual communications, and UK Border Force document analysis. The British Council applies multilingual embeddings to language assessment and educational content recommendation across their global English-language teaching operations.

Lineage and Major Families

Pre-neural distributional methods — Latent Semantic Analysis (LSA, Deerwester et al. 1990) using SVD decomposition of term-document co-occurrence matrices; Pointwise Mutual Information (PMI) matrices; Hyperspace Analogue to Language (HAL). These established the distributional hypothesis computationally but produced sparse, brittle representations.

Static word embeddings — Word2Vec (Mikolov et al., skip-gram and CBOW, 2013), GloVe (Pennington et al., global co-occurrence, 2014), FastText (Bojanowski et al., subword n-grams enabling OOV handling, 2016). Fixed vector per word type, context-independent. These became default features for NLP tasks 2014–2018.

Contextual token embeddings — ELMo (Peters et al., bidirectional character-level LSTMs, 2018), GPT-1 (Radford et al., unidirectional transformer, 2018), BERT (Devlin et al., bidirectional masked LM, 2019), RoBERTa, ALBERT, XLNet. Context-dependent per-token representations; each token’s vector is a function of the entire input sequence.

Sentence and document embeddings — Sentence-BERT (Reimers & Gurevych, 2019), Universal Sentence Encoder (Cer et al., 2018), InferSent (Conneau et al., 2017), SimCSE (Gao et al., 2021), E5 (Wang et al., 2022), BGE (Xiao et al., 2023). Fixed-size pooled representations suitable for cosine comparison at retrieval scale.

Long-context passage embeddings — NV-Embed, E5-Mistral-7B, GTE-Qwen-7B, and similar models represent entire documents up to 32K tokens using decoder-only LLM backbones with pooling modifications, improving Retrieval-Augmented Generation recall for long-form knowledge sources.

Multimodal joint embeddings — CLIP (Radford et al., 2021, 400M image-text pairs), ALIGN (Jia et al., 2021), Florence (Yuan et al., 2021), ImageBind (Girdhar et al., 2023, six modalities), Cohere Embed v4 (2025, text + image + interleaved documents), Gemini Embedding 2 (2026, text + image + video + audio + PDF in a single 3072-dim space).

Knowledge Graph Embeddings — TransE (Bordes et al., 2013), RotatE (Sun et al., 2019), ComplEx, QuatE. Represent knowledge graph entities and relations as vectors/matrices preserving structural neighbourhood properties and enabling link prediction and entity alignment.

Code embeddings — CodeBERT (Feng et al., 2020), UniXcoder, StarCoder, code-specific sentence models. Map code functions, repositories, and documentation into a joint semantic space enabling cross-lingual code search and clone detection.

Use Cases

Semantic Search — query and document embeddings compared by Cosine Similarity or approximate nearest-neighbour algorithms (HNSW, IVF) in Vector Database systems (Pinecone, Weaviate, Qdrant, Milvus, pgvector) retrieve semantically relevant results beyond keyword overlap. A query about “medication side effects” returns relevant pharmaceutical documents even when they use clinical terminology not present in the query.

Retrieval-Augmented Generation — external documents are pre-indexed as embeddings; at inference, a user query is embedded and the top-K nearest documents are retrieved and injected into the language model context, grounding generation in current factual content and reducing hallucination.

Recommendation Systems — user preference vectors and item content vectors learned jointly; product, music, video, and article recommendations reduce to approximate nearest-neighbour lookup in the joint embedding space. Spotify’s recommendation engine, Netflix’s content similarity, and Amazon’s product discovery all rely on embedding-based retrieval as a first-stage retrieval layer.

Duplicate detection and entity resolution — records referring to the same real-world entity (products, people, organisations) embed near one another, enabling efficient deduplication at scale. Vitally important for data quality in financial institutions, healthcare record linkage, and e-commerce catalogue management.

Zero-shot and few-shot classification — candidate class labels embedded alongside input examples; classification proceeds by finding the nearest label embedding, requiring no task-specific training data. Enables rapid deployment on new taxonomy categories without retraining.

Code search and completion — code models produce function and repository embeddings enabling semantic code retrieval; GitHub Copilot’s retrieval layer uses embedding-based search over opened files to provide contextually relevant completions without explicit retrieval configuration.

Fraud and Anomaly Detection — transaction sequences, user sessions, and network flows embedded as temporal sequences; anomalies appear as outliers in the learned normal-behaviour manifold. Used extensively in banking, insurance, and cybersecurity contexts.

Computational biology — protein language models (ESMFold, ProtTrans, ESM-2) produce amino-acid sequence embeddings encoding structural and functional properties, accelerating drug discovery, function annotation, and protein engineering. AlphaFold’s structural embeddings compress protein geometry into comparable vector representations.

Knowledge Graph completion — link prediction and entity alignment in knowledge graphs (Wikidata, Freebase) use embeddings to infer missing relations and merge overlapping graphs, improving the completeness of structured knowledge repositories.

Computational law and finance — regulatory documents, contracts, and financial disclosures embedded for semantic retrieval, enabling legal search systems that match provisions by conceptual intent rather than keyword presence. Used by UK legal technology firms and financial regulators.

Standards and Interoperability

There is no single formal standard governing embedding formats, training methodologies, or evaluation protocols. Interoperability relies on de facto conventions established by dominant libraries and commercial APIs. The most widely adopted serialisation formats are flat binary files of 32-bit floats in numpy .npy format, HuggingFace safetensors (a safe, portable format with metadata), and ONNX (Open Neural Network Exchange), which enables cross-framework inference of embedding models on CPU and GPU hardware from different vendors.

The Sentence-Transformers library (Reimers & Gurevych; hosted at UKPLab/sentence-transformers on GitHub, now jointly maintained by HuggingFace) has emerged as the de facto standard for open-source embedding model distribution, training, and evaluation. It provides a unified interface for over 5,000 pre-trained models, standardised evaluation utilities covering MTEB tasks, and training utilities supporting contrastive, triplet, and multiple-negatives objectives. The library’s SentenceTransformer class is the most widely used embedding API in the open-source ecosystem, adopted by downstream libraries including LangChain, LlamaIndex, Haystack, and Semantic Kernel.

Commercial embedding APIs have converged on a JSON-over-HTTPS interface pattern: POST request with text string(s), response containing array(s) of floating-point vectors. OpenAI’s Embeddings API (text-embedding-3-small, text-embedding-3-large), Cohere’s Embed API (Embed v4), Voyage AI’s embedding endpoints, and Google’s Gemini Embedding API all follow this convention. Most Vector Database vendors (Pinecone, Weaviate, Qdrant) natively consume these API responses and provide client libraries that handle the encode-then-upsert pipeline transparently.

The MTEB (Massive Text Embedding Benchmark) leaderboard functions as an informal standard for model quality: models listed on MTEB have been evaluated under controlled conditions on standard datasets, enabling objective comparison. MTEB’s evaluation code is open-source (mteb/mteb on GitHub), and community contributions of new datasets and languages are encouraged. As of 2026, MTEB covers over 1,000 languages and 58 English task datasets spanning retrieval, clustering, re-ranking, classification, pair classification, semantic textual similarity, bitext mining, and summarisation.

ONNX Runtime (Microsoft) provides a standardised inference runtime for embedding models that enables deployment across CPUs (x86, ARM), GPUs (CUDA, ROCm), and specialised accelerators without framework dependencies. This is particularly important for embedding models deployed at the network edge — in client applications, mobile devices, or IoT hardware — where Python runtimes and PyTorch are impractical.

Academic Context

The embedding literature spans foundational distributional semantics (Harris 1954; Firth 1957) through neural word representations (Bengio et al. 2003, neural language model with distributed word representations as a byproduct) to the modern industrialised form. The NeurIPS 2013 Word2Vec paper (Mikolov et al.) is among the most-cited papers in the history of machine learning, with over 30,000 citations as of 2026, making it the most-cited NLP paper in history by a substantial margin. Its impact extended far beyond word representations: the skip-gram training procedure was adapted for graph nodes (DeepWalk, Node2Vec), knowledge graph entities (TransE), code tokens (Code2Vec), and molecular substructures (mol2vec), establishing prediction-based self-supervised learning as a general paradigm for learning representations from co-occurrence structure in arbitrary relational data.

BERT (Devlin et al., NAACL 2019) catalysed an entire sub-field of pre-trained encoder models, generating thousands of derivative works adapting the architecture to specific domains, languages, and resource constraints: SciBERT (scientific text), BioBERT (biomedical), ClinicalBERT (clinical notes), LegalBERT (legal documents), FinBERT (financial news), mBERT (104 languages), XLM-R (cross-lingual with 100 languages), DistilBERT (40% smaller, 60% faster), TinyBERT (7.5× smaller via knowledge distillation). This proliferation reflects the practical usefulness of the pre-trained encoder paradigm across domains and the ease of fine-tuning on task-specific data.

The MTEB benchmark (Muennighoff et al., 2023, now maintained at the ILCC Edinburgh and HuggingFace) has become the community-accepted standard evaluation suite. The contrastive learning literature underpinning modern embedding training draws heavily on SimCLR (Chen et al., Google Brain, 2020) and MoCo (He et al., FAIR, 2020) from the vision domain, subsequently adapted for text in SimCSE (Gao et al., Princeton, 2021) and for multi-positive contrastive learning in E5 (Microsoft Research, 2022). The progression from static word-level to contextual sentence-level to instruction-tuned document-level embeddings reflects iterative problem-solving driven by the needs of production retrieval systems rather than purely theoretical motivations — each generation of embedding methods was driven by the failure of the previous generation in specific deployment contexts.

Current Landscape (2026)

The 2026 embedding landscape is characterised by four converging trends. First, open-source models have achieved near-parity with proprietary APIs: Alibaba’s Qwen3-Embedding-8B achieves MTEB score 70.6, competitive with frontier commercial offerings. The availability of high-quality open embedding models enables organisations with privacy constraints or budget limitations to deploy retrieval systems without dependence on external API providers, a significant shift from 2023 when OpenAI’s text-embedding-ada-002 was the near-universal default for production RAG systems.

Second, multimodal joint embeddings have become production-ready: Google’s Gemini Embedding 2 (released March 2026) embeds text, images, video, audio, and PDF documents into a single 3,072-dimensional space. Cohere’s Embed v4 supports interleaved text-image document embeddings for multimodal RAG, enabling retrieval over documents that mix text and diagrams — critical for technical documentation, scientific papers, and engineering drawings. This extends the Retrieval-Augmented Generation paradigm from text-only corpora to the full richness of real-world enterprise content.

Third, Matryoshka Representation Learning has been widely adopted, enabling deployments to select embedding dimensions adaptively — 128, 256, 512, 1024 or 3072 dimensions from a single model depending on recall requirements and latency budgets. Production pipelines increasingly use two-stage retrieval: 128-dimensional vectors for billion-scale candidate generation at sub-millisecond latency, followed by 1024-dimensional vectors for precise ranking of the top-1000 candidates, all from a single MRL-trained model.

Fourth, the Vector Database market has consolidated around a handful of mature systems: Pinecone (serverless cloud), Weaviate (open-source with cloud managed offering), Qdrant (Rust-based high-performance), Milvus (large-scale, CNCF project), and the pgvector PostgreSQL extension enabling vector search within existing relational database infrastructure. Annual recurring revenue in the vector database segment exceeded $200 million in 2025–2026. The MTEB leaderboard now evaluates over 200 models across more than 1,000 languages, and rankings are updated weekly, making it the authoritative signal for embedding model selection in production systems.

UK Context

UK academic institutions have made substantial contributions to the embedding research ecosystem across five key areas:

Edinburgh — The University of Edinburgh’s Institute for Language, Cognition and Computation (ILCC) produced foundational knowledge graph embedding work (Balazevic, Allen & Hospedales, hypernetwork KGEs extending the TransE family). The School of Informatics maintains research groups in distributional semantics, cross-lingual representation learning, and low-resource language embeddings.

London — University College London’s machine learning group has contributed to structured representation learning and uncertainty quantification in embeddings. Imperial College London’s Department of Computing applies embeddings to biomedical language models and protein structure prediction. King’s College London’s NLP group works on clinical text embeddings for NHS electronic health records. The Alan Turing Institute coordinates cross-institution embedding and data-centric AI research across the national network.

Cambridge — The Cambridge Language Technology Lab maintains active research in cross-lingual word embeddings and knowledge graph completion. The European Bioinformatics Institute (EMBL-EBI, Hinxton) applies protein language model embeddings to genomic annotation and drug target identification.

Northern England — In the Northern English industrial context:

• Manchester: University of Manchester’s Centre for AI Fundamentals applies embedding models to biomedical Knowledge Graph completion and drug-target interaction prediction. Manchester-based financial technology firms apply embedding-based fraud detection and customer segmentation at scale.

• Sheffield: The University of Sheffield’s NLP group (historically one of the UK’s leading computational linguistics departments under established researchers including Kalina Bontcheva and Yorick Wilks) applies embeddings to information extraction, clinical text analysis, and social media monitoring.

• Leeds: University of Leeds contributes to legal NLP using sentence embeddings for contract analysis and contributes to the NHS data-sharing infrastructure for clinical record embedding.

• Newcastle: Newcastle University’s Digital Institute applies embeddings to cultural heritage collections and social history text corpora.

  NHS and Public Sector — The National Health Service AI Lab (NHSX) uses document embedding for clinical note retrieval, patient record linkage, and clinical coding automation across NHS trusts. The NICE Evidence Reviews use embedding-based literature search to identify relevant clinical trials from PubMed, supplementing manual systematic review. UK Government’s GCHQ / NCSC applies embedding-based approaches to cyber threat intelligence, malware classification, and network traffic anomaly detection.

  Legal and Financial Services — UK legal technology firms (Luminance, RAVN Systems, Thought River) apply sentence and document embeddings for contract review, due diligence, and regulatory compliance. UK financial institutions (Barclays AI Labs, HSBC Analytics, Standard Chartered AI) apply embedding-based fraud detection, customer churn prediction, and document intelligence to process financial documentation at scale. The Financial Conduct Authority (FCA) has explored embedding-based analysis of regulatory filings to identify anomalies and compliance gaps across regulated firms.

  Emerging UK initiatives — The National AI Research and Innovation Programme (AIRP) funds embedding research for climate science (satellite image embeddings for land use change detection), heritage digitisation (embedding of archival photograph and manuscript collections for cross-collection search), and genomics (genome sequence embeddings for variant interpretation at NHS clinical genetics services). These programmes reflect the breadth of application domains where the embedding paradigm is generating productive new research at the intersection of AI and established scientific disciplines.

  The EPSRC Network Plus in Natural Language Processing (NLP4Science) coordinates UK academic embedding research across Edinburgh, Sheffield, Cambridge, Manchester, and King’s College London, focusing on domain-specific embedding evaluation protocols and reproducibility standards across low-resource and specialised language communities. The network specifically addresses the challenge of evaluating embedding quality for scientific domains where ground-truth semantic similarity is not well defined by human annotators who lack domain expertise — requiring novel evaluation protocols based on expert knowledge, experimental outcomes, and structured databases rather than crowdsourced annotation.

  Wales and other devolved nations — Cardiff University’s NLP group applies Welsh-English cross-lingual embeddings to Welsh language technology, supporting the Welsh Government’s ambition for one million Welsh speakers by 2050. Swansea University’s Medical Informatics group applies clinical note embeddings to Welsh NHS data under the SAIL Databank framework, enabling population-scale health research while preserving patient privacy through secure trusted research environments.

  Future Directions (2026–2030)

  The primary frontier is the integration of embedding infrastructure with agentic AI systems: embeddings will serve as the memory and retrieval backbone for autonomous agents, with real-time indexing of agent-generated observations into queryable vector stores. When an AI agent reads a document, processes a tool call result, or generates an intermediate reasoning step, those events can be embedded and stored in a searchable episodic memory, enabling the agent to retrieve relevant past experience when encountering analogous situations. This positions embeddings as the infrastructure layer for what AI researchers call “long-term memory” in agent architectures — the equivalent of human episodic and semantic memory, implemented as a high-throughput vector index that the agent queries with its current context.

  Long-context embedding models will extend effective context windows to full-book or full-repository scale, enabling semantic search over entire code bases or document libraries without chunking. Current production systems typically chunk documents into 256–512 token segments before embedding, losing cross-chunk semantic coherence. Models like NV-Embed and GTE-Qwen supporting 32K-token contexts begin to address this, and the next generation of long-context embedding models will handle megabyte-scale inputs as a unit. This will transform Retrieval-Augmented Generation from paragraph-level retrieval to chapter-level or file-level retrieval, dramatically improving answer quality for complex multi-hop questions spanning long documents.

  Learned sparse embeddings (SPLADE, ELSER) will complement dense embeddings in hybrid retrieval pipelines, combining the recall advantages of dense vectors with the precision advantages of sparse term matching. Hybrid search — combining BM25 sparse retrieval with dense embedding retrieval using reciprocal rank fusion — has become standard practice in production Retrieval-Augmented Generation systems, consistently outperforming either approach alone across recall and precision metrics. Future systems will make this hybridisation learnable end-to-end rather than combining independently-trained components.

  Adaptive quantisation schemes will push billion-scale embedding indices into consumer hardware, enabling privacy-preserving on-device semantic search for sensitive personal data (medical records, private communications) that cannot be transmitted to cloud APIs. Binary quantisation with learned correction factors can achieve near-lossless compression; combined with product quantisation, this enables one-billion-vector indices to fit on devices with 8GB of RAM. The convergence of embedding and Large Language Model architectures will continue, with frontier LLMs serving directly as embedding models through last-layer pooling or dedicated embedding training stages. In computational biology, protein and genome embeddings will enable systematic function prediction across novel organisms, extending the reach of Embeddings beyond language into the most fundamental data modality of living systems. The UK’s emerging AI regulatory framework (DSIT, AI Safety Institute) will drive demand for auditable, interpretable embedding representations whose geometric structure can be inspected for demographic bias, discriminatory clustering, and stereotype amplification.

  Research and Literature

  1. Harris, Z. S. (1954). Distributional structure. Word, 10(2–3), 146–162. Foundational distributional hypothesis.
  2. Firth, J. R. (1957). A synopsis of linguistic theory, 1930–1955. Studies in Linguistic Analysis. Oxford: Blackwell. “Know a word by the company it keeps.”
  3. Bengio, Y., Ducharme, R., Vincent, P., & Janvin, C. (2003). A neural probabilistic language model. Journal of Machine Learning Research, 3, 1137–1155. First neural word representations.
  4. Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6), 391–407.
  5. Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. ICLR 2013. arXiv:1301.3781. Word2Vec.
  6. Pennington, J., Socher, R., & Manning, C. D. (2014). GloVe: Global vectors for word representation. EMNLP 2014. Global co-occurrence matrix factorisation.
  7. Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017). Enriching word vectors with subword information. TACL, 5, 135–146. FastText.
  8. Peters, M., Neumann, M., Iyyer, M., et al. (2018). Deep contextualized word representations. NAACL 2018. arXiv:1802.05365. ELMo.
  9. Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. NAACL 2019. arXiv:1810.04805.
  10. Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using siamese BERT-networks. EMNLP 2019. arXiv:1908.10084.
  11. Radford, A., Kim, J. W., Hallacy, C., et al. (2021). Learning transferable visual models from natural language supervision. ICML 2021. CLIP.
  12. Gao, T., Yao, X., & Chen, D. (2021). SimCSE: Simple contrastive learning of sentence embeddings. EMNLP 2021. arXiv:2104.08821.
  13. Wang, L., Yang, N., Huang, X., et al. (2022). Text embeddings by weakly-supervised contrastive pre-training. arXiv:2212.03533. E5 embedding family.
  14. Kusupati, A., Bhatt, G., Rege, A., et al. (2022). Matryoshka representation learning. NeurIPS 2022. arXiv:2205.13147.
  15. Muennighoff, N., Tazi, N., Magne, L., & Reimers, N. (2023). MTEB: Massive text embedding benchmark. EACL 2023. arXiv:2210.07316.
  16. Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., & Yakhnenko, O. (2013). Translating embeddings for modeling multi-relational data. NeurIPS 2013. TransE.
  17. Sun, Z., Deng, Z.-H., Nie, J.-Y., & Tang, J. (2019). RotatE: Knowledge graph embedding by relational rotation in complex space. ICLR 2019. arXiv:1902.10197.
  18. Balazevic, I., Allen, C., & Hospedales, T. (2019). Hypernetwork knowledge graph embeddings. ICANN 2019. University of Edinburgh.
  19. Xiao, S., Liu, Z., Zhang, P., & Muennighoff, N. (2023). C-Pack: Packaged resources to advance general Chinese embedding. arXiv:2309.07597. BGE embedding family.
  20. Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020). A simple framework for contrastive learning of visual representations. ICML 2020. SimCLR.
  21. Girdhar, R., El-Nouby, A., Liu, Z., et al. (2023). ImageBind: One embedding space to bind them all. CVPR 2023. arXiv:2305.05665.
  22. Lin, S.-C., et al. (2023). SPLADE-v3: New baselines for SPLADE. arXiv:2309.10085. Learned sparse embeddings.
  23. Feng, Z., Guo, D., Tang, D., et al. (2020). CodeBERT: A pre-trained model for programming and natural languages. EMNLP 2020. arXiv:2002.08155.
  24. Rives, A., Meier, J., Sercu, T., et al. (2021). Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. PNAS, 118(15). ESM protein embeddings.
  25. Cohere. (2025). Embed v4: Multimodal embeddings for production RAG. Technical report. First production text+image+document embedding API.
  26. Google. (2026). Gemini Embedding 2: All-modality embeddings for text, image, video, audio, and PDF. Technical report, March 2026.
  27. Ailog RAG. (2026). MTEB 2026: State of the embeddings benchmark. Retrieved from https://app.ailog.fr/en/blog/news/rag-benchmark-mteb-2026
  28. Awesome Agents. (2026). Embedding model leaderboard: MTEB rankings March 2026. Retrieved from https://awesomeagents.ai/leaderboards/embedding-model-leaderboard-mteb-march-2026/

  Bias, Fairness, and Interpretability

  Embedding spaces learned from large corpora inherit and amplify biases present in training data. The seminal study by Bolukbasi et al. (2016) demonstrated that Word2Vec embeddings encode gender stereotypes: the embedding analogy “man : doctor :: woman : ?” consistently returns “nurse” rather than “doctor”, reflecting the statistical skew in training corpora rather than an accurate representation of real-world occupations. Similar biases have been documented for racial, national, religious, and socioeconomic categories across embedding families from Word2Vec through BERT to modern sentence encoders. These biases propagate downstream into applications: Recommendation Systems that embed user profiles and item descriptions may systematically recommend different products to users inferred to be from different demographic groups; clinical text embeddings may underperform for patient populations underrepresented in training data.

  De-biasing techniques include geometrically projecting out gender or race subspaces identified by principal component analysis (Bolukbasi et al. 2016), adversarial training with a discriminator that cannot predict protected attributes from embeddings (Zhang et al. 2018), and data augmentation with counterfactual examples (Zhao et al. 2018). However, de-biasing is technically difficult to define precisely — some geometric correlations between embedding dimensions and demographic attributes may encode legitimate distributional facts rather than harmful stereotypes — and the removal of all demographic correlates may reduce embedding quality for downstream tasks where demographic factors carry legitimate predictive weight (e.g., dialect-aware clinical NLP).

  The UK’s Equality Act 2010 and the emerging AI regulation framework (DSIT 2023 AI White Paper; pro-innovation approach with sector-specific principles) create obligations for organisations deploying embedding-based systems in employment, credit, and public service contexts to evaluate and document demographic performance disparities. The AI Safety Institute has identified embedding bias as a cross-cutting concern across multiple high-risk AI application domains.

  Formal Treatment and Mathematical Foundations

  An embedding is formally a function f: X → ℝ^d mapping from an input domain X (a vocabulary, image space, graph node set, or other discrete set) to a d-dimensional real vector space. The function f is learned by minimising an objective that encourages semantic or functional proximity to correspond to geometric proximity. For Word2Vec skip-gram, the objective maximises the log-probability of observing context words c given a target word w:

  J = Σ_{(w,c)} log σ(v_w · v_c) + k · E_{c’~P_n}[log σ(-v_w · v_{c’})]

  where v_w and v_c are the embedding vectors, σ is the sigmoid function, k is the number of negative samples, and P_n is the noise distribution. For contrastive embedding models such as CLIP and Sentence-BERT, the InfoNCE loss aligns positive pairs (x_i, x_i^+) while repelling negative pairs:

  L = -log [exp(sim(f(x_i), f(x_i^+))/τ) / Σ_j exp(sim(f(x_i), f(x_j))/τ)]

  where sim is cosine similarity and τ is a temperature hyperparameter. For Matryoshka Representation Learning, the loss is computed at multiple prefix lengths d_1 < d_2 < … < d_m = d simultaneously, jointly optimising all truncations of the embedding.

  The geometry of embedding spaces has been extensively studied. The isotropy problem (embeddings concentrating in a narrow cone of the vector space) was identified in BERT-derived representations and addressed through post-processing (whitening, PCA rotation) and improved training objectives. The intrinsic dimensionality of semantic embedding manifolds is substantially lower than the nominal embedding dimension, motivating quantisation and compression research. Cosine similarity rather than Euclidean distance is preferred for comparison, as it is scale-invariant — the magnitude of an embedding vector carries no semantic content in most architectures.

  Benchmark Datasets and Evaluation

  The primary evaluation framework is the Massive Text Embedding Benchmark (MTEB) (Muennighoff et al., 2023), which evaluates embedding models across 8 task categories: retrieval, clustering, classification, pair classification, re-ranking, semantic textual similarity (STS), summarisation, and bitext mining. MTEB covers 58 English datasets and over 1,000 languages in its multilingual extension (MMTEB). The leaderboard (mteb.clspace.com) is updated continuously and is the de facto standard for model selection in production systems.

  Key retrieval benchmarks within MTEB include MS-MARCO (passage retrieval, 8.8M passages, Microsoft), BEIR (18 heterogeneous retrieval datasets including TREC-COVID, HotpotQA, and FiQA), and Natural Questions. For semantic similarity, the STS-Benchmark (STSb) and SICK-R datasets provide human-rated sentence pair scores. For domain-specific evaluation, MTEB includes BioASQ (biomedical), SciFact (scientific fact-checking), and ArguAna (argument retrieval).

  Retrieval metrics include nDCG@10 (normalised discounted cumulative gain at rank 10), MAP (mean average precision), and Recall@K. Clustering metrics include v-measure and adjusted Rand index. Production embedding model providers additionally publish latency benchmarks (queries per second), storage footprint (bytes per vector), and accuracy-compression tradeoffs for quantised variants.

  The Sentence-Transformers library (GitHub: UKPLab/sentence-transformers) is the canonical open-source implementation, providing over 5,000 pre-trained models and standardised evaluation utilities. HuggingFace Hub hosts the majority of publicly available embedding models in safetensors format.

  Key Terminology

  Embedding space / latent space — the continuous d-dimensional vector space in which embedded representations reside. Semantic structure in the input domain is encoded as geometric structure (distance, angle, direction) in this space.

  Cosine similarity — the preferred distance metric for comparing embeddings; measures the angle between two vectors irrespective of their magnitudes, ranging from -1 (opposite directions) to +1 (identical directions). Insensitive to vector scale, suitable for comparing embeddings of varying norms.

  Approximate Nearest Neighbour (ANN) — algorithms for finding the K most similar vectors to a query in large collections without exhaustive comparison. Hierarchical Navigable Small World (HNSW) graphs and Inverted File (IVF) indices are the dominant approaches, enabling billion-scale retrieval with sub-millisecond latency.

  Contrastive learning — a self-supervised training paradigm that learns representations by pulling together positive pairs (similar items) and pushing apart negative pairs (dissimilar items). The dominant training approach for modern embedding models, used in CLIP, SimCSE, E5, and their derivatives.

  Matryoshka Representation Learning (MRL) — a training technique enabling a single model to produce embeddings that remain meaningful when truncated to any prefix length, enabling adaptive dimension selection at inference time without retraining.

  Positional encoding — a mechanism for injecting sequential order information into the position-invariant self-attention computation. Absolute sinusoidal (Vaswani et al. 2017), learned absolute, Rotary Position Embedding (RoPE), and ALiBi (Attention with Linear Biases) are the major variants.

  Fine-tuning / instruction tuning — adapting a pre-trained embedding model to a specific domain, task type, or query style by continued training on labelled or synthetically generated data. Instruction-tuned models (E5-instruct, NV-Embed-v2) condition embedding generation on a textual description of the task type.

  Vector database — a storage and retrieval system specialised for dense vector embeddings, providing ANN search, metadata filtering, and real-time insertion. Major examples include Pinecone, Weaviate, Qdrant, Milvus, Chroma, and the pgvector PostgreSQL extension.

  Quantisation — compressing floating-point embedding vectors to lower bit depths (int8, binary). Binary quantisation (1 bit per dimension) reduces storage by 32x relative to float32 while retaining approximately 95% of retrieval quality, enabling billion-scale deployments on commodity hardware.

  Knowledge graph embedding — the application of embedding techniques to knowledge graph entities and relations, mapping nodes and edges into continuous space such that link prediction reduces to geometric operations. TransE, RotatE, ComplEx, and QuatE are the foundational families.

Provenance