Experiment Tracking

Experiment tracking is the practice of recording the configuration, code, data, and results of machine learning experiments so they can be compared and reproduced, enabling teams to audit, iterate, and roll back to prior model states. It is a core discipline within MLOps.

Semantic Classification

Content

Compositional Relationships (Components)

SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:hasPart ai:HyperparameterLogging))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:hasPart ai:MetricLogging))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:hasPart ai:ArtifactVersioning))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:hasPart ai:RunComparison))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:hasPart ai:ModelRegistry))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:hasPart ai:PromptManagement))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:hasPart ai:EnvironmentSnapshot))

Dependency Relationships

SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:requires ai:ModelTraining))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:requires ai:DataVersioning))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:requires ai:Reproducibility))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:requires ai:VersionControl))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:requires ai:MetadataManagement))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:requires ai:DataPipeline))

Capability Relationships

SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:enables ai:ModelComparison))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:enables ai:ModelRegistry))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:enables ai:ContinuousTraining))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:enables ai:HyperparameterTuning))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:enables ai:AIGovernance))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:enables ai:Reproducibility))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:enables ai:CollaborativeResearch))

Implementation Relationships

SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:implements ai:ScientificMethod))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:implements ai:AuditTrail))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:implements ai:DataLineage))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:implements ai:ModelLineage))

Reduction Relationships

SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:reducesTo ai:MetadataLogging))
SubClassOf(ai:ExperimentTracking
  ObjectSomeValuesFrom(ai:reducesTo ai:RunManagement))

About

Experiment tracking emerged as a named discipline in the 2017–2019 period, coinciding with the rapid industrialisation of machine learning. Before dedicated tooling existed, practitioners tracked experiments in spreadsheets, naming conventions on file systems, or personal notebooks — approaches that broke down entirely at team scale or when experiments numbered in the hundreds. The key recognition was that machine learning research methodology differed fundamentally from classical software engineering: a model’s behaviour depends not only on code but on the specific dataset version it trained on, the exact hyperparameter configuration, the random seed, the framework version, and the hardware environment. Omitting any one of these details makes an experiment irreproducible.

The first widely adopted open-source solution was MLflow, released by Databricks in 2018, which introduced the concept of a “run” as the atomic unit of tracking and provided a minimal API: mlflow.log_param(), mlflow.log_metric(), mlflow.log_artifact(). Its simplicity and open-source licensing drove rapid adoption. Simultaneously, Weights & Biases (W&B) launched a more opinionated, cloud-native offering with richer visualisation and a developer experience designed for fast adoption by research teams. Neptune.ai positioned itself as a metadata database for production ML teams managing thousands of runs.

By 2026, experiment tracking has undergone a fundamental expansion in scope driven by the generative AI era. Classical ML experiment tracking logged scalar metrics — accuracy, loss, AUC — against hyperparameter configurations. GenAI experiment tracking must additionally handle prompt versions, LLM call chains with associated token costs, multi-turn conversation datasets used for evaluation, LLM-as-judge scores, and latency distributions across model endpoints. MLflow responded with MLflow 3.x (2025–2026), which introduced LLM Tracing built on OpenTelemetry, a Prompt Registry for versioning prompt templates, and native GenAI evaluation frameworks including LLM-as-Judge scoring. Weights & Biases launched W&B Weave as a purpose-built GenAI evaluation and tracing layer sitting alongside its classical experiment tracking infrastructure.

A significant market event in 2026 was Neptune.ai’s acquisition by OpenAI and the shutdown of its SaaS platform in March 2026, consolidating the market further around MLflow and W&B while creating migration demand for teams that had built workflows on Neptune’s API.

Components / Architecture

A complete experiment tracking system comprises the following components:

Run Management:

• A “run” is the atomic record of a single training or evaluation execution. Each run captures a unique ID, start/end time, status (running/completed/failed), and links to the commit hash of the training code.

• Runs are grouped into “experiments” (logical groupings by task or model family) and optionally into “projects” or “sweeps” for hyperparameter search campaigns.

• The run record is the primary object that all other components attach to.

  Hyperparameter Logging:

• Records all configuration values that govern a training run: model architecture choices, learning rate, batch size, regularisation coefficients, data augmentation settings, and framework-specific options.

• Hyperparameters are logged as key-value pairs before or at the start of training.

• Enables systematic comparison: given two runs with different outcomes, hyperparameter tables immediately surface the differing configuration.

• Integration with Hyperparameter Tuning frameworks (Optuna, Ray Tune, W&B Sweeps) allows automated search results to be captured in the tracking database.

  Metric Logging:

• Time-series capture of scalar metrics during training: training loss, validation loss, accuracy, F1 score, perplexity, BLEU score, ROUGE, or any custom metric.

• Step-indexed (batch or epoch) so that convergence curves can be compared across runs.

• In GenAI settings, metrics extend to token counts, latency percentiles (p50/p95/p99), cost per call, and LLM-as-Judge scores (faithfulness, relevance, hallucination rate).

• Streaming metrics allow real-time monitoring of active runs alongside historical comparisons.

  Artifact Versioning:

• Stores binary outputs of training: model weights checkpoints, serialised model files (PyTorch .pt, ONNX, SavedModel), evaluation result files, confusion matrices, SHAP value plots, and dataset samples.

• Artifacts are content-addressed (hashed) and stored in object storage (S3, GCS, Azure Blob) with the tracking server maintaining pointer metadata.

• Downstream: Model Registry consumes versioned artifacts from the tracking server to create promotable model versions.

  Environment Snapshot:

• Captures the software environment: Python version, library versions (pip freeze / conda environment), CUDA version, GPU model, Docker image tag or hash.

• Critical for reproducibility: without the environment snapshot, re-running a historical experiment may produce different results due to library API changes.

• Some systems (MLflow, DVC) also snapshot the git commit hash or diff, ensuring code reproducibility.

  Run Comparison and Visualisation:

• Side-by-side comparison tables of hyperparameters and final metrics across selected runs.

• Metric trajectory plots enabling visual identification of overfitting, instability, or faster convergence.

• Parallel coordinate plots for multi-objective hyperparameter sweep visualisation.

• W&B and MLflow both provide web UIs with interactive filtering, sorting, and grouping.

  Model Registry Integration:

• Promotion workflow: a run’s model artifact can be registered in the model registry, receiving a version tag and transitioning through stages (development → staging → production).

• Governance gate: model promotion typically requires documented evaluation results captured in the tracking run, creating an auditable link between the deployed model and its training evidence.

• Rollback: the registry maintains all previous versions, allowing one-click redeployment of any historical model version.

  LLM Tracing (GenAI Extension):

• Distributed traces capturing every step of an LLM call chain: prompt construction, LLM API call (with full request/response), tool calls, retrieval steps, and final output generation.

• Built on OpenTelemetry in MLflow 3.x, ensuring vendor-neutral trace collection compatible with standard observability infrastructure.

• Enables debugging of multi-step agent workflows, identification of latency bottlenecks, and cost attribution per sub-step.

  Prompt Registry (GenAI Extension):

• Versions prompt templates used in LLM applications, analogous to how artifact versioning versions model weights.

• Enables A/B comparison of prompt variants with evaluation results attached to each version.

• MLflow Prompt Registry (2025–2026) supports templated prompts with variable injection and links prompt versions to evaluation run records.

  Tooling Landscape (2026)

  MLflow (Databricks / open-source, v3.x): The most widely deployed open-source experiment tracking platform. MLflow’s architecture comprises a Tracking Server (REST API + UI), a backend store (SQLite, PostgreSQL, or cloud-managed), and an artifact store (local, S3, GCS, Azure Blob). MLflow 3.x (2025–2026) added full GenAI support: LLM Tracing via OpenTelemetry, Prompt Registry, native integration with LangChain/LlamaIndex/OpenAI SDK, and built-in LLM-as-Judge evaluation. Organisations preferring on-premises or self-hosted deployments favour MLflow for its licensing and Databricks integration with the Delta Lake ecosystem.

  Weights & Biases (W&B): Cloud-native experiment tracking with best-in-class visualisation, developer experience, and a strong following in research settings. W&B’s 2024–2026 pivot to GenAI produced W&B Weave: a dedicated GenAI tracing, evaluation, and monitoring layer. Weave integrates with W&B’s classical experiment tracking dashboard, providing a unified view of both classical ML runs and LLM evaluation results. W&B Sweeps provides tightly integrated hyperparameter search. W&B holds the strongest market position as of 2026 based on developer adoption surveys.

  DVC (Data Version Control, Iterative.ai): Open-source tooling focused on data and model versioning built on top of Git. DVC stores large files (datasets, model weights) in object storage with Git-tracked metadata pointers, enabling Git workflows for ML. DVC Experiments provides lightweight experiment tracking integrated with the DVC pipeline system, appealing to teams that want to keep ML versioning within their existing Git infrastructure.

  Comet ML: Cloud experiment tracking platform with strong enterprise features, code reproducibility tracking, and integrations with Databricks and SageMaker. Acquired by IBM in 2023; continues as an independent product with IBM ecosystem integration.

  Kubeflow Pipelines (KFP) + Metadata: Kubernetes-native ML workflow orchestration with integrated metadata tracking. KFP captures pipeline run metadata — component inputs/outputs, execution times, artifact lineage — in an ML Metadata (MLMD) store. Preferred in organisations with existing Kubernetes infrastructure and a need for tightly coupled pipeline orchestration and experiment metadata.

  Amazon SageMaker Experiments, Google Vertex AI Experiments, Azure ML: Cloud provider managed experiment tracking integrated into their respective ML platform stacks. Lower operational overhead for teams already committed to a cloud provider; limited portability compared to open-source alternatives.

  Use Cases / Major Families

  Hyperparameter Search Campaigns: The most common use case for experiment tracking. A hyperparameter sweep launches tens to hundreds of runs with different configurations; the tracking server captures all results and provides comparison tables and parallel coordinate plots to identify the optimal configuration. Frameworks such as Optuna, Ray Tune, and W&B Sweeps log results directly to the tracking server. Without experiment tracking, sweep results exist only in log files, making it impossible to systematically compare configurations or reproduce the best run.

  Team Collaboration on Model Development: In multi-person ML teams, experiment tracking provides a shared view of all runs across team members, preventing duplicate work and enabling async collaboration. A researcher returning after holiday can immediately see what experiments colleagues ran, what metrics they achieved, and which model artifacts were produced, without needing to ask. This is the social layer of reproducibility.

  Regulatory Compliance and AI Governance: The EU AI Act (in force from 2024) requires high-risk AI systems to maintain technical documentation linking deployed models to their training data, configuration, and evaluation results. Experiment tracking — specifically the run record linking a model artifact to its dataset version, hyperparameters, and evaluation metrics — is the primary artefact satisfying this documentation requirement. NIST AI RMF (2023) similarly emphasises traceability as a key governance control. Model cards and factsheets required by responsible AI frameworks draw directly from tracked experiment metadata.

  Continuous Training Loop Validation: In production MLOps pipelines, models are retrained periodically on fresh data. Experiment tracking validates each retraining run by capturing the new run record, comparing evaluation metrics against the previous model version’s record, and gating promotion only if the new model meets the threshold. This automated comparison relies entirely on the tracked metrics from the incumbent model’s run.

  Debugging Model Regressions: When a deployed model’s production metrics degrade, experiment tracking provides the investigative trail. Engineers can compare the current deployed model’s training run record against previous successful versions, identifying differences in data snapshot, hyperparameters, or code version that correlate with the regression. Without the full run record, root cause analysis is guesswork.

  LLM Prompt Engineering and Evaluation: In GenAI applications, practitioners iterate over prompt variants seeking improved output quality on evaluation datasets. W&B Weave and MLflow Prompt Registry treat each prompt version as a tracked experiment, recording evaluation metrics (LLM-as-Judge scores, task-specific metrics) against each prompt version. This extends the experiment tracking paradigm from parameter space to the combinatorial space of natural language prompt formulations.

  Academic Reproducibility: ML conferences increasingly require experiment tracking records as reproducibility artefacts. NeurIPS, ICML, and ICLR reproducibility tracks request that authors share tracked run records — captured in MLflow or W&B — alongside code and model checkpoints. Experiment tracking directly addresses the reproducibility crisis documented in ML research, where reported results frequently cannot be replicated even with the published code.

  Academic Context

  The intellectual roots of experiment tracking lie in the philosophy of science and the experimental record-keeping traditions of physical sciences — lab notebooks as metadata management. In machine learning, the reproducibility crisis was documented prominently by Joelle Pineau and colleagues at NeurIPS 2017–2019, who found that a substantial proportion of reported results could not be reproduced, motivating systematic logging requirements. Sculley et al.’s “Hidden Technical Debt in Machine Learning Systems” (NIPS 2015) identified the absence of systematic configuration and experiment management as a primary source of technical debt in production ML. Google’s MLOps whitepaper (2020) formally positioned experiment tracking as a level-0 requirement, the minimum baseline for any operationalised ML system.

  The connection between experiment tracking and the Scientific Method is theoretically grounded: experiment tracking implements the reproducibility, falsifiability, and documentation requirements of scientific methodology as engineering infrastructure. Amershi et al. (2019) “Software Engineering for Machine Learning: A Case Study” (Microsoft Research, ICSE) documented experiment tracking as the highest-priority tooling need identified by professional ML engineers. Zhao et al. (2022) “Operationalizing Machine Learning: An Interview Study” systematically documented how experiment tracking failures lead to months of wasted engineering effort in production teams.

  UK academic groups have contributed to reproducibility methodology. The Alan Turing Institute’s reproducibility programme, led by researchers including Kirstie Whitaker, has produced guidelines for experiment reporting in ML. The Edinburgh DataShare repository and the University of Manchester’s ReproducibilityCentre have promoted open experiment records for published research. The UK Reproducibility Network (UKRN) coordinates reproducibility standards across disciplines that are now influencing ML publication standards.

  Current Landscape (2026)

  The experiment tracking market has undergone dramatic growth and consolidation. The AI experiment tracking market was valued at approximately $1.58billionin2024andisprojectedtoreach$19.55 billion by 2032 (Intel Market Research, 2025), driven primarily by the generative AI pivot.

  GenAI Dominates the Feature Roadmap: All major platforms have pivoted their 2025–2026 roadmaps toward GenAI experiment management. MLflow 3.10 (March 2026) is now described as an “Open Source AI Platform for Agents, LLMs & Models” rather than purely an experiment tracker, reflecting the scope expansion. Classical ML metric logging is now table-stakes; differentiation occurs in LLM tracing quality, prompt versioning, and multi-turn evaluation datasets.

  Neptune Shutdown (March 2026): Neptune.ai was acquired by OpenAI and shut down its SaaS experiment tracking platform in March 2026. Teams that had built on Neptune’s API are migrating to MLflow or W&B, with ZenML, Comet, and DVC also capturing displaced workloads.

  OpenTelemetry as the Standard Substrate: MLflow’s 3.x LLM tracing layer is built directly on OpenTelemetry with GenAI semantic conventions, positioning experiment tracking as a specialised application of distributed observability infrastructure. This blurs the boundary between experiment tracking and production monitoring, which is intentional: the same trace format captures both development experiment runs and production inference traces.

  EU AI Act Compliance Driving Enterprise Adoption: The EU AI Act’s Article 9 (risk management) and Article 11 (technical documentation) requirements for high-risk AI systems have driven enterprise procurement of experiment tracking platforms with governance features — immutable run records, access controls, and audit export capabilities. Databricks MLflow with Unity Catalog integration and W&B enterprise tier with SSO and role-based access are the primary beneficiaries.

  LLM Evaluation as the New Hyperparameter Tuning: Prompt engineering workflows in 2026 involve evaluating dozens of prompt variants on curated evaluation datasets with LLM-as-Judge scorers, generating hundreds of evaluation runs per application. This mirrors the hyperparameter tuning workflow of classical ML and has made experiment tracking essential for GenAI application development teams, not just model training teams.

  UK Context

  UK adoption of experiment tracking spans both the research and enterprise sectors. The Alan Turing Institute (ATI), based at the British Library in London with nodes at partner universities including Edinburgh, Manchester, Cambridge, Oxford, and UCL, actively promotes experiment tracking standards as part of its reproducible research programme. ATI fellows working on healthcare AI — a particularly compliance-sensitive domain given NHS data governance requirements — have documented experiment tracking as a prerequisite for any clinical AI deployment.

  DeepMind (now Google DeepMind, headquartered in London) has been a major adopter and contributor to experiment tracking tooling, with internal systems that informed the design of public platforms. Its scale of experimentation — thousands of GPU-days per experiment campaign — makes systematic tracking essential. Google DeepMind uses experiment tracking to enforce reproducibility standards for published research aligned with its commitment to open science.

  In Northern England, the industrial AI cluster centred on Manchester, Leeds, and Sheffield has seen growing adoption of MLOps tooling including experiment tracking driven by regulatory requirements in the finance and healthcare sectors. The Leeds Teaching Hospitals NHS Trust and Manchester University NHS Foundation Trust have both piloted AI model governance frameworks that include experiment tracking requirements as part of their model deployment approval processes. The University of Sheffield’s Machine Learning group, historically strong in speech technology and now active in NLP and healthcare AI, uses MLflow for experiment management across its research portfolio.

  The Edinburgh DataShare platform and the University of Edinburgh’s School of Informatics have promoted open experiment records alongside published research. The Scottish AI Strategy (updated 2024) explicitly references reproducibility and model governance as requirements for public sector AI deployments, driving experiment tracking adoption in Scottish government digital services.

  The UK’s DSIT AI Safety Institute (AISI) has incorporated model documentation requirements into its Frontier AI Safety Framework that align with experiment tracking output formats, creating a governance pathway from experiment records to safety evaluation reports.

  Future Directions (2026–2030)

  Automated Experiment Intelligence: AI-assisted experiment analysis that goes beyond comparison tables, automatically identifying which hyperparameter changes drove metric improvements across a corpus of historical runs, generating natural-language summaries of experiment campaigns, and proposing next experiment configurations. MLflow and W&B have begun deploying LLM-powered experiment analysis in 2025–2026; this will mature significantly by 2028.

  Unified Classical and GenAI Tracking: The current bifurcation between classical ML experiment tracking (loss curves, hyperparameters) and GenAI experiment tracking (prompt traces, LLM judge scores) will converge into a unified experiment record schema that handles both modalities, driven by the growing prevalence of hybrid systems that combine trained models with LLM components.

  Federated Experiment Tracking: Multi-organisation collaborative research — common in healthcare and climate AI — requires experiment records that span organisational boundaries without sharing raw data. Federated experiment metadata protocols, building on the OpenTelemetry substrate, will enable cross-institutional experiment comparisons while preserving data sovereignty.

  Causal Experiment Analysis: Moving beyond correlation-based run comparison to causal attribution of metric changes to specific intervention variables, using techniques from causal inference. This addresses the fundamental limitation that comparing runs with multiple simultaneous changes cannot establish which change caused the observed improvement.

  Hardware-Aware Tracking: As AI hardware diversifies (NVIDIA GPUs, Google TPUs, AWS Trainium/Inferentia, Intel Gaudi, UK-based Graphcore IPUs), experiment tracking will need to capture hardware-specific performance profiles — memory bandwidth utilisation, tensor core occupancy, inter-chip communication latency — as first-class metadata, enabling reproducibility across heterogeneous compute environments.

  Regulatory Integration: Direct integration with regulatory submission workflows — EU AI Act technical documentation templates, MHRA guidance for medical device AI, FCA requirements for financial services AI — will transform experiment tracking from a developer tool into a compliance system with structured export formats accepted by regulatory bodies.

  Research & Literature

  1. Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., … & Dennison, D. (2015). Hidden technical debt in machine learning systems. Advances in Neural Information Processing Systems (NeurIPS), 28.
  2. Amershi, S., Begel, A., Bird, C., DeLine, R., Gall, H., Kamar, E., … & Zimmermann, T. (2019). Software engineering for machine learning: A case study. ICSE 2019: International Conference on Software Engineering.
  3. Google Cloud. (2020). MLOps: Continuous delivery and automation pipelines in machine learning. Google Cloud whitepaper. practitioners.google/guides/mlops.
  4. Kreuzberger, D., Kühl, N., & Hirschl, S. (2023). Machine learning operations (MLOps): Overview, definition, and architecture. IEEE Access, 11, 31866–31879.
  5. Zhao, L., Li, B., Le, N. D., Zheng, J., Huang, W., Zhu, Q., … & Zhang, H. (2022). Operationalizing machine learning: An interview study. arXiv:2209.09125.
  6. Pineau, J., Vincent-Lamarre, P., Sinha, K., Larivière, V., Beygelzimer, A., d’Alché-Buc, F., … & Larochelle, H. (2021). Improving reproducibility in machine learning research. Journal of Machine Learning Research, 22(1), 7459–7478.
  7. MLflow Documentation. (2026). MLflow 3.x: Open source AI platform for agents, LLMs and models. https://mlflow.org/docs/latest/
  8. Weights & Biases. (2025). Introducing W&B Weave: GenAI evaluation and tracing. https://wandb.ai/site/weave
  9. Huyen, C. (2022). Designing Machine Learning Systems. O’Reilly Media. (Chapter 6: Model Development and Offline Evaluation)
  10. Gift, N., & Deza, A. (2021). Practical MLOps. O’Reilly Media. (Chapter 4: Continuous Delivery for Machine Learning Models)
  11. Klaise, J., Van Looveren, A., & Vacanti, G. (2021). Alibi Explain: Algorithms for explaining machine learning models. Journal of Machine Learning Research, 22(181), 1–7.
  12. Intel Market Research. (2025). AI experiment tracking market outlook 2026–2034. https://www.intelmarketresearch.com/ai-experiment-tracking-market-46972
  13. Shankar, S., Garcia, R., Hellerstein, J. M., & Parameswaran, A. (2022). Operationalizing machine learning: An interview study. VLDB 2022.
  14. Vartak, M., Subramanyam, H., Lee, W.-E., Viswanath, S., Madden, S., & Zaharia, M. (2016). ModelDB: A system for machine learning model management. HILDA Workshop, SIGMOD 2016.
  15. Zaharia, M., et al. (2018). Accelerating the machine learning lifecycle with MLflow. IEEE Data Engineering Bulletin, 41(4), 39–45.
  16. NIST. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). National Institute of Standards and Technology. NIST AI 100-1.
  17. European Commission. (2024). EU Artificial Intelligence Act (Regulation 2024/1689). Official Journal of the European Union.
  18. Sculley, D., et al. (2018). Winner’s curse? On pace, progress, and empirical rigor. ICLR 2018 Workshop.
  19. Gundersen, O. E., & Kjensmo, S. (2018). State of the art: Reproducibility in artificial intelligence. AAAI 2018.
  20. Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., & Meger, D. (2018). Deep reinforcement learning that matters. AAAI 2018.
  21. Lones, M. A. (2021). How to avoid machine learning pitfalls: A guide for academic researchers. arXiv:2108.02497. (UK context: University of Edinburgh)
  22. Alan Turing Institute. (2024). Reproducibility and open research in AI. ATI Technical Report.
  23. UK Government DSIT. (2024). AISI Frontier AI Safety Framework. Department for Science, Innovation and Technology.
  24. Raghunathan, A., Bhatt, U., Leike, J., et al. (2024). Reproducibility: The new frontier in AI governance. arXiv:2510.11595.
  25. ZenML. (2025). Neptune.ai vs MLflow vs ZenML: Which ML experiment tracking stack should you use? https://www.zenml.io/blog/neptune-ai-vs-mlflow
  26. SGAnalytics. (2026). Top 20 MLOps tools in 2026. https://www.sganalytics.com/blog/mlops-tools/
  27. Kolekar, R. (2026). MLOps in 2026 — the definitive guide. https://rahulkolekar.com/mlops-in-2026-the-definitive-guide-tools-cloud-platforms-architectures-and-a-practical-playbook/

Provenance