Narrative Goldmine

Quality Assurance

7 min read

Properties

Type
  • Class
Resource
  • urn:ngm:class:quality-assurance
Domain
  • infrastructure
Status
  • stable
Maturity
  • mature
Quality
  • 0.72
Is a
Has part
Requires
Enables
Generated
  • by: process:vault-migrate/1.0 · at: 2026-09-22T12:41:39.024503178Z
Bridges to
Contrasts with
Legacy term id
  • MV-9208
Links
    Page resource
    • urn:visionflow:page:7774a5d76d809d773090d8342e053b35bd7303c2d70e43c1eb5263994c6d80aa
    Related to
    Standardized by
    Supports
    Uses

    Quality Assurance (QA) is the systematic discipline of establishing and maintaining defined standards of correctness, reliability, safety, and fitness-for-purpose across the full lifecycle of software systems, AI models, and digital infrastructure. It encompasses planned and systematic activities — including requirements analysis, process audits, test design, validation, verification, and continuous monitoring — that prevent defects from reaching production rather than merely detecting them after the fact. In AI contexts, QA extends beyond functional correctness to encompass model fairness, adversarial robustness, data quality, and distributional-shift monitoring, increasingly mandated by governance frameworks such as the EU AI Act and ISO/IEC 42001. Effective QA integrates with DevOps and MLOps pipelines through automated gates that enforce quality thresholds before any artefact is promoted to the next deployment stage.

    Overview

    • QA originated in manufacturing (W. Edwards Deming, Walter Shewhart) and was formalised in software engineering through standards such as IEEE 829 and CMM. Its defining characteristic is proactive defect prevention rather than reactive detection, achieved by embedding quality criteria into every phase of the development lifecycle — from specification through design, implementation, integration, and deployment.
    • In modern software delivery, QA is operationalised through automated Continuous Integration gates that execute test suites, Static Analysis, and security scanners on every commit. Any artefact failing a defined threshold is blocked from progressing to staging or production.
    • For AI/ML systems, QA has expanded substantially. Model Evaluation on held-out benchmarks, fairness audits across demographic subgroups, Adversarial Testing against distributional shifts and prompt injections, and data-pipeline quality checks via Data Validation are all QA responsibilities. MLOps platforms such as MLflow and Kubeflow operationalise these checks as reproducible pipeline stages.
    • Regulatory drivers — particularly the EU AI Act (2024) and ISO/IEC 42001 — now mandate documented QA evidence for high-risk AI deployments, creating a direct link between QA practice and Compliance Framework obligations.

    Key Components

    • Test Planning and Design
      • Defining test scope, coverage criteria, entry/exit conditions, and acceptance criteria aligned with Requirements Engineering artefacts.
      • Generating test cases from equivalence partitions, boundary values, decision tables, and use-case scenarios.
    • Functional Testing
      • Software Testing executed at unit, integration, system, and acceptance levels to verify functional correctness.
      • Regression Testing suites prevent previously fixed defects from re-emerging across release cycles.
    • Non-Functional Testing
      • Performance, load, and stress testing against throughput and latency requirements.
      • Security testing including penetration testing and vulnerability scanning (feeds into Risk Management).
      • Accessibility testing against WCAG standards.
    • Static Analysis and Code Quality
      • Static Analysis tools (e.g. SonarQube, ESLint, mypy) detect bugs, code smells, and security vulnerabilities without execution.
      • Code Review processes enforce coding standards and distribute knowledge.
    • Test Automation
      • Test Automation frameworks (e.g. Selenium, Cypress, pytest, JUnit) enable fast, repeatable validation inside Continuous Integration pipelines.
      • Shift-left strategies move test execution earlier in the development cycle to reduce remediation cost.
    • AI/ML-Specific QA
      • Evaluation of model performance on representative held-out datasets using metrics appropriate to the task (accuracy, F1, AUC-ROC, BLEU, etc.).
      • Fairness in AI auditing: demographic parity, equalised odds, and counterfactual fairness checks across protected attribute groups.
      • Adversarial Testing: red-teaming, robustness benchmarks, and prompt-injection testing for language models.
      • Data quality checks (Data Validation): schema validation, distribution monitoring, and label-consistency audits.
      • Model drift detection in production to trigger retraining when distributional shift degrades performance.
    • Formal Methods
      • Formal Verification techniques (model checking, theorem proving) provide mathematical guarantees for safety-critical components.
      • Contract-based design (Design by Contract) embeds pre/post-conditions as executable QA artefacts.
    • Process Audits
      • ISO/IEC 25010 quality model audits assess product characteristics: functional suitability, reliability, usability, efficiency, maintainability, portability, security.
      • CMMI, Agile QA retrospectives, and Six Sigma process reviews identify systemic quality risks.

    Applications and Use Cases

    • Enterprise Software Development — QA gates within Jira/Azure DevOps workflows enforce test coverage thresholds and static-analysis scores before merge, feeding Continuous Delivery pipelines.
    • Safety-Critical Systems — Aerospace (DO-178C), automotive (ISO 26262), and medical devices (IEC 62304) mandate rigorous QA artefacts including traceability matrices, code coverage metrics, and hazard analysis.
    • AI Model Deployment — MLOps teams run pre-deployment QA checklists covering accuracy benchmarks, fairness metrics, latency SLAs, and adversarial robustness scores before promoting a model to production. Model Governance dashboards track these over time.
    • Financial Systems — QA in algorithmic trading and credit-risk models includes backtesting, stress testing, and model-risk management reviews mandated by SR 11-7 (US Federal Reserve) and EBA guidelines.
    • Spatial Computing and XR — Observability hooks and frame-rate regression testing validate that spatial computing applications meet perceptual quality thresholds (frame time, tracking accuracy, rendering fidelity) across hardware configurations.
    • Blockchain and Smart Contracts — Formal verification and audit-driven QA processes validate smart contract logic against specification before deployment to immutable ledgers, where defects cannot be patched post-release.
    • Open-Source Projects — Community-driven QA through CI bots (GitHub Actions), mandatory review policies, and fuzz-testing pipelines maintain quality without centralised QA teams.

    Standards and Governance Context

    • ISO/IEC 25010:2023 — Defines the Systems and Software Quality Model with eight top-level quality characteristics and thirty-one sub-characteristics used to specify, measure, and evaluate product quality.
    • ISO/IEC 42001:2023 — AI Management System standard mandating documented QA processes, risk controls, and performance monitoring for organisations developing or deploying AI.
    • IEEE 829 — Standard for Software and System Test Documentation specifying structure of test plans, design specifications, case and procedure specifications, incident reports, and summary reports.
    • EU AI Act (2024) — For high-risk AI systems, requires technical documentation demonstrating that QA activities were conducted, including accuracy evaluation, robustness testing, and cybersecurity testing.
    • ISTQB (International Software Testing Qualifications Board) — De facto global certification body providing the Foundations, Advanced, and Expert syllabus levels that standardise QA practitioner knowledge.
    • CMMI (Capability Maturity Model Integration) — Process-improvement framework that benchmarks organisational QA capability across five maturity levels; commonly used in defence and government contracting.
    • Six Sigma DMAIC — Data-driven QA improvement methodology (Define, Measure, Analyse, Improve, Control) widely applied in enterprise software and hardware manufacturing contexts.

    Semantic Classification

    Current Landscape (2026)

    • The defining shift of 2025-2026 is the move from AI-assisted to agentic testing: autonomous agents that observe an application, reason about what to test, generate cases from plain-language prompts or user stories, execute them, prioritise by risk and analyse failures with minimal per-cycle human direction. Playwright Test Agents brought multi-agent frameworks from research into production, and Tricentis, Mabl, Functionize and testers.ai now ship closed-loop agentic platforms.
    • Self-healing test automation matured into a baseline expectation rather than a differentiator, with leading systems reporting 80-90% reductions in test-maintenance effort by identifying UI elements through multi-attribute ML recognition instead of rigid selectors.
    • Adoption is now mainstream: dev.to/Currents reporting cites roughly 81% of teams using AI in testing workflows, ThinkSys puts AI-first quality-engineering adoption at ~77.7%, and the global software-testing market is projected to grow from about USD 55.8B (2024) to USD 112.5B (2034) at ~7.2% CAGR.
    • The vendor field spans AI-native autonomous tools (Momentic, Meticulous, testers.ai), managed AI-plus-human QA (QA Wolf, Rainforest QA, Bug0), AI-assisted platforms (Mabl, Testsigma, LambdaTest KaneAI, ACCELQ) and visual AI (Applitools), while newer categories such as vision-based mobile automation and “vibe/experience testing” of AI-generated UIs emerged through 2026.
    • Regulation reshaped QA scope: the EU AI Act became applicable on 2 August 2026, mandating documented quality-management systems (Article 17) and conformity assessment for high-risk AI providers, with high-risk Annex III obligations extended to 2 December 2027 under the AI Omnibus simplification.
    • On standards, prEN 18286 (Artificial Intelligence - Quality Management System for EU AI Act Regulatory Purposes) entered public enquiry on 30 October 2025 as the first harmonised AI standard, sitting alongside established frameworks such as ISO/IEC 25010, ISO/IEC 5055 and ISO/IEC 12207 that do not yet explicitly address AI tooling.
    • Open challenges as of 2026 centre on trust and governance: fully autonomous “deploy-and-forget” agents remain experimental, so supervised autonomy with human review before commitment to regression suites is the realistic norm, and QA is increasingly tasked with validating the AI systems that now help build and test the software itself.

    References

    Provenance

    Graph View

    Backlinks