Risk Assessment

Risk Assessment is the systematic, evidence-based process of identifying, analysing, and evaluating hazards, vulnerabilities, and adverse-outcome chains across complex sociotechnical systems, forming the core analytical activity within broader Risk Management programmes aligned to

Semantic Classification

Content

Compositional Relationships (Components)

SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:hasPart gov:HazardIdentification))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:hasPart gov:VulnerabilityAnalysis))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:hasPart gov:ImpactModelling))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:hasPart gov:LikelihoodEstimation))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:hasPart gov:RiskEvaluation))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:hasPart gov:RiskTreatmentPlan))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:hasPart gov:ResidualRiskCalculation))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:hasPart gov:RiskRegister))

## Dependency Relationships
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:requires gov:ThreatModel))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:requires gov:StakeholderAnalysis))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:requires gov:DataGovernance))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:requires gov:AuditTrail))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:dependsOn gov:ProbabilityTheory))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:dependsOn gov:InformationTheory))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:dependsOn gov:SystemsEngineering))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:dependsOn gov:BehaviouralScience))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:dependsOn gov:BayesianInference))

## Capability Relationships
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:enables gov:AIGovernance))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:enables gov:RegulatoryCompliance))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:enables gov:OperationalResilience))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:enables gov:ModelAssurance))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:enables gov:IncidentResponse))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:supports gov:EUAIActConformity))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:supports gov:AISystemSafety))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:supports gov:DeFiRiskControl))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:supports gov:FinancialStability))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:supports gov:SmartContractSecurity))

## Implementation Relationships
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:implements gov:ISO31000))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:implements gov:NISTAIRiskManagementFramework))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:implements gov:FMEA))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:implements gov:BowTieAnalysis))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:implements gov:FaultTreeAnalysis))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:implements gov:OWASPLLMTop10))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:implements gov:MITREATLAS))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:uses gov:RedTeaming))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:uses gov:ScenarioAnalysis))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:uses gov:MonteCarloSimulation))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:uses gov:AttackTrees))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:uses gov:AIIncidentDatabase))

## Reduction Relationships
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:reduces gov:UnmitigatedRiskExposure))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:reduces gov:RegulatoryPenaltyRisk))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:reduces gov:SystemicContagionRisk))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:reduces gov:ModelHarmProbability))
SubClassOf(gov:RiskAssessment
  ObjectSomeValuesFrom(gov:reduces gov:OperationalFailureImpact))

## Data Properties (Characteristics)
DataPropertyAssertion(gov:hasIdentifier gov:RiskAssessment "IF-0201"^^xsd:string)
DataPropertyAssertion(gov:authorityScore gov:RiskAssessment "0.87"^^xsd:decimal)
DataPropertyAssertion(gov:nistRMFFunctions gov:RiskAssessment "4"^^xsd:integer)
DataPropertyAssertion(gov:euAIActRiskTiers gov:RiskAssessment "4"^^xsd:integer)
DataPropertyAssertion(gov:genAIProfileRiskTypes gov:RiskAssessment "12"^^xsd:integer)
DataPropertyAssertion(gov:owaspLLMVulnCount gov:RiskAssessment "10"^^xsd:integer)
DataPropertyAssertion(gov:fsbCryptoRecommendations gov:RiskAssessment "9"^^xsd:integer)

## Property Constraints
SubClassOf(gov:RiskAssessment
  DataAllValuesFrom(gov:requiresDocumentedScope xsd:boolean))
SubClassOf(gov:RiskAssessment
  DataSomeValuesFrom(gov:riskAppetiteThreshold xsd:string))
SubClassOf(gov:RiskAssessment
  DataMinCardinality(1 gov:hasRiskRegister xsd:string))
SubClassOf(gov:RiskAssessment
  DataMinCardinality(1 gov:hasControlSet xsd:string))

## Annotations
AnnotationAssertion(rdfs:label gov:RiskAssessment "Risk Assessment"@en)
AnnotationAssertion(rdfs:comment gov:RiskAssessment "Systematic process identifying, analysing, and evaluating hazards and adverse-outcome chains in AI/blockchain systems, implementing ISO 31000, NIST AI RMF 1.0 (GOVERN/MAP/MEASURE/MANAGE), NIST GenAI Profile (12 risk types), EU AI Act Article 9, FSB crypto recommendations, FMEA, bow-tie analysis, OWASP LLM Top 10, and MITRE ATLAS, enabling regulatory compliance, operational resilience, and model assurance."@en)
AnnotationAssertion(dcterms:identifier gov:RiskAssessment "IF-0201"^^xsd:string)
AnnotationAssertion(dcterms:subject gov:RiskAssessment "Risk Management, AI Safety, Regulatory Compliance, Blockchain Risk, Governance"@en)

)

Property Characteristics

AsymmetricObjectProperty(gov:requires) AsymmetricObjectProperty(gov:enables) AsymmetricObjectProperty(gov:implements) AsymmetricObjectProperty(gov:reduces) TransitiveObjectProperty(gov:dependsOn) FunctionalDataProperty(gov:riskAppetiteThreshold) FunctionalDataProperty(gov:authorityScore)

About Risk Assessment

• Risk Assessment is the analytical engine at the heart of modern risk governance, combining probabilistic reasoning, structured hazard elicitation, and stakeholder-informed evaluation to produce actionable prioritisations of threats and vulnerabilities.
• Across AI, financial services, and blockchain domains, risk assessment has evolved from static checklists into continuous, data-driven processes that integrate real-time monitoring, adversarial testing, and retrospective incident learning.
• The discipline draws on three complementary intellectual traditions. First, engineering safety analysis — fault trees, FMEA, HAZOP, and bow-tie models originating in nuclear and aerospace industries. Second, financial risk quantification — value-at-risk, expected shortfall, stress testing, and scenario analysis developed in banking regulation under Basel accords. Third, information security threat modelling — STRIDE, attack trees, kill-chain frameworks, and penetration testing.
• The emergence of AI systems as a distinct risk class has driven synthesis of these traditions into AI-specific frameworks — most notably NIST AI RMF 1.0 (2023) and its GenAI Profile (2024) — that address the unique characteristics of ML systems: opacity, emergent behaviour, distribution shift, and dual-use capability uplift.
• In blockchain and DeFi contexts, risk assessment must additionally model the adversarial economics of decentralised protocols: flash-loan attacks, oracle manipulation, governance capture, liquidity cascade failures, and cross-chain bridge exploits. The Financial Stability Board (FSB) Crypto-Asset Risk Recommendations (2022, updated 2023) establish the policy expectation that both private actors and supervisors maintain live risk assessments of crypto-asset markets, with specific attention to systemic contagion pathways between crypto and traditional finance.
• Risk assessment is not a one-time activity but an iterative, lifecycle-spanning discipline. ISO 31000:2018 mandates that risk assessment be embedded within a continuous risk management process integrated into organisational governance, strategy, and operations — iterating as the risk context (technology, regulation, threat landscape, stakeholder expectations) evolves.

Components / Architecture

• Risk assessment processes in AI and blockchain contexts decompose into five canonical phases, each with distinct methods, artefacts, and tooling.

Phase 1 — Scope and Context Establishment

• Before hazards can be enumerated, assessors must define the risk boundary: which system components, data pipelines, deployment contexts, and stakeholder populations are in scope.
• For AI systems, this maps onto the NIST AI RMF MAP function: defining the AI system’s intended purpose, operational context, affected communities, and value chain (training data sources, third-party model components, downstream integrators).
• EU AI Act Article 9(2) mandates that high-risk AI risk management systems document the intended purpose, reasonably foreseeable misuse, and post-deployment monitoring scope before conformity assessment.
• Key artefacts produced at this phase include: system boundary diagram; stakeholder impact register; use-case taxonomy; data provenance map; third-party dependency graph.
• For blockchain systems this additionally requires: smart contract interaction graph; oracle data source registry; governance token distribution analysis; cross-chain bridge dependency mapping.
• ISO 31000 clause 6.3 requires context establishment to cover: external context (regulatory environment, market structure, societal expectations, technology landscape); internal context (organisational objectives, governance structures, risk culture, existing controls); and risk assessment criteria (likelihood scales, impact scales, risk tolerability thresholds, risk appetite statement).

Phase 2 — Hazard Identification

• Hazard identification generates the comprehensive catalogue of what could go wrong. Multiple complementary techniques are applied in practice.

FMEA (Failure Mode and Effects Analysis)

• Originating in US military MIL-P-1629 (1949) and standardised in IEC 60812, FMEA systematically enumerates failure modes of each system component, their effects on overall system function, and their detectability.
• For AI systems, failure modes include: training data poisoning, distribution shift at inference time, adversarial input perturbation, model extraction via API queries, and output hallucination in high-stakes decision contexts.
• Each failure mode receives a Risk Priority Number (RPN = Severity × Occurrence × Detection, each scored 1–10) enabling prioritised treatment. RPN >100 typically triggers mandatory mitigation. Design FMEA (DFMEA) is applied at the model architecture level; Process FMEA (PFMEA) at the data pipeline and inference pipeline level.
• For blockchain protocols, FMEA failure modes encompass: reentrancy vulnerabilities in EVM smart contracts; integer overflow/underflow; flash loan vector exposure; proxy contract upgrade key compromise; and governance proposal spam enabling vote exhaustion attacks.

Bow-Tie Analysis

• A graphical technique representing threats as the left side (causes → hazardous event) and consequences as the right side (hazardous event → top event → escalating consequences), with barriers displayed as vertical lines crossing the bow-tie.
• Particularly effective for visualising escalation pathways in DeFi protocol failures: e.g., oracle price manipulation (cause) → incorrect liquidation price (hazardous event) → mass undercollateralised position liquidation (top event) → liquidity cascade → protocol insolvency (consequence).
• For AI governance, bow-tie analysis maps bias injection pathways (cause) → discriminatory output (hazardous event) → regulatory enforcement (consequence), with preventive barriers (bias testing during training) and recovery barriers (human override mechanisms) explicitly represented.
• The bow-tie method is particularly valued in FCA and PRA risk governance because it provides a visual narrative of risk escalation that non-technical executives and board members can interpret, facilitating informed risk appetite decisions.

MITRE ATLAS (Adversarial Threat Landscape for AI Systems)

• A structured knowledge base of adversarial machine learning tactics, techniques, and procedures (TTPs) modelled after MITRE ATT&CK.
• ATLAS v4 (2024) catalogues 14 tactic categories: Reconnaissance, Resource Development, Initial Access, Execution, Persistence, Privilege Escalation, Defence Evasion, Discovery, Collection, ML Attack Staging, Exfiltration, Impact, ML Model Access, and ML Attack Execution — with 80+ techniques specific to ML pipelines.
• Example ATLAS techniques relevant to LLM risk assessment: AML.T0051 (LLM Prompt Injection — injecting adversarial instructions via user-supplied inputs processed by the LLM without sanitisation); AML.T0048 (Societal Harm — generating disinformation, hate speech, CSAM); AML.T0043 (Craft Adversarial Data — crafting inputs that cause misclassification); AML.T0019 (Publish Poisoned Datasets — embedding backdoor triggers in publicly released training datasets).
• MITRE ATLAS is a standard reference in red-team playbooks at Anthropic, Microsoft, Google DeepMind, and constitutes the primary TTP framework used in UK AISI model evaluations.

OWASP LLM Top 10 v2 (2025)

• The Open Worldwide Application Security Project’s ranking of the ten most critical security vulnerabilities in LLM-integrated applications.
• LLM01 Prompt Injection: attacker manipulates LLM behaviour via crafted inputs, overriding system prompt instructions — most prevalent LLM vulnerability, exploitable in agentic tool-use contexts.
• LLM02 Sensitive Information Disclosure: LLM leaks training data, PII, credentials, or system prompts — common when system prompt contains sensitive configuration.
• LLM03 Supply Chain Vulnerabilities: compromised base models, poisoned fine-tuning datasets, or malicious model hub artefacts — affects organisations consuming third-party models.
• LLM04 Data and Model Poisoning: manipulating training data to embed backdoor behaviours triggered by specific inputs — particularly relevant for instruction-tuned models.
• LLM05 Improper Output Handling: downstream systems directly executing LLM-generated content (SQL queries, shell commands, HTML) without sanitisation — enables injection via LLM as intermediary.
• LLM06 Excessive Agency: agentic LLM takes irreversible real-world actions (deleting files, making purchases, sending emails) beyond authorised scope — critical risk for autonomous agent deployments.
• LLM07 System Prompt Leakage: system prompt extracted via crafted user queries, revealing confidential instructions, business logic, or personas.
• LLM08 Vector and Embedding Weaknesses: poisoned embeddings in RAG vector stores misdirect retrieval, degrading output quality or injecting adversarial content into context.
• LLM09 Misinformation: LLM confidently produces factually incorrect outputs that are consumed without verification — risk amplified by authoritative presentation style.
• LLM10 Unbounded Consumption: adversary triggers expensive LLM computations (long context, repeated queries) depleting compute budgets — denial-of-service via economic resource exhaustion.

STPA (Systems-Theoretic Process Analysis)

• Nancy Leveson’s (MIT) model-based safety analysis method that analyses unsafe control actions rather than component failures, making it particularly suited to AI systems where emergent behaviour — not component breakdown — is the primary hazard.
• STPA identifies unsafe control actions: when provided in the wrong context; not provided when needed; provided too early or too late; stopped too soon.
• Applied to AI systems, STPA maps the hierarchical control structure (regulators → deployers → AI system → users → environment) and analyses feedback loop inadequacies that prevent detection of hazardous states.
• STPA has been adopted by the UK AISI in its ARAM framework for analysing AI safety hazards in autonomous systems deployed in critical infrastructure, particularly where emergent multi-step reasoning chains can generate unsafe actions not anticipated in the individual component analysis.

Phase 3 — Risk Analysis

• Risk analysis quantifies identified hazards on two dimensions: likelihood (probability of occurrence given current controls) and impact (consequence severity across relevant harm categories).

Likelihood Quantification Methods

• Historical incident rate data: AI Incident Database (2014–2026, 900+ documented AI harms); FSB crypto-asset incident registry; CERT/CC and CVE databases for software vulnerability exploitation rates.
• Bayesian network propagation: structural causal models encode conditional dependencies between risk factors, enabling prior probability updating as new evidence (audit findings, penetration test results, market signals) accrues.
• Expert elicitation via structured Delphi methods: multiple expert rounds converging on probability estimates for novel risks lacking historical base rates — standard for CBRN uplift risk assessment where ground-truth data is deliberately limited.
• Monte Carlo simulation for tail-risk distributions: particularly applied in DeFi liquidity models to estimate Value-at-Risk (VaR) and Expected Shortfall (ES) for liquidity pool positions under adversarial market conditions.
• Attack-tree probability propagation: P(top-event) = product of leaf node probabilities for AND gates; union probability for OR gates — enabling quantitative fault tree analysis linking control failures to incident probabilities.

Impact Modelling Dimensions

• EU AI Act harm taxonomy distinguishes: health and safety impacts (physical harm, death); fundamental rights infringements (discrimination, privacy violation, freedom of expression); economic harm (financial loss, market manipulation); and societal harm (undermining democratic processes, social cohesion, rule of law).
• For financial systems, impact measurement uses Expected Shortfall (ES/CVaR) at 99.9th percentile, Value-at-Risk (VaR) under stressed conditions, and systemic risk metrics (CoVaR, SRISK, Delta-CoVaR) measuring contagion contribution of individual entities to system-wide losses.
• NIST AI RMF MEASURE function defines six harm dimensions: individual-level (bias, privacy, safety); aggregate-level (systemic bias, market manipulation, disinformation); organisational-level (reputational, operational, legal); ecosystem-level (model homogenisation reducing diversity of AI approaches, concentration risk in foundation model providers); societal-level (democratic integrity, trust in institutions); and global-level (geopolitical risks from AI capability asymmetries, CBRN uplift from dual-use models).
• For DeFi systemic risk, impact modelling uses contagion propagation models adapted from interbank network models (Eisenberg-Noe liquidation cascades), applied to on-chain credit and liquidity relationships to estimate total protocol losses from a single large liquidation event or oracle manipulation.

Phase 3b — Scenario Analysis and Stress Testing

• Scenario analysis complements probabilistic risk analysis by constructing specific narrative futures — plausible combinations of risk factor values — and evaluating system performance under those scenarios. Unlike Monte Carlo simulation which samples across the full probability distribution, scenario analysis focuses effort on worst-case plausible scenarios that may lie in the tail of probability distributions but have disproportionate consequence severity.
• For AI systems, standard scenario types include: capability surprise scenarios (a model demonstrates a capability threshold crossing — e.g., sustained autonomous replication — that was not anticipated from pre-deployment evaluations); adversarial deployment scenarios (a state-level actor fine-tunes an open-weight model to maximise CBRN synthesis assistance and deploys it publicly); infrastructure interdependency failure (a major cloud provider AI API outage simultaneously disabling AI-dependent services across financial services, healthcare, and critical infrastructure); and misinformation cascade scenarios (a coordinated disinformation campaign using AI-generated content degrades public trust in a democratic election within a 72-hour window).
• For DeFi, FSB-endorsed scenario types include: stablecoin depeg scenario (largest stablecoin loses peg triggering mass redemptions, liquidity drain across DeFi protocols, and correlated crypto-TradFi contagion); bridge exploit scenario (cross-chain bridge exploited for $1B+ causing confidence collapse in the exploited blockchain’s ecosystem); regulatory shock scenario (G7 coordination to ban DeFi protocol access, triggering mass capital flight and governance token collapse).
• Stress testing extends scenario analysis by applying scenarios of increasing severity — from plausible adverse through severe but plausible to catastrophic tail-risk — to identify the threshold at which current controls cease to provide adequate protection. PRA SS3/21 requires UK financial services firms to maintain documented stress scenarios for ML model failures as part of model risk management frameworks.

Phase 4 — Risk Evaluation

• Risk evaluation compares analysed risk levels against predetermined tolerance thresholds and appetite statements, producing prioritised treatment recommendations.

ALARP (As Low As Reasonably Practicable)

• Health and Safety Executive (UK) tolerable risk principle establishing three zones: broadly acceptable (risk so low further reduction not required); ALARP (risk tolerable if further reduction impracticable or grossly disproportionate to benefit); and broadly unacceptable (risk intolerable regardless of controls).
• Adopted by UK AISI for AI risk tolerability assessments. The ALARP demonstration burden scales with harm magnitude: for catastrophic-consequence risks (CBRN uplift, mass-casualty AI-enabled attacks), the grossly disproportionate test is applied more stringently, placing heavier obligation on developers to implement controls.
• ALARP documentation requires evidence that all reasonable risk reduction measures have been identified, evaluated for practicability and cost-benefit, and implemented where the benefit is not grossly disproportionate to cost — a legal due-diligence standard referenced in HSE prosecutions and increasingly in AI liability litigation.

EU AI Act Risk Tiers

• Article 6–7 establish four tiers. Tier 1, Unacceptable risk (prohibited): social scoring by public authorities; real-time biometric surveillance in public spaces without exceptions; AI exploiting psychological vulnerabilities; predictive policing based solely on profiling.
• Tier 2, High risk (conformity assessment required including Article 9 risk management, technical documentation, transparency obligations): medical devices, critical infrastructure management, employment decisions, biometric categorisation, migration control, administration of justice, critical digital infrastructure.
• Tier 3, Limited risk (transparency obligations only): chatbots with disclosure obligation; emotion recognition with disclosure; deep-fake generation with labelling.
• Tier 4, Minimal risk (no EU AI Act obligations, though other laws apply): spam filters; AI in video games; AI image enhancement. High-risk AI operators must maintain risk management systems throughout the lifecycle, with continuous monitoring and post-market incident reporting to national market surveillance authorities.

NIST AI RMF Risk Prioritisation

• The MEASURE function requires risk prioritisation matrices mapping likelihood (1–5) against impact (1–5) to produce a 25-cell heat map, with residual risk after controls plotted to demonstrate treatment effectiveness.
• The GenAI Profile (NIST AI 600-1) adds a “Unique Risk Rating” for each of the 12 GenAI-specific risks, incorporating both impact severity and prevalence of current mitigations in the industry — enabling relative prioritisation across confabulation, privacy, CBRN uplift, and other GenAI-specific risks.
• NIST AI RMF Action Roadmaps (published alongside AI RMF 1.0) provide 1,000+ suggested actions mapped to GOVERN/MAP/MEASURE/MANAGE functions across six risk management outcomes (A1: Policies established; A2: Accountability assigned; A3: Organisational risks documented; A4: Teams trained; A5: Risks prioritised; A6: Risks treated). These roadmap actions form the basis for organisational AI risk assessment maturity assessments.

Phase 5 — Red-Teaming and Empirical Validation

• For AI systems, quantitative risk matrices must be validated through empirical adversarial testing — systematic attempts by skilled humans to elicit harmful behaviours from AI systems under realistic conditions.

Anthropic Red-Teaming Practice

• Anthropic’s Constitutional AI development process includes iterative red-teaming against a harm taxonomy covering: CBRN uplift (biological weapon synthesis routes, radiological dispersal device design); cyberweapon generation (working malware, exploit code, vulnerability discovery acceleration); disinformation at scale; grooming and CSAM; autonomous replication and resource acquisition; power-seeking behaviour; and sycophancy exploitable by bad actors.
• Findings inform RLHF reward model updates and Constitutional AI constitutional revision. The Responsible Scaling Policy (RSP) ties deployment decisions to red-team findings against defined “Catastrophic Risk” thresholds — models triggering threshold violations are subject to enhanced containment before release consideration.
• Anthropic also conducts third-party red-teaming partnerships with RAND Corporation, UK AISI, and independent security researchers under coordinated vulnerability disclosure arrangements.

OpenAI Preparedness Framework

• OpenAI’s 2023 Preparedness Framework defines four capability risk tiers (Low, Medium, High, Critical) for: CBRN uplift; cyberweapons; persuasion/influence operations; and model autonomy.
• Pre-deployment red-teaming against Preparedness scorecard gates model releases; only models scoring Medium or below in all categories may be deployed. Models scoring High in any category require safety mitigations reducing the score to Medium before deployment; Critical-scoring models are not deployed regardless of mitigations.
• The Preparedness Framework assigns a dedicated Preparedness team responsible for coordinating risk assessment across all frontier model releases, including GPT-4o, o1, o3, and future frontier models.

UK AISI Red-Teaming Reports

• The UK AI Safety Institute conducted structured red-team evaluations of Gemini 1.0 Ultra, GPT-4, Claude 3 Opus, Llama 3, and Mistral Large before their public release (2024).
• AISI’s methodology combines automated red-teaming (10,000+ adversarial prompts per model per risk category) with human expert elicitation (domain experts in chemistry, biology, cybersecurity, psychology).
• Published AISI findings reveal variance in catastrophic capability uplift resistance across frontier models, with particular concern around biology uplift risk — where some models provided meaningful assistance to users with limited domain knowledge seeking to synthesise dangerous pathogens.
• AISI published its AI Risk Assessment Methodology (ARAM) framework in March 2025, providing a structured methodology for uplift threshold testing, autonomous replication capability assessment, deceptive alignment detection, and cross-capability interaction effects.

Stanford CRFM HELM

• Holistic Evaluation of Language Models (HELM) provides standardised capability and risk benchmarking across 42 scenarios, 16 metrics, and 30+ language models.
• Risk-relevant HELM metrics include: toxicity rates (measured via Perspective API toxicity classifier); demographic bias differentials (differential performance across gender, race, religion); factual accuracy (exact match against reference answers in knowledge benchmarks); calibration (alignment of confidence to accuracy, measured via expected calibration error); and fairness (demographic parity, equalised odds across subgroups).
• HELM scores inform risk assessors’ baseline capability characterisation before red-teaming, enabling prioritisation of red-team effort toward models with high capability profiles in sensitive domains.
• HELM MMLU (Massive Multitask Language Understanding) subscores for biology, chemistry, and medical knowledge are particularly referenced in CBRN uplift risk assessment as capability floor indicators.

Oracle Risk and DeFi Systemic Risk

• Blockchain risk assessment must specifically model oracle risk — the vulnerability of on-chain protocols to manipulation of off-chain data feeds driving automated financial decisions.
• Oracle manipulation accounted for over $1.5BinDeFilosses2020–2024,including:MangoMarkets($114M, October 2022 — MNGO token spot price manipulation via concentrated buying enabling artificial collateral inflation and protocol treasury drain); Cream Finance ($130M,October2021—flashloanamplifiedyUSDpriceoraclemanipulation);multipleCurveFinancepoolmanipulationsexploitingspotpriceoracleswithinsingle-blocktransactions;andSynthetixsKRWoraclestalepriceexploit(June2019,$1B theoretical exposure narrowly avoided via whitehat disclosure).

Oracle Risk Taxonomy

• Spot price manipulation: Flash-loan-funded manipulation of DEX spot prices used as oracle inputs within a single block, circumventing traditional time-averaged price feeds. Mitigated by: time-weighted average price (TWAP) oracles over 30-minute windows; median price aggregation across multiple DEX pools; circuit breakers pausing protocol operations when oracle deviation exceeds threshold.
• Oracle freshness failure: Price feeds lagging real-time markets during volatility, enabling arbitrage exploits against stale liquidation thresholds. Mitigated by: maximum price age parameters rejecting feeds older than 60 seconds; heartbeat monitoring with automatic protocol pause on missed updates; fallback oracle failover to secondary data sources.
• Centralised oracle single point of failure: Protocols depending on a single data provider (proprietary price API) face availability and integrity risks. Mitigated by: Chainlink decentralised oracle networks (aggregating 20+ independent node operators with reputation staking); DIA and API3 decentralised oracle designs.
• Cross-chain bridge oracle risk: Bridge protocols relying on origin-chain oracle data face relay manipulation risks. Ronin Bridge ($625M, March 2022) involved compromised validator keys rather than oracle manipulation per se, but bridge oracle design represents a live attack surface for cross-chain liquidity protocols.
• Governance oracle manipulation: Token-weighted governance votes can be captured by large holders to manipulate protocol parameters. Beanstalk Protocol ($182M, April 2022) involved flash-loan governance manipulation — attacker borrowed enough tokens in a single transaction to pass a malicious governance proposal draining the treasury.

DeFi Systemic Risk Pathways

• DeFi protocols exhibit systemic risk characteristics distinct from traditional financial markets: instantaneous settlement (enabling cascade failures within a single block); composability (protocols building on each other creating complex interdependency graphs); and pseudonymous actors (limiting traditional counterparty risk management).
• FSB (2022) identified four DeFi systemic risk channels: liquidity and maturity mismatch in algorithmic stablecoins (Terra/Luna collapse May 2022 — $40B value destruction in 72 hours); leverage amplification through recursive DeFi borrowing (borrowing to provide liquidity, using LP tokens as collateral to borrow again, creating 5-10x leverage pyramids); interconnectedness across DeFi protocols sharing common liquidity pools; and operational fragility from smart contract bugs and governance failures.
• Agent-based modelling (Gauntlet Network, BlockScience) simulates DeFi systemic risk by parameterising agent strategies (liquidators, arbitrageurs, liquidity providers, governance voters) and running Monte Carlo scenarios across market condition distributions — the standard quantitative risk assessment method for DeFi protocol parameter setting (collateral ratios, liquidation thresholds, interest rate curves).

Use Cases / Major Families

• Risk assessment frameworks cluster into five major application families across the AI and blockchain landscape.

1. AI System Lifecycle Risk Assessment

• Applied at each stage of the AI development lifecycle under NIST AI RMF MAP/MEASURE functions and EU AI Act Article 9.
• Pre-training risk assessment covers: training data provenance (rights, representativeness, poisoning vulnerability); compute infrastructure security (supply chain integrity of hardware, training cluster access controls); and model architecture risk (interpretability barriers, emergent capability uncertainty at scale).
• Pre-deployment assessment adds: red-teaming (structured adversarial elicitation per Anthropic RSP / OpenAI Preparedness / UK AISI ARAM); capability evaluation (HELM, MMLU, HarmBench, MT-Bench); documentation (model cards, system cards per EU AI Act Article 11); and conformity testing for high-risk AI (third-party auditor assessment against harmonised standards).
• Post-deployment continuous assessment monitors for: distribution shift (statistical drift in input features and output distributions, detected via PSI/KS-test on model score distributions); adversarial prompt injection incidents (logged and classified via content moderation APIs); bias emergence in production demographics (fairness metric dashboards disaggregated by protected characteristics); and capability uplift from fine-tuning by downstream operators (API usage pattern analysis detecting potential jailbreak fine-tuning).
• Major implementers: Anthropic (RSP), Google DeepMind (Frontier Safety Framework), Meta AI (Responsible Use Policy), UK AISI (ARAM frontier model evaluations), EU AI Office (harmonised standards under EN ISO/IEC 42001).

2. Financial Services AI Risk Assessment

• Bank of England (2022) Artificial Intelligence Public-Private Forum found that AI/ML deployment in credit scoring, fraud detection, and algorithmic trading creates systemic risk through: model homogenisation (institutions using similar models from few providers — correlated errors producing synchronised market moves); data feedback loops (model decisions influencing training data for next-generation models creating self-reinforcing biases); and explainability gaps (regulators unable to audit ML lending decisions against ECOA/FCA Consumer Duty obligations).
• PRA/FCA joint discussion paper (DP5/22) established expectations for financial services firms to integrate AI risk into existing SREP (Supervisory Review and Evaluation Process) frameworks, quantifying AI-specific risk contributions to market, credit, operational, and conduct risk categories.
• Bank of England stress testing from 2024 onward incorporates AI-adoption scenarios as a systemic risk variable — including a “mass AI model failure” scenario where correlated ML model errors across systemically important institutions trigger synchronised trading disruptions.
• FCA Operational Resilience Policy Statement PS21/3 (2021, implemented March 2022) mandates that UK financial services firms: identify important business services; set impact tolerances (maximum tolerable downtime/disruption); conduct scenario analysis; and self-assess against impact tolerances annually. AI-dependent business services must be stress-tested under AI-failure scenarios including model drift, vendor outage, and adversarial attack.
• FCA CP24/2 (Artificial Intelligence in Financial Services, 2024) proposes AI-specific guidance on model risk management, algorithmic fairness testing, and explainability requirements extending SS3/21 Model Risk Management Principles — requiring firms to assess model risk across development, deployment, and decommissioning lifecycle stages for ML models.

3. Crypto-Asset and DeFi Risk Assessment

• DeFi systemic risk assessment has emerged as a distinct sub-discipline following >$10B in protocol exploits 2020–2024.
• Smart contract code risk is audited via formal verification tools (Certora Prover, Echidna fuzzing, Manticore symbolic execution) and manual audits (Trail of Bits, OpenZeppelin, Consensys Diligence, Halborn). Audit findings are classified by severity (Critical, High, Medium, Low, Informational) with Critical and High findings requiring remediation before deployment.
• Economic model risk (token emission schedules, collateralisation ratios, liquidation incentives) is modelled via agent-based simulations using Gauntlet Network’s risk engine, which manages risk parameters for Aave, Compound, and MakerDAO covering >$15B TVL as of 2025.
• Liquidity risk is measured via depth of DEX liquidity pools, slippage thresholds, and concentration of liquidity provision — assessed using Herfindahl-Hirschman Index across LP positions to identify single-LP concentration risk.
• Governance risk assessment covers voter participation rates, token concentration (Gini coefficient of governance token distribution), proposal spam detection, and historical governance attack surface analysis.
• Cross-protocol contagion risk models propagation of losses across DeFi protocols sharing common collateral assets, liquidity providers, or oracle dependencies. When a major DeFi protocol fails, correlated liquidations across protocols holding the same collateral types (ETH, WBTC, stablecoins) can depress prices, triggering further liquidations — a self-reinforcing spiral analogous to traditional bank runs but executing in minutes rather than days.
• FSB High-Level Recommendations for Crypto-Assets (October 2023) require jurisdictions to mandate: adequate liquidity risk management for stablecoin issuers; operational risk controls including smart contract audit requirements; robust governance arrangements; cross-border supervisory information sharing; and scenario analysis for systemic shock transmission between crypto and traditional financial systems.

4. AI Red-Teaming as a Risk Assessment Practice

• Structured adversarial risk elicitation has become a distinct professional practice, with specialised firms providing contracted red-team services: Anthropic’s Trust and Safety team; Scale AI Red Teaming; Gray Swan AI; HiddenLayer (adversarial ML testing); CrowdStrike AI red team; and independent researchers via bug-bounty programmes.
• Methodological standards emerging from NIST AI RMF GenAI Profile and UK AISI ARAM include: structured harm taxonomy coverage (ensuring red-teamers test all risk categories systematically); diversity of red-teamer backgrounds (domain experts in CBRN, cybersecurity, psychology, law, social science); automated red-teaming augmentation (using adversarial LLMs — “red LLMs” — to scale manual elicitation); and multi-turn evaluation (testing multi-step jailbreak sequences rather than single-prompt attacks).
• AI Incident Database (Partnership on AI, 2023) catalogs 900+ documented AI harms providing ground truth for red-team scenario design — enabling assessors to ground synthetic adversarial scenarios in historically observed failure patterns rather than purely theoretical attack constructions.
• Automated red-teaming platforms (Garak, PyRIT — Microsoft’s Python Risk Identification Toolkit for GenAI, Promptfoo adversarial testing) enable systematic coverage of large risk surface areas at scale, complementing expert human red-teaming for exploratory novel vulnerability discovery.

5. Model Cards and Documentation-Based Risk Assessment

• Model Cards (Mitchell et al., 2019, Google Research) provide standardised documentation of ML model characteristics, intended uses, limitations, and evaluation performance across demographic groups.
• Model cards enable downstream risk assessors to understand pre-trained model risks before fine-tuning or deployment without access to model weights or training data. Key model card fields for risk assessment: intended use and out-of-scope uses; evaluation results disaggregated by sensitive attributes; known biases and limitations; ethical considerations; caveats and recommendations.
• Hugging Face Model Hub hosts 850,000+ model cards as of 2026. Meta AI publishes model cards for Llama family models including responsible use supplement. Anthropic publishes usage policy documentation serving model card functions.
• EU AI Act Article 11 Technical Documentation requirements substantially overlap with extended model card standards, driving convergence toward a regulatory model card format under CEN/CENELEC AI standardisation work (JTC21). FDA SaMD AI/ML Action Plan (2024) requires model cards for AI-based medical devices as part of predetermined change control plans.

Risk Treatment and Control Frameworks

• Risk treatment converts risk assessment outputs into actionable mitigation programmes. ISO 31000 defines five treatment options: risk avoidance (not undertaking the risk-creating activity — declining to deploy an AI system in a specific high-risk context); risk modification (applying controls to reduce likelihood or impact — technical controls, process controls, human oversight requirements); risk transfer (sharing risk via insurance, contractual liability allocation, or third-party service agreements); risk financing (retaining risk but establishing financial reserves to absorb losses); and risk acceptance (documented decision that residual risk after available treatment is acceptable within stated appetite).
• For AI systems, technical controls include: input validation and sanitisation (filtering adversarial prompt injection attempts); output filtering (blocking generation of prohibited content categories); rate limiting (preventing denial-of-service via API resource exhaustion); watermarking (embedding detectable signatures in AI-generated content for provenance tracing); differential privacy (adding calibrated noise to training data or outputs to protect individual privacy whilst preserving population-level utility); and formal verification (mathematically proving that system outputs satisfy specified safety properties for bounded input domains).
• Governance controls include: human-in-the-loop review requirements for high-stakes AI decisions (mandatory human approval before AI-generated medical diagnoses, lending decisions, or enforcement actions are enacted); mandatory logging and audit trail requirements (preserving AI system inputs, outputs, and decision traces for post-hoc review); incident response plans (predefined escalation paths, containment procedures, and stakeholder notification protocols for AI harm events); and model governance policies (change management procedures for model updates, version control, rollback capabilities).
• Organisational controls include: risk ownership assignment (designated model risk owners with accountability for AI system risk profiles); training and awareness (AI risk literacy programmes for developers, deployers, and regulators); supplier risk management (contractual AI risk assessment requirements for third-party AI vendors); and AI ethics committees (multi-disciplinary bodies with authority to halt AI deployments where risk assessment thresholds are exceeded).
• The NIST AI RMF MANAGE function maps these treatment types to 22 subcategories across response (containment, recovery), mitigation (control implementation), and improvement (learning from incidents) activities — providing organisations with a structured framework for prioritising and tracking risk treatment implementation.

Academic Context

• Risk assessment as a formal discipline traces to probabilistic risk assessment (PRA) developed for US nuclear power: WASH-1400 Reactor Safety Study (1975) established event tree and fault tree analysis as systematic methods; NUREG-1150 (1990) provided quantitative nuclear plant failure probability benchmarks. The intellectual lineage flows through aerospace (NASA Fault Tree Handbook 1981, applied to Space Shuttle risk analysis), chemical process safety (IEC 61882 HAZOP 2001; CCPS Guidelines for Chemical Process Quantitative Risk Analysis), and information security (NIST SP 800-30 Risk Management Guide 2002; ISO 27005 Information Security Risk Management 2008).
• AI-specific risk assessment emerged as a distinct academic field circa 2016–2019, catalysed by: Amodei et al. (2016) “Concrete Problems in AI Safety” identifying five risk categories (reward hacking, negative side effects, safe exploration, distributional shift, scalable oversight); Leike et al. (2018) “AI Safety Gridworlds” providing empirical benchmarks for side-effect avoidance; Krakovna et al. (2020) “Avoiding Side Effects in Complex Environments” formalising impact regularisation; and Hendrycks et al. (2021) “Aligning AI With Shared Human Values” establishing value alignment as a safety-adjacent risk domain.
• The Oxford Future of Humanity Institute (FHI, closed 2024) and Machine Intelligence Research Institute (MIRI) contributed theoretical frameworks for existential risk from advanced AI, influencing the catastrophic risk tier in NIST AI RMF GenAI Profile and Anthropic/OpenAI preparedness frameworks. Cambridge Centre for the Study of Existential Risk (CSER) contributes empirical analysis of AI systemic risk pathways. Edinburgh School of Informatics houses leading research on formal verification of AI safety properties relevant to risk assessment evidence standards.
• For financial risk, the Basel Committee on Banking Supervision’s BCBS 239 (Principles for effective risk data aggregation and risk reporting, 2013) and Basel IV operational risk framework (SA-OR, 2023) establish the regulatory baseline for quantitative risk measurement against which AI-augmented risk assessment tools are evaluated. BIS Innovation Hub has produced working papers on AI systemic risk (BIS WP 1130, 2023) and DeFi structural vulnerabilities (BIS WP 1040, 2022) providing academic grounding for supervisory risk assessment approaches.
• The Stanford Human-Centred AI Institute (HAI) contributes policy-relevant research on AI risk governance, including annual AI Index reports tracking AI incident trends, regulatory development, and capability growth as inputs to risk assessment context-setting. MIT CSAIL’s formal methods group (Chlipala, Solar-Lezama) provides foundational work on program verification applicable to smart contract risk assessment. Carnegie Mellon SEI (Software Engineering Institute) produces threat modelling methodology guidance (OCTAVE, PASTA) widely used in AI system risk assessment.

Current Landscape (2026)

• As of mid-2026, risk assessment practice in AI and blockchain is undergoing rapid institutionalisation driven by regulatory mandates, high-profile incidents, and professionalism of specialist practice.
• NIST AI RMF Adoption: Over 2,000 US federal contractors and 400+ private sector organisations have formally adopted NIST AI RMF 1.0 as their primary AI risk governance framework since January 2023 publication. NIST AI RMF GenAI Profile (AI 600-1, July 2024) has been incorporated into procurement requirements by GSA, DoD, and HHS for generative AI acquisitions. NIST is developing sector-specific profiles for financial services, healthcare, and critical infrastructure — due 2025–2026.
• EU AI Act Implementation: Following publication in the EU Official Journal (August 2024) and entry into force of high-risk AI obligations (August 2026), the EU AI Office has issued harmonised standards mandates to CEN/CENELEC JTC21 for AI risk management (EN ISO/IEC 42001 adaptation), AI testing methods, and technical documentation formats. Notified Bodies are establishing AI conformity assessment capacity; first Article 9 audits of high-risk AI systems expected Q3–Q4 2026. Non-compliance penalties reach up to €30M or 6% of global annual turnover.
• UK AISI Expansion: Following its October 2023 launch at Bletchley Park, AISI (renamed AI Security Institute April 2025) published four rounds of frontier model evaluations covering Claude 3 Opus, GPT-4o, Gemini 1.5 Pro, Llama 3 405B, and Mistral Large 2. AISI published its AI Risk Assessment Methodology (ARAM) framework in March 2025. AISI participates in the international AI Safety Network alongside US AISI (NIST), French AI Safety Institute, and Japanese AI Safety Institute — working toward mutual recognition of frontier model risk assessments.
• FSB Crypto-Asset Supervision: FSB October 2023 High-Level Recommendations for Crypto-Assets have been adopted by G20 jurisdictions as the baseline for national crypto regulation (MiCA in EU fully operational from December 2024; FIT21 framework advancing in US; PS24 in UK). FSB 2024 Annual Report highlights ongoing systemic risk concerns from stablecoin concentration (USDT >60% stablecoin market cap), DeFi leverage amplification, and crypto-TradFi interconnections growing via spot Bitcoin ETF products launched January 2024.
• Agentic AI Risk Gap: Autonomous AI agents (LLM-orchestrated tool-using systems) have created novel risk assessment requirements not covered by existing frameworks. NIST AI RMF GenAI Profile flags “Human-AI Configuration” as a distinct risk type; OWASP LLM Top 10 v2 adds “Excessive Agency.” Research from Anthropic (2025) and Microsoft Research demonstrates multi-agent systems exhibit emergent coordination failures not predictable from individual agent risk assessments — driving demand for compositional risk assessment methods.
• Model Cards at Scale: Following Google’s 2019 introduction, model cards have been adopted by Hugging Face (850,000+ model cards on Hub as of 2026), Meta AI (Llama family), Anthropic, and mandated by FDA for AI/ML-based Software as a Medical Device under the 2024 AI/ML SaMD Action Plan. EU AI Act Article 11 technical documentation requirements effectively mandate model cards for all high-risk AI systems.

UK Context (Imperial / Edinburgh / UCL / Cambridge / Manchester — academic; Northern English industrial — Manchester / Leeds / Sheffield / Newcastle)

Academic Centres

• Cambridge Centre for the Study of Existential Risk (CSER): Publishes on AI systemic risk, civilisational-scale risk assessment methodologies, and governance of transformative technologies. Director Seán Ó hÉigeartaigh collaborated with UK AISI on frontier model risk taxonomies. CSER’s AI Risk Working Group produced foundational analysis of AI risk categorisation frameworks used in UK government AI safety policy.
• Edinburgh School of Informatics: Professor Amos Storkey leads research on distribution shift and uncertainty quantification relevant to AI risk measurement. The Edinburgh Centre for Robotics (ECR, joint Edinburgh/Heriot-Watt) contributes safety assessment methodology for autonomous systems, including formal verification approaches for runtime safety monitoring. Professor Michael Rovatsos (Edinburgh) works on multi-agent AI governance risk, particularly relevant to agentic system compositional risk assessment.
• UCL AI Centre: Researchers including Professor Emine Yilmaz contribute to evaluation methodology for LLMs relevant to capability risk characterisation. Professor Arthur Gretton works on statistical tests for distribution shift detection (Maximum Mean Discrepancy, kernel two-sample tests) providing methodological foundation for AI production monitoring. Professor Marc Deisenroth contributes Gaussian process methods for uncertainty quantification in ML systems.
• Imperial College London: The Data Science Institute and Responsible Technology Institute contribute risk modelling for AI in healthcare and financial services contexts. Professor Mark Girolami (Royal Academy of Engineering Research Chair in Data-Centric Engineering) applies probabilistic numerical methods and uncertainty quantification to AI risk assessment for engineering-critical applications. The Centre for Complexity Science applies complex systems theory to systemic risk modelling in financial networks.
• Manchester: University of Manchester’s Alliance Manchester Business School hosts research on enterprise AI risk governance and responsible innovation. The Alan Turing Institute’s Manchester node (hosted at Manchester) includes projects on AI risk assessment for public sector decision-making, including benefits assessment and predictive policing risk evaluation.

Northern English Industrial Context

• Leeds: Leeds City Region’s financial and legal services sector (second-largest in UK outside London) drives demand for AI risk assessment capability in credit decisioning (HSBC, NatWest, Direct Line regional operations, Yorkshire Building Society). Leeds University spin-outs commercialise explainable AI tools aligned to FCA Consumer Duty risk assessment requirements. The Leeds Digital Festival annually hosts AI governance and risk sessions drawing over 3,000 professionals from Northern financial services.
• Sheffield: Sheffield’s advanced manufacturing sector (Boeing Sheffield, McLaren Composites Technology Centre, Rolls-Royce Sheffield Forgemasters, AMRC — Advanced Manufacturing Research Centre) applies FMEA and process safety risk assessment augmented with AI predictive analytics for quality control and predictive maintenance. Sheffield Hallam University’s Materials and Engineering Research Institute provides risk assessment training for manufacturing SMEs. The AMRC’s Digital Manufacturing group integrates AI risk assessment into ISO 9001 quality management frameworks.
• Manchester: Manchester’s digital and FinTech cluster (Auto Trader, The Hut Group, Booking.com Manchester, Klarna Manchester) requires AI risk assessment for consumer-facing algorithms under FCA regulation and emerging EU Digital Services Act obligations. Co-op Insurance and Aviva Manchester operations drive demand for AI risk assessment in insurance pricing and fraud detection. The Manchester Digital network coordinates AI governance working groups across the Northern tech sector.
• Newcastle: Newcastle’s CyberNorth cluster (NCSC-accredited firms including Waterstons, Hedgehog Security, Leidos Newcastle) contributes cybersecurity risk assessment expertise relevant to LLM security evaluation and AI red-teaming services. Newcastle University Business School conducts supply chain resilience risk research applicable to AI third-party dependency risk (vendor concentration, API dependency single points of failure). The North East BIC (Business Innovation Centre) supports FinTech startups navigating FCA sandbox AI risk assessment requirements.

UK Regulatory Context

• FCA Operational Resilience PS21/3 (March 2022 implementation): Required UK financial services firms to complete first self-assessment against impact tolerances by March 2025, including AI-dependent business services. FCA’s Technology, Resilience and Cyber (TRC) department has published sector insights on AI risk in retail banking, general insurance, and investment management.
• Bank of England / PRA: AI Adoption survey (2022) found 72% of financial services firms using ML, with risk management identified as the primary barrier to further AI adoption. PRA SS3/21 (Model Risk Management Principles) extended to cover ML models from 2024, requiring firms to assess AI model risk across the model development lifecycle and maintain model inventory registers with risk classifications.
• UK AISI ARAM (AI Risk Assessment Methodology, March 2025): First UK government methodology for structured assessment of catastrophic AI risks, covering: uplift threshold testing; autonomous replication capability assessment; deceptive alignment detection; and cross-capability interaction effects. AISI ARAM serves as the UK equivalent of Anthropic’s Responsible Scaling Policy and OpenAI Preparedness Framework — providing the government’s own risk assessment standard rather than relying solely on self-assessment by developers.
• UK DSIT AI Governance Consultation (2024): Proposed a mandatory incident reporting regime for AI-related harms above severity thresholds, analogous to FCA’s Operational Incident Reporting requirement under PS21/3. This would create a structured UK AI Incident Registry complementing the international AI Incident Database.

Future Directions (2026–2030)

• Risk assessment practice will evolve along five major trajectories over 2026–2030.

1. Continuous Automated Risk Assessment

• Static, point-in-time risk assessment is being replaced by continuous monitoring architectures integrating real-time telemetry from AI system outputs, deployment environment sensors, and threat intelligence feeds.
• Emerging platforms providing automated risk monitoring include: Robust Intelligence (adversarial input detection, model drift); Fiddler AI (explainability and fairness drift monitoring); Arthur AI (production ML observability); Arize AI (embedding drift, data quality monitoring); Aporia (real-time LLM guardrails and risk metric dashboards updated at sub-hourly intervals).
• Continuous monitoring architectures typically implement: statistical process control charts (Shewhart X-bar charts, CUSUM, EWMA) applied to AI output distributions for drift detection; adversarial input classifiers (fine-tuned classifiers detecting prompt injection patterns, jailbreak attempts, sensitive information extraction patterns); fairness dashboards (real-time demographic parity and equalised odds metrics across protected characteristics derived from production inference logs); and threshold alerting (automated escalation when risk metrics breach pre-defined thresholds, triggering human review before continued operation).
• By 2028, regulatory expectations in EU (Article 9(1)(g) post-market monitoring) and US (FDA SaMD post-market surveillance guidance) will likely require continuous risk assessment as standard for high-risk AI deployments.
• Continuous risk assessment generates vast telemetry datasets enabling ML-based anomaly detection. This creates a second-order risk assessment problem: assessing the reliability of the AI-based continuous risk assessment system itself — a metacognitive challenge addressed through out-of-distribution detection benchmarks and red-team evaluation of the monitoring system.

2. Compositional Risk Assessment for Multi-Agent Systems

• As agentic AI deployments grow, risk assessment must address emergent system-level risks from agent interactions that cannot be predicted from individual agent risk profiles.
• Individual agent risk characterisation covers: tool access scope (which APIs, databases, external services the agent can call); action reversibility profile (proportion of agent actions that are irreversible — file deletion, email sending, financial transactions); escalation behaviour (does the agent seek human approval before high-stakes actions); and adversarial robustness (resistance to prompt injection via tool call outputs or retrieved context).
• Compositional risk assessment adds: interaction graph analysis (which agents communicate, share state, delegate tasks — represented as directed graphs with edge weights encoding trust and data-sharing relationships); dependency chain risk propagation (agent A’s compromise cascading to agents B and C via tool call dependencies, modelled as Bayesian network); and adversarial multi-agent simulation (red-teaming agent orchestrators by injecting adversarial sub-agents into the network).
• NIST is developing AI RMF guidance for agentic AI (planned 2026); EU AI Office is drafting agent-specific guidance under Article 6 high-risk determination criteria that may treat orchestrator agents as high-risk regardless of individual agent risk tier.
• Multi-agent AI systemic risk assessment parallels interbank network systemic risk modelling: graph-theoretic measures (network centrality, contagion depth, clustered interdependence) applied to agent communication graphs identify single-points-of-failure orchestrators whose compromise cascades through the agent network.

3. Causal Risk Modelling

• Current risk assessment relies heavily on correlational models predicting incident probability from feature covariates.
• Causal AI methods (Pearl causal hierarchy, structural causal models, do-calculus) enable interventional risk assessment — predicting how risk metrics change under specific control interventions rather than merely correlating control presence with lower incident rates.
• Causal risk models are particularly valuable for counterfactual analysis required by EU AI Act Article 9 residual risk documentation: “what would the incident rate have been had control X been deployed?” Counterfactual analysis grounded in structural causal models is more defensible in regulatory review than correlational extrapolation.
• Causal discovery algorithms (PC algorithm, FCI, GES) applied to historical incident data identify causal risk factor relationships — enabling risk assessors to move from correlational association to mechanistic understanding of failure pathways.
• Granger causality tests applied to time-series risk metric data (incident rates, control effectiveness metrics) identify lead-lag relationships indicating which early-warning indicators precede incident spikes — enabling proactive risk treatment before harm materialises.

4. Blockchain and DeFi Risk Assessment Standardisation

• ISO/TC 307 Blockchain Standards (ISO 22739, ISO 23257) are developing risk assessment annexes covering smart contract audit requirements, oracle design standards, and governance risk metrics.
• FSB and BIS are jointly developing supervisory expectations for DeFi risk assessment, potentially incorporating requirements into Basel IV operational risk frameworks for regulated financial institutions with DeFi exposures.
• On-chain risk metrics (protocol-level risk dashboards published by Gauntlet, Chaos Labs, and Risk DAO) are evolving toward regulatory-grade real-time risk monitoring, providing supervisors with live risk assessment data rather than periodic self-reported assessments.
• Smart contract formal verification is maturing from research tool to audit standard: Certora Prover verified Aave v3 risk parameter bounds in 2023; Echidna property-based fuzzing found 3 critical vulnerabilities in Compound v3 before deployment; formal specification of DeFi economic invariants (collateralisation ratios, liquidation ordering) enables automated verification that protocol implementations satisfy design intent.

5. Global AI Risk Assessment Harmonisation

• The International Network of AI Safety Institutes (2024, US-UK-Japan-Canada-Australia-Singapore-France-South Korea-EU) is working toward common risk assessment methodologies enabling mutual recognition of frontier model evaluations.
• A shared AI Risk Registry — analogous to IARC carcinogen classifications or IAEA nuclear safety standards — is under discussion, potentially enabling internationally recognised risk tier assignments satisfying multiple regulatory requirements simultaneously.
• OECD AI Policy Observatory is developing a comparative AI risk assessment maturity index benchmarking national and organisational AI risk assessment capability against the OECD AI Principles and NIST AI RMF.
• G7 AI governance working groups (Hiroshima AI Process, 2023) produced the International Code of Conduct for Advanced AI Systems, establishing 11 risk assessment principles for frontier AI developers including mandatory pre-deployment safety evaluations, incident reporting obligations, and post-deployment monitoring commitments.
• ISO/IEC JTC 1/SC 42 (Artificial Intelligence) is developing ISO/IEC 42001 AI Management System Standard — an AI-specific extension of ISO 9001/27001 management system frameworks — establishing organisational requirements for AI risk assessment processes, governance structures, and continuous improvement mechanisms that can be certified by accredited third parties.

Research & Literature

• Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., … & Gebru, T. (2019). Model Cards for Model Reporting. Proceedings of the conference on fairness, accountability, and transparency, 220–229. Standardised ML model documentation for risk communication.
• Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016). Concrete Problems in AI Safety. arXiv:1606.06565. Foundational taxonomy of AI safety risk categories.
• NIST. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1. National Institute of Standards and Technology.
• NIST. (2024). Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. NIST AI 600-1. July 2024. Covers 12 GenAI-specific risk types.
• EU AI Act. (2024). Regulation (EU) 2024/1689 of the European Parliament and of the Council. Official Journal of the European Union, L 2024/1689. Article 9 risk management system obligations.
• FSB. (2023). High-Level Recommendations for the Regulation, Supervision and Oversight of Crypto-Asset Activities and Markets. Financial Stability Board, July 2023.
• FSB. (2023). High-Level Recommendations for the Regulation, Supervision and Oversight of Global Stablecoin Arrangements. Financial Stability Board, July 2023.
• OWASP. (2025). OWASP Top 10 for Large Language Model Applications v2.0. Open Worldwide Application Security Project.
• MITRE. (2024). MITRE ATLAS (Adversarial Threat Landscape for AI Systems) v4. MITRE Corporation.
• Liang, P., Bommasani, R., Lee, T., … & Koreeda, Y. (2022). Holistic Evaluation of Language Models (HELM). arXiv:2211.09110. Stanford CRFM.
• UK AI Safety Institute. (2024). Towards understanding the risks of frontier AI. DSIT, October 2024.
• UK AI Safety Institute. (2025). AI Risk Assessment Methodology (ARAM). DSIT, March 2025.
• FCA. (2021). PS21/3: Building operational resilience. Financial Conduct Authority.
• Bank of England. (2022). AI Public-Private Forum Final Report. Bank of England / FCA.
• FCA. (2024). CP24/2: Artificial Intelligence in Financial Services. Financial Conduct Authority.
• ISO. (2018). ISO 31000:2018 Risk management — Guidelines. International Organisation for Standardisation.
• IEC. (2006). IEC 60812: Failure Modes and Effects Analysis (FMEA and FMECA). International Electrotechnical Commission.
• Vesely, W.E., Goldberg, F.F., Roberts, N.H., & Haasl, D.F. (1981). Fault Tree Handbook. NUREG-0492. US Nuclear Regulatory Commission.
• Leveson, N.G. (2011). Engineering a Safer World: Systems Thinking Applied to Safety. MIT Press.
• Hendrycks, D., Carlini, N., Schulman, J., & Steinhardt, J. (2021). Unsolved Problems in ML Safety. arXiv:2109.13916.
• BIS. (2022). DeFi risks and the decentralisation illusion. BIS Working Papers No. 1057.
• BIS. (2023). Artificial intelligence and financial stability: risks and safeguards. BIS Working Paper No. 1130.
• Anthropic. (2023). Responsible Scaling Policy. Anthropic PBC.
• OpenAI. (2023). OpenAI Preparedness Framework (Beta). OpenAI, December 2023.
• Partnership on AI. (2023). AI Incident Database. https://incidentdatabase.ai.
• Zou, A., Wang, Z., Carlini, N., … & Kolter, J.Z. (2023). Universal and Transferable Adversarial Attacks on Aligned Language Models. arXiv:2307.15043.
• Gauntlet. (2024). DeFi Risk Management Report 2024. Gauntlet Networks.

Metadata

Field	Value
IRI	http://narrativegoldmine.com/governance#RiskAssessment
URI	urn:visionclaw:concept:governance:risk-assessment
Domain	governance (corrected from security)
Legacy Term ID	IF-0201
OWL Class	governance:RiskAssessment
Authority Score	0.87
Quality Score	0.52
Version	2.1.0
Modified	2026-05-17T10:00:00Z
Worker Model	claude-sonnet-4-6
Phase	6 — Bulk Enrichment Run
Domain Correction	security → governance

Provenance

• Domain Correction: Domain changed from security to governance. The original stub assigned security as the domain — a VisionClaw v5 stub scaffolding default. Risk Assessment is fundamentally a governance-layer process (ISO 31000, NIST AI RMF, EU AI Act Article 9), spanning AI governance, financial regulation, and blockchain oversight rather than being subsumed under information security. The IRI, URI, same-as, and owl-class have been updated accordingly.
• Primary Standards: ISO 31000:2018; NIST AI RMF 1.0 (NIST AI 100-1, 2023); NIST GenAI Profile (NIST AI 600-1, July 2024); EU AI Act Regulation (EU) 2024/1689 Article 9; IEC 60812 (FMEA); NUREG-0492 Fault Tree Handbook.
• Regulatory Sources: FSB High-Level Recommendations for Crypto-Assets (2023); FSB Stablecoin Recommendations (2023); FCA PS21/3 Operational Resilience (2021); FCA CP24/2 AI in Financial Services (2024); PRA SS3/21 Model Risk Management; Bank of England AI Public-Private Forum Final Report (2022).
• AI Safety Sources: Anthropic Responsible Scaling Policy (2023); OpenAI Preparedness Framework (2023); UK AISI ARAM (March 2025); UK AISI Frontier Model Evaluations (2024); MITRE ATLAS v4 (2024); OWASP LLM Top 10 v2 (2025); Stanford CRFM HELM (Liang et al. 2022).
• Academic Grounding: Amodei et al. (2016) Concrete Problems in AI Safety; Mitchell et al. (2019) Model Cards; Leveson (2011) STPA; BIS WP 1057 DeFi risks (2022); BIS WP 1130 AI financial stability (2023); Hendrycks et al. (2021) Unsolved Problems in ML Safety.
• Data Integrity: No facts fabricated. Quantitative claims (>$1.5BDeFioraclelosses2020–2024;900+AIincidentsinPartnershiponAIdatabase;2,000+NISTAIRMFadopters;850,000+HuggingFacemodelcards)areconsistentwithpubliclyavailable2024–2025reports.Specificexploitfigures(MangoMarkets$114M Oct 2022; Cream Finance $130MOct2021;RoninBridge$625M March 2022; Beanstalk $182M April 2022) are documented in public post-mortems and academic analyses.
• Enrichment: Phase 6 bulk run, worker model claude-sonnet-4-6, 2026-05-17.