Medical imaging is the acquisition, reconstruction, processing, and computational analysis of visual representations of human anatomy and physiology — including X-ray, computed tomography (CT), magnetic resonance imaging (MRI), ultrasound, positron emission tomography (PET), single-photon emission computed tomography (SPECT), and digital pathology — for the purposes of clinical diagnosis, treatment planning, surgical guidance, and longitudinal disease monitoring. The field has undergone a fundamental shift with the integration of deep learning methods, particularly convolutional neural networks, vision transformers, and diffusion-based models, which now perform organ segmentation, lesion detection, and disease classification at or near radiologist-level accuracy on constrained benchmarks. Data interoperability is standardised by DICOM (Digital Imaging and Communications in Medicine) for image storage and transfer, and by HL7 FHIR for clinical report integration; AI-based clinical decision software requires regulatory clearance such as FDA 510(k) premarket notification, EU MDR/IVDR conformity assessment, or UKCA marking.
Overview
- Medical imaging sits at the intersection of physics, signal processing, Computer Vision, and clinical medicine. Each modality exploits a different physical principle — ionising X-rays for radiography and CT, nuclear magnetic resonance for MRI, acoustic pressure waves for ultrasound, and radiotracer decay for PET/SPECT — yielding complementary information about tissue structure, function, and metabolism.
- The clinical workflow involves image acquisition on scanner hardware, raw-data reconstruction (filtered back-projection or iterative algorithms for CT; Fourier inversion for MRI), archival in DICOM-formatted files within a Picture Archiving and Communication System (PACS), radiologist or pathologist reporting, and integration of structured reports into the Electronic Health Record via HL7 FHIR.
- AI has been integrated at every stage: acquisition acceleration (compressed sensing MRI), reconstruction quality enhancement (deep unfolding networks), automated worklist prioritisation, segmentation, detection, classification, and report generation via Natural Language Processing.
- The societal value is substantial: early-stage cancer detection, reduced inter-reader variability in screening programmes, and enabling radiology services in low-resource settings through AI-assisted interpretation.
Key Components
- Image Acquisition Modalities
- X-ray / Digital Radiography (DR): high throughput, ionising radiation, excellent bone/lung contrast
- Computed Tomography (CT): volumetric cross-sectional imaging; used for oncology, trauma, cardiovascular
- Magnetic Resonance Imaging (MRI): superior soft-tissue contrast; no ionising radiation; slower acquisition
- Ultrasound: portable, real-time, no radiation; operator-dependent; limited depth penetration in some tissues
- Positron Emission Tomography (PET): functional/metabolic imaging; combined with CT (PET-CT) or MRI (PET-MRI) for anatomical co-registration
- Digital Pathology: whole-slide imaging of biopsy specimens at micron resolution; foundation for computational pathology
- Fluorescence / Optical Coherence Tomography (OCT): ophthalmic imaging, surgical guidance
- Image Processing Pipeline
- Reconstruction: raw k-space (MRI) or sinogram (CT) data converted to image volumes via mathematical inversion
- Pre-processing: noise reduction, normalisation, resampling, registration to a common anatomical space
- Image Segmentation: delineating organs, tumours, or lesions (U-Net, nnU-Net, Segment Anything adaptations)
- Object Detection: localising nodules, fractures, or haemorrhages (YOLO-family, Faster R-CNN adapted to 3-D)
- Classification: assigning a diagnostic category to a patch or whole image
- Registration: aligning images acquired at different times or modalities (rigid, deformable, deep-learning-based)
- Deep Learning Architectures
- Convolutional Neural Network: U-Net and its variants dominate segmentation; ResNet/DenseNet encoders for classification
- Transformer Architecture: Swin Transformer, TransUNet — capture long-range spatial dependencies in volumetric data
- Diffusion Models: score-based generative models for data augmentation and MRI reconstruction acceleration
- Foundation Models: large pre-trained vision encoders (MedSAM, BioViL-T) fine-tuned on specific imaging tasks
- Infrastructure & Standards
- DICOM: ISO 12052 standard covering image format, network protocol (C-STORE, C-FIND), and structured reporting (DICOM SR)
- HL7 FHIR: RESTful API standard for EHR integration of imaging reports and AI outputs
- PACS / VNA: Picture Archiving and Communication System / Vendor Neutral Archive for image storage and retrieval
- Federated Learning: privacy-preserving multi-site model training across hospital silos without raw data sharing
Applications and Use Cases
- Screening Programmes
- Lung cancer screening: low-dose CT nodule detection and volumetric tracking (e.g., Lung-RADS reporting)
- Breast cancer: mammography AI as independent second reader in national programmes (ScreenPoint, Transpara)
- Diabetic retinopathy: automated grading of fundus photographs (IDx-DR: first FDA-cleared autonomous AI diagnostic)
- Cervical cancer: AI-assisted colposcopy and cytology triage
- Oncology
- Radiotherapy treatment planning: Image Segmentation of gross tumour volume (GTV) and organs-at-risk (OAR); AI reduces contouring time from hours to minutes
- Tumour response assessment: RECIST-compliant lesion measurement from serial CT/MRI; AI automates longitudinal tracking
- Radiomics: extraction of hundreds of quantitative features from imaging volumes for prognostic biomarker development
- Digital Pathology: HER2 scoring, Ki-67 proliferation index, tumour-infiltrating lymphocyte (TIL) quantification from whole-slide images
- Cardiovascular Imaging
- Cardiac MRI: automated left-ventricular ejection fraction and strain analysis
- Coronary CT angiography (CCTA): AI-derived fractional flow reserve (FFR-CT) for stenosis haemodynamic significance
- Echocardiography: real-time segmentation for wall-motion scoring
- Neuroradiology
- Acute stroke: large-vessel occlusion (LVO) detection and automated ASPECTS scoring from CT for thrombectomy triage
- Multiple sclerosis: white-matter lesion segmentation and longitudinal tracking on MRI
- Alzheimer’s disease: amyloid PET quantification; hippocampal volumetry
- Surgical Guidance
- Intraoperative imaging: C-arm fluoroscopy, cone-beam CT; AI overlay for instrument tracking
- Augmented-reality surgical navigation: pre-operative CT/MRI surface models registered to the operative field — bridges to Augmented Reality
- Surgical Robotics: vision-based tissue tracking and force-feedback augmentation
- Digital Twin Applications
- Patient-specific anatomical models derived from CT/MRI for surgical simulation — bridges to Digital Twin
- In silico clinical trials using virtual cohorts generated by imaging-conditioned generative models
Standards and Governance
- DICOM (Digital Imaging and Communications in Medicine): the universal standard (NEMA PS3 / ISO 12052) governing image encoding, file format, and network service classes (C-STORE, C-FIND, C-MOVE, WADO). DICOM SR extensions standardise structured AI output embedding in PACS workflows.
- HL7 FHIR ImagingStudy and DiagnosticReport resources: RESTful representation of imaging encounters and AI findings for Electronic Health Record integration.
- IHE Profiles: Integrating the Healthcare Enterprise (IHE) profiles (IHE Radiology, AI Results) define workflow integration for AI tools within existing RIS/PACS infrastructure.
- FDA 510(k) / De Novo: premarket notification pathway for Software as a Medical Device (SaMD); AI/ML-based Software modifications guidance (2021) addresses continuous learning systems.
- EU MDR / IVDR: European Medical Device Regulation and In Vitro Diagnostic Regulation apply to AI imaging tools; Annex I covers general safety and performance requirements.
- UKCA marking: Post-Brexit UK conformity assessment replacing CE for Great Britain market; MHRA has published separate AI and software guidance.
- ISO 13485: Quality management system standard for medical device manufacturers; required for regulatory submissions.
- ACR AI-LAB / RSNA QIBA: American College of Radiology and Radiological Society of North America quantitative imaging biomarker programmes develop performance standards for AI tools.
- MIDRC / TCIA: Medical Imaging Data Resource Center and The Cancer Imaging Archive — major open imaging repositories for AI training and benchmarking.
- Privacy: imaging data processing under HIPAA (US), UK GDPR / NHS DSP Toolkit, and EU GDPR requires de-identification (pixel-level and metadata-level) and Data Processing Agreements with AI vendors.
Challenges and Frontiers
- Distribution shift: models trained on data from one scanner vendor or acquisition protocol can fail silently on data from different sites — motivating Federated Learning and domain-adaptation techniques.
- Annotation scarcity: expert pixel-level annotation of medical images is expensive and slow; self-supervised pre-training and few-shot learning aim to reduce labelling burden.
- Explainability: Clinical Decision Support tools require interpretable outputs; gradient-based attribution maps (Grad-CAM) and concept-based explanations are active research areas, also linked to AI Governance requirements.
- Multi-modal fusion: combining imaging with genomics, proteomics, and Electronic Health Record tabular data for richer predictive models.
- Generative models: diffusion-model-based MRI reconstruction acceleration, synthetic data generation for rare pathologies, and image-to-report generation via Natural Language Processing and vision-language models.
- Regulatory readiness for continuous learning: post-market performance monitoring and predetermined change control plans for adaptive AI devices.