Data Preprocessing

Data preprocessing is the stage of a machine learning workflow that transforms raw data into a clean, consistent form suitable for modelling. It encompasses cleaning, normalisation, encoding, imputation and feature engineering to remove noise and align scales and types. The quality of preprocessing strongly determines downstream model accuracy and is a prerequisite for reliable training.

Semantic Classification

Content

Compositional Relationships (Components)

SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:hasPart ai:DataCleaning))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:hasPart ai:FeatureEngineering))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:hasPart ai:Normalisation))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:hasPart ai:MissingValueImputation))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:hasPart ai:OutlierDetection))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:hasPart ai:CategoricalEncoding))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:hasPart ai:DimensionalityReduction))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:hasPart ai:FeatureSelection))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:hasPart ai:DataAugmentation))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:hasPart ai:TrainTestSplit))

Dependency Relationships

SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:requires ai:DataQuality))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:requires ai:DataGovernance))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:dependsOn ai:DataDistribution))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:dependsOn ai:TabularData))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:dependsOn ai:DomainKnowledge))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:dependsOn ai:LabelQuality))

Capability Relationships

SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:enables ai:SupervisedLearning))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:enables ai:DeepLearning))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:enables ai:ModelAccuracy))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:enables ai:Generalisation))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:supports ai:ComputerVision))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:supports ai:NaturalLanguageProcessing))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:supports ai:TimeSeriesAnalysis))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:supports ai:RecommendationSystems))

Implementation Relationships

SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:implements ai:FeatureStore))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:implements ai:MLOps))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:uses ai:ScikitLearn))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:uses ai:AutoML))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:uses ai:PrincipalComponentAnalysis))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:uses ai:SMOTE))
SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:uses ai:Tokenisation))

Reduction Relationships

SubClassOf(ai:DataPreprocessing
  ObjectSomeValuesFrom(ai:reducesTo ai:MachineLearningPipeline))
SubClassOf(ai:DataCleaning
  ObjectSomeValuesFrom(ai:reducesTo ai:DataPreprocessing))
SubClassOf(ai:FeatureEngineering
  ObjectSomeValuesFrom(ai:reducesTo ai:DataPreprocessing))
SubClassOf(ai:NormalisationStep
  ObjectSomeValuesFrom(ai:reducesTo ai:DataPreprocessing))
SubClassOf(ai:DataAugmentation
  ObjectSomeValuesFrom(ai:reducesTo ai:DataPreprocessing))

About

• Data Preprocessing is the indispensable foundation of every applied machine learning project. Studies consistently show that data scientists spend 60–80% of project time on data preparation activities — a proportion that has remained stubbornly constant even as automation tools have improved — and that the choice of preprocessing steps often has a greater impact on model performance than hyperparameter tuning or model architecture selection. The adage “garbage in, garbage out” captures the fundamental dependency: a perfectly specified model architecture trained on improperly preprocessed data will perform poorly, while a modest model trained on clean, well-scaled data can outperform sophisticated architectures on the same raw dataset. This is not merely anecdotal; empirical studies across tabular benchmarks consistently demonstrate that preprocessing accounts for 10–30% of final model accuracy variance, exceeding the variance attributable to algorithm choice in many domains. Preprocessing is not a one-time activity but a recurring, iterative process that evolves as new data arrives, new edge cases surface in production, and understanding of the domain deepens. The shift toward data-centric AI — championed by Andrew Ng and formalised in the data-centric AI survey (Zha et al., 2023) — frames preprocessing improvement as the primary lever for model quality gains, rather than model architecture search.
• The classical view of preprocessing as a deterministic, hand-crafted pipeline of sequential steps has been substantially augmented by automated and data-driven approaches. AutoML frameworks such as Auto-sklearn 2.0, TPOT, H2O AutoML, and Google AutoML Tables now include automated preprocessing search — selecting among imputation strategies (mean, median, MICE, KNN), encoding schemes (one-hot, ordinal, target, hashing), and scaling methods (standard, robust, min-max, log) alongside model hyperparameters via Bayesian optimisation (SMAC, Optuna) or evolutionary search (TPOT’s genetic programming). DiffPrep (arXiv:2308.10915) introduced differentiable preprocessing pipeline search for tabular data, enabling gradient-based joint optimisation of preprocessing and model parameters simultaneously without the combinatorial search burden of AutoML. Feature Store platforms (Feast, Tecton, Databricks Feature Store, AWS SageMaker Feature Store, Vertex AI Feature Store) centralise preprocessing logic and serve precomputed features at both training and inference time, eliminating the pervasive problem of training-serving skew — where features computed differently during serving than during training degrade production performance. Declarative pipeline frameworks like scikit-learn’s Pipeline API, Apache Beam, and TensorFlow Extended (TFX) enforce the critical discipline that all transformers (scalers, imputers, encoders) are fitted exclusively on training data and applied to validation and test folds with parameters fixed at fit time — a requirement that, when violated, constitutes Data Leakage, one of the most common and damaging errors in applied ML practice.
• The statistical correctness of preprocessing is as important as its functional correctness. A normalisation step that uses mean and variance computed over the entire dataset — including test samples — artificially inflates test-set performance by leaking distributional information into the model’s training context, producing evaluation metrics that do not generalise to production. This form of leakage is especially insidious in time-series contexts, where future-period statistics must never be used to preprocess past-period data, requiring strictly temporal train-test splits and point-in-time-correct feature computation. In medical contexts, label leakage — where preprocessing steps inadvertently encode information about the target variable (e.g., including a laboratory test result that is only ordered when a diagnosis is already suspected) — is a documented cause of inflated performance in AI diagnostic systems that fail to replicate in prospective clinical trials.
• Preprocessing requirements vary dramatically by data modality. For structured Tabular Data, numeric scaling, categorical encoding, and missing value handling dominate. For Computer Vision, pixel normalisation (dividing by 255 or applying ImageNet mean/std: μ = [0.485, 0.456, 0.406], σ = [0.229, 0.224, 0.225] for RGB channels), random cropping and resizing, horizontal flipping, colour jitter, and MixUp constitute the standard Data Augmentation suite. For Natural Language Processing, Tokenisation via byte-pair encoding (BPE), WordPiece, or SentencePiece, subword vocabulary construction, stop-word removal (for classical NLP), padding and attention masking for variable-length sequences, and Embedding lookups form the preprocessing stack. For Time Series Analysis, temporal alignment, resampling to uniform frequency via linear interpolation or forward-fill, differencing for stationarity (ADF test-guided), seasonal decomposition (STL), and construction of lag, rolling-window, and Fourier-basis features are essential preprocessing steps. For healthcare data under the UK’s Data (Use and Access) Act 2025 and NHS Digital standards, additional preprocessing for quasi-identifier removal (k-anonymity with k ≥ 5 for NHS research), suppression of rare combinations, and formal Differential Privacy guarantees (ε-DP noise injection) is required before any model training on patient records, ensuring compliance with NHS England’s Data Security and Protection Toolkit and UK GDPR Article 9 (special category data).

Formal Analysis

• Statistical correctness of scaling: For a training set X_train ∈ ℝ^{n×d}, Z-score standardisation computes μ_j = (1/n) Σ_i X_{ij} and σ_j = std(X_train[:,j]) from training data only. At inference time, the same μ_j, σ_j are applied to transform new observations: x’_j = (x_j − μ_j) / σ_j. This ensures the test distribution is mapped to the same normalised space as training without leaking any test-set statistics. Violation — using the full-dataset mean/std — introduces a bias of O(n_test/(n_train + n_test)) in the test-set statistics, which in small-data regimes can yield 2–5% spuriously inflated accuracy.
• Optimal scaling choice by algorithm class: Decision-tree-based models (Random Forest, XGBoost, LightGBM) are invariant to monotone feature transformations because split decisions depend only on rank ordering of values within a feature. Therefore, scaling provides no benefit for tree models and is often omitted. Distance-based models (KNN, SVM with RBF kernel, K-means clustering) are scale-sensitive because Euclidean distance weights all dimensions equally; unscaled features with large numerical ranges dominate distance computations, causing effective dimensionality collapse. Neural networks trained with gradient descent exhibit sensitivity to scale through the conditioning of the loss landscape: poorly scaled inputs produce ill-conditioned Hessians, slowing convergence and requiring smaller learning rates. The 2025 empirical study (arXiv:2506.08274) quantified these effects across 14 algorithms and 16 datasets, confirming that scaling improved accuracy by 3–15% for KNN and SVM relative to unscaled baselines while producing less than 0.5% difference for Random Forest and XGBoost.
• Imputation under different missingness mechanisms: The statistical theory of missing data (Little and Rubin, 2002) distinguishes three mechanisms: (1) Missing Completely At Random (MCAR) — missingness is independent of both observed and unobserved data; mean/median imputation is unbiased. (2) Missing At Random (MAR) — missingness depends on observed but not unobserved data; model-based imputation (MICE, regression imputation) is unbiased. (3) Missing Not At Random (MNAR) — missingness depends on the missing value itself (e.g., high earners declining to report income); no purely statistical imputation is unbiased without structural assumptions. In practice, MNAR data requires domain expertise, sensitivity analysis, and often explicit modelling of the missingness mechanism alongside the outcome model.
• Dimensionality reduction via PCA: For a data matrix X ∈ ℝ^{n×d} with sample covariance Σ = (1/(n-1)) X^T X, PCA computes the eigendecomposition Σ = V Λ V^T and projects X onto the top-k eigenvectors: Z = X V_k. The retained variance fraction is Σ_{i=1}^k λ_i / Σ_{i=1}^d λ_i. PCA is optimal in the L2 sense — minimising reconstruction error among linear projections to k dimensions. However, it assumes linear structure; non-linear variants (kernel PCA, autoencoders, t-SNE, UMAP) capture manifold structure but are computationally more expensive and lose the variance-explained interpretability.
• SMOTE formal description: Given a minority-class instance x_i, SMOTE selects one of its K nearest minority-class neighbours x_{j} uniformly at random and generates a synthetic sample x_new = x_i + λ(x_j − x_i) where λ ∈ Uniform(0, 1). This linear interpolation in feature space creates a synthetic sample that lies on the line segment between two real minority instances, preserving local geometric structure while avoiding exact duplication. SMOTE does not interpolate in label space (all synthetic samples inherit the minority label), which creates issues near the decision boundary; SMOTE-ENN and SMOTE-Tomek apply post-hoc cleaning to remove synthetic samples near the boundary.

Components and Architecture

Data Cleaning

• Duplicate removal: Identification and elimination of exact or near-duplicate records that would bias gradient updates and inflate apparent training set size.
• Schema validation: Enforcement of data type contracts, range constraints, and referential integrity checks to catch upstream ingestion errors early.
• Format standardisation: Harmonisation of date formats (ISO 8601), currency representations, unit conversions, and string normalisation (Unicode NFC, lowercasing, stripping whitespace).
• Consistency checks: Cross-column logical constraints (e.g., end date must be after start date; age must match birth year).

Missing Value Imputation

• Statistical imputation: Mean/median imputation for numeric columns (appropriate for MCAR data); mode imputation for categoricals. Computationally cheap but ignores correlations.
• Model-based imputation: Multiple Imputation by Chained Equations (MICE) iteratively fits regression/classification models for each column with missing data, conditioning on all other features. Captures joint distributions and produces calibrated uncertainty estimates.
• K-nearest-neighbour imputation: Replaces missing values with the mean of K most similar records by feature distance. Preserves local structure but is O(n²) at scale.
• Indicator variables: Appending a binary flag column for each imputed variable signals missingness to the model, enabling it to distinguish between imputed and observed values.
• Domain-specific defaults: In time series, forward-fill and backward-fill exploit temporal autocorrelation; in sensor data, interpolation is physics-motivated.

Feature Scaling

• Min-Max Normalisation: Maps each feature to [0, 1] via x’ = (x − x_min) / (x_max − x_min). Sensitive to outliers; preferred for neural networks and distance-based models (KNN, SVM with RBF kernel) when features are bounded.
• Z-score Standardisation: Transforms to zero mean and unit variance via x’ = (x − μ) / σ. Suitable for algorithms assuming normally distributed inputs (logistic regression, linear SVM, PCA). The most common choice in general-purpose pipelines.
• RobustScaler: Uses median and interquartile range instead of mean and standard deviation, providing resistance to outliers. Recommended for datasets with heavy-tailed distributions or known outlier contamination.
• MaxAbsScaler: Scales each feature by its maximum absolute value to [−1, 1]. Preserves zero entries in sparse data; widely used in text feature matrices.
• Log / Power transforms: Box-Cox and Yeo-Johnson power transforms map skewed distributions toward Gaussian; logarithm of 1 + x is standard for count data and monetary values.
• A 2024 empirical study (arXiv:2506.08274) across 14 ML algorithms and 16 datasets confirmed that scaling choice significantly affects accuracy for distance- and gradient-based models, with negligible impact on tree-based ensembles (Random Forest, XGBoost, LightGBM) that are monotonic-transformation-invariant.

Categorical Encoding

• One-hot encoding: Expands a K-valued categorical into K binary indicator columns. Creates sparse high-dimensional representations for high-cardinality features.
• Ordinal encoding: Assigns integer ranks to ordered categories. Appropriate only when natural order exists; inappropriate for nominal variables.
• Target (mean) encoding: Replaces each category with the mean of the target variable within that category, computed on training data with cross-fitting to prevent leakage. Powerful for high-cardinality features; requires regularisation.
• Learned Embedding: Categorical variables are mapped to dense continuous vectors learnt jointly with the model (entity embeddings in deep tabular models; word embeddings in NLP). Reduces dimensionality and captures inter-category similarity.
• Hashing trick: Maps categories to a fixed-dimension hash space using a hash function; handles new, unseen categories at inference without retraining.

Outlier Detection and Treatment

• Statistical methods: Z-score thresholding (|z| > 3), IQR fences (Q1 − 1.5·IQR, Q3 + 1.5·IQR), and Grubbs’ test for univariate outliers.
• Model-based methods: Isolation Forest, Local Outlier Factor (LOF), and one-class SVM provide unsupervised outlier scoring in multivariate spaces.
• Treatment options: Removal (if outliers are data errors), winsorisation (clipping to boundary percentile values), log-transformation (to compress extreme values), or flagging (retaining with an indicator variable).

Dimensionality Reduction

• Principal Component Analysis (PCA): Linear projection onto orthogonal directions of maximum variance. Reduces noise and collinearity; produces interpretable variance-explained metrics. Widely used as a preprocessing step before clustering or linear models.
• Autoencoder: Non-linear dimensionality reduction via neural encoder-decoder. The bottleneck representation captures complex non-linear manifold structure. The encoder is repurposed as a feature extractor.
• t-SNE and UMAP: Non-linear methods primarily for 2D/3D visualisation of high-dimensional data distributions; not directly used as ML features due to non-determinism and transductive nature.
• Feature Selection: Filter methods (mutual information, ANOVA F-statistic, chi-squared test); wrapper methods (recursive feature elimination, sequential feature selection); embedded methods (L1 regularisation via Lasso, tree-based feature importance from Random Forest or XGBoost). The Curse of Dimensionality motivates aggressive selection for datasets with many features and limited samples.

Handling Class Imbalance

• SMOTE (Synthetic Minority Over-sampling Technique): Generates synthetic minority-class samples by linear interpolation between existing minority instances in feature space. Reduces overfitting compared to naive duplication.
• ADASYN (Adaptive Synthetic Sampling): Extends SMOTE by generating more samples near the decision boundary where classification is hardest.
• Under-sampling: Random deletion or informed selection (Tomek links, Edited Nearest Neighbours) of majority-class samples to restore balance. Risks information loss.
• Cost-sensitive learning: Assigns higher misclassification costs to minority classes via class weights in the loss function; avoids creating synthetic data.
• Ensemble resampling: BalancedBaggingClassifier and EasyEnsemble combine resampling with ensemble methods for robust handling of severe imbalance.

Data Augmentation

• For Computer Vision: random horizontal/vertical flip, random crop and resize, colour jitter (brightness, contrast, saturation, hue), Gaussian blur, random erasure, MixUp, CutMix, AutoAugment, RandAugment.
• For Natural Language Processing: back-translation, synonym substitution, random insertion/deletion/swap, EDA (Easy Data Augmentation), paraphrase generation via Large Language Models.
• For Time Series Analysis: jittering (adding Gaussian noise), scaling, time warping, window slicing, frequency domain augmentation via STFT.
• For tabular data: SMOTE and its variants (above); Gaussian noise injection; conditional Generative Adversarial Networks (CTGANs).

Pipeline Construction and Data Leakage Prevention

• All transformation parameters (scalers, imputation statistics, encoder vocabularies, PCA components) must be fitted exclusively on training data and then applied without refitting to validation and test partitions.
• Scikit-learn’s Pipeline API enforces this discipline by wrapping estimators and transformers into a single object where fit_transform is called only in the training context and transform in evaluation contexts.
• Cross-Validation wrappers (cross_val_score, GridSearchCV) apply the full pipeline within each fold to prevent leakage through preprocessing.
• Feature Store platforms extend this to production by persisting fitted transformers and serving consistent, point-in-time-correct features.

MLOps Integration and Deployment

• In production machine learning systems, preprocessing is not a one-off notebook activity but a continuously maintained software pipeline. The MLOps discipline formalises the engineering practices required to make preprocessing reliable, reproducible, and maintainable: version control of preprocessing code (Git, DVC); automated testing of preprocessing transformations (unit tests asserting that StandardScaler produces zero-mean unit-variance output; integration tests checking that the full pipeline produces correctly shaped tensors); monitoring of data drift (detecting when the distribution of incoming inference-time features diverges from the training-time distribution, indicating that preprocessing statistics may need recalibration); and automated retraining triggers that invoke preprocessing refit and model retraining when drift is detected.
• Feature Store architectures serve as the central infrastructure for production preprocessing at scale. Offline feature computation processes historical data through preprocessing transformations and materialises features into a low-latency store (Redis, DynamoDB, Bigtable) accessible at training time. Online feature computation applies the same preprocessing logic in real time to incoming raw events (user clicks, transaction records, sensor readings), ensuring that inference-time features are computed identically to training-time features. Point-in-time correctness — ensuring that only features available at the time of the prediction are used to generate training labels — is enforced by the feature store’s time-travel capability (e.g., Feast’s point-in-time join logic). Leading feature stores as of 2026 include Databricks Feature Store (integrated with Unity Catalog and MLflow), Tecton, Feast (open-source), Hopsworks, and AWS SageMaker Feature Store.
• Data Pipeline orchestration platforms (Apache Airflow, Prefect, Dagster, Metaflow) manage the scheduling, monitoring, and retry logic for preprocessing pipelines that run on new data ingestion schedules. For streaming preprocessing, Apache Kafka + Apache Flink or Apache Spark Structured Streaming enable real-time feature computation with exactly-once semantics. The Beam model (Apache Beam, underpinning Google Cloud Dataflow) provides a unified API for batch and streaming preprocessing that can run on multiple execution engines, enabling the same preprocessing code to process historical data in batch and new data in real time without duplication.

Use Cases

• Healthcare analytics: NHS electronic health records require imputation of missing lab values (e.g., using MICE conditioned on demographics and diagnosis codes), temporal alignment of irregular observations to regular grid timestamps, normalisation of vital signs (blood pressure, heart rate, oxygen saturation) across different measurement devices and care settings, anonymisation of quasi-identifiers (patient postcode, exact date of birth, rare diagnoses) under UK GDPR and the Data (Use and Access) Act 2025, careful handling of skewed distributions in clinical measurements (lab values span orders of magnitude; log-transformation is standard), and treatment of death or discharge as competing risks rather than simply missing data. The MIMIC-III and MIMIC-IV datasets (Beth Israel Deaconess Medical Center) and the UK Biobank (500,000 participants) are canonical benchmarks for clinical preprocessing pipelines. The OpenSAFELY platform, used during the COVID-19 pandemic for population-scale NHS data analysis, processes over 24 million GP records with automated preprocessing pipelines run within a secure environment where neither researchers nor the platform operators can access identified data.
• Financial services and fraud detection: High-dimensional transaction data requires log-normalisation of monetary amounts (transaction amounts are log-normally distributed; raw values span 6+ orders of magnitude), temporal feature extraction (hour of day, day of week, days since last transaction, rolling 7/30/90-day spend aggregates, velocity features counting transactions in the last hour), one-hot encoding of merchant category codes (448 MCCs in the Visa/Mastercard taxonomy), SMOTE for severe class imbalance (fraud rates of 0.01–0.1% in card-not-present transactions), and winsorisation of extreme transaction values at the 99.9th percentile to prevent a few large legitimate corporate transactions from distorting scale-sensitive features. The FCA’s October 2024 AI LAB initiative supports regulated experimentation with ML preprocessing pipelines in financial contexts, including testing whether preprocessing choices introduce disparate impact across demographic groups.
• Natural Language Processing and Large Language Models: Text preprocessing chains for classical NLP include lowercasing, punctuation removal, stemming or lemmatisation, stop-word removal, and TF-IDF or count vectorisation. For modern transformer-based NLP, the preprocessing stack is: raw text → Unicode normalisation (NFC) → subword tokenisation (BPE/WordPiece/SentencePiece) → integer token IDs → padding to fixed length → attention masks → position IDs. For fine-tuning large pre-trained models, this preprocessing is largely handled by the HuggingFace Tokenizer library, which wraps model-specific tokenisers (LlamaTokenizer, GPT2Tokenizer, BertTokenizerFast) and handles batching, padding, and truncation automatically. Preprocessing for multi-lingual models requires language-specific Unicode normalisation, script detection, and handling of right-to-left languages (Arabic, Hebrew) with appropriate padding direction.
• Computer Vision and medical imaging: For natural images, the canonical preprocessing pipeline is: raw JPEG → decode to uint8 RGB tensor → random resize + crop to 224×224 (training) or centre crop (evaluation) → random horizontal flip (training) → convert to float32 in [0, 1] → normalise by ImageNet channel statistics (subtract mean [0.485, 0.456, 0.406], divide by std [0.229, 0.224, 0.225]). For medical imaging, DICOM files require windowing (mapping the full 12-bit Hounsfield unit range to a clinically relevant display window), resampling to isotropic voxel spacing (CT scans may have anisotropic voxel sizes: 1×1 mm in-plane, 3 mm slice thickness), intensity normalisation relative to tissue-specific reference ranges, and rigid or deformable registration to align scans from different time points. Histopathology whole-slide images (WSIs; 100,000×100,000 pixels at 40× magnification) require tiling into patches, background exclusion, and colour normalisation (Macenko or Reinhard stain normalisation) to mitigate scanner and staining variability across hospitals.
• Sensor and IoT data: Industrial sensor streams from manufacturing equipment require resampling to uniform frequency (sensor polling intervals vary across device generations), Kalman filter or Savitzky-Golay smoothing for noise reduction, lag feature construction (k = 1, 5, 60 seconds of historical readings), rolling statistics (mean, variance, peak-to-peak amplitude over windows), fast Fourier transform (FFT) features for vibration data (bearing fault detection relies on frequency-domain features at specific harmonic multiples of shaft rotation frequency), and anomaly masking (excluding periods of planned maintenance or known equipment failure from healthy-class training data). Environmental monitoring data (temperature, pressure, air quality index) demands spatial interpolation (kriging, inverse distance weighting) to fill spatial gaps in sensor networks, and seasonal decomposition (STL: Seasonal and Trend decomposition using Loess) to separate trend, seasonal, and residual components before modelling.
• Recommendation systems: User interaction logs require deduplication (removing duplicate click events from UI refresh bugs), session segmentation (grouping events separated by more than 30 minutes of inactivity into distinct sessions), frequency-based feature encoding (logarithm of item popularity; bucketised user activity quantiles), temporal decay weighting (recent interactions are more predictive of current preferences; exponential decay with half-life of 7 days is common), and negative sampling — the critical preprocessing step for implicit feedback datasets where only positive interactions are observed. For item-based collaborative filtering, preprocessing typically includes computing the user-item interaction matrix, normalising by user activity level (root-mean-square normalisation), and applying item frequency downsampling (analogous to word negative sampling in Word2Vec) to prevent popular items from dominating training. Feature hashing maps sparse categorical user and item features (URLs, product IDs, query terms) to fixed-dimensional feature vectors without maintaining an explicit vocabulary.

Academic Context

• The theoretical understanding of data preprocessing is grounded in statistical learning theory and decades of empirical research in pattern recognition and data mining. Vapnik’s structural risk minimisation framework (1995) motivates the need for controlled input representations: the effective hypothesis space of a learning algorithm interacts with the input representation, and preprocessing that reduces redundancy, noise, and scale variability can substantially shrink the effective VC dimension and tighten generalisation bounds. The universal approximation theorem establishes that sufficiently expressive models can learn arbitrary continuous functions — but the theorem assumes bounded, normalised inputs; in practice, improperly scaled inputs with high dynamic range lead to ill-conditioned gradient dynamics that prevent efficient training even for expressively sufficient architectures.
• Ben-David et al. (2010) formalised domain adaptation from a theoretical perspective, highlighting that preprocessing choices that shift input distributions directly affect the divergence between source and target domains, which upper-bounds transferability. In Transfer Learning settings, preprocessing that normalises inputs to match the distribution assumed by a pre-trained model’s input layer (e.g., ImageNet statistics for image models; subword vocabulary alignment for language models) is a critical step for effective fine-tuning. Chapelle et al. (2006) established the role of label and feature preprocessing in semi-supervised learning regimes, showing that pre-processing to align the geometry of the unlabelled and labelled manifolds significantly improves semi-supervised accuracy.
• The AutoML research community has formalised preprocessing as part of the combined algorithm selection and hyperparameter optimisation (CASH) problem, in which the joint space of preprocessing choices and model hyperparameters is searched simultaneously. The Freiburg AutoML Lab (Frank Hutter, Matthias Feurer) developed Auto-sklearn (2015), which uses SMAC (Sequential Model-based Algorithm Configuration) Bayesian optimisation over 15 preprocessors and 15 classifiers, winning three of four AutoML challenge tracks. Auto-sklearn 2.0 (2022) replaced the SMAC search with a portfolio-based meta-learning approach that selects preprocessing and model configurations based on dataset meta-features, achieving competitive performance without the wall-clock overhead of full Bayesian search. TPOT (Olson et al., 2016) used genetic programming to evolve sklearn Pipeline graphs including preprocessing steps, demonstrating that flexible pipeline topology search can discover non-obvious preprocessing strategies.
• The data-centric AI movement, championed by Andrew Ng and formalised in the landmark survey by Zha et al. (2023), argues that for mature model architectures, improvements in data quality through preprocessing and curation yield larger performance gains than model architecture improvements. This perspective has motivated dedicated research tracks at NeurIPS (DataPerf benchmark), ICML (DataComp), and ICLR on data curation, cleaning, and augmentation as primary research objects rather than implementation details. Key contributors include Chris Ré’s Snorkel team (weak supervision and programmatic labelling as preprocessing), Hazy Research (data augmentation as a systematic discipline), and the MLCommons DataPerf working group.
• Foundational empirical contributions to preprocessing research include: LeCun et al. (1998, “Efficient BackProp”) providing the first systematic guidelines for preprocessing in neural network training — centring inputs, whitening (decorrelating features), and using tanh activations — based on analysis of the Hessian conditioning; Ioffe and Szegedy (2015) on Batch Normalisation as an in-model normalisation technique that reduces sensitivity to input preprocessing by normalising activations at each layer; Snoek et al. (2012) on Bayesian hyperparameter optimisation extendable to preprocessing; Chawla et al. (2002) on SMOTE; Van Buuren and Groothuis-Oudshoorn (2011) on MICE multiple imputation; and the comprehensive 2025 MDPI review (Aborokbah et al.) providing the most systematic empirical evaluation to date across 14 algorithms, 16 datasets, and 12 preprocessing configurations for classification and regression tasks.

Components and Architecture (Continued)

Pipeline Construction and Correctness

• The scikit-learn Pipeline class chains preprocessing transformers and a final estimator, enforcing the fit/transform discipline. The fit method on the pipeline fits each transformer in sequence using training data only, then fits the estimator. The transform method applies fitted transformers without re-fitting. When combined with GridSearchCV or cross_val_score, the pipeline’s fit method is called inside each cross-validation fold, guaranteeing that preprocessing statistics are computed only on the training fold — the gold standard for unbiased evaluation.
• ColumnTransformer enables different preprocessing branches for different feature subsets: numeric columns → [StandardScaler], categorical high-cardinality → [TargetEncoder], categorical low-cardinality → [OneHotEncoder], text columns → [TfidfVectorizer]. The outputs are concatenated before the model. This modular architecture cleanly separates preprocessing concerns and enables independent tuning of each branch.
• Pipeline serialisation (joblib.dump, pickle) persists the entire fitted preprocessing + model stack, enabling deployment of a single artifact that transforms raw input at inference time identically to training. This eliminates the most common source of training-serving skew: hand-coded inference preprocessing that diverges from training preprocessing over time.

Temporal Preprocessing for Time Series

• Time-aware train-test splits: random shuffling is prohibited for time series data. Train and test sets must be temporally ordered, with the test set consisting exclusively of time periods later than the training set. Walk-forward validation (expanding or sliding window) provides multiple evaluation periods while respecting temporal order.
• Stationarity transformation: economic and many physical time series exhibit trends and seasonality (non-stationary behaviour). First-order differencing (Δy_t = y_t − y_{t-1}) removes linear trends; seasonal differencing (Δ_s y_t = y_t − y_{t-s}) removes seasonal patterns. The Augmented Dickey-Fuller (ADF) test quantifies the evidence for non-stationarity, guiding the number of differencing operations. Excessive differencing can over-remove signal; KPSS test complements ADF for robust stationarity diagnosis.
• Feature engineering for time series: lag features (y_{t-1}, y_{t-2}, …, y_{t-k}) encode autoregressive dependencies; rolling statistics (mean, std, skewness over windows of size w) capture local trends; Fourier basis functions (sin/cos at known periods) encode seasonality; calendar features (hour, day-of-week, month, is-holiday) capture temporal patterns relevant in business domains.

Text Preprocessing and Tokenisation

• Classical NLP preprocessing: lowercasing, punctuation removal, stemming (Porter, Lancaster) or lemmatisation (WordNet lemmatiser, spaCy), stop-word removal, and TF-IDF vectorisation were the standard pipeline for bag-of-words models (SVM, Naïve Bayes, logistic regression). For modern transformer-based models, these steps are largely replaced by learnt subword Tokenisation.
• Subword tokenisation: BPE (byte-pair encoding, Sennrich et al., 2016) iteratively merges the most frequent character pair in the vocabulary, producing a vocabulary of typically 32k–128k tokens that balances between character-level coverage and word-level semantic coherence. WordPiece (Schuster and Nakamura, 2012, used in BERT) and SentencePiece (Kudo and Richardson, 2018, used in T5, LLaMA) are related approaches. Subword tokenisation ensures that the Embedding vocabulary is fixed and that out-of-vocabulary words are decomposed into known subword units rather than replaced by a single unknown token.
• Special tokens and padding: transformer models require fixed-length input sequences. Padding (adding a [PAD] token to short sequences) and attention masking (setting attention weights to zero for padding positions) are essential preprocessing steps. Truncation (cutting long sequences to the maximum context length) must be handled carefully to avoid removing the most informative parts of the input.

Data Quality and Governance

• Data quality is the meta-property that determines whether preprocessing can recover useful signal from raw data. It encompasses four dimensions formalised by the ISO 25012 Data Quality Model: (1) Accuracy — the degree to which data correctly represents the real-world entity it describes; (2) Completeness — the presence of all required values; (3) Consistency — absence of contradictions within and across datasets; (4) Timeliness — data reflecting the state of the world at the appropriate time. Poor data quality in any dimension propagates through preprocessing into model behaviour: inaccurate labels produce biased classifiers; incomplete records inflate imputation error; inconsistent encoding creates spurious feature correlations; stale data produces models that reflect outdated distributions.
• Data governance frameworks establish policies, processes, and responsibilities for ensuring data quality before and during preprocessing. In the UK, the Information Commissioner’s Office (ICO) enforces UK GDPR requirements for accuracy (Article 5(1)(d)) and purpose limitation (Article 5(1)(b)) that directly constrain what preprocessing operations are permitted on personal data. NHS Digital’s Data Security and Protection Toolkit mandates specific data de-identification and pseudonymisation standards for clinical data used in ML training. The Financial Conduct Authority’s (FCA) AI guidelines for financial services specify that training data must be representative, unbiased, and traceable to authorised sources.
• Data versioning and lineage tracking — enabled by tools such as DVC (Data Version Control), Delta Lake, Apache Iceberg, and LakeFS — record exactly which version of each dataset was used to train each model, which preprocessing transformers were applied and with which parameters, and how the preprocessing pipeline evolved over time. This lineage is essential for regulatory compliance (reproducing a model’s training conditions for audit), debugging production degradations (identifying whether model drift stems from data distribution shift or preprocessing changes), and collaborative research (ensuring that benchmark results are reproducible across teams).
• Bias Mitigation in preprocessing addresses the risk that historical training data encodes discriminatory patterns that the model will learn and amplify. Pre-processing bias mitigation techniques include reweighting (assigning higher loss weights to underrepresented demographic groups), resampling (oversampling minority groups or undersampling majority groups), disparate impact removal (transforming features to achieve statistical parity across protected groups while preserving predictive utility), and learning fair representations (encoding inputs into a latent space that is maximally informative of the target while being minimally informative of protected attributes). Under the UK Equality Act 2010 and emerging AI governance frameworks, preprocessing for fairness is increasingly a legal and ethical obligation for high-stakes AI applications in hiring, lending, and healthcare.

Current Landscape (2026)

• In 2026, the boundary between preprocessing and model training has blurred substantially. Large pre-trained foundation models (Transfer Learning) absorb much of the traditional preprocessing burden: image normalisation is handled internally by model-specific processing functions; Tokenisation and Embedding are the primary NLP preprocessing steps, with tokeniser vocabularies learnt jointly with model pre-training; tabular foundation models (TabPFN 2.0, released 2025, from the AutoML group at Freiburg) perform in-context preprocessing implicitly by observing the training examples in the prompt rather than explicitly fitting preprocessing transformers. Nevertheless, preprocessing remains critical for data curation, de-biasing, and regulatory compliance, and the data-centric AI movement has elevated its status in the ML research agenda. The shift toward MLOps and Data Pipeline engineering has elevated preprocessing from an ad-hoc exploratory notebook script into a first-class software artifact that is version-controlled (DVC, LakeFS, Delta Lake), unit-tested (pytest-based preprocessing test suites), and monitored for data drift (Evidently AI, Arize, WhyLabs, NannyML).
• Feature Store adoption has accelerated markedly since 2024. Databricks Feature Store (integrated with Unity Catalog and MLflow tracking), Tecton (the leading commercial offering, serving features at <1ms P99 latency at Netflix, Stripe, and Lyft scale), Feast (open-source, deployed at Twitter/X, Shopify, and GoJek), Hopsworks (focusing on Python-native ML feature engineering), and AWS SageMaker Feature Store now serve features at sub-millisecond latency for production recommendation, fraud detection, and search-ranking systems. The adoption of feature stores has dramatically reduced the incidence of training-serving skew bugs that were previously a leading cause of silent model degradation in production.
• DiffPrep (arXiv:2308.10915) introduced differentiable preprocessing pipeline search for tabular data, enabling gradient-based joint optimisation of preprocessing and model parameters without the combinatorial search overhead of AutoML — a significant advance toward automatic preprocessing. AutoML platforms (Auto-sklearn 2.0, H2O AutoML, Google AutoML Tables, Azure AutoML) now include preprocessing as a first-class search dimension alongside model selection. Data Augmentation for Large Language Models has become a major research subfield: GPT-4o, Claude 3.5, and Gemini 1.5 Pro are used to generate synthetic training data for instruction tuning; diffusion-model-generated synthetic images (Stable Diffusion, Midjourney) augment computer vision training corpora; TabSyn and CTGAN generate synthetic tabular training data for privacy-constrained domains. Privacy-preserving preprocessing — Differential Privacy via scikit-learn-dp (wrapping DP-SGD imputation and DP statistics) and TensorFlow Privacy, and Federated Learning preprocessing via PySyft and FLWR — is gaining regulatory traction under GDPR, the UK Data (Use and Access) Act 2025, and the EU AI Act Annex III requirements for special-category data in high-risk AI applications.
• The data-centric AI paradigm, crystallised at the 2021 Data-Centric AI workshop (NeurIPS 2021, organisers: Andrew Ng, Lora Aroyo, Cody Coleman) and formalised in the DataPerf benchmark suite (ICML 2023), frames preprocessing quality improvement as the primary research question for mature model architectures. DataComp (Gadre et al., 2023) demonstrated that better data curation and filtering (a form of preprocessing) improved CLIP visual representation quality more than architecture improvements, establishing preprocessing as a first-class research contribution at top venues. The DCLM (DataComp for Language Models) project (2024) similarly showed that careful web-scraped text preprocessing — quality filtering, deduplication, toxicity removal — yields DCLM-Baseline-7B that outperforms Llama 2 7B on standard benchmarks despite using the same architecture and training compute.

UK Context

• The UK has a distinctive data preprocessing landscape shaped by strong public sector data assets, stringent data protection regulation, and a vibrant academic ML community. The NHS produces one of the world’s richest longitudinal health datasets, but access for ML requires preprocessing pipelines that satisfy the UK GDPR (as retained in UK law post-Brexit and extended by the Data (Use and Access) Act 2025), NHS Digital data governance frameworks, and the newly established NHS Federated Data Platform (FDP) standards. Preprocessing for clinical AI in UK research typically occurs within Trusted Research Environments (TREs) such as OpenSAFELY, SAIL Databank (Swansea), and the NHS England Secure Data Environment, where data never leaves the environment and preprocessing scripts are validated by governance teams.
• Academic contributions to preprocessing research come from: the Alan Turing Institute (ATI) in London, which hosts work on automated data quality and fairness-aware preprocessing; University College London’s Centre for Medical Image Computing (CMIC), which leads on medical image preprocessing standards; the University of Edinburgh’s Informatics department, which contributes to temporal and text preprocessing for clinical NLP; King’s College London’s Department of Informatics and the NIHR Maudsley BioMedical Research Centre, which work on mental health data preprocessing; and the University of Manchester’s COSMIC group, which focuses on preprocessing for biological and genomic data. The Northern England industrial context includes BT Group’s AI labs in Bradford preprocessing telecommunications network data, Siemens Healthineers (Manchester) developing CT/MRI preprocessing pipelines, AMRC Castings Group applying ML preprocessing to manufacturing sensor data in Sheffield, and the National Innovation Centre for Data (NICD) in Newcastle supporting SMEs in building production preprocessing pipelines.

Future Directions (2026–2030)

• Foundation model–native preprocessing: Pre-trained foundation models will progressively internalise task-specific preprocessing, incorporating learned tokenisers (adaptive BPE vocabularies that expand as new terminology emerges), task-conditioned normalisation (attention-based input scaling that adapts to task context), and multi-modal alignment layers that map heterogeneous input types into a unified representation space. The practical effect will be that preprocessing for standard modalities (images, text, tabular structured data) will become largely implicit, shifting the preprocessing burden toward data curation and quality assurance rather than transformation engineering.
• Automated bias detection and mitigation as standard practice: Preprocessing pipelines will incorporate automated fairness auditing as a mandatory step before model training in regulated sectors. Algorithmic auditing frameworks (Aequitas, Fairlearn, IBM AI Fairness 360) will be integrated into preprocessing pipeline libraries, automatically detecting demographic skew, protected attribute proxies (postcode as a proxy for ethnicity; name as a proxy for gender), and representation imbalances in training data. Built-in mitigation steps (reweighting, resampling, adversarial debiasing, disparate impact remover) will be applied when auditing flags violations, guided by UK Equality Act 2010 protected characteristics and the EU AI Act’s prohibited practices list.
• Privacy-preserving preprocessing at scale: Differentially private preprocessing — DP-StandardScaler (using the Laplace or Gaussian mechanism to compute mean and standard deviation estimates with (ε, δ)-DP guarantees), DP-PCA (via the exponential mechanism or random projections), and locally private categorical encoding (using the Randomised Response protocol or RAPPOR for local DP) — will become first-class features in major preprocessing libraries (scikit-learn-dp, TensorFlow Privacy, OpenDP). This will enable organisations to preprocess sensitive data (patient records, financial transactions, biometric data) with formal privacy guarantees satisfying GDPR Article 25 (data protection by design and by default).
• Streaming and online preprocessing with drift detection: Real-time preprocessing for high-throughput streaming data (Apache Kafka + Apache Flink, Kinesis + Lambda, Pub/Sub + Dataflow) will use streaming estimators (online mean/variance estimation via Welford’s algorithm; sliding-window quantile estimation via GK summary; streaming k-means for cluster-based encoding) that update incrementally without reprocessing historical data. Integrated drift detection (ADWIN, Page-Hinkley test, Kolmogorov-Smirnov test on feature marginals) will trigger automatic preprocessing refit and model retraining when statistically significant distribution shift is detected, enabling robust preprocessing under evolving data distributions.
• Synthetic data as primary preprocessing augmentation: High-fidelity tabular and multimodal synthetic data generators — diffusion models (TabDDPM, SynthCity), flow-based models (RealNVP, Glow), and conditional Generative Adversarial Networks (CTGAN, TVAE) for tabular data; diffusion models (Stable Diffusion 3, DALL-E 4) for images; large language models for text — will serve as a preprocessing augmentation layer that dramatically increases effective dataset size, particularly for rare events (less than 0.01% occurrence), highly imbalanced classes, and privacy-constrained domains where real data cannot be shared. The challenge of evaluating synthetic data quality — ensuring that synthetic samples preserve the joint distribution of real data (fidelity) while not simply memorising training examples (privacy) and providing useful augmentation signal (utility) — is an active research problem addressed by the SynthEval, SDMetrics, and Synthetic Data Vault libraries.
• Multi-modal preprocessing unification: Cross-modal preprocessing pipelines aligning image patches, text tokens, audio spectrograms, sensor time series, and graph structures into unified embedding spaces will support Multimodal Learning architectures. Unified preprocessors will handle modality-specific transformations (image normalisation, audio mel-spectrogram extraction, text tokenisation, graph adjacency normalisation) and produce embeddings in a shared latent space via modality-specific projection heads, enabling joint training on heterogeneous datasets without custom modality-switching logic.
• Preprocessing-aware fairness constraints as first-class ML objective: Formalising preprocessing choices as part of constrained optimisation frameworks, where the pipeline is jointly optimised for predictive performance subject to formal fairness constraints (demographic parity, equalised odds, individual fairness via Lipschitz constraints). Research groups at the Alan Turing Institute, MIT, and CMU are developing preprocessing algorithms that solve this joint optimisation problem, producing preprocessing pipelines that are provably fair with respect to specified protected attributes while minimising predictive performance trade-offs under the Pareto frontier of the fairness-utility curve.

Key Terminology

• Data Leakage: The inadvertent use of test-set or future-time information during training, most commonly caused by fitting preprocessing transformers on the full dataset before the train-test split. Leads to optimistic evaluation metrics that do not generalise to production. In time-series contexts, temporal leakage occurs when future data values are used in a feature (e.g., rolling average centred on the target time point rather than trailing). Label leakage occurs when a preprocessing feature encodes the target variable (e.g., including a diagnostic code that is only assigned after the outcome occurs).
• Training-Serving Skew: Discrepancy between preprocessing applied at training time and at inference time in production, causing degraded model performance. Feature Store architectures eliminate this by centralising preprocessing logic and serving the same computed features at both train and inference time. Common sources: different code paths for batch training vs. online inference; different handling of null values; different ordering of pipeline steps.
• Missing Value Imputation: The process of estimating and filling in missing data values using statistical (mean/median/mode), model-based (MICE, regression), or distance-based (KNN) techniques. Choice of imputation strategy depends on the missingness mechanism (MCAR, MAR, MNAR) and the downstream model’s sensitivity to imputation error.
• Winsorisation: Clipping extreme values in a distribution to a specified percentile boundary (e.g., 1st and 99th percentile) to limit the influence of outliers without removing observations. Preferred over outlier removal when observations may be genuine extreme values rather than data errors (e.g., very high transaction amounts in financial data may be legitimate corporate purchases).
• Train-Test Split / Cross-Validation: Partition of available data into training, validation, and test sets. Preprocessing transformers must be fitted only on the training partition and applied without refitting to validation and test data. Cross-Validation (k-fold, stratified, time-series split) provides multiple train-validation partitions for more reliable performance estimation in small datasets.
• Curse of Dimensionality: The phenomenon whereby high-dimensional feature spaces become increasingly sparse as dimensionality grows, requiring exponentially more training samples for reliable density estimation and model fitting. Motivates Feature Selection, Dimensionality Reduction, and regularisation. Described by Richard Bellman in 1957 in the context of dynamic programming; relevant to distance-based ML algorithms (KNN, SVM) and clustering.
• SMOTE (Synthetic Minority Over-sampling Technique): Creates interpolated synthetic minority-class observations by linear combination of existing minority instances and their K-nearest neighbours, addressing Class Imbalance in training datasets. Variants include ADASYN (density-adaptive), SMOTE-ENN (with boundary cleaning), and SMOTENC (for nominal and continuous features).
• One-hot encoding: Binarisation of a K-class categorical variable into a K-dimensional indicator vector, with exactly one entry set to 1. Creates sparse representations for high-cardinality categories; for vocabulary sizes > 1000, Embedding or hashing is preferred. Also called dummy coding in statistics.
• RobustScaler: A scikit-learn scaler that uses the median and interquartile range (Q3 − Q1) instead of mean and standard deviation, providing resistance to outliers. Recommended for datasets with outlier contamination exceeding 5–10% of observations, where StandardScaler’s statistics would be distorted.
• Feature Store: A data engineering system that computes, stores, versions, and serves preprocessing feature transformations for consistent reuse across model training and production serving. Enforces point-in-time correctness, eliminating both Data Leakage and training-serving skew. Examples: Tecton, Feast, Databricks Feature Store, AWS SageMaker Feature Store.
• Data Distribution: The statistical distribution of values within a feature. Preprocessing choices (scaling method, imputation strategy, encoding approach) should be guided by the empirical distribution: Gaussian → StandardScaler; heavy-tailed → RobustScaler or log transform; bounded → MinMaxScaler; multimodal → cluster-based encoding or non-parametric methods.
• Data Pipeline: An orchestrated sequence of data ingestion, validation, preprocessing, feature computation, and storage steps that prepares data for model training and serving. Implemented with Apache Airflow, Prefect, Dagster, Apache Beam, or TFX. Pipeline as code ensures reproducibility and enables version control of preprocessing logic.
• Label Quality: The accuracy and consistency of ground-truth labels in training data. Noisy labels (incorrect annotations) act as a form of data poisoning that preprocessing cannot directly address; label quality improvement techniques include confident learning (Northcutt et al., 2021), cleanlab, and multi-annotator agreement via inter-rater reliability metrics (Cohen’s kappa, Fleiss’ kappa).

Research and Literature

• 1. Duda, R.O., Hart, P.E., & Stork, D.G. (2001). Pattern Classification (2nd ed.). Wiley. [Foundational text on feature preprocessing and representation]
• 2. Bishop, C.M. (2006). Pattern Recognition and Machine Learning. Springer. [Comprehensive treatment of preprocessing in probabilistic ML]
• 3. Pedregosa, F., Varoquaux, G., Gramfort, A., et al. (2011). Scikit-learn: machine learning in Python. JMLR, 12, 2825–2830. [The canonical preprocessing API]
• 4. Ioffe, S. & Szegedy, C. (2015). Batch normalization: accelerating deep network training by reducing internal covariate shift. ICML 2015. [In-model normalisation mitigating dependence on input preprocessing]
• 5. Feurer, M., Klein, A., Eggensperger, K., et al. (2015). Efficient and robust automated machine learning. NeurIPS 2015. [Auto-sklearn: preprocessing as part of CASH problem]
• 6. Feurer, M., Eggensperger, K., Falkner, S., et al. (2022). Auto-sklearn 2.0: hands-free AutoML via meta-learning. JMLR, 23, 1–61. [Auto-sklearn 2.0 with automated preprocessing]
• 7. Chawla, N.V., Bowyer, K.W., Hall, L.O., & Kegelmeyer, W.P. (2002). SMOTE: synthetic minority over-sampling technique. JAIR, 16, 321–357. [SMOTE original paper]
• 8. Van Buuren, S. & Groothuis-Oudshoorn, K. (2011). mice: multivariate imputation by chained equations in R. Journal of Statistical Software, 45(3). [MICE imputation]
• 9. Jolliffe, I.T. (2002). Principal Component Analysis (2nd ed.). Springer. [PCA theory and applications]
• 10. LeCun, Y., Bottou, L., Orr, G.B., & Müller, K.-R. (1998). Efficient backprop. In Neural Networks: Tricks of the Trade. Springer. [Early preprocessing guidelines for neural networks]
• 11. García, S., Luengo, J., & Herrera, F. (2016). Tutorial on practical tips of the most influential data preprocessing algorithms in data mining. Knowledge-Based Systems, 98, 1–29. [Comprehensive preprocessing tutorial]
• 12. Zheng, A. & Casari, A. (2018). Feature Engineering for Machine Learning. O’Reilly Media. [Practitioner reference for preprocessing and feature engineering]
• 13. Molnar, C. (2022). Interpretable Machine Learning (2nd ed.). Online. [Preprocessing and its effect on model interpretability]
• 14. Ben-David, S., Blitzer, J., Crammer, K., et al. (2010). A theory of learning from different domains. Machine Learning, 79(1–2), 151–175. [Domain adaptation: preprocessing distribution shift]
• 15. He, H. & Garcia, E.A. (2009). Learning from imbalanced data. IEEE TKDE, 21(9), 1263–1284. [Survey of class imbalance handling]
• 16. Chandrashekar, G. & Sahin, F. (2014). A survey on feature selection methods. Computers and Electrical Engineering, 40(1), 16–28. [Feature selection survey]
• 17. Shorten, C. & Khoshgoftaar, T.M. (2019). A survey on image data augmentation for deep learning. Journal of Big Data, 6(1). [Data augmentation for CV]
• 18. Wei, J. & Zou, K. (2019). EDA: easy data augmentation techniques for boosting performance on text classification tasks. EMNLP-IJCNLP 2019. [NLP data augmentation]
• 19. Aborokbah, M.M., et al. (2025). Data preprocessing and feature engineering for data mining: techniques, tools, and best practices. AI, 6(10), 257. MDPI. [Comprehensive 2025 survey]
• 20. Zha, D., Bhat, Z.P., Lai, K.H., et al. (2023). Data-centric artificial intelligence: a survey. arXiv:2303.10158. [Data-centric AI: preprocessing as primary quality lever]
• 21. Falagas, M.E., Pitsouni, E.I., Malietzis, G.A., & Pappas, G. (2008). Comparison of PubMed, Scopus, Web of Science, and Google Scholar. FASEB Journal. [Bibliometric context for preprocessing research]
• 22. Xu, R., Wunsch, D. (2005). Survey of clustering algorithms. IEEE TNN, 16(3), 645–678. [Clustering as a preprocessing/exploratory step]
• 23. Salvador, S. & Chan, P. (2007). Toward accurate dynamic time warping in linear time and space. Intelligent Data Analysis, 11(5). [DTW for time series preprocessing]
• 24. Lemaître, G., Nogueira, F., & Aridas, C.K. (2017). Imbalanced-learn: a Python toolbox to tackle the curse of imbalanced datasets in machine learning. JMLR, 18(17). [Imbalanced-learn library including SMOTE]
• 25. Dodge, Y. (Ed.) (2003). The Oxford Dictionary of Statistical Terms. OUP. [Reference for statistical preprocessing terminology]
• 26. Arxiv (2024). DiffPrep: differentiable data preprocessing pipeline search for learning over tabular data. arXiv:2308.10915. [Differentiable preprocessing]
• 27. Arxiv (2025). The impact of feature scaling in machine learning: effects on regression and classification tasks. arXiv:2506.08274. [Empirical study across 14 algorithms]
• 28. NHS England (2025). Data (Use and Access) Act 2025 guidance for health data preprocessing and anonymisation. NHS Digital / NHS England. [UK regulatory context for health data preprocessing]

Provenance