3D Generation

3D Generation refers to the automated or semi-automated creation of three-dimensional geometry, appearance, and scene representations using machine learning models, procedural algorithms, or hybrid approaches. Techniques include text-to-3D, image-to-3D, and scene-level generation via neural radiance fields, Gaussian splatting, diffusion models, and large multi-modal networks. The field aims to dramatically reduce the time and skill threshold required to produce photorealistic or stylised 3D content for gaming, film, spatial computing, e-commerce, digital twins, and robotics simulation.

Semantic Classification

Content

Compositional Relationships (Components)

SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:hasPart ai:TextTo3D))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:hasPart ai:NeuralRadianceField))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:hasPart ai:GaussianSplatting))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:hasPart ai:MeshGeneration))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:hasPart ai:NovelViewSynthesis))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:hasPart ai:SceneGeneration))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:hasPart ai:TextureSynthesis))

Dependency Relationships

SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:requires ai:GPUCompute))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:requires ai:Objaverse))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:requires ai:DifferentiableRendering))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:dependsOn ai:DiffusionModel))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:dependsOn ai:ImplicitNeuralRepresentation))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:dependsOn ai:Triplane))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:dependsOn ai:CLIP))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:dependsOn ai:DeepLearning))

Capability Relationships

SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:enables ai:3DContentCreation))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:enables ai:DigitalTwinGeneration))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:enables ai:GameAssetGeneration))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:enables ai:SceneGeneration))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:enables ai:SpatialComputing))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:enables ai:ExtendedReality))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:supports ai:ECommerce))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:supports ai:RoboticsSimulation))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:supports ai:FilmPrevisualisation))

Implementation Relationships

SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:implements ai:ScoreDistillationSampling))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:implements ai:FeedForwardReconstruction))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:implements ai:GaussianSplatting))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:implements ai:NeuralRadianceField))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:uses ai:DiffusionModel))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:uses ai:CLIP))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:uses ai:UNet))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:uses ai:VisionTransformer))

Reduction Relationships

SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:reducesTo ai:ImageGeneration))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:reducesTo ai:GenerativeAdversarialNetwork))
SubClassOf(ai:3DGeneration
  ObjectSomeValuesFrom(ai:reducesTo ai:VariationalAutoencoder))

About

3D Generation encompasses the full pipeline from conditioning signal (text, image, video, point cloud) to a usable three-dimensional asset or scene representation. The challenge it solves is both perceptual — ensuring novel viewpoints of the generated scene look photorealistic or stylistically coherent — and geometric — ensuring the underlying 3D structure is consistent, watertight, and suitable for downstream tasks such as real-time rendering, physics simulation, or fabrication. Unlike Image Generation, which operates in a well-structured 2D pixel domain, 3D generation must contend with the fundamental ambiguities of projective geometry: an infinite family of 3D scenes can produce the same 2D image, so learning a generative model that produces geometrically plausible 3D structure requires either strong 3D supervision, multi-view consistency constraints, or the kind of diffusion-model-lifted priors that Score Distillation Sampling provides.

The intellectual genealogy of the field runs from classical procedural modelling (L-systems, constructive solid geometry, fractal terrain, 1970s–1990s) through statistical shape models (active shape models, morphable face models, Blanz & Vetter 3D Morphable Model 1999, 2000s), early generative deep learning on 3D voxel grids and Point Cloud structures (Wu et al. 3D-GAN 2016; Fan et al. PointNet 2017; Achlioptas et al. 2018), the implicit-representation revolution initiated by Occupancy Network (Mescheder et al. 2019) and Signed Distance Function-based methods (Park et al. 2019 DeepSDF), the neural radiance field breakthrough (Mildenhall et al. 2020 NeRF), and the subsequent explosion of diffusion-based and feed-forward reconstruction systems from 2022 onward. Each generation of technique inherited the prior generation’s evaluation metrics and benchmark datasets while dramatically expanding quality and scope.

The pivotal scaling moment arrived when the Objaverse dataset (Deitke et al. 2023) made 800K+ annotated 3D assets publicly available, followed by Objaverse-XL expanding to 10M+ objects. This gave researchers the large-scale 3D training corpus needed to train feed-forward reconstruction networks capable of generalising across object categories. Previously, models trained on ShapeNet (55 categories, ~50K models) showed poor cross-category transfer and were largely confined to cars, chairs, and aeroplanes. Objaverse scale broke this bottleneck, enabling Zero123, MVDream, CRM, and TRELLIS to generalise to arbitrary object types.

As of 2026, the object-level single-image-to-3D subsector has reached near-production maturity. Microsoft’s TRELLIS model (released December 2024, open-sourced under MIT licence) is a 4-billion-parameter structured latent diffusion model using a triplane-Gaussian hybrid representation that produces 1536-resolution 3D assets in under 20 seconds on a single 24 GB GPU, with PBR texture output and mesh topology suitable for game engine import. Meshy has generated over 30 million 3D assets for more than 3 million creators globally. The Tripo AI, Hitem3D, and Kaedim platforms offer competing API services. Full Scene Generation — producing cohesive environments with consistent material properties, lighting, and semantic structure across many objects — remains an active frontier where compositional generation, layout-conditioned diffusion, and world-model approaches (Google’s Genie 2, DeepMind’s DreamerV3) are competing paradigms.

The economic impact is beginning to materialise: the global 3D content creation market (estimated at $3.2 billion in 2024 by market research firms) is projected to grow at 18% CAGR through 2030, with AI-generated content expected to constitute 30–40% of all newly created 3D assets by 2028. Game studios, architectural visualisation firms, and e-commerce platforms are the early adopters; digital twin generation for manufacturing and infrastructure is the emerging enterprise wedge.

The technical barriers that remain significant in 2026 are: (1) material and lighting disentanglement — separating geometry, albedo, roughness, metallicity, and lighting from appearance observations is an ill-posed problem still not fully solved; (2) topological quality — automatically generating production-quality mesh topology (quad-dominant, suitable for animation rigging) rather than dense triangle soups remains challenging; (3) scale consistency — generated objects often have ambiguous real-world scale; (4) scene-level coherence — multi-object scenes require relational reasoning beyond object-centric generation; and (5) physical plausibility — generated assets frequently violate structural physics (floating objects, interpenetrating geometry) in ways that require post-processing or constrained generation.

Components and Architecture

3D Representation Families

• Neural Radiance Fields (NeRF): Implicitly encode a scene as a continuous volumetric function mapping 3D coordinates (x,y,z) and viewing directions (θ,φ) to colour (r,g,b) and volume density σ. Rendered via differentiable ray marching: for each camera ray, sample points along the ray, evaluate the MLP at each sample, and composite colour and density using the volume rendering equation C(r) = ∫ T(t) σ(r(t)) c(r(t),d) dt where T(t) = exp(-∫₀ᵗ σ(r(s)) ds) is the accumulated transmittance. High photorealism but slow per-scene optimisation (hours to days in the original 2020 implementation). Instant-NGP (Müller et al. 2022) replaced the single MLP with a hash-grid encoding, reducing training to seconds on a single GPU. Key subsequent variants: Zip-NeRF (2023), Nerfacto (Nerfstudio), and mip-NeRF 360 for unbounded scenes.

• Gaussian Splatting (3DGS): Represent scenes as collections of parametric 3D Gaussian primitives, each characterised by a 3D centre position μ, a full 3×3 covariance matrix Σ (decomposed as Σ = RSS^T R^T for rotation R and scale S), colour represented as low-degree spherical harmonics coefficients, and an opacity scalar α. Rendering proceeds by projecting 3D Gaussians to 2D screen-space ellipses and alpha-compositing them in depth-sorted order using a tile-based rasteriser. Training requires only minutes; rendering at 100+ FPS on consumer GPUs; representation is explicit and directly editable. Widely adopted for capture (Luma AI, Polycam) and is becoming the preferred format for real-time XR experiences in 2025–2026.

• Triplane Representations: Factor the 3D volume into three axis-aligned 2D feature planes (XY, XZ, YZ). For a 3D query point p=(x,y,z), the triplane feature is obtained by projecting p onto each plane and bilinearly interpolating, then concatenating the three feature vectors and decoding with a small MLP. This hybrid approach allows efficient use of 2D convolutional backbones (e.g., U-Net or Vision Transformer) for generating 3D content, since triplane features are simply three channel images. Used in EG3D (Chan et al. 2022), GET3D (Gao et al. 2022), and as the structured latent format in TRELLIS (Xiang et al. 2024) which denoises triplane tokens with a DiT (Diffusion Transformer) backbone.

• Mesh-Based Representations: Polyhedral meshes with UV-mapped textures remain the industry standard for real-time game engines, VFX pipelines, and physical manufacturing. Most 3D generation pipelines convert their intermediate representation to mesh as a final step. Methods include classical marching cubes (limited to axis-aligned voxels), Deep Marching Tetrahedra (DMTet, Shen et al. 2021) which learns a differentiable mesh extraction from SDF fields, and FlexiCubes (Shen et al. 2023) which allows gradient flow through mesh topology. CraftsMan and TRELLIS produce UV-ready meshes suitable for direct import into Unreal Engine or Blender without manual retopology.

• Point Cloud Methods: Direct generation over sets of 3D points (unordered). Point-E (OpenAI 2022) uses a two-stage pipeline: a text-conditioned CLIP-guided image generator followed by a point cloud diffusion model conditioned on the generated image. Fast (seconds) but lower quality than mesh-based methods; primarily useful for initialising downstream refinement.

  Generative Conditioning Mechanisms

• Score Distillation Sampling (SDS): The key insight of DreamFusion (Poole et al. 2022) is that a pre-trained 2D diffusion model implicitly defines a probability distribution over images that A* can treat as a score function. Given a 3D representation θ (e.g., NeRF parameters), differentiable rendering produces a 2D image x from a random camera viewpoint. The score gradient from the diffusion model — evaluated at a randomly perturbed version of x at noise level t — provides a signal ∇_θ L_SDS = E_t,ε[w(t)(ε̂_φ(x_t; y, t) - ε)∂x/∂θ] that pushes the 3D scene to look like images the diffusion model assigns high probability. Original SDS produces over-saturated, low-diversity, sometimes multi-faced (Janus problem) outputs. Improved variants: Variational Score Distillation (VSD, Wang et al. 2023 ProlificDreamer) treats the 3D parameters as samples from a variational distribution, dramatically improving quality and diversity. Score Distillation via Reparametrised DDIM (SRDD, 2024) reparametrises the SDS objective to avoid the mode-collapse tendency.

• Feed-Forward Multi-View Reconstruction (Large Reconstruction Models): Train a large Transformer Architecture directly on (image, camera pose, 3D representation) triples drawn from rendered Objaverse assets. At inference, a single conditioning image (or a text-to-image generation from Stable Diffusion / DALL-E) is passed through an image encoder; camera-conditioned cross-attention decodes a 3D representation in a single forward pass. Key examples: Zero123 (Liu et al. 2023) fine-tuned Stable Diffusion for novel-view synthesis; CRM (Wang et al. 2024) uses a convolutional reconstruction model outputting a triplane mesh; TRELLIS (Xiang et al. 2024) denoises structured triplane latents with a 4B-parameter DiT.

• Multi-View Consistency Networks: Generate multiple consistent views of an object simultaneously, then fuse into 3D via traditional multi-view stereo or Gaussian splatting. MVDream (Shi et al. 2023) generates four consistent views from a text prompt; Zero123++ (Liu et al. 2023) generates six views from a single image. Consistent multi-view generation is the current best-practice first stage before 3D reconstruction.

• Native 3D Diffusion: Directly diffuse over 3D representations without 2D lifting. Point-E (OpenAI 2022) diffuses over point clouds; Shap-E (OpenAI 2023) diffuses over implicit function parameters; GaussianDreamer (Yi et al. 2024) and LGM (Tang et al. 2024) diffuse over Gaussian parameters. Quality historically lagged 2D-lifted approaches but the gap is narrowing rapidly; native 3D diffusion avoids the multi-view inconsistency problem inherent in SDS.

  Training Data Ecosystem

• Objaverse (Deitke et al. 2023, CVPR): 800K+ 3D assets scraped from Sketchfab under CC licences; includes diverse object categories with text annotations and rendered images. The dominant training corpus for object-level generation from 2023 onward.

• Objaverse-XL (Deitke et al. 2023, Stability AI): Extends to 10M+ objects across GitHub, Thingiverse, Sketchfab, and other web sources. Provides the scale needed for large foundation models.

• Rendered multi-view datasets: Objaverse rendered at 32–256 viewpoints per object under variable lighting provides paired (image, 3D) supervision; Zero123’s training used 100M+ such renders; CRM used 280 views per object.

• ShapeNet (Chang et al. 2015): The original benchmark dataset, 55 categories and ~50K models; now considered too small and category-limited for modern foundation model training but still used for evaluation.

• ScanNet and ScanNet++: Real-world indoor scene scans used for scene-level reconstruction and generation research.

  Mathematical Foundations and Technical Deep Dive

  The Score Distillation Sampling (SDS) Objective DreamFusion’s key insight is that a pre-trained 2D diffusion model ε_φ(x_t; y, t) — which predicts the noise added to a clean sample x₀ at timestep t — implicitly defines a distribution over images that can be used as a score function to guide 3D optimisation. Given a 3D representation θ (NeRF parameters, Gaussian parameters, etc.), differentiable rendering g(θ, c) produces an image x = g(θ, c) for camera pose c. The SDS gradient with respect to θ is:

  ∇_θ L_SDS(φ, x=g(θ)) = E_{t,ε,c} [w(t)(ε_φ(x_t; y, t) − ε) ∂x/∂θ]

  where x_t = √ᾱ_t x + √(1−ᾱ_t) ε is the noisy image, ᾱ_t is the signal-to-noise schedule, w(t) is a weighting function, y is the text conditioning, and ε is the sampled noise. Crucially, the Jacobian ∂ε_φ/∂x is omitted because computing it requires backpropagation through the entire diffusion U-Net — computationally prohibitive. This omission introduces the “variance of SDS” problem (over-saturation and mode collapse) addressed by Variational Score Distillation (ProlificDreamer).

  NeRF Volume Rendering NeRF encodes a scene as a function F_θ: (x, d) → (c, σ) mapping 3D position x ∈ ℝ³ and viewing direction d ∈ S² to RGB colour c ∈ [0,1]³ and volume density σ ∈ ℝ⁺. The rendered colour C(r) for camera ray r(t) = o + td is:

  C(r) = ∫₀^∞ T(t) σ(r(t)) c(r(t), d) dt

  where T(t) = exp(−∫₀^t σ(r(s)) ds) is the accumulated transmittance. In discrete form with stratified sampling at N points:

  C(r) ≈ Σᵢ Tᵢ (1 − exp(−σᵢδᵢ)) cᵢ

  where Tᵢ = exp(−Σⱼ<ᵢ σⱼδⱼ) and δᵢ is the distance between adjacent samples. The training objective minimises MSE between rendered pixels and ground-truth images: L = Σᵣ ||C(r) − C_gt(r)||₂².

  Gaussian Splatting Rendering Each 3D Gaussian primitive is defined by: (μ ∈ ℝ³, Σ ∈ ℝ³ˣ³, α ∈ [0,1], sh ∈ ℝ^(3k)) where μ is position, Σ = RSS^T R^T is the covariance (R rotation, S diagonal scale), α is opacity, and sh are spherical harmonic coefficients for view-dependent colour. Rendering to a 2D pixel p proceeds by:

  1. Project 3D Gaussian to 2D: Σ’ = JWΣWᵀJᵀ where J is the Jacobian of the projective transformation and W is the view matrix.
  2. Evaluate the 2D Gaussian at pixel p: exp(−½(p−μ’)^T Σ’⁻¹ (p−μ’))
  3. Alpha-composite Gaussians sorted by depth: C(p) = Σᵢ cᵢ αᵢ Πⱼ<ᵢ (1−αⱼ)

  Training minimises a combination of L1 and D-SSIM loss: L = (1−λ)L1 + λ L_D-SSIM with λ=0.2 typically. Adaptive densification adds new Gaussians in under-reconstructed regions.

  3D Representation Comparison Table

• NeRF: continuous, differentiable, slow to train (minutes–hours), slow to render (ms/frame), no native mesh output

• Instant-NGP: continuous with hash grid, fast to train (seconds), moderate render speed, no native mesh

• 3D Gaussian Splatting: explicit Gaussian primitives, fast to train (minutes), real-time render (30–150 FPS), editable but no native mesh; convert via GaussianObject or SuGaR

• Triplane: hybrid 2D/3D, amenable to 2D CNNs, fast inference with feed-forward decoders, mesh via SDF

• DMTet/FlexiCubes: differentiable mesh extraction, production-quality topology, not standalone representations

• Point Cloud: simple, unordered, easy to generate with MLP diffusion, low quality for rendering

  Evaluation Metrics 3D generation quality is evaluated using:

• PSNR (Peak Signal-to-Noise Ratio): dB value comparing rendered novel-view images to ground truth; typical values 25–35 dB for state-of-the-art.

• SSIM / LPIPS: Structural similarity and perceptual image patch similarity for rendered appearance.

• Chamfer Distance (CD): Point-to-point mean squared distance between generated and ground-truth point clouds; lower is better.

• Earth Mover’s Distance (EMD): Distribution distance between point clouds; more sensitive to outliers than CD.

• FID (Fréchet Inception Distance): Distribution-level quality metric comparing rendered image distributions; adapted from 2D image generation evaluation.

• Mesh quality metrics: Watertightness (no holes), manifoldness (no non-manifold edges), regularity of triangle size distribution.

• User studies: Human preference ratings comparing generated assets to baselines and to human-created assets; increasingly used for text-to-3D evaluation where ground-truth 3D is unavailable.

  Use Cases and Major Families

  Text-to-3D Users provide a natural language description (“a Victorian armchair with brass feet and green velvet upholstery”) and receive a 3D asset. Early work (DreamFusion, 2022) required 1.5–6 hours per object on a single A100 GPU with mediocre quality (over-saturation, Janus problem, poor topology). By 2025, commercial APIs from Meshy (Meshy 5, July 2025) and TRELLIS produce multi-thousand-polygon meshes with PBR textures in 15–30 seconds. Quality is competitive with entry-level freelance modelling for background assets and props. The primary bottleneck for production-grade results is consistency of fine-grained geometric detail (stitching on upholstery, engraving patterns) and topologically clean mesh output suitable for animation rigs.

  The text-to-3D pipeline typically chains: (1) text → multi-view images via a 2D diffusion model conditioned on camera poses; (2) multi-view images → 3D mesh via a feed-forward reconstruction network; optionally (3) texture baking and PBR material estimation. End-to-end text-to-3D models that avoid the intermediate image stage (native 3D diffusion) are actively researched but are not yet dominant commercially.

  Image-to-3D Single reference photograph → full 3D mesh. This is the most commercially mature modality in 2025–2026. Luma AI uses Gaussian Splatting and NeRF reconstruction for product photography capture, enabling photo-realistic 3D product viewers from a single smartphone photograph. E-commerce platforms (Shopify Magic 3D, Amazon AR product viewer) use image-to-3D to generate interactive product visualisations at scale. Zero123++, TripoSR (OpenAI / Tripo AI collaboration, 2024), and CRM achieve full-pipeline reconstruction times under 10 seconds on a consumer GPU. The dominant limiting factor is the inherent ambiguity: the unseen back and bottom of objects must be hallucinated, leading to “hallucinated geometry” artifacts that require quality checks or user-guided disambiguation.

  Video-to-3D and Photogrammetric Capture Luma AI’s video-based NeRF and Gaussian Splatting capture pipeline (2023–2025) allows consumers to reconstruct 3D scenes from 30–120 second smartphone video walkthrough footage, automatically extracting camera poses via COLMAP or deep-pose-estimation networks, and fitting a 3D Gaussian representation. Outputs are used in Unreal Engine (via Luma plugin), Unity (via Luma SDK), and WebXR experiences served directly from the browser. Polycam and RealityCapture offer competing photogrammetry pipelines. For cultural heritage applications, high-quality multi-image photogrammetric capture (100–400 DSLR images) with LiDAR fusion produces millimetre-accuracy 3D models; neural generation is complementing classical photogrammetry by filling occlusion gaps and super-resolving textures.

  Scene-Level Generation Moving beyond single-object generation to full scene synthesis with consistent spatial layout, inter-object occlusion, and coherent lighting. Current approaches: (1) Compositional diffusion (GALA3D, Set-the-Scene, Compositional 3D Scene Generation using Locally Conditioned Diffusion, 2024) places objects via a layout-conditioned diffusion model and then individually generates and inserts assets; (2) World models (Google Genie 2, 2024; DreamerV3) generate interactive 3D-consistent environments from video priors, primarily targeting game-like interactive environments; (3) Procedural hybrid (Infinigen, Holodeck, 2023–2024) uses LLM-guided procedural asset placement with neural texture generation. Scene-level generation for gaming environments — block-out of a dungeon, forest clearing, or urban block from a text prompt — is an active commercial target in 2026, with Epic Games AI Lab, Ubisoft, and startup studios (Latitude, Skybox AI, Blockade Labs) competing for this market segment.

  Human and Avatar Generation Generating photorealistic 3D human avatars from images or text is a high-commercial-value application for gaming, film, telepresence, and XR. Key systems: EVA3D (Hong et al. 2023) generates clothed 3D humans from monocular video; HumanGaussian (2024) uses 3D Gaussian Splatting for real-time avatar representation; Gaussian Head Avatar (2024) specialises on high-fidelity head reconstruction. The challenge is simultaneously capturing geometry (body shape, clothing drape, hair strands), appearance (skin subsurface scattering, specular highlights, cloth material), and dynamics (animation blend shapes, hair simulation). Imperial College London’s Stefanos Zafeiriou group (SmartLab) leads UK academic research on generative digital humans, with work on 3D face generation, expression transfer, and photorealistic rendering presented at CVPR 2024 and SIGGRAPH 2025.

  Robotics Simulation and Digital Twin Generating diverse photorealistic 3D environments for sim-to-real transfer is a high-priority application for training Robotics and Autonomous Navigation models. The domain randomisation approach (OpenAI Rubik’s cube dexterous manipulation, 2019) demonstrated that training on procedurally generated variations improves real-world transfer. Neural 3D generation extends this by creating higher-realism environments beyond what hand-crafted procedural systems can produce. Infinigen (Raistrick et al. 2023, Princeton/Berkeley) generates infinite photorealistic procedural 3D environments including terrain, vegetation, water, and atmospheric effects. Digital Twin generation — producing a 3D-accurate replica of an industrial plant, building interior, or infrastructure asset from sensor data — is emerging as an enterprise application, with companies such as Matterport, Synaptive Medical, and Bentley Systems integrating neural 3D generation into their asset-capture pipelines.

  E-Commerce and Retail Visualisation The transition from 2D product photography to 3D interactive product viewers and AR try-before-you-buy is driving commercial deployment. Image-to-3D APIs are enabling retailers to generate 3D models from existing product photographs without 3D studio shoots. IKEA, Wayfair, and Shopify have all announced AI 3D asset generation initiatives. The glTF 2.0 format (used in Gaussian-splat-to-mesh export pipelines) is the dominant interchange standard for web-delivered 3D content.

  Key Challenges and Open Problems

  The Janus Problem (Multi-Face Artifacts) Score Distillation Sampling optimises 3D scenes by rendering from random viewpoints, but the 2D diffusion model has no notion of 3D consistency across views. Consequently, it can “satisfy” the diffusion loss by placing a face on every side of a generated head — the infamous Janus Problem (named after the two-faced Roman god). Solutions include: multi-view consistent diffusion models (MVDream, Zero123++) that generate all views jointly; 3D-native diffusion (Shap-E, TRELLIS) that avoids SDS entirely; and view-specific masking strategies that enforce geometric consistency.

  Texture Quality and UV Mapping Generated 3D meshes often have baked (view-dependent) textures rather than proper UV-mapped PBR (Physically Based Rendering) materials. Baked textures produce correct appearance from the viewpoint used during generation but fail under relighting, novel viewpoints, or material editing. Generating UV-unwrapped, albedo-separated, metallic/roughness-decomposed PBR textures from a single-pass network is an active research problem. CraftsMan and TRELLIS represent 2025–2026 state-of-the-art for direct PBR output; most other methods require a separate texture-baking postprocessing step.

  Mesh Topology for Animation Most 3D generation methods produce dense triangle meshes (150K–2M triangles) that are topologically unsuitable for animation rigging: they lack clean edge loops around joints, have irregular polygon density, and cannot be easily subdivided. Professional character animation requires quad-dominant topology with carefully placed edge loops. Automatic retopology tools (ZRemesher, Instant Meshes, Neural Mesh Simplification) partially address this but cannot fully replicate the manual craft of a rigged character mesh. CraftsMan’s topology-aware generation is a significant step toward directly useful animation-ready assets.

  Scene-Level Coherence Single-object generation has matured substantially, but coherent multi-object scene generation remains significantly harder. Key sub-problems: (1) spatial layout consistency — objects must be properly grounded (on surfaces, not floating), correctly scaled relative to each other, and plausibly arranged; (2) material consistency — a scene generated from text should have a consistent lighting environment (all shadows cast in the same direction, consistent ambient light level); (3) semantic coherence — a kitchen should contain kitchen items at plausible positions, not random objects at random scales; (4) physical plausibility — generated scenes should pass basic sanity checks (furniture not intersecting walls, books on shelves not floating, characters standing on ground planes).

  Ambiguity in Single-Image-to-3D Single-view reconstruction is fundamentally ill-posed — infinitely many 3D scenes can produce the same 2D image. Feed-forward reconstruction networks necessarily hallucinate the unseen parts of the object (back, underside, interior). The quality of this hallucination depends on prior knowledge from training data: objects in categories well-represented in Objaverse (furniture, vehicles, common animals) are hallucinated plausibly; rare or unusual objects produce arbitrary geometry on occluded faces. This limits applicability to production workflows where all faces of an asset must be geometrically correct.

  Computational Cost Despite significant progress, 3D generation remains substantially more expensive than 2D generation:

• TRELLIS: 20 seconds on NVIDIA 4090 (24 GB VRAM) — fast but requires high-end consumer GPU

• SDS-based methods (ProlificDreamer): 30–60 minutes on A100 — research-grade only

• Multi-view rendering + Gaussian fitting: 5–30 minutes for high quality — moderate

• Point-E/Shap-E (2022 OpenAI, point cloud diffusion): 1–20 seconds — fast but low quality

  Democratising 3D generation to consumer hardware (mobile, low-end GPU) remains a significant engineering challenge requiring model compression, efficient representations, and hardware-specific optimisation.

  Academic Context

  3D generation sits at the confluence of computer graphics, computer vision, and machine learning, drawing theoretical foundations from differential geometry, probabilistic generative modelling, and representation learning. The primary venues are SIGGRAPH and SIGGRAPH Asia (for rendering and geometry contributions), CVPR, ICCV, ECCV (for computer vision contributions), NeurIPS and ICLR (for machine learning contributions), and the specialist 3DOR (3D Object Retrieval) and Shape Modelling International workshops.

  The founding papers of the modern neural 3D generation era are:

  1. Occupancy Networks (Mescheder et al., NeurIPS 2019): Introduced the implicit occupancy function f(x, z) → [0,1] where z is a latent shape code, enabling continuous 3D shape generation via learned implicit surfaces without fixed voxel grids.
  2. DeepSDF (Park et al., CVPR 2019): Signed distance functions as latent-conditioned implicit surfaces, enabling shape completion, interpolation, and continuous 3D shape spaces.
  3. NeRF (Mildenhall et al., ECCV 2020): Neural Radiance Fields achieved state-of-the-art novel view synthesis by encoding scenes as continuous volumetric functions via MLP, trained with a differentiable volume rendering loss.
  4. pi-GAN (Chan et al., CVPR 2021): Implicit neural representations for 3D-consistent face generation from 2D supervision only, establishing the template for GAN-based neural 3D generation.
  5. Instant-NGP (Müller et al., SIGGRAPH 2022): Hash-grid encoding reduced NeRF training from 12–24 hours to 5–15 seconds, democratising per-scene neural reconstruction.
  6. DreamFusion (Poole et al., ICLR 2023): Score Distillation Sampling coupled NeRF optimisation with frozen 2D diffusion priors for text-to-3D generation without any 3D training data, opening the zero-shot text-conditioned regime.
  7. 3D Gaussian Splatting (Kerbl et al., SIGGRAPH 2023): Explicit radiance-field representation achieving real-time (30–150 FPS) rendering while maintaining NeRF-quality appearance; winner of the SIGGRAPH 2023 Best Paper Award.
  8. Zero123 (Liu et al., ICCV 2023): Fine-tuned Stable Diffusion on Objaverse multi-view renders for image-conditioned novel-view synthesis, establishing the template for the “image-to-3D via 2D diffusion prior” approach.
  9. ProlificDreamer (Wang et al., NeurIPS 2023): Variational Score Distillation improved text-to-3D diversity and quality substantially, reducing over-saturation by treating 3D parameters as samples from a variational posterior.
  10. TRELLIS (Xiang et al., Microsoft Research Asia, arXiv:2412.01506, 2024): 4B-parameter structured latent diffusion model with SLAT (Structured LATent) representation; production-quality 3D asset generation at 1536 resolution in <20 seconds under MIT licence.

  Key research groups: MIT CSAIL (visual scene understanding, NeRF variants), Stanford Graphics Lab (differentiable rendering, neural rendering), Berkeley AI Research (compositional generation, Instant3D), CMU Robotics Institute (sim-to-real, robot manipulation), Google DeepMind Visual Geometry Group (NeRF, depth estimation, world models), Microsoft Research Asia (TRELLIS, shape generation), ETH Zürich (geometry processing, neural rendering), Max Planck Institute for Intelligent Systems (human avatar generation, implicit representations).

  Current Landscape (2026)

  The year 2025 marked the inflection point where 3D generation transitioned from primarily research demonstrations to production-adjacent tooling. The structural shift is twofold: open-source models have reached production quality, and the commercial moat has migrated from model weights to tooling, integrations, and fine-tuning services.

  Foundation Models for 3D

• TRELLIS (Microsoft Research Asia, December 2024, MIT licence): 4B-parameter structured latent diffusion model using a DiT (Diffusion Transformer) backbone and a SLAT (Structured LATent) triplane-Gaussian hybrid representation. Produces 1536-resolution 3D assets in under 20 seconds on a single NVIDIA 4090 (24 GB), with physically-based rendering (PBR) texture output and export to GLB, OBJ, or Gaussian Splat formats. TRELLIS v2 (2025) further scales the model and improves texture quality; benchmarks on Google Scanned Objects and OmniObject3D show PSNR improvements of 2–3 dB over prior state-of-the-art.

• CraftsMan (2025): Focuses on production-quality topology — the generated meshes use intelligent edge flow suitable for subdivision surface smoothing and animation rigging, addressing a key pain point of earlier methods that produced irregular dense triangle soups.

• Tripo AI, Hitem3D: Independent commercial platforms competing in the text/image-to-3D API market, targeting game developers and e-commerce providers.

  Commercial Platform Ecosystem

• Meshy (Meshy 5, July 2025): Over 30 million 3D assets generated for 3 million+ creators globally. Meshy 5 introduced smarter material decomposition, improved animation potential (blend shape generation from reference poses), and API access for integration into game engines and pipeline tools.

• Luma AI: Dominates photorealistic Gaussian-splat and NeRF capture from consumer video; expanding into text-to-3D generation via a generative model trained on its large capture dataset. The Luma Unreal Engine plugin and Luma Web SDK provide direct integration pathways.

• Kaedim: SaaS platform targeting 3D game asset production; accepts sketch or image references and delivers production-quality mesh with materials in hours, combining AI generation with human artist QA.

• Scenario, Blockade Labs, Skybox AI: Specialise in AI-generated game environments and skybox textures; beginning to integrate 3D generation (scene blocking, asset population) as the next layer above 2D generation.

  XR Integration Progress

• Gaussian Splatting assets are deployable in WebXR via three.js, Babylon.js, and PlayCanvas libraries, enabling browser-based 3D visualisation without plugin installation.

• Apple Vision Pro’s visionOS 2.1 (2025) added native Gaussian splat rendering via RealityKit, enabling direct capture → spatial computing workflows.

• Meta Quest’s OpenXR integration supports GLB and USD import, with community tooling for Gaussian splat → USD conversion.

  Industry Adoption Patterns

• Gaming: Ubisoft AI Lab, EA’s SEED research group, Epic Games’ AI initiatives, and startup studios are testing 3D generation for environment blocking (rough 3D layout of game levels), secondary asset population (rocks, crates, debris, vegetation), and concept visualisation. The workflow model is “AI draft → human artist polish” rather than full AI replacement.

• E-commerce: Shopify Magic, Amazon’s product 3D viewer, and Wayfair Room Planner are using image-to-3D APIs to create interactive product visualisations at scale. The glTF 2.0 format is the interchange standard for this use case.

• Architectural visualisation: ArchViz firms are using AI-generated 3D props and furniture to accelerate scene population; the primary adoption driver is reducing the time to populate a rendered interior from days to hours.

• Film and TV pre-production: VFX studios and animation houses are evaluating 3D generation for pre-visualisation (previs) — rough 3D animatics showing scene composition and camera moves before full production begins.

  Open-Source Ecosystem The consolidation of high-quality open-source models (TRELLIS MIT, CRM Apache 2.0, Zero123++ community licence, GaussianObject, LGM) means that model weights are no longer a defensible commercial moat. The competitive differentiators in 2026 are user experience, pipeline integration, fine-tuning services for domain-specific assets, quality assurance workflows, and enterprise support.

  UK Context

  The UK has a significant and growing academic and industrial presence in 3D generation and neural rendering, anchored by several major funding programmes and industrial clusters:

  EPSRC AI Hub in Generative Models (£80M, 2024–2029) The UK’s most significant direct investment in 3D generation research is the EPSRC-funded AI Hub in Generative Models, launched February 2024 and fully operational from March 2025. Led by UCL and bringing together Imperial College London, Cardiff University, the University of Cambridge, the University of Oxford, the University of Manchester, the University of Edinburgh, and the University of Surrey with 40+ industry partners, this hub specifically targets next-generation generative AI including diffusion models, neural rendering, and 3D generation. The hub is funded until 2029 and aims to deliver projects across image/video/3D generation, geometric deep learning, and generative models for scientific applications (drug discovery, materials science, climate modelling). The hub’s 3D generation research is concentrated at UCL, Imperial, and Edinburgh.

  UCL Department of Computer Science UCL hosts the 3DOR (3D Object Retrieval) workshop series and the 3dor.cs.ucl.ac.uk research programme, spanning 3D shape analysis, 3D generation, and geometry processing. UCL’s Computer Graphics, Vision and Imaging (CGVI) group has active research in neural rendering, differentiable rendering, and generative 3D. UCL also hosts the AI Centre’s Generative AI hub coordination, with researchers working on diffusion model theory and large-scale generative pre-training.

  Imperial College London — SmartLab (Zafeiriou Group) The Intelligent Behaviour Understanding Group (iBUG, led by Stefanos Zafeiriou at Imperial’s Department of Computing) is the UK’s leading academic group for 3D face and human generation. Research areas include 3D Morphable Models for faces, video-to-3D reconstruction of faces and bodies, generative models for digital humans, and expression/identity transfer. Zafeiriou delivered a keynote on “Generative Models for Digital Humans” at the 3DOR 2025 workshop. Imperial’s work has direct commercial relevance to film VFX, gaming avatar creation, and telepresence systems.

  University of Edinburgh — School of Informatics Edinburgh has historically strong research in geometry processing, character animation, and physically-based simulation (ACM SIGGRAPH publications from the IPAB and CISA3 groups). In 3D generation specifically, Changjian Li’s group works on generating 3D animation from 2D storyboards and related shape-from-sketch problems. Edinburgh’s connection to the EPSRC Generative AI Hub and to the Alan Turing Institute reinforces its position as a leading 3D generation research site in the UK.

  Northern England Industrial Context Northern English universities are applying 3D generation to manufacturing-adjacent digital twin and inspection problems:

• University of Manchester: Digital twin research for advanced manufacturing; collaborations with Siemens Energy and BAE Systems on AI-assisted 3D modelling from sensor data; applying NeRF-based reconstruction to nuclear decommissioning inspection (hazardous environments inaccessible to humans).

• University of Sheffield — AMRC (Advanced Manufacturing Research Centre): 3D generation and reconstruction applied to quality inspection, defect detection, and digital twin creation for aerospace components manufactured at the Airbus Sheffield facility.

• University of Newcastle: Digital Heritage and Cultural Computing; applying photogrammetry and NeRF-based 3D capture to heritage building documentation (Northumberland castles, Roman sites along Hadrian’s Wall).

  UK Industrial Players

• Foundry (London): Developer of Mari, Katana, and Modo — professional 3D tools for VFX; investing in AI-assisted texturing, material generation, and 3D asset generation integrations.

• Rebellion Developments (Oxford): AAA game studio evaluating AI-generated 3D environments and assets for their game production pipeline.

• Improbable (London): Spatial computing and simulation; using AI-generated 3D environments for virtual worlds and defence simulation.

• Digital Catapult (London): National innovation centre facilitating industry-academia collaboration on XR and 3D generation applications for UK creative industries.

  Future Directions (2026–2030)

  Unification of Generation Modalities The near-term trajectory (2026–2028) is convergence of image, video, and 3D generation into unified world models. Systems like Google Genie 2 and OpenAI’s rumoured “4D” models aim to generate time-coherent 3D scenes from text or video prompts, with physically plausible dynamics (falling objects, flowing water, character locomotion). The architecture is typically a video generation backbone (spatial-temporal transformer) with an additional 3D geometry prediction head or depth estimation module.

  Real-Time Generation on Edge Hardware Distillation, quantisation, and architecture search are targeting 3D generation on consumer and embedded hardware:

• Apple Silicon (M4, M5 series Neural Engine): Target is sub-5-second text-to-3D on device, enabling AR authoring directly on iPhone or iPad.

• NVIDIA RTX 4070/5080 laptops (8–12 GB VRAM): Quantised TRELLIS-style models are expected to run at 30–60 seconds per asset by 2027.

• Meta Quest 4 (2026 expected): Onboard neural processing enabling real-time Gaussian splat capture and lightweight generation for spatial computing authoring.

  Physics-Consistent and Simulation-Ready Generation A key gap in 2026 is that generated 3D assets are visually plausible but physically inconsistent — objects can float, interpenetrate, or have mass distributions that make them fall over. Research directions: (1) physics simulation loss during generation (differentiable rigid-body simulation); (2) structural-integrity constraints (ensuring generated furniture, architecture, and vehicles could physically support their stated loads); (3) material property prediction alongside geometry (density, elasticity, friction coefficient) to enable physics simulation.

  Agentic 3D Content Pipelines The long-term vision (2027–2030) is LLM-orchestrated multi-step agentic pipelines that treat 3D generation as one tool among many. An agent might decompose “a medieval village square” into: select a cobblestone texture from a texture library; generate a fountain object via text-to-3D; generate surrounding buildings with OpenAI Point-E and refine with TRELLIS; generate market stalls and wooden carts; assemble all assets in Blender via a Python API; validate physical plausibility with a simulation pass; and iterate based on user feedback. This represents a shift from “model that generates 3D” to “system that manages 3D content production”.

  Fine-Grained Editing and Part-Level Control Moving beyond “generate then manually fix” to interactive, semantically-aware editing:

• Part segmentation during generation: automatically identifying and labelling semantic parts (chair leg, seat, backrest) in the generated mesh, enabling selective replacement or restyling.

• Material and appearance transfer: applying the material style of a reference photograph to a generated geometry.

• Pose and animation control: generating 3D characters in specified poses or with specified skeletal rigs from text or image references.

• Geometric constraints: generating objects with specified measurements (a table exactly 75 cm high and 120 cm wide) or from engineering specifications (dimensions from a 2D drawing).

  Regulatory and Intellectual Property Landscape The training data for most 3D generation models includes Sketchfab assets under Creative Commons licences; Objaverse’s 800K+ assets are CC-BY licenced but many originate from commercial sources. The UK’s Intellectual Property Office is actively consulting (as of 2025–2026) on AI-generated works and training data copyright, with specific relevance to 3D assets embedded in commercial products (games, films, architectural designs). The EU AI Act’s transparency requirements for training data are creating additional compliance overhead for commercial 3D generation service providers operating in European markets.

  Broader Impact, Ethical Considerations, and Standardisation

  Creative Industry Disruption 3D Generation is reshaping the economics of 3D content production. For high-volume, lower-complexity 3D assets (game environment props, e-commerce product models, background architecture) AI generation is beginning to substitute for and redirect the labour of 3D generalist artists. For high-complexity, hero assets (main characters, hero vehicles, architectural centrepieces) AI tools are acting as accelerators for human artist workflows — generating rough drafts, reference variations, or texture options that artists refine. The net employment impact is contested: proponents argue for productivity amplification and new role creation (3D generation operators, AI output curators, model fine-tuning specialists); critics note that entry-level 3D modelling roles (the training ground for senior artists) are at particular risk.

  Deepfake and Misuse Concerns 3D Generation of realistic human faces and bodies raises significant concerns about deepfake creation, non-consensual avatar generation, and identity theft in metaverse and Extended Reality contexts. Apple Vision Pro’s Persona system and Meta’s codec avatars are beginning to intersect with AI-generated avatar technology. Watermarking and provenance tracking of AI-generated 3D assets (analogous to C2PA standards for 2D imagery) are active technical and policy discussions. The UK Online Safety Act (2023) and emerging EU AI Act provisions both have relevance to synthetic 3D human generation.

  Standardisation and Interoperability The 3D format landscape in 2026 is fragmented:

• glTF 2.0 (Khronos Group): The dominant open standard for web and game engine delivery; supported by Babylon.js, three.js, PlayCanvas, Unity, Unreal, Blender.

• USD (Universal Scene Description) (Pixar/Apple): Increasingly dominant for film VFX pipelines and Apple Vision Pro spatial computing; Apple’s RealityKit uses USDZ (zip-compressed USD).

• 3MF: Additive manufacturing (3D printing) format; not relevant to neural 3D generation but important for physical fabrication workflows downstream of generation.

• Gaussian Splat formats: No unified standard as of 2026; .ply (PLY with Gaussian properties) is de facto, with active Khronos working group developing an extension to glTF for Gaussian Splatting.

• OpenXR and WebXR: The cross-platform XR standards used for spatial computing; glTF support is native; Gaussian splat support is achieved via WebGL-based splatting renderers.

  The lack of a unified 3D generation output standard means that assets generated by TRELLIS, Meshy, or Luma AI require format conversion steps before import into production pipelines, creating friction that tool vendors are addressing via direct plugin integrations (Luma → Unreal, Meshy → Blender, TRELLIS → Blender/Unity).

  Dataset Ethics and Copyright Training 3D generation models on Objaverse requires careful consideration of the Creative Commons licence conditions: CC-BY (attribution), CC-BY-SA (attribution + share-alike), CC-BY-NC (non-commercial only), and CC0 (public domain). Commercial 3D generation services must ensure their training datasets do not include CC-BY-NC or CC-BY-SA assets if commercial outputs are intended. The UK Intellectual Property Office’s 2023 consultation on AI and copyright concluded (2024) that training on copyright material without authorisation infringes existing copyright law even for non-commercial research purposes, with a narrow text-and-data mining exception; the policy landscape for commercial 3D generation training data remains uncertain and is subject to ongoing legal challenge and legislation.

  Research and Literature

  1. Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., & Geiger, A. (2019). Occupancy Networks: Learning 3D Reconstruction in Function Space. CVPR 2019.
  2. Park, J.J., Florence, P., Straub, J., Newcombe, R., & Lovegrove, S. (2019). DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. CVPR 2019.
  3. Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., & Ng, R. (2020). NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. ECCV 2020.
  4. Müller, T., Evans, A., Schied, C., & Keller, A. (2022). Instant Neural Graphics Primitives with a Multiresolution Hash Encoding. SIGGRAPH 2022.
  5. Poole, B., Jain, A., Barron, J.T., & Mildenhall, B. (2022). DreamFusion: Text-to-3D using 2D Diffusion. ICLR 2023.
  6. Kerbl, B., Kopanas, G., Leimkühler, T., & Drettakis, G. (2023). 3D Gaussian Splatting for Real-Time Radiance Field Rendering. SIGGRAPH 2023.
  7. Liu, R., Wu, R., Van Hoorick, B., Tokmakov, P., Zakharov, S., & Vondrick, C. (2023). Zero-1-to-3: Zero-Shot One Image to 3D Object. ICCV 2023.
  8. Wang, Z., Lu, C., Wang, Y., Liu, F., Yu, J., Lin, D., & Dahua, L. (2023). ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation. NeurIPS 2023.
  9. Deitke, M., et al. (2023). Objaverse: A Universe of Annotated 3D Objects. CVPR 2023.
  10. Deitke, M., et al. (2023). Objaverse-XL: A Universe of 10M+ 3D Objects. Stability AI Technical Report.
  11. Shi, Y., et al. (2023). MVDream: Multi-view Diffusion for 3D Generation. ICLR 2024.
  12. Wang, Y., et al. (2024). CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Model. ECCV 2024.
  13. Xiang, J., et al. (2024). TRELLIS: Structured 3D Latents for Scalable and Versatile 3D Generation. Microsoft Research Asia, arXiv:2412.01506.
  14. Stability AI. (2024). Stable Zero123: Quality 3D Object Generation from Single Images. Stability AI Blog.
  15. Gao, J., et al. (2022). GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images. NeurIPS 2022.
  16. Chan, E.R., et al. (2022). Efficient Geometry-aware 3D Generative Adversarial Networks (EG3D). CVPR 2022.
  17. Jun, H., & Nichol, A. (2023). Shap-E: Generating Conditional 3D Implicit Functions. arXiv:2305.02463. OpenAI.
  18. Liu, M., et al. (2024). One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion. CVPR 2024.
  19. Tang, J., et al. (2024). LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation. ECCV 2024.
  20. Li, Z., et al. (2025). DirectTriGS: Triplane-based Gaussian Splatting Field Representation for 3D Generation. arXiv:2503.06900.
  21. UCL Centre for Artificial Intelligence. (2025). Launch of Research Hub to Help Deliver the Next Generation of Generative AI Models. UCL News, March 2025.
  22. 3DAI Studio. (2026). State of AI 3D Generation 2026 — Market, Models, Open Source, MCP & APIs. 3daistudio.com.
  23. Meshy. (2025). Introducing Meshy 5 Preview: Smarter AI, Cleaner Models, Bigger Animation Potential. ACN Newswire, July 2025.
  24. Advances in Extended Reality with 3D Gaussian Splatting. (2024). arXiv:2412.06257.
  25. Survey: Recent Advances in 3D Object and Scene Generation. (2025). arXiv:2504.11734.
  26. 3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities. (2024). arXiv:2407.17418.
  27. Shen, T., et al. (2021). Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis (DMTet). NeurIPS 2021.
  28. Wang, P., et al. (2024). Score Distillation via Reparametrized DDIM. arXiv:2405.15891.

  Key Terminology

• SDS (Score Distillation Sampling): Technique for distilling 2D diffusion model knowledge into 3D optimisation via score gradient backpropagation; introduced in DreamFusion (2022).

• VSD (Variational Score Distillation): Improvement over SDS that treats 3D parameters as samples from a variational posterior, reducing over-saturation; from ProlificDreamer (2023).

• Janus Problem: The artifact in SDS-based text-to-3D where the generated object has multiple faces (e.g., a face on both front and back of a generated head) because the 2D prior has no 3D consistency.

• PBR (Physically Based Rendering): Material representation using albedo, roughness, metallicity, and normal maps that enables correct appearance under arbitrary lighting; the target output format for production 3D generation.

• UV Mapping: The process of projecting a 3D mesh’s surface onto a 2D texture atlas; required for PBR materials and production asset workflows.

• Triplane: A 3D feature representation using three perpendicular 2D feature planes; enables efficient hybrid 2D/3D generative networks.

• SDF (Signed Distance Function): Implicit surface representation where the value at each point encodes the signed distance to the nearest surface; negative inside, positive outside.

• Novel View Synthesis: The task of rendering a 3D scene from a new camera viewpoint given one or more reference views; the core evaluation task for NeRF-based methods.

• Objaverse: The dominant large-scale 3D asset dataset (800K+ CC-licenced objects from Sketchfab), released by Deitke et al. 2023; the training backbone for most modern 3D generation models.

• Gaussian Splat: A single 3D Gaussian primitive in a 3DGS scene representation; characterised by position, covariance (encoding shape and orientation), opacity, and spherical harmonic colour coefficients.

Provenance