Narrative Goldmine

Data Format Standard

4 min read

Properties

Type
  • Class
Resource
  • urn:ngm:class:data-format-standard
Domain
  • standards
Status
  • stable
Maturity
  • draft
Quality
  • 0.7
Is a
Part of
Has part
Requires
Enables
Generated
  • by: process:vault-migrate/1.0 · at: 2026-09-22T12:41:39.024503178Z
Bridges to
Contrasts with
Depends on
Implements
Legacy term id
  • MV-9838
Links
Page resource
  • urn:visionflow:page:7c88be9331a1f32da49a25b91d5c65d95e42070a5b3154ec256c6e24ed6291be
Related to
Standardized by
Supports
Uses

Technical specifications defining how data is structured, encoded, and exchanged across metaverse platforms and 3D applications, including standards like glTF for efficient 3D asset transmission and Universal Scene Description (USD) for complex scene composition and collaboration.

Semantic Classification

Content

Technical Details

  • Key 3D Format Standards:
    • glTF: “JPEG of 3D” - fast, efficient, opinionated format by Khronos Group
    • USD: “Photoshop of 3D” - rich, extensible format for authoring and collaboration (Pixar)
    • MaterialX: Common language for shading and material definition
  • Standards Organizations:
    • Metaverse Standards Forum: 3D Asset Interoperability Domain Group
    • Alliance for OpenUSD (AOUSD): Pixar, Adobe, Apple, Autodesk, NVIDIA
    • Khronos Group: glTF development and promotion
  • 2024 Progress:
    • USD and glTF communities building synergies to reduce fragmentation
    • NVIDIA added glTF plugin for direct USD scene referencing
    • Exploring NERFs and Gaussian Splats for new 3D representations
  • Scope Expansion: Beyond geometry to behaviors, sound, physics, scene composition

Applications

  • 3D asset exchange between platforms
  • Metaverse content creation workflows
  • Game engine asset pipelines
  • Virtual production environments
  • Digital twin visualization

Current Landscape (2026)

  • The Apache Iceberg v3 table-format spec was ratified in mid-2025 and rolled out across the 1.8.0 (Feb 2025), 1.9.0 (Apr 2025) and 1.10.0 (Sep 2025) releases, adding binary deletion vectors, default column values, row-level lineage, nanosecond timestamps, and new VARIANT and geospatial types; the 1.10.1 patch shipped in December 2025 with full PyIceberg MERGE support.
  • VARIANT became the ratified open standard for semi-structured data in the Apache Parquet community in October 2025 (Parquet format 2.12.0 / Parquet-Java 1.16.0), with Databricks contributing over 9,600 lines to parquet-java and a shredding technique that reads roughly 8x faster than plain Variant and 30x faster than JSON strings; it is now shared across Parquet, Iceberg v3 and Delta Lake.
  • Apache Parquet-Java 1.16.0 (Sep 2025) enabled vectorised reads by default and dropped Hadoop 2, while the community began scoping a Parquet v3 format for 2026 (FSST string encoding, cleaner metadata, faster bloom filters) after finalising the long tail of v2 features.
  • Apache Arrow shipped 20.0.0, 21.0.0 and 22.0.0 across 2025 and formally extended its columnar spec with a canonical TimestampWithOffset type to remove timezone ambiguity, plus Variant and geospatial extension types so semi-structured data can travel Parquet-to-Iceberg-to-application without re-encoding.
  • Format convergence accelerated after Databricks acquired Tabular (the original Iceberg creators) in June 2024 for a reported 1 to 2 billion dollars: Delta Lake UniForm now publishes Delta tables with Iceberg-readable metadata, Databricks ships Iceberg v3 as first-class under Unity Catalog, and Delta Lake 4.0 (Sep 2025) plus 4.1.0 (Mar 2026) added a variant type, coordinated commits and catalog-managed tables.
  • Iceberg has become the de facto multi-engine standard: AWS defaults to it across Athena, Glue, EMR and native S3 Tables, and Snowflake Horizon reached general availability for Iceberg reads in February 2026, while Apache Polaris (incubating) matured its REST catalog through 1.3.0-incubating with generic-table GA for cataloguing external formats like Hudi and Delta.
  • Open frontiers as of 2026 include Iceberg Format V4 (native indexing, single-file/one-file commits, JDK 17 baseline), hardening the REST catalog spec (ETag optimistic concurrency, idempotency keys, HTTP 429 handling) for predictable multi-engine behaviour, and unease over whether Iceberg’s vendor-neutral governance survives with its creators now inside Databricks.

References

Provenance

Graph View