Amazon S3 Tables now support all Apache Iceberg V3 data types
AWS announced that its S3 Tables service can store Iceberg V3 tables with full compatibility for the new binary deletion‑vector format and the additional column types. The upgrade path lets a retailer replace a compliance‑driven delete of 50,000 rows in a 2‑billion‑row dataset with a single, compact deletion‑vector file, eliminating the cascade of tiny positional delete files that previously forced costly compaction cycles. At the same time, the variant type enables a click‑stream table to hold heterogeneous JSON payloads without schema‑on‑write gymnastics, while nanosecond‑precision timestamps and geometry/geography columns remove the need to encode these values as strings or integers. Row‑lineage fields (_row_id, _last_updated_sequence_number) are added automatically, giving downstream pipelines a cheap way to pull only changed rows.
The move positions AWS as the most feature‑rich cloud provider for open‑source lakehouse formats. Iceberg has become the de‑facto standard for petabyte‑scale analytics, and competitors such as Databricks, Google Cloud, and Snowflake have all added varying degrees of Iceberg support. By integrating V3 capabilities across its stack—EMR Spark for writes, Glue Data Catalog for metadata, and Redshift for queries—AWS offers a seamless end‑to‑end path that many customers lack elsewhere. This reflects a broader industry push to bring true lakehouse functionality (row‑level deletes, time travel, semi‑structured and geospatial handling) into object‑storage‑backed data lakes, reducing reliance on proprietary formats.
While the technical gains are clear, adoption will hinge on ecosystem readiness. All engines that read the upgraded tables must understand V3 semantics; otherwise, legacy workloads could break, and the upgrade is irreversible because Iceberg does not support downgrading from V3 to V2. Organizations will need to validate Spark, Redshift, and any third‑party tools before flipping the format‑version flag. Watch for early‑stage performance data on deletion‑vector handling at massive scale and for any latency spikes in variant column pruning, which could influence whether the promised cost savings materialize.
Key Takeaways
S3 Tables now store Iceberg V3 tables with native support for variant, nanosecond timestamps, geometry, geography, and unknown types.
Deletion vectors collapse thousands of positional delete files into a single binary file, dramatically cutting compaction overhead.
Row‑lineage metadata enables incremental pipelines to query only rows changed since the last checkpoint.
The upgrade is one‑way and requires all downstream engines to be V3‑compatible, making thorough testing essential before migration.
About the Source
This analysis is based on reporting by AWS Blog. Here is a short excerpt for context:
Amazon S3 Tables now support all data types in the Apache Iceberg V3 spec, along with column default values, deletion vectors, and row lineage. Upgrade existing V2 tables or create new V3 tables with built-in compaction, maintenance, and engine compatibility.Read the original at AWS Blog