Skip to content
Insight

Before upgrading an Iceberg table to format v3

Snowflake's in-place upgrade to Iceberg format v3 cannot be reversed, and engine support for v3 is still uneven. Leave a table on v2 until it needs a v3 feature, and confirm every reader and writer before upgrading it.

Rosera Technologies10 minute read

On 16 October 2026 Snowflake made in-place upgrades from Apache Iceberg format version 2 to version 3 generally available. The statement is short, ALTER ICEBERG TABLE ... UPGRADE TO ICEBERG VERSION 3, and the table keeps its name, grants, policies and downstream references, although streams that reference METADATA$ROW_ID stop working and dependent dynamic tables run one full refresh. Before running it, check every other engine that reads or writes the same table. The upgrade cannot be reversed, and the Iceberg specification requires a reader to reject a table whose format version is newer than the reader supports.

A table on v2 does not have to move. Snowflake's system default is still v2, as is the default in the Iceberg Java library, so a new Snowflake-managed table is created as v3 only when ICEBERG_VERSION or ICEBERG_VERSION_DEFAULT is set to 3, and an existing table becomes v3 only when someone runs the upgrade. Other engines can raise the version as a side effect. Databricks upgrades a managed Iceberg table when deletion vectors, variant or geospatial columns are enabled on it, and an engine that writes to an externally managed table can raise its format version with a property change. We recommend upgrading a table when it needs something that only v3 provides, and agreeing in advance which team may change the format version of a shared table.

What version 3 adds

The v3 specification adds deletion vectors, which record deleted rows as bitmaps and replace the position delete files that v2 writers add. It adds row lineage, which gives each row an identifier and a last-updated sequence number that change tracking across engines can build on. Existing rows receive these fields in the first commit after the upgrade, and snapshots from before the upgrade have none. Columns can carry default values without data files being rewritten. There are also new types: variant, geometry, geography, nanosecond timestamps and unknown. Two other parts of the specification, table encryption keys and multi-argument transforms for partitioning and sorting, are not supported by Snowflake.

If a table needs variant, geometry, geography or nanosecond timestamp columns, column defaults written to Iceberg metadata, or row lineage, v3 is required, because Snowflake supports each of these only on v3 tables. A table that needs none of them can stay on v2, although v2 has Snowflake limits of its own. Streams on a Snowflake-managed v2 table cannot be replicated, and a replicated v2 table cannot have streams created on it or serve as the source of a dynamic table. For a Snowflake-managed table the upgrade removes all three limits, and on an externally managed table it adds support for standard streams, where v2 allows only insert-only streams.

Tables that Snowflake can upgrade

Snowflake supports the upgrade for Snowflake-managed tables and for externally managed tables with CATALOG_SOURCE = ICEBERG_REST, including tables in catalog-linked databases. Tables that use a native AWS Glue or object-storage catalog source are not eligible, and neither are Delta Direct tables, Parquet Direct tables or Snowflake-managed tables whose external volume does not allow writes. An externally managed table with an expression, sequence or identity default fails to upgrade. An externally managed table can also be upgraded by the engine that owns it. That engine must then commit a new snapshot with row lineage, for example through a DML statement, because a property change alone creates no snapshot and Snowflake refreshes of the table fail until one exists.

Active position delete files can also block the upgrade until ALTER ICEBERG TABLE ... OPTIMIZE (REMOVE_POSITION_DELETES) has been run as a separate statement. That step rewrites the affected partitions and does not preserve the table's clustering, so a recluster may be needed afterward.

The upgrade cannot be reversed

Snowflake states that the upgrade is permanent. Google and AWS document the same rule for their engines, Dremio states it in its v3 announcement, and the Iceberg Java library, as of release 1.12.0, rejects a change that lowers a table's format version. The one exception we found is Databricks, which documents restoring its own managed Iceberg tables to a version from before the upgrade. That route does not suit a table that Snowflake also reads, because Snowflake's next refresh fails once an external engine lowers the format version.

For a Snowflake-managed table, the documented recovery is a Time Travel clone taken at a timestamp before the upgrade. The clone is a new v2 table. Changes made after that timestamp have to be applied to it by hand, streams have to be recreated, and COPY GRANTS does not carry ownership across. Externally managed tables cannot be cloned, so Snowflake documents no recovery path for them. Rolling the external catalog back to a snapshot from before the upgrade does not help, because Snowflake refreshes of the table then fail until a new v3 snapshot exists.

Reader and writer support for v3

Engine support for v3 was uneven when we checked the documentation on 24 October 2026. Each row below is taken from the vendor's or the project's own pages.

Enginev3 status on 24 October 2026
Amazon AthenaCreates and works with v2 tables only, and cannot read v3 tables.
Trino 484Support for v3 is marked experimental. Geometry and geography are not supported.
PyIceberg 0.12.0Reads v3 tables. Cannot yet create v3 tables, write to them or upgrade v2 tables, so a PyIceberg job that writes to a table fails once the table is upgraded.
Google BigQuerySupport for v3 in external tables is in Preview, and a query can process at most 100,000 deletion vector entries across a table, with each data file that has a deletion vector counting as one entry. In the Lakehouse runtime catalog, BigQuery can read v3 tables but cannot write to them, and the new v3 types, default values and row lineage are not supported there.
Amazon RedshiftSupports v3 only on Serverless (except the 4 RPU size) and on provisioned clusters with RG instance types. It cannot use struct, list, map, binary, uuid or time columns in v3 tables, or the new v3 types (variant, geometry, geography, nanosecond timestamps and unknown). It cannot use v3 tables that contain equality deletes and cannot create materialized views on any v3 table. After an upgrade, timestamptz columns map to TIMESTAMPTZ instead of TIMESTAMP.
Apache SparkReads and writes v3 through the Iceberg runtime. The variant type needs Spark 4.0 or later, and Iceberg's Spark documentation lists nanosecond timestamps, geometry and geography as not supported, although Iceberg 1.12.0 added geometry and geography for Spark 4.1.
DatabricksSupports managed v3 tables on Databricks Runtime 18 LTS and above, without initial defaults, nanosecond timestamps, the unknown type or multi-argument transforms. The pages we read do not say whether Databricks reads v3 tables that another catalog manages, so test that path directly.
AWS Glue and Amazon EMRSupport v3 from Glue 5.1 and EMR 7.12. The variant type needs Glue 6.0 or EMR 8.0.
DuckDBReads and writes v3 features, including the variant type and deletion vectors. Geography is not supported.
Microsoft Fabric (OneLake table shortcuts)Supports v3 source tables only in part. Deletion vectors are converted, and equality deletes make the conversion fail. Columns with v3 types or an initial default are dropped while the conversion still succeeds, and the dropped columns are listed in the conversion log. Microsoft also notes that some Fabric engines cannot read the table at all when its data files contain v3 types, even after those columns are dropped.

Athena's "engine version 3" is the name of an Athena release and is unrelated to the Iceberg format version. The two terms are easy to confuse in Athena's documentation.

Apache Iceberg 1.12.0, released on 29 September 2026, removed support for Spark 3.4 and Flink 2.0, so a team on either version must upgrade Spark or Flink before it can move to Iceberg 1.12.0. Because Iceberg's v3 support arrived in stages from release 1.8.0 to 1.12.0, confirm that the Iceberg release each job runs supports the v3 features its tables use. Iceberg's Flink documentation also says that jobs using the dynamic sink should not be running while a table is upgraded from v2 to v3.

Under Snowflake's default setting, a Snowflake-managed table moves from copy-on-write on v2 to merge-on-read with deletion vectors on v3. If an engine can open v3 tables but cannot apply deletion vectors, Snowflake recommends setting ICEBERG_MERGE_ON_READ_BEHAVIOR to 'DISABLED', which makes all Snowflake writes use copy-on-write. The parameter governs only Snowflake's own writes, and Trino, DuckDB and Redshift write deletion vectors on v3 tables, so the setting is enough only while Snowflake is the sole engine that updates or deletes rows in the table.

What changes inside Snowflake

Each table is upgraded in a single atomic transaction. While it runs, UPDATE, DELETE and MERGE are blocked, INSERT and COPY commits (Snowpipe included) are retried automatically, and SELECT continues to work. A DML statement that starts after the upgrade has begun waits up to the LOCK_TIMEOUT limit and then times out, so a job running with a short limit can fail. Snowflake recommends a larger warehouse for large tables, because upgrading a very large table on a small warehouse can take minutes.

After the upgrade, streams that reference METADATA$ROW_ID stop working and must be recreated. Dynamic tables that read the upgraded table run one full refresh before returning to incremental refreshes, so we recommend allowing warehouse capacity for that refresh when the source table is large. Snowflake also lists features that are not supported for Iceberg v3, among them dbt Projects on Snowflake, schema inference, the classic Snowpipe Streaming architecture and SnowGov Regions. For v3 tables, Snowflake requires version 4.0 or newer of the Snowflake Connector for Kafka and version 1.33.0 or newer of Snowpark. We recommend adding these items to the inventory described in the final section.

When a Snowflake-managed table has a default that only Snowflake can evaluate, such as CURRENT_DATE() or a sequence, the upgrade sets ALLOW_V3_EXPRESSION_DEFAULTS on the table automatically. While such a default exists, Snowflake does not allow SET WRITE DEFAULT on any column of that table, even a column with a constant default. External engines do not evaluate Snowflake-only defaults, so their writes must supply a value for a NOT NULL column and either a value or NULL for a nullable column, even where Snowflake would have filled the column from a sequence. The upgrade also writes literal defaults such as DEFAULT 1 to Iceberg metadata as initial and write defaults, and Microsoft Fabric drops any column that has an initial default when it reads the table through a OneLake shortcut.

An order of work

Start with an inventory of every engine and library version that touches each candidate table. Include notebooks, PyIceberg jobs, OneLake shortcuts in Microsoft Fabric and the queries other teams run from Athena or BigQuery, and check each of them against the table above.

For a Snowflake-managed table, upgrade a clone and point each engine at the clone, which is the test Snowflake itself recommends. Snowflake cannot clone an externally managed table, so create the test copy with the engine that owns the table and upgrade that copy. Under Snowflake's default setting, an upgraded Snowflake-managed table gains deletion vectors only as rows are later updated or deleted, and only for data files larger than about 1.6 MB in which fewer than about 5% of rows are deleted. We recommend running a few small UPDATE, DELETE or MERGE statements on the upgraded copy before the engines read it, because a read test on a copy with no deletion vectors does not show whether an engine can apply them.

List the streams and dynamic tables that depend on the table. Streams that reference METADATA$ROW_ID stop working, and a recreated stream records changes only from the time it is created, so pause every write to the table, Snowpipe loads included, have the consumers of those streams process their pending changes before the upgrade, and recreate the streams before the writes resume. Other streams continue to work, and dependent dynamic tables need no step beyond the one full refresh described in the previous section.

Choose a window with no UPDATE, DELETE or MERGE jobs and no external writers committing, stop any Flink jobs that use the dynamic sink, and record the exact upgrade time. If the table is in a replication or failover group, upgrade it in the source account and avoid a failover until the next replication refresh has delivered the v3 table. For a Snowflake-managed table, set DATA_RETENTION_TIME_IN_DAYS to cover the whole validation period, so that a Time Travel clone from before the upgrade remains available. A retention period longer than one day requires Enterprise Edition or higher.

The same problem on your own platform

Send us your version of it, with whatever detail you are able to share. We will tell you how we would approach it and what we would need to look at first.