fix(gcp): a BigQuery replace or rollback keeps the table's governance

AGENTS.md

@@ -51,6 +51,8 @@ Every contract is delivered through an 11-stage pipeline. Each stage is a hard g
 
 `--allow-data-loss` is required for `replace*` when `FLUID_ENV != dev` OR target has rows. `--no-verify-plan-binding` is an emergency DR waiver for plan-binding checks (logged at WARNING level so audit trails catch it).
 
+The `replace` rows describe the native engine. On the OpenTofu engine (aws, gcp, snowflake, confluent) `replace` emits the same module as `amend` and adds the data-loss gate; a table is replaced only where `tofu` plans it, and on GCP its row access policies and table grants are recreated with it (`replace_triggered_by`). A BigQuery table is never recreated to replace its rows: the planner's replace and `fluid rollback` run a `WRITE_TRUNCATE_DATA` query job into the existing table (a `CREATE OR REPLACE TABLE` drops its row access policies and policy tags), and the pre-replace backup is a `CREATE SNAPSHOT TABLE … CLONE`. dbt's `table` materialisation still recreates a governed table (known gap; `docs/governance-parity.md`). See `docs/apply.md`.
+
 ### Plan-Binding (Terraform-Style "Apply Consumes Exact Plan")
 
 Stage 6 `fluid plan` emits two cryptographic fields in `plan.json`:

CHANGELOG.md

@@ -48,6 +48,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   (`lakeformation-tag-definition`); and tags on a binding that names no Glue table
   (`lakeformation-tag-association`). The `tags` and `tagDefinitions` descriptions in
   fluid-schema 0.7.3 to 0.7.6 say so.
+- **A BigQuery replace or rollback keeps the table's governance.** Nothing forge-cli
+  runs recreates a governed BigQuery table any more. `CREATE OR REPLACE TABLE … AS
+  SELECT`, which BigQuery documents drops every row access policy, and whose SELECT
+  schema has no policy tag, left every filtered or restricted principal reading every
+  row and column until the next apply. dbt's `table` materialisation still does (a
+  known gap, documented in `docs/governance-parity.md`).
+  - `fluid rollback` restores with one query job, `SELECT * FROM <backup>` into the
+    live table with `WRITE_TRUNCATE_DATA` and `CREATE_NEVER` (`WRITE_TRUNCATE` on an
+    emulator). The destination is a table reference built from the validated
+    identifiers, and the recorded `ddl[]` is still never run. A backup the restoring
+    identity cannot read in full, because a row access policy shows it only some rows,
+    is refused (`rollback_bigquery_backup_filtered`) instead of restoring that subset.
+  - `GcpProvider.restore_ddl` records no restore SQL, as AWS does.
+  - The GCP planner's replace emits a SQL build as the SELECT with a structured
+    destination and `WRITE_TRUNCATE_DATA` / `CREATE_NEVER`, keeps the table's
+    `bq.ensure_table`, and backs the table up with a zero-copy
+    `CREATE SNAPSHOT TABLE … CLONE` that expires after 30 days, not a plain copy with
+    no policy and no tag. SQL that names its own sink passes through as before.
+  - When `tofu` replaces a BigQuery table (partitioning or a key added to a live
+    table), its row access policies and table IAM members are recreated with it in
+    the same apply (`lifecycle.replace_triggered_by` on the table's `id`). BigQuery
+    deletes them with the table, and `tofu` planned them as unchanged. An in-place
+    update of the table leaves them alone, so a label change still needs no
+    `--allow-data-loss`.
 - **A row filter on an AWS binding that is not a Glue-catalog table is refused, not
   dropped.** On AWS a row filter is a Lake Formation data cells filter on a Glue table,
   and the emitter writes no Lake Formation resource for any other format. So

docs/apply.md

@@ -21,7 +21,7 @@ fluid apply contract.fluid.yaml --mode amend --yes
 | `create-only` | `CREATE IF NOT EXISTS`, fails if the target exists | no | untouched | no |
 | `amend` *(default)* | additive — `ALTER … ADD COLUMN IF NOT EXISTS`; views `CREATE OR REPLACE` | no | preserved; new columns `NULL` | no |
 | `amend-and-build` | same as `amend` | yes | preserved; transforms re-run | no |
-| `replace` | drop + recreate the target | no | **dropped** | **yes** |
+| `replace` | drop + recreate the target (native engine; see [the OpenTofu engine](#apply-engines-and-what-that-changes)) | no | **dropped** | **yes** |
 | `replace-and-build` | same as `replace` | yes (`dbt --full-refresh`) | **dropped**, then rebuilt | **yes** |
 
 `--mode` defaults to `amend`. `--dry-run` is an ergonomic alias for
@@ -64,8 +64,8 @@ The engine is resolved per provider — there is no user-facing switch.
 | `aws`, `gcp`, `snowflake`, `confluent` | OpenTofu (`.tf.json` + `tofu init/plan/apply`) |
 | `local` | native in-process apply |
 
-Two mode behaviours differ by engine. Both are reported at run time, but
-know them before you plan a destructive change:
+Three mode behaviours differ by engine. Know them before you plan a
+destructive change:
 
 **1. No pre-replace snapshot on the OpenTofu engine.** The native path
 plans a pre-flight zero-copy snapshot (`<target>__backup_<ts>`) and records
@@ -91,6 +91,36 @@ matches the live table will therefore plan clean — including under
 
 `fluid verify --strict` is the gate — it exits non-zero on a type mismatch.
 
+**3. `replace` provisions what `amend` provisions.** The OpenTofu engine
+emits the same module in every mode, so `--mode replace` drops and
+recreates nothing by itself: it adds the data-loss gate and the drift
+report to a normal `tofu apply`. A table is replaced only when `tofu`
+plans it (a change the platform cannot make in place, such as partitioning
+an existing BigQuery table or giving it a key), which the gate refuses
+without `--allow-data-loss`. On GCP a replaced table comes back with its
+governance in the same apply: its policy tags are in the table's schema,
+and its row access policies and table-level grants, which BigQuery deletes
+with the table, are recreated with it (`replace_triggered_by`; see
+[governance parity](governance-parity.md#changes-a-live-table-cannot-take-in-place)).
+
+`replace-and-build` then runs the builds as a full refresh. dbt gets
+`--full-refresh`. An embedded-SQL build into BigQuery lands with one
+`WRITE_TRUNCATE_DATA` load into the existing table, which replaces the
+rows and keeps the table, its policy tags and its row access policies.
+dbt does not: dbt-bigquery materialises a `table` model (and, under
+`--full-refresh`, an `incremental` one) with `create or replace table`,
+which drops the row access policies and, unless the dbt project re-applies
+them, the policy tags `tofu` made. That is a known gap; see
+[governance parity](governance-parity.md).
+
+`fluid rollback` restores a BigQuery table the same way it keeps it: one
+query job, `SELECT * FROM <backup>` into the live table with
+`WRITE_TRUNCATE_DATA` and `CREATE_NEVER`, never `CREATE OR REPLACE TABLE`.
+The restoring identity must read the whole backup: a row access policy
+that shows it only some rows makes the rollback refuse
+(`rollback_bigquery_backup_filtered`), and a policy-tagged column needs
+Fine-Grained Reader on its tag.
+
 **Drift against the apply's own state.** `fluid diff` (and `fluid verify
 --state-drift`) run the refresh `fluid apply` runs, without applying: from
 the same workdir and backend (`--workspace-dir`, `--state-backend`,

docs/governance-parity.md

@@ -188,12 +188,31 @@ policy:
   filter, with the principal's excluded columns as its column wildcard's exclusions,
   instead of on the table. The masked view of a filtered principal applies the same
   predicate, since the view reads the table as its definer.
-* **A build keeps them.** BigQuery removes every row access policy of a table that a
-  `WRITE_TRUNCATE` load writes, so a full-refresh build loads with
+* **An embedded-SQL build keeps them.** BigQuery removes every row access policy of a
+  table that a `WRITE_TRUNCATE` load writes, so a full-refresh build loads with
   `WRITE_TRUNCATE_DATA`, which replaces the rows and keeps the table's policies, tags
   and partitioning (measured on BigQuery, 5 October 2026, on an unpartitioned and on a
   DAY-partitioned table with a partition expiration). It is the disposition dlt's
   BigQuery destination uses for the same reason.
+* **A replace and a rollback keep them.** Nothing forge-cli runs recreates a governed
+  BigQuery table. `fluid rollback` restores one with a single query job, `SELECT * FROM
+  <backup>` into the live table with `WRITE_TRUNCATE_DATA` and `CREATE_NEVER`; it was
+  `CREATE OR REPLACE TABLE … AS SELECT`, which BigQuery documents drops every row
+  access policy of the table, and whose SELECT schema carried no policy tag. It refuses
+  a backup the restoring identity cannot read in full, which would otherwise restore
+  only the rows a policy shows it. The GCP planner's replace plans the same query job
+  for a SQL build, keeps the table definition planned, and backs the table up with a
+  zero-copy `CREATE SNAPSHOT TABLE … CLONE`, not the plain copy it made, which had no
+  policy and no tag.
+* **Known gap: dbt still recreates the table.** dbt-bigquery materialises a `table`
+  model with `create or replace table`, in every build mode, and an `incremental` model
+  the same way under `--mode replace-and-build`, where forge-cli passes `--full-refresh`.
+  That runs after `tofu apply` made the row access policies and policy tags: the row
+  access policies are dropped, and the policy tags too unless the dbt project re-applies
+  them (`persist_docs` with `policy_tags`, which dbt does after the replace). Until the
+  next `fluid apply`, every principal a filter or a restriction limits reads every row
+  and column of a table a dbt build recreated. forge-cli does not change dbt's
+  materialisation yet; that is a follow-up.
 
 ## Dataset grants are no longer authoritative
 
@@ -285,6 +304,16 @@ partitioning's shape. Changing `retention` later is an in-place update of
 also refuses without `--allow-data-loss`; BigQuery cannot un-partition a table, so to
 keep data longer, set a longer `retention` instead.
 
+A replaced table comes back governed. BigQuery deletes a table's row access policies
+and its IAM policy with the table, and `tofu` did not see it in the same apply: a row
+access policy's `table_id` is the same string before and after, so it planned the
+policies as unchanged and the new table stood with none (and, in a shared dataset,
+without its table-level grants) until the next apply. Each row access policy and
+table IAM member now has `lifecycle.replace_triggered_by` on the table's `id`, which
+is unknown in the plan only when a new table is created, so they are recreated with
+the table and left alone by an in-place update. Their replacement adds removals the
+data-loss gate counts, but only in a plan that already replaces the table.
+
 ## Lake Formation tags the platform owns
 
 AWS recommends one LF-tag ontology per account: a data steward or the platform
@@ -337,7 +366,10 @@ to tag its own tables. A contract follows either model, per key:
   in-process stand-in for the BigQuery REST API, shows adding retention or a key to a
   live table plans its replacement (the data-loss gate refuses it), a new retention is
   in place, the governed table then plans clean, and the dataset's own access entries
-  survive the grants (`tests/iac/test_iac_gcp_governance_plan.py`).
+  survive the grants (`tests/iac/test_iac_gcp_governance_plan.py`). The stand-in
+  deletes a table's row access policies and IAM policy with it, as BigQuery does: a
+  replaced table gets both back in the same apply, and an in-place update (a label, a
+  new retention) leaves them.
 * A real `tofu plan` against moto accepts the Lake Formation grants with excluded
   columns, and `fluid verify`'s Lake Formation check runs against moto's stored grants
   (`tests/iac/test_iac_aws_column_restrictions.py`).
@@ -389,6 +421,24 @@ to tag its own tables. A contract follows either model, per key:
 * `tofu plan` with terraform-provider-aws 6.67 against the live state of the lab's
   eleven AWS products planned no change for any of them, with only the provider pin
   moved from `~> 5.0`.
+* **The replace and the rollback, measured on BigQuery**, 6 October 2026, on a
+  temporary DAY-partitioned table with a row access policy and a policy-tagged column,
+  driven through forge-cli's own planner action and `fluid rollback` code:
+  * the pre-replace `CREATE SNAPSHOT TABLE … CLONE` copied both row access policies and
+    the policy tag to the snapshot (not the partition expiration, as BigQuery
+    documents);
+  * the replace's `WRITE_TRUNCATE_DATA` query job, and the rollback's restore from the
+    snapshot, each kept the row access policy, the policy tag, the partitioning and
+    the partition expiration; afterwards a reader with only the filtered policy saw 2
+    of 3 rows and was refused the tagged column;
+  * the query job maps the SELECT's columns by name, not by position, and a missing
+    NULLABLE column lands as NULL;
+  * an identity without full row access or Fine-Grained Reader is refused both the
+    replace job and the restore (403): it fails closed and changes nothing;
+  * the control, the old `CREATE OR REPLACE TABLE … AS SELECT` on an unpartitioned
+    copy, dropped the row access policy and the policy tag, and every row and the
+    tagged column became readable. On a partitioned table BigQuery refuses it outright
+    ("Cannot replace a table with a different partitioning spec").
 * **Not proven yet**: an apply of the masking, row filter and label resources exactly
   as forge-cli emits them, on either cloud.
 * **Not proven**: associating a tag the platform owns on a real account, and how Lake

fluid_build/cli/_rollback_writer.py

@@ -89,10 +89,11 @@ def _restore_ddl_for_snapshot(snapshot_meta: Mapping[str, Any], provider: str) -
     """Build the DDL ``cli/rollback.py`` will run to restore the snapshot.
 
     Delegates to ``provider.restore_ddl(snapshot)`` — every in-tree
-    provider owns its restore semantics (Snowflake CLONE, BigQuery
-    CTAS, AWS S3-prefix copy, Redshift transactional CTAS). Adding a
-    new provider is a single subclass override, not a central
-    if/elif edit here.
+    provider owns its restore semantics (Snowflake CLONE, AWS S3-prefix
+    copy, Redshift transactional CTAS; BigQuery records none, because
+    ``fluid rollback`` restores it with a data-only query job that keeps
+    the table and its governance). Adding a new provider is a single
+    subclass override, not a central if/elif edit here.
 
     Returns ``[]`` when no DDL applies — the rollback CLI surfaces
     that as a typed ``rollback_provider_unsupported`` error.