fix(aws): a Lake Formation tag the contract does not define is associated, and the module validates

.github/workflows/iac-tests.yml

@@ -137,6 +137,17 @@ jobs:
             -q -p no:randomly --junitxml=state-drift.xml
           .venv/bin/python scripts/ci/assert_lane_coverage.py state-drift.xml \
             --require "state drift=tests/iac/test_iac_state_drift_moto.py"
+      # Lake Formation tags: a tag the contract defines is named by reference, one
+      # the platform owns by its literal key. Only a real apply and plan show that
+      # an association applied with depends_on plans no change under the
+      # reference, and that a platform-owned tag is associated and survives the
+      # destroy. Fails if the file only skipped.
+      - name: Stage 1 — Lake Formation tags apply + plan (tofu vs moto, creds-free)
+        run: |
+          .venv/bin/python -m pytest tests/iac/test_iac_lakeformation_tags_moto.py \
+            -q -p no:randomly --junitxml=lf-tags-moto.xml
+          .venv/bin/python scripts/ci/assert_lane_coverage.py lf-tags-moto.xml \
+            --require "LF tags moto=tests/iac/test_iac_lakeformation_tags_moto.py"
       # The default state key names the provider, and the first apply after the
       # upgrade moves the old key's state with `tofu init -migrate-state`. Only a
       # real apply, move and plan prove the plan after it changes nothing.

CHANGELOG.md

@@ -18,6 +18,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
+- **A Lake Formation tag the contract does not define is associated, and the module
+  validates.** A key in a binding's `governance.lakeFormation.tags` with no
+  `governance.lakeFormation.tagDefinitions` entry got a `depends_on` on an
+  `aws_lakeformation_lf_tag` the module never declared, so `tofu validate` and
+  `fluid apply` failed with "Reference to undeclared resource". Such a key is now a
+  tag the platform owns: associated as written, with no `depends_on`, no warning and no
+  refusal. It must already exist in the account, and the identity running `fluid apply`
+  needs `ASSOCIATE` on it. A key the contract defines is named by reference,
+  `key = aws_lakeformation_lf_tag.<name>.key`, the provider's documented pattern,
+  instead of a `depends_on` edge, so the association still waits for the tag. The
+  reference evaluates to the same key, so an association applied from 0.19.0 plans no
+  change (applied and planned against moto).
+  Refused at `fluid validate` and at emit, where each used to fail at apply or be
+  dropped: a value the contract's definition of the tag does not allow, compared
+  case-insensitively (`lakeformation-tag-value`); a definition with no values, or two
+  that would be one resource name, such as `a-b` and `a_b`
+  (`lakeformation-tag-definition`); and tags on a binding that names no Glue table
+  (`lakeformation-tag-association`). The `tags` and `tagDefinitions` descriptions in
+  fluid-schema 0.7.3 to 0.7.6 say so.
 - **A row filter on an AWS binding that is not a Glue-catalog table is refused, not
   dropped.** On AWS a row filter is a Lake Formation data cells filter on a Glue table,
   and the emitter writes no Lake Formation resource for any other format. So

HONESTLY_TESTED.md

@@ -45,6 +45,7 @@ Markers declared in `pyproject.toml`.
 | Stage 3 cross-account proxy | `tests/iac/test_iac_aws_real_cross_account_e2e.py` | ✅ 2/2 before `bucketPolicy` existed — consumer role assumes + LF + S3 bucket policy + Athena SELECT; un-granted principal correctly denied. The consumer is in the deployer's account, so under the new default (`bucketPolicy: cross-account`) it gets **no** bucket-policy statement and the test now asserts that; that version has **not yet been re-run live**. Uses the existing `binding.governance.lakeFormation.grants[]` block. |
 | Stage 1 LF bucket-policy modes | `tests/iac/test_iac_lakeformation_bucket_policy.py`, `tests/iac/test_iac_lakeformation_bucket_policy_plan.py`, `tests/iac/test_iac_tofu_validate.py::test_lakeformation_bucket_policy_modes_pass_tofu_validate` | ✅ — the three `bucketPolicy` modes rendered (all-grantees byte-identical to the previous emit), `tofu validate` of each, and a real `tofu plan` against moto's STS proving the default keeps a cross-account grantee's statements and plans no bucket policy for a same-account one. |
 | Stage 1 LF column grants | `tests/iac/test_iac_lakeformation_column_grants.py`, `tests/iac/test_iac_lakeformation_column_permissions.py`, `tests/iac/test_iac_lakeformation_column_grants_plan.py` | ✅ — the three grant shapes rendered (no columns: `table`; `columns`: `table_with_columns.column_names`, no wildcard; `excludedColumns`: `wildcard = true` + `excluded_column_names`; refused at emit: both together, a column name the expose's schema does not declare, an exclusion of every column, and a column limit on a binding with no `location.table`), and a real `tofu plan` against moto of all three. Plan, not validate: the excluded shape without the wildcard passes `tofu validate` and fails `tofu plan` with "Missing required argument", and the file pins that too. A column-limited grant is emitted and planned with `SELECT` only: Lake Formation takes nothing else on `table_with_columns` ("Permissions modification is invalid" on the demo lab's apply, 28 Sept 2026), a `DESCRIBE` beside it is dropped because the column-limited `SELECT` already lets the principal see the table and Lake Formation refuses `DESCRIBE` to a principal holding a partial `SELECT`, and `ALTER`, `DROP`, `DELETE`, `INSERT` or `ALL` beside a column limit is refused at emit, as is a second grant for the same principal on the same table (Lake Formation refuses the table-level permissions to it, a table-level `SELECT` would read the withheld columns, and the provider reads the pair back as one). `DESCRIBE`'s grant option beside a column limit is refused (no resource can carry it). `SELECT`'s grant option is kept beside `columns` and refused beside `excludedColumns`: the guide's pages disagree (the permissions reference: "you can't include the grant option if column filtering is applied"; the console page offers it for simple column-based access), and *Data filtering limitations* is the specific rule followed: "To grant SELECT with the grant option and column filtering, you must use an include list, not an exclude list." **Not proven here**: moto stores any permission on any resource and `tofu plan` checks none, so the permission rules rest on the AWS LF developer guide and that failed apply; the fixed grant has **not yet been applied live**, and `SELECT` with the grant option on an include list has **not been tried live** (a live `GrantPermissions` of it settles the conflict). |
+| Stage 1 LF tags | `tests/iac/test_iac_lakeformation_tags.py`, `tests/iac/test_iac_lakeformation_tags_moto.py` | ✅ — a tag the contract defines is named in its association by reference (`key = aws_lakeformation_lf_tag.<name>.key`, the provider's documented pattern) with no `depends_on`, and a tag the platform owns by its literal key; the refusals at emit and at `fluid validate`; `tofu validate` of a platform-owned association. A real `tofu apply` / `plan` / `destroy` against moto: an association applied as 0.19.0 emitted it (literal keys and `depends_on`) plans no change under the references, and the state then records the tags as its dependencies; a tag the platform created beforehand is associated beside one the contract creates, reads back through `GetResourceLFTags`, plans clean and survives the destroy; a platform-owned key that does not exist fails the apply with `EntityNotFoundException`. **Not proven here**: moto checks that an associated key exists but not its value, so a wrong value on a platform-owned tag, which Lake Formation refuses, is not proven to fail; moto enforces no `ASSOCIATE` permission; and the platform-owned path has **not been applied live**. |
 | Stage 1 retention + encryption at rest | `tests/iac/test_iac_aws_retention_encryption.py`, `tests/iac/test_iac_aws_retention_encryption_moto.py`, `tests/iac/test_iac_tofu_validate.py::test_retention_and_kms_pass_tofu_validate`, `tests/cli/test_verify_athena.py` (storage section) | ✅ — `exposes[].lifecycle {retention, expire: true}` and `binding.encryption.kms` (fluid-schema 0.7.6) rendered (prefix-scoped rules, never a whole-bucket rule; key, alias, default SSE-KMS with a bucket key; key policy), `tofu validate` of each shape, and a real `tofu plan` / `apply` / `destroy` against moto: the key policy evaluated against the applying account, an object written with no encryption header landing SSE-KMS under the product key, `fluid verify`'s storage checks passing and then failing on a changed rule and an SSE-S3 object, and dropping the fields deleting the lifecycle configuration and scheduling the key's deletion 7 days out. An existing key is planned against moto too: named by alias, S3 is handed its key ARN; a key pending deletion and an RSA key fail the plan on the SSE configuration's preconditions; the AWS managed key named by its key ARN (which passes the name check) fails it with registerLocation (moto records every key as customer managed, so the test sets its record of alias/aws/s3 to `AWS`). A brownfield bucket carrying an operator's lifecycle rule and SSE-S3 is adopted through `fluid apply`'s own `_adopt_existing`, and the plan shows both configurations as in-place updates. `fluid verify` is covered with botocore stubs, including a key that is not `Enabled` and lifecycle rules filtered by tag, size or a narrower prefix that expire objects sooner. **Not proven here**: moto enforces neither key policies nor Lake Formation, so Athena reading an SSE-KMS location through Lake Formation-vended credentials (the service-linked role's key-policy statement) rests on the AWS documentation and has **not been run live**. |
 | Stage 1 GCP governance parity (retention, CMEK, column restrictions, mapped principals) | `tests/iac/test_iac_gcp_governance.py`, `tests/iac/test_iac_gcp_governance_plan.py`, `tests/iac/test_iac_aws_column_restrictions.py`, `tests/cli/test_verify_bigquery_governance.py` | ✅ — rendered and `tofu validate`d: partition expiration (never a table TTL), a Cloud KMS key ring, key, service-agent grant and the dataset and table keys, a Data Catalog taxonomy, policy tags and fine-grained readers, and non-authoritative dataset members with the contract's logical principals mapped by `binding.principals` (placeholders and unmapped principals refused). A real `tofu plan` / `apply` with terraform-provider-google against an in-process stand-in for the BigQuery REST API (`tests/iac/_fake_bigquery.py`; the goccy emulator crashes the provider on apply): adding retention or a key to a live table plans its replacement, which the data-loss gate refuses without `--allow-data-loss`; a new retention is in place; the governed table plans clean; the dataset keeps its own access entries. The same stand-in shows a revoked reader is a member destroy the data-loss gate lets through, and that the first apply after the authoritative access list of 0.16.6 and earlier revokes a grant removed in the same change and then plans clean. AWS: the restriction becomes each Lake Formation grant's excluded columns, a real `tofu plan` against moto accepts them, and `fluid verify`'s Lake Formation check runs against moto's stored grants (and against a listed `TableWildcard` grant). `fluid verify` on BigQuery is covered with real `google.cloud.bigquery` Table/Dataset objects and a fake Data Catalog session. **Not proven here**: nothing ran against real BigQuery, Cloud KMS, Data Catalog or Lake Formation; no emulator enforces policy tags, keys or IAM, so a denied principal's refusal, the service agent's use of the key and a load into a tagged column rest on the providers' documentation. |
 | Stage 2 cross-account (two-account LocalStack) | `tests/integration/test_iac_cross_account_localstack.py` | ✅ 10 tests, gated on `FLUID_IAC_LIVE_XACCT=1` + a second LocalStack Pro carrying `lakeformation`. Producer stack applies in account A (`000000000000`); every grant names a role in account B (`222222222222`) — LocalStack derives the account from the access-key id. Verified live: LF permission + LF prefix-registration + `aws_s3_bucket_policy` all land and read back; the shared-**pool** bucket is referenced via `data.aws_s3_bucket`, its `GetObject` grant is scoped to `location.path` and its `ListBucket` carries the `s3:prefix` condition. Under `ENFORCE_IAM=1` the emitted bucket policy is proven to be the **deciding** access control: with a deliberately broad identity policy on the account-B role, the granted prefix reads and the sibling tenant's prefix is denied — and widening only the bucket policy flips that denial (causal control). **Still NOT proven here**: LF cross-account *authorization* (LocalStack rejects `GrantPermissions` under IAM enforcement even for a registered `DataLakeAdmin` — `docs/upstream-issues/localstack-lakeformation-grant-auth.md`), and cross-account Glue catalog sharing (no AWS RAM in LocalStack; account B gets `EntityNotFoundException`, which is state isolation, not a denial). |