From 5804c9cd69aab545eff0443cbe0ac82270157c35 Mon Sep 17 00:00:00 2001 From: Speculator55005 <50082482+fas89@users.noreply.github.com> Date: Mon, 12 Oct 2026 22:30:28 +0200 Subject: [PATCH 1/2] docs(apply): every engine validates the contract before planning; MWAA refusal on AWS (forge-cli #714, #715) forge-cli #714: fluid apply schema-validates the contract on both engines and in every mode, dry-run and the build modes included, and refuses a plan.json that embeds no contract (apply_plan_missing_contract). apply.md gains "Contract validation" (captured apply_contract_invalid and apply_plan_missing_contract output, the event table, what is checked for a contract and for a plan.json, where the check sits, the checked copy is the applied copy), with pointers from "Which engine runs", "Plan binding" and the errors table. plan.md's contract row, the error-code and typed-error pages, production troubleshooting, and the GCP metadata.policies and Snowflake security: notes, which fluid apply now refuses, say the same. forge-cli #715: an aws contract with orchestration.engine airflow and an orchestration.schedule fails fluid apply and fluid generate iac with unsupported_binding, kind mwaa-environment-not-emitted, instead of reporting success with no MWAA environment. aws.md gains "Scheduling on AWS" (captured refusal, the named-environment remediation, what each orchestration shape emits, edge cases, the EventBridge path and its caveats, and the planned operations now named in a WARNING line); generate-iac.md, airflow.md ("MWAA on AWS", with a captured schedule-sync dry run), schedule-sync.md (MWAA is not a Path-B scheduler) and production troubleshooting are updated. upgrading.md gains "Unreleased changes after 0.18.1" with both changes, and aws.md's upgrade list a bullet for each. --- docs/advanced/airflow.md | 30 +++++++ docs/advanced/error-codes.md | 2 + docs/advanced/production-troubleshooting.md | 5 ++ docs/advanced/typed-cli-errors.md | 2 +- docs/cli/apply.md | 51 ++++++++++++ docs/cli/generate-iac.md | 13 ++- docs/cli/plan.md | 2 +- docs/cli/schedule-sync.md | 2 +- docs/providers/aws.md | 88 +++++++++++++++++++++ docs/providers/gcp.md | 7 ++ docs/providers/snowflake.md | 2 +- docs/upgrading.md | 25 ++++++ 12 files changed, 224 insertions(+), 5 deletions(-) diff --git a/docs/advanced/airflow.md b/docs/advanced/airflow.md index 354dff0..b8fe65b 100644 --- a/docs/advanced/airflow.md +++ b/docs/advanced/airflow.md @@ -92,6 +92,36 @@ fluid schedule-sync --scheduler airflow --dags-dir dist/artifacts/schedule --des `--dags-dir` must contain one directory per product. With the default `--delete-scope product`, a loose file in `--dags-dir` is refused with `schedule_sync_dags_dir_not_product_scoped` and exit code 2, so one product's sync cannot delete another's DAGs. If you generated with `fluid generate schedule`, write into `-o //` as in the quick start. `schedule-sync` takes no contract argument: it needs `--scheduler` and `--dags-dir`, plus `--destination` for `airflow` and `mwaa`. +## MWAA on AWS + +*([forge-cli #715](https://github.com/Agenticstiger/forge-cli/pull/715), unreleased)* `fluid apply` does not create an Amazon MWAA environment. Create the environment yourself, in your own VPC, or use one you already run, and push the DAGs to the S3 folder it reads. Schedule the builds with `builds[].execution.trigger`, and leave `orchestration.schedule` out: + +```yaml +orchestration: + engine: airflow +builds: + - id: load_orders + execution: + trigger: + type: schedule + cron: "0 6 * * *" +``` + +```bash +fluid generate schedule contract.fluid.yaml -o dags/sales_orders_daily +fluid schedule-sync --scheduler mwaa --dags-dir dags --destination s3://acme-airflow-dags/dags/ --dry-run +``` + +```text +[schedule-sync] scheduler=mwaa dags-dir=.../dags env=dev delete-scope=product dry-run=True +[schedule-sync] → /usr/local/bin/aws s3 sync .../dags/sales_orders_daily/ s3://acme-airflow-dags/dags/sales_orders_daily/ --delete +[schedule-sync] ✔ mwaa sync complete (1 subprocess(es)) +``` + +Drop `--dry-run` to push. `--destination` is the S3 folder the environment reads its DAGs from, in its source bucket. Each scheduled run executes `fluid apply` on the MWAA worker, which needs what [What the DAG does](#what-the-dag-does) lists. + +An `aws` contract that keeps `orchestration.schedule` with `orchestration.engine: airflow` asks `fluid apply` for the environment itself, and is refused with `unsupported_binding`, kind `mwaa-environment-not-emitted`, before anything is applied; see [Scheduling on AWS](../providers/aws.md#scheduling-on-aws). Before this change the apply reported success and no environment existed. The refusal's own next step names `fluid generate schedule --scheduler airflow --output dags`, which writes the DAG straight into `dags/`; `fluid schedule-sync` refuses that layout with `schedule_sync_dags_dir_not_product_scoped` under its default `--delete-scope product`, so write into `dags//` as above. + ## `fluid generate-airflow` (deprecated) `fluid generate-airflow` still runs, and prints a warning that points at its replacement: diff --git a/docs/advanced/error-codes.md b/docs/advanced/error-codes.md index 9564cbf..19dc5f7 100644 --- a/docs/advanced/error-codes.md +++ b/docs/advanced/error-codes.md @@ -226,6 +226,8 @@ As of 0.18.1 the second suggestion names `azure`, but `fluid providers` lists `a The state refusals (`state_shared_with_another_provider`, `state_migration_ambiguous` and the other `state_*` events) are described on [OpenTofu state](../concepts/state.md). The refusal for an env with no overlay (`overlay_declared_but_missing`) and the `bundle_env_mismatch` refusal are on [Environments and overlays](../concepts/environments-and-overlays.md); `plan_env_mismatch` is in [Production troubleshooting](./production-troubleshooting.md). +*(unreleased, [forge-cli #714](https://github.com/Agenticstiger/forge-cli/pull/714) and [forge-cli #715](https://github.com/Agenticstiger/forge-cli/pull/715))* `apply_contract_invalid`, `apply_plan_missing_contract` and `unsupported_binding` (kind `mwaa-environment-not-emitted`) are not catalogued, so they print no curated suggestions. [Production troubleshooting](./production-troubleshooting.md#environment-mode-and-plan-file-errors) explains the first two, and [Scheduling on AWS](../providers/aws.md#scheduling-on-aws) the third. `contract_version_unsupported`, catalogued [above](#contract-version-unsupported), is now raised by `fluid apply` on both engines and in every mode. + ### planner_failed `ERR_PLANNER_FAILED`. The documentation link lands on [Production troubleshooting](./production-troubleshooting.md). diff --git a/docs/advanced/production-troubleshooting.md b/docs/advanced/production-troubleshooting.md index 7f9a3a0..8cd4d19 100644 --- a/docs/advanced/production-troubleshooting.md +++ b/docs/advanced/production-troubleshooting.md @@ -65,6 +65,9 @@ When the bundle is unrecoverable, `fluid apply --no-verify-plan-binding` skips t | `apply_build_id_requires_build_mode` | `--build-id` selects which build runs, and the mode (for example `dry-run`) runs none | Use `--mode amend-and-build` or `replace-and-build`, or drop `--build-id` | | `apply_plan_unreadable` | The plan file is missing, is not JSON, or is truncated | Regenerate it with `fluid plan --out` | | `apply_plan_mode_mismatch` | The plan was made for a different `--mode` than apply was given | Re-run `fluid plan --mode ` with the mode you will apply | +| `apply_contract_invalid` | *(unreleased, [forge-cli #714](https://github.com/Agenticstiger/forge-cli/pull/714))* The contract, or the contract a `plan.json` embeds, is not valid against its schema; the `errors:` line lists why. `fluid apply` checks it on both engines and in every mode, so a cloud or build apply that used to exit 0 on such a contract now stops here | Run `fluid validate --env `, fix what it reports, and apply again. No flag skips the check. See [Contract validation](../cli/apply.md#contract-validation) | +| `contract_version_unsupported` | The contract's `fluidVersion` is below 0.7. *(unreleased, [forge-cli #714](https://github.com/Agenticstiger/forge-cli/pull/714))* `fluid apply` raises it on both engines and in every mode | Migrate the contract to a 0.7.x schema | +| `apply_plan_missing_contract` | *(unreleased, [forge-cli #714](https://github.com/Agenticstiger/forge-cli/pull/714))* The `plan.json` embeds no contract, so there is nothing to validate. Refused with or without `--no-verify-plan-binding`; such a plan used to run its actions unchecked on the native engine | Re-run `fluid plan --out .json` and apply the new plan | | `contract_tests_baseline_missing`, `contract_tests_bad_baseline` | `fluid contract-tests --baseline` names a file that does not exist, or whose content is not a baseline | Create one with `fluid contract-tests --write-baseline baseline.schema.json` | ### The federation check @@ -203,6 +206,8 @@ The organization comes from `organization_id`, else `FLUID_CC_ORG_ID`; with neit | `dbt_tests_refusing_overwrite` | `fluid generate dbt-tests --out` names an existing file that lacks fluid's managed-by header, so it is not overwritten | Delete the file, or pass a different `--out` | | `refusing_to_overwrite_existing_file_pass_force_to_override` (exit 2) | `fluid scaffold-ide` would overwrite a file that exists | Pass `--force`; see [`fluid scaffold-ide`](../cli/scaffold-ide.md) | | `principal-placeholder` (the kind of an `unsupported_binding` error) | On a `gcp` binding, a principal is a placeholder (a reserved top-level domain) or is not an IAM member and `binding.principals` does not map it | Map it in `binding.principals` to a real group or service account, or write the IAM member in `accessPolicy`; see [`fluid generate iac`](../cli/generate-iac.md) | +| `mwaa-environment-not-emitted` (the kind of an `unsupported_binding` error) | *(unreleased, [forge-cli #715](https://github.com/Agenticstiger/forge-cli/pull/715))* An `aws` contract with `orchestration.engine: airflow` and an `orchestration.schedule` asks for an MWAA environment, which the OpenTofu module cannot create: it needs a VPC network the contract does not carry. `fluid apply` and `fluid generate iac` stop before anything is applied; the apply used to report success with no environment | To schedule without Airflow, set `orchestration.engine: none` and keep the schedule. To keep Airflow, run the DAGs on an MWAA environment you own and drop `orchestration.schedule`. See [Scheduling on AWS](../providers/aws.md#scheduling-on-aws) | +| `aws_planned_op_not_emitted`, `aws_orchestration_task_not_emitted`, `planned_op_not_emitted` (WARNING) | *(unreleased, [forge-cli #715](https://github.com/Agenticstiger/forge-cli/pull/715))* The AWS planner planned an operation the OpenTofu module has no resource for, and the apply goes on without it. The line names the operation and its id | Read which operation it is in [Planned operations the module does not create](../providers/aws.md#planned-operations-the-module-does-not-create), and provision it another way if you need it | | `schedule_sync_dags_dir_not_product_scoped` (exit 2) | `fluid schedule-sync` defaults to `--delete-scope product`, which mirrors each top-level directory of `--dags-dir` and never deletes another product's DAGs. The directory has loose files | Put the files in `//`. `fluid generate artifacts` does; for `fluid generate schedule` use `-o //`. Or pass `--delete-scope none` to copy without deleting | | `lakeformation-grant-columns` | A Lake Formation grant gives a principal read access, but the contract's column restrictions let it read no column | Remove the grant, or allow the principal at least one column in `policy.authz.columnRestrictions` | | `generate_iac_aws_account_required` | `fluid generate iac` for AWS needs the account id and does not look it up | Set `AWS_ACCOUNT_ID`. `fluid apply` resolves the account itself | diff --git a/docs/advanced/typed-cli-errors.md b/docs/advanced/typed-cli-errors.md index f179bf2..6a3679a 100644 --- a/docs/advanced/typed-cli-errors.md +++ b/docs/advanced/typed-cli-errors.md @@ -78,7 +78,7 @@ Each class has a `for_*` factory that builds the `what`, `why`, `fix` and `doc` `SchemaValidationError` is also the name of a different class in the forge agent layer; see [Typed Errors](./typed-errors.md). The two are not the same type. -The route table also sends four catalogued events here: `invalid_schema_version`, `contract_version_unsupported`, `invalid_min_version` and `invalid_max_version`. Set `fluidVersion` to a version the installed CLI bundles. The latest stable contract schema is 0.7.5; 0.7.6 is a preview that a contract opts in to with `fluidVersion: "0.7.6"`. `fluid validate` reports the version it validated against. +The route table also sends four catalogued events here: `invalid_schema_version`, `contract_version_unsupported`, `invalid_min_version` and `invalid_max_version`. Set `fluidVersion` to a version the installed CLI bundles. The latest stable contract schema is 0.7.5; 0.7.6 is a preview that a contract opts in to with `fluidVersion: "0.7.6"`. `fluid validate` reports the version it validated against. *([forge-cli #714](https://github.com/Agenticstiger/forge-cli/pull/714), unreleased)* `fluid apply` raises `contract_version_unsupported` before it plans, on both engines and in every mode; see [Contract validation](../cli/apply.md#contract-validation). ### Capability negotiation diff --git a/docs/cli/apply.md b/docs/cli/apply.md index acce81f..4296535 100644 --- a/docs/cli/apply.md +++ b/docs/cli/apply.md @@ -124,6 +124,8 @@ Two things differ between the engines, and the rest of this page says which one - **Modes.** `--mode dry-run` stops after `tofu plan` on the OpenTofu engine. Beyond that the OpenTofu engine does not branch on `--mode`: `amend`, `create-only` and `replace` run the same `tofu apply` of the emitted module. What stays mode-specific is the [data-loss gate](#safety-gates) and the build phase of the `*-and-build` modes. - **Options.** Some options are read only by the native engine. They are marked in the [options tables](#options). +*([forge-cli #714](https://github.com/Agenticstiger/forge-cli/pull/714), unreleased)* The contract check does not differ: both engines validate the contract before they plan, in every mode. See [Contract validation](#contract-validation). + ## Apply mode | `--mode` | What it does | @@ -152,6 +154,49 @@ amend-and-build (or replace-and-build) to run that build, or drop --build-id. Before 0.16.3 the flag was accepted and dropped, so a DDL-only apply exited 0. +## Contract validation + +*([forge-cli #714](https://github.com/Agenticstiger/forge-cli/pull/714), unreleased)* `fluid apply` checks the contract against its schema before it plans anything, on both engines and in every mode, `--dry-run` and the build modes included. A GCP contract that still carries the deprecated `metadata.policies` mapping stops before `tofu` runs: + +```bash +fluid apply contract.fluid.yaml --dry-run --yes +``` + +```text +❌ apply_contract_invalid [ERR_APPLY_CONTRACT_INVALID] + command: apply + error_count: 1 + errors: ["metadata: Additional properties are not allowed ('policies' was unexpected)"] + message: Contract validation failed with 1 error(s); ``fluid apply`` will not proceed on a schema-invalid contract. Run ``fluid validate`` for the full report. +``` + +It is the check `fluid validate` and `fluid plan` run: a `fluidVersion` below 0.7 is refused, then the contract is validated against the schema its `fluidVersion` names. + +| The input | Event | Exit | +|---|---|---| +| A contract whose `fluidVersion` is below 0.7 | `contract_version_unsupported` | 1 | +| A contract that is not valid against its schema | `apply_contract_invalid` | 1 | +| A `plan.json` that embeds no contract | `apply_plan_missing_contract` | 1 | + +Before this change only the native engine ran the check, and not in the build modes. On the OpenTofu engine (`aws`, `gcp`, `snowflake`, `confluent` and IaC plugin clouds), and under `--mode amend-and-build` or `replace-and-build` on either engine, a contract `fluid validate` rejects reached `tofu` or the build runner and could apply with exit 0. + +What is checked: + +- **A contract file or a bundle.** The contract with its `--env` overlay, as `fluid validate --env ` sees it: `{{ env.* }}` placeholders are still literal. +- **A `plan.json`.** The contract embedded in the plan, the copy its `planDigest` covers. Every plan `fluid plan` writes embeds it (see [What `plan.json` records](./plan.md#what-plan-json-records)). + +A plan whose `contract` is missing, empty or not an object is refused, with or without `--no-verify-plan-binding`. There is nothing to check, and the `planDigest` is an unkeyed hash: it proves the file has not changed since it was written, not who wrote it. Before this change such a plan fell back to the native engine and ran its actions unchecked. + +```text +❌ apply_plan_missing_contract [ERR_APPLY_PLAN_MISSING_CONTRACT] + path: .../runtime/plan.json + hint: plan.json embeds no contract, so there is nothing to validate before applying it. Re-run ``fluid plan --out .json`` with a current forge-cli. +``` + +The check runs after the [plan binding](#plan-binding) checks and the [mode gate](#the-mode-gate), and before the [apply hooks](#extension-point-apply-hooks), `tofu`, the native provider and any build, so a hook never sees a refused contract. `fluid apply` reads its input once, and the engine choice, the hooks, `tofu`, the build runner and the native provider all take that copy: the contract that is applied is the contract that was checked, even if the file changes while the apply runs. + +There is no flag to skip the check. Run `fluid validate --env ` to see every error, fix them, and apply again. + ## Plan binding When you pass a saved plan (`runtime/plan.json`) instead of a contract, `apply` runs three gates on it before it builds or provisions anything, on both engines: @@ -174,6 +219,8 @@ This is the Terraform-style "apply consumes exact plan" guarantee, enforced cryp Pass the bundle with `--bundle `: the tgz this `plan.json` was generated against. When omitted, `apply` auto-discovers a single sibling `.tgz` / `.tar.gz` next to the `plan.json`. If several are present it logs a warning and leaves the bundle unresolved, so pass `--bundle` to disambiguate. A `plan.json` made from a raw contract has an empty `bundleDigest`, so only `planDigest` is verified; see [`fluid plan`](./plan.md#plan-binding). +*([forge-cli #714](https://github.com/Agenticstiger/forge-cli/pull/714), unreleased)* After these gates the contract the plan embeds is validated against its schema, and a plan that embeds none is refused with `apply_plan_missing_contract`. See [Contract validation](#contract-validation). + ### The plan's mode `fluid plan --mode X` records `X` in `plan.json`. `fluid apply plan.json --mode Y` with a different `Y` is refused with `apply_plan_mode_mismatch` and exit 1, before any build runs. This is why every non-default example above plans with the same `--mode`. See [`fluid plan --mode`](./plan.md#key-options). @@ -516,6 +563,9 @@ Each failure prints the typed event name and an `[ERR_]` slug, and exits | --- | --- | | `apply_plan_digest_plan_tamper`, `apply_plan_digest_bundle_mismatch`, `apply_plan_digest_bundle_missing` | A saved plan fails its digest checks. See [Plan binding](#plan-binding). | | `apply_plan_mode_mismatch` | `--mode` differs from the mode the plan was made for. | +| `apply_contract_invalid` | *(unreleased, [forge-cli #714](https://github.com/Agenticstiger/forge-cli/pull/714))* The contract, or the contract a `plan.json` embeds, is not valid against its schema. Raised on both engines and in every mode. See [Contract validation](#contract-validation). | +| `contract_version_unsupported` | *(unreleased, [forge-cli #714](https://github.com/Agenticstiger/forge-cli/pull/714))* The contract's `fluidVersion` is below 0.7. Raised on both engines and in every mode. | +| `apply_plan_missing_contract` | *(unreleased, [forge-cli #714](https://github.com/Agenticstiger/forge-cli/pull/714))* A `plan.json` embeds no contract. Re-run `fluid plan` and apply the new plan. | | `plan_env_mismatch` | A build mode is given an `--env` that differs from the plan's. | | `bundle_env_mismatch` | `--env` differs from the env a bundle was built for. | | `apply_build_id_requires_build_mode` | `--build-id` with a mode that runs no build. | @@ -527,6 +577,7 @@ Each failure prints the typed event name and an `[ERR_]` slug, and exits | `opentofu_dataset_access_unreconciled` | State holds a BigQuery dataset an older forge-cli applied with an authoritative access list, and every entry of it is a grant the contract no longer makes. Revoke those entries on the dataset by hand, or keep one of them in `accessPolicy` for one apply. | | `apply_state_backend_invalid`, `state_shared_with_another_provider`, `state_migration_*` | See [Remote state](#remote-state). | | `generate_iac_provider_mismatch` | `--provider` contradicts the contract's binding. | +| `unsupported_binding` | The OpenTofu module cannot provision part of the contract as declared, and refuses it rather than drop it; the `kind:` line says which part. *(unreleased, [forge-cli #715](https://github.com/Agenticstiger/forge-cli/pull/715))* Kind `mwaa-environment-not-emitted`: an `aws` contract with `orchestration.engine: airflow` and an `orchestration.schedule` asks for an MWAA environment, which the module cannot create. See [Scheduling on AWS](../providers/aws.md#scheduling-on-aws). | | `opentofu_engine_no_tofu` | The OpenTofu engine needs `tofu` and it is not on `PATH`. Install it, or pass `--ensure-opentofu`. | ## Notes diff --git a/docs/cli/generate-iac.md b/docs/cli/generate-iac.md index e70af01..3e043b9 100644 --- a/docs/cli/generate-iac.md +++ b/docs/cli/generate-iac.md @@ -92,7 +92,7 @@ Engine selection is automatic and per-provider (`apply.py::resolve_apply_engine` | Provider | Resources emitted | |---|---| -| `aws` | S3 buckets / IAM roles + policies / Glue databases + tables + column comments (for an Iceberg expose, only when its catalog is Glue: [forge-cli #707](https://github.com/Agenticstiger/forge-cli/pull/707), unreleased) / Athena workgroups / Lake Formation grants, with a contract's `columnRestrictions` written as excluded columns *(0.17.0)* / KMS keys and aliases for `binding.encryption.kms` and S3 lifecycle rules for `lifecycle.expire` *(0.16.5)* | +| `aws` | S3 buckets / IAM roles + policies / Glue databases + tables + column comments (for an Iceberg expose, only when its catalog is Glue: [forge-cli #707](https://github.com/Agenticstiger/forge-cli/pull/707), unreleased) / Athena workgroups / Lake Formation grants, with a contract's `columnRestrictions` written as excluded columns *(0.17.0)* / KMS keys and aliases for `binding.encryption.kms` and S3 lifecycle rules for `lifecycle.expire` *(0.16.5)*. No MWAA environment: a contract that asks for one is refused ([forge-cli #715](https://github.com/Agenticstiger/forge-cli/pull/715), unreleased; see [Scheduling on AWS](../providers/aws.md#scheduling-on-aws)) | | `gcp` | BigQuery datasets + tables / `google_bigquery_dataset_iam_member` grants *(0.17.0)* / GCS buckets / BigLake-Iceberg warehouse buckets *(0.14.0)* / Pub/Sub topics, with the binding's region as `message_storage_policy.allowed_persistence_regions` / Cloud KMS key rings and keys, Data Catalog taxonomies and policy tags, and a `terraform_data` partition-replace trigger *(0.17.0)* | | `snowflake` | Databases / schemas / tables / column comments (Horizon-aware) / file formats / stages / external volumes + Glue catalog integrations for Iceberg exposes *(0.13.1)* | @@ -130,6 +130,17 @@ Principals are refused when they cannot be real identities. A principal in a res Other refusal kinds exist for principals and governance fields: `principal-unmapped`, `column-restriction-*` and `encryption-kms-*`. See the [GCP provider page](../providers/gcp.md) for the provider's own behaviour. +*([forge-cli #715](https://github.com/Agenticstiger/forge-cli/pull/715), unreleased)* On AWS, a contract with `orchestration.engine: airflow` and an `orchestration.schedule` is refused with kind `mwaa-environment-not-emitted`, before the output directory is created: + +```text +❌ unsupported_binding [ERR_UNSUPPORTED_BINDING] + kind: mwaa-environment-not-emitted + error: The contract asks for the MWAA environment 'mwaa-sales_orders_daily' (orchestration.engine airflow or mwaa with orchestration.schedule), but the OpenTofu engine cannot create an aws_mwaa_environment: it needs a VPC network_configuration (subnets and security groups) that the contract does not carry. Applying would report success with no Airflow environment, so nothing was applied. + remediation: ["To schedule without Airflow, set orchestration.engine to 'none' and keep orchestration.schedule: fluid apply then creates an EventBridge Scheduler schedule.", ...] +``` + +The module used to be written without an `aws_mwaa_environment`, and `fluid apply` reported success. The refusal holds when the AWS planner fails on another field, and under `--provider aws` on a multi-cloud contract whose first expose is on another cloud. With `--shadow` the command stops at the refusal; the shadow comparison counts a planned `mwaa.ensure_environment` as an `environment` resource, so an environment missing from a module would show as a native-only gap. A planned operation the AWS module does not create is named in a WARNING line instead of being dropped without a word. Both ways out, and the warnings, are in [Scheduling on AWS](../providers/aws.md#scheduling-on-aws). + ### Access grants on GCP *(0.17.0)* [Governance parity](../concepts/governance-parity.md) lists the grants, policy tags, keys and retention each cloud receives from one contract. diff --git a/docs/cli/plan.md b/docs/cli/plan.md index 413c829..08c8916 100644 --- a/docs/cli/plan.md +++ b/docs/cli/plan.md @@ -89,7 +89,7 @@ Besides the digests, `plan.json` carries what `apply` needs to refuse a plan tha | `mode` | Always; `null` without `--mode`. | `fluid apply plan.json --mode Y` with a `Y` other than the recorded mode is refused with `apply_plan_mode_mismatch`, before any build. A missing mode and `amend` count as the same. | | `contract_metadata.env` | `--env` was given, or the plan was made from a bundle that records an env. | In a build mode (`amend-and-build`, `replace-and-build`), `apply` without `--env` runs the builds in that env, and a different `--env` is refused with `plan_env_mismatch`. See [`fluid apply`](./apply.md#the-plan-s-environment). | | `contract_metadata.source_contract` | The plan was made from a bundle. | Names the source contract file the bundle was built from. | -| `contract` | Always. | Apply runs the embedded contract, which already has the overlay applied. | +| `contract` | Always. | Apply runs the embedded contract, which already has the overlay applied. *([forge-cli #714](https://github.com/Agenticstiger/forge-cli/pull/714), unreleased)* It validates it against its schema first, on both engines, and refuses a plan with no embedded contract with `apply_plan_missing_contract`. See [Contract validation](./apply.md#contract-validation). | A bundle is never re-overlaid: `fluid plan bundle.tgz --env prod` on a bundle built for `dev` is refused with `bundle_env_mismatch`. Plan the bundle without `--env`, or rebuild it with `fluid bundle --env prod --format tgz`. diff --git a/docs/cli/schedule-sync.md b/docs/cli/schedule-sync.md index 3eccc45..e0d2387 100644 --- a/docs/cli/schedule-sync.md +++ b/docs/cli/schedule-sync.md @@ -250,4 +250,4 @@ fluid schedule-sync \ - Each dispatch needs its tool on `PATH`: `aws` (`s3://` destinations and `mwaa`), `gsutil` (`gs://`), `az` (`az://`), `rsync` (`file://`, `ssh://`, `git+ssh://`), `scp` (`scp://`), `git` (`git+ssh://`), `gcloud` (`composer`), `astro`, `prefect` or `dagster-cloud`. [`fluid doctor`](./doctor.md) does not check for `astro`, `prefect` or `dagster-cloud`. A missing tool fails `schedule-sync` with `schedule_sync_binary_not_on_path` (exit `2`) before anything is pushed, `--dry-run` included. - `--dry-run` short-circuits before the subprocess runs and emits the (redacted) planned argv. Nothing about the destination changes. - Exit codes: `0` = pushed (or dry run complete); `1` = push failure (a subprocess exited non-zero, or the bundle signature did not verify); `2` = config error: a missing or empty `--dags-dir`, a `--dags-dir` that is not product-scoped, an invalid identifier, a bad scheduler choice or destination, or a scheduler binary missing from `PATH`. -- Path-B schedulers (EventBridge, Snowflake Tasks, MWAA-native) apply their schedules inside stage-7 apply via `SchedulePlanner`. Stage 11 is a no-op for those contracts. +- Path-B schedulers (EventBridge, Snowflake Tasks) apply their schedules inside stage-7 apply via `SchedulePlanner`. Stage 11 is a no-op for those contracts. *([forge-cli #715](https://github.com/Agenticstiger/forge-cli/pull/715), unreleased)* MWAA is not one of them. On `aws`, `fluid apply` refuses a contract that asks it for an MWAA environment (`orchestration.engine: airflow` with an `orchestration.schedule`) with `unsupported_binding`, kind `mwaa-environment-not-emitted`; it used to report success and create none. Run the DAGs on an environment you own and push them with `--scheduler mwaa`; see [MWAA on AWS](../advanced/airflow.md#mwaa-on-aws). diff --git a/docs/providers/aws.md b/docs/providers/aws.md index 16290f6..cf297a9 100644 --- a/docs/providers/aws.md +++ b/docs/providers/aws.md @@ -37,6 +37,7 @@ The AWS provider turns a FLUID contract into cloud resources: - **Verification** — `fluid verify` reads Glue, runs an Athena `COUNT(*)` and checks the governance above on the live account. See [Verify on AWS](#verify-on-aws). - **Sovereignty validation** — region allow and deny lists are enforced before deployment. See [Data Sovereignty](#data-sovereignty). - **IAM policy compilation** — `fluid policy-compile` writes IAM bindings from `accessPolicy`, as advisory output only. See [accessPolicy on AWS](#accesspolicy-on-aws). +- **Scheduling** — with `orchestration.engine: none`, `orchestration.schedule` becomes an EventBridge Scheduler schedule. *(forge-cli [#715](https://github.com/Agenticstiger/forge-cli/pull/715), unreleased)* With `orchestration.engine: airflow` it asks for an MWAA environment, which `fluid apply` refuses rather than report a success it did not create. See [Scheduling on AWS](#scheduling-on-aws). ## Working example: Bitcoin price tracker @@ -359,6 +360,91 @@ Contract SQL on the DuckDB engine can read and write only the contract's own dir For acquisition build properties, see [Source-aligned acquisition](../advanced/source-aligned-acquisition.md); the same landing rules are in [where the build lands data](../advanced/source-aligned-acquisition.md#where-the-build-lands-data). +## Scheduling on AWS + +*([forge-cli #715](https://github.com/Agenticstiger/forge-cli/pull/715), unreleased)* An `aws` contract with `orchestration.engine: airflow` and an `orchestration.schedule` asks for an Amazon MWAA environment, and `fluid apply` and `fluid generate iac` refuse it: + +```yaml +orchestration: + engine: airflow + schedule: "cron(0 6 * * ? *)" +``` + +```bash +fluid generate iac contract.fluid.yaml --out ./review +``` + +```text +❌ unsupported_binding [ERR_UNSUPPORTED_BINDING] + kind: mwaa-environment-not-emitted + error: The contract asks for the MWAA environment 'mwaa-sales_orders_daily' (orchestration.engine airflow or mwaa with orchestration.schedule), but the OpenTofu engine cannot create an aws_mwaa_environment: it needs a VPC network_configuration (subnets and security groups) that the contract does not carry. Applying would report success with no Airflow environment, so nothing was applied. + remediation: ["To schedule without Airflow, set orchestration.engine to 'none' and keep orchestration.schedule: fluid apply then creates an EventBridge Scheduler schedule.", "To keep Airflow, create the MWAA environment 'mwaa-sales_orders_daily' yourself (it needs your VPC), then push this contract's DAG to it with `fluid generate schedule --scheduler airflow --output dags` and `fluid schedule-sync --scheduler mwaa --dags-dir dags --destination s3:///dags/`. fluid apply still plans the environment while the contract has orchestration.schedule, so apply a copy of the contract without it."] +``` + +`fluid apply contract.fluid.yaml` prints the same refusal before `tofu init` and writes no module, and `fluid generate iac` writes no output directory. Before this change the module had no `aws_mwaa_environment`, `fluid apply` reported success, and no Airflow environment existed. The OpenTofu engine cannot create one: `aws_mwaa_environment` needs a VPC `network_configuration` (private subnets and security groups), and a contract carries no network. + +The environment is named `mwaa-`, or `orchestration.mwaa.environmentName` when the contract sets it. A contract that names one is told to push to that environment rather than create it, with `--destination` filled in from `orchestration.mwaa.dagS3Path`: + +```text + remediation: [..., "Your contract names the existing MWAA environment 'shared-airflow': push this contract's DAG to it with `fluid generate schedule --scheduler airflow --output dags` and `fluid schedule-sync --scheduler mwaa --dags-dir dags --destination s3://acme-airflow-dags/dags/`. fluid apply still plans the environment while the contract has orchestration.schedule, so apply a copy of the contract without it."] +``` + +What `orchestration` makes `fluid apply` do on AWS: + +| `orchestration` | What the module gets | +|---|---| +| `engine: airflow` and a `schedule` | Nothing: the contract is refused with `mwaa-environment-not-emitted` | +| `engine: none` (or any engine but `airflow`) and a `schedule` | An EventBridge Scheduler schedule and the Lambda function it invokes; see [Scheduling without Airflow](#scheduling-without-airflow) | +| `engine: airflow` and no `schedule` | No schedule resource. Builds scheduled with `builds[].execution.trigger` run from the DAGs `fluid generate schedule` writes; see [MWAA on AWS](../advanced/airflow.md#mwaa-on-aws) | + +Edge cases: + +- **`fluid validate` and `fluid plan` pass the contract.** `airflow` is a valid engine, and the refusal comes from the module emitter. Run `fluid generate iac` to see it without touching AWS; it needs no `AWS_ACCOUNT_ID` to refuse. +- **The refusal follows the contract, not the plan.** It holds when the AWS planner fails on another field (a Glue table name it rejects, `orchestration.tasks` that depend on each other), and under `--provider aws` on a multi-cloud contract whose first expose is on GCP or Snowflake, where that cloud's planner runs instead and never plans the environment. +- **`engine: mwaa`** asks for the environment too, but `mwaa` is not one of the schema's `orchestration.engine` values: `fluid validate` rejects it, and with [forge-cli #714](https://github.com/Agenticstiger/forge-cli/pull/714) `fluid apply` stops on it with `apply_contract_invalid` first. `fluid generate iac`, which does not validate, refuses it as above. +- **`orchestration.triggers` with no `schedule`** does not ask for MWAA and is not refused. +- **`--shadow`.** `fluid generate iac --shadow` stops at the refusal. The shadow comparison now counts a planned `mwaa.ensure_environment` as an `environment` resource, so an environment the module lacked would show as a native-only gap. + +### Scheduling without Airflow + +Set `orchestration.engine: none` and keep the schedule: + +```yaml +orchestration: + engine: none + schedule: "cron(0 6 * * ? *)" +``` + +```text +Planned EventBridge schedule: fluid-schedule-sales_orders_daily +... +Wrote OpenTofu module: ./review/main.tf.json (provider: aws, 6 resources) +``` + +Next to the bucket, the Glue database and the Glue table, the module gets an `aws_scheduler_schedule` named `fluid-schedule-`, an `aws_lambda_function` named `fluid-workflow-`, and the `aws_lambda_permission` that lets the scheduler invoke it. Before you rely on it (measured on the forge-cli #715 branch): + +- The schedule and the function name IAM roles the module does not create: `arn:aws:iam:::role/EventBridgeSchedulerRole` and `arn:aws:iam:::role/fluid-workflow-execution`. They have to exist in the account. +- The function logs each `orchestration.tasks` entry of type `provider_action` and returns. It calls no AWS API and runs none of the contract's builds. +- Lambda function names take letters, digits, hyphens and underscores only. With a dot in the contract id (`sales.orders_daily`) the AWS planner fails, and the module is written without the schedule, the function and the permission, with exit 0. Use an id without dots for a scheduled AWS contract. + +To keep Airflow, run the DAGs on an MWAA environment you create in your own VPC, or one you already run: [MWAA on AWS](../advanced/airflow.md#mwaa-on-aws) has the steps. + +### Planned operations the module does not create + +The AWS planner plans some operations the OpenTofu module has no resource for. They are no longer dropped without a word: each is named in a WARNING line, and the apply goes on. + +```text +aws_orchestration_task_not_emitted op='athena.execute_query' id='refresh_orders': orchestration tasks run in the scheduler, and fluid apply provisions nothing for them +``` + +| Warning | What it names | +|---|---| +| `aws_planned_op_not_emitted` | An operation nothing in the module provisions yet: IAM bindings from `metadata.policies` (`iam.bind_glue_database`, `iam.bind_s3_bucket`), Athena views (`athena.create_view`), Redshift schemas and tables (`redshift.ensure_schema`, `redshift.ensure_table`), and a top-level `execution.trigger` schedule (`events.ensure_schedule`) | +| `aws_orchestration_task_not_emitted` | An `orchestration.tasks` entry. Tasks run in the scheduler, and the module provisions nothing for them | +| `planned_op_not_emitted` | An operation the AWS module does not know, such as another cloud's under `--provider aws` on a multi-cloud contract | + +The operations the module builds from `exposes[]` (the bucket, the Glue database and tables) and the build steps the build runner executes pass without a warning. + ## Credentials setup ### Jenkins CI (recommended) @@ -951,6 +1037,8 @@ The release notes for [`0.16.0`](../RELEASE_NOTES_0.16.0.md) and [`0.17.0`](../R - **`0.18.0`:** contract SQL runs in a DuckDB sandbox and `$ref` stays inside the contract's directory tree; see [Where a build lands data](#where-a-build-lands-data). - **Unreleased, [forge-cli #707](https://github.com/Agenticstiger/forge-cli/pull/707):** an Iceberg expose whose `location.catalog` is not `glue` keeps its S3 bucket and loses its Glue database and table. On a contract an earlier release applied, `fluid apply` stops with `iceberg_catalog_move_blocked` and prints the `tofu state rm` commands that release them; see [Upgrading an AWS contract that names another catalog](../advanced/source-aligned-acquisition.md#upgrading-an-aws-contract-that-names-another-catalog). A Kafka Connect sink on Glue starts, where it failed at startup before. - **Unreleased, [forge-cli #709](https://github.com/Agenticstiger/forge-cli/pull/709):** the `iceberg_catalog_move_blocked` guard flags an expose's Glue database and table when that expose's own Glue table is in state, so a Glue database left by a removed expose no longer blocks the apply. An expose that names a `location.database` and no `location.table` has no Glue table, so its Glue database is flagged when state holds it and the module no longer declares it. When the guard cannot read the state, it logs an `iceberg_catalog_move_probe_skipped` WARNING. `fluid diff` refuses an Iceberg `location.catalog` value FLUID does not know, as `fluid validate` and `fluid apply` do, and a typo no longer also draws a Lake Formation refusal. See [Upgrading an AWS contract that names another catalog](../advanced/source-aligned-acquisition.md#upgrading-an-aws-contract-that-names-another-catalog). +- **Unreleased, [forge-cli #714](https://github.com/Agenticstiger/forge-cli/pull/714):** `fluid apply` validates the contract against its schema before it plans, in every mode. A contract `fluid validate` rejects used to reach `tofu` and could apply with exit 0; it now stops with `apply_contract_invalid`. Run `fluid validate contract.fluid.yaml --env ` before you upgrade. See [Contract validation](../cli/apply.md#contract-validation). +- **Unreleased, [forge-cli #715](https://github.com/Agenticstiger/forge-cli/pull/715):** a contract with `orchestration.engine: airflow` and an `orchestration.schedule` fails `fluid apply` and `fluid generate iac` with `unsupported_binding`, kind `mwaa-environment-not-emitted`. It used to apply with exit 0 and create no MWAA environment. Set `orchestration.engine: none` to schedule through EventBridge Scheduler, or run the DAGs on an MWAA environment you own; see [Scheduling on AWS](#scheduling-on-aws). Planned operations the module does not create are now named in a WARNING line. ## How far this has been exercised diff --git a/docs/providers/gcp.md b/docs/providers/gcp.md index bfc1c2e..9bb665b 100644 --- a/docs/providers/gcp.md +++ b/docs/providers/gcp.md @@ -559,6 +559,13 @@ contracts keep working, but it has never been schema-valid and fails mid-migration does not drop half its grants; duplicate grants collapse. See [Release Notes `0.13.0`](../RELEASE_NOTES_0.13.0.md#accesspolicy-is-now-the-iac-access-grant-surface). +*([forge-cli #714](https://github.com/Agenticstiger/forge-cli/pull/714), unreleased)* +`fluid apply` now validates the contract before it plans, so a contract that still +carries `metadata.policies` stops with `apply_contract_invalid` +(`metadata: Additional properties are not allowed ('policies' was unexpected)`) +instead of applying. Only `fluid generate iac`, which does not validate, still emits +it. See [Contract validation](../cli/apply.md#contract-validation). + ### Shared vs. isolated containers By default the product **owns** the BigQuery dataset and GCS bucket it creates. A diff --git a/docs/providers/snowflake.md b/docs/providers/snowflake.md index c4b7299..7e105fe 100644 --- a/docs/providers/snowflake.md +++ b/docs/providers/snowflake.md @@ -556,7 +556,7 @@ What 0.18.1 does with each governance field on a Snowflake binding: | `sovereignty` | checked against the binding's region before deployment | | `metadata.layer`, `productType`, `domain`, `fluidVersion` | written into the table COMMENT | -The Snowflake module can emit `snowflake_masking_policy`, `snowflake_row_access_policy` and `snowflake_grant_privileges_to_account_role` objects, but only from a top-level `security:` block (`security.policies`, `security.access_control.grants`). The contract schema rejects that block, so a contract carrying it fails `fluid validate`, and even then the policies are created without being attached to a column or table. +The Snowflake module can emit `snowflake_masking_policy`, `snowflake_row_access_policy` and `snowflake_grant_privileges_to_account_role` objects, but only from a top-level `security:` block (`security.policies`, `security.access_control.grants`). The contract schema rejects that block, so a contract carrying it fails `fluid validate`, and even then the policies are created without being attached to a column or table. *([forge-cli #714](https://github.com/Agenticstiger/forge-cli/pull/714), unreleased)* `fluid apply` now validates the contract before it plans and stops on that block with `apply_contract_invalid`, so only `fluid generate iac`, which does not validate, still emits these objects. See [Contract validation](../cli/apply.md#contract-validation). ## CI/CD Pipeline diff --git a/docs/upgrading.md b/docs/upgrading.md index 608af5c..3852082 100644 --- a/docs/upgrading.md +++ b/docs/upgrading.md @@ -294,6 +294,31 @@ Background: [Composing a contract with `$ref`](./concepts/contract-refs.md), [DuckDB sandbox for contract SQL](./advanced/duckdb-sandbox.md), [Contract loading API](./advanced/contract-loading-api.md) (new in `fluid_build.api` `1.1`). +## Unreleased changes after 0.18.1 + +These changes are in forge-cli pull requests that are not in a release yet. Each row names +its pull request. + +| Change | What to do | How to check | +|---|---|---| +| `fluid apply` validates the contract against its schema on both engines and in every mode ([forge-cli #714](https://github.com/Agenticstiger/forge-cli/pull/714)). A cloud apply (`aws`, `gcp`, `snowflake`, `confluent`) or a build mode skipped the check, so a contract `fluid validate` rejects could apply with exit 0. It now stops before planning with `apply_contract_invalid`, or `contract_version_unsupported` below `fluidVersion` 0.7 | Run `fluid validate --env ` for each overlay you apply, and fix what it reports. No flag skips the check | `fluid apply --env --dry-run` stops before `tofu plan` with the event | +| A `plan.json` that embeds no contract is refused with `apply_plan_missing_contract` ([forge-cli #714](https://github.com/Agenticstiger/forge-cli/pull/714)). It used to run its actions unchecked on the native engine | Re-run `fluid plan` and apply the new plan | The apply fails before anything runs | +| An `aws` contract with `orchestration.engine: airflow` and an `orchestration.schedule` fails `fluid apply` and `fluid generate iac` with `unsupported_binding`, kind `mwaa-environment-not-emitted` ([forge-cli #715](https://github.com/Agenticstiger/forge-cli/pull/715)). The apply reported success and created no MWAA environment | Set `orchestration.engine: none` to schedule through EventBridge Scheduler, or run the DAGs on an MWAA environment you own and drop `orchestration.schedule`. See [Scheduling on AWS](./providers/aws.md#scheduling-on-aws) | `fluid generate iac contract.fluid.yaml` refuses with the kind and writes no module | + +A key the schema does not have is one way a contract fails the first row. With +`fluid apply --dry-run`, a GCP contract that still uses the deprecated `metadata.policies` +went on to `tofu plan` on forge-cli main before #714, and now stops: + +```text +❌ apply_contract_invalid [ERR_APPLY_CONTRACT_INVALID] + command: apply + error_count: 1 + errors: ["metadata: Additional properties are not allowed ('policies' was unexpected)"] + message: Contract validation failed with 1 error(s); ``fluid apply`` will not proceed on a schema-invalid contract. Run ``fluid validate`` for the full report. +``` + +Background: [Contract validation](./cli/apply.md#contract-validation). + ## Older releases Upgrading from before `0.15.0`? Start with the From 795422c1fe417f1f987a400b7a113af40f9a14a4 Mon Sep 17 00:00:00 2001 From: Speculator55005 <50082482+fas89@users.noreply.github.com> Date: Mon, 12 Oct 2026 22:34:25 +0200 Subject: [PATCH 2/2] docs(aws): the MWAA refusal's keep-Airflow commands generate into dags/ Follows forge-cli #715's e6260e2a: the printed generate/schedule-sync pair now uses the product-scoped layout schedule-sync requires, so it runs as printed. --- docs/advanced/airflow.md | 2 +- docs/providers/aws.md | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/advanced/airflow.md b/docs/advanced/airflow.md index b8fe65b..8912357 100644 --- a/docs/advanced/airflow.md +++ b/docs/advanced/airflow.md @@ -120,7 +120,7 @@ fluid schedule-sync --scheduler mwaa --dags-dir dags --destination s3://acme-air Drop `--dry-run` to push. `--destination` is the S3 folder the environment reads its DAGs from, in its source bucket. Each scheduled run executes `fluid apply` on the MWAA worker, which needs what [What the DAG does](#what-the-dag-does) lists. -An `aws` contract that keeps `orchestration.schedule` with `orchestration.engine: airflow` asks `fluid apply` for the environment itself, and is refused with `unsupported_binding`, kind `mwaa-environment-not-emitted`, before anything is applied; see [Scheduling on AWS](../providers/aws.md#scheduling-on-aws). Before this change the apply reported success and no environment existed. The refusal's own next step names `fluid generate schedule --scheduler airflow --output dags`, which writes the DAG straight into `dags/`; `fluid schedule-sync` refuses that layout with `schedule_sync_dags_dir_not_product_scoped` under its default `--delete-scope product`, so write into `dags//` as above. +An `aws` contract that keeps `orchestration.schedule` with `orchestration.engine: airflow` asks `fluid apply` for the environment itself, and is refused with `unsupported_binding`, kind `mwaa-environment-not-emitted`, before anything is applied; see [Scheduling on AWS](../providers/aws.md#scheduling-on-aws). Before this change the apply reported success and no environment existed. The refusal's own next step names `fluid generate schedule --scheduler airflow --output dags/` and `fluid schedule-sync --scheduler mwaa --dags-dir dags`, the product-scoped layout above, so the two commands run as printed. ## `fluid generate-airflow` (deprecated) diff --git a/docs/providers/aws.md b/docs/providers/aws.md index cf297a9..640b660 100644 --- a/docs/providers/aws.md +++ b/docs/providers/aws.md @@ -378,15 +378,15 @@ fluid generate iac contract.fluid.yaml --out ./review ❌ unsupported_binding [ERR_UNSUPPORTED_BINDING] kind: mwaa-environment-not-emitted error: The contract asks for the MWAA environment 'mwaa-sales_orders_daily' (orchestration.engine airflow or mwaa with orchestration.schedule), but the OpenTofu engine cannot create an aws_mwaa_environment: it needs a VPC network_configuration (subnets and security groups) that the contract does not carry. Applying would report success with no Airflow environment, so nothing was applied. - remediation: ["To schedule without Airflow, set orchestration.engine to 'none' and keep orchestration.schedule: fluid apply then creates an EventBridge Scheduler schedule.", "To keep Airflow, create the MWAA environment 'mwaa-sales_orders_daily' yourself (it needs your VPC), then push this contract's DAG to it with `fluid generate schedule --scheduler airflow --output dags` and `fluid schedule-sync --scheduler mwaa --dags-dir dags --destination s3:///dags/`. fluid apply still plans the environment while the contract has orchestration.schedule, so apply a copy of the contract without it."] + remediation: ["To schedule without Airflow, set orchestration.engine to 'none' and keep orchestration.schedule: fluid apply then creates an EventBridge Scheduler schedule.", "To keep Airflow, create the MWAA environment 'mwaa-sales_orders_daily' yourself (it needs your VPC), then push this contract's DAG to it with `fluid generate schedule --scheduler airflow --output dags/sales_orders_daily` and `fluid schedule-sync --scheduler mwaa --dags-dir dags --destination s3:///dags/`. fluid apply still plans the environment while the contract has orchestration.schedule, so apply a copy of the contract without it."] ``` `fluid apply contract.fluid.yaml` prints the same refusal before `tofu init` and writes no module, and `fluid generate iac` writes no output directory. Before this change the module had no `aws_mwaa_environment`, `fluid apply` reported success, and no Airflow environment existed. The OpenTofu engine cannot create one: `aws_mwaa_environment` needs a VPC `network_configuration` (private subnets and security groups), and a contract carries no network. -The environment is named `mwaa-`, or `orchestration.mwaa.environmentName` when the contract sets it. A contract that names one is told to push to that environment rather than create it, with `--destination` filled in from `orchestration.mwaa.dagS3Path`: +The environment is named `mwaa-`, or `orchestration.mwaa.environmentName` when the contract sets it. The DAG is generated into `dags//`, because `fluid schedule-sync` mirrors one directory per product and refuses loose files; a contract id that is not a valid directory name prints `dags/` for you to fill in. A contract that names one is told to push to that environment rather than create it, with `--destination` filled in from `orchestration.mwaa.dagS3Path`: ```text - remediation: [..., "Your contract names the existing MWAA environment 'shared-airflow': push this contract's DAG to it with `fluid generate schedule --scheduler airflow --output dags` and `fluid schedule-sync --scheduler mwaa --dags-dir dags --destination s3://acme-airflow-dags/dags/`. fluid apply still plans the environment while the contract has orchestration.schedule, so apply a copy of the contract without it."] + remediation: [..., "Your contract names the existing MWAA environment 'shared-airflow': push this contract's DAG to it with `fluid generate schedule --scheduler airflow --output dags/sales_orders_daily` and `fluid schedule-sync --scheduler mwaa --dags-dir dags --destination s3://acme-airflow-dags/dags/`. fluid apply still plans the environment while the contract has orchestration.schedule, so apply a copy of the contract without it."] ``` What `orchestration` makes `fluid apply` do on AWS: