design(terraform): reconcile — plan drift as a PR today, HCL regeneration through import blocks later

[lex00]

Sub-issue 8 of #2081. Not blocked, filed for later.

Reconcile is the middle setting on the dial (docs/src/content/docs/concepts/lifecycle-models.mdx, "The dial"): observe reports, reconcile pulls live reality back into source, authoritative pushes source into the cloud. docs/src/content/docs/guide/reconciling-lifecycle.mdx covers the cloud-to-code direction, and lexicons/temporal/src/composites/reconcile-op.ts implements it as Snapshot, Plan, Reconcile, where reconcilePr regenerates TypeScript via chant import --from <env> and opens a PR whose diff is the regenerated source.

That mechanism does not carry over. The terraform lexicon does not synthesize HCL and has no typed model to regenerate from, so chant import has nothing to emit. Cloud-to-code here means writing HCL someone else authored, in their style and their module layout. There is no path from live state to .tf that chant owns.

The v1 is what #2087 already covers: TerraformWatchOp with findingMode: "issue" or "pull-request", carrying the -no-color human plan. A human reads it and decides whether source or the cloud is wrong.

The candidate later path is Terraform's own generator. terraform plan -generate-config-out=<file> with import {} blocks (Terraform 1.5 and up) emits HCL for resources present in the cloud but absent from state. That fits a reconcilePr-shaped activity: run the generator, commit the generated file plus the import blocks, open a PR. The output is a starting point rather than finished code, which is the right thing to put in front of a reviewer.

Open questions:

• Where do the import {} blocks come from? Terraform will not enumerate unmanaged resources; something has to name the addresses and cloud ids first.
• Which cloud does the enumeration? Reusing the aws/gcp/azure lexicons' live reads is one option and drags in a cross-lexicon dependency.
• Where does the generated file land in someone else's module layout, and who decides?
• Does the ownership answer hold? Everything reconciled in is by definition not in state yet, so it reports unknown until the import lands.
• Is this a terraform-lexicon concern at all, or a chant import backend?

What would turn this into an implementable issue: a decision on the enumeration source (question 2), and one worked end-to-end example on a small AWS root showing generated HCL plus import blocks that terraform plan accepts as a no-op afterwards.

[lex00]

#2105 ships reconcile for the choudoufu backend, which answers this issue for that backend and leaves the stock Terraform question exactly where it stands.

The mechanism is different, so the open questions above do not transfer. On choudoufu ownership is two tags on the live resource (live/MARKERS.md), not an entry in a state file, so cloud-to-code is "claim what the configuration already describes" rather than "write HCL for something the configuration does not mention". live-plan already enumerates the live resources sitting at a declared identity with no marker, and prints the tofu-estate/tofu-address pair that would claim each one, so nothing has to be generated and nothing has to be authored in somebody else's style.

What landed:

• TerraformWatchOp with live: true plans through choudoufuLivePlan, publishes Drift, Unowned and Adoptable off that one live read, and carries the adoption ledger under the plan text in the issue and pull-request bodies.
• TerraformAdoptOp, new: Check (live-check), Ledger (live-plan with -adoption-only), Gate (always), Adopt. The adopt step writes the two markers through the paste-ready tagging command choudoufu itself printed. An address more than one live resource sits at is reported and never adopted, and compensate is refused without a command, since un-adopting is untag.
• live-import is documented on the lexicon's ops page as the one-time migration step that precedes all of this, and is deliberately not wrapped in an Op: it reads a state file once, and chant does not host one.

Against the open questions on this issue:

• Enumeration source: choudoufu's own estate-wide sweep, in the same run as the plan. No cross-lexicon dependency, no aws/gcp/azure live read.
• Where the generated file lands: nothing is generated. The configuration is already there; the write is a tag on the resource.
• Ownership answer: it holds immediately. The marker is the ownership answer, so an adopted resource reads owned on the next plan with no import to land first (terraform: observe on choudoufu — describeResources() over live-plan and live-ls, ownership from the marker tags #2104 covers describeResources() on a live root).

Stock Terraform and OpenTofu are unchanged by this. terraform plan -generate-config-out with import {} blocks is still the candidate path there, and questions 1, 2, 3 and 5 above are still the ones that would have to be settled first. This issue stays open for that.

One caveat on the choudoufu side: end-to-end adoption against a real estate waits on INTENTIUS/choudoufu#894, which makes live-plan -json reachable on a configuration that declares its own estate. The emulator acceptance test is written and skips with that reason until it lands.

[lex00]

Research spike done. The worked example ran for real against floci (floci/floci:latest on :4597, Terraform v1.15.8, hashicorp/aws v5.100.0): a bucket created outside Terraform, an import {} block naming it, terraform plan -generate-config-out=generated.tf, apply, and then terraform plan -detailed-exitcode exiting 0. Full write-up, with the five questions answered against file:line evidence and every command and output recorded, is in docs/design/terraform-reconcile-spike.md on branch spike/2089-terraform-reconcile (committed, not pushed).

Recommendation

Do not implement the stock-root generator yet. The example proves the second half works end to end (import {} plus -generate-config-out plus apply gives a clean no-op plan) and proves the first half is missing: nothing in chant produces (address, id) pairs for an undeclared AWS resource, and the AWS exporter (lexicons/aws/src/import/live-export.ts:15) reads a CloudFormation stack a Terraform estate does not have. The root-module restriction makes the generated file useless for a modular estate without a human edit, capping the value of automating it at "saves typing the attribute list". Implementing now means either an account-wide Cloud Control sweep that belongs to the aws lexicon, or vendoring terraformer's naming, both larger decisions than this issue. Two pieces are worth landing regardless: write down that a stock plan reports unmanaged resources as no drift at all, and give reconcilePr the ability to open a PR over files a caller already wrote, which the generator would need and which TerraformWatchOp's pull-request mode wants today. Revisit when the aws lexicon has an account-scope enumerator, or when Terraform lifts the root-module restriction.

Three findings that drove that, all reproduced:

• terraform plan -detailed-exitcode exits 0 on a root with an unmanaged bucket sitting next to it. Unmanaged is not drift, it is nothing, so enumeration cannot come from the plan.
• import { to = module.buckets.aws_s3_bucket.in_module } fails generation outright: "Only resources within the root module are eligible for config generation." And a second run at the same path fails with Target generated file already exists.
• Ownership holds. classifyStateOwnership (lexicons/terraform/src/describe-resources.ts:241) is index.rows.has(address) ? "owned" : "unknown"; with the generated block committed but the import not applied, terraform show -json has no row for the address, so it reads unknown, and after apply the same call reads owned.

Proposed sub-issues

1. docs(terraform): a stock plan is silent about unmanaged resources. Acceptance: the observation page states that plan -detailed-exitcode exits 0 for a live resource no state row covers, that describeResources cannot see it either, and names import {} as the only cloud-to-code route on a stock root.
2. core(op): reconcilePr commits files the caller wrote instead of running chant import. Acceptance: a mode that commits named paths and opens the PR, existing callers unchanged, with a test pinning that chant import is not invoked in it.
3. aws: account-scope enumeration through Cloud Control listResources. Acceptance: listResources (lexicons/aws/src/api/read-client.ts:517) reachable unscoped over a declared type list, paginated and rate-limited, returning type plus primary identifier, with no terraform dependency.
4. terraform: terraformGenerateConfig activity (blocked on 2 and 3). Acceptance: given (address, id) pairs it writes imports.tf, runs plan -generate-config-out, refuses a non-root-module address with Terraform's own message, and returns both paths, with an emulator test reproducing the no-op plan after apply.

@lex00 should this land in-tree under docs/design/, alongside 365-typed-crd-status-readiness.md and committed-encrypted-sops-provenance.md? It is committed on the spike branch and nothing is pushed yet.