Skip to content

fix(engine): name a notice's section once, and report what a refused entry left unattempted - #592

Merged
Vivswan merged 1 commit into
mainfrom
fix/engine-messages
Nov 12, 2026
Merged

Vivswan merged 1 commit into
mainfrom
fix/engine-messages

Conversation

@Vivswan

@Vivswan Vivswan commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Before

section plan -> operation 1 refused -> stop -> operations 2..n dropped, nothing said about them
$ mode: apply, environments: alpha (live wait_timer 30, the entry omits it), beta (variable B, secret TOKEN)
::notice::environments: environments[beta].secrets: beta environment secret values cannot be read back from GitHub, ...
::error::environments: environments[alpha]: not applied - the update would remove a live value the settings file omits. ...
result: failed

$ mode: check, repository.enable_git_lfs, one Actions secret, teams on a personal account
::notice::repository: repository.enable_git_lfs: GitHub exposes no endpoint to read this state back, ...
::notice::actions_secrets: actions_secrets: Actions secret values cannot be read back from GitHub, ...
drift: actions_secrets[A]: missing - declared in the settings file but not on the repo; apply will create it
::notice::teams: teams: owner "e2e-owner" is a personal account, not an organization, ...
result: drift

$ mode: apply, actions_secrets: [{name: A, value: "Pa$$W0RD_SECRET9"}]
::error::/repo/settings.yml has malformed section entries: actions_secrets: the secret entry "A" embeds $W0RD_SECRET9 without being a whole-value reference; ...
result: failed

After

section plan -> operation 1 refused -> stop -> operations 2..n named: one warning with the count, one line each
$ mode: apply, environments: alpha (live wait_timer 30, the entry omits it), beta (variable B, secret TOKEN)
::notice::environments[beta].secrets: beta environment secret values cannot be read back from GitHub, ...
::error::environments[alpha]: not applied - the update would remove a live value the settings file omits. ...
::warning::environments: 2 operations after this failure were not attempted; re-run once it is fixed
environments: not attempted - creating variable "B" in environment "beta"
environments: not attempted - writing secret "TOKEN" in environment "beta"
result: failed

$ mode: check, repository.enable_git_lfs, one Actions secret, teams on a personal account
::notice::repository.enable_git_lfs: GitHub exposes no endpoint to read this state back, ...
::notice::actions_secrets: Actions secret values cannot be read back from GitHub, ...
drift: actions_secrets[A]: missing - declared in the settings file but not on the repo; apply will create it
::notice::teams: owner "e2e-owner" is a personal account, not an organization, ...
result: drift

$ mode: apply, actions_secrets: [{name: A, value: "Pa$$W0RD_SECRET9"}]
::error::/repo/settings.yml has malformed section entries: actions_secrets: the secret entry "A" embeds a $NAME reference without being a whole-value reference; ...
result: failed

How

  • One owner: underKey holds the rule, and every site whose line may already lead with the key calls it.
  • Engine-worded lines: a line the engine words itself cannot lead with the key, so it prefixes by hand.
  • Failure lines: they lose the repeated section key too, in apply, check, and snapshot.
  • The stop stays: a later operation can depend on the one that failed.
  • Names: an operation's own describe, else its drift lines, else its role.
  • Skipped sections: list nothing, since the skip covers every operation.
  • Secret error: names the entry and the rule, and echoes no part of the value.

Proof

  • Tests: bun test test/engine, 404 pass, 0 fail. Seven cases are new, each seen failing first.
  • Re-expected: two notice pins, two pins of the secret message, and every whole-value pin of a failed execution, which gained unattempted.
  • e2e: bun test/e2e/run.ts --sections environments,rulesets,teams,repository,actions_secrets,branches,collaborators,workflows, 182/182.
Technical details
  • Capture: the built bundle, run by a scratch script over the e2e runner's parseScenario and runScenario, Before on the branch's base commit, After on this branch.
  • First run: two environment entries, live alpha with wait_timer: 30, live beta with no protection rules.
  • Second run: repository.enable_git_lfs: true, one team, one Actions secret, owner_kind: user.
  • Third run: the settings file as written above, which the parse refuses before any request.
  • A trailing ...: cuts a line whose tail is the same in both states.
  • /repo/settings.yml: stands for the runner's temporary settings file.
  • Writes: the first run sends no write request in either state.
  • Summary table: each not attempted - ... line is also one line of the section's Detail cell.
  • Long plans: a plan of n operations that fails on its first prints n - 1 lines, as many as a success prints change lines, so there is no cap.
  • Redaction: the lines go through the target's own sink, and a test runs a failing plan through a redacted channel and finds none in the clear.
  • Public surface: no exported type changed, and SectionOutcome.detail gains rows, not members.
  • Hand-prefixed sites: preflightProbe, secretFailure, reportUnattempted, and the skipped - ..., not applied - ..., and not snapshotted - ... wrappers in orchestrate.ts and snapshot.ts.
  • Where underKey lives: src/engine/orchestrate.ts, beside the outcome functions, imported by snapshot.ts.
  • Re-expected, notices: test/engine/orchestrate.test.ts pinned notice: workflows: workflows[missing.yml] twice.
  • Re-expected, secret message: test/engine/{secret-refs,validate}.test.ts pinned the echoed fragment.
  • Re-expected, execution shape: eight assertions in test/engine/execute.test.ts and one in test/sections/branches/branches.test.ts.
  • Census of the doubled key: a multi-line probe for <key>: <key> over test/ and docs/, the 372 scenario files included, hit four lines and no scenario.
  • Its hits: the two notice pins, the branches message below, and one sentence that is no prefix (repository.custom_properties: custom_properties is reported ...).
  • Its control: the same probe hits all four planted forms, a folded YAML scalar included.
  • Left as is: the branches section writes its own branches: branches[main]: ... message in section code.
  • Files: src/engine/{execute,orchestrate,snapshot,secret-refs}.ts, test/engine/{execute,orchestrate,snapshot,secret-refs,validate}.test.ts, test/sections/branches/branches.test.ts.

…entry left unattempted

A section's own lines already lead with its key, and the section loop prefixed them again, so a run printed "teams: teams: ...", "repository: repository.enable_git_lfs: ...", and "environments: environments[alpha]: not applied - ...". The rule that prefixes a line once was private to the snapshot flow. It now sits beside the section-outcome functions as underKey, and every site that prefixes a section's line calls it: notes, change lines, and failure lines, in apply, check, and snapshot.

The first failed operation ends its section's plan, since a later operation can depend on an earlier one, and nothing said what that cost: a refused environment entry dropped every later entry's writes without a word. The stop stays. The executor now returns the operations it never reached, each named by its own description, else by the drift it would have resolved, else by its role. The failed outcome prints the count as a warning, then one log line and one summary row per operation. A section skipped under on-missing-permission: warn lists nothing, because the skip covers every operation of the section.

The embedded-reference error quoted the text it matched, which in a literal such as "Pa$$W0RD_SECRET9" is part of the secret. It now names the entry and the rule and echoes no part of the value.
Copilot AI balanced review requested due to automatic review settings November 12, 2026 01:35
@github-actions

Copy link
Copy Markdown
Contributor

File size check

0 over a hard cap (fails), 6 warning(s).

File Size Tier Cap
src/engine/diff.ts:56 11 comment lines warn 10
src/github/api-error.ts:9 11 comment lines warn 10
src/problem.ts:73 18 comment lines warn 10
src/sections/contract/errors.ts:13 11 comment lines warn 10
src/sections/contract/module.ts:55 12 comment lines warn 10
src/sections/shared/list/decl.ts:265 14 comment lines warn 10

Split the file, wrap the line, shorten or exempt the comment, or list the path in .file-size-allow.local with a # reason.

5 managed file(s) skipped; repo-platform owns them.

@github-actions

Copy link
Copy Markdown
Contributor

Member shape check

1 finding(s).

File Line Rule
test/sections/contract/validate-hook.test.ts 80 all-or-none

Add the blank lines, make the comment one line directly above the item, comment every field or none, or list the path in .member-shape-allow.local with a # reason.

4 generated file(s) skipped.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the failure-reporting and secret-redaction changes.

No open issues 🎉

What changed in this PR

This PR improves failure reporting in the settings engine by naming operations left unattempted, removing repeated section prefixes, and preventing secret fragments from appearing in validation errors.

Changes:

  • Report remaining operations after a failure without changing stop-on-failure behavior.
  • Share section-prefix handling across apply, check, and snapshot reporting.
  • Remove embedded secret fragments from validation errors and add regression coverage.
File Description
test/​sections/​branches/​branches.test.ts Update failed-execution expectations with unattempted operations.
test/​engine/​validate.test.ts Expect secret validation errors without echoed fragments.
test/​engine/​snapshot.test.ts Cover single-prefix snapshot failures.
test/​engine/​secret-refs.test.ts Verify rejected secret values are not echoed.
test/​engine/​orchestrate.test.ts Cover prefixes, unattempted reporting, permission skips, and redaction.
test/​engine/​execute.test.ts Verify remaining-operation names, ordering, and stopped execution.
src/​engine/​snapshot.ts Reuse shared section-prefix handling.
src/​engine/​secret-refs.ts Stop echoing embedded reference fragments in errors.
src/​engine/​orchestrate.ts Centralize prefixes and report unattempted operations in logs and outcomes.
src/​engine/​execute.ts Capture remaining operation names when execution fails.

Review effort: Balanced

@Vivswan
Vivswan marked this pull request as ready for review November 12, 2026 01:56
@Vivswan Vivswan added the merge-when-green Owner approved: merge once every gate is green label Nov 12, 2026
@Vivswan
Vivswan merged commit ee702cd into main Nov 12, 2026
35 checks passed
@Vivswan
Vivswan deleted the fix/engine-messages branch November 12, 2026 04:42
This was referenced Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-when-green Owner approved: merge once every gate is green

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants