[open-actions-self-update] Prompt Tuning: open-actions-config-update routes implementation lessons to an AgentConfig no implementing agent loads #1626

🤖 Kelos Agent @gjkim42

Area

Prompt Tuning

Problem and impact

open-actions-config-update is the only loop that turns Open Actions PR review feedback into agent configuration. Its prompt calls self-development/open-actions/agentconfig.yaml the "shared Open Actions role instructions" and sends "Shared changes (affect all Open Actions agents)" there. The fleet is not wired that way:

• open-actions-dev-agent (agentconfig.yaml) is loaded only by open-actions-triage and open-actions-squash-commits. Triage is read-only ("Do NOT open PRs, push code, or modify any files"), and squash-commits must "NOT modify any source code".
• Reviewed implementation output comes from authors that never load it. 35 of the 40 most recent Open Actions PRs (#145–#193) came from maintainer-named branches such as fix/action-download-retries, including every PR from Support opencode #184 through Documentation: Remove duplicate AGENTS.md file #193. None of local-run.sh may remove CRDs #185–Documentation: Remove duplicate AGENTS.md file #193 received a /kelos pick-up, and no Kelos AgentConfig reaches those authors. The other 5 (#148, #156, #157, #166, #183) use the open-actions-workers Session's open-actions-task-<N> branches. That Session and open-actions-pr-responder load only base-agent and take Open Actions' conventions from AGENTS.md in the kelos-dev/open-actions checkout. Both now run Codex, which writes base-agent to $CODEX_HOME/AGENTS.md and reads the checkout's AGENTS.md directly. Their base-agent-only agentConfigRefs date from the fleet's creation in b169c1a (2026-08-09), and internal/examples/self_development_test.go pins it. open-actions-reviewer runs on the maintainer's /kelos review on any open PR, so it is the one configured agent that sees every implementation.
• The planner, reviewer, API reviewer, fake-user, and fake-strategist each load their own inline AgentConfig.

So the most common kind of review lesson, one about implementation output, lands in a file that no implementing agent reads. The PR looks like a fix but changes nothing for the agents that made the mistake. The loop also uses that file to decide a finding is "already covered" and adds nothing more. The prompt has no path to withdraw misrouted changes either, so they keep piling up in one open PR.

Evidence from the workflows and agent activity

• The misrouted PR keeps growing. PR #1627 has been open and unassigned since 2026-08-09. It now has 16 commits, and the last two repeat the pattern. cfb2fa77 (2026-10-06T18:03Z) extends the agentconfig.yaml Console bullet after the open-actions#192 review (P3) found that automatic status refreshes drop keyboard focus and text selection. docs: Add comprehensive README for self-development examples #192 came from the maintainer branch fix/console-status-refresh, and its author fixed the finding before merge (ece583d). de591892 (2026-10-13T18:07Z) extends the producer/consumer test bullet after the open-actions#193 review suggested building a runner test's executor with NewExecutor instead of assigning executor.logger. Documentation: Remove duplicate AGENTS.md file #193 came from the maintainer branch fix/action-download-retries. Its author force-pushed the fix four minutes after the review (78a1e9a), so the merged test already uses NewExecutor and asserts the Console's open_actions_runner marker. The Improve Open Actions compatibility reviews and implementation guidance #1627 description reasons that "The code reviewer caught each one, so only the implementation guidance changes", so the only edit for this lesson went to a file no implementer loads. Its diff adds 25 implementation bullet lines to agentconfig.yaml (2 of them rewrite existing bullets), covering Console form and keyboard tests, controller watch handlers, Git semantics, Helm upgrades, matrix identities, and more. It also changes open-actions-reviewer.yaml by +20/−4 lines and open-actions-api-reviewer.yaml by +6/−1. Its description says "Shared implementation guidance lives in agentconfig.yaml". Of all that, only the reviewer and API-reviewer lines reach an agent that acts on them.
• "Already covered" points at guidance nobody loads. The Improve Open Actions compatibility reviews and implementation guidance #1627 description skips new rules for Support gemini cli #172/notice the user before overriding the existing config, credentials #176's Console form testing, Support gemini CLI agent type #173's watch filtering, and CLI UX: Missing client-side validation for enum flags (--type, --credential-type) #182's non-idempotent retries because "existing guidance already covers" them. That guidance exists only in agentconfig.yaml. Commit 2cda9401 records that the reviews of open-actions#183 and #187 "found the same defect in the Console Run workflow form". The loop responded by extending the agentconfig.yaml bullet again.
• Merged rules did not reach the implementers. #1625, a config-update PR merged 2026-08-09, added the two implementation bullets now on main (stale informer reads and tests for stale-state decisions). Eleven days later, the open-actions#93 review flagged a P2 retry-idempotency defect in the rerun path. That PR came from a workers Session (branch open-actions-task-32). Improve Open Actions compatibility reviews and implementation guidance #1627 cites this review as the reason for yet another agentconfig.yaml rule.
• Where a check can reach, it works. Commit 36aa1fc1 turns the open-actions#191 startup lesson into a "Controller efficiency" check in open-actions-reviewer.yaml, which reviews every Session PR. The agentconfig.yaml half of the same commit reaches no implementer.
• Prompt: open-actions-config-update.yaml file map, lines 68–72, routing, lines 98–107, and existing-PR handling, lines 125–130.
• Wiring: open-actions-workers.yaml lines 56–57, open-actions-pr-responder.yaml lines 63–64, open-actions-triage.yaml lines 49–51, open-actions-squash-commits.yaml lines 59–61, and the test pins, lines 227–232.
• The README's How It Works section describes the wiring correctly. Its config-update section, lines 300–307, repeats the prompt's "shared role instructions in agentconfig.yaml" wording.

Proposed change

In self-development/open-actions/open-actions-config-update.yaml:

1. Replace the file-map bullets (lines 70–72) with a map of which agent loads which file:

   • self-development/base-agent.yaml: shared instructions and skills, loaded by every resource
   • self-development/open-actions/agentconfig.yaml: open-actions-dev-agent, loaded only by open-actions-triage and open-actions-squash-commits
   • The inline AgentConfigs in open-actions-planner.yaml, open-actions-reviewer.yaml, open-actions-api-reviewer.yaml, open-actions-fake-user.yaml, and open-actions-fake-strategist.yaml, each loaded only by its own spawner
   • The open-actions-workers and open-actions-pr-responder Sessions load only base-agent. They read Open Actions' conventions from AGENTS.md in kelos-dev/open-actions, which this agent does not edit. PRs the maintainer pushes from their own branches load no file here either

2. Replace the "Shared changes" and "Task-specific changes" blocks (lines 98–107) with routing by the agent that produced the output under review. Keep the base-agent skills rule:

   Route each lesson to the configuration that the agent behind the reviewed output actually loads:

   • Triage or squash-commits behavior: agentconfig.yaml
   • Planner, reviewer, API reviewer, fake-user, or fake-strategist behavior: that spawner's prompt or inline AgentConfig
   • Implementation output (code, tests, docs, commits, PR descriptions), whether from the open-actions-workers or open-actions-pr-responder Sessions or from branches the maintainer pushes: no file here reaches its author. Do not add these rules to agentconfig.yaml, and do not expand the Sessions' minimal initialPrompt. If a review-time check would catch the problem and is not already in open-actions-reviewer.yaml (or open-actions-api-reviewer.yaml for API surfaces), add it there. Otherwise, make no change for that lesson.
   • Treat a recurring finding as already covered only when the covering guidance is in a configuration the producing agent loads.
   • Keep shared skills only in self-development/base-agent.yaml. Do not duplicate them in Open Actions role AgentConfigs.

3. Extend the existing-PR constraint (lines 125–130) with:

   If an existing unassigned PR from this agent contains changes that this routing does not allow, or that the evidence no longer supports, remove them. If no changes remain, close the PR with a comment explaining why.

Then update lines 300–307 of self-development/open-actions/README.md to describe the same routing.

Acceptance criteria

• open-actions-config-update.yaml states which agents load agentconfig.yaml, which load each inline AgentConfig, and which load only base-agent. It no longer says agentconfig.yaml affects all Open Actions agents.
• Lessons are routed by the agent that produced the reviewed output. Rules for implementation output never go to agentconfig.yaml, whether a Session or the maintainer wrote it. They go to open-actions-reviewer.yaml or open-actions-api-reviewer.yaml when a review-time check fits and is not already there.
• A finding counts as "already covered" only by guidance the producing agent loads.
• The agent may remove unsupported changes from its own unassigned PR, and close the PR when nothing remains. The assigned-PR guard, the one-PR-per-run rule, /kind cleanup, and release-note: NONE are unchanged.
• self-development/open-actions/README.md describes the same routing.
• agentConfigRefs do not change, so internal/examples/self_development_test.go needs no update. Only files under self-development/open-actions/ change, and make verify passes.
• Post-merge check: the next open-actions-config-update run removes from PR #1627 the agentconfig.yaml bullets that target implementation output. It keeps or adds reviewer and API-reviewer checks only where the evidence supports them.

Related work and scope

• PR #1627 is the misrouted PR itself. This issue fixes the prompt that produced it. Trimming Improve Open Actions compatibility reviews and implementation guidance #1627 is the expected outcome, not a separate change.
• #1734 proposes the same fix for agora-config-update and explicitly leaves Open Actions out of scope. Using the same routing keeps the two fleets consistent.
• #766 covers kelos-config-update, a different fleet.
• Adding open-actions-dev-agent to the two SessionSpawners is not proposed, for three reasons. The base-agent-only Session wiring is deliberate and pinned by the test. open-actions-dev-agent carries ephemeral-container and bot-identity text that does not fit Sessions running on the maintainer's token with a persistent volume. And its Makefile-target and commit-message rules repeat conventions the Sessions already read from Open Actions' AGENTS.md.
• The two implementation bullets that Update Open Actions agent conventions #1625 merged into agentconfig.yaml stay where they are. Moving them is left to the maintainer.
• Open question for the maintainer: should lessons that no review-time check can catch be proposed for Open Actions' AGENTS.md, for example as a kelos-dev/open-actions issue? The maintainer already edits that file directly (open-actions#177, merged 2026-09-16, changed only AGENTS.md). This proposal leaves that out.
• Also considered, ranked lower:
  • The previous candidate in this slot: make single-slot publication clock-stamped, literal-safe, and idempotent. The fake-user, fake-strategist, and self-update prompts still name no time source for Checked at. However, the last four fake-user refreshes of open-actions#76 (2026-09-29 through 2026-10-02) and every fake-strategist refresh of open-actions#168 through 2026-10-03T00:06:16Z recorded real times. The shell-expansion corruption (2026-08-17 and 2026-09-06) and the duplicate create (open-actions#75/add kind labels #76, 2026-08-14) have not recurred.
  • The reviewer template's **Verdict**: APPROVE / REQUEST CHANGES / COMMENT (sticky comment only) was copied verbatim, (sticky comment only) included, into 3 of the last 10 reviews (open-actions#179, #185, #187). This is cosmetic.
  • The reviewer and API reviewer cite the nonexistent config/crd/bases/. Open Actions' CRDs live in internal/manifests/charts/open-actions/templates/crds. No review has been misled by it yet.
  • Triage created no task for open-actions#180 (2026-09-24), and failed on E2E test anchor issue #117–Speed up image builds by pre-compiling Go binaries on host #119 (2026-08-23). This looks like a delivery or infrastructure problem, not a prompt problem.

Latest verdict

• Status: VALID

• Checked at: 2026-10-26T06:01:20Z

• Evidence: Kelos main moved from b1e0398 to 68257b1. The new commits are agent image bumps and e7e6184 (support files in inline AgentConfig skills, #1841). None of them touch self-development/ or internal/examples/self_development_test.go. 3040ff1 (2026-10-15, models only) is still the last commit to change those paths, and git diff 40dedd4 origin/main over them is still empty, so the 40dedd4 permalinks above still show the current content. open-actions-config-update.yaml still calls agentconfig.yaml the "shared Open Actions role instructions" (line 71) and still sends "Shared changes (affect all Open Actions agents)" there (lines 101–102). Only open-actions-triage and open-actions-squash-commits load open-actions-dev-agent. The open-actions-workers and open-actions-pr-responder Sessions still load only base-agent, and lines 227–232 of the test still pin this. README lines 300–307 still describe "the shared role instructions in agentconfig.yaml". PR #1627 is still open, unassigned, and unreviewed: 16 commits, +51/−7 (agentconfig.yaml +25/−2, open-actions-reviewer.yaml +20/−4, open-actions-api-reviewer.yaml +6/−1), head de591892 (2026-10-13T18:07Z). It is still the only open open-actions-config-update PR. Open Actions main is still 87db1d2. No Open Actions PR has been opened since #193 and no issue since #180, so there is no new review evidence that changes the ranking. Open Actions' AGENTS.md has not changed since 0bb03ef (2026-09-16). Lower-ranked alternatives, rechecked:

  • open-actions-reviewer.yaml line 215 and open-actions-api-reviewer.yaml line 246 still cite config/crd/bases/, which still returns HTTP 404 in Open Actions. The CRDs are still in internal/manifests/charts/open-actions/templates/crds. No review has been misled by it yet.
  • No Open Actions prompt mentions docs/reference.md, which Open Actions' AGENTS.md (line 15) names as the user-facing reference. No recent review is known to have missed a needed update there, so this ranks lower.
  • The reviewer template's (sticky comment only) suffix (line 325) is cosmetic. The fake-user and fake-strategist slots still record real times (open-actions#76 at 2026-10-25T09:00:32Z, open-actions#168 at 2026-10-26T00:12:08Z), so the clock-stamp alternative stays weak.

  No other open generated-by-kelos issue overlaps. #1734 (Agora) and #766 (Kelos) are still open and unassigned, and both cover other fleets.