Skip to content

develop action: update test plan checklist in PR description after PR creation #88

Description

@monkut

Problem

When the develop action implements an issue and opens a PR, the PR description includes a ## Test plan checklist (generated by Claude Code's standard PR template). However, the develop action does not check off any of those items — even when its own verification gate (tests, lint, type checker) and the new tests it wrote already satisfy them.

The pr-review action already updates the test plan (see askcc/definitions.py:496-503), but by then the PR has been sitting with an entirely unchecked test plan, which misrepresents the actual state of verification at PR creation time.

Expected behaviour

After develop opens the PR, it should:

  • Read the PR description with gh pr view <number> --json body -q .body
  • Find the ## Test plan section (if present)
  • For each - [ ] item, decide if it is already satisfied by the implementation, new tests, or the verification gate run
  • Replace - [ ] with - [x] for satisfied items
  • Leave items requiring manual/external verification (browser QA, deployment validation, etc.) unchecked
  • Update the PR body with gh pr edit <number> --body "<updated body>"
  • Skip silently if no ## Test plan section exists

Actual behaviour

develop opens the PR with the test plan items left as - [ ] regardless of what was already verified locally.

Suggested fix

Mirror the test-plan-update logic already present in REVIEWPR_AGENT_PROMPT (askcc/definitions.py:496-503), but for DEVELOP_AGENT_PROMPT:

  1. Modify the "On completion" block to add a "Update the test plan checklist in the PR description" step between PR creation and the issue comment.
  2. Append a new "Test plan update" section explaining the read → mark → edit flow, with the same conservative rule: only check off items genuinely satisfied by what was just done.

Add tests in tests/test_askcc.py mirroring TestReviewprPromptMergeGuard to assert the new strings exist in DEVELOP_AGENT_PROMPT.

Acceptance Criteria

  • DEVELOP_AGENT_PROMPT instructs the agent to update test plan items in the PR description after PR creation
  • Instruction explicitly says to leave items requiring manual/external verification unchecked
  • Instruction is a no-op when no ## Test plan section exists
  • Unit tests assert the new instruction strings are present in DEVELOP_AGENT_PROMPT
  • uv run pytest, uv run ruff check, and uv run pyright all pass

Dependencies

None identified. Self-contained prompt + test change in askcc/definitions.py and tests/test_askcc.py. Mirrors the pattern established by issue #86 / PR #87 which added the CHANGES_REQUESTED guard to REVIEWPR_AGENT_PROMPT with TestReviewprPromptMergeGuard.

Implementation Plan

1. Update the "On completion" step list in DEVELOP_AGENT_PROMPT

File: askcc/definitions.py, lines 281–285.

Insert a new bullet between the PR-push step and the issue-comment step so the order becomes:

  1. Run /simplify or /refactor.
  2. Commit, push the feature branch, and open a PR linked to the issue.
  3. Update the test plan checklist in the PR description (see "Test plan update" below).
  4. Add an issue comment summarizing what was implemented.

This creates a forward reference to the detailed section added in step 2.

2. Append a "Test plan update" section to DEVELOP_AGENT_PROMPT

File: askcc/definitions.py, immediately after the existing "On completion" block (after line 285).

Mirror the structure used in REVIEWPR_AGENT_PROMPT (lines 496–503), adjusted to the develop context:

Test plan update:
- After opening the PR, read the PR description with `gh pr view <number> --json body -q .body`.
- Look for a `## Test plan` section containing checklist items (`- [ ]` checkboxes).
- For each test plan task, decide whether it is satisfied by the implementation,
  the new tests you wrote, or the verification gate run (pytest/ruff/pyright).
- Check off satisfied tasks by replacing `- [ ]` with `- [x]` in the PR body.
- Leave items requiring manual/external verification (browser QA, deployment
  validation, etc.) unchecked.
- Update the PR description with `gh pr edit <number> --body "<updated body>"`.
- If no `## Test plan` section exists, skip this step silently.

The wording must contain the literal substrings the tests will assert on (see step 3): the heading Test plan update, the no-op clause If no \## Test plan` section exists, skip this step, and the manual-verification clause Leave items requiring manual/external verification ... unchecked`.

3. Add a TestDevelopPromptTestPlanUpdate test class in tests/test_askcc.py

File: tests/test_askcc.py, immediately after TestDevelopPromptTddContent (line 251–279) and before TestReviewprPromptMergeGuard (line 282), to keep develop-related tests grouped.

Mirror TestReviewprPromptMergeGuard (lines 282–298) with develop-specific assertions:

class TestDevelopPromptTestPlanUpdate:
    """The develop prompt must update the PR test plan after PR creation (issue #88)."""

    def test_includes_test_plan_update_section_heading(self):
        assert "Test plan update:" in DEVELOP_AGENT_PROMPT

    def test_includes_pr_body_read_command(self):
        assert "gh pr view <number> --json body -q .body" in DEVELOP_AGENT_PROMPT

    def test_includes_pr_body_edit_command(self):
        assert 'gh pr edit <number> --body "<updated body>"' in DEVELOP_AGENT_PROMPT

    def test_includes_manual_verification_carveout(self):
        assert "manual/external verification" in DEVELOP_AGENT_PROMPT
        assert "unchecked" in DEVELOP_AGENT_PROMPT

    def test_includes_noop_when_no_test_plan_section(self):
        assert "If no `## Test plan` section exists, skip this step" in DEVELOP_AGENT_PROMPT

    def test_completion_step_references_test_plan_update(self):
        assert "Update the test plan checklist in the PR description" in DEVELOP_AGENT_PROMPT

No new imports required — DEVELOP_AGENT_PROMPT is already imported (line 15).

4. Run the verification gate

uv run pytest tests/test_askcc.py -k "TestDevelopPromptTestPlanUpdate"  # new tests pass
uv run pytest                                                           # full suite green
uv run ruff check
uv run pyright

5. Open PR

Branch: feature/88-develop-update-pr-test-plan (or similar — develop action picks the slug). PR title should reference issue #88. Standard ## Verification and ## Key Flows sections per DEVELOP_AGENT_PROMPT requirements; the change is small enough that the Key Flows diagram can be skipped (single-string-edit + new test class).

Risks / open questions

  • String drift risk: tests assert on literal substrings. If a future edit rewords the prompt, tests will fail loudly — that is the intended behaviour (per the TestReviewprPromptMergeGuard precedent), but document it in the test class docstring so reviewers don't "fix" the assertions away.
  • Step ordering: the new test plan update step runs after PR creation but before the issue comment, so the issue comment can also confirm the test plan was updated. The plan reflects this ordering.
  • No behavioural code change: this is a prompt-only change — no Python logic in askcc/functions.py is affected, and no new CLI flags are introduced.

Activity

  1. self-assigned this
    on Apr 27, 2026
  2. monkut commented on Apr 27, 2026

    @monkut
    OwnerAuthor

    Plan summary

    Updated the issue description with finalized Acceptance Criteria, Dependencies, and a step-by-step Implementation Plan. Assigned to @monkut.

    Approach

    Prompt-only change in askcc/definitions.py mirroring the pattern from PR #87 (CHANGES_REQUESTED guard):

    1. Insert a new "Update the test plan checklist in the PR description" bullet into the DEVELOP_AGENT_PROMPT "On completion" block (between PR push and issue comment).
    2. Append a "Test plan update:" section after the "On completion" block — same read → mark → edit flow as REVIEWPR_AGENT_PROMPT (lines 496–503), with explicit carveouts for manual/external verification items and a silent no-op when no ## Test plan section exists.
    3. Add TestDevelopPromptTestPlanUpdate in tests/test_askcc.py (placed between TestDevelopPromptTddContent and TestReviewprPromptMergeGuard) asserting on the new literal substrings.

    Files touched

    • askcc/definitions.py — DEVELOP_AGENT_PROMPT (lines 281–285 and a new appended block)
    • tests/test_askcc.py — new TestDevelopPromptTestPlanUpdate class

    Verification

    uv run pytest, uv run ruff check, uv run pyright — all must pass.

    Risks / open questions

    • Test assertions are literal-substring matches; intentional brittleness so prompt-wording changes surface in CI (same trade-off TestReviewprPromptMergeGuard already accepts).
    • No behavioural Python code is changed and no new CLI flags are introduced — scope stays inside the two files above.

    :tokens-used: model: N/A, input: 17, output: 7945

  3. added
    action:developIssue is ready for development
    and removed
    action:planIssue is ready for planning
    on Apr 27, 2026
  4. monkut commented on Apr 27, 2026

    @monkut
    OwnerAuthor

    Implementation summary

    Implemented in PR #90 (branch feature/88-develop-update-pr-test-plan).

    Changes

    • askcc/definitions.py — DEVELOP_AGENT_PROMPT:
      • Added the bullet Update the test plan checklist in the PR description (see "Test plan update" below). to the "On completion" block, between PR creation and the issue-comment step.
      • Appended a new "Test plan update:" section mirroring REVIEWPR_AGENT_PROMPT (lines 496–503): read with gh pr view <number> --json body -q .body, check off - [ ] items satisfied by the implementation/new tests/verification gate, leave manual/external verification items unchecked, edit with gh pr edit <number> --body "<updated body>", and skip silently if no ## Test plan section exists.
    • tests/test_askcc.py — added TestDevelopPromptTestPlanUpdate (between TestDevelopPromptTddContent and TestReviewprPromptMergeGuard) with six literal-substring assertions covering the new heading, read/edit gh commands, manual-verification carveout, no-op clause, and the completion-step bullet. Class docstring documents the intentional brittleness (mirroring the TestReviewprPromptMergeGuard precedent).

    Acceptance criteria

    • DEVELOP_AGENT_PROMPT instructs the agent to update test plan items in the PR description after PR creation
    • Instruction explicitly says to leave items requiring manual/external verification unchecked
    • Instruction is a no-op when no ## Test plan section exists
    • Unit tests assert the new instruction strings are present in DEVELOP_AGENT_PROMPT
    • uv run pytest (192 passed), uv run ruff check (clean), and uv run pyright (0 errors) all pass

    No behavioural Python code was changed — prompt-only edit plus test class, exactly as scoped in the implementation plan.

    :tokens-used: model: N/A, input: 39, output: 9926

  5. added and removed
    action:developIssue is ready for development
    on Apr 27, 2026
  6. added a commit that references this issue on Apr 27, 2026
    92817b4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions