Skip to content

BAN-148: Fix Sonnet continuing conversation instead of summarising - #4

Merged
RemoteCTO merged 2 commits into
mainfrom
BAN-148
Feb 16, 2026
Merged

RemoteCTO merged 2 commits into
mainfrom
BAN-148

Conversation

@RemoteCTO

@RemoteCTO RemoteCTO commented Feb 16, 2026 •

Copy link
Copy Markdown
Owner

Sonnet treated USER:/ASSISTANT: transcript entries as a
conversation to continue, producing ~700 char in-character
responses instead of structured summaries. Both retry attempts
failed validation, leaving sessions unresumable.

Root cause: The system prompt and input framing were too
weak — Sonnet couldn't distinguish between "here is a
conversation to summarise" and "here is a conversation to
continue".

Changes

  • System prompt reframed as "SUMMARISATION TOOL" with
    explicit anti-roleplay instructions. Conditional: uses
    generic format instructions when SEAMLESS_PROMPT_FILE
    is active to avoid heading conflicts with custom prompts.
  • Transcript wrapping in <transcript> XML tags creates
    a clear data boundary between instructions and input.
  • Closing instruction appended after the transcript:
    "Produce the structured summary now."
  • User prompt restructured: numbered section prefixes
    removed (### 1. → ## ), explicit "include ALL six
    sections" rule, 2000 char minimum target.
  • Plugin manifest version synced with package.json.

Test plan

  • Manual compaction of failed session db080aef —
    produced 15,735 chars valid output (was 676/716 chars)
  • All 158 existing tests passing
  • Biome lint clean
  • Validated against real 75K char infrastructure
    transcript (326 entries)

Sonnet treated USER:/ASSISTANT: transcript entries as a conversation
to continue, producing ~700 char in-character responses instead of
structured summaries. Three changes prevent this:

- System prompt reframed as "SUMMARISATION TOOL" with explicit
  anti-roleplay instructions
- Transcript wrapped in <transcript> XML tags to create a clear
  data boundary between instructions and input data
- Closing instruction added: "Produce the structured summary now"
Copilot AI review requested due to automatic review settings February 16, 2026 08:01

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a critical bug where Claude Sonnet was treating the conversation transcript as an ongoing conversation to continue rather than data to analyze, resulting in ~700 character in-character responses instead of comprehensive structured summaries. The fix implements three defensive measures: reframing the system prompt as a "SUMMARISATION TOOL" with explicit anti-roleplay instructions, wrapping the transcript in XML tags to create a clear data boundary, and adding a closing instruction to produce the summary.

Changes:

  • Updated system and user prompts with anti-roleplay instructions and explicit section requirements
  • Wrapped transcript input in <transcript> XML tags instead of using bare text after a separator
  • Version bumped to 0.2.1 with corresponding CHANGELOG entries

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
scripts/compactor.mjs Updated PROMPT and SYSTEM_PROMPT with anti-roleplay instructions; wrapped transcript in XML tags; added closing instruction; removed section numbering
package.json Version bump from 0.2.0 to 0.2.1
.claude-plugin/plugin.json Version sync from 0.1.1 to 0.2.1 to match package.json
CHANGELOG.md Added comprehensive 0.2.1 release notes documenting the bug fix and changes

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread CHANGELOG.md
Comment thread scripts/compactor.mjs Outdated
System prompt now uses generic format instructions when a custom
prompt file is active, avoiding conflict between hardcoded heading
names in the system prompt and different sections in the user's
custom prompt.
@RemoteCTO
RemoteCTO merged commit f67ec94 into main Feb 16, 2026
2 checks passed
@RemoteCTO
RemoteCTO deleted the BAN-148 branch February 16, 2026 08:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants