fix(agent-runtime): a spam verdict withholds the reply instead of sending it - #715
Open
kmonsoe wants to merge 1 commit into
Open
fix(agent-runtime): a spam verdict withholds the reply instead of sending it#715kmonsoe wants to merge 1 commit into
kmonsoe wants to merge 1 commit into
Conversation
…ding it The audit pass runs after generation but before delivery, so mark_spam flipped the conversation to spam and then posted the generated reply anyway. The verdict now gates delivery: the reply is withheld and parked as a draft_reply, so a misclassification is one click from recovery. Adds conv.setDraftReply + POST /v1/conversations/:id/draft-reply to author a draft without requesting a handover. Also lifts the seed prompt's no-redirect rule out of the handover paragraph, where it never bound on replies that weren't handovers. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011HwhYb3yU3qnGjtFRodQxm
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A cold pitch to
hello@getmunin.comgot marked as spam and answered — with a polite note telling the sender to forward it to the very address they had just written to.What was actually broken
Not instruction-following. Control flow.
Spam isn't a tool the agent calls; it's a separate audit pass (
packages/agent-runtime/src/audit.ts) invoked after generation but before delivery. It marked the conversationspamviachangeStatus, returned, and then the handler unconditionally posted the reply it had already generated. The model did what it was told; the runtime ignored its own verdict.shouldRespondalready skips non-open conversations, so the leak was only ever the in-flight turn.The fix
runAuditPassnow reports the spam verdict alongside the handover reason, and delivery is gated on it. On spam the reply is withheld and parked as adraft_replyinstead of discarded — so if the classification was wrong, the reply already exists and a human sends it with one click. No regeneration, and a human looks at it before it goes out, which matters precisely because the trigger is a classifier that just proved unreliable.Parking needs a way to author a draft without requesting a handover (
requestHandoversetsneedsHumanAttention, which contradicts "this is spam"), soconv.setDraftReplyandPOST /v1/conversations/:id/draft-replyare new. The endpoint replaces any existing draft rather than stacking, and pre-checks the conversation so an unknown id is a 404 rather than a poisoned outer transaction.Prompt
system.tsalready forbade redirecting people off-channel and volunteering contact details — but the paragraph opened with "When you flag a conversation for a human," so the model read the whole rule as conditional on handover. The pitch reply wasn't a handover, so nothing bound.That rule is now unconditional, and adds the bit that is genuinely new: never name a contact address the sender has already written to — inbound mail has by definition already reached the right inbox. Plus a short rule that pitches get a two-sentence decline with no redirect, or a handover if they look like real business.
Follow-up not in this PR
Live orgs keep whatever prompt is already in their KB —
ensureDocumentonly seeds when the document is absent, so this reaches new orgs only. Prod, dev and local each need a manual KB edit.Testing
pnpm -F @getmunin/agent-runtime test— 302 pass, including three new cases: spam withholds and parks, a non-spam audit action still sends, and a failed park still withholds.pnpm -F @getmunin/backend-core test— 1339 pass, including two new integration tests for the endpoint (replace-not-stack, 404 on unknown id).pnpm typecheck,lint,openapi:generateclean.🤖 Generated with Claude Code
https://claude.ai/code/session_011HwhYb3yU3qnGjtFRodQxm