Fix delegate_editor: writes-vs-claim verification guard (raffle-app fabricated-completion incident) - #154
Merged
Merged
Conversation
…hout duplicating the claim-extraction logic
…writes-vs-claim guard
…or the fabricated-completion-report failure mode)
…guard, same pattern as agent_delegate.js)
…inal result, not just buried in the transcript
… final answer, and surface fallbackModelUsed on the persisted/returned result
…t too, for a polling caller
…s -- a successful draft answer now gets one extra verification round-trip (tools re-enabled) before being trusted, same pattern as agent_delegate.js's own pendingVerification
…ynchronous-run checkpoint test
…es a fabricated completion report (the raffle-app incident's exact failure shape), lets the model self-correct by actually writing during the verification round, stays single-fire when the model insists, and surfaces fallbackModelUsed on the final result
…aim heuristic The prompt used to unconditionally tell the model "you still have tool access this turn" even when it might not (the final step / stuck-loop force withholds tools). If the verification round itself lands on a tools-withheld step and the model believes the prompt and tries to write_file anyway, the loop discards the whole run as failed. Now the prompt's wording matches whether tools are actually available. Also adds a small heuristic to detect a completion-sounding answer, used next to gate a no-tools fallback verification path and a final safety check on the way out.
…o skip entirely Previously the guard only fired when `!withholdTools && step < cappedSteps` -- meaning a fabricated completion claim landing on the run's actual last step (tools withheld, no budget for another loop iteration) bypassed the guard completely. That's the most likely trigger shape (model burns its whole step budget reading, then has to answer with no tools left) and matches the original raffle-app incident if its fabricated summary landed on the final allowed step. Now: when the normal tools-enabled corrective round isn't possible, but writtenFiles is empty and the answer looks like a completion claim, run one inline no-tools corrective call instead of silently trusting the draft. And regardless of which path was taken (or none, if pendingVerification was already true and the model just repeats the same claim), a final safety check flags -- rather than silently accepts -- a still-completion- claiming, zero-write answer on the way out.
…w flagged, not silently accepted Also add coverage for the two gaps found in review: a fabricated completion claim landing on the run's actual final step (previously skipped the guard entirely), and an honest "nothing needed" answer with zero writes correctly NOT being flagged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Incident
A
delegate_editorrun onallocsys/raffle-app(Telegram Stars payment confirmation) took 9read_filesteps, wrote nothing, then produced a confident, detailed completion report — specific functions and files claimed updated, none of which existed in the diff. Root cause: every step was served by the fallback modelgemini-3.5-flash-lite(primary Gemini unavailable), and the loop's completion path trusted the model's own final text with zero cross-check againstwrittenFiles.Fix
writtenFilesand any tool-result text already gathered this run.agent_delegate.js:extractMechanicalClaims/findUnverifiedClaims(provider-agnostic, proven on Gemini) and the single-firependingVerificationpattern (tools re-enabled, not withheld, for the corrective round).UNVERIFIED_COMPLETION_CLAIM:rather than passed through clean.fallbackModelUsedsurfaced on the final result (and on checkpoint resumes), not just buried in the transcript.Explicitly out of scope:
detectToolCallLeakage, step/char caps,reasoningEffortoverride — all bai-specific, not applicable to this Gemini failure mode.Testing
fallbackModelUsedsurfaced correctlyCI (
verify) is green on this branch.