fix(translation): stop same-format replay from dropping router additions - #321
fix(translation): stop same-format replay from dropping router additions#321sabhatinas wants to merge 2 commits into
Conversation
A same-format hop replays the exact inbound body, which is what keeps it lossless. That replay was unconditional, so anything a routing algorithm added to the request IR after decoding never reached the wire: the codec returned the body captured before the addition. stage_router's efficient_system_prompt was silently dropped whenever the inbound request and the selected target shared a format, and handoff notes went the same way. All three buffered codecs had the same short-circuit, so this was not OpenAI-specific — an Anthropic-inbound request routed to an Anthropic target dropped its prompt too, and Responses likewise. Cross-format hops escaped only by accident: the preserved body is keyed by inbound format, so the target codec found nothing to replay and re-encoded from normalized fields. PreservationMetadata now carries a seal recorded when the body is captured. Codecs seal after decoding; exact_preserved_request refuses to replay once the IR no longer matches. Callers that mutate the IR need do nothing and cannot forget — the previous contract asked them to clear the entry by hand, which is what libsy was not doing. `model` is excluded from the seal: routing rewrites it on every hop and the client stamps the resolved name onto the encoded body afterwards, so a replayed body is never wrong about it. Note: exact_preserved_request now takes &LlmRequest rather than &PreservationMetadata. Out-of-tree codecs implementing BufferedCodec need the same one-line change, plus a seal_preservation() call after decoding if they want exact replay. Verified against the reproduction in SWITCH-1224 using a stub upstream that echoes the system message it received: before, the efficient tier saw no system message; after, it sees the configured prompt. Fixes SWITCH-1224 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… pair Covers the guarantee the fix has to make, not just the reported symptom: both tier prompts (capable and efficient, and anything else an algorithm wires into the processor chain) reach the wire on all nine source/target format pairs. Same-format is the case that regressed; cross-format never had the replay shortcut available, so those rows pin existing behaviour rather than change it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (8)
WalkthroughThe change adds request fingerprints to preservation metadata, seals decoded requests, and rejects exact replay when the request IR changes. Tests cover configured prompts, notes, same-format replay, and cross-format instruction propagation. ChangesPreserved Request Validity
Estimated code review effort: 3 (Moderate) | ~25 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Comment |
Summary
A same-format hop replays the exact inbound body, which is what keeps it lossless. That replay was unconditional, so anything a routing algorithm added to the request IR after decoding never reached the wire — the codec returned the body captured before the addition. stage_router's efficient_system_prompt and capable_system_prompt were silently dropped whenever the inbound request and the selected target shared a format, and handoff notes went the same way.
This is not OpenAI-specific. All three buffered codecs had the identical short-circuit — anthropic/buffered.rs:165, openai_chat/buffered.rs:180, responses/buffered.rs:122. An Anthropic-inbound request routed to an Anthropic target dropped its prompt too, and Responses likewise. Cross-format hops escaped only by accident: the preserved body is keyed by inbound format, so the target codec found nothing to replay and re-encoded from normalized fields.
Fix
PreservationMetadata now carries a seal recorded when the body is captured. Codecs seal after decoding; exact_preserved_request refuses to replay once the IR no longer matches. Callers that mutate the IR need do nothing and cannot forget — the previous contract asked them to clear the entry by hand, which is what libsy was not doing.
model is excluded from the seal: routing rewrites it on every hop and send_encoded stamps the resolved name onto the encoded body afterwards, so a replayed body is never wrong about it.
Verification
Reproduced the ticket end-to-end with a stub upstream that echoes the system message it received:
┌───────────────┬──────────────────────────────────────────────────────────────────┐
│ build │ efficient tier sees │
├───────────────┼──────────────────────────────────────────────────────────────────┤
│ upstream main │ │
├───────────────┼──────────────────────────────────────────────────────────────────┤
│ this branch │ Respond with exactly EFFICIENT_SYSTEM_SENTINEL and nothing else. │
└───────────────┴──────────────────────────────────────────────────────────────────┘
Tests added:
The two mutation-shape tests were confirmed to fail on unpatched code. Full workspace green, clippy clean, cargo fmt --check clean.
Breaking change
exact_preserved_request now takes &LlmRequest rather than &PreservationMetadata. Out-of-tree codecs implementing BufferedCodec need that one-line change, plus a seal_preservation() call after decoding if they want exact replay.
Fixes SWITCH-1224
Summary by CodeRabbit
Bug Fixes
Tests