Skip to content

fix(translation): stop same-format replay from dropping router additions - #321

Closed
sabhatinas wants to merge 2 commits into
mainfrom
sabhatinas/switch-1224-preserve-seal
Closed

fix(translation): stop same-format replay from dropping router additions#321
sabhatinas wants to merge 2 commits into
mainfrom
sabhatinas/switch-1224-preserve-seal

Conversation

@sabhatinas

@sabhatinas sabhatinas commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Summary

A same-format hop replays the exact inbound body, which is what keeps it lossless. That replay was unconditional, so anything a routing algorithm added to the request IR after decoding never reached the wire — the codec returned the body captured before the addition. stage_router's efficient_system_prompt and capable_system_prompt were silently dropped whenever the inbound request and the selected target shared a format, and handoff notes went the same way.

This is not OpenAI-specific. All three buffered codecs had the identical short-circuit — anthropic/buffered.rs:165, openai_chat/buffered.rs:180, responses/buffered.rs:122. An Anthropic-inbound request routed to an Anthropic target dropped its prompt too, and Responses likewise. Cross-format hops escaped only by accident: the preserved body is keyed by inbound format, so the target codec found nothing to replay and re-encoded from normalized fields.

Fix

PreservationMetadata now carries a seal recorded when the body is captured. Codecs seal after decoding; exact_preserved_request refuses to replay once the IR no longer matches. Callers that mutate the IR need do nothing and cannot forget — the previous contract asked them to clear the entry by hand, which is what libsy was not doing.

model is excluded from the seal: routing rewrites it on every hop and send_encoded stamps the resolved name onto the encoded body afterwards, so a replayed body is never wrong about it.

Verification

Reproduced the ticket end-to-end with a stub upstream that echoes the system message it received:

┌───────────────┬──────────────────────────────────────────────────────────────────┐
│ build │ efficient tier sees │
├───────────────┼──────────────────────────────────────────────────────────────────┤
│ upstream main │ │
├───────────────┼──────────────────────────────────────────────────────────────────┤
│ this branch │ Respond with exactly EFFICIENT_SYSTEM_SENTINEL and nothing else. │
└───────────────┴──────────────────────────────────────────────────────────────────┘

Tests added:

  • both tier prompts invalidate exact replay; an unprompted target keeps it
  • handoff notes likewise
  • same-format encoding carries the instruction for all three formats
  • a router-added instruction reaches the wire across all nine source/target format pairs

The two mutation-shape tests were confirmed to fail on unpatched code. Full workspace green, clippy clean, cargo fmt --check clean.

Breaking change

exact_preserved_request now takes &LlmRequest rather than &PreservationMetadata. Out-of-tree codecs implementing BufferedCodec need that one-line change, plus a seal_preservation() call after decoding if they want exact replay.

Fixes SWITCH-1224

Summary by CodeRabbit

  • Bug Fixes

    • Prevented stale preserved request bodies from being replayed after request content changes.
    • Ensured added instructions are retained during same-format and cross-format request encoding.
    • Improved exact request replay behavior across supported Anthropic, OpenAI Chat, and Responses formats.
  • Tests

    • Added regression coverage for modified requests, configured prompts, one-off notes, and instruction propagation.

sabhatinas and others added 2 commits August 6, 2026 10:12
A same-format hop replays the exact inbound body, which is what keeps it
lossless. That replay was unconditional, so anything a routing algorithm
added to the request IR after decoding never reached the wire: the codec
returned the body captured before the addition. stage_router's
efficient_system_prompt was silently dropped whenever the inbound request
and the selected target shared a format, and handoff notes went the same
way.

All three buffered codecs had the same short-circuit, so this was not
OpenAI-specific — an Anthropic-inbound request routed to an Anthropic
target dropped its prompt too, and Responses likewise. Cross-format hops
escaped only by accident: the preserved body is keyed by inbound format,
so the target codec found nothing to replay and re-encoded from
normalized fields.

PreservationMetadata now carries a seal recorded when the body is
captured. Codecs seal after decoding; exact_preserved_request refuses to
replay once the IR no longer matches. Callers that mutate the IR need do
nothing and cannot forget — the previous contract asked them to clear the
entry by hand, which is what libsy was not doing.

`model` is excluded from the seal: routing rewrites it on every hop and
the client stamps the resolved name onto the encoded body afterwards, so
a replayed body is never wrong about it.

Note: exact_preserved_request now takes &LlmRequest rather than
&PreservationMetadata. Out-of-tree codecs implementing BufferedCodec need
the same one-line change, plus a seal_preservation() call after decoding
if they want exact replay.

Verified against the reproduction in SWITCH-1224 using a stub upstream
that echoes the system message it received: before, the efficient tier
saw no system message; after, it sees the configured prompt.

Fixes SWITCH-1224

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… pair

Covers the guarantee the fix has to make, not just the reported symptom:
both tier prompts (capable and efficient, and anything else an algorithm
wires into the processor chain) reach the wire on all nine source/target
format pairs.

Same-format is the case that regressed; cross-format never had the replay
shortcut available, so those rows pin existing behaviour rather than
change it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@sabhatinas
sabhatinas requested a review from a team as a code owner August 6, 2026 17:22
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 46f35135-236f-44e9-8c02-a355bb63778a

📥 Commits

Reviewing files that changed from the base of the PR and between 759658e and a845afc.

📒 Files selected for processing (8)
  • crates/libsy/src/algorithms/util/prompts.rs
  • crates/protocol/src/llm.rs
  • crates/switchyard-translation/src/codecs/anthropic/buffered.rs
  • crates/switchyard-translation/src/codecs/openai_chat/buffered.rs
  • crates/switchyard-translation/src/codecs/responses/buffered.rs
  • crates/switchyard-translation/src/util.rs
  • crates/switchyard-translation/tests/extension_points.rs
  • crates/switchyard-translation/tests/request_translation.rs

Walkthrough

The change adds request fingerprints to preservation metadata, seals decoded requests, and rejects exact replay when the request IR changes. Tests cover configured prompts, notes, same-format replay, and cross-format instruction propagation.

Changes

Preserved Request Validity

Layer / File(s) Summary
Request fingerprint contract
crates/protocol/src/llm.rs
PreservationMetadata stores a request seal. LlmRequest computes the normalized request fingerprint and checks whether preserved data remains current.
Translation replay integration
crates/switchyard-translation/src/codecs/*/buffered.rs, crates/switchyard-translation/src/util.rs, crates/switchyard-translation/tests/extension_points.rs
Provider decoders seal preserved requests. exact_preserved_request receives the full request and rejects stale preserved bodies.
Preservation regression coverage
crates/libsy/src/algorithms/util/prompts.rs, crates/switchyard-translation/tests/request_translation.rs
Tests cover prompt and note mutations, exact same-format replay, and instruction propagation across wire formats.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Poem

I’m a rabbit with a sealed request,
Keeping old bytes at their best.
When instructions hop inside,
Exact replay must step aside.
Fresh fingerprints guide the way—
New encoded words now stay.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main fix: preventing same-format replay from dropping router-added request fields.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@sabhatinas sabhatinas closed this Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant