fix(libsy): keep tier prompts and handoff notes on a same-format hop - #322
fix(libsy): keep tier prompts and handoff notes on a same-format hop#322sabhatinas wants to merge 1 commit into
Conversation
WalkthroughThe change adds request fingerprint seals to preserved LLM requests. Decoders seal normalized requests, and exact replay now requires a current seal. Regression tests cover mutations from prompts, notes, and router-added instructions. ChangesPreserved Request Freshness
Estimated code review effort: 4 (Complex) | ~45 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/protocol/src/llm.rs`:
- Around line 351-354: Correct the freshness documentation around
Self::seal_preservation to match the fingerprint behavior: remove “rewriting the
model” from the changes that invalidate preservation, or explicitly state that
model changes are excluded.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 5f270b69-4ebc-47df-82cc-2a1a7afb2932
📒 Files selected for processing (8)
crates/libsy/src/algorithms/util/prompts.rscrates/protocol/src/llm.rscrates/switchyard-translation/src/codecs/anthropic/buffered.rscrates/switchyard-translation/src/codecs/openai_chat/buffered.rscrates/switchyard-translation/src/codecs/responses/buffered.rscrates/switchyard-translation/src/util.rscrates/switchyard-translation/tests/extension_points.rscrates/switchyard-translation/tests/request_translation.rs
3787e0e to
8b2cde9
Compare
d6d8c83 to
cb810de
Compare
A codec replays the preserved inbound body verbatim when the target format matches the source, which is what keeps a same-format hop lossless. That body is captured at decode, before a routing algorithm has added anything, so replaying it discarded whatever was added: stage_router's tier system prompts and its handoff notes never reached the model whenever the inbound request and the selected target shared a format. The encode path was already correct — it just was not reached. Dropping the preserved body once something has been added sends the codec down its normal path, which encodes from the request itself. Not OpenAI-specific: all three buffered codecs short-circuit the same way, so Anthropic-to-Anthropic and Responses-to-Responses dropped it too. Cross-format escaped only because the body is keyed by inbound format, so the target codec's lookup missed and fell through to the same normal path. Fixes SWITCH-1224 Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
cb810de to
83b5fc3
Compare
Tier system prompts and handoff notes never reached the model on a same-format hop.
Why. A codec stores the verbatim inbound body at decode, the algorithm then mutates the canonical IR only, and at encode the codec replays that stored body whenever one exists for the target format — returning a snapshot taken before the addition, without ever reading the IR. All three buffered codecs do this, so Anthropic→Anthropic and Responses→Responses dropped it too. Cross-format usually misses the lookup, but not always: preservation carries bodies embedded by earlier hops, so a multi-hop request can hit the same short-circuit.
Fix. Drop the preserved bodies once something has been added, which sends the codec down its normal encode path. Two call sites and one helper. The whole map is cleared, not just the inbound entry — the addition is missing from every stored body equally.
Verified against the ticket's reproduction with a stub upstream:
<NO SYSTEM MESSAGE>onmain, the configured prompt on this branch. No new tests — the existing fixtures now seed a preserved body, so the tests already covering each mutation assert what happens to it, and fail on unpatched code.Fixes SWITCH-1224