Skip to content

fix(libsy): keep tier prompts and handoff notes on a same-format hop - #322

Open
sabhatinas wants to merge 1 commit into
mainfrom
sabhatinas/switch-1224-preserve-seal
Open

fix(libsy): keep tier prompts and handoff notes on a same-format hop#322
sabhatinas wants to merge 1 commit into
mainfrom
sabhatinas/switch-1224-preserve-seal

Conversation

@sabhatinas

@sabhatinas sabhatinas commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Tier system prompts and handoff notes never reached the model on a same-format hop.

Why. A codec stores the verbatim inbound body at decode, the algorithm then mutates the canonical IR only, and at encode the codec replays that stored body whenever one exists for the target format — returning a snapshot taken before the addition, without ever reading the IR. All three buffered codecs do this, so Anthropic→Anthropic and Responses→Responses dropped it too. Cross-format usually misses the lookup, but not always: preservation carries bodies embedded by earlier hops, so a multi-hop request can hit the same short-circuit.

Fix. Drop the preserved bodies once something has been added, which sends the codec down its normal encode path. Two call sites and one helper. The whole map is cleared, not just the inbound entry — the addition is missing from every stored body equally.

Verified against the ticket's reproduction with a stub upstream: <NO SYSTEM MESSAGE> on main, the configured prompt on this branch. No new tests — the existing fixtures now seed a preserved body, so the tests already covering each mutation assert what happens to it, and fail on unpatched code.

Fixes SWITCH-1224

@sabhatinas
sabhatinas requested a review from a team as a code owner August 6, 2026 17:40
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

The change adds request fingerprint seals to preserved LLM requests. Decoders seal normalized requests, and exact replay now requires a current seal. Regression tests cover mutations from prompts, notes, and router-added instructions.

Changes

Preserved Request Freshness

Layer / File(s) Summary
Request fingerprint contract
crates/protocol/src/llm.rs
PreservationMetadata stores an optional request seal. LlmRequest seals request state and detects changes to the serialized request shape.
Translation replay freshness
crates/switchyard-translation/src/codecs/*/buffered.rs, crates/switchyard-translation/src/util.rs, crates/switchyard-translation/tests/extension_points.rs
Request decoders seal normalized requests. exact_preserved_request rejects stale requests before replay.
Replay invalidation coverage
crates/libsy/src/algorithms/util/prompts.rs, crates/switchyard-translation/tests/request_translation.rs
Tests cover configured prompts, conversation notes, same-format mutations, and router-added instructions across wire formats.

Estimated code review effort: 4 (Complex) | ~45 minutes

Poem

I’m a rabbit with a seal in my hat,
Fresh requests stay exactly at that.
If notes hop in line,
Replay says, “Not mine!”
New instructions march out—just like that.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main fix: preserving tier prompts and handoff notes during same-format request hops.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/protocol/src/llm.rs`:
- Around line 351-354: Correct the freshness documentation around
Self::seal_preservation to match the fingerprint behavior: remove “rewriting the
model” from the changes that invalidate preservation, or explicitly state that
model changes are excluded.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5f270b69-4ebc-47df-82cc-2a1a7afb2932

📥 Commits

Reviewing files that changed from the base of the PR and between 759658e and a845afc.

📒 Files selected for processing (8)
  • crates/libsy/src/algorithms/util/prompts.rs
  • crates/protocol/src/llm.rs
  • crates/switchyard-translation/src/codecs/anthropic/buffered.rs
  • crates/switchyard-translation/src/codecs/openai_chat/buffered.rs
  • crates/switchyard-translation/src/codecs/responses/buffered.rs
  • crates/switchyard-translation/src/util.rs
  • crates/switchyard-translation/tests/extension_points.rs
  • crates/switchyard-translation/tests/request_translation.rs

Comment thread crates/protocol/src/llm.rs Outdated
@sabhatinas
sabhatinas force-pushed the sabhatinas/switch-1224-preserve-seal branch from 3787e0e to 8b2cde9 Compare August 6, 2026 18:07
Comment thread crates/switchyard-translation/tests/request_translation.rs Outdated
Comment thread crates/protocol/src/llm.rs Outdated
@sabhatinas sabhatinas changed the title fix(translation): stop same-format replay from dropping router additions fix(libsy): keep tier prompts and handoff notes on a same-format hop Aug 6, 2026
@sabhatinas
sabhatinas force-pushed the sabhatinas/switch-1224-preserve-seal branch 2 times, most recently from d6d8c83 to cb810de Compare August 6, 2026 18:36
A codec replays the preserved inbound body verbatim when the target format
matches the source, which is what keeps a same-format hop lossless. That
body is captured at decode, before a routing algorithm has added anything,
so replaying it discarded whatever was added: stage_router's tier system
prompts and its handoff notes never reached the model whenever the inbound
request and the selected target shared a format.

The encode path was already correct — it just was not reached. Dropping the
preserved body once something has been added sends the codec down its
normal path, which encodes from the request itself.

Not OpenAI-specific: all three buffered codecs short-circuit the same way,
so Anthropic-to-Anthropic and Responses-to-Responses dropped it too.
Cross-format escaped only because the body is keyed by inbound format, so
the target codec's lookup missed and fell through to the same normal path.

Fixes SWITCH-1224

Signed-off-by: Sabhatina Selvam <sabhatinas@nvidia.com>
@sabhatinas
sabhatinas force-pushed the sabhatinas/switch-1224-preserve-seal branch from cb810de to 83b5fc3 Compare August 6, 2026 18:50
@sabhatinas
sabhatinas enabled auto-merge (squash) August 6, 2026 19:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants