Skip to content

fix: harden session affinity identity handling - #308

Open
joelagnel wants to merge 4 commits into
NVIDIA-NeMo:mainfrom
joelagnel:aow/affinity-session-fixes
Open

fix: harden session affinity identity handling#308
joelagnel wants to merge 4 commits into
NVIDIA-NeMo:mainfrom
joelagnel:aow/affinity-session-fixes

Conversation

@joelagnel

@joelagnel joelagnel commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Scope clarification

The repeatedly reclassified TB-Lite run was specifically Harbor running Hermes with session_affinity = true and message_hash_fallback omitted/false. Hermes supplied no native session header, while Harbor's proxy-generated proxy_x_session_id was previously used only for routing statistics.

With message_hash_fallback = true, classifier affinity already worked on a best-effort basis by hashing the first user message. Recognizing Harbor's proxy ID is therefore the exact-identity fix for Harbor/Hermes when the fallback is disabled, and an upgrade from heuristic to exact identity when it is enabled. The empty-ID hardening and diagnostics in this PR remain general.

What

  • Treat empty session IDs as absent when deriving affinity keys.
  • Warn once when affinity is enabled but a request has no usable identity.
  • Distinguish absent and empty session IDs in server request logs.
  • Normalize Harbor's per-attempt proxy_x_session_id into request metadata so Harbor-proxied agents such as Hermes receive exact session affinity.

Why

Session affinity could silently classify every turn again when a harness supplied no recognized session identity, or collapse unrelated requests onto one assignment when a caller supplied an empty identity. Harbor already generates a unique session ID per trial attempt, but Switchyard previously used it only for routing statistics rather than affinity.

Closes #301

How tested

  • cargo fmt --all -- --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test --workspace
  • Regression test observed failing before the Harbor session-header alias was added

Checklist

  • Unit tests added for the behavior changes.
  • Header precedence is covered: canonical/native IDs win, Harbor wins over generic session-id.
  • All four commits carry DCO sign-offs.
  • No customer-facing configuration or CLI surface changed.

Notes for reviewers

Harbor's proxy generates and overwrites proxy_x_session_id once per trial attempt. Hermes itself does not currently send a Switchyard-recognized live session header. Normalizing the proxy header therefore gives Harbor-proxied Hermes runs an exact affinity key while preserving message_hash_fallback for clients that genuinely lack an identity.

Summary by CodeRabbit

  • Bug Fixes
    • Prevented requests with empty session IDs from being incorrectly grouped together.
    • Improved session detection for Harbor proxy requests, including correct header precedence.
    • Clarified request logs by displaying - when no session ID is provided.
    • Reduced duplicate warnings for requests without usable session information.

Signed-off-by: Joel Fernandes <joel@joelfernandes.org>
@joelagnel
joelagnel requested a review from a team as a code owner August 5, 2026 18:20
@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

Changes

Session handling

Layer / File(s) Summary
Harbor session header resolution
crates/protocol/src/metadata.rs
Session lookup now includes proxy_x_session_id between OpenCode and the generic session-id header. Tests cover extraction and precedence.
Affinity keying and warning behavior
crates/libsy/src/algorithms/util/affinity.rs
Empty session IDs no longer create shared affinity keys. Unkeyable requests emit a warning once, while intentional root abstention remains silent. Tests cover these cases.
Session ID request logging
crates/switchyard-server/src/lib.rs
Missing session IDs now log as -; explicitly empty session IDs remain empty.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Poem

A rabbit checks the session trail,
Empty keys no longer prevail.
One warning hops, then rests from sight,
Harbor headers choose the right.
Missing logs wear a dash so neat.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The PR addresses all three requirements in [#301]: empty IDs are ignored, missing keys warn once, and absent IDs log as "-".
Out of Scope Changes check ✅ Passed Harbor header normalization directly supports session affinity identity handling and does not introduce unrelated behavior.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main session affinity identity handling changes.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
crates/libsy/src/algorithms/util/affinity.rs (1)

121-167: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Warn when a child request has no usable affinity identity.

If metadata.is_subagent is true and agent_id is absent, lines 121-125 return None. If its session ID is absent or empty, lines 138-141 also return None. Neither path reaches the one-time warning at line 154.

Return early only for intentional subagents_only root abstention. Preserve child message-hash fallback abstention, but let child requests without a usable {session_id, agent_id} pair consume the warning. Add regressions for both paths.

As per PR objectives, affinity must warn once when it cannot derive a key.

Also applies to: 515-577

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/libsy/src/algorithms/util/affinity.rs` around lines 121 - 167, Update
the affinity key derivation flow so child requests lacking a usable agent_id or
session identity reach the one-time unkeyed warning via should_warn_unkeyed.
Keep the intentional subagents_only root-request abstention and child
message-hash fallback abstention behavior, but ensure both missing-identity
paths warn once before returning None. Add regressions covering a subagent
without agent_id and a subagent with an absent or empty session ID.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@crates/libsy/src/algorithms/util/affinity.rs`:
- Around line 121-167: Update the affinity key derivation flow so child requests
lacking a usable agent_id or session identity reach the one-time unkeyed warning
via should_warn_unkeyed. Keep the intentional subagents_only root-request
abstention and child message-hash fallback abstention behavior, but ensure both
missing-identity paths warn once before returning None. Add regressions covering
a subagent without agent_id and a subagent with an absent or empty session ID.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e6a1aa02-41be-4822-a300-e3afe2b3eca8

📥 Commits

Reviewing files that changed from the base of the PR and between 54d4b66 and 0e2fda8.

📒 Files selected for processing (3)
  • crates/libsy/src/algorithms/util/affinity.rs
  • crates/protocol/src/metadata.rs
  • crates/switchyard-server/src/lib.rs

Affinity without recognized session metadata and with message_hash_fallback disabled cannot derive a key, so every turn is classified again despite session_affinity being enabled.

Emit this warning once. This was observed with Harbor running Hermes: Hermes supplied no native session header, and the Harbor proxy ID was not yet normalized for affinity. Configurations with message_hash_fallback enabled already retain best-effort affinity.

Signed-off-by: Joel Fernandes <joel@joelfernandes.org>
…quest logs

Signed-off-by: Joel Fernandes <joel@joelfernandes.org>
Harbor's proxy generates a stable UUID per trial attempt and sends it as proxy_x_session_id. Hermes does not supply a native session header in this path.

Normalize the proxy header into request metadata so affinity uses the exact Harbor attempt identity. This fixes Harbor/Hermes configurations with message_hash_fallback disabled; configurations with it enabled already had best-effort message-hash affinity and now prefer the exact ID.

Signed-off-by: Joel Fernandes <joel@joelfernandes.org>
@joelagnel
joelagnel force-pushed the aow/affinity-session-fixes branch from a20a712 to 5f80d1d Compare August 6, 2026 17:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Session affinity keys on an empty session id, and is silent when it cannot key at all

1 participant