[AIGNLAI]

Scope

Only the five previously confirmed bugs found during Routerbench reproduction, plus regression tests and compatibility documentation.

1. Batch RouteLLM_BERT prediction to bound inference memory instead of tokenizing/forwarding the entire test set at once.
2. Honor RouteLLM_BERT top-level epochs/lr/batch_size with per-key training overrides taking precedence. The existing JSON configuration is no longer silently ignored.
3. Use the complete query-aligned training split as RouteFM Context by default for both BGE and Qwen configurations. An integer context_size remains an explicit fixed-K ablation, with one shared row sample across candidates. Missing observations are masked.
4. Detect Routerbench query IDs with inconsistent original inputs across candidates before pivot/groupby aggregation. The clean protocol excludes and logs those IDs based only on input consistency, not outcomes.
5. Use normalized-input-group splitting for Routerbench in-domain evaluation so repeated query text cannot cross train/test.

Data protocol / compatibility

• Routerbench explicitly defaults to dataset.protocol="grouped-clean".
• dataset.protocol="legacy" preserves historical first-prompt aggregation and seeded row-random in-domain splitting, with warnings.
• Group keys only normalize whitespace; original prompts, candidate identities/order, quality labels and monetary costs are retained.
• Other benchmark loaders and Routerbench's out-of-domain task splitter are unchanged.
• This corrects a data protocol and therefore changes results; it is not presented as the untouched historical benchmark protocol.
• No claim is made that cleaning this experiment split removes external pretrained-model exposure.

Deliberately excluded

No reproduction runners, artifact downloaders, encoder caches, benchmark result files, checkpoints, local paths, credentials, NIRT gateway changes, new dependencies, AUC changes, shared-cost changes, or other router algorithm changes.

Validation

• Python source compilation: PASS.
• Full local unit/offline smoke suite: 118 tests PASS.
• Full verified official Routerbench artifact through the production loader: 14 conflicting query IDs excluded, all 11 candidates retained, 7,296 train / 29,187 test queries. Every row, original prompt, score and cost exactly matches the previously audited frozen clean snapshots.
• Local bounded real-weight smoke for RouteFM-BGE and RouteLLM_BERT: 128 actual training / 16 actual held-out queries each, genuine pretrained encoders/router weights, finite predictions and native evaluation JSON output. Local artifact loading and actual data slices were supplied at the I/O boundary; no substitute vectors or tiny model weights were used in these two checks.
• Regression coverage includes BERT configuration precedence and bounded batch equivalence, complete/aligned RouteFM Context and missing-cell masks, input conflict handling, deterministic disjoint grouped splits, and legacy compatibility.