Title
signals: follow-ups deferred from PR #903
Summary
PR #903 landed the Rust port of the Signals paper taxonomy into brightstaff
with 100% parity against the Python reference on 2,000 lmsys-chat-1m samples.
Three items came up during review that were intentionally scoped out of that
PR. This issue tracks them so they don't drop on the floor.
Follow-ups
1. Documentation for the new OTel attribute set — P1
The layered attribute keys (signals.interaction.misalignment.count,
signals.execution.failure.severity, etc.) and per-instance span events
(signal.<type> with confidence/snippet/metadata) added in #903 are not yet
documented. Legacy keys (signals.frustration.*, signals.escalation.requested,
etc.) are being kept in dual-emit for backward compatibility and should be
flagged as deprecated in the docs.
Acceptance:
2. Upstream Python reference: grouped summary format — P3, optional
In #903 the Rust generate_summary was refactored to produce a
taxonomy-grouped output. The Python reference at katanemo/signals
still emits the flat format. The parity comparator classifies summary
as Tier-C so this is not a correctness issue, but it's the one known
intentional divergence and worth aligning upstream.
Acceptance:
3. Legacy OTel key sunset — P3
Decided offline during #903 review to keep signals.frustration.*,
signals.escalation.requested, signals.follow_up.repair.*,
signals.positive_feedback.count, signals.repetition.count for one
release for backward compatibility. This issue tracks the eventual removal.
Acceptance:
Context
Title
signals: follow-ups deferred from PR #903
Summary
PR #903 landed the Rust port of the Signals paper taxonomy into
brightstaffwith 100% parity against the Python reference on 2,000 lmsys-chat-1m samples.
Three items came up during review that were intentionally scoped out of that
PR. This issue tracks them so they don't drop on the floor.
Follow-ups
1. Documentation for the new OTel attribute set — P1
The layered attribute keys (
signals.interaction.misalignment.count,signals.execution.failure.severity, etc.) and per-instance span events(
signal.<type>with confidence/snippet/metadata) added in #903 are not yetdocumented. Legacy keys (
signals.frustration.*,signals.escalation.requested,etc.) are being kept in dual-emit for backward compatibility and should be
flagged as deprecated in the docs.
Acceptance:
docs/source/guides/observability/has a signals attributes referencelisting all layered keys with descriptions and example values
docs/source/observability/monitoring.rstlead example updated to showlayered keys
2. Upstream Python reference: grouped summary format — P3, optional
In #903 the Rust
generate_summarywas refactored to produce ataxonomy-grouped output. The Python reference at
katanemo/signalsstill emits the flat format. The parity comparator classifies
summaryas Tier-C so this is not a correctness issue, but it's the one known
intentional divergence and worth aligning upstream.
Acceptance:
katanemo/signalsporting the grouped-summaryformat (same layer headers, same severity notation, same alert tail)
3. Legacy OTel key sunset — P3
Decided offline during #903 review to keep
signals.frustration.*,signals.escalation.requested,signals.follow_up.repair.*,signals.positive_feedback.count,signals.repetition.countfor onerelease for backward compatibility. This issue tracks the eventual removal.
Acceptance:
samplers, any downstream ingest)
emit_legacy_attributesfromcrates/brightstaff/src/signals/otel.rscrates/brightstaff/src/tracing/constants.rsCLAUDE.mdContext
c7f31879..cee89950)katanemo/signals@0.1.0tests/parity/signals/+.github/workflows/parity-signals.ymlseed 42