feat(taxonomy): add Step.agent_id — Phase 1 of the MAST multi-agent plan - #13
Merged
Merged
Conversation
Phase 1 of docs/concepts/multi-agent-failures.md's recommended phasing. Step.agent_id: str | None = None — records which agent produced a step; optional, None by default, zero behavior change for existing single-agent callers. triage.observability.otel_ingest. trajectory_from_spans() populates it from a span's gen_ai.agent.id (preferred) or gen_ai.agent.name attribute when present — the same extraction pattern already used for tool_called/metadata["http_status"]. The finding worth calling out, and why this commit corrects the scoping doc's own earlier estimate rather than just implementing it: RulesClassifier's LOOP_DETECTED needed ZERO changes to catch a step repeated across two different agents (MAST's "Step Repetition"), not the small extension originally estimated. _is_loop_window() was already agent-identity-agnostic — it only ever compared tool_called/tool_input, because there was no agent field to look at — so adding Step.agent_id made the existing single-agent matching logic correct for the multi-agent case for free. Verified, not assumed: see test_loop_detected_across_different_agent_ids in tests/test_classifier_rules.py. Documented as deliberate in rules.py's comments and RulesClassifier's docstring — same-agent-only matching would be the wrong default, since it would silently break single-agent loop detection for any caller who happens to tag steps with an agent id, and the whole point of this phase is that a loop spanning a handoff is still a loop. Also wires agent_id through triage/suspension.py's serialize_run/deserialize_run so a suspended run's trajectory round-trips it correctly — caught by the existing test_serialize_covers_all_step_fields regression guard, which failed until this was added. triage/checkpoint/base.py's checkpoint serialization has a pre-existing, separate gap here (it already didn't round-trip idempotent/partial either, before this change, and has no completeness test guarding it) — left as-is since fixing it is unrelated to this phase, not a regression introduced by it. Updates docs/concepts/multi-agent-failures.md (phase 1 marked done, its own "small code change" estimate corrected to "zero code change" for the loop-matching logic specifically), docs/known-limitations.md's "Multi-agent systems" section, and CLAUDE.md (repo layout, rule priority table, and a new design-decision entry explaining why agent-agnostic loop matching is intentional). Verification: ruff check/format clean, pytest 806 passed (was 801 — +5 new tests, 0 skipped), mypy --strict clean, mkdocs build --strict clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M4WNEkbnKSx9mTg5Q1jX39
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this change and why?
Phase 1 of
docs/concepts/multi-agent-failures.md's recommended phasing (from #12).Step.agent_id: str | None = None— records which agent produced a step; optional,Noneby default, zero behavior change for existing single-agent callers.triage.observability.otel_ingest.trajectory_from_spans()populates it from a span'sgen_ai.agent.id(preferred) orgen_ai.agent.nameattribute when present — the same extraction pattern already used fortool_called/metadata["http_status"].The finding worth calling out, and why this commit corrects the scoping doc's own earlier estimate rather than just implementing it:
RulesClassifier'sLOOP_DETECTEDneeded zero changes to catch a step repeated across two different agents (MAST's "Step Repetition"), not the small extensionmulti-agent-failures.mdoriginally estimated._is_loop_window()was already agent-identity-agnostic — it only ever comparedtool_called/tool_input, because there was no agent field to look at — so addingStep.agent_idmade the existing single-agent matching logic correct for the multi-agent case for free. Verified, not assumed: seetest_loop_detected_across_different_agent_idsintests/test_classifier_rules.py. Documented as deliberate inrules.py's comments andRulesClassifier's docstring — same-agent-only matching would be the wrong default, since it would silently break single-agent loop detection for any caller who happens to tag steps with an agent id, and the whole point of this phase is that a loop spanning a handoff is still a loop.Also wires
agent_idthroughtriage/suspension.py'sserialize_run/deserialize_runso a suspended run's trajectory round-trips it correctly — caught by the existingtest_serialize_covers_all_step_fieldsregression guard, which failed until this was added.triage/checkpoint/base.py's checkpoint serialization has a pre-existing, separate gap here (it already didn't round-tripidempotent/partialeither, before this change, and has no completeness test guarding it) — left as-is since fixing it is unrelated to this phase, not a regression introduced by it.Updates
docs/concepts/multi-agent-failures.md(phase 1 marked done, its own "small code change" estimate corrected to "zero code change" for the loop-matching logic specifically),docs/known-limitations.md's "Multi-agent systems" section, andCLAUDE.md(repo layout, rule priority table, and a new design-decision entry explaining why agent-agnostic loop matching is intentional).Related issue
None (Phase 1 of the plan added in #12)
Type of change
triage/classifier/rules.py,triage/scorer/,tests/data/error_corpus_*.json)Checklist
description are clean. I am the author and I am responsible for this change.
(N/A — see the "Ownership" exception in this template for PRs an automated agent
session opens directly. Every other item below still applies in full.)
pytest tests/ -x --tb=shortpasses locally (806 passed, up from 801 — +5 new tests, 0 skipped)ruff check .,ruff format --check ., andmypy triage/ --strictare all cleanopenai/anthropic/langchain/langgraph/opentelemetry/etc. insidetriage/core — no new imports at all;otel_ingest.py's existing lazy OTel import isuntouched, this only adds a new attribute lookup within it
Stepgains a new optional field (agent_id),additive per the stability commitment (existing field names/behavior unchanged);
docs/concepts/multi-agent-failures.md,docs/known-limitations.md, andCHANGELOG.md(under
[Unreleased]) all updatedFailureTypeorRecoveryAction— N/A, and deliberately not yet (seemulti-agent-failures.md's explicit recommendation against newFailureTypemembersuntil a classifier can actually disambiguate them)
If this touches
rules.pyor an error corpus — touchesrules.py(comments only, nomatching-logic change) but not via corpus tuning:
rules.pyagainst any corpus's misses — theLOOP_DETECTEDcommentadditions document existing, already-correct behavior; no pattern or mapping changed
scripts/classifier_accuracy.pyre-run — N/A, no corpus contains multi-agenttrajectories or
agent_iddata, so this change can't move any corpus's score; verifiedby re-running it anyway and confirming the numbers are unchanged from feat(classifier): score corpus E — structured error codes generalize for JSON-RPC, not HTTP #9's last report
Anything reviewers should look at closely?
The
triage/checkpoint/base.pygap I found but didn't fix (checkpoint round-tripping alreadysilently dropped
idempotent/partialbefore this change, and now also won't carryagent_id) — confirmed viagrepthat no completeness test covers it the waytest_serialize_covers_all_step_fieldscovers suspension. Worth a follow-up issue if someoneagrees it should be fixed, but I judged it out of scope for this phase since it's pre-existing
and unrelated to
agent_idspecifically.Attribution note (not from this repo's policy): same exception as prior PRs on this
branch — see the "Ownership" section of this template for PRs an automated agent session
opens directly.
🤖 Generated with Claude Code
https://claude.ai/code/session_01M4WNEkbnKSx9mTg5Q1jX39
Generated by Claude Code