fix(evaluator): require nemo-relay 0.7.3 so Relay teardown cannot fail a good trial

[SandyChapman]

Summary

NVBug 6562846: a Fabric DeepAgents invocation completed its turn — model calls, tool calls, a final assistant response — and was reported as failed, because NeMo Relay's telemetry teardown raised RuntimeError: invalid argument: scope handle is not at the top of the stack. QA saw 31 such trials in the 2026-08-05 Evaluator regression. Both halves of the root cause are now fixed upstream (nemo-relay 0.7.3, nemo-fabric 0.2.0) and this repo already locks both. What it still declared was nemo-relay>=0.7.2, which admits the last broken release on any install that does not use uv.lock. This raises the declared floor to 0.7.3.

Related Issue

AALGO-495. NVBug 6562846.

Changes

• nemo-relay floor >=0.7.2 → >=0.7.3 at all four declaration sites: packages/nemo_evaluator_sdk, packages/nemo_platform, packages/nemo_platform_plugin, and the vendored sdk/python/nemo-platform mirror (regenerated by make vendor, not hand-edited into divergence).
• .github/wheel-constraints/nemo-platform-services.txt moved off its nemo-relay==0.7.2 pin, which would otherwise have made the wheel install-smoke-test unsatisfiable against the new floor — and had it validating the broken version.
• uv.lock relocked with the platform uv (0.9.14). The resolved version was already 0.7.3, so the lock diff is four specifier lines.

No code changes: nine lines in, nine out.

Root cause, for the record

Relay's LangChain callback handler popped chain scopes in callback order rather than stack order. LangGraph schedules sibling chain runs concurrently, so a run can end while scopes opened after it are still open; the out-of-LIFO pop was rejected and swallowed, the child scope was stranded live on the stack, and the enclosing scope's teardown then raised. Fabric's DeepAgents adapter caught that teardown exception in the same try that guarded the agent invocation, so an observability fault was reported as the agent's failure.

nemo-relay 0.7.3 defers a completed scope until it reaches the top of the stack; nemo-fabric 0.2.0 separates telemetry_error from the invocation failure domain.

Why no regression test

A test in this suite would have to drive nemo-relay's own callback handler — no code in this repo is involved — so it could only ever fail on a dependency change, which the floor already prevents. The floor is the control; the validation below is the confirmation.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with documentation updates
• Documentation only
• Contributor tooling or automation
• CI, build, or test infrastructure

Quality Gates

• Tests added or updated for changed behavior
• Existing tests cover changed behavior — justification:
• Tests not applicable — justification: this changes a declared dependency floor, not repository code. The behaviour it protects lives in nemo-relay, so the only meaningful verification is running the defect against both versions, recorded below.
• Documentation updated for user-visible behavior
• Documentation not applicable — justification: no user-facing surface, config, or CLI change.

Verification

• Pull request title follows the repository's Conventional Commit format
• Every commit includes an appropriate Signed-off-by: trailer
• uv run pre-commit run -a passes, or any blocked checks are identified below
• Targeted tests pass, or tests are marked not applicable above
• No secrets, API keys, or credentials are included

Targeted validation:

The fix is in 0.7.3, not 0.7.2 — confirmed at source. nemo_relay/integrations/langchain/callbacks.py goes from 112 lines in 0.7.2 to 244 in 0.7.3. _CompletedScope and _close_completed_scopes are absent in 0.7.2 and present in 0.7.3; 0.7.2's _pop_scope calls nemo_relay.scope.pop directly, while 0.7.3 records the completion and drains only when that scope reaches the top. The two are consecutive releases, so there is no earlier version carrying the fix.

And confirmed by behaviour. Running the bug's minimal repro — two overlapping LangChain chain lifecycles inside one Relay scope, through the real NemoRelayCallbackHandler:

• nemo-relay 0.7.2 — the child scope.pop is rejected and logged, the stack top is left on the stranded chain-A, and leaving the outer deepagents-request scope raises RuntimeError: invalid argument: scope handle is not at the top of the stack.
• nemo-relay 0.7.3 (this floor) — the stack top is back to deepagents-request and the scope exits normally.

Live end-to-end. A real DeepAgents agent driven through FabricAgentRuntime → Fabric 0.2.0 → Relay 0.7.3 with telemetry enabled (openai/gpt-oss-120b via integrate.api.nvidia.com, two rounds of tool calls): trial COMPLETED, agent_ok=True, Fabric status=succeeded, adapter telemetry_error=None, and the full evidence set attached (result, workspace, relay_config, relay_atif, relay_atof, trace; ATIF 133 KB). That is the scenario that produced the failed QA trials.

• uv run --frozen pytest packages/nemo_evaluator_sdk/tests/agent_eval/ -q — 776 passed, 10 skipped.
• uv run ruff check packages/nemo_evaluator_sdk/ — clean.
• tools/lint/lint-python-types.sh (the exact CI type-check entrypoint) — all checks passed.
• uv run pre-commit run -a with the platform uv (0.9.14) on PATH — all 15 hooks pass, including uv-lock, uv.lock drift, ty, copyright headers, Helm Docs, and UI lint-staged.
• uv lock --check with uv 0.9.14 — clean.
• tools/lint/lint-sdk-vendored.sh cannot execute on this host (mapfile: command not found; macOS ships bash 3.2). Its two invariants were verified directly instead: make vendor and make generate-cli-reference-docs both leave the tree clean, so sdk/python/ and docs/cli are in sync.

Known-stale, deliberately not touched

agents/nemo-studio-assistant/uv.lock still carries nemo-relay>=0.7.2,<0.8 in its copy of the evaluator SDK's metadata. That lock was already out of date before this change (uv lock --check --project agents/nemo-studio-assistant fails on unmodified main), nothing in CI or the Makefile gates it, and Dockerfile.fabric-local installs from constraints.fabric-local.txt rather than the lock. Relocking it re-resolves the whole project and drags in ~190 lines of unrelated upgrades (langchain-core, langgraph, langsmith), which does not belong in this PR. It resolves nemo-relay 0.7.3 today, so it is not exposed to the defect.

[github-actions]

Suite	Lines Covered	Line Rate	Branch Rate
Unit Tests	36931/46840	78.8%	63.3%
Integration Tests	21712/44078	49.3%	22.4%

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4316dd7e-9056-4bc8-97f5-45f867affc30

📥 Commits

Reviewing files that changed from the base of the PR and between fe8c7b8 and c9453a8.

⛔ Files ignored due to path filters (2)

• sdk/python/nemo-platform/pyproject.toml is excluded by !sdk/**
• uv.lock is excluded by !**/*.lock

📒 Files selected for processing (4)

• .github/wheel-constraints/nemo-platform-services.txt
• packages/nemo_evaluator_sdk/pyproject.toml
• packages/nemo_platform/pyproject.toml
• packages/nemo_platform_plugin/pyproject.toml

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

📝 Walkthrough

Walkthrough

The pull request raises the minimum nemo-relay version from 0.7.2 to 0.7.3 across package metadata and wheel install-smoke-test constraints. The <0.8 upper bound remains unchanged.

Changes

Dependency alignment

Layer / File(s)	Summary
Update nemo-relay constraints .github/wheel-constraints/nemo-platform-services.txt, packages/*/pyproject.toml	All listed constraints now require nemo-relay version 0.7.3 or later, with the existing <0.8 upper bound retained.

Suggested reviewers: a2bondar

Merge Risk: ⚪ Minimal · up to c9453

The PR raises the required Relay version to prevent the known teardown failure and updates the corresponding lock and wheel constraints; no actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the evaluator fix, the required nemo-relay version, and the Relay teardown failure it addresses.

Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (4 skipped: 4 unsupported.)

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch aalgo-495-relay-scope-teardown-floor/schapman

---

_{Comment @coderabbitai help to get the list of available commands.}

[svvarom]

The wheel build checks pass. LGTM.