From ed1fdca6a80654a89fb304cc30a75d125d390692 Mon Sep 17 00:00:00 2001 From: Will Killian Date: Wed, 5 Aug 2026 10:55:35 -0400 Subject: [PATCH 1/3] docs: finalize 0.7 release documentation Signed-off-by: Will Killian --- docs/about-nemo-relay/release-notes/index.mdx | 84 +++++++++++++++++-- docs/reference/migration-guides.mdx | 26 ++++++ 2 files changed, 105 insertions(+), 5 deletions(-) diff --git a/docs/about-nemo-relay/release-notes/index.mdx b/docs/about-nemo-relay/release-notes/index.mdx index 6d9ed0ae7..584ca70bd 100644 --- a/docs/about-nemo-relay/release-notes/index.mdx +++ b/docs/about-nemo-relay/release-notes/index.mdx @@ -27,8 +27,10 @@ Relay](/about-nemo-relay/overview). ## Release 0.7 -NVIDIA NeMo Relay 0.7 release notes are in preparation. The following -compatibility information applies to the current 0.7 prerelease. +NVIDIA NeMo Relay 0.7 adds asynchronous middleware, codec-aware observability +sanitization, typed OpenTelemetry exporters, cross-boundary context +propagation, and opt-in exact-match LLM response caching. It also expands CLI +configuration, packaging, logging, and diagnostics workflows. ### Highlights @@ -36,6 +38,13 @@ compatibility information applies to the current 0.7 prerelease. start, authenticate, evaluate, or deliver a response. Lifecycle and after-the-fact hooks explicitly fail open. Reinstall each coding-agent integration with `nemo-relay install --force` after upgrading. +- Rust middleware callbacks are asynchronous. Python middleware can return an + awaitable, and Node.js middleware can return a Promise. Relay awaits the + merged global and scope-local middleware chain in priority order. +- Scope and mark lifecycle APIs remain synchronous, but sanitization and + subscriber publication run on an ordered background dispatcher. Python + applications can use `flush_async()` when delivery depends on asynchronous + callbacks. - LLM observability sanitizers now receive the active request or response codec for each managed call. Sanitizers can normalize built-in, runtime-registered, and opaque codec payloads without changing the @@ -56,6 +65,22 @@ compatibility information applies to the current 0.7 prerelease. - OpenTelemetry exporter dependencies, including OpenInference semantic support, are always enabled instead of being controlled by Cargo features. - OpenTelemetry Rust is upgraded to `0.32`. +- Relay propagation values can seed isolated scope stacks across process and + runtime boundaries. Python applications can also fork the active scope stack + for concurrent `asyncio` tasks without sharing mutable task-local context. +- The Adaptive plugin provides opt-in exact-match LLM response caching with + in-memory and Redis stores. The cache is disabled unless `response_cache` is + configured. +- ATIF filename templates can route trajectories with path-safe values from + top-level scope metadata and use configured fallbacks when metadata is + optional. +- The CLI adds a structured `nemo-relay config edit` workflow, size-based + operational log rotation, and offline `doctor` probes. Python, Node.js, and + Go bindings initialize default operational logging from the same environment + controls as the CLI. +- Prebuilt CLI binaries are available from PyPI through + `nemo-relay-cli-bin` and the exact-version `nemo-relay[cli]` extra. The 0.7 + release does not publish a standalone CLI package to npm. - Top-level plugin component `config` lists and the observability destination lists `atof.sinks`, `opentelemetry.endpoints`, and `atif.storage` concatenate across configuration layers, with higher-precedence entries @@ -84,6 +109,11 @@ Before upgrading from 0.6, follow the For the new callback contract and codec operations, refer to [Codec-Aware LLM Sanitizers](/about-nemo-relay/concepts/middleware#codec-aware-llm-sanitizers). +Rust callers that exhaustively match `FlowError` must handle the new +`CallbackException` variant. It retains Python and JavaScript exception types +for OpenTelemetry projection while preserving the existing `internal_error` +classification. + Observability configuration version 3 is a breaking change. It removes the standalone `openinference` section and standalone OpenInference subscriber APIs, and requires a type and endpoint for every OpenTelemetry exporter. Legacy mark @@ -100,8 +130,52 @@ Process-global `OTEL_EXPORTER_OTLP_HEADERS` and `OTEL_EXPORTER_OTLP_TRACES_HEADERS` now fail construction or activation because their values cannot be isolated between endpoints. +The experimental Relay-owned Switchyard plugin is deprecated in 0.7. It is +excluded from default CLI builds and is scheduled for removal in 0.8, when a +Switchyard-owned native plugin is expected to replace it. + ### Fixed Known Issues in 0.7 +- Sanitizer callback errors, panics, rejected awaitables or Promises, and + invalid return values now fail closed. Relay withholds the governed + observability payload instead of emitting the original unsanitized value. +- Relay removes known LLM credential headers before emitting request events, + and the PII redaction plugin rejects malformed RFC 6901 `target_paths` before + activation. +- Generated coding-agent enforcement hooks now fail closed for transport, + startup, authentication, evaluation, malformed-response, and policy-delivery + failures. Lifecycle and after-the-fact hooks continue to fail open. +- OpenTelemetry exporters now report finite batch-queue span loss through + operational logs and plugin runtime diagnostics, including an exact dropped + span count during graceful shutdown. +- Typed OpenTelemetry configuration now rejects colliding destinations after + canonical endpoint resolution, validates attribute mappings before + activation, and completes bare OTLP HTTP endpoints with `/v1/traces`. +- OpenTelemetry and OpenInference prefer response-side model attribution, + preserve binding callback exception types, and map structured `FlowError` + values to useful `error.type` attributes. OpenInference emits system content + as ordered input messages. +- ATIF no longer duplicates user steps for same-turn continuations. Model + fallback matches the span exporters, and metadata-routing and remote-delivery + failures remain visible and recoverable through flush and teardown. +- Explicit configuration editor targets are preserved, file layers follow one + documented priority model, disabled plugin entries do not contribute + configuration, and programmatic component enablement takes precedence over + discovered files. +- Library initialization reports inherited `plugins.toml` files without + exposing their values. `nemo-relay doctor` reports invalid explicit and + logging configuration, supports offline exporter validation, and reports + effective upstream authentication presence without exposing credentials. +- Transparent gateways preserve non-success streaming provider responses and + have bounded, interruptible shutdown behavior. +- Node.js preserves asynchronous propagation context, includes queued callback + delivery in flush completion, and isolates subscriber exceptions instead of + aborting the process. +- Python plugin teardown is safe from a running `asyncio` loop, cancelled + scopes are reported as errors, and completion callbacks no longer target a + closed event loop during cancellation. +- Prebuilt CLI binaries carry the intended release version, so an upgraded CLI + does not silently reuse a gateway started by a differently identified build. - Embedded Python hosts can use `plugin.load_dynamic_plugin_activation_specs(path)` to convert the standard `[[plugins.dynamic]]` records in one explicitly selected `plugins.toml` into @@ -138,9 +212,9 @@ their values cannot be isolated between endpoints. - Codex 0.143 does not expose `SessionEnd`, and Codex multi-agent v2 encrypts delegated-task payloads that Relay cannot decrypt or reliably link. - The Node.js binding and package workflows require Node.js 24 or later. -- Switchyard is experimental, excluded from default CLI builds, and requires a - separately managed compatible service and a named authenticated ATOF HTTP - stream sink. +- The deprecated Switchyard plugin is experimental, excluded from default CLI + builds, and requires a separately managed compatible service and a named + authenticated ATOF HTTP stream sink. It is scheduled for removal in 0.8. - OpenClaw has public hook-backed telemetry. Its security and optimization coverage is partial because it does not own a managed execution path. - The NeMo Guardrails remote backend inherits its configured service's diff --git a/docs/reference/migration-guides.mdx b/docs/reference/migration-guides.mdx index ed76d4bb1..9d403d41c 100644 --- a/docs/reference/migration-guides.mdx +++ b/docs/reference/migration-guides.mdx @@ -31,6 +31,32 @@ Rust code that constructs `AdaptiveConfig` with an exhaustive struct literal must add `response_cache: None`. Prefer `..AdaptiveConfig::default()` when the literal should remain compatible with new optional fields. +### Update Exhaustive FlowError Matches + +Rust callers that exhaustively match `FlowError` must add the new +`FlowError::CallbackException` variant or use a wildcard arm. The variant +retains the originating Python or JavaScript exception type for observability +while preserving the existing `internal_error` classification. + +### Reinstall Coding-Agent Enforcement Hooks + +Generated enforcement hooks now fail closed when Relay cannot start, +authenticate, evaluate, or deliver a response. Generated lifecycle and +after-the-fact hooks continue to fail open. + +After upgrading, reinstall every persistent coding-agent integration so its +generated hooks use the 0.7 policy: + +```bash +nemo-relay install claude-code --force +nemo-relay install codex --force +nemo-relay install hermes --force +``` + +Custom enforcement hooks must pass `--fail-closed` or set +`NEMO_RELAY_FAIL_CLOSED=1`. Do not apply fail-closed behavior to lifecycle or +after-the-fact hooks that cannot prevent the action they observe. + ### Migrate Middleware Callbacks The following callback families are now asynchronous: conditional execution From 70d32f539e91388be837d2262dde9c2a99f41bc0 Mon Sep 17 00:00:00 2001 From: Will Killian Date: Wed, 5 Aug 2026 11:05:11 -0400 Subject: [PATCH 2/3] docs: curate 0.7 release highlights Signed-off-by: Will Killian --- docs/about-nemo-relay/release-notes/index.mdx | 76 ++++++------------- 1 file changed, 24 insertions(+), 52 deletions(-) diff --git a/docs/about-nemo-relay/release-notes/index.mdx b/docs/about-nemo-relay/release-notes/index.mdx index 584ca70bd..bd53fd2ef 100644 --- a/docs/about-nemo-relay/release-notes/index.mdx +++ b/docs/about-nemo-relay/release-notes/index.mdx @@ -34,59 +34,29 @@ configuration, packaging, logging, and diagnostics workflows. ### Highlights -- Generated coding-agent enforcement hooks now fail closed when Relay cannot - start, authenticate, evaluate, or deliver a response. Lifecycle and - after-the-fact hooks explicitly fail open. Reinstall each coding-agent - integration with `nemo-relay install --force` after upgrading. -- Rust middleware callbacks are asynchronous. Python middleware can return an - awaitable, and Node.js middleware can return a Promise. Relay awaits the - merged global and scope-local middleware chain in priority order. -- Scope and mark lifecycle APIs remain synchronous, but sanitization and - subscriber publication run on an ordered background dispatcher. Python - applications can use `flush_async()` when delivery depends on asynchronous - callbacks. -- LLM observability sanitizers now receive the active request or response codec - for each managed call. Sanitizers can normalize built-in, - runtime-registered, and opaque codec payloads without changing the - client-visible request or response. -- Sanitizer callback failures now fail closed across mark, scope, tool, and - LLM surfaces: Relay withholds the governed observability payload while - preserving application execution. -- The PII redaction plugin now selects the active codec per call. One component - can safely sanitize mixed OpenAI Chat, OpenAI Responses, and Anthropic - Messages traffic without a fixed provider codec. -- Native and worker plugins can resolve invocation-scoped codec capabilities. - Request sanitizers can decode and encode requests, and response sanitizers - can decode responses. -- The Observability plugin now uses one typed, multi-endpoint OpenTelemetry - section for `full`, `gen_ai`, and `openinference` trace projections. -- The `gen_ai` projection implements the pinned +- **Asynchronous middleware and ordered event delivery:** Rust middleware + callbacks can await work directly, Python callbacks can return awaitables, + and Node.js callbacks can return Promises. Scope and mark APIs remain + synchronous while Relay publishes sanitized events through an ordered + background dispatcher. +- **Codec-aware privacy controls:** LLM sanitizers receive the active per-call + codec across in-process, native-plugin, and worker boundaries. The PII + redaction plugin can protect mixed OpenAI Chat, OpenAI Responses, and + Anthropic Messages traffic without a fixed provider codec. +- **Unified OpenTelemetry export:** One typed, multi-endpoint configuration + supports `full`, `gen_ai`, and `openinference` projections. The `gen_ai` + projection implements a pinned [OpenTelemetry GenAI semantic-conventions v1.42-era snapshot](https://github.com/open-telemetry/semantic-conventions-genai/tree/43633a68ef8f8ed87a1d5eb205990311ca708bf1). -- OpenTelemetry exporter dependencies, including OpenInference semantic - support, are always enabled instead of being controlled by Cargo features. -- OpenTelemetry Rust is upgraded to `0.32`. -- Relay propagation values can seed isolated scope stacks across process and - runtime boundaries. Python applications can also fork the active scope stack - for concurrent `asyncio` tasks without sharing mutable task-local context. -- The Adaptive plugin provides opt-in exact-match LLM response caching with - in-memory and Redis stores. The cache is disabled unless `response_cache` is - configured. -- ATIF filename templates can route trajectories with path-safe values from - top-level scope metadata and use configured fallbacks when metadata is - optional. -- The CLI adds a structured `nemo-relay config edit` workflow, size-based - operational log rotation, and offline `doctor` probes. Python, Node.js, and - Go bindings initialize default operational logging from the same environment - controls as the CLI. -- Prebuilt CLI binaries are available from PyPI through - `nemo-relay-cli-bin` and the exact-version `nemo-relay[cli]` extra. The 0.7 - release does not publish a standalone CLI package to npm. -- Top-level plugin component `config` lists and the observability destination - lists `atof.sinks`, `opentelemetry.endpoints`, and `atif.storage` - concatenate across configuration layers, with higher-precedence entries - first. -- `object_store` is upgraded to `0.14.1`, which removes the temporary - `RUSTSEC-2026-0194` and `RUSTSEC-2026-0195` advisory exceptions. +- **Portable Relay context:** Transport-neutral propagation can seed isolated + scope stacks across process and runtime boundaries. Python applications can + also fork the active scope stack for concurrent `asyncio` tasks. +- **Opt-in LLM response caching:** The Adaptive plugin can serve eligible + exact-match calls from in-memory or Redis stores. The cache remains disabled + unless `response_cache` is configured. +- **CLI and operational improvements:** The CLI adds structured configuration + editing, PyPI distribution, size-based log rotation, and offline `doctor` + probes. Python, Node.js, and Go bindings initialize default operational + logging from the same environment controls as the CLI. ### Support Matrix and Compatibility Updates @@ -176,6 +146,8 @@ Switchyard-owned native plugin is expected to replace it. closed event loop during cancellation. - Prebuilt CLI binaries carry the intended release version, so an upgraded CLI does not silently reuse a gateway started by a differently identified build. +- `object_store` is upgraded to `0.14.1`, which removes the temporary + `RUSTSEC-2026-0194` and `RUSTSEC-2026-0195` advisory exceptions. - Embedded Python hosts can use `plugin.load_dynamic_plugin_activation_specs(path)` to convert the standard `[[plugins.dynamic]]` records in one explicitly selected `plugins.toml` into From 7fd5e92b8999122a7dd161acca31c7e10731488c Mon Sep 17 00:00:00 2001 From: Will Killian Date: Wed, 5 Aug 2026 11:22:34 -0400 Subject: [PATCH 3/3] docs: add release-note list introductions Signed-off-by: Will Killian --- docs/about-nemo-relay/release-notes/index.mdx | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/about-nemo-relay/release-notes/index.mdx b/docs/about-nemo-relay/release-notes/index.mdx index bd53fd2ef..f9495fca8 100644 --- a/docs/about-nemo-relay/release-notes/index.mdx +++ b/docs/about-nemo-relay/release-notes/index.mdx @@ -34,6 +34,9 @@ configuration, packaging, logging, and diagnostics workflows. ### Highlights +The following highlights summarize the most significant user-facing changes in +0.7: + - **Asynchronous middleware and ordered event delivery:** Rust middleware callbacks can await work directly, Python callbacks can return awaitables, and Node.js callbacks can return Promises. Scope and mark APIs remain @@ -106,6 +109,8 @@ Switchyard-owned native plugin is expected to replace it. ### Fixed Known Issues in 0.7 +The following known issues are fixed in 0.7: + - Sanitizer callback errors, panics, rejected awaitables or Promises, and invalid return values now fail closed. Relay withholds the governed observability payload instead of emitting the original unsanitized value.