feat(python): initialize plugins from TOML

[bbednarski9]

Overview

Adds an owned Python initialization path for ordinary plugins.toml files, including lifecycle-enabled dynamic plugins, without requiring an embedded host to parse [[plugins.dynamic]] or construct activation specs itself.

This is the first PR in a two-PR stack. It introduces the binding-neutral resolver and activation foundation and exposes it in Python. A stacked follow-up will migrate the CLI runtime owner to the same activation plan while preserving the CLI control plane and bootstrap behavior.

• I confirm this contribution is my own work, or I have the right to submit it under this project's license.
• I searched existing issues and open pull requests, and this does not duplicate existing work.

Details

• Add the versioned nemo-relay-plugin-host-config workspace crate. It shares Relay's user/project/system source selection, component merge rules, dynamic host policy, lifecycle state, trust checks, managed-environment attestation, and immutable activation snapshots with embedded hosts.
• Add nemo_relay.plugin.initialize_from_plugins_toml(...) and the owned PluginFileActivation async lifetime. The API supports default discovery or a selected plugins.toml, a highest-precedence static overlay, inactive no-input handles, owned static-only handles, deterministic/idempotent close, async context management, and best-effort finalization.
• Treat [[plugins.dynamic]] as declarations only. Each declaration is validated and reconciled against the same source file's sibling .dynamic-plugins.json; only live records with spec.enabled == true enter the activation plan. Initialization never installs packages, creates environments, changes enablement, or edits plugins.toml.
• Persist reconciliation atomically before any code load. Source ownership is tracked in a hidden lifecycle-state sidecar without repurposing public config_ref; duplicate IDs and ambiguous live source moves fail closed.
• Pin configuration and lifecycle sources to one physical path while keeping ordinary Windows paths portable for child runtimes. Reserved, overlong, UNC, and otherwise ambiguous paths retain their verbatim representation instead of being rewritten unsafely.
• Add a hidden binding-neutral PluginHostActivationPlan that carries resolved static config, ordered dynamic specs, redacted diagnostics, and Arc-owned snapshot resources into Relay's cancellation-resistant mutation executor.
• Harden native and worker ownership so callbacks, registrations, runtimes, libraries, managed environments, and snapshots are released in dependency order. Unsafe rollback or teardown retains the process lease and resources fail closed.
• Keep relocatable macOS CPython workers self-contained by snapshotting the exact libpython<major>.<minor>.dylib selected by the attestation-verified pyvenv.cfg; the copied runtime library is then covered by the immutable snapshot digest and protection.
• Preserve the existing initialize(), initialize_with_dynamic_plugins(), explicit activation specs, generic clear APIs, and non-Python bindings unchanged.
• Add Python API documentation, discovery/lifecycle/security guidance, 0.7 release notes, workspace packaging/publishing metadata, and CI coverage for the new crate.

Security model: calling the new initializer is explicit consent to load lifecycle-enabled trusted native libraries or worker processes from the selected configuration scope. Enabled Python workers must already have a valid Relay-managed environment and attestation; this API is not an installer or control plane. Diagnostics include source paths and plugin IDs but do not include component values or credentials.

Validation completed locally:

• cargo test -p nemo-relay-plugin-host-config
• cargo test -p nemo-relay-cli plugins::lifecycle::tests (79 lifecycle/control-plane tests)
• cargo test -p nemo-relay --features worker-grpc --test native_plugin_integration --test worker_plugin_integration
• just test-rust
• just test-python-plugin
• just test-python-plugin-e2e under the CI-pinned uv CPython 3.11.14 runtime (managed gRPC worker add, enable, activation, middleware execution, and teardown)
• just test-python
• just test-go
• just test-node
• cargo fmt --all -- --check
• cargo clippy --workspace --all-targets -- -D warnings
• just package-rust
• just package-python
• just package-python-sdist (verified the shared crate is present in the sdist)
• just docs
• just docs-linkcheck
• uv run pre-commit run --all-files

Breaking changes: none. This is an additive Python API and a document-hidden Rust host contract. Node, FFI, and Go retain their existing explicit-spec APIs.

Where should the reviewer start?

Start with crates/plugin-host-config/src/activation.rs and crates/plugin-host-config/src/resolver.rs for the file-backed contract, then review crates/core/src/plugin/dynamic/host.rs for atomic ownership and teardown. The Python surface is wired in crates/python/src/py_plugin.rs and python/nemo_relay/plugin.py.

The most important design invariant is that the same immutable snapshot verified immediately before load remains owned by the native or worker runtime through the last in-flight callback. Lifecycle reconciliation is durable configuration state, not part of the runtime activation transaction.

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• Closes [Enhancement]: Initialize discovered [[plugins.dynamic]] without caller-built specs in Python #673
• Downstream: feat(relay): initialize static and dynamic plugin components NousResearch/hermes-agent#77915

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 63b85384-dea8-487c-bfb2-bef9479ee749

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Fern docs preview: https://nvidia-preview-pull-request-684.docs.buildwithfern.com/nemo/relay/dev

[github-actions]

License Diff

Compared against origin/release/0.7.

Lockfile license changes

Lockfile License Changes

Rust

Added

• None

Removed

• None

Updated/Changed

• None

Node

Added

• None

Removed

• None

Updated/Changed

• None

Python

Added

• None

Removed

• None

Updated/Changed

• None

Status output

[license-diff] selected languages: rust, node, python
[license-diff] generating current inventory
[license-diff] current: generating Rust inventory
[license-diff] current: Rust inventory complete (449 packages)
[license-diff] current: generating Node inventory
[license-diff] current: Node inventory complete (367 packages)
[license-diff] current: generating Python inventory
[license-diff] current: Python inventory complete (105 packages)
[license-diff] current inventory complete
[license-diff] checking out base ref origin/release/0.7 into a temporary worktree
[license-diff] base: generating Rust inventory
[license-diff] base: Rust inventory complete (449 packages)
[license-diff] base: generating Node inventory
[license-diff] base: Node inventory complete (367 packages)
[license-diff] base: generating Python inventory
[license-diff] base: Python inventory complete (105 packages)
[license-diff] base inventory complete
[license-diff] removing temporary base worktree
[license-diff] comparing inventories
[license-diff] rendering Markdown output
[license-diff] done