Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
8466b60
refactor(relay): trim plugin-only surface
bbednarski9 Aug 4, 2026
ccdcb53
docs(relay): keep plugin scope isolated
bbednarski9 Aug 4, 2026
2e974a7
fix(relay): prevent repeated stream fallback
bbednarski9 Aug 4, 2026
e3407b8
fix(relay): reject empty translated streams
bbednarski9 Aug 4, 2026
8930c2f
fix(relay): back off routing retries
bbednarski9 Aug 4, 2026
a8a97aa
test(relay): cover single stream fallback
bbednarski9 Aug 4, 2026
41c7547
fix(relay): harden response finalization
bbednarski9 Aug 4, 2026
5ef6952
chore(relay): update SDK to 0.7.0-rc.5
bbednarski9 Aug 4, 2026
eb0becd
fix(relay): flush marks before fallback errors
bbednarski9 Aug 4, 2026
2fa016d
fix(relay): require environment-backed target headers
bbednarski9 Aug 4, 2026
1d11cbc
chore(relay): use stable 0.7.0 SDK
afourniernv Aug 5, 2026
6202e02
fix(relay): cap runtime compatibility at 0.8
afourniernv Aug 5, 2026
e707944
fix(relay): drop unsupported caller extra body
bbednarski9 Aug 6, 2026
9085873
fix(relay): restore async native middleware bridge
bbednarski9 Aug 6, 2026
54a936f
fix(relay): adapt plugin to current library contracts
bbednarski9 Aug 6, 2026
5d9d329
fix(relay): align plugin schema with classifier config
bbednarski9 Aug 6, 2026
c19d9ab
feat(relay): add native routing plugin
bbednarski9 Aug 3, 2026
e477a1c
docs(relay): document native plugin integration
bbednarski9 Aug 3, 2026
cdfb8b7
refactor(relay): emit cdylib only
bbednarski9 Aug 3, 2026
230a91d
refactor(relay): minimize plugin bundle
bbednarski9 Aug 3, 2026
f1ae243
refactor(relay): use public Rust plugin SDK
bbednarski9 Aug 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,14 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added

- **NeMo Relay native plugin** — a dynamically loaded integration that runs
libsy's weighted-random and LLM-classifier algorithms in process while
Switchyard owns provider HTTP dispatch, credentials, translation, retries,
and fallback. Managed calls require NeMo Relay 0.7 or newer and do not depend
on `switchyard-server`.

### Removed

- **Latency-aware router** — the `latency_service` route type and its
Expand Down
141 changes: 141 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ members = [
"crates/switchyard-server",
"crates/switchyard-skill-distillation",
"crates/switchyard-translation",
"crates/switchyard-nemo-relay-plugin",
]

[workspace.package]
Expand All @@ -23,6 +24,7 @@ repository = "https://github.com/NVIDIA-NeMo/Switchyard"
rust-version = "1.96.1"

[workspace.dependencies]
async-channel = "2"
async-stream = "0.3"
async-trait = "0.1"
futures = "0.3"
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ algorithm you write yourself.
- **Protocol Translation**: convert between OpenAI Chat, Anthropic Messages, and OpenAI Responses formats
- **Multi-Backend Routing**: random routing, LLM-as-classifier routing, signal-driven stage-router, or your own algorithm
- **Operational Metrics**: Prometheus metrics cover requests, errors, latency, tokens, and routing overhead
- **NeMo Relay Plugin**: run random or LLM-classifier routing in Relay while Switchyard owns provider HTTP dispatch

## Quick Start

Expand Down Expand Up @@ -119,6 +120,7 @@ configured LLM client selects one upstream format.
- **[`switchyard-libsy`](crates/libsy/README.md)**: embed routing algorithms in a Rust application
- **[`switchyard-protocol`](crates/protocol/README.md)**: provider-neutral request, response, and streaming types
- **[`switchyard-translation`](crates/switchyard-translation/README.md)**: request, response, and stream translation
- **[`switchyard-nemo-relay-plugin`](crates/switchyard-nemo-relay-plugin/README.md)**: install Switchyard as a native NeMo Relay plugin

## Community

Expand Down
30 changes: 30 additions & 0 deletions crates/switchyard-nemo-relay-plugin/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

[package]
name = "switchyard-nemo-relay-plugin"
version = "0.1.0"
description = "Switchyard-owned HTTP routing plugin for NeMo Relay"
authors.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
rust-version.workspace = true
publish = false

[lib]
crate-type = ["cdylib"]

[dependencies]
async-channel.workspace = true
async-trait.workspace = true
futures-util.workspace = true
http = "1"
nemo-relay-plugin = "=0.7.0"
serde.workspace = true
serde_json.workspace = true
switchyard-libsy.workspace = true
switchyard-llm-client.workspace = true
switchyard-protocol.workspace = true
switchyard-translation.workspace = true
tokio.workspace = true
Loading
Loading