Problem
The OSS CLI has workloads route to configure traffic routing for an existing workload, but no workloads create command to register a new workload on the gateway. This blocks the end-to-end flow described in skills/use-understudy-gateway/SKILL.md:
- An agent runs
understudy capture-evidence workload-card → local workload-card.json is created ✅
- Agent tries to route traffic:
understudy workloads route workload-001 --model-id gemma-4-31b-it --traffic-pct 100 → 404 "Workload not found" ❌
- The workload was never registered on the gateway server.
Current behavior
$ understudy workloads route workload-001 --project-id proj_xxx --model-id gemma-4-31b-it --traffic-pct 100
error: Workload not found.
The workload-card.json exists locally but the gateway has no record of it.
Expected behavior
# Register the local workload card on the gateway
understudy workloads create --project-id proj_xxx --from-card .understudy/workload-discovery/workload-card.json
# Then route traffic to it
understudy workloads route workload-001 --project-id proj_xxx --model-id gemma-4-31b-it --traffic-pct 100
Impact
Without this, an agent using only the OSS CLI cannot:
- Set up a workload route through the gateway
- Use
understudy run with the gateway to drive an eval against an Understudy model
- Follow the A/B model routing recipe in the skill
The workaround is to either use the private CLI (which has automationbench run-official --provider lilac that handles this internally) or bypass the gateway entirely and call a provider API directly.
Related
Suggested implementation
Add a workloads create subcommand to src/commands/workloads.ts that:
- Reads a local workload-card.json
- Calls the gateway API to register the workload
- Returns the server-assigned workload-id for use with
workloads route
Problem
The OSS CLI has
workloads routeto configure traffic routing for an existing workload, but noworkloads createcommand to register a new workload on the gateway. This blocks the end-to-end flow described inskills/use-understudy-gateway/SKILL.md:understudy capture-evidence workload-card→ localworkload-card.jsonis created ✅understudy workloads route workload-001 --model-id gemma-4-31b-it --traffic-pct 100→ 404 "Workload not found" ❌Current behavior
The workload-card.json exists locally but the gateway has no record of it.
Expected behavior
Impact
Without this, an agent using only the OSS CLI cannot:
understudy runwith the gateway to drive an eval against an Understudy modelThe workaround is to either use the private CLI (which has
automationbench run-official --provider lilacthat handles this internally) or bypass the gateway entirely and call a provider API directly.Related
skills/use-understudy-gateway/SKILL.md— describes the workload route flow but assumes a workload existssrc/commands/workloads.ts— current implementation has onlyroutesubcommandSuggested implementation
Add a
workloads createsubcommand tosrc/commands/workloads.tsthat:workloads route