Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,9 +86,7 @@ jobs:
run: python3 eng/verify-vault-publish-boundary.py

- name: Test local decision service and evaluation tools
run: |
python3 -B -m unittest discover -s tests/laya-service -v
python3 -B -m unittest discover -s tests/routing-eval -p test_jev_report.py
run: dotnet test tools/laya_service/tests/LayaService.Tests.csproj -c Release --no-build

- name: Test standard targets
run: dotnet test --no-build -c Release --verbosity normal --logger "trx;LogFileName=results.trx" src/OpenClaw.Tests
Expand Down
7 changes: 7 additions & 0 deletions OpenClaw.Net.slnx
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,13 @@
<Project Path="src/OpenClaw.WhatsApp.BaileysWorker/OpenClaw.WhatsApp.BaileysWorker.csproj" />
<Project Path="src/OpenClawNet.Sandbox.OpenSandbox/OpenClawNet.Sandbox.OpenSandbox.csproj" />
</Folder>
<Folder Name="/tools/" />
<Folder Name="/tools/laya_service/">
<Project Path="tools/laya_service/LayaService.csproj" />
</Folder>
<Folder Name="/tools/laya_service/tests/">
<Project Path="tools/laya_service/tests/LayaService.Tests.csproj" />
</Folder>
<Project Path="src/OpenClaw.Agent/OpenClaw.Agent.csproj" />
<Project Path="src/OpenClaw.Channels/OpenClaw.Channels.csproj" />
<Project Path="src/OpenClaw.Core/OpenClaw.Core.csproj" />
Expand Down
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ Use this page as the map. If you are unsure where to go next, the groups below a
| [opensquilla-meta-skill-migration.md](opensquilla-meta-skill-migration.md) | OpenClaw.NET meta-skill migration notes for OpenSquilla-style DAG orchestration, current parity, and remaining migration gaps. |
| [dynamic-turn-routing-model-profiles.md](dynamic-turn-routing-model-profiles.md) | Collaboration playbook for how Dynamic Turn Routing and Model Profiles work together (precedence, field projection, fallback, and restore semantics). |
| [laya-routing.md](laya-routing.md) | Local Laya decision routing, model preparation, calibration, attribution, and rollback. |
| [zh-CN/integrations/laya-routing.md](zh-CN/integrations/laya-routing.md) | 本地 Laya 路由服务、.NET CLI、模型校验、校准与回滚(简体中文)。 |
| [jev-routing.md](jev-routing.md) | Optional Jev shadow evaluation and model routing, configuration, diagnostics, quality reports, and rollback. |
| [EXTERNAL_CLI_CONNECTORS.md](EXTERNAL_CLI_CONNECTORS.md) | Governed external CLI connectors, optional presets, named command allowlists, approvals, redaction, and audit behavior. |
| [plugins/payment.md](plugins/payment.md) | Native payment tool, virtual cards, machine payments, providers, and safe agent-facing actions. |
Expand Down
6 changes: 6 additions & 0 deletions docs/SITE_MAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,9 @@ Use this map when turning the Markdown docs into a documentation website. It kee
| Integrations | Microsoft Teams | [TEAMS_SETUP.md](TEAMS_SETUP.md) |
| Integrations | WhatsApp | [WHATSAPP_SETUP.md](WHATSAPP_SETUP.md) |
| Integrations | A2A | [a2a.md](a2a.md) |
| Integrations | Local Laya Decision Routing | [laya-routing.md](laya-routing.md) |
| Integrations | Jev Decision Routing | [jev-routing.md](jev-routing.md) |
| Integrations | Local Laya Routing (zh-CN) | [zh-CN/integrations/laya-routing.md](zh-CN/integrations/laya-routing.md) |
| Integrations | External Coding Backends | [external-coding-backends.md](external-coding-backends.md) |
| Integrations | Nacos MCP Router (PoC) | [nacos-mcp-router.md](nacos-mcp-router.md) |
| Integrations | Nacos MCP Router (PoC, zh-CN) | [zh-CN/nacos-mcp-router.md](zh-CN/nacos-mcp-router.md) |
Expand Down Expand Up @@ -146,6 +149,9 @@ Integrations
Microsoft Teams
WhatsApp
A2A
Local Laya Decision Routing
Jev Decision Routing
Local Laya Routing (zh-CN)
External Coding Backends
Nacos MCP Router (PoC)
Tailscale Deployment
Expand Down
8 changes: 8 additions & 0 deletions docs/cli/routing.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,14 @@ Router mode behavior:

`routing status` also reports `jevMode`, `jevModel`, `layaMode`, `layaModel`, and `layaCalibrationId`. See [local Laya routing](../laya-routing.md) for offline setup and calibration. Jev is disabled by default; its `shadow` and `active` modes are configured through `DynamicTurnRouting.Jev`. See [Jev routing](../jev-routing.md) for the evaluation workflow. The existing `diagnostics on|off` command controls the ONNX policy flag; Jev journaling is controlled separately by `Jev.DiagnosticsPath`.

Jev and Laya journal summaries share the standalone .NET report tool. It can write JSON and optional ScottPlot PNG reliability/risk-coverage charts; see the provider guides for label schema and examples:

```bash
dotnet run --project tools/laya_service -c Release -- report \
/path/to/decisions.snapshot.jsonl --labels /path/to/labels.jsonl \
--output /path/to/report.json --plot /path/to/reliability.png
```

Examples:

```bash
Expand Down
9 changes: 6 additions & 3 deletions docs/jev-routing.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,8 @@ The tier/profile fields describe routing preferences before the existing model s
Copy a completed journal snapshot and summarize it:

```bash
python3 scripts/evaluate-jev-routing.py /path/to/jev-decisions.snapshot.jsonl --output /tmp/jev-report.json
dotnet run --project tools/laya_service -c Release -- report \
/path/to/jev-decisions.snapshot.jsonl --output /tmp/jev-report.json
```

For representative decisions, use existing session records to assign human labels in a separate JSONL file:
Expand All @@ -72,7 +73,9 @@ For representative decisions, use existing session records to assign human label
```

```bash
python3 scripts/evaluate-jev-routing.py /path/to/jev-decisions.snapshot.jsonl --labels /path/to/labels.jsonl --output /tmp/jev-quality.json
dotnet run --project tools/laya_service -c Release -- report \
/path/to/jev-decisions.snapshot.jsonl --labels /path/to/labels.jsonl \
--output /tmp/jev-quality.json
```

Without labels the report includes coverage, failures, disagreement, latency and estimated decision spend; it makes no accuracy claim. With labels it compares the baseline, always-T2, and Jev with safeguards/fallback using accuracy, confusion matrices, per-tier F1, under-routing, and high-risk capability retention. Label coverage and per-tier sample counts matter. Separate model/rubric cohorts before calibrating thresholds.
Expand Down Expand Up @@ -104,7 +107,7 @@ Restart afterward and remove any environment override that would re-enable Jev.

```bash
dotnet test src/OpenClaw.Tests/OpenClaw.Tests.csproj -p:OpenClawSkipDashboardBuild=true --filter 'FullyQualifiedName~JevRoutingTests|FullyQualifiedName~TurnRoutingPolicyTests|FullyQualifiedName~RoutingCommandsTests'
python3 -m unittest discover -s tests/routing-eval -p test_jev_report.py
dotnet test tools/laya_service/tests/LayaService.Tests.csproj -c Release --filter 'FullyQualifiedName~RoutingJournalReportTests|FullyQualifiedName~ReliabilityPlotTests'
```

The transport follows the [TypeSafe API](https://docs.typesafe.ai/api). The model pin and initial estimated price come from its [model reference](https://docs.typesafe.ai/models), checked September 20, 2026. See [confidence semantics](https://docs.typesafe.ai/confidence) and [known model limitations](https://docs.typesafe.ai/model-jaggedness/jev-1.13) before tuning policy.
Expand Down
Loading
Loading