diff --git a/CHANGELOG.md b/CHANGELOG.md index 7f0b268c9..7187e194b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,12 @@ and this project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html) ## [0.2.0] - Unreleased +### Added + +- Add a dry-run-default OpenRouter free-model canary. Its opt-in live mode + requires explicit request, output-token, timeout, evidence-path, and retention + bounds and selects only freshly discovered, completely zero-priced chat rows. + ### Deprecated - Internal callers now use diff --git a/README.md b/README.md index 6eedf5755..f97979228 100644 --- a/README.md +++ b/README.md @@ -133,10 +133,39 @@ Model-based conduct verification requires `fast-mlsirm` in the same runtime and The agent pool is manageable at runtime: `POST`/`PATCH`/`DELETE` on `/api/v1/agent_pools/default/worker_agents[/{id}]` add, govern, and remove model-group members. Pass `--agents-db PATH` (or `CONTEXTUAL_ORCHESTRATOR_AGENTS_DB`) to persist those changes to a stdlib sqlite file — stored changes overlay the seed agents file at startup, and removals write disabled tombstones so they survive restarts; without it the pool is in-memory as before. Beyond the local MLX/llama.cpp discovery above, `python -m contextual_orchestrator discover-models [--agents-db PATH]` discovers models from remote providers (OpenAI, OpenRouter, NVIDIA NIM ×2 keys, Bytez, and an allowlisted OpenAI-compatible gateway) for any subset of their KV-registered credentials, and can persist them into the same `--agents-db` sqlite file, added disabled by default. See [docs/kv-credentials.md](docs/kv-credentials.md#multi-provider-auto-discovery) for the credential-name table and cost-based auto-selection. -Seed the credential into the KV once at bootstrap: +`python -m contextual_orchestrator openrouter-free-canary` is a dry-run-only +catalog check by default. It selects no pinned model: the lexically first +current OpenRouter chat row with explicit zero prompt and completion prices, +USD currency, and no incomparable unit price is reported without issuing a +completion. Live mode is deliberately verbose and unscheduled: ```bash -echo "$OPENAI_API_KEY" | python -m contextual_orchestrator register-credential --name OPENAI_API_KEY --value-stdin +python -m contextual_orchestrator openrouter-free-canary --live \ + --max-requests 1 --max-output-tokens 8 --timeout-seconds 10 \ + --evidence-output ./openrouter-canary.json --retention-days 7 +``` + +Every live cap and the evidence path/retention choice is mandatory. The command +uses only the KV-registered `OPENROUTER_API_KEY`, disables retries, makes one +fixed prompt request, and writes atomic JSON evidence containing neither the +prompt, response, nor credential. It records the attempt as `pending` before +transport, then atomically records the validated `OK` outcome. Before transport +it removes expired evidence at the same output path and proves that a +mode-`0600` atomic write is possible; unrelated, malformed, or unexpired files +are never overwritten; +retention cleanup therefore runs whenever the canary is invoked. The file also +records `expires_at`; run +`python -m contextual_orchestrator openrouter-free-canary +--prune-expired-evidence ./openrouter-canary.json` from the operator's chosen +retention lifecycle. Cleanup reads only that local file, verifies the canary +schema/provider/mode identity, removes it at or after its deadline, and never +resolves a credential, discovers a model, or calls a provider. This repository +deliberately adds no scheduler. + +Seed the OpenRouter credential into the KV once at bootstrap: + +```bash +echo "$OPENROUTER_API_KEY" | python -m contextual_orchestrator register-credential --name OPENROUTER_API_KEY --value-stdin ``` For a persistent KV-backed server token, seed a credential such as diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index d145a0b1d..5c45b2e7d 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -2234,10 +2234,15 @@ National Institute of Standards and Technology. (2024). *Artificial intelligence risk management framework: Generative artificial intelligence profile* (NIST AI 600-1). https://doi.org/10.6028/NIST.AI.600-1 +National Institute of Standards and Technology. (2020, updated 2023). +*Security and privacy controls for information systems and organizations* +(NIST SP 800-53 Rev. 5, release 5.1.1). https://doi.org/10.6028/NIST.SP.800-53r5 + These sources support the current product shape, OpenAI-compatible wire -honesty, deep-versus-shallow orchestration allocation, cache safety, and -generative-AI risk evidence. PDFs are attached only when redistribution is -permitted; otherwise the canonical citation and link are retained. +honesty, deep-versus-shallow orchestration allocation, cache safety, +audit-evidence protection and retention, and generative-AI risk evidence. PDFs +are attached only when redistribution is permitted; otherwise the canonical +citation and link are retained. ## 9. Design and ecosystem record @@ -2380,9 +2385,15 @@ Buyer-visible gaps now prioritized: keyboard/native-form REST editor; DB membership is normalized and legacy JSON membership migrates without data loss. Authenticated deployed-browser runtime evidence remains a release/UAT gate rather than an implementation gap. -3. Free-model tests are deterministic catalog-contract tests. Add an opt-in, - spend-capped live OpenRouter canary selected from current zero-price metadata; - never pin a transient free model identifier in production or CI. +3. **Implemented on protected `main`:** the OpenRouter free-model + canary is dry-run-only by default and selects a current chat row only when + prompt and completion prices are explicitly zero and comparable. Live mode + is unscheduled and requires positive request, output-token, timeout, evidence + retention, and output-path choices; it disables retries, pins no model id, + and persists neither credentials, prompts, nor responses. Its atomic, + path-serialized attempt ledger and operator-chosen retention implement the + audit-record protection and retention intent of NIST SP 800-53 Rev. 5.1 + controls AU-9 and AU-11. 4. Multi-instance routing observations remain process-local. Add a time-windowed durable observation model with calibrated decay before horizontal scaling. 5. Protected main, not a feature-stack merge, remains the release boundary; do