diff --git a/showcase/kleos-research/README.md b/showcase/kleos-research/README.md new file mode 100644 index 0000000..26afaaf --- /dev/null +++ b/showcase/kleos-research/README.md @@ -0,0 +1,99 @@ +# Kleos Research + +The layer between agents and the models they run on. Two things live here: +**0xCopilot**, a local-first desktop harness, and **Kaleidoscope**, memory for +agents that outlives the session. + +## 0xCopilot — the harness + +An Electron desktop app that supervises an embedded PostgreSQL and three Python +services on `localhost`. Nothing is hosted: the runtime, the store, and the +conversation history sit on the user's own disk. + +Relevant to EconomyOS: it runs on **your own key or a Virtuals ACP key**. The +model picker is built from Virtuals' live catalogue — roughly sixty models from +ten vendors behind one OpenAI-wire endpoint — and carries **real per-Mtok +pricing on every row**, so the cost of a task is legible before it is spent +rather than after. The catalogue is discovered rather than hardcoded, so a model +Virtuals adds appears without a code change. + +`$CPILOT` is tokenised through Virtuals +([listing 113720](https://app.virtuals.io/virtuals/113720)). + +## Kaleidoscope — the memory + +Filesystem-native, continually adapting knowledge for agent harnesses: +user-owned inspectable files, no database server, no memory-owned model, and +harness-neutral integration. It preserves decisions, evidence, and outcomes +across sessions, then retrieves a small task-scoped context under explicit +latency and token bounds. + +It is measured on **BEAM 100K** — ten memory abilities over conversations from +100K tokens up — through an +[open-source evaluation suite](https://github.com/kleos-research/kaleidoscope-benchmarks) +that shells out to the real `kscope` binary rather than reimplementing it, so +what gets measured is what ships. + +Held-out results are published in +[Optimising for memory recall](https://kleosresearch.xyz/research/optimising-for-memory-recall.pdf): + +- a full scan of **100,000 memories in 3.04 ms**, with no index +- **32 bytes per memory retains 99.7%** of the retrieval quality of 1024 +- **96% of the recall of a model four times its size**, from 7.2 MB + +Caveats travel with the numbers: the encoder results are configured on a train +split and reported on held-out test over externally-authored corpora, and the +timing is one machine — the ratios travel, the absolute milliseconds do not. + +## The skill + +[`skills/0xcopilot-local-workspace`](skills/0xcopilot-local-workspace) — stand up +a local agent workspace and point it at Virtuals compute, so work runs on +EconomyOS credits with per-Mtok pricing visible before a task is spent. + +It is deliberately a workspace skill and not a task-runner: the CLI is a +launcher and installer with no `copilot run ""` verb, and the skill says so +under "when NOT to use this" rather than implying a pipeline that does not exist. +Two steps are marked as human-only handoffs — signing in, and entering the +provider key — because an agent should not be typing an operator's credential. + +**There is no Kaleidoscope skill, and that is not an oversight.** The evaluation +suite is open source, but it requires the `kscope` binary on `PATH`, and +Kaleidoscope itself is not yet published. A skill whose first step no reader can +perform is worse than no skill. (Note also that the `kscope` package on PyPI is +an unrelated project by another author — it is not this one.) When Kaleidoscope +ships publicly, a memory skill belongs here. + +## Why the proof is shaped like this + +There is no demo video here, and the benchmark artifact is a findings file +rather than a headline. That is deliberate. + +[`tools/harness-bench/FINDINGS.md`](https://github.com/0x-copilot-dev/0x-copilot/blob/main/tools/harness-bench/FINDINGS.md) +is the harness benchmark, measured on the packaged app against a live model and +scored from the same records the product bills from. Its headline is a real +win — raising an inherited step ceiling took task completion from **3-of-4 to +4-of-4 for +0.1% tokens**, because the ceiling had been terminating real work +with `recursion_limit_exceeded`. It also carries, in the same file, the +correction of an earlier version of itself that had declared that finding +falsified on the strength of a metric structurally blind to the failure it was +measuring. + +The same discipline is why nothing here is quoted against another memory +system. A comparison that looked significant was withdrawn once an aggregation +fault was found — one side had been scored across nine abilities and the other +across ten — and the corrected interval crosses zero. It is not published +because it is not settled. + +Claims that survive replication are the ones on the card. + +## Links + +| | | +| --- | --- | +| Harness source | https://github.com/0x-copilot-dev/0x-copilot | +| CLI | https://www.npmjs.com/package/@0x-copilot/cli | +| Desktop app | https://copilot.kleosresearch.xyz | +| Kaleidoscope | https://memory.kleosresearch.xyz | +| Evaluation suite | https://github.com/kleos-research/kaleidoscope-benchmarks | +| The lab | https://kleosresearch.xyz | diff --git a/showcase/kleos-research/assets/hero-card.png b/showcase/kleos-research/assets/hero-card.png new file mode 100644 index 0000000..091a626 Binary files /dev/null and b/showcase/kleos-research/assets/hero-card.png differ diff --git a/showcase/kleos-research/showcase.json b/showcase/kleos-research/showcase.json new file mode 100644 index 0000000..4c204c1 --- /dev/null +++ b/showcase/kleos-research/showcase.json @@ -0,0 +1,110 @@ +{ + "slug": "kleos-research", + "title": "Kleos Research", + "tagline": "Runs agents locally on your own keys or Virtuals compute, meters what every run actually costs, and gives them a filesystem-native memory that outlives the session", + "description": "Kleos Research builds the layer between agents and the models they run on. 0xCopilot is the local-first desktop harness: bring your own key, or a Virtuals ACP key, and pick from a catalogue carrying live per-Mtok pricing so the cost of a task is visible before you spend it. Kaleidoscope is filesystem-native memory for agents — harness-agnostic, user-owned files, no database server and no memory-owned model — which scans 100,000 memories in 3.04 ms with no index, and holds 99.7% of its retrieval quality at 32 bytes per memory. It is measured on the BEAM 100K tier through an open-source evaluation suite that shells out to the real binary rather than reimplementing it. Both publish their measurements in the open, retractions included: the harness benchmark records a step-ceiling fix worth 3-of-4 to 4-of-4 task completion for +0.1% tokens, and prints the claims it has withdrawn beside the ones that held.", + "status": "live", + "topic": "agents", + "topics": [ + "agents", + "memory", + "acp", + "compute", + "virtuals", + "token", + "robinhood-chain", + "developer-tools", + "privacy" + ], + "builder": { + "name": "Kleos Research", + "url": "https://kleosresearch.xyz" + }, + "links": { + "repo": "https://github.com/0x-copilot-dev/0x-copilot", + "share": "https://x.com/Copilot0x/status/2078188003479195781", + "feedback": "https://github.com/Virtual-Protocol/acp-cli-demos/issues/new?title=Feedback%3A%20Kleos%20Research", + "demo": "https://copilot.kleosresearch.xyz" + }, + "primitives": [ + "acp", + "token" + ], + "visual": { + "kind": "local-first agent harness with metered runs and portable memory", + "eyebrow": "local · byok · virtuals compute", + "title": "the layer between agents and models", + "posterUrl": "https://raw.githubusercontent.com/Virtual-Protocol/acp-cli-demos/main/showcase/kleos-research/assets/hero-card.png" + }, + "skills": [ + { + "name": "0xcopilot-local-workspace", + "href": "https://github.com/Virtual-Protocol/acp-cli-demos/tree/main/showcase/kleos-research/skills/0xcopilot-local-workspace", + "sourcePath": "showcase/kleos-research/skills/0xcopilot-local-workspace", + "summary": "Stand up a local-first agent workspace on the operator's own machine and point it at Virtuals compute, so work runs on EconomyOS credits with per-Mtok pricing visible before a task is spent. Covers install, the human-only credential handoff, validation, and the stop conditions — including that the CLI is a launcher, not a batch runner.", + "install": "cp -R showcase/kleos-research/skills/0xcopilot-local-workspace ~/.agents/skills/\ncp -R showcase/kleos-research/skills/0xcopilot-local-workspace ~/.claude/skills/" + } + ], + "artifacts": [ + { + "label": "Harness benchmark findings — a measured completion win, the real cost structure, and two published retractions", + "href": "https://github.com/0x-copilot-dev/0x-copilot/blob/main/tools/harness-bench/FINDINGS.md", + "kind": "proof" + }, + { + "label": "0xCopilot source — Electron desktop app supervising an embedded Postgres and three Python services on localhost", + "href": "https://github.com/0x-copilot-dev/0x-copilot", + "kind": "repo" + }, + { + "label": "Virtuals compute integration — catalog discovery against compute.virtuals.io with live per-Mtok pricing", + "href": "https://github.com/0x-copilot-dev/0x-copilot/blob/main/services/ai-backend/src/agent_runtime/api/virtuals_model_source.py", + "kind": "proof" + }, + { + "label": "0xCopilot CLI releases — install the desktop app from npm", + "href": "https://github.com/0x-copilot-dev/0x-copilot/releases", + "kind": "demo" + }, + { + "label": "Optimising for memory recall — held-out results behind the 3.04 ms scan and the 32-byte memory (PDF)", + "href": "https://kleosresearch.xyz/research/optimising-for-memory-recall.pdf", + "kind": "proof" + }, + { + "label": "Kaleidoscope — filesystem-native, continually adapting memory for agent harnesses", + "href": "https://memory.kleosresearch.xyz", + "kind": "docs" + }, + { + "label": "Kaleidoscope BEAM evaluation suite — open-source harness that runs the BEAM 100K tier against the real binary", + "href": "https://github.com/kleos-research/kaleidoscope-benchmarks", + "kind": "repo" + }, + { + "label": "0xCopilot CLI on npm — install the desktop app with one command", + "href": "https://www.npmjs.com/package/@0x-copilot/cli", + "kind": "demo" + }, + { + "label": "Kaleidoscope memory-recall paper — announcement", + "href": "https://x.com/Copilot0x/status/2089381387048472685", + "kind": "post" + }, + { + "label": "Kleos Research — the lab behind 0xCopilot and Kaleidoscope", + "href": "https://kleosresearch.xyz", + "kind": "docs" + }, + { + "label": "$CPILOT on Virtuals — the live listing", + "href": "https://app.virtuals.io/virtuals/113720", + "kind": "demo" + } + ], + "feedbackPrompts": [ + "Does live per-Mtok pricing in the model picker change which model you would pick for a task?", + "What would you need to see before trusting an agent memory that lives as plain files on your own disk?", + "Which matters more in a harness you pay for: a lower cost per task, or a higher share of tasks that actually finish?" + ] +} diff --git a/showcase/kleos-research/skills/0xcopilot-local-workspace/SKILL.md b/showcase/kleos-research/skills/0xcopilot-local-workspace/SKILL.md new file mode 100644 index 0000000..73d2563 --- /dev/null +++ b/showcase/kleos-research/skills/0xcopilot-local-workspace/SKILL.md @@ -0,0 +1,105 @@ +# 0xCopilot local workspace on Virtuals compute + +Stand up a local-first agent workspace on the operator's own machine and point +it at Virtuals compute, so agent work runs on EconomyOS credits instead of a +personal provider bill — with the model catalogue carrying live per-Mtok pricing +so the cost of a task is legible before it is spent. + +## When to use this + +- The operator has Virtuals credits (or their own provider key) and wants a + local harness rather than a hosted one. +- Conversations, runs, and artifacts must stay on the operator's disk. +- You are asked to verify or repair an existing 0xCopilot install. + +## When NOT to use this + +- **You need a headless, scriptable agent run.** This CLI is a launcher and + installer, not a batch runner — there is no `copilot run ""`. Work is + given to the agent in the app. If you need automation, drive the product's + HTTP API instead; do not pretend this skill gives you a pipeline. +- The target is Linux, or Windows on ARM. Not supported. +- The operator wants a shared/multi-tenant deployment. This is single-user and + local by design. + +## Preconditions + +| | | +| --- | --- | +| Node.js | 20+ | +| OS | macOS (Apple Silicon or Intel) or Windows x64 | +| Network | required on first launch | +| Disk | several hundred MB for the runtime, plus app data | +| Credential | a Virtuals ACP key, or an OpenAI / Anthropic / Google key | + +## Approval gates — stop and ask before each + +1. **First install downloads several hundred megabytes.** Get explicit consent + before running `copilot` or `copilot install` on a metered or constrained + connection. +2. **Running agent tasks spends credits.** Virtuals compute draws down the + operator's balance. Confirm a budget before starting work, not after. +3. **`copilot uninstall` deletes all local 0xCopilot data** — conversations, + runs, artifacts. It is not recoverable. Never run it to "clean up" without an + explicit instruction naming the consequence. Prefer `copilot repair`, which + recovers a stuck launch while keeping data. +4. **Never type, echo, or store the operator's provider key.** It is entered by + a human, in the app, and lives in the OS keychain. + +## Steps + +```bash +npm install -g @0x-copilot/cli # or: bun add -g @0x-copilot/cli +copilot # prepares the runtime, then launches the app +``` + +Then **hand off to the human** for two steps an agent must not perform: + +1. Sign in (wallet, or Google where enabled). +2. **Settings → Models & keys → Provider keys** → add the Virtuals key. Selecting + Virtuals as the provider exposes its catalogue — roughly sixty models across + ten vendors behind one endpoint, each row carrying real per-Mtok pricing. + +Give the agent a goal in the composer once a key is saved. + +## Stop conditions + +- `copilot doctor` reports an unmet requirement → report its output verbatim and + stop. Do not attempt to work around a missing runtime. +- The launch hangs → `copilot repair` (keeps data). Add `--session` only if the + operator agrees to clear saved sign-ins. Escalate rather than reinstalling. +- A run fails with a provider or service error → this is a credential or + upstream-availability problem, not a task problem. Re-check the key before + retrying, and never retry in a loop against a paid endpoint. + +## Validation + +```bash +copilot version # prints the installed version +copilot doctor # diagnoses the setup and prints what it finds +``` + +The install is good when `doctor` reports no unmet requirements and the app +launches to a workspace. The *key* is good when the composer's model pill +resolves to a real model rather than a placeholder — a saved key is not proof of +a working one, because the catalogue endpoint answers publicly and says nothing +about your credential. + +## Output contract + +On success, report: + +- installed CLI version, +- `doctor` verdict (unmet requirements, if any), +- whether a provider key is configured — **as a boolean, never the value**, +- the local data location, so the operator knows what `uninstall` would remove. + +On failure, report the failing command, its exact output, and which stop +condition above was hit. Do not summarise an error into prose — the product's +typed errors are more precise than a paraphrase of them. + +## Links + +- CLI package — https://www.npmjs.com/package/@0x-copilot/cli +- Source — https://github.com/0x-copilot-dev/0x-copilot +- App — https://copilot.kleosresearch.xyz