Skip to content

feat: memory map, usage metering and value metering — GET /memories/field, /spend and /value - #2071

Merged
erni-a merged 11 commits into
mainfrom
CAURA-000-value-metering
Oct 13, 2026
Merged

feat: memory map, usage metering and value metering — GET /memories/field, /spend and /value#2071
erni-a merged 11 commits into
mainfrom
CAURA-000-value-metering

Conversation

@erni-a

@erni-a erni-a commented Oct 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

The memory map and metering for the enterprise Prism › Map, Home and Insights › Value, in one PR to save serial CI cycles (it replaces #2069 and #2070). Eleven commits in three parts, each readable on its own:

1. GET /api/v1/memories/field — the memory map (2 commits)

  • Two-level PCA over the memory embeddings, k-means clusters labelled from the entities their memories mention, percentile normalisation so outlying clusters don't squeeze the rest into a corner.
  • core-storage gains field-page / field-signature reads; the layout is cached in-process for 10 minutes, keyed by that signature.
  • docs/api-reference.md documents the route.

2. Usage metering — migration 066, GET /api/v1/spend (3 commits)

  • common/usage: a scope (@scoped) and an aggregator; provider hooks read usage from OpenAI, Vertex/Gemini (usage_metadata) and the rest.
  • Migration 066: llm_usage_minutes, value_usage_minutes, usage_flush_batches.
  • Savings recorded: recall's source-vs-returned tokens, cross-agent reuse, crystallizer and near-duplicate retirements. Nothing compared with "the whole corpus", no hours estimates.
  • Storage: POST /usage/flush?tenant_id= and POST /usage/query?tenant_id=, both bound to one tenant. A writer sends one flush per tenant (common.usage.per_tenant); each tenant's batch id is a uuid5 of the batch's, so a re-sent batch, even one that half landed, is counted once.
  • GET /api/v1/spend (tenant or admin credentials only). Named spend because the enterprise gateway already owns /api/v1/usage for plan counters.
  • observability.usage_metering_enabled gates it; docs/usage-metering.md documents what is and isn't metered.

3. Value metering — migration 067, GET /api/v1/value (6 commits)

  • Sessions: MCP issues an Mcp-Session-Id label, REST takes X-Caura-Session-Id, and a broker's metadata.session_id wins.
  • Migration 067: agent_sessions, memory_origins, memory_reuse (tenant-bound routes, no allowlist growth).
  • Origin of a memory: minutes since the session started or last wrote, and calls (a lower bound), plus tokens where a hook, the OpenClaw plugin or the SDKs report them (MemoryCreate.origin).
  • Reuse is counted once per (session, memory), never in the memory's own session. "Sessions helped" counts only reuse of memories with a recorded origin; "mistakes prevented" counts resolved contradictions and retired outdated memories, with pending ones shown as waiting.
  • Evolve outcomes confirm reuse (memory_reuse.outcome).
  • GET /api/v1/value, read by the enterprise Insights › Value page and Home.
  • The plugin sends X-Caura-Session-Id and per-turn origin; the Python and TypeScript SDKs take session_id / origin.

Type of Change

  • New feature (non-breaking change that adds functionality)

How Has This Been Tested?

All of CI's static checks pass locally on this branch, the tenant-scope gate included (lock files, broker OpenAPI, every ruff and format scope, the MCP guards, mypy ×6, the legacy-name ratchet, do-not-touch, the evidence registry). Each part's tests passed on its own branch: root suite 10,087 / 10,094 / 10,110, core-api + core-storage 1,035 / 1,044, migrations 065 → 066 → 067 on a fresh database, plugin 574/576 (the two resolveAgentId fallback warn tests fail the same way on main on the test machine), Python client 229, TypeScript client 68. The three parts combined without a conflict.

Additional Notes

No tenant-scope allowlist growth. Every new storage path is bound to one tenant: the flush takes a required tenant_id and writes every row under it, so a flush can't reach another tenant's totals (test_a_flush_writes_only_the_tenant_it_names).

  • The memory field is computed per process with no table yet. Before broad rollout it needs a bound on concurrent builds and persisted coordinates (prod core-api is 4 CPU / 8 GiB, concurrency 40). The enterprise map is behind features.prism_field, off by default.
  • Named /spend because the enterprise gateway already owns /api/v1/usage. caura-ops' Model Spend page reads these tables once 066 is in production.
  • Fixed while rebasing: plugin/src/origin-usage.ts wasn't in the plugin install lists (_plugin_files, the install script's SRC_FILES, FALLBACK_SRC_FILES), so fresh installs would fail with TS2307; test_plugin_source_manifest.py covers it.
  • Deferred: tokens from the Claude Code / Codex broker hook need origin on BulkMemoryItem, part of the frozen broker contract.

@erni-a
erni-a requested a review from a team as a code owner October 13, 2026 09:34
@erni-a
erni-a force-pushed the CAURA-000-value-metering branch from 15d7b67 to 3ad1578 Compare October 13, 2026 10:11
@erni-a erni-a changed the title feat(value): agent sessions, memory origins and reuse, and GET /api/v1/value feat: memory map, usage metering and value metering — GET /memories/field, /spend and /value Oct 13, 2026
@erni-a
erni-a changed the base branch from CAURA-000-usage-metering to main October 13, 2026 10:11
This was referenced Oct 13, 2026
Closed
Closed
@github-actions

github-actions Bot commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Claude Code Review — skipped: PR author 'erni-a' is not a public member of the 'caura-ai' org

1 similar comment
@github-actions

github-actions Bot commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Claude Code Review — skipped: PR author 'erni-a' is not a public member of the 'caura-ai' org

erni-a added 2 commits October 13, 2026 13:17
@erni-a
feat(core-api): GET /memories/field — 2D map layout of a tenant's mem…
d4db36f 
…ories

Adds a read that lays out the memories a caller can list on a 2D map,
computed from their embeddings with numpy only: PCA to <=50 dims, k-means
(k = clamp(round(sqrt(n/2)), 2, 24), fixed seed), centroids placed by a
PCA of the centroids plus repulsion, members placed by each cluster's
local 2D PCA at a radius proportional to sqrt(size), normalised to [0, 1]
with 6% padding. Deterministic for the same inputs. Memories with no
embedding yet appear with cluster -1 near the centre. Clusters carry a
short label from their most distinctive linked entities (tf-idf style),
falling back to memory type plus a frequent title word.

Same tenant check, fleet-read gate and visibility scoping as
GET /memories; superseded and archived rows are included, deleted and
held rows are not. Results are cached in-process for 10 minutes per
caller scope and storage signature (count, embedded count, newest
write/status change). Persisting the fitted model is a later step.

core-storage-api gains POST /memories/field-signature and
POST /memories/field-page (keyset pages of up to 2000 compact rows with
base64 float32 embeddings, linked entity ids, 240-char preview and
content length). _numpy_kmeans moves to services/kmeans.py, shared with
the insights discover mode.

Signed-off-by: erni <erni@caura.ai>
@erni-a
fix(core-api): memory field fills the square despite outlying clusters
a81d9f6 
One far-off tiny cluster squeezed every other cluster into a corner,
because the final normalisation used min/max and the centroid PCA let one
outlying centroid set the scale. Now:

- each axis is normalised by the 3rd-97th percentile of the point
  coordinates to [0.06, 0.94], with outliers clamped into [0.02, 0.98];
- centroid anchors are centred on their size-weighted mean and
  compressed per axis (signed square root) before repulsion;
- repulsion never moves an anchor more than 0.5 from its PCA position.

Also registers the field reads' fleet predicate in the C27 strict fleet
scoping inventory (strict, org-blind, as memory_list_by_filters).

Signed-off-by: erni <erni@caura.ai>
@erni-a
erni-a force-pushed the CAURA-000-value-metering branch from 799b13c to c0122b8 Compare October 13, 2026 10:17
@github-actions

github-actions Bot commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Claude Code Review — skipped: PR author 'erni-a' is not a public member of the 'caura-ai' org

erni-a added 9 commits October 13, 2026 13:27
@erni-a
feat(usage): record every model call's tokens per tenant, agent and o…
868b878 
…peration

Every LLM and embedding call the server makes is now counted with the
provider's own token figures (input, output, reasoning) and attributed to
the tenant, agent, fleet and operation it was made for.

- common/usage: a usage scope (ContextVar, plus a @Scoped decorator for
  entry points), record_model_call / record_value, and a per-minute
  in-memory aggregator flushed every 15 s.
- Providers report usage: OpenAI-compatible chat and embeddings (the
  counts were already read, then only logged), Vertex and Gemini
  (usage_metadata was discarded). The retry layer counts failed attempts
  separately, so a billed-then-rejected response is never two calls.
- Attribution at the entry points: writes, search, recall, enrichment,
  entity extraction, contradiction, crystallization, forge, ingest,
  evolve, interview, digests, queue consumers and lifecycle actions;
  tracked background tasks inherit their tenant. Request paths fall back
  to the signed-in tenant and the MCP agent. A call with no tenant is
  recorded as _unattributed rather than dropped.
- Storage (migration 066): llm_usage_minutes and value_usage_minutes as
  additive per-minute upserts, and usage_flush_batches so a re-sent
  flush is counted once. POST /usage/flush (cross-tenant by design, like
  the capability and tenant usage flushes) and POST /usage/query, bound
  to one tenant. Both tables join the tenant and fleet purges.
- observability.usage_metering_enabled (default on) turns it off per
  workspace; storage drops that tenant's rows at flush time, for
  core-api and core-worker alike.

Counts only, never content, kept in the deployment's own database.

Signed-off-by: erni <erni@caura.ai>
@erni-a
feat(usage): record the recall and dedup savings Caura can stand behind
8294f35 
Only measures with both sides known exactly:

- Recall (RecordRecallValue, the search pipeline's last step, and the
  legacy path so the hotfix lever keeps it): recalls, results, tokens
  returned, results written by another agent, and for results that are a
  piece of a larger memory (auto-chunks and atomic facts, which name
  their parent through the reserved parent_memory_id) the tokens returned
  and the full size of the distinct parents. A whole memory returned as
  itself claims no saving; a deleted parent claims nothing; diagnostic
  searches are not recalls. Runs after the response, one storage read
  for the parent sizes.
- Duplicate merges: crystallization records the memories and tokens it
  retired and the tokens it wrote back, once the sources are archived;
  the write-path near-duplicate merge records the row it retired (the
  agent's own new write is not offset against it).

Not recorded: savings against sending every memory, or anything not
observed. Imported documents are left out: the server keeps no
verifiable size for the source, and a client-supplied one would be
forgeable. Tokens are tiktoken cl100k_base, which core-api already ships.

Signed-off-by: erni <erni@caura.ai>
@erni-a
feat(usage): GET /api/v1/spend, and document what is metered
abdde01 
GET /api/v1/spend returns one workspace's model calls (provider-reported
tokens in, out and reasoning, failed attempts) and recall savings over a
window, grouped by day, agent, fleet, operation, kind, provider, model or
metric, with a summary: total tokens, context saved (parent tokens less
returned), cross-agent results and duplicates retired.

Spend per agent is an admin-plane report, gated like tenant settings:
tenant and admin credentials read it; agent keys and org members get 403;
a tenant outside the caller's readable set is refused.

docs/usage-metering.md says what is recorded and what is deliberately
not, how it is stored and counted, and how a workspace turns it off.

Signed-off-by: erni <erni@caura.ai>
@erni-a
feat(value): agent sessions, memory origins and reuse, and GET /api/v…
1b93c02 
…1/value

Value metering: what the memory does for a customer's agents, counted per
session (docs/plans/value-metering.md).

- Sessions. MCP stays stateless but now issues a session label
  (Mcp-Session-Id) on the first response and reads it back; REST and SDKs
  send X-Caura-Session-Id; the broker's metadata.session_id wins on its
  writes. A per-process tracker counts calls per session and flushes per
  tenant.
- Origin. After a write, the memory's session work is recorded once:
  minutes since the session started or last wrote, calls to Caura in that
  stretch (a lower bound on turns), and tokens/turns when a hook, plugin
  or SDK reported them in the new `origin` field. The MCP write tool does
  not take it: a model cannot observe its own token use.
- Reuse. After a recall, each memory from another session is recorded the
  first time this session recalls it; the primary key makes "once per
  session" structural, and a session never reuses its own memory.
- Storage (migration 067): agent_sessions, memory_origins, memory_reuse,
  every route bound to one tenant; origins and reuse cascade with their
  memory and are listed in the tenant purge.
- GET /api/v1/value: sessions helped, work reused (minutes, calls,
  reported tokens with their coverage), knowledge shared with the agents
  others learn from most and who learns most, mistakes prevented, and
  Caura's own cost as one line. Tenant and admin credentials only.

Signed-off-by: erni <erni@caura.ai>
@erni-a
fix(value): count help only where the work is known, and only resolve…
3dab964 
…d contradictions

Sessions helped now needs a recalled memory with a recorded origin; memories
older than value metering still count as reuse, reported separately. Mistakes
prevented counts contradictions a person resolved (dismissed ones were false
alarms), and reports the ones still waiting for review instead of counting
them.

Signed-off-by: erni <erni@caura.ai>
@erni-a
feat(value): report the work behind a memory from the OpenClaw plugin…
a6129d1 
… and the SDKs

- OpenClaw plugin: each afterTurn adds the provider-reported usage of the
  assistant messages not counted before (by timestamp, so a history handed
  over again is not counted twice), per OpenClaw session. The plugin's
  next write in that session sends it as `origin` (source plugin) with
  the turn count, then starts again. Its writes and recall searches carry
  the session as X-Caura-Session-Id. Turns without reported usage send
  turns only, never a guess.
- Python and TypeScript clients: `session_id` / `sessionId` and
  `set_session()` / `setSession()` put calls in a session; `write()` takes
  `origin` (tokens in, out, turns from the caller's own provider usage),
  sent as SDK-reported.

Signed-off-by: erni <erni@caura.ai>
@erni-a
fix(value): record the origin of memories written in bulk, the broker…
190cbba 
…'s path

The broker writes through /memories/bulk with its session on each item's
metadata, and bulk writes recorded no origin. Each item this attempt created
now gets one, a microsecond after the previous item so a batch is one stretch
of work carried by its first item rather than repeated for every item.

Signed-off-by: erni <erni@caura.ai>
@erni-a
feat(value): confirm reuse from Evolve outcomes
12ff91d 
An Evolve report names the memories the outcome relied on. When the reporting
session had reused one of them from another session, that reuse is marked with
the outcome (latest report wins); naming a memory the session never reused
marks nothing. GET /api/v1/value adds work_reused.confirmed: the reuses, minutes
and reported tokens behind reported successes, and the reuses that preceded a
failure.

Signed-off-by: erni <erni@caura.ai>
@erni-a
docs(value): plan status — D and E done, broker hook deferred
67cf25e 
Signed-off-by: erni <erni@caura.ai>
@erni-a
erni-a force-pushed the CAURA-000-value-metering branch from c0122b8 to 67cf25e Compare October 13, 2026 10:27
@github-actions

github-actions Bot commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Claude Code Review — skipped: PR author 'erni-a' is not a public member of the 'caura-ai' org

@erni-a

erni-a commented Oct 13, 2026

Copy link
Copy Markdown
Contributor Author

@ran-taig Please approve

@erni-a

erni-a commented Oct 13, 2026

Copy link
Copy Markdown
Contributor Author

@arkash20 please approve

@erni-a
erni-a merged commit da9fc57 into main Oct 13, 2026
24 checks passed
@erni-a
erni-a deleted the CAURA-000-value-metering branch October 13, 2026 10:42
@caura-deploy-bot caura-deploy-bot Bot mentioned this pull request Oct 13, 2026
Merged
erni-a pushed a commit that referenced this pull request Oct 13, 2026
@caura-deploy-bot
chore: release main (#2072)
cece4cc 
🤖 I have created a release *beep* *boop*
---


<details><summary>backend: 3.33.0</summary>

##
[3.33.0](backend-v3.32.0...backend-v3.33.0)
(2026-10-13)


### Features

* memory map, usage metering and value metering — GET /memories/field,
/spend and /value
([#2071](#2071))
([da9fc57](da9fc57))


### Documentation

* **skill:** three lessons from real agents — a miss isn't proof, don't
save found-nothing, leave memory_type to the server
([#2068](#2068))
([f6e7578](f6e7578))
</details>

<details><summary>plugin: 2.25.0</summary>

##
[2.25.0](plugin-v2.24.7...plugin-v2.25.0)
(2026-10-13)


### Features

* memory map, usage metering and value metering — GET /memories/field,
/spend and /value
([#2071](#2071))
([da9fc57](da9fc57))


### Documentation

* **skill:** three lessons from real agents — a miss isn't proof, don't
save found-nothing, leave memory_type to the server
([#2068](#2068))
([f6e7578](f6e7578))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Signed-off-by: release-please[bot] <release-please[bot]@users.noreply.github.com>
Co-authored-by: caura-deploy-bot[bot] <265395343+caura-deploy-bot[bot]@users.noreply.github.com>
@eldad-caura-ai eldad-caura-ai mentioned this pull request Oct 13, 2026
Merged
1 task
eldad-caura-ai added a commit that referenced this pull request Oct 15, 2026
@eldad-caura-ai @claude
release(clients): caura-client 1.1.0 for Python and TypeScript (#2079)
5f6d805 
## Summary

Bumps both SDK clients to **1.1.0** so the next publish carries what has
merged since the last releases (O14). Neither client has been published
since `caura-client` 1.0.3 on PyPI (tag `caura-client-v1.0.3`) and
`@caura/client` 1.0.2 on npm (tag `caura-client-ts-v1.0.2`).

Versions only: `clients/python/pyproject.toml` and
`caura_client/_version.py`; `clients/typescript/package.json`,
`package-lock.json` (its own two entries) and `src/version.ts`.

## What 1.1.0 carries

Both clients:
- #1807: refuse to send the API key over plain HTTP to another host
- #2010: parse HTTP-date `Retry-After` values
- #2012: `search()` keeps the response envelope (`SearchResult`, still a
list or array), `recall()` sends `items_alias: false`, and
`caller_agent_id` / `callerAgentId` lets a tenant-key client read as an
agent (opt-in)
- #2035: `stats()`
- #2071: the memory map, spend and value methods

Python: #2013 (`CauraAPIError` for a non-JSON 2xx body), plus #1775 and
#1909's client parts. TypeScript: #2014 (`CauraApiError` for a non-JSON
2xx body), #1667 (typing and response timeouts).

**Why minor:** new methods and fields. `search()` still returns a list
subclass (an array in TypeScript), so existing callers keep working. One
visible change: `recall()`'s `raw` no longer contains `items` unless
`items_alias` is passed.

**Server versions:** `stats()` needs a backend with #2035 (3.32.0 or
later) and the memory map, spend and value methods need #2071 (3.33.0 or
later). Against an older server those calls get a 404; everything else
works as before.

## Release, after merge

Each tag runs its publish workflow:
- `caura-client-v1.1.0` → `publish-python-client.yml` (PyPI
`caura-client`)
- `caura-client-ts-v1.1.0` → `publish-npm-client.yml` (npm
`@caura/client`)

The `caura`, `caura-sdk` (PyPI) and unscoped `caura-client` (npm)
packages depend on `caura-client>=1.0.0` / `@caura/client ^1.0.1`, so
they pick up 1.1.0 without a release of their own.

## Test plan

- [ ] CI: the Python client's version test (`__version__` agrees with
`pyproject.toml`) and the TypeScript client's build and tests pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: eldad-caura <eldad@caura.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@eldad-caura-ai eldad-caura-ai mentioned this pull request Oct 19, 2026
Merged
eldad-caura-ai added a commit that referenced this pull request Oct 20, 2026
@eldad-caura-ai @claude
fix(field): bound memory-map builds and see edits in its cache (#2110)
c222908 
## Why

From a recheck of what `main` gained since the audit's last reviewed
head (`0ea70e7f`, 2026-10-08). #2071 added `GET
/api/v1/memories/field`, and enterprise Home now calls it on every load.

- **M-125 (Medium, availability):** the route has no rate limit (limits
are attached only to specific write and search routes), takes no
per-tenant slot, and doesn't share builds.
- **Cost per build:** an uncached request pages up to 50,000 memories
with their 1024-d embeddings from the read replica, about 11 MB of JSON
per 2,000 rows, decoded on the event loop. PCA and k-means then run on
the shared thread pool.
- **An easy-to-bypass cache:** its key includes `limit` (1–50,000) and
`fleet_id`, and it holds 32 entries.
- **Impact:** any credential of any tenant can start any number of
builds at once, slowing or exhausting a shared core-api instance for
every tenant. For an active tenant, every write also changes the
signature, so normal Home traffic rebuilds the map too. #2071's own
description says the route "needs a bound on concurrent builds" before
broad rollout.
- **L-236 (Low):** the cache's change detector (`count`, `embedded`,
newest `created_at`/`status_changed_at`) doesn't move on an edit. An
edited or redacted memory keeps its old 240-character preview, title and
visibility on the map for up to the 10-minute TTL.

## What

- **Shared builds:** requests for the same map (same cache key) wait on
one build.
- **Per-process caps:** at most `memory_field_max_concurrent_builds`
builds (default 2), and one per tenant. Anything beyond gets **429**
with `Retry-After`. Home already treats a failed map request quietly,
and the Map page retries.
- **Shared signature reads** (from the Claude review): concurrent
requests for one visibility scope share a single signature scan, and
nothing is kept once it finishes. The signature doesn't depend on
`limit`, so varying `limit` no longer multiplies scans either.
- **Edit detection:** the storage signature adds a hash over each
visible row's `content_hash`, title, visibility, type, weight, fleet,
agent, status and `supersedes_id`, so an edit invalidates the cache.

## Verification

Staged in CI. The tests-only commit ([run
39328014463](https://github.com/caura-ai/caura/actions/runs/39328014463))
failed exactly its four new cases, each for its reason, with the other
10,269 tests passing:
- **`test_an_edit_misses_the_cache`:** `computed_at` was unchanged after
the edit, so the stale cache was served.
- **`test_requests_for_one_map_share_its_build`:** the two requests got
different `computed_at` values, so two builds ran.
- **`test_a_tenant_builds_one_map_at_a_time`:** `TimeoutError`; the
second map started a build instead of answering 429.
- **`test_builds_are_capped_per_process`:** `TimeoutError`; the third
tenant built instead of answering 429.

The review fix was staged the same way.
`test_concurrent_requests_share_one_signature_scan` was pushed without
the sharing code ([run
39330490412](https://github.com/caura-ai/caura/actions/runs/39330490412))
and failed with three scans for three requests (`assert 3 == 1`); the
other 10,273 tests passed, including the four above.

The test harness wraps the storage client: each build's first page fetch
counts the build and then holds on an `asyncio.Event`, so the tests need
no threads and no polling.

Earlier pushes of the tests-only stage stopped at ruff (ASYNC109,
ASYNC110, then blank-line formatting) before any test ran. The final
commit is `12f71f9a`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: eldad-caura <eldad@caura.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@arkash20 arkash20 arkash20 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@erni-a @arkash20