Repository navigation
feat(observability): durable tenant CostStore with Postgres backend - #1835
Conversation
Add the CostStore port (reserve, commit, release, window) with an in-memory store, an optional Drizzle/Postgres subpath with atomic cap admission and a usage ledger, and an exported contract suite. The three cost guards accept an optional store and warn in production when spend is process-local.
Lighthouse
Run by |
Local gates, re-run with the real hooks (no
|
| Gate | Command | Result |
|---|---|---|
| Pre-push hook, standalone | sh .husky/pre-push (doc-bridge index, check:quality-gates, lint, build) |
exit 0, 12m57s |
| Pre-push hook, on the real push | git push origin EmersonBraun/chat-t04 |
exit 0, 7m10s, 3f2e242c..88722d4d |
| Quality gates | pnpm check:quality-gates |
✓ all 52 quality gates passed |
| Lint | pnpm lint |
60/60 turbo tasks |
| Build | pnpm build |
44/44 turbo tasks |
| Repository tests | pnpm test |
77/77 turbo tasks, exit 0 |
| Package tests + coverage | pnpm --filter @agentskit/observability test:coverage |
373 passed, 1 skipped (the Postgres test, run separately below); lines 94.44% |
| CostStore contract, PostgreSQL 16.15 | AK_POSTGRES_TEST_PORT=… pnpm --filter @agentskit/observability test:postgres |
1/1, then 10 more consecutive runs, 10/10 |
| CostStore contract, workerd | pnpm --filter @agentskit/observability test:workerd |
all 8 criteria passed over wrangler dev --local against the same PostgreSQL 16 |
| Size budget | size-core-closure, gen-size-limit-subpaths --check/--build, size-limit |
exit 0, 110 entries within budget |
One thing the hook run turned up: the docs build regenerates apps/docs-next/lib/api-symbol-routes.json with the 17 new public symbols of this PR, and the file was not committed. 88722d4d adds it. No other file changed after the full run.
Quality gates (tail)
▶ Math.random identifier rule tests
RUN v4.1.11 /Users/rebecabraun/orca/workspaces/agentskit/chat-t04
Test Files 1 passed (1)
Tests 5 passed (5)
Start at 22:21:39
Duration 161ms (transform 34ms, setup 0ms, import 49ms, tests 3ms, environment 0ms)
Math.random identifier rule tests: 0.7s
────────────────────────────────────────────────────────
✓ all 52 quality gates passed
Lint and build summaries
Tasks: 60 successful, 60 total
Cached: 15 cached, 60 total
Time: 43.961s
Tasks: 44 successful, 44 total
Cached: 27 cached, 44 total
Time: 3m25.511s
sh .husky/pre-push 737.87s user 249.96s system 127% cpu 12:57.42 total
EXIT=0
pnpm test (tail)
Tasks: 77 successful, 77 total
Cached: 43 cached, 77 total
Time: 2m54.149s
pnpm test 274.49s user 128.23s system 229% cpu 2:55.40 total
EXIT=0
Observability tests and coverage
Test Files 29 passed | 1 skipped (30)
Tests 373 passed | 1 skipped (374)
Start at 22:26:08
Duration 3.16s (transform 1.93s, setup 0ms, import 3.78s, tests 3.21s, environment 2ms)
% Coverage report from v8
-------------------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
-------------------|---------|----------|---------|---------|-------------------
All files | 93.28 | 86.84 | 94.71 | 94.44 |
audit-log.ts | 97.67 | 100 | 94.44 | 100 |
CostStore contract on PostgreSQL 16
16.15 (Debian 16.15-1.pgdg13+2)
Test Files 1 passed (1)
Tests 1 passed (1)
Start at 22:26:25
Duration 1.62s (transform 70ms, setup 0ms, import 389ms, tests 1.13s, environment 0ms)
EXIT=0
CostStore contract on workerd
workerd + PostgreSQL: CostStore contract, ledger and replay race passed
EXIT=0
Size budget, observability entries
@agentskit/observability (ESM)
Size limit: 17 kB
Size: 16.57 kB gzipped
@agentskit/observability (ESM export .)
Size limit: 22.5 kB
Size: 21.87 kB gzipped
@agentskit/observability (ESM export ./trace-tracker)
Size limit: 2 kB
Size: 1.4 kB gzipped
@agentskit/observability (ESM export ./cost-guard)
Size limit: 2.5 kB
Size: 2.37 kB gzipped
@agentskit/observability (ESM export ./langfuse)
Size limit: 3 kB
Size: 2.45 kB gzipped
@agentskit/observability (ESM export ./postgres)
Size limit: 3 kB
Size: 2.89 kB gzipped
@agentskit/observability (ESM export ./cost-store-contract)
Size limit: 2.5 kB
Size: 2.08 kB gzipped
Turbo served part of pnpm test (43 of 77 tasks) and of the hook's lint/build from its local cache, for inputs unchanged since an earlier run on this machine; the observability tasks and both contract runs executed fresh.
Not covered here: Neon and Hyperdrive (the PRD gate for RF-26 asks for Neon; this run is local PostgreSQL 16 only).
Lighthouse
Run by |
- match reservation errors by code in the contract suite; each CommonJS subpath bundles its own copy of the class. Add a CommonJS smoke test. - recordCostSpend releases its hold when the commit fails - expirePostgresCostReservations returns holds that were never settled - a guard without onError warns once when the store fails - warnWithoutStore, and Cloudflare Workers detection for the missing-store warning - reserve rejects a released reservation id instead of replaying it as admitted - validate release and window inputs in the PostgreSQL store - ledger token columns are bigint; Drizzle schema carries the CHECK and index - run the contract in workerd on a pool, and in CI with the CommonJS smoke
Review follow-up at
|
| Gate | Result |
|---|---|
sh .husky/pre-push standalone |
exit 0, 15m01s: ✓ all 52 quality gates passed, lint 60/60, build 44/44 |
Same hook on git push |
exit 0, 7m58s, 88722d4d..fdb1e758 |
pnpm --filter @agentskit/observability test:coverage |
387 passed, lines 94.58% |
test:postgres (PostgreSQL 16.15 (Debian 16.15-1.pgdg13+2)) |
1/1, nine criteria incl. ledger, replay race, expiry |
test:cjs |
in-memory and PostgreSQL stores pass the contract through require |
test:workerd |
pool of 5 connections, nine criteria |
size-limit |
exit 0 after raising ./cost-guard to 3 KB and ./postgres to 3.5 KB |
The first hook run on these changes failed three gates, all expected for new exports and an edited README: public API JSDoc coverage (snapshot needed the two new /postgres symbols), README Standard freshness, and its test. I updated the snapshot and refreshed only the changed source hash (--preserve-review-dates), then re-ran.
To check that the CommonJS smoke test catches M1, I restored the instanceof comparison, rebuilt, and ran it: it fails with CostStore contract: commit after release is rejected. With the fix it passes.
The repository-wide pnpm test (77/77) was run on 88722d4d, before these changes; after them I ran the observability suite and the three contract runs, not the whole repository again. The two new CI steps (test:cjs, test:workerd) run on GitHub for the first time with this push.
Outputs
Test Files 30 passed (30)
Tests 387 passed (387)
All files | 93.42 | 86.9 | 95.07 | 94.58 |
### EXIT 0 (pnpm --filter @agentskit/observability test:coverage)
Test Files 1 passed (1)
Tests 1 passed (1)
### EXIT 0 (pnpm --filter @agentskit/observability test:postgres)
CommonJS: in-memory CostStore passed the contract
CommonJS: PostgreSQL CostStore passed the contract
### EXIT 0 (pnpm --filter @agentskit/observability test:cjs)
workerd + PostgreSQL (pool of 5): CostStore contract, ledger, replay race and expiry passed
### EXIT 0 (pnpm --filter @agentskit/observability test:workerd)
✓ all 52 quality gates passed
sh .husky/pre-push 611.20s user 223.81s system 92% cpu 15:01.83 total
EXIT=0
88722d4d..fdb1e758 EmersonBraun/chat-t04 -> EmersonBraun/chat-t04
git push origin EmersonBraun/chat-t04 341.83s user 89.10s system 90% cpu 7:58.22 total
EXIT=0
Lighthouse
Run by |
Track 04, stage 4. Covers RF-25, RF-26, RF-28 and the ledger of RF-29 in
@agentskit/observability.Not in this PR, tracked elsewhere:
QUOTA_EXCEEDED, warning at 80%) is application behavior and lives in the chat handler: feat(chat): action decisions across requests and turn cost policy agentskit-chat#205. Here the cap is an argument ofreserve.What changes
CostStoreport:reserve,commit,release,window, all tenant-scoped and idempotent perreservationId.reservechecks an estimate against a cap passed by the caller;commitwrites real spend and usage ledger rows;releasereturns the hold.reservewith the id of a released reservation throwsAK_COST_RESERVATION_RELEASED.createInMemoryCostStorefor development and tests.@agentskit/observability/postgres:postgresCostStore({ db })on BYO Drizzle and node-postgres, with exported tables and idempotent DDL. Each operation is one statement; admission is the conditional updatespent + reserved + amount <= cap.expirePostgresCostReservations({ db, olderThan })releases reservations that were never settled.@agentskit/observability/cost-store-contract:costStoreContract, a framework-free suite for custom stores. It matches errors bycode.costGuard,multiTenantCostGuardandcreateAdvancedCostGuardaccept an optionalstore. With it, each priced call is written to the store and the budget is checked against the stored window total. Without it, behavior is unchanged except for a one-time warning in production runtimes.test:postgres,test:cjs,test:workerd).Decisions a reviewer should check
reserve, not stored state. The host reads it from its plan data.commitrecords real spend even above the reservation. The cap is a hard ceiling on reservations only.reserveitself.expirePostgresCostReservations. A cutoff shorter than a call in flight releases its hold, and that call'scommitis then rejected.reserveand host transactions. A concurrent duplicate id ends in a unique violation that the store absorbs. Inside a host transaction that still aborts the transaction, soreservebelongs outside it or in a savepoint. Documented, not changed: admitting the hold and writing the reservation row in one statement is what makes the cap atomic.NODE_ENV=production, and on Cloudflare Workers whenNODE_ENVis not set.warnWithoutStore: true | falseoverrides it.onError. A guard withoutonErrorlogs the first failure once.@agentskit/observability (ESM)16.5 → 17 KB,./cost-guard2.5 → 3 KB,./postgres3 → 3.5 KB.drizzle-ormandpgare new optional peers, used only by/postgres.Review follow-up (second commit set)
instanceofacross CommonJS entrieserror.code.tests/cjs-smoke.cjsruns the contract throughrequireagainst the in-memory and the PostgreSQL store; with the old check it fails on "commit after release is rejected".recordCostSpendexpirePostgresCostReservationscovers holds no process can release.console.warnper guard whenonErroris absent.test:workerdandtest:cjsadded to the PostgreSQL job. The worker fixture uses a pool of 5, so the 50-reservation race runs on several connections.warnWithoutStore.bigint.releaseandwindowvalidate tenant and id in the PostgreSQL store. Replay ofreserveafterreleaseis rejected. The Drizzle schema carries the statusCHECKand the ledger index. The integration test loads the built package only when it runs. The 23505-in-transaction case is documented.Local validation
Run on macOS, Node 26.8.1, pnpm 10.34.4, with the real pre-push hook (no
HUSKY=0). Outputs are in the comments.sh .husky/pre-push(52 quality gates,lint,build)pnpm test(whole repository)pnpm --filter @agentskit/observability test:coveragetest:cjs)test:workerd)size-limitNot validated
Neon and Hyperdrive, database row-level security, throughput, and the provider-invoice comparison of RF-29. The two new CI steps have not run on GitHub yet; the first CI run of this push is their first.
No release or publication from this PR.