Obsivra

Open-source observability for LLM applications — trace every model call, see exactly where latency and cost come from.

Obsivra ingests structured traces from LLM applications via a lightweight TypeScript SDK, computes token usage and cost against a versioned pricing table, and renders nested agent/tool/model executions as a waterfall.

Generic APM models a span as a timed operation with attributes. It has no concept of a prompt/completion token split, no TTFT, no prompt-cache semantics, no notion of a tool call nested inside a model call. Obsivra makes an LLM call a first-class observable unit.

Status: pre-v0.1. The ingest pipeline, cost engine, SDK, and dashboard all work end-to-end and are covered by 113 tests. Not yet deployed publicly.

Architecture

flowchart TB
    subgraph client ["Your application"]
        direction LR
        SDK["<b>@obsivra/sdk</b><br/><i>wrapOpenAI · batched · fail-open</i>"]
        RAW["<b>Any HTTP client</b><br/><i>raw JSON</i>"]
    end

    API["<b>INGEST API</b> · Fastify<br/>auth → validate → enqueue → <b>202</b><br/><i>hot path · no DB write</i>"]
    REDIS[("<b>Redis Streams</b><br/><i>consumer group</i>")]
    WORKER["<b>WORKER</b> · idempotent<br/>normalise tokens → resolve pricing<br/>→ cost → insert → rollup"]
    DLQ[("<b>Dead-letter</b>")]
    PG[("<b>Postgres</b><br/>projects · api_keys · model_pricing<br/><b>spans</b> partitioned monthly<br/>rollup_hourly")]
    QAPI["<b>QUERY API</b><br/><i>keyset paging · rollup-served</i>"]
    WEB["<b>Next.js dashboard</b><br/><i>trace list · waterfall</i>"]

    SDK --> API
    RAW --> API
    API -->|XADD| REDIS
    REDIS -->|XREADGROUP| WORKER
    WORKER -.->|N failed attempts| DLQ
    WORKER -->|ON CONFLICT DO NOTHING| PG
    PG --> QAPI
    QAPI --> WEB

    style API fill:#4c1d95,stroke:#7c3aed,color:#fff
    style WORKER fill:#065f46,stroke:#10b981,color:#fff
    style PG fill:#1e3a5f,stroke:#3b82f6,color:#fff
    style REDIS fill:#7f1d1d,stroke:#ef4444,color:#fff
    style DLQ fill:#78350f,stroke:#f59e0b,color:#fff

The load-bearing idea: this is a high-write append-only telemetry pipeline with a low-write control plane attached. Those halves have opposite requirements, so a control-plane query must never block the ingest path — hence the queue, and hence cost being computed by the worker rather than at read time.

Quickstart

Prerequisites: Node 20+ (22 recommended), pnpm 10+, Docker.

1. Install and start infrastructure

git clone https://github.com/Ashutosh-code-arch/obsivra && cd obsivra
pnpm install
cp .env.example .env
docker compose up -d        # Postgres 17 + Redis 7

2. Create the schema

pnpm turbo build && pnpm db:migrate

This applies the drizzle-kit migrations, then the hand-written partition DDL, then provisions this month's and the next three months' spans partitions.

3. Create a project and an API key

pnpm db:seed

Prints an obs_… key once. Copy it — only its SHA-256 hash is stored.

4. Run the API + worker

node apps/api/dist/main.js

Serves on :3001 and starts the queue consumer in the same process. For development with hot reload use pnpm --filter @obsivra/api dev.

5. Load demo data (optional, recommended)

pnpm db:seed:demo obs_your_key_here

Posts 570 spans across 25 traces through the real ingest API — nested agent runs, cache hits, streaming TTFT, an unpriced model, errors, and one 450-span trace to exercise the waterfall.

6. Run the dashboard

cp apps/web/.env.example apps/web/.env.local   # then paste your key into it
pnpm --filter @obsivra/web dev

Open http://localhost:3000.

Shutting down

docker compose down          # stops Postgres + Redis, keeps data
docker compose down -v       # also deletes the volumes

Using the SDK

pnpm add @obsivra/sdk

import OpenAI from 'openai';
import { Obsivra, wrapOpenAI } from '@obsivra/sdk';

const obsivra = new Obsivra({
  apiKey: process.env.OBSIVRA_API_KEY!,
  baseUrl: 'http://localhost:3001',
  // redactIO: true,   // NFR-7: drop prompt/response bodies, keep cost + timings
});

const openai = wrapOpenAI(new OpenAI(), obsivra.tracer);

// Every chat completion now emits a span. No other code changes.
const res = await openai.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [{ role: 'user', content: 'hello' }],
  stream_options: { include_usage: true },   // needed for token counts on streams
});

await obsivra.shutdown();   // graceful exit
// await obsivra.flush();   // serverless: before the handler returns

Manual spans, for tools and retrieval:

await obsivra.trace({ name: 'tool.search_docs', kind: 'tool' }, async () => {
  return searchDocs(query);   // nests under the current span automatically
});

The SDK never throws into your code. Network failures, 5xx, timeouts, and an unreachable URL are all swallowed and logged via onError. The buffer is bounded at 1000 spans and drops oldest on overflow — during an incident the recent spans are the ones you need.

API

Method	Path
`POST`	`/v1/traces`	Ingest a batch → `202`
`GET`	`/v1/traces`	List + filter + paginate
`GET`	`/v1/traces/:id`	One trace with all spans
`GET`	`/v1/stats/overview`	Rollup metrics
`GET`	`/health`	Liveness

Auth on every /v1 route: Authorization: Bearer obs_<key>.

curl -X POST localhost:3001/v1/traces \
  -H "authorization: Bearer $OBSIVRA_API_KEY" \
  -H 'content-type: application/json' \
  -d '{"spans":[{
        "trace_id":"t1","span_id":"s1","parent_span_id":null,
        "name":"chat.completion","span_kind":"llm",
        "start_time":"2026-08-09T10:00:00.000Z",
        "end_time":"2026-08-09T10:00:02.340Z",
        "status":"ok","model":"gpt-4o-mini","provider":"openai",
        "usage":{"input_tokens":1204,"output_tokens":88,
                 "cache_read_tokens":1024,"cache_write_tokens":0,
                 "reasoning_tokens":0},
        "ttft_ms":210}]}'

A malformed envelope returns 400 and enqueues nothing. An individual invalid span is reported in errors[] while the rest of the batch still returns 202 — partial success by design.

Design decisions worth reading

Full reasoning lives in docs/adr/ADR-LOG.md. The four that shaped the most code:

Cost is computed at ingest and frozen (ADR-0006)

Pricing rows carry valid_from / valid_to / version. The worker resolves the row in force at the span's start_time, computes the cost, and stores the cost and the pricing_version used. Computing at query time would mean last quarter's spend chart silently changes the day a provider adjusts prices.

An unknown model stores cost_usd = NULL and pricing_status = 'unknown_model' — never zero. A stored 0 is indistinguishable from "we priced it and it was free"; the dashboard renders — and shows an unpriced count next to every total.

Five token classes, normalised per provider (ADR-0008)

The two providers disagree about what "input tokens" means, and the difference is not cosmetic:

	Anthropic	OpenAI
Cached tokens are…	siblings of `input_tokens`	a subset of `prompt_tokens`
Cache write	1.25×–2× base input	not billed separately
Cache read	~0.1× base input	discounted

Summing all fields is correct for Anthropic and double-counts for OpenAI. And a cache write costs more than base input while a read costs a tenth — so one "cached" rate cannot represent both. A provider adapter normalises to disjoint counts before pricing; the cost engine contains no provider conditional, and input + cache_read + cache_write is the prompt total for every provider.

Money is computed in integer pico-USD with BigInt, never floats.

Idempotency is a database constraint, not application logic (ADR-0007)

Redis Streams gives at-least-once delivery, so duplicates are normal rather than exceptional. The worker inserts with ON CONFLICT DO NOTHING against a unique index. A SELECT-then-INSERT check would be a race; a constraint is not.

A caveat worth knowing: the requirement originally specified UNIQUE (project_id, trace_id, span_id). Postgres rejects that on a table partitioned by start_time — every unique constraint must contain the partition key. The real key is therefore four columns. Redelivery still collides correctly (an identical payload has an identical start_time), but a client re-sending the same span with a corrected timestamp gets two rows.

Rollups recompute instead of increment

ON CONFLICT DO NOTHING makes a duplicate insert harmless. An increment is not harmless to re-apply — at-least-once delivery would inflate every counter. So the worker recomputes the affected (project, hour, model) bucket from raw spans and upserts it. Running it twice produces the same numbers.

Development

pnpm turbo build typecheck lint test   # everything (84 unit tests)
pnpm test:integration                  # 29 tests, needs Docker up + db:migrate
pnpm lint:fix                          # Biome autofix

Two test tiers, deliberately. Unit tests are mocked and fast. Integration tests run against real Postgres and Redis, because the bugs that mattered most here were invisible to mocks: raw SQL returns timestamps as strings where the typed query builder returns Dates, and container PIDs are almost always 1 so a PID-based Redis consumer name collides across every deployed instance.

packages/shared   Zod schemas + wire types — the contract every package imports
packages/db       Drizzle schema, migrations, partition provisioning
packages/sdk      the published client library
apps/api          Fastify ingest + query + the worker
apps/web          Next.js 16 dashboard

The monorepo exists for coordinated breakage, not code reuse: when the span payload changes, TypeScript fails in the API, the SDK, and the web app simultaneously.

Deployment

Vercel (web) + Render (API + worker) + Neon (Postgres) + Upstash (Redis), targeting $0–7/month.

API + worker: render.yaml — one service; Render's free tier has no background-worker primitive, so both entry points run in one process
Dashboard: apps/web/vercel.json and apps/web/DEPLOY.md

Deploy the API first — the dashboard's origin has to be added to the API's CORS_ORIGINS, and you don't know that origin until the Vercel project exists.

Explicitly NOT built

These are deliberate subtractions, not a roadmap. v0.1 is one engineer, four weeks, 80 hours.

Excluded	Why
AI Gateway / proxy mode	A different product. Helicone + LiteLLM territory.
RAG Studio, Agent Studio, memory system	That's a framework, not an observability tool.
Fine-tuning, model training	Separate discipline, near-zero shared code.
Orgs, RBAC, multi-tenancy	Single-project. API key → project. That's it.
OTLP protobuf ingestion	JSON only. Protobuf is post-v0.1.
Python SDK	TypeScript only. Halves the surface area.
ClickHouse	Postgres + monthly partitions. Migrate when the data justifies it.
Kubernetes, Terraform	Render + Vercel. Not on the critical path.

Known limitations, tracked rather than hidden (see HANDOFF.md §6): overview percentiles average per-bucket percentiles and so understate the tail; the dashboard ships the API key to the browser, which is fine for single-project use and nowhere else; cache-write TTL tiers collapse to one rate; and trace-list paging is disabled under the duration and cost sorts because keyset cursors only line up with the default ordering.

Licence

MIT

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
apps		apps
docs		docs
packages		packages
.env.example		.env.example
.gitignore		.gitignore
CONTEXT.md		CONTEXT.md
HANDOFF.md		HANDOFF.md
README.md		README.md
biome.json		biome.json
docker-compose.yml		docker-compose.yml
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml
render.yaml		render.yaml
tsconfig.base.json		tsconfig.base.json
turbo.json		turbo.json

Navigation Menu

Repository files navigation

Obsivra

Architecture

Quickstart

1. Install and start infrastructure

2. Create the schema

3. Create a project and an API key

4. Run the API + worker

5. Load demo data (optional, recommended)

6. Run the dashboard

Shutting down

Using the SDK

API

Design decisions worth reading

Cost is computed at ingest and frozen (ADR-0006)

Five token classes, normalised per provider (ADR-0008)

Idempotency is a database constraint, not application logic (ADR-0007)

Rollups recompute instead of increment

Development

Deployment

Explicitly NOT built

Licence

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

Obsivra

Architecture

Quickstart

1. Install and start infrastructure

2. Create the schema

3. Create a project and an API key

4. Run the API + worker

5. Load demo data (optional, recommended)

6. Run the dashboard

Shutting down

Using the SDK

API

Design decisions worth reading

Cost is computed at ingest and frozen (ADR-0006)

Five token classes, normalised per provider (ADR-0008)

Idempotency is a database constraint, not application logic (ADR-0007)

Rollups recompute instead of increment

Development

Deployment

Explicitly NOT built

Licence

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages