Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 18 additions & 42 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# Obyflow

**AI-native, CLI-first observability platform** for tracing, debugging, and understanding modern applicationsLLM calls, vector-store queries, and framework (LangChain) steps included.
**AI-native, CLI-first observability for modern applications.** Trace and debug LLM calls, vector-store queries, and LangChain steps from your terminal.

Obyflow captures structured events locally (SQLite, no external backend required), correlates them into traces, and uses an LLM of your choice to turn raw evidence into a plain-English investigation of what went wrong.
Obyflow stores structured events locally in SQLite, connects them into traces, and can use an LLM of your choice to help investigate what went wrong.

[![npm version](https://img.shields.io/npm/v/obyflow.svg)](https://www.npmjs.com/package/obyflow)
[![PyPI version](https://img.shields.io/pypi/v/obyflow-python.svg)](https://pypi.org/project/obyflow-python/)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)

If you find Obyflow useful, please consider ⭐ starring the repo — it helps others discover the project.
If Obyflow is useful to you, consider giving the repo a ⭐.

![Obyflow interactive terminal](./docs/obyflow-terminal.png)

Expand All @@ -22,43 +22,21 @@ pip install obyflow-python # Python SDK - https://pypi.org/project/obyflow-

## Features

- **Structured event model** — 10 typed event kinds (`trace`, `log`, `metric`, `error`, `embedding`, `vector_op`, `chain`, `tool_call`, `llm_call`, `custom`), each with typed attributes (e.g. `llm_call` captures model/provider/token counts/latency; `vector_op` captures db provider, similarity scores, result counts)
- **Trace correlation** — joins events by trace/span/request id into a single correlated trace; the Node and Python SDKs auto-propagate `x-obyflow-trace-id`/`x-obyflow-parent-span-id` headers across outbound HTTP/fetch calls, so multi-service traces stay linked with zero manual header wiring
- **Anomaly detection** — statistical baselining (mean/stddev and median/MAD) with z-scored deviations against historical norms
- **Evidence graph & diagnosis engines** — purpose-built diagnosis for LangChain/LangGraph/LlamaIndex chain failures (failed steps, tool-call timeouts, empty retriever results, step-duration regressions) and vector-DB retrieval issues (empty results, low similarity, slow queries, embedding latency)
- **"What changed" correlation** — correlates incidents against deployments, git commits, config changes, feature flags, model version changes, and dependency changes; git correlation reads real commit metadata (author, files changed, insertions/deletions) from a local repo
- **Confidence scoring** — HIGH/MEDIUM/LOW investigation confidence based on evidence volume, anomaly severity, correlated services, and deployment correlation
- **Incident memory** — fingerprints incidents and surfaces similar past incidents, learning from recorded resolutions over time
- **Telemetry health checks** — detects dropped events and coverage gaps in your instrumentation
- **AI-assisted investigation** — ask a question or investigate a trace, get an evidence-backed root-cause summary, with grounding validation (flags LLM citations that don't match real evidence) and token-budget-aware context trimming
- **Token usage & cost tracking** — per-request prompt/completion token counts on every `llm_call` event, rolled up by the `usage` command into per-service totals and estimated USD cost via built-in pricing tables (Claude, GPT-4o/5, Gemini); also tracks per-model context-window limits and warns when investigation context is approaching a model's limit
- **Resilient LLM calls** — automatic retry with exponential backoff on rate limits (429), overloaded/unavailable errors (503), and transient network failures (`ECONNRESET`/`ETIMEDOUT`/`ECONNREFUSED`)
- **Local-first storage** via SQLite, zero external infra to get started
- **Redaction** — configurable field-level redaction (passwords, tokens, credit cards, SSNs, API keys) applied at ingestion or evidence time, plus value-pattern detection (Luhn-validated credit card numbers, SSN format, bearer tokens) that redacts sensitive-looking values even when the field name doesn't match a configured field
- **Automatic resource attributes** — every emitted event is tagged with hostname, PID, and Node/Python runtime version, plus the current git commit SHA (read from CI env vars like `GITHUB_SHA`/`VERCEL_GIT_COMMIT_SHA` or a local `git rev-parse HEAD`) — this is what powers commit-based "what changed" correlation without any manual instrumentation
- **Pluggable LLM providers** — Anthropic, OpenAI, Gemini, Ollama, or none (evidence-only mode)
- **Node.js and Python SDKs** with matching instrumentation for LangChain and six vector databases (Pinecone, Qdrant, Weaviate, Chroma, pgvector, Milvus) plus embedding calls (OpenAI, Anthropic, Cohere). Inbound HTTP tracing differs by design: Node auto-instruments any `http`-based server (Express, Koa, Fastify, raw `http.createServer`) via a runtime patch installed by `start()`, with zero extra code; Python requires explicit middleware registration — `ObyflowASGIMiddleware` for FastAPI/Starlette or `ObyflowWSGIMiddleware` for Flask/Django (sync) — added once alongside `start()`.
- **Data export** — JSON, CSV, or OpenTelemetry OTLP
- **Live-updating CLI views** — `traces`/`logs`/`metrics`/`errors` support `--watch [seconds]` to poll and re-render, plus `--detail` for full detail cards instead of a table

## Repository layout
- **Structured events** — Traces, logs, metrics, errors, LLM calls, tools, embeddings, vectors, chains, and custom events.
- **Distributed tracing** — Follow requests across services with automatic trace propagation.
- **Anomaly detection** — Spot unusual behavior against historical baselines.
- **Root-cause analysis** — Diagnose chain failures, tool timeouts, retriever issues, and slow vector queries.
- **Change tracking** — See what changed across deployments, commits, configs, flags, models, and dependencies.
- **Incident history** — Find similar incidents and recorded resolutions.
- **AI investigations** — Investigate traces and incidents using your telemetry as evidence.
- **LLM usage** — Track tokens, costs, context usage, and model limits.
- **Automatic retries** — Handle rate limits, temporary provider errors, and network failures.
- **Local-first** — Store everything in SQLite with no external backend.
- **Redaction** — Automatically remove sensitive values before they're stored.
- **Node.js & Python** — SDKs with LangChain, vector DB, and embedding integrations.
- **Export** — JSON, CSV, or OpenTelemetry OTLP.
- **CLI** — Browse, filter, watch, and inspect your telemetry from the terminal.

```
packages/
core/ shared event model, storage, config, evidence & anomaly logic
cli/ the `obyflow` CLI (init, start, traces, logs, investigate, ask, incident, ...)
node-sdk/ @obyflow/node — instrumentation SDK for Node.js apps
adapters/
adapter-framework/ LangChain callback handler adapter
adapter-vectordb/ Pinecone / Qdrant / Weaviate / Chroma / pgvector / Milvus adapters
llm/
llm-core/ shared LLM adapter interface
llm-anthropic/ llm-openai/ llm-gemini/ llm-ollama/
python/
obyflow-python/ Python SDK (ASGI middleware, LangChain callback, vector/analysis helpers)
```

This is a pnpm + Turborepo monorepo for the TypeScript packages, plus a standalone Python package.

## Getting started

Expand Down Expand Up @@ -104,9 +82,7 @@ npx obyflow config get llm.provider # read a single config value
npx obyflow config set llm.model <model-id> # set and persist a config value
```

The read commands (`traces`, `logs`, `metrics`, `errors`, `usage`, `export`) accept `--db <path>` (defaults to `obyflow.db`), `--service <name>`, and `--since <window>` (e.g. `15m`, `2h`, `1d`) to scope results; `traces`, `logs`, `metrics`, and `errors` additionally support `--limit <n>`, `--detail` (full detail cards instead of a table), and `--watch [seconds]` (poll and re-render, default every 2s). `investigate` and `incident summarize` accept `--git-repo <path>` to correlate incidents against real commit metadata (author, files changed, insertions/deletions) from a local git repository, and `--no-llm` to show evidence and anomalies only, skipping LLM synthesis. Run `npx obyflow <command> --help` for the full flag list on any command.

Supported LLM providers: `anthropic`, `openai`, `gemini`, `ollama`, or `none` (evidence-only mode, no LLM key required).
Supported LLM providers: `gemini`, `anthropic`, `openai`, `ollama`, or `none` (evidence-only mode, no LLM key required).

Example using Gemini:

Expand Down
Loading