Skip to content

Releases: Siddharthpatni/Sentinel

Sentinel v0.1.0 — Phase 3: Observability, Verifications, Routing, Evals, Audit & Alerts

28 May 12:19
@Siddharthpatni Siddharthpatni
v0.1.0
005b574

Choose a tag to compare

View all tags
Sentinel v0.1.0 — Phase 3: Observability, Verifications, Routing, Evals, Audit & Alerts Latest
Latest

🛡️ Sentinel v0.1.0 — first public release

Sentinel is an open-source, self-hostable LLM observability + agent reliability platform. Drop-in proxy for OpenAI / Anthropic that gives you traces, evals, routing, EU AI Act audit logs, and alerts — with zero code changes beyond two import lines.

✨ What's included in this release

  • Observability — every LLM call traced with cost, latency, tokens, and full request/response bodies in a live dashboard. 24h cost sparkline included.
    • Verifications — declarative rules re-check primary calls with a judge model. Sampled, async, never blocks the caller.
    • Routing & fallback — per-request model overrides with ordered candidate fallback (up to 3 attempts). Streaming bypasses preserved.
    • Evals — YAML-defined regression suites with 7 assertion types (contains / equals / regex / max-latency / max-cost / json-schema / llm-judge), run-history UI, pass-rate trend endpoint, and a CI entrypoint for GitHub Actions.
    • EU AI Act audit log — risk-tier classifiers tag inbound calls; every tagged call lands in a SHA-256-chained ledger an auditor can verify offline. NDJSON export, server-side /verify.
    • Alerts — threshold checks on cost-per-hour, error-rate, and p95 latency over rolling windows.
    • BYOK — per-project provider keys (OpenAI / Anthropic / OpenRouter / Gemini), encrypted at rest with Fernet, live-validated against provider model-list endpoints.
    • Python SDK — pip install sentinel-sdk. Drop-in OpenAI / Anthropic clients plus a programmatic control plane for rules, routing, and evals.

🚀 Quick start

git clone https://github.com/Siddharthpatni/Sentinel.git
cd Sentinel
cp .env.example .env  # fill in OPENAI_API_KEY / ANTHROPIC_API_KEY
docker compose up -d
# Open http://localhost:3000 — traces appear in real time

📐 Architecture

FastAPI gateway · Next.js dashboard · PostgreSQL · Redis + Celery worker. MIT-licensed, self-hostable, production-shaped.

📖 Docs

  • docs/verifications.md · docs/routing.md · docs/evals.md · docs/audit.md · docs/alerts.md
    • Conceptual learning notes in docs/learn/

💬 Feedback

This is the first public cut — feedback, issues, and PRs are very welcome. If you're shipping LLM features in production and have a war story about agent reliability, I'd love to hear it.

License: MIT

Assets 2
Loading