From d7c66f5041489a8e171055c42e5284d2d5b0d55c Mon Sep 17 00:00:00 2001 From: Vivek Rao Date: Fri, 17 Apr 2026 00:47:22 +0530 Subject: [PATCH] docs(readme): rewrite to match actual shipped features Strip unimplemented feature claims (FlowBuilder, bidirectional adapters, WSS agent gateway, email polling, SSO/multi-region Cloud features). Add Roadmap section for planned features. Add SigOps Cloud section with features that have code. Add SigOps CLI section. --- README.md | 396 +++++++++++++++++++----------------------------------- 1 file changed, 141 insertions(+), 255 deletions(-) diff --git a/README.md b/README.md index 27c2727..c77096c 100644 --- a/README.md +++ b/README.md @@ -1,352 +1,238 @@ -# SigOps +# SigOps — Infrastructure Execution OS -> The Open Source Infrastructure Execution OS +Open-source platform for automated infrastructure operations. +Ingest signals → match rules → execute workflows → verify results. -SigOps is a governed, AI-assisted execution OS that converts any operational signal into safe, automated actions. It observes infrastructure signals (Prometheus, Grafana, Datadog, webhooks, email), matches them against rules, executes workflows written in SEL (SigOps Execution Language), and verifies the outcome — all in a closed loop. The Rust-based agent runs on customer infrastructure with outbound-only WebSocket connections. +[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE) --- -## Quick Reference +## What SigOps Does -| | | -|---|---| -| **Entity** | SigOps Technologies Pvt Ltd | -| **Repo** | `sigops` | -| **Visibility** | PUBLIC | -| **License** | MIT (Open Source) | -| **Product ID** | `sigops` | -| **Tables** | ~10 | -| **API Endpoints** | ~25 | -| **Backend Port** | 4200 | -| **Frontend Port** | 4201 | -| **Stack** | Hono · Drizzle · PostgreSQL · Redis · Zod · React 18 · MUI · Vite | - ---- +SigOps is an **infrastructure execution OS** — a governed, closed-loop system that turns operational signals (alerts, webhooks, metrics breaches) into automated, auditable remediation. Instead of paging a human at 3am to run the same five commands, SigOps matches the signal against a rule, runs a workflow written in **SEL** (the SigOps Execution Language), dispatches steps to a Rust agent running on the target host, and verifies the outcome. -## What This Product Does +The target audience is platform / SRE / DevOps teams who already have monitoring (Prometheus, Grafana, etc.) but are drowning in manual remediation toil. SigOps does not replace your observability stack — it plugs into it and acts on what it sees. -### Core Features (Open Source) +At a high level: a signal enters through the ingest API, the rules engine matches it, a workflow executes step-by-step through an agent on the customer host, each step is tracked in a state machine with retries and risk scoring, and the resulting execution is visible in the dashboard for audit and replay. -- **Signal Ingestion** — Webhook receiver, Prometheus adapter, email polling, deduplication, fingerprinting -- **Signal Rules Engine** — Pattern matching, severity filtering, regex on titles, JSONPath on body, duration conditions -- **SEL Language** — First-class execution language with .sel files, parser, runtime, tooling -- **Execution Engine** — Closed-loop: signal → match → execute → verify → resolve. Risk scoring, approval gates -- **Agent Gateway** — WebSocket server for Rust agents. Heartbeat, command routing, tool discovery -- **Tool Registry** — Typed, versioned, reusable actions. 5 built-in tools (restart, http, notify_slack, wait, condition) -- **Workflow Engine** — CRUD, scheduling (cron), versioning, tags -- **FlowBuilder** — Visual workflow editor (basic, React Flow based) -- **Adapter Layer** — Coexist with PagerDuty, Datadog, Grafana (bidirectional adapters) -- **Dashboard** — Signal list, execution viewer, agent management, stats +--- -For production features (on-call, war rooms, postmortems, status pages, AI -advisor, knowledge engine, marketplace, enterprise SSO), see the **SigOps -Cloud** section near the end of this README. +## Core Features + +Only features that exist in code on `main` today are listed here. Items planned but not shipped are in the **Roadmap** section. + +- **Signal Ingestion** — Webhook receiver + Prometheus Alertmanager adapter. Fingerprinting and deduplication. +- **Signal Rules Engine** — Pattern matching, severity filtering, regex on titles, JSONPath on body. +- **Signal State Machine** — `OPEN → ACK → RESOLVED → SUPPRESSED` lifecycle with audit trail. +- **SEL Language** — Parser, evaluator, and built-in tool surface. Workflows are `.sel` files. +- **Workflows** — CRUD, SEL validation, versioning, tags. +- **Workflow Schedules** — Cron-stored schedules (CRUD layer). +- **Executions + Execution Steps** — State machine, retries, risk scoring, step-level tracking. +- **Agents** — HTTP heartbeat polling with backoff and token rotation on each cycle. +- **Built-in Tools (5)** — `restart`, `http`, `notify_slack`, `wait`, `condition`. +- **Agent Tools / Tool Registry** — Discovery + registry per agent. +- **Stats / Dashboard API** — Recent executions, severity breakdown, agent status. +- **Auth** — JWT HS256 verification via an external identity provider (see below). +- **UI Dashboard** — 10 pages (signals, rules, workflows, executions, agents, tools, schedules, stats, settings, auth) built with React 18 + Vite + MUI. +- **Docker** — Multi-stage `server/Dockerfile` and `ui/Dockerfile` (nginx:alpine). `docker-compose.yml` for Postgres + Redis locally. --- ## Architecture -Full architecture document: `docs/architecture/SigOps-Architecture-v1.2.0.pdf` -Build instructions: `CLAUDE.md` (root of this repo) -Project memory: `docs/memory/memory.md` -Session logs: `docs/session/` +``` +signal → rules engine → SEL workflow → agent execution → verification + │ + ▼ + execution state machine + (OPEN → RUNNING → SUCCEEDED/FAILED) +``` + +- **Signal-driven.** All work starts with a signal. No cron-triggered actions outside of scheduled workflows. +- **Agent-first execution model.** Steps run on a Rust agent on the target host. The server never executes directly against customer infrastructure. +- **Closed-loop.** Every execution is tracked end-to-end — match, dispatch, step results, verification, resolution. +- **State machine driven.** Signals and executions both have explicit state machines; no ad-hoc status columns. + +Full architecture document: `docs/architecture/SigOps-Architecture-v1.2.0.pdf`. --- -## Repository Structure +## Tech Stack -``` -sigops/ -├── CLAUDE.md # Build instructions for Claude Code -├── CLAUDE-CODE-AUDIT-PROMPT.md # Universal protocol enforcement -├── README.md # This file -├── docker-compose.yml # PostgreSQL + Redis -├── .env.example # Environment variables template -├── .gitignore -├── pnpm-workspace.yaml # Links server/ + ui/ -│ -├── server/ # Hono backend -│ ├── package.json -│ ├── tsconfig.json -│ ├── drizzle.config.ts -│ ├── vitest.config.ts # Test config (>90% coverage) -│ └── src/ -│ ├── index.ts # App entry + identity-provider registration -│ ├── config/index.ts # Env validation (Zod) -│ ├── db/ -│ │ ├── schema.ts # ALL Drizzle table definitions -│ │ ├── migrations/ # Drizzle migrations -│ │ └── seed.ts # Seed data -│ ├── lib/ -│ │ ├── uap-client.ts # @cluster/uap-client (identity provider HTTP client) -│ │ └── auth.ts # @cluster/auth-middleware -│ ├── modules/ -│ │ └── / -│ │ ├── routes.ts # Hono routes (CRUD + SEARCH) -│ │ ├── service.ts # Business logic -│ │ ├── validation.ts # Zod schemas -│ │ └── __tests__/ # Unit + integration tests -│ └── shared/ -│ ├── types.ts -│ ├── errors.ts -│ ├── pagination.ts -│ └── middleware.ts -│ -├── ui/ # React frontend -│ ├── package.json -│ ├── vite.config.ts -│ ├── tsconfig.json -│ └── src/ -│ ├── App.tsx -│ ├── main.tsx -│ ├── stores/ # Zustand stores -│ ├── hooks/ # TanStack Query hooks -│ ├── components/ # Shared components (@cluster/ui-kit) -│ ├── pages/ # Route pages (List + Detail + Form) -│ └── lib/ -│ ├── api.ts # Axios wrapper -│ └── auth.ts # JWT handling -│ -└── docs/ - ├── architecture/ # Architecture PDF + DOCX - ├── memory/memory.md # Project memory (single source of truth) - └── session/ # Session logs (one per Claude Code session) -``` +- **Runtime:** Node.js 20+ / TypeScript +- **Server:** Hono +- **Database:** PostgreSQL + Drizzle ORM +- **Cache / Queue:** Redis +- **Frontend:** React 18 + Vite + Zustand + MUI + TanStack Query +- **Agent:** Rust (separate repo — [sigops-agent](https://github.com/diskhacker/sigops-agent)) --- -## Getting Started +## Requirements -### Prerequisites +- Node.js **>= 20** +- PostgreSQL **>= 15** +- Redis **>= 7** +- Docker (optional, for the bundled `docker-compose.yml`) -- **Node.js** 20+ (`winget install OpenJS.NodeJS.LTS`) -- **pnpm** (`npm install -g pnpm`) -- **Docker Desktop** (for PostgreSQL + Redis) -- **Git** + **GitHub CLI** (`winget install GitHub.cli`) +--- -### Setup +## Quick Start ```bash # 1. Clone -git clone https://github.com//sigops.git +git clone https://github.com/diskhacker/sigops.git cd sigops -# 2. Environment +# 2. Install +pnpm install + +# 3. Configure cp .env.example .env -# Edit .env: set JWT_SECRET, AUTH_PROVIDER_URL, DATABASE_URL, REDIS_URL +# Edit .env: set JWT_SECRET, DATABASE_URL, REDIS_URL, AUTH_PROVIDER_URL -# 3. Start databases +# 4. Start Postgres + Redis docker compose up -d -# PostgreSQL on port 5432, Redis on port 6372 -# 4. Install dependencies -cd server && pnpm install -cd ../ui && pnpm install -cd .. - -# 5. Create database tables +# 5. Create tables cd server && pnpm db:push -# 6. Seed initial data -pnpm db:seed - -# 7. Start development server +# 6. Run the server (port 4200) pnpm dev -# Backend running on http://localhost:4200 -# 8. Start frontend (separate terminal) -cd ui && pnpm dev -# Frontend running on http://localhost:4201 -# API proxied to http://localhost:4200 +# 7. Run the UI (separate terminal, port 4201) +cd ../ui && pnpm dev ``` -### Verify +Health check: ```bash -# Health check curl http://localhost:4200/health # → { "status": "ok", "db": "connected", "redis": "connected" } - -# Run tests -cd server && pnpm test - -# Run tests with coverage (must be >90%) -pnpm test:coverage ``` --- -## Environment Variables +## Configuration -```bash -# Database -DATABASE_URL=postgresql://postgres:postgres@localhost:5432/sigops_dev - -# Redis -REDIS_URL=redis://localhost:6372 - -# Identity provider (required — issues HS256 JWT tokens) -AUTH_PROVIDER_URL= -AUTH_PROVIDER_API_KEY= -JWT_SECRET= - -# Product Identity -PRODUCT_ID=sigops -PRODUCT_NAME=SigOps -PORT=4200 - -# Environment -NODE_ENV=development -LOG_LEVEL=debug -``` +| Env var | Required | Default | Description | +|---|---|---|---| +| `DATABASE_URL` | yes | — | Postgres connection string | +| `REDIS_URL` | yes | — | Redis connection string | +| `JWT_SECRET` | yes | — | HS256 shared signing secret (min 32 chars). Must match your identity provider. | +| `AUTH_PROVIDER_URL` | yes | — | URL of your identity provider (UAP or compatible). Used for product registration at boot. | +| `AUTH_PROVIDER_API_KEY` | yes | — | Service key used to register this product with the identity provider. | +| `PRODUCT_ID` | no | `sigops` | Identifier used in JWT audience claim. | +| `PORT` | no | `4200` | Backend port. | +| `NODE_ENV` | no | `development` | `development` \| `production`. | +| `LOG_LEVEL` | no | `info` | `debug` \| `info` \| `warn` \| `error`. | --- ## Authentication -SigOps requires an external identity provider that issues HS256 JWT tokens. -Configure via `JWT_SECRET` (shared secret) and `AUTH_PROVIDER_URL` env vars. +SigOps requires an external identity provider that issues HS256 JWT tokens. Configure via `JWT_SECRET` (shared signing secret) and `AUTH_PROVIDER_URL`. -SigOps deliberately does **not** include its own auth, tenants, billing, -RBAC, notifications, or audit-logging stack — those are the identity -provider's concern. Compatible with any provider that: +SigOps verifies tokens locally with the shared secret — there is no per-request auth round-trip. Compatible with any provider that: -- issues HS256 JWT tokens -- includes `sub` (user id), `tid` (tenant id), and `permissions` claims -- verifies locally via shared secret (no per-request auth roundtrip) +- Issues HS256 JWT tokens +- Includes `sub` (user id), `tid` (tenant id), and `permissions` claims +- Exposes a `POST /products/register` endpoint for product self-registration -ClusterAssets operates a managed identity layer — see **SigOps Cloud** below. +At boot, SigOps registers itself with the configured provider so administrators can map roles, plans, and notification templates from a single control plane. -At boot, SigOps registers itself with the configured provider -(`POST ${AUTH_PROVIDER_URL}/products/register`) so that admins can map roles, -plans, and notification templates from a single control plane. +For managed identity, billing, RBAC, and audit — see **SigOps Cloud** below. --- -## Shared Packages +## SigOps CLI -This product uses shared `@cluster/*` packages from the `cluster-shared` repo: +A local task registry for automating scripts. Works standalone without a server. Connected mode adds remote execution via agents. + +```bash +npm install -g @sigops/cli +sigops init +sigops add my-task --run "node scripts/deploy.js --env={{env}}" +sigops run my-task --env=production +sigops ui # web dashboard at http://localhost:3939 +``` -| Package | Purpose | -|---------|---------| -| `@cluster/uap-client` | HTTP client for the configured identity provider | -| `@cluster/auth-middleware` | JWT verification + RBAC + tenant isolation | -| `@cluster/drizzle-utils` | Pagination, audit logging, soft delete helpers | -| `@cluster/zod-schemas` | Shared validation schemas (pagination, errors) | -| `@cluster/ui-kit` | DataTable, FormDialog, AppShell, ThemeProvider | -| `@cluster/test-utils` | Test JWT minting, DB setup, API client | -| `@cluster/eslint-config` | Shared ESLint + Prettier config | +See [sigops-cli](https://github.com/diskhacker/sigops-cli) for full docs. --- -## Development Protocols +## SigOps Cloud -Every feature follows this mandatory process: +SigOps open-source gives you the execution engine. **SigOps Cloud** adds what production teams need on top: -``` -AUDIT → Check what exists (memory, sessions, codebase) -REVIEW → Cross-check against architecture document -CONFIRM → Confirm approach before coding -REUSE → Check @cluster/* packages first -IMPLEMENT → Write code following standards -``` +### Intelligence +- **AI Advisor** — incident context summaries and suggested remediation +- **Knowledge Engine** — runbook search over your documentation -A feature is **complete** only when: -- ✅ Backend API works (all CRUD + SEARCH endpoints) -- ✅ Zod validation on all request/response -- ✅ Frontend UI works (List + Detail + Create + Edit + Search pages) -- ✅ UI ↔ Backend wired via TanStack Query hooks -- ✅ Unit tests >90% coverage -- ✅ Integration tests on all endpoints -- ✅ Responsive (mobile + tablet + desktop) -- ✅ Session log updated -- ✅ Memory updated +### Incident Management +- **On-Call** schedules with escalation chains +- **War Room** — collaboration during active incidents +- **Postmortem** workflow with templates +- **Status Page** (public + private) +- **SLA Tracker** with breach alerts ---- +### Enterprise +- Managed identity, billing, and RBAC (via UAP) +- Audit log with export +- Multi-tenant isolation -## Testing +### Marketplace +- Curated tool registry +- Pre-built integrations (Slack, JIRA, PagerDuty, and more) -```bash -# Unit + integration tests -cd server && pnpm test +[sigops.clusterassets.com](https://sigops.clusterassets.com) -# With coverage report -pnpm test:coverage +--- -# Watch mode (during development) -pnpm test:watch +## Roadmap -# Specific module -pnpm test -- --grep "clients" -``` +Features planned but not yet shipped: -Coverage thresholds (enforced in vitest.config.ts): -- Lines: 90% -- Functions: 90% -- Branches: 85% -- Statements: 90% +- [ ] **FlowBuilder** — visual workflow editor +- [ ] **Bidirectional adapters** for PagerDuty, Datadog, and Grafana (currently inbound-only Prometheus) +- [ ] **WebSocket agent transport** (currently HTTP polling) +- [ ] **Rate limiting** on signal ingestion +- [ ] **OpenTelemetry / Prometheus metrics** export +- [ ] **Pattern Learning** — auto-detect recurring incidents +- [ ] **Predictive Incidents** — early warning system +- [ ] **Email signal polling** --- ## Scripts | Command | Description | -|---------|-------------| +|---|---| | `pnpm dev` | Start dev server with hot reload | -| `pnpm build` | TypeScript compile to dist/ | +| `pnpm build` | TypeScript compile to `dist/` | | `pnpm start` | Run production build | | `pnpm test` | Run all tests | -| `pnpm test:coverage` | Run tests with coverage | -| `pnpm db:generate` | Generate Drizzle migration | -| `pnpm db:push` | Push schema to database | +| `pnpm test:coverage` | Run tests with coverage (>=90% lines/functions/statements, >=85% branches) | +| `pnpm db:generate` | Generate a Drizzle migration | +| `pnpm db:push` | Push schema to database (dev only — do NOT use in production) | +| `pnpm db:migrate` | Apply generated migrations (production path) | | `pnpm db:seed` | Seed initial data | | `pnpm lint` | Run ESLint | | `pnpm typecheck` | TypeScript type checking | --- -## SigOps Cloud - -SigOps open-source gives you the execution engine. **SigOps Cloud** adds what -production teams need on top: - -### Intelligence - -- **AI Advisor** — incident context summaries, suggested remediation -- **Knowledge Engine** — runbook RAG over your Confluence / docs - -### Incident Management - -- **On-Call** schedules with escalation chains -- **War Room** — real-time collaboration on active incidents -- **Postmortem** workflow with templates -- **Status Page** (public + private) -- **SLA Tracker** with breach alerts - -### Enterprise - -- SSO (SAML, OIDC) -- Audit log export (S3, SIEM integrations) -- Advanced RBAC with custom roles -- Multi-region deployment -- Dedicated support SLA - -### Marketplace +## Contributing -- Curated tool registry -- Pre-built integrations (PagerDuty, Datadog, Slack, JIRA, and more) -- Community playbooks +1. Read `CLAUDE.md` before making any changes. +2. Follow naming conventions (kebab-case files, snake_case tables, PascalCase components). +3. Every PR must include: backend + frontend + tests. +4. Update `docs/session/` with what you changed and `docs/memory/memory.md` if decisions were made. -**Managed hosting · Identity included · [Learn more →](https://sigops.clusterassets.com)** +See `CONTRIBUTING.md` if present. --- -## Contributing - -1. Read `CLAUDE.md` before making any changes -2. Follow naming conventions (kebab-case files, snake_case tables, PascalCase components) -3. Every PR must include: backend + frontend + tests -4. Update `docs/session/` with what you changed -5. Update `docs/memory/memory.md` if decisions were made +## License ---- +MIT — ClusterAssets Innovation Pvt. Ltd.