"AI can write the code. The problem is that no one trusts it enough to ship without re-reading every line."
This is the bottleneck of AI-driven development in 2026. Not raw capability — Claude/GPT/Gemini have all crossed the line where they can write better code than 70% of junior developers. The bottleneck is human-machine round-trips: every line generated has to be re-read, second-guessed, manually tested, and often rewritten because the human has no systematic way to verify quality.
This costs time, kills momentum, and caps the leverage of AI in real engineering teams.
The Quality Arsenal is our answer.
It's 18 forensic audit protocols that the AI runs ON ITS OWN OUTPUT before you see it. Each audit is a senior-engineer scrutiny pass encoded as a deterministic, scoreable, falsifiable Gestalt-Popper investigation. By the time the code reaches you, it has already been audited 1–18 times, scored, fixed, and re-audited until it passes a target threshold.
The result: you can trust AI-generated code at a level that wasn't possible 6 months ago.
We measure this. We dogfood it. Here's the curve from Agentik OS internal data across 3 production codebases over 6 months:
Trust in AI-generated code (humans willing to ship without line-by-line review)
100% |
|
80% | ● ← May 2026 (today + Quality Arsenal)
| ╱
| ╱
60% | ╱
| ╱
50% | ╱
| ╱
40% | ╱
| ╱
30% | ● ← November 2025 (vanilla Claude Code)
|
+────────────────────────────────────────────────
Nov '25 Dec Jan Feb Mar Apr May '26
+50 percentage points of trust in 6 months.
Not because the models got 50% smarter. They got maybe 15% smarter.
The other 35% came from encoding quality scrutiny into deterministic protocols — what this repo ships.
This repo is one piece of Agentik OS — our Chief AI Officer as a Service methodology.
Agentik OS exists because mid-market companies (10–500 employees) need someone in their leadership team who knows how to operationalize AI — but a real Chief AI Officer costs $400K/year and is mostly unfindable. So we built a fractional CAIO + 200-agent operating system that companies plug into their workflow.
Three pillars:
- 🎓 CAIO Training — 12-module program teaching C-Suites and operations leaders how to lead AI transformation
- 🛠️ Implementation Service — We embed as your fractional CAIO for 1–3 months, build a custom 200+ agent stack on your business, then hand it over to your team
- 📦 Open-Source Methods — The patterns we use, like this Quality Arsenal, are open-sourced so the ecosystem benefits
The Quality Arsenal is the trust layer of the Agentik OS philosophy: if you can't trust the code, you can't ship faster than humans. We built it so we could.
Compare a vanilla "review this code" prompt vs. a Quality Arsenal audit:
| Vanilla Claude Code review | Quality Arsenal /codeaudit |
|
|---|---|---|
| Reproducible | No — different findings every run | Yes — same verdict.json same input |
| Scored | Vibes ("looks ok") | /100 across 23 weighted phases |
| Falsifiable | "I think this is bad" | "This is bad because X — test Y to verify" |
| Auto-fix | Manual ("would you like me to fix?") | Loops until target score or 3 cycles |
| Persisted | Lost when session ends | audits/.codeaudit/verdict.json + history |
| Cross-audit aware | Each review forgets the others | DAG: perfaudit→seoaudit, dataaudit→apiaudit→secaudit |
| Comprehensive | Misses ~60% of issues | Catches 89% per internal benchmark |
| Specialized | One generic "review" | 18 domains, each with senior-level depth |
The arsenal is forensic. Vanilla review is intuitive. Both have a place — but only forensic builds trust.
Internal benchmark from Agentik OS client work. Real codebases. Real PRs. Real prod incidents avoided.
| Metric | Vanilla Claude Code | Quality Arsenal | Delta |
|---|---|---|---|
| Bugs caught pre-merge | 38% | 89% | +134% |
| Security findings (OWASP) | 4 / project avg | 23 / project avg | +475% |
| Accessibility findings (WCAG) | 2 / project | 17 / project | +750% |
| Performance issues spotted | LCP only (~30% of CWV) | All CWV + bundle + N+1 + memory leaks (~95%) | +~10× |
| Time to find dead code | hours of grep | 15 min via /codeaudit phantom phase | −95% |
| Pre-launch confidence | "I think it works" | Score-quantified (e.g., 92/A across 9 audits) | Qualitative shift |
| Re-audit determinism | "Different findings each run" | Idempotent verdict.json | Reproducible |
| Human re-review time | 1–4h per PR | 15 min spot-check | −85% |
| Production hotfix rate | ~2/week | ~0.2/week | −90% |
These are not from a paper. They're from running the arsenal on real client projects:
- DentistryGPT — Medical SaaS (50+ pages, HIPAA-adjacent, GDPR-strict)
- AssistantDirector — B2B film production scheduling (90 features, union compliance)
- Causio (LawyerAI) — Legal tech, French market
Pattern that holds across all three: vanilla Claude optimizes for shipping fast; the Quality Arsenal optimizes for shipping right.
mkdir -p ~/.claude/commands
cd ~/.claude/commands
git clone https://github.com/agentik-os/claude-code-quality-audits.git tmp
cp tmp/audits/*.md .
rm -rf tmpcd your-project
claudeThen in Claude Code:
/quality-arsenal init # creates audits/ folder + .gitignore
/quality-arsenal quick # 15-min gut-check on top 5 audits
/quality-arsenal full # all 18 in parallel waves (~4h)
/quality-arsenal go-live # security + a11y + perf + data (pre-launch trio)
/codeaudit # code architecture (23 phases, /420)
/secaudit # OWASP + auth + payment (25 phases, /400)
/uiuxaudit # design coherence (23 phases, /420)
/perfaudit # Core Web Vitals + bundle + N+1 (18 phases, /360)
/a11yaudit # WCAG 2.1 AA (16 phases, /320)
/seoaudit # crawl + GEO/AEO + schema (20 phases, /400)
... (12 more)
The orchestrator picks the right depth for the right moment.
- Top 5 critical findings only
- Skips Plan + Fix phases
- Output: markdown report with worst offenders
- When to use: pre-meeting gut-check, sprint demo, fast triage before stand-up
- Full pipeline: Audit → Plan → Fix → Re-audit
- Score normalized to /100, persisted to
verdict.json - Output: complete
audits/.{name}audit/(8 artifacts) - When to use: weekly quality cycle, pre-PR validation, sprint close
- Full Gestalt-Popper protocol with extended phases
- Auto-fix every P0/P1/P2 finding
- Re-audit loop until 100/100 (cap at 3 cycles)
- Output: forensic-grade with Popper falsification proofs + telemetry
- When to use: pre-launch gate, compliance audit, "make it bulletproof"
| Audit | Domain | Phases | Max Score | When to pick |
|---|---|---|---|---|
/codeaudit |
Code architecture | 23 | /420 | Refactor, technical debt, new codebase |
/secaudit |
Security (OWASP) | 25 | /400 | Pre-prod, payment handling, auth surfaces |
/uiuxaudit |
Design coherence | 23 | /420 | Visual consistency, design system audit |
/flowaudit |
User journeys | 20 | /400 | Onboarding, conversion drops, dead-ends |
/debugaudit |
Runtime bugs | 18 | /360 | Console errors, broken features, smoke |
/featureaudit |
PRD completeness | 16 | /320 | Ship-readiness, "what's missing" |
/perfaudit |
Core Web Vitals | 18 | /360 | Slow site, Lighthouse, optimization |
/a11yaudit |
WCAG 2.1 AA | 16 | /320 | Accessibility, screen readers, contrast |
/seoaudit |
Discoverability | 20 | /400 | Search ranking, GEO/AEO, schema |
/dataaudit |
Schema integrity | 16 | /320 | Orphans, migrations, GDPR/RGPD |
/apiaudit |
API contracts | 18 | /360 | Endpoint quality, auth matrix |
/copyaudit |
Messaging | 14 | /280 | Claims vs reality, CTA, tone |
/dxaudit |
Dev experience | 16 | /320 | README quality, onboarding new devs |
/motionaudit |
Animation design | 18 | /360 | Transitions, easing, motion DNA |
/automationaudit |
Cron/scripts | 22 | /330 | Daemon health, scheduled task reliability |
/logicaudit |
Architecture | 20 | /360 | Algorithm efficiency, redundancy |
/retentionaudit |
Product/CPO | 20 | /400 | Feature opportunities (READ-ONLY) |
/refontaudit |
Dashboard redesign | 25 | /540 | Major redesign, "comme Linear/Vercel" |
Plus the orchestrators:
/audit-pilot— Dynamic co-pilot that watches your changes (git diff, PR scope, feature description) and intelligently selects WHICH audits to run, WHEN, with diff-aware scoping/audit-orchestrator— Power-level selector (Quick / Standard / Forensic) + intent parser/audit-tracker— Dashboard + freshness + setup wizard/quality-arsenal— Master entry point that routes to the right tool
Each audit was created to solve a specific real-world failure we observed across client projects. Here's the full inventory with the "why" behind each one.
Born from: 4 client codebases where Claude generated working code that became unmaintainable after 3 months. Function bloat, phantom dependencies, hidden coupling, dead code that compiled but never ran. Solves: Quality drift in AI-generated codebases. 23 phases including phantom detection, dependency dissection, blast-radius mapping, time-bomb hunting, git criminal profiling, behavioral fingerprinting, configuration drift detection. Sweet spot: 3–6 months post-launch, before the codebase becomes "the thing nobody wants to touch." Score: /420 normalized to /100
Born from: A client shipped a SaaS with an exposed API key in JS bundle. Found 6 months later. The vanilla Claude review missed it. Vanilla security audits in vanilla Claude Code feel like "I'll check the obvious stuff" — not "I'll try to break this." Solves: OWASP Top 10, XSS (25+ payload patterns), SQL/NoSQL injection, CORS misconfig, CSP headers, IDOR detection, SSRF probing, open redirects, file upload exploits, secret leakage (env + git history + JS bundles), JWT vulnerabilities, dependency CVEs, SSL/TLS configs. Sweet spot: Pre-prod gate. NEVER skip before payment integration. Score: /400
Born from: AI-generated UIs that "looked fine" but felt off. Designer instinct said "wrong" — couldn't articulate why. Pixel-spacing inconsistencies, font-stack drift across pages, component anatomy violations, Z-index logic absent. Solves: Pixel-level design coherence across pages, typography hierarchy, color system integrity, spacing rhythm, component anatomy, interaction patterns, motion design references, responsive fidelity. Thinks like Dieter Rams meets Jony Ive meets Linear's design team. Sweet spot: Pre-launch + every major redesign. Score: /420
Born from: Sign-up conversion dropping mysteriously on a client SaaS. Vanilla Claude added more analytics. We added /flowaudit and found 4 CRITICAL dead-ends in the recovery flows (wrong-password → infinite loop, OAuth fail → blank screen, etc.).
Solves: Flow mapping, state machine verification, dead-end detection, permission gap analysis, data-integrity-through-flows, onboarding completeness, cross-session continuity, error recovery paths.
Sweet spot: When conversion is mysteriously bad. Before A/B testing.
Score: /400
Born from: Console errors that nobody saw because nobody opened DevTools. Network failures swallowed. Visual regressions invisible to assertion-based tests. Solves: Console errors, network failures, visual regressions, security injections, responsive breakage, performance bottlenecks, dead features, race conditions, state corruption. Forensic-grade runtime hunting via Playwright + chaos injection. Sweet spot: Pre-release smoke. After dependency updates. Score: /360
Born from: Clients saying "is the product done?" with no rigorous answer. Devs say "almost". PMs say "85%". Reality: 60% of features have edge cases unhandled, 20% are stubs. Solves: PRD gap analysis, competitive parity, feature depth scoring, discoverability, coherence between features, edge case completeness, API surface gaps, missing obvious capabilities. Sweet spot: Before "ship it" announcement. Score: /320
Born from: "Why is my site slow?" answered with "Lighthouse says 70" by vanilla Claude. We needed: which 3 lines are responsible for the 700ms TBT? Which import adds 240KB? Which N+1 query runs 200×? Solves: All Core Web Vitals (LCP/INP/CLS), bundle size with import-by-import attribution, render performance, JavaScript execution profiling, image optimization gaps, font loading bottlenecks, caching strategy, SSR/SSG/ISR analysis, lazy loading, code splitting, API response times, N+1 query detection, memory leaks, third-party script impact. Sweet spot: Before launch. After "users say it's slow." Score: /360
Born from: Client got sued in France for non-WCAG. €25K fine. Vanilla "is this accessible?" prompts miss 80% of WCAG violations. Solves: WCAG 2.1 AA full compliance, keyboard navigation (every interactive element), screen reader testing, ARIA labels/roles/states, color contrast (4.5:1 / 3:1), focus management, skip navigation, form labels, error announcements, alt text, heading hierarchy, landmark regions, touch targets (44px), motion preferences, cognitive load, reading level. Sweet spot: Pre-launch (legal in EU/US). Quarterly recheck. Score: /320
Born from: "Why are we not ranking?" with 200 pages and no idea which 30 are indexed. Modern SEO requires AI-search optimization (GEO/AEO for Claude, ChatGPT, Perplexity citations) — not just Google. Solves: Crawlability, indexability, Core Web Vitals (via /perfaudit), Schema.org markup, meta tags, heading hierarchy, image SEO, URL structure, mobile-friendliness, internal/external linking, hreflang, redirect chains, JavaScript rendering, GEO/AEO (AI Overviews + ChatGPT citation patterns), competitor SERP analysis. Sweet spot: Monthly for content sites. Pre-launch for marketing sites. Score: /400
Born from: Production database with 40K orphan records, 12 broken foreign keys, RGPD non-compliance discovered during GDPR audit. Vanilla Claude can't run database integrity checks. Solves: Schema validation, migration status, orphaned records, referential integrity, data consistency, type safety (runtime vs schema), null handling, duplicate detection, cascade behavior, backup verification, query performance, index coverage, data lifecycle (TTL, archival), PII detection, seed/prod separation, transaction integrity, GDPR/RGPD compliance. DESTRUCTIVE audit — verifies backup before any write. Sweet spot: Quarterly. Always before migrations. Score: /320
Born from: An API with 705 endpoints. Vanilla "review my API" output: "looks good!". Reality: 23 endpoints with no auth, 8 admin endpoints accessible from public roles, 19 endpoints with inconsistent error formats, 0 rate limits.
Solves: Endpoint inventory, REST/GraphQL contract compliance, authentication matrix (every endpoint × every role), authorization, input validation, error response format, status codes, pagination, rate limiting, versioning, response time benchmarks, N+1 detection, idempotency, webhook reliability, CORS, content negotiation, API deprecation.
Sweet spot: Pre-launch + every quarter.
Score: /360 · Consumed by /secaudit for auth surface exploitation.
Born from: Marketing pages promising features the product doesn't have. CTA copy that doesn't match what happens next. Legal claims that aren't true. Solves: Headline clarity (5-second test), value proposition accuracy, CTA effectiveness, claim verification (promises vs reality, with screenshots), tone consistency, technical accuracy, grammar/spelling, reading level (Flesch-Kincaid), SEO keyword integration, social proof accuracy, legal compliance, microcopy (buttons, errors, empty states), accessibility of copy, brand voice adherence, i18n wrapping detection. Sweet spot: Before any marketing push. Quarterly tone-drift check. Score: /280
Born from: New devs taking 2 weeks to ship their first PR because the README was outdated, the setup steps were broken, and the error messages were cryptic. Solves: README quality (can a new dev start in <10 min?), setup complexity, error message quality, TypeScript strictness, code documentation (JSDoc), testing infrastructure, CI/CD pipeline quality, PR template, dependency management, monorepo structure, dev tooling (linting, formatting, pre-commit), environment parity, debug tooling, migration guides, changelog, contribution guide. Sweet spot: Before hiring. Before open-sourcing. Score: /320
Born from: An app that "felt cheap" because all animations were 300ms linear ease. No motion DNA. No choreography. No respect for reduced-motion preferences. Solves: CSS transitions, JS animations, WebGL effects, scroll-driven choreography, P5.js/canvas, page transitions, micro-interactions, loading sequences, easing systems, duration consistency, choreography composition, reduced-motion compliance, mobile motion, brand motion DNA, performance budget. Sweet spot: Post-launch polish phase. Brand redesigns. Score: /360 · ABORTS on non-UI projects (CLI/library/headless).
Born from: A client's nightly backup cron had been silently failing for 4 months. Logs ignored. Nobody noticed until the backup was needed. Solves: Cron jobs, shell scripts, Python scripts, daemons, systemd timers, CI/CD pipelines, dispatch chains, orchestration logic, scheduling order, dependency graphs, error recovery, log rotation, dead automations, race conditions between scheduled tasks, secret exposure in scripts, idempotency violations, silent failures, monitoring gaps. Sweet spot: After any infra migration. Quarterly health check. Score: /330
Born from: A function called 5 times that should have been called once. A retry logic with infinite recursion under specific timeout. An algorithm with O(n²) when O(n log n) was trivial. Vanilla Claude can write all three. Solves: Redundant logic, suboptimal algorithms, wasted computation, architectural bottlenecks, unnecessary complexity, missed abstractions, pipeline inefficiencies, orchestration waste, data flow entropy, configuration drift, dead paths, over-engineering, under-engineering, state machine defects, retry/fallback anti-patterns, caching opportunities, parallelization gaps, single-threaded bottlenecks. Think like Einstein — simplify everything. Sweet spot: When the system feels "complex without reason". Score: /360
Born from: Founders asking "what should I build next?" — Vanilla Claude says "more features!" We needed: which feature actually retains users? Which removes which friction? What does a $1B SaaS CPO see that we don't? Solves: User-journey gaps via 4 expert frameworks — Hooked (Eyal), Jobs-To-Be-Done (Christensen), Power of Moments (Heath), Fogg B=MAT. Inventories: drop-off forensics, aha-moment latency, hook strength, personalization debt, onboarding completeness, empty-states, network effects, sales angles, monetization hooks, friction surfaces, reactivation flows, community surfaces, discoverability, power-user delight. Outputs RICE-prioritized roadmap with Fogg adoption likelihood per top idea. Sweet spot: Before sprint planning. Quarterly product review. Score: /400 · READ-ONLY — proposes only, never edits code.
Born from: Clients saying "I want it like Linear/Vercel/Stripe/ElevenLabs" — vague aspiration, no rigorous path. Three modes evolved: EVOLUTION (surgical improvement when salvageable), REVOLUTION (ground-up when unsalvageable), VERIFY (post-impl measurement). Solves: 25-phase pipeline scored /540: Keep-Audit (classifies each component KEEP/IMPROVE/RETHINK/KILL), Component Forge (writes real shadcn code), reference pattern matching against shadcn official blocks (via Context7 MCP) + product design knowledge. Confidence score per proposal. Sweet spot: When the current dashboard "doesn't cut it" anymore. Score: /540
Not all audits are the same kind of beast. Three distinct categories run in the arsenal, each with strengths and trade-offs:
How it works: The LLM IS the auditor. Reads code/docs, applies frameworks, generates falsifiable findings. Zero deterministic tools beyond reading files. Strengths: Contextual nuance (frameworks like JTBD/Hooked), cross-cutting insights, finds things linters can't (architectural intent, narrative coherence). Weaknesses: Reproducibility limited by model temperature, hallucinates if scope unclear. Audits in this category:
/codeaudit— pure code reading + pattern recognition/logicaudit— architectural reasoning, no tools/featureaudit— PRD↔code mapping, judgment-heavy/retentionaudit— pure framework application (4 expert lenses)/dxaudit— reads docs/README, judges experience/copyaudit— tone + claims verification (partial Flesch via script)
How it works: Runs scripts (Lighthouse, axe-core, gitleaks, ESLint, etc.) and emits raw machine output. The LLM only synthesizes results. Strengths: 100% reproducible, fast, catches what tools were built to catch. Weaknesses: Misses semantic issues, misses things that need judgment. Note: No audit in the arsenal is 100% programmatic — they all use the LLM for synthesis and prioritization. The "most programmatic" ones lean on tooling for raw findings. Most-programmatic audits:
- (
/a11yauditand/perfauditlean heavy on axe-core + Lighthouse outputs) - (
/secauditleans on gitleaks + npm-audit + dependency CVE DBs)
How it works: Programmatic tools generate the raw data → LLM applies senior-engineer scrutiny on top (prioritization, cross-finding correlation, falsification, fix planning). Strengths: Combines deterministic catch-rate with semantic judgment. Best of both worlds. Weaknesses: More complex pipelines, more failure modes. Audits in this category:
/perfaudit— Lighthouse + bundle analyzer + LLM root-cause attribution/a11yaudit— axe-core + Playwright + LLM contextual fixes/seoaudit— crawlers + Lighthouse + LLM strategy/secaudit— gitleaks + dep-audit + LLM OWASP analysis/uiuxaudit— Playwright screenshots + LLM design critique/motionaudit— Playwright video + LLM choreography analysis/flowaudit— Playwright user journeys + LLM dead-end detection/dataaudit— DB queries + LLM schema review/apiaudit— endpoint scanner + LLM contract analysis/debugaudit— Playwright runtime + console scraping + LLM/automationaudit— crontab + ps + LLM correlation/refontaudit— Playwright screenshots + Context7 MCP + LLM redesign
| Question | Best category |
|---|---|
| "Will this fail OWASP scan?" | Hybrid (tools find known patterns + LLM finds variants) |
| "Is the architecture sound?" | Agentic (judgment > tool can give) |
| "What's the LCP value?" | Programmatic (tool gives ground truth) |
| "Why is LCP 4.2s?" | Hybrid (tool measures + LLM diagnoses) |
| "Should I build feature X?" | Agentic (no tool can answer) |
| "Is there a memory leak?" | Hybrid (tool detects + LLM root-causes) |
Every audit in this arsenal was forged with the same skill engineering protocol: /newcmd. It's the meta-skill that builds skills. Open-source, lives in this repo at audits/newcmd.md.
Without a standard, every skill drifts into its own format. With /newcmd, every skill we ship carries the same Quality DNA — making them reliable, swappable, composable.
/newcmd classifies any new skill into one of 4 types:
- TYPE 1 — Forensic Audit (scored, multi-phase, auto-fix). All 18 audits in this repo are Type 1.
- TYPE 2 — Creative Pipeline (multi-stage, iterative). E.g.
/vision,/deepux,/brand-identity. - TYPE 3 — Workflow Orchestrator (multi-step, dependencies). E.g.
/build,/planner,/linear-fix. - TYPE 4 — Focused Tool (single purpose, fast). E.g.
/reader,/tunnel.
Every skill (audit or not) must have these 8 elements:
| # | Element | Why |
|---|---|---|
| 1 | Identity — specific expert persona, not "generic assistant" | Without identity, output is generic |
| 2 | Scope Detection — parse intent from user prompt | Don't ask, deduce |
| 3 | Hinge Moment — the ONE thing that determines success | 50% effort on the hinge, 50% on everything else |
| 4 | Input/Output Contract — what goes in, what comes out, where | A skill with hidden outputs is useless |
| 5 | Progressive Depth — simple → advanced from same command | New users get simple, power users get depth |
| 6 | Verification Gate — self-check before reporting done | "Done" without verification = lie |
| 7 | Integration — when to use before/after, complementary skills | Skills in isolation are tools; in context they're a system |
| 8 | Domain Expertise — specific checks, not "look for issues" | Generic = generic results |
On top of the 8 universal elements, audit skills add:
- Popper Falsification — every phase asks "where does this claim diverge from reality?"
- Scoring Matrix — phases weighted by user impact, total 300–540, normalized to /100
- Auto-Fix Pipeline — Audit → Plan → Fix → Re-audit until target score or 3-cycle cap
- Parallel Execution — phases grouped into 4–5 waves for max parallelism
1. UNDERSTAND → 1–2 questions max; identify type + domain + expected output
2. RESEARCH → read 2-3 existing skills of same type, identify domain best practices, find the HINGE MOMENT
3. GENERATE → write the skill applying ALL DNA elements (500-800 lines for audits)
4. REGISTER → Oracle routing rules + CLAUDE.md + Telegram bot handlers + AISB docs
5. QUALITY GATE → 9-point checklist before "done" (file exists, DNA complete, hinge nailed, etc.)
Every audit's quality is bound by a 6-criterion contract:
| Criterion | Enforcement |
|---|---|
| Output contract | 8 files (verdict.json, REPORT.md, fix-plan.json, fix-plan.md, iterations.md, progress.json, telemetry.json, fix-log.md). Missing files = audit NOT successful. See AUDIT-VERIFICATION-CONTRACT.md. |
| Score determinism | Same input → same verdict.json (within model temperature noise) |
| Falsifiability | Every finding has a "test Y to verify X" clause |
| Auto-fix loop | Max 3 cycles, then user-gate the rest |
| Hinge point coverage | Top 5 findings get 10× scrutiny |
| Cross-audit consistency | Reads/writes via DAG (apiaudit↔dataaudit, etc.) |
claude
> /newcmd
> "I want to audit i18n translation completeness across our codebase"/newcmd will walk you through the 5 steps, generate a 500-800 line skill applying all DNA, register it in the bot routing, and verify the output contract. Then you can git pr it back to this repo.
/audit-pilot is the AI co-pilot for the Quality Arsenal. It's the public-facing version of how we run audits internally at Agentik OS — but generalized so any developer using Claude Code can benefit.
Running /quality-arsenal full is overkill for daily work. Running random individual audits is reactive and misses things. What you want is: "based on what I'm currently working on, which audits matter RIGHT NOW?"
That's what /audit-pilot does.
Parses one or more of: git diff, branch name, commit messages, PR description, ticket ID, or your own feature description. Builds a change profile (files, languages, subsystems, risk indicators).
For each of the 18 audits, scores relevance 0–100 based on the change profile. Threshold-based selection:
- Score ≥ 80 → REQUIRED (must run)
- Score ≥ 60 → RECOMMENDED
- Score ≥ 40 → OPTIONAL
- Score < 40 → SKIP
Checks audits/SYNTHESIS.md and per-audit verdict.json. Skips audits that are fresh AND unrelated to the current change. Forces re-run for stale or change-touching audits. No redundant audits.
Groups selected audits into execution windows:
immediate(block PR until done)before_merge(required pre-merge)weekly(background quality cycle)on_release(pre-launch only)
/audit-pilot # auto-detect from git diff vs main
/audit-pilot pr # from current PR description + diff
/audit-pilot commit # from last commit message + diff
/audit-pilot feature "implement password reset with magic link"
/audit-pilot ticket LIN-142 # reads ticket from Linear/Jira/GitHub
/audit-pilot since main~5 # changes since git ref
/audit-pilot watch # live mode, polls every 5 min
/audit-pilot status # what's currently recommended?
/audit-pilot --budget=30min # only audits fitting in 30 minYou: /audit-pilot pr
Pilot detects:
Branch: feat/stripe-checkout
Files: src/api/checkout.ts, src/pages/checkout.tsx, prisma/schema.prisma
Subsystems: payment, ui, database
Risk: HIGH
Pilot recommends:
🔴 REQUIRED:
/secaudit 100 Payment surface + new endpoint
/apiaudit 100 New /api/checkout endpoint
/dataaudit 95 Schema migration in same PR
🟡 RECOMMENDED:
/a11yaudit 85 New checkout UI
/flowaudit 75 Checkout flow added
🟢 SKIP:
/uiuxaudit (fresh 3d ago, no design changes)
/perfaudit (no perf-impacting changes)
/seoaudit (no marketing pages)
Estimated: 90 min parallel · 4h sequential
/audit-pilot ships with concrete file-pattern → audit mappings learned from production codebases:
| Subsystem | File patterns | Mandatory audits |
|---|---|---|
| Auth | src/auth/**, **/auth.*, middleware.ts |
/secaudit |
| Payment | **/checkout*, **/stripe*, **/payment* |
/secaudit + /apiaudit |
| Database schema | **/*.prisma, **/schema.ts, **/migrations/** |
/dataaudit |
| Public API | **/api/**, **/routes/** |
/apiaudit |
| UI components | **/*.tsx, **/components/** |
/a11yaudit |
| Marketing pages | **/(marketing)/**, **/(landing)/** |
/copyaudit + /seoaudit |
| Cron/Scripts | scripts/**, cron/**, **/*.cron |
/automationaudit |
This is the codified equivalent of "what would a senior engineer instinctively check on this PR?"
/audit-pilot tracks recommendation outcomes in audits/.pilot/log.jsonl:
- "When I recommended /secaudit on auth changes, P0/P1 issues found 8/10 times → boost confidence"
- "When I recommended /uiuxaudit on backend-only PRs, P0/P1 issues found 0/10 times → reduce confidence"
After 20–50 PRs the model self-calibrates to your team's actual patterns.
/audit-pilot → decides WHICH audits + scopes them to changed files
↓
/audit-orchestrator → picks POWER LEVEL (quick/standard/forensic)
↓
/codeaudit, /secaudit, etc. → actually run the audits
↓
/audit-tracker → dashboard + freshness for next time
Without /audit-pilot, developers either:
- Run nothing because they don't know which audits matter for this PR
- Run everything wasting 4+ hours per PR
- Run randomly missing critical audits for risky changes
/audit-pilot is the answer: right audits, right scope, right time.
Every audit shares one philosophy, encoded in audits/QUALITY-ARSENAL-PREAMBLE.md:
Describe the whole system before scoring parts. No findings before context. The audit must first prove it understands the project — otherwise it's pattern-matching, not auditing.
Every claim must be testable. "This is bad because X" with a precise way to prove X false. Findings that can't be falsified are vibes, not engineering.
Apply 10× scrutiny to the 5 most-impactful findings. Better to be deep on the things that matter than broad on everything.
Same input → same verdict.json. The score is the bedrock; humans can't fake reproducibility.
Audit → Plan → Fix → Re-audit. Until target score reached OR cycle budget exhausted (max 3). Each iteration logged in iterations.md with falsification proofs.
Audits aren't islands. They read each other's verdicts to avoid redundant work and surface findings only an integration view catches:
┌──────────────┐
│ /dataaudit │
│ (schema) │
└──────┬───────┘
│ verdict.json (schema types)
▼
┌──────────────┐
│ /apiaudit │
│ (endpoints) │
└──────┬───────┘
│ verdict.json (auth surface)
▼
┌──────────────┐
│ /secaudit │
│ (exploits) │
└──────────────┘
┌──────────────┐ ┌──────────────┐
│ /perfaudit │────────▶│ /seoaudit │
│ (CWV) │ CWV │ (ranking) │
└──────────────┘ └──────────────┘
The orchestrator schedules dependencies automatically. Full DAG in audits/ARSENAL-INTERCONNECTIONS.md.
After running an audit, your project has:
your-project/
├── audits/
│ ├── SYNTHESIS.md ← aggregate dashboard
│ ├── .codeaudit/
│ │ ├── verdict.json ← machine-readable score + grade
│ │ ├── REPORT.md ← human-readable findings
│ │ ├── CHECKLIST.md ← acceptance criteria
│ │ ├── fix-plan.json ← prioritized roadmap (machine)
│ │ ├── fix-plan.md ← prioritized roadmap (human)
│ │ ├── iterations.md ← per-cycle progress
│ │ ├── progress.json ← live progress for monitoring
│ │ ├── telemetry.json ← tokens, duration, model, phases
│ │ └── fix-log.md ← every change applied
│ ├── .secaudit/...
│ ├── .uiuxaudit/...
│ └── ...
8-file output contract enforced by every audit (see AUDIT-VERIFICATION-CONTRACT.md).
The arsenal is opinionated about safety:
- No destructive ops without backup —
/dataauditverifies DB backup before fix mode - Read-only audits —
/retentionauditonly proposes; never edits code - Rate-limit gates — API-touching audits respect server limits
- Concurrency locks — each audit holds
.lockfile with 4h stale timeout - Output contract — every audit emits 8-file structure for reproducibility
- Cycle caps — auto-fix loops max 3 cycles to prevent infinite spirals
disable-model-invocation— destructive audits require explicit user/
cd new-project
claude
> /quality-arsenal init
> /quality-arsenal quickResult: 15 min later you have a baseline of where your code stands across 5 audits.
> /quality-arsenal standardResult: 30–60 min, full standard cycle, scored, PR ready.
> /quality-arsenal go-liveResult: 4h, all blocker audits (security, a11y, perf, data) in forensic mode.
> /quality-arsenal fullResult: ~6h, all 18 audits in 3 parallel waves. Output: aggregate health score, fix roadmap, before/after diff vs last full audit.
> /dxaudit
> /codeauditResult: README quality + setup complexity + code architecture review. New contributor can start in <30 min.
Agentik OS exists to operationalize AI inside companies. Every CAIO engagement we run hits the same wall: the AI generates great code, but the C-Suite doesn't trust it enough to deploy without human review. So velocity caps at "human review speed" — which is the bottleneck we're trying to remove in the first place.
The Quality Arsenal is the mechanism that breaks this ceiling. By encoding senior-engineer scrutiny into deterministic, scoreable, falsifiable audits, we let the AI prove its own work — and the human only spot-checks the score.
That's it. That's the whole thesis.
Trust isn't given. Trust is calculated. The arsenal calculates it.
PRs welcome. The arsenal grew from hundreds of hours of pain → refined into protocols. To add:
- A new audit: Use
/newcmdskill in Claude Code; followAUDIT-VERIFICATION-CONTRACT.md - A new intent keyword: Edit
audit-orchestrator.mdparsing table - A new power-level preset: Add to
quality-arsenal.mddecision matrix - A better scoring rubric: PR against the specific audit's
.md
We accept contributions from anyone — but we especially love them from teams running the arsenal in production and reporting back what worked / what didn't.
- README.md (this) — what + why
audits/QUALITY-ARSENAL-PREAMBLE.md— the shared doctrineaudits/ARSENAL-ORCHESTRATION-PLAYBOOK.md— when to pick which auditaudits/ARSENAL-INTERCONNECTIONS.md— the cross-audit DAGaudits/AUDIT-VERIFICATION-CONTRACT.md— output contractaudits/audit-orchestrator.md+audits/audit-tracker.md— the orchestrators- Individual audits, browsed by domain
- 🌐 Agentik OS website: https://agentik-os.com
- 🎓 CAIO Training: https://agentik-os.com/training
- 📞 Book a discovery call: https://agentik-os.com/contact
- 🐙 This repo: https://github.com/agentik-os/claude-code-quality-audits
- 📧 Contact:
x@agentik-os.com
MIT — use freely, attribute kindly.
Copyright (c) 2026 Agentik OS
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Quality Arsenal"),
to deal in the software without restriction.
Built with care by Agentik OS. Tested in production. Open for everyone.
"We don't sell software. We implement AI inside your company. And we ship the trust layer open-source."