Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

## v3.5.0 (2026-07-22)
### Changed
- **CODEC Pilot parked.** The browser-automation pillar is withdrawn from the product: the Pilot tab is removed from the dashboard and its product card from the Cortex map. Pilot needs a far deeper build than the rest of CODEC to be trustworthy — Google blocks account sign-in from any CDP-controlled browser (not a setting we can change), and cookie walls plus bot challenges make a large share of real sites unusable. The code is parked, not deleted: it lives in its own `codec-pilot` repo and the runner is untouched, so it can return when it earns its place.
- **CODEC Pilot parked.** The browser-automation pillar is withdrawn from the product: the Pilot tab is removed from the dashboard and its product card from the Cortex map. Pilot needs a far deeper build than the rest of CODEC to be trustworthy — Google blocks account sign-in from any CDP-controlled browser (not a setting we can change), and cookie walls plus bot challenges make a large share of real sites unusable. The code is parked, not deleted: the skill, route proxy, and PM2 runner were unhooked from the repo and preserved in git history (see `docs/PILOT-PARKED.md`), so it can return when it earns its place.
- **CODEC Project folded into CODEC Overview.** Project mode is a capability of the dashboard rather than a product in its own right. No functionality changes — plan → approve → autonomous run all work exactly as before.
- **9 products → 7.** Core · Chat · Dashboard · Vibe · Agents · Dictate · Instant.
- **9 products → 7.** Core · Dictate · Instant · Chat · Vibe · Voice · Overview.

### Added
- **Claim-to-artifact matching (`codec_claim_check`).** CODEC may not claim what it did not do. Every real action is already recorded, so a claim of action with no corresponding action is false by construction — not by opinion. Catches both impossible capabilities ("I'll remember this for future sessions") and unbacked actions ("saved to your Desktop" with no file skill run). Covers the streaming and non-streaming chat paths. Tuned for false negatives over false positives: half the tests are false-positive guards.
Expand Down
25 changes: 12 additions & 13 deletions DEMO_SCRIPT.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
> Status: ✅ verified · ⚠️ needs a decision/check before filming ·
> 🔑 needs Google auth · 🧑 you run (voice/hardware).
>
> **Last sealed:** 2026-07-16.
> **Last sealed:** 2026-07-28.

## Act 1 — It hears you, acts, and types for you

Expand All @@ -26,7 +26,7 @@
| # | Feature | TRY (live) | EXPECT | Status |
|---|---|---|---|---|
| 7 | **Vision Mouse Control (UI-TARS)** — showstopper | On Cloudflare DNS: "Click the DNS button." | Cursor moves to the button and clicks — pixel coords, no accessibility API | ✅ |
| 8 | Think mode + reveal reasoning | Ask a logic trap (rate / car-wash) with Think on → "Reveal train of thought" | Correct answer + readable reasoning panel | ✅ |
| 8 | Think mode + reveal reasoning | Think on → "If 5 machines take 5 minutes to make 5 widgets, how long would 100 machines take to make 100 widgets?" → "Reveal train of thought" | Answers **"5 minutes"** (not 100) + a readable reasoning panel | ✅ |

## Act 3 — Autonomous agents (work while you watch)

Expand Down Expand Up @@ -71,37 +71,36 @@
| 3-agent zero-dep concurrency | 9 | "Up to 3 background agents on a custom sub-800-line thread pool — no LangChain, no CrewAI." |
| Plan-hash tamper + R/W grants | 10 | "The plan is sha256-hashed on approval — if the agent alters its own goals mid-run it auto-aborts. Reads and writes are glob-sandboxed to `~/codec-projects/`." |
| ~~Self-healing replay + HITL takeover~~ (parked with Pilot) | — | "Your click is recorded as an element, not a pixel — so replay survives the page moving. It falls back XPath → CSS → local-LLM rescue; hit a captcha and you take the wheel, then hand back." |
| CCF compression + temporal memory | 14 / 19 | "~65% token reduction; an FTS5 SQLite fact store with valid_from/valid_until — tell it your plans changed and it supersedes the old fact." |
| Watchdog + blocked_on_qwen recovery | 14 / 15 | "A watchdog kills zombie >500MB/<0.5%-CPU processes; if the local model drops mid-project, the agent auto-resumes the moment the port recovers." |
| CCF compression + temporal memory | 13 / 18 | "~65% token reduction; an FTS5 SQLite fact store with valid_from/valid_until — tell it your plans changed and it supersedes the old fact." |
| Watchdog + blocked_on_qwen recovery | 13 / 14 | "A watchdog kills zombie >500MB/<0.5%-CPU processes; if the local model drops mid-project, the agent auto-resumes the moment the port recovers." |
| AppKit overlay over fullscreen | any notification | "A native NSPanel floats status over *any* fullscreen app — watch it appear while I'm full-screen." |
| Proactive nudges → iMessage/Telegram | 19 | "The observer can nudge you — a doc you've dwelled on — to a macOS banner, iMessage, or Telegram. The desktop agent reaches your pocket." |
| Proactive nudges → iMessage/Telegram | 18 | "The observer can nudge you — a doc you've dwelled on — to a macOS banner, iMessage, or Telegram. The desktop agent reaches your pocket." |
| Ed25519 signed self-updates | settings | "Updates verify an Ed25519-signed Sparkle appcast against an embedded key — hardened from execution all the way to updates." |

## Cut from the demo
- **Compare across models** (ex-beat 21) — removed 2026-07-21: needs a licensed machine (AVA cloud) to show 3+ columns; on an unlicensed Mac it shows one and makes no comparison.
- **CODEC Pilot** (ex-beat 13) — parked 2026-07-22. Google blocks account sign-in from any CDP-controlled browser, and cookie walls / bot challenges make a large share of real sites unusable. It needs a far deeper build to be trustworthy; the code is parked in the `codec-pilot` repo, not deleted.
- **CODEC Pilot** (ex-beat 13) — parked 2026-07-22, unhooked from production in v3.5. Google blocks account sign-in from any CDP-controlled browser, and cookie walls / bot challenges make a large share of real sites unusable. It needs a far deeper build to be trustworthy; the code is preserved in git history (see `docs/PILOT-PARKED.md`), not deleted.
- **Live webcam vision** (ex-beat 8) — removed 2026-07-16. The capture was soft (the Anker C200's lens, not code) and it landed as a gadget rather than a capability. Nothing else depends on it.

## Decide before filming
- **Beat 18** — sign in to Notion once in the Connector tab (it persists now). Toggle **GitHub off** — it can't OAuth (needs a PAT).
- Voice interrupt (now beat 21) — **confirmed working**.
- **Beat 17** — sign in to Notion once in the Connector tab (it persists now). Toggle **GitHub off** — it can't OAuth (needs a PAT).
- Voice interrupt (now beat 20) — **confirmed working**.

## Pre-record checklist
- Chrome tabs L→R: `opencodec.org` → GitHub → Gmail → WhatsApp Web → Cloudflare → CODEC Chat
- An empty text field / Notes open for the **F5 live-typing** beat (3)
- Safari: Time Magazine article open
- Claude Desktop: CODEC connector active **and set to Always Allow** (Act 5) — otherwise beat 17 stalls on a per-call approval prompt
- Claude Desktop: CODEC connector active **and set to Always Allow** (Act 5) — otherwise beat 16 stalls on a per-call approval prompt
- Phone: `codec.avadigital.ai` loaded, Touch ID ready
- Notion signed in (Connector tab, beat 18); Philips Hue reachable; Spotify authorized (finale)
- Notion signed in (Connector tab, beat 17); Philips Hue reachable; Spotify authorized (finale)
- Vision Mouse tested 10× on the Cloudflare DNS button
- Observer running with a populated buffer
- **Pilot: do NOT rely on Google login** — Google blocks account sign-in from automated browsers. Beat 13 runs fully on public/anonymous pages (teach-mode + anonymous Gemini text). Skip Flow video-generation, which needs an account.
- Cortex map open on a 2nd screen (persistent visual anchor)
- Fresh Deep-Research + Project chat windows open

## Director's notes
- Act 1 is the emotional hook — promise (1), deliver, and F5 (3) is an unexpected early "wow".
- Showstopper risk: 7 (Vision Mouse) — if it fails live, skip to 8.
- Core "works while you watch" pair: 10 Project, 12 Vibe.
- Act 5 is THE differentiator: 17 (server) + 18 (client) back-to-back = bidirectional MCP.
- Save Hue + Spotify (23) for the very last frame.
- Act 5 is THE differentiator: 16 (server) + 17 (client) back-to-back = bidirectional MCP.
- Save Hue + Spotify (22) for the very last frame.
2 changes: 1 addition & 1 deletion FEATURES.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Sovereign AI Workstation — Full Product Breakdown

> Engine: **CODEC v3.5** — 370 features · 90 skills · 2000+ tests · 52K+ lines of production code · 7 products
> Engine: **CODEC v3.5** — 370 features · 89 skills · 2000+ tests · 52K+ lines of production code · 7 products

The product name is **Sovereign AI Workstation**. Throughout this document
and the codebase, **CODEC** refers to the underlying open-source engine /
Expand Down
20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<p align="center">
<a href="https://github.com/AVADSA25/codec/actions/workflows/ci.yml"><img src="https://github.com/AVADSA25/codec/actions/workflows/ci.yml/badge.svg" alt="CI"/></a>
<a href="https://github.com/AVADSA25/codec/discussions"><img src="https://img.shields.io/badge/community-Discussions-7057ff?style=flat-square&logo=github" alt="GitHub Discussions"/></a>
<a href="FEATURES.md"><img src="https://img.shields.io/badge/features-400+-blue?style=flat-square" alt="400+ Features"/></a>
<a href="FEATURES.md"><img src="https://img.shields.io/badge/features-370-blue?style=flat-square" alt="370 Features"/></a>
<img src="https://img.shields.io/badge/skills-76-orange?style=flat-square" alt="76 Skills"/>
<img src="https://img.shields.io/badge/tests-2000+-green?style=flat-square" alt="2,000+ Tests"/>
<img src="https://img.shields.io/badge/lines-52K+-purple?style=flat-square" alt="52K+ Lines"/>
Expand Down Expand Up @@ -46,7 +46,7 @@ It listens, sees the screen, speaks back, controls apps, writes code, drafts mes

No cloud dependency. No data leaving the machine unless you choose. No subscription on the open-source build. MIT licensed.

> **Sovereign AI Workstation** is the product brand. **CODEC** (v3.2) is the open-source engine that powers it — the codename you'll see in code paths, skill registries, the `codec_*` PM2 services, and the `~/.codec/` config directory. *Sovereign AI Workstation* is what you ship; *CODEC* is what you ship with. Same way iPhone runs on Darwin, or Tesla Model S runs on Roadster components — one is the product, the other is the engine.
> **Sovereign AI Workstation** is the product brand. **CODEC** (v3.5) is the open-source engine that powers it — the codename you'll see in code paths, skill registries, the `codec_*` PM2 services, and the `~/.codec/` config directory. *Sovereign AI Workstation* is what you ship; *CODEC* is what you ship with. Same way iPhone runs on Darwin, or Tesla Model S runs on Roadster components — one is the product, the other is the engine.

---

Expand All @@ -62,11 +62,11 @@ And all of it runs on *your* machine: no data leaves unless you explicitly route

---

## 9 Products. One System.
## 7 Products. One System.

| # | Product | What It Does |
|:-:|---|---|
| 1 | **CODEC Core** | Voice command layer + vision mouse control — 75 skills, screen clicks by voice |
| 1 | **CODEC Core** | Voice command layer + vision mouse control — 89 skills, screen clicks by voice |
| 2 | **CODEC Dictate** | Hold, speak, paste — hands-free F5 live typing at cursor, draft refinement, floating overlays |
| 3 | **CODEC Instant** | Right-click → 8 AI services system-wide — proofread, translate, reply, explain |
| 4 | **CODEC Chat** | 250K-context conversational AI + 12 autonomous agent crews |
Expand All @@ -80,7 +80,7 @@ And all of it runs on *your* machine: no data leaves unless you explicitly route

Always-on voice assistant. Say *"Hey CODEC"* or press F13 to activate. F18 for voice commands. F16 for text input.

75 skills fire instantly: Google Calendar, Gmail, Drive, Docs, Sheets, Tasks, Keep, Chrome automation, web search, Hue lights, timers, Spotify, clipboard, terminal commands, PM2 control, and more. Most skills bypass the LLM entirely — direct action, zero latency. Skills are matched by trigger specificity — longer, more specific triggers always win over generic ones.
89 skills fire instantly: Google Calendar, Gmail, Drive, Docs, Sheets, Tasks, Keep, Chrome automation, web search, Hue lights, timers, Spotify, clipboard, terminal commands, PM2 control, and more. Most skills bypass the LLM entirely — direct action, zero latency. Skills are matched by trigger specificity — longer, more specific triggers always win over generic ones.

**Vision Mouse Control — See & Click**

Expand Down Expand Up @@ -114,7 +114,7 @@ Full conversational AI. Long context. File uploads (drag-and-drop). Image analys

Voice input via continuous microphone with stop button. Streaming responses with typing and thinking indicators.

**Four modes** in the chat composer: **Chat / Think / Agents / Project**. The first three live here — Project mode is documented as its own product in [§9 below](#9-codec-project--drop-a-project-autonomy) since it runs autonomously on a dedicated daemon for hours rather than turn-by-turn in the chat thread.
**Four modes** in the chat composer: **Chat / Think / Agents / Project**. The first three live here — Project mode is documented within [§4 CODEC Chat](#4-codec-chat--250k-context--12-agent-crews--drop-a-project-autonomy) since it runs autonomously on a dedicated daemon for hours rather than turn-by-turn in the chat thread.

Plus 12 autonomous agent crews — not single prompts, full multi-step workflows. Say *"research the latest AI agent frameworks and write a report."* Minutes later there's a formatted Google Doc in Drive with sources, images, and recommendations.

Expand Down Expand Up @@ -158,7 +158,7 @@ Full transcript saved to memory. Every conversation becomes searchable context f
Private dashboard accessible from any device, anywhere. Cloudflare Tunnel or Tailscale VPN — no port forwarding, no third-party relay. 135+ API endpoints. Send commands, view the screen, launch voice calls, manage agents — all from a browser. Installable as a PWA on mobile and desktop.

**Cortex — System Nerve Center**
Visual command center showing all 9 CODEC products in an interactive grid. Neural network SVG map, real-time activity feed, searchable skills panel, and detailed event log viewer. The single-pane-of-glass view of the entire system.
Visual command center showing all 7 CODEC products in an interactive grid. Neural network SVG map, real-time activity feed, searchable skills panel, and detailed event log viewer. The single-pane-of-glass view of the entire system.

**Audit — Full Event Trail**
Every action CODEC takes is logged across 16 categories: command, skill, llm, auth, error, scheduled, voice, vision, tts, stt, system, security, hotkey, screenshot, config, draft. Filterable by category pills, searchable, with colored timeline dots and expandable event details. JSON-line storage with 50MB rotation. Default 24h time range with 1h/6h/24h/7d quick filters.
Expand Down Expand Up @@ -282,7 +282,7 @@ Three smart agents ship built-in: Daily Briefing, Restaurant Decider (location-a
</p>
<p align="center">
<img src="docs/screenshots/terminal.png" alt="Terminal" width="400"/><br/>
<em>75 skills loaded at startup</em>
<em>89 skills loaded at startup</em>
</p>
<p align="center">
<img src="docs/screenshots/cortex.png" alt="Cortex Neural Map" width="720"/><br/>
Expand Down Expand Up @@ -703,7 +703,7 @@ setup_codec.py — Setup wizard (9 steps)
ecosystem.config.js — PM2 process management (16 services)
```

**CODEC Pilot** is parked (v3.5) and no longer part of the product. Its code and PM2 service live on in the `codec-pilot` repo.
**CODEC Pilot** is parked (v3.5) and no longer part of the product. Its code, skill, route proxy, and PM2 service were unhooked from this repo in v3.5 and preserved in git history (see [docs/PILOT-PARKED.md](docs/PILOT-PARKED.md)).

---

Expand Down Expand Up @@ -743,7 +743,7 @@ python3 setup_codec.py

> **Project mode** (drop-a-project autonomy) is live and ships as part of **CODEC Overview**.
>
> **CODEC Pilot** (browser automation) is **parked as of v3.5.** Google blocks account sign-in from any CDP-controlled browser, and cookie walls plus bot challenges make a large share of real sites unusable — it needs a considerably deeper build before it can be trusted as a product. The code is preserved in the `codec-pilot` repo rather than deleted.
> **CODEC Pilot** (browser automation) is **parked as of v3.5.** Google blocks account sign-in from any CDP-controlled browser, and cookie walls plus bot challenges make a large share of real sites unusable — it needs a considerably deeper build before it can be trusted as a product. The code is preserved in git history (see [docs/PILOT-PARKED.md](docs/PILOT-PARKED.md)) rather than deleted.

**Phase 3.5 (in progress)** — UX + polish on top of the autonomous-agent substrate:

Expand Down
Loading