diff --git a/.env.example b/.env.example index 44004b88..b47e22e1 100644 --- a/.env.example +++ b/.env.example @@ -1,59 +1,87 @@ -# Шаблон переменных окружения для deploy.sh, verify.sh и локальных CLI-скриптов. -# Рекомендуемый путь: +# GhostRoute environment example +# +# Recommended operator-local path: # mkdir -p secrets # cp .env.example secrets/router.env # -# Директория secrets/ добавлена в .gitignore и никогда не попадёт в git. -# Для обратной совместимости скрипты всё ещё читают .env, если secrets/router.env нет. - -# Профиль SSH-доступа к роутеру. -# По умолчанию auto выбирает: -# - home LAN/Wi-Fi: ROUTER_LAN:ROUTER_LAN_PORT, только если маршрут к LAN-адресу -# идет не через VPN-интерфейс; -# - off-LAN/VPN: ROUTER:ROUTER_WAN_PORT, то есть выделенный WAN/remote SSH endpoint. +# `secrets/` and real `.env*` files are gitignored. Keep real router/VPS hosts, +# ports, users, keys and provider details out of git. Use placeholders in public +# docs and commits. + +# ----------------------------------------------------------------------------- +# Router SSH access profile +# ----------------------------------------------------------------------------- + +# Default: auto chooses LAN when the control machine is really on the home LAN, +# otherwise it keeps the dedicated remote/WAN SSH endpoint profile. # ROUTER_ACCESS_MODE=auto -# LAN-адрес роутера для домашней сети. Используется только когда control machine -# реально находится в home LAN/Wi-Fi; если маршрут идет через utun/tun/wg, auto -# считает это VPN/off-LAN и выбирает WAN-профиль ниже. +# LAN-only router access. Use only when the control machine is actually on the +# home LAN/Wi-Fi; if the route goes through utun/tun/wg/Tailscale, auto mode +# treats the machine as off-LAN. # ROUTER_LAN= # ROUTER_LAN_PORT=22 -# WAN/remote SSH endpoint роутера. Для удаленной работы храните здесь DDNS или -# текущий WAN fallback host и отдельный внешний порт. +# Remote/off-LAN router SSH endpoint. Store only operator-local values here. # ROUTER= # ROUTER_WAN_PORT= # ROUTER_PORT= -# auto выполняет короткий SSH-preflight. Если WAN endpoint принимает TCP, но не -# отдает SSH banner, helper пробует direct LAN/Wi-Fi fallback перед ошибкой. +# SSH preflight checks the banner before selecting a fallback path. # ROUTER_ACCESS_PREFLIGHT=ssh -# SSH-пользователь на роутере (обычно admin). +# SSH identity. Prefer a key under gitignored `secrets/router-remote-ssh/` or a +# private workstation path; never commit the key. # ROUTER_USER=admin - -# Путь к SSH-ключу для подключения к роутеру. # SSH_IDENTITY_FILE=~/.ssh/id_rsa - -# Таймаут подключения в секундах. # CONNECT_TIMEOUT=5 -# Логирование DNS-запросов для автоматического обнаружения доменов. -# Лог пишется в /opt/var/log/dnsmasq.log (требует Entware на USB). -# Скрипт domain-auto-add.sh запускается каждые 4 часа через cron, -# парсит DNS-лог и добавляет новые домены в VPN автоматически. -# -# Просмотр результатов: -# ./scripts/domain-report # сводка + последний запуск -# ./scripts/domain-report --log # подробный лог активности -# ./scripts/domain-report --all # все авто-добавленные домены -# ./scripts/domain-report --reset # удалить все авто-добавленные домены -# -# Отключить логирование: ENABLE_DNSMASQ_LOGGING=0 +# Optional override when a script should read another gitignored env file. +# GHOSTROUTE_ROUTER_ENV_FILE=secrets/router.env + +# ----------------------------------------------------------------------------- +# DNS / catalog discovery +# ----------------------------------------------------------------------------- + +# Enables router-side dnsmasq query logging for domain discovery. Logs are stored +# on router storage, not in git. See the module-native command paths below. # ENABLE_DNSMASQ_LOGGING=1 -# URL списка заблокированных доменов для фильтрации auto-discovery. -# По умолчанию: community.antifilter.download (кураторский список ~500 доменов). -# Скрипт update-blocked-list.sh скачивает его ежедневно через VPN. -# Если список не скачан, domain-auto-add.sh работает в fallback-режиме (добавляет всё). +# Public blocked-domain list used as an admission filter for auto-discovery. +# If the list cannot be fetched, the discovery script documents its fallback +# behavior in module reports. # BLOCKED_LIST_URL=https://community.antifilter.download/list/domains.lst + +# Module-native inspection commands: +# ./modules/dns-catalog-intelligence/bin/domain-report +# ./modules/dns-catalog-intelligence/bin/domain-report --log +# ./modules/dns-catalog-intelligence/bin/domain-report --all +# ./modules/dns-catalog-intelligence/bin/domain-report --reset +# ./modules/dns-catalog-intelligence/bin/catalog-review-report + +# ----------------------------------------------------------------------------- +# GhostRoute Console local development +# ----------------------------------------------------------------------------- + +# Local Console data directory. The default module path is gitignored. +# GHOSTROUTE_CONSOLE_DATA_DIR=modules/ghostroute-console/data + +# Synthetic GUI test data and quota knobs. Safe placeholders only. +# GHOSTROUTE_CONSOLE_VPS_QUOTA_GB=1024 +# GHOSTROUTE_CONSOLE_LTE_QUOTA_GB=50 +# GHOSTROUTE_CONSOLE_BILLING_RESET_DAY=1 + +# Live UI refresh interval for server-sent events. Long default by design; the +# Console reads prepared snapshots rather than becoming a real-time control loop. +# GHOSTROUTE_LIVE_UI_REFRESH_MS=600000 + +# Diagnostic rollback switch for Console prepared windows. Production should use +# prepared windows; set to 0 only while debugging a read-model regression. +# GHOSTROUTE_CONSOLE_USE_PREPARED_WINDOWS=1 + +# Console public deployment values belong in Vault or gitignored operator files, +# never in tracked docs: +# +# +# +# diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 00000000..443e5753 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,53 @@ +## Summary + +- +- + +## Scope + +Touched areas: + +- + +Runtime impact: + +- [ ] Docs-only / no runtime behavior changed +- [ ] Console read-only UI/API/read-model change +- [ ] Router data plane / sing-box / dnsmasq / iptables +- [ ] VPS edge / Caddy / Xray / Reality +- [ ] Ansible / deploy / generated artifacts +- [ ] Traffic Observatory / machine contract +- [ ] Secrets / Vault / local operator files + +## Tests + +Run: + +```bash + +``` + +Not run, and why: + +- + +## Risk and rollback + +Blast radius: + +- + +Rollback path: + +- + +Safety tag needed before deploy? + +- [ ] No +- [ ] Yes: `pre--` + +## Secrets and public-safety check + +- [ ] No real endpoints, listener ports, credentials, UUIDs, Reality keys, short IDs, admin paths, QR payloads, VLESS URIs, provider details or personal device identifiers are included. +- [ ] Generated artifacts under `ansible/out/`, local reports and private operator notes remain gitignored. +- [ ] Public examples use placeholders only. diff --git a/.gitignore b/.gitignore index 6d400757..7b273444 100644 --- a/.gitignore +++ b/.gitignore @@ -1,12 +1,16 @@ -# Персональные переменные окружения (IP роутера, SSH-ключ и т.п.) +# Local environment and operator secrets .env +.env.* +!.env.example +*.local.env +/secrets/ -# Конфиги WireGuard с приватными ключами +# WireGuard / VPN configs with private keys *.conf.wg wireguard/ wg*.conf -# Любые файлы с приватными/чувствительными данными +# Keys, certificates and credential material *.key *.pem *.p12 @@ -16,23 +20,24 @@ id_ed25519 id_ecdsa *.pub -# Claude Code +# AI assistant local state .claude/ -# VPN domain journal (local-only, не для репозитория) +# Local reports, journals and private operator notes +reports/ docs/vpn-domain-journal.md docs/router-health-latest.md docs/catalog-review-latest.md docs/private/ configs/private/ -reports/ + +# Keep tracked Next.js route folders named reports/ inside the Console app !modules/ghostroute-console/app/src/app/reports/ !modules/ghostroute-console/app/src/app/reports/** !modules/ghostroute-console/app/src/app/api/reports/ !modules/ghostroute-console/app/src/app/api/reports/** -# Локальные secrets / overrides с персональными IP, ключами и bypass-правилами -/secrets/ +# Ansible Vault and generated client artifacts /ansible/secrets/*.yml !/ansible/secrets/*.yml.example /ansible/secrets/*.backup-* @@ -61,19 +66,30 @@ reports/ /ansible/out/clients-home-backup-*/ ansible/*.retry -# macOS -.DS_Store -__pycache__/ - -# Node/Next module-local artifacts +# Node / Next.js / Playwright artifacts node_modules/ .next/ +out/ +coverage/ +npm-debug.log* +yarn-debug.log* +yarn-error.log* +pnpm-debug.log* modules/ghostroute-console/app/node_modules/ modules/ghostroute-console/app/.next/ +modules/ghostroute-console/app/out/ +modules/ghostroute-console/app/coverage/ modules/ghostroute-console/app/test-results/ +modules/ghostroute-console/app/playwright-report/ modules/ghostroute-console/data/ -# Редакторы +# Python / shell local artifacts +__pycache__/ +*.py[cod] +.pytest_cache/ + +# OS and editor noise +.DS_Store *.swp *.swo *~ diff --git a/.nvmrc b/.nvmrc new file mode 100644 index 00000000..2bd5a0a9 --- /dev/null +++ b/.nvmrc @@ -0,0 +1 @@ +22 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 02babc65..035d1d03 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,38 +1,65 @@ # Contributing to GhostRoute GhostRoute is a single-operator routing platform. This file documents how the -operator (and any AI agents collaborating on the repo) should make changes — -locally, in PRs, and at the doc / commit level. +operator and AI agents should make changes locally, in PRs and at the docs / +commit level. + +The default posture is conservative: understand the owning module, keep the diff +small, prove the change with the narrowest safe checks and never publish real +runtime state. ## Project context Read these first; the rest of this file assumes the rules they document. -- [`AGENTS.md`](AGENTS.md) — primary contract for how to work in this repo - (Karpathy-style workflow, safety rules, secret handling, architecture - invariants, where things live). +- [`AGENTS.md`](AGENTS.md) — primary contract for working in this repo: + workflow, safety rules, secret handling, architecture invariants and directory + ownership. - [`CLAUDE.md`](CLAUDE.md) — Claude-specific working notes; imports `AGENTS.md`. Both files are equivalent for non-Claude agents. -- [`SECURITY.md`](SECURITY.md) — protected assets, threat model, secret policy, - recovery boundaries. -- [`docs/operational-modules.md`](docs/operational-modules.md) — module map. +- [`SECURITY.md`](SECURITY.md) — protected assets, threat model, secret policy + and recovery boundaries. +- [`docs/product-requirements.md`](docs/product-requirements.md) — product brief: + users, goals, non-goals and quality attributes. +- [`docs/testing.md`](docs/testing.md) — test layers, CI contract and live-check + boundaries. +- [`docs/operational-modules.md`](docs/operational-modules.md) — canonical module + map and ownership table. ## Development setup This repo is operated from a control machine (macOS/Linux) and targets an ASUS -Asuswrt-Merlin router and a small VPS. Local-only checks do not require any -router or VPS access. +Asuswrt-Merlin router plus a small VPS. Local-only checks do not require router, +VPS or Vault access. Prerequisites: -- Node.js 22 (matches `.github/workflows/ci.yml`). If you use `nvm`, run - `nvm use 22` (a `.nvmrc` is on the backlog). +- Node.js 22. Run `nvm use` if you use nvm; `.nvmrc` pins the local version to + the same major version used in GitHub Actions. - Python 3.x for Ansible-related tooling. -- POSIX shell (`bash` for control-machine scripts, BusyBox `ash` compatibility - required for anything running on the router). -- `ripgrep` (`rg`) for the doc-syntax scan. +- POSIX shell. Control-machine scripts may use Bash; router-side scripts must + remain compatible with BusyBox `ash` unless a file clearly opts into Bash. +- `ripgrep` (`rg`) for static checks. +- Optional: `pre-commit` for local hooks. + +Recommended local setup: + +```bash +nvm use +npm --prefix modules/ghostroute-console/app ci +cp .env.example secrets/router.env # then fill placeholders locally only +``` + +`secrets/router.env`, `.env.local`, generated client artifacts and Console data +are gitignored. Do not commit real endpoints, ports, users, keys, UUIDs, VLESS +URIs, QR payloads, provider details or device names. -Local-only verification: +## Verification + +Use the narrowest check that proves the change. See [`docs/testing.md`](docs/testing.md) +for the full matrix. + +Local-only repo checks: ```bash ./modules/secrets-management/bin/secret-scan @@ -41,20 +68,49 @@ Local-only verification: bash -n verify.sh tests/run-all.sh tests/run-fast.sh ``` +Console-specific checks: + +```bash +npm --prefix modules/ghostroute-console/app test +npm --prefix modules/ghostroute-console/app run build +npm --prefix modules/ghostroute-console/app run test:e2e:gui +npm --prefix modules/ghostroute-console/app run test:perf +``` + +Ansible syntax checks for touched surfaces: + +```bash +cd ansible +ansible-playbook --syntax-check playbooks/20-stealth-router.yml +ansible-playbook --syntax-check playbooks/21-channel-b-router.yml +ansible-playbook --syntax-check playbooks/22-channel-c-router.yml +ansible-playbook --syntax-check playbooks/24-channel-d-router.yml +ansible-playbook --syntax-check playbooks/99-verify.yml +``` + +Live checks (`./verify.sh --verbose`, `99-verify.yml`, health reports, traffic +reports and deploy gates) require real operator infrastructure. They are +read-only unless the command documentation says otherwise, but they are not +public CI requirements. + ## Documentation conventions - `README.md` is the developer-facing ground truth. `README-ru.md` is the localized operator-facing summary; if EN and RU drift, EN wins. -- Use [language tags](docs/repo-review-2026-05-10.md) for non-EN-primary docs: - `[RU primary]` at the top of any doc that is operator-Russian, plus a - cross-link to its EN entry point if one exists. -- Internal links use repo-root paths (e.g. `[docs/architecture.md](docs/architecture.md)`) - so they resolve identically on GitHub and locally. +- Use language tags for non-EN-primary docs: `[RU primary]` at the top of any + operator-Russian doc, plus a cross-link to its English entry point if one + exists. +- Internal links use repo-root paths from root docs and depth-correct relative + paths from nested module docs so they render both on GitHub and in local + previews. - Module-owned deep dives live in `modules//docs/`. Cross-cutting docs live in `docs/`. -- Sensitive values must use placeholders only (``, - ``, `example.invalid`). Never paste real endpoints, ports, - UUIDs, Reality keys, short IDs, admin paths, QR payloads, or VLESS URIs. +- Planning documents must say when they are planning or draft material. Do not + imply future work is already implemented. +- Sensitive values must use placeholders only: ``, + ``, ``, ``, `example.invalid`, + or documentation-only IPs. Never paste real endpoints, ports, UUIDs, Reality + keys, short IDs, admin paths, QR payloads or VLESS URIs. ## Commit conventions @@ -63,78 +119,83 @@ bash -n verify.sh tests/run-all.sh tests/run-fast.sh client registry`. Existing history uses plain imperative English without prefixes; do not rewrite it. - **Allowed types**: `feat`, `fix`, `docs`, `chore`, `refactor`, `test`, - `perf`, `build`, `ci`. Use a `scope` for module-bounded changes - (`console`, `routing-core`, `traffic-observatory`, `health-monitor`, - `dns-catalog`, `recovery`, `secrets`, `client-profiles`, `ansible`, `docs`). + `perf`, `build`, `ci`. Use a `scope` for module-bounded changes such as + `console`, `routing-core`, `traffic-observatory`, `health-monitor`, + `dns-catalog`, `recovery`, `secrets`, `client-profiles`, `ansible` or `docs`. - **Subject line**: imperative, ≤72 chars, no trailing period. -- **Body** (optional): explain *why*, not *what*. Reference ADRs, runbooks or - AGENTS.md sections for non-obvious decisions. +- **Body**: explain *why*, not just *what*. Reference ADRs, runbooks or + `AGENTS.md` sections for non-obvious decisions. - **Co-authorship**: when an AI agent meaningfully contributed, include a - `Co-Authored-By: ` trailer. Otherwise omit it. Do - not retroactively add or remove these. -- **Safety tags**: before any potentially breaking migration (router data - plane, channel ownership, secrets layout), create a tag of the form + `Co-Authored-By: ` trailer. Otherwise omit it. Do not + retroactively add or remove these. +- **Safety tags**: before any potentially breaking migration (router data plane, + channel ownership, secrets layout), create a tag of the form `pre--` so the prior state is recoverable. ## Pull requests -- Title: same convention as commit subject. -- Body should answer: - - *Summary*: what changes and why (1–3 bullets). - - *Scope*: which modules / files are affected; explicitly call out anything - that touches the router data plane, sing-box, dnsmasq, iptables, Reality, - VPS, or `traffic-observatory`. - - *Test plan*: which checks were run locally, and which deliberately not - (live deploy, broad Ansible runs). - - *Risk*: rollback plan, blast radius, irreversible steps. -- Keep PRs small and module-scoped. If a refactor crosses modules, split it. -- All PRs should pass the GitHub Actions `CI` workflow (`tests/run-fast.sh` + - `tests/run-smoke.sh`). Performance suite runs only on - `workflow_dispatch`. +Keep PRs small and module-scoped. A good PR body answers: + +- **Summary**: what changed and why. +- **Scope**: modules/files affected; explicitly call out anything that touches + router data plane, sing-box, dnsmasq, iptables, Reality, VPS or + `traffic-observatory` contracts. +- **Test plan**: checks run locally, checks skipped intentionally and why. +- **Risk**: rollback plan, blast radius, irreversible steps and whether a safety + tag is needed. +- **Secrets**: confirm no real endpoints, credentials, QR payloads, UUIDs or + generated artifacts are included. + +All PRs should pass the GitHub Actions `CI` workflow (`tests/run-fast.sh` plus +`tests/run-smoke.sh`). The heavier Console performance suite runs only on manual +`workflow_dispatch` unless the PR specifically needs that proof. ## Pre-commit hooks `.pre-commit-config.yaml` runs: - `ghostroute-secret-scan` — `./modules/secrets-management/bin/secret-scan` - catches real endpoints, UUIDs, keys, ports, public IPs and admin paths - before they reach a commit. + catches real endpoints, UUIDs, keys, ports, public IPs and admin paths before + they reach a commit. - `ghostroute-shell-syntax` — `./tests/check-shell-syntax.sh` validates shell - scripts for both BusyBox `ash` (router) and `bash` (control machine) - compatibility. + scripts for Bash or BusyBox `sh` parsing based on each file's shebang. + +Install once: + +```bash +pre-commit install +pre-commit run --all-files +``` -Install once: `pre-commit install`. Do not skip hooks (`--no-verify`) unless -the operator explicitly authorizes it for an emergency commit. +Do not skip hooks (`--no-verify`) unless the operator explicitly authorizes it +for an emergency commit. ## Safety boundaries -These are restated in `AGENTS.md` and are enforced for all contributors, -human or AI: +These rules are restated in `AGENTS.md` and apply to human and AI contributors: -- Never run `git commit`, `git push`, `./deploy.sh`, mutating Ansible - playbooks (`00-*`, `10-*`, `11-*`, `20-*`, `21-*`, `22-*`, - `30-generate-client-profiles.yml`), or any router/VPS-mutating SSH/SCP/rsync +- Never run `git commit`, `git push`, `./deploy.sh`, mutating Ansible playbooks + (`00-*`, `10-*`, `11-*`, `20-*`, `21-*`, `22-*`, + `30-generate-client-profiles.yml`) or any router/VPS-mutating SSH/SCP/rsync command without explicit operator permission. - Never reintroduce `VPN_DOMAINS`, `RC_VPN_ROUTE`, `0x1000`, active `wgs1` or active `wgc1` as production state. - Never deploy if relevant local checks have not passed; if checks were intentionally skipped, say so and ask before deploying. -- Never disable hooks (`--no-verify`, `--no-gpg-sign`) without explicit - operator authorization. +- Never disable hooks (`--no-verify`, `--no-gpg-sign`) without explicit operator + authorization. ## Where to put new findings - A specific bug or regression: open a focused PR or issue. - A long-term improvement: add to [`docs/future-improvements-backlog.md`](docs/future-improvements-backlog.md) - with an appropriate phase marker (`✓ done` / `◐ in progress` / - `○ deferred`). -- A repo-wide review: write a snapshot under - `docs/repo-review-YYYY-MM-DD.md` (latest: - [`docs/repo-review-2026-05-10.md`](docs/repo-review-2026-05-10.md)). + with an appropriate phase marker (`✓ done` / `◐ in progress` / `○ deferred`). +- A repo-wide review: write a snapshot under `docs/repo-review-YYYY-MM-DD.md` + and link it from `docs/README.md`. ## Reporting security issues -This is a personal operations repo. For sensitive findings, do not open a -public issue. Share only sanitized evidence with placeholders. See -[`SECURITY.md`](SECURITY.md) for the full policy. +This is a personal operations repo. For sensitive findings, do not open a public +issue. Share only sanitized evidence with placeholders. See [`SECURITY.md`](SECURITY.md) +for the full policy. diff --git a/docs/README.md b/docs/README.md index 9f10abe5..96b25088 100644 --- a/docs/README.md +++ b/docs/README.md @@ -15,14 +15,19 @@ local runbooks or historical planning notes. ## Start Here +- [product-requirements.md](product-requirements.md) - product brief: users, goals, non-goals, requirements and success metrics. - [operational-modules.md](operational-modules.md) - canonical module map and ownership table. - [architecture.md](architecture.md) - high-level GhostRoute architecture. - [router-runtime-map.md](router-runtime-map.md) - sanitized map of what is installed on the ASUS Merlin router, with diagrams and runtime guardrails. - [praefectus-ai/docs/vps-runtime-map.md](https://github.com/eiler2005/praefectus-ai/blob/main/docs/vps-runtime-map.md) - companion VPS-side runtime map for Channel M, Console and routing/app surfaces. -- [channels.md](channels.md) - compact handoff view of Channel A, B and C. +- [channels.md](channels.md) - compact handoff view of Channel A, B, C and D. +- [routing-policy-principles.md](routing-policy-principles.md) - compact routing decision contract across endpoints, channels, router and egress. - [dns-policy.md](dns-policy.md) - DNS leak and fingerprint policy for Channel A/B/C proofs. - [channel-c.md](channel-c.md) - detailed Channel C C1 native Naive and Shadowrocket compatibility status. - [SECURITY.md](../SECURITY.md) - threat model, protected assets, non-goals and security workflow. +- [testing.md](testing.md) - test layers, CI contract, local verification and live-check boundaries. +- [deployment-and-rollback.md](deployment-and-rollback.md) - pre-deploy checklist, rollback triggers and recovery paths. +- [operational-slos.md](operational-slos.md) - availability, correctness, privacy and recovery targets. - [ansible/README.md](../ansible/README.md) - deployment, Vault, profile generation and verification control plane. - [getting-started.md](getting-started.md) - first deploy and local setup workflow. - [troubleshooting.md](troubleshooting.md) - cross-module incident diagnostics. @@ -54,3 +59,8 @@ today." - [traffic-facts-v3-and-pyramid-plan.md](traffic-facts-v3-and-pyramid-plan.md) - traffic-facts v3 machine-contract and Console pyramid refactor plan. - [traffic-intelligence-layer-plan.md](traffic-intelligence-layer-plan.md) - traffic intelligence layer design on top of the v3 pipeline. - [managed-domain-app-family-draft.md](managed-domain-app-family-draft.md) - draft app-family catalog over the active managed domains. + +## Review Snapshots + +- [repo-review-2026-06-16.md](repo-review-2026-06-16.md) - public-repo polish review after the latest docs/schema/API commits. +- [repo-review-2026-05-10.md](repo-review-2026-05-10.md) - previous repository quality audit and prioritized fixlist. diff --git a/docs/product-requirements.md b/docs/product-requirements.md new file mode 100644 index 00000000..a55ec401 --- /dev/null +++ b/docs/product-requirements.md @@ -0,0 +1,141 @@ +# Product Requirements Brief + +GhostRoute is a personal, single-operator routing platform for an ASUS Merlin +edge router and a VPS-managed Reality egress. This document frames the existing +system as a product: who it serves, what it must guarantee, what it deliberately +does not try to do, and which engineering constraints matter for future changes. + +It is intentionally a brief, not a marketing roadmap. Runtime behavior is still +defined by the architecture docs, ADRs, module READMEs and tests. + +## Problem + +A home network can contain many devices that should not each need a VPN app, +manual proxy profile, or per-device debugging. At the same time, selected mobile +clients need a home-first entry point and selected destinations need managed +Reality egress through an active VPS backend. + +The hard parts are not only packet routing. The operator also needs safe deploys, +clear recovery boundaries, read-only observability, explicit secret handling and +a public repository that explains the architecture without leaking production +state. + +## Users + +| User | Need | Constraint | +|---|---|---| +| Primary operator | Run and debug the routing platform from a control machine. | Must not leak real endpoints, ports, keys, UUIDs, QR payloads or provider details. | +| Home LAN / Wi-Fi devices | Use ordinary apps without installing VPN clients. | Routing policy must live on the router and avoid broad accidental capture. | +| Selected remote clients | Enter through the home endpoint first, then follow managed split policy. | Generated client artifacts are credentials and stay outside git. | +| Reviewer / hiring manager | Understand the architecture, safety model and engineering maturity quickly. | Public docs must distinguish implemented behavior from plans and local operator notes. | + +## Goals + +1. **Home-first managed routing.** Keep the first hop for selected clients at the + home endpoint, while selected managed destinations use the active Reality + egress and non-managed traffic can remain direct. +2. **Module-native operations.** Keep routing, health, traffic, catalog, + client-profile, secrets, Console and recovery ownership separated. +3. **Read-only observability by default.** Reports, health snapshots and Console + read models explain runtime state without becoming hidden deploy mechanisms. +4. **Explicit safety boundaries.** Preserve no-auto-failover channel semantics, + manual cold fallback, deploy gates and recoverable rollback paths. +5. **Public-safe documentation.** Explain enough architecture to be auditable + while using placeholders for every sensitive deployment-specific value. + +## Non-goals + +- Public VPN service, multi-tenant access, billing or user management. +- Automatic B/C/D failover into Channel A or automatic WireGuard recovery. +- Public disclosure of real infrastructure, provider mapping, listener values, + device identities, QR payloads, UUIDs or credentials. +- High-availability guarantees beyond the single-operator, home-WAN and VPS + assumptions documented in `docs/operational-slos.md`. +- Console-driven router/VPS runtime mutation. The Console is a read-only evidence + surface except for documented, audited operator-state overlays. + +## Current capabilities + +| Capability | Current status | Primary references | +|---|---|---| +| Channel A router data plane | Production | `README.md`, `docs/architecture.md`, `modules/routing-core/docs/` | +| Channel A selected full-VPS override | Implemented for selected LAN/Wi-Fi devices and Home Reality profiles | `docs/channel-a-selected-full-vps.md`, ADR-0010 | +| Channel B home-first selected-client lane | Production for selected profiles | `docs/channels.md`, Ansible `21-*` playbook docs | +| Channel C compatibility lane | C1-Shadowrocket live-proven; native sing-box Naive blocked by client support | `docs/channel-c.md`, ADR-0008 | +| Channel D NaiveProxy lab | Experimental, disabled by default | `docs/channel-d.md`, `playbooks/24-channel-d-router.yml` | +| Channel M MAX service egress | Dedicated service lane, not client failover | `docs/channel-m-environment.md` | +| Health / traffic / catalog observability | Implemented as read-only modules and reports | `docs/operational-modules.md`, module docs | +| GhostRoute Console | Read-only prepared-data workbench with desktop and mobile routes | `modules/ghostroute-console/README.md` | +| Secrets management | Vault and gitignored generated artifacts with repo-specific scanning | `SECURITY.md`, `modules/secrets-management/docs/secrets-management.md` | + +## Functional requirements + +### Routing policy + +- Preserve the managed split: `STEALTH_DOMAINS` and `VPN_STATIC_NETS` are the + public names for the managed domain and static CIDR catalogs. +- Keep Channel A/B/C/D ownership isolated. A selected-client lane must not + silently mutate Channel A REDIRECT, router DNS, TUN state or recovery hooks. +- Keep Channel M separate from managed client routing. It exists for MAX service + egress through the home WAN and never uses `reality-out`. +- Keep legacy WireGuard as explicit cold fallback only. + +### Operations + +- Every mutating path must have a documented pre-check and post-check. +- Read-only checks must be runnable without generated client profiles whenever + possible. +- Live checks that require router/VPS access must remain explicit operator steps, + not hidden CI requirements. +- Rollback instructions must name the smallest affected component: router data + plane, selected-client channel, VPS edge, DNS/catalog, Console or cold fallback. + +### Observability + +- Machine contracts should prefer JSON facts and prepared read models over ad-hoc + parsing of human reports. +- Console request paths should read bounded prepared data, not scan large raw + snapshots on every page render. +- Health, traffic and catalog reports must remain safe for LLM handoff only after + redaction and placeholder rules are satisfied. + +### Security and privacy + +- Real production values live in Ansible Vault or gitignored local files only. +- Public docs and examples must use placeholders such as ``, + ``, ``, `example.invalid` or RFC 5737-style + documentation IPs when an example must look concrete. +- Generated client profiles, QR images, VLESS URIs, keys and local reports are + credentials or private evidence and must not be committed. + +## Quality attributes + +| Attribute | Requirement | Evidence | +|---|---|---| +| Maintainability | Changes stay module-scoped and preserve documented ownership. | `AGENTS.md`, `CONTRIBUTING.md`, module READMEs. | +| Testability | Repo-only checks cover syntax, static routing invariants, fixtures, Console smoke and secret hygiene. | `docs/testing.md`, `.github/workflows/ci.yml`. | +| Recoverability | Manual rollback paths and cold fallback are explicit. | `docs/deployment-and-rollback.md`, `SECURITY.md`. | +| Auditability | Architecture decisions are recorded as ADRs; planning docs are marked as non-runtime direction. | `docs/adr/`, `docs/README.md`. | +| Public safety | Docs describe roles and placeholders, not real endpoints or provider mapping. | `SECURITY.md`, secret scan. | + +## Success metrics + +These are repository and operator-quality metrics rather than SaaS business +metrics: + +- `./tests/run-fast.sh` passes for repo-only changes. +- GitHub Actions `CI` passes on pull requests. +- `./modules/secrets-management/bin/secret-scan` reports clean before push. +- `./verify.sh` and `ansible/playbooks/99-verify.yml` pass when live targets are + intentionally checked by the operator. +- Console performance and post-deploy checks stay within the targets documented + in module runbooks. +- Public documentation can answer: what the system does, what it does not do, + how it is tested, how it fails, how it recovers and where secrets live. + +## Future direction + +The backlog lives in `docs/future-improvements-backlog.md`. Planning documents +under `docs/` are design direction unless they explicitly state that a feature is +implemented. New work should promote a backlog item into an ADR or module doc +only when the behavior is concrete, tested and safe to describe publicly. diff --git a/docs/repo-review-2026-06-16.md b/docs/repo-review-2026-06-16.md new file mode 100644 index 00000000..9a8539d8 --- /dev/null +++ b/docs/repo-review-2026-06-16.md @@ -0,0 +1,190 @@ +# GhostRoute Repo Review — 2026-06-16 + +## Scope + +This review looked at the latest public `ghostroute` commits, repository landing +surfaces and safety documentation. The goal was not to add runtime features. The +goal was to make the repository easier for a reviewer or hiring manager to read +as a production-like personal infrastructure project without exposing private +operator state. + +Reviewed areas: + +- latest commit history and recent docs/API/schema changes; +- root README and docs navigation; +- product framing, testing strategy and contributor workflow; +- secret hygiene surfaces (`.env.example`, `.gitignore`, `SECURITY.md`); +- Console public API/schema documentation added in the latest commits. + +No router, VPS, Vault, generated client profile or local report data was read or +modified. + +## Latest commit assessment + +### `7d5562e` — docs/schema/API polish + +The latest commit is directionally strong for public presentation. It fixed +broken internal links, removed root-relative link assumptions, separated stable +planning from runtime docs, and added Console database/API reference docs. That +is a good hiring signal because it shows the repository is not just code: it has +contracts, navigation and evidence that docs are verified. + +The most important improvement is the explicit Console schema/API documentation: +`modules/ghostroute-console/docs/database-schema.md` and +`modules/ghostroute-console/docs/api-contracts.md`. Those docs turn a large +Next.js/SQLite module into a reviewable interface. They should remain concise +references, not copies of every route response shape or DDL column. + +### `a516971` — architecture diagram update + +The diagram update improves first-impression readability, but binary diagrams are +harder to review and diff than Mermaid/ASCII. Since diagrams already exist, do +not churn them further unless a runtime/architecture change requires it. Future +architecture explanations should prefer text or Mermaid when possible. + +### `3fe802d` — traffic-facts v3 / Console pyramid planning + +The traffic-facts v3 plan is a strong architecture signal because it clarifies a +machine contract, separates raw evidence from presentation reports and keeps +Console from consuming unstable human-oriented output. Keep its status as a plan +until implementation lands; avoid language that makes the future pyramid sound +fully production if it is still a refactor plan. + +## What was changed in this polish branch + +### Product framing + +Created `docs/product-requirements.md`. + +Why: the repo already had rich architecture docs, but no compact product lens for +reviewers. The new brief defines the problem, users, goals, non-goals, functional +requirements, quality attributes and success metrics without inventing features. + +Hiring signal strengthened: product thinking, scope control, stakeholder clarity +and explicit non-goals. + +### Testing strategy + +Created `docs/testing.md`. + +Why: test commands existed across README, CONTRIBUTING, CI and module docs, but a +reviewer needed one place to understand the test pyramid and the boundary between +public CI and live operator verification. + +Hiring signal strengthened: testability, CI maturity, separation of fixture tests +from live infrastructure, and realistic deploy verification. + +### Documentation navigation + +Updated `docs/README.md`. + +Why: `docs/` is the natural reviewer entry point after the README. The index now +links product requirements, testing, deployment/rollback, SLOs and review +snapshots, and clearly separates stable docs from planning/future direction. + +Hiring signal strengthened: technical writing, information architecture and +maintainability. + +### Environment examples + +Updated root `.env.example` and added +`modules/ghostroute-console/app/.env.example`. + +Why: the root example had the right security intent but still contained stale +script paths and did not make the Console-local environment obvious. The new +examples use placeholders only, point to module-native commands and separate +router SSH, DNS/catalog and Console local-development knobs. + +Hiring signal strengthened: developer experience, onboarding safety and secret +hygiene. + +### Ignore rules + +Updated `.gitignore`. + +Why: the repo ignored the critical secret/artifact paths already, but did not +cover several common local env variants and browser/test artifacts such as +`.env.local`, `.env.production`, Console Playwright reports, coverage and npm +logs. The update keeps `.env.example` trackable while ignoring real `.env*` +files. + +Hiring signal strengthened: operational safety and practical repo hygiene. + +### Node version and contribution flow + +Added `.nvmrc` with Node 22 and updated `CONTRIBUTING.md`. + +Why: CI already uses Node 22, while the contribution guide described `.nvmrc` as +a backlog item. The new guide ties setup, docs conventions, tests, PR content, +pre-commit hooks and safety boundaries together. + +Hiring signal strengthened: developer experience, consistency between local and +CI environments, and PR discipline. + +### Pull request template + +Added `.github/pull_request_template.md`. + +Why: contributor docs are useful, but a PR template makes the expected behavior +visible at the moment of change: scope, tests, risk, rollback and secret review. + +Hiring signal strengthened: code-review process, operational risk management and +security review discipline. + +## Risks and findings + +| Finding | Risk | Remediation | +|---|---|---| +| Root README is excellent but very dense. | A recruiter may bounce before reaching the architecture/testing/security signals. | Keep README as the deep landing page, but rely on `docs/product-requirements.md` and `docs/testing.md` as reviewer shortcuts. Consider a short "Reviewer map" block in README later. | +| Console is described as read-only, but API docs include POST action endpoints. | A reviewer could misunderstand "read-only" as "no writes anywhere" rather than "no router/VPS runtime mutation". | Keep wording precise: "read-only runtime surface" / "audited operator-state overlays". Avoid implying POST endpoints deploy or mutate routing. | +| Latest diagram commit added binary images. | Binary diffs are harder to audit and conflict with a text-first documentation preference. | Do not add more generated images for this polish pass. Prefer Mermaid/ASCII for future diagram changes unless screenshots are truly necessary. | +| Planning docs are powerful but numerous. | Future designs may be mistaken for implemented runtime behavior. | Keep `docs/README.md` Planning & Future Direction section and status badges/notes up to date. | +| Full secret scan could not be run from this GitHub-only review session. | I cannot prove the entire tree is clean from local tooling here. | Before merge, run `./modules/secrets-management/bin/secret-scan` and let CI run `./tests/run-fast.sh`. | + +## Sensitive data handling + +No secrets, tokens, credentials, private endpoints, UUIDs, VLESS URIs, QR payloads +or personal device identifiers were added. All examples use placeholders such as +``, ``, `` and +``. + +No existing secret value was intentionally displayed in this review. The files +inspected in the GitHub app used placeholders and public role names where +expected. A full repository secret scan should still be run before merge because +this review did not execute local tooling. + +## Recommended next improvements + +1. Add a short "Reviewer map" near the top of `README.md` linking directly to + Product Requirements, Architecture, Testing, Security, SLOs, Deployment / + Rollback and Console API/Schema docs. +2. Consider a `docs/public-sanitization-checklist.md` only if secret-review steps + keep spreading across docs. For now `SECURITY.md`, `CONTRIBUTING.md`, the PR + template and `docs/testing.md` are enough. +3. Keep future Console API docs synchronized with route changes. If response + shapes grow, document representative schemas close to route handlers rather + than bloating the top-level API table. +4. Run the full repo checks and CI before merging this docs/hygiene branch. + +## Verification to run before merge + +```bash +./modules/secrets-management/bin/secret-scan +./tests/run-fast.sh +./tests/run-smoke.sh +bash -n verify.sh tests/run-all.sh tests/run-fast.sh + +npm --prefix modules/ghostroute-console/app ci +npm --prefix modules/ghostroute-console/app test +npm --prefix modules/ghostroute-console/app run build +``` + +For live/operator validation only when router and VPS are intentionally in scope: + +```bash +./verify.sh --verbose +cd ansible && ansible-playbook playbooks/99-verify.yml +cd .. +./modules/ghostroute-health-monitor/bin/router-health-report +./modules/traffic-observatory/bin/traffic-report check +``` diff --git a/docs/testing.md b/docs/testing.md new file mode 100644 index 00000000..74740970 --- /dev/null +++ b/docs/testing.md @@ -0,0 +1,137 @@ +# Testing Strategy + +GhostRoute tests are split by risk and by access boundary. Public CI proves the +repository contract without requiring a router, VPS, Vault password or generated +client artifacts. Live verification is deliberately separate and must be run only +by the operator when real targets are reachable. + +## Test layers + +| Layer | Scope | Requires live infrastructure | Primary command | +|---|---|---:|---| +| Secret hygiene | Tracked-file scan for endpoints, UUIDs, keys, ports, VLESS/QR-style payloads and production literals. | No | `./modules/secrets-management/bin/secret-scan` | +| Static/syntax checks | Shell syntax, BusyBox/router script compatibility, static routing invariants and module entrypoints. | No | `./tests/run-fast.sh` | +| Console unit/contract checks | Next.js/Node selectors, migrations, aggregate contracts and seeded GUI database behavior. | No | `npm --prefix modules/ghostroute-console/app test` | +| Console smoke | Build plus Playwright smoke over a seeded local GUI database. | No | `./tests/run-smoke.sh` | +| Console performance | Deterministic local Playwright performance budget on seeded data. | No | `npm --prefix modules/ghostroute-console/app run test:perf` | +| Ansible syntax | Playbook parse checks for the selected router/VPS surface. | No | `cd ansible && ansible-playbook --syntax-check playbooks/.yml` | +| Live read-only verification | Router/VPS facts, channel invariants, health and traffic checks. | Yes | `./verify.sh`, `ansible-playbook playbooks/99-verify.yml` | +| Mutating deploy gate | Pre/post canary before runtime mutation. | Yes | `./modules/ghostroute-health-monitor/bin/live-check --active-probe --deploy-gate` | + +## Recommended local flow + +For documentation-only or small static changes: + +```bash +./modules/secrets-management/bin/secret-scan +./tests/run-fast.sh +``` + +For Console UI, selector, database or API changes: + +```bash +npm --prefix modules/ghostroute-console/app ci +npm --prefix modules/ghostroute-console/app test +npm --prefix modules/ghostroute-console/app run build +./tests/run-smoke.sh +``` + +For broad Console read-model or performance-sensitive changes: + +```bash +npm --prefix modules/ghostroute-console/app run test:gui:all +``` + +For Ansible/router changes, add the smallest syntax set that matches the touched +surface: + +```bash +cd ansible +ansible-playbook --syntax-check playbooks/20-stealth-router.yml +ansible-playbook --syntax-check playbooks/21-channel-b-router.yml +ansible-playbook --syntax-check playbooks/22-channel-c-router.yml +ansible-playbook --syntax-check playbooks/24-channel-d-router.yml +ansible-playbook --syntax-check playbooks/99-verify.yml +``` + +Do not run mutating playbooks as a test. Use syntax checks and read-only verify +first; deploy remains a separate operator decision. + +## CI contract + +`.github/workflows/ci.yml` runs two pull-request jobs by default: + +1. **Repo checks** install Node/Python/ripgrep, install Console dependencies and + run `./tests/run-fast.sh`. +2. **Console smoke** installs Console dependencies, installs Playwright Chromium + and runs `./tests/run-smoke.sh`. + +The Console performance job is manual (`workflow_dispatch`) so ordinary PRs do +not pay the heavier browser-performance cost unless the change needs it. + +## What the tests protect + +| Risk | Covered by | +|---|---| +| Secret or private deployment value committed to public repo | `secret-scan`, `.gitignore`, pre-commit hook | +| Router shell script no longer parses on the target shell | `tests/check-shell-syntax.sh`, `run-fast` | +| Channel B/C/D accidentally mutates Channel A ownership | Static channel tests, `AGENTS.md` invariants, `99-verify.yml` live checks | +| Legacy WireGuard or `RC_VPN_ROUTE` returns as production state | Static grep-style invariant tests and live verify | +| Console read model breaks accounting invariants | Console tests, aggregate verification, post-deploy checks | +| Console UI only works with live data | Seeded GUI database plus Playwright smoke/perf tests | +| Public docs expose real values | Secret scan plus documentation placeholder policy | + +## Live checks + +Live checks are intentionally not part of public CI. They may require the current +router access profile, Vault values, a reachable VPS and fresh operator network +state. + +Read-only live sequence: + +```bash +./verify.sh --verbose +cd ansible && ansible-playbook playbooks/99-verify.yml +cd .. +./modules/ghostroute-health-monitor/bin/router-health-report +./modules/traffic-observatory/bin/traffic-report check +./modules/traffic-observatory/bin/traffic-report today +``` + +Mutating deploys must follow `docs/deployment-and-rollback.md`. A green local or +CI suite does not authorize deployment by itself. + +## Pre-commit hooks + +`.pre-commit-config.yaml` provides two local hooks: + +```bash +pre-commit install +pre-commit run --all-files +``` + +The hooks run the GhostRoute secret scan and shell syntax checks. Do not bypass +hooks for normal work; if an emergency requires bypassing, document why in the PR +or operator note. + +## Evidence to include in PRs + +A good PR test plan should say exactly what ran and what did not run. Example for +a docs-only PR: + +```text +Tests: +- Not run: docs-only change; no runtime, Console or Ansible behavior changed. +- Recommended before merge: ./modules/secrets-management/bin/secret-scan +``` + +Example for a Console PR: + +```text +Tests: +- npm --prefix modules/ghostroute-console/app test +- npm --prefix modules/ghostroute-console/app run build +- ./tests/run-smoke.sh +Not run: +- Live router/VPS verification; change is local Console read-model/UI only. +``` diff --git a/modules/ghostroute-console/app/.env.example b/modules/ghostroute-console/app/.env.example new file mode 100644 index 00000000..7951b11c --- /dev/null +++ b/modules/ghostroute-console/app/.env.example @@ -0,0 +1,27 @@ +# GhostRoute Console local development example +# +# Copy to `.env.local` only if you need local overrides: +# cp .env.example .env.local +# +# `.env.local` is gitignored by the root ignore rules. Do not put production +# Basic Auth credentials, router SSH values, real egress metadata or private +# device labels in tracked files. + +# Local seeded GUI database used by dev:gui and browser tests. +# GHOSTROUTE_CONSOLE_DATA_DIR=../data/gui-test + +# Synthetic quota knobs for local UI development. +# GHOSTROUTE_CONSOLE_VPS_QUOTA_GB=1024 +# GHOSTROUTE_CONSOLE_LTE_QUOTA_GB=50 +# GHOSTROUTE_CONSOLE_BILLING_RESET_DAY=1 + +# Server-sent event refresh interval. Keep it conservative; live data is based on +# append-only snapshots, not a control-plane stream. +# GHOSTROUTE_LIVE_UI_REFRESH_MS=600000 + +# Prepared-window diagnostic switch. Production should leave this enabled. +# GHOSTROUTE_CONSOLE_USE_PREPARED_WINDOWS=1 + +# Public deployment placeholders only. Real values live in Vault/operator config. +# GHOSTROUTE_CONSOLE_PUBLIC_HOST= +# GHOSTROUTE_CONSOLE_PUBLIC_PORT=