diff --git a/.env.example b/.env.example index fa7aeead..d5bbd871 100644 --- a/.env.example +++ b/.env.example @@ -70,6 +70,55 @@ PARSAR_AGENT_DAEMON_WS_URL=ws://parsar-server:8080/agent-daemon/ws # Bot's own open_id (ou_xxx). Required for group chat @Bot to work. PARSAR_FEISHU_DEFAULT_BOT_OPEN_ID= +# ----------------------------------------------------------------------------- +# Cloud sandbox — e2b.app (optional; powers the "Cloud isolation" agent mode) +# ----------------------------------------------------------------------------- +# Both values are required to enable cloud isolation. With either empty the +# server logs a warning at boot and cloud-isolation agents fail fast, while +# local-device agents keep working. +# +# Build the template first (writes only gitignored build artifacts): +# make e2b-template +# then paste the template id it reports here. Rebuilding the same template +# name keeps the id, so this only changes on first creation. +AGENT_DAEMON_SANDBOX_TEMPLATE= +PARSAR_E2B_API_KEY= + +# Optional larger tier, selected per agent via agents.config.sandbox_size="xl". +# An agent asking for a size with no template configured degrades to the +# standard one with a warning rather than failing the acquire. +AGENT_DAEMON_SANDBOX_TEMPLATE_XL= + +# Optional deployment-wide sandbox lifetime. Uses Go duration syntax (for +# example 30m, 1h, or 24h) and defaults to 1h. An agent may override this with +# agents.config.sandbox_ttl. Parsar does not impose a provider-specific maximum; +# configure a value supported by your sandbox provider and account. +AGENT_DAEMON_SANDBOX_TTL= + +# Deployment-wide default for periodic best-effort renewal. An agent may +# override this with agents.config.sandbox_auto_renew. Renewal requires a TTL +# longer than the five-minute scan interval. Enabled by default so a +# continuously-used agent's sandbox is not reaped mid-conversation; set to +# false to opt out (e.g. for a provider that does not support renewal). +AGENT_DAEMON_SANDBOX_AUTO_RENEW=true + +# Deprecated compatibility setting. Used only when the duration above is +# empty; prefer AGENT_DAEMON_SANDBOX_TTL for new deployments. +AGENT_DAEMON_SANDBOX_TTL_HOURS= + +# Self-hosted / proxied e2b only. Leave empty for e2b.app. +PARSAR_E2B_API_BASE_URL= +PARSAR_E2B_SANDBOX_HOST= +# PEM of a private CA, when the sandbox gateway serves a non-public cert. +PARSAR_E2B_CA_CERT= + +# IMPORTANT for local testing: the daemon runs INSIDE the cloud sandbox and +# dials back to PARSAR_PUBLIC_URL, so a loopback URL can never work — the +# sandbox resolves 127.0.0.1 to itself and pairing times out. Expose the dev +# server through a tunnel and set PARSAR_PUBLIC_URL to that hostname, e.g. +# ngrok http 18080 +# PARSAR_PUBLIC_URL=https://.ngrok-free.dev + # ----------------------------------------------------------------------------- # Network proxy (optional — only if your host needs a proxy for internet) # ----------------------------------------------------------------------------- diff --git a/.gitignore b/.gitignore index 7c23b234..1a5f6373 100644 --- a/.gitignore +++ b/.gitignore @@ -51,6 +51,7 @@ go.work.sum # Reviewer / verifier session artifacts (workspace-only, never commit) reviewer-report.md +semantic-review/ /.fork-point # Claude Code IDE state (per-developer scratch: plans, worktree pointers, @@ -72,3 +73,12 @@ reviewer-report.md !/parsar-bootstrap/ /httprunner !/httprunner/ +# `cd server && go build ./cmd/server` drops its output here. +/server/server +!/server/server/ + +# Cross-compiled binaries staged for the E2B template build +# (`make e2b-template`). E2B has no multi-stage build support, so +# parsar-daemon / parsar are compiled on the host and COPY'd in by +# infra/sandbox/e2b.Dockerfile. Build artefacts, never committed. +infra/sandbox/.build/ diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 84d458c3..826d96ad 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -248,6 +248,47 @@ description and keep ownership on the side listed here. - Eager acquisition must be best-effort. Failure to prewarm a sandbox should surface as runtime health/provisioning state, not crash unrelated startup paths. +- Cloud sandbox maintenance runs once at server startup and every five minutes. + Automatic renewal requires a TTL longer than that interval; interrupted + renewals remain retryable, while a provider rejection disables the policy. +- A `spawning` sandbox binding holds that agent's only reservation slot + (`uk_sandboxes_active_per_agent` is partial on `killed_at is null`), and the + loser path waits on `spawning` indefinitely. Any code that reserves a slot + must therefore guarantee a terminal transition, and an acquire that finds a + reservation older than the cold-start bound must be able to reclaim it — + otherwise one crashed cold start wedges the agent permanently. + +### Sandbox images + +- `infra/sandbox/Dockerfile` (local Docker + generic) and + `infra/sandbox/e2b.Dockerfile` (e2b.app) must keep their shared runtime + payload, CLI versions, and hooks aligned; provider-specific bootstrap and + build mechanics may differ. + Agent CLI installs live only in `infra/sandbox/scripts/install-agents.sh`, + which both images run; do not inline per-CLI `npm install -g` / download + steps in either Dockerfile. That script owns the version pins and the Node + force-relink that keeps a base image's bundled Node from shadowing ours. +- The image must ship the hook scripts at the absolute paths + `server/internal/connector/agentdaemon/sandbox_seed.go` seeds into + `settings.json` (`/opt/parsar/hooks/claude/...`). The hooks fail open, so a + missing script degrades spec/memory injection silently instead of erroring — + changing one side means changing the other. +- e2b's template builder is not BuildKit. It rejects multi-stage builds (hence + the prebuilt binaries in `infra/sandbox/.build/`, staged by + `make e2b-template`), it does not persist `/tmp` between layers, and it + lowers `ARG FOO="bar"` keeping the quotes as literal characters — so version + ARGs in `e2b.Dockerfile` must stay unquoted. +- Build templates with `make e2b-template`. It writes only into + `infra/sandbox/.build/` (gitignored), never the repo root. + +### Testing cloud isolation locally + +- The daemon runs inside the cloud sandbox and dials back to + `PARSAR_PUBLIC_URL`, so a loopback URL cannot work: the sandbox resolves + `127.0.0.1` to itself and pairing times out. Expose the dev server through a + tunnel and set `PARSAR_PUBLIC_URL` to that hostname. +- `AGENT_DAEMON_SANDBOX_TEMPLATE` + `PARSAR_E2B_API_KEY` are the two required + values; see `.env.example` for the full set and their defaults. ### API, DB, and generated surfaces diff --git a/Makefile b/Makefile index 87c5e8ad..14e9edbc 100644 --- a/Makefile +++ b/Makefile @@ -18,7 +18,7 @@ endif PARSAR_IMAGE ?= parsar PARSAR_IMAGE_TAG ?= dev -.PHONY: help setup node-deps dev dev-db check check-setup check-sqlc check-go check-store check-web check-cli check-hygiene test test-fast test-go test-web typecheck-web lint-web-design lint-web test-cli typecheck reset-dev clean-dev paths migrate-dev sqlc-generate server web cli devgateway http-runner-once http-runner-loop dev-all smoke e2e-http-agent e2e-feishu-gateway dev-server-up dev-server-down dev-server-log bootstrap docker-build docker-build-no-cache openapi +.PHONY: help setup node-deps dev dev-db check check-setup check-sqlc check-go check-store check-web check-cli check-hygiene test test-fast test-go test-web typecheck-web lint-web-design lint-web test-cli typecheck reset-dev clean-dev paths migrate-dev sqlc-generate server web cli devgateway http-runner-once http-runner-loop dev-all smoke e2e-http-agent e2e-feishu-gateway dev-server-up dev-server-down dev-server-log bootstrap docker-build docker-build-no-cache openapi e2b-template e2b-template-binaries help: @printf '%s\n' \ @@ -43,7 +43,11 @@ help: ' make check-go Run sqlc drift check and non-store Go tests' \ ' make check-store Run migration and store integration tests' \ ' make check-web Run web typecheck and design lint' \ - ' make check-cli Typecheck CLI/plugin packages' + ' make check-cli Typecheck CLI/plugin packages' \ + '' \ + 'Cloud sandbox (e2b):' \ + ' make e2b-template Cross-compile binaries and build the e2b sandbox template' \ + ' make e2b-template E2B_TEMPLATE_NAME=my-sandbox' setup: ./scripts/setup.sh @@ -243,12 +247,13 @@ docker-build-no-cache: # go install github.com/swaggo/swag/cmd/swag@v1.16.4 # The recipe auto-installs on first use. SWAG_VERSION ?= v1.16.4 +SWAG ?= $(shell command -v swag 2>/dev/null || printf '%s/bin/swag' "$$(go env GOPATH)") openapi: - @command -v swag >/dev/null 2>&1 || \ + @command -v $(SWAG) >/dev/null 2>&1 || \ go install github.com/swaggo/swag/cmd/swag@$(SWAG_VERSION) @mkdir -p docs/openapi - swag init \ + $(SWAG) init \ -g server/cmd/server/main.go \ --dir . \ --output docs/openapi/gen \ @@ -260,3 +265,60 @@ openapi: @rmdir docs/openapi/gen 2>/dev/null || true @echo "openapi: wrote docs/openapi/openapi.yaml" @echo "openapi: paths=$$(grep -c '^ /' docs/openapi/openapi.yaml)" + +# --- E2B sandbox template ---------------------------------------------- +# +# Builds the cloud-isolation sandbox template on e2b.app. Collapses the +# manual sequence (cross-compile two Go binaries, then invoke the e2b +# CLI from the right directory) into one command. +# +# Why the binaries are cross-compiled here rather than in the image: +# e2b's builder rejects multi-stage Dockerfiles, so infra/sandbox/ +# e2b.Dockerfile cannot compile Go itself and instead COPYs prebuilt +# binaries out of infra/sandbox/.build/ (gitignored). linux/amd64 is +# hardcoded because e2b templates are amd64-only. +# +# The build context is infra/sandbox/ (not the repo root) so the upload +# stays small; every COPY source in e2b.Dockerfile lives under it. +# +# Requires the e2b CLI and an API key: +# npm install -g @e2b/cli +# export E2B_API_KEY=e2b_... +# PARSAR_E2B_API_KEY is accepted as a fallback so the same value already +# in .env (used by the server) works without being re-exported, and .env +# is sourced when present. +# +# After a successful build, point the server at the template: +# AGENT_DAEMON_SANDBOX_TEMPLATE=