6b: merge upstream master into the Go port (mechanical) - #4
Merged
Merged
Conversation
Addresses VirtusLab#84. * Move basic auth handling from cursor to common mitmproxy module --> applicable for all agents * Extend docstrings of respective functions
test_claude_does_not_touch_basic_auth asserted the pre-VirtusLab#85 behavior where ClaudeAddon inherited empty stubs from the common addon and therefore left Basic Auth headers untouched. VirtusLab#85 moved the real implementation into the common addon, so ClaudeAddon now correctly decodes, substitutes the placeholder, and re-encodes — the assertion inverted. Rename to test_claude_also_substitutes_basic_auth and assert the new behavior. Rename the enclosing class from TestCursorBasicAuth to TestBasicAuth since Basic Auth substitution is no longer cursor-specific. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
## Message from human ~at 1am~ ~**Sigh**. I fat fingered the PR button and opened this one before I meant to. This is AI slop until I review it and decide I can speak to it, at which point I will remove the draft status.~ This PR uses an env var to tell `uv` (a Python package manager) to use system TLS certificates. ## PR Message from Claude (edited): uv bundles its own root CA store (it's a Rust binary using rustls) and doesn't consult the system trust store by default, so it fails TLS verification against mitmproxy's intercepting CA. Selecting the python stack now sets ~UV_NATIVE_TLS and~ UV_SYSTEM_CERTS on the agent service automatically. Adds stack_env_entries() alongside the existing stack_extension() pattern, and customize_compose_stack_environment() to merge stack env vars into services.agent.environment. The existing per-agent environment merge (CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC etc.) was changed from an overwrite to an append so agent- and stack-contributed entries coexist regardless of call order. Corrects the README's TLS/CA section, which claimed Python "works out of the box" without mentioning the situation with `uv`. --------- Co-authored-by: Claude <noreply@anthropic.com>
I don't think it was intended for the agent to have root access in the container. Currently the agent can perform mischief such as writing a setuid binary in the project directory on the host. The no_new_privs flag does not prevent app-init.sh from running as root. It does affect all setuid binaries, not just sudo. Without this patch: ```bash vscode ➜ /workspaces/t-sandbox $ sudo whoami root ``` With this patch: ```bash vscode ➜ /workspaces/t-sandbox $ sudo whoami sudo: The "no new privileges" flag is set, which prevents sudo from running as root. sudo: If sudo is running in a container, you may need to adjust the container configuration to disable the flag. ``` Interestingly, this setting also *enables* a new capability in the container: the ability to "install" (activate?) seccomp filters. I don't know if there is any use for this. The kernel documentation [2] says: > Filters installed for the seccomp mode 2 sandbox persist across execve and can change the behavior of newly-executed programs. Unprivileged users are therefore only allowed to install such filters if no_new_privs is set. References: 1. https://docs.docker.com/reference/cli/docker/container/run/#security-opt 2. https://www.kernel.org/doc/Documentation/prctl/no_new_privs.txt
…xy to restart (VirtusLab#93) Restart the agent after wg-client on `sandcat restart` so its shared network namespace re-links to the fresh wg-client (fixes VirtusLab#69), and rename the command restart-proxy -> restart since it now restarts the full stack (mitmproxy + wg-client + agent).
…tusLab#104) Follow-up to VirtusLab#97: the README still described the pre-split layout for sandcat.env, cursor-cli-config.json, the agent's read-only mount, and the volumes diagram.
…tusLab#92) Closes VirtusLab#54 (the cert half; the DNS half landed via VirtusLab#55's extra_hosts). Rebased onto master with a fix: apply_upstream_ca_bundles now composes the CA install on top of the template's entrypoint instead of substituting a fixed pre-VirtusLab#97 one, preserving the mitmproxy-public cert publication the healthcheck gates on.
Closes VirtusLab#20. Pins mitmproxy to 12.2.3 across compose/Dockerfiles with a two-file version contract enforced by CI, pins the rtk installer (script SHA + RTK_VERSION with checksum verification), and simplifies the tag policy per review: every published image carries the pinned base, ghcr latest = newest master build, weekly cron is a build-only canary against upstream mitmproxy:latest.
Closes #2. ## Summary Adds **network presets** — `{"preset": "<name>"}` entries in the `network` policy that expand in place to predefined allow-rule groups, as requested in #2 (modeled on agent-sandbox's enforcer lists and tsk's squid.conf). - Expansion preserves rule order, so first-match-wins works across presets (a `deny` before a preset shadows its hosts). - Fail-loud: unknown preset name or `preset` combined with other keys raises at `load()` — the proxy refuses to start rather than run a policy different from the one asked for. - Presets are host-only (no method restriction) — matching the domain-level lists they're modeled on; method-tightening registries breaks legitimate flows (npm audit POSTs) for marginal gain. - One preset per `init` stack (python/node/java/scala/go/rust/ruby/dotnet/zig — a test keeps this in sync with `STACK_NAMES`), plus `nix`, `vscode`, `jetbrains`, `github`, `anthropic`, `openai`. - Ecosystem presets are self-contained (`scala` repeats Maven hosts); duplicates across combined presets are harmless under first-match-wins. This PR does **not** change the default policy — the project template keeps `allow * GET`. A follow-up PR will add an opt-in strict mode that replaces the wildcard with stack presets. ## Test plan - [x] pytest: 337 passed (14 new preset tests × both addon variants), incl. in-place ordering, fail-loud on unknown/mixed keys, definitions-vs-STACK_NAMES sync guard - [x] Full bats: 16/16 suites - [x] Hands-on integration in a real container: project policy set to `{"preset":"python"}, {"preset":"github"}` (no wildcard) → `pip download` from pypi succeeds through the proxy, `curl https://example.com` → 403, DNS for non-preset hosts REFUSED; unknown preset → mitmproxy fails to become healthy (fail-loud verified live) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ildcard (VirtusLab#106) Stacked on VirtusLab#105 (network presets) — the base branch is `feat/2-network-presets`; retarget to master after VirtusLab#105 merges. Follow-up to #2; no separate issue, so no close keyword. ## Summary `sandcat init --features strict-network` (or `SANDCAT_STRICT_NETWORK=true`) generates project settings whose `network` list holds one `{"preset": "<stack>"}` entry per resolved stack and **no allow-all-GET wildcard**. Everything beyond the stack registries and the user-settings layer (the agent's own API hosts) is then denied by default — including DNS resolution, so blocked hosts never even resolve. - Opt-in by design: flipping the default would surprise every new project with "sandcat blocks the internet". The init summary states which policy the project got (`Network: strict — stack presets: python, java` vs `default (allow all GET; tighten with --features strict-network)`). - `init settings` grows `--strict-network` / `--stacks` flags; with no stacks the strict list is empty (user-settings layer only). - Only preset *names* land in the project file — the addon expands them at proxy start, so domain lists update with the sandcat version instead of freezing per project. - `scala` resolves to `java scala` via the existing stack-deps mechanism, so both presets are seeded. ## Test plan - [x] bats: init 167/167 (new: feature parsing incl. error-message listing, strict/default summary lines, settings --strict-network seeding, empty-stacks case, unknown-option rejection; interactive stubs updated for the new feature label) - [x] Full bats: 16/16 suites - [x] Hands-on integration in a real container: `init --stacks python --features strict-network` generates `{\"network\":[{\"preset\":\"python\"}]}` (no wildcard); stack healthy; pypi.org **200** (stack preset), www.anthropic.com + github.com **200** (presets expanded from the *user-settings layer* — expansion works across layers), example.com **DNS REFUSED** (rc=6 before HTTP); real PyPI package fetch end-to-end through the proxy (JSON API → wheel from files.pythonhosted.org, valid zip). Note: `pip download` itself is unavailable in the devbox/nix python (no ensurepip) — toolchain quirk, not policy; the fetch above covers the network path. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Fixes VirtusLab#102. Carries VirtusLab#108 by @atirna — the original commit is cherry-picked with authorship preserved; opened from an in-repo branch so CI runs without the fork-approval step. On top of it there is one cosmetic follow-up commit renaming the test-seam variable `SANDCAT_HOME` → `SANDCAT_USER_HOME` (the former already means "host CLI install dir" in `install.sh`, so one name carried two meanings). ## Summary (from VirtusLab#108) Java trust-store exports move out of `.bashrc` into a guarded `/etc/profile.d/sandcat-java.sh`. The entrypoint sources it after `app-user-init.sh` refreshes the writable trust store, so the agent process — and everything it spawns — inherits `JAVA_HOME` and `JAVA_TOOL_OPTIONS`. Login shells get it via `/etc/profile`, VS Code terminals via the existing `sandcat-*.sh` sourcing block in `/etc/bash.bashrc`. ## Verification - bats: init 162/162, run 9/9 (incl. the PR's three new tests) - Hands-on integration on a real container with `--stacks java` (temurin via devbox): - PID1 (agent process) environment contains both `JAVA_HOME` and `JAVA_TOOL_OPTIONS` (`/proc/1/environ`) - end-to-end: a JVM child with the inherited env fetches `https://example.com` **through the proxy → HTTP 200** (mitmproxy CA honored via the sandcat trust store) - negative control: same fetch with `env -u JAVA_TOOL_OPTIONS` → `PKIX path building failed` — the exact VirtusLab#102 symptom - login shell exports present; `cacerts` refreshed by app-user-init; baseline (GET 200 / POST 403 / no sudoers / no CA private key) unregressed - Reviewer note: `docker compose exec … printenv` can NOT observe these variables — `docker exec` always gets the image's Config.Env, never the entrypoint's; `/proc/1/environ` is the correct probe. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Atirna <288419661+atirna@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ns (VirtusLab#79) sandcat init --ide jetbrains --stacks <s> now seeds customizations.jetbrains.plugins with the stack's Marketplace plugin ID (scala → org.intellij.scala, python → PythonCore, go/ruby/zig accordingly), so Gateway auto-installs it into the backend IDE — the symmetric counterpart of stack_extension() on the VS Code path. Rebased across the mise→devbox migration; the branch's stale mise customize_dockerfile was dropped during conflict resolution.
…irtusLab#111) Closes VirtusLab#110. Closes VirtusLab#112. RTD scaffold modeled on softwaremill/chimp (minus the mdoc indirection) plus the full content migration: every README section as a docs page, cli/README.md folded in as the CLI reference, README pointing at https://sandcat.virtuslab.com with the local-preview instructions dropped per review. Strict Sphinx build (-W) clean; mermaid renders via myst_fence_as_directive; llms.txt published.
…it gaps (VirtusLab#115) Closes VirtusLab#114. Top-level Agents and IDE integration chapters (incl. the previously undocumented JetBrains path: customizations.jetbrains, per-stack plugins, overrideCommand:false warning), VS Code hardening folded into its IDE page, audit gap fixes (Basic Auth substitution walkthrough, agent container hardening), Getting started collapsed to a single Quick start with Installation as a top-level entry and the init details split into Configuration pages (stacks, volume-mounts, caches, gitignore). Strict Sphinx build clean throughout.
…irtusLab#116) collapse_navigation:False + navigation_depth:3 keep the tree expanded; the Quick start page sits under a Getting started caption, level with the other groups.
Scheduled canaries and GHCR publishes only belong on VirtusLab/sandcat. Co-authored-by: Michał Wiącek <wiacekm@virtuslab.com>
Mechanical sync of 24 upstream commits (c16d8fd..4a451ba) into the bash tree and templates. Merged cleanly with no conflicts; nothing on the Go side is touched here. The Go changes needed to restore parity with the updated bash follow in a separate stacked PR, so the mechanical merge and the substantive port changes can be reviewed apart. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5
…tusLab#99) Upstream pinned the mitmproxy image (SCT_MITMPROXY_VERSION, rendered into both the template placeholder and the secret-provider image tags) and the rtk installer script and binary. The version lives in compose.MitmproxyVersion, with a contract test against images/mitmproxy.env mirroring cli/test/compat/mitmproxy_version.bats; the rtk block is regenerated from bash. The compose-agent.yml split (VirtusLab#99) leaves compose-all.yml declaring `agent: {}` in flow style. yaml.v3 preserves that style when children are added, rendering the populated service on one line; yq switches an empty flow collection to block style as soon as it gains content. appendContent now does the same, and only under that condition, so a non-empty flow node still keeps its style as it would under yq. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5
The python stack now sets UV_SYSTEM_CERTS=1 on the agent service so uv trusts mitmproxy's intercepting CA. Upstream reworked the environment plumbing to make room for it: services.agent.environment is merged (appended to) rather than set, with the stack entries written before the agent's. SetAgentEnvironment becomes MergeAgentEnvironment and Generate follows the bash call order, which decides the entry order in the file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5
Stacks now carry a JetBrains Marketplace plugin id, the counterpart of the VS Code extension for the JetBrains devcontainer path. Generate fills the `"plugins": []` array that the JetBrains customizations block declares, after CustomizeJSON has emitted it. Stacks whose language support is bundled, or whose JetBrains IDE is a standalone product with no IntelliJ plugin, contribute nothing and leave the array empty. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5
, VirtusLab#96) `--features strict-network` (or SANDCAT_STRICT_NETWORK=true) replaces the project template's allow-all-GET wildcard with one `{"preset": s}` entry per resolved stack, expanded to concrete rules by the mitmproxy addon at start so the domain lists track the sandcat version. That path goes through `yq -o=json` in the bash and therefore reformats the whole file; jsonfile reproduces the layout, and a parity test covers the plain copy, strict with stacks, and strict with none. init also scaffolds an empty .sandcat/settings.local.json — the highest-precedence layer in the addon's settings merge and gitignored by the Sandcat block — and never overwrites one that exists, since it holds real credentials. The harness gains three strict-network cases: 30/30. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5
Fourth supported agent. The table entry carries its extension, token help, mitmproxy addon and host config paths; the Dockerfile, home-prep and user-init fragments are captured from bash like the others, and the dump script now enumerates agents from sct_available_agents so a fifth one cannot be missed. The compose mounts follow upstream: mcp-config.json read-only, session-state read-write because Copilot CLI writes session events there and fails with EROFS otherwise. Harness and parity matrices gain copilot rows: 33/33. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5
Users behind a corporate TLS interceptor can list PEM bundles under `upstream_ca_bundles` in user settings or the project's settings.local.json. init validates each (absolute, readable, contains a certificate block — the document is left untouched on any failure), bind-mounts them read-only into mitmproxy, and prepends their installation to the entrypoint. The install runs into both the OS store and certifi's bundle, because mitmproxy reads its trust store from certifi; and it is joined with `|| exit 1;` rather than `&&` so it stays outside the entrypoint's backgrounded list and fails loud. The harness never exercises this (no bundle under a throwaway HOME), so it gets a dedicated parity test with fixture bundles split across the two settings files, plus a rejection test. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5
) The command restarts the agent as well when it was running: the agent uses network_mode: service:wg-client, which Docker resolves to a netns fd at start, so after wg-client restarts the agent's fd points at a torn-down namespace and every outbound connection breaks. The re-link waits for wg-client to be healthy first. Unlike the bash, the old name is kept as a hidden deprecated alias so existing scripts and docs don't break on upgrade. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5
Upstream added four template files (compose-agent.yml, java-env.sh, the copilot addon and user-settings template). The hand-maintained expected list in embed_test.go would have kept passing without them, so it now walks cli/templates and requires the embedded tree to match file for file and byte for byte. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5
…o plans/) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5 # Conflicts: # .github/workflows/go.yml # CLAUDE.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5 # Conflicts: # internal/cli/root_test.go
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5 # Conflicts: # internal/cli/root_test.go
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5
6b: Go changes tracking the upstream merge
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #3. Pure merge of
master(the fork's mirror of VirtusLabmaster,c16d8fd..4a451ba, 24 commits) into the port branch. No conflicts, no Go changes.CI is red on purpose. The differential harness is the parity oracle for the Go port, and after this merge it reports 0/27 with all three parity suites failing — upstream shipped 17 CLI commits since our base (Copilot CLI agent,
--features strict-network, JetBrains stack plugins,restart-proxy→restart,settings.local.jsonscaffold,compose-agent.ymlsplit, pinned installers, …). The Go changes that track them follow in a separate stacked PR, one commit per upstream feature, so the mechanical merge and the substantive work can be reviewed apart.Merge order: merge the follow-up PR into this branch first, which turns this one green, then this into
go-port, then #3 intomain. See GO-PORT-PLAN.md §4.1 step 6b.🤖 Generated with Claude Code
https://claude.ai/code/session_01YKma722KMnAX3LcDLUYDQ5