From 7955cceaf9f2e64ec77c4abb10d5d46ac1bbeb8d Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Wed, 4 Nov 2026 11:55:18 +0200 Subject: [PATCH 01/20] Open the linux-arm64 workstream with its design note Co-Authored-By: Claude Opus 5.5 --- CURRENT-STATUS.md | 1 + .../2026-11-04-design-linux-arm64.md | 68 +++++++++++++++++++ .../2026-11-04-linux-arm64/CURRENT-STATUS.md | 62 +++++++++++++++++ .../intake-dispositions.md | 4 ++ .../2026-11-04-linux-arm64/intake/README.md | 9 +++ 5 files changed, 144 insertions(+) create mode 100644 engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md create mode 100644 engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md create mode 100644 engineering-docs/wip/2026-11-04-linux-arm64/intake-dispositions.md create mode 100644 engineering-docs/wip/2026-11-04-linux-arm64/intake/README.md diff --git a/CURRENT-STATUS.md b/CURRENT-STATUS.md index 33ce5a78..30d114c8 100644 --- a/CURRENT-STATUS.md +++ b/CURRENT-STATUS.md @@ -20,6 +20,7 @@ coordination branch. Detailed state belongs in each linked status file. | `website` | 2026-09-16 | Deliver the website autonomy experiment: repository-owned landing/docs/blog content, independently developable website submodule, local preview and publishing mechanism | paused 2026-09-30; producer migration merged in PR #152 and the versioned documentation live; W04-C, W09-C, W11-C retained | `ws-website/contract-v1` | [workstream status](engineering-docs/wip/2026-09-16-website/CURRENT-STATUS.md) | | `capsule-webconsole` | 2026-10-09 | Serve a token-gated web console from every capsule: configuration, versions, info, live resources, later records and workflow state | active; work order accepted 2026-10-09; the first iteration, deliverables 1 and 2, is three stacked pull requests, #184, #186 and the runtime-and-base slice, each reviewed by the Codex pair; the owner merges | `ws-capsule-webconsole/first-slice` | [workstream status](engineering-docs/wip/2026-10-09-capsule-webconsole/CURRENT-STATUS.md) | | `wrapped-projects` | 2026-11-02 | Wrap a foreign repository as a submodule of a small wrapper project and work on it in a capsule with three commands; FRC learning project first | active; PR1 (#333) accepted by the owner 2026-11-03, awaiting merge; the project-capabilities design handed to workflow-improvements | `ws-wrapped-projects/wrap` | [workstream status](engineering-docs/wip/2026-11-02-wrapped-projects/CURRENT-STATUS.md) | +| `linux-arm64` | 2026-11-04 | Run capsules as `linux-arm64` as well as `linux-amd64`: natively on Apple Silicon Macs and on ARM Linux hosts | active 2026-11-04; registered; slice 1, the second capsule platform | `ws-linux-arm64/capsules` | [workstream status](engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md) | | `website` | 2026-09-16 | Deliver the website autonomy experiment: repository-owned landing/docs/blog content, independently developable website submodule, local preview and publishing mechanism | paused 2026-09-20 at owner request to switch to maintenance; experiment accepted, A−; blog integrated and published; content/parent-integration follow-ups remain | `ws-website/initial-cut`; renamed from `website/initial-cut`; preparation and backlog-handoff exceptions in status | [workstream status](engineering-docs/wip/2026-09-16-website/CURRENT-STATUS.md) | Paused and blocked workstreams remain open until they conclude successfully or diff --git a/engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md b/engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md new file mode 100644 index 00000000..36f10f50 --- /dev/null +++ b/engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md @@ -0,0 +1,68 @@ +# Design: linux-arm64 Capsules + +Workstream: `linux-arm64`. Written 2026-11-04 from two read-only surveys of the launcher and +the base recipe, on top of `macos-launcher`'s platform split. File references are to +`devcapsule-src/`. + +## 1. What exists + +- **Resolution is already per platform.** `MATRICES` (`resolution_matrix.py`) maps each + `Platform` to one matrix of base pins, component pins, verified edges, capabilities and + materialization recipes. Locks are `devcapsule..lock`; a project may commit one + per platform, and a checkout holds one version set. +- **The base recipe is arch-neutral.** `images/base.py` renders a Dockerfile from + `ubuntu:24.04` and Ubuntu packages, which exist for arm64; Node and Temurin choose their + archive by `dpkg --print-architecture`, and Temurin's arm64 digest is already pinned. +- **Discovery already speaks arm64.** `components/discovery.py` and `npm_channel.py` map + `linux-arm64` for JetBrains, Eclipse, VSCodium, VS Code, Claude Code and npm packages. +- **What is amd64-only.** `Platform` has one member; every component pin is an amd64 + archive; Eclipse refuses other platforms (`components/eclipse.py`); Codex's npm alias map + names only `linux-amd64` (`components/codex.py`); the JetBrains runtime check names + `libskiko-linux-x64.so` (`assets/*/check-runtime-deps.sh`); `build-pex.sh` defaults to a + `linux-x86_64` scie; the published base is a single-platform amd64 manifest. + +## 2. Decisions taken by the human, 2026-11-04 + +1. Every component at once, not a first slice. +2. The arm64 base is built locally and approved per checkout; the owner publishes later. +3. On Apple Silicon, a project runs as `linux-arm64` when it has a `linux-arm64` lock, and + otherwise as `linux-amd64` under emulation. +4. A new workstream on the macOS branch. + +## 3. Design + +1. **`Platform.LINUX_ARM64`.** The capsule platform a host can run, in preference order: + `linux-amd64` hosts run amd64; `linux-arm64` hosts run arm64; Apple Silicon prefers arm64 + and can emulate amd64; Intel Macs run amd64. A project's platform is the first preferred + platform it has a lock for; a project with no lock yet takes the first. Where callers now + ask `Platform.current()` without a project, the design keeps one answer per process, + chosen when the project is known; slice 1 settles the mechanism after reading the callers. +2. **The runtime.** An arm64 capsule needs a `linux-aarch64` executable. A launcher whose own + binary runs on the capsule platform delivers itself (R-RUNTIME-001). Otherwise the user + names the runtime: `DEVCAPSULE_RUNTIME_PEX` keeps naming one file, and a directory holding + `devcapsule-linux-amd64` and `devcapsule-linux-arm64` serves both platforms from one + setting. +3. **The base.** Built natively on this Mac by `devcapsule images build --type base`, and + approved per checkout by the documented local-base path. Whether the `linux-arm64` matrix + can resolve without a published base pin is slice 2's first question. +4. **Pins.** The same versions as the amd64 matrix, each with its arm64 archive's URL and a + sha256 computed from the downloaded bytes, never copied from a page alone. Eclipse's + native closure comes from `ports.ubuntu.com`. Playwright's Chromium has no + Chrome-for-Testing arm64 build; its arm64 source is slice 3's open question, and + `browser-automation` is unavailable on arm64 with a warning (R-CONFIG-001) if none passes. + +## 4. Slices + +1. **Second platform.** `LINUX_ARM64`, the per-project selection, the runtime directory, the + `linux-aarch64` scie, the Eclipse, Codex and skiko hard-codes, tests. Done when the unit + suite passes here and on Linux, and `project init` on this Mac writes an arm64 lock + against a stub matrix. +2. **Base.** Build the arm64 base here; settle how the arm64 matrix and a local base meet. +3. **Pins.** Every component, by script: download, hash, record. +4. **Live and documentation.** `nox -s ide-smoke` for every surface natively on this Mac; + the Linux, macOS and install pages. + +## 5. What this does not decide + +- Publishing an arm64 or multi-platform base, or arm64 release executables: the owner's. +- Whether ARM Linux becomes a supported release platform: the owner's. diff --git a/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md new file mode 100644 index 00000000..f41663d2 --- /dev/null +++ b/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md @@ -0,0 +1,62 @@ +# Workstream Current Status: linux-arm64 Capsules + +Mnemonic: `linux-arm64` + +Start date: 2026-11-04 + +State: active; registered 2026-11-04; slice 1, the second capsule platform, next + +Branch association: `ws-linux-arm64/capsules`, forked from local `main` at b3f10b7d, with +`ws-macos-launcher/host-port` merged in as its foundation + +Integration target: `main`, after `macos-launcher`'s pull request + +Delivery method: pull request, merge commit; the human opens and merges it + +Requirements: R-RUNTIME-001 (the runtime delivered matches the capsule); R-COMPAT-001 +(projects locked for `linux-amd64` keep working); R-CONFIG-001 + +## Goal And Scope + +Run capsules as `linux-arm64` as well as `linux-amd64`: natively on Apple Silicon Macs +through the `macos-launcher` work, and on ARM Linux hosts. + +In scope, by the human's answers of 2026-11-04: + +- every component the `linux-amd64` matrix offers, pinned for `linux-arm64` in one + workstream: PyCharm, IntelliJ IDEA, Rider and the .NET SDK, Eclipse and its native + package closure, VSCodium, VS Code, the Codex, Claude Code and Antigravity CLIs, and + Playwright; +- an arm64 base image built locally from the existing recipe and approved per checkout as a + local base; publishing it, and pinning its digest, is the owner's; +- on an Apple Silicon Mac, a project runs as `linux-arm64` when it has a `linux-arm64` lock, + and otherwise as `linux-amd64` under emulation, as today; +- a `linux-aarch64` runtime executable for arm64 capsules. + +Out of scope: publishing images or executables; release automation. + +## Current State + +**2026-11-04, opened.** The human, running capsules on the Mac under emulation, directed +native ARM capsules: "ok we need linux arm64", then answered four questions: everything at +once; build the base locally now; arm64 when locked, else amd64; a new workstream built on +the macOS branch. Two read-only surveys mapped the work; the plan is in the +[design note](2026-11-04-design-linux-arm64.md). + +## Next Resumable Task + +Slice 1 of the design note: the second capsule platform. + +## Open Threads + +- The SSH agent on this Mac holds no identity, so the workstream cannot be published or + pushed; registration is local, as is `macos-launcher`'s. +- This branch builds on `ws-macos-launcher/host-port`, which is not yet on `main`. The + registration commit is forked from `main` as topic 7.5 requires, and the macOS branch is + merged after it; this pull request follows that one. +- An arm64 base on `docker.io/mycodespaceai`, and its digest in the matrix, need the owner. + +## Document Index + +- [2026-11-04-design-linux-arm64.md](2026-11-04-design-linux-arm64.md): what the second + platform needs, the decisions, and the slices. Open before each slice. diff --git a/engineering-docs/wip/2026-11-04-linux-arm64/intake-dispositions.md b/engineering-docs/wip/2026-11-04-linux-arm64/intake-dispositions.md new file mode 100644 index 00000000..c67569ac --- /dev/null +++ b/engineering-docs/wip/2026-11-04-linux-arm64/intake-dispositions.md @@ -0,0 +1,4 @@ +# Intake Decisions: `linux-arm64` + +Record each removed intake item, its decision, reasoning, and resulting +status file requirement or task here. diff --git a/engineering-docs/wip/2026-11-04-linux-arm64/intake/README.md b/engineering-docs/wip/2026-11-04-linux-arm64/intake/README.md new file mode 100644 index 00000000..69ba7553 --- /dev/null +++ b/engineering-docs/wip/2026-11-04-linux-arm64/intake/README.md @@ -0,0 +1,9 @@ +# Intake: `linux-arm64` + +Work other workstreams have delivered to `linux-arm64` and that it has not +yet decided. Senders add one file per item, named +`YYYY-MM-DD--.md`, and deliver it by mail on the +coordination branch, never through `main` or by editing this directory. Each +item is acknowledged into the status file or forwarded as `WORKFLOW.md` +permits, and every decision is recorded in `intake-dispositions.md` before the +item is removed. From c5d627823cc1c502549d42b2db6b855a8a206ec6 Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Tue, 3 Nov 2026 20:28:19 +0200 Subject: [PATCH 02/20] Open the macos-launcher workstream with its design note Run the launcher natively on Apple Silicon Macs against Docker Desktop, capsules as linux-amd64 under emulation, at the human's direction of 2026-11-03. The design note records the host assumptions found and the slice plan. Co-authored-by: Claude Opus 5.5 --- CURRENT-STATUS.md | 1 + .../2026-11-03-design-macos-host-port.md | 76 +++++++++++++++++++ .../CURRENT-STATUS.md | 64 ++++++++++++++++ .../intake-dispositions.md | 4 + .../intake/README.md | 9 +++ index.md | 1 + 6 files changed, 155 insertions(+) create mode 100644 engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md create mode 100644 engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md create mode 100644 engineering-docs/wip/2026-11-03-macos-launcher/intake-dispositions.md create mode 100644 engineering-docs/wip/2026-11-03-macos-launcher/intake/README.md diff --git a/CURRENT-STATUS.md b/CURRENT-STATUS.md index 30d114c8..5ad089a8 100644 --- a/CURRENT-STATUS.md +++ b/CURRENT-STATUS.md @@ -20,6 +20,7 @@ coordination branch. Detailed state belongs in each linked status file. | `website` | 2026-09-16 | Deliver the website autonomy experiment: repository-owned landing/docs/blog content, independently developable website submodule, local preview and publishing mechanism | paused 2026-09-30; producer migration merged in PR #152 and the versioned documentation live; W04-C, W09-C, W11-C retained | `ws-website/contract-v1` | [workstream status](engineering-docs/wip/2026-09-16-website/CURRENT-STATUS.md) | | `capsule-webconsole` | 2026-10-09 | Serve a token-gated web console from every capsule: configuration, versions, info, live resources, later records and workflow state | active; work order accepted 2026-10-09; the first iteration, deliverables 1 and 2, is three stacked pull requests, #184, #186 and the runtime-and-base slice, each reviewed by the Codex pair; the owner merges | `ws-capsule-webconsole/first-slice` | [workstream status](engineering-docs/wip/2026-10-09-capsule-webconsole/CURRENT-STATUS.md) | | `wrapped-projects` | 2026-11-02 | Wrap a foreign repository as a submodule of a small wrapper project and work on it in a capsule with three commands; FRC learning project first | active; PR1 (#333) accepted by the owner 2026-11-03, awaiting merge; the project-capabilities design handed to workflow-improvements | `ws-wrapped-projects/wrap` | [workstream status](engineering-docs/wip/2026-11-02-wrapped-projects/CURRENT-STATUS.md) | +| `macos-launcher` | 2026-11-03 | Run the DevCapsule launcher natively on Apple Silicon Macs against Docker Desktop, with capsules as `linux-amd64` under emulation | active 2026-11-03; registered; slice 1, the capsule-platform split | `ws-macos-launcher/host-port` | [workstream status](engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md) | | `linux-arm64` | 2026-11-04 | Run capsules as `linux-arm64` as well as `linux-amd64`: natively on Apple Silicon Macs and on ARM Linux hosts | active 2026-11-04; registered; slice 1, the second capsule platform | `ws-linux-arm64/capsules` | [workstream status](engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md) | | `website` | 2026-09-16 | Deliver the website autonomy experiment: repository-owned landing/docs/blog content, independently developable website submodule, local preview and publishing mechanism | paused 2026-09-20 at owner request to switch to maintenance; experiment accepted, A−; blog integrated and published; content/parent-integration follow-ups remain | `ws-website/initial-cut`; renamed from `website/initial-cut`; preparation and backlog-handoff exceptions in status | [workstream status](engineering-docs/wip/2026-09-16-website/CURRENT-STATUS.md) | diff --git a/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md b/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md new file mode 100644 index 00000000..245f375b --- /dev/null +++ b/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md @@ -0,0 +1,76 @@ +# Design: The Launcher On macOS + +Workstream: `macos-launcher`. Written 2026-11-03 from a read of the launcher and probes on +an Apple Silicon Mac with Docker Desktop 29.3.1. Statements marked *to verify* are read from +code or recalled Docker Desktop behaviour, not yet observed on this host. + +## 1. The two platforms + +Today one value plays two roles. `Platform.current()` (`devcapsule/platforms.py`) derives a +key from the host's `system` and `machine`, and every caller uses it to name and select the +platform lock and the resolution matrix: + +- `configuration/storage.py`: the lock path `devcapsule..lock`, and the refusal of a + version set recorded for another platform; +- `configuration/derivation.py`, `configuration/operations.py`, + `configuration/owner_commands.py`, `version_sets.py`: `MATRICES[Platform.current()]`. + +None of those callers asks about the host. They ask which platform the capsules run as. On +Linux the two coincide. On a Mac the host is `darwin-arm64` and the capsules are Linux. + +Design: + +1. `Platform` keeps its meaning, the wire format of a lock: a capsule platform. Its members + stay append-only; no `darwin-*` member is added, because no lock is ever for macOS. +2. `Platform.current()` keeps its name and its callers, and its contract becomes *the + platform this host runs capsules as*. On Linux it is the host's own platform, as now. On + macOS it is `linux-amd64` on both architectures: Intel natively, Apple Silicon under + Docker's emulation. When `linux-arm64` exists, Apple Silicon maps to it and emulation + becomes the fallback, not the rule. The mapping lives in `platforms.py` alone. +3. A new `HostOs` (`linux`, `darwin`) names the host's operating system for the host-side + mechanisms of section 2. Feature code asks `HostOs.current()`, never `sys.platform`. +4. `Platform.docker_platform` gives Docker's spelling (`linux/amd64`), and + `Platform.emulated()` says whether this host's CPU differs from the capsule platform's. + When it does, every `docker run` and every buildx build the launcher composes carries + `--platform`, so that Docker does not pick the host's architecture for a multi-platform + image or warn on a single-platform one. + +Rejected: renaming `current()` to `for_capsules()`. The name is accurate once the contract is +written down, and the rename touches 27 call and test sites for no behaviour. + +## 2. Host mechanisms that assume Linux + +Found by reading the launch path from `run_pycharm` (`launch/pycharm/_launcher.py`) outward. + +| Mechanism | Where | On macOS | Plan | +|---|---|---|---| +| Lock and matrix key | section 1 | refuses `darwin-arm64` | slice 1 | +| Docker `--platform` | `_launcher.py` `docker run`; `images/build.py` buildx | Docker picks arm64 or warns | slice 1 | +| Host-browser bridge | `host_open.py` | peer check uses `SO_PEERCRED`, Linux only; the broker socket is a host Unix socket bind-mounted into the capsule, which Docker Desktop's file sharing does not carry (*to verify*); opener is `xdg-open` | slice 2: on macOS the bridge is an unavailable optional integration, with a warning, per R-CONFIG-001; the opener is `open`; a TCP transport is a later decision | +| Host Docker socket | `_launcher.py` `host_docker_socket`, `append_docker_mode_args` | the context socket `~/.docker/run/docker.sock` is a host path; the bind source must be the VM's `/var/run/docker.sock`; `--group-add` takes the host socket's gid, which means nothing inside the VM (*to verify*) | slice 3 | +| Container identity | `current_host_user`, `project_runtime_plan.py` | the capsule user becomes 501:20; gid 20 is `dialout` in Ubuntu; ownership through Docker Desktop's file sharing differs from Linux bind mounts (*to verify*) | slice 3 | +| Container detection | `host_daemon.py` reads `/proc/self/mountinfo` | absent on macOS; must read as "not in a container" | slice 1, with a test | +| Executable | `scripts/build-pex.sh` | `DEVCAPSULE_SCIE_PLATFORM=macos-aarch64` | slice 4 | +| Ports | `--publish 127.0.0.1:` | Docker Desktop forwards loopback publishes (*to verify*) | slice 3, by a live launch | +| Project's host networking rule | `WORKFLOW-LOCAL.md` *Local Launch Networking* | Docker Desktop's host networking is an opt-in setting (*to verify*) | record the fallback when a launch needs it | + +## 3. Slices + +1. **Capsule platform.** Section 1, `--platform` on run and build, `/proc` absence. Done when + `project config show` and `project run --print-command` answer on this Mac, and the unit + tests pass here and keep passing on Linux. +2. **Browser bridge.** Unavailable-with-warning on macOS; `open` as the opener for when a + transport exists. +3. **Live launch.** A real `project run` of this repository on this Mac: Docker socket, + identity, ownership of written files, ports, the contained desktop in a Mac browser. + Findings decide the code; this slice is mostly evidence. +4. **Executable and documentation.** A `macos-aarch64` scie, smoke-tested here; the macOS + platform page and the install page rewritten for what works; release automation is the + owner's call. + +## 4. What this does not decide + +- Whether macOS becomes a *supported* platform in a release. That is the owner's ruling; + this workstream produces the evidence for it. +- Signing and notarization of a macOS executable. +- `linux-arm64` capsules. diff --git a/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md new file mode 100644 index 00000000..8129afd0 --- /dev/null +++ b/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md @@ -0,0 +1,64 @@ +# Workstream Current Status: macOS Launcher + +Mnemonic: `macos-launcher` + +Start date: 2026-11-03 + +State: active; registered 2026-11-03; slice 1, the capsule-platform split, in progress + +Branch association: `ws-macos-launcher/host-port`, forked from local `main` at b3f10b7d + +Integration target: `main` + +Delivery method: pull request, merge commit; the human opens and merges it + +Requirements: R-PRODUCT-002 (host boundaries stay explicit on the new host); R-CONFIG-001 + +## Goal And Scope + +Run the DevCapsule launcher natively on an Apple Silicon Mac (`darwin-arm64`) against +Docker Desktop, with capsules running as `linux-amd64` under Docker's x86 emulation. + +In scope: the launcher on macOS, the existing `linux-amd64` lock, pins and base image used +unchanged, the host-side mechanisms that assume Linux, a `macos-aarch64` executable, and the +macOS platform page. + +Out of scope, by the human's direction of 2026-11-03 ("do stage 2 no need for stage 1"): +`linux-arm64` capsules, their resolution matrix, pins and base image. Native-speed +capsules on Apple Silicon need that work later; this workstream leaves room for it. + +## Current State + +**2026-11-03, opened.** The human asked whether DevCapsule builds for ARM, then for native +macOS on Apple Silicon, and directed the macOS host port without ARM capsules. Evidence on +the human's Mac (macOS 27.0, arm64, Docker Desktop 29.3.1, `desktop-linux` context at +`~/.docker/run/docker.sock`, 14 CPUs, 47 GiB): + +- `docker run --platform linux/amd64 ubuntu:24.04 uname -m` prints `x86_64`: emulation works. +- The locked base `devcapsule-base@sha256:8c97d308…` is a single-platform manifest. +- The development CLI runs (`version --json` answers), and every project command stops at + `Unsupported platform 'darwin-arm64'; supported platforms: linux-amd64.` + +The work plan, one slice each, is in the +[design note](2026-11-03-design-macos-host-port.md). + +## Next Resumable Task + +Slice 1: separate the capsule platform, which keys locks and matrices, from the host +operating system, and pass `--platform linux/amd64` to Docker when they differ. + +## Open Threads + +- The SSH agent on this Mac holds no identity, so `git fetch`, `workflow claim` and + `workflow publish` fail with `Permission denied (publickey)`. The human loads the key; + until then the registration is local only and `main` could not be re-verified. +- No `python3.12` exists on this Mac. `.venv` was created with `uv venv --python 3.12` + (CPython 3.12.12) instead of `python3.12 -m venv`; the executable still embeds 3.12.14. +- Adding a supported host platform is a product commitment. The repository owner has not + yet ruled on it; the human started the work. + +## Document Index + +- [2026-11-03-design-macos-host-port.md](2026-11-03-design-macos-host-port.md): the host + assumptions found, the design of the platform split, and the slice plan. Open before + each slice. diff --git a/engineering-docs/wip/2026-11-03-macos-launcher/intake-dispositions.md b/engineering-docs/wip/2026-11-03-macos-launcher/intake-dispositions.md new file mode 100644 index 00000000..e11d9eb8 --- /dev/null +++ b/engineering-docs/wip/2026-11-03-macos-launcher/intake-dispositions.md @@ -0,0 +1,4 @@ +# Intake Decisions: `macos-launcher` + +Record each removed intake item, its decision, reasoning, and resulting +status file requirement or task here. diff --git a/engineering-docs/wip/2026-11-03-macos-launcher/intake/README.md b/engineering-docs/wip/2026-11-03-macos-launcher/intake/README.md new file mode 100644 index 00000000..34538b70 --- /dev/null +++ b/engineering-docs/wip/2026-11-03-macos-launcher/intake/README.md @@ -0,0 +1,9 @@ +# Intake: `macos-launcher` + +Work other workstreams have delivered to `macos-launcher` and that it has not +yet decided. Senders add one file per item, named +`YYYY-MM-DD--.md`, and deliver it by mail on the +coordination branch, never through `main` or by editing this directory. Each +item is acknowledged into the status file or forwarded as `WORKFLOW.md` +permits, and every decision is recorded in `intake-dispositions.md` before the +item is removed. diff --git a/index.md b/index.md index 5698399e..59eb3188 100644 --- a/index.md +++ b/index.md @@ -178,6 +178,7 @@ status file; internal WIP/archive documents use the local index in that status. - [Maintenance current status](engineering-docs/wip/2026-09-18-maintenance/CURRENT-STATUS.md) - [Capsule web console current status](engineering-docs/wip/2026-10-09-capsule-webconsole/CURRENT-STATUS.md) - [Wrapped projects current status](engineering-docs/wip/2026-11-02-wrapped-projects/CURRENT-STATUS.md) +- [macOS launcher current status](engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md) - [Sample demo projects current status](engineering-docs/wip/2026-08-14-sample-projects/CURRENT-STATUS.md) - [Contained display current status](engineering-docs/wip/2026-08-19-contained-display/CURRENT-STATUS.md) From 0914467fc4bc3c5ae4c635ae3daae78c89210e36 Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Tue, 3 Nov 2026 20:46:43 +0200 Subject: [PATCH 03/20] Run the launcher on macOS against linux-amd64 capsules Platform.current() now names the platform this host runs capsules as, linux-amd64 on a Mac, and HostOs names the host for host mechanisms. Docker runs, builds and pulls carry --platform when the host emulates the capsule platform; native Linux commands are unchanged. A launcher whose own binary cannot run in the capsule delivers the runtime that DEVCAPSULE_RUNTIME_PEX names, or refuses with that remedy. On macOS the host-browser bridge is unavailable with a warning, since Docker Desktop refuses a host Unix socket as a bind source; its peer check reads LOCAL_PEERCRED and two macOS socket races are fixed. The capsule user joins group 0 for Docker Desktop's socket. The test suite gets a short temporary root on macOS and skips container-side and X11-dependent tests there. Co-authored-by: Claude Opus 5.5 --- .../devcapsule/environment_realization.py | 7 +- devcapsule-src/devcapsule/host_open.py | 53 +++++++++-- devcapsule-src/devcapsule/images/build.py | 4 + .../devcapsule/launch/pycharm/_launcher.py | 16 +++- devcapsule-src/devcapsule/platforms.py | 91 ++++++++++++++++--- .../devcapsule/recursive_successor_plan.py | 8 +- devcapsule-src/devcapsule/runtime_artifact.py | 57 +++++++++++- devcapsule-src/tests/conftest.py | 14 +++ .../tests/test_environment_realization.py | 15 +++ devcapsule-src/tests/test_host_open.py | 15 +++ devcapsule-src/tests/test_platforms.py | 44 ++++++++- devcapsule-src/tests/test_pycharm.py | 44 +++++++++ .../tests/test_recursive_orchestrator.py | 10 ++ devcapsule-src/tests/test_runtime_artifact.py | 52 ++++++++++- devcapsule-src/tests/test_supervisor.py | 4 + .../2026-11-03-design-macos-host-port.md | 33 ++++++- .../CURRENT-STATUS.md | 23 ++++- 17 files changed, 452 insertions(+), 38 deletions(-) diff --git a/devcapsule-src/devcapsule/environment_realization.py b/devcapsule-src/devcapsule/environment_realization.py index a26126e9..43edc8f7 100644 --- a/devcapsule-src/devcapsule/environment_realization.py +++ b/devcapsule-src/devcapsule/environment_realization.py @@ -10,6 +10,7 @@ from python_on_whales.exceptions import DockerException from devcapsule.compat import CliError +from devcapsule.platforms import Platform from devcapsule.images.build import BuildxImageBuilder, ImageBuildSpec from devcapsule.images.metadata import inspect_local_image from devcapsule.materialization import ( @@ -181,7 +182,11 @@ def realize_environment( def ensure_local_image(reference: str) -> ImageDetails: try: if not docker.image.exists(reference): - docker.image.pull(reference) + capsule = Platform.current() + if capsule.emulated(): + docker.image.pull(reference, platform=capsule.docker_platform) + else: + docker.image.pull(reference) except DockerException as exc: raise CliError(f"Cannot obtain Docker image {reference!r}: {exc}") from exc return required_local_image(reference) diff --git a/devcapsule-src/devcapsule/host_open.py b/devcapsule-src/devcapsule/host_open.py index db25b8e0..e6ea1e07 100644 --- a/devcapsule-src/devcapsule/host_open.py +++ b/devcapsule-src/devcapsule/host_open.py @@ -5,6 +5,7 @@ from collections import deque from collections.abc import Iterator, Mapping, Sequence from contextlib import contextmanager +import errno import json import os from pathlib import Path @@ -12,6 +13,7 @@ import stat import struct import subprocess +import sys import tempfile from threading import Event, Thread import time @@ -19,6 +21,7 @@ from urllib.parse import urlsplit from devcapsule.host_daemon import in_container +from devcapsule.platforms import HostOs HOST_OPEN_SOCKET_ENV = "DEVCAPSULE_HOST_OPEN_SOCKET" @@ -32,6 +35,13 @@ RATE_LIMIT_REQUESTS = 30 RATE_LIMIT_WINDOW_SECONDS = 60.0 _UNIX_PATH_LIMIT = 100 +HOST_OPEN_UNAVAILABLE_ON_MACOS = ( + "devcapsule: warning: host-browser is unavailable on macOS: Docker Desktop does not carry the " + "host's Unix sockets into a capsule. Links open in the capsule's own browser instead." +) +# macOS: SOL_LOCAL, and the leading fields of struct xucred (cr_version, cr_uid). +_SOL_LOCAL = 0 +_XUCRED_SIZE = 76 class HostOpenError(Exception): @@ -79,7 +89,13 @@ def open_host_url( connection.settimeout(timeout) connection.connect(str(path)) connection.sendall(request) - connection.shutdown(socket.SHUT_WR) + try: + connection.shutdown(socket.SHUT_WR) + except OSError as exc: + # macOS refuses shutdown once the broker has answered and + # closed; the answer is still buffered and is read below. + if exc.errno != errno.ENOTCONN: + raise response = _read_frame(connection, MAX_RESPONSE_BYTES) except (OSError, TimeoutError) as exc: raise HostOpenError("host-browser broker is unavailable") from exc @@ -188,6 +204,7 @@ def _handle(self, connection: socket.socket) -> None: detail = str(exc) or "request rejected" try: connection.sendall(_encode_frame({"ok": False, "error": detail})) + _drain(connection) except OSError: pass @@ -244,6 +261,13 @@ def host_open_bridge( # It may only propagate a bridge explicitly inherited from its owner. yield None return + if HostOs.current() is not HostOs.LINUX: + # The broker is a host Unix socket bound into the capsule. Docker + # Desktop refuses that bind source (observed on 29.3.1, macOS 27), so + # the optional integration is unavailable rather than fatal. + print(HOST_OPEN_UNAVAILABLE_ON_MACOS, file=sys.stderr) + yield None + return parent = _runtime_parent(environ) with _broker_socket_path(parent) as path: @@ -288,11 +312,15 @@ def _broker_socket_path(parent: str | None) -> Iterator[Path]: def _peer_uid(connection: socket.socket) -> int: - if not hasattr(socket, "SO_PEERCRED"): - raise HostOpenError("peer credentials are unavailable on this platform") - credentials = connection.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, struct.calcsize("3i")) - _pid, uid, _gid = struct.unpack("3i", credentials) - return uid + if hasattr(socket, "SO_PEERCRED"): + credentials = connection.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, struct.calcsize("3i")) + _pid, uid, _gid = struct.unpack("3i", credentials) + return uid + if hasattr(socket, "LOCAL_PEERCRED"): + credentials = connection.getsockopt(_SOL_LOCAL, socket.LOCAL_PEERCRED, _XUCRED_SIZE) + _version, uid = struct.unpack_from("=II", credentials) + return uid + raise HostOpenError("peer credentials are unavailable on this platform") def _is_socket(path: Path) -> bool: @@ -308,6 +336,19 @@ def _encode_frame(value: Mapping[str, Any]) -> bytes: ).encode("utf-8") +def _drain(connection: socket.socket) -> None: + """Consume what a rejected client sent, up to its end or the frame limit. + + Closing a socket with unread input resets the connection on macOS, and + the client then loses the rejection it was just sent. + """ + + connection.shutdown(socket.SHUT_WR) + remaining = MAX_FRAME_BYTES + while remaining > 0 and (chunk := connection.recv(min(4096, remaining))): + remaining -= len(chunk) + + def _read_frame(connection: socket.socket, limit: int) -> bytes: chunks: list[bytes] = [] size = 0 diff --git a/devcapsule-src/devcapsule/images/build.py b/devcapsule-src/devcapsule/images/build.py index 6dc09616..b43cb9ff 100644 --- a/devcapsule-src/devcapsule/images/build.py +++ b/devcapsule-src/devcapsule/images/build.py @@ -19,6 +19,7 @@ from python_on_whales.exceptions import DockerException from devcapsule.compat import CliError, copy_resource_tree +from devcapsule.platforms import Platform @dataclass(frozen=True) @@ -266,6 +267,9 @@ def build(self, spec: ImageBuildSpec, *, network: str = "default") -> None: file=rendered.dockerfile, tags=[plan.image], load=True, + # Native hosts build as before; an emulating host must + # build for the capsule platform, not its own CPU. + platforms=[capsule.docker_platform] if (capsule := Platform.current()).emulated() else None, network=network, build_contexts={name: str(path) for name, path in rendered.named_contexts.items()}, allow=["network.host"] if network == "host" else [], diff --git a/devcapsule-src/devcapsule/launch/pycharm/_launcher.py b/devcapsule-src/devcapsule/launch/pycharm/_launcher.py index 2109ef40..907a8104 100644 --- a/devcapsule-src/devcapsule/launch/pycharm/_launcher.py +++ b/devcapsule-src/devcapsule/launch/pycharm/_launcher.py @@ -22,6 +22,7 @@ from typing import Callable, Mapping, Sequence from ...docker_arguments import inline_environment_names, redact_environment +from ...platforms import HostOs, Platform from ...container_runtime.contract import ( CONTAINED_DISPLAY_TRANSPORT, INFRASTRUCTURE_FAILURE_EXIT_CODE, @@ -299,6 +300,13 @@ class PycharmRunConfig: libgl_dri3_disable: str = "1" +# Docker Desktop presents its socket inside every container as root:root, +# mode 0660, whatever the macOS file's owner (observed on 29.3.1). The capsule +# user joins group 0 to use it. A capsule granted the host daemon already +# holds root on Docker Desktop's VM, so the group adds no authority. +DOCKER_DESKTOP_SOCKET_GID = 0 + + def run_pycharm(options: PycharmRunOptions, env: Mapping[str, str] | None = None) -> int: """Run PyCharm through the translated Python launcher.""" @@ -957,7 +965,10 @@ def build_run_config(options: PycharmRunOptions, env: Mapping[str, str]) -> Pych "Start Docker on the host, set HOST_DOCKER_SOCKET, or launch with --docker-in-docker / --no-docker." ) host_docker_socket = host_docker_socket.resolve() - host_docker_gid = host_docker_socket.stat().st_gid + host_docker_gid = ( + DOCKER_DESKTOP_SOCKET_GID if HostOs.current() is HostOs.DARWIN + else host_docker_socket.stat().st_gid + ) if enable_sudo: while ide_sudo_gid in {0, host_user.gid, host_docker_gid}: @@ -1293,6 +1304,7 @@ def build_docker_args( args = [ "--rm", "-i", + *Platform.current().docker_platform_args(), "--name", config.name, "--workdir", @@ -1816,7 +1828,7 @@ def write_user_files(config: PycharmRunConfig, files: TempRuntimeFiles) -> None: "root:x:0:", f"{host_user.group_name}:x:{host_user.gid}:", ] - if config.host_docker_gid is not None and config.host_docker_gid != host_user.gid: + if config.host_docker_gid not in (None, 0, host_user.gid): group_lines.append(f"host-docker:x:{config.host_docker_gid}:") if config.enable_sudo: group_lines.append(f"ide-sudo:x:{config.ide_sudo_gid}:{host_user.name}") diff --git a/devcapsule-src/devcapsule/platforms.py b/devcapsule-src/devcapsule/platforms.py index 2c1cbfa4..8e1f4d96 100644 --- a/devcapsule-src/devcapsule/platforms.py +++ b/devcapsule-src/devcapsule/platforms.py @@ -20,36 +20,101 @@ import platform as host_platform from typing import Mapping -__all__ = ["Platform", "UnsupportedPlatformError", "XdgHomes"] +__all__ = ["HostOs", "Platform", "UnsupportedPlatformError", "XdgHomes", "host_architecture"] class UnsupportedPlatformError(ValueError): """A platform key outside the supported set; the message is displayable.""" +def host_architecture() -> str: + """This host's CPU architecture in the platform keys' spelling.""" + + machine = host_platform.machine().lower() + return {"x86_64": "amd64", "aarch64": "arm64"}.get(machine, machine) + + +class HostOs(StrEnum): + """The host operating system, for the mechanisms that differ by host. + + Locks never carry it: capsules are Linux on every host, and the lock + names the capsule platform (``Platform``). + """ + + LINUX = "linux" + DARWIN = "darwin" + + @classmethod + def current(cls) -> HostOs: + system = host_platform.system().lower() + try: + return cls(system) + except ValueError: + supported = ", ".join(member.value for member in cls) + raise UnsupportedPlatformError( + f"Unsupported host operating system {system!r}; supported: {supported}." + ) from None + + +# The capsule platform a non-Linux host runs. Apple Silicon runs linux-amd64 +# under Docker's emulation until a linux-arm64 platform exists. +_CAPSULE_PLATFORM_OF_HOST = { + "darwin-arm64": "linux-amd64", + "darwin-amd64": "linux-amd64", +} + + class Platform(StrEnum): - """The supported-platform declaration and its wire format. + """The supported capsule platforms and their wire format. - Members are the single authority on what platforms DevCapsule supports; - values are the wire format serialized into lock filenames, lock - documents, and artifact tables. Because committed locks carry these - values, members are append-only and never renamed. + A platform is what capsules run as, not what the launcher runs on: a + Mac launches Linux capsules. Members are the single authority on what + capsule platforms DevCapsule supports; values are the wire format + serialized into lock filenames, lock documents, and artifact tables. + Because committed locks carry these values, members are append-only and + never renamed. A client never constructs a key from parts: it obtains one from - ``current()`` (the platform it is running on) or ``parse()`` (the - platform a lock records), and only carries it between those points. + ``current()`` (the platform this host runs capsules as) or ``parse()`` + (the platform a lock records), and only carries it between those points. """ LINUX_AMD64 = "linux-amd64" @classmethod def current(cls) -> Platform: - """The platform this process is running on.""" + """The platform this host runs capsules as. - system = host_platform.system().lower() - machine = host_platform.machine().lower() - architecture = {"x86_64": "amd64", "aarch64": "arm64"}.get(machine, machine) - return cls.parse(f"{system}-{architecture}") + On Linux, the host's own platform. On macOS, the Linux platform + Docker Desktop runs, natively or under emulation. + """ + + host = f"{host_platform.system().lower()}-{host_architecture()}" + return cls.parse(_CAPSULE_PLATFORM_OF_HOST.get(host, host)) + + @property + def architecture(self) -> str: + return self.value.split("-", 1)[1] + + @property + def docker_platform(self) -> str: + """Docker's spelling of this platform, as ``--platform`` takes it.""" + + return self.value.replace("-", "/", 1) + + def emulated(self) -> bool: + """Whether this host's CPU runs capsules of this platform only under emulation.""" + + return host_architecture() != self.architecture + + def docker_platform_args(self) -> list[str]: + """The ``--platform`` option every run and build carries under emulation. + + Empty on a native host, so a Linux launch composes the same command + as before the option existed. + """ + + return [f"--platform={self.docker_platform}"] if self.emulated() else [] @classmethod def parse(cls, key: str) -> Platform: diff --git a/devcapsule-src/devcapsule/recursive_successor_plan.py b/devcapsule-src/devcapsule/recursive_successor_plan.py index b400b87e..241db940 100644 --- a/devcapsule-src/devcapsule/recursive_successor_plan.py +++ b/devcapsule-src/devcapsule/recursive_successor_plan.py @@ -43,9 +43,11 @@ } ) _BOOLEAN_FLAGS = frozenset({"--detach", "--privileged", "--read-only", "--rm", "-i"}) -# Flags that describe how Docker obtains the image rather than the resulting -# container. They are deliberately not part of the comparable plan. -_IGNORED_FLAGS = frozenset({"--pull"}) +# Flags that describe how Docker obtains or executes the image rather than the +# resulting container. They are deliberately not part of the comparable plan. +# The launcher emits each as one inline token (``--platform=linux/amd64`` on +# a host that emulates the capsule platform). +_IGNORED_FLAGS = frozenset({"--platform", "--pull"}) class SuccessorPlanError(Exception): diff --git a/devcapsule-src/devcapsule/runtime_artifact.py b/devcapsule-src/devcapsule/runtime_artifact.py index 0dc7353a..64338b0a 100644 --- a/devcapsule-src/devcapsule/runtime_artifact.py +++ b/devcapsule-src/devcapsule/runtime_artifact.py @@ -8,15 +8,32 @@ from devcapsule.build_info import BuildInfoError, read_pex_build_info from devcapsule.compat import CliError +from devcapsule.platforms import Platform + +RUNTIME_PEX_ENV = "DEVCAPSULE_RUNTIME_PEX" + +# ELF e_machine values, by the architecture spelling of platform keys. +_ELF_MACHINES = {0x3E: "amd64", 0xB7: "arm64"} +_MACH_O_MAGICS = (b"\xcf\xfa\xed\xfe", b"\xce\xfa\xed\xfe", b"\xca\xfe\xba\xbe") def runtime_artifact() -> Path: + """The DevCapsule executable this launch delivers into the capsule. + + A packaged launcher delivers itself when its binary runs on the capsule + platform. A launcher that cannot, a macOS executable launching Linux + capsules, delivers the runtime ``DEVCAPSULE_RUNTIME_PEX`` names. A + source-form launcher must name its built runtime the same way. + """ + + capsule = Platform.current() # scie preserves the executable's path in SCIE; PEX points to the original # archive too. sys.argv[0] alone can name the extracted console script. - # A source-form launcher must explicitly select its built runtime. - selected = (os.environ.get("SCIE") or os.environ.get("PEX") - or os.environ.get("DEVCAPSULE_RUNTIME_PEX") or sys.argv[0]) - path = Path(selected).expanduser().resolve() + own = os.environ.get("SCIE") or os.environ.get("PEX") + explicit = os.environ.get(RUNTIME_PEX_ENV) + path = _resolved(own or explicit or sys.argv[0]) + if own and explicit and executable_platform(path) not in (None, str(capsule)): + path = _resolved(explicit) try: read_pex_build_info(path) except BuildInfoError as exc: @@ -25,4 +42,36 @@ def runtime_artifact() -> Path: "or build with 'nox -s pex' and set DEVCAPSULE_RUNTIME_PEX to its path " "when running from source." ) from exc + found = executable_platform(path) + if found not in (None, str(capsule)): + raise CliError( + f"{path} is a {found} executable and cannot run inside {capsule} capsules. " + f"Set {RUNTIME_PEX_ENV} to a {capsule} DevCapsule executable of the same version." + ) return path + + +def executable_platform(path: Path) -> str | None: + """The platform a native executable runs on, from its header. + + ``darwin`` for any Mach-O file, ``linux-`` for an ELF file of a + known architecture, and None for anything else, such as a plain PEX + archive, whose interpreter the capsule supplies. + """ + + try: + with path.open("rb") as handle: + header = handle.read(20) + except OSError: + return None + if header[:4] in _MACH_O_MAGICS: + return "darwin" + if header[:4] == b"\x7fELF" and len(header) >= 20: + machine = int.from_bytes(header[18:20], "little" if header[5] == 1 else "big") + architecture = _ELF_MACHINES.get(machine) + return f"linux-{architecture}" if architecture else "linux-unknown" + return None + + +def _resolved(selected: str) -> Path: + return Path(selected).expanduser().resolve() diff --git a/devcapsule-src/tests/conftest.py b/devcapsule-src/tests/conftest.py index 5c50403e..d34213bd 100644 --- a/devcapsule-src/tests/conftest.py +++ b/devcapsule-src/tests/conftest.py @@ -2,10 +2,24 @@ import os from pathlib import Path +import sys +import tempfile import pytest +def pytest_configure(config: pytest.Config) -> None: + """Give a macOS run a short temporary root. + + macOS limits a Unix socket path to 104 bytes, and its per-user temporary + directory under /var/folders leaves too little of that for the sockets + the tests bind beneath tmp_path. An explicit --basetemp still wins. + """ + + if sys.platform == "darwin" and not config.option.basetemp: + config.option.basetemp = tempfile.mkdtemp(prefix="dcpt-", dir="/tmp") + + @pytest.fixture(autouse=True) def host_launch_by_default(monkeypatch: pytest.MonkeyPatch) -> None: """Make launcher tests behave as if they run on the host. diff --git a/devcapsule-src/tests/test_environment_realization.py b/devcapsule-src/tests/test_environment_realization.py index c5cec44b..d5178715 100644 --- a/devcapsule-src/tests/test_environment_realization.py +++ b/devcapsule-src/tests/test_environment_realization.py @@ -266,6 +266,7 @@ def test_realize_environment_allows_terms_bound_claude_download_authorization( def test_ensure_local_image_pulls_only_when_missing(already_local: bool) -> None: details = base_image() with ( + patch("devcapsule.platforms.host_architecture", return_value="amd64"), patch( "devcapsule.environment_realization.docker.image.exists", return_value=already_local, @@ -285,6 +286,20 @@ def test_ensure_local_image_pulls_only_when_missing(already_local: bool) -> None require.assert_called_once_with(BASE_REFERENCE) +def test_an_emulating_host_pulls_the_capsule_platform() -> None: + # Apple Silicon runs linux-amd64 capsules; the pull must not resolve a + # multi-platform reference to the host's arm64 image. + with ( + patch("devcapsule.platforms.host_platform.system", return_value="Darwin"), + patch("devcapsule.platforms.host_architecture", return_value="arm64"), + patch("devcapsule.environment_realization.docker.image.exists", return_value=False), + patch("devcapsule.environment_realization.docker.image.pull") as pull, + patch("devcapsule.environment_realization.required_local_image", return_value=base_image()), + ): + ensure_local_image(BASE_REFERENCE) + pull.assert_called_once_with(BASE_REFERENCE, platform="linux/amd64") + + def test_realize_environment_allows_explicit_managed_base_override(tmp_path: Path) -> None: selected = resolved_project(tmp_path, authorized=False) local_reference = "local/devcapsule-base:test" diff --git a/devcapsule-src/tests/test_host_open.py b/devcapsule-src/tests/test_host_open.py index 7d623d59..3d58f0b6 100644 --- a/devcapsule-src/tests/test_host_open.py +++ b/devcapsule-src/tests/test_host_open.py @@ -11,6 +11,7 @@ import pytest from devcapsule import cli +from devcapsule.platforms import HostOs from devcapsule.host_open import ( HOST_OPEN_SOCKET_ENV, MAX_URL_BYTES, @@ -175,6 +176,7 @@ def test_physical_host_bridge_owns_and_cleans_broker(tmp_path: Path) -> None: with ( patch("devcapsule.host_open.in_container", return_value=False), + patch("devcapsule.host_open.HostOs.current", return_value=HostOs.LINUX), patch("devcapsule.host_open.subprocess.run", run), host_open_bridge(env, enabled=True, opener=("test-opener",)) as path, ): @@ -186,6 +188,19 @@ def test_physical_host_bridge_owns_and_cleans_broker(tmp_path: Path) -> None: assert not root.exists() +def test_macos_host_bridge_is_unavailable_with_a_warning(tmp_path: Path, capsys) -> None: + # Docker Desktop refuses a host Unix socket as a bind source, so the + # optional integration degrades instead of failing the launch. + with ( + patch("devcapsule.host_open.in_container", return_value=False), + patch("devcapsule.host_open.HostOs.current", return_value=HostOs.DARWIN), + host_open_bridge({"XDG_RUNTIME_DIR": str(tmp_path)}, enabled=True) as path, + ): + assert path is None + assert "host-browser is unavailable on macOS" in capsys.readouterr().err + assert list(tmp_path.iterdir()) == [] + + def test_hidden_host_open_command_uses_authorized_environment(tmp_path: Path, capsys) -> None: assert "host-open" not in cli.command_names() assert "host-open" in cli.command_names(include_hidden=True) diff --git a/devcapsule-src/tests/test_platforms.py b/devcapsule-src/tests/test_platforms.py index d47a77c0..4b9c022c 100644 --- a/devcapsule-src/tests/test_platforms.py +++ b/devcapsule-src/tests/test_platforms.py @@ -6,7 +6,8 @@ import pytest -from devcapsule.platforms import Platform, UnsupportedPlatformError, XdgHomes +from devcapsule import platforms +from devcapsule.platforms import HostOs, Platform, UnsupportedPlatformError, XdgHomes pytestmark = pytest.mark.legacy @@ -36,6 +37,47 @@ def test_current_returns_a_member_on_a_supported_host() -> None: assert Platform.parse(current.value) is current +def _host(monkeypatch: pytest.MonkeyPatch, system: str, machine: str) -> None: + monkeypatch.setattr(platforms.host_platform, "system", lambda: system) + monkeypatch.setattr(platforms.host_platform, "machine", lambda: machine) + + +@pytest.mark.parametrize("system,machine,emulated", [ + ("Linux", "x86_64", False), + ("Darwin", "x86_64", False), + ("Darwin", "arm64", True), +]) +def test_current_is_the_capsule_platform_of_the_host(monkeypatch, system, machine, emulated) -> None: + # A Mac runs Linux capsules: the lock it reads is linux-amd64, natively + # on Intel and under Docker's emulation on Apple Silicon. + _host(monkeypatch, system, machine) + assert Platform.current() is Platform.LINUX_AMD64 + assert Platform.current().emulated() is emulated + assert Platform.current().docker_platform_args() == (["--platform=linux/amd64"] if emulated else []) + + +@pytest.mark.parametrize("system,machine", [("Linux", "aarch64"), ("Windows", "AMD64"), ("Darwin", "ppc")]) +def test_current_refuses_a_host_without_a_capsule_platform(monkeypatch, system, machine) -> None: + _host(monkeypatch, system, machine) + with pytest.raises(UnsupportedPlatformError, match="supported platforms: linux-amd64"): + Platform.current() + + +def test_docker_spelling_and_architecture() -> None: + assert Platform.LINUX_AMD64.docker_platform == "linux/amd64" + assert Platform.LINUX_AMD64.architecture == "amd64" + + +def test_host_os_names_linux_and_macos_only(monkeypatch) -> None: + _host(monkeypatch, "Darwin", "arm64") + assert HostOs.current() is HostOs.DARWIN + _host(monkeypatch, "Linux", "x86_64") + assert HostOs.current() is HostOs.LINUX + _host(monkeypatch, "Windows", "AMD64") + with pytest.raises(UnsupportedPlatformError, match="'windows'"): + HostOs.current() + + def test_xdg_homes_honor_explicit_overrides() -> None: homes = XdgHomes.from_environment( { diff --git a/devcapsule-src/tests/test_pycharm.py b/devcapsule-src/tests/test_pycharm.py index 269414ce..f438e708 100644 --- a/devcapsule-src/tests/test_pycharm.py +++ b/devcapsule-src/tests/test_pycharm.py @@ -19,6 +19,7 @@ build_run_config, ) from devcapsule.container_runtime.contract import INFRASTRUCTURE_FAILURE_EXIT_CODE +from devcapsule.platforms import HostOs from devcapsule.launch.pycharm._launcher import ( AttachedResult, display_disclosure, @@ -466,6 +467,7 @@ def test_physical_host_launcher_owns_browser_broker_for_docker_lifetime( with ( patch("devcapsule.host_open.in_container", return_value=False), + patch("devcapsule.host_open.HostOs.current", return_value=HostOs.LINUX), patch("devcapsule.launch.pycharm._launcher.shutil.which", return_value=None), patch("devcapsule.launch.pycharm._launcher.run_attached") as run, ): @@ -621,6 +623,48 @@ def test_authorized_runtime_effects_build_the_complete_docker_plan(tmp_path: Pat assert "SYS_ADMIN" not in args +@pytest.mark.parametrize("host_os,expected_gid", [(HostOs.LINUX, "file"), (HostOs.DARWIN, 0)]) +def test_host_docker_group_follows_where_the_socket_is_owned(tmp_path: Path, host_os, expected_gid) -> None: + # On Linux the capsule joins the socket file's group. Docker Desktop shows + # its socket inside the VM as root:root 0660, so a Mac host joins group 0 + # and writes no second group for it. + project = tmp_path / "project" + project.mkdir() + env = base_env(tmp_path) + docker_socket = tmp_path / "docker.sock" + docker_socket.touch() + gid = docker_socket.stat().st_gid if expected_gid == "file" else expected_gid + host_user = HostUser(uid=501, gid=20, name="developer", group_name="staff") + with ( + patch("devcapsule.launch.pycharm._launcher.current_host_user", return_value=host_user), + patch("devcapsule.launch.pycharm._launcher.is_socket", return_value=True), + patch("devcapsule.launch.pycharm._launcher.HostOs.current", return_value=host_os), + ): + config = build_run_config( + PycharmRunOptions( + project=project, + project_mount="/workspace/project", + docker_mode=DockerMode.host, + host_docker_socket=docker_socket, + network_mode="bridge", + runtime_plan=external_runtime_plan(), + use_image_process=True, + ), + env, + ) + with patch("devcapsule.launch.pycharm._launcher.write_xauthority"): + files = prepare_temp_runtime_files(config, env) + try: + args = build_docker_args(config, files, env) + groups = files.group_file.read_text(encoding="utf-8") + finally: + cleanup_temp_runtime_files(files) + + assert config.host_docker_gid == gid + assert ["--group-add", str(gid)] == args[args.index("--group-add") : args.index("--group-add") + 2] + assert ("host-docker:x:" in groups) is (gid not in (0, host_user.gid)) + + def test_unauthorized_runtime_effects_keep_safe_docker_defaults(tmp_path: Path) -> None: project = tmp_path / "project" project.mkdir() diff --git a/devcapsule-src/tests/test_recursive_orchestrator.py b/devcapsule-src/tests/test_recursive_orchestrator.py index 58898f20..faa90a60 100644 --- a/devcapsule-src/tests/test_recursive_orchestrator.py +++ b/devcapsule-src/tests/test_recursive_orchestrator.py @@ -24,6 +24,12 @@ pytestmark = pytest.mark.legacy +# The recursive dogfood validates the host's X11 socket directory even for a +# contained display; a macOS host has none, and the flow is Linux-only. +requires_host_x11 = pytest.mark.skipif( + not Path("/tmp/.X11-unix").is_dir(), reason="recursive dogfood needs a host X11 socket directory" +) + @dataclass class RecursiveFixture: @@ -152,6 +158,7 @@ def test_normal_project_launch_derives_only_a_downgradable_readiness_marker( assert unauthorized == {RECURSIVE_E2E_ENABLED_ENV: "0"} +@requires_host_x11 def test_public_dry_run_is_unique_redacted_and_cleans_owned_staging( recursive_fixture: RecursiveFixture, ) -> None: @@ -180,6 +187,7 @@ def test_public_dry_run_is_unique_redacted_and_cleans_owned_staging( assert not (workspace / second.run_id).exists() +@requires_host_x11 def test_public_dry_run_cleans_after_preparation_failure( recursive_fixture: RecursiveFixture, ) -> None: @@ -203,6 +211,7 @@ def test_public_dry_run_cleans_after_preparation_failure( assert not run_root.exists() +@requires_host_x11 def test_keep_on_failure_preserves_only_the_owned_run( recursive_fixture: RecursiveFixture, ) -> None: @@ -228,6 +237,7 @@ def test_keep_on_failure_preserves_only_the_owned_run( assert run_root.is_dir() +@requires_host_x11 def test_contained_display_dry_run_forwards_no_host_x11( recursive_fixture: RecursiveFixture, tmp_path: Path ) -> None: diff --git a/devcapsule-src/tests/test_runtime_artifact.py b/devcapsule-src/tests/test_runtime_artifact.py index 4eda6d3a..32b1b357 100644 --- a/devcapsule-src/tests/test_runtime_artifact.py +++ b/devcapsule-src/tests/test_runtime_artifact.py @@ -7,7 +7,8 @@ from devcapsule.compat import CliError from devcapsule.materialization import ArtifactSpec, formation_descriptor, formation_identity -from devcapsule.runtime_artifact import runtime_artifact +from devcapsule import platforms +from devcapsule.runtime_artifact import executable_platform, runtime_artifact from devcapsule.runtime_command import RuntimeCommand from tests.test_base_image import pex_fixture @@ -32,6 +33,55 @@ def test_source_launcher_requires_explicit_built_runtime(tmp_path: Path, monkeyp assert runtime_artifact() == launcher +MACH_O_ARM64 = b"\xcf\xfa\xed\xfe" + bytes(16) +ELF_AMD64 = b"\x7fELF\x02\x01\x01" + bytes(11) + (0x3E).to_bytes(2, "little") +ELF_ARM64 = b"\x7fELF\x02\x01\x01" + bytes(11) + (0xB7).to_bytes(2, "little") + + +def native(path: Path, header: bytes) -> Path: + """A PEX fixture whose first bytes are a native executable's header. + + A scie is a native launcher with the PEX appended; the build information + is read from the archive, so the header and the archive coexist. + """ + + pex = pex_fixture(path) + pex.write_bytes(header + pex.read_bytes()) + return pex + + +def _apple_silicon(monkeypatch) -> None: + monkeypatch.setattr(platforms.host_platform, "system", lambda: "Darwin") + monkeypatch.setattr(platforms.host_platform, "machine", lambda: "arm64") + + +def test_executable_platform_reads_native_headers(tmp_path: Path) -> None: + assert executable_platform(native(tmp_path / "mac", MACH_O_ARM64)) == "darwin" + assert executable_platform(native(tmp_path / "amd64", ELF_AMD64)) == "linux-amd64" + assert executable_platform(native(tmp_path / "arm64", ELF_ARM64)) == "linux-arm64" + assert executable_platform(pex_fixture(tmp_path / "plain.pex")) is None + assert executable_platform(tmp_path / "absent") is None + + +def test_macos_launcher_delivers_the_named_linux_runtime(tmp_path: Path, monkeypatch) -> None: + _apple_silicon(monkeypatch) + monkeypatch.setenv("SCIE", str(native(tmp_path / "devcapsule-macos", MACH_O_ARM64))) + linux = native(tmp_path / "devcapsule-linux", ELF_AMD64) + monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(linux)) + assert runtime_artifact() == linux + + +def test_macos_launcher_without_a_linux_runtime_says_what_to_set(tmp_path: Path, monkeypatch) -> None: + _apple_silicon(monkeypatch) + monkeypatch.setenv("SCIE", str(native(tmp_path / "devcapsule-macos", MACH_O_ARM64))) + monkeypatch.delenv("DEVCAPSULE_RUNTIME_PEX", raising=False) + with pytest.raises(CliError, match="darwin executable and cannot run inside linux-amd64 capsules.*DEVCAPSULE_RUNTIME_PEX"): + runtime_artifact() + monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(native(tmp_path / "wrong-arch", ELF_ARM64))) + with pytest.raises(CliError, match="linux-arm64 executable"): + runtime_artifact() + + def test_runtime_update_changes_formation_identity_without_changing_components() -> None: artifact = ArtifactSpec("1", "https://example.test/ide", "a" * 64) before = formation_descriptor(platform="linux-amd64", base_identity="sha256:base", artifact=artifact, diff --git a/devcapsule-src/tests/test_supervisor.py b/devcapsule-src/tests/test_supervisor.py index 0bb50b36..9b55a8d1 100644 --- a/devcapsule-src/tests/test_supervisor.py +++ b/devcapsule-src/tests/test_supervisor.py @@ -20,6 +20,10 @@ import pytest +# The supervisor is the capsule's entry process: it runs only inside Linux +# capsules and relies on Linux signal primitives (sigwaitinfo). +pytestmark = pytest.mark.skipif(sys.platform != "linux", reason="container-side code; capsules are Linux") + from devcapsule import logs from devcapsule.container_runtime.supervisor import ( INFRASTRUCTURE_FAILURE_EXIT_CODE, diff --git a/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md b/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md index 245f375b..b6d1caf2 100644 --- a/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md +++ b/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md @@ -46,14 +46,39 @@ Found by reading the launch path from `run_pycharm` (`launch/pycharm/_launcher.p |---|---|---|---| | Lock and matrix key | section 1 | refuses `darwin-arm64` | slice 1 | | Docker `--platform` | `_launcher.py` `docker run`; `images/build.py` buildx | Docker picks arm64 or warns | slice 1 | -| Host-browser bridge | `host_open.py` | peer check uses `SO_PEERCRED`, Linux only; the broker socket is a host Unix socket bind-mounted into the capsule, which Docker Desktop's file sharing does not carry (*to verify*); opener is `xdg-open` | slice 2: on macOS the bridge is an unavailable optional integration, with a warning, per R-CONFIG-001; the opener is `open`; a TCP transport is a later decision | -| Host Docker socket | `_launcher.py` `host_docker_socket`, `append_docker_mode_args` | the context socket `~/.docker/run/docker.sock` is a host path; the bind source must be the VM's `/var/run/docker.sock`; `--group-add` takes the host socket's gid, which means nothing inside the VM (*to verify*) | slice 3 | -| Container identity | `current_host_user`, `project_runtime_plan.py` | the capsule user becomes 501:20; gid 20 is `dialout` in Ubuntu; ownership through Docker Desktop's file sharing differs from Linux bind mounts (*to verify*) | slice 3 | -| Container detection | `host_daemon.py` reads `/proc/self/mountinfo` | absent on macOS; must read as "not in a container" | slice 1, with a test | +| Host-browser bridge | `host_open.py` | peer check used `SO_PEERCRED`, Linux only; the broker socket is a host Unix socket bind-mounted into the capsule, which Docker Desktop refuses (section 2.1) | slice 2: unavailable on macOS with a warning, per R-CONFIG-001; the peer check also reads macOS `LOCAL_PEERCRED`; a TCP transport is a later decision. The contained desktop's own opener is Python's `webbrowser`, which already uses `open` | +| Host Docker socket | `_launcher.py` `host_docker_socket`, `write_user_files` | both socket paths bind; `--group-add` took the host file's gid (`staff`), but the socket is `0:0 0660` inside (section 2.1) | slice 3: on macOS the capsule user joins group 0; the capsule already holds the daemon, root on the VM, so the root group adds no authority beyond it | +| Container identity | `current_host_user`, `write_user_files` | 501:20; the generated `/etc/group` avoids the image's `dialout`; ownership round-trips (section 2.1) | none found; re-check in the live launch | +| Container detection | `host_daemon.py` | a marker file, not `/proc`; already correct (section 2.1) | none | +| Runtime artifact | `runtime_artifact.py` | the launcher delivers its own executable into the capsule; a macOS binary cannot run there | slice 1: deliver `DEVCAPSULE_RUNTIME_PEX` when the launcher's own binary is foreign, refuse with the remedy otherwise; how a release ships the Linux runtime beside the macOS launcher is slice 4 | | Executable | `scripts/build-pex.sh` | `DEVCAPSULE_SCIE_PLATFORM=macos-aarch64` | slice 4 | | Ports | `--publish 127.0.0.1:` | Docker Desktop forwards loopback publishes (*to verify*) | slice 3, by a live launch | | Project's host networking rule | `WORKFLOW-LOCAL.md` *Local Launch Networking* | Docker Desktop's host networking is an opt-in setting (*to verify*) | record the fallback when a launch needs it | +### 2.1 Observed on this Mac, 2026-11-03 + +Docker Desktop 29.3.1, macOS 27.0, arm64, `desktop-linux` context. + +- **Host Unix sockets.** A host socket as a bind source fails with `invalid mount config for + type "bind": bind source path does not exist: /socket_mnt/`, for a socket in the + session scratch, in `$TMPDIR` (`/var/folders/…/T/devcapsule-host-open.*/broker.sock`, where + the broker would bind it, 90 bytes) and in `~/.cache`. Docker Desktop rewrites the source to + `/socket_mnt/…` and does not find it. The host-browser bridge cannot reach a capsule. +- **The Docker socket** is the exception: `/var/run/docker.sock` on the Mac is a symlink to + `~/.docker/run/docker.sock`, and binding either path gives the capsule a socket owned + `0:0`, mode `0660`. The host file's group means nothing inside. +- **Ownership.** A capsule process running as `501:20` writes files that the Mac shows as + owned by uid 501; the group is the parent directory's, as macOS assigns it. +- **Identity.** The launcher writes the capsule's `/etc/passwd` and `/etc/group` whole, so the + host's `staff` (20) does not collide with the image's `dialout` (20). +- **`/proc`.** `host_daemon.in_container()` tests a marker file; nothing on the host path reads + `/proc`. The row above is void. +- **Test suite.** Before slice 1: 411 failed and 461 errors of 2,743. After the platform + split: 30 failed and 24 errors, in four groups: the supervisor (container-side, + `signal.sigwaitinfo` is Linux only); Unix socket paths beyond macOS's 104 bytes under the + per-user temporary directory; the recursive dogfood, which validates a host X11 socket + directory; the host-browser bridge. + ## 3. Slices 1. **Capsule platform.** Section 1, `--platform` on run and build, `/proc` absence. Done when diff --git a/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md index 8129afd0..7668fbb3 100644 --- a/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md @@ -4,7 +4,7 @@ Mnemonic: `macos-launcher` Start date: 2026-11-03 -State: active; registered 2026-11-03; slice 1, the capsule-platform split, in progress +State: active; slices 1 and 2 committed 2026-11-03; slice 3, the live launch, next Branch association: `ws-macos-launcher/host-port`, forked from local `main` at b3f10b7d @@ -42,10 +42,27 @@ the human's Mac (macOS 27.0, arm64, Docker Desktop 29.3.1, `desktop-linux` conte The work plan, one slice each, is in the [design note](2026-11-03-design-macos-host-port.md). +**2026-11-03, slices 1 and 2.** `Platform.current()` now means the platform this host runs +capsules as (`linux-amd64` on a Mac), with `HostOs` for host mechanisms. Every `docker run`, +buildx build and base pull carries `--platform linux/amd64` when the host emulates it, and a +native Linux launch composes the same command as before. A launcher whose own binary cannot +run in the capsule delivers `DEVCAPSULE_RUNTIME_PEX` instead, or refuses with that remedy. +The host-browser bridge is unavailable on macOS with a warning; its peer check reads +`LOCAL_PEERCRED`, and two macOS socket races in its protocol are fixed (an early rejection +closed with unread input; `shutdown` after the broker closed). The capsule user joins group 0 +for Docker Desktop's socket. `project config show` answers on this Mac. + +Validation on this Mac: the unit suite, 2,725 passed, 19 skipped (the container-side +supervisor and the X11-dependent recursive dogfood, by marker), from 411 failed and 461 errors +before; the host-open module 20 runs clean; `mypy --platform linux` clean over 269 files. `nox` +cannot run its sessions here, having no `python3.12` on `PATH`, so the commands were run from +`.venv` directly. The unit suite on Linux is still to run, inside an amd64 container. + ## Next Resumable Task -Slice 1: separate the capsule platform, which keys locks and matrices, from the host -operating system, and pass `--platform linux/amd64` to Docker when they differ. +Slice 3: in a `linux/amd64` container from this checkout, run the unit suite and build the +`linux-x86_64` executable; then launch this repository on this Mac with +`DEVCAPSULE_RUNTIME_PEX` naming that executable, and record what the launch shows. ## Open Threads From b4c99ffe646ade519f3c47a68504b695c0e961a8 Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Tue, 3 Nov 2026 20:54:12 +0200 Subject: [PATCH 04/20] Fall back to bridge networking on macOS Docker Desktop's host network is its VM's: under --network host the contained desktop and console listen where the Mac cannot reach them (observed on 29.3.1), while a bridge publish to 127.0.0.1 answers. An authorized network host therefore launches on bridge with a warning. The unit suite pins the host OS to Linux by default, as it already pins the host-not-container default; macOS tests patch it, and the real_host_os marker opts a test out. Co-authored-by: Claude Opus 5.5 --- .../devcapsule/launch/pycharm/_launcher.py | 13 +++++++++- devcapsule-src/pyproject.toml | 1 + devcapsule-src/tests/conftest.py | 16 ++++++++++++ devcapsule-src/tests/test_platforms.py | 1 + devcapsule-src/tests/test_pycharm.py | 22 ++++++++++++++++ .../2026-11-03-design-macos-host-port.md | 13 ++++++++-- .../CURRENT-STATUS.md | 26 ++++++++++++++++--- 7 files changed, 85 insertions(+), 7 deletions(-) diff --git a/devcapsule-src/devcapsule/launch/pycharm/_launcher.py b/devcapsule-src/devcapsule/launch/pycharm/_launcher.py index 907a8104..b24e1b69 100644 --- a/devcapsule-src/devcapsule/launch/pycharm/_launcher.py +++ b/devcapsule-src/devcapsule/launch/pycharm/_launcher.py @@ -16,7 +16,7 @@ import tempfile from threading import Event import time -from dataclasses import dataclass, field +from dataclasses import dataclass, field, replace from enum import Enum from pathlib import Path from typing import Callable, Mapping, Sequence @@ -305,6 +305,11 @@ class PycharmRunConfig: # user joins group 0 to use it. A capsule granted the host daemon already # holds root on Docker Desktop's VM, so the group adds no authority. DOCKER_DESKTOP_SOCKET_GID = 0 +HOST_NETWORK_UNAVAILABLE_ON_MACOS = ( + "devcapsule: warning: host networking is unavailable on macOS: Docker Desktop's host network " + "is its virtual machine's, out of the Mac's reach. This launch uses bridge networking; " + "services on the Mac are reachable from the capsule as host.docker.internal." +) def run_pycharm(options: PycharmRunOptions, env: Mapping[str, str] | None = None) -> int: @@ -824,6 +829,12 @@ def build_run_config(options: PycharmRunOptions, env: Mapping[str, str]) -> Pych ) if options.network_mode not in {"bridge", "host", "none"}: raise PycharmRunError("The Docker network mode must be bridge, host, or none.") + if options.network_mode == "host" and HostOs.current() is HostOs.DARWIN: + # Docker Desktop's host network is its VM's: a listener on loopback + # there is unreachable from the Mac (observed on 29.3.1), so the + # desktop and console would never answer. Bridge is the narrower grant. + print(HOST_NETWORK_UNAVAILABLE_ON_MACOS, file=sys.stderr) + options = replace(options, network_mode="bridge") if options.use_image_process and options.runtime_plan is None: raise PycharmRunError("Using the image process requires an external runtime plan.") contained_display = options.display_transport == CONTAINED_DISPLAY_TRANSPORT diff --git a/devcapsule-src/pyproject.toml b/devcapsule-src/pyproject.toml index e468bdeb..324f3335 100644 --- a/devcapsule-src/pyproject.toml +++ b/devcapsule-src/pyproject.toml @@ -95,6 +95,7 @@ markers = [ "ide_smoke: launches a real IDE session per surface with the executable under test; opt-in through nox -s ide-smoke", "flaky: executes for visibility but has a temporarily non-blocking result", "legacy: predates the unit-test conventions of WORKFLOW-LOCAL.md; the file only loses tests, see scripts/validate_unittests_follow_conventions.py", + "real_host_os: sees the real host operating system instead of the suite's Linux default", ] [tool.coverage.run] diff --git a/devcapsule-src/tests/conftest.py b/devcapsule-src/tests/conftest.py index d34213bd..44349fa3 100644 --- a/devcapsule-src/tests/conftest.py +++ b/devcapsule-src/tests/conftest.py @@ -38,6 +38,22 @@ def host_launch_by_default(monkeypatch: pytest.MonkeyPatch) -> None: ) +@pytest.fixture(autouse=True) +def linux_host_by_default(request: pytest.FixtureRequest, monkeypatch: pytest.MonkeyPatch) -> None: + """Make host-mechanism tests take the Linux path wherever they run. + + The launcher takes macOS paths for the host-browser bridge, host + networking and the Docker socket's group. Tests of the Linux behaviour + would otherwise fail on a Mac; tests of the macOS behaviour patch + ``HostOs.current`` themselves. Docker-marked tests keep the real host. + """ + + if any(request.node.get_closest_marker(marker) for marker in (*DOCKER_MARKERS, "real_host_os")): + return + from devcapsule.platforms import HostOs + monkeypatch.setattr(HostOs, "current", classmethod(lambda cls: HostOs.LINUX)) + + DOCKER_MARKERS = ("e2e", "base_build_e2e", "recursive_e2e", "contributor_e2e", "ide_smoke") diff --git a/devcapsule-src/tests/test_platforms.py b/devcapsule-src/tests/test_platforms.py index 4b9c022c..7c734ded 100644 --- a/devcapsule-src/tests/test_platforms.py +++ b/devcapsule-src/tests/test_platforms.py @@ -68,6 +68,7 @@ def test_docker_spelling_and_architecture() -> None: assert Platform.LINUX_AMD64.architecture == "amd64" +@pytest.mark.real_host_os def test_host_os_names_linux_and_macos_only(monkeypatch) -> None: _host(monkeypatch, "Darwin", "arm64") assert HostOs.current() is HostOs.DARWIN diff --git a/devcapsule-src/tests/test_pycharm.py b/devcapsule-src/tests/test_pycharm.py index f438e708..d68fca46 100644 --- a/devcapsule-src/tests/test_pycharm.py +++ b/devcapsule-src/tests/test_pycharm.py @@ -665,6 +665,28 @@ def test_host_docker_group_follows_where_the_socket_is_owned(tmp_path: Path, hos assert ("host-docker:x:" in groups) is (gid not in (0, host_user.gid)) +@pytest.mark.parametrize("host_os,effective", [(HostOs.LINUX, "host"), (HostOs.DARWIN, "bridge")]) +def test_host_networking_falls_back_to_bridge_on_macos(tmp_path: Path, capsys, host_os, effective) -> None: + # Docker Desktop's host network is its VM's; the contained desktop would + # listen where the Mac cannot reach it. + project = tmp_path / "project" + project.mkdir() + with patch("devcapsule.launch.pycharm._launcher.HostOs.current", return_value=host_os): + config = build_run_config( + PycharmRunOptions( + project=project, + project_mount="/workspace/project", + docker_mode=DockerMode.none, + network_mode="host", + runtime_plan=external_runtime_plan(), + use_image_process=True, + ), + base_env(tmp_path), + ) + assert config.network_mode == effective + assert ("host networking is unavailable on macOS" in capsys.readouterr().err) is (effective == "bridge") + + def test_unauthorized_runtime_effects_keep_safe_docker_defaults(tmp_path: Path) -> None: project = tmp_path / "project" project.mkdir() diff --git a/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md b/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md index b6d1caf2..7a45ae15 100644 --- a/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md +++ b/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md @@ -52,8 +52,8 @@ Found by reading the launch path from `run_pycharm` (`launch/pycharm/_launcher.p | Container detection | `host_daemon.py` | a marker file, not `/proc`; already correct (section 2.1) | none | | Runtime artifact | `runtime_artifact.py` | the launcher delivers its own executable into the capsule; a macOS binary cannot run there | slice 1: deliver `DEVCAPSULE_RUNTIME_PEX` when the launcher's own binary is foreign, refuse with the remedy otherwise; how a release ships the Linux runtime beside the macOS launcher is slice 4 | | Executable | `scripts/build-pex.sh` | `DEVCAPSULE_SCIE_PLATFORM=macos-aarch64` | slice 4 | -| Ports | `--publish 127.0.0.1:` | Docker Desktop forwards loopback publishes (*to verify*) | slice 3, by a live launch | -| Project's host networking rule | `WORKFLOW-LOCAL.md` *Local Launch Networking* | Docker Desktop's host networking is an opt-in setting (*to verify*) | record the fallback when a launch needs it | +| Ports | `--publish 127.0.0.1:` | forwarded to the Mac's loopback (section 2.1) | none | +| Host networking | `_launcher.py` `build_run_config` | a listener on loopback under `--network host` is unreachable from the Mac; a bridge publish to `127.0.0.1` answers (section 2.1) | slice 3: on macOS an authorized `network host` launches on bridge with a warning; the Mac's services are `host.docker.internal`. This is the concrete reason and fallback *Local Launch Networking* asks to record, and a product default for the owner to confirm | ### 2.1 Observed on this Mac, 2026-11-03 @@ -73,6 +73,15 @@ Docker Desktop 29.3.1, macOS 27.0, arm64, `desktop-linux` context. host's `staff` (20) does not collide with the image's `dialout` (20). - **`/proc`.** `host_daemon.in_container()` tests a marker file; nothing on the host path reads `/proc`. The row above is void. +- **Networking.** A `linux/amd64` HTTP server bound to `127.0.0.1` under `--network host` is + unreachable from the Mac (`curl` HTTP 000); the same server on bridge, published to + `127.0.0.1:18766`, answers HTTP 200. `host.docker.internal` resolves inside a capsule. +- **Executables.** `build-pex.sh` with `DEVCAPSULE_SCIE_PLATFORM=macos-aarch64` builds natively + here: a 31 MB Mach-O arm64 scie, CPython 3.12.14, that answers `version --json` and + `project config show`. The `linux-x86_64` runtime, 40 MB, builds inside a `linux/amd64` + `python:3.12-slim` container from an export of the commit. With both, `runtime_artifact()` + delivers the Linux one, and without `DEVCAPSULE_RUNTIME_PEX` it refuses with the remedy. +- **Disk.** Docker Desktop's VM disk is 32 GB with 5.9 GB free; the base is 0.89 GB compressed. - **Test suite.** Before slice 1: 411 failed and 461 errors of 2,743. After the platform split: 30 failed and 24 errors, in four groups: the supervisor (container-side, `signal.sigwaitinfo` is Linux only); Unix socket paths beyond macOS's 104 bytes under the diff --git a/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md index 7668fbb3..22cf3557 100644 --- a/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md @@ -4,7 +4,7 @@ Mnemonic: `macos-launcher` Start date: 2026-11-03 -State: active; slices 1 and 2 committed 2026-11-03; slice 3, the live launch, next +State: active; slices 1 and 2 and the slice-3 fixes committed 2026-11-03; the live launch waits on the human's Docker disk decision Branch association: `ws-macos-launcher/host-port`, forked from local `main` at b3f10b7d @@ -58,11 +58,29 @@ before; the host-open module 20 runs clean; `mypy --platform linux` clean over 2 cannot run its sessions here, having no `python3.12` on `PATH`, so the commands were run from `.venv` directly. The unit suite on Linux is still to run, inside an amd64 container. +**2026-11-03, slice 3 so far.** Host networking falls back to bridge on macOS with a warning: +under `--network host` the contained desktop and console would listen on the VM's loopback, +out of the Mac's reach. The unit suite pins the host OS to Linux by default, as it pins the +host-not-container default, and the macOS tests opt in; `real_host_os` opts a test out. + +Both executables exist: `devcapsule-src/dist/devcapsule-macos-arm64`, built natively, and the +`linux-x86_64` runtime, built in an amd64 container from an export of 062782bf. In that +container the unit suite ran on Linux: 2,738 passed, 1 failed, +`test_recursive_host.py::test_staging_cleans_up_after_preparation_failure`, which expects a +`chmod 0500` directory to refuse a write and runs as root there; not caused by this change. +On this Mac after the fallback: 2,727 passed, 19 skipped; `mypy --platform linux` clean. + +The live launch is not run. Docker Desktop's disk has 5.9 GB free and eight of the human's +containers are running; the base (0.89 GB compressed) and a PyCharm formation would likely +fill it. The decision is the human's. + ## Next Resumable Task -Slice 3: in a `linux/amd64` container from this checkout, run the unit suite and build the -`linux-x86_64` executable; then launch this repository on this Mac with -`DEVCAPSULE_RUNTIME_PEX` naming that executable, and record what the launch shows. +Slice 3, the live launch, once the human has made Docker disk room: +`DEVCAPSULE_PEX_UNDER_TEST=devcapsule-src/dist/devcapsule-macos-arm64`, +`DEVCAPSULE_RUNTIME_PEX=`, `PATH` with a `python3.12`, then +`.venv/bin/python -m nox -s ide-smoke -- --surface pycharm`, and record what it shows. The +smoke initializes with `--authorize network host`, which now falls back to bridge on macOS. ## Open Threads From c25e381c809f22cdd2a0a9181d2a8012759f29a7 Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Tue, 3 Nov 2026 23:34:11 +0200 Subject: [PATCH 05/20] Pass the PyCharm smoke on macOS and record the live launch The IDE smoke matched only the ready line a launcher inside a capsule prints, which names the URL. A launcher outside a container opens the URL instead and prints the ready line without it, so the smoke waited out its timeout on every host but a capsule. It now takes either line, with the URL from the launcher's earlier announcement. Its runtime digest check compares with the runtime the launcher delivers, which on macOS is the Linux one DEVCAPSULE_RUNTIME_PEX names. With both, nox -s ide-smoke -- --surface pycharm passes on an Apple Silicon Mac against Docker Desktop. The design note records the launch and two probes beside it: the desktop in a Mac browser, identity and ownership, and Docker access through group 0. Co-Authored-By: Claude Opus 5.5 --- devcapsule-src/tests/e2e/ide_session.py | 23 +++++++--- .../tests/e2e/test_ide_comes_alive.py | 15 ++++++- .../2026-11-03-design-macos-host-port.md | 43 +++++++++++++++++-- .../CURRENT-STATUS.md | 28 ++++++++---- 4 files changed, 91 insertions(+), 18 deletions(-) diff --git a/devcapsule-src/tests/e2e/ide_session.py b/devcapsule-src/tests/e2e/ide_session.py index fa461d38..5d336a76 100644 --- a/devcapsule-src/tests/e2e/ide_session.py +++ b/devcapsule-src/tests/e2e/ide_session.py @@ -60,9 +60,17 @@ class IdeSurface: ) CREATOR = "e2e@devcapsule.test" -DESKTOP_READY = re.compile(r"Contained display is ready; open it in a browser: (http://127\.0\.0\.1:\d+/vnc\.html\?\S+)") +DESKTOP_READY = re.compile( + r"Contained display is ready; (?:open it in a browser: (http://127\.0\.0\.1:\d+/vnc\.html\?\S+)" + r"|opened in your browser\.|opened through the host-browser bridge\.)" +) """The launcher's readiness line. It also announces the URL earlier, before the -bridge listens; only the ready line means the page answers.""" +bridge listens; only the ready line means the page answers. Inside a capsule +without a bridge the ready line names the URL. Elsewhere the launcher opens it, +which the smoke's ``BROWSER=true`` makes a no-op, and the URL is the earlier +announcement's.""" +DESKTOP_ANNOUNCED = re.compile(r"Contained display: (http://127\.0\.0\.1:\d+/vnc\.html\?\S+)") +"""The launcher's first mention of the desktop URL, printed before it answers.""" CONSOLE_READY = re.compile(r"Web console is ready; open it in a browser: (http://127\.0\.0\.1:\d+/\?token=\S+)") """The launcher's console readiness line; printed once the console answered its port.""" LAUNCH_TIMEOUT = 600.0 @@ -241,7 +249,7 @@ def launched_ide(executable: Path, surface: IdeSurface, workspace: Path, def wait_for_desktop_url(launcher: subprocess.Popen[bytes], launcher_log: Path) -> str: """Return the desktop URL once the launcher prints it; fail if it exits first.""" - return _wait_for_ready_url(launcher, launcher_log, DESKTOP_READY, "desktop") + return _wait_for_ready_url(launcher, launcher_log, DESKTOP_READY, "desktop", announced=DESKTOP_ANNOUNCED) def wait_for_console_url(launcher: subprocess.Popen[bytes], launcher_log: Path) -> str: @@ -250,13 +258,18 @@ def wait_for_console_url(launcher: subprocess.Popen[bytes], launcher_log: Path) def _wait_for_ready_url(launcher: subprocess.Popen[bytes], launcher_log: Path, pattern: re.Pattern[str], - what: str, timeout: float = LAUNCH_TIMEOUT) -> str: + what: str, timeout: float = LAUNCH_TIMEOUT, + announced: re.Pattern[str] | None = None) -> str: deadline = time.monotonic() + timeout while True: text = launcher_log.read_text(encoding="utf-8", errors="replace") found = pattern.search(text) - if found: + if found and found.group(1): return found.group(1) + if found and announced and (earlier := announced.search(text, 0, found.start())): + return earlier.group(1) + if found: + raise AssertionError(f"the {what} became ready without a URL announced before it:\n{text[-4000:]}") if launcher.poll() is not None: raise AssertionError(f"launcher exited {launcher.returncode} before publishing a {what} URL:\n{text}") if time.monotonic() > deadline: diff --git a/devcapsule-src/tests/e2e/test_ide_comes_alive.py b/devcapsule-src/tests/e2e/test_ide_comes_alive.py index 01a65a59..936a70f2 100644 --- a/devcapsule-src/tests/e2e/test_ide_comes_alive.py +++ b/devcapsule-src/tests/e2e/test_ide_comes_alive.py @@ -18,6 +18,7 @@ import pytest +from devcapsule.runtime_artifact import RUNTIME_PEX_ENV, executable_platform from tests.e2e.ide_session import SURFACES, IdeSurface, capture_desktop, command, desktop_page_answers, ide_session, launched_ide, stop_session, wait_for_console_url, wait_for_ide_window pytestmark = pytest.mark.legacy @@ -79,7 +80,8 @@ def test_ide_comes_alive(surface: IdeSurface, built_pex: Path, tmp_path: Path, e identity = command("docker", "inspect", "--format", '{{json .Id}} {{json .Image}}', session.container) (evidence / "container-identity.txt").write_text(identity.stdout, encoding="utf-8") child_digest = command("docker", "exec", session.container, "sha256sum", "/opt/devcapsule/bin/devcapsule.pex").stdout.split()[0] - assert child_digest == hashlib.sha256(built_pex.read_bytes()).hexdigest(), "Child runtime differs from selected executable" + assert child_digest == hashlib.sha256(delivered_runtime(built_pex).read_bytes()).hexdigest(), \ + "Child runtime differs from the runtime the selected executable delivers" if surface.name == "eclipse": package = command("docker", "exec", "--user", f"{os.getuid()}:{os.getgid()}", session.container, "python3", "-c", @@ -154,6 +156,17 @@ def test_ide_comes_alive(surface: IdeSurface, built_pex: Path, tmp_path: Path, e }, indent=2) + "\n", encoding="utf-8") +def delivered_runtime(executable: Path) -> Path: + """The executable a launch from ``executable`` delivers into the capsule. + + Itself when it runs on the capsule platform; a macOS launcher delivers the + Linux runtime its environment names instead. + """ + if executable_platform(executable) == "darwin": + return Path(os.environ[RUNTIME_PEX_ENV]).expanduser().resolve() + return executable + + def editor_font_size(container: str) -> float: probe = command("docker", "exec", container, "python3", "-c", "import xml.etree.ElementTree as E; " diff --git a/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md b/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md index 7a45ae15..72f02ffd 100644 --- a/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md +++ b/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md @@ -47,8 +47,8 @@ Found by reading the launch path from `run_pycharm` (`launch/pycharm/_launcher.p | Lock and matrix key | section 1 | refuses `darwin-arm64` | slice 1 | | Docker `--platform` | `_launcher.py` `docker run`; `images/build.py` buildx | Docker picks arm64 or warns | slice 1 | | Host-browser bridge | `host_open.py` | peer check used `SO_PEERCRED`, Linux only; the broker socket is a host Unix socket bind-mounted into the capsule, which Docker Desktop refuses (section 2.1) | slice 2: unavailable on macOS with a warning, per R-CONFIG-001; the peer check also reads macOS `LOCAL_PEERCRED`; a TCP transport is a later decision. The contained desktop's own opener is Python's `webbrowser`, which already uses `open` | -| Host Docker socket | `_launcher.py` `host_docker_socket`, `write_user_files` | both socket paths bind; `--group-add` took the host file's gid (`staff`), but the socket is `0:0 0660` inside (section 2.1) | slice 3: on macOS the capsule user joins group 0; the capsule already holds the daemon, root on the VM, so the root group adds no authority beyond it | -| Container identity | `current_host_user`, `write_user_files` | 501:20; the generated `/etc/group` avoids the image's `dialout`; ownership round-trips (section 2.1) | none found; re-check in the live launch | +| Host Docker socket | `_launcher.py` `host_docker_socket`, `write_user_files` | both socket paths bind; `--group-add` took the host file's gid (`staff`), but the socket is `0:0 0660` inside (section 2.1) | slice 3: on macOS the capsule user joins group 0; the capsule already holds the daemon, root on the VM, so the root group adds no authority beyond it; confirmed in the live launch (section 2.2) | +| Container identity | `current_host_user`, `write_user_files` | 501:20; the generated `/etc/group` avoids the image's `dialout`; ownership round-trips (sections 2.1, 2.2) | none | | Container detection | `host_daemon.py` | a marker file, not `/proc`; already correct (section 2.1) | none | | Runtime artifact | `runtime_artifact.py` | the launcher delivers its own executable into the capsule; a macOS binary cannot run there | slice 1: deliver `DEVCAPSULE_RUNTIME_PEX` when the launcher's own binary is foreign, refuse with the remedy otherwise; how a release ships the Linux runtime beside the macOS launcher is slice 4 | | Executable | `scripts/build-pex.sh` | `DEVCAPSULE_SCIE_PLATFORM=macos-aarch64` | slice 4 | @@ -81,7 +81,44 @@ Docker Desktop 29.3.1, macOS 27.0, arm64, `desktop-linux` context. `project config show`. The `linux-x86_64` runtime, 40 MB, builds inside a `linux/amd64` `python:3.12-slim` container from an export of the commit. With both, `runtime_artifact()` delivers the Linux one, and without `DEVCAPSULE_RUNTIME_PEX` it refuses with the remedy. -- **Disk.** Docker Desktop's VM disk is 32 GB with 5.9 GB free; the base is 0.89 GB compressed. +- **Disk.** Docker Desktop's VM disk was 32 GB with 5.9 GB free; the base is 0.89 GB + compressed. The human raised it to 96 GB before the live launch. + +### 2.2 The live launch, 2026-11-03 + +The macOS launcher and the `linux-x86_64` runtime, both built from 4e42d073, against Docker +Desktop on this Mac. Evidence under `devcapsule-src/dist/e2e-evidence/` (not committed). + +- **`nox -s ide-smoke -- --surface pycharm` passes.** The first run took 10 minutes: it built + the PyCharm formation under emulation, the build's last step alone 63 s. It failed only in + the harness, which matched the ready line a capsule prints, not the one a host prints; see + below. The second run, from the built image, passed in 13 s: desktop page 200, console 200 + with its token and 403 without, a `jetbrains-pycharm` window, the capsule's runtime equal to + the Linux runtime delivered, container removed, project records removed. +- **The launch**, as the launcher printed it: the host-networking warning, then bridge, the + desktop on container port 6080 published to `127.0.0.1:`, the console on 6081 the + same way. `docker inspect` shows network mode `bridge`, platform `linux`, both ports bound + to `127.0.0.1`. +- **In a Mac browser.** The desktop URL opened in Chrome on this Mac shows the capsule's + desktop with PyCharm's User Agreement, the first window of a fresh configuration, 16 s + after launch: splash at 10 s, the agreement at 16 s. Nobody accepted it, so the project + window was not reached. +- **Identity and ownership.** `uname -m` in the capsule is `x86_64`. PyCharm's JVM runs as + uid 501, gid 20. A file the IDE user writes in the project is uid 501 on the Mac, group + the parent directory's; a host file reads as `501 20` inside. The workspace was under + `/private/tmp`, which Docker Desktop shares by default. +- **Docker socket.** With `--authorize docker-daemon host-socket`, the container gets + `--group-add 0`, PyCharm's process carries groups `0 20`, and the IDE user's + `docker version` reaches Docker Desktop's daemon (server 29.3.1, `arm64`) and `docker ps` + lists the host's containers. The daemon is the VM's: an image the IDE user runs without + `--platform` is `arm64`, unlike the capsule itself. +- **Host-browser bridge.** Not exercised: unavailable on macOS with its warning (section 2.1). +- **The harness.** The smoke's readiness pattern required the ready line that names the URL, + which a launcher inside a capsule prints. A launcher outside a container opens the URL + instead, a no-op under the smoke's `BROWSER=true`, and prints the ready line without it; + on any host but a capsule the smoke waited out its 600 s. The harness now takes either + ready line and the URL from the launcher's earlier announcement. The child-runtime + digest check compares with the runtime the launcher delivers. - **Test suite.** Before slice 1: 411 failed and 461 errors of 2,743. After the platform split: 30 failed and 24 errors, in four groups: the supervisor (container-side, `signal.sigwaitinfo` is Linux only); Unix socket paths beyond macOS's 104 bytes under the diff --git a/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md index 22cf3557..35af81a2 100644 --- a/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md @@ -4,7 +4,7 @@ Mnemonic: `macos-launcher` Start date: 2026-11-03 -State: active; slices 1 and 2 and the slice-3 fixes committed 2026-11-03; the live launch waits on the human's Docker disk decision +State: active; slices 1 to 3 done 2026-11-03, the live PyCharm launch passing on this Mac; slice 4, the documentation, next Branch association: `ws-macos-launcher/host-port`, forked from local `main` at b3f10b7d @@ -70,17 +70,27 @@ container the unit suite ran on Linux: 2,738 passed, 1 failed, `chmod 0500` directory to refuse a write and runs as root there; not caused by this change. On this Mac after the fallback: 2,727 passed, 19 skipped; `mypy --platform linux` clean. -The live launch is not run. Docker Desktop's disk has 5.9 GB free and eight of the human's -containers are running; the base (0.89 GB compressed) and a PyCharm formation would likely -fill it. The decision is the human's. +**2026-11-03, slice 3, the live launch.** The human raised Docker Desktop's disk to 96 GB. +Both executables rebuilt from 4e42d073. `nox -s ide-smoke -- --surface pycharm` passes on +this Mac with the macOS launcher delivering the Linux runtime: the first run built the +PyCharm formation under emulation in 10 minutes and failed only in the harness, the second +passed in 13 s. Two probe launches added what the smoke does not check: the capsule's desktop +open in Chrome on this Mac, PyCharm at its User Agreement 16 s after launch; `x86_64` in the +capsule; files written as uid 501 on both sides; with `docker-daemon` authorized, the IDE +user reaches Docker Desktop's daemon through group 0. The record is section 2.2 of the +design note. The User Agreement was left unaccepted; accepting it is the human's. + +The harness had matched only the ready line a capsule prints; a launcher outside a container +prints another, so the smoke could not pass on any host but a capsule. It now takes either. +Its runtime digest check compares with the runtime the launcher delivers. ## Next Resumable Task -Slice 3, the live launch, once the human has made Docker disk room: -`DEVCAPSULE_PEX_UNDER_TEST=devcapsule-src/dist/devcapsule-macos-arm64`, -`DEVCAPSULE_RUNTIME_PEX=`, `PATH` with a `python3.12`, then -`.venv/bin/python -m nox -s ide-smoke -- --surface pycharm`, and record what it shows. The -smoke initializes with `--authorize network host`, which now falls back to bridge on macOS. +Slice 4: rewrite `docs/platforms/macos.md` and the install page's macOS line for what works, +as built from source: the macOS launcher, the Linux runtime beside it in +`DEVCAPSULE_RUNTIME_PEX`, Docker Desktop, emulated capsules, the bridge fallback, the +unavailable host-browser bridge, and Docker access reaching an `arm64` daemon. Whether a +release ships the pair is the owner's ruling, not this page's. ## Open Threads From 40d3e452b707f6f3fe5ea5c0dacbc473152d1cc5 Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Tue, 3 Nov 2026 23:36:54 +0200 Subject: [PATCH 06/20] Document the experimental macOS launcher built from source The macOS platform page said DevCapsule does not run on macOS. The source tree's launcher now runs on Apple Silicon against Docker Desktop, so the page says what the release still lacks, how the Mac differs from Linux, and how to build both executables. Its commands were run on an Apple Silicon Mac. The install page sends Mac readers there. The workstream's design note records that a supported macOS release reopens D-0009: the launcher cannot deliver its own bytes there, and today's path is the explicit override the decision admits. Co-Authored-By: Claude Opus 5.5 --- CURRENT-STATUS.md | 2 +- docs/getting-started/install.md | 6 +- docs/platforms/macos.md | 89 +++++++++++++++---- .../2026-11-03-design-macos-host-port.md | 12 ++- .../CURRENT-STATUS.md | 29 ++++-- 5 files changed, 111 insertions(+), 27 deletions(-) diff --git a/CURRENT-STATUS.md b/CURRENT-STATUS.md index 5ad089a8..0c5629d7 100644 --- a/CURRENT-STATUS.md +++ b/CURRENT-STATUS.md @@ -20,7 +20,7 @@ coordination branch. Detailed state belongs in each linked status file. | `website` | 2026-09-16 | Deliver the website autonomy experiment: repository-owned landing/docs/blog content, independently developable website submodule, local preview and publishing mechanism | paused 2026-09-30; producer migration merged in PR #152 and the versioned documentation live; W04-C, W09-C, W11-C retained | `ws-website/contract-v1` | [workstream status](engineering-docs/wip/2026-09-16-website/CURRENT-STATUS.md) | | `capsule-webconsole` | 2026-10-09 | Serve a token-gated web console from every capsule: configuration, versions, info, live resources, later records and workflow state | active; work order accepted 2026-10-09; the first iteration, deliverables 1 and 2, is three stacked pull requests, #184, #186 and the runtime-and-base slice, each reviewed by the Codex pair; the owner merges | `ws-capsule-webconsole/first-slice` | [workstream status](engineering-docs/wip/2026-10-09-capsule-webconsole/CURRENT-STATUS.md) | | `wrapped-projects` | 2026-11-02 | Wrap a foreign repository as a submodule of a small wrapper project and work on it in a capsule with three commands; FRC learning project first | active; PR1 (#333) accepted by the owner 2026-11-03, awaiting merge; the project-capabilities design handed to workflow-improvements | `ws-wrapped-projects/wrap` | [workstream status](engineering-docs/wip/2026-11-02-wrapped-projects/CURRENT-STATUS.md) | -| `macos-launcher` | 2026-11-03 | Run the DevCapsule launcher natively on Apple Silicon Macs against Docker Desktop, with capsules as `linux-amd64` under emulation | active 2026-11-03; registered; slice 1, the capsule-platform split | `ws-macos-launcher/host-port` | [workstream status](engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md) | +| `macos-launcher` | 2026-11-03 | Run the DevCapsule launcher natively on Apple Silicon Macs against Docker Desktop, with capsules as `linux-amd64` under emulation | active 2026-11-03; slices 1 to 4 done, the PyCharm smoke passing on an Apple Silicon Mac; waits on publication and the owner's rulings | `ws-macos-launcher/host-port` | [workstream status](engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md) | | `linux-arm64` | 2026-11-04 | Run capsules as `linux-arm64` as well as `linux-amd64`: natively on Apple Silicon Macs and on ARM Linux hosts | active 2026-11-04; registered; slice 1, the second capsule platform | `ws-linux-arm64/capsules` | [workstream status](engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md) | | `website` | 2026-09-16 | Deliver the website autonomy experiment: repository-owned landing/docs/blog content, independently developable website submodule, local preview and publishing mechanism | paused 2026-09-20 at owner request to switch to maintenance; experiment accepted, A−; blog integrated and published; content/parent-integration follow-ups remain | `ws-website/initial-cut`; renamed from `website/initial-cut`; preparation and backlog-handoff exceptions in status | [workstream status](engineering-docs/wip/2026-09-16-website/CURRENT-STATUS.md) | diff --git a/docs/getting-started/install.md b/docs/getting-started/install.md index 7a298168..31342016 100644 --- a/docs/getting-started/install.md +++ b/docs/getting-started/install.md @@ -1,7 +1,7 @@ --- description: Check prerequisites, download the DevCapsule executable for this release, verify its SHA-256 checksum, and install it under ~/.local/bin. weight: 1 -updated: 2026-10-30 +updated: 2026-11-03 --- # Install DevCapsule @@ -9,7 +9,9 @@ DevCapsule ships as a single executable for **Linux (`x86_64` / Intel-AMD 64-bit)**. It requires a web browser and a working Docker daemon on the same machine. It does not run natively on macOS, Windows, or ARM. **On Windows, read [Windows via WSL2](../platforms/windows-wsl2.md) first** and run every command -below inside your WSL2 Linux terminal. +below inside your WSL2 Linux terminal. On an Apple Silicon Mac, read +[macOS](../platforms/macos.md) for the experimental launcher you build from +source; the steps below do not apply. ## Check your computer diff --git a/docs/platforms/macos.md b/docs/platforms/macos.md index 7e92b680..9483dd59 100644 --- a/docs/platforms/macos.md +++ b/docs/platforms/macos.md @@ -1,24 +1,81 @@ --- -description: Why DevCapsule does not run on macOS yet, and what a native macOS port would require. +description: DevCapsule ships no macOS executable; an experimental launcher built from source runs on Apple Silicon Macs against Docker Desktop. What works, what differs, and how to build it. weight: 3 -updated: 2026-10-30 +updated: 2026-11-03 --- # macOS -DevCapsule {{tag}} does not run on macOS (`darwin-arm64` or `darwin-amd64`). +DevCapsule {{tag}} ships no macOS executable. The release executable is for +64-bit Intel/AMD Linux and does not run on macOS. -Today, both the release executable and the locked component catalog target -`linux-amd64` only, and the launcher expects a Linux Docker host sharing a -native Linux filesystem with the checkout. Supporting macOS properly will -require: +The launcher in the source tree runs natively on **Apple Silicon Macs** +(`darwin-arm64`) against **Docker Desktop**, built by you. This is +experimental and not a supported platform in {{tag}}. Capsules stay the same +`linux-amd64` environments Linux users get, run by Docker Desktop under x86-64 +emulation. Intel Macs are untested. -- Native macOS launcher builds and `linux-arm64` / `linux-amd64` container - resolution rules for Apple Silicon and Intel Macs. -- End-to-end verification of Docker Desktop (or Lima/Colima) bind-mount - semantics, permissions, and loopback port forwarding. -- Contained-desktop and [web console](../sessions/web-console.md) validation in - macOS browsers. +## What differs from Linux -None of that work is in {{tag}}; when macOS support begins, the supported -platforms in `.devcapsule/devcapsule..lock` and [Install](../getting-started/install.md) -will announce it explicitly. +- **Two executables.** On Linux the launcher copies itself into the capsule. A + macOS executable cannot run there, so you build a Linux runtime beside it and + name it in `DEVCAPSULE_RUNTIME_PEX`. The launcher delivers that file instead, + and stops with that remedy when the variable is unset. Build both from the + same commit. +- **Emulation.** The first launch of an IDE builds its image under emulation: + about ten minutes for PyCharm on a 14-core Mac. Later launches reuse the + image and start in seconds. The IDE itself runs slower than on a native host. +- **Networking.** Docker Desktop's host network belongs to its virtual machine, + out of your Mac's reach. A project authorized for `network host` launches on + bridge networking instead, with a warning. The contained desktop and the + [web console](../sessions/web-console.md) are published to `127.0.0.1` as on + Linux, and the capsule reaches services on your Mac as `host.docker.internal`. +- **No host-browser bridge.** Docker Desktop cannot share a Mac socket with a + container, so links the IDE opens cannot reach your Mac's browser. The launch + warns and continues. The contained desktop still opens in your browser. +- **Docker inside the capsule.** With `docker-daemon` authorized, the capsule's + Docker client reaches Docker Desktop's daemon, whose engine is `arm64`: an + image you run from inside the capsule is the ARM variant unless you pass + `--platform linux/amd64`. +- **File sharing.** Keep projects under a directory Docker Desktop shares, by + default `/Users`, `/Volumes`, `/private`, `/tmp` and `/var/folders`. Files the + capsule writes there belong to your user. + +## Build and install + +You need Docker Desktop with several gigabytes free under **Settings → +Resources → Disk usage limit**, Git, and Python 3.12. From a clean checkout: + +```bash +git clone https://github.com/ccozianu/devcapsule.git +cd devcapsule/devcapsule-src +python3.12 -m venv .venv +.venv/bin/pip install -r dev-requirements.txt +.venv/bin/pip install -e . --no-deps +DEVCAPSULE_SCIE_PLATFORM=macos-aarch64 PYTHON=.venv/bin/python \ + scripts/build-pex.sh --allow-local-source --output dist/devcapsule-macos-arm64 +git -C .. archive HEAD | docker run --rm -i --platform linux/amd64 \ + -v "$PWD/dist:/out" python:3.12-slim sh -c ' + mkdir /build && tar -x -C /build && cd /build/devcapsule-src && + python -m venv /venv && /venv/bin/pip install -q -r dev-requirements.txt && + /venv/bin/pip install -q -e . --no-deps && + PYTHON=/venv/bin/python scripts/build-pex.sh --allow-local-source \ + --output /out/devcapsule-linux-amd64' +``` + +The first build writes the macOS launcher. The second builds the Linux runtime +from the same commit inside an emulated Linux container. Install both: + +```bash +mkdir -p ~/.local/bin ~/.local/lib/devcapsule +install -m 0755 dist/devcapsule-macos-arm64 ~/.local/bin/devcapsule +install -m 0755 dist/devcapsule-linux-amd64 ~/.local/lib/devcapsule/ +export DEVCAPSULE_RUNTIME_PEX="$HOME/.local/lib/devcapsule/devcapsule-linux-amd64" +~/.local/bin/devcapsule version +``` + +Add the `export` line, and `$HOME/.local/bin` on your `PATH`, to your shell +profile. Then continue with [your first session](../getting-started/first-session.md). + +If something fails on your Mac, please [open an issue](https://github.com/ccozianu/devcapsule/issues/new) +with your macOS and Docker Desktop versions and the error output, omitting +private session URLs and tokens. diff --git a/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md b/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md index 72f02ffd..59a91d1f 100644 --- a/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md +++ b/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md @@ -137,11 +137,21 @@ Desktop on this Mac. Evidence under `devcapsule-src/dist/e2e-evidence/` (not com Findings decide the code; this slice is mostly evidence. 4. **Executable and documentation.** A `macos-aarch64` scie, smoke-tested here; the macOS platform page and the install page rewritten for what works; release automation is the - owner's call. + owner's call. Done 2026-11-03: the page describes the experimental from-source launcher, + its differences from Linux, and a build of both executables whose commands were run on + this Mac; the install page sends Mac readers there. ## 4. What this does not decide - Whether macOS becomes a *supported* platform in a release. That is the owner's ruling; this workstream produces the evidence for it. +- [D-0009](../../decisions/product/d-0009-launcher-delivers-identical-runtime.md), one + executable outside and inside. A macOS launcher cannot deliver its own bytes, so on macOS + the delivered runtime is the user's explicit choice in `DEVCAPSULE_RUNTIME_PEX`, which + D-0009's third point admits. As a product default it would not hold: D-0009's *Reopen If* + names this case, a supported execution environment that cannot run the launcher inside its + container. A macOS release needs the owner to reopen it and say how identity holds, for + example a macOS launcher that carries the exact Linux runtime bytes and delivers them, or + one that fetches the release's Linux executable against a digest it pins. - Signing and notarization of a macOS executable. - `linux-arm64` capsules. diff --git a/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md index 35af81a2..43a77665 100644 --- a/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md @@ -4,7 +4,7 @@ Mnemonic: `macos-launcher` Start date: 2026-11-03 -State: active; slices 1 to 3 done 2026-11-03, the live PyCharm launch passing on this Mac; slice 4, the documentation, next +State: active; slices 1 to 4 done 2026-11-03; waits on the SSH key to publish, and on the owner's rulings Branch association: `ws-macos-launcher/host-port`, forked from local `main` at b3f10b7d @@ -84,13 +84,23 @@ The harness had matched only the ready line a capsule prints; a launcher outside prints another, so the smoke could not pass on any host but a capsule. It now takes either. Its runtime digest check compares with the runtime the launcher delivers. +**2026-11-03, slice 4.** `docs/platforms/macos.md` now describes the experimental launcher +built from source on Apple Silicon: no macOS executable in the release, the two executables, +emulation, the bridge fallback, no host-browser bridge, Docker access reaching an `arm64` +daemon, file sharing, and a build of both executables. The page's commands were run on this +Mac, the Linux build from `git archive` of the commit into a `linux/amd64` container. The +install page sends Mac readers there. `nox -s docs-contract` skipped itself: the website +submodule is not initialized here, and fetching it needs the SSH key. + +Reading for slice 4 found that a supported macOS release reopens +[D-0009](../../decisions/product/d-0009-launcher-delivers-identical-runtime.md): the +launcher cannot deliver its own bytes there. Today's path is the explicit user override +D-0009 admits. The design note's section 4 states the question for the owner. + ## Next Resumable Task -Slice 4: rewrite `docs/platforms/macos.md` and the install page's macOS line for what works, -as built from source: the macOS launcher, the Linux runtime beside it in -`DEVCAPSULE_RUNTIME_PEX`, Docker Desktop, emulated capsules, the bridge fallback, the -unavailable host-browser bridge, and Docker access reaching an `arm64` daemon. Whether a -release ships the pair is the owner's ruling, not this page's. +None in the plan; the slices are done. Publish the workstream and push the branch once the +SSH key is loaded, give the human the compare link, and wait for the owner's rulings below. ## Open Threads @@ -100,7 +110,12 @@ release ships the pair is the owner's ruling, not this page's. - No `python3.12` exists on this Mac. `.venv` was created with `uv venv --python 3.12` (CPython 3.12.12) instead of `python3.12 -m venv`; the executable still embeds 3.12.14. - Adding a supported host platform is a product commitment. The repository owner has not - yet ruled on it; the human started the work. + yet ruled on it; the human started the work. It includes reopening D-0009 (design note, + section 4) and how a release would ship the Linux runtime beside the macOS launcher. +- The macOS default for an authorized `network host`, bridge with a warning, was put to the + human on 2026-11-03 with refusal as the alternative; no answer yet, so the warning stands. +- The PyCharm User Agreement in the probe sessions was left unaccepted, so the project + window was not reached on this Mac. ## Document Index From 6b2c66e085a6ac5fc1aeee46a75fc43acde296c3 Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Wed, 4 Nov 2026 11:42:25 +0200 Subject: [PATCH 07/20] Give development sudo a root-owned policy on macOS sudo ignores a policy file root does not own. On Linux the launcher chowns its generated policy to root, through a constrained helper container when the user cannot. Docker Desktop shares Mac files with its virtual machine as the Mac user, so inside any container a bind-mounted file stays the user's and not even root can chown it: a macOS launch with development sudo stopped with "Permission denied". On macOS the policy is now written by root into a Docker volume and mounted from there, read-only, at the same path. The volume is named after the container and removed when the session ends. The Linux helper also gains --platform when the host emulates the capsule. Co-Authored-By: Claude Opus 5.5 --- .../devcapsule/launch/pycharm/_launcher.py | 98 ++++++++++++++++++- devcapsule-src/tests/test_pycharm.py | 83 +++++++++++++++- 2 files changed, 174 insertions(+), 7 deletions(-) diff --git a/devcapsule-src/devcapsule/launch/pycharm/_launcher.py b/devcapsule-src/devcapsule/launch/pycharm/_launcher.py index b24e1b69..836e7970 100644 --- a/devcapsule-src/devcapsule/launch/pycharm/_launcher.py +++ b/devcapsule-src/devcapsule/launch/pycharm/_launcher.py @@ -128,10 +128,15 @@ class TempRuntimeFiles: # The console's configuration channel: a directory the capsule writes # requests into and reads the launch-time listing from; CHANNEL_PATH inside. configuration_directory: Path | None = None + # On macOS, the Docker volume that holds the root-owned sudo policy. + sudoers_volume: str | None = None SUDOERS_POLICY_PATH = "/etc/sudoers.d/devcapsule-development-sudo" SUDOERS_POLICY = "%ide-sudo ALL=(ALL:ALL) NOPASSWD: ALL\n" +SUDOERS_VOLUME_FILE = "policy" +"""The policy's name inside its macOS volume.""" +SUDOERS_VOLUME_LABEL = "devcapsule.purpose=development-sudo-policy" @@ -416,6 +421,7 @@ def run_pycharm(options: PycharmRunOptions, env: Mapping[str, str] | None = None print(session_result_text(config, result), file=sys.stderr, flush=True) return exit_code finally: + remove_sudoers_policy_volume(files.sudoers_volume, runtime_env) cleanup_temp_runtime_files(files) except HostOpenError as exc: raise PycharmRunError(str(exc)) from exc @@ -1417,12 +1423,12 @@ def build_docker_args( "Development sudo requires generated shadow and sudoers files." ) args.extend(["--mount", f"type=bind,src={files.shadow_file},dst=/etc/shadow,ro"]) - args.extend( - [ - "--mount", - f"type=bind,src={files.sudoers_file},dst={SUDOERS_POLICY_PATH},ro", - ] + policy_source = ( + f"type=volume,src={files.sudoers_volume},volume-subpath={SUDOERS_VOLUME_FILE}" + if files.sudoers_volume + else f"type=bind,src={files.sudoers_file}" ) + args.extend(["--mount", f"{policy_source},dst={SUDOERS_POLICY_PATH},ro"]) if files.runtime_plan_file is not None: args.extend( [ @@ -1868,6 +1874,9 @@ def ensure_sudoers_policy_ownership( policy = files.sudoers_file if not config.enable_sudo or policy is None: raise PycharmRunError("Development sudo requires a generated sudoers policy.") + if HostOs.current() is HostOs.DARWIN: + files.sudoers_volume = prepare_sudoers_policy_volume(config, policy, env) + return try: os.chown(policy, 0, 0) except PermissionError: @@ -1877,6 +1886,7 @@ def ensure_sudoers_policy_ownership( "run", "--rm", "--pull=never", + *Platform.current().docker_platform_args(), "--network", "none", "--read-only", @@ -1918,6 +1928,84 @@ def ensure_sudoers_policy_ownership( ) +def prepare_sudoers_policy_volume(config: PycharmRunConfig, policy: Path, env: Mapping[str, str]) -> str: + """Write the sudo policy into a Docker volume as root; return the volume's name. + + sudo ignores a policy root does not own. Docker Desktop shares Mac files + with its virtual machine as the Mac user: a bind-mounted file is owned by + that user inside every container, and not even root there can chown it. + A volume lives on the virtual machine's own disk, where root's ownership + holds. The name follows the container's, so a volume a killed launcher + left behind is rewritten and then removed by the next launch. + """ + + volume = f"{config.name}-sudoers" + created = subprocess.run( + ["docker", "volume", "create", "--label", SUDOERS_VOLUME_LABEL, volume], + check=False, env=dict(env), stdout=subprocess.DEVNULL, stderr=subprocess.PIPE, text=True, + ) + if created.returncode != 0: + detail = created.stderr.strip() + raise PycharmRunError( + "Could not create the volume for the development-sudo policy" + (f": {detail}" if detail else ".") + ) + staged = f"/policy/{SUDOERS_VOLUME_FILE}.new" + command = [ + "docker", + "run", + "--rm", + "-i", + "--pull=never", + *Platform.current().docker_platform_args(), + "--network", + "none", + "--read-only", + "--pids-limit", + "64", + "--cap-drop", + "ALL", + "--security-opt", + "no-new-privileges", + "--user", + "0:0", + "--mount", + f"type=volume,src={volume},dst=/policy", + "--entrypoint", + "/bin/sh", + config.image, + "-ec", + # 0440 root:root, as sudo requires. A policy left by an earlier + # launch is read-only even to root without capabilities: replace it. + f"umask 0337; rm -f {staged}; cat > {staged}; mv -f {staged} /policy/{SUDOERS_VOLUME_FILE}", + ] + completed = subprocess.run( + command, + check=False, + env=dict(env), + input=policy.read_text(encoding="utf-8"), + stdout=subprocess.DEVNULL, + stderr=subprocess.PIPE, + text=True, + ) + if completed.returncode != 0: + remove_sudoers_policy_volume(volume, env) + detail = completed.stderr.strip() + raise PycharmRunError( + "Could not prepare the root-owned development-sudo policy" + (f": {detail}" if detail else ".") + ) + return volume + + +def remove_sudoers_policy_volume(volume: str | None, env: Mapping[str, str]) -> None: + """Remove the policy volume; a capsule still using it keeps it until the next launch.""" + + if volume: + subprocess.run( + ["docker", "volume", "rm", "--force", volume], + check=False, env=dict(env), stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, + ) + + def file_owner_uid(path: Path) -> int: return path.stat().st_uid diff --git a/devcapsule-src/tests/test_pycharm.py b/devcapsule-src/tests/test_pycharm.py index d68fca46..0ae91528 100644 --- a/devcapsule-src/tests/test_pycharm.py +++ b/devcapsule-src/tests/test_pycharm.py @@ -591,6 +591,9 @@ def test_authorized_runtime_effects_build_the_complete_docker_plan(tmp_path: Pat assert sudoers_file.read_text(encoding="utf-8") == SUDOERS_POLICY assert sudoers_file.stat().st_mode & 0o777 == 0o440 args = build_docker_args(config, files, env) + files.sudoers_volume = "capsule-sudoers" + volume_args = build_docker_args(config, files, env) + files.sudoers_volume = None files.sudoers_file = None with pytest.raises(PycharmRunError, match="shadow and sudoers files"): build_docker_args(config, files, env) @@ -603,6 +606,9 @@ def test_authorized_runtime_effects_build_the_complete_docker_plan(tmp_path: Pat assert ["--user", "1000:1000"] == args[args.index("--user") : args.index("--user") + 2] assert "ENABLE_SUDO=1" in args assert f"type=bind,src={sudoers_file},dst={SUDOERS_POLICY_PATH},ro" in args + # On macOS the policy comes from its volume, not the Mac's file. + assert f"type=volume,src=capsule-sudoers,volume-subpath=policy,dst={SUDOERS_POLICY_PATH},ro" in volume_args + assert f"type=bind,src={sudoers_file},dst={SUDOERS_POLICY_PATH},ro" not in volume_args assert not sudoers_file.exists() assert not sudoers_directory.exists() assert "DOCKER_HOST=unix:///run/host-docker.sock" in args @@ -787,6 +793,75 @@ def test_sudoers_ownership_helper_failure_is_actionable(tmp_path: Path) -> None: ensure_sudoers_policy_ownership(config, files, {}) +def _macos_sudo_files(tmp_path: Path) -> tuple[PycharmRunConfig, TempRuntimeFiles]: + policy = tmp_path / "sudoers" + policy.write_text(SUDOERS_POLICY, encoding="utf-8") + files = TempRuntimeFiles( + xauth_file=tmp_path / "xauth", + passwd_file=tmp_path / "passwd", + group_file=tmp_path / "group", + sudoers_file=policy, + ) + config = cast( + PycharmRunConfig, + SimpleNamespace(enable_sudo=True, image="devcapsule-local-pycharm:canonical", name="capsule"), + ) + return config, files + + +def test_macos_sudo_policy_is_written_into_a_volume_by_root( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, +) -> None: + # Docker Desktop shares Mac files as the Mac user: no chown makes a + # bind-mounted policy root's, so the policy lives in a volume. + monkeypatch.setattr(HostOs, "current", classmethod(lambda cls: HostOs.DARWIN)) + monkeypatch.setattr("devcapsule.platforms.host_architecture", lambda: "arm64") + config, files = _macos_sudo_files(tmp_path) + + with ( + patch("devcapsule.launch.pycharm._launcher.os.chown") as chown, + patch("devcapsule.launch.pycharm._launcher.subprocess.run") as run, + ): + run.return_value = SimpleNamespace(returncode=0, stderr="") + ensure_sudoers_policy_ownership(config, files, {"PATH": "/usr/bin"}) + + chown.assert_not_called() + assert files.sudoers_volume == "capsule-sudoers" + create, write = (call.args[0] for call in run.call_args_list) + assert create == ["docker", "volume", "create", "--label", "devcapsule.purpose=development-sudo-policy", + "capsule-sudoers"] + assert write[:4] == ["docker", "run", "--rm", "-i"] + assert "--platform=linux/amd64" in write + assert ["--network", "none"] == write[write.index("--network") : write.index("--network") + 2] + assert ["--cap-drop", "ALL"] == write[write.index("--cap-drop") : write.index("--cap-drop") + 2] + assert "--cap-add" not in write + assert ["--user", "0:0"] == write[write.index("--user") : write.index("--user") + 2] + assert {"--read-only", "--pull=never", "no-new-privileges"} <= set(write) + assert "type=volume,src=capsule-sudoers,dst=/policy" in write + assert write[-1].startswith("umask 0337;") + assert write[-1].endswith("/policy/policy") + assert run.call_args_list[1].kwargs["input"] == SUDOERS_POLICY + + +def test_macos_sudo_policy_failure_removes_its_volume( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.setattr(HostOs, "current", classmethod(lambda cls: HostOs.DARWIN)) + config, files = _macos_sudo_files(tmp_path) + + with patch("devcapsule.launch.pycharm._launcher.subprocess.run") as run: + run.side_effect = [ + SimpleNamespace(returncode=0, stderr=""), + SimpleNamespace(returncode=125, stderr="no such image"), + SimpleNamespace(returncode=0, stderr=""), + ] + with pytest.raises(PycharmRunError, match="no such image"): + ensure_sudoers_policy_ownership(config, files, {}) + + assert run.call_args_list[-1].args[0] == ["docker", "volume", "rm", "--force", "capsule-sudoers"] + assert files.sudoers_volume is None + + def test_sudo_banner_is_printed_after_complete_policy_preparation( tmp_path: Path, capsys: pytest.CaptureFixture[str], @@ -799,8 +874,10 @@ def test_sudo_banner_is_printed_after_complete_policy_preparation( with ( patch("devcapsule.launch.pycharm._launcher.shutil.which", return_value=None), patch( - "devcapsule.launch.pycharm._launcher.ensure_sudoers_policy_ownership" + "devcapsule.launch.pycharm._launcher.ensure_sudoers_policy_ownership", + side_effect=lambda _config, files, _env: setattr(files, "sudoers_volume", "capsule-sudoers"), ) as ensure, + patch("devcapsule.launch.pycharm._launcher.remove_sudoers_policy_volume") as remove, patch("devcapsule.launch.pycharm._launcher.run_attached") as run, ): run.return_value = AttachedResult(0, interrupted=False, stopped=True) @@ -821,7 +898,9 @@ def test_sudo_banner_is_printed_after_complete_policy_preparation( ensure.assert_called_once() assert "DEVELOPMENT SUDO IS ENABLED" in capsys.readouterr().err command = run.call_args.args[0] - assert any(SUDOERS_POLICY_PATH in argument for argument in command) + assert any(SUDOERS_POLICY_PATH in argument and "src=capsule-sudoers" in argument for argument in command) + # The policy volume outlives nothing but the session. + assert remove.call_args.args[0] == "capsule-sudoers" def test_sudo_policy_preparation_failure_cleans_files_without_enabled_banner( From dc88f4839974a6ff6bd44d25f7840128b1618417 Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Wed, 4 Nov 2026 11:44:24 +0200 Subject: [PATCH 08/20] Record the macOS development-sudo fix in the workstream Co-Authored-By: Claude Opus 5.5 --- .../2026-11-03-design-macos-host-port.md | 1 + .../wip/2026-11-03-macos-launcher/CURRENT-STATUS.md | 10 ++++++++++ 2 files changed, 11 insertions(+) diff --git a/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md b/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md index 59a91d1f..111e9139 100644 --- a/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md +++ b/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md @@ -52,6 +52,7 @@ Found by reading the launch path from `run_pycharm` (`launch/pycharm/_launcher.p | Container detection | `host_daemon.py` | a marker file, not `/proc`; already correct (section 2.1) | none | | Runtime artifact | `runtime_artifact.py` | the launcher delivers its own executable into the capsule; a macOS binary cannot run there | slice 1: deliver `DEVCAPSULE_RUNTIME_PEX` when the launcher's own binary is foreign, refuse with the remedy otherwise; how a release ships the Linux runtime beside the macOS launcher is slice 4 | | Executable | `scripts/build-pex.sh` | `DEVCAPSULE_SCIE_PLATFORM=macos-aarch64` | slice 4 | +| Development sudo | `ensure_sudoers_policy_ownership`, `build_docker_args` | the policy must be root's; a bind-mounted Mac file stays the Mac user's, and the helper's `chown` is refused even to root (found by the human, 2026-11-04) | fixed: on macOS root writes the policy into a Docker volume named after the container, mounted read-only from it and removed with the session; `sudo -n id -u` prints 0 in a live launch | | Ports | `--publish 127.0.0.1:` | forwarded to the Mac's loopback (section 2.1) | none | | Host networking | `_launcher.py` `build_run_config` | a listener on loopback under `--network host` is unreachable from the Mac; a bridge publish to `127.0.0.1` answers (section 2.1) | slice 3: on macOS an authorized `network host` launches on bridge with a warning; the Mac's services are `host.docker.internal`. This is the concrete reason and fallback *Local Launch Networking* asks to record, and a product default for the owner to confirm | diff --git a/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md index 43a77665..7d3eb985 100644 --- a/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md @@ -97,6 +97,16 @@ Reading for slice 4 found that a supported macOS release reopens launcher cannot deliver its own bytes there. Today's path is the explicit user override D-0009 admits. The design note's section 4 states the question for the owner. +**2026-11-04, development sudo.** The human's first own launch on this Mac, a project with +development sudo, stopped at "Permission denied": the sudo policy must be root's, and Docker +Desktop shares a Mac file as the Mac user, which even root in the helper container cannot +change. Neither smoke nor probe had enabled sudo. On macOS root now writes the policy into a +Docker volume named after the container, mounted read-only and removed when the session +ends (e5bd8acd); the Linux helper gained `--platform` when emulating. A live VSCodium launch +with development sudo: the policy `0 0 0440` in the capsule, `sudo -n id -u` as the IDE user +prints 0, the volume gone afterwards. Unit suite on this Mac: 2,729 passed, 19 skipped. The +fixed pair is installed in the human's `~/.local`. + ## Next Resumable Task None in the plan; the slices are done. Publish the workstream and push the branch once the From e8e67a1a3ee86e99996212194bcf86d276514764 Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Wed, 4 Nov 2026 12:05:59 +0200 Subject: [PATCH 09/20] Add linux-arm64 as a second capsule platform Platform gains LINUX_ARM64. A host now runs a list of capsule platforms, preferred first: a Linux host its own, Apple Silicon linux-arm64 natively and linux-amd64 under emulation, an Intel Mac linux-amd64. Every project command selects, for its process, the first of those the project has a lock for, else the preferred one, so a Mac runs a project locked only for linux-amd64 as before and a new one natively. DEVCAPSULE_RUNTIME_PEX may name a directory holding one runtime per capsule platform as devcapsule-. Eclipse accepts arm64 native packages, Codex maps its arm64 npm package, and the JetBrains runtime check names the machine's skiko library. The linux-arm64 resolution matrix shares amd64's vocabulary and recipes and has, for now, no bases or pins of its own. The unit suite runs capsules as linux-amd64 by default, as it runs the Linux host by default; real_host_os tests keep the real host. Co-Authored-By: Claude Opus 5.5 --- .../docker4pycharm/check-runtime-deps.sh | 6 +- .../assets/pycharm/check-runtime-deps.sh | 6 +- devcapsule-src/devcapsule/commands/project.py | 20 ++++++ devcapsule-src/devcapsule/components/codex.py | 3 +- .../devcapsule/components/eclipse.py | 4 +- devcapsule-src/devcapsule/platforms.py | 58 ++++++++++++--- .../devcapsule/resolution_matrix.py | 15 ++++ devcapsule-src/devcapsule/runtime_artifact.py | 16 ++++- devcapsule-src/tests/conftest.py | 19 +++++ devcapsule-src/tests/test_platforms.py | 72 +++++++++++++++---- devcapsule-src/tests/test_runtime_artifact.py | 22 ++++++ 11 files changed, 213 insertions(+), 28 deletions(-) diff --git a/devcapsule-src/devcapsule/assets/docker4pycharm/check-runtime-deps.sh b/devcapsule-src/devcapsule/assets/docker4pycharm/check-runtime-deps.sh index 06bac67c..b3cacf90 100755 --- a/devcapsule-src/devcapsule/assets/docker4pycharm/check-runtime-deps.sh +++ b/devcapsule-src/devcapsule/assets/docker4pycharm/check-runtime-deps.sh @@ -2,7 +2,11 @@ set -euo pipefail IDE_HOME="${IDE_HOME:-${PYCHARM_HOME:-/opt/pycharm}}" -SKIKO_LIB="$IDE_HOME/lib/skiko-awt-runtime-all/libskiko-linux-x64.so" +case "$(uname -m)" in + aarch64 | arm64) SKIKO_ARCH=arm64 ;; + *) SKIKO_ARCH=x64 ;; +esac +SKIKO_LIB="$IDE_HOME/lib/skiko-awt-runtime-all/libskiko-linux-$SKIKO_ARCH.so" FAILURES=0 pass() { diff --git a/devcapsule-src/devcapsule/assets/pycharm/check-runtime-deps.sh b/devcapsule-src/devcapsule/assets/pycharm/check-runtime-deps.sh index 06bac67c..b3cacf90 100644 --- a/devcapsule-src/devcapsule/assets/pycharm/check-runtime-deps.sh +++ b/devcapsule-src/devcapsule/assets/pycharm/check-runtime-deps.sh @@ -2,7 +2,11 @@ set -euo pipefail IDE_HOME="${IDE_HOME:-${PYCHARM_HOME:-/opt/pycharm}}" -SKIKO_LIB="$IDE_HOME/lib/skiko-awt-runtime-all/libskiko-linux-x64.so" +case "$(uname -m)" in + aarch64 | arm64) SKIKO_ARCH=arm64 ;; + *) SKIKO_ARCH=x64 ;; +esac +SKIKO_LIB="$IDE_HOME/lib/skiko-awt-runtime-all/libskiko-linux-$SKIKO_ARCH.so" FAILURES=0 pass() { diff --git a/devcapsule-src/devcapsule/commands/project.py b/devcapsule-src/devcapsule/commands/project.py index d6e5fa73..918d2350 100644 --- a/devcapsule-src/devcapsule/commands/project.py +++ b/devcapsule-src/devcapsule/commands/project.py @@ -2577,6 +2577,7 @@ def make_context(cls, arguments: argparse.Namespace, parent: object | None) -> o context = replace(context, capsule_root=runtime_configuration.capsule_project_root(Path("."))) if access.needs_launcher: runtime_configuration.require_launcher(context.start_path(), tokens) + _select_capsule_platform(context) return context @classmethod @@ -2596,6 +2597,25 @@ def subcommands(cls) -> Mapping[str, type[Command] | type[Group]]: } +def _select_capsule_platform(context: ProjectCommandContext) -> None: + """Run the selected project as the platform it is locked for on this host. + + See ``Platform.select_for_project``: where a host runs two platforms, the + project's locks choose. Without an enclosing project, as for ``init`` in + a fresh directory, the host's preferred platform stands. + """ + from devcapsule.platforms import Platform, UnsupportedPlatformError + + try: + root: Path | None = discover_project(context.start_path()) + except ProjectConfigurationError: + root = None + try: + Platform.select_for_project(root) + except UnsupportedPlatformError: + pass # Each platform-dependent step reports it in its own terms. + + def _capsule_access(leaf: type[Command]) -> CapsuleAccess: """The access a ``project`` leaf declares for itself. diff --git a/devcapsule-src/devcapsule/components/codex.py b/devcapsule-src/devcapsule/components/codex.py index f90bc744..00f87ae9 100644 --- a/devcapsule-src/devcapsule/components/codex.py +++ b/devcapsule-src/devcapsule/components/codex.py @@ -71,7 +71,8 @@ class CodexComponent(ComponentDefinition): """Trusted Codex component implementation.""" def distribution_channel(self) -> NpmChannel: - return NpmChannel(CODEX_PACKAGE, {"linux-amd64": "@openai/codex-linux-x64"}) + return NpmChannel(CODEX_PACKAGE, {"linux-amd64": "@openai/codex-linux-x64", + "linux-arm64": "@openai/codex-linux-arm64"}) @property def id(self) -> str: diff --git a/devcapsule-src/devcapsule/components/eclipse.py b/devcapsule-src/devcapsule/components/eclipse.py index 3d2d0dd1..be4b344b 100644 --- a/devcapsule-src/devcapsule/components/eclipse.py +++ b/devcapsule-src/devcapsule/components/eclipse.py @@ -59,8 +59,8 @@ def locked_artifacts(self, metadata: Mapping[str, object], platform: str) -> tup # dependencies use the same acquisition/hash verification before an # offline dpkg installation. The set is specific to the Ubuntu 24.04 base. packages = metadata.get("native-packages") - if platform != "linux-amd64" or not isinstance(packages, dict) or not packages: - raise CliError("Eclipse requires pinned native-packages for linux-amd64.") + if platform not in ("linux-amd64", "linux-arm64") or not isinstance(packages, dict) or not packages: + raise CliError(f"Eclipse requires pinned native-packages for {platform}.") artifacts = [] for package, entry in packages.items(): if not isinstance(entry, dict) or any(not isinstance(entry.get(key), str) or not entry[key] diff --git a/devcapsule-src/devcapsule/platforms.py b/devcapsule-src/devcapsule/platforms.py index 8e1f4d96..52c17c94 100644 --- a/devcapsule-src/devcapsule/platforms.py +++ b/devcapsule-src/devcapsule/platforms.py @@ -13,6 +13,7 @@ from __future__ import annotations +from contextvars import ContextVar from dataclasses import dataclass, field from enum import StrEnum import os @@ -56,13 +57,18 @@ def current(cls) -> HostOs: ) from None -# The capsule platform a non-Linux host runs. Apple Silicon runs linux-amd64 -# under Docker's emulation until a linux-arm64 platform exists. -_CAPSULE_PLATFORM_OF_HOST = { - "darwin-arm64": "linux-amd64", - "darwin-amd64": "linux-amd64", +# The capsule platforms each host runs, preferred first. A Linux host runs its +# own platform. Apple Silicon runs linux-arm64 natively and linux-amd64 under +# Docker Desktop's emulation; an Intel Mac runs linux-amd64. +_CAPSULE_PLATFORMS_OF_HOST: Mapping[str, tuple[str, ...]] = { + "darwin-arm64": ("linux-arm64", "linux-amd64"), + "darwin-amd64": ("linux-amd64",), } +# The platform selected for the project this process works on, if any; see +# Platform.select_for_project. +_selected_platform: ContextVar[Platform | None] = ContextVar("devcapsule_capsule_platform", default=None) + class Platform(StrEnum): """The supported capsule platforms and their wire format. @@ -80,17 +86,51 @@ class Platform(StrEnum): """ LINUX_AMD64 = "linux-amd64" + LINUX_ARM64 = "linux-arm64" @classmethod def current(cls) -> Platform: - """The platform this host runs capsules as. + """The platform this process runs capsules as. - On Linux, the host's own platform. On macOS, the Linux platform - Docker Desktop runs, natively or under emulation. + The platform selected for its project, when one was; otherwise the + host's preferred platform: on Linux the host's own, on Apple Silicon + linux-arm64. """ + return _selected_platform.get() or cls.runnable()[0] + + @classmethod + def runnable(cls) -> tuple[Platform, ...]: + """The platforms this host runs capsules as, preferred first.""" + host = f"{host_platform.system().lower()}-{host_architecture()}" - return cls.parse(_CAPSULE_PLATFORM_OF_HOST.get(host, host)) + return tuple(cls.parse(key) for key in _CAPSULE_PLATFORMS_OF_HOST.get(host, (host,))) + + @classmethod + def select_for_project(cls, root: Path | None) -> Platform: + """Select, for this process, the platform the project at ``root`` runs as. + + The first platform this host runs that the project has a lock for; + without one, as for a project not yet initialized, the preferred + platform. Apple Silicon so runs a project locked for linux-arm64 + natively and one locked only for linux-amd64 under emulation. + """ + + runnable = cls.runnable() + locks = root / ".devcapsule" if root is not None else None + selected = next( + (platform for platform in runnable + if locks is not None and (locks / f"devcapsule.{platform}.lock").is_file()), + runnable[0], + ) + _selected_platform.set(selected) + return selected + + @classmethod + def clear_selection(cls) -> None: + """Forget the project selection; ``current()`` is the host's preference again.""" + + _selected_platform.set(None) @property def architecture(self) -> str: diff --git a/devcapsule-src/devcapsule/resolution_matrix.py b/devcapsule-src/devcapsule/resolution_matrix.py index 3df46d48..e9b19fa7 100644 --- a/devcapsule-src/devcapsule/resolution_matrix.py +++ b/devcapsule-src/devcapsule/resolution_matrix.py @@ -1702,12 +1702,27 @@ def _formation( }, ) +# linux-arm64 offers the same vocabulary and the same materialization +# recipes as linux-amd64; its bases, pins and verified edges are its own, +# because a validation on one architecture establishes nothing on another. +_LINUX_ARM64_MATRIX = ResolutionMatrix( + platform=Platform.LINUX_ARM64, + matrix_version=_MATRIX_VERSION, + bases=(), + components={}, + edges=(), + couplings=(), + capability_table=_LINUX_AMD64_MATRIX._capability_table, + materialization=_LINUX_AMD64_MATRIX._materialization, +) + # The map is the platform authority: its keys are the supported platforms, # total over the Platform enum (D-0006/D-0007). Clients index it with a key # from Platform.current() or Platform.parse(), never one built from parts. MATRICES: Mapping[Platform, ResolutionMatrix] = MappingProxyType( { Platform.LINUX_AMD64: _LINUX_AMD64_MATRIX, + Platform.LINUX_ARM64: _LINUX_ARM64_MATRIX, } ) diff --git a/devcapsule-src/devcapsule/runtime_artifact.py b/devcapsule-src/devcapsule/runtime_artifact.py index 64338b0a..72f4d87d 100644 --- a/devcapsule-src/devcapsule/runtime_artifact.py +++ b/devcapsule-src/devcapsule/runtime_artifact.py @@ -23,7 +23,9 @@ def runtime_artifact() -> Path: A packaged launcher delivers itself when its binary runs on the capsule platform. A launcher that cannot, a macOS executable launching Linux capsules, delivers the runtime ``DEVCAPSULE_RUNTIME_PEX`` names. A - source-form launcher must name its built runtime the same way. + source-form launcher must name its built runtime the same way. The + variable names one executable, or a directory holding one per capsule + platform as ``devcapsule-``, for a host that runs two. """ capsule = Platform.current() @@ -31,6 +33,15 @@ def runtime_artifact() -> Path: # archive too. sys.argv[0] alone can name the extracted console script. own = os.environ.get("SCIE") or os.environ.get("PEX") explicit = os.environ.get(RUNTIME_PEX_ENV) + if explicit and Path(explicit).expanduser().is_dir(): + directory = Path(explicit).expanduser() + explicit = str(directory / f"devcapsule-{capsule}") + if not Path(explicit).is_file() and executable_platform(_resolved(own or sys.argv[0])) != str(capsule): + raise CliError( + f"{RUNTIME_PEX_ENV} names {directory}, which holds no devcapsule-{capsule}: " + f"this launch runs a {capsule} capsule. Put a {capsule} DevCapsule executable of the same " + f"version there, or a directory holding it as devcapsule-{capsule}." + ) path = _resolved(own or explicit or sys.argv[0]) if own and explicit and executable_platform(path) not in (None, str(capsule)): path = _resolved(explicit) @@ -46,7 +57,8 @@ def runtime_artifact() -> Path: if found not in (None, str(capsule)): raise CliError( f"{path} is a {found} executable and cannot run inside {capsule} capsules. " - f"Set {RUNTIME_PEX_ENV} to a {capsule} DevCapsule executable of the same version." + f"Set {RUNTIME_PEX_ENV} to a {capsule} DevCapsule executable of the same version, " + f"or to a directory holding it as devcapsule-{capsule}." ) return path diff --git a/devcapsule-src/tests/conftest.py b/devcapsule-src/tests/conftest.py index 44349fa3..52e26b02 100644 --- a/devcapsule-src/tests/conftest.py +++ b/devcapsule-src/tests/conftest.py @@ -1,6 +1,7 @@ from __future__ import annotations import os +from collections.abc import Iterator from pathlib import Path import sys import tempfile @@ -54,6 +55,24 @@ def linux_host_by_default(request: pytest.FixtureRequest, monkeypatch: pytest.Mo monkeypatch.setattr(HostOs, "current", classmethod(lambda cls: HostOs.LINUX)) +@pytest.fixture(autouse=True) +def amd64_capsules_by_default(request: pytest.FixtureRequest, monkeypatch: pytest.MonkeyPatch) -> Iterator[None]: + """Make capsule-platform tests run as linux-amd64 wherever they run. + + An Apple Silicon host prefers linux-arm64; the suite's fixtures and + golden locks are linux-amd64. Tests of other platforms patch + ``Platform.runnable`` themselves; Docker-marked and ``real_host_os`` + tests keep the real host. No project selection outlives its test. + """ + + from devcapsule.platforms import Platform + Platform.clear_selection() + if not any(request.node.get_closest_marker(marker) for marker in (*DOCKER_MARKERS, "real_host_os")): + monkeypatch.setattr(Platform, "runnable", classmethod(lambda cls: (Platform.LINUX_AMD64,))) + yield + Platform.clear_selection() + + DOCKER_MARKERS = ("e2e", "base_build_e2e", "recursive_e2e", "contributor_e2e", "ide_smoke") diff --git a/devcapsule-src/tests/test_platforms.py b/devcapsule-src/tests/test_platforms.py index 7c734ded..f95cc158 100644 --- a/devcapsule-src/tests/test_platforms.py +++ b/devcapsule-src/tests/test_platforms.py @@ -17,6 +17,7 @@ def test_platform_values_are_the_wire_format() -> None: # StrEnum: the member formats as its value wherever locks and filenames # interpolate it. assert f"devcapsule.{Platform.LINUX_AMD64}.lock" == "devcapsule.linux-amd64.lock" + assert Platform.LINUX_ARM64.value == "linux-arm64" def test_parse_accepts_every_member_and_only_members() -> None: @@ -42,30 +43,77 @@ def _host(monkeypatch: pytest.MonkeyPatch, system: str, machine: str) -> None: monkeypatch.setattr(platforms.host_platform, "machine", lambda: machine) -@pytest.mark.parametrize("system,machine,emulated", [ - ("Linux", "x86_64", False), - ("Darwin", "x86_64", False), - ("Darwin", "arm64", True), +@pytest.mark.real_host_os +@pytest.mark.parametrize("system,machine,runnable", [ + ("Linux", "x86_64", ("linux-amd64",)), + ("Linux", "aarch64", ("linux-arm64",)), + ("Darwin", "x86_64", ("linux-amd64",)), + ("Darwin", "arm64", ("linux-arm64", "linux-amd64")), ]) -def test_current_is_the_capsule_platform_of_the_host(monkeypatch, system, machine, emulated) -> None: - # A Mac runs Linux capsules: the lock it reads is linux-amd64, natively - # on Intel and under Docker's emulation on Apple Silicon. +def test_each_host_runs_its_capsule_platforms_preferring_native(monkeypatch, system, machine, runnable) -> None: + # A Mac runs Linux capsules: Apple Silicon natively as linux-arm64 and + # under Docker's emulation as linux-amd64; Intel natively as linux-amd64. _host(monkeypatch, system, machine) - assert Platform.current() is Platform.LINUX_AMD64 - assert Platform.current().emulated() is emulated - assert Platform.current().docker_platform_args() == (["--platform=linux/amd64"] if emulated else []) + assert tuple(str(platform) for platform in Platform.runnable()) == runnable + assert Platform.current() is Platform.parse(runnable[0]) + assert not Platform.current().emulated() + assert Platform.current().docker_platform_args() == [] + + +@pytest.mark.real_host_os +def test_apple_silicon_names_the_emulated_platform_to_docker(monkeypatch) -> None: + _host(monkeypatch, "Darwin", "arm64") + assert Platform.LINUX_AMD64.emulated() + assert Platform.LINUX_AMD64.docker_platform_args() == ["--platform=linux/amd64"] + assert Platform.LINUX_ARM64.docker_platform_args() == [] -@pytest.mark.parametrize("system,machine", [("Linux", "aarch64"), ("Windows", "AMD64"), ("Darwin", "ppc")]) +@pytest.mark.real_host_os +@pytest.mark.parametrize("system,machine", [("Linux", "riscv64"), ("Windows", "AMD64"), ("Darwin", "ppc")]) def test_current_refuses_a_host_without_a_capsule_platform(monkeypatch, system, machine) -> None: _host(monkeypatch, system, machine) - with pytest.raises(UnsupportedPlatformError, match="supported platforms: linux-amd64"): + with pytest.raises(UnsupportedPlatformError, match="supported platforms: linux-amd64, linux-arm64"): Platform.current() +@pytest.mark.real_host_os +@pytest.mark.parametrize("locks,selected", [ + ((), "linux-arm64"), + (("linux-amd64",), "linux-amd64"), + (("linux-arm64",), "linux-arm64"), + (("linux-amd64", "linux-arm64"), "linux-arm64"), +]) +def test_apple_silicon_runs_a_project_as_the_platform_it_is_locked_for( + tmp_path: Path, monkeypatch, locks, selected, +) -> None: + # The human's ruling of 2026-11-04: natively when the project has an + # arm64 lock, else its amd64 lock under emulation; a new project native. + _host(monkeypatch, "Darwin", "arm64") + (tmp_path / ".devcapsule").mkdir() + for platform in locks: + (tmp_path / ".devcapsule" / f"devcapsule.{platform}.lock").write_text("") + assert Platform.select_for_project(tmp_path) is Platform.parse(selected) + assert Platform.current() is Platform.parse(selected) + Platform.clear_selection() + assert Platform.current() is Platform.LINUX_ARM64 + + +@pytest.mark.real_host_os +def test_a_host_selects_only_a_platform_it_runs(tmp_path: Path, monkeypatch) -> None: + # An amd64 Linux host cannot run an arm64-only project; it keeps its own + # platform, and the missing lock is reported where the lock is read. + _host(monkeypatch, "Linux", "x86_64") + (tmp_path / ".devcapsule").mkdir() + (tmp_path / ".devcapsule" / "devcapsule.linux-arm64.lock").write_text("") + assert Platform.select_for_project(tmp_path) is Platform.LINUX_AMD64 + assert Platform.select_for_project(None) is Platform.LINUX_AMD64 + + def test_docker_spelling_and_architecture() -> None: assert Platform.LINUX_AMD64.docker_platform == "linux/amd64" assert Platform.LINUX_AMD64.architecture == "amd64" + assert Platform.LINUX_ARM64.docker_platform == "linux/arm64" + assert Platform.LINUX_ARM64.architecture == "arm64" @pytest.mark.real_host_os diff --git a/devcapsule-src/tests/test_runtime_artifact.py b/devcapsule-src/tests/test_runtime_artifact.py index 32b1b357..fc7b85d9 100644 --- a/devcapsule-src/tests/test_runtime_artifact.py +++ b/devcapsule-src/tests/test_runtime_artifact.py @@ -147,3 +147,25 @@ def test_development_mode_removes_only_its_own_inherited_link(tmp_path, monkeypa else: assert command.read_text() == "development command" assert command.is_symlink() == (existing == "development-link") + + +@pytest.mark.real_host_os +def test_a_runtime_directory_serves_each_capsule_platform(tmp_path: Path, monkeypatch) -> None: + # Apple Silicon runs both platforms; one setting names both runtimes, and + # the project's locks choose which one a launch delivers. + _apple_silicon(monkeypatch) + monkeypatch.setenv("SCIE", str(native(tmp_path / "devcapsule-macos", MACH_O_ARM64))) + runtimes = tmp_path / "runtimes" + runtimes.mkdir() + amd64 = native(runtimes / "devcapsule-linux-amd64", ELF_AMD64) + arm64 = native(runtimes / "devcapsule-linux-arm64", ELF_ARM64) + monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(runtimes)) + assert runtime_artifact() == arm64 + project = tmp_path / "project" + (project / ".devcapsule").mkdir(parents=True) + (project / ".devcapsule" / "devcapsule.linux-amd64.lock").write_text("") + platforms.Platform.select_for_project(project) + assert runtime_artifact() == amd64 + amd64.unlink() + with pytest.raises(CliError, match="directory holding it as devcapsule-linux-amd64"): + runtime_artifact() From b6794612ae0527732f102c91f138d1ea2320e6b0 Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Wed, 4 Nov 2026 12:08:57 +0200 Subject: [PATCH 10/20] Pin the locally built arm64 base and let a command name its platform The linux-arm64 matrix gains its recipe-11 base: the amd64 recipe built natively on an Apple Silicon Mac by a launcher at public main b3f10b7d. It is not published. Its reference is the digest of the image index in the builder's containerd store, which a push of that exact image keeps; until the owner pushes it, a checkout selects the local image as a local base. `project --platform PLATFORM` names the capsule platform for one command, refused when the host cannot run it. `project --platform linux-arm64 init` is how a project locked for linux-amd64 gains an arm64 lock, since init completes what a project lacks; on Apple Silicon `--platform linux-amd64` runs a project under emulation on purpose. Co-Authored-By: Claude Opus 5.5 --- devcapsule-src/devcapsule/commands/project.py | 21 +++++- devcapsule-src/devcapsule/platforms.py | 11 +++ .../devcapsule/resolution_matrix.py | 35 +++++++++- .../tests/test_project_platform_selection.py | 67 +++++++++++++++++++ 4 files changed, 130 insertions(+), 4 deletions(-) create mode 100644 devcapsule-src/tests/test_project_platform_selection.py diff --git a/devcapsule-src/devcapsule/commands/project.py b/devcapsule-src/devcapsule/commands/project.py index 918d2350..1a728c08 100644 --- a/devcapsule-src/devcapsule/commands/project.py +++ b/devcapsule-src/devcapsule/commands/project.py @@ -2537,6 +2537,13 @@ def configure(cls, parser: argparse.ArgumentParser) -> None: type=Path, help="Project root or descendant; defaults to discovery from the current directory.", ) + parser.add_argument( + "--platform", + dest="capsule_platform", + metavar="PLATFORM", + help="Capsule platform for this command, such as linux-arm64; defaults to the first one " + "this host runs that the project has a lock for.", + ) @classmethod def make_context(cls, arguments: argparse.Namespace, parent: object | None) -> object | None: @@ -2577,7 +2584,7 @@ def make_context(cls, arguments: argparse.Namespace, parent: object | None) -> o context = replace(context, capsule_root=runtime_configuration.capsule_project_root(Path("."))) if access.needs_launcher: runtime_configuration.require_launcher(context.start_path(), tokens) - _select_capsule_platform(context) + _select_capsule_platform(context, getattr(arguments, "capsule_platform", None)) return context @classmethod @@ -2597,15 +2604,23 @@ def subcommands(cls) -> Mapping[str, type[Command] | type[Group]]: } -def _select_capsule_platform(context: ProjectCommandContext) -> None: +def _select_capsule_platform(context: ProjectCommandContext, named: str | None = None) -> None: """Run the selected project as the platform it is locked for on this host. See ``Platform.select_for_project``: where a host runs two platforms, the project's locks choose. Without an enclosing project, as for ``init`` in - a fresh directory, the host's preferred platform stands. + a fresh directory, the host's preferred platform stands. ``--platform`` + names one instead: ``project --platform linux-arm64 init`` adds that + platform's lock to a project, as init completes what a project lacks. """ from devcapsule.platforms import Platform, UnsupportedPlatformError + if named is not None: + try: + Platform.select(Platform.parse(named)) + except UnsupportedPlatformError as exc: + raise ProjectConfigurationError(str(exc)) from exc + return try: root: Path | None = discover_project(context.start_path()) except ProjectConfigurationError: diff --git a/devcapsule-src/devcapsule/platforms.py b/devcapsule-src/devcapsule/platforms.py index 52c17c94..04aa7e81 100644 --- a/devcapsule-src/devcapsule/platforms.py +++ b/devcapsule-src/devcapsule/platforms.py @@ -126,6 +126,17 @@ def select_for_project(cls, root: Path | None) -> Platform: _selected_platform.set(selected) return selected + @classmethod + def select(cls, platform: Platform) -> Platform: + """Select ``platform`` for this process, as the user named it; refuse one this host cannot run.""" + + runnable = cls.runnable() + if platform not in runnable: + names = ", ".join(str(member) for member in runnable) + raise UnsupportedPlatformError(f"This host runs {names} capsules, not {platform}.") + _selected_platform.set(platform) + return platform + @classmethod def clear_selection(cls) -> None: """Forget the project selection; ``current()`` is the host's preference again.""" diff --git a/devcapsule-src/devcapsule/resolution_matrix.py b/devcapsule-src/devcapsule/resolution_matrix.py index e9b19fa7..0c3fbe47 100644 --- a/devcapsule-src/devcapsule/resolution_matrix.py +++ b/devcapsule-src/devcapsule/resolution_matrix.py @@ -1702,13 +1702,46 @@ def _formation( }, ) +# The linux-arm64 base, recipe 11: the amd64 recipe-11 recipe built natively +# on an Apple Silicon Mac (Docker Desktop 29.3.1) on 2026-11-04 by +# `images build --type base --recipe ubuntu-24.04` from a launcher built at +# the public main commit b3f10b7d, whose devcapsule-webconsole the image +# installed. Inspected: linux/arm64, recipe 11 ready, display=contained, +# runtime=launcher-supplied, all six services, the web console root. +# NOT YET PUBLISHED. The reference is the digest of the OCI image index in +# the builder's containerd store, which a push of this exact image to +# mycodespaceai/devcapsule-base keeps; a rebuild gets another digest and +# must be pinned again. Until the owner pushes it, a checkout selects the +# local image: `project config authorize base-image `. +_LINUX_ARM64_RECIPE_11_BASE = _BasePin( + mnemonic="v0.3.1.dev0-local-macos-aarch64", + source_revision="b3f10b7d95aec659f268afecc03d0912a2e89d2f", + base_family=_BASE_FAMILY_UBUNTU_24_04, + satisfies={ + "python": "3.12", "docker-cli": None, "node": NODE_VERSION, + "java": TEMURIN_VERSION, "maven": MAVEN_VERSION, "postgresql-client": None, + }, + recipe=11, + display=CONTAINED_DISPLAY, + runtime=LAUNCHER_SUPPLIED_RUNTIME, + lock_table={ + "reference": ( + "docker.io/mycodespaceai/devcapsule-base" + "@sha256:7d32268065b60b6ad75139d23c1e6fbb2daa43b3f2c53d2856c5e2e39891652b" + ), + "build-mnemonic": "v0.3.1.dev0-local-macos-aarch64", + "contract": "ubuntu-24.04@11", + "recipe-source": RECIPE_SOURCE_PATH, + }, +) + # linux-arm64 offers the same vocabulary and the same materialization # recipes as linux-amd64; its bases, pins and verified edges are its own, # because a validation on one architecture establishes nothing on another. _LINUX_ARM64_MATRIX = ResolutionMatrix( platform=Platform.LINUX_ARM64, matrix_version=_MATRIX_VERSION, - bases=(), + bases=(_LINUX_ARM64_RECIPE_11_BASE,), components={}, edges=(), couplings=(), diff --git a/devcapsule-src/tests/test_project_platform_selection.py b/devcapsule-src/tests/test_project_platform_selection.py new file mode 100644 index 00000000..3c48e5dd --- /dev/null +++ b/devcapsule-src/tests/test_project_platform_selection.py @@ -0,0 +1,67 @@ +"""The capsule platform a project command runs as, chosen by the project's locks or by --platform.""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +from devcapsule import cli +from devcapsule.platforms import Platform + + +def _apple_silicon(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr(Platform, "runnable", classmethod(lambda cls: (Platform.LINUX_ARM64, Platform.LINUX_AMD64))) + + +def _project(root: Path, *platforms: str) -> Path: + (root / ".devcapsule").mkdir(parents=True) + (root / ".devcapsule" / "devcapsule.toml").write_text("") + for platform in platforms: + (root / ".devcapsule" / f"devcapsule.{platform}.lock").write_text("") + return root + + +def _selected_by(arguments: list[str], monkeypatch: pytest.MonkeyPatch) -> Platform: + seen: list[Platform] = [] + + def record(cls: object, arguments: object, context: object) -> int: + seen.append(Platform.current()) + return 0 + + monkeypatch.setattr("devcapsule.commands.project.ProjectInfoCommand.run", classmethod(record)) + assert cli.main(arguments) == 0 + return seen[0] + + +def test_a_project_command_runs_as_the_platform_the_project_is_locked_for( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, +) -> None: + _apple_silicon(monkeypatch) + amd64_only = _project(tmp_path / "amd64", "linux-amd64") + both = _project(tmp_path / "both", "linux-amd64", "linux-arm64") + assert _selected_by(["project", "--path", str(amd64_only), "info"], monkeypatch) is Platform.LINUX_AMD64 + assert _selected_by(["project", "--path", str(both), "info"], monkeypatch) is Platform.LINUX_ARM64 + + +def test_platform_option_names_the_platform_for_one_command(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + # `project --platform linux-arm64 init` is how an amd64 project gains an + # arm64 lock; `--platform linux-amd64` runs it under emulation on purpose. + _apple_silicon(monkeypatch) + amd64_only = _project(tmp_path / "amd64", "linux-amd64") + both = _project(tmp_path / "both", "linux-amd64", "linux-arm64") + assert _selected_by(["project", "--platform", "linux-arm64", "--path", str(amd64_only), "info"], + monkeypatch) is Platform.LINUX_ARM64 + assert _selected_by(["project", "--platform", "linux-amd64", "--path", str(both), "info"], + monkeypatch) is Platform.LINUX_AMD64 + + +def test_platform_option_refuses_a_platform_this_host_cannot_run( + tmp_path: Path, capsys: pytest.CaptureFixture[str], +) -> None: + # The suite's host runs linux-amd64 only. + project = _project(tmp_path / "project", "linux-amd64") + assert cli.main(["project", "--platform", "linux-arm64", "--path", str(project), "info"]) != 0 + assert "This host runs linux-amd64 capsules, not linux-arm64." in capsys.readouterr().err + assert cli.main(["project", "--platform", "windows-amd64", "--path", str(project), "info"]) != 0 + assert "Unsupported platform 'windows-amd64'" in capsys.readouterr().err From 667d00e23fbac482add3f5d838b9e1ea7508d036 Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Wed, 4 Nov 2026 12:16:05 +0200 Subject: [PATCH 11/20] Pin every component's arm64 build for linux-arm64 The linux-arm64 matrix now pins the same component versions as linux-amd64 with their arm64 artifacts: PyCharm, IntelliJ IDEA and Rider, the .NET SDK, Eclipse and its WebKitGTK package closure, VSCodium, VS Code, the Codex, Claude Code and Antigravity CLIs, and Playwright with Chromium. Each sha256 was computed from the downloaded bytes and matched every vendor checksum, npm integrity value and signed Ubuntu index available; components/linux_arm64_pins.py records where each came from. An amd64 version added without an arm64 pin fails at import. Ubuntu removed libsoup 3.4.4-5ubuntu0.8 from every noble pocket on 2026-10-08, so the arm64 Eclipse closure pins 3.4.4-5ubuntu0.10. The amd64 closure's libsoup URLs return 404 as well; that is maintenance's. The matrix has no verified edges yet: a validation on amd64 establishes nothing on arm64, so resolution needs --unverified until arm64 smokes record their evidence. Co-Authored-By: Claude Opus 5.5 --- .../devcapsule/components/linux_arm64_pins.py | 471 ++++++++++++++++++ .../devcapsule/resolution_matrix.py | 28 +- .../tests/test_linux_arm64_matrix.py | 76 +++ 3 files changed, 571 insertions(+), 4 deletions(-) create mode 100644 devcapsule-src/devcapsule/components/linux_arm64_pins.py create mode 100644 devcapsule-src/tests/test_linux_arm64_matrix.py diff --git a/devcapsule-src/devcapsule/components/linux_arm64_pins.py b/devcapsule-src/devcapsule/components/linux_arm64_pins.py new file mode 100644 index 00000000..4a1a19a5 --- /dev/null +++ b/devcapsule-src/devcapsule/components/linux_arm64_pins.py @@ -0,0 +1,471 @@ +"""linux-arm64 artifact pins, collected 2026-11-04. + +Each entry is the arm64 build of a version the linux-amd64 matrix pins: the +same version, the vendor's arm64 artifact, and a sha256 computed from the +downloaded bytes, which matched every vendor checksum, npm integrity value +and signed Ubuntu index available (Ubuntu's InRelease files verified with +gpgv against the archive signing key). Notes on particular entries: + +- Eclipse's WebKitGTK closure is the amd64 closure's package versions from + ports.ubuntu.com, except libsoup: Ubuntu removed 3.4.4-5ubuntu0.8 from + every noble pocket on 2026-10-08, so both libsoup packages are + 3.4.4-5ubuntu0.10 from noble-security. Architecture-independent packages + are the amd64 closure's own bytes. +- Playwright 1.63.0's Chromium comes from Chrome for Testing's linux-arm64 + build of 153.0.8010.12 on Playwright's CDN, byte-identical to Google's. +- Antigravity's arm64 builds live under ``linux-arm/``; VS Code's arm64 + archives are named by the update service, not by the x64 names' pattern. + +Keys are ``(component id, version)``, as the linux-amd64 matrix pins them. +""" + +from typing import Any + +ECLIPSE_NATIVE_PACKAGES: dict[str, dict[str, str]] = {'bubblewrap': {'sha256': 'dce3a2e5f0302eab717a64fe9c9411fe2d40469e2a7946e0f2ce1420a61025c3', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/b/bubblewrap/bubblewrap_0.9.0-1ubuntu0.3_arm64.deb', + 'version': '0.9.0-1ubuntu0.3'}, + 'dictionaries-common': {'sha256': '601e7cb4d662aca433785eaafd8094e51b1386aab5f5aef6572a1a565b80f21e', + 'url': 'https://archive.ubuntu.com/ubuntu/pool/main/d/dictionaries-common/dictionaries-common_1.29.7_all.deb', + 'version': '1.29.7'}, + 'emacsen-common': {'sha256': 'd802bbf9167ad1d8aa633b65684439e08672379f15bf752662d7fd7b9798dce1', + 'url': 'https://archive.ubuntu.com/ubuntu/pool/main/e/emacsen-common/emacsen-common_3.0.5_all.deb', + 'version': '3.0.5'}, + 'glib-networking': {'sha256': 'f229b9b01b384d73bf17d50a549802432bbcca36bb3eb74f0dac11f628f5c380', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/g/glib-networking/glib-networking_2.80.0-1build1_arm64.deb', + 'version': '2.80.0-1build1'}, + 'glib-networking-common': {'sha256': 'c9bc3b5c3dc8a75e752efcf246e5ef2d3b9e80cd1301fd37fcb0c36562062931', + 'url': 'https://archive.ubuntu.com/ubuntu/pool/main/g/glib-networking/glib-networking-common_2.80.0-1build1_all.deb', + 'version': '2.80.0-1build1'}, + 'glib-networking-services': {'sha256': 'a93e273d521c2a06ab579a17af4bfc1e4270f48b01af5cd957f7a1c82202c5fd', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/g/glib-networking/glib-networking-services_2.80.0-1build1_arm64.deb', + 'version': '2.80.0-1build1'}, + 'gsettings-desktop-schemas': {'sha256': '4d15441e9197f95b29c0d2845d8fcdbf63fa4ac0c68fb221820ccd24131e3436', + 'url': 'https://archive.ubuntu.com/ubuntu/pool/main/g/gsettings-desktop-schemas/gsettings-desktop-schemas_46.1-0ubuntu1_all.deb', + 'version': '46.1-0ubuntu1'}, + 'gstreamer1.0-plugins-base': {'sha256': 'dbbe644b135e95ef2cb035c27afb6eb05ba91c72ef63ef12a15de221a264616e', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/g/gst-plugins-base1.0/gstreamer1.0-plugins-base_1.24.2-1ubuntu0.5_arm64.deb', + 'version': '1.24.2-1ubuntu0.5'}, + 'gstreamer1.0-plugins-good': {'sha256': 'c1dba2da2de061f8df51eaedf3ee22ac4aefaae6356d79b0fd9918cb64b8beae', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/g/gst-plugins-good1.0/gstreamer1.0-plugins-good_1.24.2-1ubuntu1.8_arm64.deb', + 'version': '1.24.2-1ubuntu1.8'}, + 'hunspell-en-us': {'sha256': 'd231c94b54792215038a4f719c562deafe8efe0a7710c0c7218974cab513330c', + 'url': 'https://archive.ubuntu.com/ubuntu/pool/main/s/scowl/hunspell-en-us_2020.12.07-2_all.deb', + 'version': '1:2020.12.07-2'}, + 'iso-codes': {'sha256': 'cafca0a3a1f07b44fbba53ab7a1f1d9755e6b80cc041173afe2b52e53bed97a7', + 'url': 'https://archive.ubuntu.com/ubuntu/pool/main/i/iso-codes/iso-codes_4.16.0-1_all.deb', + 'version': '4.16.0-1'}, + 'libaa1': {'sha256': 'c6ba7e5a2d291a92fc3f9604ac298193e0102dc02c3b7d67c112819a7a9a2511', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/a/aalib/libaa1_1.4p5-51.1_arm64.deb', + 'version': '1.4p5-51.1'}, + 'libaspell15': {'sha256': '7a3da894c71c254c7ed7074284bb035eabb44f0e56cabaf96186e345813d1d6f', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/a/aspell/libaspell15_0.60.8.1-1build1_arm64.deb', + 'version': '0.60.8.1-1build1'}, + 'libasyncns0': {'sha256': 'acda11529a3bdc32961cec888060417d835be9626b00ab8f34607f7935cda11f', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/liba/libasyncns/libasyncns0_0.8-6build4_arm64.deb', + 'version': '0.8-6build4'}, + 'libavc1394-0': {'sha256': 'b707ec893678e9dec0b80e4481ed2dc581eac3d21de2be65984121640e3d5afe', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/liba/libavc1394/libavc1394-0_0.5.4-5build3_arm64.deb', + 'version': '0.5.4-5build3'}, + 'libcaca0': {'sha256': '95dffb776d959bee43bc2db755c1e4330547b9449eb20f48f8ba1dfddf435cf2', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libc/libcaca/libcaca0_0.99.beta20-4ubuntu0.2_arm64.deb', + 'version': '0.99.beta20-4ubuntu0.2'}, + 'libcdparanoia0': {'sha256': 'eb23ce43347dc8f734f66c24109198309d4844d9d9045ae9ca97cfa25061f63e', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/c/cdparanoia/libcdparanoia0_3.10.2%2bdebian-14build3_arm64.deb', + 'version': '3.10.2+debian-14build3'}, + 'libduktape207': {'sha256': '21cae9d4994eba43aa41623e55186df6434301faa47176196dbf33987550fec1', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/d/duktape/libduktape207_2.7.0%2btests-0ubuntu3_arm64.deb', + 'version': '2.7.0+tests-0ubuntu3'}, + 'libdv4t64': {'sha256': '0d91e6739333a460c61467e55156b38ac426e40c92bda606197e91b6f98864be', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libd/libdv/libdv4t64_1.0.0-17.1build1_arm64.deb', + 'version': '1.0.0-17.1build1'}, + 'libenchant-2-2': {'sha256': 'a8334fe77eabef07ead474657e38faeb7590da494d012480feb230e05a70b7ed', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/e/enchant-2/libenchant-2-2_2.3.3-2build2_arm64.deb', + 'version': '2.3.3-2build2'}, + 'libevdev2': {'sha256': '4fd17e552a00e3a2dc5d39a6578b26a1ac973afc80384d165647d72441f3a064', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libe/libevdev/libevdev2_1.13.1%2bdfsg-1build1_arm64.deb', + 'version': '1.13.1+dfsg-1build1'}, + 'libflac12t64': {'sha256': '9738c071c167b2baa8dd1a509d85076150108953a1429de839280f87b0b01b83', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/f/flac/libflac12t64_1.4.3%2bds-2.1ubuntu2_arm64.deb', + 'version': '1.4.3+ds-2.1ubuntu2'}, + 'libgstreamer-gl1.0-0': {'sha256': '569e78b1fbfb50fa57271157dc45cc3ebd682d3a8a940e1116cbf64c12cd043f', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/g/gst-plugins-base1.0/libgstreamer-gl1.0-0_1.24.2-1ubuntu0.5_arm64.deb', + 'version': '1.24.2-1ubuntu0.5'}, + 'libgstreamer-plugins-base1.0-0': {'sha256': 'cf52b2547943fc6444a6d674c2fe1fc2f9cc9ff1e3ff5f4a88b8f07da4a55dd4', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/g/gst-plugins-base1.0/libgstreamer-plugins-base1.0-0_1.24.2-1ubuntu0.5_arm64.deb', + 'version': '1.24.2-1ubuntu0.5'}, + 'libgstreamer-plugins-good1.0-0': {'sha256': 'a67bd34f000e34163221749b20aa605487ba1a3fd562a07b81d2060eb3006d5d', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/g/gst-plugins-good1.0/libgstreamer-plugins-good1.0-0_1.24.2-1ubuntu1.8_arm64.deb', + 'version': '1.24.2-1ubuntu1.8'}, + 'libgstreamer1.0-0': {'sha256': '48c930493054b261621ca7ea887ee4c49488bb216f0a42d8572189e5a3e291f0', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/g/gstreamer1.0/libgstreamer1.0-0_1.24.2-1ubuntu0.1_arm64.deb', + 'version': '1.24.2-1ubuntu0.1'}, + 'libgudev-1.0-0': {'sha256': 'ce395eea11fdbd542597c35d10cb931e288243467e9d2e14654225cd8e80dac5', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libg/libgudev/libgudev-1.0-0_238-5ubuntu1_arm64.deb', + 'version': '1:238-5ubuntu1'}, + 'libharfbuzz-icu0': {'sha256': '92f1311987c69b90432408c19e28b4796ff7584927cb55c9cec74bd3bab9efb4', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/h/harfbuzz/libharfbuzz-icu0_8.3.0-2build2_arm64.deb', + 'version': '8.3.0-2build2'}, + 'libhunspell-1.7-0': {'sha256': '905d26e0c736b7334027ff9a0e12f8faa913d33ff861c0ca5c9c20b43071bca7', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/h/hunspell/libhunspell-1.7-0_1.7.2%2breally1.7.2-10build3_arm64.deb', + 'version': '1.7.2+really1.7.2-10build3'}, + 'libhyphen0': {'sha256': 'ac159cf84661f8027d0f4274d9ff559eb41ab9e2a476d5d63cf2573bda542c94', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/h/hyphen/libhyphen0_2.8.8-7build3_arm64.deb', + 'version': '2.8.8-7build3'}, + 'libiec61883-0': {'sha256': '83644951ab8aff10f530066e74129431f723dc5b810e14025520b4f70bb58bbe', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libi/libiec61883/libiec61883-0_1.2.0-6build1_arm64.deb', + 'version': '1.2.0-6build1'}, + 'libjavascriptcoregtk-4.1-0': {'sha256': '936ed381c7cfa24ee753c01da8c4bc539e8911d696c95e2bc9ded98e44cd47ca', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/w/webkit2gtk/libjavascriptcoregtk-4.1-0_2.52.6-0ubuntu0.24.04.1_arm64.deb', + 'version': '2.52.6-0ubuntu0.24.04.1'}, + 'libmanette-0.2-0': {'sha256': '9e5585c1d791e192f48ed4d6579c5dbdbc79f2330afc4f7998fd5e72e654bf2b', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libm/libmanette/libmanette-0.2-0_0.2.7-1build2_arm64.deb', + 'version': '0.2.7-1build2'}, + 'libmp3lame0': {'sha256': 'a12e5e953294f424fba8a9fa0c47e081a0216150b277c4b02da1468d4b78507f', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/l/lame/libmp3lame0_3.100-6build1_arm64.deb', + 'version': '3.100-6build1'}, + 'libmpg123-0t64': {'sha256': '7f78498103c42cebe15f1283fb2dd98286069a59f29aca4de372bc98f70d06d2', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/m/mpg123/libmpg123-0t64_1.32.5-1ubuntu1.1_arm64.deb', + 'version': '1.32.5-1ubuntu1.1'}, + 'libogg0': {'sha256': 'ed92eb736754ccdea645db7a12734961d2e686369f4fb4c070126b70462b02e1', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libo/libogg/libogg0_1.3.5-3build1_arm64.deb', + 'version': '1.3.5-3build1'}, + 'libopus0': {'sha256': '9e0b07689ce79772e62f8d4d1849bde1b1fc5b041bef85b791b61d7e3fee0ff7', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/o/opus/libopus0_1.4-1build1_arm64.deb', + 'version': '1.4-1build1'}, + 'liborc-0.4-0t64': {'sha256': 'fb8248d89d37a06145b8112aed200ffd2f4960d294d95ddc8471a0b43beafb3c', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/o/orc/liborc-0.4-0t64_0.4.38-1ubuntu0.1_arm64.deb', + 'version': '1:0.4.38-1ubuntu0.1'}, + 'libproxy1v5': {'sha256': '4ea61580d2b5c6ec4f05445ed63618130743256d0208501cd5af14c28764040b', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libp/libproxy/libproxy1v5_0.5.4-4build1_arm64.deb', + 'version': '0.5.4-4build1'}, + 'libpulse0': {'sha256': '863bf58ca712b687873c9f57bac2968555b0f1bcdeeb79265da4130d0102fea3', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/p/pulseaudio/libpulse0_16.1%2bdfsg1-2ubuntu10.1_arm64.deb', + 'version': '1:16.1+dfsg1-2ubuntu10.1'}, + 'libraw1394-11': {'sha256': '3a55e774189eb21b220e5cbf14a635e627f370dba0a52bbe6eea6b5a785e0a87', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libr/libraw1394/libraw1394-11_2.1.2-2build3_arm64.deb', + 'version': '2.1.2-2build3'}, + 'libsecret-1-0': {'sha256': 'b489847c20b4990e9e97dc701215c2af40ac071d9f1d465196a8a215b17ae2a4', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libs/libsecret/libsecret-1-0_0.21.4-1build3_arm64.deb', + 'version': '0.21.4-1build3'}, + 'libsecret-common': {'sha256': '41866e9026e451fd25a4cbe887fa56c43219f9f3eda88d1831452ba6d6b8bb5b', + 'url': 'https://archive.ubuntu.com/ubuntu/pool/main/libs/libsecret/libsecret-common_0.21.4-1build3_all.deb', + 'version': '0.21.4-1build3'}, + 'libshout3': {'sha256': '3099f4bdaf6bf7ee418a4db7b7d3a191c9a36902164dc22d62e7e10b40ffe94f', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libs/libshout/libshout3_2.4.6-1build2_arm64.deb', + 'version': '2.4.6-1build2'}, + 'libslang2': {'sha256': '77b4b81dd3d5783772dc0b742c79680ce1eef18a771cf1f421e9a9e5eaf2d71e', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/s/slang2/libslang2_2.3.3-3build2_arm64.deb', + 'version': '2.3.3-3build2'}, + 'libsndfile1': {'sha256': '3fc5c0cb78110c739a0aa9cedad1812c04a5b3ee9068a4775233b51d8dc5c0f7', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libs/libsndfile/libsndfile1_1.2.2-1ubuntu5.24.04.1_arm64.deb', + 'version': '1.2.2-1ubuntu5.24.04.1'}, + 'libsoup-3.0-0': {'sha256': '3831f49dc17ed43600c531d3d77c5bcec83e3aba6bc719fc0e83c58234115595', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libs/libsoup3/libsoup-3.0-0_3.4.4-5ubuntu0.10_arm64.deb', + 'version': '3.4.4-5ubuntu0.10'}, + 'libsoup-3.0-common': {'sha256': '89f066fe021cb2991750dc9a65fb64cc24a467ab32b5a1a722e377244b6fda4c', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libs/libsoup3/libsoup-3.0-common_3.4.4-5ubuntu0.10_all.deb', + 'version': '3.4.4-5ubuntu0.10'}, + 'libspeex1': {'sha256': '8ba1a6feb898b9f28f1c071c96ad0a992b4d449ad75aecb791d1ebd2a43c0ada', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/s/speex/libspeex1_1.2.1-2ubuntu2.24.04.1_arm64.deb', + 'version': '1.2.1-2ubuntu2.24.04.1'}, + 'libtag1v5': {'sha256': 'd6d23bb62694e291775b65ce618a04393d9b2ca61c88c765a05cd9df6e1cba81', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/t/taglib/libtag1v5_1.13.1-1build1_arm64.deb', + 'version': '1.13.1-1build1'}, + 'libtag1v5-vanilla': {'sha256': '27db0f02f005a969e7e557e3ca7670a1f215814f4f446bab155446ace17d496c', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/t/taglib/libtag1v5-vanilla_1.13.1-1build1_arm64.deb', + 'version': '1.13.1-1build1'}, + 'libtext-iconv-perl': {'sha256': '25e749dbe3331ec8c04dfde36cc3e24d4871a2f7a46dff4282d521973f4698b6', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libt/libtext-iconv-perl/libtext-iconv-perl_1.7-8build3_arm64.deb', + 'version': '1.7-8build3'}, + 'libtheora0': {'sha256': '78ebaa1c851465dac9e13532623a4e41d28ed55f3df0353daf7c29d96a2e2b87', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libt/libtheora/libtheora0_1.1.1%2bdfsg.1-16.1build3_arm64.deb', + 'version': '1.1.1+dfsg.1-16.1build3'}, + 'libtwolame0': {'sha256': 'f2fffdedec9f9bc58fc700ccbc146c50a5a1d5ae2e6881e501bd727234aa5d16', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/t/twolame/libtwolame0_0.4.0-2build3_arm64.deb', + 'version': '0.4.0-2build3'}, + 'libv4l-0t64': {'sha256': '0efed54458a4a3da075defb2675b243d7123115d26219ea2ffb88750e777f0e3', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/v/v4l-utils/libv4l-0t64_1.26.1-4build3_arm64.deb', + 'version': '1.26.1-4build3'}, + 'libv4lconvert0t64': {'sha256': 'cfb01d4343d1ed4a92476df431d4ef89f6871708c8befd6ef8e711a40dfc0808', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/v/v4l-utils/libv4lconvert0t64_1.26.1-4build3_arm64.deb', + 'version': '1.26.1-4build3'}, + 'libvisual-0.4-0': {'sha256': 'c9b6a8bcf0e9d57c1a32e363dc391d3ad3db1a01b51a406b5c847d32680baa72', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libv/libvisual/libvisual-0.4-0_0.4.2-2build1_arm64.deb', + 'version': '0.4.2-2build1'}, + 'libvorbis0a': {'sha256': 'e21bd74f3bf593b4697f77c6b52bfbd34cb5dfc5ed5f115545d31e43599c1ef8', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libv/libvorbis/libvorbis0a_1.3.7-1build3_arm64.deb', + 'version': '1.3.7-1build3'}, + 'libvorbisenc2': {'sha256': 'a6ab660d303d44fbf040c07816a7072a40240d1639c37f6246624dba8d28407a', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libv/libvorbis/libvorbisenc2_1.3.7-1build3_arm64.deb', + 'version': '1.3.7-1build3'}, + 'libvpx9': {'sha256': '0a11569cb76d303978bb8d1a97cb012d96baadb5e093e6936f5b53de802241ec', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libv/libvpx/libvpx9_1.14.0-1ubuntu2.3_arm64.deb', + 'version': '1.14.0-1ubuntu2.3'}, + 'libwavpack1': {'sha256': 'fd86ac4ecc6aaa218744e38ba7e8c0d4bb13f45c5508c3247e059fdcee33ad55', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/w/wavpack/libwavpack1_5.6.0-1build1_arm64.deb', + 'version': '5.6.0-1build1'}, + 'libwayland-server0': {'sha256': '2b6dec0d40070685f4ce346d20b064fc5e118a3149f30be88465251985b496ed', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/w/wayland/libwayland-server0_1.22.0-2.1build1_arm64.deb', + 'version': '1.22.0-2.1build1'}, + 'libwebkit2gtk-4.1-0': {'sha256': '508f97ca47c45464c0e43480ebe983a0b4b484dd1a76031272f762662943b7c9', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/w/webkit2gtk/libwebkit2gtk-4.1-0_2.52.6-0ubuntu0.24.04.1_arm64.deb', + 'version': '2.52.6-0ubuntu0.24.04.1'}, + 'libxslt1.1': {'sha256': '1e8b809859ddf2c827de14152741e74ea2f83907c7c06cba470c6cc75624ba49', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libx/libxslt/libxslt1.1_1.1.39-0exp1ubuntu0.24.04.3_arm64.deb', + 'version': '1.1.39-0exp1ubuntu0.24.04.3'}, + 'session-migration': {'sha256': 'b310677c0cdd3410d13ae66515c5c0836ea0c36e02f2dbde89a9c76ebcefd565', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/s/session-migration/session-migration_0.3.9build1_arm64.deb', + 'version': '0.3.9build1'}, + 'xdg-dbus-proxy': {'sha256': '9ff35a693536fd898ed78be84c45a668e5edbfa30dd53884004ebef7212deebf', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/x/xdg-dbus-proxy/xdg-dbus-proxy_0.1.5-1ubuntu0.2_arm64.deb', + 'version': '0.1.5-1ubuntu0.2'}} + +PLAYWRIGHT_PIN: dict[str, Any] = {'version': '1.63.0', + 'delivery-policy': 'local-materialization', + 'license': 'Apache-2.0', + 'platform': 'linux-arm64', + 'artifacts': {'greenlet': {'format': 'python-wheel', + 'url': 'https://files.pythonhosted.org/packages/27/89/366d2af5061eeefa5012f510d95a99c8620dcc457609838db4d538820318/greenlet-3.5.6-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl', + 'sha256': 'e85880b538e59a59f55117b81f208a6660ad5ac328aad9305f812d9b8bc67a0f', + 'destination': '/opt/playwright'}, + 'playwright': {'format': 'python-wheel', + 'url': 'https://files.pythonhosted.org/packages/f3/82/3d85505284c5a210f2da6c07b8f757524e79d1fba9cfdafe1eafb766ae59/playwright-1.63.0-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl', + 'sha256': '354e15b29503565fc598b89f16fbe070459343bef9d7498a93e304864000c6a7', + 'destination': '/opt/playwright'}, + 'pyee': {'format': 'python-wheel', + 'url': 'https://files.pythonhosted.org/packages/a0/c4/b4d4827c93ef43c01f599ef31453ccc1c132b353284fc6c87d535c233129/pyee-13.0.1-py3-none-any.whl', + 'sha256': 'af2f8fede4171ef667dfded53f96e2ed0d6e6bd7ee3bb46437f77e3b57689228', + 'destination': '/opt/playwright'}, + 'typing_extensions': {'format': 'python-wheel', + 'url': 'https://files.pythonhosted.org/packages/49/d3/b8441a820a491ddfc024b0b0cf0393375b75ea13866d9c66727e54c2fc80/typing_extensions-4.16.0-py3-none-any.whl', + 'sha256': '481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8', + 'destination': '/opt/playwright'}, + 'chromium-1243': {'format': 'zip-directory', + 'url': 'https://cdn.playwright.dev/builds/cft/153.0.8010.12/linux-arm64/chrome-linux-arm64.zip', + 'sha256': '7d8a4b4ff289efe44a06501a519df142c4c18fff7aaf1c1401a3fbb12b3bd069', + 'destination': '/opt/playwright/browsers/chromium-1243'}, + 'chromium_headless_shell-1243': {'format': 'zip-directory', + 'url': 'https://cdn.playwright.dev/builds/cft/153.0.8010.12/linux-arm64/chrome-headless-shell-linux-arm64.zip', + 'sha256': 'd433c45172c7836e38124fe545f767b02210bfb43a6262f08a297473a8e91c99', + 'destination': '/opt/playwright/browsers/chromium_headless_shell-1243'}, + 'ffmpeg-1011': {'format': 'zip-directory', + 'url': 'https://cdn.playwright.dev/dbazure/download/playwright/builds/ffmpeg/1011/ffmpeg-linux-arm64.zip', + 'sha256': '2628c03f05318ff812c8c9baaf207dea2ddf53e818c0dc936714b0fbe3afb009', + 'destination': '/opt/playwright/browsers/ffmpeg-1011'}}} + +COMPONENT_LOCK_TABLES: dict[tuple[str, str], dict[str, Any]] = { + ('pycharm', '2026.2.0.1'): {'version': '2026.2.0.1', + 'variant': 'professional', + 'delivery-policy': 'local-materialization', + 'url': 'https://download.jetbrains.com/python/pycharm-2026.2.0.1-aarch64.tar.gz', + 'sha256': 'f29b6a782a58fabb232715db8fef9765bfdaebc6eb590ed462c4f96562227427'}, + ('pycharm', '2026.2.4'): {'version': '2026.2.4', + 'variant': 'professional', + 'delivery-policy': 'local-materialization', + 'url': 'https://download.jetbrains.com/python/pycharm-2026.2.4-aarch64.tar.gz', + 'sha256': '9ce06a9551e17c68a5741ee70ae4203dcc530935fa5e105df171d600e55a2bbf'}, + ('intellij', '2026.2.3'): {'version': '2026.2.3', + 'variant': 'unified', + 'delivery-policy': 'local-materialization', + 'url': 'https://download.jetbrains.com/idea/idea-2026.2.3-aarch64.tar.gz', + 'sha256': '873286dd6406971a2311827edd66c56060c345582a4a6b1774e3925183b02ec5'}, + ('intellij', '2026.2.4'): {'version': '2026.2.4', + 'variant': 'unified', + 'delivery-policy': 'local-materialization', + 'url': 'https://download.jetbrains.com/idea/idea-2026.2.4-aarch64.tar.gz', + 'sha256': '158b97dfbc3a88075fa14fcf5f9f2c17b339f4527e4245b3a33caf18050c2ae2'}, + ('eclipse', '2026-09-R'): {'version': '2026-09-R', + 'variant': 'java', + 'delivery-policy': 'local-materialization', + 'terms-url': 'https://www.eclipse.org/legal/epl-2.0/', + 'url': 'https://download.eclipse.org/technology/epp/downloads/release/2026-09/R/eclipse-java-2026-09-R-linux-gtk-aarch64.tar.gz', + 'sha256': '37f8bff91ce2af38bb691ab0608e1184af48cb48ad04da2bb09a9568ba905f8d', + 'upstream-sha512': 'c95453e0170efff6294a746b53e2f83f857aeb2309dbc20b933de18664f7d7e717f071a08a96d46797c2191014e1c8505da8370e45fff83a91396636d29a3091'}, + ('rider', '2026.2.3.1'): {'version': '2026.2.3.1', + 'delivery-policy': 'local-materialization', + 'license': 'Proprietary', + 'terms-url': 'https://www.jetbrains.com/legal/docs/toolbox/license/', + 'url': 'https://download.jetbrains.com/rider/JetBrains.Rider-2026.2.3.1-aarch64.tar.gz', + 'sha256': '8239c1e7c0352a9fb79dcbf68fb9550d64b215606a635ed88981291d92995a61'}, + ('rider', '2026.2.4'): {'version': '2026.2.4', + 'delivery-policy': 'local-materialization', + 'license': 'Proprietary', + 'terms-url': 'https://www.jetbrains.com/legal/docs/toolbox/license/', + 'url': 'https://download.jetbrains.com/rider/JetBrains.Rider-2026.2.4-aarch64.tar.gz', + 'sha256': 'adaaadc42dbfe2efa8e3aa96e487ba7b7ea33858b6bc1aad8ac5024205555ecd'}, + ('dotnet-sdk', '10.0.401'): {'version': '10.0.401', + 'platform': 'linux-arm64', + 'delivery-policy': 'local-materialization', + 'license': 'MIT', + 'url': 'https://builds.dotnet.microsoft.com/dotnet/Sdk/10.0.401/dotnet-sdk-10.0.401-linux-arm64.tar.gz', + 'sha256': '91d3d67f2ed065909bd2e1ba50a37192aff6df15178304939d5b55634d72da6a', + 'upstream-sha512': '58ace73ced6b4360754689a686bdfb8a317f4da6cb8bb416dbc7d0ba9f47e43e3c09f5eb1f1a1cfaacbd10df9558da4882bf2a5e195d6ab56a02c1f9f76102ed'}, + ('codium', '1.126.04524'): {'version': '1.126.04524', + 'delivery-policy': 'local-materialization', + 'license': 'MIT', + 'url': 'https://github.com/VSCodium/vscodium/releases/download/1.126.04524/VSCodium-linux-arm64-1.126.04524.tar.gz', + 'sha256': '73d87d46d4dc208fe12c0497dc607aab0a6e2bf332f54a0b6826a3a1aa32bc34'}, + ('codium', '1.143.17232'): {'version': '1.143.17232', + 'delivery-policy': 'local-materialization', + 'license': 'MIT', + 'url': 'https://github.com/VSCodium/vscodium/releases/download/1.143.17232/VSCodium-linux-arm64-1.143.17232.tar.gz', + 'sha256': '9f270ae242561812acd02be9cf5c13c848da359b356327adfe44b815a9532309'}, + ('vscode', '1.143.1'): {'version': '1.143.1', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'vscode-download', + 'license': 'Proprietary', + 'terms-url': 'https://code.visualstudio.com/license', + 'url': 'https://vscode.download.prss.microsoft.com/dbazure/download/stable/e646868227c3e3a9e3f24277c8c7220be42c0be7/code-stable-arm64-1792691733.tar.gz', + 'sha256': 'e84bf20d112fdb73c29b85b317ba50d5a48bea05c0711509669c8466639e9eb6'}, + ('vscode', '1.144.0'): {'version': '1.144.0', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'vscode-download', + 'license': 'Proprietary', + 'terms-url': 'https://code.visualstudio.com/license', + 'url': 'https://vscode.download.prss.microsoft.com/dbazure/download/stable/2c33b50b16c472efe58357d1d2a8a5414bc23fbf/code-stable-arm64-1793133773.tar.gz', + 'sha256': 'c1758f2144c68b295fb2a3543e93bc03ab99f8ea715cfd0560377092aba0f198'}, + ('codex', '0.145.0'): {'version': '0.145.0', + 'delivery-policy': 'local-materialization', + 'license': 'Apache-2.0', + 'npm-package': '@openai/codex', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.145.0.tgz', + 'sha256': '416399796cac371d1a033b17f34b08ba9b25c8f298a5b9d00e10f72c3b128c8d', + 'artifacts': {'linux-arm64': {'npm-package': '@openai/codex-linux-arm64', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.145.0-linux-arm64.tgz', + 'sha256': 'b78c57e172b2f18e5969ae26183253cd3cdd9abb3b424a8f7334f4b5530c2b27'}}}, + ('codex', '0.153.0'): {'version': '0.153.0', + 'delivery-policy': 'local-materialization', + 'license': 'Apache-2.0', + 'npm-package': '@openai/codex', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.153.0.tgz', + 'sha256': '0dc1968cc6075929d70d7ab1421122743a9f8237a8cc9ac69e2e8f2768798fef', + 'artifacts': {'linux-arm64': {'npm-package': '@openai/codex-linux-arm64', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.153.0-linux-arm64.tgz', + 'sha256': '1c65fda30a919908688147b67b1b195ab635bd4a5eac49ffa7be527a13af0a09'}}}, + ('codex', '0.153.4'): {'version': '0.153.4', + 'delivery-policy': 'local-materialization', + 'license': 'Apache-2.0', + 'npm-package': '@openai/codex', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.153.4.tgz', + 'sha256': 'fd04263c1adfa1d285c6c0ad86a97cab508d3012ee9eab80a99f773cc4b2fb3a', + 'artifacts': {'linux-arm64': {'npm-package': '@openai/codex-linux-arm64', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.153.4-linux-arm64.tgz', + 'sha256': '439c0dd0d6923f607b4e5cd1e3079c12f0b86f6e5007f07e377d6ad25e2d7bb9'}}}, + ('codex', '0.157.1'): {'version': '0.157.1', + 'delivery-policy': 'local-materialization', + 'license': 'Apache-2.0', + 'npm-package': '@openai/codex', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.157.1.tgz', + 'sha256': '813e2a944f4474b7e1826d9ce8bf696ad8d80c68f43f7a7067beec51fa8f5b73', + 'artifacts': {'linux-arm64': {'npm-package': '@openai/codex-linux-arm64', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.157.1-linux-arm64.tgz', + 'sha256': 'b73d482c0963a69d5cece5fe16e919473359937c22fb4499702ea35870e1824f'}}}, + ('codex', '0.163.0'): {'version': '0.163.0', + 'delivery-policy': 'local-materialization', + 'license': 'Apache-2.0', + 'npm-package': '@openai/codex', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.163.0.tgz', + 'sha256': '1a71db36560bb446d2c2e63ae675c4a3018c38ace38ca533f8d1b25d3e6555b4', + 'artifacts': {'linux-arm64': {'npm-package': '@openai/codex-linux-arm64', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.163.0-linux-arm64.tgz', + 'sha256': 'dc6e7c4d6de71c23a0f60d05e2577ff7eef615b76f22630d58ad384b84763522'}}}, + ('codex', '0.166.0'): {'version': '0.166.0', + 'delivery-policy': 'local-materialization', + 'license': 'Apache-2.0', + 'npm-package': '@openai/codex', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.166.0.tgz', + 'sha256': '67ee6f4069d9328956401f54ac202c20369233159058af9f4ec1af731b87f5b7', + 'artifacts': {'linux-arm64': {'npm-package': '@openai/codex-linux-arm64', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.166.0-linux-arm64.tgz', + 'sha256': '9044e840397864fc6c539d13719b25fb26abb35d5353366cbabf7b4555406acd'}}}, + ('claude-code', '2.1.227'): {'version': '2.1.227', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'claude-code-download', + 'license': 'Proprietary', + 'terms-url': 'https://www.anthropic.com/legal/commercial-terms', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://downloads.claude.ai/claude-code-releases/2.1.227/linux-arm64/claude', + 'sha256': 'db47335532cbcab67a4b3ab16d8f3f77976bf85d53c7d79f8296538aa22bfce6'}}}, + ('claude-code', '2.1.236'): {'version': '2.1.236', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'claude-code-download', + 'license': 'Proprietary', + 'terms-url': 'https://www.anthropic.com/legal/commercial-terms', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://downloads.claude.ai/claude-code-releases/2.1.236/linux-arm64/claude', + 'sha256': 'c38d37deaf1643083326c48a6acc0afb09dada126e6bda77ef1a4410ae60ca12'}}}, + ('claude-code', '2.1.261'): {'version': '2.1.261', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'claude-code-download', + 'license': 'Proprietary', + 'terms-url': 'https://www.anthropic.com/legal/commercial-terms', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://downloads.claude.ai/claude-code-releases/2.1.261/linux-arm64/claude', + 'sha256': '7bbed5a9b0fc2e4ec67bad3490d06ca91b86d6b037d47520b7898951757d1b8a'}}}, + ('claude-code', '2.1.283'): {'version': '2.1.283', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'claude-code-download', + 'license': 'Proprietary', + 'terms-url': 'https://www.anthropic.com/legal/commercial-terms', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://downloads.claude.ai/claude-code-releases/2.1.283/linux-arm64/claude', + 'sha256': '346d294f0103d6fc0de11ac953579b5c62dfa90698a4cfc486b6f927c615e697'}}}, + ('claude-code', '2.1.302'): {'version': '2.1.302', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'claude-code-download', + 'license': 'Proprietary', + 'terms-url': 'https://www.anthropic.com/legal/commercial-terms', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://downloads.claude.ai/claude-code-releases/2.1.302/linux-arm64/claude', + 'sha256': '2bd4f4e5ed5cb09955b490098f745321cb5491e7ea8c752ba1e3bb010cdadb10'}}}, + ('claude-code', '2.1.314'): {'version': '2.1.314', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'claude-code-download', + 'license': 'Proprietary', + 'terms-url': 'https://www.anthropic.com/legal/commercial-terms', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://downloads.claude.ai/claude-code-releases/2.1.314/linux-arm64/claude', + 'sha256': 'a66c0a8f5d0c87a439953d797985192bbcbaefbce2e0d7b2eaa8df30ef0f3ef2'}}}, + ('antigravity-cli', '1.1.24'): {'version': '1.1.24', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'antigravity-download', + 'license': 'Proprietary', + 'terms-url': 'https://antigravity.google/terms/', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://storage.googleapis.com/antigravity-public/antigravity-cli/1.1.24-6130423206641664/linux-arm/cli_linux_arm64.tar.gz', + 'sha256': 'e601bde6dcb9caebf8a83de235a0e10a21192c7f9ca9ed459a120714e7d42399', + 'archive-member': 'antigravity', + 'upstream-sha512': '316ca00d50389a08b162c66066b4e2db201e4ffb85acea05029e3c4532c69d5b8f7c741cf027325889f898ea8f747af8cd15c802e15fcf5d73b7137b6e2420a1'}}}, + ('antigravity-cli', '1.2.12'): {'version': '1.2.12', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'antigravity-download', + 'license': 'Proprietary', + 'terms-url': 'https://antigravity.google/terms/', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://storage.googleapis.com/antigravity-public/antigravity-cli/1.2.12-5784551402897408/linux-arm/cli_linux_arm64.tar.gz', + 'sha256': 'bd338c9d19ab963d9d2bc027e4e797b470ea84bae02080e4fde4555357ea9444', + 'archive-member': 'antigravity', + 'upstream-sha512': 'e2f10960197efbf2455edb1284ae686821e244908367f507bcb6a0d0f616e945d3eec5cdf29b41bbcd7f630826d0e9e233f457184d1696fa91e81d6a7ed440c9'}}}, + ('antigravity-cli', '1.4.2'): {'version': '1.4.2', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'antigravity-download', + 'license': 'Proprietary', + 'terms-url': 'https://antigravity.google/terms/', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://storage.googleapis.com/antigravity-public/antigravity-cli/1.4.2-4835981233356800/linux-arm/cli_linux_arm64.tar.gz', + 'sha256': '08dd4a2ca52d531fef1dfae4ec2fbafcd56342bfa694c8193a60da2b26fb1898', + 'archive-member': 'antigravity', + 'upstream-sha512': 'f4655468d3303e6039cbcf1a30769e3cb5c4e16d149fba0d08e83f91f0f2b07cb623a4adb8615ca7a8fc73bf2445301e0e1ab82ea89f6aa44cbd9a9e96703e19'}}}, + ('antigravity-cli', '1.4.12'): {'version': '1.4.12', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'antigravity-download', + 'license': 'Proprietary', + 'terms-url': 'https://antigravity.google/terms/', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://storage.googleapis.com/antigravity-public/antigravity-cli/1.4.12-4768735748292608/linux-arm/cli_linux_arm64.tar.gz', + 'sha256': '30dd61ebe1f0a8829057ea2de29dad08c710dd972ce4c02949f51e632135a86b', + 'archive-member': 'antigravity', + 'upstream-sha512': '7cae317fb6a02eb387982f06bfb234b366879cf9a80ad8969b571e5a55d1e37a6a5a49cd6b703c0443bb417937cf5e3b14a8c80cb26917ad79eef723a9f374c7'}}}, +} +COMPONENT_LOCK_TABLES[("eclipse", "2026-09-R")]["native-packages"] = ECLIPSE_NATIVE_PACKAGES +COMPONENT_LOCK_TABLES[("playwright", PLAYWRIGHT_PIN["version"])] = PLAYWRIGHT_PIN diff --git a/devcapsule-src/devcapsule/resolution_matrix.py b/devcapsule-src/devcapsule/resolution_matrix.py index 0c3fbe47..99ec6220 100644 --- a/devcapsule-src/devcapsule/resolution_matrix.py +++ b/devcapsule-src/devcapsule/resolution_matrix.py @@ -59,6 +59,7 @@ from devcapsule.components.catalog import COMPONENTS from devcapsule.components.playwright_pin import PIN as PLAYWRIGHT_PIN from devcapsule.components.eclipse_native_pin import PACKAGES as ECLIPSE_NATIVE_PACKAGES +from devcapsule.components.linux_arm64_pins import COMPONENT_LOCK_TABLES as LINUX_ARM64_LOCK_TABLES from devcapsule.configuration.file_formats import ( ProjectConfigurationError, canonical_digest, @@ -1735,14 +1736,33 @@ def _formation( }, ) -# linux-arm64 offers the same vocabulary and the same materialization -# recipes as linux-amd64; its bases, pins and verified edges are its own, -# because a validation on one architecture establishes nothing on another. +def _on_linux_arm64(pin: _ComponentPin) -> _ComponentPin: + """The arm64 build of a linux-amd64 pin: same version, its own artifacts. + + A base-delivered component carries no artifact and is the same pin on + both platforms; every other pin must have its arm64 table, so an amd64 + version added without one fails here rather than at a user's launch. + """ + + table = LINUX_ARM64_LOCK_TABLES.get((pin.component_id, pin.version)) + if table is None and pin.lock_table.get("delivery-policy") == "base-image": + return pin + if table is None: + raise RuntimeError(f"{pin.component_id} {pin.version} has no linux-arm64 pin in components/linux_arm64_pins.py") + return _ComponentPin(pin.component_id, pin.version, table) + + +# linux-arm64 offers the same vocabulary, versions and materialization +# recipes as linux-amd64; its bases, artifacts and verified edges are its +# own, because a validation on one architecture establishes nothing on another. _LINUX_ARM64_MATRIX = ResolutionMatrix( platform=Platform.LINUX_ARM64, matrix_version=_MATRIX_VERSION, bases=(_LINUX_ARM64_RECIPE_11_BASE,), - components={}, + components={ + component: tuple(_on_linux_arm64(pin) for pin in pins) + for component, pins in _LINUX_AMD64_MATRIX._components.items() + }, edges=(), couplings=(), capability_table=_LINUX_AMD64_MATRIX._capability_table, diff --git a/devcapsule-src/tests/test_linux_arm64_matrix.py b/devcapsule-src/tests/test_linux_arm64_matrix.py new file mode 100644 index 00000000..ba6acdc5 --- /dev/null +++ b/devcapsule-src/tests/test_linux_arm64_matrix.py @@ -0,0 +1,76 @@ +"""The linux-arm64 matrix pins the linux-amd64 versions with arm64 artifacts only.""" + +from __future__ import annotations + +import json +import re +from typing import Any + +from devcapsule.platforms import Platform +from devcapsule.resolution_matrix import MATRICES + +AMD64 = MATRICES[Platform.LINUX_AMD64] +ARM64 = MATRICES[Platform.LINUX_ARM64] +AMD64_MARKERS = re.compile(r"x86[_-]64|[_-]x64|amd64|linux64/|cft/[^/]+/linux64|ffmpeg-linux\.zip") + + +def _urls(value: Any) -> list[str]: + if isinstance(value, dict): + return [url for key, item in value.items() for url in ([item] if key == "url" else _urls(item))] + if isinstance(value, (list, tuple)): + return [url for item in value for url in _urls(item)] + return [] + + +def test_arm64_pins_the_same_component_versions_as_amd64() -> None: + assert {component: [pin.version for pin in pins] for component, pins in ARM64._components.items()} == { + component: [pin.version for pin in pins] for component, pins in AMD64._components.items() + } + + +def test_no_arm64_pin_names_an_amd64_artifact() -> None: + for component, pins in ARM64._components.items(): + for pin in pins: + for url in _urls(pin.lock_table): + assert not AMD64_MARKERS.search(url), (component, pin.version, url) + + +def test_every_arm64_artifact_is_hash_pinned_and_differs_from_amd64() -> None: + for component, pins in ARM64._components.items(): + for arm64, amd64 in zip(pins, AMD64._components[component]): + if arm64.lock_table.get("delivery-policy") == "base-image": + assert arm64 is amd64 + continue + assert re.fullmatch(r"[0-9a-f]{64}", str(arm64.lock_table.get("sha256", "0" * 64))) + if "url" in arm64.lock_table and not str(arm64.lock_table["url"]).endswith("codex-" + arm64.version + ".tgz"): + assert arm64.lock_table["sha256"] != amd64.lock_table["sha256"], (component, arm64.version) + + +def test_platform_fields_and_artifact_keys_name_arm64() -> None: + for component, pins in ARM64._components.items(): + for pin in pins: + table = pin.lock_table + if "platform" in table: + assert table["platform"] == "linux-arm64", component + if component in {"codex", "claude-code", "antigravity-cli"}: + assert list(table["artifacts"]) == ["linux-arm64"], component + assert "linux-amd64" not in json.dumps([pin.lock_table for pins in ARM64._components.values() for pin in pins]) + + +def test_eclipse_native_closure_is_arm64_or_architecture_independent() -> None: + (eclipse,) = ARM64._components["eclipse"] + packages = eclipse.lock_table["native-packages"] + assert len(packages) == len(AMD64._components["eclipse"][0].lock_table["native-packages"]) + for name, entry in packages.items(): + # Architecture-independent packages keep the amd64 closure's bytes. + if entry["url"].endswith("_all.deb"): + continue + assert entry["url"].startswith("https://ports.ubuntu.com/ubuntu-ports/pool/"), name + assert entry["url"].endswith("_arm64.deb"), name + + +def test_arm64_base_is_its_own_unpublished_recipe_11_pin() -> None: + (base,) = ARM64.base_images() + assert base.contract.recipe == 11 + assert base.reference.startswith("docker.io/mycodespaceai/devcapsule-base@sha256:") + assert base.reference not in {image.reference for image in AMD64.base_images()} From 86fbff857e403d3965614cf69bf4384bf8b9a2cd Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Wed, 4 Nov 2026 12:32:45 +0200 Subject: [PATCH 12/20] Re-pin a checkout's personal choices for the platform it runs as A checkout keeps one set of personal pins, and on Apple Silicon a project may run as linux-arm64 or linux-amd64. Adding an arm64 lock to a project whose checkout had chosen VSCodium under linux-amd64 kept the x86-64 VSCodium in the checkout's arm64 pins, because the retained choice's table was reused as it stood; the capsule then failed at start with "rosetta error: failed to open elf at /lib64/ld-linux-x86-64.so.2". Personal pins now take the platform's own embedded table for every component version that platform pins, both when a selection is pinned and when pins are laid over a lock, which also heals pins already written wrongly. A version the platform does not pin, such as a channel upgrade, is kept on its own platform and otherwise asks for a new choice. The macOS test session also removes the short temporary root it creates; each run had left one in /tmp. Co-Authored-By: Claude Opus 5.5 --- .../configuration/capability_selection.py | 36 +++++++++++++++++++ .../devcapsule/resolution_matrix.py | 8 +++++ devcapsule-src/tests/conftest.py | 18 ++++++++++ .../tests/e2e/test_ide_comes_alive.py | 17 +++++---- .../tests/test_linux_arm64_matrix.py | 34 ++++++++++++++++++ 5 files changed, 107 insertions(+), 6 deletions(-) diff --git a/devcapsule-src/devcapsule/configuration/capability_selection.py b/devcapsule-src/devcapsule/configuration/capability_selection.py index 7db8d5d1..3b3e47ae 100644 --- a/devcapsule-src/devcapsule/configuration/capability_selection.py +++ b/devcapsule-src/devcapsule/configuration/capability_selection.py @@ -111,6 +111,40 @@ def local_pins(checkout: Mapping[str, Any]) -> dict[str, Any]: raise ProjectConfigurationError(f"Invalid local capability pins: {exc}") from exc +def pins_for_platform(pins: Mapping[str, Any], matrix: ResolutionMatrix) -> dict[str, Any]: + """Return personal pins with each component's artifacts for ``matrix``'s platform. + + A checkout keeps one set of personal pins, and a project may run as two + platforms: Apple Silicon runs linux-arm64 and linux-amd64. A component + version both platforms pin takes this platform's embedded table, so pins + written for one platform never deliver its artifacts to the other. A + version this platform does not pin is kept only when the pins were + written for this platform; otherwise ProjectConfigurationError names the + remedy. The input is unchanged. + """ + if not pins: + return {} + result = deepcopy(dict(pins)) + platform = str(matrix.platform) + components = result.get("components") + if isinstance(components, dict): + for name, metadata in components.items(): + if not isinstance(metadata, dict): + continue + native = matrix.component_pin(name, metadata.get("version")) + if native is not None: + components[name] = deepcopy(dict(native)) + elif str(pins.get("platform")) != platform: + raise ProjectConfigurationError( + f"This checkout's {name} {metadata.get('version')} is pinned for {pins.get('platform')}, " + f"and {platform} has no pin for that version; choose it again for {platform}, " + f"for example with 'project --platform {platform} config ide IDE'." + ) + if "platform" in result: + result["platform"] = platform + return result + + def pin_selection(matrix: ResolutionMatrix, policy: CapabilityPolicy, selection: LocalCapabilities, source: Mapping[str, Any], previous: LocalCapabilities, previous_pins: Mapping[str, Any], *, allow_unverified: bool, recommendations: Mapping[str, str] | None = None) -> dict[str, Any]: @@ -123,6 +157,7 @@ def pin_selection(matrix: ResolutionMatrix, policy: CapabilityPolicy, selection: selector and materialization follow the IDE component's document. Catalog resolution failures raise ProjectConfigurationError. Inputs are unchanged. """ + previous_pins = pins_for_platform(previous_pins, matrix) active_names = set(known_policy_names(policy, matrix).values()) | {matrix.canonical_name(name) for name in selection.selected} reset = (set(previous.providers) - set(selection.providers)) & active_names capabilities = tuple(dict.fromkeys((*selection.selected, *selection.providers, *sorted(reset)))) @@ -191,6 +226,7 @@ def compose_lock(manifest: Mapping[str, Any], shared: Mapping[str, Any], if not pins: raise ProjectConfigurationError("Local capability pins are missing; choose it again with 'project config ide IDE'.") matrix = matrix_for(lock) + pins = pins_for_platform(pins, matrix) if policy is None: policy = CapabilityPolicy.read(manifest) canonical = {**known_policy_names(policy, matrix), **{name: matrix.canonical_name(name) for name in local.selected}} diff --git a/devcapsule-src/devcapsule/resolution_matrix.py b/devcapsule-src/devcapsule/resolution_matrix.py index 99ec6220..155dc5cf 100644 --- a/devcapsule-src/devcapsule/resolution_matrix.py +++ b/devcapsule-src/devcapsule/resolution_matrix.py @@ -395,6 +395,14 @@ def component_pins(self) -> dict[str, Mapping[str, Any]]: """The latest embedded table for each component on this platform.""" return {name: pins[-1].lock_table for name, pins in self._components.items() if pins} + @property + def platform(self) -> Platform: + return self._platform + + def component_pin(self, component: str, version: object) -> Mapping[str, Any] | None: + """This platform's embedded table for one component version, or None.""" + return next((pin.lock_table for pin in self._components.get(component, ()) if pin.version == version), None) + def component_closure(self, capability: str, minimum: str | None = None, lock: Mapping[str, Any] | None = None, *, choice: str | None = None, recommended: str | None = None) -> tuple[str, ...]: diff --git a/devcapsule-src/tests/conftest.py b/devcapsule-src/tests/conftest.py index 52e26b02..0928a195 100644 --- a/devcapsule-src/tests/conftest.py +++ b/devcapsule-src/tests/conftest.py @@ -3,6 +3,7 @@ import os from collections.abc import Iterator from pathlib import Path +import shutil import sys import tempfile @@ -19,6 +20,23 @@ def pytest_configure(config: pytest.Config) -> None: if sys.platform == "darwin" and not config.option.basetemp: config.option.basetemp = tempfile.mkdtemp(prefix="dcpt-", dir="/tmp") + config.stash[_SHORT_BASETEMP] = Path(config.option.basetemp) + + +_SHORT_BASETEMP = pytest.StashKey[Path]() + + +def pytest_unconfigure(config: pytest.Config) -> None: + """Remove the short temporary root this session created. + + pytest prunes its own default roots, but not one it was given; a root per + session would otherwise outlive every run, and a full disk fails Docker + Desktop's virtual machine with it. + """ + + created = config.stash.get(_SHORT_BASETEMP, None) + if created is not None: + shutil.rmtree(created, ignore_errors=True) @pytest.fixture(autouse=True) diff --git a/devcapsule-src/tests/e2e/test_ide_comes_alive.py b/devcapsule-src/tests/e2e/test_ide_comes_alive.py index 936a70f2..cdac4242 100644 --- a/devcapsule-src/tests/e2e/test_ide_comes_alive.py +++ b/devcapsule-src/tests/e2e/test_ide_comes_alive.py @@ -18,6 +18,7 @@ import pytest +from devcapsule.platforms import Platform from devcapsule.runtime_artifact import RUNTIME_PEX_ENV, executable_platform from tests.e2e.ide_session import SURFACES, IdeSurface, capture_desktop, command, desktop_page_answers, ide_session, launched_ide, stop_session, wait_for_console_url, wait_for_ide_window @@ -80,7 +81,7 @@ def test_ide_comes_alive(surface: IdeSurface, built_pex: Path, tmp_path: Path, e identity = command("docker", "inspect", "--format", '{{json .Id}} {{json .Image}}', session.container) (evidence / "container-identity.txt").write_text(identity.stdout, encoding="utf-8") child_digest = command("docker", "exec", session.container, "sha256sum", "/opt/devcapsule/bin/devcapsule.pex").stdout.split()[0] - assert child_digest == hashlib.sha256(delivered_runtime(built_pex).read_bytes()).hexdigest(), \ + assert child_digest == hashlib.sha256(delivered_runtime(built_pex, session.workspace).read_bytes()).hexdigest(), \ "Child runtime differs from the runtime the selected executable delivers" if surface.name == "eclipse": package = command("docker", "exec", "--user", f"{os.getuid()}:{os.getgid()}", @@ -156,15 +157,19 @@ def test_ide_comes_alive(surface: IdeSurface, built_pex: Path, tmp_path: Path, e }, indent=2) + "\n", encoding="utf-8") -def delivered_runtime(executable: Path) -> Path: +def delivered_runtime(executable: Path, workspace: Path) -> Path: """The executable a launch from ``executable`` delivers into the capsule. Itself when it runs on the capsule platform; a macOS launcher delivers the - Linux runtime its environment names instead. + Linux runtime its environment names instead: the file, or, from a + directory, the one for the platform the project's locks select. """ - if executable_platform(executable) == "darwin": - return Path(os.environ[RUNTIME_PEX_ENV]).expanduser().resolve() - return executable + if executable_platform(executable) != "darwin": + return executable + named = Path(os.environ[RUNTIME_PEX_ENV]).expanduser().resolve() + if named.is_dir(): + return named / f"devcapsule-{Platform.select_for_project(workspace)}" + return named def editor_font_size(container: str) -> float: diff --git a/devcapsule-src/tests/test_linux_arm64_matrix.py b/devcapsule-src/tests/test_linux_arm64_matrix.py index ba6acdc5..d57c4a5d 100644 --- a/devcapsule-src/tests/test_linux_arm64_matrix.py +++ b/devcapsule-src/tests/test_linux_arm64_matrix.py @@ -74,3 +74,37 @@ def test_arm64_base_is_its_own_unpublished_recipe_11_pin() -> None: assert base.contract.recipe == 11 assert base.reference.startswith("docker.io/mycodespaceai/devcapsule-base@sha256:") assert base.reference not in {image.reference for image in AMD64.base_images()} + + +def _personal_pins(platform: str, component: str, matrix: Any) -> dict[str, Any]: + pin = matrix.component_pins()[component] + return {"platform": platform, "components": {component: dict(pin), "interactive-surface": component}} + + +def test_personal_pins_take_the_artifacts_of_the_platform_they_run_on() -> None: + # A checkout chose codium while the project ran as linux-amd64; the + # project then gained an arm64 lock. Its pins must deliver arm64 bytes. + from devcapsule.configuration.capability_selection import pins_for_platform + + amd64 = _personal_pins("linux-amd64", "codium", AMD64) + on_arm64 = pins_for_platform(amd64, ARM64) + assert on_arm64["platform"] == "linux-arm64" + assert on_arm64["components"]["codium"] == dict(ARM64.component_pins()["codium"]) + assert on_arm64["components"]["interactive-surface"] == "codium" + assert amd64["components"]["codium"] == dict(AMD64.component_pins()["codium"]) # input unchanged + # Pins already written wrongly, arm64 platform with amd64 bytes, are healed too. + mislabelled = dict(amd64, platform="linux-arm64") + assert pins_for_platform(mislabelled, ARM64)["components"]["codium"] == dict(ARM64.component_pins()["codium"]) + assert pins_for_platform(on_arm64, AMD64)["components"]["codium"] == dict(AMD64.component_pins()["codium"]) + + +def test_a_personal_version_the_other_platform_lacks_asks_for_a_new_choice() -> None: + import pytest + + from devcapsule.configuration.capability_selection import pins_for_platform + from devcapsule.configuration.storage import ProjectConfigurationError + + channel = {"platform": "linux-amd64", "components": {"codex": {"version": "9.9.9", "url": "https://example.test/x64.tgz"}}} + with pytest.raises(ProjectConfigurationError, match="codex 9.9.9 is pinned for linux-amd64.*linux-arm64 has no pin"): + pins_for_platform(channel, ARM64) + assert pins_for_platform(channel, AMD64)["components"]["codex"]["version"] == "9.9.9" From 30cb5c7df5950a1d88afc2161291cd8086d5deb9 Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Wed, 4 Nov 2026 12:34:16 +0200 Subject: [PATCH 13/20] Add libunwind8 to the arm64 Eclipse package closure arm64's libgstreamer1.0-0 depends on libunwind8, which the amd64 closure does not carry; the Eclipse smoke's offline dpkg install stopped at it. Pinned from noble-updates, the index checked against the signed InRelease and the package bytes against the index. Co-Authored-By: Claude Opus 5.5 --- .../devcapsule/components/linux_arm64_pins.py | 13 ++++++++++++- devcapsule-src/tests/test_linux_arm64_matrix.py | 3 ++- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/devcapsule-src/devcapsule/components/linux_arm64_pins.py b/devcapsule-src/devcapsule/components/linux_arm64_pins.py index 4a1a19a5..0744ae39 100644 --- a/devcapsule-src/devcapsule/components/linux_arm64_pins.py +++ b/devcapsule-src/devcapsule/components/linux_arm64_pins.py @@ -10,7 +10,8 @@ ports.ubuntu.com, except libsoup: Ubuntu removed 3.4.4-5ubuntu0.8 from every noble pocket on 2026-10-08, so both libsoup packages are 3.4.4-5ubuntu0.10 from noble-security. Architecture-independent packages - are the amd64 closure's own bytes. + are the amd64 closure's own bytes. libunwind8 is added: arm64's + libgstreamer1.0-0 depends on it. - Playwright 1.63.0's Chromium comes from Chrome for Testing's linux-arm64 build of 153.0.8010.12 on Playwright's CDN, byte-identical to Google's. - Antigravity's arm64 builds live under ``linux-arm/``; VS Code's arm64 @@ -220,6 +221,16 @@ 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/x/xdg-dbus-proxy/xdg-dbus-proxy_0.1.5-1ubuntu0.2_arm64.deb', 'version': '0.1.5-1ubuntu0.2'}} +# arm64 only: libgstreamer1.0-0 depends on libunwind8 on arm64, which the +# amd64 closure does not carry. Found by the Eclipse smoke on 2026-11-04 +# ("Package libunwind8 is not installed"); noble-updates, its index verified +# against the gpgv-checked InRelease and the bytes against the index. +ECLIPSE_NATIVE_PACKAGES["libunwind8"] = { + "sha256": "26f2bf9c0d9b42d8230aa2c070076054b0ed06519bb9141c05a9c3164732fef1", + "url": "https://ports.ubuntu.com/ubuntu-ports/pool/main/libu/libunwind/libunwind8_1.6.2-3build1.1_arm64.deb", + "version": "1.6.2-3build1.1", +} + PLAYWRIGHT_PIN: dict[str, Any] = {'version': '1.63.0', 'delivery-policy': 'local-materialization', 'license': 'Apache-2.0', diff --git a/devcapsule-src/tests/test_linux_arm64_matrix.py b/devcapsule-src/tests/test_linux_arm64_matrix.py index d57c4a5d..66eb322a 100644 --- a/devcapsule-src/tests/test_linux_arm64_matrix.py +++ b/devcapsule-src/tests/test_linux_arm64_matrix.py @@ -60,7 +60,8 @@ def test_platform_fields_and_artifact_keys_name_arm64() -> None: def test_eclipse_native_closure_is_arm64_or_architecture_independent() -> None: (eclipse,) = ARM64._components["eclipse"] packages = eclipse.lock_table["native-packages"] - assert len(packages) == len(AMD64._components["eclipse"][0].lock_table["native-packages"]) + # The same packages, plus what arm64's own dependencies add. + assert set(packages) - set(AMD64._components["eclipse"][0].lock_table["native-packages"]) == {"libunwind8"} for name, entry in packages.items(): # Architecture-independent packages keep the amd64 closure's bytes. if entry["url"].endswith("_all.deb"): From f14f32b5a89da1ee3aafd058c34522e5b243d93c Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Wed, 4 Nov 2026 12:36:13 +0200 Subject: [PATCH 14/20] Record linux-arm64 slices 1 to 3 and document --platform Co-Authored-By: Claude Opus 5.5 --- docs/reference/cli.md | 8 +++-- .../2026-11-04-linux-arm64/CURRENT-STATUS.md | 33 +++++++++++++++++-- 2 files changed, 37 insertions(+), 4 deletions(-) diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 078cbca1..09d04825 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -1,7 +1,7 @@ --- description: The command tree of the devcapsule launcher, with one line of intent per command and usage for init, run and config. weight: 1 -updated: 2026-10-30 +updated: 2026-11-04 --- # Command reference @@ -11,7 +11,11 @@ checks even when given `--help`. `devcapsule project` commands discover the project from the current directory. To target another directory, pass `--path` before the subcommand: -`devcapsule project --path PROJECT config show`. Inside a capsule, inspection +`devcapsule project --path PROJECT config show`. On a host that runs two capsule +platforms, an Apple Silicon Mac, a project runs as the first one it has a lock +for, `linux-arm64` before `linux-amd64`; `--platform PLATFORM` before the +subcommand names one for that command, so `devcapsule project --platform +linux-arm64 init` adds an arm64 lock to a project. Inside a capsule, inspection commands distinguish captured launch data from next-launch choices where available. Commands that need launcher-owned state print the exact `devcapsule` command to run outside. diff --git a/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md index f41663d2..c6143bf0 100644 --- a/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md @@ -4,7 +4,7 @@ Mnemonic: `linux-arm64` Start date: 2026-11-04 -State: active; registered 2026-11-04; slice 1, the second capsule platform, next +State: active; slices 1 to 3 committed 2026-11-04; native smokes in progress Branch association: `ws-linux-arm64/capsules`, forked from local `main` at b3f10b7d, with `ws-macos-launcher/host-port` merged in as its foundation @@ -43,9 +43,33 @@ once; build the base locally now; arm64 when locked, else amd64; a new workstrea the macOS branch. Two read-only surveys mapped the work; the plan is in the [design note](2026-11-04-design-linux-arm64.md). +**2026-11-04, slices 1 to 3.** `Platform.LINUX_ARM64`; a project runs as the first platform +the host runs that it has a lock for, and `project --platform` names one (f1bba6c9, +7a32f1c6). `DEVCAPSULE_RUNTIME_PEX` may name a directory of runtimes. The arm64 base was +built natively here from a launcher at public `main` b3f10b7d and pinned at its local index +digest, unpublished (7a32f1c6). Every component is pinned for arm64 at the amd64 versions, +102 artifacts, 15.6 GB hashed from their bytes (87e71b19), and the Eclipse closure gained +`libunwind8`, which arm64's GStreamer needs (b12e2516). The human's own switch of +`~/hello-devcapsule` to arm64 found that a checkout's personal pins kept the amd64 VSCodium; +personal pins are now re-pinned per platform (52fb1a1a). + +Native arm64 smokes on this Mac with the local base: VSCodium passed in 58 s with its main +window; VS Code passed. Eclipse failed at `libunwind8`, since fixed; PyCharm, Rider and +IntelliJ failed on Docker I/O errors when the Mac's disk filled. The disk filled because +these runs built five formations at once on top of a 20 GB artifact cache, and Docker +Desktop's VM failed with it; a restart recovered it. With the human's consent the test +images and build cache were removed, and the reruns go one surface at a time, checking free +space first and removing each image after. The macOS test session now removes its temporary +root; each run had left one in `/tmp`. + +Found for `maintenance`, not fixed here: the amd64 Eclipse closure's `libsoup-3.0-0` and +`libsoup-3.0-common` 3.4.4-5ubuntu0.8 URLs return 404, since Ubuntu removed that version from +every noble pocket on 2026-10-08, so Eclipse cannot be acquired afresh on amd64. + ## Next Resumable Task -Slice 1 of the design note: the second capsule platform. +Record the rerun smokes; add arm64 verified edges for the surfaces that pass, with their +evidence, so resolution stops needing `--unverified`; then the documentation slice. ## Open Threads @@ -55,6 +79,11 @@ Slice 1 of the design note: the second capsule platform. registration commit is forked from `main` as topic 7.5 requires, and the macOS branch is merged after it; this pull request follows that one. - An arm64 base on `docker.io/mycodespaceai`, and its digest in the matrix, need the owner. + The pin holds the digest a push of this Mac's exact image keeps; a rebuild must re-pin. +- The libsoup 404 on amd64 goes to `maintenance` by intake mail once the coordination branch + can be pushed. +- Docker access from an arm64 capsule reaches an arm64 daemon either way; on amd64 capsules + under emulation it also defaults to arm64 images (`docs/platforms/macos.md`). ## Document Index From e2e2fbccfafe16e97146c5f082dcd2298ba28582 Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Wed, 4 Nov 2026 13:52:52 +0200 Subject: [PATCH 15/20] Fix the review of the macOS and linux-arm64 branch The human's /review of every change since main rejected the branch with seven high, fourteen medium and three low findings. Each is fixed: - A Mac creates linux-amd64 projects again, as the human's ruling says (arm64 when the project is locked for it). Platform.current() is host detection: the platform a host creates projects for; runnable() lists what it runs. Which platform a project runs as is the configuration's question, storage.project_platform, read-only and answered from the project's locks; project --platform and images build --platform name one. The process-wide ContextVar selection is gone, so images build follows the project's locks too. - Below the configuration the lock's platform is passed down: the base pull, the buildx build, the delivered runtime and the docker run arguments. Emulation is judged against the Docker daemon's architecture, asked once per process. Known-good version sets are the lock platform's only. - A delivered runtime that is not the launcher's own bytes must be its build: the same version, and the same revision where both record one (D-0009, R-RUNTIME-001, whose record is updated). A plain PEX on a non-Linux host is not delivered. - The arm64 base pin is marked unpublished: init says so, and a launch without a local base stops with the remedy instead of a failed pull. - Personal pins are remapped only when written for another platform. - The macOS sudo-policy volume is labelled, and each launch prunes unused ones, so an interrupted launch no longer leaks it. - Docker Desktop's conventions are HostOs properties: the socket group, an unreachable host network, unbindable Unix sockets, bind mounts that refuse chown. The host-open code that only tests on a Mac reached is removed and those tests are Linux-only. - The successor plan compares --platform in either spelling. - Tests: the arm64 hash test walks every artifact; the suite's default host is amd64 Linux down to the daemon; real_host_os is renamed own_host_patches; the recursive orchestrator skips off Linux only; a failed macOS session keeps its temporary root. - The macOS page describes linux-amd64 by default and linux-arm64 per project with a local base; the design note records the review. Co-Authored-By: Claude Opus 5.5 --- devcapsule-src/devcapsule/commands/images.py | 17 ++ devcapsule-src/devcapsule/commands/project.py | 42 ++--- .../configuration/capability_commands.py | 8 +- .../configuration/capability_selection.py | 45 ++--- .../devcapsule/configuration/derivation.py | 6 +- .../devcapsule/configuration/operations.py | 32 ++-- .../configuration/owner_commands.py | 4 +- .../devcapsule/configuration/storage.py | 61 ++++++- .../devcapsule/environment_realization.py | 32 +++- devcapsule-src/devcapsule/host_daemon.py | 23 +++ devcapsule-src/devcapsule/host_open.py | 32 +--- devcapsule-src/devcapsule/images/base.py | 5 +- devcapsule-src/devcapsule/images/build.py | 13 +- .../devcapsule/launch/pycharm/_launcher.py | 40 +++-- devcapsule-src/devcapsule/platforms.py | 163 +++++++++++------- .../devcapsule/recursive_successor_plan.py | 20 ++- .../devcapsule/resolution_matrix.py | 8 + devcapsule-src/devcapsule/runtime_artifact.py | 128 +++++++++----- devcapsule-src/devcapsule/version_sets.py | 10 +- devcapsule-src/pyproject.toml | 2 +- devcapsule-src/tests/conftest.py | 44 +++-- .../tests/e2e/test_ide_comes_alive.py | 15 +- devcapsule-src/tests/test_base_image.py | 2 +- devcapsule-src/tests/test_cli.py | 2 +- .../tests/test_environment_realization.py | 17 +- devcapsule-src/tests/test_host_open.py | 12 ++ .../tests/test_linux_arm64_matrix.py | 42 +++-- devcapsule-src/tests/test_platforms.py | 91 ++++------ .../tests/test_project_platform_selection.py | 65 ++++--- devcapsule-src/tests/test_pycharm.py | 20 ++- .../tests/test_recursive_orchestrator.py | 16 +- .../tests/test_recursive_successor_plan.py | 14 ++ devcapsule-src/tests/test_runtime_artifact.py | 149 ++++++++++------ devcapsule-src/tests/test_version_sets.py | 6 +- docs/platforms/macos.md | 103 +++++++---- docs/reference/cli.md | 7 +- ...r-runtime-001-launcher-runtime-identity.md | 25 ++- .../2026-11-04-design-linux-arm64.md | 41 +++-- .../2026-11-04-linux-arm64/CURRENT-STATUS.md | 14 +- 39 files changed, 886 insertions(+), 490 deletions(-) diff --git a/devcapsule-src/devcapsule/commands/images.py b/devcapsule-src/devcapsule/commands/images.py index 43e3f04d..f7669b12 100644 --- a/devcapsule-src/devcapsule/commands/images.py +++ b/devcapsule-src/devcapsule/commands/images.py @@ -110,9 +110,25 @@ def configure(cls, parser: argparse.ArgumentParser) -> None: default="default", help="Build network mode forwarded to Docker buildx (default: default).", ) + parser.add_argument( + "--platform", + dest="capsule_platform", + metavar="PLATFORM", + help="Capsule platform to build for, such as linux-arm64. An environment build defaults to " + "the first one this host runs that the project has a lock for; a base build to the one " + "this host creates projects for.", + ) @classmethod def run(cls, arguments: argparse.Namespace, context: object | None) -> int: + from devcapsule.configuration.storage import name_platform + from devcapsule.platforms import Platform, UnsupportedPlatformError + + try: + capsule_platform = Platform.parse(arguments.capsule_platform) if arguments.capsule_platform else None + except UnsupportedPlatformError as exc: + raise CliError(str(exc)) from exc + name_platform(capsule_platform) if arguments.image_type == "environment": return _build_environment( project=arguments.project, @@ -153,6 +169,7 @@ def run(cls, arguments: argparse.Namespace, context: object | None) -> int: build_base_image( options, network=arguments.network, + platform=capsule_platform, ) image = inspect_local_image(arguments.tag) labels = dict(image.config.labels or {}) diff --git a/devcapsule-src/devcapsule/commands/project.py b/devcapsule-src/devcapsule/commands/project.py index 1a728c08..1f9ece3d 100644 --- a/devcapsule-src/devcapsule/commands/project.py +++ b/devcapsule-src/devcapsule/commands/project.py @@ -53,6 +53,7 @@ from devcapsule import runtime_configuration from devcapsule.runtime_configuration import CapsuleAccess from devcapsule.compat import CliError, DecisionPending +from devcapsule.platforms import Platform import difflib from devcapsule.configuration.capabilities import CapabilityPolicy, LocalCapabilities from devcapsule.configuration.console_requests import deferred_rows @@ -755,9 +756,9 @@ def _load_checkout_documents(context: object | None) -> CheckoutDocuments | runt def _assess(documents: CheckoutDocuments) -> Listing: from devcapsule.configuration.authorization import authorized_base_selection from devcapsule.configuration.trust import HomeTrust - from devcapsule.platforms import Platform + from devcapsule.configuration.storage import project_platform from devcapsule.resolution_matrix import MATRICES, known_base_image - matrix = MATRICES[Platform.current()] + matrix = MATRICES[project_platform(documents.root)] answers = documents.checkout.get("authorization", {}) base_answer = answers.get("base-image", {}) if isinstance(answers, dict) else {} base_reference = base_answer.get("reference") if isinstance(base_answer, dict) else None @@ -1702,12 +1703,12 @@ def run(cls, arguments: argparse.Namespace, context: object | None) -> int: def _local_selection(context: object | None) -> tuple[Path, dict[str, Any], LocalCapabilities, dict[str, str]]: """The checkout's current selection and the catalog's IDE names, for the selection verbs.""" - from devcapsule.platforms import Platform + from devcapsule.configuration.storage import project_platform from devcapsule.resolution_matrix import MATRICES root, manifest = manifest_for(_project_context(context).start_path()) record = find_checkout_record(manifest, root) current = LocalCapabilities.read(load_checkout(record, manifest, root)) if record is not None else LocalCapabilities() - matrix = MATRICES[Platform.current()] + matrix = MATRICES[project_platform(root)] surfaces = {name: matrix.resolve_provider(name) or name for name in matrix.ide_capabilities()} return root, manifest, current, surfaces @@ -2436,6 +2437,7 @@ def prepare_display(base: ImageDetails) -> None: enable_host_browser=selected_host_browser, display_transport=display_transport, web_console=web_console, + capsule_platform=Platform.parse(str(lock["platform"])), ) ) if command_report is not None: @@ -2558,6 +2560,7 @@ def make_context(cls, arguments: argparse.Namespace, parent: object | None) -> o launcher. """ context = ProjectCommandContext(arguments.selected_path) + _name_capsule_platform(getattr(arguments, "capsule_platform", None)) tokens: list[str] = list(arguments.rest) parsed_tokens = tokens[:tokens.index("--")] if "--" in tokens else tokens if any(token in {"-h", "--help"} for token in parsed_tokens): @@ -2584,7 +2587,6 @@ def make_context(cls, arguments: argparse.Namespace, parent: object | None) -> o context = replace(context, capsule_root=runtime_configuration.capsule_project_root(Path("."))) if access.needs_launcher: runtime_configuration.require_launcher(context.start_path(), tokens) - _select_capsule_platform(context, getattr(arguments, "capsule_platform", None)) return context @classmethod @@ -2604,31 +2606,21 @@ def subcommands(cls) -> Mapping[str, type[Command] | type[Group]]: } -def _select_capsule_platform(context: ProjectCommandContext, named: str | None = None) -> None: - """Run the selected project as the platform it is locked for on this host. +def _name_capsule_platform(named: str | None) -> None: + """Record the platform ``--platform`` named for this command, or clear it. - See ``Platform.select_for_project``: where a host runs two platforms, the - project's locks choose. Without an enclosing project, as for ``init`` in - a fresh directory, the host's preferred platform stands. ``--platform`` - names one instead: ``project --platform linux-arm64 init`` adds that - platform's lock to a project, as init completes what a project lacks. + Without one, each step asks ``project_platform`` for the project's + platform, which its locks decide on a host that runs two. Naming one is + how ``project --platform linux-arm64 init`` adds that platform's lock to a + project, since init completes what a project lacks. """ + from devcapsule.configuration.storage import name_platform from devcapsule.platforms import Platform, UnsupportedPlatformError - if named is not None: - try: - Platform.select(Platform.parse(named)) - except UnsupportedPlatformError as exc: - raise ProjectConfigurationError(str(exc)) from exc - return try: - root: Path | None = discover_project(context.start_path()) - except ProjectConfigurationError: - root = None - try: - Platform.select_for_project(root) - except UnsupportedPlatformError: - pass # Each platform-dependent step reports it in its own terms. + name_platform(Platform.parse(named) if named is not None else None) + except UnsupportedPlatformError as exc: + raise ProjectConfigurationError(str(exc)) from exc def _capsule_access(leaf: type[Command]) -> CapsuleAccess: diff --git a/devcapsule-src/devcapsule/configuration/capability_commands.py b/devcapsule-src/devcapsule/configuration/capability_commands.py index bb2edcd4..4b773f42 100644 --- a/devcapsule-src/devcapsule/configuration/capability_commands.py +++ b/devcapsule-src/devcapsule/configuration/capability_commands.py @@ -22,7 +22,9 @@ from .manifest_document import capability_reasons, capability_table, manifest_preamble, render_manifest from .manifest_schemas import ManifestV2 from .nodes import build_node_registry -from .storage import atomic_write, checkout_record_paths, discover_project, load_toml, require_settled +from .storage import ( + atomic_write, checkout_record_paths, discover_project, load_toml, platform_lock_path, project_platform, require_settled, +) def paths(root: Path) -> tuple[Path, Path, Path]: @@ -31,7 +33,7 @@ def paths(root: Path) -> tuple[Path, Path, Path]: ``root`` is the project directory. This only constructs paths. """ directory = root / ".devcapsule" - return (directory / "devcapsule.toml", directory / f"devcapsule.{Platform.current()}.lock", + return (directory / "devcapsule.toml", platform_lock_path(root, project_platform(root)), directory / ".capability-transaction.toml") @@ -192,7 +194,7 @@ def configure(start: Path, *, required: Sequence[str] | None = None, local_edit = local is not None or without is not None or providers is not None if project_edit == local_edit: raise ProjectConfigurationError("Choose either project flags (--required/--optional) or local flags (--local/--without).") - matrix = MATRICES[Platform.current()] + matrix = MATRICES[project_platform(root)] if project_edit: return configure_project(root, manifest, lock, matrix, required=required, optional=optional, preview=preview, allow_unverified=allow_unverified) diff --git a/devcapsule-src/devcapsule/configuration/capability_selection.py b/devcapsule-src/devcapsule/configuration/capability_selection.py index 3b3e47ae..8d773071 100644 --- a/devcapsule-src/devcapsule/configuration/capability_selection.py +++ b/devcapsule-src/devcapsule/configuration/capability_selection.py @@ -112,37 +112,38 @@ def local_pins(checkout: Mapping[str, Any]) -> dict[str, Any]: def pins_for_platform(pins: Mapping[str, Any], matrix: ResolutionMatrix) -> dict[str, Any]: - """Return personal pins with each component's artifacts for ``matrix``'s platform. + """Return personal pins written for another platform, remapped to ``matrix``'s platform. A checkout keeps one set of personal pins, and a project may run as two - platforms: Apple Silicon runs linux-arm64 and linux-amd64. A component - version both platforms pin takes this platform's embedded table, so pins - written for one platform never deliver its artifacts to the other. A - version this platform does not pin is kept only when the pins were - written for this platform; otherwise ProjectConfigurationError names the - remedy. The input is unchanged. + platforms: Apple Silicon runs linux-arm64 and linux-amd64. Pins written + for this platform are returned as they stand, records not to be + re-derived. Pins written for the other take this platform's embedded + table for each component version, so one platform's artifacts never reach + the other's capsules; a version this platform does not pin raises + ProjectConfigurationError naming how to choose again. The input is unchanged. """ - if not pins: - return {} - result = deepcopy(dict(pins)) platform = str(matrix.platform) - components = result.get("components") + if not pins or str(pins.get("platform")) == platform: + return deepcopy(dict(pins)) + repinned = deepcopy(dict(pins)) + components = repinned.get("components") + surface = components.get("interactive-surface") if isinstance(components, dict) else None if isinstance(components, dict): - for name, metadata in components.items(): + for component, metadata in components.items(): if not isinstance(metadata, dict): continue - native = matrix.component_pin(name, metadata.get("version")) - if native is not None: - components[name] = deepcopy(dict(native)) - elif str(pins.get("platform")) != platform: + matrix_table = matrix.component_pin(component, metadata.get("version")) + if matrix_table is None: + remedy = (f"'devcapsule project --platform {platform} config ide {component}'" if component == surface + else f"'devcapsule project --platform {platform} config use' for the capability it provides") raise ProjectConfigurationError( - f"This checkout's {name} {metadata.get('version')} is pinned for {pins.get('platform')}, " - f"and {platform} has no pin for that version; choose it again for {platform}, " - f"for example with 'project --platform {platform} config ide IDE'." + f"This checkout's {component} {metadata.get('version')} is pinned for {pins.get('platform')}, " + f"and {platform} has no pin for that version; choose it again with {remedy}." ) - if "platform" in result: - result["platform"] = platform - return result + components[component] = deepcopy(dict(matrix_table)) + if "platform" in repinned: + repinned["platform"] = platform + return repinned def pin_selection(matrix: ResolutionMatrix, policy: CapabilityPolicy, selection: LocalCapabilities, diff --git a/devcapsule-src/devcapsule/configuration/derivation.py b/devcapsule-src/devcapsule/configuration/derivation.py index 4b49112d..26f3e09a 100644 --- a/devcapsule-src/devcapsule/configuration/derivation.py +++ b/devcapsule-src/devcapsule/configuration/derivation.py @@ -41,6 +41,7 @@ from .manifest_document import Recommendations from .trust import HomeTrust from .storage import ( + project_platform, ResolvedProject, atomic_write, checkout_record_paths, @@ -108,7 +109,8 @@ class CheckoutState: def __init__(self, root: Path, manifest: dict[str, Any]) -> None: self.root = root self.manifest = manifest - self.matrix = MATRICES[Platform.current()] + self.platform = project_platform(root) + self.matrix = MATRICES[self.platform] self.surfaces = {name: self.matrix.resolve_provider(name) or name for name in self.matrix.ide_capabilities()} self.lock_path, self.shared_lock = recommendation_lock_for(root, manifest) self.home = load_settings() @@ -158,7 +160,7 @@ def composed_lock(self, checkout: Mapping[str, Any] | None = None) -> dict[str, """The shared lock, or the checkout's version set, with the checkout's pins laid over it.""" document = self.checkout if checkout is None else checkout selected = selected_version_lock(document) - if selected is not None and selected.get("platform") != str(Platform.current()): + if selected is not None and selected.get("platform") != str(self.platform): raise ProjectConfigurationError("Local version set is for another platform; follow the project explicitly.") return compose_lock(self.manifest, selected if selected is not None else self.shared_lock, document, policy=self.policy) diff --git a/devcapsule-src/devcapsule/configuration/operations.py b/devcapsule-src/devcapsule/configuration/operations.py index 0753e7d4..3dfacb6b 100644 --- a/devcapsule-src/devcapsule/configuration/operations.py +++ b/devcapsule-src/devcapsule/configuration/operations.py @@ -66,6 +66,8 @@ build_node_registry, ) from .storage import ( + platform_lock_path, + project_platform, atomic_write, checkout_record_paths, load_checkout, @@ -80,13 +82,10 @@ from .values import checkout_omitted_values, normalize_configuration_value -def _current_matrix() -> ResolutionMatrix: - """This host's resolution matrix, with platform failures as user errors.""" +def _project_matrix(root: Path) -> ResolutionMatrix: + """The resolution matrix of the platform this host runs the project at ``root`` as.""" - try: - return MATRICES[Platform.current()] - except UnsupportedPlatformError as exc: - raise ProjectConfigurationError(str(exc)) from exc + return MATRICES[project_platform(root)] __all__ = [ "CheckoutRecord", @@ -313,6 +312,8 @@ class InitializeReport: checkout_record: Path # None when the owner's checkout has no IDE yet: nothing to resolve. resolve: ResolveReport | None + # The lock pins a base its registry does not hold yet (see ResolutionMatrix.base_unpublished). + base_unpublished: bool = False def render(self) -> str: required = " ".join(self.required) or "none" @@ -326,6 +327,9 @@ def render(self) -> str: if self.offered: lines.append(f"Offers: {' '.join(self.offered)}") lines.append(f"{self.lock_action} {self.lock_path} (base {self.base_display})") + if self.base_unpublished: + lines.append("Its base is not published yet: a checkout runs this lock only on a locally built base of " + "the same recipe, selected with 'devcapsule project config authorize base-image '.") for name, value, justification in self.recommendations: line = f"Recommended {name} = {value} for every checkout" lines.append(f"{line} ({justification})." if justification else f"{line}.") @@ -402,12 +406,9 @@ def initialize_project( raise ProjectConfigurationError(f"Project directory does not exist: {root}") require_settled(root) manifest_path = root / ".devcapsule" / "devcapsule.toml" - try: - platform = Platform.current() - except UnsupportedPlatformError as exc: - raise ProjectConfigurationError(str(exc)) from exc - lock_path = root / ".devcapsule" / f"devcapsule.{platform}.lock" - matrix = _current_matrix() + platform = project_platform(root) + lock_path = platform_lock_path(root, platform) + matrix = MATRICES[platform] existing_manifest: dict[str, Any] | None = None existing_lock: dict[str, Any] | None = None @@ -546,6 +547,7 @@ def persist() -> None: base_display=_base_display(shared_lock), recommendations=recommendations, selection=selection, answered=(), carried=(), recommended=(), checkout_record=record.input_path, resolve=None, + base_unpublished=matrix.base_unpublished(str(shared_lock.get("base", {}).get("reference"))), ) lock = effective_lock(manifest, compose_lock(manifest, shared_lock, record.document), record.document, warn=False) declarations = authorization_declarations(manifest, lock) @@ -573,6 +575,7 @@ def persist() -> None: answered=tuple(settled.answered), carried=tuple(settled.carried), recommended=accepted_recommendations, checkout_record=record.input_path, resolve=resolve_report, + base_unpublished=matrix.base_unpublished(str(shared_lock.get("base", {}).get("reference"))), ) @@ -580,7 +583,7 @@ def _initialize_report( manifest_path: Path, manifest_action: str, identity: _ProjectIdentity, lock_path: Path, lock_action: str, *, base_display: str, recommendations: list[tuple[str, str]], selection: _OwnerSelection, answered: tuple[str, ...], carried: tuple[str, ...], recommended: tuple[str, ...], - checkout_record: Path, resolve: ResolveReport | None, + checkout_record: Path, resolve: ResolveReport | None, base_unpublished: bool = False, ) -> InitializeReport: return InitializeReport( manifest_path=manifest_path, @@ -605,6 +608,7 @@ def _initialize_report( recommended=recommended, checkout_record=checkout_record, resolve=resolve, + base_unpublished=base_unpublished, ) @@ -653,7 +657,7 @@ def _existing_selection(root: Path, manifest: Mapping[str, Any] | None) -> _Owne if not input_path.is_file(): return _OwnerSelection(None, ()) local = LocalCapabilities.read(load_toml(input_path)) - matrix = _current_matrix() + matrix = _project_matrix(root) surfaces = set(matrix.ide_capabilities()) ide = next((name for name in local.selected if name in surfaces), None) return _OwnerSelection(ide, tuple(name for name in local.selected if name not in surfaces)) diff --git a/devcapsule-src/devcapsule/configuration/owner_commands.py b/devcapsule-src/devcapsule/configuration/owner_commands.py index 91f0ce9b..d2fad3cc 100644 --- a/devcapsule-src/devcapsule/configuration/owner_commands.py +++ b/devcapsule-src/devcapsule/configuration/owner_commands.py @@ -37,7 +37,7 @@ environment_name, ) from .sources import WrappedSource, WrappedSources, has_control_characters, java_line, source_path -from .storage import discover_project, load_toml, require_settled +from .storage import discover_project, load_toml, project_platform, require_settled from .values import normalize_memory_size __all__ = [ @@ -59,7 +59,7 @@ def _project(start: Path, *, check_resolutions: bool = True) -> tuple[Path, dict manifest_path, lock_path, _ = paths(root) manifest, lock = current_manifest(load_toml(manifest_path), manifest_path).document(), load_toml(lock_path) check_documents(manifest, lock, check_resolutions=check_resolutions) - return root, manifest, lock, MATRICES[Platform.current()] + return root, manifest, lock, MATRICES[project_platform(root)] def _policy_edit( diff --git a/devcapsule-src/devcapsule/configuration/storage.py b/devcapsule-src/devcapsule/configuration/storage.py index 3fbae07f..dc27781a 100644 --- a/devcapsule-src/devcapsule/configuration/storage.py +++ b/devcapsule-src/devcapsule/configuration/storage.py @@ -382,6 +382,50 @@ def manifest_source_digest(root: Path) -> str | None: return None +# The platform a command was told to use (`project --platform`), if any. A +# command runs for one project, so this is the process's; see project_platform. +_named_platform: Platform | None = None + + +def name_platform(platform: Platform | None) -> None: + """Make ``platform`` the one project commands in this process use, or clear it. + + It must be one this host runs; the error names the ones it does. + """ + global _named_platform + if platform is not None and platform not in Platform.runnable(): + names = ", ".join(str(member) for member in Platform.runnable()) + raise ProjectConfigurationError(f"This host runs {names} capsules, not {platform}.") + _named_platform = platform + + +def platform_lock_path(root: Path, platform: Platform) -> Path: + """The path of the project's lock for ``platform``.""" + return root / ".devcapsule" / f"devcapsule.{platform}.lock" + + +def project_platform(root: Path | None) -> Platform: + """The capsule platform this host runs the project at ``root`` as. + + The platform named for this command, if any; otherwise the first platform + this host runs that the project has a lock for, so Apple Silicon runs a + project locked for linux-arm64 natively and one locked only for + linux-amd64 under emulation; otherwise, as for a project not yet + initialized, the platform this host creates projects for. Reads only. + """ + if _named_platform is not None: + return _named_platform + try: + runnable = Platform.runnable() + if root is not None: + for platform in runnable: + if platform_lock_path(root, platform).is_file(): + return platform + return Platform.current() + except UnsupportedPlatformError as exc: + raise ProjectConfigurationError(str(exc)) from exc + + def recommendation_lock_for(root: Path, manifest: Mapping[str, Any]) -> tuple[Path, dict[str, Any]]: """Load the committed platform lock for this host's platform. @@ -404,15 +448,13 @@ def recommendation_lock_for(root: Path, manifest: Mapping[str, Any]) -> tuple[Pa to keep existing committed locks working. """ - try: - current = Platform.current() - except UnsupportedPlatformError as exc: - raise ProjectConfigurationError(str(exc)) from exc - path = root / ".devcapsule" / f"devcapsule.{current}.lock" + platform = project_platform(root) + path = platform_lock_path(root, platform) if not path.is_file(): raise ProjectConfigurationError( - f"Missing {path}: this project carries no platform lock for {current}. " - "The platform lock is authored on the project side and committed with the project." + f"Missing {path}: this project carries no platform lock for {platform}. " + "The platform lock is authored on the project side and committed with the project; " + f"'devcapsule project --platform {platform} init' adds it." ) value = load_toml(path) validate_file_format(value, ConfigurationFileKind.lock, path) @@ -439,9 +481,10 @@ def composition_for(root: Path, manifest: Mapping[str, Any]) -> tuple[Path, dict checkout = load_checkout(record, manifest, root) if record is not None else {} selected = selected_version_lock(checkout) if selected is not None: - if selected.get("platform") != str(Platform.current()): + platform = project_platform(root) + if selected.get("platform") != str(platform): raise ProjectConfigurationError("Local version set is for another platform; follow the project explicitly.") - path, shared = root / ".devcapsule" / f"devcapsule.{Platform.current()}.lock", selected + path, shared = platform_lock_path(root, platform), selected else: path, shared = recommendation_lock_for(root, manifest) return path, compose_lock(manifest, shared, checkout), checkout diff --git a/devcapsule-src/devcapsule/environment_realization.py b/devcapsule-src/devcapsule/environment_realization.py index 43edc8f7..32e7f7ca 100644 --- a/devcapsule-src/devcapsule/environment_realization.py +++ b/devcapsule-src/devcapsule/environment_realization.py @@ -3,6 +3,7 @@ from __future__ import annotations from dataclasses import dataclass +from functools import partial from pathlib import Path from typing import Any, Callable @@ -10,6 +11,7 @@ from python_on_whales.exceptions import DockerException from devcapsule.compat import CliError +from devcapsule import host_daemon from devcapsule.platforms import Platform from devcapsule.images.build import BuildxImageBuilder, ImageBuildSpec from devcapsule.images.metadata import inspect_local_image @@ -74,6 +76,7 @@ def realize_environment( if base_override != selected.lock.get("base", {}).get("reference"): raise CliError("A base override cannot establish the project's minimum-version guarantee; use its pinned compatible base.") locked = parse_locked_environment(selected.lock) + capsule_platform = Platform.parse(locked.platform) from devcapsule.configuration.storage import load_settings if trust is None: trust = HomeTrust.read(load_settings()) @@ -110,12 +113,22 @@ def realize_environment( selection.local_image_identity if local_authorization else locked.base_identity ) + if not local_authorization and base_override is None: + from devcapsule.resolution_matrix import MATRICES + + if MATRICES[capsule_platform].base_unpublished(base_reference): + raise CliError( + f"The lock's base {base_reference} is not published yet, so it cannot be pulled. Build the " + f"{capsule_platform} base locally ('devcapsule images build --type base --recipe ubuntu-24.04 " + f"--platform {capsule_platform} --tag --source-revision ') and select it " + "for this checkout ('devcapsule project config authorize base-image ')." + ) if obtain_image is not None: obtain = obtain_image elif local_authorization: obtain = required_local_image else: - obtain = ensure_local_image + obtain = partial(ensure_local_image, platform=capsule_platform) inspect = inspect_image or optional_local_image require = require_image or required_local_image base = obtain(base_reference) @@ -139,7 +152,7 @@ def realize_environment( # One context root for every build: BuildKit's incremental transfer of # the IDE trees is keyed on this path, see BuildxImageBuilder. builder = BuildxImageBuilder(context_root=selected_cache / "build-contexts" / "context") - build = lambda spec: builder.build(spec, network="none") + build = partial(builder.build, network="none", platform=capsule_platform) materialize_environment = materialize or ensure_materialized_surface # Explanation/reporting hooks ride only the real materializer; injected # test materializers keep their narrower signature. @@ -147,7 +160,7 @@ def realize_environment( {} if materialize is not None else { - "runtime_pex": runtime_artifact(), + "runtime_pex": runtime_artifact(capsule_platform), "runtime_command": RuntimeCommand(runtime.get("devcapsule-command", RuntimeCommand.STANDARD)), "report": report, "list_formations": lambda: component_formations(locked.component_id), @@ -179,12 +192,17 @@ def realize_environment( ) -def ensure_local_image(reference: str) -> ImageDetails: +def ensure_local_image(reference: str, platform: Platform | None = None) -> ImageDetails: + """The local image at ``reference``, pulled for ``platform`` when absent. + + ``platform`` is the lock's; without one, the platform this host creates + projects for. A daemon that runs it natively pulls as before. + """ + capsule_platform = platform or Platform.current() try: if not docker.image.exists(reference): - capsule = Platform.current() - if capsule.emulated(): - docker.image.pull(reference, platform=capsule.docker_platform) + if capsule_platform.emulated(host_daemon.daemon_architecture()): + docker.image.pull(reference, platform=capsule_platform.docker_platform) else: docker.image.pull(reference) except DockerException as exc: diff --git a/devcapsule-src/devcapsule/host_daemon.py b/devcapsule-src/devcapsule/host_daemon.py index adfcb76e..e3845ace 100644 --- a/devcapsule-src/devcapsule/host_daemon.py +++ b/devcapsule-src/devcapsule/host_daemon.py @@ -21,6 +21,7 @@ from __future__ import annotations from collections.abc import Mapping, Sequence +from functools import cache import json from pathlib import Path, PurePosixPath import re @@ -207,3 +208,25 @@ def _docker(arguments: Sequence[str], env: Mapping[str, str]) -> subprocess.Comp def _unescape(value: str) -> str: return re.sub(r"\\([0-7]{3})", lambda match: chr(int(match.group(1), 8)), value) + + +@cache +def daemon_architecture() -> str: + """The Docker daemon's CPU architecture, in the platform keys' spelling. + + Emulation is the daemon's, not the launcher's: Docker Desktop on Apple + Silicon is arm64 whether the launcher runs natively on the Mac or inside + an emulated amd64 capsule granted the daemon. Asked once per process; + this host's architecture when the daemon does not answer. + """ + from devcapsule.platforms import architecture_key, host_architecture + + try: + completed = subprocess.run( + ["docker", "version", "--format", "{{.Server.Arch}}"], + capture_output=True, text=True, timeout=30, check=False, + ) + except (OSError, subprocess.TimeoutExpired): + return host_architecture() + reported = completed.stdout.strip() + return architecture_key(reported) if completed.returncode == 0 and reported else host_architecture() diff --git a/devcapsule-src/devcapsule/host_open.py b/devcapsule-src/devcapsule/host_open.py index e6ea1e07..202e4275 100644 --- a/devcapsule-src/devcapsule/host_open.py +++ b/devcapsule-src/devcapsule/host_open.py @@ -5,7 +5,6 @@ from collections import deque from collections.abc import Iterator, Mapping, Sequence from contextlib import contextmanager -import errno import json import os from pathlib import Path @@ -39,9 +38,6 @@ "devcapsule: warning: host-browser is unavailable on macOS: Docker Desktop does not carry the " "host's Unix sockets into a capsule. Links open in the capsule's own browser instead." ) -# macOS: SOL_LOCAL, and the leading fields of struct xucred (cr_version, cr_uid). -_SOL_LOCAL = 0 -_XUCRED_SIZE = 76 class HostOpenError(Exception): @@ -89,13 +85,7 @@ def open_host_url( connection.settimeout(timeout) connection.connect(str(path)) connection.sendall(request) - try: - connection.shutdown(socket.SHUT_WR) - except OSError as exc: - # macOS refuses shutdown once the broker has answered and - # closed; the answer is still buffered and is read below. - if exc.errno != errno.ENOTCONN: - raise + connection.shutdown(socket.SHUT_WR) response = _read_frame(connection, MAX_RESPONSE_BYTES) except (OSError, TimeoutError) as exc: raise HostOpenError("host-browser broker is unavailable") from exc @@ -261,10 +251,10 @@ def host_open_bridge( # It may only propagate a bridge explicitly inherited from its owner. yield None return - if HostOs.current() is not HostOs.LINUX: + if not HostOs.current().binds_unix_sockets: # The broker is a host Unix socket bound into the capsule. Docker - # Desktop refuses that bind source (observed on 29.3.1, macOS 27), so - # the optional integration is unavailable rather than fatal. + # Desktop refuses that bind source, so the optional integration is + # unavailable rather than fatal. print(HOST_OPEN_UNAVAILABLE_ON_MACOS, file=sys.stderr) yield None return @@ -312,15 +302,11 @@ def _broker_socket_path(parent: str | None) -> Iterator[Path]: def _peer_uid(connection: socket.socket) -> int: - if hasattr(socket, "SO_PEERCRED"): - credentials = connection.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, struct.calcsize("3i")) - _pid, uid, _gid = struct.unpack("3i", credentials) - return uid - if hasattr(socket, "LOCAL_PEERCRED"): - credentials = connection.getsockopt(_SOL_LOCAL, socket.LOCAL_PEERCRED, _XUCRED_SIZE) - _version, uid = struct.unpack_from("=II", credentials) - return uid - raise HostOpenError("peer credentials are unavailable on this platform") + if not hasattr(socket, "SO_PEERCRED"): + raise HostOpenError("peer credentials are unavailable on this platform") + credentials = connection.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, struct.calcsize("3i")) + _pid, uid, _gid = struct.unpack("3i", credentials) + return uid def _is_socket(path: Path) -> bool: diff --git a/devcapsule-src/devcapsule/images/base.py b/devcapsule-src/devcapsule/images/base.py index 6bb874c7..e964a94d 100644 --- a/devcapsule-src/devcapsule/images/base.py +++ b/devcapsule-src/devcapsule/images/base.py @@ -2,6 +2,8 @@ from __future__ import annotations +from devcapsule.platforms import Platform + import hashlib from collections.abc import Callable from dataclasses import dataclass @@ -317,8 +319,9 @@ def build_base_image( *, network: str = "default", source_verifier: Callable[[BuildInfo], None] = verify_public_github_revision, + platform: Platform | None = None, ) -> None: spec = build_base_image_spec(options) if not options.allow_local_source: source_verifier(pex_build_info(options)) - (builder or BuildxImageBuilder()).build(spec, network=network) + (builder or BuildxImageBuilder()).build(spec, network=network, platform=platform) diff --git a/devcapsule-src/devcapsule/images/build.py b/devcapsule-src/devcapsule/images/build.py index b43cb9ff..48a35529 100644 --- a/devcapsule-src/devcapsule/images/build.py +++ b/devcapsule-src/devcapsule/images/build.py @@ -19,6 +19,7 @@ from python_on_whales.exceptions import DockerException from devcapsule.compat import CliError, copy_resource_tree +from devcapsule import host_daemon from devcapsule.platforms import Platform @@ -257,8 +258,11 @@ class BuildxImageBuilder: def __init__(self, context_root: Path | None = None) -> None: self.context_root = context_root - def build(self, spec: ImageBuildSpec, *, network: str = "default") -> None: + def build(self, spec: ImageBuildSpec, *, network: str = "default", platform: Platform | None = None) -> None: + """Build ``spec`` for ``platform``: a project's lock names it; a base build without one + is for the platform this host creates projects for.""" plan = spec.build_plan() + capsule_platform = platform or Platform.current() try: with self._context_directory() as context_root: rendered = render_build_context(plan, context_root) @@ -267,9 +271,10 @@ def build(self, spec: ImageBuildSpec, *, network: str = "default") -> None: file=rendered.dockerfile, tags=[plan.image], load=True, - # Native hosts build as before; an emulating host must - # build for the capsule platform, not its own CPU. - platforms=[capsule.docker_platform] if (capsule := Platform.current()).emulated() else None, + # A daemon of the capsule platform's architecture builds as + # before; another must be told the platform, not left its own. + platforms=([capsule_platform.docker_platform] + if capsule_platform.emulated(host_daemon.daemon_architecture()) else None), network=network, build_contexts={name: str(path) for name, path in rendered.named_contexts.items()}, allow=["network.host"] if network == "host" else [], diff --git a/devcapsule-src/devcapsule/launch/pycharm/_launcher.py b/devcapsule-src/devcapsule/launch/pycharm/_launcher.py index 836e7970..c2af6f1f 100644 --- a/devcapsule-src/devcapsule/launch/pycharm/_launcher.py +++ b/devcapsule-src/devcapsule/launch/pycharm/_launcher.py @@ -58,6 +58,7 @@ print_only_opener, watch_display_ready, ) +from ... import host_daemon from ...host_daemon import ( current_container, docker_socket, @@ -236,6 +237,8 @@ class PycharmRunOptions: # caller (commands/project.py); without it the launcher announces the # absence once and plans no console: no port, no token, no wait. web_console: WebConsoleAvailability = WebConsoleAvailability(True, None) + # The lock's platform; without a lock, the one this host creates projects for. + capsule_platform: Platform | None = None @dataclass @@ -289,6 +292,8 @@ class PycharmRunConfig: console_token: str = "" open_console_url: Callable[[str], None] | None = None web_console: WebConsoleAvailability = WebConsoleAvailability(True, None) + # The lock's platform; without a lock, the one this host creates projects for. + capsule_platform: Platform = field(default_factory=Platform.current) # This session's log directory on the host, bound into the capsule at # SESSION_LOG_DESTINATION: the launcher's own log, the supervisor's, and # one file per supervised child (see devcapsule.logs). @@ -305,11 +310,6 @@ class PycharmRunConfig: libgl_dri3_disable: str = "1" -# Docker Desktop presents its socket inside every container as root:root, -# mode 0660, whatever the macOS file's owner (observed on 29.3.1). The capsule -# user joins group 0 to use it. A capsule granted the host daemon already -# holds root on Docker Desktop's VM, so the group adds no authority. -DOCKER_DESKTOP_SOCKET_GID = 0 HOST_NETWORK_UNAVAILABLE_ON_MACOS = ( "devcapsule: warning: host networking is unavailable on macOS: Docker Desktop's host network " "is its virtual machine's, out of the Mac's reach. This launch uses bridge networking; " @@ -835,7 +835,7 @@ def build_run_config(options: PycharmRunOptions, env: Mapping[str, str]) -> Pych ) if options.network_mode not in {"bridge", "host", "none"}: raise PycharmRunError("The Docker network mode must be bridge, host, or none.") - if options.network_mode == "host" and HostOs.current() is HostOs.DARWIN: + if options.network_mode == "host" and not HostOs.current().host_network_reachable: # Docker Desktop's host network is its VM's: a listener on loopback # there is unreachable from the Mac (observed on 29.3.1), so the # desktop and console would never answer. Bridge is the narrower grant. @@ -982,10 +982,12 @@ def build_run_config(options: PycharmRunOptions, env: Mapping[str, str]) -> Pych "Start Docker on the host, set HOST_DOCKER_SOCKET, or launch with --docker-in-docker / --no-docker." ) host_docker_socket = host_docker_socket.resolve() - host_docker_gid = ( - DOCKER_DESKTOP_SOCKET_GID if HostOs.current() is HostOs.DARWIN - else host_docker_socket.stat().st_gid - ) + # Where the host decides the socket's group inside containers (Docker + # Desktop: root), the capsule user joins it; a capsule granted the + # daemon already holds root on its virtual machine, so the group adds + # no authority. Elsewhere the host socket file's group applies. + socket_gid = HostOs.current().docker_socket_gid + host_docker_gid = socket_gid if socket_gid is not None else host_docker_socket.stat().st_gid if enable_sudo: while ide_sudo_gid in {0, host_user.gid, host_docker_gid}: @@ -1192,6 +1194,7 @@ def build_run_config(options: PycharmRunOptions, env: Mapping[str, str]) -> Pych console_token=console_token, open_console_url=options.open_console_url, web_console=options.web_console, + capsule_platform=options.capsule_platform or Platform.current(), interactive_state_mounts=tuple(interactive_state_mounts), additional_state_mounts=tuple(additional_state_mounts), additional_environment=tuple(additional_environment), @@ -1321,7 +1324,7 @@ def build_docker_args( args = [ "--rm", "-i", - *Platform.current().docker_platform_args(), + *config.capsule_platform.docker_platform_args(host_daemon.daemon_architecture()), "--name", config.name, "--workdir", @@ -1874,7 +1877,7 @@ def ensure_sudoers_policy_ownership( policy = files.sudoers_file if not config.enable_sudo or policy is None: raise PycharmRunError("Development sudo requires a generated sudoers policy.") - if HostOs.current() is HostOs.DARWIN: + if not HostOs.current().chown_reaches_bind_mounts: files.sudoers_volume = prepare_sudoers_policy_volume(config, policy, env) return try: @@ -1886,7 +1889,7 @@ def ensure_sudoers_policy_ownership( "run", "--rm", "--pull=never", - *Platform.current().docker_platform_args(), + *config.capsule_platform.docker_platform_args(host_daemon.daemon_architecture()), "--network", "none", "--read-only", @@ -1935,11 +1938,16 @@ def prepare_sudoers_policy_volume(config: PycharmRunConfig, policy: Path, env: M with its virtual machine as the Mac user: a bind-mounted file is owned by that user inside every container, and not even root there can chown it. A volume lives on the virtual machine's own disk, where root's ownership - holds. The name follows the container's, so a volume a killed launcher - left behind is rewritten and then removed by the next launch. + holds. The session removes it at its end; one a killed or interrupted + launch left behind carries the label, and the next launch prunes every + such volume no container uses before it creates its own. """ volume = f"{config.name}-sudoers" + subprocess.run( + ["docker", "volume", "prune", "--all", "--force", "--filter", f"label={SUDOERS_VOLUME_LABEL}"], + check=False, env=dict(env), stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, + ) created = subprocess.run( ["docker", "volume", "create", "--label", SUDOERS_VOLUME_LABEL, volume], check=False, env=dict(env), stdout=subprocess.DEVNULL, stderr=subprocess.PIPE, text=True, @@ -1956,7 +1964,7 @@ def prepare_sudoers_policy_volume(config: PycharmRunConfig, policy: Path, env: M "--rm", "-i", "--pull=never", - *Platform.current().docker_platform_args(), + *config.capsule_platform.docker_platform_args(host_daemon.daemon_architecture()), "--network", "none", "--read-only", diff --git a/devcapsule-src/devcapsule/platforms.py b/devcapsule-src/devcapsule/platforms.py index 04aa7e81..a49fa077 100644 --- a/devcapsule-src/devcapsule/platforms.py +++ b/devcapsule-src/devcapsule/platforms.py @@ -13,7 +13,6 @@ from __future__ import annotations -from contextvars import ContextVar from dataclasses import dataclass, field from enum import StrEnum import os @@ -21,25 +20,34 @@ import platform as host_platform from typing import Mapping -__all__ = ["HostOs", "Platform", "UnsupportedPlatformError", "XdgHomes", "host_architecture"] +__all__ = ["HostOs", "Platform", "UnsupportedPlatformError", "XdgHomes", "architecture_key", "host_architecture"] class UnsupportedPlatformError(ValueError): """A platform key outside the supported set; the message is displayable.""" +def architecture_key(machine: str) -> str: + """A CPU architecture as a machine or a Docker daemon reports it, in the platform keys' spelling.""" + + lowered = machine.lower() + return {"x86_64": "amd64", "aarch64": "arm64"}.get(lowered, lowered) + + def host_architecture() -> str: """This host's CPU architecture in the platform keys' spelling.""" - machine = host_platform.machine().lower() - return {"x86_64": "amd64", "aarch64": "arm64"}.get(machine, machine) + return architecture_key(host_platform.machine()) class HostOs(StrEnum): """The host operating system, for the mechanisms that differ by host. Locks never carry it: capsules are Linux on every host, and the lock - names the capsule platform (``Platform``). + names the capsule platform (``Platform``). On macOS the container runtime + is Docker Desktop's virtual machine; its conventions, observed on Docker + Desktop 29.3.1, are the properties below, which feature code consults + instead of asking which host it is on. """ LINUX = "linux" @@ -56,18 +64,66 @@ def current(cls) -> HostOs: f"Unsupported host operating system {system!r}; supported: {supported}." ) from None + @property + def docker_socket_gid(self) -> int | None: + """The group owning the Docker socket inside a container, when the host decides it. + + Docker Desktop presents its socket as ``root:root``, mode 0660, in every + container, whatever the Mac file's owner. None on Linux, where the host + socket file's own group applies. + """ + + return 0 if self is HostOs.DARWIN else None + + @property + def host_network_reachable(self) -> bool: + """Whether a listener on Docker's host network is reachable from this host. + + Docker Desktop's host network is its virtual machine's, out of the Mac's reach. + """ + + return self is HostOs.LINUX + + @property + def binds_unix_sockets(self) -> bool: + """Whether a host Unix socket can be bind-mounted into a container. + + Docker Desktop rewrites the source under ``/socket_mnt`` and refuses it. + """ + + return self is HostOs.LINUX -# The capsule platforms each host runs, preferred first. A Linux host runs its -# own platform. Apple Silicon runs linux-arm64 natively and linux-amd64 under + @property + def chown_reaches_bind_mounts(self) -> bool: + """Whether root in a container can change a bind-mounted host file's owner. + + Docker Desktop shares Mac files as the Mac user; a chown inside is refused. + """ + + return self is HostOs.LINUX + + +# The capsule platforms each host runs, native first. A Linux host runs its +# own platform; Apple Silicon runs linux-arm64 natively and linux-amd64 under # Docker Desktop's emulation; an Intel Mac runs linux-amd64. -_CAPSULE_PLATFORMS_OF_HOST: Mapping[str, tuple[str, ...]] = { - "darwin-arm64": ("linux-arm64", "linux-amd64"), - "darwin-amd64": ("linux-amd64",), +_CAPSULE_PLATFORMS_OF_HOST: Mapping[tuple[HostOs, str], tuple[str, ...]] = { + (HostOs.LINUX, "amd64"): ("linux-amd64",), + (HostOs.LINUX, "arm64"): ("linux-arm64",), + (HostOs.DARWIN, "arm64"): ("linux-arm64", "linux-amd64"), + (HostOs.DARWIN, "amd64"): ("linux-amd64",), +} + +# The platform a host creates a new project for, where it is not the native +# one. Apple Silicon creates linux-amd64 projects, as before linux-arm64 +# existed, while the linux-arm64 base is unpublished: the human's ruling of +# 2026-11-04 runs a project as linux-arm64 when it is locked for it, and a +# developer adds that lock deliberately (project --platform linux-arm64 init). +_NEW_PROJECT_PLATFORM_OF_HOST: Mapping[tuple[HostOs, str], str] = { + (HostOs.DARWIN, "arm64"): "linux-amd64", } -# The platform selected for the project this process works on, if any; see -# Platform.select_for_project. -_selected_platform: ContextVar[Platform | None] = ContextVar("devcapsule_capsule_platform", default=None) +# The platform whose executables carry each ELF e_machine value. +_PLATFORM_OF_ELF_MACHINE: Mapping[int, str] = {0x3E: "linux-amd64", 0xB7: "linux-arm64"} class Platform(StrEnum): @@ -80,9 +136,12 @@ class Platform(StrEnum): Because committed locks carry these values, members are append-only and never renamed. - A client never constructs a key from parts: it obtains one from - ``current()`` (the platform this host runs capsules as) or ``parse()`` - (the platform a lock records), and only carries it between those points. + A client never constructs a key from parts. Host detection gives + ``current()`` (the platform this host creates a project for) and + ``runnable()`` (every platform it runs); ``parse()`` gives the platform a + lock records, and ``of_elf_machine()`` the one an executable is for. + Which platform a project runs as is the configuration's question + (``configuration.storage.project_platform``), answered from its locks. """ LINUX_AMD64 = "linux-amd64" @@ -90,58 +149,30 @@ class Platform(StrEnum): @classmethod def current(cls) -> Platform: - """The platform this process runs capsules as. + """The platform this host creates a new project for. - The platform selected for its project, when one was; otherwise the - host's preferred platform: on Linux the host's own, on Apple Silicon - linux-arm64. + On Linux the host's own. On a Mac linux-amd64, natively on Intel and + under Docker Desktop's emulation on Apple Silicon. """ - return _selected_platform.get() or cls.runnable()[0] + host = (HostOs.current(), host_architecture()) + created = _NEW_PROJECT_PLATFORM_OF_HOST.get(host) + return cls.parse(created) if created is not None else cls.runnable()[0] @classmethod def runnable(cls) -> tuple[Platform, ...]: - """The platforms this host runs capsules as, preferred first.""" - - host = f"{host_platform.system().lower()}-{host_architecture()}" - return tuple(cls.parse(key) for key in _CAPSULE_PLATFORMS_OF_HOST.get(host, (host,))) - - @classmethod - def select_for_project(cls, root: Path | None) -> Platform: - """Select, for this process, the platform the project at ``root`` runs as. - - The first platform this host runs that the project has a lock for; - without one, as for a project not yet initialized, the preferred - platform. Apple Silicon so runs a project locked for linux-arm64 - natively and one locked only for linux-amd64 under emulation. - """ - - runnable = cls.runnable() - locks = root / ".devcapsule" if root is not None else None - selected = next( - (platform for platform in runnable - if locks is not None and (locks / f"devcapsule.{platform}.lock").is_file()), - runnable[0], - ) - _selected_platform.set(selected) - return selected - - @classmethod - def select(cls, platform: Platform) -> Platform: - """Select ``platform`` for this process, as the user named it; refuse one this host cannot run.""" + """Every platform this host runs capsules as, native first.""" - runnable = cls.runnable() - if platform not in runnable: - names = ", ".join(str(member) for member in runnable) - raise UnsupportedPlatformError(f"This host runs {names} capsules, not {platform}.") - _selected_platform.set(platform) - return platform + host_os, architecture = HostOs.current(), host_architecture() + keys = _CAPSULE_PLATFORMS_OF_HOST.get((host_os, architecture), (f"{host_os}-{architecture}",)) + return tuple(cls.parse(key) for key in keys) @classmethod - def clear_selection(cls) -> None: - """Forget the project selection; ``current()`` is the host's preference again.""" + def of_elf_machine(cls, machine: int) -> Platform | None: + """The platform an ELF executable of ``machine`` runs on, or None for another architecture.""" - _selected_platform.set(None) + key = _PLATFORM_OF_ELF_MACHINE.get(machine) + return cls(key) if key is not None else None @property def architecture(self) -> str: @@ -153,19 +184,19 @@ def docker_platform(self) -> str: return self.value.replace("-", "/", 1) - def emulated(self) -> bool: - """Whether this host's CPU runs capsules of this platform only under emulation.""" + def emulated(self, daemon_architecture: str) -> bool: + """Whether a Docker daemon of ``daemon_architecture`` runs this platform only under emulation.""" - return host_architecture() != self.architecture + return daemon_architecture != self.architecture - def docker_platform_args(self) -> list[str]: - """The ``--platform`` option every run and build carries under emulation. + def docker_platform_args(self, daemon_architecture: str) -> list[str]: + """The ``--platform`` option every run, build and pull carries under emulation. - Empty on a native host, so a Linux launch composes the same command - as before the option existed. + Empty when the daemon runs this platform natively, so a native launch + composes the same command as before the option existed. """ - return [f"--platform={self.docker_platform}"] if self.emulated() else [] + return [f"--platform={self.docker_platform}"] if self.emulated(daemon_architecture) else [] @classmethod def parse(cls, key: str) -> Platform: diff --git a/devcapsule-src/devcapsule/recursive_successor_plan.py b/devcapsule-src/devcapsule/recursive_successor_plan.py index 241db940..7c808d7b 100644 --- a/devcapsule-src/devcapsule/recursive_successor_plan.py +++ b/devcapsule-src/devcapsule/recursive_successor_plan.py @@ -35,6 +35,7 @@ "--name", "--network", "--pids-limit", + "--platform", "--restart", "--security-opt", "--tmpfs", @@ -43,11 +44,9 @@ } ) _BOOLEAN_FLAGS = frozenset({"--detach", "--privileged", "--read-only", "--rm", "-i"}) -# Flags that describe how Docker obtains or executes the image rather than the -# resulting container. They are deliberately not part of the comparable plan. -# The launcher emits each as one inline token (``--platform=linux/amd64`` on -# a host that emulates the capsule platform). -_IGNORED_FLAGS = frozenset({"--platform", "--pull"}) +# Flags that describe how Docker obtains the image rather than the resulting +# container. They are deliberately not part of the comparable plan. +_IGNORED_FLAGS = frozenset({"--pull"}) class SuccessorPlanError(Exception): @@ -111,6 +110,9 @@ class ExpectedSuccessorPlan: security_opt: tuple[str, ...] runtime_plan_destination: str runtime_plan_digest: str + # The planned ``--platform`` (``linux/amd64``), when the daemon emulates the + # capsule platform; None when the image's own platform is the daemon's. + platform: str | None = None def to_mapping(self, *, show_host_paths: bool = False) -> dict[str, Any]: return { @@ -272,6 +274,7 @@ def from_docker_args( security_opt=tuple(sorted(parsed.repeated("--security-opt"))), runtime_plan_destination=runtime_plan_destination, runtime_plan_digest=runtime_plan_digest, + platform=parsed.optional("--platform"), ) @@ -293,6 +296,13 @@ def compare_inspection( raise SuccessorPlanError("successor name does not match the deterministic run name") if inspection.get("Image") != plan.image_identity: raise SuccessorPlanError("successor image ID does not match materialization evidence") + # The image ID fixes the architecture; where the daemon also reports the + # platform it ran, an emulated plan must have run as the one it named. + descriptor = inspection.get("ImageManifestDescriptor") + reported = descriptor.get("platform") if isinstance(descriptor, Mapping) else None + if plan.platform is not None and isinstance(reported, Mapping): + if f"{reported.get('os')}/{reported.get('architecture')}" != plan.platform: + raise SuccessorPlanError("successor platform does not match the planned --platform") if config.get("Image") != plan.image_reference: raise SuccessorPlanError("successor image reference does not match the canonical environment") diff --git a/devcapsule-src/devcapsule/resolution_matrix.py b/devcapsule-src/devcapsule/resolution_matrix.py index 155dc5cf..b99b06a6 100644 --- a/devcapsule-src/devcapsule/resolution_matrix.py +++ b/devcapsule-src/devcapsule/resolution_matrix.py @@ -122,6 +122,9 @@ class _BasePin: display: str = HOST_X11_ONLY_DISPLAY runtime: str = LAUNCHER_SUPPLIED_RUNTIME source_revision: str | None = None + # False while the reference names an image its registry does not hold + # yet: only a locally built base of the same recipe stands in for it. + published: bool = True # A clean public-source builder need not have a release tag. @property @@ -247,6 +250,10 @@ def base_family(self, lock: Mapping[str, Any]) -> str | None: return next((base.base_family for base in self._bases if base.lock_table.get("reference") == reference), None) + def base_unpublished(self, reference: str) -> bool: + """Whether the pinned base at ``reference`` is not in its registry yet.""" + return any(base.lock_table.get("reference") == reference and not base.published for base in self._bases) + def base_image(self, reference: str) -> BaseImage | None: """The pinned base at ``reference``, described by its contract.""" return next((base.image for base in self._bases @@ -1733,6 +1740,7 @@ def _formation( recipe=11, display=CONTAINED_DISPLAY, runtime=LAUNCHER_SUPPLIED_RUNTIME, + published=False, lock_table={ "reference": ( "docker.io/mycodespaceai/devcapsule-base" diff --git a/devcapsule-src/devcapsule/runtime_artifact.py b/devcapsule-src/devcapsule/runtime_artifact.py index 72f4d87d..072f8f48 100644 --- a/devcapsule-src/devcapsule/runtime_artifact.py +++ b/devcapsule-src/devcapsule/runtime_artifact.py @@ -6,69 +6,66 @@ from pathlib import Path import sys -from devcapsule.build_info import BuildInfoError, read_pex_build_info +from devcapsule.build_info import BuildInfo, BuildInfoError, current_build_info, read_pex_build_info from devcapsule.compat import CliError -from devcapsule.platforms import Platform +from devcapsule import platforms +from devcapsule.platforms import HostOs, Platform RUNTIME_PEX_ENV = "DEVCAPSULE_RUNTIME_PEX" -# ELF e_machine values, by the architecture spelling of platform keys. -_ELF_MACHINES = {0x3E: "amd64", 0xB7: "arm64"} _MACH_O_MAGICS = (b"\xcf\xfa\xed\xfe", b"\xce\xfa\xed\xfe", b"\xca\xfe\xba\xbe") +_UNKNOWN_REVISION = "unknown" -def runtime_artifact() -> Path: - """The DevCapsule executable this launch delivers into the capsule. +def runtime_artifact(capsule_platform: Platform) -> Path: + """The DevCapsule executable a launch delivers into a ``capsule_platform`` capsule. - A packaged launcher delivers itself when its binary runs on the capsule - platform. A launcher that cannot, a macOS executable launching Linux - capsules, delivers the runtime ``DEVCAPSULE_RUNTIME_PEX`` names. A - source-form launcher must name its built runtime the same way. The - variable names one executable, or a directory holding one per capsule - platform as ``devcapsule-``, for a host that runs two. + A packaged launcher delivers itself when it runs on the capsule platform + (D-0009). A launcher that cannot, such as a macOS executable launching + Linux capsules, delivers the runtime ``DEVCAPSULE_RUNTIME_PEX`` names: one + executable, or a directory holding one per capsule platform as + ``devcapsule-``. A source-form launcher names its built runtime + the same way. A delivered runtime that is not the launcher's own bytes + must be the launcher's build: the same version, and the same source + revision wherever both record one (R-RUNTIME-001). """ - capsule = Platform.current() # scie preserves the executable's path in SCIE; PEX points to the original # archive too. sys.argv[0] alone can name the extracted console script. - own = os.environ.get("SCIE") or os.environ.get("PEX") - explicit = os.environ.get(RUNTIME_PEX_ENV) - if explicit and Path(explicit).expanduser().is_dir(): - directory = Path(explicit).expanduser() - explicit = str(directory / f"devcapsule-{capsule}") - if not Path(explicit).is_file() and executable_platform(_resolved(own or sys.argv[0])) != str(capsule): - raise CliError( - f"{RUNTIME_PEX_ENV} names {directory}, which holds no devcapsule-{capsule}: " - f"this launch runs a {capsule} capsule. Put a {capsule} DevCapsule executable of the same " - f"version there, or a directory holding it as devcapsule-{capsule}." - ) - path = _resolved(own or explicit or sys.argv[0]) - if own and explicit and executable_platform(path) not in (None, str(capsule)): - path = _resolved(explicit) + own_variable = os.environ.get("SCIE") or os.environ.get("PEX") + own = _resolved(own_variable) if own_variable else None + named = os.environ.get(RUNTIME_PEX_ENV) + if own is not None and _runs_in(own, capsule_platform): + delivered = own + elif named: + delivered = _named_runtime(Path(named).expanduser(), capsule_platform) + else: + delivered = own or _resolved(sys.argv[0]) try: - read_pex_build_info(path) + delivered_build = read_pex_build_info(delivered) except BuildInfoError as exc: raise CliError( "Environment materialization needs a DevCapsule PEX. Run the built PEX, " "or build with 'nox -s pex' and set DEVCAPSULE_RUNTIME_PEX to its path " "when running from source." ) from exc - found = executable_platform(path) - if found not in (None, str(capsule)): + if not _runs_in(delivered, capsule_platform): raise CliError( - f"{path} is a {found} executable and cannot run inside {capsule} capsules. " - f"Set {RUNTIME_PEX_ENV} to a {capsule} DevCapsule executable of the same version, " - f"or to a directory holding it as devcapsule-{capsule}." + f"{delivered} is {_describe(executable_platform(delivered))} and cannot run inside " + f"{capsule_platform} capsules. Set {RUNTIME_PEX_ENV} to a {capsule_platform} DevCapsule " + f"executable of the same version, or to a directory holding it as devcapsule-{capsule_platform}." ) - return path + if delivered != own: + _require_same_build(delivered, delivered_build, current_build_info()) + return delivered -def executable_platform(path: Path) -> str | None: - """The platform a native executable runs on, from its header. +def executable_platform(path: Path) -> Platform | HostOs | None: + """What a native executable runs on, from its header. - ``darwin`` for any Mach-O file, ``linux-`` for an ELF file of a - known architecture, and None for anything else, such as a plain PEX - archive, whose interpreter the capsule supplies. + The ``Platform`` of an ELF executable of a known architecture, + ``HostOs.LINUX`` for one of another architecture, ``HostOs.DARWIN`` for any + Mach-O file, and None for anything else, such as a plain PEX archive. """ try: @@ -77,13 +74,60 @@ def executable_platform(path: Path) -> str | None: except OSError: return None if header[:4] in _MACH_O_MAGICS: - return "darwin" + return HostOs.DARWIN if header[:4] == b"\x7fELF" and len(header) >= 20: machine = int.from_bytes(header[18:20], "little" if header[5] == 1 else "big") - architecture = _ELF_MACHINES.get(machine) - return f"linux-{architecture}" if architecture else "linux-unknown" + return Platform.of_elf_machine(machine) or HostOs.LINUX return None +def _runs_in(path: Path, capsule_platform: Platform) -> bool: + """Whether the executable at ``path`` runs inside ``capsule_platform`` capsules. + + A plain PEX bundles native wheels for the machine it was built on, so it + counts as the capsule platform's only on a Linux host of that architecture. + """ + + runs_on = executable_platform(path) + if runs_on is None: + return HostOs.current() is HostOs.LINUX and platforms.host_architecture() == capsule_platform.architecture + return runs_on == capsule_platform + + +def _named_runtime(named: Path, capsule_platform: Platform) -> Path: + if not named.is_dir(): + return _resolved(str(named)) + candidate = named / f"devcapsule-{capsule_platform}" + if not candidate.is_file(): + raise CliError( + f"{RUNTIME_PEX_ENV} names {named}, which holds no devcapsule-{capsule_platform}: this launch " + f"runs a {capsule_platform} capsule. Put a {capsule_platform} DevCapsule executable of the same " + f"version there, or a directory holding it as devcapsule-{capsule_platform}." + ) + return _resolved(str(candidate)) + + +def _require_same_build(delivered: Path, delivered_build: BuildInfo, launcher_build: BuildInfo) -> None: + revisions = (delivered_build.source_revision, launcher_build.source_revision) + revisions_differ = _UNKNOWN_REVISION not in revisions and revisions[0] != revisions[1] + if delivered_build.version != launcher_build.version or revisions_differ: + raise CliError( + f"{delivered} is DevCapsule {delivered_build.version} from {delivered_build.source_revision}, " + f"but this launcher is {launcher_build.version} from {launcher_build.source_revision}. The runtime " + "a launch delivers runs the plan this launcher writes, so it must be the same build: rebuild " + f"it from the launcher's commit and point {RUNTIME_PEX_ENV} at it." + ) + + +def _describe(runs_on: Platform | HostOs | None) -> str: + if runs_on is HostOs.DARWIN: + return "a macOS executable" + if runs_on is HostOs.LINUX: + return "a Linux executable of an architecture no capsule platform names" + if runs_on is None: + return "a plain PEX built for another machine" + return f"a {runs_on} executable" + + def _resolved(selected: str) -> Path: return Path(selected).expanduser().resolve() diff --git a/devcapsule-src/devcapsule/version_sets.py b/devcapsule-src/devcapsule/version_sets.py index 883a559f..101e1c3a 100644 --- a/devcapsule-src/devcapsule/version_sets.py +++ b/devcapsule-src/devcapsule/version_sets.py @@ -137,7 +137,7 @@ def _validation(lock: Mapping[str, Any], checkout: Mapping[str, Any]) -> tuple[t missing = tuple(f"{name} {metadata.get('version')} on local base {local_base}" for name, metadata in lock["components"].items() if isinstance(metadata, dict)) return (), missing - return MATRICES[Platform.current()].validation_evidence(lock) + return MATRICES[Platform.parse(str(lock.get("platform")))].validation_evidence(lock) INSPECTION_SCHEMA_VERSION = 1 @@ -410,7 +410,7 @@ def preview(start: Path, component: str, version: str, *, report: Callable[[str] if channel is None: raise CliError(f"{component}: {definition.channel_omission_reason() or 'no distribution channel declared'}") selection = channel.select(version, str(workspace.lock["platform"])) - matrix = MATRICES[Platform.current()] + matrix = MATRICES[Platform.parse(str(workspace.lock["platform"]))] if workspace.lock["platform"] not in selection.platforms: raise CliError("Candidate does not support the selected platform.") if selection.base_families and matrix.base_family(workspace.lock) not in selection.base_families: @@ -634,6 +634,9 @@ def _known(workspace: Workspace) -> list[tuple[str, dict[str, Any], dict[str, An raise CliError(f"Cannot interpret known-good record {path}: {exc}. Inspect the record; current selection is unchanged.") from exc if set_id(lock, document.get("base", {}).get("image-id")) != path.stem: raise CliError(f"Known-good record was modified: {path}. Inspect it before recovery.") + # A checkout may run as two platforms; only this platform's sets are its history. + if lock.get("platform") != workspace.lock.get("platform"): + continue result.append((path.stem, lock, document)) return sorted(result, key=lambda entry: entry[2]["last-success"], reverse=True) @@ -673,7 +676,8 @@ def rollback(start: Path, identity: str | None = None, *, reacquire: bool = Fals descriptor = formation_descriptor(platform=parsed.platform, base_identity=details.identity, artifact=parsed.artifact, ancillary_artifacts=parsed.ancillary_artifacts, recipe_id=parsed.recipe_id, recipe_version=parsed.recipe_version, - component_id=parsed.component_id, runtime_sha256=sha256_file(runtime_artifact())) + component_id=parsed.component_id, + runtime_sha256=sha256_file(runtime_artifact(Platform.parse(parsed.platform)))) if missing: name = canonical_image_name(descriptor, parsed.component_id) verify_materialized_image(required_local_image(name), descriptor=descriptor, canonical_name=name) diff --git a/devcapsule-src/pyproject.toml b/devcapsule-src/pyproject.toml index 324f3335..ec0dd382 100644 --- a/devcapsule-src/pyproject.toml +++ b/devcapsule-src/pyproject.toml @@ -95,7 +95,7 @@ markers = [ "ide_smoke: launches a real IDE session per surface with the executable under test; opt-in through nox -s ide-smoke", "flaky: executes for visibility but has a temporarily non-blocking result", "legacy: predates the unit-test conventions of WORKFLOW-LOCAL.md; the file only loses tests, see scripts/validate_unittests_follow_conventions.py", - "real_host_os: sees the real host operating system instead of the suite's Linux default", + "own_host_patches: the test sets up its host itself; the suite's amd64 Linux host defaults stay off", ] [tool.coverage.run] diff --git a/devcapsule-src/tests/conftest.py b/devcapsule-src/tests/conftest.py index 0928a195..d4a5fab3 100644 --- a/devcapsule-src/tests/conftest.py +++ b/devcapsule-src/tests/conftest.py @@ -26,17 +26,19 @@ def pytest_configure(config: pytest.Config) -> None: _SHORT_BASETEMP = pytest.StashKey[Path]() -def pytest_unconfigure(config: pytest.Config) -> None: - """Remove the short temporary root this session created. +def pytest_sessionfinish(session: pytest.Session, exitstatus: int) -> None: + """Remove the short temporary root this session created, once every test passed. - pytest prunes its own default roots, but not one it was given; a root per - session would otherwise outlive every run, and a full disk fails Docker - Desktop's virtual machine with it. + pytest prunes its own default roots, but not one it was given, so a root + per session would otherwise outlive every run and fill the disk; a failed + session keeps its root, and its path, for inspection. """ - created = config.stash.get(_SHORT_BASETEMP, None) - if created is not None: + created = session.config.stash.get(_SHORT_BASETEMP, None) + if created is not None and exitstatus == 0: shutil.rmtree(created, ignore_errors=True) + elif created is not None: + print(f"\nTemporary files of this failed session: {created}") @pytest.fixture(autouse=True) @@ -67,7 +69,7 @@ def linux_host_by_default(request: pytest.FixtureRequest, monkeypatch: pytest.Mo ``HostOs.current`` themselves. Docker-marked tests keep the real host. """ - if any(request.node.get_closest_marker(marker) for marker in (*DOCKER_MARKERS, "real_host_os")): + if any(request.node.get_closest_marker(marker) for marker in (*DOCKER_MARKERS, "own_host_patches")): return from devcapsule.platforms import HostOs monkeypatch.setattr(HostOs, "current", classmethod(lambda cls: HostOs.LINUX)) @@ -75,20 +77,24 @@ def linux_host_by_default(request: pytest.FixtureRequest, monkeypatch: pytest.Mo @pytest.fixture(autouse=True) def amd64_capsules_by_default(request: pytest.FixtureRequest, monkeypatch: pytest.MonkeyPatch) -> Iterator[None]: - """Make capsule-platform tests run as linux-amd64 wherever they run. - - An Apple Silicon host prefers linux-arm64; the suite's fixtures and - golden locks are linux-amd64. Tests of other platforms patch - ``Platform.runnable`` themselves; Docker-marked and ``real_host_os`` - tests keep the real host. No project selection outlives its test. + """Make capsule-platform tests run on an amd64 host with an amd64 daemon. + + With ``linux_host_by_default`` the suite's host is then an amd64 Linux + machine wherever it runs: it creates and runs linux-amd64 capsules, and + its Docker daemon needs no ``--platform``, as on the Linux CI. Tests of + other hosts patch the host themselves and carry ``own_host_patches``; + Docker-marked tests keep the real host. No ``--platform`` choice outlives + its test. """ - from devcapsule.platforms import Platform - Platform.clear_selection() - if not any(request.node.get_closest_marker(marker) for marker in (*DOCKER_MARKERS, "real_host_os")): - monkeypatch.setattr(Platform, "runnable", classmethod(lambda cls: (Platform.LINUX_AMD64,))) + from devcapsule.configuration import storage + + storage.name_platform(None) + if not any(request.node.get_closest_marker(marker) for marker in (*DOCKER_MARKERS, "own_host_patches")): + monkeypatch.setattr("devcapsule.platforms.host_architecture", lambda: "amd64") + monkeypatch.setattr("devcapsule.host_daemon.daemon_architecture", lambda: "amd64") yield - Platform.clear_selection() + storage.name_platform(None) DOCKER_MARKERS = ("e2e", "base_build_e2e", "recursive_e2e", "contributor_e2e", "ide_smoke") diff --git a/devcapsule-src/tests/e2e/test_ide_comes_alive.py b/devcapsule-src/tests/e2e/test_ide_comes_alive.py index cdac4242..d3566838 100644 --- a/devcapsule-src/tests/e2e/test_ide_comes_alive.py +++ b/devcapsule-src/tests/e2e/test_ide_comes_alive.py @@ -18,7 +18,8 @@ import pytest -from devcapsule.platforms import Platform +from devcapsule.configuration.storage import project_platform +from devcapsule.platforms import HostOs from devcapsule.runtime_artifact import RUNTIME_PEX_ENV, executable_platform from tests.e2e.ide_session import SURFACES, IdeSurface, capture_desktop, command, desktop_page_answers, ide_session, launched_ide, stop_session, wait_for_console_url, wait_for_ide_window @@ -158,17 +159,17 @@ def test_ide_comes_alive(surface: IdeSurface, built_pex: Path, tmp_path: Path, e def delivered_runtime(executable: Path, workspace: Path) -> Path: - """The executable a launch from ``executable`` delivers into the capsule. + """The executable a launch from ``executable`` should deliver into the capsule. - Itself when it runs on the capsule platform; a macOS launcher delivers the - Linux runtime its environment names instead: the file, or, from a - directory, the one for the platform the project's locks select. + Derived independently of the launcher's choice: itself when it is a Linux + executable; for a macOS launcher, the Linux runtime its environment names, + from a directory the one for the platform the workspace's locks select. """ - if executable_platform(executable) != "darwin": + if executable_platform(executable) is not HostOs.DARWIN: return executable named = Path(os.environ[RUNTIME_PEX_ENV]).expanduser().resolve() if named.is_dir(): - return named / f"devcapsule-{Platform.select_for_project(workspace)}" + return named / f"devcapsule-{project_platform(workspace)}" return named diff --git a/devcapsule-src/tests/test_base_image.py b/devcapsule-src/tests/test_base_image.py index c17f548f..d66be7ed 100644 --- a/devcapsule-src/tests/test_base_image.py +++ b/devcapsule-src/tests/test_base_image.py @@ -182,7 +182,7 @@ def test_base_image_build_forwards_host_network_to_buildx(tmp_path: Path) -> Non source_verifier.assert_called_once() builder.build.assert_called_once() assert builder.build.call_args.args[0].image == "test-base:latest" - assert builder.build.call_args.kwargs == {"network": "host"} + assert builder.build.call_args.kwargs == {"network": "host", "platform": None} def test_base_image_rejects_unreachable_revision_before_buildx(tmp_path: Path) -> None: diff --git a/devcapsule-src/tests/test_cli.py b/devcapsule-src/tests/test_cli.py index 8da2fa6b..20ee5e50 100644 --- a/devcapsule-src/tests/test_cli.py +++ b/devcapsule-src/tests/test_cli.py @@ -370,7 +370,7 @@ def test_images_build_base_maps_cli_options(tmp_path: Path, capsys) -> None: assert options.source_revision == "revision-1" assert options.allow_local_source is False assert options.recipe == "ubuntu-24.04" - assert build.call_args.kwargs == {"network": "host"} + assert build.call_args.kwargs == {"network": "host", "platform": None} output = capsys.readouterr().out assert "Image ID: sha256:abc123" in output assert "Runtime: supplied by the launcher" in output diff --git a/devcapsule-src/tests/test_environment_realization.py b/devcapsule-src/tests/test_environment_realization.py index d5178715..2e4a9368 100644 --- a/devcapsule-src/tests/test_environment_realization.py +++ b/devcapsule-src/tests/test_environment_realization.py @@ -11,6 +11,7 @@ CLAUDE_CODE_TERMS_URL, ) from devcapsule.environment_realization import ensure_local_image, realize_environment +from devcapsule.platforms import Platform from devcapsule.materialization import ImageDetails from devcapsule.runtime_command import RuntimeCommand from devcapsule.configuration.storage import ( @@ -286,18 +287,18 @@ def test_ensure_local_image_pulls_only_when_missing(already_local: bool) -> None require.assert_called_once_with(BASE_REFERENCE) -def test_an_emulating_host_pulls_the_capsule_platform() -> None: - # Apple Silicon runs linux-amd64 capsules; the pull must not resolve a - # multi-platform reference to the host's arm64 image. +@pytest.mark.parametrize("daemon,expected", [("arm64", {"platform": "linux/amd64"}), ("amd64", {})]) +def test_a_daemon_of_another_architecture_pulls_the_lock_platform(daemon: str, expected: dict[str, str]) -> None: + # Docker Desktop on Apple Silicon is an arm64 daemon; a linux-amd64 lock's + # base must not resolve to its arm64 variant. A native daemon pulls as before. with ( - patch("devcapsule.platforms.host_platform.system", return_value="Darwin"), - patch("devcapsule.platforms.host_architecture", return_value="arm64"), + patch("devcapsule.host_daemon.daemon_architecture", return_value=daemon), patch("devcapsule.environment_realization.docker.image.exists", return_value=False), patch("devcapsule.environment_realization.docker.image.pull") as pull, patch("devcapsule.environment_realization.required_local_image", return_value=base_image()), ): - ensure_local_image(BASE_REFERENCE) - pull.assert_called_once_with(BASE_REFERENCE, platform="linux/amd64") + ensure_local_image(BASE_REFERENCE, Platform.LINUX_AMD64) + pull.assert_called_once_with(BASE_REFERENCE, **expected) def test_realize_environment_allows_explicit_managed_base_override(tmp_path: Path) -> None: @@ -413,7 +414,7 @@ def test_realizer_uses_resolved_shipped_command_choice(tmp_path, monkeypatch, co selected.resolution["runtime"]["devcapsule-command"] = command materialize = Mock(return_value=(CANONICAL_IMAGE, True)) monkeypatch.setattr("devcapsule.environment_realization.ensure_materialized_surface", materialize) - monkeypatch.setattr("devcapsule.environment_realization.runtime_artifact", lambda: tmp_path / "runtime.pex") + monkeypatch.setattr("devcapsule.environment_realization.runtime_artifact", lambda _platform: tmp_path / "runtime.pex") realize_environment(selected, root=tmp_path / "cache", obtain_image=lambda _ref: base_image(), require_image=lambda _ref: completed_image(), build=Mock(), inspect_image=Mock()) assert materialize.call_args.kwargs["runtime_command"] is RuntimeCommand(command or "devcapsule") diff --git a/devcapsule-src/tests/test_host_open.py b/devcapsule-src/tests/test_host_open.py index 3d58f0b6..417ffaa6 100644 --- a/devcapsule-src/tests/test_host_open.py +++ b/devcapsule-src/tests/test_host_open.py @@ -8,6 +8,8 @@ from types import SimpleNamespace from unittest.mock import Mock, patch +import sys + import pytest from devcapsule import cli @@ -24,7 +26,12 @@ pytestmark = pytest.mark.legacy +# The broker reads its peer's credentials with SO_PEERCRED and runs only on +# Linux hosts; on macOS the bridge is unavailable before any broker exists. +linux_broker = pytest.mark.skipif(sys.platform != "linux", reason="the host-open broker runs only on Linux hosts") + +@linux_broker def test_broker_forwards_one_exact_url_without_shell_parsing(tmp_path: Path) -> None: path = tmp_path / "host-open.sock" run = Mock(return_value=SimpleNamespace(returncode=0)) @@ -90,6 +97,7 @@ def test_broker_rejects_malformed_protocol_without_invoking_opener(tmp_path: Pat run.assert_not_called() +@linux_broker def test_broker_applies_a_bounded_request_rate(tmp_path: Path) -> None: path = tmp_path / "host-open.sock" run = Mock(return_value=SimpleNamespace(returncode=0)) @@ -121,6 +129,7 @@ def test_broker_rejects_a_different_peer_uid(tmp_path: Path) -> None: run.assert_not_called() +@linux_broker def test_broker_reports_host_opener_timeout(tmp_path: Path) -> None: path = tmp_path / "host-open.sock" run = Mock(side_effect=subprocess.TimeoutExpired(["xdg-open"], 0.1)) @@ -133,6 +142,7 @@ def test_broker_reports_host_opener_timeout(tmp_path: Path) -> None: open_host_url("https://example.test", socket_path=path) +@linux_broker def test_client_reports_missing_and_failed_brokers_without_exposing_url(tmp_path: Path) -> None: url = "https://example.test/path?secret=do-not-report" with pytest.raises(HostOpenError, match="absent") as missing: @@ -170,6 +180,7 @@ def test_bridge_reuses_only_an_explicit_inherited_socket(tmp_path: Path) -> None assert disabled is None +@linux_broker def test_physical_host_bridge_owns_and_cleans_broker(tmp_path: Path) -> None: run = Mock(return_value=SimpleNamespace(returncode=0)) env = {"XDG_RUNTIME_DIR": str(tmp_path), "DISPLAY": ":1"} @@ -201,6 +212,7 @@ def test_macos_host_bridge_is_unavailable_with_a_warning(tmp_path: Path, capsys) assert list(tmp_path.iterdir()) == [] +@linux_broker def test_hidden_host_open_command_uses_authorized_environment(tmp_path: Path, capsys) -> None: assert "host-open" not in cli.command_names() assert "host-open" in cli.command_names(include_hidden=True) diff --git a/devcapsule-src/tests/test_linux_arm64_matrix.py b/devcapsule-src/tests/test_linux_arm64_matrix.py index 66eb322a..f3dc8dbb 100644 --- a/devcapsule-src/tests/test_linux_arm64_matrix.py +++ b/devcapsule-src/tests/test_linux_arm64_matrix.py @@ -35,15 +35,35 @@ def test_no_arm64_pin_names_an_amd64_artifact() -> None: assert not AMD64_MARKERS.search(url), (component, pin.version, url) -def test_every_arm64_artifact_is_hash_pinned_and_differs_from_amd64() -> None: +def _artifacts(value: Any) -> list[dict[str, Any]]: + """Every table at any depth that names a download by ``url``.""" + if isinstance(value, dict): + own = [value] if "url" in value else [] + return own + [found for item in value.values() for found in _artifacts(item)] + if isinstance(value, (list, tuple)): + return [found for item in value for found in _artifacts(item)] + return [] + + +def test_every_arm64_artifact_is_hash_pinned() -> None: for component, pins in ARM64._components.items(): - for arm64, amd64 in zip(pins, AMD64._components[component]): - if arm64.lock_table.get("delivery-policy") == "base-image": - assert arm64 is amd64 - continue - assert re.fullmatch(r"[0-9a-f]{64}", str(arm64.lock_table.get("sha256", "0" * 64))) - if "url" in arm64.lock_table and not str(arm64.lock_table["url"]).endswith("codex-" + arm64.version + ".tgz"): - assert arm64.lock_table["sha256"] != amd64.lock_table["sha256"], (component, arm64.version) + for pin in pins: + for artifact in _artifacts(pin.lock_table): + assert re.fullmatch(r"[0-9a-f]{64}", str(artifact.get("sha256"))), (component, pin.version, artifact) + + +def test_arm64_artifacts_are_not_amd64_bytes() -> None: + # Architecture-independent files are shared: Codex's JavaScript package, + # pure-Python wheels, and Ubuntu _all packages. Everything else differs. + shared = {artifact["sha256"] for pins in AMD64._components.values() for pin in pins + for artifact in _artifacts(pin.lock_table)} + for component, pins in ARM64._components.items(): + for pin in pins: + for artifact in _artifacts(pin.lock_table): + url = str(artifact["url"]) + independent = url.endswith(("_all.deb", "-none-any.whl")) or url.endswith(f"codex-{pin.version}.tgz") + if not independent: + assert artifact["sha256"] not in shared, (component, pin.version, url) def test_platform_fields_and_artifact_keys_name_arm64() -> None: @@ -93,9 +113,9 @@ def test_personal_pins_take_the_artifacts_of_the_platform_they_run_on() -> None: assert on_arm64["components"]["codium"] == dict(ARM64.component_pins()["codium"]) assert on_arm64["components"]["interactive-surface"] == "codium" assert amd64["components"]["codium"] == dict(AMD64.component_pins()["codium"]) # input unchanged - # Pins already written wrongly, arm64 platform with amd64 bytes, are healed too. - mislabelled = dict(amd64, platform="linux-arm64") - assert pins_for_platform(mislabelled, ARM64)["components"]["codium"] == dict(ARM64.component_pins()["codium"]) + # Pins written for the platform they run on are records, returned as they stand. + recorded = {**amd64, "components": {**amd64["components"], "codium": {**amd64["components"]["codium"], "note": "kept"}}} + assert pins_for_platform(recorded, AMD64) == recorded assert pins_for_platform(on_arm64, AMD64)["components"]["codium"] == dict(AMD64.component_pins()["codium"]) diff --git a/devcapsule-src/tests/test_platforms.py b/devcapsule-src/tests/test_platforms.py index f95cc158..0df3b333 100644 --- a/devcapsule-src/tests/test_platforms.py +++ b/devcapsule-src/tests/test_platforms.py @@ -43,70 +43,53 @@ def _host(monkeypatch: pytest.MonkeyPatch, system: str, machine: str) -> None: monkeypatch.setattr(platforms.host_platform, "machine", lambda: machine) -@pytest.mark.real_host_os -@pytest.mark.parametrize("system,machine,runnable", [ - ("Linux", "x86_64", ("linux-amd64",)), - ("Linux", "aarch64", ("linux-arm64",)), - ("Darwin", "x86_64", ("linux-amd64",)), - ("Darwin", "arm64", ("linux-arm64", "linux-amd64")), +@pytest.mark.own_host_patches +@pytest.mark.parametrize("system,machine,runnable,created", [ + ("Linux", "x86_64", ("linux-amd64",), "linux-amd64"), + ("Linux", "aarch64", ("linux-arm64",), "linux-arm64"), + ("Darwin", "x86_64", ("linux-amd64",), "linux-amd64"), + ("Darwin", "arm64", ("linux-arm64", "linux-amd64"), "linux-amd64"), ]) -def test_each_host_runs_its_capsule_platforms_preferring_native(monkeypatch, system, machine, runnable) -> None: - # A Mac runs Linux capsules: Apple Silicon natively as linux-arm64 and - # under Docker's emulation as linux-amd64; Intel natively as linux-amd64. +def test_each_host_runs_its_platforms_and_creates_projects_for_one(monkeypatch, system, machine, runnable, created) -> None: + # A Mac runs Linux capsules. Apple Silicon runs linux-arm64 natively and + # linux-amd64 under emulation, and creates linux-amd64 projects while the + # linux-arm64 base is unpublished (the human's ruling of 2026-11-04). _host(monkeypatch, system, machine) assert tuple(str(platform) for platform in Platform.runnable()) == runnable - assert Platform.current() is Platform.parse(runnable[0]) - assert not Platform.current().emulated() - assert Platform.current().docker_platform_args() == [] + assert Platform.current() is Platform.parse(created) -@pytest.mark.real_host_os -def test_apple_silicon_names_the_emulated_platform_to_docker(monkeypatch) -> None: - _host(monkeypatch, "Darwin", "arm64") - assert Platform.LINUX_AMD64.emulated() - assert Platform.LINUX_AMD64.docker_platform_args() == ["--platform=linux/amd64"] - assert Platform.LINUX_ARM64.docker_platform_args() == [] +def test_emulation_is_the_daemons() -> None: + # Docker Desktop on Apple Silicon is an arm64 daemon, whatever runs the launcher. + assert Platform.LINUX_AMD64.emulated("arm64") + assert Platform.LINUX_AMD64.docker_platform_args("arm64") == ["--platform=linux/amd64"] + assert Platform.LINUX_AMD64.docker_platform_args("amd64") == [] + assert Platform.LINUX_ARM64.docker_platform_args("arm64") == [] -@pytest.mark.real_host_os -@pytest.mark.parametrize("system,machine", [("Linux", "riscv64"), ("Windows", "AMD64"), ("Darwin", "ppc")]) -def test_current_refuses_a_host_without_a_capsule_platform(monkeypatch, system, machine) -> None: +@pytest.mark.own_host_patches +@pytest.mark.parametrize("system,machine,message", [ + ("Linux", "riscv64", "supported platforms: linux-amd64, linux-arm64"), + ("Darwin", "ppc", "supported platforms: linux-amd64, linux-arm64"), + ("Windows", "AMD64", "supported: linux, darwin"), +]) +def test_current_refuses_a_host_without_a_capsule_platform(monkeypatch, system, machine, message) -> None: _host(monkeypatch, system, machine) - with pytest.raises(UnsupportedPlatformError, match="supported platforms: linux-amd64, linux-arm64"): + with pytest.raises(UnsupportedPlatformError, match=message): Platform.current() -@pytest.mark.real_host_os -@pytest.mark.parametrize("locks,selected", [ - ((), "linux-arm64"), - (("linux-amd64",), "linux-amd64"), - (("linux-arm64",), "linux-arm64"), - (("linux-amd64", "linux-arm64"), "linux-arm64"), -]) -def test_apple_silicon_runs_a_project_as_the_platform_it_is_locked_for( - tmp_path: Path, monkeypatch, locks, selected, -) -> None: - # The human's ruling of 2026-11-04: natively when the project has an - # arm64 lock, else its amd64 lock under emulation; a new project native. - _host(monkeypatch, "Darwin", "arm64") - (tmp_path / ".devcapsule").mkdir() - for platform in locks: - (tmp_path / ".devcapsule" / f"devcapsule.{platform}.lock").write_text("") - assert Platform.select_for_project(tmp_path) is Platform.parse(selected) - assert Platform.current() is Platform.parse(selected) - Platform.clear_selection() - assert Platform.current() is Platform.LINUX_ARM64 - - -@pytest.mark.real_host_os -def test_a_host_selects_only_a_platform_it_runs(tmp_path: Path, monkeypatch) -> None: - # An amd64 Linux host cannot run an arm64-only project; it keeps its own - # platform, and the missing lock is reported where the lock is read. - _host(monkeypatch, "Linux", "x86_64") - (tmp_path / ".devcapsule").mkdir() - (tmp_path / ".devcapsule" / "devcapsule.linux-arm64.lock").write_text("") - assert Platform.select_for_project(tmp_path) is Platform.LINUX_AMD64 - assert Platform.select_for_project(None) is Platform.LINUX_AMD64 +def test_an_elf_machine_names_its_platform() -> None: + assert Platform.of_elf_machine(0x3E) is Platform.LINUX_AMD64 + assert Platform.of_elf_machine(0xB7) is Platform.LINUX_ARM64 + assert Platform.of_elf_machine(0xF3) is None + + +def test_docker_desktop_facts_are_the_macos_hosts() -> None: + assert HostOs.DARWIN.docker_socket_gid == 0 and HostOs.LINUX.docker_socket_gid is None + assert not HostOs.DARWIN.host_network_reachable and HostOs.LINUX.host_network_reachable + assert not HostOs.DARWIN.binds_unix_sockets and HostOs.LINUX.binds_unix_sockets + assert not HostOs.DARWIN.chown_reaches_bind_mounts and HostOs.LINUX.chown_reaches_bind_mounts def test_docker_spelling_and_architecture() -> None: @@ -116,7 +99,7 @@ def test_docker_spelling_and_architecture() -> None: assert Platform.LINUX_ARM64.architecture == "arm64" -@pytest.mark.real_host_os +@pytest.mark.own_host_patches def test_host_os_names_linux_and_macos_only(monkeypatch) -> None: _host(monkeypatch, "Darwin", "arm64") assert HostOs.current() is HostOs.DARWIN diff --git a/devcapsule-src/tests/test_project_platform_selection.py b/devcapsule-src/tests/test_project_platform_selection.py index 3c48e5dd..c1b81101 100644 --- a/devcapsule-src/tests/test_project_platform_selection.py +++ b/devcapsule-src/tests/test_project_platform_selection.py @@ -1,4 +1,4 @@ -"""The capsule platform a project command runs as, chosen by the project's locks or by --platform.""" +"""The capsule platform a project runs as: its locks decide, or --platform names one.""" from __future__ import annotations @@ -6,27 +6,54 @@ import pytest -from devcapsule import cli +from devcapsule import cli, platforms +from devcapsule.configuration.storage import project_platform from devcapsule.platforms import Platform -def _apple_silicon(monkeypatch: pytest.MonkeyPatch) -> None: - monkeypatch.setattr(Platform, "runnable", classmethod(lambda cls: (Platform.LINUX_ARM64, Platform.LINUX_AMD64))) +def _host(monkeypatch: pytest.MonkeyPatch, system: str, machine: str) -> None: + monkeypatch.setattr(platforms.host_platform, "system", lambda: system) + monkeypatch.setattr(platforms.host_platform, "machine", lambda: machine) -def _project(root: Path, *platforms: str) -> Path: +def _project(root: Path, *locked: str) -> Path: (root / ".devcapsule").mkdir(parents=True) (root / ".devcapsule" / "devcapsule.toml").write_text("") - for platform in platforms: + for platform in locked: (root / ".devcapsule" / f"devcapsule.{platform}.lock").write_text("") return root -def _selected_by(arguments: list[str], monkeypatch: pytest.MonkeyPatch) -> Platform: +@pytest.mark.own_host_patches +@pytest.mark.parametrize("locked,expected", [ + ((), "linux-amd64"), + (("linux-amd64",), "linux-amd64"), + (("linux-arm64",), "linux-arm64"), + (("linux-amd64", "linux-arm64"), "linux-arm64"), +]) +def test_apple_silicon_runs_a_project_as_the_platform_it_is_locked_for( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, locked: tuple[str, ...], expected: str, +) -> None: + # The human's ruling of 2026-11-04: natively when the project has an + # arm64 lock, else its amd64 lock under emulation; a new project amd64. + _host(monkeypatch, "Darwin", "arm64") + assert project_platform(_project(tmp_path / "project", *locked)) is Platform.parse(expected) + + +@pytest.mark.own_host_patches +def test_a_host_runs_a_project_only_as_a_platform_it_runs(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + # An amd64 Linux host cannot run an arm64-only project; it keeps its own + # platform, and the missing lock is reported where the lock is read. + _host(monkeypatch, "Linux", "x86_64") + assert project_platform(_project(tmp_path / "project", "linux-arm64")) is Platform.LINUX_AMD64 + assert project_platform(None) is Platform.LINUX_AMD64 + + +def _platform_seen_by(arguments: list[str], monkeypatch: pytest.MonkeyPatch) -> Platform: seen: list[Platform] = [] def record(cls: object, arguments: object, context: object) -> int: - seen.append(Platform.current()) + seen.append(project_platform(Path(str(getattr(context, "selected_path"))))) return 0 monkeypatch.setattr("devcapsule.commands.project.ProjectInfoCommand.run", classmethod(record)) @@ -34,26 +61,18 @@ def record(cls: object, arguments: object, context: object) -> int: return seen[0] -def test_a_project_command_runs_as_the_platform_the_project_is_locked_for( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch, -) -> None: - _apple_silicon(monkeypatch) - amd64_only = _project(tmp_path / "amd64", "linux-amd64") - both = _project(tmp_path / "both", "linux-amd64", "linux-arm64") - assert _selected_by(["project", "--path", str(amd64_only), "info"], monkeypatch) is Platform.LINUX_AMD64 - assert _selected_by(["project", "--path", str(both), "info"], monkeypatch) is Platform.LINUX_ARM64 - - +@pytest.mark.own_host_patches def test_platform_option_names_the_platform_for_one_command(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: # `project --platform linux-arm64 init` is how an amd64 project gains an # arm64 lock; `--platform linux-amd64` runs it under emulation on purpose. - _apple_silicon(monkeypatch) + _host(monkeypatch, "Darwin", "arm64") amd64_only = _project(tmp_path / "amd64", "linux-amd64") both = _project(tmp_path / "both", "linux-amd64", "linux-arm64") - assert _selected_by(["project", "--platform", "linux-arm64", "--path", str(amd64_only), "info"], - monkeypatch) is Platform.LINUX_ARM64 - assert _selected_by(["project", "--platform", "linux-amd64", "--path", str(both), "info"], - monkeypatch) is Platform.LINUX_AMD64 + assert _platform_seen_by(["project", "--path", str(both), "info"], monkeypatch) is Platform.LINUX_ARM64 + assert _platform_seen_by(["project", "--platform", "linux-arm64", "--path", str(amd64_only), "info"], + monkeypatch) is Platform.LINUX_ARM64 + assert _platform_seen_by(["project", "--platform", "linux-amd64", "--path", str(both), "info"], + monkeypatch) is Platform.LINUX_AMD64 def test_platform_option_refuses_a_platform_this_host_cannot_run( diff --git a/devcapsule-src/tests/test_pycharm.py b/devcapsule-src/tests/test_pycharm.py index 0ae91528..dbefa42a 100644 --- a/devcapsule-src/tests/test_pycharm.py +++ b/devcapsule-src/tests/test_pycharm.py @@ -19,7 +19,7 @@ build_run_config, ) from devcapsule.container_runtime.contract import INFRASTRUCTURE_FAILURE_EXIT_CODE -from devcapsule.platforms import HostOs +from devcapsule.platforms import HostOs, Platform from devcapsule.launch.pycharm._launcher import ( AttachedResult, display_disclosure, @@ -747,7 +747,7 @@ def test_sudoers_ownership_helper_is_constrained(tmp_path: Path) -> None: ) config = cast( PycharmRunConfig, - SimpleNamespace(enable_sudo=True, image="devcapsule-local-pycharm:canonical"), + SimpleNamespace(enable_sudo=True, image="devcapsule-local-pycharm:canonical", capsule_platform=Platform.LINUX_AMD64), ) with ( @@ -781,7 +781,7 @@ def test_sudoers_ownership_helper_failure_is_actionable(tmp_path: Path) -> None: ) config = cast( PycharmRunConfig, - SimpleNamespace(enable_sudo=True, image="devcapsule-local-pycharm:canonical"), + SimpleNamespace(enable_sudo=True, image="devcapsule-local-pycharm:canonical", capsule_platform=Platform.LINUX_AMD64), ) with ( @@ -804,7 +804,8 @@ def _macos_sudo_files(tmp_path: Path) -> tuple[PycharmRunConfig, TempRuntimeFile ) config = cast( PycharmRunConfig, - SimpleNamespace(enable_sudo=True, image="devcapsule-local-pycharm:canonical", name="capsule"), + SimpleNamespace(enable_sudo=True, image="devcapsule-local-pycharm:canonical", name="capsule", + capsule_platform=Platform.LINUX_AMD64), ) return config, files @@ -815,7 +816,7 @@ def test_macos_sudo_policy_is_written_into_a_volume_by_root( # Docker Desktop shares Mac files as the Mac user: no chown makes a # bind-mounted policy root's, so the policy lives in a volume. monkeypatch.setattr(HostOs, "current", classmethod(lambda cls: HostOs.DARWIN)) - monkeypatch.setattr("devcapsule.platforms.host_architecture", lambda: "arm64") + monkeypatch.setattr("devcapsule.host_daemon.daemon_architecture", lambda: "arm64") config, files = _macos_sudo_files(tmp_path) with ( @@ -827,7 +828,11 @@ def test_macos_sudo_policy_is_written_into_a_volume_by_root( chown.assert_not_called() assert files.sudoers_volume == "capsule-sudoers" - create, write = (call.args[0] for call in run.call_args_list) + prune, create, write = (call.args[0] for call in run.call_args_list) + # Volumes earlier launches left behind, killed or interrupted, go first; + # prune removes only those no container uses. + assert prune == ["docker", "volume", "prune", "--all", "--force", "--filter", + "label=devcapsule.purpose=development-sudo-policy"] assert create == ["docker", "volume", "create", "--label", "devcapsule.purpose=development-sudo-policy", "capsule-sudoers"] assert write[:4] == ["docker", "run", "--rm", "-i"] @@ -840,7 +845,7 @@ def test_macos_sudo_policy_is_written_into_a_volume_by_root( assert "type=volume,src=capsule-sudoers,dst=/policy" in write assert write[-1].startswith("umask 0337;") assert write[-1].endswith("/policy/policy") - assert run.call_args_list[1].kwargs["input"] == SUDOERS_POLICY + assert run.call_args_list[2].kwargs["input"] == SUDOERS_POLICY def test_macos_sudo_policy_failure_removes_its_volume( @@ -851,6 +856,7 @@ def test_macos_sudo_policy_failure_removes_its_volume( with patch("devcapsule.launch.pycharm._launcher.subprocess.run") as run: run.side_effect = [ + SimpleNamespace(returncode=0, stderr=""), SimpleNamespace(returncode=0, stderr=""), SimpleNamespace(returncode=125, stderr="no such image"), SimpleNamespace(returncode=0, stderr=""), diff --git a/devcapsule-src/tests/test_recursive_orchestrator.py b/devcapsule-src/tests/test_recursive_orchestrator.py index faa90a60..f6bea71e 100644 --- a/devcapsule-src/tests/test_recursive_orchestrator.py +++ b/devcapsule-src/tests/test_recursive_orchestrator.py @@ -2,6 +2,7 @@ from dataclasses import dataclass from pathlib import Path +import sys import socket import pytest @@ -25,10 +26,9 @@ pytestmark = pytest.mark.legacy # The recursive dogfood validates the host's X11 socket directory even for a -# contained display; a macOS host has none, and the flow is Linux-only. -requires_host_x11 = pytest.mark.skipif( - not Path("/tmp/.X11-unix").is_dir(), reason="recursive dogfood needs a host X11 socket directory" -) +# contained display. The flow is Linux-only; a Linux runner without one fails +# these tests rather than skipping them. +linux_only = pytest.mark.skipif(sys.platform != "linux", reason="the recursive dogfood flow is Linux-only") @dataclass @@ -158,7 +158,7 @@ def test_normal_project_launch_derives_only_a_downgradable_readiness_marker( assert unauthorized == {RECURSIVE_E2E_ENABLED_ENV: "0"} -@requires_host_x11 +@linux_only def test_public_dry_run_is_unique_redacted_and_cleans_owned_staging( recursive_fixture: RecursiveFixture, ) -> None: @@ -187,7 +187,7 @@ def test_public_dry_run_is_unique_redacted_and_cleans_owned_staging( assert not (workspace / second.run_id).exists() -@requires_host_x11 +@linux_only def test_public_dry_run_cleans_after_preparation_failure( recursive_fixture: RecursiveFixture, ) -> None: @@ -211,7 +211,7 @@ def test_public_dry_run_cleans_after_preparation_failure( assert not run_root.exists() -@requires_host_x11 +@linux_only def test_keep_on_failure_preserves_only_the_owned_run( recursive_fixture: RecursiveFixture, ) -> None: @@ -237,7 +237,7 @@ def test_keep_on_failure_preserves_only_the_owned_run( assert run_root.is_dir() -@requires_host_x11 +@linux_only def test_contained_display_dry_run_forwards_no_host_x11( recursive_fixture: RecursiveFixture, tmp_path: Path ) -> None: diff --git a/devcapsule-src/tests/test_recursive_successor_plan.py b/devcapsule-src/tests/test_recursive_successor_plan.py index 9e2d0ac3..7860bc5c 100644 --- a/devcapsule-src/tests/test_recursive_successor_plan.py +++ b/devcapsule-src/tests/test_recursive_successor_plan.py @@ -566,3 +566,17 @@ def test_duplicate_mount_destination_is_rejected() -> None: inspection["Mounts"].append(dict(inspection["Mounts"][0])) assert "declared more than once" in rejected(inspection) + + +@pytest.mark.parametrize("spelling", [["--platform=linux/amd64"], ["--platform", "linux/amd64"]]) +def test_an_emulated_launch_plans_its_platform_and_the_successor_must_run_it(spelling: list[str]) -> None: + # The launcher names the platform when the daemon emulates it; both of + # Docker's spellings parse, and a successor that ran another is refused. + plan = build_plan([*docker_args(), *spelling]) + assert plan.platform == "linux/amd64" + inspection = matching_inspection(plan) + inspection["ImageManifestDescriptor"] = {"platform": {"os": "linux", "architecture": "amd64"}} + compare_inspection(plan, inspection) + inspection["ImageManifestDescriptor"] = {"platform": {"os": "linux", "architecture": "arm64"}} + with pytest.raises(SuccessorPlanError, match="successor platform does not match the planned --platform"): + compare_inspection(plan, inspection) diff --git a/devcapsule-src/tests/test_runtime_artifact.py b/devcapsule-src/tests/test_runtime_artifact.py index fc7b85d9..31191c0c 100644 --- a/devcapsule-src/tests/test_runtime_artifact.py +++ b/devcapsule-src/tests/test_runtime_artifact.py @@ -8,6 +8,8 @@ from devcapsule.compat import CliError from devcapsule.materialization import ArtifactSpec, formation_descriptor, formation_identity from devcapsule import platforms +from devcapsule.build_info import BuildInfo +from devcapsule.platforms import HostOs, Platform from devcapsule.runtime_artifact import executable_platform, runtime_artifact from devcapsule.runtime_command import RuntimeCommand from tests.test_base_image import pex_fixture @@ -15,71 +17,130 @@ pytestmark = pytest.mark.legacy -def test_packaged_launcher_supplies_itself_despite_source_override(tmp_path: Path, monkeypatch) -> None: - launcher = pex_fixture(tmp_path / "launcher.pex") - monkeypatch.setenv("SCIE", str(launcher)) - monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(tmp_path / "different.pex")) - assert runtime_artifact() == launcher - - -def test_source_launcher_requires_explicit_built_runtime(tmp_path: Path, monkeypatch) -> None: - monkeypatch.delenv("SCIE", raising=False) - monkeypatch.delenv("PEX", raising=False) - monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(tmp_path / "missing.pex")) - with pytest.raises(CliError, match="DEVCAPSULE_RUNTIME_PEX"): - runtime_artifact() - launcher = pex_fixture(tmp_path / "runtime.pex") - monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(launcher)) - assert runtime_artifact() == launcher - - +AMD64 = Platform.LINUX_AMD64 +ARM64 = Platform.LINUX_ARM64 MACH_O_ARM64 = b"\xcf\xfa\xed\xfe" + bytes(16) ELF_AMD64 = b"\x7fELF\x02\x01\x01" + bytes(11) + (0x3E).to_bytes(2, "little") ELF_ARM64 = b"\x7fELF\x02\x01\x01" + bytes(11) + (0xB7).to_bytes(2, "little") +ELF_RISCV = b"\x7fELF\x02\x01\x01" + bytes(11) + (0xF3).to_bytes(2, "little") -def native(path: Path, header: bytes) -> Path: +def pex_with_native_header(path: Path, header: bytes, *, revision: str = "a" * 40) -> Path: """A PEX fixture whose first bytes are a native executable's header. A scie is a native launcher with the PEX appended; the build information is read from the archive, so the header and the archive coexist. """ - pex = pex_fixture(path) + pex = pex_fixture(path, revision=revision) pex.write_bytes(header + pex.read_bytes()) return pex -def _apple_silicon(monkeypatch) -> None: +@pytest.fixture(autouse=True) +def launcher_of_the_fixtures_build(monkeypatch: pytest.MonkeyPatch) -> None: + """The running launcher is the build the fixtures record: 0.1.0 at revision a…a.""" + + monkeypatch.setattr("devcapsule.runtime_artifact.current_build_info", lambda: BuildInfo( + schema_version=2, version="0.1.0", source_repository="https://github.com/example/devcapsule", + source_revision="a" * 40, source_url="unknown")) + + +def _apple_silicon(monkeypatch: pytest.MonkeyPatch) -> None: monkeypatch.setattr(platforms.host_platform, "system", lambda: "Darwin") monkeypatch.setattr(platforms.host_platform, "machine", lambda: "arm64") +def test_packaged_launcher_supplies_itself_despite_source_override(tmp_path: Path, monkeypatch) -> None: + launcher = pex_fixture(tmp_path / "launcher.pex") + monkeypatch.setenv("SCIE", str(launcher)) + monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(tmp_path / "different.pex")) + assert runtime_artifact(AMD64) == launcher + + +def test_source_launcher_requires_explicit_built_runtime(tmp_path: Path, monkeypatch) -> None: + monkeypatch.delenv("SCIE", raising=False) + monkeypatch.delenv("PEX", raising=False) + monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(tmp_path / "missing.pex")) + with pytest.raises(CliError, match="DEVCAPSULE_RUNTIME_PEX"): + runtime_artifact(AMD64) + launcher = pex_fixture(tmp_path / "runtime.pex") + monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(launcher)) + assert runtime_artifact(AMD64) == launcher + + def test_executable_platform_reads_native_headers(tmp_path: Path) -> None: - assert executable_platform(native(tmp_path / "mac", MACH_O_ARM64)) == "darwin" - assert executable_platform(native(tmp_path / "amd64", ELF_AMD64)) == "linux-amd64" - assert executable_platform(native(tmp_path / "arm64", ELF_ARM64)) == "linux-arm64" + assert executable_platform(pex_with_native_header(tmp_path / "mac", MACH_O_ARM64)) is HostOs.DARWIN + assert executable_platform(pex_with_native_header(tmp_path / "amd64", ELF_AMD64)) is AMD64 + assert executable_platform(pex_with_native_header(tmp_path / "arm64", ELF_ARM64)) is ARM64 + assert executable_platform(pex_with_native_header(tmp_path / "riscv", ELF_RISCV)) is HostOs.LINUX assert executable_platform(pex_fixture(tmp_path / "plain.pex")) is None assert executable_platform(tmp_path / "absent") is None +@pytest.mark.own_host_patches def test_macos_launcher_delivers_the_named_linux_runtime(tmp_path: Path, monkeypatch) -> None: _apple_silicon(monkeypatch) - monkeypatch.setenv("SCIE", str(native(tmp_path / "devcapsule-macos", MACH_O_ARM64))) - linux = native(tmp_path / "devcapsule-linux", ELF_AMD64) + monkeypatch.setenv("SCIE", str(pex_with_native_header(tmp_path / "devcapsule-macos", MACH_O_ARM64))) + linux = pex_with_native_header(tmp_path / "devcapsule-linux", ELF_AMD64) monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(linux)) - assert runtime_artifact() == linux + assert runtime_artifact(AMD64) == linux +@pytest.mark.own_host_patches def test_macos_launcher_without_a_linux_runtime_says_what_to_set(tmp_path: Path, monkeypatch) -> None: _apple_silicon(monkeypatch) - monkeypatch.setenv("SCIE", str(native(tmp_path / "devcapsule-macos", MACH_O_ARM64))) + monkeypatch.setenv("SCIE", str(pex_with_native_header(tmp_path / "devcapsule-macos", MACH_O_ARM64))) monkeypatch.delenv("DEVCAPSULE_RUNTIME_PEX", raising=False) - with pytest.raises(CliError, match="darwin executable and cannot run inside linux-amd64 capsules.*DEVCAPSULE_RUNTIME_PEX"): - runtime_artifact() - monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(native(tmp_path / "wrong-arch", ELF_ARM64))) - with pytest.raises(CliError, match="linux-arm64 executable"): - runtime_artifact() + with pytest.raises(CliError, match="a macOS executable and cannot run inside linux-amd64 capsules.*DEVCAPSULE_RUNTIME_PEX"): + runtime_artifact(AMD64) + monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(pex_with_native_header(tmp_path / "wrong-arch", ELF_ARM64))) + with pytest.raises(CliError, match="a linux-arm64 executable"): + runtime_artifact(AMD64) + + +@pytest.mark.own_host_patches +def test_a_plain_pex_launcher_on_a_mac_is_not_delivered(tmp_path: Path, monkeypatch) -> None: + # A PEX bundles native wheels for the machine it was built on. + _apple_silicon(monkeypatch) + monkeypatch.delenv("SCIE", raising=False) + monkeypatch.setenv("PEX", str(pex_fixture(tmp_path / "launcher.pex"))) + linux = pex_with_native_header(tmp_path / "devcapsule-linux", ELF_AMD64) + monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(linux)) + assert runtime_artifact(AMD64) == linux + monkeypatch.delenv("DEVCAPSULE_RUNTIME_PEX") + with pytest.raises(CliError, match="a plain PEX built for another machine"): + runtime_artifact(AMD64) + + +@pytest.mark.own_host_patches +def test_a_runtime_of_another_build_is_refused(tmp_path: Path, monkeypatch) -> None: + # D-0009: the runtime runs the plan this launcher writes. A runtime left + # from an earlier build, after the launcher was updated, is refused. + _apple_silicon(monkeypatch) + monkeypatch.setenv("SCIE", str(pex_with_native_header(tmp_path / "devcapsule-macos", MACH_O_ARM64))) + stale = pex_with_native_header(tmp_path / "devcapsule-linux", ELF_AMD64, revision="b" * 40) + monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(stale)) + with pytest.raises(CliError, match="is DevCapsule 0.1.0 from b+, but this launcher is 0.1.0 from a+"): + runtime_artifact(AMD64) + + +@pytest.mark.own_host_patches +def test_a_runtime_directory_serves_each_capsule_platform(tmp_path: Path, monkeypatch) -> None: + # Apple Silicon runs both platforms; one setting names both runtimes, and + # each launch delivers the one for its lock's platform. + _apple_silicon(monkeypatch) + monkeypatch.setenv("SCIE", str(pex_with_native_header(tmp_path / "devcapsule-macos", MACH_O_ARM64))) + runtimes = tmp_path / "runtimes" + runtimes.mkdir() + amd64 = pex_with_native_header(runtimes / "devcapsule-linux-amd64", ELF_AMD64) + arm64 = pex_with_native_header(runtimes / "devcapsule-linux-arm64", ELF_ARM64) + monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(runtimes)) + assert runtime_artifact(ARM64) == arm64 + assert runtime_artifact(AMD64) == amd64 + amd64.unlink() + with pytest.raises(CliError, match="holds no devcapsule-linux-amd64"): + runtime_artifact(AMD64) def test_runtime_update_changes_formation_identity_without_changing_components() -> None: @@ -147,25 +208,3 @@ def test_development_mode_removes_only_its_own_inherited_link(tmp_path, monkeypa else: assert command.read_text() == "development command" assert command.is_symlink() == (existing == "development-link") - - -@pytest.mark.real_host_os -def test_a_runtime_directory_serves_each_capsule_platform(tmp_path: Path, monkeypatch) -> None: - # Apple Silicon runs both platforms; one setting names both runtimes, and - # the project's locks choose which one a launch delivers. - _apple_silicon(monkeypatch) - monkeypatch.setenv("SCIE", str(native(tmp_path / "devcapsule-macos", MACH_O_ARM64))) - runtimes = tmp_path / "runtimes" - runtimes.mkdir() - amd64 = native(runtimes / "devcapsule-linux-amd64", ELF_AMD64) - arm64 = native(runtimes / "devcapsule-linux-arm64", ELF_ARM64) - monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(runtimes)) - assert runtime_artifact() == arm64 - project = tmp_path / "project" - (project / ".devcapsule").mkdir(parents=True) - (project / ".devcapsule" / "devcapsule.linux-amd64.lock").write_text("") - platforms.Platform.select_for_project(project) - assert runtime_artifact() == amd64 - amd64.unlink() - with pytest.raises(CliError, match="directory holding it as devcapsule-linux-amd64"): - runtime_artifact() diff --git a/devcapsule-src/tests/test_version_sets.py b/devcapsule-src/tests/test_version_sets.py index 16b1a733..c0723c0b 100644 --- a/devcapsule-src/tests/test_version_sets.py +++ b/devcapsule-src/tests/test_version_sets.py @@ -152,8 +152,8 @@ def open_payload(url, **kwargs): selected = ExecutionConfiguration.load(root).project runtime = tmp_path / "runtime.pex" runtime.write_bytes(b"fixture-runtime") - monkeypatch.setattr("devcapsule.environment_realization.runtime_artifact", lambda: runtime) - monkeypatch.setattr("devcapsule.runtime_artifact.runtime_artifact", lambda: runtime) + monkeypatch.setattr("devcapsule.environment_realization.runtime_artifact", lambda _platform: runtime) + monkeypatch.setattr("devcapsule.runtime_artifact.runtime_artifact", lambda _platform: runtime) monkeypatch.setattr("devcapsule.materialization.read_pex_build_info", lambda _: SimpleNamespace( build_mnemonic="fixture", source_repository="fixture", source_revision="fixture", source_url="fixture")) base = ImageDetails(lock["base"]["reference"], "sha256:" + "a" * 64, { @@ -163,7 +163,7 @@ def open_payload(url, **kwargs): state = SimpleNamespace(fail_build=False, exit_code=0, during_run=None, launched=[], built=[], images=images, root=root, record=selected.checkout_path, resolution=selected.resolution_path, component=component, lock=path, original=path.read_bytes(), metadata=metadata, base=base) - def require(reference): + def require(reference, platform=None): if reference not in images: raise CliError("local image missing") return images[reference] diff --git a/docs/platforms/macos.md b/docs/platforms/macos.md index 9483dd59..3632dd64 100644 --- a/docs/platforms/macos.md +++ b/docs/platforms/macos.md @@ -1,7 +1,7 @@ --- -description: DevCapsule ships no macOS executable; an experimental launcher built from source runs on Apple Silicon Macs against Docker Desktop. What works, what differs, and how to build it. +description: DevCapsule ships no macOS executable; an experimental launcher built from source runs on Apple Silicon Macs against Docker Desktop, with capsules under emulation or, per project, natively as linux-arm64. What works, what differs, and how to build it. weight: 3 -updated: 2026-11-03 +updated: 2026-11-04 --- # macOS @@ -10,20 +10,30 @@ DevCapsule {{tag}} ships no macOS executable. The release executable is for The launcher in the source tree runs natively on **Apple Silicon Macs** (`darwin-arm64`) against **Docker Desktop**, built by you. This is -experimental and not a supported platform in {{tag}}. Capsules stay the same -`linux-amd64` environments Linux users get, run by Docker Desktop under x86-64 -emulation. Intel Macs are untested. +experimental and not a supported platform in {{tag}}. Intel Macs are untested. + +A project runs as one of two capsule platforms: + +- **`linux-amd64`, the default.** The same environments Linux users get, run by + Docker Desktop under x86-64 emulation. Every new project starts here. +- **`linux-arm64`, natively, per project.** Faster, but its base image is not + published yet: you build it on your Mac and select it for each checkout. + +A project runs as `linux-arm64` when it has a `linux-arm64` lock, and +otherwise as `linux-amd64`. `devcapsule project --platform PLATFORM ...` names +one for a single command. ## What differs from Linux -- **Two executables.** On Linux the launcher copies itself into the capsule. A - macOS executable cannot run there, so you build a Linux runtime beside it and - name it in `DEVCAPSULE_RUNTIME_PEX`. The launcher delivers that file instead, - and stops with that remedy when the variable is unset. Build both from the - same commit. -- **Emulation.** The first launch of an IDE builds its image under emulation: - about ten minutes for PyCharm on a 14-core Mac. Later launches reuse the - image and start in seconds. The IDE itself runs slower than on a native host. +- **Two executables, or three.** On Linux the launcher copies itself into the + capsule. A macOS executable cannot run there, so you build the Linux runtime + for each capsule platform you use and put them in one directory that + `DEVCAPSULE_RUNTIME_PEX` names. Each launch delivers the one for its + project's platform and refuses a runtime from another build, so rebuild + them whenever you rebuild the launcher. +- **Emulation.** Under `linux-amd64`, the first launch of an IDE builds its + image under emulation: about ten minutes for PyCharm on a 14-core Mac. Later + launches reuse the image. The IDE runs slower than on a native host. - **Networking.** Docker Desktop's host network belongs to its virtual machine, out of your Mac's reach. A project authorized for `network host` launches on bridge networking instead, with a warning. The contained desktop and the @@ -33,9 +43,9 @@ emulation. Intel Macs are untested. container, so links the IDE opens cannot reach your Mac's browser. The launch warns and continues. The contained desktop still opens in your browser. - **Docker inside the capsule.** With `docker-daemon` authorized, the capsule's - Docker client reaches Docker Desktop's daemon, whose engine is `arm64`: an - image you run from inside the capsule is the ARM variant unless you pass - `--platform linux/amd64`. + Docker client reaches Docker Desktop's daemon, whose engine is `arm64` on + either capsule platform: an image you run from inside the capsule is the ARM + variant unless you pass `--platform linux/amd64`. - **File sharing.** Keep projects under a directory Docker Desktop shares, by default `/Users`, `/Volumes`, `/private`, `/tmp` and `/var/folders`. Files the capsule writes there belong to your user. @@ -43,7 +53,8 @@ emulation. Intel Macs are untested. ## Build and install You need Docker Desktop with several gigabytes free under **Settings → -Resources → Disk usage limit**, Git, and Python 3.12. From a clean checkout: +Resources → Disk usage limit**, and about as much free on the Mac itself, plus +Git and Python 3.12. From a clean checkout of a published commit: ```bash git clone https://github.com/ccozianu/devcapsule.git @@ -51,30 +62,62 @@ cd devcapsule/devcapsule-src python3.12 -m venv .venv .venv/bin/pip install -r dev-requirements.txt .venv/bin/pip install -e . --no-deps +revision="$(git rev-parse HEAD)" DEVCAPSULE_SCIE_PLATFORM=macos-aarch64 PYTHON=.venv/bin/python \ - scripts/build-pex.sh --allow-local-source --output dist/devcapsule-macos-arm64 -git -C .. archive HEAD | docker run --rm -i --platform linux/amd64 \ - -v "$PWD/dist:/out" python:3.12-slim sh -c ' - mkdir /build && tar -x -C /build && cd /build/devcapsule-src && - python -m venv /venv && /venv/bin/pip install -q -r dev-requirements.txt && - /venv/bin/pip install -q -e . --no-deps && - PYTHON=/venv/bin/python scripts/build-pex.sh --allow-local-source \ - --output /out/devcapsule-linux-amd64' + scripts/build-pex.sh --source-revision "$revision" --allow-unpublished-revision \ + --output dist/devcapsule-macos-arm64 +for target in amd64:linux-x86_64 arm64:linux-aarch64; do + git -C .. archive HEAD | docker run --rm -i --platform "linux/${target%%:*}" \ + -v "$PWD/dist:/out" python:3.12-slim sh -c " + mkdir /build && tar -x -C /build && cd /build/devcapsule-src && + python -m venv /venv && /venv/bin/pip install -q -r dev-requirements.txt && + /venv/bin/pip install -q -e . --no-deps && + DEVCAPSULE_SCIE_PLATFORM=${target#*:} PYTHON=/venv/bin/python scripts/build-pex.sh \ + --allow-local-source --output /out/devcapsule-linux-${target%%:*}" +done ``` -The first build writes the macOS launcher. The second builds the Linux runtime -from the same commit inside an emulated Linux container. Install both: +This writes the macOS launcher and a Linux runtime for each capsule platform, +all from the same commit. `--allow-unpublished-revision` skips only the check +that the commit is a branch tip; the commit itself must be public, since a base +build installs the web console from it. Install them: ```bash mkdir -p ~/.local/bin ~/.local/lib/devcapsule install -m 0755 dist/devcapsule-macos-arm64 ~/.local/bin/devcapsule -install -m 0755 dist/devcapsule-linux-amd64 ~/.local/lib/devcapsule/ -export DEVCAPSULE_RUNTIME_PEX="$HOME/.local/lib/devcapsule/devcapsule-linux-amd64" +install -m 0755 dist/devcapsule-linux-amd64 dist/devcapsule-linux-arm64 ~/.local/lib/devcapsule/ +export DEVCAPSULE_RUNTIME_PEX="$HOME/.local/lib/devcapsule" ~/.local/bin/devcapsule version ``` Add the `export` line, and `$HOME/.local/bin` on your `PATH`, to your shell -profile. Then continue with [your first session](../getting-started/first-session.md). +profile. Then continue with [your first session](../getting-started/first-session.md); +its projects run as `linux-amd64`. + +## Run a project natively as linux-arm64 + +Build the arm64 base once, natively, with the launcher you installed and the +revision it was built from (`devcapsule version` shows it): + +```bash +devcapsule images build --type base --recipe ubuntu-24.04 --platform linux-arm64 \ + --tag devcapsule-base-local:arm64 --source-revision "$revision" +``` + +Then, in each project, add the arm64 lock and select your local base for the +checkout: + +```bash +devcapsule project --platform linux-arm64 init --unverified --authorize base-image default +devcapsule project config authorize base-image devcapsule-base-local:arm64 +devcapsule project run +``` + +`--unverified` is needed while no arm64 combination of IDE and base has +recorded validation evidence; init says so. The arm64 lock names a base that is +not published yet, so do not commit it to a shared repository: anyone without +a local arm64 base cannot run it. To run the project under emulation again for +one launch, use `devcapsule project --platform linux-amd64 run`. If something fails on your Mac, please [open an issue](https://github.com/ccozianu/devcapsule/issues/new) with your macOS and Docker Desktop versions and the error output, omitting diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 09d04825..ed13c40a 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -13,9 +13,10 @@ checks even when given `--help`. To target another directory, pass `--path` before the subcommand: `devcapsule project --path PROJECT config show`. On a host that runs two capsule platforms, an Apple Silicon Mac, a project runs as the first one it has a lock -for, `linux-arm64` before `linux-amd64`; `--platform PLATFORM` before the -subcommand names one for that command, so `devcapsule project --platform -linux-arm64 init` adds an arm64 lock to a project. Inside a capsule, inspection +for, `linux-arm64` before `linux-amd64`, and a new project as `linux-amd64`; +`--platform PLATFORM` before the subcommand names one for that command, so +`devcapsule project --platform linux-arm64 init` adds an arm64 lock to a +project. `devcapsule images build` takes `--platform` too. Inside a capsule, inspection commands distinguish captured launch data from next-launch choices where available. Commands that need launcher-owned state print the exact `devcapsule` command to run outside. diff --git a/engineering-docs/requirements/devcapsule/r-runtime-001-launcher-runtime-identity.md b/engineering-docs/requirements/devcapsule/r-runtime-001-launcher-runtime-identity.md index bbb111cc..8e6fd616 100644 --- a/engineering-docs/requirements/devcapsule/r-runtime-001-launcher-runtime-identity.md +++ b/engineering-docs/requirements/devcapsule/r-runtime-001-launcher-runtime-identity.md @@ -50,10 +50,21 @@ dependency cycle and keeps runtime-plan production and execution in one build. - Source-form launchers can explicitly select a built PEX using `DEVCAPSULE_RUNTIME_PEX`; running that PEX directly gives the ordinary identical-executable path. Source Python files are not themselves a PEX. -- The packaged launcher currently takes precedence over - `DEVCAPSULE_RUNTIME_PEX`. That variable is a source-development selection, - not a general packaged-launcher runtime override. No new override interface - is implemented by recording this decision. +- A packaged launcher that runs on the capsule platform takes precedence + over `DEVCAPSULE_RUNTIME_PEX`. That variable is otherwise a source-development + selection, not a general packaged-launcher runtime override. +- Since the `linux-arm64` workstream (2026-11-04, unreleased): a packaged + launcher that cannot run inside the capsule, a macOS executable launching + Linux capsules, delivers the runtime `DEVCAPSULE_RUNTIME_PEX` names instead, + and stops with that remedy when it is unset. The variable may name a + directory holding one runtime per capsule platform as + `devcapsule-`; the lock's platform selects. This is the explicit + user choice D-0009 admits, not a new default: the outside and inside bytes + differ, so the delivered runtime must be the launcher's build, the same + version, and the same source revision wherever both record one. A local + build without a recorded revision can be matched by version only. A + release shipping macOS launchers would reopen D-0009, whose *Reopen If* + names this case. ## Verification @@ -74,3 +85,9 @@ These fixture-image checks do not stand in for actual IDE-terminal acceptance. Future changes must preserve these properties. Any additional override interface needs its own explicit-selection and default-regression coverage; this record does not claim that interface has been validated. + +The macOS delivery path is covered by `tests/test_runtime_artifact.py`: the +named runtime and the directory form, refusal of a foreign or missing runtime, +of a plain PEX on a non-Linux host, and of a runtime from another build. The +IDE smoke compares the capsule's runtime digest with the runtime the workspace's +lock selects. Neither is a release validation of macOS launchers. diff --git a/engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md b/engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md index 36f10f50..697ebf19 100644 --- a/engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md +++ b/engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md @@ -31,17 +31,22 @@ the base recipe, on top of `macos-launcher`'s platform split. File references ar ## 3. Design -1. **`Platform.LINUX_ARM64`.** The capsule platform a host can run, in preference order: - `linux-amd64` hosts run amd64; `linux-arm64` hosts run arm64; Apple Silicon prefers arm64 - and can emulate amd64; Intel Macs run amd64. A project's platform is the first preferred - platform it has a lock for; a project with no lock yet takes the first. Where callers now - ask `Platform.current()` without a project, the design keeps one answer per process, - chosen when the project is known; slice 1 settles the mechanism after reading the callers. +1. **`Platform.LINUX_ARM64`.** Host detection stays in `platforms.py`: `runnable()`, every + platform a host runs, native first (Apple Silicon: arm64, then amd64 under emulation), + and `current()`, the platform a host creates a new project for, `linux-amd64` on a Mac + while the arm64 base is unpublished, per the ruling in section 2. Which platform a + project runs as is the configuration's question: `configuration.storage.project_platform`, + read-only, answers the first runnable platform the project has a lock for, else + `current()`; `project --platform` and `images build --platform` name one for a command. + Below the configuration, the platform is the lock's, passed down explicitly to the base + pull, the image build, the runtime choice and the launch (D-0006's consequence). + Emulation is judged against the Docker daemon's architecture, not the launcher's CPU. 2. **The runtime.** An arm64 capsule needs a `linux-aarch64` executable. A launcher whose own binary runs on the capsule platform delivers itself (R-RUNTIME-001). Otherwise the user - names the runtime: `DEVCAPSULE_RUNTIME_PEX` keeps naming one file, and a directory holding - `devcapsule-linux-amd64` and `devcapsule-linux-arm64` serves both platforms from one - setting. + names the runtime: `DEVCAPSULE_RUNTIME_PEX` names one file, or a directory holding + `devcapsule-linux-amd64` and `devcapsule-linux-arm64`. A runtime that is not the + launcher's own bytes must be its build: the same version, and the same revision where + both record one. 3. **The base.** Built natively on this Mac by `devcapsule images build --type base`, and approved per checkout by the documented local-base path. Whether the `linux-arm64` matrix can resolve without a published base pin is slice 2's first question. @@ -62,7 +67,23 @@ the base recipe, on top of `macos-launcher`'s platform split. File references ar 4. **Live and documentation.** `nox -s ide-smoke` for every surface natively on this Mac; the Linux, macOS and install pages. -## 5. What this does not decide +## 5. Review, 2026-11-04 + +The human's `/review` of all changes since `main`, run by a separate agent, rejected the +branch with seven high findings: a Mac's new projects defaulting to the unpublished arm64 +base; `images build` ignoring the project's locks; the platform read from a process-wide +selection below the configuration instead of from the lock; no version check on the +delivered runtime; personal pins re-derived on their own platform; a leaking sudo-policy +volume; a hash test that skipped three components. Fourteen medium and three low findings +followed. The human said "fix all"; each is fixed as described in section 3, except that a +lock naming an unpublished base is reported, not refused, since the human's ruling has the +checkout select a local base for it. + +## 6. What this does not decide - Publishing an arm64 or multi-platform base, or arm64 release executables: the owner's. - Whether ARM Linux becomes a supported release platform: the owner's. +- D-0006 says host detection is `current()` and parsing a lock is `parse()`, with + deliberately no third way to obtain a `Platform`. `runnable()`, a host's list, and + `of_elf_machine()`, an executable's platform, are two more; amending D-0006 to name them + is the owner's. diff --git a/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md index c6143bf0..f56fd1a5 100644 --- a/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md @@ -4,7 +4,7 @@ Mnemonic: `linux-arm64` Start date: 2026-11-04 -State: active; slices 1 to 3 committed 2026-11-04; native smokes in progress +State: active; slices 1 to 3 and the review's fixes committed 2026-11-04; native smokes pending Docker Branch association: `ws-linux-arm64/capsules`, forked from local `main` at b3f10b7d, with `ws-macos-launcher/host-port` merged in as its foundation @@ -66,6 +66,18 @@ Found for `maintenance`, not fixed here: the amd64 Eclipse closure's `libsoup-3. `libsoup-3.0-common` 3.4.4-5ubuntu0.8 URLs return 404, since Ubuntu removed that version from every noble pocket on 2026-10-08, so Eclipse cannot be acquired afresh on amd64. +**2026-11-04, review and fixes.** The human ran the `/review` skill of `~/work/teachme` on all +changes since `main`, through a separate agent: rejected, seven high, fourteen medium, three +low findings. The human said "fix all"; the design note's section 5 records them, and +section 3 the design they produced. In short: a Mac creates `linux-amd64` projects again and +runs a project as the first platform it has a lock for; the configuration answers which +(`storage.project_platform`), and below it the lock's platform is passed down explicitly; a +delivered runtime must be the launcher's build; the unpublished arm64 base is reported at +init and refused at launch without a local base; Docker Desktop's conventions are `HostOs` +properties; emulation is the daemon's. Unit suite on this Mac: 2,749 passed, 25 skipped. +The human's `~/hello-devcapsule` checkout holds amd64 VSCodium pins labelled arm64, written +by the bug since fixed; re-choosing its IDE through DevCapsule commands repairs it. + ## Next Resumable Task Record the rerun smokes; add arm64 verified edges for the surfaces that pass, with their From 868369bb027f33bd50d26e14dc63cac8363e40d7 Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Wed, 4 Nov 2026 13:54:20 +0200 Subject: [PATCH 16/20] Remap a personal pin that is another platform's embedded table pins_for_platform now judges each component table by what it is, not by the pins' label: a table that is exactly another platform's embedded pin for its version takes this platform's; one that matches no embedded table is a record kept as it stands on its labelled platform. This keeps the review's rule, never re-deriving same-platform records, and heals the pins an earlier build wrote with an amd64 VSCodium under a linux-arm64 label, which re-choosing the IDE could not. Co-Authored-By: Claude Opus 5.5 --- .../configuration/capability_selection.py | 33 ++++++++++++++----- .../tests/test_linux_arm64_matrix.py | 4 +++ 2 files changed, 28 insertions(+), 9 deletions(-) diff --git a/devcapsule-src/devcapsule/configuration/capability_selection.py b/devcapsule-src/devcapsule/configuration/capability_selection.py index 8d773071..8f34c072 100644 --- a/devcapsule-src/devcapsule/configuration/capability_selection.py +++ b/devcapsule-src/devcapsule/configuration/capability_selection.py @@ -112,19 +112,22 @@ def local_pins(checkout: Mapping[str, Any]) -> dict[str, Any]: def pins_for_platform(pins: Mapping[str, Any], matrix: ResolutionMatrix) -> dict[str, Any]: - """Return personal pins written for another platform, remapped to ``matrix``'s platform. + """Return personal pins with every component's artifacts for ``matrix``'s platform. A checkout keeps one set of personal pins, and a project may run as two - platforms: Apple Silicon runs linux-arm64 and linux-amd64. Pins written - for this platform are returned as they stand, records not to be - re-derived. Pins written for the other take this platform's embedded - table for each component version, so one platform's artifacts never reach - the other's capsules; a version this platform does not pin raises + platforms: Apple Silicon runs linux-arm64 and linux-amd64. A component + table that is exactly another platform's embedded table for its version + is that platform's pin, whatever the pins' label says, and takes this + platform's table for the version; one that matches no embedded table is a + record this launcher did not write, kept as it stands on the platform its + pins name. Everything else is returned unchanged. A component written for + another platform at a version this platform does not pin raises ProjectConfigurationError naming how to choose again. The input is unchanged. """ + if not pins: + return {} platform = str(matrix.platform) - if not pins or str(pins.get("platform")) == platform: - return deepcopy(dict(pins)) + labelled = str(pins.get("platform")) repinned = deepcopy(dict(pins)) components = repinned.get("components") surface = components.get("interactive-surface") if isinstance(components, dict) else None @@ -132,12 +135,15 @@ def pins_for_platform(pins: Mapping[str, Any], matrix: ResolutionMatrix) -> dict for component, metadata in components.items(): if not isinstance(metadata, dict): continue + written_for = _embedded_platform_of(component, metadata) or labelled + if written_for == platform: + continue matrix_table = matrix.component_pin(component, metadata.get("version")) if matrix_table is None: remedy = (f"'devcapsule project --platform {platform} config ide {component}'" if component == surface else f"'devcapsule project --platform {platform} config use' for the capability it provides") raise ProjectConfigurationError( - f"This checkout's {component} {metadata.get('version')} is pinned for {pins.get('platform')}, " + f"This checkout's {component} {metadata.get('version')} is pinned for {written_for}, " f"and {platform} has no pin for that version; choose it again with {remedy}." ) components[component] = deepcopy(dict(matrix_table)) @@ -146,6 +152,15 @@ def pins_for_platform(pins: Mapping[str, Any], matrix: ResolutionMatrix) -> dict return repinned +def _embedded_platform_of(component: str, table: Mapping[str, Any]) -> str | None: + """The platform whose embedded table for this component version ``table`` is, if any.""" + for platform, embedded in MATRICES.items(): + candidate = embedded.component_pin(component, table.get("version")) + if candidate is not None and dict(candidate) == dict(table): + return str(platform) + return None + + def pin_selection(matrix: ResolutionMatrix, policy: CapabilityPolicy, selection: LocalCapabilities, source: Mapping[str, Any], previous: LocalCapabilities, previous_pins: Mapping[str, Any], *, allow_unverified: bool, recommendations: Mapping[str, str] | None = None) -> dict[str, Any]: diff --git a/devcapsule-src/tests/test_linux_arm64_matrix.py b/devcapsule-src/tests/test_linux_arm64_matrix.py index f3dc8dbb..f1ceb18a 100644 --- a/devcapsule-src/tests/test_linux_arm64_matrix.py +++ b/devcapsule-src/tests/test_linux_arm64_matrix.py @@ -113,6 +113,10 @@ def test_personal_pins_take_the_artifacts_of_the_platform_they_run_on() -> None: assert on_arm64["components"]["codium"] == dict(ARM64.component_pins()["codium"]) assert on_arm64["components"]["interactive-surface"] == "codium" assert amd64["components"]["codium"] == dict(AMD64.component_pins()["codium"]) # input unchanged + # A table that is another platform's embedded pin is that platform's, + # whatever the label says: pins written wrongly by an earlier build heal. + mislabelled = dict(amd64, platform="linux-arm64") + assert pins_for_platform(mislabelled, ARM64)["components"]["codium"] == dict(ARM64.component_pins()["codium"]) # Pins written for the platform they run on are records, returned as they stand. recorded = {**amd64, "components": {**amd64["components"], "codium": {**amd64["components"]["codium"], "note": "kept"}}} assert pins_for_platform(recorded, AMD64) == recorded From 398fc3cefb81e3567abe1acb214b03a9a19d62f7 Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Wed, 4 Nov 2026 14:00:13 +0200 Subject: [PATCH 17/20] Let the IDE smoke create its project for a named platform; record the Linux runs Co-Authored-By: Claude Opus 5.5 --- devcapsule-src/tests/e2e/ide_session.py | 5 ++++- .../wip/2026-11-04-linux-arm64/CURRENT-STATUS.md | 11 +++++++++++ 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/devcapsule-src/tests/e2e/ide_session.py b/devcapsule-src/tests/e2e/ide_session.py index 5d336a76..a036a994 100644 --- a/devcapsule-src/tests/e2e/ide_session.py +++ b/devcapsule-src/tests/e2e/ide_session.py @@ -179,8 +179,11 @@ def initialize_project(executable: Path, surface: IdeSurface, workspace: Path, e """ environment = dict(os.environ, BROWSER="true") # webbrowser runs `true URL`: no tab opens built_base = os.environ.get("DEVCAPSULE_E2E_BASE_IMAGE") + # DEVCAPSULE_SMOKE_PLATFORM creates the project for that capsule platform, + # such as linux-arm64 on Apple Silicon, whose new projects are linux-amd64. + platform = os.environ.get("DEVCAPSULE_SMOKE_PLATFORM") init = subprocess.run( - [str(executable), "project", "init", + [str(executable), "project", *(["--platform", platform] if platform else []), "init", *(flag for need in surface.needs for flag in ("--ide" if need.endswith("-ide") else "--require", need)), "--creator", CREATOR, "--slug", slug, "--authorize", "base-image", "default", diff --git a/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md index f56fd1a5..00374b27 100644 --- a/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md @@ -78,6 +78,17 @@ properties; emulation is the daemon's. Unit suite on this Mac: 2,749 passed, 25 The human's `~/hello-devcapsule` checkout holds amd64 VSCodium pins labelled arm64, written by the bug since fixed; re-choosing its IDE through DevCapsule commands repairs it. +**2026-11-04, after the fixes.** A personal pin is remapped when its table is exactly +another platform's embedded pin, whatever its label, so the human's checkout healed without +a command (4d2aeb28): their `~/hello-devcapsule` capsule runs natively, `aarch64`, an arm64 +VSCodium. The unit suite in Linux containers: amd64, emulated, 2,770 passed, 4 failed; arm64, +native, 2,757 passed, 17 failed. Both fail three recursive-orchestrator tests that need a host +X11 directory, as `main` does in a headless container now that the skip is Linux-only. The +amd64 run fails a console test whose 0.3 s timeout emulated Python misses; it passes natively. +The arm64 run fails 14 console-channel tests whose real `devcapsule` subprocess sees the +arm64 host while the suite's fixtures are amd64: a limit of the suite on ARM Linux, not CI. +The IDE smoke takes `DEVCAPSULE_SMOKE_PLATFORM` to create its project for a platform. + ## Next Resumable Task Record the rerun smokes; add arm64 verified edges for the surfaces that pass, with their From eb53ce9a638163a86b077ecbdead5e9911eb0688 Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Wed, 4 Nov 2026 14:58:08 +0200 Subject: [PATCH 18/20] Run a project as the platform its locks name; regenerate every lock A second review of all changes since main, and a review against the codebase's own conventions, found that one checkout could switch platform per command and that owner edits on a Mac left the committed linux-amd64 lock behind its manifest. The design note's sections 3 and 5 record the outcome: - project --platform and the module-global selection behind it are gone. A project runs as the first platform this host runs that it has a lock for; project init --platform adds that platform's lock, and images build --platform names a base build's platform. One argparse type parses both and refuses a platform the host cannot run. - An owner's edit of the shared contract regenerates every lock the project carries under one journal. Recovery still reads the single-lock journal an older launcher wrote. - Below the configuration the lock's platform is required: the base pull, the build, the run options and the recursive successor take it. The Docker daemon's architecture is asked once per launch, not cached for the process; Platform.emulated_docker_platform is the one emulation decision. - The macOS sudo-policy sweep removes only unused policy volumes older than a day, so a concurrent launch's fresh volume survives. - The retained successor plan carries its planned --platform, so the independent inspection checks it too. - linux-arm64 records native smokes of VSCodium, VS Code, Eclipse, PyCharm, Rider and IntelliJ IDEA as verified edges; the .NET SDK and Playwright stay unverified. - The arm64 pin module holds plain literals under unambiguous names, libunwind8 in its sorted place. - Conventions: Warning: prefixes, owner-free comments, upper-case placeholders, inline Linux-only skips, one host fixture and helper in the tests, matrix_for and lock_platform for a lock's platform, and the macOS-only drain in host_open removed. - Docs: the CLI reference and the macOS page describe init --platform; the unit and end-to-end test pages list the fixtures, the marker and the smoke's variables; index.md lists the linux-arm64 status. Co-Authored-By: Claude Opus 5.5 --- .../devcapsule/commands/framework.py | 20 ++- devcapsule-src/devcapsule/commands/images.py | 21 +-- devcapsule-src/devcapsule/commands/project.py | 36 +--- .../devcapsule/components/linux_arm64_pins.py | 23 +-- .../configuration/capability_commands.py | 160 +++++++++++++----- .../configuration/capability_selection.py | 22 ++- .../devcapsule/configuration/derivation.py | 1 - .../devcapsule/configuration/operations.py | 45 ++++- .../configuration/owner_commands.py | 52 +++--- .../devcapsule/configuration/storage.py | 49 ++---- .../devcapsule/environment_realization.py | 25 +-- devcapsule-src/devcapsule/host_daemon.py | 9 +- devcapsule-src/devcapsule/host_open.py | 16 +- devcapsule-src/devcapsule/images/base.py | 5 +- devcapsule-src/devcapsule/images/build.py | 14 +- .../devcapsule/launch/pycharm/_image_build.py | 3 +- .../devcapsule/launch/pycharm/_launcher.py | 73 ++++++-- devcapsule-src/devcapsule/platforms.py | 27 +-- .../devcapsule/recursive_successor.py | 2 + .../devcapsule/recursive_successor_plan.py | 14 +- .../devcapsule/resolution_matrix.py | 57 ++++++- devcapsule-src/devcapsule/runtime_artifact.py | 6 +- devcapsule-src/devcapsule/version_sets.py | 7 +- devcapsule-src/noxfile.py | 3 + .../tests/configuration/test_contract.py | 3 +- .../tests/configuration/test_invariants.py | 5 +- devcapsule-src/tests/conftest.py | 39 ++--- devcapsule-src/tests/e2e/ide_session.py | 2 +- .../tests/e2e/test_ide_comes_alive.py | 6 +- devcapsule-src/tests/test_base_image.py | 12 +- devcapsule-src/tests/test_cli.py | 3 +- .../tests/test_environment_realization.py | 15 +- devcapsule-src/tests/test_host_open.py | 19 +-- devcapsule-src/tests/test_image_build.py | 12 +- .../tests/test_linux_arm64_matrix.py | 98 +++++------ devcapsule-src/tests/test_owner_commands.py | 57 ++++++- devcapsule-src/tests/test_platforms.py | 18 +- .../tests/test_project_platform_selection.py | 75 ++++---- devcapsule-src/tests/test_pycharm.py | 98 ++++++++--- .../tests/test_recursive_orchestrator.py | 15 +- .../tests/test_recursive_successor_plan.py | 6 + devcapsule-src/tests/test_runtime_artifact.py | 17 +- devcapsule-src/tests/test_supervisor.py | 8 +- docs/platforms/macos.md | 28 +-- docs/reference/cli.md | 21 ++- engineering-docs/development/e2e-tests.md | 2 + engineering-docs/development/unit-tests.md | 14 +- .../2026-11-04-design-linux-arm64.md | 34 +++- index.md | 1 + 49 files changed, 786 insertions(+), 512 deletions(-) diff --git a/devcapsule-src/devcapsule/commands/framework.py b/devcapsule-src/devcapsule/commands/framework.py index 3c704429..8f149c14 100644 --- a/devcapsule-src/devcapsule/commands/framework.py +++ b/devcapsule-src/devcapsule/commands/framework.py @@ -6,8 +6,9 @@ - The framework owns the command *tree*: discovery-independent group dispatch, group help, hidden commands, group-scoped options such as - ``project --path``, and the carrier-option grammar shared by ``init``, - ``run``, and the ``config`` family. + ``project --path``, the carrier-option grammar shared by ``init``, + ``run``, and the ``config`` family, and the ``--platform`` value shared by + ``project init`` and ``images build``. - :mod:`argparse` owns only *leaf* option parsing, declared per command in ``configure``. @@ -31,6 +32,7 @@ from typing import Any, ClassVar, Mapping, NoReturn, Sequence from devcapsule.compat import CliError +from devcapsule.platforms import Platform, UnsupportedPlatformError class UsageError(CliError): @@ -303,3 +305,17 @@ def carrier_answers( ) ) return tuple(answers) + + +def capsule_platform_argument(value: str) -> Platform: + """A ``--platform`` value: a capsule platform this host runs.""" + + try: + platform = Platform.parse(value) + runnable = Platform.runnable() + except UnsupportedPlatformError as exc: + raise argparse.ArgumentTypeError(str(exc)) from exc + if platform not in runnable: + names = ", ".join(str(member) for member in runnable) + raise argparse.ArgumentTypeError(f"this host runs {names} capsules, not {platform}") + return platform diff --git a/devcapsule-src/devcapsule/commands/images.py b/devcapsule-src/devcapsule/commands/images.py index f7669b12..92ea9b9b 100644 --- a/devcapsule-src/devcapsule/commands/images.py +++ b/devcapsule-src/devcapsule/commands/images.py @@ -19,11 +19,12 @@ build_base_image, resolved_root_image, ) -from devcapsule.commands.framework import Command, Group +from devcapsule.commands.framework import Command, Group, capsule_platform_argument from devcapsule.compat import CliError from devcapsule.environment_realization import optional_local_image, realize_environment from devcapsule.images.metadata import LocalImageRecord, inspect_local_image, list_local_images from devcapsule.materialization import ImageDetails +from devcapsule.platforms import Platform from devcapsule.configuration.execution import ( fresh_resolved_project, ) @@ -113,23 +114,17 @@ def configure(cls, parser: argparse.ArgumentParser) -> None: parser.add_argument( "--platform", dest="capsule_platform", + type=capsule_platform_argument, metavar="PLATFORM", - help="Capsule platform to build for, such as linux-arm64. An environment build defaults to " - "the first one this host runs that the project has a lock for; a base build to the one " - "this host creates projects for.", + help="Capsule platform of a base build, such as linux-arm64; default: the one this host " + "creates projects for. An environment build is for its project's platform.", ) @classmethod def run(cls, arguments: argparse.Namespace, context: object | None) -> int: - from devcapsule.configuration.storage import name_platform - from devcapsule.platforms import Platform, UnsupportedPlatformError - - try: - capsule_platform = Platform.parse(arguments.capsule_platform) if arguments.capsule_platform else None - except UnsupportedPlatformError as exc: - raise CliError(str(exc)) from exc - name_platform(capsule_platform) if arguments.image_type == "environment": + if arguments.capsule_platform is not None: + raise CliError("--platform applies only to base builds; an environment build is for its project's platform.") return _build_environment( project=arguments.project, base_override=arguments.base_override, @@ -169,7 +164,7 @@ def run(cls, arguments: argparse.Namespace, context: object | None) -> int: build_base_image( options, network=arguments.network, - platform=capsule_platform, + platform=arguments.capsule_platform or Platform.current(), ) image = inspect_local_image(arguments.tag) labels = dict(image.config.labels or {}) diff --git a/devcapsule-src/devcapsule/commands/project.py b/devcapsule-src/devcapsule/commands/project.py index 1f9ece3d..e9c5cbfe 100644 --- a/devcapsule-src/devcapsule/commands/project.py +++ b/devcapsule-src/devcapsule/commands/project.py @@ -32,6 +32,7 @@ Command, Group, add_carrier_options, + capsule_platform_argument, carrier_answers, ) from devcapsule.components.catalog import COMPONENTS @@ -53,8 +54,8 @@ from devcapsule import runtime_configuration from devcapsule.runtime_configuration import CapsuleAccess from devcapsule.compat import CliError, DecisionPending -from devcapsule.platforms import Platform import difflib +from devcapsule.configuration.capability_selection import lock_platform from devcapsule.configuration.capabilities import CapabilityPolicy, LocalCapabilities from devcapsule.configuration.console_requests import deferred_rows from devcapsule.configuration.listing import Listing, assess_checkout @@ -311,6 +312,11 @@ def configure(cls, parser: argparse.ArgumentParser) -> None: help="A capability the project offers and a developer may leave out, such as browser-automation. Repeatable.", ) _add_owner_options(parser) + parser.add_argument( + "--platform", type=capsule_platform_argument, metavar="PLATFORM", + help="Add the lock for this capsule platform, such as linux-arm64; the project then runs as it here. " + "Default: the platform this host runs the project as.", + ) _retired_option(parser, "--agent", "devcapsule project init --with CAPABILITY") parser.add_argument( "--regenerate", @@ -352,6 +358,7 @@ def run(cls, arguments: argparse.Namespace, context: object | None) -> int: answers=answers, regenerate=arguments.regenerate, allow_unverified=arguments.allow_unverified, + platform=arguments.platform, ) ) print(report.render()) @@ -2437,7 +2444,7 @@ def prepare_display(base: ImageDetails) -> None: enable_host_browser=selected_host_browser, display_transport=display_transport, web_console=web_console, - capsule_platform=Platform.parse(str(lock["platform"])), + capsule_platform=lock_platform(lock), ) ) if command_report is not None: @@ -2539,13 +2546,6 @@ def configure(cls, parser: argparse.ArgumentParser) -> None: type=Path, help="Project root or descendant; defaults to discovery from the current directory.", ) - parser.add_argument( - "--platform", - dest="capsule_platform", - metavar="PLATFORM", - help="Capsule platform for this command, such as linux-arm64; defaults to the first one " - "this host runs that the project has a lock for.", - ) @classmethod def make_context(cls, arguments: argparse.Namespace, parent: object | None) -> object | None: @@ -2560,7 +2560,6 @@ def make_context(cls, arguments: argparse.Namespace, parent: object | None) -> o launcher. """ context = ProjectCommandContext(arguments.selected_path) - _name_capsule_platform(getattr(arguments, "capsule_platform", None)) tokens: list[str] = list(arguments.rest) parsed_tokens = tokens[:tokens.index("--")] if "--" in tokens else tokens if any(token in {"-h", "--help"} for token in parsed_tokens): @@ -2606,23 +2605,6 @@ def subcommands(cls) -> Mapping[str, type[Command] | type[Group]]: } -def _name_capsule_platform(named: str | None) -> None: - """Record the platform ``--platform`` named for this command, or clear it. - - Without one, each step asks ``project_platform`` for the project's - platform, which its locks decide on a host that runs two. Naming one is - how ``project --platform linux-arm64 init`` adds that platform's lock to a - project, since init completes what a project lacks. - """ - from devcapsule.configuration.storage import name_platform - from devcapsule.platforms import Platform, UnsupportedPlatformError - - try: - name_platform(Platform.parse(named) if named is not None else None) - except UnsupportedPlatformError as exc: - raise ProjectConfigurationError(str(exc)) from exc - - def _capsule_access(leaf: type[Command]) -> CapsuleAccess: """The access a ``project`` leaf declares for itself. diff --git a/devcapsule-src/devcapsule/components/linux_arm64_pins.py b/devcapsule-src/devcapsule/components/linux_arm64_pins.py index 0744ae39..d48a3b9f 100644 --- a/devcapsule-src/devcapsule/components/linux_arm64_pins.py +++ b/devcapsule-src/devcapsule/components/linux_arm64_pins.py @@ -22,7 +22,7 @@ from typing import Any -ECLIPSE_NATIVE_PACKAGES: dict[str, dict[str, str]] = {'bubblewrap': {'sha256': 'dce3a2e5f0302eab717a64fe9c9411fe2d40469e2a7946e0f2ce1420a61025c3', +LINUX_ARM64_ECLIPSE_NATIVE_PACKAGES: dict[str, dict[str, str]] = {'bubblewrap': {'sha256': 'dce3a2e5f0302eab717a64fe9c9411fe2d40469e2a7946e0f2ce1420a61025c3', 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/b/bubblewrap/bubblewrap_0.9.0-1ubuntu0.3_arm64.deb', 'version': '0.9.0-1ubuntu0.3'}, 'dictionaries-common': {'sha256': '601e7cb4d662aca433785eaafd8094e51b1386aab5f5aef6572a1a565b80f21e', @@ -184,6 +184,9 @@ 'libtwolame0': {'sha256': 'f2fffdedec9f9bc58fc700ccbc146c50a5a1d5ae2e6881e501bd727234aa5d16', 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/t/twolame/libtwolame0_0.4.0-2build3_arm64.deb', 'version': '0.4.0-2build3'}, + 'libunwind8': {'sha256': '26f2bf9c0d9b42d8230aa2c070076054b0ed06519bb9141c05a9c3164732fef1', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libu/libunwind/libunwind8_1.6.2-3build1.1_arm64.deb', + 'version': '1.6.2-3build1.1'}, 'libv4l-0t64': {'sha256': '0efed54458a4a3da075defb2675b243d7123115d26219ea2ffb88750e777f0e3', 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/v/v4l-utils/libv4l-0t64_1.26.1-4build3_arm64.deb', 'version': '1.26.1-4build3'}, @@ -221,17 +224,7 @@ 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/x/xdg-dbus-proxy/xdg-dbus-proxy_0.1.5-1ubuntu0.2_arm64.deb', 'version': '0.1.5-1ubuntu0.2'}} -# arm64 only: libgstreamer1.0-0 depends on libunwind8 on arm64, which the -# amd64 closure does not carry. Found by the Eclipse smoke on 2026-11-04 -# ("Package libunwind8 is not installed"); noble-updates, its index verified -# against the gpgv-checked InRelease and the bytes against the index. -ECLIPSE_NATIVE_PACKAGES["libunwind8"] = { - "sha256": "26f2bf9c0d9b42d8230aa2c070076054b0ed06519bb9141c05a9c3164732fef1", - "url": "https://ports.ubuntu.com/ubuntu-ports/pool/main/libu/libunwind/libunwind8_1.6.2-3build1.1_arm64.deb", - "version": "1.6.2-3build1.1", -} - -PLAYWRIGHT_PIN: dict[str, Any] = {'version': '1.63.0', +LINUX_ARM64_PLAYWRIGHT_PIN: dict[str, Any] = {'version': '1.63.0', 'delivery-policy': 'local-materialization', 'license': 'Apache-2.0', 'platform': 'linux-arm64', @@ -291,7 +284,8 @@ 'terms-url': 'https://www.eclipse.org/legal/epl-2.0/', 'url': 'https://download.eclipse.org/technology/epp/downloads/release/2026-09/R/eclipse-java-2026-09-R-linux-gtk-aarch64.tar.gz', 'sha256': '37f8bff91ce2af38bb691ab0608e1184af48cb48ad04da2bb09a9568ba905f8d', - 'upstream-sha512': 'c95453e0170efff6294a746b53e2f83f857aeb2309dbc20b933de18664f7d7e717f071a08a96d46797c2191014e1c8505da8370e45fff83a91396636d29a3091'}, + 'upstream-sha512': 'c95453e0170efff6294a746b53e2f83f857aeb2309dbc20b933de18664f7d7e717f071a08a96d46797c2191014e1c8505da8370e45fff83a91396636d29a3091', + 'native-packages': LINUX_ARM64_ECLIPSE_NATIVE_PACKAGES}, ('rider', '2026.2.3.1'): {'version': '2026.2.3.1', 'delivery-policy': 'local-materialization', 'license': 'Proprietary', @@ -477,6 +471,5 @@ 'sha256': '30dd61ebe1f0a8829057ea2de29dad08c710dd972ce4c02949f51e632135a86b', 'archive-member': 'antigravity', 'upstream-sha512': '7cae317fb6a02eb387982f06bfb234b366879cf9a80ad8969b571e5a55d1e37a6a5a49cd6b703c0443bb417937cf5e3b14a8c80cb26917ad79eef723a9f374c7'}}}, + ('playwright', '1.63.0'): LINUX_ARM64_PLAYWRIGHT_PIN, } -COMPONENT_LOCK_TABLES[("eclipse", "2026-09-R")]["native-packages"] = ECLIPSE_NATIVE_PACKAGES -COMPONENT_LOCK_TABLES[("playwright", PLAYWRIGHT_PIN["version"])] = PLAYWRIGHT_PIN diff --git a/devcapsule-src/devcapsule/configuration/capability_commands.py b/devcapsule-src/devcapsule/configuration/capability_commands.py index 4b773f42..9fbaaffe 100644 --- a/devcapsule-src/devcapsule/configuration/capability_commands.py +++ b/devcapsule-src/devcapsule/configuration/capability_commands.py @@ -13,7 +13,7 @@ import tomllib from typing import Any, Mapping, Sequence -from devcapsule.platforms import Platform +from devcapsule.platforms import Platform, UnsupportedPlatformError from devcapsule.resolution_matrix import MATRICES, ResolutionMatrix from .capabilities import CapabilityPolicy, LocalCapabilities, capability_notes, resolution_recommendations from .capability_selection import compose_lock, effective_lock, generate_lock, local_pins, usable_lock, matrix_for, pin_selection @@ -23,7 +23,8 @@ from .manifest_schemas import ManifestV2 from .nodes import build_node_registry from .storage import ( - atomic_write, checkout_record_paths, discover_project, load_toml, platform_lock_path, project_platform, require_settled, + atomic_write, checkout_record_paths, discover_project, load_toml, locked_platforms, platform_lock_path, + project_platform, require_settled, ) @@ -40,70 +41,137 @@ def paths(root: Path) -> tuple[Path, Path, Path]: def recover(root: Path) -> None: """Finish an interrupted shared edit, or do nothing if no journal exists. - The manifest chooses the before/after lock to restore. Refuse malformed - journals or independent edits with ProjectConfigurationError. Successful - recovery removes the journal. Use the same platform that began the edit; - callers must exclude concurrent writers. Filesystem errors propagate. + The manifest chooses the before or after side of every lock the journal + holds. Refuse malformed journals or independent edits with + ProjectConfigurationError. Successful recovery removes the journal. + Callers must exclude concurrent writers. Filesystem errors propagate. """ - manifest_path, lock_path, journal = paths(root) + manifest_path, _, journal = paths(root) if not journal.exists(): return transaction = load_toml(journal) - if set(transaction) != {"before-manifest", "after-manifest", "before-lock", "after-lock"} or any( - not isinstance(value, str) for value in transaction.values() - ): - raise ProjectConfigurationError("Malformed capability transaction; preserve it for recovery.") + locks = _journaled_locks(root, transaction) current = manifest_path.read_bytes().decode("utf-8") if manifest_path.exists() else "" - current_lock = lock_path.read_bytes().decode("utf-8") if lock_path.exists() else "" - if current_lock not in {transaction["before-lock"], transaction["after-lock"]}: - raise ProjectConfigurationError("Lock changed outside the pending transaction; refusing to overwrite it.") + for lock_path, (before, after) in locks.items(): + current_lock = lock_path.read_bytes().decode("utf-8") if lock_path.exists() else "" + if current_lock not in {before, after}: + raise ProjectConfigurationError("Lock changed outside the pending transaction; refusing to overwrite it.") if current == transaction["after-manifest"]: - chosen = "after-lock" + committed = True elif current == transaction["before-manifest"]: - chosen = "before-lock" + committed = False else: raise ProjectConfigurationError("Configuration changed outside the pending transaction; refusing to overwrite it.") - # Manifest-only edits preserve even an absent or empty lock. - if transaction["before-lock"] != transaction["after-lock"]: - if transaction[chosen]: - atomic_write(lock_path, transaction[chosen], mode=0o644) - else: - lock_path.unlink(missing_ok=True) + for lock_path, (before, after) in locks.items(): + restored = after if committed else before + # Manifest-only edits preserve even an absent or empty lock. + if before != after: + if restored: + atomic_write(lock_path, restored, mode=0o644) + else: + lock_path.unlink(missing_ok=True) journal.unlink() -def promote(root: Path, manifest: dict[str, Any], lock: dict[str, Any] | None = None) -> None: - """Persist an already-validated manifest, and lock when given, with a recovery journal. +def _journaled_locks(root: Path, transaction: Mapping[str, Any]) -> dict[Path, tuple[str, str]]: + """Each lock path the journal holds, with its before and after text. + + A journal holds every platform lock under ``before-locks`` and + ``after-locks``; one written by a launcher of a single platform holds + that platform's lock as ``before-lock`` and ``after-lock``. + """ + malformed = ProjectConfigurationError("Malformed capability transaction; preserve it for recovery.") + manifests = (transaction.get("before-manifest"), transaction.get("after-manifest")) + if not all(isinstance(value, str) for value in manifests): + raise malformed + if set(transaction) == {"before-manifest", "after-manifest", "before-lock", "after-lock"}: + sides = {str(project_platform(root)): (transaction["before-lock"], transaction["after-lock"])} + elif set(transaction) == {"before-manifest", "after-manifest", "before-locks", "after-locks"}: + before, after = transaction["before-locks"], transaction["after-locks"] + if not isinstance(before, dict) or not isinstance(after, dict) or set(before) != set(after): + raise malformed + sides = {key: (before[key], after[key]) for key in before} + else: + raise malformed + locks: dict[Path, tuple[str, str]] = {} + for key, (before_text, after_text) in sides.items(): + if not isinstance(before_text, str) or not isinstance(after_text, str): + raise malformed + try: + platform = Platform.parse(key) + except UnsupportedPlatformError: + raise malformed from None + locks[platform_lock_path(root, platform)] = (before_text, after_text) + return locks + + +def promote(root: Path, manifest: dict[str, Any], locks: Mapping[Platform, dict[str, Any]] | None = None) -> None: + """Persist an already-validated manifest, and the locks when given, with a recovery journal. Call ``check_documents`` first and exclude concurrent writers. Write the - lock before the manifest; attempt recovery on OSError. Failures propagate. - A pending journal raises ProjectConfigurationError. This updates shared - files only; it neither writes nor resolves a developer's checkout. + locks before the manifest; attempt recovery on OSError. Failures + propagate. A pending journal raises ProjectConfigurationError. This + updates shared files only; it neither writes nor resolves a developer's + checkout. The manifest is written by the one manifest writer: bare keys, the owner's table order, the leading comment block of the existing file - kept, and the supplied schema version. A ``lock`` of ``None`` keeps the + kept, and the supplied schema version. ``locks`` of ``None`` keeps every lock file's bytes exactly, for an edit that changes the manifest alone. """ - manifest_path, lock_path, journal = paths(root) + manifest_path, _, journal = paths(root) ManifestV2.read(manifest, manifest_path) # The one writer writes schema 2 and nothing else. require_settled(root) before_manifest = manifest_path.read_bytes().decode("utf-8") if manifest_path.exists() else "" - before_lock = lock_path.read_bytes().decode("utf-8") if lock_path.exists() else "" - transaction = {"before-manifest": before_manifest, "before-lock": before_lock, - "after-manifest": render_manifest(manifest, preamble=manifest_preamble(before_manifest)), - "after-lock": before_lock if lock is None else render_toml(lock)} - atomic_write(journal, render_toml(transaction)) + after_manifest = render_manifest(manifest, preamble=manifest_preamble(before_manifest)) + before_locks: dict[str, str] = {} + after_locks: dict[str, str] = {} + for platform, lock in (locks or {}).items(): + lock_path = platform_lock_path(root, platform) + before_locks[str(platform)] = lock_path.read_bytes().decode("utf-8") if lock_path.exists() else "" + after_locks[str(platform)] = render_toml(lock) + atomic_write(journal, render_toml({"before-manifest": before_manifest, "after-manifest": after_manifest, + "before-locks": before_locks, "after-locks": after_locks})) try: - if lock is not None: - atomic_write(lock_path, transaction["after-lock"], mode=0o644) - atomic_write(manifest_path, transaction["after-manifest"], mode=0o644) + for platform, lock in (locks or {}).items(): + atomic_write(platform_lock_path(root, platform), after_locks[str(platform)], mode=0o644) + atomic_write(manifest_path, after_manifest, mode=0o644) except OSError: recover(root) raise journal.unlink() +def project_locks(root: Path) -> dict[Platform, dict[str, Any]]: + """Every platform lock the project carries, by platform. + + An edit of the shared contract regenerates them all, so a lock committed + for another platform never falls behind the manifest (R-COMPAT-001). A + project with none fails as reading this host's lock does. + """ + platforms = locked_platforms(root) or (project_platform(root),) + return {platform: load_toml(platform_lock_path(root, platform)) for platform in platforms} + + +def regenerate_locks(policy: CapabilityPolicy, locks: Mapping[Platform, dict[str, Any]], *, + recommendations: Mapping[str, str], allow_unverified: bool = False) -> dict[Platform, dict[str, Any]]: + """Each of ``locks`` regenerated for ``policy`` from its own platform's matrix. + + A lock that cannot follow refuses the whole edit; with several, the + error names the platform. + """ + generated: dict[Platform, dict[str, Any]] = {} + for platform, lock in locks.items(): + try: + generated[platform] = generate_lock(policy, MATRICES[platform], previous=lock, + allow_unverified=allow_unverified, recommendations=recommendations) + except ProjectConfigurationError as exc: + if len(locks) == 1: + raise + raise ProjectConfigurationError(f"The project's {platform} lock cannot follow this edit: {exc}") from exc + return generated + + def check_documents(manifest: dict[str, Any], lock: dict[str, Any], *, check_resolutions: bool = True) -> tuple[str, ...]: """Validate shared documents offline and return optional-omission warnings. @@ -196,18 +264,21 @@ def configure(start: Path, *, required: Sequence[str] | None = None, raise ProjectConfigurationError("Choose either project flags (--required/--optional) or local flags (--local/--without).") matrix = MATRICES[project_platform(root)] if project_edit: - return configure_project(root, manifest, lock, matrix, required=required, optional=optional, + locks = project_locks(root) + for other in locks.values(): + check_documents(manifest, other) + return configure_project(root, manifest, locks, required=required, optional=optional, preview=preview, allow_unverified=allow_unverified) return configure_local(root, manifest, lock, matrix, local=local, without=without, providers=providers, preview=preview, allow_unverified=allow_unverified) -def configure_project(root: Path, manifest: dict[str, Any], lock: dict[str, Any], matrix: ResolutionMatrix, *, +def configure_project(root: Path, manifest: dict[str, Any], locks: Mapping[Platform, dict[str, Any]], *, required: Sequence[str] | None, optional: Sequence[str] | None, preview: bool, allow_unverified: bool) -> str: - """Replace the shared policy and regenerate the platform lock; see ``configure``. + """Replace the shared policy and regenerate every platform lock; see ``configure``. - ``manifest`` and ``lock`` are the validated current shared documents and + ``manifest`` and ``locks`` are the validated current shared documents and are not changed; ``manifest`` is used as the candidate's template. """ policy = CapabilityPolicy.read(manifest) @@ -219,9 +290,10 @@ def configure_project(root: Path, manifest: dict[str, Any], lock: dict[str, Any] candidate_manifest["capabilities"] = declarations candidate = CapabilityPolicy.read(candidate_manifest) candidate_manifest["capabilities"] = capability_table(candidate, capability_reasons(manifest), previous=manifest["capabilities"]) - generated = generate_lock(candidate, matrix, previous=lock, allow_unverified=allow_unverified, - recommendations=resolution_recommendations(candidate_manifest)) - check_documents(candidate_manifest, generated) + generated = regenerate_locks(candidate, locks, allow_unverified=allow_unverified, + recommendations=resolution_recommendations(candidate_manifest)) + for lock in generated.values(): + check_documents(candidate_manifest, lock) if not preview: promote(root, candidate_manifest, generated) return ("Preview; no files changed.\n" if preview else "Updated shared capability policy and lock.\n") + render_toml(candidate_manifest["capabilities"]) diff --git a/devcapsule-src/devcapsule/configuration/capability_selection.py b/devcapsule-src/devcapsule/configuration/capability_selection.py index 8f34c072..227e98f4 100644 --- a/devcapsule-src/devcapsule/configuration/capability_selection.py +++ b/devcapsule-src/devcapsule/configuration/capability_selection.py @@ -15,23 +15,31 @@ from typing import Any, Mapping from devcapsule.components.catalog import COMPONENTS, INTERACTIVE_SURFACES -from devcapsule.platforms import Platform +from devcapsule.platforms import Platform, UnsupportedPlatformError from devcapsule.resolution_matrix import MATRICES, ResolutionMatrix from .capabilities import CapabilityPolicy, LocalCapabilities, names, meets_minimum, resolution_recommendations from .file_formats import ProjectConfigurationError, legacy_lock, table -def matrix_for(lock: Mapping[str, Any]) -> ResolutionMatrix: - """Return this launcher's catalog for the lock's declared platform. +def lock_platform(lock: Mapping[str, Any]) -> Platform: + """Return the lock's declared platform. Raise ProjectConfigurationError if that platform is absent or unsupported. """ try: - return MATRICES[Platform(str(lock.get("platform")))] - except (ValueError, KeyError): + return Platform.parse(str(lock.get("platform"))) + except UnsupportedPlatformError: raise ProjectConfigurationError(f"Unsupported lock platform {lock.get('platform')!r}.") from None +def matrix_for(lock: Mapping[str, Any]) -> ResolutionMatrix: + """Return this launcher's catalog for the lock's declared platform. + + Raise ProjectConfigurationError if that platform is absent or unsupported. + """ + return MATRICES[lock_platform(lock)] + + def known_policy_names(policy: CapabilityPolicy, matrix: ResolutionMatrix) -> dict[str, str]: """Canonical names for the shared policy; unknown offers stay opaque.""" result = {name: matrix.canonical_name(name) for name in policy.required} @@ -140,8 +148,8 @@ def pins_for_platform(pins: Mapping[str, Any], matrix: ResolutionMatrix) -> dict continue matrix_table = matrix.component_pin(component, metadata.get("version")) if matrix_table is None: - remedy = (f"'devcapsule project --platform {platform} config ide {component}'" if component == surface - else f"'devcapsule project --platform {platform} config use' for the capability it provides") + remedy = (f"'devcapsule project config ide {component}'" if component == surface + else f"'devcapsule project config use CAPABILITY {component}'") raise ProjectConfigurationError( f"This checkout's {component} {metadata.get('version')} is pinned for {written_for}, " f"and {platform} has no pin for that version; choose it again with {remedy}." diff --git a/devcapsule-src/devcapsule/configuration/derivation.py b/devcapsule-src/devcapsule/configuration/derivation.py index 26f3e09a..89afb695 100644 --- a/devcapsule-src/devcapsule/configuration/derivation.py +++ b/devcapsule-src/devcapsule/configuration/derivation.py @@ -26,7 +26,6 @@ from devcapsule.compat import DecisionPending from devcapsule.materialization import validate_base_image -from devcapsule.platforms import Platform from devcapsule.project import sanitize_name from devcapsule.resolution_matrix import MATRICES, ResolutionMatrix, known_base_image diff --git a/devcapsule-src/devcapsule/configuration/operations.py b/devcapsule-src/devcapsule/configuration/operations.py index 3dfacb6b..903ae576 100644 --- a/devcapsule-src/devcapsule/configuration/operations.py +++ b/devcapsule-src/devcapsule/configuration/operations.py @@ -27,7 +27,7 @@ from devcapsule.environment_realization import required_local_image from devcapsule.materialization import validate_base_image from devcapsule.project import sanitize_name -from devcapsule.platforms import Platform, UnsupportedPlatformError +from devcapsule.platforms import Platform from devcapsule.resolution_matrix import MATRICES, ResolutionMatrix from .trust import HomeTrust @@ -42,7 +42,7 @@ ) from .capabilities import CapabilityPolicy, LocalCapabilities, requirement from .capability_commands import check_documents, pin_selection -from .capability_selection import compose_lock, effective_lock, generate_lock, local_pins +from .capability_selection import compose_lock, effective_lock, generate_lock, local_pins, matrix_for from .file_formats import ( AuthorizationScalar, ConfigurationFileKind, @@ -278,6 +278,8 @@ class InitializeRequest: allow_unverified: bool = False # None: decide from whether stdin is a terminal. interactive: bool | None = None + # The platform whose lock to write; None: the one the project runs as here. + platform: Platform | None = None @dataclass(frozen=True) @@ -313,7 +315,7 @@ class InitializeReport: # None when the owner's checkout has no IDE yet: nothing to resolve. resolve: ResolveReport | None # The lock pins a base its registry does not hold yet (see ResolutionMatrix.base_unpublished). - base_unpublished: bool = False + base_unpublished: bool def render(self) -> str: required = " ".join(self.required) or "none" @@ -329,7 +331,7 @@ def render(self) -> str: lines.append(f"{self.lock_action} {self.lock_path} (base {self.base_display})") if self.base_unpublished: lines.append("Its base is not published yet: a checkout runs this lock only on a locally built base of " - "the same recipe, selected with 'devcapsule project config authorize base-image '.") + "the same recipe, selected with 'devcapsule project config authorize base-image IMAGE'.") for name, value, justification in self.recommendations: line = f"Recommended {name} = {value} for every checkout" lines.append(f"{line} ({justification})." if justification else f"{line}.") @@ -406,7 +408,7 @@ def initialize_project( raise ProjectConfigurationError(f"Project directory does not exist: {root}") require_settled(root) manifest_path = root / ".devcapsule" / "devcapsule.toml" - platform = project_platform(root) + platform = _platform_to_lock(root, request.platform) lock_path = platform_lock_path(root, platform) matrix = MATRICES[platform] @@ -547,7 +549,7 @@ def persist() -> None: base_display=_base_display(shared_lock), recommendations=recommendations, selection=selection, answered=(), carried=(), recommended=(), checkout_record=record.input_path, resolve=None, - base_unpublished=matrix.base_unpublished(str(shared_lock.get("base", {}).get("reference"))), + base_unpublished=_pins_unpublished_base(shared_lock), ) lock = effective_lock(manifest, compose_lock(manifest, shared_lock, record.document), record.document, warn=False) declarations = authorization_declarations(manifest, lock) @@ -575,15 +577,36 @@ def persist() -> None: answered=tuple(settled.answered), carried=tuple(settled.carried), recommended=accepted_recommendations, checkout_record=record.input_path, resolve=resolve_report, - base_unpublished=matrix.base_unpublished(str(shared_lock.get("base", {}).get("reference"))), + base_unpublished=_pins_unpublished_base(shared_lock), ) +def _platform_to_lock(root: Path, named: Platform | None) -> Platform: + """The platform whose lock init writes: ``named``, or the one the project runs as here. + + Init completes the checkout for the lock it writes, so a named platform + must be the one the project runs as once that lock exists. + """ + if named is None: + return project_platform(root) + runnable = Platform.runnable() + if named not in runnable: + names = ", ".join(str(member) for member in runnable) + raise ProjectConfigurationError(f"This host runs {names} capsules, not {named}.") + runs_as = next(platform for platform in runnable if platform is named or platform_lock_path(root, platform).is_file()) + if runs_as is not named: + raise ProjectConfigurationError( + f"On this host the project runs as {runs_as}, which it has a lock for, " + f"so init cannot add a {named} lock here." + ) + return named + + def _initialize_report( manifest_path: Path, manifest_action: str, identity: _ProjectIdentity, lock_path: Path, lock_action: str, *, base_display: str, recommendations: list[tuple[str, str]], selection: _OwnerSelection, answered: tuple[str, ...], carried: tuple[str, ...], recommended: tuple[str, ...], - checkout_record: Path, resolve: ResolveReport | None, base_unpublished: bool = False, + checkout_record: Path, resolve: ResolveReport | None, base_unpublished: bool, ) -> InitializeReport: return InitializeReport( manifest_path=manifest_path, @@ -612,6 +635,11 @@ def _initialize_report( ) +def _pins_unpublished_base(lock: Mapping[str, Any]) -> bool: + """Whether ``lock`` pins a base its registry does not hold yet.""" + return matrix_for(lock).base_unpublished(str(lock.get("base", {}).get("reference"))) + + def _base_display(lock: Mapping[str, Any]) -> str: base = lock.get("base") if isinstance(base, dict) and isinstance(base.get("reference"), str): @@ -1585,6 +1613,7 @@ def _apply_answers_to_standing_checkout( base_display=base_display, recommendations=[], selection=_existing_selection(root, manifest), answered=tuple(settled.answered), carried=tuple(settled.carried), recommended=(), checkout_record=record.input_path, resolve=resolve_report, + base_unpublished=_pins_unpublished_base(lock), ) diff --git a/devcapsule-src/devcapsule/configuration/owner_commands.py b/devcapsule-src/devcapsule/configuration/owner_commands.py index d2fad3cc..5972b998 100644 --- a/devcapsule-src/devcapsule/configuration/owner_commands.py +++ b/devcapsule-src/devcapsule/configuration/owner_commands.py @@ -2,9 +2,9 @@ Each operation edits the shared manifest through one writer (:func:`manifest_document.render_manifest`) and, when the tools change, -regenerates the platform lock, validating the candidate pair before anything -is written and promoting both files under the capability transaction journal -of :mod:`capability_commands`. The rc4 IDE recommendation command stays; agents are ordinary shared +regenerates every platform lock the project carries, validating each +candidate pair before anything is written and promoting the files under the +capability transaction journal of :mod:`capability_commands`. The rc4 IDE recommendation command stays; agents are ordinary shared capabilities. Component acquisition still requires the developer's consent. The verbs are the ones the CLI UX redesign work order of 2026-10-24 settled: @@ -24,8 +24,7 @@ from .authorization import CURATED_HOST_RECOMMENDATIONS, render_authorization_value from .capabilities import CapabilityPolicy, requirement, resolution_recommendations -from .capability_commands import canonical_tool_name, check_documents, paths, promote -from .capability_selection import generate_lock +from .capability_commands import canonical_tool_name, check_documents, paths, project_locks, promote, regenerate_locks from .file_formats import ProjectConfigurationError from .manifest import current_manifest from .manifest_document import ( @@ -53,20 +52,24 @@ ] -def _project(start: Path, *, check_resolutions: bool = True) -> tuple[Path, dict[str, Any], dict[str, Any], ResolutionMatrix]: +def _project(start: Path, *, check_resolutions: bool = True) -> tuple[ + Path, dict[str, Any], dict[Platform, dict[str, Any]], ResolutionMatrix, +]: + """The project's root, manifest and every lock, validated, and this host's matrix for its vocabulary.""" root = discover_project(start) require_settled(root) - manifest_path, lock_path, _ = paths(root) - manifest, lock = current_manifest(load_toml(manifest_path), manifest_path).document(), load_toml(lock_path) - check_documents(manifest, lock, check_resolutions=check_resolutions) - return root, manifest, lock, MATRICES[project_platform(root)] + manifest_path = paths(root)[0] + manifest, locks = current_manifest(load_toml(manifest_path), manifest_path).document(), project_locks(root) + for lock in locks.values(): + check_documents(manifest, lock, check_resolutions=check_resolutions) + return root, manifest, locks, MATRICES[project_platform(root)] def _policy_edit( start: Path, *, change: str, names: Sequence[str], because: str | None, preview: bool, allow_unverified: bool, native: bool = False, ) -> str: - root, manifest, lock, matrix = _project(start) + root, manifest, locks, matrix = _project(start) policy = CapabilityPolicy.read(manifest) reasons = capability_reasons(manifest) required, optional = set(policy.required), set(policy.optional) @@ -127,14 +130,16 @@ def ordered(values: set[str], previous: list[str]) -> tuple[str, ...]: candidate = CapabilityPolicy(ordered(required, required_order), ordered(optional, optional_order), tuple(sorted(minimums.items()))) candidate_manifest = deepcopy(manifest) candidate_manifest["capabilities"] = capability_table(candidate, reasons, previous=manifest["capabilities"]) - generated = generate_lock(candidate, matrix, previous=lock, allow_unverified=allow_unverified, - recommendations=resolution_recommendations(candidate_manifest)) - check_documents(candidate_manifest, generated) + generated = regenerate_locks(candidate, locks, allow_unverified=allow_unverified, + recommendations=resolution_recommendations(candidate_manifest)) + for lock in generated.values(): + check_documents(candidate_manifest, lock) if not preview: promote(root, candidate_manifest, generated) verb = {"require": "Required", "offer": "Offered", "drop": "Dropped"}[change] lines = [ - ("Preview; nothing written." if preview else f"{verb} {', '.join(canonical)}; the manifest and the lock are updated."), + ("Preview; nothing written." if preview else + f"{verb} {', '.join(canonical)}; the manifest and {'the lock' if len(generated) == 1 else 'its locks'} are updated."), f"Required: {' '.join(candidate.requirements()) or 'none'}", f"Offered: {' '.join(candidate.optional) or 'none'}", ] @@ -170,7 +175,7 @@ def _refresh_own_checkout(root: Path, manifest: Mapping[str, Any]) -> list[str]: def require_tools(start: Path, names: Sequence[str], *, because: str | None = None, preview: bool = False, allow_unverified: bool = False) -> str: - """Make ``names`` required tools of the project, with an optional reason; regenerate the lock.""" + """Make ``names`` required tools of the project, with an optional reason; regenerate the locks.""" return _policy_edit(start, change="require", names=names, because=because, preview=preview, allow_unverified=allow_unverified) @@ -186,7 +191,7 @@ def drop_tools(start: Path, names: Sequence[str], *, preview: bool = False, allo def _environment_edit(start: Path, *, change: str, names: Sequence[str], because: str | None, preview: bool) -> str: - root, manifest, _lock, _matrix = _project(start) + root, manifest, _locks, _matrix = _project(start) environment = deepcopy(manifest.get("environment", {})) required, optional = environment.get("required", {}), environment.get("optional", {}) canonical = tuple(environment_name(name, field_name="environment variable") for name in names) @@ -251,7 +256,7 @@ def recommend(start: Path, name: str, value: str, *, because: str | None = None, recommendation. A host recommendation requires ``because``: the reason is what the developer reads when asked. """ - root, manifest, lock, matrix = _project(start, check_resolutions=False) + root, manifest, locks, matrix = _project(start, check_resolutions=False) candidate_manifest = deepcopy(manifest) removing = value.strip().lower() == "none" generated = None @@ -330,11 +335,11 @@ def recommend(start: Path, name: str, value: str, *, because: str | None = None, capabilities.pop("resolution", None) if resolutions: capabilities["resolution"] = resolutions - generated = generate_lock(CapabilityPolicy.read(candidate_manifest), matrix, previous=lock, - recommendations=resolutions) + generated = regenerate_locks(CapabilityPolicy.read(candidate_manifest), locks, recommendations=resolutions) effect = "the provider default; acquisition still needs the developer's trust or consent" _validate(candidate_manifest) - check_documents(candidate_manifest, generated if generated is not None else lock) + for lock in (generated if generated is not None else locks).values(): + check_documents(candidate_manifest, lock) if not preview: promote(root, candidate_manifest, generated) head = "Preview; nothing written." if preview else ( @@ -408,7 +413,7 @@ def declare_source(start: Path, path: str, *, upstream: str | None = None, java: not know. The lock is untouched: a source adds no capability by itself; ``require_tools`` does that, and ``project wrap`` calls both. """ - root, manifest, _lock, _matrix = _project(start, check_resolutions=False) + root, manifest, locks, _matrix = _project(start, check_resolutions=False) path = source_path(path, field_name="source path") if upstream is not None and (not isinstance(upstream, str) or not upstream.strip() or has_control_characters(upstream)): raise ProjectConfigurationError("The upstream is the upstream repository's URL, on one line; it cannot be blank.") @@ -422,7 +427,8 @@ def declare_source(start: Path, path: str, *, upstream: str | None = None, java: candidate_manifest = deepcopy(manifest) candidate_manifest["sources"] = candidate.document(previous=previous) WrappedSources.read(candidate_manifest) - check_documents(candidate_manifest, _lock, check_resolutions=False) + for lock in locks.values(): + check_documents(candidate_manifest, lock, check_resolutions=False) if not preview: promote(root, candidate_manifest) verb = "Redeclared" if declared.get(path) is not None else "Declared" diff --git a/devcapsule-src/devcapsule/configuration/storage.py b/devcapsule-src/devcapsule/configuration/storage.py index dc27781a..b9dedfe9 100644 --- a/devcapsule-src/devcapsule/configuration/storage.py +++ b/devcapsule-src/devcapsule/configuration/storage.py @@ -382,52 +382,35 @@ def manifest_source_digest(root: Path) -> str | None: return None -# The platform a command was told to use (`project --platform`), if any. A -# command runs for one project, so this is the process's; see project_platform. -_named_platform: Platform | None = None - - -def name_platform(platform: Platform | None) -> None: - """Make ``platform`` the one project commands in this process use, or clear it. - - It must be one this host runs; the error names the ones it does. - """ - global _named_platform - if platform is not None and platform not in Platform.runnable(): - names = ", ".join(str(member) for member in Platform.runnable()) - raise ProjectConfigurationError(f"This host runs {names} capsules, not {platform}.") - _named_platform = platform - - def platform_lock_path(root: Path, platform: Platform) -> Path: """The path of the project's lock for ``platform``.""" return root / ".devcapsule" / f"devcapsule.{platform}.lock" -def project_platform(root: Path | None) -> Platform: +def project_platform(root: Path) -> Platform: """The capsule platform this host runs the project at ``root`` as. - The platform named for this command, if any; otherwise the first platform - this host runs that the project has a lock for, so Apple Silicon runs a - project locked for linux-arm64 natively and one locked only for - linux-amd64 under emulation; otherwise, as for a project not yet - initialized, the platform this host creates projects for. Reads only. + The first platform this host runs that the project has a lock for, so + Apple Silicon runs a project locked for linux-arm64 natively and one + locked only for linux-amd64 under emulation; for a project with no such + lock, the platform this host creates projects for. Reads only. """ - if _named_platform is not None: - return _named_platform try: - runnable = Platform.runnable() - if root is not None: - for platform in runnable: - if platform_lock_path(root, platform).is_file(): - return platform + for platform in Platform.runnable(): + if platform_lock_path(root, platform).is_file(): + return platform return Platform.current() except UnsupportedPlatformError as exc: raise ProjectConfigurationError(str(exc)) from exc +def locked_platforms(root: Path) -> tuple[Platform, ...]: + """Every platform the project at ``root`` has a lock for, on any host.""" + return tuple(platform for platform in Platform if platform_lock_path(root, platform).is_file()) + + def recommendation_lock_for(root: Path, manifest: Mapping[str, Any]) -> tuple[Path, dict[str, Any]]: - """Load the committed platform lock for this host's platform. + """Load the committed lock of the platform this host runs the project as. The lock is the project-side record of one resolution: the version set and property defaults selected for one platform. It is a record, not a @@ -454,7 +437,7 @@ def recommendation_lock_for(root: Path, manifest: Mapping[str, Any]) -> tuple[Pa raise ProjectConfigurationError( f"Missing {path}: this project carries no platform lock for {platform}. " "The platform lock is authored on the project side and committed with the project; " - f"'devcapsule project --platform {platform} init' adds it." + "'devcapsule project init' adds it." ) value = load_toml(path) validate_file_format(value, ConfigurationFileKind.lock, path) @@ -471,7 +454,7 @@ def recommendation_lock_for(root: Path, manifest: Mapping[str, Any]) -> tuple[Pa def composition_for(root: Path, manifest: Mapping[str, Any]) -> tuple[Path, dict[str, Any], dict[str, Any]]: """The checkout's persistent software selection, before execution filtering. - Return the current platform's shared lock path, the composed selection (the + Return the project platform's shared lock path, the composed selection (the explicit version set, or the shared lock with personal pins laid over it) and the loaded checkout record, empty when the checkout is unregistered. A version set for another platform raises ProjectConfigurationError. diff --git a/devcapsule-src/devcapsule/environment_realization.py b/devcapsule-src/devcapsule/environment_realization.py index 32e7f7ca..bdb7eab4 100644 --- a/devcapsule-src/devcapsule/environment_realization.py +++ b/devcapsule-src/devcapsule/environment_realization.py @@ -3,7 +3,6 @@ from __future__ import annotations from dataclasses import dataclass -from functools import partial from pathlib import Path from typing import Any, Callable @@ -11,7 +10,7 @@ from python_on_whales.exceptions import DockerException from devcapsule.compat import CliError -from devcapsule import host_daemon +from devcapsule.host_daemon import daemon_architecture from devcapsule.platforms import Platform from devcapsule.images.build import BuildxImageBuilder, ImageBuildSpec from devcapsule.images.metadata import inspect_local_image @@ -120,15 +119,15 @@ def realize_environment( raise CliError( f"The lock's base {base_reference} is not published yet, so it cannot be pulled. Build the " f"{capsule_platform} base locally ('devcapsule images build --type base --recipe ubuntu-24.04 " - f"--platform {capsule_platform} --tag --source-revision ') and select it " - "for this checkout ('devcapsule project config authorize base-image ')." + f"--platform {capsule_platform} --tag IMAGE --source-revision REVISION') and select it " + "for this checkout ('devcapsule project config authorize base-image IMAGE')." ) if obtain_image is not None: obtain = obtain_image elif local_authorization: obtain = required_local_image else: - obtain = partial(ensure_local_image, platform=capsule_platform) + obtain = lambda reference: ensure_local_image(reference, capsule_platform) inspect = inspect_image or optional_local_image require = require_image or required_local_image base = obtain(base_reference) @@ -152,7 +151,7 @@ def realize_environment( # One context root for every build: BuildKit's incremental transfer of # the IDE trees is keyed on this path, see BuildxImageBuilder. builder = BuildxImageBuilder(context_root=selected_cache / "build-contexts" / "context") - build = partial(builder.build, network="none", platform=capsule_platform) + build = lambda spec: builder.build(spec, network="none", platform=capsule_platform) materialize_environment = materialize or ensure_materialized_surface # Explanation/reporting hooks ride only the real materializer; injected # test materializers keep their narrower signature. @@ -192,19 +191,11 @@ def realize_environment( ) -def ensure_local_image(reference: str, platform: Platform | None = None) -> ImageDetails: - """The local image at ``reference``, pulled for ``platform`` when absent. - - ``platform`` is the lock's; without one, the platform this host creates - projects for. A daemon that runs it natively pulls as before. - """ - capsule_platform = platform or Platform.current() +def ensure_local_image(reference: str, platform: Platform) -> ImageDetails: + """The local image at ``reference``, pulled for the lock's ``platform`` when absent.""" try: if not docker.image.exists(reference): - if capsule_platform.emulated(host_daemon.daemon_architecture()): - docker.image.pull(reference, platform=capsule_platform.docker_platform) - else: - docker.image.pull(reference) + docker.image.pull(reference, platform=platform.emulated_docker_platform(daemon_architecture())) except DockerException as exc: raise CliError(f"Cannot obtain Docker image {reference!r}: {exc}") from exc return required_local_image(reference) diff --git a/devcapsule-src/devcapsule/host_daemon.py b/devcapsule-src/devcapsule/host_daemon.py index e3845ace..ff9844e4 100644 --- a/devcapsule-src/devcapsule/host_daemon.py +++ b/devcapsule-src/devcapsule/host_daemon.py @@ -16,12 +16,14 @@ actually backs it. When a bind source is not backed by any mount, translation is impossible and the launch fails loudly rather than silently producing an empty directory. + +The daemon's architecture is the other fact about it a launch needs: a daemon +of another architecture than the capsule platform runs it under emulation. """ from __future__ import annotations from collections.abc import Mapping, Sequence -from functools import cache import json from pathlib import Path, PurePosixPath import re @@ -210,14 +212,13 @@ def _unescape(value: str) -> str: return re.sub(r"\\([0-7]{3})", lambda match: chr(int(match.group(1), 8)), value) -@cache def daemon_architecture() -> str: """The Docker daemon's CPU architecture, in the platform keys' spelling. Emulation is the daemon's, not the launcher's: Docker Desktop on Apple Silicon is arm64 whether the launcher runs natively on the Mac or inside - an emulated amd64 capsule granted the daemon. Asked once per process; - this host's architecture when the daemon does not answer. + an emulated amd64 capsule granted the daemon. This host's architecture + when the daemon does not answer. """ from devcapsule.platforms import architecture_key, host_architecture diff --git a/devcapsule-src/devcapsule/host_open.py b/devcapsule-src/devcapsule/host_open.py index 202e4275..c59c510a 100644 --- a/devcapsule-src/devcapsule/host_open.py +++ b/devcapsule-src/devcapsule/host_open.py @@ -35,7 +35,7 @@ RATE_LIMIT_WINDOW_SECONDS = 60.0 _UNIX_PATH_LIMIT = 100 HOST_OPEN_UNAVAILABLE_ON_MACOS = ( - "devcapsule: warning: host-browser is unavailable on macOS: Docker Desktop does not carry the " + "Warning: host-browser is unavailable on macOS: Docker Desktop does not carry the " "host's Unix sockets into a capsule. Links open in the capsule's own browser instead." ) @@ -194,7 +194,6 @@ def _handle(self, connection: socket.socket) -> None: detail = str(exc) or "request rejected" try: connection.sendall(_encode_frame({"ok": False, "error": detail})) - _drain(connection) except OSError: pass @@ -322,19 +321,6 @@ def _encode_frame(value: Mapping[str, Any]) -> bytes: ).encode("utf-8") -def _drain(connection: socket.socket) -> None: - """Consume what a rejected client sent, up to its end or the frame limit. - - Closing a socket with unread input resets the connection on macOS, and - the client then loses the rejection it was just sent. - """ - - connection.shutdown(socket.SHUT_WR) - remaining = MAX_FRAME_BYTES - while remaining > 0 and (chunk := connection.recv(min(4096, remaining))): - remaining -= len(chunk) - - def _read_frame(connection: socket.socket, limit: int) -> bytes: chunks: list[bytes] = [] size = 0 diff --git a/devcapsule-src/devcapsule/images/base.py b/devcapsule-src/devcapsule/images/base.py index e964a94d..00c8b789 100644 --- a/devcapsule-src/devcapsule/images/base.py +++ b/devcapsule-src/devcapsule/images/base.py @@ -2,8 +2,6 @@ from __future__ import annotations -from devcapsule.platforms import Platform - import hashlib from collections.abc import Callable from dataclasses import dataclass @@ -24,6 +22,7 @@ from devcapsule.build_info import BuildInfo, BuildInfoError, current_build_info, read_pex_build_info from devcapsule.compat import CliError from devcapsule.launch.pycharm._image_build import BASE_APT_PACKAGES +from devcapsule.platforms import Platform from devcapsule.images.build import ( AptPackagesComponent, BaseImageComponent, @@ -317,9 +316,9 @@ def build_base_image( options: BaseImageBuildOptions, builder: BuildxImageBuilder | None = None, *, + platform: Platform, network: str = "default", source_verifier: Callable[[BuildInfo], None] = verify_public_github_revision, - platform: Platform | None = None, ) -> None: spec = build_base_image_spec(options) if not options.allow_local_source: diff --git a/devcapsule-src/devcapsule/images/build.py b/devcapsule-src/devcapsule/images/build.py index 48a35529..46fcd72e 100644 --- a/devcapsule-src/devcapsule/images/build.py +++ b/devcapsule-src/devcapsule/images/build.py @@ -19,7 +19,7 @@ from python_on_whales.exceptions import DockerException from devcapsule.compat import CliError, copy_resource_tree -from devcapsule import host_daemon +from devcapsule.host_daemon import daemon_architecture from devcapsule.platforms import Platform @@ -258,11 +258,10 @@ class BuildxImageBuilder: def __init__(self, context_root: Path | None = None) -> None: self.context_root = context_root - def build(self, spec: ImageBuildSpec, *, network: str = "default", platform: Platform | None = None) -> None: - """Build ``spec`` for ``platform``: a project's lock names it; a base build without one - is for the platform this host creates projects for.""" + def build(self, spec: ImageBuildSpec, *, platform: Platform, network: str = "default") -> None: + """Build ``spec`` for the capsule ``platform``.""" plan = spec.build_plan() - capsule_platform = platform or Platform.current() + emulated = platform.emulated_docker_platform(daemon_architecture()) try: with self._context_directory() as context_root: rendered = render_build_context(plan, context_root) @@ -271,10 +270,7 @@ def build(self, spec: ImageBuildSpec, *, network: str = "default", platform: Pla file=rendered.dockerfile, tags=[plan.image], load=True, - # A daemon of the capsule platform's architecture builds as - # before; another must be told the platform, not left its own. - platforms=([capsule_platform.docker_platform] - if capsule_platform.emulated(host_daemon.daemon_architecture()) else None), + platforms=[emulated] if emulated is not None else None, network=network, build_contexts={name: str(path) for name, path in rendered.named_contexts.items()}, allow=["network.host"] if network == "host" else [], diff --git a/devcapsule-src/devcapsule/launch/pycharm/_image_build.py b/devcapsule-src/devcapsule/launch/pycharm/_image_build.py index 5f1f4f63..0e3a7868 100644 --- a/devcapsule-src/devcapsule/launch/pycharm/_image_build.py +++ b/devcapsule-src/devcapsule/launch/pycharm/_image_build.py @@ -24,6 +24,7 @@ resource_to_tempdir, ) from devcapsule.images.tooling import NODE_CURRENT_BIN, node_tooling_component +from devcapsule.platforms import Platform DEFAULT_BASE_IMAGE = "ubuntu:24.04" DEFAULT_IMAGE = "pycharm-isolated:latest" @@ -162,7 +163,7 @@ def build_pycharm_image(options: PycharmImageBuildOptions, builder: BuildxImageB build_root = Path(stack.enter_context(tempfile.TemporaryDirectory(prefix="devcapsule-pycharm-build-"))) pycharm_root = normalize_pycharm_source(options.pycharm, build_root) spec = build_pycharm_image_spec(options, pycharm_root=pycharm_root, assets_root=assets_dir) - builder.build(spec, network=options.network) + builder.build(spec, platform=Platform.current(), network=options.network) return 0 diff --git a/devcapsule-src/devcapsule/launch/pycharm/_launcher.py b/devcapsule-src/devcapsule/launch/pycharm/_launcher.py index c2af6f1f..e8d6fabb 100644 --- a/devcapsule-src/devcapsule/launch/pycharm/_launcher.py +++ b/devcapsule-src/devcapsule/launch/pycharm/_launcher.py @@ -17,6 +17,7 @@ from threading import Event import time from dataclasses import dataclass, field, replace +from datetime import datetime, timedelta, timezone from enum import Enum from pathlib import Path from typing import Callable, Mapping, Sequence @@ -58,9 +59,9 @@ print_only_opener, watch_display_ready, ) -from ... import host_daemon from ...host_daemon import ( current_container, + daemon_architecture, docker_socket, is_host_backed, requires_translation, @@ -138,6 +139,8 @@ class TempRuntimeFiles: SUDOERS_VOLUME_FILE = "policy" """The policy's name inside its macOS volume.""" SUDOERS_VOLUME_LABEL = "devcapsule.purpose=development-sudo-policy" +ABANDONED_SUDOERS_VOLUME_AGE = timedelta(days=1) +"""How old a policy volume no container uses must be to count as a killed launch's.""" @@ -174,6 +177,8 @@ def web_console_notice(availability: WebConsoleAvailability) -> str: @dataclass class PycharmRunOptions: project: Path + # The lock's platform: what the capsule runs as. + capsule_platform: Platform # Project configuration is complete; only the legacy command inherits # ambient launcher options. Host observations and explicit secret sources # remain available in either mode. @@ -237,8 +242,6 @@ class PycharmRunOptions: # caller (commands/project.py); without it the launcher announces the # absence once and plans no console: no port, no token, no wait. web_console: WebConsoleAvailability = WebConsoleAvailability(True, None) - # The lock's platform; without a lock, the one this host creates projects for. - capsule_platform: Platform | None = None @dataclass @@ -276,6 +279,9 @@ class PycharmRunConfig: memory_limit_bytes: int | None runtime_plan: RuntimePlan | None use_image_process: bool + capsule_platform: Platform + # The Docker daemon's, asked once per launch: it decides emulation. + daemon_architecture: str cpus: int | None = None launch_configuration: LaunchConfiguration | None = None host_browser_socket: Path | None = None @@ -292,8 +298,6 @@ class PycharmRunConfig: console_token: str = "" open_console_url: Callable[[str], None] | None = None web_console: WebConsoleAvailability = WebConsoleAvailability(True, None) - # The lock's platform; without a lock, the one this host creates projects for. - capsule_platform: Platform = field(default_factory=Platform.current) # This session's log directory on the host, bound into the capsule at # SESSION_LOG_DESTINATION: the launcher's own log, the supervisor's, and # one file per supervised child (see devcapsule.logs). @@ -311,7 +315,7 @@ class PycharmRunConfig: HOST_NETWORK_UNAVAILABLE_ON_MACOS = ( - "devcapsule: warning: host networking is unavailable on macOS: Docker Desktop's host network " + "Warning: host networking is unavailable on macOS: Docker Desktop's host network " "is its virtual machine's, out of the Mac's reach. This launch uses bridge networking; " "services on the Mac are reachable from the capsule as host.docker.internal." ) @@ -1194,7 +1198,8 @@ def build_run_config(options: PycharmRunOptions, env: Mapping[str, str]) -> Pych console_token=console_token, open_console_url=options.open_console_url, web_console=options.web_console, - capsule_platform=options.capsule_platform or Platform.current(), + capsule_platform=options.capsule_platform, + daemon_architecture=daemon_architecture(), interactive_state_mounts=tuple(interactive_state_mounts), additional_state_mounts=tuple(additional_state_mounts), additional_environment=tuple(additional_environment), @@ -1324,7 +1329,7 @@ def build_docker_args( args = [ "--rm", "-i", - *config.capsule_platform.docker_platform_args(host_daemon.daemon_architecture()), + *config.capsule_platform.docker_platform_args(config.daemon_architecture), "--name", config.name, "--workdir", @@ -1889,7 +1894,7 @@ def ensure_sudoers_policy_ownership( "run", "--rm", "--pull=never", - *config.capsule_platform.docker_platform_args(host_daemon.daemon_architecture()), + *config.capsule_platform.docker_platform_args(config.daemon_architecture), "--network", "none", "--read-only", @@ -1938,16 +1943,12 @@ def prepare_sudoers_policy_volume(config: PycharmRunConfig, policy: Path, env: M with its virtual machine as the Mac user: a bind-mounted file is owned by that user inside every container, and not even root there can chown it. A volume lives on the virtual machine's own disk, where root's ownership - holds. The session removes it at its end; one a killed or interrupted - launch left behind carries the label, and the next launch prunes every - such volume no container uses before it creates its own. + holds. The session removes it at its end; the next launch removes one a + killed launch left behind (``remove_abandoned_sudoers_volumes``). """ volume = f"{config.name}-sudoers" - subprocess.run( - ["docker", "volume", "prune", "--all", "--force", "--filter", f"label={SUDOERS_VOLUME_LABEL}"], - check=False, env=dict(env), stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, - ) + remove_abandoned_sudoers_volumes(env) created = subprocess.run( ["docker", "volume", "create", "--label", SUDOERS_VOLUME_LABEL, volume], check=False, env=dict(env), stdout=subprocess.DEVNULL, stderr=subprocess.PIPE, text=True, @@ -1964,7 +1965,7 @@ def prepare_sudoers_policy_volume(config: PycharmRunConfig, policy: Path, env: M "--rm", "-i", "--pull=never", - *config.capsule_platform.docker_platform_args(host_daemon.daemon_architecture()), + *config.capsule_platform.docker_platform_args(config.daemon_architecture), "--network", "none", "--read-only", @@ -2004,8 +2005,44 @@ def prepare_sudoers_policy_volume(config: PycharmRunConfig, policy: Path, env: M return volume +def remove_abandoned_sudoers_volumes(env: Mapping[str, str]) -> None: + """Remove the policy volumes of launches that ended without removing their own. + + A concurrent launch's volume is in no container's use between its + creation and its capsule's start, so only a volume older than + ``ABANDONED_SUDOERS_VOLUME_AGE`` is taken for abandoned; Docker refuses + to remove one a container still uses. + """ + + listed = subprocess.run( + ["docker", "volume", "ls", "--quiet", "--filter", f"label={SUDOERS_VOLUME_LABEL}"], + check=False, env=dict(env), capture_output=True, text=True, + ) + volumes = listed.stdout.split() if listed.returncode == 0 else [] + if not volumes: + return + inspected = subprocess.run( + ["docker", "volume", "inspect", "--format", "{{.Name}} {{.CreatedAt}}", *volumes], + check=False, env=dict(env), capture_output=True, text=True, + ) + cutoff = datetime.now(timezone.utc) - ABANDONED_SUDOERS_VOLUME_AGE + abandoned = [] + for line in inspected.stdout.splitlines(): + volume, _, created = line.partition(" ") + try: + if datetime.fromisoformat(created) < cutoff: + abandoned.append(volume) + except ValueError: + continue # A creation time Docker did not report is no evidence of age. + if abandoned: + subprocess.run( + ["docker", "volume", "rm", *abandoned], + check=False, env=dict(env), stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, + ) + + def remove_sudoers_policy_volume(volume: str | None, env: Mapping[str, str]) -> None: - """Remove the policy volume; a capsule still using it keeps it until the next launch.""" + """Remove the policy volume; one a capsule still uses is left for a later launch to find abandoned.""" if volume: subprocess.run( diff --git a/devcapsule-src/devcapsule/platforms.py b/devcapsule-src/devcapsule/platforms.py index a49fa077..da126116 100644 --- a/devcapsule-src/devcapsule/platforms.py +++ b/devcapsule-src/devcapsule/platforms.py @@ -114,10 +114,9 @@ def chown_reaches_bind_mounts(self) -> bool: } # The platform a host creates a new project for, where it is not the native -# one. Apple Silicon creates linux-amd64 projects, as before linux-arm64 -# existed, while the linux-arm64 base is unpublished: the human's ruling of -# 2026-11-04 runs a project as linux-arm64 when it is locked for it, and a -# developer adds that lock deliberately (project --platform linux-arm64 init). +# one. Apple Silicon creates linux-amd64 projects while the linux-arm64 base +# is unpublished; a developer adds a linux-arm64 lock deliberately +# (project init --platform linux-arm64), and the project then runs as it. _NEW_PROJECT_PLATFORM_OF_HOST: Mapping[tuple[HostOs, str], str] = { (HostOs.DARWIN, "arm64"): "linux-amd64", } @@ -184,19 +183,21 @@ def docker_platform(self) -> str: return self.value.replace("-", "/", 1) - def emulated(self, daemon_architecture: str) -> bool: - """Whether a Docker daemon of ``daemon_architecture`` runs this platform only under emulation.""" + def emulated_docker_platform(self, daemon_architecture: str) -> str | None: + """The platform every run, build and pull names to a daemon that emulates this one. - return daemon_architecture != self.architecture + A Docker daemon of ``daemon_architecture`` defaults to its own + architecture, so it must be told this platform when the two differ. + None when it runs this platform natively and is told nothing. + """ - def docker_platform_args(self, daemon_architecture: str) -> list[str]: - """The ``--platform`` option every run, build and pull carries under emulation. + return self.docker_platform if daemon_architecture != self.architecture else None - Empty when the daemon runs this platform natively, so a native launch - composes the same command as before the option existed. - """ + def docker_platform_args(self, daemon_architecture: str) -> list[str]: + """``emulated_docker_platform`` as the option of a ``docker run`` command line.""" - return [f"--platform={self.docker_platform}"] if self.emulated(daemon_architecture) else [] + emulated = self.emulated_docker_platform(daemon_architecture) + return [f"--platform={emulated}"] if emulated is not None else [] @classmethod def parse(cls, key: str) -> Platform: diff --git a/devcapsule-src/devcapsule/recursive_successor.py b/devcapsule-src/devcapsule/recursive_successor.py index 30f68ed8..b07c9ee3 100644 --- a/devcapsule-src/devcapsule/recursive_successor.py +++ b/devcapsule-src/devcapsule/recursive_successor.py @@ -25,6 +25,7 @@ from devcapsule.display_client import display_url, select_display_transport from devcapsule.environment_realization import realize_environment from devcapsule.materialization import RUNTIME_PLAN_PATH as SUCCESSOR_RUNTIME_PLAN_PATH +from devcapsule.configuration.capability_selection import lock_platform from devcapsule.configuration.storage import ( atomic_write, ) @@ -505,6 +506,7 @@ def _resolved_run_options( ) return PycharmRunOptions( project=selected.root, + capsule_platform=lock_platform(selected.lock), project_mount=str(runtime["project-mount"]), image=image, name=name, diff --git a/devcapsule-src/devcapsule/recursive_successor_plan.py b/devcapsule-src/devcapsule/recursive_successor_plan.py index 7c808d7b..1c1b9796 100644 --- a/devcapsule-src/devcapsule/recursive_successor_plan.py +++ b/devcapsule-src/devcapsule/recursive_successor_plan.py @@ -115,7 +115,7 @@ class ExpectedSuccessorPlan: platform: str | None = None def to_mapping(self, *, show_host_paths: bool = False) -> dict[str, Any]: - return { + mapping = { "schema_version": 1, "name": self.name, "image_reference": self.image_reference, @@ -143,6 +143,11 @@ def to_mapping(self, *, show_host_paths: bool = False) -> dict[str, Any]: "runtime_plan_destination": self.runtime_plan_destination, "runtime_plan_digest": self.runtime_plan_digest, } + # Only an emulated plan carries the key, so a native plan's digest is + # the one a launcher without emulation computed for it. + if self.platform is not None: + mapping["platform"] = self.platform + return mapping def to_json(self, *, show_host_paths: bool = False) -> str: return json.dumps( @@ -188,6 +193,7 @@ def from_mapping(cls, value: Mapping[str, Any]) -> ExpectedSuccessorPlan: security_opt=_string_tuple(value.get("security_opt")), runtime_plan_destination=_required_string(value, "runtime_plan_destination"), runtime_plan_digest=_required_string(value, "runtime_plan_digest"), + platform=_optional_string(value.get("platform"), "platform"), ) @classmethod @@ -584,6 +590,12 @@ def _required_bool(value: Mapping[str, Any], name: str) -> bool: return found +def _optional_string(value: Any, name: str) -> str | None: + if value is not None and (not isinstance(value, str) or not value): + raise SuccessorPlanError(f"retained expected plan has no usable {name}") + return value + + def _optional_int(value: Any, name: str) -> int | None: if value is None: return None diff --git a/devcapsule-src/devcapsule/resolution_matrix.py b/devcapsule-src/devcapsule/resolution_matrix.py index b99b06a6..05c1eea9 100644 --- a/devcapsule-src/devcapsule/resolution_matrix.py +++ b/devcapsule-src/devcapsule/resolution_matrix.py @@ -101,7 +101,7 @@ class Capability: @dataclass(frozen=True) class _BasePin: - """One published base image and the capabilities it ships.""" + """One base image and the capabilities it ships.""" mnemonic: str # The family of base releases this pin belongs to — see _VerifiedEdge: @@ -122,10 +122,10 @@ class _BasePin: display: str = HOST_X11_ONLY_DISPLAY runtime: str = LAUNCHER_SUPPLIED_RUNTIME source_revision: str | None = None + # A clean public-source builder need not have a release tag. # False while the reference names an image its registry does not hold # yet: only a locally built base of the same recipe stands in for it. published: bool = True - # A clean public-source builder need not have a release tag. @property def contract(self) -> BaseContract: @@ -1724,11 +1724,11 @@ def _formation( # the public main commit b3f10b7d, whose devcapsule-webconsole the image # installed. Inspected: linux/arm64, recipe 11 ready, display=contained, # runtime=launcher-supplied, all six services, the web console root. -# NOT YET PUBLISHED. The reference is the digest of the OCI image index in +# Not published yet. The reference is the digest of the OCI image index in # the builder's containerd store, which a push of this exact image to # mycodespaceai/devcapsule-base keeps; a rebuild gets another digest and # must be pinned again. Until the owner pushes it, a checkout selects the -# local image: `project config authorize base-image `. +# local image: `project config authorize base-image IMAGE`. _LINUX_ARM64_RECIPE_11_BASE = _BasePin( mnemonic="v0.3.1.dev0-local-macos-aarch64", source_revision="b3f10b7d95aec659f268afecc03d0912a2e89d2f", @@ -1752,6 +1752,7 @@ def _formation( }, ) + def _on_linux_arm64(pin: _ComponentPin) -> _ComponentPin: """The arm64 build of a linux-amd64 pin: same version, its own artifacts. @@ -1764,7 +1765,7 @@ def _on_linux_arm64(pin: _ComponentPin) -> _ComponentPin: if table is None and pin.lock_table.get("delivery-policy") == "base-image": return pin if table is None: - raise RuntimeError(f"{pin.component_id} {pin.version} has no linux-arm64 pin in components/linux_arm64_pins.py") + raise AssertionError(f"{pin.component_id} {pin.version} has no linux-arm64 pin in components/linux_arm64_pins.py") return _ComponentPin(pin.component_id, pin.version, table) @@ -1779,15 +1780,55 @@ def _on_linux_arm64(pin: _ComponentPin) -> _ComponentPin: component: tuple(_on_linux_arm64(pin) for pin in pins) for component, pins in _LINUX_AMD64_MATRIX._components.items() }, - edges=(), + edges=( + # Native IDE smokes on Apple Silicon (Docker Desktop 29.3.1) against + # the local, unpublished arm64 recipe-11 base above. They establish + # the IDE only: Playwright and the .NET SDK rode along unexercised. + _VerifiedEdge( + "codium", "1.143.17232", _BASE_FAMILY_UBUNTU_24_04, + "noVNC HTTP/window startup passed natively on the local arm64 recipe-11 base, " + "2026-11-04 run 20261104T120158Z-e8f78f; VSCodium window; no pixel, saved-edit " + "or persistence check", + ), + _VerifiedEdge( + "vscode", "1.144.0", _BASE_FAMILY_UBUNTU_24_04, + "noVNC HTTP/window startup passed natively on the local arm64 recipe-11 base, " + "2026-11-04 run 20261104T120217Z-ee18ce; Visual Studio Code window; no pixel, " + "saved-edit or persistence check", + ), + _VerifiedEdge( + "eclipse", "2026-09-R", _BASE_FAMILY_UBUNTU_24_04, + "noVNC HTTP/window startup passed natively on the local arm64 recipe-11 base, " + "2026-11-04 run 20261104T120256Z-aa127a; workspace window; no pixel, saved-edit " + "or persistence check", + ), + _VerifiedEdge( + "pycharm", "2026.2.4", _BASE_FAMILY_UBUNTU_24_04, + "noVNC HTTP/window startup passed natively on the local arm64 recipe-11 base, " + "2026-11-04 run 20261104T120351Z-38cdd0; splash window; first-use agreement and " + "editor acceptance remain", + ), + _VerifiedEdge( + "rider", "2026.2.4", _BASE_FAMILY_UBUNTU_24_04, + "noVNC HTTP/window startup passed natively on the local arm64 recipe-11 base, " + "2026-11-04 run 20261104T120546Z-c9c1b3; splash window; first-use agreement and " + "editor acceptance remain", + ), + _VerifiedEdge( + "intellij", "2026.2.4", _BASE_FAMILY_UBUNTU_24_04, + "noVNC HTTP/window startup passed natively on the local arm64 recipe-11 base, " + "2026-11-04 run 20261104T121007Z-289109; splash window; first-use agreement and " + "editor acceptance remain", + ), + ), couplings=(), capability_table=_LINUX_AMD64_MATRIX._capability_table, materialization=_LINUX_AMD64_MATRIX._materialization, ) # The map is the platform authority: its keys are the supported platforms, -# total over the Platform enum (D-0006/D-0007). Clients index it with a key -# from Platform.current() or Platform.parse(), never one built from parts. +# total over the Platform enum (D-0006/D-0007). Clients index it with a +# Platform member, from host detection or a lock, never a key built from parts. MATRICES: Mapping[Platform, ResolutionMatrix] = MappingProxyType( { Platform.LINUX_AMD64: _LINUX_AMD64_MATRIX, diff --git a/devcapsule-src/devcapsule/runtime_artifact.py b/devcapsule-src/devcapsule/runtime_artifact.py index 072f8f48..476a16c5 100644 --- a/devcapsule-src/devcapsule/runtime_artifact.py +++ b/devcapsule-src/devcapsule/runtime_artifact.py @@ -8,7 +8,6 @@ from devcapsule.build_info import BuildInfo, BuildInfoError, current_build_info, read_pex_build_info from devcapsule.compat import CliError -from devcapsule import platforms from devcapsule.platforms import HostOs, Platform RUNTIME_PEX_ENV = "DEVCAPSULE_RUNTIME_PEX" @@ -85,12 +84,13 @@ def _runs_in(path: Path, capsule_platform: Platform) -> bool: """Whether the executable at ``path`` runs inside ``capsule_platform`` capsules. A plain PEX bundles native wheels for the machine it was built on, so it - counts as the capsule platform's only on a Linux host of that architecture. + counts as the capsule platform's only on a Linux host, whose one capsule + platform is its own. """ runs_on = executable_platform(path) if runs_on is None: - return HostOs.current() is HostOs.LINUX and platforms.host_architecture() == capsule_platform.architecture + return HostOs.current() is HostOs.LINUX and Platform.runnable() == (capsule_platform,) return runs_on == capsule_platform diff --git a/devcapsule-src/devcapsule/version_sets.py b/devcapsule-src/devcapsule/version_sets.py index 101e1c3a..4d1ae252 100644 --- a/devcapsule-src/devcapsule/version_sets.py +++ b/devcapsule-src/devcapsule/version_sets.py @@ -23,7 +23,7 @@ from devcapsule.configuration.file_formats import canonical_digest, render_toml, selected_version_lock from devcapsule.configuration.model import Configuration from devcapsule.configuration.capabilities import CapabilityPolicy, LocalCapabilities -from devcapsule.configuration.capability_selection import usable_lock +from devcapsule.configuration.capability_selection import matrix_for, usable_lock from devcapsule.configuration.trust import HomeTrust from devcapsule.configuration.storage import ( load_settings, @@ -33,7 +33,6 @@ from devcapsule.environment_realization import RealizedEnvironment, realize_environment from devcapsule.materialization import ArtifactSpec, acquire_artifact, cache_root, parse_locked_environment, sha256_file from devcapsule.platforms import Platform, XdgHomes -from devcapsule.resolution_matrix import MATRICES from devcapsule import runtime_configuration from devcapsule.build_info import current_build_info from devcapsule.component_status import CompatibilityLookup @@ -137,7 +136,7 @@ def _validation(lock: Mapping[str, Any], checkout: Mapping[str, Any]) -> tuple[t missing = tuple(f"{name} {metadata.get('version')} on local base {local_base}" for name, metadata in lock["components"].items() if isinstance(metadata, dict)) return (), missing - return MATRICES[Platform.parse(str(lock.get("platform")))].validation_evidence(lock) + return matrix_for(lock).validation_evidence(lock) INSPECTION_SCHEMA_VERSION = 1 @@ -410,7 +409,7 @@ def preview(start: Path, component: str, version: str, *, report: Callable[[str] if channel is None: raise CliError(f"{component}: {definition.channel_omission_reason() or 'no distribution channel declared'}") selection = channel.select(version, str(workspace.lock["platform"])) - matrix = MATRICES[Platform.parse(str(workspace.lock["platform"]))] + matrix = matrix_for(workspace.lock) if workspace.lock["platform"] not in selection.platforms: raise CliError("Candidate does not support the selected platform.") if selection.base_families and matrix.base_family(workspace.lock) not in selection.base_families: diff --git a/devcapsule-src/noxfile.py b/devcapsule-src/noxfile.py index 743e2d4d..72849429 100644 --- a/devcapsule-src/noxfile.py +++ b/devcapsule-src/noxfile.py @@ -457,6 +457,7 @@ def ide_smoke(session: nox.Session) -> None: parser.add_argument("--max-actions", type=int, default=30) parser.add_argument("--agent-timeout", type=int, default=900) parser.add_argument("--surface", action="append", default=[], help="limit to a surface name; repeatable") + parser.add_argument("--platform", help="create each project for this capsule platform, such as linux-arm64") parser.add_argument("--required-ide", action="store_true", help="instead of the surfaces: a project requiring PyCharm launches Codium once with " "'run --ide codium --force', then PyCharm again plainly (work order of 2026-10-28)") @@ -472,6 +473,8 @@ def ide_smoke(session: nox.Session) -> None: session.error("--scenario cannot be combined with --agent: the scenario leaves the IDE in its final state") if len(options.surface) != 1: session.error("--scenario requires exactly one --surface, the scenario's own") + if options.platform: + session.env["DEVCAPSULE_SMOKE_PLATFORM"] = options.platform if options.relaunch: if not options.agent or options.surface not in (["intellij"], ["rider"]): session.error("--relaunch requires --agent --surface intellij or rider") diff --git a/devcapsule-src/tests/configuration/test_contract.py b/devcapsule-src/tests/configuration/test_contract.py index 74d1b78c..e1eeb182 100644 --- a/devcapsule-src/tests/configuration/test_contract.py +++ b/devcapsule-src/tests/configuration/test_contract.py @@ -44,6 +44,7 @@ replace_answers, ) from devcapsule.launch.pycharm import DockerMode, PycharmRunOptions, build_run_config +from devcapsule.platforms import Platform from tests.test_pycharm import base_env pytestmark = pytest.mark.legacy @@ -299,7 +300,7 @@ def test_launcher_environment_cannot_override_explicit_sudo_denial(checkout, tmp # ProjectRunCommand passes enable_sudo=False for the persisted denial. # Check the NEXT boundary, rather than replacing it with a launch mock. config = build_run_config( - PycharmRunOptions(project=project, docker_mode=DockerMode.none, enable_sudo=False), env, + PycharmRunOptions(project=project, capsule_platform=Platform.LINUX_AMD64, docker_mode=DockerMode.none, enable_sudo=False), env, ) assert config.enable_sudo is False diff --git a/devcapsule-src/tests/configuration/test_invariants.py b/devcapsule-src/tests/configuration/test_invariants.py index b598cc45..923859f2 100644 --- a/devcapsule-src/tests/configuration/test_invariants.py +++ b/devcapsule-src/tests/configuration/test_invariants.py @@ -43,6 +43,7 @@ initialize_project, ) from devcapsule.launch.pycharm import DockerMode, PycharmRunOptions, build_run_config +from devcapsule.platforms import Platform from tests.test_pycharm import base_env from tests.test_upgrade_recovery import checkout, invoke, install_external_fakes @@ -232,7 +233,7 @@ def test_project_launch_cannot_inherit_unbound_paths_or_credentials(checkout, tm 'DOCKER_MODE': 'invalid', }) config = build_run_config(PycharmRunOptions( - project=project, inherit_legacy_configuration=False, + project=project, capsule_platform=Platform.LINUX_AMD64, inherit_legacy_configuration=False, docker_mode=DockerMode.none, enable_sudo=False, network_mode='bridge', ), env) assert config.persistent_home != unbound and config.ide_config != unbound and config.plugins != unbound @@ -247,7 +248,7 @@ def test_explicit_sudo_decision_overrides_legacy_environment(value, tmp_path): project.mkdir() env = base_env(tmp_path) env['PYCHARM_ENABLE_SUDO'] = '1' - config = build_run_config(PycharmRunOptions(project=project, docker_mode=DockerMode.none, + config = build_run_config(PycharmRunOptions(project=project, capsule_platform=Platform.LINUX_AMD64, docker_mode=DockerMode.none, enable_sudo=value), env) assert config.enable_sudo is (True if value is None else value) diff --git a/devcapsule-src/tests/conftest.py b/devcapsule-src/tests/conftest.py index d4a5fab3..3fc4067b 100644 --- a/devcapsule-src/tests/conftest.py +++ b/devcapsule-src/tests/conftest.py @@ -1,7 +1,6 @@ from __future__ import annotations import os -from collections.abc import Iterator from pathlib import Path import shutil import sys @@ -60,41 +59,25 @@ def host_launch_by_default(monkeypatch: pytest.MonkeyPatch) -> None: @pytest.fixture(autouse=True) -def linux_host_by_default(request: pytest.FixtureRequest, monkeypatch: pytest.MonkeyPatch) -> None: - """Make host-mechanism tests take the Linux path wherever they run. +def amd64_linux_host_by_default(request: pytest.FixtureRequest, monkeypatch: pytest.MonkeyPatch) -> None: + """Make host-dependent tests run on an amd64 Linux host with an amd64 Docker daemon. The launcher takes macOS paths for the host-browser bridge, host - networking and the Docker socket's group. Tests of the Linux behaviour - would otherwise fail on a Mac; tests of the macOS behaviour patch - ``HostOs.current`` themselves. Docker-marked tests keep the real host. + networking, the Docker socket's group and the sudo policy, and an Apple + Silicon host runs linux-arm64 capsules natively. Tests of the CI's host + would otherwise depend on where they run; tests of another host patch it + themselves and carry ``own_host_patches``. Docker-marked tests keep the + real host. """ if any(request.node.get_closest_marker(marker) for marker in (*DOCKER_MARKERS, "own_host_patches")): return from devcapsule.platforms import HostOs - monkeypatch.setattr(HostOs, "current", classmethod(lambda cls: HostOs.LINUX)) - -@pytest.fixture(autouse=True) -def amd64_capsules_by_default(request: pytest.FixtureRequest, monkeypatch: pytest.MonkeyPatch) -> Iterator[None]: - """Make capsule-platform tests run on an amd64 host with an amd64 daemon. - - With ``linux_host_by_default`` the suite's host is then an amd64 Linux - machine wherever it runs: it creates and runs linux-amd64 capsules, and - its Docker daemon needs no ``--platform``, as on the Linux CI. Tests of - other hosts patch the host themselves and carry ``own_host_patches``; - Docker-marked tests keep the real host. No ``--platform`` choice outlives - its test. - """ - - from devcapsule.configuration import storage - - storage.name_platform(None) - if not any(request.node.get_closest_marker(marker) for marker in (*DOCKER_MARKERS, "own_host_patches")): - monkeypatch.setattr("devcapsule.platforms.host_architecture", lambda: "amd64") - monkeypatch.setattr("devcapsule.host_daemon.daemon_architecture", lambda: "amd64") - yield - storage.name_platform(None) + monkeypatch.setattr(HostOs, "current", classmethod(lambda cls: HostOs.LINUX)) + monkeypatch.setattr("devcapsule.platforms.host_architecture", lambda: "amd64") + for module in ("environment_realization", "images.build", "launch.pycharm._launcher"): + monkeypatch.setattr(f"devcapsule.{module}.daemon_architecture", lambda: "amd64") DOCKER_MARKERS = ("e2e", "base_build_e2e", "recursive_e2e", "contributor_e2e", "ide_smoke") diff --git a/devcapsule-src/tests/e2e/ide_session.py b/devcapsule-src/tests/e2e/ide_session.py index a036a994..9e2df352 100644 --- a/devcapsule-src/tests/e2e/ide_session.py +++ b/devcapsule-src/tests/e2e/ide_session.py @@ -183,7 +183,7 @@ def initialize_project(executable: Path, surface: IdeSurface, workspace: Path, e # such as linux-arm64 on Apple Silicon, whose new projects are linux-amd64. platform = os.environ.get("DEVCAPSULE_SMOKE_PLATFORM") init = subprocess.run( - [str(executable), "project", *(["--platform", platform] if platform else []), "init", + [str(executable), "project", "init", *(["--platform", platform] if platform else []), *(flag for need in surface.needs for flag in ("--ide" if need.endswith("-ide") else "--require", need)), "--creator", CREATOR, "--slug", slug, "--authorize", "base-image", "default", diff --git a/devcapsule-src/tests/e2e/test_ide_comes_alive.py b/devcapsule-src/tests/e2e/test_ide_comes_alive.py index d3566838..989c26fe 100644 --- a/devcapsule-src/tests/e2e/test_ide_comes_alive.py +++ b/devcapsule-src/tests/e2e/test_ide_comes_alive.py @@ -18,7 +18,6 @@ import pytest -from devcapsule.configuration.storage import project_platform from devcapsule.platforms import HostOs from devcapsule.runtime_artifact import RUNTIME_PEX_ENV, executable_platform from tests.e2e.ide_session import SURFACES, IdeSurface, capture_desktop, command, desktop_page_answers, ide_session, launched_ide, stop_session, wait_for_console_url, wait_for_ide_window @@ -163,13 +162,14 @@ def delivered_runtime(executable: Path, workspace: Path) -> Path: Derived independently of the launcher's choice: itself when it is a Linux executable; for a macOS launcher, the Linux runtime its environment names, - from a directory the one for the platform the workspace's locks select. + from a directory the one for the platform of the one lock the smoke's init wrote. """ if executable_platform(executable) is not HostOs.DARWIN: return executable named = Path(os.environ[RUNTIME_PEX_ENV]).expanduser().resolve() if named.is_dir(): - return named / f"devcapsule-{project_platform(workspace)}" + (lock,) = (workspace / ".devcapsule").glob("devcapsule.*.lock") + return named / f"devcapsule-{lock.name.split('.')[1]}" return named diff --git a/devcapsule-src/tests/test_base_image.py b/devcapsule-src/tests/test_base_image.py index d66be7ed..238311c9 100644 --- a/devcapsule-src/tests/test_base_image.py +++ b/devcapsule-src/tests/test_base_image.py @@ -20,6 +20,7 @@ from devcapsule.compat import CliError from devcapsule.images.build import render_build_context from devcapsule.images.tooling import MAVEN_VERSION, TEMURIN_VERSION +from devcapsule.platforms import Platform pytestmark = pytest.mark.legacy @@ -177,12 +178,13 @@ def test_base_image_build_forwards_host_network_to_buildx(tmp_path: Path) -> Non builder = Mock() source_verifier = Mock() - build_base_image(options, builder=builder, network="host", source_verifier=source_verifier) + build_base_image(options, builder=builder, platform=Platform.LINUX_AMD64, network="host", + source_verifier=source_verifier) source_verifier.assert_called_once() builder.build.assert_called_once() assert builder.build.call_args.args[0].image == "test-base:latest" - assert builder.build.call_args.kwargs == {"network": "host", "platform": None} + assert builder.build.call_args.kwargs == {"network": "host", "platform": Platform.LINUX_AMD64} def test_base_image_rejects_unreachable_revision_before_buildx(tmp_path: Path) -> None: @@ -192,7 +194,7 @@ def test_base_image_rejects_unreachable_revision_before_buildx(tmp_path: Path) - source_verifier = Mock(side_effect=CliError("source revision is not publicly reachable")) with pytest.raises(CliError, match="not publicly reachable"): - build_base_image(options, builder=builder, source_verifier=source_verifier) + build_base_image(options, builder=builder, platform=Platform.LINUX_AMD64, source_verifier=source_verifier) builder.build.assert_not_called() @@ -239,7 +241,8 @@ def test_console_base_rejects_missing_source_before_starting_the_builder(tmp_pat pex = pex_fixture(tmp_path / "devcapsule.pex", revision="unknown", public=False) builder = Mock() with pytest.raises(CliError, match="--allow-local-source cannot supply the console source"): - build_base_image(BaseImageBuildOptions(pex, allow_local_source=True), builder=builder) + build_base_image(BaseImageBuildOptions(pex, allow_local_source=True), builder=builder, + platform=Platform.LINUX_AMD64) builder.build.assert_not_called() @@ -251,6 +254,7 @@ def test_base_image_local_source_escape_hatch_skips_public_verification(tmp_path build_base_image( BaseImageBuildOptions(pex, allow_local_source=True), builder=builder, + platform=Platform.LINUX_AMD64, source_verifier=source_verifier, ) diff --git a/devcapsule-src/tests/test_cli.py b/devcapsule-src/tests/test_cli.py index 20ee5e50..e342f180 100644 --- a/devcapsule-src/tests/test_cli.py +++ b/devcapsule-src/tests/test_cli.py @@ -16,6 +16,7 @@ from devcapsule.launch.pycharm._image_build import PycharmImageBuildOptions from devcapsule.environment_realization import RealizedEnvironment from devcapsule.images.metadata import LocalImageRecord +from devcapsule.platforms import Platform from devcapsule.materialization import ImageDetails, parse_locked_environment from devcapsule.configuration.file_formats import ( canonical_digest, @@ -370,7 +371,7 @@ def test_images_build_base_maps_cli_options(tmp_path: Path, capsys) -> None: assert options.source_revision == "revision-1" assert options.allow_local_source is False assert options.recipe == "ubuntu-24.04" - assert build.call_args.kwargs == {"network": "host", "platform": None} + assert build.call_args.kwargs == {"network": "host", "platform": Platform.LINUX_AMD64} output = capsys.readouterr().out assert "Image ID: sha256:abc123" in output assert "Runtime: supplied by the launcher" in output diff --git a/devcapsule-src/tests/test_environment_realization.py b/devcapsule-src/tests/test_environment_realization.py index 2e4a9368..19494ae6 100644 --- a/devcapsule-src/tests/test_environment_realization.py +++ b/devcapsule-src/tests/test_environment_realization.py @@ -267,7 +267,6 @@ def test_realize_environment_allows_terms_bound_claude_download_authorization( def test_ensure_local_image_pulls_only_when_missing(already_local: bool) -> None: details = base_image() with ( - patch("devcapsule.platforms.host_architecture", return_value="amd64"), patch( "devcapsule.environment_realization.docker.image.exists", return_value=already_local, @@ -278,27 +277,27 @@ def test_ensure_local_image_pulls_only_when_missing(already_local: bool) -> None return_value=details, ) as require, ): - assert ensure_local_image(BASE_REFERENCE) == details + assert ensure_local_image(BASE_REFERENCE, Platform.LINUX_AMD64) == details if already_local: pull.assert_not_called() else: - pull.assert_called_once_with(BASE_REFERENCE) + pull.assert_called_once_with(BASE_REFERENCE, platform=None) require.assert_called_once_with(BASE_REFERENCE) -@pytest.mark.parametrize("daemon,expected", [("arm64", {"platform": "linux/amd64"}), ("amd64", {})]) -def test_a_daemon_of_another_architecture_pulls_the_lock_platform(daemon: str, expected: dict[str, str]) -> None: +@pytest.mark.parametrize("daemon,expected", [("arm64", "linux/amd64"), ("amd64", None)]) +def test_a_daemon_of_another_architecture_pulls_the_lock_platform(daemon: str, expected: str | None) -> None: # Docker Desktop on Apple Silicon is an arm64 daemon; a linux-amd64 lock's - # base must not resolve to its arm64 variant. A native daemon pulls as before. + # base must not resolve to its arm64 variant. with ( - patch("devcapsule.host_daemon.daemon_architecture", return_value=daemon), + patch("devcapsule.environment_realization.daemon_architecture", return_value=daemon), patch("devcapsule.environment_realization.docker.image.exists", return_value=False), patch("devcapsule.environment_realization.docker.image.pull") as pull, patch("devcapsule.environment_realization.required_local_image", return_value=base_image()), ): ensure_local_image(BASE_REFERENCE, Platform.LINUX_AMD64) - pull.assert_called_once_with(BASE_REFERENCE, **expected) + pull.assert_called_once_with(BASE_REFERENCE, platform=expected) def test_realize_environment_allows_explicit_managed_base_override(tmp_path: Path) -> None: diff --git a/devcapsule-src/tests/test_host_open.py b/devcapsule-src/tests/test_host_open.py index 417ffaa6..31707e76 100644 --- a/devcapsule-src/tests/test_host_open.py +++ b/devcapsule-src/tests/test_host_open.py @@ -5,11 +5,10 @@ from pathlib import Path import socket import subprocess +import sys from types import SimpleNamespace from unittest.mock import Mock, patch -import sys - import pytest from devcapsule import cli @@ -26,12 +25,8 @@ pytestmark = pytest.mark.legacy -# The broker reads its peer's credentials with SO_PEERCRED and runs only on -# Linux hosts; on macOS the bridge is unavailable before any broker exists. -linux_broker = pytest.mark.skipif(sys.platform != "linux", reason="the host-open broker runs only on Linux hosts") - -@linux_broker +@pytest.mark.skipif(sys.platform != "linux", reason="the broker reads SO_PEERCRED, which only Linux has") def test_broker_forwards_one_exact_url_without_shell_parsing(tmp_path: Path) -> None: path = tmp_path / "host-open.sock" run = Mock(return_value=SimpleNamespace(returncode=0)) @@ -97,7 +92,7 @@ def test_broker_rejects_malformed_protocol_without_invoking_opener(tmp_path: Pat run.assert_not_called() -@linux_broker +@pytest.mark.skipif(sys.platform != "linux", reason="the broker reads SO_PEERCRED, which only Linux has") def test_broker_applies_a_bounded_request_rate(tmp_path: Path) -> None: path = tmp_path / "host-open.sock" run = Mock(return_value=SimpleNamespace(returncode=0)) @@ -129,7 +124,7 @@ def test_broker_rejects_a_different_peer_uid(tmp_path: Path) -> None: run.assert_not_called() -@linux_broker +@pytest.mark.skipif(sys.platform != "linux", reason="the broker reads SO_PEERCRED, which only Linux has") def test_broker_reports_host_opener_timeout(tmp_path: Path) -> None: path = tmp_path / "host-open.sock" run = Mock(side_effect=subprocess.TimeoutExpired(["xdg-open"], 0.1)) @@ -142,7 +137,7 @@ def test_broker_reports_host_opener_timeout(tmp_path: Path) -> None: open_host_url("https://example.test", socket_path=path) -@linux_broker +@pytest.mark.skipif(sys.platform != "linux", reason="the broker reads SO_PEERCRED, which only Linux has") def test_client_reports_missing_and_failed_brokers_without_exposing_url(tmp_path: Path) -> None: url = "https://example.test/path?secret=do-not-report" with pytest.raises(HostOpenError, match="absent") as missing: @@ -180,7 +175,7 @@ def test_bridge_reuses_only_an_explicit_inherited_socket(tmp_path: Path) -> None assert disabled is None -@linux_broker +@pytest.mark.skipif(sys.platform != "linux", reason="the broker reads SO_PEERCRED, which only Linux has") def test_physical_host_bridge_owns_and_cleans_broker(tmp_path: Path) -> None: run = Mock(return_value=SimpleNamespace(returncode=0)) env = {"XDG_RUNTIME_DIR": str(tmp_path), "DISPLAY": ":1"} @@ -212,7 +207,7 @@ def test_macos_host_bridge_is_unavailable_with_a_warning(tmp_path: Path, capsys) assert list(tmp_path.iterdir()) == [] -@linux_broker +@pytest.mark.skipif(sys.platform != "linux", reason="the broker reads SO_PEERCRED, which only Linux has") def test_hidden_host_open_command_uses_authorized_environment(tmp_path: Path, capsys) -> None: assert "host-open" not in cli.command_names() assert "host-open" in cli.command_names(include_hidden=True) diff --git a/devcapsule-src/tests/test_image_build.py b/devcapsule-src/tests/test_image_build.py index f52260a3..51d1466b 100644 --- a/devcapsule-src/tests/test_image_build.py +++ b/devcapsule-src/tests/test_image_build.py @@ -12,6 +12,7 @@ parse_pycharm_build_options, ) from devcapsule.images.build import BuildxImageBuilder, ImageBuildSpec, normalize_pycharm_source, render_build_context +from devcapsule.platforms import Platform pytestmark = pytest.mark.legacy @@ -137,9 +138,9 @@ def test_buildx_builder_reuses_one_context_root_and_attaches_named_contexts(tmp_ spec = ImageBuildSpec(image="result:test", base_image="sha256:base", components=(DirectoryComponent(tree, "/opt/tree"),)) with patch("devcapsule.images.build.docker.build") as build: - BuildxImageBuilder(context_root=context_root).build(spec, network="none") + BuildxImageBuilder(context_root=context_root).build(spec, platform=Platform.LINUX_AMD64, network="none") (context_root / "stale-from-last-build").write_text("x") - BuildxImageBuilder(context_root=context_root).build(spec, network="host") + BuildxImageBuilder(context_root=context_root).build(spec, platform=Platform.LINUX_AMD64, network="host") first, second = build.call_args_list assert first.args[0] == context_root and second.args[0] == context_root assert first.kwargs["build_contexts"] == {"copy-dir-0": str(tree)} @@ -152,7 +153,8 @@ def test_buildx_builder_reuses_one_context_root_and_attaches_named_contexts(tmp_ def test_buildx_builder_without_a_root_uses_a_temporary_context(tmp_path: Path) -> None: with patch("devcapsule.images.build.docker.build") as build: - BuildxImageBuilder().build(ImageBuildSpec(image="result:test", base_image="sha256:base"), network="none") + BuildxImageBuilder().build(ImageBuildSpec(image="result:test", base_image="sha256:base"), + platform=Platform.LINUX_AMD64, network="none") used_context = build.call_args.args[0] assert used_context.name.startswith("devcapsule-buildx-context-") and not used_context.exists() assert build.call_args.kwargs["build_contexts"] == {} @@ -195,9 +197,9 @@ def test_reused_context_preserves_sources_and_recovers_after_build_failure(tmp_p spec = ImageBuildSpec("result:test", "base:test", (DirectoryComponent(source, "/opt/tool"),)) with patch("devcapsule.images.build.docker.build", side_effect=OSError("build failed")): with pytest.raises(CliError, match="build failed"): - BuildxImageBuilder(context).build(spec) + BuildxImageBuilder(context).build(spec, platform=Platform.LINUX_AMD64) with patch("devcapsule.images.build.docker.build") as build: - BuildxImageBuilder(context).build(spec) + BuildxImageBuilder(context).build(spec, platform=Platform.LINUX_AMD64) assert build.call_count == 1 assert (source / "payload").read_text() == "keep" assert sorted(path.name for path in context.iterdir()) == ["Dockerfile"] diff --git a/devcapsule-src/tests/test_linux_arm64_matrix.py b/devcapsule-src/tests/test_linux_arm64_matrix.py index f1ceb18a..8f342328 100644 --- a/devcapsule-src/tests/test_linux_arm64_matrix.py +++ b/devcapsule-src/tests/test_linux_arm64_matrix.py @@ -6,6 +6,11 @@ import re from typing import Any +import pytest + +from devcapsule.components.linux_arm64_pins import COMPONENT_LOCK_TABLES +from devcapsule.configuration.capability_selection import pins_for_platform +from devcapsule.configuration.file_formats import ProjectConfigurationError from devcapsule.platforms import Platform from devcapsule.resolution_matrix import MATRICES @@ -14,27 +19,6 @@ AMD64_MARKERS = re.compile(r"x86[_-]64|[_-]x64|amd64|linux64/|cft/[^/]+/linux64|ffmpeg-linux\.zip") -def _urls(value: Any) -> list[str]: - if isinstance(value, dict): - return [url for key, item in value.items() for url in ([item] if key == "url" else _urls(item))] - if isinstance(value, (list, tuple)): - return [url for item in value for url in _urls(item)] - return [] - - -def test_arm64_pins_the_same_component_versions_as_amd64() -> None: - assert {component: [pin.version for pin in pins] for component, pins in ARM64._components.items()} == { - component: [pin.version for pin in pins] for component, pins in AMD64._components.items() - } - - -def test_no_arm64_pin_names_an_amd64_artifact() -> None: - for component, pins in ARM64._components.items(): - for pin in pins: - for url in _urls(pin.lock_table): - assert not AMD64_MARKERS.search(url), (component, pin.version, url) - - def _artifacts(value: Any) -> list[dict[str, Any]]: """Every table at any depth that names a download by ``url``.""" if isinstance(value, dict): @@ -45,43 +29,51 @@ def _artifacts(value: Any) -> list[dict[str, Any]]: return [] +def test_every_arm64_pin_is_a_version_amd64_pins_and_the_matrix_serves() -> None: + # The converse, an amd64 version without an arm64 pin, fails at import. + for (component, version), table in COMPONENT_LOCK_TABLES.items(): + assert AMD64.component_pin(component, version) is not None, (component, version) + assert ARM64.component_pin(component, version) == table, (component, version) + + +def test_no_arm64_pin_names_an_amd64_artifact() -> None: + for key, table in COMPONENT_LOCK_TABLES.items(): + for artifact in _artifacts(table): + assert not AMD64_MARKERS.search(str(artifact["url"])), (key, artifact["url"]) + + def test_every_arm64_artifact_is_hash_pinned() -> None: - for component, pins in ARM64._components.items(): - for pin in pins: - for artifact in _artifacts(pin.lock_table): - assert re.fullmatch(r"[0-9a-f]{64}", str(artifact.get("sha256"))), (component, pin.version, artifact) + for key, table in COMPONENT_LOCK_TABLES.items(): + for artifact in _artifacts(table): + assert re.fullmatch(r"[0-9a-f]{64}", str(artifact.get("sha256"))), (key, artifact) def test_arm64_artifacts_are_not_amd64_bytes() -> None: # Architecture-independent files are shared: Codex's JavaScript package, # pure-Python wheels, and Ubuntu _all packages. Everything else differs. - shared = {artifact["sha256"] for pins in AMD64._components.values() for pin in pins - for artifact in _artifacts(pin.lock_table)} - for component, pins in ARM64._components.items(): - for pin in pins: - for artifact in _artifacts(pin.lock_table): - url = str(artifact["url"]) - independent = url.endswith(("_all.deb", "-none-any.whl")) or url.endswith(f"codex-{pin.version}.tgz") - if not independent: - assert artifact["sha256"] not in shared, (component, pin.version, url) + shared = {artifact["sha256"] for (component, version) in COMPONENT_LOCK_TABLES + for artifact in _artifacts(AMD64.component_pin(component, version))} + for (component, version), table in COMPONENT_LOCK_TABLES.items(): + for artifact in _artifacts(table): + url = str(artifact["url"]) + independent = url.endswith(("_all.deb", "-none-any.whl")) or url.endswith(f"codex-{version}.tgz") + if not independent: + assert artifact["sha256"] not in shared, (component, version, url) def test_platform_fields_and_artifact_keys_name_arm64() -> None: - for component, pins in ARM64._components.items(): - for pin in pins: - table = pin.lock_table - if "platform" in table: - assert table["platform"] == "linux-arm64", component - if component in {"codex", "claude-code", "antigravity-cli"}: - assert list(table["artifacts"]) == ["linux-arm64"], component - assert "linux-amd64" not in json.dumps([pin.lock_table for pins in ARM64._components.values() for pin in pins]) + for (component, _version), table in COMPONENT_LOCK_TABLES.items(): + if "platform" in table: + assert table["platform"] == "linux-arm64", component + if component in {"codex", "claude-code", "antigravity-cli"}: + assert list(table["artifacts"]) == ["linux-arm64"], component + assert "linux-amd64" not in json.dumps(list(COMPONENT_LOCK_TABLES.values())) def test_eclipse_native_closure_is_arm64_or_architecture_independent() -> None: - (eclipse,) = ARM64._components["eclipse"] - packages = eclipse.lock_table["native-packages"] + packages = ARM64.component_pin("eclipse", "2026-09-R")["native-packages"] # The same packages, plus what arm64's own dependencies add. - assert set(packages) - set(AMD64._components["eclipse"][0].lock_table["native-packages"]) == {"libunwind8"} + assert set(packages) - set(AMD64.component_pin("eclipse", "2026-09-R")["native-packages"]) == {"libunwind8"} for name, entry in packages.items(): # Architecture-independent packages keep the amd64 closure's bytes. if entry["url"].endswith("_all.deb"): @@ -95,6 +87,13 @@ def test_arm64_base_is_its_own_unpublished_recipe_11_pin() -> None: assert base.contract.recipe == 11 assert base.reference.startswith("docker.io/mycodespaceai/devcapsule-base@sha256:") assert base.reference not in {image.reference for image in AMD64.base_images()} + assert ARM64.base_unpublished(base.reference) + assert not any(AMD64.base_unpublished(image.reference) for image in AMD64.base_images()) + + +@pytest.mark.parametrize("ide", ["frontend-ide", "vscode-ide", "eclipse-ide", "python-ide", "java-ide"]) +def test_each_ide_smoked_natively_on_arm64_resolves_verified(ide: str) -> None: + assert ARM64.resolve([ide]).unverified == () def _personal_pins(platform: str, component: str, matrix: Any) -> dict[str, Any]: @@ -105,8 +104,6 @@ def _personal_pins(platform: str, component: str, matrix: Any) -> dict[str, Any] def test_personal_pins_take_the_artifacts_of_the_platform_they_run_on() -> None: # A checkout chose codium while the project ran as linux-amd64; the # project then gained an arm64 lock. Its pins must deliver arm64 bytes. - from devcapsule.configuration.capability_selection import pins_for_platform - amd64 = _personal_pins("linux-amd64", "codium", AMD64) on_arm64 = pins_for_platform(amd64, ARM64) assert on_arm64["platform"] == "linux-arm64" @@ -114,7 +111,7 @@ def test_personal_pins_take_the_artifacts_of_the_platform_they_run_on() -> None: assert on_arm64["components"]["interactive-surface"] == "codium" assert amd64["components"]["codium"] == dict(AMD64.component_pins()["codium"]) # input unchanged # A table that is another platform's embedded pin is that platform's, - # whatever the label says: pins written wrongly by an earlier build heal. + # whatever the label says, so a mislabelled record takes arm64's bytes too. mislabelled = dict(amd64, platform="linux-arm64") assert pins_for_platform(mislabelled, ARM64)["components"]["codium"] == dict(ARM64.component_pins()["codium"]) # Pins written for the platform they run on are records, returned as they stand. @@ -124,11 +121,6 @@ def test_personal_pins_take_the_artifacts_of_the_platform_they_run_on() -> None: def test_a_personal_version_the_other_platform_lacks_asks_for_a_new_choice() -> None: - import pytest - - from devcapsule.configuration.capability_selection import pins_for_platform - from devcapsule.configuration.storage import ProjectConfigurationError - channel = {"platform": "linux-amd64", "components": {"codex": {"version": "9.9.9", "url": "https://example.test/x64.tgz"}}} with pytest.raises(ProjectConfigurationError, match="codex 9.9.9 is pinned for linux-amd64.*linux-arm64 has no pin"): pins_for_platform(channel, ARM64) diff --git a/devcapsule-src/tests/test_owner_commands.py b/devcapsule-src/tests/test_owner_commands.py index 27125725..d1b1d298 100644 --- a/devcapsule-src/tests/test_owner_commands.py +++ b/devcapsule-src/tests/test_owner_commands.py @@ -11,13 +11,14 @@ from devcapsule import cli from devcapsule.configuration import owner_commands -from devcapsule.configuration.capability_commands import paths +from devcapsule.configuration.capability_commands import paths, recover from devcapsule.configuration.file_formats import ProjectConfigurationError, render_toml from devcapsule.configuration.manifest_document import render_manifest from devcapsule.configuration.storage import load_toml from devcapsule.configuration.storage import checkout_record_paths from devcapsule.platforms import Platform from devcapsule.resolution_matrix import MATRICES +from tests.test_platforms import on_host pytestmark = pytest.mark.legacy @@ -423,3 +424,57 @@ def test_sources_survive_other_owner_edits_and_do_not_change_pending_decisions(p {"kind": "source", "name": "notes", "effective": "no toolchain declared", "source": "project: upstream, read-only", "pending": "", "command": ""}, {"kind": "source", "name": "vendor/robot.code", "effective": "java 17", "source": "project: fork of https://example.test/up", "pending": "", "command": ""}, ] + + +@pytest.fixture +def two_platform_project(project: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """The project on Apple Silicon, with a linux-arm64 lock added beside its linux-amd64 one.""" + on_host(monkeypatch, "Darwin", "arm64") + assert cli.main(["project", "--path", str(project), "init", "--platform", "linux-arm64"]) == 0 + return project + + +def _locked_components(root: Path) -> dict[str, set[str]]: + return {path.name: set(load_toml(path).get("components", {})) + for path in sorted((root / ".devcapsule").glob("devcapsule.*.lock"))} + + +@pytest.mark.own_host_patches +def test_an_owner_edit_regenerates_every_platform_lock(two_platform_project: Path, capsys) -> None: + # Edited on the Mac, the committed linux-amd64 lock follows the manifest too. + assert cli.main(["project", "--path", str(two_platform_project), "config", "require", "dotnet", "--unverified"]) == 0 + assert "the manifest and its locks are updated" in capsys.readouterr().out + assert _locked_components(two_platform_project) == { + "devcapsule.linux-amd64.lock": {"dotnet-sdk"}, "devcapsule.linux-arm64.lock": {"dotnet-sdk"}, + } + + +@pytest.mark.own_host_patches +def test_an_edit_one_lock_cannot_follow_changes_nothing(two_platform_project: Path) -> None: + # The .NET SDK has no linux-arm64 validation yet; linux-amd64 alone would accept it. + before = {path: path.read_bytes() for path in (two_platform_project / ".devcapsule").iterdir() if path.is_file()} + with pytest.raises(ProjectConfigurationError, match="linux-arm64 lock cannot follow this edit"): + owner_commands.require_tools(two_platform_project, ["dotnet"]) + assert {path: path.read_bytes() for path in before} == before + + +@pytest.mark.own_host_patches +@pytest.mark.parametrize("committed", [False, True]) +def test_an_interrupted_edit_recovers_every_lock_to_one_side(two_platform_project: Path, committed: bool) -> None: + devcapsule = two_platform_project / ".devcapsule" + manifest, locks = paths(two_platform_project)[0], sorted(devcapsule.glob("devcapsule.*.lock")) + before = {path: path.read_text() for path in (manifest, *locks)} + owner_commands.require_tools(two_platform_project, ["dotnet"], allow_unverified=True) + after = {path: path.read_text() for path in (manifest, *locks)} + assert all(before[path] != after[path] for path in locks) + journal = {"before-manifest": before[manifest], "after-manifest": after[manifest], + "before-locks": {path.name.split(".")[1]: before[path] for path in locks}, + "after-locks": {path.name.split(".")[1]: after[path] for path in locks}} + paths(two_platform_project)[2].write_text(render_toml(journal)) + if not committed: + manifest.write_text(before[manifest]) + locks[0].write_text(before[locks[0]]) # Interrupted between the two lock writes. + recover(two_platform_project) + expected = after if committed else before + assert {path: path.read_text() for path in (manifest, *locks)} == expected + assert not paths(two_platform_project)[2].exists() diff --git a/devcapsule-src/tests/test_platforms.py b/devcapsule-src/tests/test_platforms.py index 0df3b333..92a47be9 100644 --- a/devcapsule-src/tests/test_platforms.py +++ b/devcapsule-src/tests/test_platforms.py @@ -38,7 +38,8 @@ def test_current_returns_a_member_on_a_supported_host() -> None: assert Platform.parse(current.value) is current -def _host(monkeypatch: pytest.MonkeyPatch, system: str, machine: str) -> None: +def on_host(monkeypatch: pytest.MonkeyPatch, system: str, machine: str) -> None: + """Make this host the one ``platform.system()`` and ``platform.machine()`` would name.""" monkeypatch.setattr(platforms.host_platform, "system", lambda: system) monkeypatch.setattr(platforms.host_platform, "machine", lambda: machine) @@ -53,15 +54,16 @@ def _host(monkeypatch: pytest.MonkeyPatch, system: str, machine: str) -> None: def test_each_host_runs_its_platforms_and_creates_projects_for_one(monkeypatch, system, machine, runnable, created) -> None: # A Mac runs Linux capsules. Apple Silicon runs linux-arm64 natively and # linux-amd64 under emulation, and creates linux-amd64 projects while the - # linux-arm64 base is unpublished (the human's ruling of 2026-11-04). - _host(monkeypatch, system, machine) + # linux-arm64 base is unpublished. + on_host(monkeypatch, system, machine) assert tuple(str(platform) for platform in Platform.runnable()) == runnable assert Platform.current() is Platform.parse(created) def test_emulation_is_the_daemons() -> None: # Docker Desktop on Apple Silicon is an arm64 daemon, whatever runs the launcher. - assert Platform.LINUX_AMD64.emulated("arm64") + assert Platform.LINUX_AMD64.emulated_docker_platform("arm64") == "linux/amd64" + assert Platform.LINUX_ARM64.emulated_docker_platform("arm64") is None assert Platform.LINUX_AMD64.docker_platform_args("arm64") == ["--platform=linux/amd64"] assert Platform.LINUX_AMD64.docker_platform_args("amd64") == [] assert Platform.LINUX_ARM64.docker_platform_args("arm64") == [] @@ -74,7 +76,7 @@ def test_emulation_is_the_daemons() -> None: ("Windows", "AMD64", "supported: linux, darwin"), ]) def test_current_refuses_a_host_without_a_capsule_platform(monkeypatch, system, machine, message) -> None: - _host(monkeypatch, system, machine) + on_host(monkeypatch, system, machine) with pytest.raises(UnsupportedPlatformError, match=message): Platform.current() @@ -101,11 +103,11 @@ def test_docker_spelling_and_architecture() -> None: @pytest.mark.own_host_patches def test_host_os_names_linux_and_macos_only(monkeypatch) -> None: - _host(monkeypatch, "Darwin", "arm64") + on_host(monkeypatch, "Darwin", "arm64") assert HostOs.current() is HostOs.DARWIN - _host(monkeypatch, "Linux", "x86_64") + on_host(monkeypatch, "Linux", "x86_64") assert HostOs.current() is HostOs.LINUX - _host(monkeypatch, "Windows", "AMD64") + on_host(monkeypatch, "Windows", "AMD64") with pytest.raises(UnsupportedPlatformError, match="'windows'"): HostOs.current() diff --git a/devcapsule-src/tests/test_project_platform_selection.py b/devcapsule-src/tests/test_project_platform_selection.py index c1b81101..a45adec2 100644 --- a/devcapsule-src/tests/test_project_platform_selection.py +++ b/devcapsule-src/tests/test_project_platform_selection.py @@ -1,4 +1,4 @@ -"""The capsule platform a project runs as: its locks decide, or --platform names one.""" +"""The capsule platform a project runs as: the first one this host runs that it has a lock for.""" from __future__ import annotations @@ -6,14 +6,13 @@ import pytest -from devcapsule import cli, platforms -from devcapsule.configuration.storage import project_platform +from devcapsule import cli +from devcapsule.configuration.file_formats import ProjectConfigurationError +from devcapsule.configuration.operations import InitializeRequest, initialize_project +from devcapsule.configuration.storage import load_toml, project_platform from devcapsule.platforms import Platform - - -def _host(monkeypatch: pytest.MonkeyPatch, system: str, machine: str) -> None: - monkeypatch.setattr(platforms.host_platform, "system", lambda: system) - monkeypatch.setattr(platforms.host_platform, "machine", lambda: machine) +from devcapsule.resolution_matrix import MATRICES +from tests.test_platforms import on_host def _project(root: Path, *locked: str) -> Path: @@ -34,9 +33,9 @@ def _project(root: Path, *locked: str) -> Path: def test_apple_silicon_runs_a_project_as_the_platform_it_is_locked_for( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, locked: tuple[str, ...], expected: str, ) -> None: - # The human's ruling of 2026-11-04: natively when the project has an - # arm64 lock, else its amd64 lock under emulation; a new project amd64. - _host(monkeypatch, "Darwin", "arm64") + # Natively when the project has an arm64 lock, else its amd64 lock under + # emulation; a project without a lock is created amd64. + on_host(monkeypatch, "Darwin", "arm64") assert project_platform(_project(tmp_path / "project", *locked)) is Platform.parse(expected) @@ -44,43 +43,41 @@ def test_apple_silicon_runs_a_project_as_the_platform_it_is_locked_for( def test_a_host_runs_a_project_only_as_a_platform_it_runs(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: # An amd64 Linux host cannot run an arm64-only project; it keeps its own # platform, and the missing lock is reported where the lock is read. - _host(monkeypatch, "Linux", "x86_64") + on_host(monkeypatch, "Linux", "x86_64") assert project_platform(_project(tmp_path / "project", "linux-arm64")) is Platform.LINUX_AMD64 - assert project_platform(None) is Platform.LINUX_AMD64 -def _platform_seen_by(arguments: list[str], monkeypatch: pytest.MonkeyPatch) -> Platform: - seen: list[Platform] = [] +@pytest.mark.own_host_patches +def test_init_platform_adds_that_lock_and_the_project_runs_as_it(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + on_host(monkeypatch, "Darwin", "arm64") + project = tmp_path / "project" + project.mkdir() + initialize_project(InitializeRequest(directory=project, creator="owner@example.test", interactive=False)) + initialize_project(InitializeRequest(directory=project, platform=Platform.LINUX_ARM64, interactive=False)) + locks = sorted(path.name for path in (project / ".devcapsule").glob("*.lock")) + assert locks == ["devcapsule.linux-amd64.lock", "devcapsule.linux-arm64.lock"] + arm64_lock = load_toml(project / ".devcapsule" / "devcapsule.linux-arm64.lock") + assert MATRICES[Platform.LINUX_ARM64].base_unpublished(arm64_lock["base"]["reference"]) + assert project_platform(project) is Platform.LINUX_ARM64 - def record(cls: object, arguments: object, context: object) -> int: - seen.append(project_platform(Path(str(getattr(context, "selected_path"))))) - return 0 - monkeypatch.setattr("devcapsule.commands.project.ProjectInfoCommand.run", classmethod(record)) - assert cli.main(arguments) == 0 - return seen[0] +@pytest.mark.own_host_patches +def test_init_refuses_a_lock_the_project_would_not_run_as(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + # Init completes the checkout for the lock it writes; with an arm64 lock + # the project runs as arm64 here, so an amd64 lock cannot be added here. + on_host(monkeypatch, "Darwin", "arm64") + project = _project(tmp_path / "project", "linux-arm64") + with pytest.raises(ProjectConfigurationError, match="runs as linux-arm64"): + initialize_project(InitializeRequest(directory=project, platform=Platform.LINUX_AMD64, interactive=False)) + assert not (project / ".devcapsule" / "devcapsule.linux-amd64.lock").exists() -@pytest.mark.own_host_patches -def test_platform_option_names_the_platform_for_one_command(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: - # `project --platform linux-arm64 init` is how an amd64 project gains an - # arm64 lock; `--platform linux-amd64` runs it under emulation on purpose. - _host(monkeypatch, "Darwin", "arm64") - amd64_only = _project(tmp_path / "amd64", "linux-amd64") - both = _project(tmp_path / "both", "linux-amd64", "linux-arm64") - assert _platform_seen_by(["project", "--path", str(both), "info"], monkeypatch) is Platform.LINUX_ARM64 - assert _platform_seen_by(["project", "--platform", "linux-arm64", "--path", str(amd64_only), "info"], - monkeypatch) is Platform.LINUX_ARM64 - assert _platform_seen_by(["project", "--platform", "linux-amd64", "--path", str(both), "info"], - monkeypatch) is Platform.LINUX_AMD64 - - -def test_platform_option_refuses_a_platform_this_host_cannot_run( +def test_init_platform_refuses_a_platform_this_host_cannot_run( tmp_path: Path, capsys: pytest.CaptureFixture[str], ) -> None: # The suite's host runs linux-amd64 only. project = _project(tmp_path / "project", "linux-amd64") - assert cli.main(["project", "--platform", "linux-arm64", "--path", str(project), "info"]) != 0 - assert "This host runs linux-amd64 capsules, not linux-arm64." in capsys.readouterr().err - assert cli.main(["project", "--platform", "windows-amd64", "--path", str(project), "info"]) != 0 + assert cli.main(["project", "--path", str(project), "init", "--platform", "linux-arm64"]) != 0 + assert "this host runs linux-amd64 capsules, not linux-arm64" in capsys.readouterr().err + assert cli.main(["project", "--path", str(project), "init", "--platform", "windows-amd64"]) != 0 assert "Unsupported platform 'windows-amd64'" in capsys.readouterr().err diff --git a/devcapsule-src/tests/test_pycharm.py b/devcapsule-src/tests/test_pycharm.py index dbefa42a..036c22e0 100644 --- a/devcapsule-src/tests/test_pycharm.py +++ b/devcapsule-src/tests/test_pycharm.py @@ -1,5 +1,6 @@ from __future__ import annotations +from datetime import datetime, timedelta, timezone from pathlib import Path import socket import time @@ -37,6 +38,7 @@ ensure_sudoers_policy_ownership, prepare_temp_runtime_files, print_storage_summary, + remove_abandoned_sudoers_volumes, run_pycharm, write_user_files, ) @@ -82,6 +84,7 @@ def test_external_runtime_plan_is_readable_and_mounted_read_only(tmp_path: Path) config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -146,6 +149,7 @@ def test_plan_driven_surface_mounts_declared_slots_instead_of_pycharm_paths( config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -196,6 +200,7 @@ def test_interactive_state_mounts_require_a_runtime_plan(tmp_path: Path) -> None build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -212,6 +217,7 @@ def test_interactive_state_mounts_must_belong_to_the_plan_component(tmp_path: Pa build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -235,6 +241,7 @@ def test_no_sudo_launches_keep_full_hardening_on_every_surface( config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -276,6 +283,7 @@ def test_sudo_launches_grant_no_capabilities_beyond_dockers_default( config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -312,6 +320,7 @@ def test_plan_driven_storage_summary_names_the_surface_and_its_slots( config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -341,6 +350,7 @@ def test_detached_lifecycle_changes_only_the_docker_process_flags(tmp_path: Path config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -372,6 +382,7 @@ def test_jcef_disclosure_is_printed_for_component_policy( config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -403,6 +414,7 @@ def test_host_browser_bridge_is_explicit_in_plan_mount_environment_and_summary( config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -448,6 +460,7 @@ def test_host_browser_bridge_can_be_disabled_despite_inherited_environment( config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, docker_mode=DockerMode.none, enable_host_browser=False, ), @@ -475,6 +488,7 @@ def test_physical_host_launcher_owns_browser_broker_for_docker_lifetime( assert run_pycharm( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, docker_mode=DockerMode.none, enable_host_browser=True, ), @@ -515,6 +529,7 @@ def test_selected_codex_state_and_explicit_secret_are_delivered(tmp_path: Path) config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -563,6 +578,7 @@ def test_authorized_runtime_effects_build_the_complete_docker_plan(tmp_path: Pat config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", persistent_home=home, ide_config=config_dir, @@ -649,6 +665,7 @@ def test_host_docker_group_follows_where_the_socket_is_owned(tmp_path: Path, hos config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.host, host_docker_socket=docker_socket, @@ -681,6 +698,7 @@ def test_host_networking_falls_back_to_bridge_on_macos(tmp_path: Path, capsys, h config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="host", @@ -707,6 +725,7 @@ def test_unauthorized_runtime_effects_keep_safe_docker_defaults(tmp_path: Path) config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -747,7 +766,8 @@ def test_sudoers_ownership_helper_is_constrained(tmp_path: Path) -> None: ) config = cast( PycharmRunConfig, - SimpleNamespace(enable_sudo=True, image="devcapsule-local-pycharm:canonical", capsule_platform=Platform.LINUX_AMD64), + SimpleNamespace(enable_sudo=True, image="devcapsule-local-pycharm:canonical", + capsule_platform=Platform.LINUX_AMD64, daemon_architecture="amd64"), ) with ( @@ -781,7 +801,8 @@ def test_sudoers_ownership_helper_failure_is_actionable(tmp_path: Path) -> None: ) config = cast( PycharmRunConfig, - SimpleNamespace(enable_sudo=True, image="devcapsule-local-pycharm:canonical", capsule_platform=Platform.LINUX_AMD64), + SimpleNamespace(enable_sudo=True, image="devcapsule-local-pycharm:canonical", + capsule_platform=Platform.LINUX_AMD64, daemon_architecture="amd64"), ) with ( @@ -805,7 +826,7 @@ def _macos_sudo_files(tmp_path: Path) -> tuple[PycharmRunConfig, TempRuntimeFile config = cast( PycharmRunConfig, SimpleNamespace(enable_sudo=True, image="devcapsule-local-pycharm:canonical", name="capsule", - capsule_platform=Platform.LINUX_AMD64), + capsule_platform=Platform.LINUX_AMD64, daemon_architecture="arm64"), ) return config, files @@ -816,23 +837,20 @@ def test_macos_sudo_policy_is_written_into_a_volume_by_root( # Docker Desktop shares Mac files as the Mac user: no chown makes a # bind-mounted policy root's, so the policy lives in a volume. monkeypatch.setattr(HostOs, "current", classmethod(lambda cls: HostOs.DARWIN)) - monkeypatch.setattr("devcapsule.host_daemon.daemon_architecture", lambda: "arm64") config, files = _macos_sudo_files(tmp_path) with ( patch("devcapsule.launch.pycharm._launcher.os.chown") as chown, patch("devcapsule.launch.pycharm._launcher.subprocess.run") as run, ): - run.return_value = SimpleNamespace(returncode=0, stderr="") + run.return_value = SimpleNamespace(returncode=0, stderr="", stdout="") ensure_sudoers_policy_ownership(config, files, {"PATH": "/usr/bin"}) chown.assert_not_called() assert files.sudoers_volume == "capsule-sudoers" - prune, create, write = (call.args[0] for call in run.call_args_list) - # Volumes earlier launches left behind, killed or interrupted, go first; - # prune removes only those no container uses. - assert prune == ["docker", "volume", "prune", "--all", "--force", "--filter", - "label=devcapsule.purpose=development-sudo-policy"] + listed, create, write = (call.args[0] for call in run.call_args_list) + assert listed == ["docker", "volume", "ls", "--quiet", "--filter", + "label=devcapsule.purpose=development-sudo-policy"] assert create == ["docker", "volume", "create", "--label", "devcapsule.purpose=development-sudo-policy", "capsule-sudoers"] assert write[:4] == ["docker", "run", "--rm", "-i"] @@ -856,7 +874,7 @@ def test_macos_sudo_policy_failure_removes_its_volume( with patch("devcapsule.launch.pycharm._launcher.subprocess.run") as run: run.side_effect = [ - SimpleNamespace(returncode=0, stderr=""), + SimpleNamespace(returncode=0, stderr="", stdout=""), SimpleNamespace(returncode=0, stderr=""), SimpleNamespace(returncode=125, stderr="no such image"), SimpleNamespace(returncode=0, stderr=""), @@ -868,6 +886,21 @@ def test_macos_sudo_policy_failure_removes_its_volume( assert files.sudoers_volume is None +def test_only_an_old_policy_volume_counts_as_abandoned() -> None: + # A concurrent launch's fresh volume is unused until its capsule starts. + old = (datetime.now(timezone.utc) - timedelta(days=2)).isoformat() + fresh = datetime.now(timezone.utc).isoformat() + with patch("devcapsule.launch.pycharm._launcher.subprocess.run") as run: + run.side_effect = [ + SimpleNamespace(returncode=0, stdout="killed-sudoers\nconcurrent-sudoers\n"), + SimpleNamespace(returncode=0, stdout=f"killed-sudoers {old}\nconcurrent-sudoers {fresh}\n"), + SimpleNamespace(returncode=0), + ] + remove_abandoned_sudoers_volumes({}) + + assert run.call_args_list[-1].args[0] == ["docker", "volume", "rm", "killed-sudoers"] + + def test_sudo_banner_is_printed_after_complete_policy_preparation( tmp_path: Path, capsys: pytest.CaptureFixture[str], @@ -891,6 +924,7 @@ def test_sudo_banner_is_printed_after_complete_policy_preparation( run_pycharm( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, image="devcapsule-local-pycharm:canonical", docker_mode=DockerMode.none, network_mode="bridge", @@ -933,6 +967,7 @@ def test_sudo_policy_preparation_failure_cleans_files_without_enabled_banner( run_pycharm( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, image="devcapsule-local-pycharm:canonical", docker_mode=DockerMode.none, network_mode="bridge", @@ -965,6 +1000,7 @@ def test_bound_secret_must_exist_on_host(tmp_path: Path) -> None: build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, docker_mode=DockerMode.none, secret_environment=("OPENAI_API_KEY",), ), @@ -987,6 +1023,7 @@ def test_image_process_uses_oci_command_and_cleans_all_temporary_files(tmp_path: run_pycharm( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", image=image, docker_mode=DockerMode.none, @@ -1036,6 +1073,7 @@ def test_launch_preparation_failure_cleans_runtime_and_identity_files(tmp_path: run_pycharm( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", image="devcapsule-local-pycharm:canonical", docker_mode=DockerMode.none, @@ -1109,6 +1147,7 @@ def test_ide_config_option_implies_custom_config_mode(tmp_path: Path) -> None: config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, ide_config=custom_config, docker_mode=DockerMode.none, ), @@ -1128,6 +1167,7 @@ def test_explicit_shared_config_mode_ignores_env_ide_config_dir(tmp_path: Path) config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, config_mode=IdeConfigMode.shared, docker_mode=DockerMode.none, ), @@ -1147,7 +1187,7 @@ def test_env_ide_config_dir_implies_custom_when_mode_is_unset(tmp_path: Path) -> env = base_env(tmp_path) env["PYCHARM_IDE_CONFIG_DIR"] = str(tmp_path / "env-config") - config = build_run_config(PycharmRunOptions(project=project, docker_mode=DockerMode.none), env) + config = build_run_config(PycharmRunOptions(project=project, capsule_platform=Platform.LINUX_AMD64, docker_mode=DockerMode.none), env) assert config.ide_config_mode == "custom" assert config.ide_config == (tmp_path / "env-config").resolve() @@ -1162,6 +1202,7 @@ def test_profile_sets_global_settings_and_plugins_defaults(tmp_path: Path) -> No config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, profile="codex", docker_mode=DockerMode.none, ), @@ -1181,6 +1222,7 @@ def test_project_state_root_mirrors_project_path_under_root_parent(tmp_path: Pat config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_state_root=workspace / ".state", docker_mode=DockerMode.none, ), @@ -1197,6 +1239,7 @@ def test_explicit_project_state_overrides_project_state_root(tmp_path: Path) -> config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_state=tmp_path / "explicit-state", project_state_root=tmp_path / ".state", docker_mode=DockerMode.none, @@ -1217,6 +1260,7 @@ def test_dogfood_legacy_roots_map_to_persistent_home_and_component_slots(tmp_pat config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, global_settings=global_settings, plugins=plugins, project_state=project_state, @@ -1240,6 +1284,7 @@ def test_persistent_home_defaults_to_checkout_scoped_xdg_data(tmp_path: Path) -> config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, docker_mode=DockerMode.none, ), base_env(tmp_path), @@ -1287,6 +1332,7 @@ def contained_config(tmp_path: Path, network_mode: str = "bridge", **overrides: env.pop("DISPLAY") # the contained display needs no host X session at all options: dict[str, object] = dict( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode=network_mode, @@ -1374,11 +1420,12 @@ def test_host_x11_passthrough_still_needs_a_host_display_and_binds_it(tmp_path: env = base_env(tmp_path) env.pop("DISPLAY") with pytest.raises(PycharmRunError, match="DISPLAY is not set"): - build_run_config(PycharmRunOptions(project=project, project_mount="/workspace/project"), env) + build_run_config(PycharmRunOptions(project=project, capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project"), env) env = base_env(tmp_path) config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -1417,6 +1464,7 @@ def test_display_disclosure_states_the_transport(tmp_path: Path) -> None: build_run_config( PycharmRunOptions( project=tmp_path / "project", + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -1458,6 +1506,7 @@ def fake_docker_run(command: list[str], env: dict[str, str], **_kwargs: object) exit_code = run_pycharm( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -1494,7 +1543,7 @@ def test_launcher_mounts_live_configuration_directory_and_private_snapshot_read_ env = base_env(tmp_path) env["XDG_RUNTIME_DIR"] = str(tmp_path / "runtime") config = build_run_config(PycharmRunOptions( - project=project, project_mount="/workspace/project", docker_mode=DockerMode.none, + project=project, capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, runtime_plan=external_runtime_plan(), use_image_process=True, launch_configuration=LaunchConfiguration(records, document), ), env) @@ -1522,7 +1571,7 @@ def test_launcher_mounts_live_configuration_directory_and_private_snapshot_read_ assert not files.launch_context_file.exists() and not files.configuration_directory.exists() listing = {"schema-version": 2, "rows": [], "pending": [], "notes": []} config = build_run_config(PycharmRunOptions( - project=project, project_mount="/workspace/project", docker_mode=DockerMode.none, + project=project, capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, runtime_plan=external_runtime_plan(), use_image_process=True, launch_configuration=LaunchConfiguration(records, document, listing), ), env) @@ -1731,7 +1780,7 @@ def test_console_launcher_cleans_up_after_command_reporting_or_docker_failure( runtime = tmp_path / "runtime" env = {**base_env(tmp_path), "XDG_RUNTIME_DIR": str(runtime)} report = Mock() - options = PycharmRunOptions(project=config.project, command_report=report if report_only else None) + options = PycharmRunOptions(project=config.project, capsule_platform=Platform.LINUX_AMD64, command_report=report if report_only else None) with ( patch("devcapsule.launch.pycharm._launcher.build_run_config", return_value=config), patch("devcapsule.launch.pycharm._launcher.write_user_files"), @@ -1767,7 +1816,7 @@ def test_web_console_is_absent_without_a_runtime_plan(tmp_path: Path) -> None: project = tmp_path / "project" project.mkdir() config = build_run_config( - PycharmRunOptions(project=project, project_mount="/workspace/project", docker_mode=DockerMode.none), + PycharmRunOptions(project=project, capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none), base_env(tmp_path), ) assert config.console_host_port is None and config.console_token == "" @@ -1828,7 +1877,7 @@ def ready(port: int, url: str, opener: object, stop: Event, **kwargs: object) -> patch("devcapsule.launch.pycharm._launcher.default_opener", return_value=default_console_opener) as default, ): assert run_pycharm(PycharmRunOptions( - project=project, project_mount="/workspace/project", docker_mode=DockerMode.none, + project=project, capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", runtime_plan=external_runtime_plan(), use_image_process=True, display_transport=display_transport, open_display_url=display_opener, open_console_url=console_opener if custom_console_opener else None, @@ -1883,6 +1932,7 @@ def session_config(tmp_path: Path, **environment: str) -> tuple[PycharmRunConfig config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -1992,7 +2042,7 @@ def test_unusable_session_directory_does_not_add_a_mount_or_stop_launch(tmp_path patch.object(launcher, "watch_display_ready"), patch.object(launcher, "run_attached", return_value=AttachedResult(7, interrupted=False, stopped=True)) as docker, ): - assert run_pycharm(PycharmRunOptions(project=config.project), env) == 7 + assert run_pycharm(PycharmRunOptions(project=config.project, capsule_platform=Platform.LINUX_AMD64), env) == 7 command = docker.call_args.args[0] assert not any(SESSION_LOG_DESTINATION in arg for arg in command) assert "cannot use session logs" in capsys.readouterr().err @@ -2074,7 +2124,7 @@ def wait(self, timeout=None): patch.object(launcher, "watch_display_ready"), patch.object(launcher.subprocess, "Popen", Client), ): - assert run_pycharm(PycharmRunOptions(project=config.project), env) == 0 + assert run_pycharm(PycharmRunOptions(project=config.project, capsule_platform=Platform.LINUX_AMD64), env) == 0 # Docker receives the literal arguments, in order, before the image. [command] = commands image = command.index(config.image) @@ -2119,7 +2169,7 @@ def wait(self, timeout=None): patch.object(launcher, "watch_display_ready", side_effect=lambda port, url, opener, stop, **kw: stops.append(stop)), patch.object(launcher.subprocess, "Popen", return_value=Client()), ): - assert run_pycharm(PycharmRunOptions(project=config.project), env) == launcher.INTERRUPTED_EXIT_CODE + assert run_pycharm(PycharmRunOptions(project=config.project, capsule_platform=Platform.LINUX_AMD64), env) == launcher.INTERRUPTED_EXIT_CODE assert stopped_before_return assert stops and all(stop.is_set() for stop in stops) assert list((tmp_path / "runtime").iterdir()) == [] @@ -2403,7 +2453,7 @@ def test_the_launcher_records_the_sessions_end_in_its_manifest(tmp_path: Path) - patch.object(launcher, "watch_display_ready"), patch.object(launcher, "run_attached", return_value=AttachedResult(7, interrupted=False, stopped=True)), ): - assert run_pycharm(PycharmRunOptions(project=config.project), env) == 7 + assert run_pycharm(PycharmRunOptions(project=config.project, capsule_platform=Platform.LINUX_AMD64), env) == 7 manifest = json.loads((config.session_log_directory / "session.json").read_text(encoding="utf-8")) assert manifest["session"] == config.session_id and manifest["exit-code"] == 7 assert manifest["ended-at"].endswith("Z") and logs.session_ended(config.session_log_directory) @@ -2414,7 +2464,7 @@ def test_the_launcher_records_the_sessions_end_in_its_manifest(tmp_path: Path) - patch.object(launcher, "watch_display_ready"), patch.object(launcher, "run_attached", return_value=AttachedResult(143, interrupted=True, stopped=True)), ): - assert run_pycharm(PycharmRunOptions(project=config.project), env) == launcher.INTERRUPTED_EXIT_CODE + assert run_pycharm(PycharmRunOptions(project=config.project, capsule_platform=Platform.LINUX_AMD64), env) == launcher.INTERRUPTED_EXIT_CODE manifest = json.loads((config.session_log_directory / "session.json").read_text(encoding="utf-8")) assert manifest["exit-code"] == launcher.INTERRUPTED_EXIT_CODE and manifest["ended-at"] finally: @@ -2441,7 +2491,7 @@ def test_interrupted_launch_with_unstopped_capsule_leaves_session_manifest_unend return_value=AttachedResult(launcher.INTERRUPTED_EXIT_CODE, interrupted=True, stopped=False), ), ): - assert run_pycharm(PycharmRunOptions(project=config.project), env) == launcher.INTERRUPTED_EXIT_CODE + assert run_pycharm(PycharmRunOptions(project=config.project, capsule_platform=Platform.LINUX_AMD64), env) == launcher.INTERRUPTED_EXIT_CODE manifest = json.loads((config.session_log_directory / "session.json").read_text(encoding="utf-8")) assert "ended-at" not in manifest and "exit-code" not in manifest assert not logs.session_ended(config.session_log_directory) diff --git a/devcapsule-src/tests/test_recursive_orchestrator.py b/devcapsule-src/tests/test_recursive_orchestrator.py index f6bea71e..9c4026ad 100644 --- a/devcapsule-src/tests/test_recursive_orchestrator.py +++ b/devcapsule-src/tests/test_recursive_orchestrator.py @@ -2,8 +2,8 @@ from dataclasses import dataclass from pathlib import Path -import sys import socket +import sys import pytest @@ -25,11 +25,6 @@ pytestmark = pytest.mark.legacy -# The recursive dogfood validates the host's X11 socket directory even for a -# contained display. The flow is Linux-only; a Linux runner without one fails -# these tests rather than skipping them. -linux_only = pytest.mark.skipif(sys.platform != "linux", reason="the recursive dogfood flow is Linux-only") - @dataclass class RecursiveFixture: @@ -158,7 +153,7 @@ def test_normal_project_launch_derives_only_a_downgradable_readiness_marker( assert unauthorized == {RECURSIVE_E2E_ENABLED_ENV: "0"} -@linux_only +@pytest.mark.skipif(sys.platform != "linux", reason="the recursive dogfood flow is Linux-only") def test_public_dry_run_is_unique_redacted_and_cleans_owned_staging( recursive_fixture: RecursiveFixture, ) -> None: @@ -187,7 +182,7 @@ def test_public_dry_run_is_unique_redacted_and_cleans_owned_staging( assert not (workspace / second.run_id).exists() -@linux_only +@pytest.mark.skipif(sys.platform != "linux", reason="the recursive dogfood flow is Linux-only") def test_public_dry_run_cleans_after_preparation_failure( recursive_fixture: RecursiveFixture, ) -> None: @@ -211,7 +206,7 @@ def test_public_dry_run_cleans_after_preparation_failure( assert not run_root.exists() -@linux_only +@pytest.mark.skipif(sys.platform != "linux", reason="the recursive dogfood flow is Linux-only") def test_keep_on_failure_preserves_only_the_owned_run( recursive_fixture: RecursiveFixture, ) -> None: @@ -237,7 +232,7 @@ def test_keep_on_failure_preserves_only_the_owned_run( assert run_root.is_dir() -@linux_only +@pytest.mark.skipif(sys.platform != "linux", reason="the recursive dogfood flow is Linux-only") def test_contained_display_dry_run_forwards_no_host_x11( recursive_fixture: RecursiveFixture, tmp_path: Path ) -> None: diff --git a/devcapsule-src/tests/test_recursive_successor_plan.py b/devcapsule-src/tests/test_recursive_successor_plan.py index 7860bc5c..2c1532f3 100644 --- a/devcapsule-src/tests/test_recursive_successor_plan.py +++ b/devcapsule-src/tests/test_recursive_successor_plan.py @@ -10,6 +10,7 @@ from devcapsule.components.pycharm import runtime_template as pycharm_runtime_template from devcapsule.launch.pycharm import DockerMode, PycharmRunOptions +from devcapsule.platforms import Platform from devcapsule.launch.pycharm._launcher import ( TempRuntimeFiles, build_docker_args, @@ -184,6 +185,7 @@ def test_plan_models_every_flag_the_real_launcher_emits(tmp_path: Path, enable_s config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="host", @@ -580,3 +582,7 @@ def test_an_emulated_launch_plans_its_platform_and_the_successor_must_run_it(spe inspection["ImageManifestDescriptor"] = {"platform": {"os": "linux", "architecture": "arm64"}} with pytest.raises(SuccessorPlanError, match="successor platform does not match the planned --platform"): compare_inspection(plan, inspection) + # The independent inspection reloads the retained plan, platform included; + # a native plan's retained form is unchanged. + assert ExpectedSuccessorPlan.from_mapping(plan.to_mapping(show_host_paths=True)) == plan + assert "platform" not in build_plan(docker_args()).to_mapping() diff --git a/devcapsule-src/tests/test_runtime_artifact.py b/devcapsule-src/tests/test_runtime_artifact.py index 31191c0c..e8857ac7 100644 --- a/devcapsule-src/tests/test_runtime_artifact.py +++ b/devcapsule-src/tests/test_runtime_artifact.py @@ -7,12 +7,12 @@ from devcapsule.compat import CliError from devcapsule.materialization import ArtifactSpec, formation_descriptor, formation_identity -from devcapsule import platforms from devcapsule.build_info import BuildInfo from devcapsule.platforms import HostOs, Platform from devcapsule.runtime_artifact import executable_platform, runtime_artifact from devcapsule.runtime_command import RuntimeCommand from tests.test_base_image import pex_fixture +from tests.test_platforms import on_host pytestmark = pytest.mark.legacy @@ -46,11 +46,6 @@ def launcher_of_the_fixtures_build(monkeypatch: pytest.MonkeyPatch) -> None: source_revision="a" * 40, source_url="unknown")) -def _apple_silicon(monkeypatch: pytest.MonkeyPatch) -> None: - monkeypatch.setattr(platforms.host_platform, "system", lambda: "Darwin") - monkeypatch.setattr(platforms.host_platform, "machine", lambda: "arm64") - - def test_packaged_launcher_supplies_itself_despite_source_override(tmp_path: Path, monkeypatch) -> None: launcher = pex_fixture(tmp_path / "launcher.pex") monkeypatch.setenv("SCIE", str(launcher)) @@ -80,7 +75,7 @@ def test_executable_platform_reads_native_headers(tmp_path: Path) -> None: @pytest.mark.own_host_patches def test_macos_launcher_delivers_the_named_linux_runtime(tmp_path: Path, monkeypatch) -> None: - _apple_silicon(monkeypatch) + on_host(monkeypatch, "Darwin", "arm64") monkeypatch.setenv("SCIE", str(pex_with_native_header(tmp_path / "devcapsule-macos", MACH_O_ARM64))) linux = pex_with_native_header(tmp_path / "devcapsule-linux", ELF_AMD64) monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(linux)) @@ -89,7 +84,7 @@ def test_macos_launcher_delivers_the_named_linux_runtime(tmp_path: Path, monkeyp @pytest.mark.own_host_patches def test_macos_launcher_without_a_linux_runtime_says_what_to_set(tmp_path: Path, monkeypatch) -> None: - _apple_silicon(monkeypatch) + on_host(monkeypatch, "Darwin", "arm64") monkeypatch.setenv("SCIE", str(pex_with_native_header(tmp_path / "devcapsule-macos", MACH_O_ARM64))) monkeypatch.delenv("DEVCAPSULE_RUNTIME_PEX", raising=False) with pytest.raises(CliError, match="a macOS executable and cannot run inside linux-amd64 capsules.*DEVCAPSULE_RUNTIME_PEX"): @@ -102,7 +97,7 @@ def test_macos_launcher_without_a_linux_runtime_says_what_to_set(tmp_path: Path, @pytest.mark.own_host_patches def test_a_plain_pex_launcher_on_a_mac_is_not_delivered(tmp_path: Path, monkeypatch) -> None: # A PEX bundles native wheels for the machine it was built on. - _apple_silicon(monkeypatch) + on_host(monkeypatch, "Darwin", "arm64") monkeypatch.delenv("SCIE", raising=False) monkeypatch.setenv("PEX", str(pex_fixture(tmp_path / "launcher.pex"))) linux = pex_with_native_header(tmp_path / "devcapsule-linux", ELF_AMD64) @@ -117,7 +112,7 @@ def test_a_plain_pex_launcher_on_a_mac_is_not_delivered(tmp_path: Path, monkeypa def test_a_runtime_of_another_build_is_refused(tmp_path: Path, monkeypatch) -> None: # D-0009: the runtime runs the plan this launcher writes. A runtime left # from an earlier build, after the launcher was updated, is refused. - _apple_silicon(monkeypatch) + on_host(monkeypatch, "Darwin", "arm64") monkeypatch.setenv("SCIE", str(pex_with_native_header(tmp_path / "devcapsule-macos", MACH_O_ARM64))) stale = pex_with_native_header(tmp_path / "devcapsule-linux", ELF_AMD64, revision="b" * 40) monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(stale)) @@ -129,7 +124,7 @@ def test_a_runtime_of_another_build_is_refused(tmp_path: Path, monkeypatch) -> N def test_a_runtime_directory_serves_each_capsule_platform(tmp_path: Path, monkeypatch) -> None: # Apple Silicon runs both platforms; one setting names both runtimes, and # each launch delivers the one for its lock's platform. - _apple_silicon(monkeypatch) + on_host(monkeypatch, "Darwin", "arm64") monkeypatch.setenv("SCIE", str(pex_with_native_header(tmp_path / "devcapsule-macos", MACH_O_ARM64))) runtimes = tmp_path / "runtimes" runtimes.mkdir() diff --git a/devcapsule-src/tests/test_supervisor.py b/devcapsule-src/tests/test_supervisor.py index 9b55a8d1..3ae83194 100644 --- a/devcapsule-src/tests/test_supervisor.py +++ b/devcapsule-src/tests/test_supervisor.py @@ -20,10 +20,6 @@ import pytest -# The supervisor is the capsule's entry process: it runs only inside Linux -# capsules and relies on Linux signal primitives (sigwaitinfo). -pytestmark = pytest.mark.skipif(sys.platform != "linux", reason="container-side code; capsules are Linux") - from devcapsule import logs from devcapsule.container_runtime.supervisor import ( INFRASTRUCTURE_FAILURE_EXIT_CODE, @@ -34,6 +30,10 @@ pytestmark = pytest.mark.legacy +# The supervisor is the capsule's entry process: it runs only inside Linux +# capsules and relies on Linux signal primitives (sigwaitinfo). +pytestmark = pytest.mark.skipif(sys.platform != "linux", reason="container-side code; capsules are Linux") + DRIVER = Path(__file__).parent / "resources" / "supervisor_driver.py" REPO_ROOT = Path(__file__).resolve().parents[1] diff --git a/docs/platforms/macos.md b/docs/platforms/macos.md index 3632dd64..8dce0f2e 100644 --- a/docs/platforms/macos.md +++ b/docs/platforms/macos.md @@ -1,5 +1,5 @@ --- -description: DevCapsule ships no macOS executable; an experimental launcher built from source runs on Apple Silicon Macs against Docker Desktop, with capsules under emulation or, per project, natively as linux-arm64. What works, what differs, and how to build it. +description: An experimental launcher you build from source runs on Apple Silicon Macs against Docker Desktop, with capsules emulated or native. weight: 3 updated: 2026-11-04 --- @@ -20,8 +20,7 @@ A project runs as one of two capsule platforms: published yet: you build it on your Mac and select it for each checkout. A project runs as `linux-arm64` when it has a `linux-arm64` lock, and -otherwise as `linux-amd64`. `devcapsule project --platform PLATFORM ...` names -one for a single command. +otherwise as `linux-amd64`. ## What differs from Linux @@ -29,8 +28,9 @@ one for a single command. capsule. A macOS executable cannot run there, so you build the Linux runtime for each capsule platform you use and put them in one directory that `DEVCAPSULE_RUNTIME_PEX` names. Each launch delivers the one for its - project's platform and refuses a runtime from another build, so rebuild - them whenever you rebuild the launcher. + project's platform and refuses one of another DevCapsule version. Rebuild + them whenever you rebuild the launcher: runtimes of the same version from + another commit are not told apart. - **Emulation.** Under `linux-amd64`, the first launch of an IDE builds its image under emulation: about ten minutes for PyCharm on a 14-core Mac. Later launches reuse the image. The IDE runs slower than on a native host. @@ -94,7 +94,7 @@ Add the `export` line, and `$HOME/.local/bin` on your `PATH`, to your shell profile. Then continue with [your first session](../getting-started/first-session.md); its projects run as `linux-amd64`. -## Run a project natively as linux-arm64 +## Run a project natively as `linux-arm64` Build the arm64 base once, natively, with the launcher you installed and the revision it was built from (`devcapsule version` shows it): @@ -108,16 +108,22 @@ Then, in each project, add the arm64 lock and select your local base for the checkout: ```bash -devcapsule project --platform linux-arm64 init --unverified --authorize base-image default +devcapsule project init --platform linux-arm64 --authorize base-image default devcapsule project config authorize base-image devcapsule-base-local:arm64 devcapsule project run ``` -`--unverified` is needed while no arm64 combination of IDE and base has -recorded validation evidence; init says so. The arm64 lock names a base that is +VSCodium, VS Code, Eclipse, PyCharm, IntelliJ IDEA and Rider have each started +natively on this base. A component no arm64 run has validated yet, such as the +.NET SDK or Playwright, needs `--unverified` on `init` or on the `config` +command that adds it; the command says so. The arm64 lock names a base that is not published yet, so do not commit it to a shared repository: anyone without -a local arm64 base cannot run it. To run the project under emulation again for -one launch, use `devcapsule project --platform linux-amd64 run`. +a local arm64 base cannot run it. A change to the project's requirements made +on your Mac updates both locks. + +To run the project under emulation again, delete +`.devcapsule/devcapsule.linux-arm64.lock` and select the project's base for +your checkout again with `devcapsule project config authorize base-image default`. If something fails on your Mac, please [open an issue](https://github.com/ccozianu/devcapsule/issues/new) with your macOS and Docker Desktop versions and the error output, omitting diff --git a/docs/reference/cli.md b/docs/reference/cli.md index ed13c40a..eddaf3ee 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -11,12 +11,7 @@ checks even when given `--help`. `devcapsule project` commands discover the project from the current directory. To target another directory, pass `--path` before the subcommand: -`devcapsule project --path PROJECT config show`. On a host that runs two capsule -platforms, an Apple Silicon Mac, a project runs as the first one it has a lock -for, `linux-arm64` before `linux-amd64`, and a new project as `linux-amd64`; -`--platform PLATFORM` before the subcommand names one for that command, so -`devcapsule project --platform linux-arm64 init` adds an arm64 lock to a -project. `devcapsule images build` takes `--platform` too. Inside a capsule, inspection +`devcapsule project --path PROJECT config show`. Inside a capsule, inspection commands distinguish captured launch data from next-launch choices where available. Commands that need launcher-owned state print the exact `devcapsule` command to run outside. @@ -60,7 +55,8 @@ home instead and disables this fallback to launcher settings. ```text devcapsule project init [--require NAME[>=VERSION]]... [--offer CAPABILITY]... [--ide IDE] [--with CAPABILITY]... [--name NAME] [--slug SLUG] [--creator CREATOR] [--project-mount PATH] - [--regenerate] [--unverified] [--set NAME VALUE]... [--authorize NAME VALUE]... [--bind NAME VALUE]... + [--platform PLATFORM] [--regenerate] [--unverified] + [--set NAME VALUE]... [--authorize NAME VALUE]... [--bind NAME VALUE]... ``` `--require` and `--offer` write shared project capabilities into @@ -77,6 +73,13 @@ IDE is selected, `init` can ask for base or vendor consent; non-interactive scripts can pass `--authorize NAME VALUE`. `--set`, `--bind`, and `--authorize` also accept one optional justification argument after the value. +A project has one lock per capsule platform it runs as, such as +`devcapsule.linux-amd64.lock`. A project runs as the first platform this host +runs that it has a lock for. An Apple Silicon Mac runs `linux-arm64` before +`linux-amd64`, and creates new projects as `linux-amd64`. `--platform PLATFORM` +adds that platform's lock, so on the Mac +`devcapsule project init --platform linux-arm64` lets a project run natively. + ### `wrap` ```text @@ -199,7 +202,7 @@ Shared project contract (for repository owners; writes schema 2): Every shared-contract command accepts `--preview` to validate and print the resulting change without writing files. Capability changes and provider -recommendations update the manifest and platform lock together. Capability +recommendations update the manifest and every platform lock together. Capability commands accept `--unverified`; `recommend` does not. Environment declarations and non-provider recommendations update only `.devcapsule/devcapsule.toml`. @@ -253,7 +256,7 @@ Each command below follows `devcapsule`: | Command | Intent | |---|---| | `version [--json]` | Print the launcher version, package version, and source revision provenance. | -| `images build --type {base,environment} ...` | Build a curated base image or materialize a project environment image. | +| `images build --type {base,environment} ...` | Build a curated base image, for the capsule platform `--platform` names, or materialize a project environment image. | | `images list [--include-legacy]` | List DevCapsule base and environment images in the local Docker store. | | `bootstrap [--refresh-workflow-definition] project` | Install or refresh DevCapsule workflow files and initialize project status records. | | `workflow brief` | Print the active workstream summary, live claims, waiting mail, sync distance, and validation commands. | diff --git a/engineering-docs/development/e2e-tests.md b/engineering-docs/development/e2e-tests.md index cdba9525..80595fbe 100644 --- a/engineering-docs/development/e2e-tests.md +++ b/engineering-docs/development/e2e-tests.md @@ -76,6 +76,8 @@ to installing `xprop` in every image. | `DEVCAPSULE_EARLY_EXIT_E2E_IMAGE`, `DEVCAPSULE_CONTRIBUTOR_E2E_IMAGE`, `DEVCAPSULE_PEX_CLEAN_MACHINE_IMAGE` | the sessions | Disposable images the tests build or reuse | | `DEVCAPSULE_E2E_BUILD_NETWORK` | `-- --build-network` | The network mode of image builds | | `DEVCAPSULE_SMOKE_SCENARIO` | `-- --scenario` | Catalog ID or JSON path of a recorded IDE scenario to replay | +| `DEVCAPSULE_SMOKE_PLATFORM` | `-- --platform` | The capsule platform the IDE smoke creates each project for, such as `linux-arm64` | +| `DEVCAPSULE_RUNTIME_PEX` | you, for a macOS launcher | The Linux runtime it delivers, or a directory of `devcapsule-` runtimes | | `PLAYWRIGHT_BROWSERS_PATH` | you, optionally | Where `ide-smoke -- --display` finds or installs Chromium | ## Evidence diff --git a/engineering-docs/development/unit-tests.md b/engineering-docs/development/unit-tests.md index 9126e306..c552869b 100644 --- a/engineering-docs/development/unit-tests.md +++ b/engineering-docs/development/unit-tests.md @@ -26,7 +26,8 @@ with a terminal report of missing lines, and the default selection `-m "not integration and not e2e"`. So a bare `pytest` under `tests/` is the unit suite; the heavier suites are chosen by marker, never by accident. The `flaky` marker runs a test for visibility with a temporarily non-blocking -result; the other markers belong to the heavier suites. +result; `own_host_patches` marks a unit test that patches the host itself +(see Fixtures); the other markers belong to the heavier suites. On a pull request the hosted workflow also writes both coverage figures, changed lines covered and the whole package, into the pull request's @@ -122,11 +123,16 @@ so prefer the existing laws and an acceptance step for what the user sees. ## Fixtures -`tests/conftest.py` holds three fixtures and nothing else: +`tests/conftest.py` holds the suite's shared fixtures and hooks: - `host_launch_by_default`, autouse: makes launcher tests behave as if they run on the host, since the launcher translates bind sources when it finds itself inside a capsule and the suite is often run from one. +- `amd64_linux_host_by_default`, autouse: makes the host an amd64 Linux + machine with an amd64 Docker daemon, as on the CI, since the launcher takes + other paths on macOS and Apple Silicon runs linux-arm64 capsules. A test of + another host patches the host itself and carries `own_host_patches`; + Docker-marked tests keep the real host. - `private_user_directories`, autouse: moves `HOME` and every XDG directory under the test's `tmp_path` for non-Docker tests. Inside a capsule this is not yet a private home: a home with no settings of its own reads the @@ -136,6 +142,10 @@ so prefer the existing laws and an acceptance step for what the user sees. set, otherwise `dist/devcapsule-local.pex`; it fails with the Nox session to run when the file is missing. The integration and end-to-end suites use it; the unit suite does not need an artifact. +- `pytest_configure` and `pytest_sessionfinish`: on macOS, a short temporary + root under `/tmp`, because a Unix socket path there is limited to 104 + bytes. A session whose tests all passed removes it; a failed one keeps it + and prints its path. Everything else is local to its module: fakes for Docker and vendor services, temporary XDG roots, and the `tmp_path` pytest provides. diff --git a/engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md b/engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md index 697ebf19..e495d6db 100644 --- a/engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md +++ b/engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md @@ -37,10 +37,14 @@ the base recipe, on top of `macos-launcher`'s platform split. File references ar while the arm64 base is unpublished, per the ruling in section 2. Which platform a project runs as is the configuration's question: `configuration.storage.project_platform`, read-only, answers the first runnable platform the project has a lock for, else - `current()`; `project --platform` and `images build --platform` name one for a command. - Below the configuration, the platform is the lock's, passed down explicitly to the base - pull, the image build, the runtime choice and the launch (D-0006's consequence). - Emulation is judged against the Docker daemon's architecture, not the launcher's CPU. + `current()`. `project init --platform` adds a platform's lock, and the project then runs + as it; `images build --platform` names the platform of a base build. A checkout so runs + one platform until its project's locks change. An owner's edit of the shared contract + regenerates every lock the project carries, so a committed lock never falls behind the + manifest (R-COMPAT-001). Below the configuration, the platform is the lock's, a required + parameter of the base pull, the image build, the runtime choice and the launch (D-0006's + consequence). Emulation is judged against the Docker daemon's architecture, asked once + per launch, not the launcher's CPU. 2. **The runtime.** An arm64 capsule needs a `linux-aarch64` executable. A launcher whose own binary runs on the capsule platform delivers itself (R-RUNTIME-001). Otherwise the user names the runtime: `DEVCAPSULE_RUNTIME_PEX` names one file, or a directory holding @@ -67,9 +71,9 @@ the base recipe, on top of `macos-launcher`'s platform split. File references ar 4. **Live and documentation.** `nox -s ide-smoke` for every surface natively on this Mac; the Linux, macOS and install pages. -## 5. Review, 2026-11-04 +## 5. Reviews, 2026-11-04 -The human's `/review` of all changes since `main`, run by a separate agent, rejected the +A review of all changes since `main`, run by a separate agent, rejected the branch with seven high findings: a Mac's new projects defaulting to the unpublished arm64 base; `images build` ignoring the project's locks; the platform read from a process-wide selection below the configuration instead of from the lock; no version check on the @@ -79,6 +83,24 @@ followed. The human said "fix all"; each is fixed as described in section 3, exc lock naming an unpublished base is reported, not refused, since the human's ruling has the checkout select a local base for it. +A second review of all changes since `main`, and a review of the branch against the +codebase's own conventions, followed the fixes. The first rejected the branch again with +three high findings: an owner's edit on a Mac rewrote only the lock of the platform the +project ran as, leaving the committed `linux-amd64` lock behind its manifest; a checkout's +one base-image answer applied to whichever lock `project --platform` named, so switching +platforms per command broke the launch; and each macOS launch's sweep of sudo-policy +volumes could remove a concurrent launch's fresh one. Nine medium and three low findings +followed. The second found seventeen departures, the first that `--platform` was kept in a +module global, process-wide state the codebase nowhere else has. The fixes are section 3's +design: `project --platform` is gone, owner edits regenerate every lock under one journal +that still reads the single-lock form an older launcher wrote, the sweep removes only +unused policy volumes older than a day, and the retained successor plan carries its +planned `--platform`. Comments, messages, tests and the generated pin module now follow the +surrounding code. + +Left as it is: a checkout keeps one base-image answer. A developer who deletes the arm64 +lock to return to emulation answers the base again (`docs/platforms/macos.md`). + ## 6. What this does not decide - Publishing an arm64 or multi-platform base, or arm64 release executables: the owner's. diff --git a/index.md b/index.md index 59eb3188..2e2516bd 100644 --- a/index.md +++ b/index.md @@ -179,6 +179,7 @@ status file; internal WIP/archive documents use the local index in that status. - [Capsule web console current status](engineering-docs/wip/2026-10-09-capsule-webconsole/CURRENT-STATUS.md) - [Wrapped projects current status](engineering-docs/wip/2026-11-02-wrapped-projects/CURRENT-STATUS.md) - [macOS launcher current status](engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md) +- [linux-arm64 capsules current status](engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md) - [Sample demo projects current status](engineering-docs/wip/2026-08-14-sample-projects/CURRENT-STATUS.md) - [Contained display current status](engineering-docs/wip/2026-08-19-contained-display/CURRENT-STATUS.md) From 5f3f87458edb432efe34d0e93f8dae26be0410ca Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Wed, 4 Nov 2026 15:08:35 +0200 Subject: [PATCH 19/20] Record the native arm64 smokes and the second review's fixes Co-Authored-By: Claude Opus 5.5 --- CURRENT-STATUS.md | 2 +- .../2026-11-04-linux-arm64/CURRENT-STATUS.md | 31 +++++++++++++++---- 2 files changed, 26 insertions(+), 7 deletions(-) diff --git a/CURRENT-STATUS.md b/CURRENT-STATUS.md index 0c5629d7..6ed04b08 100644 --- a/CURRENT-STATUS.md +++ b/CURRENT-STATUS.md @@ -21,7 +21,7 @@ coordination branch. Detailed state belongs in each linked status file. | `capsule-webconsole` | 2026-10-09 | Serve a token-gated web console from every capsule: configuration, versions, info, live resources, later records and workflow state | active; work order accepted 2026-10-09; the first iteration, deliverables 1 and 2, is three stacked pull requests, #184, #186 and the runtime-and-base slice, each reviewed by the Codex pair; the owner merges | `ws-capsule-webconsole/first-slice` | [workstream status](engineering-docs/wip/2026-10-09-capsule-webconsole/CURRENT-STATUS.md) | | `wrapped-projects` | 2026-11-02 | Wrap a foreign repository as a submodule of a small wrapper project and work on it in a capsule with three commands; FRC learning project first | active; PR1 (#333) accepted by the owner 2026-11-03, awaiting merge; the project-capabilities design handed to workflow-improvements | `ws-wrapped-projects/wrap` | [workstream status](engineering-docs/wip/2026-11-02-wrapped-projects/CURRENT-STATUS.md) | | `macos-launcher` | 2026-11-03 | Run the DevCapsule launcher natively on Apple Silicon Macs against Docker Desktop, with capsules as `linux-amd64` under emulation | active 2026-11-03; slices 1 to 4 done, the PyCharm smoke passing on an Apple Silicon Mac; waits on publication and the owner's rulings | `ws-macos-launcher/host-port` | [workstream status](engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md) | -| `linux-arm64` | 2026-11-04 | Run capsules as `linux-arm64` as well as `linux-amd64`: natively on Apple Silicon Macs and on ARM Linux hosts | active 2026-11-04; registered; slice 1, the second capsule platform | `ws-linux-arm64/capsules` | [workstream status](engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md) | +| `linux-arm64` | 2026-11-04 | Run capsules as `linux-arm64` as well as `linux-amd64`: natively on Apple Silicon Macs and on ARM Linux hosts | active 2026-11-04; slices 1 to 4 done, six IDEs running natively on Apple Silicon; waits on publication, the owner's arm64 base and rulings | `ws-linux-arm64/capsules` | [workstream status](engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md) | | `website` | 2026-09-16 | Deliver the website autonomy experiment: repository-owned landing/docs/blog content, independently developable website submodule, local preview and publishing mechanism | paused 2026-09-20 at owner request to switch to maintenance; experiment accepted, A−; blog integrated and published; content/parent-integration follow-ups remain | `ws-website/initial-cut`; renamed from `website/initial-cut`; preparation and backlog-handoff exceptions in status | [workstream status](engineering-docs/wip/2026-09-16-website/CURRENT-STATUS.md) | Paused and blocked workstreams remain open until they conclude successfully or diff --git a/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md index 00374b27..96a1752e 100644 --- a/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md @@ -4,7 +4,7 @@ Mnemonic: `linux-arm64` Start date: 2026-11-04 -State: active; slices 1 to 3 and the review's fixes committed 2026-11-04; native smokes pending Docker +State: active; slices 1 to 4 done 2026-11-04, six IDEs running natively; waits on publication and the owner Branch association: `ws-linux-arm64/capsules`, forked from local `main` at b3f10b7d, with `ws-macos-launcher/host-port` merged in as its foundation @@ -66,9 +66,8 @@ Found for `maintenance`, not fixed here: the amd64 Eclipse closure's `libsoup-3. `libsoup-3.0-common` 3.4.4-5ubuntu0.8 URLs return 404, since Ubuntu removed that version from every noble pocket on 2026-10-08, so Eclipse cannot be acquired afresh on amd64. -**2026-11-04, review and fixes.** The human ran the `/review` skill of `~/work/teachme` on all -changes since `main`, through a separate agent: rejected, seven high, fourteen medium, three -low findings. The human said "fix all"; the design note's section 5 records them, and +**2026-11-04, review and fixes.** The human had all changes since `main` reviewed by a +separate agent: rejected, seven high, fourteen medium, three low findings. The human said "fix all"; the design note's section 5 records them, and section 3 the design they produced. In short: a Mac creates `linux-amd64` projects again and runs a project as the first platform it has a lock for; the configuration answers which (`storage.project_platform`), and below it the lock's platform is passed down explicitly; a @@ -89,10 +88,28 @@ The arm64 run fails 14 console-channel tests whose real `devcapsule` subprocess arm64 host while the suite's fixtures are amd64: a limit of the suite on ARM Linux, not CI. The IDE smoke takes `DEVCAPSULE_SMOKE_PLATFORM` to create its project for a platform. +**2026-11-04, native smokes and a second review.** One at a time, each after a check of free +space and followed by removing its images, every IDE passed natively as `linux-arm64` with +the local base: VSCodium, VS Code, Eclipse, PyCharm, Rider and IntelliJ IDEA, runs +20261104T120158Z-e8f78f to 20261104T121007Z-289109. The JetBrains IDEs were seen at their +splash window. The six are the `linux-arm64` matrix's first verified edges; the .NET SDK and +Playwright rode along unexercised and stay unverified. A second review of all changes since +`main`, and a review against the codebase's own conventions, followed; the design note's +section 5 records both, and 9d285a5a their fixes: `project --platform` and its module-global +selection are gone and `project init --platform` adds a lock; an owner's edit regenerates +every lock; the lock's platform is required below the configuration; the sudo-policy sweep +spares a concurrent launch's volume; comments, messages and tests follow the surrounding +code. Unit suite on this Mac: 2,760 passed, 25 skipped; in a `linux/amd64` container, 2,779 +passed and 5 failed, the same five recursive tests that fail on `main` there, for want of a +host X11 directory, a non-root user and a Docker group. From 9d285a5a, VSCodium passed +natively (20261104T130201Z-392364, created with `init --platform linux-arm64` and verified +without `--unverified`) and under emulation (20261104T130042Z-36f56a). + ## Next Resumable Task -Record the rerun smokes; add arm64 verified edges for the surfaces that pass, with their -evidence, so resolution stops needing `--unverified`; then the documentation slice. +Publish this workstream and `macos-launcher` once the SSH agent holds a key, and hand both +pull requests to the human, the macOS one first. The owner's rulings and the arm64 base's +publication follow. ## Open Threads @@ -107,6 +124,8 @@ evidence, so resolution stops needing `--unverified`; then the documentation sli can be pushed. - Docker access from an arm64 capsule reaches an arm64 daemon either way; on amd64 capsules under emulation it also defaults to arm64 images (`docs/platforms/macos.md`). +- A checkout keeps one base-image answer. Deleting the arm64 lock to return to emulation + needs the base answered again, as the macOS page says. ## Document Index From dcc2a7db9ad57efe03e61cca5270386b1af68c7a Mon Sep 17 00:00:00 2001 From: Florin Munteanu Date: Wed, 4 Nov 2026 15:15:51 +0200 Subject: [PATCH 20/20] Ask a host of several platforms which one a new project is for On an Apple Silicon Mac, init chose linux-amd64 for a new project without a question. Init now asks before the identity questions when the host runs more than one capsule platform, with the native one, Platform.current(), as the default and the answer of a run without a terminal. --platform answers it; a project with a lock the host runs is not asked, and a Linux host never is. The macOS page, the CLI reference and the linux-arm64 records follow. Co-Authored-By: Claude Opus 5.5 --- .../devcapsule/configuration/operations.py | 87 ++++++++++++++----- devcapsule-src/devcapsule/platforms.py | 24 ++--- devcapsule-src/tests/test_owner_commands.py | 14 ++- devcapsule-src/tests/test_platforms.py | 5 +- .../tests/test_project_platform_selection.py | 53 ++++++++++- docs/platforms/macos.md | 23 ++--- docs/reference/cli.md | 10 ++- .../2026-11-04-design-linux-arm64.md | 9 +- .../2026-11-04-linux-arm64/CURRENT-STATUS.md | 7 ++ 9 files changed, 164 insertions(+), 68 deletions(-) diff --git a/devcapsule-src/devcapsule/configuration/operations.py b/devcapsule-src/devcapsule/configuration/operations.py index 903ae576..23d2c149 100644 --- a/devcapsule-src/devcapsule/configuration/operations.py +++ b/devcapsule-src/devcapsule/configuration/operations.py @@ -27,7 +27,7 @@ from devcapsule.environment_realization import required_local_image from devcapsule.materialization import validate_base_image from devcapsule.project import sanitize_name -from devcapsule.platforms import Platform +from devcapsule.platforms import Platform, UnsupportedPlatformError from devcapsule.resolution_matrix import MATRICES, ResolutionMatrix from .trust import HomeTrust @@ -278,7 +278,8 @@ class InitializeRequest: allow_unverified: bool = False # None: decide from whether stdin is a terminal. interactive: bool | None = None - # The platform whose lock to write; None: the one the project runs as here. + # The platform whose lock to write; None: the one the project runs as + # here, or, for a new project on a host of several platforms, asked. platform: Platform | None = None @@ -408,7 +409,14 @@ def initialize_project( raise ProjectConfigurationError(f"Project directory does not exist: {root}") require_settled(root) manifest_path = root / ".devcapsule" / "devcapsule.toml" - platform = _platform_to_lock(root, request.platform) + elicitor = Elicitor( + _init_command_line(request), + interactive=request.interactive, + input_stream=input_stream, + output_stream=output_stream, + ) + # The platform comes first: its matrix answers every later question. + platform = _elicit_platform(elicitor, root) lock_path = platform_lock_path(root, platform) matrix = MATRICES[platform] @@ -453,12 +461,6 @@ def initialize_project( "the 'config' family changes individual answers." ) - elicitor = Elicitor( - _init_command_line(request), - interactive=request.interactive, - input_stream=input_stream, - output_stream=output_stream, - ) identity = _elicit_identity(elicitor, root, existing_manifest, matrix) authored = frozenset(answer.name for answer in request.answers if answer.justification is not None) # Fail now if identity or requirements are unanswered: the lock cannot be @@ -581,25 +583,64 @@ def persist() -> None: ) -def _platform_to_lock(root: Path, named: Platform | None) -> Platform: - """The platform whose lock init writes: ``named``, or the one the project runs as here. +def _elicit_platform(elicitor: Elicitor, root: Path) -> Platform: + """The platform whose lock init writes, asked when this host runs several. - Init completes the checkout for the lock it writes, so a named platform - must be the one the project runs as once that lock exists. + A project with a lock this host runs keeps that platform unless + ``--platform`` names another; otherwise a host of one platform takes it, + and a host of several asks, its native platform the default and the + unattended answer. Init completes the checkout for the lock it writes, so + the platform must be the one the project runs as once that lock exists. """ - if named is None: - return project_platform(root) runnable = Platform.runnable() - if named not in runnable: - names = ", ".join(str(member) for member in runnable) - raise ProjectConfigurationError(f"This host runs {names} capsules, not {named}.") - runs_as = next(platform for platform in runnable if platform is named or platform_lock_path(root, platform).is_file()) - if runs_as is not named: + native = runnable[0] + locked = next((platform for platform in runnable if platform_lock_path(root, platform).is_file()), None) + choices = " or ".join( + f"{platform} ({'native' if platform is native else 'emulated'}" + + ("; its base is built locally" if _base_unpublished(MATRICES[platform]) else "") + ")" + for platform in runnable + ) + answer = elicitor.seek( + "platform", + description=f"Capsule platform for this project: {choices}", + remedy="--platform PLATFORM", + existing=str(locked) if locked is not None else None, + default=str(native) if len(runnable) == 1 else None, + empty_answer=str(native), + omitted_answer=str(native), + validate=_platform_validator(runnable), + ) + if answer is None: # Input ended at the question. + elicitor.finish(require_all_consumed=False) + raise AssertionError("an unanswered platform question must fail the elicitation") + platform = Platform.parse(answer.value) + runs_as = next(candidate for candidate in runnable + if candidate is platform or platform_lock_path(root, candidate).is_file()) + if runs_as is not platform: raise ProjectConfigurationError( f"On this host the project runs as {runs_as}, which it has a lock for, " - f"so init cannot add a {named} lock here." + f"so init cannot add a {platform} lock here." ) - return named + return platform + + +def _platform_validator(runnable: tuple[Platform, ...]) -> Any: + def validate(value: str) -> str: + try: + candidate = Platform.parse(value.strip()) + except UnsupportedPlatformError as exc: + raise ProjectConfigurationError(str(exc)) from exc + if candidate not in runnable: + names = ", ".join(str(platform) for platform in runnable) + raise ProjectConfigurationError(f"This host runs {names} capsules, not {candidate}.") + return str(candidate) + + return validate + + +def _base_unpublished(matrix: ResolutionMatrix) -> bool: + """Whether every base ``matrix`` pins is still unpublished.""" + return all(matrix.base_unpublished(base.reference) for base in matrix.base_images()) def _initialize_report( @@ -808,6 +849,8 @@ def _init_command_line(request: InitializeRequest) -> dict[AnswerKey, str]: command_line[("offer", "value")] = " ".join(request.offer) if request.ide is not None: command_line[("ide", "value")] = request.ide + if request.platform is not None: + command_line[("platform", "value")] = str(request.platform) for key, value in ( ("name", request.project_name), ("slug", request.slug), diff --git a/devcapsule-src/devcapsule/platforms.py b/devcapsule-src/devcapsule/platforms.py index da126116..46914079 100644 --- a/devcapsule-src/devcapsule/platforms.py +++ b/devcapsule-src/devcapsule/platforms.py @@ -113,14 +113,6 @@ def chown_reaches_bind_mounts(self) -> bool: (HostOs.DARWIN, "amd64"): ("linux-amd64",), } -# The platform a host creates a new project for, where it is not the native -# one. Apple Silicon creates linux-amd64 projects while the linux-arm64 base -# is unpublished; a developer adds a linux-arm64 lock deliberately -# (project init --platform linux-arm64), and the project then runs as it. -_NEW_PROJECT_PLATFORM_OF_HOST: Mapping[tuple[HostOs, str], str] = { - (HostOs.DARWIN, "arm64"): "linux-amd64", -} - # The platform whose executables carry each ELF e_machine value. _PLATFORM_OF_ELF_MACHINE: Mapping[int, str] = {0x3E: "linux-amd64", 0xB7: "linux-arm64"} @@ -136,9 +128,9 @@ class Platform(StrEnum): never renamed. A client never constructs a key from parts. Host detection gives - ``current()`` (the platform this host creates a project for) and - ``runnable()`` (every platform it runs); ``parse()`` gives the platform a - lock records, and ``of_elf_machine()`` the one an executable is for. + ``current()`` (the platform this host runs natively) and ``runnable()`` + (every platform it runs); ``parse()`` gives the platform a lock records, + and ``of_elf_machine()`` the one an executable is for. Which platform a project runs as is the configuration's question (``configuration.storage.project_platform``), answered from its locks. """ @@ -148,15 +140,9 @@ class Platform(StrEnum): @classmethod def current(cls) -> Platform: - """The platform this host creates a new project for. - - On Linux the host's own. On a Mac linux-amd64, natively on Intel and - under Docker Desktop's emulation on Apple Silicon. - """ + """The platform this host runs natively, and a new project's unless the developer chooses another.""" - host = (HostOs.current(), host_architecture()) - created = _NEW_PROJECT_PLATFORM_OF_HOST.get(host) - return cls.parse(created) if created is not None else cls.runnable()[0] + return cls.runnable()[0] @classmethod def runnable(cls) -> tuple[Platform, ...]: diff --git a/devcapsule-src/tests/test_owner_commands.py b/devcapsule-src/tests/test_owner_commands.py index d1b1d298..5bdea0f4 100644 --- a/devcapsule-src/tests/test_owner_commands.py +++ b/devcapsule-src/tests/test_owner_commands.py @@ -427,11 +427,17 @@ def test_sources_survive_other_owner_edits_and_do_not_change_pending_decisions(p @pytest.fixture -def two_platform_project(project: Path, monkeypatch: pytest.MonkeyPatch) -> Path: - """The project on Apple Silicon, with a linux-arm64 lock added beside its linux-amd64 one.""" +def two_platform_project(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """A project on Apple Silicon with a linux-amd64 lock and a linux-arm64 lock added beside it.""" + monkeypatch.setenv("XDG_CONFIG_HOME", str(tmp_path / "config")) + monkeypatch.setenv("HOME", str(tmp_path / "home")) on_host(monkeypatch, "Darwin", "arm64") - assert cli.main(["project", "--path", str(project), "init", "--platform", "linux-arm64"]) == 0 - return project + root = tmp_path / "project" + root.mkdir() + assert cli.main(["project", "--path", str(root), "init", "--platform", "linux-amd64", "--require", "python", + "--creator", "mailto:owner@example.test"]) == 0 + assert cli.main(["project", "--path", str(root), "init", "--platform", "linux-arm64"]) == 0 + return root def _locked_components(root: Path) -> dict[str, set[str]]: diff --git a/devcapsule-src/tests/test_platforms.py b/devcapsule-src/tests/test_platforms.py index 92a47be9..6011c41c 100644 --- a/devcapsule-src/tests/test_platforms.py +++ b/devcapsule-src/tests/test_platforms.py @@ -49,12 +49,11 @@ def on_host(monkeypatch: pytest.MonkeyPatch, system: str, machine: str) -> None: ("Linux", "x86_64", ("linux-amd64",), "linux-amd64"), ("Linux", "aarch64", ("linux-arm64",), "linux-arm64"), ("Darwin", "x86_64", ("linux-amd64",), "linux-amd64"), - ("Darwin", "arm64", ("linux-arm64", "linux-amd64"), "linux-amd64"), + ("Darwin", "arm64", ("linux-arm64", "linux-amd64"), "linux-arm64"), ]) def test_each_host_runs_its_platforms_and_creates_projects_for_one(monkeypatch, system, machine, runnable, created) -> None: # A Mac runs Linux capsules. Apple Silicon runs linux-arm64 natively and - # linux-amd64 under emulation, and creates linux-amd64 projects while the - # linux-arm64 base is unpublished. + # linux-amd64 under emulation; a new project defaults to the native one. on_host(monkeypatch, system, machine) assert tuple(str(platform) for platform in Platform.runnable()) == runnable assert Platform.current() is Platform.parse(created) diff --git a/devcapsule-src/tests/test_project_platform_selection.py b/devcapsule-src/tests/test_project_platform_selection.py index a45adec2..a1c2d61c 100644 --- a/devcapsule-src/tests/test_project_platform_selection.py +++ b/devcapsule-src/tests/test_project_platform_selection.py @@ -2,6 +2,7 @@ from __future__ import annotations +import io from pathlib import Path import pytest @@ -25,7 +26,7 @@ def _project(root: Path, *locked: str) -> Path: @pytest.mark.own_host_patches @pytest.mark.parametrize("locked,expected", [ - ((), "linux-amd64"), + ((), "linux-arm64"), (("linux-amd64",), "linux-amd64"), (("linux-arm64",), "linux-arm64"), (("linux-amd64", "linux-arm64"), "linux-arm64"), @@ -34,7 +35,7 @@ def test_apple_silicon_runs_a_project_as_the_platform_it_is_locked_for( tmp_path: Path, monkeypatch: pytest.MonkeyPatch, locked: tuple[str, ...], expected: str, ) -> None: # Natively when the project has an arm64 lock, else its amd64 lock under - # emulation; a project without a lock is created amd64. + # emulation; a project without a lock is created for the native platform. on_host(monkeypatch, "Darwin", "arm64") assert project_platform(_project(tmp_path / "project", *locked)) is Platform.parse(expected) @@ -52,7 +53,8 @@ def test_init_platform_adds_that_lock_and_the_project_runs_as_it(tmp_path: Path, on_host(monkeypatch, "Darwin", "arm64") project = tmp_path / "project" project.mkdir() - initialize_project(InitializeRequest(directory=project, creator="owner@example.test", interactive=False)) + initialize_project(InitializeRequest(directory=project, creator="owner@example.test", + platform=Platform.LINUX_AMD64, interactive=False)) initialize_project(InitializeRequest(directory=project, platform=Platform.LINUX_ARM64, interactive=False)) locks = sorted(path.name for path in (project / ".devcapsule").glob("*.lock")) assert locks == ["devcapsule.linux-amd64.lock", "devcapsule.linux-arm64.lock"] @@ -81,3 +83,48 @@ def test_init_platform_refuses_a_platform_this_host_cannot_run( assert "this host runs linux-amd64 capsules, not linux-arm64" in capsys.readouterr().err assert cli.main(["project", "--path", str(project), "init", "--platform", "windows-amd64"]) != 0 assert "Unsupported platform 'windows-amd64'" in capsys.readouterr().err + + +def _init_answering(project: Path, answers: str) -> str: + """Run an interactive init of ``project``, answering each question in turn; return what it asked.""" + asked = io.StringIO() + initialize_project( + InitializeRequest(directory=project, creator="owner@example.test", interactive=True), + input_stream=io.StringIO(answers), output_stream=asked, + ) + return asked.getvalue() + + +@pytest.mark.own_host_patches +@pytest.mark.parametrize("answer,expected", [("", "linux-arm64"), ("linux-amd64", "linux-amd64")]) +def test_init_asks_a_host_of_two_platforms_the_native_one_by_default( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, answer: str, expected: str, +) -> None: + on_host(monkeypatch, "Darwin", "arm64") + project = tmp_path / "project" + project.mkdir() + asked = _init_answering(project, f"{answer}\n" + "\n" * 8) + assert ("Capsule platform for this project: linux-arm64 (native; its base is built locally) " + "or linux-amd64 (emulated) [linux-arm64]:") in asked + assert [path.name for path in (project / ".devcapsule").glob("*.lock")] == [f"devcapsule.{expected}.lock"] + + +@pytest.mark.own_host_patches +def test_init_keeps_the_platform_of_a_lock_the_host_runs_without_asking( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, +) -> None: + on_host(monkeypatch, "Darwin", "arm64") + project = tmp_path / "project" + project.mkdir() + initialize_project(InitializeRequest(directory=project, creator="owner@example.test", + platform=Platform.LINUX_AMD64, interactive=False)) + assert "Capsule platform" not in _init_answering(project, "\n" * 8) + assert [path.name for path in (project / ".devcapsule").glob("*.lock")] == ["devcapsule.linux-amd64.lock"] + + +def test_a_host_of_one_platform_is_not_asked(tmp_path: Path) -> None: + project = tmp_path / "project" + project.mkdir() + assert "Capsule platform" not in _init_answering(project, "\n" * 8) + assert (project / ".devcapsule" / "devcapsule.linux-amd64.lock").is_file() + diff --git a/docs/platforms/macos.md b/docs/platforms/macos.md index 8dce0f2e..c89837fd 100644 --- a/docs/platforms/macos.md +++ b/docs/platforms/macos.md @@ -14,13 +14,15 @@ experimental and not a supported platform in {{tag}}. Intel Macs are untested. A project runs as one of two capsule platforms: -- **`linux-amd64`, the default.** The same environments Linux users get, run by - Docker Desktop under x86-64 emulation. Every new project starts here. -- **`linux-arm64`, natively, per project.** Faster, but its base image is not - published yet: you build it on your Mac and select it for each checkout. +- **`linux-arm64`, natively, the default.** Faster, but its base image is not + published yet: you build it on your Mac once and select it for each checkout. +- **`linux-amd64`, under emulation.** The same environments Linux users get, + with the published base, run by Docker Desktop under x86-64 emulation. -A project runs as `linux-arm64` when it has a `linux-arm64` lock, and -otherwise as `linux-amd64`. +`devcapsule project init` asks which one a new project is for; Enter takes +`linux-arm64`, and `--platform PLATFORM` answers without the question. A +project runs as `linux-arm64` when it has a `linux-arm64` lock, and otherwise +as `linux-amd64`. ## What differs from Linux @@ -91,8 +93,9 @@ export DEVCAPSULE_RUNTIME_PEX="$HOME/.local/lib/devcapsule" ``` Add the `export` line, and `$HOME/.local/bin` on your `PATH`, to your shell -profile. Then continue with [your first session](../getting-started/first-session.md); -its projects run as `linux-amd64`. +profile. Then build the arm64 base below, or continue with +[your first session](../getting-started/first-session.md) and answer +`linux-amd64` when init asks for the platform. ## Run a project natively as `linux-arm64` @@ -104,8 +107,8 @@ devcapsule images build --type base --recipe ubuntu-24.04 --platform linux-arm64 --tag devcapsule-base-local:arm64 --source-revision "$revision" ``` -Then, in each project, add the arm64 lock and select your local base for the -checkout: +Then, in each project, create the arm64 lock, or add it beside an existing +`linux-amd64` one, and select your local base for the checkout: ```bash devcapsule project init --platform linux-arm64 --authorize base-image default diff --git a/docs/reference/cli.md b/docs/reference/cli.md index eddaf3ee..04cb7fa6 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -75,10 +75,12 @@ also accept one optional justification argument after the value. A project has one lock per capsule platform it runs as, such as `devcapsule.linux-amd64.lock`. A project runs as the first platform this host -runs that it has a lock for. An Apple Silicon Mac runs `linux-arm64` before -`linux-amd64`, and creates new projects as `linux-amd64`. `--platform PLATFORM` -adds that platform's lock, so on the Mac -`devcapsule project init --platform linux-arm64` lets a project run natively. +runs that it has a lock for; an Apple Silicon Mac runs `linux-arm64` before +`linux-amd64`. On a host that runs more than one, init asks which platform a +new project is for, the native one by default. `--platform PLATFORM` answers +without the question, and adds that platform's lock to a project locked for +another, so on the Mac `devcapsule project init --platform linux-arm64` lets an +emulated project run natively. ### `wrap` diff --git a/engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md b/engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md index e495d6db..d8b8c027 100644 --- a/engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md +++ b/engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md @@ -26,15 +26,18 @@ the base recipe, on top of `macos-launcher`'s platform split. File references ar 1. Every component at once, not a first slice. 2. The arm64 base is built locally and approved per checkout; the owner publishes later. 3. On Apple Silicon, a project runs as `linux-arm64` when it has a `linux-arm64` lock, and - otherwise as `linux-amd64` under emulation. + otherwise as `linux-amd64` under emulation. Amended the same day, after the human's + first init of a new project: on a host that runs several platforms, init asks which a + new project is for, the native one by default ("depending on the current os just ask", + "default arm64 on arm"). 4. A new workstream on the macOS branch. ## 3. Design 1. **`Platform.LINUX_ARM64`.** Host detection stays in `platforms.py`: `runnable()`, every platform a host runs, native first (Apple Silicon: arm64, then amd64 under emulation), - and `current()`, the platform a host creates a new project for, `linux-amd64` on a Mac - while the arm64 base is unpublished, per the ruling in section 2. Which platform a + and `current()`, the host's native platform, a new project's default, per section 2's + amended ruling; init asks on a host of several platforms. Which platform a project runs as is the configuration's question: `configuration.storage.project_platform`, read-only, answers the first runnable platform the project has a lock for, else `current()`. `project init --platform` adds a platform's lock, and the project then runs diff --git a/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md index 96a1752e..5c1e0a31 100644 --- a/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md @@ -105,6 +105,13 @@ host X11 directory, a non-root user and a Docker group. From 9d285a5a, VSCodium natively (20261104T130201Z-392364, created with `init --platform linux-arm64` and verified without `--unverified`) and under emulation (20261104T130042Z-36f56a). +**2026-11-04, init asks for the platform.** The human's first init of a new project on the +Mac chose `linux-amd64` without a question; they ruled that init ask on a host of several +platforms, the native one by default ("depending on the current os just ask", "default +arm64 on arm"). `Platform.current()` is now the native platform, and init asks through its +elicitor before the identity questions: Enter, or a run without a terminal, takes it. A +project with a lock the host runs is not asked, and a Linux host never is. + ## Next Resumable Task Publish this workstream and `macos-launcher` once the SSH agent holds a key, and hand both