diff --git a/CURRENT-STATUS.md b/CURRENT-STATUS.md index 33ce5a78..6ed04b08 100644 --- a/CURRENT-STATUS.md +++ b/CURRENT-STATUS.md @@ -20,6 +20,8 @@ coordination branch. Detailed state belongs in each linked status file. | `website` | 2026-09-16 | Deliver the website autonomy experiment: repository-owned landing/docs/blog content, independently developable website submodule, local preview and publishing mechanism | paused 2026-09-30; producer migration merged in PR #152 and the versioned documentation live; W04-C, W09-C, W11-C retained | `ws-website/contract-v1` | [workstream status](engineering-docs/wip/2026-09-16-website/CURRENT-STATUS.md) | | `capsule-webconsole` | 2026-10-09 | Serve a token-gated web console from every capsule: configuration, versions, info, live resources, later records and workflow state | active; work order accepted 2026-10-09; the first iteration, deliverables 1 and 2, is three stacked pull requests, #184, #186 and the runtime-and-base slice, each reviewed by the Codex pair; the owner merges | `ws-capsule-webconsole/first-slice` | [workstream status](engineering-docs/wip/2026-10-09-capsule-webconsole/CURRENT-STATUS.md) | | `wrapped-projects` | 2026-11-02 | Wrap a foreign repository as a submodule of a small wrapper project and work on it in a capsule with three commands; FRC learning project first | active; PR1 (#333) accepted by the owner 2026-11-03, awaiting merge; the project-capabilities design handed to workflow-improvements | `ws-wrapped-projects/wrap` | [workstream status](engineering-docs/wip/2026-11-02-wrapped-projects/CURRENT-STATUS.md) | +| `macos-launcher` | 2026-11-03 | Run the DevCapsule launcher natively on Apple Silicon Macs against Docker Desktop, with capsules as `linux-amd64` under emulation | active 2026-11-03; slices 1 to 4 done, the PyCharm smoke passing on an Apple Silicon Mac; waits on publication and the owner's rulings | `ws-macos-launcher/host-port` | [workstream status](engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md) | +| `linux-arm64` | 2026-11-04 | Run capsules as `linux-arm64` as well as `linux-amd64`: natively on Apple Silicon Macs and on ARM Linux hosts | active 2026-11-04; slices 1 to 4 done, six IDEs running natively on Apple Silicon; waits on publication, the owner's arm64 base and rulings | `ws-linux-arm64/capsules` | [workstream status](engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md) | | `website` | 2026-09-16 | Deliver the website autonomy experiment: repository-owned landing/docs/blog content, independently developable website submodule, local preview and publishing mechanism | paused 2026-09-20 at owner request to switch to maintenance; experiment accepted, A−; blog integrated and published; content/parent-integration follow-ups remain | `ws-website/initial-cut`; renamed from `website/initial-cut`; preparation and backlog-handoff exceptions in status | [workstream status](engineering-docs/wip/2026-09-16-website/CURRENT-STATUS.md) | Paused and blocked workstreams remain open until they conclude successfully or diff --git a/devcapsule-src/devcapsule/assets/docker4pycharm/check-runtime-deps.sh b/devcapsule-src/devcapsule/assets/docker4pycharm/check-runtime-deps.sh index 06bac67c..b3cacf90 100755 --- a/devcapsule-src/devcapsule/assets/docker4pycharm/check-runtime-deps.sh +++ b/devcapsule-src/devcapsule/assets/docker4pycharm/check-runtime-deps.sh @@ -2,7 +2,11 @@ set -euo pipefail IDE_HOME="${IDE_HOME:-${PYCHARM_HOME:-/opt/pycharm}}" -SKIKO_LIB="$IDE_HOME/lib/skiko-awt-runtime-all/libskiko-linux-x64.so" +case "$(uname -m)" in + aarch64 | arm64) SKIKO_ARCH=arm64 ;; + *) SKIKO_ARCH=x64 ;; +esac +SKIKO_LIB="$IDE_HOME/lib/skiko-awt-runtime-all/libskiko-linux-$SKIKO_ARCH.so" FAILURES=0 pass() { diff --git a/devcapsule-src/devcapsule/assets/pycharm/check-runtime-deps.sh b/devcapsule-src/devcapsule/assets/pycharm/check-runtime-deps.sh index 06bac67c..b3cacf90 100644 --- a/devcapsule-src/devcapsule/assets/pycharm/check-runtime-deps.sh +++ b/devcapsule-src/devcapsule/assets/pycharm/check-runtime-deps.sh @@ -2,7 +2,11 @@ set -euo pipefail IDE_HOME="${IDE_HOME:-${PYCHARM_HOME:-/opt/pycharm}}" -SKIKO_LIB="$IDE_HOME/lib/skiko-awt-runtime-all/libskiko-linux-x64.so" +case "$(uname -m)" in + aarch64 | arm64) SKIKO_ARCH=arm64 ;; + *) SKIKO_ARCH=x64 ;; +esac +SKIKO_LIB="$IDE_HOME/lib/skiko-awt-runtime-all/libskiko-linux-$SKIKO_ARCH.so" FAILURES=0 pass() { diff --git a/devcapsule-src/devcapsule/commands/framework.py b/devcapsule-src/devcapsule/commands/framework.py index 3c704429..8f149c14 100644 --- a/devcapsule-src/devcapsule/commands/framework.py +++ b/devcapsule-src/devcapsule/commands/framework.py @@ -6,8 +6,9 @@ - The framework owns the command *tree*: discovery-independent group dispatch, group help, hidden commands, group-scoped options such as - ``project --path``, and the carrier-option grammar shared by ``init``, - ``run``, and the ``config`` family. + ``project --path``, the carrier-option grammar shared by ``init``, + ``run``, and the ``config`` family, and the ``--platform`` value shared by + ``project init`` and ``images build``. - :mod:`argparse` owns only *leaf* option parsing, declared per command in ``configure``. @@ -31,6 +32,7 @@ from typing import Any, ClassVar, Mapping, NoReturn, Sequence from devcapsule.compat import CliError +from devcapsule.platforms import Platform, UnsupportedPlatformError class UsageError(CliError): @@ -303,3 +305,17 @@ def carrier_answers( ) ) return tuple(answers) + + +def capsule_platform_argument(value: str) -> Platform: + """A ``--platform`` value: a capsule platform this host runs.""" + + try: + platform = Platform.parse(value) + runnable = Platform.runnable() + except UnsupportedPlatformError as exc: + raise argparse.ArgumentTypeError(str(exc)) from exc + if platform not in runnable: + names = ", ".join(str(member) for member in runnable) + raise argparse.ArgumentTypeError(f"this host runs {names} capsules, not {platform}") + return platform diff --git a/devcapsule-src/devcapsule/commands/images.py b/devcapsule-src/devcapsule/commands/images.py index 43e3f04d..92ea9b9b 100644 --- a/devcapsule-src/devcapsule/commands/images.py +++ b/devcapsule-src/devcapsule/commands/images.py @@ -19,11 +19,12 @@ build_base_image, resolved_root_image, ) -from devcapsule.commands.framework import Command, Group +from devcapsule.commands.framework import Command, Group, capsule_platform_argument from devcapsule.compat import CliError from devcapsule.environment_realization import optional_local_image, realize_environment from devcapsule.images.metadata import LocalImageRecord, inspect_local_image, list_local_images from devcapsule.materialization import ImageDetails +from devcapsule.platforms import Platform from devcapsule.configuration.execution import ( fresh_resolved_project, ) @@ -110,10 +111,20 @@ def configure(cls, parser: argparse.ArgumentParser) -> None: default="default", help="Build network mode forwarded to Docker buildx (default: default).", ) + parser.add_argument( + "--platform", + dest="capsule_platform", + type=capsule_platform_argument, + metavar="PLATFORM", + help="Capsule platform of a base build, such as linux-arm64; default: the one this host " + "creates projects for. An environment build is for its project's platform.", + ) @classmethod def run(cls, arguments: argparse.Namespace, context: object | None) -> int: if arguments.image_type == "environment": + if arguments.capsule_platform is not None: + raise CliError("--platform applies only to base builds; an environment build is for its project's platform.") return _build_environment( project=arguments.project, base_override=arguments.base_override, @@ -153,6 +164,7 @@ def run(cls, arguments: argparse.Namespace, context: object | None) -> int: build_base_image( options, network=arguments.network, + platform=arguments.capsule_platform or Platform.current(), ) image = inspect_local_image(arguments.tag) labels = dict(image.config.labels or {}) diff --git a/devcapsule-src/devcapsule/commands/project.py b/devcapsule-src/devcapsule/commands/project.py index d6e5fa73..e9c5cbfe 100644 --- a/devcapsule-src/devcapsule/commands/project.py +++ b/devcapsule-src/devcapsule/commands/project.py @@ -32,6 +32,7 @@ Command, Group, add_carrier_options, + capsule_platform_argument, carrier_answers, ) from devcapsule.components.catalog import COMPONENTS @@ -54,6 +55,7 @@ from devcapsule.runtime_configuration import CapsuleAccess from devcapsule.compat import CliError, DecisionPending import difflib +from devcapsule.configuration.capability_selection import lock_platform from devcapsule.configuration.capabilities import CapabilityPolicy, LocalCapabilities from devcapsule.configuration.console_requests import deferred_rows from devcapsule.configuration.listing import Listing, assess_checkout @@ -310,6 +312,11 @@ def configure(cls, parser: argparse.ArgumentParser) -> None: help="A capability the project offers and a developer may leave out, such as browser-automation. Repeatable.", ) _add_owner_options(parser) + parser.add_argument( + "--platform", type=capsule_platform_argument, metavar="PLATFORM", + help="Add the lock for this capsule platform, such as linux-arm64; the project then runs as it here. " + "Default: the platform this host runs the project as.", + ) _retired_option(parser, "--agent", "devcapsule project init --with CAPABILITY") parser.add_argument( "--regenerate", @@ -351,6 +358,7 @@ def run(cls, arguments: argparse.Namespace, context: object | None) -> int: answers=answers, regenerate=arguments.regenerate, allow_unverified=arguments.allow_unverified, + platform=arguments.platform, ) ) print(report.render()) @@ -755,9 +763,9 @@ def _load_checkout_documents(context: object | None) -> CheckoutDocuments | runt def _assess(documents: CheckoutDocuments) -> Listing: from devcapsule.configuration.authorization import authorized_base_selection from devcapsule.configuration.trust import HomeTrust - from devcapsule.platforms import Platform + from devcapsule.configuration.storage import project_platform from devcapsule.resolution_matrix import MATRICES, known_base_image - matrix = MATRICES[Platform.current()] + matrix = MATRICES[project_platform(documents.root)] answers = documents.checkout.get("authorization", {}) base_answer = answers.get("base-image", {}) if isinstance(answers, dict) else {} base_reference = base_answer.get("reference") if isinstance(base_answer, dict) else None @@ -1702,12 +1710,12 @@ def run(cls, arguments: argparse.Namespace, context: object | None) -> int: def _local_selection(context: object | None) -> tuple[Path, dict[str, Any], LocalCapabilities, dict[str, str]]: """The checkout's current selection and the catalog's IDE names, for the selection verbs.""" - from devcapsule.platforms import Platform + from devcapsule.configuration.storage import project_platform from devcapsule.resolution_matrix import MATRICES root, manifest = manifest_for(_project_context(context).start_path()) record = find_checkout_record(manifest, root) current = LocalCapabilities.read(load_checkout(record, manifest, root)) if record is not None else LocalCapabilities() - matrix = MATRICES[Platform.current()] + matrix = MATRICES[project_platform(root)] surfaces = {name: matrix.resolve_provider(name) or name for name in matrix.ide_capabilities()} return root, manifest, current, surfaces @@ -2436,6 +2444,7 @@ def prepare_display(base: ImageDetails) -> None: enable_host_browser=selected_host_browser, display_transport=display_transport, web_console=web_console, + capsule_platform=lock_platform(lock), ) ) if command_report is not None: diff --git a/devcapsule-src/devcapsule/components/codex.py b/devcapsule-src/devcapsule/components/codex.py index f90bc744..00f87ae9 100644 --- a/devcapsule-src/devcapsule/components/codex.py +++ b/devcapsule-src/devcapsule/components/codex.py @@ -71,7 +71,8 @@ class CodexComponent(ComponentDefinition): """Trusted Codex component implementation.""" def distribution_channel(self) -> NpmChannel: - return NpmChannel(CODEX_PACKAGE, {"linux-amd64": "@openai/codex-linux-x64"}) + return NpmChannel(CODEX_PACKAGE, {"linux-amd64": "@openai/codex-linux-x64", + "linux-arm64": "@openai/codex-linux-arm64"}) @property def id(self) -> str: diff --git a/devcapsule-src/devcapsule/components/eclipse.py b/devcapsule-src/devcapsule/components/eclipse.py index 3d2d0dd1..be4b344b 100644 --- a/devcapsule-src/devcapsule/components/eclipse.py +++ b/devcapsule-src/devcapsule/components/eclipse.py @@ -59,8 +59,8 @@ def locked_artifacts(self, metadata: Mapping[str, object], platform: str) -> tup # dependencies use the same acquisition/hash verification before an # offline dpkg installation. The set is specific to the Ubuntu 24.04 base. packages = metadata.get("native-packages") - if platform != "linux-amd64" or not isinstance(packages, dict) or not packages: - raise CliError("Eclipse requires pinned native-packages for linux-amd64.") + if platform not in ("linux-amd64", "linux-arm64") or not isinstance(packages, dict) or not packages: + raise CliError(f"Eclipse requires pinned native-packages for {platform}.") artifacts = [] for package, entry in packages.items(): if not isinstance(entry, dict) or any(not isinstance(entry.get(key), str) or not entry[key] diff --git a/devcapsule-src/devcapsule/components/linux_arm64_pins.py b/devcapsule-src/devcapsule/components/linux_arm64_pins.py new file mode 100644 index 00000000..d48a3b9f --- /dev/null +++ b/devcapsule-src/devcapsule/components/linux_arm64_pins.py @@ -0,0 +1,475 @@ +"""linux-arm64 artifact pins, collected 2026-11-04. + +Each entry is the arm64 build of a version the linux-amd64 matrix pins: the +same version, the vendor's arm64 artifact, and a sha256 computed from the +downloaded bytes, which matched every vendor checksum, npm integrity value +and signed Ubuntu index available (Ubuntu's InRelease files verified with +gpgv against the archive signing key). Notes on particular entries: + +- Eclipse's WebKitGTK closure is the amd64 closure's package versions from + ports.ubuntu.com, except libsoup: Ubuntu removed 3.4.4-5ubuntu0.8 from + every noble pocket on 2026-10-08, so both libsoup packages are + 3.4.4-5ubuntu0.10 from noble-security. Architecture-independent packages + are the amd64 closure's own bytes. libunwind8 is added: arm64's + libgstreamer1.0-0 depends on it. +- Playwright 1.63.0's Chromium comes from Chrome for Testing's linux-arm64 + build of 153.0.8010.12 on Playwright's CDN, byte-identical to Google's. +- Antigravity's arm64 builds live under ``linux-arm/``; VS Code's arm64 + archives are named by the update service, not by the x64 names' pattern. + +Keys are ``(component id, version)``, as the linux-amd64 matrix pins them. +""" + +from typing import Any + +LINUX_ARM64_ECLIPSE_NATIVE_PACKAGES: dict[str, dict[str, str]] = {'bubblewrap': {'sha256': 'dce3a2e5f0302eab717a64fe9c9411fe2d40469e2a7946e0f2ce1420a61025c3', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/b/bubblewrap/bubblewrap_0.9.0-1ubuntu0.3_arm64.deb', + 'version': '0.9.0-1ubuntu0.3'}, + 'dictionaries-common': {'sha256': '601e7cb4d662aca433785eaafd8094e51b1386aab5f5aef6572a1a565b80f21e', + 'url': 'https://archive.ubuntu.com/ubuntu/pool/main/d/dictionaries-common/dictionaries-common_1.29.7_all.deb', + 'version': '1.29.7'}, + 'emacsen-common': {'sha256': 'd802bbf9167ad1d8aa633b65684439e08672379f15bf752662d7fd7b9798dce1', + 'url': 'https://archive.ubuntu.com/ubuntu/pool/main/e/emacsen-common/emacsen-common_3.0.5_all.deb', + 'version': '3.0.5'}, + 'glib-networking': {'sha256': 'f229b9b01b384d73bf17d50a549802432bbcca36bb3eb74f0dac11f628f5c380', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/g/glib-networking/glib-networking_2.80.0-1build1_arm64.deb', + 'version': '2.80.0-1build1'}, + 'glib-networking-common': {'sha256': 'c9bc3b5c3dc8a75e752efcf246e5ef2d3b9e80cd1301fd37fcb0c36562062931', + 'url': 'https://archive.ubuntu.com/ubuntu/pool/main/g/glib-networking/glib-networking-common_2.80.0-1build1_all.deb', + 'version': '2.80.0-1build1'}, + 'glib-networking-services': {'sha256': 'a93e273d521c2a06ab579a17af4bfc1e4270f48b01af5cd957f7a1c82202c5fd', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/g/glib-networking/glib-networking-services_2.80.0-1build1_arm64.deb', + 'version': '2.80.0-1build1'}, + 'gsettings-desktop-schemas': {'sha256': '4d15441e9197f95b29c0d2845d8fcdbf63fa4ac0c68fb221820ccd24131e3436', + 'url': 'https://archive.ubuntu.com/ubuntu/pool/main/g/gsettings-desktop-schemas/gsettings-desktop-schemas_46.1-0ubuntu1_all.deb', + 'version': '46.1-0ubuntu1'}, + 'gstreamer1.0-plugins-base': {'sha256': 'dbbe644b135e95ef2cb035c27afb6eb05ba91c72ef63ef12a15de221a264616e', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/g/gst-plugins-base1.0/gstreamer1.0-plugins-base_1.24.2-1ubuntu0.5_arm64.deb', + 'version': '1.24.2-1ubuntu0.5'}, + 'gstreamer1.0-plugins-good': {'sha256': 'c1dba2da2de061f8df51eaedf3ee22ac4aefaae6356d79b0fd9918cb64b8beae', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/g/gst-plugins-good1.0/gstreamer1.0-plugins-good_1.24.2-1ubuntu1.8_arm64.deb', + 'version': '1.24.2-1ubuntu1.8'}, + 'hunspell-en-us': {'sha256': 'd231c94b54792215038a4f719c562deafe8efe0a7710c0c7218974cab513330c', + 'url': 'https://archive.ubuntu.com/ubuntu/pool/main/s/scowl/hunspell-en-us_2020.12.07-2_all.deb', + 'version': '1:2020.12.07-2'}, + 'iso-codes': {'sha256': 'cafca0a3a1f07b44fbba53ab7a1f1d9755e6b80cc041173afe2b52e53bed97a7', + 'url': 'https://archive.ubuntu.com/ubuntu/pool/main/i/iso-codes/iso-codes_4.16.0-1_all.deb', + 'version': '4.16.0-1'}, + 'libaa1': {'sha256': 'c6ba7e5a2d291a92fc3f9604ac298193e0102dc02c3b7d67c112819a7a9a2511', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/a/aalib/libaa1_1.4p5-51.1_arm64.deb', + 'version': '1.4p5-51.1'}, + 'libaspell15': {'sha256': '7a3da894c71c254c7ed7074284bb035eabb44f0e56cabaf96186e345813d1d6f', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/a/aspell/libaspell15_0.60.8.1-1build1_arm64.deb', + 'version': '0.60.8.1-1build1'}, + 'libasyncns0': {'sha256': 'acda11529a3bdc32961cec888060417d835be9626b00ab8f34607f7935cda11f', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/liba/libasyncns/libasyncns0_0.8-6build4_arm64.deb', + 'version': '0.8-6build4'}, + 'libavc1394-0': {'sha256': 'b707ec893678e9dec0b80e4481ed2dc581eac3d21de2be65984121640e3d5afe', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/liba/libavc1394/libavc1394-0_0.5.4-5build3_arm64.deb', + 'version': '0.5.4-5build3'}, + 'libcaca0': {'sha256': '95dffb776d959bee43bc2db755c1e4330547b9449eb20f48f8ba1dfddf435cf2', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libc/libcaca/libcaca0_0.99.beta20-4ubuntu0.2_arm64.deb', + 'version': '0.99.beta20-4ubuntu0.2'}, + 'libcdparanoia0': {'sha256': 'eb23ce43347dc8f734f66c24109198309d4844d9d9045ae9ca97cfa25061f63e', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/c/cdparanoia/libcdparanoia0_3.10.2%2bdebian-14build3_arm64.deb', + 'version': '3.10.2+debian-14build3'}, + 'libduktape207': {'sha256': '21cae9d4994eba43aa41623e55186df6434301faa47176196dbf33987550fec1', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/d/duktape/libduktape207_2.7.0%2btests-0ubuntu3_arm64.deb', + 'version': '2.7.0+tests-0ubuntu3'}, + 'libdv4t64': {'sha256': '0d91e6739333a460c61467e55156b38ac426e40c92bda606197e91b6f98864be', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libd/libdv/libdv4t64_1.0.0-17.1build1_arm64.deb', + 'version': '1.0.0-17.1build1'}, + 'libenchant-2-2': {'sha256': 'a8334fe77eabef07ead474657e38faeb7590da494d012480feb230e05a70b7ed', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/e/enchant-2/libenchant-2-2_2.3.3-2build2_arm64.deb', + 'version': '2.3.3-2build2'}, + 'libevdev2': {'sha256': '4fd17e552a00e3a2dc5d39a6578b26a1ac973afc80384d165647d72441f3a064', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libe/libevdev/libevdev2_1.13.1%2bdfsg-1build1_arm64.deb', + 'version': '1.13.1+dfsg-1build1'}, + 'libflac12t64': {'sha256': '9738c071c167b2baa8dd1a509d85076150108953a1429de839280f87b0b01b83', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/f/flac/libflac12t64_1.4.3%2bds-2.1ubuntu2_arm64.deb', + 'version': '1.4.3+ds-2.1ubuntu2'}, + 'libgstreamer-gl1.0-0': {'sha256': '569e78b1fbfb50fa57271157dc45cc3ebd682d3a8a940e1116cbf64c12cd043f', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/g/gst-plugins-base1.0/libgstreamer-gl1.0-0_1.24.2-1ubuntu0.5_arm64.deb', + 'version': '1.24.2-1ubuntu0.5'}, + 'libgstreamer-plugins-base1.0-0': {'sha256': 'cf52b2547943fc6444a6d674c2fe1fc2f9cc9ff1e3ff5f4a88b8f07da4a55dd4', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/g/gst-plugins-base1.0/libgstreamer-plugins-base1.0-0_1.24.2-1ubuntu0.5_arm64.deb', + 'version': '1.24.2-1ubuntu0.5'}, + 'libgstreamer-plugins-good1.0-0': {'sha256': 'a67bd34f000e34163221749b20aa605487ba1a3fd562a07b81d2060eb3006d5d', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/g/gst-plugins-good1.0/libgstreamer-plugins-good1.0-0_1.24.2-1ubuntu1.8_arm64.deb', + 'version': '1.24.2-1ubuntu1.8'}, + 'libgstreamer1.0-0': {'sha256': '48c930493054b261621ca7ea887ee4c49488bb216f0a42d8572189e5a3e291f0', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/g/gstreamer1.0/libgstreamer1.0-0_1.24.2-1ubuntu0.1_arm64.deb', + 'version': '1.24.2-1ubuntu0.1'}, + 'libgudev-1.0-0': {'sha256': 'ce395eea11fdbd542597c35d10cb931e288243467e9d2e14654225cd8e80dac5', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libg/libgudev/libgudev-1.0-0_238-5ubuntu1_arm64.deb', + 'version': '1:238-5ubuntu1'}, + 'libharfbuzz-icu0': {'sha256': '92f1311987c69b90432408c19e28b4796ff7584927cb55c9cec74bd3bab9efb4', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/h/harfbuzz/libharfbuzz-icu0_8.3.0-2build2_arm64.deb', + 'version': '8.3.0-2build2'}, + 'libhunspell-1.7-0': {'sha256': '905d26e0c736b7334027ff9a0e12f8faa913d33ff861c0ca5c9c20b43071bca7', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/h/hunspell/libhunspell-1.7-0_1.7.2%2breally1.7.2-10build3_arm64.deb', + 'version': '1.7.2+really1.7.2-10build3'}, + 'libhyphen0': {'sha256': 'ac159cf84661f8027d0f4274d9ff559eb41ab9e2a476d5d63cf2573bda542c94', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/h/hyphen/libhyphen0_2.8.8-7build3_arm64.deb', + 'version': '2.8.8-7build3'}, + 'libiec61883-0': {'sha256': '83644951ab8aff10f530066e74129431f723dc5b810e14025520b4f70bb58bbe', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libi/libiec61883/libiec61883-0_1.2.0-6build1_arm64.deb', + 'version': '1.2.0-6build1'}, + 'libjavascriptcoregtk-4.1-0': {'sha256': '936ed381c7cfa24ee753c01da8c4bc539e8911d696c95e2bc9ded98e44cd47ca', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/w/webkit2gtk/libjavascriptcoregtk-4.1-0_2.52.6-0ubuntu0.24.04.1_arm64.deb', + 'version': '2.52.6-0ubuntu0.24.04.1'}, + 'libmanette-0.2-0': {'sha256': '9e5585c1d791e192f48ed4d6579c5dbdbc79f2330afc4f7998fd5e72e654bf2b', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libm/libmanette/libmanette-0.2-0_0.2.7-1build2_arm64.deb', + 'version': '0.2.7-1build2'}, + 'libmp3lame0': {'sha256': 'a12e5e953294f424fba8a9fa0c47e081a0216150b277c4b02da1468d4b78507f', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/l/lame/libmp3lame0_3.100-6build1_arm64.deb', + 'version': '3.100-6build1'}, + 'libmpg123-0t64': {'sha256': '7f78498103c42cebe15f1283fb2dd98286069a59f29aca4de372bc98f70d06d2', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/m/mpg123/libmpg123-0t64_1.32.5-1ubuntu1.1_arm64.deb', + 'version': '1.32.5-1ubuntu1.1'}, + 'libogg0': {'sha256': 'ed92eb736754ccdea645db7a12734961d2e686369f4fb4c070126b70462b02e1', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libo/libogg/libogg0_1.3.5-3build1_arm64.deb', + 'version': '1.3.5-3build1'}, + 'libopus0': {'sha256': '9e0b07689ce79772e62f8d4d1849bde1b1fc5b041bef85b791b61d7e3fee0ff7', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/o/opus/libopus0_1.4-1build1_arm64.deb', + 'version': '1.4-1build1'}, + 'liborc-0.4-0t64': {'sha256': 'fb8248d89d37a06145b8112aed200ffd2f4960d294d95ddc8471a0b43beafb3c', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/o/orc/liborc-0.4-0t64_0.4.38-1ubuntu0.1_arm64.deb', + 'version': '1:0.4.38-1ubuntu0.1'}, + 'libproxy1v5': {'sha256': '4ea61580d2b5c6ec4f05445ed63618130743256d0208501cd5af14c28764040b', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libp/libproxy/libproxy1v5_0.5.4-4build1_arm64.deb', + 'version': '0.5.4-4build1'}, + 'libpulse0': {'sha256': '863bf58ca712b687873c9f57bac2968555b0f1bcdeeb79265da4130d0102fea3', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/p/pulseaudio/libpulse0_16.1%2bdfsg1-2ubuntu10.1_arm64.deb', + 'version': '1:16.1+dfsg1-2ubuntu10.1'}, + 'libraw1394-11': {'sha256': '3a55e774189eb21b220e5cbf14a635e627f370dba0a52bbe6eea6b5a785e0a87', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libr/libraw1394/libraw1394-11_2.1.2-2build3_arm64.deb', + 'version': '2.1.2-2build3'}, + 'libsecret-1-0': {'sha256': 'b489847c20b4990e9e97dc701215c2af40ac071d9f1d465196a8a215b17ae2a4', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libs/libsecret/libsecret-1-0_0.21.4-1build3_arm64.deb', + 'version': '0.21.4-1build3'}, + 'libsecret-common': {'sha256': '41866e9026e451fd25a4cbe887fa56c43219f9f3eda88d1831452ba6d6b8bb5b', + 'url': 'https://archive.ubuntu.com/ubuntu/pool/main/libs/libsecret/libsecret-common_0.21.4-1build3_all.deb', + 'version': '0.21.4-1build3'}, + 'libshout3': {'sha256': '3099f4bdaf6bf7ee418a4db7b7d3a191c9a36902164dc22d62e7e10b40ffe94f', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libs/libshout/libshout3_2.4.6-1build2_arm64.deb', + 'version': '2.4.6-1build2'}, + 'libslang2': {'sha256': '77b4b81dd3d5783772dc0b742c79680ce1eef18a771cf1f421e9a9e5eaf2d71e', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/s/slang2/libslang2_2.3.3-3build2_arm64.deb', + 'version': '2.3.3-3build2'}, + 'libsndfile1': {'sha256': '3fc5c0cb78110c739a0aa9cedad1812c04a5b3ee9068a4775233b51d8dc5c0f7', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libs/libsndfile/libsndfile1_1.2.2-1ubuntu5.24.04.1_arm64.deb', + 'version': '1.2.2-1ubuntu5.24.04.1'}, + 'libsoup-3.0-0': {'sha256': '3831f49dc17ed43600c531d3d77c5bcec83e3aba6bc719fc0e83c58234115595', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libs/libsoup3/libsoup-3.0-0_3.4.4-5ubuntu0.10_arm64.deb', + 'version': '3.4.4-5ubuntu0.10'}, + 'libsoup-3.0-common': {'sha256': '89f066fe021cb2991750dc9a65fb64cc24a467ab32b5a1a722e377244b6fda4c', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libs/libsoup3/libsoup-3.0-common_3.4.4-5ubuntu0.10_all.deb', + 'version': '3.4.4-5ubuntu0.10'}, + 'libspeex1': {'sha256': '8ba1a6feb898b9f28f1c071c96ad0a992b4d449ad75aecb791d1ebd2a43c0ada', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/s/speex/libspeex1_1.2.1-2ubuntu2.24.04.1_arm64.deb', + 'version': '1.2.1-2ubuntu2.24.04.1'}, + 'libtag1v5': {'sha256': 'd6d23bb62694e291775b65ce618a04393d9b2ca61c88c765a05cd9df6e1cba81', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/t/taglib/libtag1v5_1.13.1-1build1_arm64.deb', + 'version': '1.13.1-1build1'}, + 'libtag1v5-vanilla': {'sha256': '27db0f02f005a969e7e557e3ca7670a1f215814f4f446bab155446ace17d496c', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/t/taglib/libtag1v5-vanilla_1.13.1-1build1_arm64.deb', + 'version': '1.13.1-1build1'}, + 'libtext-iconv-perl': {'sha256': '25e749dbe3331ec8c04dfde36cc3e24d4871a2f7a46dff4282d521973f4698b6', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libt/libtext-iconv-perl/libtext-iconv-perl_1.7-8build3_arm64.deb', + 'version': '1.7-8build3'}, + 'libtheora0': {'sha256': '78ebaa1c851465dac9e13532623a4e41d28ed55f3df0353daf7c29d96a2e2b87', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libt/libtheora/libtheora0_1.1.1%2bdfsg.1-16.1build3_arm64.deb', + 'version': '1.1.1+dfsg.1-16.1build3'}, + 'libtwolame0': {'sha256': 'f2fffdedec9f9bc58fc700ccbc146c50a5a1d5ae2e6881e501bd727234aa5d16', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/t/twolame/libtwolame0_0.4.0-2build3_arm64.deb', + 'version': '0.4.0-2build3'}, + 'libunwind8': {'sha256': '26f2bf9c0d9b42d8230aa2c070076054b0ed06519bb9141c05a9c3164732fef1', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libu/libunwind/libunwind8_1.6.2-3build1.1_arm64.deb', + 'version': '1.6.2-3build1.1'}, + 'libv4l-0t64': {'sha256': '0efed54458a4a3da075defb2675b243d7123115d26219ea2ffb88750e777f0e3', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/v/v4l-utils/libv4l-0t64_1.26.1-4build3_arm64.deb', + 'version': '1.26.1-4build3'}, + 'libv4lconvert0t64': {'sha256': 'cfb01d4343d1ed4a92476df431d4ef89f6871708c8befd6ef8e711a40dfc0808', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/v/v4l-utils/libv4lconvert0t64_1.26.1-4build3_arm64.deb', + 'version': '1.26.1-4build3'}, + 'libvisual-0.4-0': {'sha256': 'c9b6a8bcf0e9d57c1a32e363dc391d3ad3db1a01b51a406b5c847d32680baa72', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libv/libvisual/libvisual-0.4-0_0.4.2-2build1_arm64.deb', + 'version': '0.4.2-2build1'}, + 'libvorbis0a': {'sha256': 'e21bd74f3bf593b4697f77c6b52bfbd34cb5dfc5ed5f115545d31e43599c1ef8', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libv/libvorbis/libvorbis0a_1.3.7-1build3_arm64.deb', + 'version': '1.3.7-1build3'}, + 'libvorbisenc2': {'sha256': 'a6ab660d303d44fbf040c07816a7072a40240d1639c37f6246624dba8d28407a', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libv/libvorbis/libvorbisenc2_1.3.7-1build3_arm64.deb', + 'version': '1.3.7-1build3'}, + 'libvpx9': {'sha256': '0a11569cb76d303978bb8d1a97cb012d96baadb5e093e6936f5b53de802241ec', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libv/libvpx/libvpx9_1.14.0-1ubuntu2.3_arm64.deb', + 'version': '1.14.0-1ubuntu2.3'}, + 'libwavpack1': {'sha256': 'fd86ac4ecc6aaa218744e38ba7e8c0d4bb13f45c5508c3247e059fdcee33ad55', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/w/wavpack/libwavpack1_5.6.0-1build1_arm64.deb', + 'version': '5.6.0-1build1'}, + 'libwayland-server0': {'sha256': '2b6dec0d40070685f4ce346d20b064fc5e118a3149f30be88465251985b496ed', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/w/wayland/libwayland-server0_1.22.0-2.1build1_arm64.deb', + 'version': '1.22.0-2.1build1'}, + 'libwebkit2gtk-4.1-0': {'sha256': '508f97ca47c45464c0e43480ebe983a0b4b484dd1a76031272f762662943b7c9', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/w/webkit2gtk/libwebkit2gtk-4.1-0_2.52.6-0ubuntu0.24.04.1_arm64.deb', + 'version': '2.52.6-0ubuntu0.24.04.1'}, + 'libxslt1.1': {'sha256': '1e8b809859ddf2c827de14152741e74ea2f83907c7c06cba470c6cc75624ba49', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/libx/libxslt/libxslt1.1_1.1.39-0exp1ubuntu0.24.04.3_arm64.deb', + 'version': '1.1.39-0exp1ubuntu0.24.04.3'}, + 'session-migration': {'sha256': 'b310677c0cdd3410d13ae66515c5c0836ea0c36e02f2dbde89a9c76ebcefd565', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/s/session-migration/session-migration_0.3.9build1_arm64.deb', + 'version': '0.3.9build1'}, + 'xdg-dbus-proxy': {'sha256': '9ff35a693536fd898ed78be84c45a668e5edbfa30dd53884004ebef7212deebf', + 'url': 'https://ports.ubuntu.com/ubuntu-ports/pool/main/x/xdg-dbus-proxy/xdg-dbus-proxy_0.1.5-1ubuntu0.2_arm64.deb', + 'version': '0.1.5-1ubuntu0.2'}} + +LINUX_ARM64_PLAYWRIGHT_PIN: dict[str, Any] = {'version': '1.63.0', + 'delivery-policy': 'local-materialization', + 'license': 'Apache-2.0', + 'platform': 'linux-arm64', + 'artifacts': {'greenlet': {'format': 'python-wheel', + 'url': 'https://files.pythonhosted.org/packages/27/89/366d2af5061eeefa5012f510d95a99c8620dcc457609838db4d538820318/greenlet-3.5.6-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl', + 'sha256': 'e85880b538e59a59f55117b81f208a6660ad5ac328aad9305f812d9b8bc67a0f', + 'destination': '/opt/playwright'}, + 'playwright': {'format': 'python-wheel', + 'url': 'https://files.pythonhosted.org/packages/f3/82/3d85505284c5a210f2da6c07b8f757524e79d1fba9cfdafe1eafb766ae59/playwright-1.63.0-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl', + 'sha256': '354e15b29503565fc598b89f16fbe070459343bef9d7498a93e304864000c6a7', + 'destination': '/opt/playwright'}, + 'pyee': {'format': 'python-wheel', + 'url': 'https://files.pythonhosted.org/packages/a0/c4/b4d4827c93ef43c01f599ef31453ccc1c132b353284fc6c87d535c233129/pyee-13.0.1-py3-none-any.whl', + 'sha256': 'af2f8fede4171ef667dfded53f96e2ed0d6e6bd7ee3bb46437f77e3b57689228', + 'destination': '/opt/playwright'}, + 'typing_extensions': {'format': 'python-wheel', + 'url': 'https://files.pythonhosted.org/packages/49/d3/b8441a820a491ddfc024b0b0cf0393375b75ea13866d9c66727e54c2fc80/typing_extensions-4.16.0-py3-none-any.whl', + 'sha256': '481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8', + 'destination': '/opt/playwright'}, + 'chromium-1243': {'format': 'zip-directory', + 'url': 'https://cdn.playwright.dev/builds/cft/153.0.8010.12/linux-arm64/chrome-linux-arm64.zip', + 'sha256': '7d8a4b4ff289efe44a06501a519df142c4c18fff7aaf1c1401a3fbb12b3bd069', + 'destination': '/opt/playwright/browsers/chromium-1243'}, + 'chromium_headless_shell-1243': {'format': 'zip-directory', + 'url': 'https://cdn.playwright.dev/builds/cft/153.0.8010.12/linux-arm64/chrome-headless-shell-linux-arm64.zip', + 'sha256': 'd433c45172c7836e38124fe545f767b02210bfb43a6262f08a297473a8e91c99', + 'destination': '/opt/playwright/browsers/chromium_headless_shell-1243'}, + 'ffmpeg-1011': {'format': 'zip-directory', + 'url': 'https://cdn.playwright.dev/dbazure/download/playwright/builds/ffmpeg/1011/ffmpeg-linux-arm64.zip', + 'sha256': '2628c03f05318ff812c8c9baaf207dea2ddf53e818c0dc936714b0fbe3afb009', + 'destination': '/opt/playwright/browsers/ffmpeg-1011'}}} + +COMPONENT_LOCK_TABLES: dict[tuple[str, str], dict[str, Any]] = { + ('pycharm', '2026.2.0.1'): {'version': '2026.2.0.1', + 'variant': 'professional', + 'delivery-policy': 'local-materialization', + 'url': 'https://download.jetbrains.com/python/pycharm-2026.2.0.1-aarch64.tar.gz', + 'sha256': 'f29b6a782a58fabb232715db8fef9765bfdaebc6eb590ed462c4f96562227427'}, + ('pycharm', '2026.2.4'): {'version': '2026.2.4', + 'variant': 'professional', + 'delivery-policy': 'local-materialization', + 'url': 'https://download.jetbrains.com/python/pycharm-2026.2.4-aarch64.tar.gz', + 'sha256': '9ce06a9551e17c68a5741ee70ae4203dcc530935fa5e105df171d600e55a2bbf'}, + ('intellij', '2026.2.3'): {'version': '2026.2.3', + 'variant': 'unified', + 'delivery-policy': 'local-materialization', + 'url': 'https://download.jetbrains.com/idea/idea-2026.2.3-aarch64.tar.gz', + 'sha256': '873286dd6406971a2311827edd66c56060c345582a4a6b1774e3925183b02ec5'}, + ('intellij', '2026.2.4'): {'version': '2026.2.4', + 'variant': 'unified', + 'delivery-policy': 'local-materialization', + 'url': 'https://download.jetbrains.com/idea/idea-2026.2.4-aarch64.tar.gz', + 'sha256': '158b97dfbc3a88075fa14fcf5f9f2c17b339f4527e4245b3a33caf18050c2ae2'}, + ('eclipse', '2026-09-R'): {'version': '2026-09-R', + 'variant': 'java', + 'delivery-policy': 'local-materialization', + 'terms-url': 'https://www.eclipse.org/legal/epl-2.0/', + 'url': 'https://download.eclipse.org/technology/epp/downloads/release/2026-09/R/eclipse-java-2026-09-R-linux-gtk-aarch64.tar.gz', + 'sha256': '37f8bff91ce2af38bb691ab0608e1184af48cb48ad04da2bb09a9568ba905f8d', + 'upstream-sha512': 'c95453e0170efff6294a746b53e2f83f857aeb2309dbc20b933de18664f7d7e717f071a08a96d46797c2191014e1c8505da8370e45fff83a91396636d29a3091', + 'native-packages': LINUX_ARM64_ECLIPSE_NATIVE_PACKAGES}, + ('rider', '2026.2.3.1'): {'version': '2026.2.3.1', + 'delivery-policy': 'local-materialization', + 'license': 'Proprietary', + 'terms-url': 'https://www.jetbrains.com/legal/docs/toolbox/license/', + 'url': 'https://download.jetbrains.com/rider/JetBrains.Rider-2026.2.3.1-aarch64.tar.gz', + 'sha256': '8239c1e7c0352a9fb79dcbf68fb9550d64b215606a635ed88981291d92995a61'}, + ('rider', '2026.2.4'): {'version': '2026.2.4', + 'delivery-policy': 'local-materialization', + 'license': 'Proprietary', + 'terms-url': 'https://www.jetbrains.com/legal/docs/toolbox/license/', + 'url': 'https://download.jetbrains.com/rider/JetBrains.Rider-2026.2.4-aarch64.tar.gz', + 'sha256': 'adaaadc42dbfe2efa8e3aa96e487ba7b7ea33858b6bc1aad8ac5024205555ecd'}, + ('dotnet-sdk', '10.0.401'): {'version': '10.0.401', + 'platform': 'linux-arm64', + 'delivery-policy': 'local-materialization', + 'license': 'MIT', + 'url': 'https://builds.dotnet.microsoft.com/dotnet/Sdk/10.0.401/dotnet-sdk-10.0.401-linux-arm64.tar.gz', + 'sha256': '91d3d67f2ed065909bd2e1ba50a37192aff6df15178304939d5b55634d72da6a', + 'upstream-sha512': '58ace73ced6b4360754689a686bdfb8a317f4da6cb8bb416dbc7d0ba9f47e43e3c09f5eb1f1a1cfaacbd10df9558da4882bf2a5e195d6ab56a02c1f9f76102ed'}, + ('codium', '1.126.04524'): {'version': '1.126.04524', + 'delivery-policy': 'local-materialization', + 'license': 'MIT', + 'url': 'https://github.com/VSCodium/vscodium/releases/download/1.126.04524/VSCodium-linux-arm64-1.126.04524.tar.gz', + 'sha256': '73d87d46d4dc208fe12c0497dc607aab0a6e2bf332f54a0b6826a3a1aa32bc34'}, + ('codium', '1.143.17232'): {'version': '1.143.17232', + 'delivery-policy': 'local-materialization', + 'license': 'MIT', + 'url': 'https://github.com/VSCodium/vscodium/releases/download/1.143.17232/VSCodium-linux-arm64-1.143.17232.tar.gz', + 'sha256': '9f270ae242561812acd02be9cf5c13c848da359b356327adfe44b815a9532309'}, + ('vscode', '1.143.1'): {'version': '1.143.1', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'vscode-download', + 'license': 'Proprietary', + 'terms-url': 'https://code.visualstudio.com/license', + 'url': 'https://vscode.download.prss.microsoft.com/dbazure/download/stable/e646868227c3e3a9e3f24277c8c7220be42c0be7/code-stable-arm64-1792691733.tar.gz', + 'sha256': 'e84bf20d112fdb73c29b85b317ba50d5a48bea05c0711509669c8466639e9eb6'}, + ('vscode', '1.144.0'): {'version': '1.144.0', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'vscode-download', + 'license': 'Proprietary', + 'terms-url': 'https://code.visualstudio.com/license', + 'url': 'https://vscode.download.prss.microsoft.com/dbazure/download/stable/2c33b50b16c472efe58357d1d2a8a5414bc23fbf/code-stable-arm64-1793133773.tar.gz', + 'sha256': 'c1758f2144c68b295fb2a3543e93bc03ab99f8ea715cfd0560377092aba0f198'}, + ('codex', '0.145.0'): {'version': '0.145.0', + 'delivery-policy': 'local-materialization', + 'license': 'Apache-2.0', + 'npm-package': '@openai/codex', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.145.0.tgz', + 'sha256': '416399796cac371d1a033b17f34b08ba9b25c8f298a5b9d00e10f72c3b128c8d', + 'artifacts': {'linux-arm64': {'npm-package': '@openai/codex-linux-arm64', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.145.0-linux-arm64.tgz', + 'sha256': 'b78c57e172b2f18e5969ae26183253cd3cdd9abb3b424a8f7334f4b5530c2b27'}}}, + ('codex', '0.153.0'): {'version': '0.153.0', + 'delivery-policy': 'local-materialization', + 'license': 'Apache-2.0', + 'npm-package': '@openai/codex', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.153.0.tgz', + 'sha256': '0dc1968cc6075929d70d7ab1421122743a9f8237a8cc9ac69e2e8f2768798fef', + 'artifacts': {'linux-arm64': {'npm-package': '@openai/codex-linux-arm64', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.153.0-linux-arm64.tgz', + 'sha256': '1c65fda30a919908688147b67b1b195ab635bd4a5eac49ffa7be527a13af0a09'}}}, + ('codex', '0.153.4'): {'version': '0.153.4', + 'delivery-policy': 'local-materialization', + 'license': 'Apache-2.0', + 'npm-package': '@openai/codex', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.153.4.tgz', + 'sha256': 'fd04263c1adfa1d285c6c0ad86a97cab508d3012ee9eab80a99f773cc4b2fb3a', + 'artifacts': {'linux-arm64': {'npm-package': '@openai/codex-linux-arm64', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.153.4-linux-arm64.tgz', + 'sha256': '439c0dd0d6923f607b4e5cd1e3079c12f0b86f6e5007f07e377d6ad25e2d7bb9'}}}, + ('codex', '0.157.1'): {'version': '0.157.1', + 'delivery-policy': 'local-materialization', + 'license': 'Apache-2.0', + 'npm-package': '@openai/codex', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.157.1.tgz', + 'sha256': '813e2a944f4474b7e1826d9ce8bf696ad8d80c68f43f7a7067beec51fa8f5b73', + 'artifacts': {'linux-arm64': {'npm-package': '@openai/codex-linux-arm64', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.157.1-linux-arm64.tgz', + 'sha256': 'b73d482c0963a69d5cece5fe16e919473359937c22fb4499702ea35870e1824f'}}}, + ('codex', '0.163.0'): {'version': '0.163.0', + 'delivery-policy': 'local-materialization', + 'license': 'Apache-2.0', + 'npm-package': '@openai/codex', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.163.0.tgz', + 'sha256': '1a71db36560bb446d2c2e63ae675c4a3018c38ace38ca533f8d1b25d3e6555b4', + 'artifacts': {'linux-arm64': {'npm-package': '@openai/codex-linux-arm64', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.163.0-linux-arm64.tgz', + 'sha256': 'dc6e7c4d6de71c23a0f60d05e2577ff7eef615b76f22630d58ad384b84763522'}}}, + ('codex', '0.166.0'): {'version': '0.166.0', + 'delivery-policy': 'local-materialization', + 'license': 'Apache-2.0', + 'npm-package': '@openai/codex', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.166.0.tgz', + 'sha256': '67ee6f4069d9328956401f54ac202c20369233159058af9f4ec1af731b87f5b7', + 'artifacts': {'linux-arm64': {'npm-package': '@openai/codex-linux-arm64', + 'url': 'https://registry.npmjs.org/@openai/codex/-/codex-0.166.0-linux-arm64.tgz', + 'sha256': '9044e840397864fc6c539d13719b25fb26abb35d5353366cbabf7b4555406acd'}}}, + ('claude-code', '2.1.227'): {'version': '2.1.227', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'claude-code-download', + 'license': 'Proprietary', + 'terms-url': 'https://www.anthropic.com/legal/commercial-terms', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://downloads.claude.ai/claude-code-releases/2.1.227/linux-arm64/claude', + 'sha256': 'db47335532cbcab67a4b3ab16d8f3f77976bf85d53c7d79f8296538aa22bfce6'}}}, + ('claude-code', '2.1.236'): {'version': '2.1.236', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'claude-code-download', + 'license': 'Proprietary', + 'terms-url': 'https://www.anthropic.com/legal/commercial-terms', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://downloads.claude.ai/claude-code-releases/2.1.236/linux-arm64/claude', + 'sha256': 'c38d37deaf1643083326c48a6acc0afb09dada126e6bda77ef1a4410ae60ca12'}}}, + ('claude-code', '2.1.261'): {'version': '2.1.261', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'claude-code-download', + 'license': 'Proprietary', + 'terms-url': 'https://www.anthropic.com/legal/commercial-terms', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://downloads.claude.ai/claude-code-releases/2.1.261/linux-arm64/claude', + 'sha256': '7bbed5a9b0fc2e4ec67bad3490d06ca91b86d6b037d47520b7898951757d1b8a'}}}, + ('claude-code', '2.1.283'): {'version': '2.1.283', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'claude-code-download', + 'license': 'Proprietary', + 'terms-url': 'https://www.anthropic.com/legal/commercial-terms', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://downloads.claude.ai/claude-code-releases/2.1.283/linux-arm64/claude', + 'sha256': '346d294f0103d6fc0de11ac953579b5c62dfa90698a4cfc486b6f927c615e697'}}}, + ('claude-code', '2.1.302'): {'version': '2.1.302', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'claude-code-download', + 'license': 'Proprietary', + 'terms-url': 'https://www.anthropic.com/legal/commercial-terms', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://downloads.claude.ai/claude-code-releases/2.1.302/linux-arm64/claude', + 'sha256': '2bd4f4e5ed5cb09955b490098f745321cb5491e7ea8c752ba1e3bb010cdadb10'}}}, + ('claude-code', '2.1.314'): {'version': '2.1.314', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'claude-code-download', + 'license': 'Proprietary', + 'terms-url': 'https://www.anthropic.com/legal/commercial-terms', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://downloads.claude.ai/claude-code-releases/2.1.314/linux-arm64/claude', + 'sha256': 'a66c0a8f5d0c87a439953d797985192bbcbaefbce2e0d7b2eaa8df30ef0f3ef2'}}}, + ('antigravity-cli', '1.1.24'): {'version': '1.1.24', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'antigravity-download', + 'license': 'Proprietary', + 'terms-url': 'https://antigravity.google/terms/', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://storage.googleapis.com/antigravity-public/antigravity-cli/1.1.24-6130423206641664/linux-arm/cli_linux_arm64.tar.gz', + 'sha256': 'e601bde6dcb9caebf8a83de235a0e10a21192c7f9ca9ed459a120714e7d42399', + 'archive-member': 'antigravity', + 'upstream-sha512': '316ca00d50389a08b162c66066b4e2db201e4ffb85acea05029e3c4532c69d5b8f7c741cf027325889f898ea8f747af8cd15c802e15fcf5d73b7137b6e2420a1'}}}, + ('antigravity-cli', '1.2.12'): {'version': '1.2.12', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'antigravity-download', + 'license': 'Proprietary', + 'terms-url': 'https://antigravity.google/terms/', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://storage.googleapis.com/antigravity-public/antigravity-cli/1.2.12-5784551402897408/linux-arm/cli_linux_arm64.tar.gz', + 'sha256': 'bd338c9d19ab963d9d2bc027e4e797b470ea84bae02080e4fde4555357ea9444', + 'archive-member': 'antigravity', + 'upstream-sha512': 'e2f10960197efbf2455edb1284ae686821e244908367f507bcb6a0d0f616e945d3eec5cdf29b41bbcd7f630826d0e9e233f457184d1696fa91e81d6a7ed440c9'}}}, + ('antigravity-cli', '1.4.2'): {'version': '1.4.2', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'antigravity-download', + 'license': 'Proprietary', + 'terms-url': 'https://antigravity.google/terms/', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://storage.googleapis.com/antigravity-public/antigravity-cli/1.4.2-4835981233356800/linux-arm/cli_linux_arm64.tar.gz', + 'sha256': '08dd4a2ca52d531fef1dfae4ec2fbafcd56342bfa694c8193a60da2b26fb1898', + 'archive-member': 'antigravity', + 'upstream-sha512': 'f4655468d3303e6039cbcf1a30769e3cb5c4e16d149fba0d08e83f91f0f2b07cb623a4adb8615ca7a8fc73bf2445301e0e1ab82ea89f6aa44cbd9a9e96703e19'}}}, + ('antigravity-cli', '1.4.12'): {'version': '1.4.12', + 'delivery-policy': 'local-materialization', + 'acquisition-authorization': 'antigravity-download', + 'license': 'Proprietary', + 'terms-url': 'https://antigravity.google/terms/', + 'distribution': 'user-acquired-not-redistributed', + 'artifacts': {'linux-arm64': {'url': 'https://storage.googleapis.com/antigravity-public/antigravity-cli/1.4.12-4768735748292608/linux-arm/cli_linux_arm64.tar.gz', + 'sha256': '30dd61ebe1f0a8829057ea2de29dad08c710dd972ce4c02949f51e632135a86b', + 'archive-member': 'antigravity', + 'upstream-sha512': '7cae317fb6a02eb387982f06bfb234b366879cf9a80ad8969b571e5a55d1e37a6a5a49cd6b703c0443bb417937cf5e3b14a8c80cb26917ad79eef723a9f374c7'}}}, + ('playwright', '1.63.0'): LINUX_ARM64_PLAYWRIGHT_PIN, +} diff --git a/devcapsule-src/devcapsule/configuration/capability_commands.py b/devcapsule-src/devcapsule/configuration/capability_commands.py index bb2edcd4..9fbaaffe 100644 --- a/devcapsule-src/devcapsule/configuration/capability_commands.py +++ b/devcapsule-src/devcapsule/configuration/capability_commands.py @@ -13,7 +13,7 @@ import tomllib from typing import Any, Mapping, Sequence -from devcapsule.platforms import Platform +from devcapsule.platforms import Platform, UnsupportedPlatformError from devcapsule.resolution_matrix import MATRICES, ResolutionMatrix from .capabilities import CapabilityPolicy, LocalCapabilities, capability_notes, resolution_recommendations from .capability_selection import compose_lock, effective_lock, generate_lock, local_pins, usable_lock, matrix_for, pin_selection @@ -22,7 +22,10 @@ from .manifest_document import capability_reasons, capability_table, manifest_preamble, render_manifest from .manifest_schemas import ManifestV2 from .nodes import build_node_registry -from .storage import atomic_write, checkout_record_paths, discover_project, load_toml, require_settled +from .storage import ( + atomic_write, checkout_record_paths, discover_project, load_toml, locked_platforms, platform_lock_path, + project_platform, require_settled, +) def paths(root: Path) -> tuple[Path, Path, Path]: @@ -31,77 +34,144 @@ def paths(root: Path) -> tuple[Path, Path, Path]: ``root`` is the project directory. This only constructs paths. """ directory = root / ".devcapsule" - return (directory / "devcapsule.toml", directory / f"devcapsule.{Platform.current()}.lock", + return (directory / "devcapsule.toml", platform_lock_path(root, project_platform(root)), directory / ".capability-transaction.toml") def recover(root: Path) -> None: """Finish an interrupted shared edit, or do nothing if no journal exists. - The manifest chooses the before/after lock to restore. Refuse malformed - journals or independent edits with ProjectConfigurationError. Successful - recovery removes the journal. Use the same platform that began the edit; - callers must exclude concurrent writers. Filesystem errors propagate. + The manifest chooses the before or after side of every lock the journal + holds. Refuse malformed journals or independent edits with + ProjectConfigurationError. Successful recovery removes the journal. + Callers must exclude concurrent writers. Filesystem errors propagate. """ - manifest_path, lock_path, journal = paths(root) + manifest_path, _, journal = paths(root) if not journal.exists(): return transaction = load_toml(journal) - if set(transaction) != {"before-manifest", "after-manifest", "before-lock", "after-lock"} or any( - not isinstance(value, str) for value in transaction.values() - ): - raise ProjectConfigurationError("Malformed capability transaction; preserve it for recovery.") + locks = _journaled_locks(root, transaction) current = manifest_path.read_bytes().decode("utf-8") if manifest_path.exists() else "" - current_lock = lock_path.read_bytes().decode("utf-8") if lock_path.exists() else "" - if current_lock not in {transaction["before-lock"], transaction["after-lock"]}: - raise ProjectConfigurationError("Lock changed outside the pending transaction; refusing to overwrite it.") + for lock_path, (before, after) in locks.items(): + current_lock = lock_path.read_bytes().decode("utf-8") if lock_path.exists() else "" + if current_lock not in {before, after}: + raise ProjectConfigurationError("Lock changed outside the pending transaction; refusing to overwrite it.") if current == transaction["after-manifest"]: - chosen = "after-lock" + committed = True elif current == transaction["before-manifest"]: - chosen = "before-lock" + committed = False else: raise ProjectConfigurationError("Configuration changed outside the pending transaction; refusing to overwrite it.") - # Manifest-only edits preserve even an absent or empty lock. - if transaction["before-lock"] != transaction["after-lock"]: - if transaction[chosen]: - atomic_write(lock_path, transaction[chosen], mode=0o644) - else: - lock_path.unlink(missing_ok=True) + for lock_path, (before, after) in locks.items(): + restored = after if committed else before + # Manifest-only edits preserve even an absent or empty lock. + if before != after: + if restored: + atomic_write(lock_path, restored, mode=0o644) + else: + lock_path.unlink(missing_ok=True) journal.unlink() -def promote(root: Path, manifest: dict[str, Any], lock: dict[str, Any] | None = None) -> None: - """Persist an already-validated manifest, and lock when given, with a recovery journal. +def _journaled_locks(root: Path, transaction: Mapping[str, Any]) -> dict[Path, tuple[str, str]]: + """Each lock path the journal holds, with its before and after text. + + A journal holds every platform lock under ``before-locks`` and + ``after-locks``; one written by a launcher of a single platform holds + that platform's lock as ``before-lock`` and ``after-lock``. + """ + malformed = ProjectConfigurationError("Malformed capability transaction; preserve it for recovery.") + manifests = (transaction.get("before-manifest"), transaction.get("after-manifest")) + if not all(isinstance(value, str) for value in manifests): + raise malformed + if set(transaction) == {"before-manifest", "after-manifest", "before-lock", "after-lock"}: + sides = {str(project_platform(root)): (transaction["before-lock"], transaction["after-lock"])} + elif set(transaction) == {"before-manifest", "after-manifest", "before-locks", "after-locks"}: + before, after = transaction["before-locks"], transaction["after-locks"] + if not isinstance(before, dict) or not isinstance(after, dict) or set(before) != set(after): + raise malformed + sides = {key: (before[key], after[key]) for key in before} + else: + raise malformed + locks: dict[Path, tuple[str, str]] = {} + for key, (before_text, after_text) in sides.items(): + if not isinstance(before_text, str) or not isinstance(after_text, str): + raise malformed + try: + platform = Platform.parse(key) + except UnsupportedPlatformError: + raise malformed from None + locks[platform_lock_path(root, platform)] = (before_text, after_text) + return locks + + +def promote(root: Path, manifest: dict[str, Any], locks: Mapping[Platform, dict[str, Any]] | None = None) -> None: + """Persist an already-validated manifest, and the locks when given, with a recovery journal. Call ``check_documents`` first and exclude concurrent writers. Write the - lock before the manifest; attempt recovery on OSError. Failures propagate. - A pending journal raises ProjectConfigurationError. This updates shared - files only; it neither writes nor resolves a developer's checkout. + locks before the manifest; attempt recovery on OSError. Failures + propagate. A pending journal raises ProjectConfigurationError. This + updates shared files only; it neither writes nor resolves a developer's + checkout. The manifest is written by the one manifest writer: bare keys, the owner's table order, the leading comment block of the existing file - kept, and the supplied schema version. A ``lock`` of ``None`` keeps the + kept, and the supplied schema version. ``locks`` of ``None`` keeps every lock file's bytes exactly, for an edit that changes the manifest alone. """ - manifest_path, lock_path, journal = paths(root) + manifest_path, _, journal = paths(root) ManifestV2.read(manifest, manifest_path) # The one writer writes schema 2 and nothing else. require_settled(root) before_manifest = manifest_path.read_bytes().decode("utf-8") if manifest_path.exists() else "" - before_lock = lock_path.read_bytes().decode("utf-8") if lock_path.exists() else "" - transaction = {"before-manifest": before_manifest, "before-lock": before_lock, - "after-manifest": render_manifest(manifest, preamble=manifest_preamble(before_manifest)), - "after-lock": before_lock if lock is None else render_toml(lock)} - atomic_write(journal, render_toml(transaction)) + after_manifest = render_manifest(manifest, preamble=manifest_preamble(before_manifest)) + before_locks: dict[str, str] = {} + after_locks: dict[str, str] = {} + for platform, lock in (locks or {}).items(): + lock_path = platform_lock_path(root, platform) + before_locks[str(platform)] = lock_path.read_bytes().decode("utf-8") if lock_path.exists() else "" + after_locks[str(platform)] = render_toml(lock) + atomic_write(journal, render_toml({"before-manifest": before_manifest, "after-manifest": after_manifest, + "before-locks": before_locks, "after-locks": after_locks})) try: - if lock is not None: - atomic_write(lock_path, transaction["after-lock"], mode=0o644) - atomic_write(manifest_path, transaction["after-manifest"], mode=0o644) + for platform, lock in (locks or {}).items(): + atomic_write(platform_lock_path(root, platform), after_locks[str(platform)], mode=0o644) + atomic_write(manifest_path, after_manifest, mode=0o644) except OSError: recover(root) raise journal.unlink() +def project_locks(root: Path) -> dict[Platform, dict[str, Any]]: + """Every platform lock the project carries, by platform. + + An edit of the shared contract regenerates them all, so a lock committed + for another platform never falls behind the manifest (R-COMPAT-001). A + project with none fails as reading this host's lock does. + """ + platforms = locked_platforms(root) or (project_platform(root),) + return {platform: load_toml(platform_lock_path(root, platform)) for platform in platforms} + + +def regenerate_locks(policy: CapabilityPolicy, locks: Mapping[Platform, dict[str, Any]], *, + recommendations: Mapping[str, str], allow_unverified: bool = False) -> dict[Platform, dict[str, Any]]: + """Each of ``locks`` regenerated for ``policy`` from its own platform's matrix. + + A lock that cannot follow refuses the whole edit; with several, the + error names the platform. + """ + generated: dict[Platform, dict[str, Any]] = {} + for platform, lock in locks.items(): + try: + generated[platform] = generate_lock(policy, MATRICES[platform], previous=lock, + allow_unverified=allow_unverified, recommendations=recommendations) + except ProjectConfigurationError as exc: + if len(locks) == 1: + raise + raise ProjectConfigurationError(f"The project's {platform} lock cannot follow this edit: {exc}") from exc + return generated + + def check_documents(manifest: dict[str, Any], lock: dict[str, Any], *, check_resolutions: bool = True) -> tuple[str, ...]: """Validate shared documents offline and return optional-omission warnings. @@ -192,20 +262,23 @@ def configure(start: Path, *, required: Sequence[str] | None = None, local_edit = local is not None or without is not None or providers is not None if project_edit == local_edit: raise ProjectConfigurationError("Choose either project flags (--required/--optional) or local flags (--local/--without).") - matrix = MATRICES[Platform.current()] + matrix = MATRICES[project_platform(root)] if project_edit: - return configure_project(root, manifest, lock, matrix, required=required, optional=optional, + locks = project_locks(root) + for other in locks.values(): + check_documents(manifest, other) + return configure_project(root, manifest, locks, required=required, optional=optional, preview=preview, allow_unverified=allow_unverified) return configure_local(root, manifest, lock, matrix, local=local, without=without, providers=providers, preview=preview, allow_unverified=allow_unverified) -def configure_project(root: Path, manifest: dict[str, Any], lock: dict[str, Any], matrix: ResolutionMatrix, *, +def configure_project(root: Path, manifest: dict[str, Any], locks: Mapping[Platform, dict[str, Any]], *, required: Sequence[str] | None, optional: Sequence[str] | None, preview: bool, allow_unverified: bool) -> str: - """Replace the shared policy and regenerate the platform lock; see ``configure``. + """Replace the shared policy and regenerate every platform lock; see ``configure``. - ``manifest`` and ``lock`` are the validated current shared documents and + ``manifest`` and ``locks`` are the validated current shared documents and are not changed; ``manifest`` is used as the candidate's template. """ policy = CapabilityPolicy.read(manifest) @@ -217,9 +290,10 @@ def configure_project(root: Path, manifest: dict[str, Any], lock: dict[str, Any] candidate_manifest["capabilities"] = declarations candidate = CapabilityPolicy.read(candidate_manifest) candidate_manifest["capabilities"] = capability_table(candidate, capability_reasons(manifest), previous=manifest["capabilities"]) - generated = generate_lock(candidate, matrix, previous=lock, allow_unverified=allow_unverified, - recommendations=resolution_recommendations(candidate_manifest)) - check_documents(candidate_manifest, generated) + generated = regenerate_locks(candidate, locks, allow_unverified=allow_unverified, + recommendations=resolution_recommendations(candidate_manifest)) + for lock in generated.values(): + check_documents(candidate_manifest, lock) if not preview: promote(root, candidate_manifest, generated) return ("Preview; no files changed.\n" if preview else "Updated shared capability policy and lock.\n") + render_toml(candidate_manifest["capabilities"]) diff --git a/devcapsule-src/devcapsule/configuration/capability_selection.py b/devcapsule-src/devcapsule/configuration/capability_selection.py index 7db8d5d1..227e98f4 100644 --- a/devcapsule-src/devcapsule/configuration/capability_selection.py +++ b/devcapsule-src/devcapsule/configuration/capability_selection.py @@ -15,23 +15,31 @@ from typing import Any, Mapping from devcapsule.components.catalog import COMPONENTS, INTERACTIVE_SURFACES -from devcapsule.platforms import Platform +from devcapsule.platforms import Platform, UnsupportedPlatformError from devcapsule.resolution_matrix import MATRICES, ResolutionMatrix from .capabilities import CapabilityPolicy, LocalCapabilities, names, meets_minimum, resolution_recommendations from .file_formats import ProjectConfigurationError, legacy_lock, table -def matrix_for(lock: Mapping[str, Any]) -> ResolutionMatrix: - """Return this launcher's catalog for the lock's declared platform. +def lock_platform(lock: Mapping[str, Any]) -> Platform: + """Return the lock's declared platform. Raise ProjectConfigurationError if that platform is absent or unsupported. """ try: - return MATRICES[Platform(str(lock.get("platform")))] - except (ValueError, KeyError): + return Platform.parse(str(lock.get("platform"))) + except UnsupportedPlatformError: raise ProjectConfigurationError(f"Unsupported lock platform {lock.get('platform')!r}.") from None +def matrix_for(lock: Mapping[str, Any]) -> ResolutionMatrix: + """Return this launcher's catalog for the lock's declared platform. + + Raise ProjectConfigurationError if that platform is absent or unsupported. + """ + return MATRICES[lock_platform(lock)] + + def known_policy_names(policy: CapabilityPolicy, matrix: ResolutionMatrix) -> dict[str, str]: """Canonical names for the shared policy; unknown offers stay opaque.""" result = {name: matrix.canonical_name(name) for name in policy.required} @@ -111,6 +119,56 @@ def local_pins(checkout: Mapping[str, Any]) -> dict[str, Any]: raise ProjectConfigurationError(f"Invalid local capability pins: {exc}") from exc +def pins_for_platform(pins: Mapping[str, Any], matrix: ResolutionMatrix) -> dict[str, Any]: + """Return personal pins with every component's artifacts for ``matrix``'s platform. + + A checkout keeps one set of personal pins, and a project may run as two + platforms: Apple Silicon runs linux-arm64 and linux-amd64. A component + table that is exactly another platform's embedded table for its version + is that platform's pin, whatever the pins' label says, and takes this + platform's table for the version; one that matches no embedded table is a + record this launcher did not write, kept as it stands on the platform its + pins name. Everything else is returned unchanged. A component written for + another platform at a version this platform does not pin raises + ProjectConfigurationError naming how to choose again. The input is unchanged. + """ + if not pins: + return {} + platform = str(matrix.platform) + labelled = str(pins.get("platform")) + repinned = deepcopy(dict(pins)) + components = repinned.get("components") + surface = components.get("interactive-surface") if isinstance(components, dict) else None + if isinstance(components, dict): + for component, metadata in components.items(): + if not isinstance(metadata, dict): + continue + written_for = _embedded_platform_of(component, metadata) or labelled + if written_for == platform: + continue + matrix_table = matrix.component_pin(component, metadata.get("version")) + if matrix_table is None: + remedy = (f"'devcapsule project config ide {component}'" if component == surface + else f"'devcapsule project config use CAPABILITY {component}'") + raise ProjectConfigurationError( + f"This checkout's {component} {metadata.get('version')} is pinned for {written_for}, " + f"and {platform} has no pin for that version; choose it again with {remedy}." + ) + components[component] = deepcopy(dict(matrix_table)) + if "platform" in repinned: + repinned["platform"] = platform + return repinned + + +def _embedded_platform_of(component: str, table: Mapping[str, Any]) -> str | None: + """The platform whose embedded table for this component version ``table`` is, if any.""" + for platform, embedded in MATRICES.items(): + candidate = embedded.component_pin(component, table.get("version")) + if candidate is not None and dict(candidate) == dict(table): + return str(platform) + return None + + def pin_selection(matrix: ResolutionMatrix, policy: CapabilityPolicy, selection: LocalCapabilities, source: Mapping[str, Any], previous: LocalCapabilities, previous_pins: Mapping[str, Any], *, allow_unverified: bool, recommendations: Mapping[str, str] | None = None) -> dict[str, Any]: @@ -123,6 +181,7 @@ def pin_selection(matrix: ResolutionMatrix, policy: CapabilityPolicy, selection: selector and materialization follow the IDE component's document. Catalog resolution failures raise ProjectConfigurationError. Inputs are unchanged. """ + previous_pins = pins_for_platform(previous_pins, matrix) active_names = set(known_policy_names(policy, matrix).values()) | {matrix.canonical_name(name) for name in selection.selected} reset = (set(previous.providers) - set(selection.providers)) & active_names capabilities = tuple(dict.fromkeys((*selection.selected, *selection.providers, *sorted(reset)))) @@ -191,6 +250,7 @@ def compose_lock(manifest: Mapping[str, Any], shared: Mapping[str, Any], if not pins: raise ProjectConfigurationError("Local capability pins are missing; choose it again with 'project config ide IDE'.") matrix = matrix_for(lock) + pins = pins_for_platform(pins, matrix) if policy is None: policy = CapabilityPolicy.read(manifest) canonical = {**known_policy_names(policy, matrix), **{name: matrix.canonical_name(name) for name in local.selected}} diff --git a/devcapsule-src/devcapsule/configuration/derivation.py b/devcapsule-src/devcapsule/configuration/derivation.py index 4b49112d..89afb695 100644 --- a/devcapsule-src/devcapsule/configuration/derivation.py +++ b/devcapsule-src/devcapsule/configuration/derivation.py @@ -26,7 +26,6 @@ from devcapsule.compat import DecisionPending from devcapsule.materialization import validate_base_image -from devcapsule.platforms import Platform from devcapsule.project import sanitize_name from devcapsule.resolution_matrix import MATRICES, ResolutionMatrix, known_base_image @@ -41,6 +40,7 @@ from .manifest_document import Recommendations from .trust import HomeTrust from .storage import ( + project_platform, ResolvedProject, atomic_write, checkout_record_paths, @@ -108,7 +108,8 @@ class CheckoutState: def __init__(self, root: Path, manifest: dict[str, Any]) -> None: self.root = root self.manifest = manifest - self.matrix = MATRICES[Platform.current()] + self.platform = project_platform(root) + self.matrix = MATRICES[self.platform] self.surfaces = {name: self.matrix.resolve_provider(name) or name for name in self.matrix.ide_capabilities()} self.lock_path, self.shared_lock = recommendation_lock_for(root, manifest) self.home = load_settings() @@ -158,7 +159,7 @@ def composed_lock(self, checkout: Mapping[str, Any] | None = None) -> dict[str, """The shared lock, or the checkout's version set, with the checkout's pins laid over it.""" document = self.checkout if checkout is None else checkout selected = selected_version_lock(document) - if selected is not None and selected.get("platform") != str(Platform.current()): + if selected is not None and selected.get("platform") != str(self.platform): raise ProjectConfigurationError("Local version set is for another platform; follow the project explicitly.") return compose_lock(self.manifest, selected if selected is not None else self.shared_lock, document, policy=self.policy) diff --git a/devcapsule-src/devcapsule/configuration/operations.py b/devcapsule-src/devcapsule/configuration/operations.py index 0753e7d4..23d2c149 100644 --- a/devcapsule-src/devcapsule/configuration/operations.py +++ b/devcapsule-src/devcapsule/configuration/operations.py @@ -42,7 +42,7 @@ ) from .capabilities import CapabilityPolicy, LocalCapabilities, requirement from .capability_commands import check_documents, pin_selection -from .capability_selection import compose_lock, effective_lock, generate_lock, local_pins +from .capability_selection import compose_lock, effective_lock, generate_lock, local_pins, matrix_for from .file_formats import ( AuthorizationScalar, ConfigurationFileKind, @@ -66,6 +66,8 @@ build_node_registry, ) from .storage import ( + platform_lock_path, + project_platform, atomic_write, checkout_record_paths, load_checkout, @@ -80,13 +82,10 @@ from .values import checkout_omitted_values, normalize_configuration_value -def _current_matrix() -> ResolutionMatrix: - """This host's resolution matrix, with platform failures as user errors.""" +def _project_matrix(root: Path) -> ResolutionMatrix: + """The resolution matrix of the platform this host runs the project at ``root`` as.""" - try: - return MATRICES[Platform.current()] - except UnsupportedPlatformError as exc: - raise ProjectConfigurationError(str(exc)) from exc + return MATRICES[project_platform(root)] __all__ = [ "CheckoutRecord", @@ -279,6 +278,9 @@ class InitializeRequest: allow_unverified: bool = False # None: decide from whether stdin is a terminal. interactive: bool | None = None + # The platform whose lock to write; None: the one the project runs as + # here, or, for a new project on a host of several platforms, asked. + platform: Platform | None = None @dataclass(frozen=True) @@ -313,6 +315,8 @@ class InitializeReport: checkout_record: Path # None when the owner's checkout has no IDE yet: nothing to resolve. resolve: ResolveReport | None + # The lock pins a base its registry does not hold yet (see ResolutionMatrix.base_unpublished). + base_unpublished: bool def render(self) -> str: required = " ".join(self.required) or "none" @@ -326,6 +330,9 @@ def render(self) -> str: if self.offered: lines.append(f"Offers: {' '.join(self.offered)}") lines.append(f"{self.lock_action} {self.lock_path} (base {self.base_display})") + if self.base_unpublished: + lines.append("Its base is not published yet: a checkout runs this lock only on a locally built base of " + "the same recipe, selected with 'devcapsule project config authorize base-image IMAGE'.") for name, value, justification in self.recommendations: line = f"Recommended {name} = {value} for every checkout" lines.append(f"{line} ({justification})." if justification else f"{line}.") @@ -402,12 +409,16 @@ def initialize_project( raise ProjectConfigurationError(f"Project directory does not exist: {root}") require_settled(root) manifest_path = root / ".devcapsule" / "devcapsule.toml" - try: - platform = Platform.current() - except UnsupportedPlatformError as exc: - raise ProjectConfigurationError(str(exc)) from exc - lock_path = root / ".devcapsule" / f"devcapsule.{platform}.lock" - matrix = _current_matrix() + elicitor = Elicitor( + _init_command_line(request), + interactive=request.interactive, + input_stream=input_stream, + output_stream=output_stream, + ) + # The platform comes first: its matrix answers every later question. + platform = _elicit_platform(elicitor, root) + lock_path = platform_lock_path(root, platform) + matrix = MATRICES[platform] existing_manifest: dict[str, Any] | None = None existing_lock: dict[str, Any] | None = None @@ -450,12 +461,6 @@ def initialize_project( "the 'config' family changes individual answers." ) - elicitor = Elicitor( - _init_command_line(request), - interactive=request.interactive, - input_stream=input_stream, - output_stream=output_stream, - ) identity = _elicit_identity(elicitor, root, existing_manifest, matrix) authored = frozenset(answer.name for answer in request.answers if answer.justification is not None) # Fail now if identity or requirements are unanswered: the lock cannot be @@ -546,6 +551,7 @@ def persist() -> None: base_display=_base_display(shared_lock), recommendations=recommendations, selection=selection, answered=(), carried=(), recommended=(), checkout_record=record.input_path, resolve=None, + base_unpublished=_pins_unpublished_base(shared_lock), ) lock = effective_lock(manifest, compose_lock(manifest, shared_lock, record.document), record.document, warn=False) declarations = authorization_declarations(manifest, lock) @@ -573,14 +579,75 @@ def persist() -> None: answered=tuple(settled.answered), carried=tuple(settled.carried), recommended=accepted_recommendations, checkout_record=record.input_path, resolve=resolve_report, + base_unpublished=_pins_unpublished_base(shared_lock), ) +def _elicit_platform(elicitor: Elicitor, root: Path) -> Platform: + """The platform whose lock init writes, asked when this host runs several. + + A project with a lock this host runs keeps that platform unless + ``--platform`` names another; otherwise a host of one platform takes it, + and a host of several asks, its native platform the default and the + unattended answer. Init completes the checkout for the lock it writes, so + the platform must be the one the project runs as once that lock exists. + """ + runnable = Platform.runnable() + native = runnable[0] + locked = next((platform for platform in runnable if platform_lock_path(root, platform).is_file()), None) + choices = " or ".join( + f"{platform} ({'native' if platform is native else 'emulated'}" + + ("; its base is built locally" if _base_unpublished(MATRICES[platform]) else "") + ")" + for platform in runnable + ) + answer = elicitor.seek( + "platform", + description=f"Capsule platform for this project: {choices}", + remedy="--platform PLATFORM", + existing=str(locked) if locked is not None else None, + default=str(native) if len(runnable) == 1 else None, + empty_answer=str(native), + omitted_answer=str(native), + validate=_platform_validator(runnable), + ) + if answer is None: # Input ended at the question. + elicitor.finish(require_all_consumed=False) + raise AssertionError("an unanswered platform question must fail the elicitation") + platform = Platform.parse(answer.value) + runs_as = next(candidate for candidate in runnable + if candidate is platform or platform_lock_path(root, candidate).is_file()) + if runs_as is not platform: + raise ProjectConfigurationError( + f"On this host the project runs as {runs_as}, which it has a lock for, " + f"so init cannot add a {platform} lock here." + ) + return platform + + +def _platform_validator(runnable: tuple[Platform, ...]) -> Any: + def validate(value: str) -> str: + try: + candidate = Platform.parse(value.strip()) + except UnsupportedPlatformError as exc: + raise ProjectConfigurationError(str(exc)) from exc + if candidate not in runnable: + names = ", ".join(str(platform) for platform in runnable) + raise ProjectConfigurationError(f"This host runs {names} capsules, not {candidate}.") + return str(candidate) + + return validate + + +def _base_unpublished(matrix: ResolutionMatrix) -> bool: + """Whether every base ``matrix`` pins is still unpublished.""" + return all(matrix.base_unpublished(base.reference) for base in matrix.base_images()) + + def _initialize_report( manifest_path: Path, manifest_action: str, identity: _ProjectIdentity, lock_path: Path, lock_action: str, *, base_display: str, recommendations: list[tuple[str, str]], selection: _OwnerSelection, answered: tuple[str, ...], carried: tuple[str, ...], recommended: tuple[str, ...], - checkout_record: Path, resolve: ResolveReport | None, + checkout_record: Path, resolve: ResolveReport | None, base_unpublished: bool, ) -> InitializeReport: return InitializeReport( manifest_path=manifest_path, @@ -605,9 +672,15 @@ def _initialize_report( recommended=recommended, checkout_record=checkout_record, resolve=resolve, + base_unpublished=base_unpublished, ) +def _pins_unpublished_base(lock: Mapping[str, Any]) -> bool: + """Whether ``lock`` pins a base its registry does not hold yet.""" + return matrix_for(lock).base_unpublished(str(lock.get("base", {}).get("reference"))) + + def _base_display(lock: Mapping[str, Any]) -> str: base = lock.get("base") if isinstance(base, dict) and isinstance(base.get("reference"), str): @@ -653,7 +726,7 @@ def _existing_selection(root: Path, manifest: Mapping[str, Any] | None) -> _Owne if not input_path.is_file(): return _OwnerSelection(None, ()) local = LocalCapabilities.read(load_toml(input_path)) - matrix = _current_matrix() + matrix = _project_matrix(root) surfaces = set(matrix.ide_capabilities()) ide = next((name for name in local.selected if name in surfaces), None) return _OwnerSelection(ide, tuple(name for name in local.selected if name not in surfaces)) @@ -776,6 +849,8 @@ def _init_command_line(request: InitializeRequest) -> dict[AnswerKey, str]: command_line[("offer", "value")] = " ".join(request.offer) if request.ide is not None: command_line[("ide", "value")] = request.ide + if request.platform is not None: + command_line[("platform", "value")] = str(request.platform) for key, value in ( ("name", request.project_name), ("slug", request.slug), @@ -1581,6 +1656,7 @@ def _apply_answers_to_standing_checkout( base_display=base_display, recommendations=[], selection=_existing_selection(root, manifest), answered=tuple(settled.answered), carried=tuple(settled.carried), recommended=(), checkout_record=record.input_path, resolve=resolve_report, + base_unpublished=_pins_unpublished_base(lock), ) diff --git a/devcapsule-src/devcapsule/configuration/owner_commands.py b/devcapsule-src/devcapsule/configuration/owner_commands.py index 91f0ce9b..5972b998 100644 --- a/devcapsule-src/devcapsule/configuration/owner_commands.py +++ b/devcapsule-src/devcapsule/configuration/owner_commands.py @@ -2,9 +2,9 @@ Each operation edits the shared manifest through one writer (:func:`manifest_document.render_manifest`) and, when the tools change, -regenerates the platform lock, validating the candidate pair before anything -is written and promoting both files under the capability transaction journal -of :mod:`capability_commands`. The rc4 IDE recommendation command stays; agents are ordinary shared +regenerates every platform lock the project carries, validating each +candidate pair before anything is written and promoting the files under the +capability transaction journal of :mod:`capability_commands`. The rc4 IDE recommendation command stays; agents are ordinary shared capabilities. Component acquisition still requires the developer's consent. The verbs are the ones the CLI UX redesign work order of 2026-10-24 settled: @@ -24,8 +24,7 @@ from .authorization import CURATED_HOST_RECOMMENDATIONS, render_authorization_value from .capabilities import CapabilityPolicy, requirement, resolution_recommendations -from .capability_commands import canonical_tool_name, check_documents, paths, promote -from .capability_selection import generate_lock +from .capability_commands import canonical_tool_name, check_documents, paths, project_locks, promote, regenerate_locks from .file_formats import ProjectConfigurationError from .manifest import current_manifest from .manifest_document import ( @@ -37,7 +36,7 @@ environment_name, ) from .sources import WrappedSource, WrappedSources, has_control_characters, java_line, source_path -from .storage import discover_project, load_toml, require_settled +from .storage import discover_project, load_toml, project_platform, require_settled from .values import normalize_memory_size __all__ = [ @@ -53,20 +52,24 @@ ] -def _project(start: Path, *, check_resolutions: bool = True) -> tuple[Path, dict[str, Any], dict[str, Any], ResolutionMatrix]: +def _project(start: Path, *, check_resolutions: bool = True) -> tuple[ + Path, dict[str, Any], dict[Platform, dict[str, Any]], ResolutionMatrix, +]: + """The project's root, manifest and every lock, validated, and this host's matrix for its vocabulary.""" root = discover_project(start) require_settled(root) - manifest_path, lock_path, _ = paths(root) - manifest, lock = current_manifest(load_toml(manifest_path), manifest_path).document(), load_toml(lock_path) - check_documents(manifest, lock, check_resolutions=check_resolutions) - return root, manifest, lock, MATRICES[Platform.current()] + manifest_path = paths(root)[0] + manifest, locks = current_manifest(load_toml(manifest_path), manifest_path).document(), project_locks(root) + for lock in locks.values(): + check_documents(manifest, lock, check_resolutions=check_resolutions) + return root, manifest, locks, MATRICES[project_platform(root)] def _policy_edit( start: Path, *, change: str, names: Sequence[str], because: str | None, preview: bool, allow_unverified: bool, native: bool = False, ) -> str: - root, manifest, lock, matrix = _project(start) + root, manifest, locks, matrix = _project(start) policy = CapabilityPolicy.read(manifest) reasons = capability_reasons(manifest) required, optional = set(policy.required), set(policy.optional) @@ -127,14 +130,16 @@ def ordered(values: set[str], previous: list[str]) -> tuple[str, ...]: candidate = CapabilityPolicy(ordered(required, required_order), ordered(optional, optional_order), tuple(sorted(minimums.items()))) candidate_manifest = deepcopy(manifest) candidate_manifest["capabilities"] = capability_table(candidate, reasons, previous=manifest["capabilities"]) - generated = generate_lock(candidate, matrix, previous=lock, allow_unverified=allow_unverified, - recommendations=resolution_recommendations(candidate_manifest)) - check_documents(candidate_manifest, generated) + generated = regenerate_locks(candidate, locks, allow_unverified=allow_unverified, + recommendations=resolution_recommendations(candidate_manifest)) + for lock in generated.values(): + check_documents(candidate_manifest, lock) if not preview: promote(root, candidate_manifest, generated) verb = {"require": "Required", "offer": "Offered", "drop": "Dropped"}[change] lines = [ - ("Preview; nothing written." if preview else f"{verb} {', '.join(canonical)}; the manifest and the lock are updated."), + ("Preview; nothing written." if preview else + f"{verb} {', '.join(canonical)}; the manifest and {'the lock' if len(generated) == 1 else 'its locks'} are updated."), f"Required: {' '.join(candidate.requirements()) or 'none'}", f"Offered: {' '.join(candidate.optional) or 'none'}", ] @@ -170,7 +175,7 @@ def _refresh_own_checkout(root: Path, manifest: Mapping[str, Any]) -> list[str]: def require_tools(start: Path, names: Sequence[str], *, because: str | None = None, preview: bool = False, allow_unverified: bool = False) -> str: - """Make ``names`` required tools of the project, with an optional reason; regenerate the lock.""" + """Make ``names`` required tools of the project, with an optional reason; regenerate the locks.""" return _policy_edit(start, change="require", names=names, because=because, preview=preview, allow_unverified=allow_unverified) @@ -186,7 +191,7 @@ def drop_tools(start: Path, names: Sequence[str], *, preview: bool = False, allo def _environment_edit(start: Path, *, change: str, names: Sequence[str], because: str | None, preview: bool) -> str: - root, manifest, _lock, _matrix = _project(start) + root, manifest, _locks, _matrix = _project(start) environment = deepcopy(manifest.get("environment", {})) required, optional = environment.get("required", {}), environment.get("optional", {}) canonical = tuple(environment_name(name, field_name="environment variable") for name in names) @@ -251,7 +256,7 @@ def recommend(start: Path, name: str, value: str, *, because: str | None = None, recommendation. A host recommendation requires ``because``: the reason is what the developer reads when asked. """ - root, manifest, lock, matrix = _project(start, check_resolutions=False) + root, manifest, locks, matrix = _project(start, check_resolutions=False) candidate_manifest = deepcopy(manifest) removing = value.strip().lower() == "none" generated = None @@ -330,11 +335,11 @@ def recommend(start: Path, name: str, value: str, *, because: str | None = None, capabilities.pop("resolution", None) if resolutions: capabilities["resolution"] = resolutions - generated = generate_lock(CapabilityPolicy.read(candidate_manifest), matrix, previous=lock, - recommendations=resolutions) + generated = regenerate_locks(CapabilityPolicy.read(candidate_manifest), locks, recommendations=resolutions) effect = "the provider default; acquisition still needs the developer's trust or consent" _validate(candidate_manifest) - check_documents(candidate_manifest, generated if generated is not None else lock) + for lock in (generated if generated is not None else locks).values(): + check_documents(candidate_manifest, lock) if not preview: promote(root, candidate_manifest, generated) head = "Preview; nothing written." if preview else ( @@ -408,7 +413,7 @@ def declare_source(start: Path, path: str, *, upstream: str | None = None, java: not know. The lock is untouched: a source adds no capability by itself; ``require_tools`` does that, and ``project wrap`` calls both. """ - root, manifest, _lock, _matrix = _project(start, check_resolutions=False) + root, manifest, locks, _matrix = _project(start, check_resolutions=False) path = source_path(path, field_name="source path") if upstream is not None and (not isinstance(upstream, str) or not upstream.strip() or has_control_characters(upstream)): raise ProjectConfigurationError("The upstream is the upstream repository's URL, on one line; it cannot be blank.") @@ -422,7 +427,8 @@ def declare_source(start: Path, path: str, *, upstream: str | None = None, java: candidate_manifest = deepcopy(manifest) candidate_manifest["sources"] = candidate.document(previous=previous) WrappedSources.read(candidate_manifest) - check_documents(candidate_manifest, _lock, check_resolutions=False) + for lock in locks.values(): + check_documents(candidate_manifest, lock, check_resolutions=False) if not preview: promote(root, candidate_manifest) verb = "Redeclared" if declared.get(path) is not None else "Declared" diff --git a/devcapsule-src/devcapsule/configuration/storage.py b/devcapsule-src/devcapsule/configuration/storage.py index 3fbae07f..b9dedfe9 100644 --- a/devcapsule-src/devcapsule/configuration/storage.py +++ b/devcapsule-src/devcapsule/configuration/storage.py @@ -382,8 +382,35 @@ def manifest_source_digest(root: Path) -> str | None: return None +def platform_lock_path(root: Path, platform: Platform) -> Path: + """The path of the project's lock for ``platform``.""" + return root / ".devcapsule" / f"devcapsule.{platform}.lock" + + +def project_platform(root: Path) -> Platform: + """The capsule platform this host runs the project at ``root`` as. + + The first platform this host runs that the project has a lock for, so + Apple Silicon runs a project locked for linux-arm64 natively and one + locked only for linux-amd64 under emulation; for a project with no such + lock, the platform this host creates projects for. Reads only. + """ + try: + for platform in Platform.runnable(): + if platform_lock_path(root, platform).is_file(): + return platform + return Platform.current() + except UnsupportedPlatformError as exc: + raise ProjectConfigurationError(str(exc)) from exc + + +def locked_platforms(root: Path) -> tuple[Platform, ...]: + """Every platform the project at ``root`` has a lock for, on any host.""" + return tuple(platform for platform in Platform if platform_lock_path(root, platform).is_file()) + + def recommendation_lock_for(root: Path, manifest: Mapping[str, Any]) -> tuple[Path, dict[str, Any]]: - """Load the committed platform lock for this host's platform. + """Load the committed lock of the platform this host runs the project as. The lock is the project-side record of one resolution: the version set and property defaults selected for one platform. It is a record, not a @@ -404,15 +431,13 @@ def recommendation_lock_for(root: Path, manifest: Mapping[str, Any]) -> tuple[Pa to keep existing committed locks working. """ - try: - current = Platform.current() - except UnsupportedPlatformError as exc: - raise ProjectConfigurationError(str(exc)) from exc - path = root / ".devcapsule" / f"devcapsule.{current}.lock" + platform = project_platform(root) + path = platform_lock_path(root, platform) if not path.is_file(): raise ProjectConfigurationError( - f"Missing {path}: this project carries no platform lock for {current}. " - "The platform lock is authored on the project side and committed with the project." + f"Missing {path}: this project carries no platform lock for {platform}. " + "The platform lock is authored on the project side and committed with the project; " + "'devcapsule project init' adds it." ) value = load_toml(path) validate_file_format(value, ConfigurationFileKind.lock, path) @@ -429,7 +454,7 @@ def recommendation_lock_for(root: Path, manifest: Mapping[str, Any]) -> tuple[Pa def composition_for(root: Path, manifest: Mapping[str, Any]) -> tuple[Path, dict[str, Any], dict[str, Any]]: """The checkout's persistent software selection, before execution filtering. - Return the current platform's shared lock path, the composed selection (the + Return the project platform's shared lock path, the composed selection (the explicit version set, or the shared lock with personal pins laid over it) and the loaded checkout record, empty when the checkout is unregistered. A version set for another platform raises ProjectConfigurationError. @@ -439,9 +464,10 @@ def composition_for(root: Path, manifest: Mapping[str, Any]) -> tuple[Path, dict checkout = load_checkout(record, manifest, root) if record is not None else {} selected = selected_version_lock(checkout) if selected is not None: - if selected.get("platform") != str(Platform.current()): + platform = project_platform(root) + if selected.get("platform") != str(platform): raise ProjectConfigurationError("Local version set is for another platform; follow the project explicitly.") - path, shared = root / ".devcapsule" / f"devcapsule.{Platform.current()}.lock", selected + path, shared = platform_lock_path(root, platform), selected else: path, shared = recommendation_lock_for(root, manifest) return path, compose_lock(manifest, shared, checkout), checkout diff --git a/devcapsule-src/devcapsule/environment_realization.py b/devcapsule-src/devcapsule/environment_realization.py index a26126e9..bdb7eab4 100644 --- a/devcapsule-src/devcapsule/environment_realization.py +++ b/devcapsule-src/devcapsule/environment_realization.py @@ -10,6 +10,8 @@ from python_on_whales.exceptions import DockerException from devcapsule.compat import CliError +from devcapsule.host_daemon import daemon_architecture +from devcapsule.platforms import Platform from devcapsule.images.build import BuildxImageBuilder, ImageBuildSpec from devcapsule.images.metadata import inspect_local_image from devcapsule.materialization import ( @@ -73,6 +75,7 @@ def realize_environment( if base_override != selected.lock.get("base", {}).get("reference"): raise CliError("A base override cannot establish the project's minimum-version guarantee; use its pinned compatible base.") locked = parse_locked_environment(selected.lock) + capsule_platform = Platform.parse(locked.platform) from devcapsule.configuration.storage import load_settings if trust is None: trust = HomeTrust.read(load_settings()) @@ -109,12 +112,22 @@ def realize_environment( selection.local_image_identity if local_authorization else locked.base_identity ) + if not local_authorization and base_override is None: + from devcapsule.resolution_matrix import MATRICES + + if MATRICES[capsule_platform].base_unpublished(base_reference): + raise CliError( + f"The lock's base {base_reference} is not published yet, so it cannot be pulled. Build the " + f"{capsule_platform} base locally ('devcapsule images build --type base --recipe ubuntu-24.04 " + f"--platform {capsule_platform} --tag IMAGE --source-revision REVISION') and select it " + "for this checkout ('devcapsule project config authorize base-image IMAGE')." + ) if obtain_image is not None: obtain = obtain_image elif local_authorization: obtain = required_local_image else: - obtain = ensure_local_image + obtain = lambda reference: ensure_local_image(reference, capsule_platform) inspect = inspect_image or optional_local_image require = require_image or required_local_image base = obtain(base_reference) @@ -138,7 +151,7 @@ def realize_environment( # One context root for every build: BuildKit's incremental transfer of # the IDE trees is keyed on this path, see BuildxImageBuilder. builder = BuildxImageBuilder(context_root=selected_cache / "build-contexts" / "context") - build = lambda spec: builder.build(spec, network="none") + build = lambda spec: builder.build(spec, network="none", platform=capsule_platform) materialize_environment = materialize or ensure_materialized_surface # Explanation/reporting hooks ride only the real materializer; injected # test materializers keep their narrower signature. @@ -146,7 +159,7 @@ def realize_environment( {} if materialize is not None else { - "runtime_pex": runtime_artifact(), + "runtime_pex": runtime_artifact(capsule_platform), "runtime_command": RuntimeCommand(runtime.get("devcapsule-command", RuntimeCommand.STANDARD)), "report": report, "list_formations": lambda: component_formations(locked.component_id), @@ -178,10 +191,11 @@ def realize_environment( ) -def ensure_local_image(reference: str) -> ImageDetails: +def ensure_local_image(reference: str, platform: Platform) -> ImageDetails: + """The local image at ``reference``, pulled for the lock's ``platform`` when absent.""" try: if not docker.image.exists(reference): - docker.image.pull(reference) + docker.image.pull(reference, platform=platform.emulated_docker_platform(daemon_architecture())) except DockerException as exc: raise CliError(f"Cannot obtain Docker image {reference!r}: {exc}") from exc return required_local_image(reference) diff --git a/devcapsule-src/devcapsule/host_daemon.py b/devcapsule-src/devcapsule/host_daemon.py index adfcb76e..ff9844e4 100644 --- a/devcapsule-src/devcapsule/host_daemon.py +++ b/devcapsule-src/devcapsule/host_daemon.py @@ -16,6 +16,9 @@ actually backs it. When a bind source is not backed by any mount, translation is impossible and the launch fails loudly rather than silently producing an empty directory. + +The daemon's architecture is the other fact about it a launch needs: a daemon +of another architecture than the capsule platform runs it under emulation. """ from __future__ import annotations @@ -207,3 +210,24 @@ def _docker(arguments: Sequence[str], env: Mapping[str, str]) -> subprocess.Comp def _unescape(value: str) -> str: return re.sub(r"\\([0-7]{3})", lambda match: chr(int(match.group(1), 8)), value) + + +def daemon_architecture() -> str: + """The Docker daemon's CPU architecture, in the platform keys' spelling. + + Emulation is the daemon's, not the launcher's: Docker Desktop on Apple + Silicon is arm64 whether the launcher runs natively on the Mac or inside + an emulated amd64 capsule granted the daemon. This host's architecture + when the daemon does not answer. + """ + from devcapsule.platforms import architecture_key, host_architecture + + try: + completed = subprocess.run( + ["docker", "version", "--format", "{{.Server.Arch}}"], + capture_output=True, text=True, timeout=30, check=False, + ) + except (OSError, subprocess.TimeoutExpired): + return host_architecture() + reported = completed.stdout.strip() + return architecture_key(reported) if completed.returncode == 0 and reported else host_architecture() diff --git a/devcapsule-src/devcapsule/host_open.py b/devcapsule-src/devcapsule/host_open.py index db25b8e0..c59c510a 100644 --- a/devcapsule-src/devcapsule/host_open.py +++ b/devcapsule-src/devcapsule/host_open.py @@ -12,6 +12,7 @@ import stat import struct import subprocess +import sys import tempfile from threading import Event, Thread import time @@ -19,6 +20,7 @@ from urllib.parse import urlsplit from devcapsule.host_daemon import in_container +from devcapsule.platforms import HostOs HOST_OPEN_SOCKET_ENV = "DEVCAPSULE_HOST_OPEN_SOCKET" @@ -32,6 +34,10 @@ RATE_LIMIT_REQUESTS = 30 RATE_LIMIT_WINDOW_SECONDS = 60.0 _UNIX_PATH_LIMIT = 100 +HOST_OPEN_UNAVAILABLE_ON_MACOS = ( + "Warning: host-browser is unavailable on macOS: Docker Desktop does not carry the " + "host's Unix sockets into a capsule. Links open in the capsule's own browser instead." +) class HostOpenError(Exception): @@ -244,6 +250,13 @@ def host_open_bridge( # It may only propagate a bridge explicitly inherited from its owner. yield None return + if not HostOs.current().binds_unix_sockets: + # The broker is a host Unix socket bound into the capsule. Docker + # Desktop refuses that bind source, so the optional integration is + # unavailable rather than fatal. + print(HOST_OPEN_UNAVAILABLE_ON_MACOS, file=sys.stderr) + yield None + return parent = _runtime_parent(environ) with _broker_socket_path(parent) as path: diff --git a/devcapsule-src/devcapsule/images/base.py b/devcapsule-src/devcapsule/images/base.py index 6bb874c7..00c8b789 100644 --- a/devcapsule-src/devcapsule/images/base.py +++ b/devcapsule-src/devcapsule/images/base.py @@ -22,6 +22,7 @@ from devcapsule.build_info import BuildInfo, BuildInfoError, current_build_info, read_pex_build_info from devcapsule.compat import CliError from devcapsule.launch.pycharm._image_build import BASE_APT_PACKAGES +from devcapsule.platforms import Platform from devcapsule.images.build import ( AptPackagesComponent, BaseImageComponent, @@ -315,10 +316,11 @@ def build_base_image( options: BaseImageBuildOptions, builder: BuildxImageBuilder | None = None, *, + platform: Platform, network: str = "default", source_verifier: Callable[[BuildInfo], None] = verify_public_github_revision, ) -> None: spec = build_base_image_spec(options) if not options.allow_local_source: source_verifier(pex_build_info(options)) - (builder or BuildxImageBuilder()).build(spec, network=network) + (builder or BuildxImageBuilder()).build(spec, network=network, platform=platform) diff --git a/devcapsule-src/devcapsule/images/build.py b/devcapsule-src/devcapsule/images/build.py index 6dc09616..46fcd72e 100644 --- a/devcapsule-src/devcapsule/images/build.py +++ b/devcapsule-src/devcapsule/images/build.py @@ -19,6 +19,8 @@ from python_on_whales.exceptions import DockerException from devcapsule.compat import CliError, copy_resource_tree +from devcapsule.host_daemon import daemon_architecture +from devcapsule.platforms import Platform @dataclass(frozen=True) @@ -256,8 +258,10 @@ class BuildxImageBuilder: def __init__(self, context_root: Path | None = None) -> None: self.context_root = context_root - def build(self, spec: ImageBuildSpec, *, network: str = "default") -> None: + def build(self, spec: ImageBuildSpec, *, platform: Platform, network: str = "default") -> None: + """Build ``spec`` for the capsule ``platform``.""" plan = spec.build_plan() + emulated = platform.emulated_docker_platform(daemon_architecture()) try: with self._context_directory() as context_root: rendered = render_build_context(plan, context_root) @@ -266,6 +270,7 @@ def build(self, spec: ImageBuildSpec, *, network: str = "default") -> None: file=rendered.dockerfile, tags=[plan.image], load=True, + platforms=[emulated] if emulated is not None else None, network=network, build_contexts={name: str(path) for name, path in rendered.named_contexts.items()}, allow=["network.host"] if network == "host" else [], diff --git a/devcapsule-src/devcapsule/launch/pycharm/_image_build.py b/devcapsule-src/devcapsule/launch/pycharm/_image_build.py index 5f1f4f63..0e3a7868 100644 --- a/devcapsule-src/devcapsule/launch/pycharm/_image_build.py +++ b/devcapsule-src/devcapsule/launch/pycharm/_image_build.py @@ -24,6 +24,7 @@ resource_to_tempdir, ) from devcapsule.images.tooling import NODE_CURRENT_BIN, node_tooling_component +from devcapsule.platforms import Platform DEFAULT_BASE_IMAGE = "ubuntu:24.04" DEFAULT_IMAGE = "pycharm-isolated:latest" @@ -162,7 +163,7 @@ def build_pycharm_image(options: PycharmImageBuildOptions, builder: BuildxImageB build_root = Path(stack.enter_context(tempfile.TemporaryDirectory(prefix="devcapsule-pycharm-build-"))) pycharm_root = normalize_pycharm_source(options.pycharm, build_root) spec = build_pycharm_image_spec(options, pycharm_root=pycharm_root, assets_root=assets_dir) - builder.build(spec, network=options.network) + builder.build(spec, platform=Platform.current(), network=options.network) return 0 diff --git a/devcapsule-src/devcapsule/launch/pycharm/_launcher.py b/devcapsule-src/devcapsule/launch/pycharm/_launcher.py index 2109ef40..e8d6fabb 100644 --- a/devcapsule-src/devcapsule/launch/pycharm/_launcher.py +++ b/devcapsule-src/devcapsule/launch/pycharm/_launcher.py @@ -16,12 +16,14 @@ import tempfile from threading import Event import time -from dataclasses import dataclass, field +from dataclasses import dataclass, field, replace +from datetime import datetime, timedelta, timezone from enum import Enum from pathlib import Path from typing import Callable, Mapping, Sequence from ...docker_arguments import inline_environment_names, redact_environment +from ...platforms import HostOs, Platform from ...container_runtime.contract import ( CONTAINED_DISPLAY_TRANSPORT, INFRASTRUCTURE_FAILURE_EXIT_CODE, @@ -59,6 +61,7 @@ ) from ...host_daemon import ( current_container, + daemon_architecture, docker_socket, is_host_backed, requires_translation, @@ -127,10 +130,17 @@ class TempRuntimeFiles: # The console's configuration channel: a directory the capsule writes # requests into and reads the launch-time listing from; CHANNEL_PATH inside. configuration_directory: Path | None = None + # On macOS, the Docker volume that holds the root-owned sudo policy. + sudoers_volume: str | None = None SUDOERS_POLICY_PATH = "/etc/sudoers.d/devcapsule-development-sudo" SUDOERS_POLICY = "%ide-sudo ALL=(ALL:ALL) NOPASSWD: ALL\n" +SUDOERS_VOLUME_FILE = "policy" +"""The policy's name inside its macOS volume.""" +SUDOERS_VOLUME_LABEL = "devcapsule.purpose=development-sudo-policy" +ABANDONED_SUDOERS_VOLUME_AGE = timedelta(days=1) +"""How old a policy volume no container uses must be to count as a killed launch's.""" @@ -167,6 +177,8 @@ def web_console_notice(availability: WebConsoleAvailability) -> str: @dataclass class PycharmRunOptions: project: Path + # The lock's platform: what the capsule runs as. + capsule_platform: Platform # Project configuration is complete; only the legacy command inherits # ambient launcher options. Host observations and explicit secret sources # remain available in either mode. @@ -267,6 +279,9 @@ class PycharmRunConfig: memory_limit_bytes: int | None runtime_plan: RuntimePlan | None use_image_process: bool + capsule_platform: Platform + # The Docker daemon's, asked once per launch: it decides emulation. + daemon_architecture: str cpus: int | None = None launch_configuration: LaunchConfiguration | None = None host_browser_socket: Path | None = None @@ -299,6 +314,13 @@ class PycharmRunConfig: libgl_dri3_disable: str = "1" +HOST_NETWORK_UNAVAILABLE_ON_MACOS = ( + "Warning: host networking is unavailable on macOS: Docker Desktop's host network " + "is its virtual machine's, out of the Mac's reach. This launch uses bridge networking; " + "services on the Mac are reachable from the capsule as host.docker.internal." +) + + def run_pycharm(options: PycharmRunOptions, env: Mapping[str, str] | None = None) -> int: """Run PyCharm through the translated Python launcher.""" @@ -403,6 +425,7 @@ def run_pycharm(options: PycharmRunOptions, env: Mapping[str, str] | None = None print(session_result_text(config, result), file=sys.stderr, flush=True) return exit_code finally: + remove_sudoers_policy_volume(files.sudoers_volume, runtime_env) cleanup_temp_runtime_files(files) except HostOpenError as exc: raise PycharmRunError(str(exc)) from exc @@ -816,6 +839,12 @@ def build_run_config(options: PycharmRunOptions, env: Mapping[str, str]) -> Pych ) if options.network_mode not in {"bridge", "host", "none"}: raise PycharmRunError("The Docker network mode must be bridge, host, or none.") + if options.network_mode == "host" and not HostOs.current().host_network_reachable: + # Docker Desktop's host network is its VM's: a listener on loopback + # there is unreachable from the Mac (observed on 29.3.1), so the + # desktop and console would never answer. Bridge is the narrower grant. + print(HOST_NETWORK_UNAVAILABLE_ON_MACOS, file=sys.stderr) + options = replace(options, network_mode="bridge") if options.use_image_process and options.runtime_plan is None: raise PycharmRunError("Using the image process requires an external runtime plan.") contained_display = options.display_transport == CONTAINED_DISPLAY_TRANSPORT @@ -957,7 +986,12 @@ def build_run_config(options: PycharmRunOptions, env: Mapping[str, str]) -> Pych "Start Docker on the host, set HOST_DOCKER_SOCKET, or launch with --docker-in-docker / --no-docker." ) host_docker_socket = host_docker_socket.resolve() - host_docker_gid = host_docker_socket.stat().st_gid + # Where the host decides the socket's group inside containers (Docker + # Desktop: root), the capsule user joins it; a capsule granted the + # daemon already holds root on its virtual machine, so the group adds + # no authority. Elsewhere the host socket file's group applies. + socket_gid = HostOs.current().docker_socket_gid + host_docker_gid = socket_gid if socket_gid is not None else host_docker_socket.stat().st_gid if enable_sudo: while ide_sudo_gid in {0, host_user.gid, host_docker_gid}: @@ -1164,6 +1198,8 @@ def build_run_config(options: PycharmRunOptions, env: Mapping[str, str]) -> Pych console_token=console_token, open_console_url=options.open_console_url, web_console=options.web_console, + capsule_platform=options.capsule_platform, + daemon_architecture=daemon_architecture(), interactive_state_mounts=tuple(interactive_state_mounts), additional_state_mounts=tuple(additional_state_mounts), additional_environment=tuple(additional_environment), @@ -1293,6 +1329,7 @@ def build_docker_args( args = [ "--rm", "-i", + *config.capsule_platform.docker_platform_args(config.daemon_architecture), "--name", config.name, "--workdir", @@ -1394,12 +1431,12 @@ def build_docker_args( "Development sudo requires generated shadow and sudoers files." ) args.extend(["--mount", f"type=bind,src={files.shadow_file},dst=/etc/shadow,ro"]) - args.extend( - [ - "--mount", - f"type=bind,src={files.sudoers_file},dst={SUDOERS_POLICY_PATH},ro", - ] + policy_source = ( + f"type=volume,src={files.sudoers_volume},volume-subpath={SUDOERS_VOLUME_FILE}" + if files.sudoers_volume + else f"type=bind,src={files.sudoers_file}" ) + args.extend(["--mount", f"{policy_source},dst={SUDOERS_POLICY_PATH},ro"]) if files.runtime_plan_file is not None: args.extend( [ @@ -1816,7 +1853,7 @@ def write_user_files(config: PycharmRunConfig, files: TempRuntimeFiles) -> None: "root:x:0:", f"{host_user.group_name}:x:{host_user.gid}:", ] - if config.host_docker_gid is not None and config.host_docker_gid != host_user.gid: + if config.host_docker_gid not in (None, 0, host_user.gid): group_lines.append(f"host-docker:x:{config.host_docker_gid}:") if config.enable_sudo: group_lines.append(f"ide-sudo:x:{config.ide_sudo_gid}:{host_user.name}") @@ -1845,6 +1882,9 @@ def ensure_sudoers_policy_ownership( policy = files.sudoers_file if not config.enable_sudo or policy is None: raise PycharmRunError("Development sudo requires a generated sudoers policy.") + if not HostOs.current().chown_reaches_bind_mounts: + files.sudoers_volume = prepare_sudoers_policy_volume(config, policy, env) + return try: os.chown(policy, 0, 0) except PermissionError: @@ -1854,6 +1894,7 @@ def ensure_sudoers_policy_ownership( "run", "--rm", "--pull=never", + *config.capsule_platform.docker_platform_args(config.daemon_architecture), "--network", "none", "--read-only", @@ -1895,6 +1936,121 @@ def ensure_sudoers_policy_ownership( ) +def prepare_sudoers_policy_volume(config: PycharmRunConfig, policy: Path, env: Mapping[str, str]) -> str: + """Write the sudo policy into a Docker volume as root; return the volume's name. + + sudo ignores a policy root does not own. Docker Desktop shares Mac files + with its virtual machine as the Mac user: a bind-mounted file is owned by + that user inside every container, and not even root there can chown it. + A volume lives on the virtual machine's own disk, where root's ownership + holds. The session removes it at its end; the next launch removes one a + killed launch left behind (``remove_abandoned_sudoers_volumes``). + """ + + volume = f"{config.name}-sudoers" + remove_abandoned_sudoers_volumes(env) + created = subprocess.run( + ["docker", "volume", "create", "--label", SUDOERS_VOLUME_LABEL, volume], + check=False, env=dict(env), stdout=subprocess.DEVNULL, stderr=subprocess.PIPE, text=True, + ) + if created.returncode != 0: + detail = created.stderr.strip() + raise PycharmRunError( + "Could not create the volume for the development-sudo policy" + (f": {detail}" if detail else ".") + ) + staged = f"/policy/{SUDOERS_VOLUME_FILE}.new" + command = [ + "docker", + "run", + "--rm", + "-i", + "--pull=never", + *config.capsule_platform.docker_platform_args(config.daemon_architecture), + "--network", + "none", + "--read-only", + "--pids-limit", + "64", + "--cap-drop", + "ALL", + "--security-opt", + "no-new-privileges", + "--user", + "0:0", + "--mount", + f"type=volume,src={volume},dst=/policy", + "--entrypoint", + "/bin/sh", + config.image, + "-ec", + # 0440 root:root, as sudo requires. A policy left by an earlier + # launch is read-only even to root without capabilities: replace it. + f"umask 0337; rm -f {staged}; cat > {staged}; mv -f {staged} /policy/{SUDOERS_VOLUME_FILE}", + ] + completed = subprocess.run( + command, + check=False, + env=dict(env), + input=policy.read_text(encoding="utf-8"), + stdout=subprocess.DEVNULL, + stderr=subprocess.PIPE, + text=True, + ) + if completed.returncode != 0: + remove_sudoers_policy_volume(volume, env) + detail = completed.stderr.strip() + raise PycharmRunError( + "Could not prepare the root-owned development-sudo policy" + (f": {detail}" if detail else ".") + ) + return volume + + +def remove_abandoned_sudoers_volumes(env: Mapping[str, str]) -> None: + """Remove the policy volumes of launches that ended without removing their own. + + A concurrent launch's volume is in no container's use between its + creation and its capsule's start, so only a volume older than + ``ABANDONED_SUDOERS_VOLUME_AGE`` is taken for abandoned; Docker refuses + to remove one a container still uses. + """ + + listed = subprocess.run( + ["docker", "volume", "ls", "--quiet", "--filter", f"label={SUDOERS_VOLUME_LABEL}"], + check=False, env=dict(env), capture_output=True, text=True, + ) + volumes = listed.stdout.split() if listed.returncode == 0 else [] + if not volumes: + return + inspected = subprocess.run( + ["docker", "volume", "inspect", "--format", "{{.Name}} {{.CreatedAt}}", *volumes], + check=False, env=dict(env), capture_output=True, text=True, + ) + cutoff = datetime.now(timezone.utc) - ABANDONED_SUDOERS_VOLUME_AGE + abandoned = [] + for line in inspected.stdout.splitlines(): + volume, _, created = line.partition(" ") + try: + if datetime.fromisoformat(created) < cutoff: + abandoned.append(volume) + except ValueError: + continue # A creation time Docker did not report is no evidence of age. + if abandoned: + subprocess.run( + ["docker", "volume", "rm", *abandoned], + check=False, env=dict(env), stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, + ) + + +def remove_sudoers_policy_volume(volume: str | None, env: Mapping[str, str]) -> None: + """Remove the policy volume; one a capsule still uses is left for a later launch to find abandoned.""" + + if volume: + subprocess.run( + ["docker", "volume", "rm", "--force", volume], + check=False, env=dict(env), stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, + ) + + def file_owner_uid(path: Path) -> int: return path.stat().st_uid diff --git a/devcapsule-src/devcapsule/platforms.py b/devcapsule-src/devcapsule/platforms.py index 2c1cbfa4..46914079 100644 --- a/devcapsule-src/devcapsule/platforms.py +++ b/devcapsule-src/devcapsule/platforms.py @@ -20,36 +20,170 @@ import platform as host_platform from typing import Mapping -__all__ = ["Platform", "UnsupportedPlatformError", "XdgHomes"] +__all__ = ["HostOs", "Platform", "UnsupportedPlatformError", "XdgHomes", "architecture_key", "host_architecture"] class UnsupportedPlatformError(ValueError): """A platform key outside the supported set; the message is displayable.""" -class Platform(StrEnum): - """The supported-platform declaration and its wire format. +def architecture_key(machine: str) -> str: + """A CPU architecture as a machine or a Docker daemon reports it, in the platform keys' spelling.""" + + lowered = machine.lower() + return {"x86_64": "amd64", "aarch64": "arm64"}.get(lowered, lowered) + + +def host_architecture() -> str: + """This host's CPU architecture in the platform keys' spelling.""" - Members are the single authority on what platforms DevCapsule supports; - values are the wire format serialized into lock filenames, lock - documents, and artifact tables. Because committed locks carry these - values, members are append-only and never renamed. + return architecture_key(host_platform.machine()) - A client never constructs a key from parts: it obtains one from - ``current()`` (the platform it is running on) or ``parse()`` (the - platform a lock records), and only carries it between those points. + +class HostOs(StrEnum): + """The host operating system, for the mechanisms that differ by host. + + Locks never carry it: capsules are Linux on every host, and the lock + names the capsule platform (``Platform``). On macOS the container runtime + is Docker Desktop's virtual machine; its conventions, observed on Docker + Desktop 29.3.1, are the properties below, which feature code consults + instead of asking which host it is on. + """ + + LINUX = "linux" + DARWIN = "darwin" + + @classmethod + def current(cls) -> HostOs: + system = host_platform.system().lower() + try: + return cls(system) + except ValueError: + supported = ", ".join(member.value for member in cls) + raise UnsupportedPlatformError( + f"Unsupported host operating system {system!r}; supported: {supported}." + ) from None + + @property + def docker_socket_gid(self) -> int | None: + """The group owning the Docker socket inside a container, when the host decides it. + + Docker Desktop presents its socket as ``root:root``, mode 0660, in every + container, whatever the Mac file's owner. None on Linux, where the host + socket file's own group applies. + """ + + return 0 if self is HostOs.DARWIN else None + + @property + def host_network_reachable(self) -> bool: + """Whether a listener on Docker's host network is reachable from this host. + + Docker Desktop's host network is its virtual machine's, out of the Mac's reach. + """ + + return self is HostOs.LINUX + + @property + def binds_unix_sockets(self) -> bool: + """Whether a host Unix socket can be bind-mounted into a container. + + Docker Desktop rewrites the source under ``/socket_mnt`` and refuses it. + """ + + return self is HostOs.LINUX + + @property + def chown_reaches_bind_mounts(self) -> bool: + """Whether root in a container can change a bind-mounted host file's owner. + + Docker Desktop shares Mac files as the Mac user; a chown inside is refused. + """ + + return self is HostOs.LINUX + + +# The capsule platforms each host runs, native first. A Linux host runs its +# own platform; Apple Silicon runs linux-arm64 natively and linux-amd64 under +# Docker Desktop's emulation; an Intel Mac runs linux-amd64. +_CAPSULE_PLATFORMS_OF_HOST: Mapping[tuple[HostOs, str], tuple[str, ...]] = { + (HostOs.LINUX, "amd64"): ("linux-amd64",), + (HostOs.LINUX, "arm64"): ("linux-arm64",), + (HostOs.DARWIN, "arm64"): ("linux-arm64", "linux-amd64"), + (HostOs.DARWIN, "amd64"): ("linux-amd64",), +} + +# The platform whose executables carry each ELF e_machine value. +_PLATFORM_OF_ELF_MACHINE: Mapping[int, str] = {0x3E: "linux-amd64", 0xB7: "linux-arm64"} + + +class Platform(StrEnum): + """The supported capsule platforms and their wire format. + + A platform is what capsules run as, not what the launcher runs on: a + Mac launches Linux capsules. Members are the single authority on what + capsule platforms DevCapsule supports; values are the wire format + serialized into lock filenames, lock documents, and artifact tables. + Because committed locks carry these values, members are append-only and + never renamed. + + A client never constructs a key from parts. Host detection gives + ``current()`` (the platform this host runs natively) and ``runnable()`` + (every platform it runs); ``parse()`` gives the platform a lock records, + and ``of_elf_machine()`` the one an executable is for. + Which platform a project runs as is the configuration's question + (``configuration.storage.project_platform``), answered from its locks. """ LINUX_AMD64 = "linux-amd64" + LINUX_ARM64 = "linux-arm64" @classmethod def current(cls) -> Platform: - """The platform this process is running on.""" + """The platform this host runs natively, and a new project's unless the developer chooses another.""" - system = host_platform.system().lower() - machine = host_platform.machine().lower() - architecture = {"x86_64": "amd64", "aarch64": "arm64"}.get(machine, machine) - return cls.parse(f"{system}-{architecture}") + return cls.runnable()[0] + + @classmethod + def runnable(cls) -> tuple[Platform, ...]: + """Every platform this host runs capsules as, native first.""" + + host_os, architecture = HostOs.current(), host_architecture() + keys = _CAPSULE_PLATFORMS_OF_HOST.get((host_os, architecture), (f"{host_os}-{architecture}",)) + return tuple(cls.parse(key) for key in keys) + + @classmethod + def of_elf_machine(cls, machine: int) -> Platform | None: + """The platform an ELF executable of ``machine`` runs on, or None for another architecture.""" + + key = _PLATFORM_OF_ELF_MACHINE.get(machine) + return cls(key) if key is not None else None + + @property + def architecture(self) -> str: + return self.value.split("-", 1)[1] + + @property + def docker_platform(self) -> str: + """Docker's spelling of this platform, as ``--platform`` takes it.""" + + return self.value.replace("-", "/", 1) + + def emulated_docker_platform(self, daemon_architecture: str) -> str | None: + """The platform every run, build and pull names to a daemon that emulates this one. + + A Docker daemon of ``daemon_architecture`` defaults to its own + architecture, so it must be told this platform when the two differ. + None when it runs this platform natively and is told nothing. + """ + + return self.docker_platform if daemon_architecture != self.architecture else None + + def docker_platform_args(self, daemon_architecture: str) -> list[str]: + """``emulated_docker_platform`` as the option of a ``docker run`` command line.""" + + emulated = self.emulated_docker_platform(daemon_architecture) + return [f"--platform={emulated}"] if emulated is not None else [] @classmethod def parse(cls, key: str) -> Platform: diff --git a/devcapsule-src/devcapsule/recursive_successor.py b/devcapsule-src/devcapsule/recursive_successor.py index 30f68ed8..b07c9ee3 100644 --- a/devcapsule-src/devcapsule/recursive_successor.py +++ b/devcapsule-src/devcapsule/recursive_successor.py @@ -25,6 +25,7 @@ from devcapsule.display_client import display_url, select_display_transport from devcapsule.environment_realization import realize_environment from devcapsule.materialization import RUNTIME_PLAN_PATH as SUCCESSOR_RUNTIME_PLAN_PATH +from devcapsule.configuration.capability_selection import lock_platform from devcapsule.configuration.storage import ( atomic_write, ) @@ -505,6 +506,7 @@ def _resolved_run_options( ) return PycharmRunOptions( project=selected.root, + capsule_platform=lock_platform(selected.lock), project_mount=str(runtime["project-mount"]), image=image, name=name, diff --git a/devcapsule-src/devcapsule/recursive_successor_plan.py b/devcapsule-src/devcapsule/recursive_successor_plan.py index b400b87e..1c1b9796 100644 --- a/devcapsule-src/devcapsule/recursive_successor_plan.py +++ b/devcapsule-src/devcapsule/recursive_successor_plan.py @@ -35,6 +35,7 @@ "--name", "--network", "--pids-limit", + "--platform", "--restart", "--security-opt", "--tmpfs", @@ -109,9 +110,12 @@ class ExpectedSuccessorPlan: security_opt: tuple[str, ...] runtime_plan_destination: str runtime_plan_digest: str + # The planned ``--platform`` (``linux/amd64``), when the daemon emulates the + # capsule platform; None when the image's own platform is the daemon's. + platform: str | None = None def to_mapping(self, *, show_host_paths: bool = False) -> dict[str, Any]: - return { + mapping = { "schema_version": 1, "name": self.name, "image_reference": self.image_reference, @@ -139,6 +143,11 @@ def to_mapping(self, *, show_host_paths: bool = False) -> dict[str, Any]: "runtime_plan_destination": self.runtime_plan_destination, "runtime_plan_digest": self.runtime_plan_digest, } + # Only an emulated plan carries the key, so a native plan's digest is + # the one a launcher without emulation computed for it. + if self.platform is not None: + mapping["platform"] = self.platform + return mapping def to_json(self, *, show_host_paths: bool = False) -> str: return json.dumps( @@ -184,6 +193,7 @@ def from_mapping(cls, value: Mapping[str, Any]) -> ExpectedSuccessorPlan: security_opt=_string_tuple(value.get("security_opt")), runtime_plan_destination=_required_string(value, "runtime_plan_destination"), runtime_plan_digest=_required_string(value, "runtime_plan_digest"), + platform=_optional_string(value.get("platform"), "platform"), ) @classmethod @@ -270,6 +280,7 @@ def from_docker_args( security_opt=tuple(sorted(parsed.repeated("--security-opt"))), runtime_plan_destination=runtime_plan_destination, runtime_plan_digest=runtime_plan_digest, + platform=parsed.optional("--platform"), ) @@ -291,6 +302,13 @@ def compare_inspection( raise SuccessorPlanError("successor name does not match the deterministic run name") if inspection.get("Image") != plan.image_identity: raise SuccessorPlanError("successor image ID does not match materialization evidence") + # The image ID fixes the architecture; where the daemon also reports the + # platform it ran, an emulated plan must have run as the one it named. + descriptor = inspection.get("ImageManifestDescriptor") + reported = descriptor.get("platform") if isinstance(descriptor, Mapping) else None + if plan.platform is not None and isinstance(reported, Mapping): + if f"{reported.get('os')}/{reported.get('architecture')}" != plan.platform: + raise SuccessorPlanError("successor platform does not match the planned --platform") if config.get("Image") != plan.image_reference: raise SuccessorPlanError("successor image reference does not match the canonical environment") @@ -572,6 +590,12 @@ def _required_bool(value: Mapping[str, Any], name: str) -> bool: return found +def _optional_string(value: Any, name: str) -> str | None: + if value is not None and (not isinstance(value, str) or not value): + raise SuccessorPlanError(f"retained expected plan has no usable {name}") + return value + + def _optional_int(value: Any, name: str) -> int | None: if value is None: return None diff --git a/devcapsule-src/devcapsule/resolution_matrix.py b/devcapsule-src/devcapsule/resolution_matrix.py index 3df46d48..05c1eea9 100644 --- a/devcapsule-src/devcapsule/resolution_matrix.py +++ b/devcapsule-src/devcapsule/resolution_matrix.py @@ -59,6 +59,7 @@ from devcapsule.components.catalog import COMPONENTS from devcapsule.components.playwright_pin import PIN as PLAYWRIGHT_PIN from devcapsule.components.eclipse_native_pin import PACKAGES as ECLIPSE_NATIVE_PACKAGES +from devcapsule.components.linux_arm64_pins import COMPONENT_LOCK_TABLES as LINUX_ARM64_LOCK_TABLES from devcapsule.configuration.file_formats import ( ProjectConfigurationError, canonical_digest, @@ -100,7 +101,7 @@ class Capability: @dataclass(frozen=True) class _BasePin: - """One published base image and the capabilities it ships.""" + """One base image and the capabilities it ships.""" mnemonic: str # The family of base releases this pin belongs to — see _VerifiedEdge: @@ -122,6 +123,9 @@ class _BasePin: runtime: str = LAUNCHER_SUPPLIED_RUNTIME source_revision: str | None = None # A clean public-source builder need not have a release tag. + # False while the reference names an image its registry does not hold + # yet: only a locally built base of the same recipe stands in for it. + published: bool = True @property def contract(self) -> BaseContract: @@ -246,6 +250,10 @@ def base_family(self, lock: Mapping[str, Any]) -> str | None: return next((base.base_family for base in self._bases if base.lock_table.get("reference") == reference), None) + def base_unpublished(self, reference: str) -> bool: + """Whether the pinned base at ``reference`` is not in its registry yet.""" + return any(base.lock_table.get("reference") == reference and not base.published for base in self._bases) + def base_image(self, reference: str) -> BaseImage | None: """The pinned base at ``reference``, described by its contract.""" return next((base.image for base in self._bases @@ -394,6 +402,14 @@ def component_pins(self) -> dict[str, Mapping[str, Any]]: """The latest embedded table for each component on this platform.""" return {name: pins[-1].lock_table for name, pins in self._components.items() if pins} + @property + def platform(self) -> Platform: + return self._platform + + def component_pin(self, component: str, version: object) -> Mapping[str, Any] | None: + """This platform's embedded table for one component version, or None.""" + return next((pin.lock_table for pin in self._components.get(component, ()) if pin.version == version), None) + def component_closure(self, capability: str, minimum: str | None = None, lock: Mapping[str, Any] | None = None, *, choice: str | None = None, recommended: str | None = None) -> tuple[str, ...]: @@ -1702,12 +1718,121 @@ def _formation( }, ) +# The linux-arm64 base, recipe 11: the amd64 recipe-11 recipe built natively +# on an Apple Silicon Mac (Docker Desktop 29.3.1) on 2026-11-04 by +# `images build --type base --recipe ubuntu-24.04` from a launcher built at +# the public main commit b3f10b7d, whose devcapsule-webconsole the image +# installed. Inspected: linux/arm64, recipe 11 ready, display=contained, +# runtime=launcher-supplied, all six services, the web console root. +# Not published yet. The reference is the digest of the OCI image index in +# the builder's containerd store, which a push of this exact image to +# mycodespaceai/devcapsule-base keeps; a rebuild gets another digest and +# must be pinned again. Until the owner pushes it, a checkout selects the +# local image: `project config authorize base-image IMAGE`. +_LINUX_ARM64_RECIPE_11_BASE = _BasePin( + mnemonic="v0.3.1.dev0-local-macos-aarch64", + source_revision="b3f10b7d95aec659f268afecc03d0912a2e89d2f", + base_family=_BASE_FAMILY_UBUNTU_24_04, + satisfies={ + "python": "3.12", "docker-cli": None, "node": NODE_VERSION, + "java": TEMURIN_VERSION, "maven": MAVEN_VERSION, "postgresql-client": None, + }, + recipe=11, + display=CONTAINED_DISPLAY, + runtime=LAUNCHER_SUPPLIED_RUNTIME, + published=False, + lock_table={ + "reference": ( + "docker.io/mycodespaceai/devcapsule-base" + "@sha256:7d32268065b60b6ad75139d23c1e6fbb2daa43b3f2c53d2856c5e2e39891652b" + ), + "build-mnemonic": "v0.3.1.dev0-local-macos-aarch64", + "contract": "ubuntu-24.04@11", + "recipe-source": RECIPE_SOURCE_PATH, + }, +) + + +def _on_linux_arm64(pin: _ComponentPin) -> _ComponentPin: + """The arm64 build of a linux-amd64 pin: same version, its own artifacts. + + A base-delivered component carries no artifact and is the same pin on + both platforms; every other pin must have its arm64 table, so an amd64 + version added without one fails here rather than at a user's launch. + """ + + table = LINUX_ARM64_LOCK_TABLES.get((pin.component_id, pin.version)) + if table is None and pin.lock_table.get("delivery-policy") == "base-image": + return pin + if table is None: + raise AssertionError(f"{pin.component_id} {pin.version} has no linux-arm64 pin in components/linux_arm64_pins.py") + return _ComponentPin(pin.component_id, pin.version, table) + + +# linux-arm64 offers the same vocabulary, versions and materialization +# recipes as linux-amd64; its bases, artifacts and verified edges are its +# own, because a validation on one architecture establishes nothing on another. +_LINUX_ARM64_MATRIX = ResolutionMatrix( + platform=Platform.LINUX_ARM64, + matrix_version=_MATRIX_VERSION, + bases=(_LINUX_ARM64_RECIPE_11_BASE,), + components={ + component: tuple(_on_linux_arm64(pin) for pin in pins) + for component, pins in _LINUX_AMD64_MATRIX._components.items() + }, + edges=( + # Native IDE smokes on Apple Silicon (Docker Desktop 29.3.1) against + # the local, unpublished arm64 recipe-11 base above. They establish + # the IDE only: Playwright and the .NET SDK rode along unexercised. + _VerifiedEdge( + "codium", "1.143.17232", _BASE_FAMILY_UBUNTU_24_04, + "noVNC HTTP/window startup passed natively on the local arm64 recipe-11 base, " + "2026-11-04 run 20261104T120158Z-e8f78f; VSCodium window; no pixel, saved-edit " + "or persistence check", + ), + _VerifiedEdge( + "vscode", "1.144.0", _BASE_FAMILY_UBUNTU_24_04, + "noVNC HTTP/window startup passed natively on the local arm64 recipe-11 base, " + "2026-11-04 run 20261104T120217Z-ee18ce; Visual Studio Code window; no pixel, " + "saved-edit or persistence check", + ), + _VerifiedEdge( + "eclipse", "2026-09-R", _BASE_FAMILY_UBUNTU_24_04, + "noVNC HTTP/window startup passed natively on the local arm64 recipe-11 base, " + "2026-11-04 run 20261104T120256Z-aa127a; workspace window; no pixel, saved-edit " + "or persistence check", + ), + _VerifiedEdge( + "pycharm", "2026.2.4", _BASE_FAMILY_UBUNTU_24_04, + "noVNC HTTP/window startup passed natively on the local arm64 recipe-11 base, " + "2026-11-04 run 20261104T120351Z-38cdd0; splash window; first-use agreement and " + "editor acceptance remain", + ), + _VerifiedEdge( + "rider", "2026.2.4", _BASE_FAMILY_UBUNTU_24_04, + "noVNC HTTP/window startup passed natively on the local arm64 recipe-11 base, " + "2026-11-04 run 20261104T120546Z-c9c1b3; splash window; first-use agreement and " + "editor acceptance remain", + ), + _VerifiedEdge( + "intellij", "2026.2.4", _BASE_FAMILY_UBUNTU_24_04, + "noVNC HTTP/window startup passed natively on the local arm64 recipe-11 base, " + "2026-11-04 run 20261104T121007Z-289109; splash window; first-use agreement and " + "editor acceptance remain", + ), + ), + couplings=(), + capability_table=_LINUX_AMD64_MATRIX._capability_table, + materialization=_LINUX_AMD64_MATRIX._materialization, +) + # The map is the platform authority: its keys are the supported platforms, -# total over the Platform enum (D-0006/D-0007). Clients index it with a key -# from Platform.current() or Platform.parse(), never one built from parts. +# total over the Platform enum (D-0006/D-0007). Clients index it with a +# Platform member, from host detection or a lock, never a key built from parts. MATRICES: Mapping[Platform, ResolutionMatrix] = MappingProxyType( { Platform.LINUX_AMD64: _LINUX_AMD64_MATRIX, + Platform.LINUX_ARM64: _LINUX_ARM64_MATRIX, } ) diff --git a/devcapsule-src/devcapsule/runtime_artifact.py b/devcapsule-src/devcapsule/runtime_artifact.py index 0dc7353a..476a16c5 100644 --- a/devcapsule-src/devcapsule/runtime_artifact.py +++ b/devcapsule-src/devcapsule/runtime_artifact.py @@ -6,23 +6,128 @@ from pathlib import Path import sys -from devcapsule.build_info import BuildInfoError, read_pex_build_info +from devcapsule.build_info import BuildInfo, BuildInfoError, current_build_info, read_pex_build_info from devcapsule.compat import CliError +from devcapsule.platforms import HostOs, Platform +RUNTIME_PEX_ENV = "DEVCAPSULE_RUNTIME_PEX" + +_MACH_O_MAGICS = (b"\xcf\xfa\xed\xfe", b"\xce\xfa\xed\xfe", b"\xca\xfe\xba\xbe") +_UNKNOWN_REVISION = "unknown" + + +def runtime_artifact(capsule_platform: Platform) -> Path: + """The DevCapsule executable a launch delivers into a ``capsule_platform`` capsule. + + A packaged launcher delivers itself when it runs on the capsule platform + (D-0009). A launcher that cannot, such as a macOS executable launching + Linux capsules, delivers the runtime ``DEVCAPSULE_RUNTIME_PEX`` names: one + executable, or a directory holding one per capsule platform as + ``devcapsule-``. A source-form launcher names its built runtime + the same way. A delivered runtime that is not the launcher's own bytes + must be the launcher's build: the same version, and the same source + revision wherever both record one (R-RUNTIME-001). + """ -def runtime_artifact() -> Path: # scie preserves the executable's path in SCIE; PEX points to the original # archive too. sys.argv[0] alone can name the extracted console script. - # A source-form launcher must explicitly select its built runtime. - selected = (os.environ.get("SCIE") or os.environ.get("PEX") - or os.environ.get("DEVCAPSULE_RUNTIME_PEX") or sys.argv[0]) - path = Path(selected).expanduser().resolve() + own_variable = os.environ.get("SCIE") or os.environ.get("PEX") + own = _resolved(own_variable) if own_variable else None + named = os.environ.get(RUNTIME_PEX_ENV) + if own is not None and _runs_in(own, capsule_platform): + delivered = own + elif named: + delivered = _named_runtime(Path(named).expanduser(), capsule_platform) + else: + delivered = own or _resolved(sys.argv[0]) try: - read_pex_build_info(path) + delivered_build = read_pex_build_info(delivered) except BuildInfoError as exc: raise CliError( "Environment materialization needs a DevCapsule PEX. Run the built PEX, " "or build with 'nox -s pex' and set DEVCAPSULE_RUNTIME_PEX to its path " "when running from source." ) from exc - return path + if not _runs_in(delivered, capsule_platform): + raise CliError( + f"{delivered} is {_describe(executable_platform(delivered))} and cannot run inside " + f"{capsule_platform} capsules. Set {RUNTIME_PEX_ENV} to a {capsule_platform} DevCapsule " + f"executable of the same version, or to a directory holding it as devcapsule-{capsule_platform}." + ) + if delivered != own: + _require_same_build(delivered, delivered_build, current_build_info()) + return delivered + + +def executable_platform(path: Path) -> Platform | HostOs | None: + """What a native executable runs on, from its header. + + The ``Platform`` of an ELF executable of a known architecture, + ``HostOs.LINUX`` for one of another architecture, ``HostOs.DARWIN`` for any + Mach-O file, and None for anything else, such as a plain PEX archive. + """ + + try: + with path.open("rb") as handle: + header = handle.read(20) + except OSError: + return None + if header[:4] in _MACH_O_MAGICS: + return HostOs.DARWIN + if header[:4] == b"\x7fELF" and len(header) >= 20: + machine = int.from_bytes(header[18:20], "little" if header[5] == 1 else "big") + return Platform.of_elf_machine(machine) or HostOs.LINUX + return None + + +def _runs_in(path: Path, capsule_platform: Platform) -> bool: + """Whether the executable at ``path`` runs inside ``capsule_platform`` capsules. + + A plain PEX bundles native wheels for the machine it was built on, so it + counts as the capsule platform's only on a Linux host, whose one capsule + platform is its own. + """ + + runs_on = executable_platform(path) + if runs_on is None: + return HostOs.current() is HostOs.LINUX and Platform.runnable() == (capsule_platform,) + return runs_on == capsule_platform + + +def _named_runtime(named: Path, capsule_platform: Platform) -> Path: + if not named.is_dir(): + return _resolved(str(named)) + candidate = named / f"devcapsule-{capsule_platform}" + if not candidate.is_file(): + raise CliError( + f"{RUNTIME_PEX_ENV} names {named}, which holds no devcapsule-{capsule_platform}: this launch " + f"runs a {capsule_platform} capsule. Put a {capsule_platform} DevCapsule executable of the same " + f"version there, or a directory holding it as devcapsule-{capsule_platform}." + ) + return _resolved(str(candidate)) + + +def _require_same_build(delivered: Path, delivered_build: BuildInfo, launcher_build: BuildInfo) -> None: + revisions = (delivered_build.source_revision, launcher_build.source_revision) + revisions_differ = _UNKNOWN_REVISION not in revisions and revisions[0] != revisions[1] + if delivered_build.version != launcher_build.version or revisions_differ: + raise CliError( + f"{delivered} is DevCapsule {delivered_build.version} from {delivered_build.source_revision}, " + f"but this launcher is {launcher_build.version} from {launcher_build.source_revision}. The runtime " + "a launch delivers runs the plan this launcher writes, so it must be the same build: rebuild " + f"it from the launcher's commit and point {RUNTIME_PEX_ENV} at it." + ) + + +def _describe(runs_on: Platform | HostOs | None) -> str: + if runs_on is HostOs.DARWIN: + return "a macOS executable" + if runs_on is HostOs.LINUX: + return "a Linux executable of an architecture no capsule platform names" + if runs_on is None: + return "a plain PEX built for another machine" + return f"a {runs_on} executable" + + +def _resolved(selected: str) -> Path: + return Path(selected).expanduser().resolve() diff --git a/devcapsule-src/devcapsule/version_sets.py b/devcapsule-src/devcapsule/version_sets.py index 883a559f..4d1ae252 100644 --- a/devcapsule-src/devcapsule/version_sets.py +++ b/devcapsule-src/devcapsule/version_sets.py @@ -23,7 +23,7 @@ from devcapsule.configuration.file_formats import canonical_digest, render_toml, selected_version_lock from devcapsule.configuration.model import Configuration from devcapsule.configuration.capabilities import CapabilityPolicy, LocalCapabilities -from devcapsule.configuration.capability_selection import usable_lock +from devcapsule.configuration.capability_selection import matrix_for, usable_lock from devcapsule.configuration.trust import HomeTrust from devcapsule.configuration.storage import ( load_settings, @@ -33,7 +33,6 @@ from devcapsule.environment_realization import RealizedEnvironment, realize_environment from devcapsule.materialization import ArtifactSpec, acquire_artifact, cache_root, parse_locked_environment, sha256_file from devcapsule.platforms import Platform, XdgHomes -from devcapsule.resolution_matrix import MATRICES from devcapsule import runtime_configuration from devcapsule.build_info import current_build_info from devcapsule.component_status import CompatibilityLookup @@ -137,7 +136,7 @@ def _validation(lock: Mapping[str, Any], checkout: Mapping[str, Any]) -> tuple[t missing = tuple(f"{name} {metadata.get('version')} on local base {local_base}" for name, metadata in lock["components"].items() if isinstance(metadata, dict)) return (), missing - return MATRICES[Platform.current()].validation_evidence(lock) + return matrix_for(lock).validation_evidence(lock) INSPECTION_SCHEMA_VERSION = 1 @@ -410,7 +409,7 @@ def preview(start: Path, component: str, version: str, *, report: Callable[[str] if channel is None: raise CliError(f"{component}: {definition.channel_omission_reason() or 'no distribution channel declared'}") selection = channel.select(version, str(workspace.lock["platform"])) - matrix = MATRICES[Platform.current()] + matrix = matrix_for(workspace.lock) if workspace.lock["platform"] not in selection.platforms: raise CliError("Candidate does not support the selected platform.") if selection.base_families and matrix.base_family(workspace.lock) not in selection.base_families: @@ -634,6 +633,9 @@ def _known(workspace: Workspace) -> list[tuple[str, dict[str, Any], dict[str, An raise CliError(f"Cannot interpret known-good record {path}: {exc}. Inspect the record; current selection is unchanged.") from exc if set_id(lock, document.get("base", {}).get("image-id")) != path.stem: raise CliError(f"Known-good record was modified: {path}. Inspect it before recovery.") + # A checkout may run as two platforms; only this platform's sets are its history. + if lock.get("platform") != workspace.lock.get("platform"): + continue result.append((path.stem, lock, document)) return sorted(result, key=lambda entry: entry[2]["last-success"], reverse=True) @@ -673,7 +675,8 @@ def rollback(start: Path, identity: str | None = None, *, reacquire: bool = Fals descriptor = formation_descriptor(platform=parsed.platform, base_identity=details.identity, artifact=parsed.artifact, ancillary_artifacts=parsed.ancillary_artifacts, recipe_id=parsed.recipe_id, recipe_version=parsed.recipe_version, - component_id=parsed.component_id, runtime_sha256=sha256_file(runtime_artifact())) + component_id=parsed.component_id, + runtime_sha256=sha256_file(runtime_artifact(Platform.parse(parsed.platform)))) if missing: name = canonical_image_name(descriptor, parsed.component_id) verify_materialized_image(required_local_image(name), descriptor=descriptor, canonical_name=name) diff --git a/devcapsule-src/noxfile.py b/devcapsule-src/noxfile.py index 743e2d4d..72849429 100644 --- a/devcapsule-src/noxfile.py +++ b/devcapsule-src/noxfile.py @@ -457,6 +457,7 @@ def ide_smoke(session: nox.Session) -> None: parser.add_argument("--max-actions", type=int, default=30) parser.add_argument("--agent-timeout", type=int, default=900) parser.add_argument("--surface", action="append", default=[], help="limit to a surface name; repeatable") + parser.add_argument("--platform", help="create each project for this capsule platform, such as linux-arm64") parser.add_argument("--required-ide", action="store_true", help="instead of the surfaces: a project requiring PyCharm launches Codium once with " "'run --ide codium --force', then PyCharm again plainly (work order of 2026-10-28)") @@ -472,6 +473,8 @@ def ide_smoke(session: nox.Session) -> None: session.error("--scenario cannot be combined with --agent: the scenario leaves the IDE in its final state") if len(options.surface) != 1: session.error("--scenario requires exactly one --surface, the scenario's own") + if options.platform: + session.env["DEVCAPSULE_SMOKE_PLATFORM"] = options.platform if options.relaunch: if not options.agent or options.surface not in (["intellij"], ["rider"]): session.error("--relaunch requires --agent --surface intellij or rider") diff --git a/devcapsule-src/pyproject.toml b/devcapsule-src/pyproject.toml index e468bdeb..ec0dd382 100644 --- a/devcapsule-src/pyproject.toml +++ b/devcapsule-src/pyproject.toml @@ -95,6 +95,7 @@ markers = [ "ide_smoke: launches a real IDE session per surface with the executable under test; opt-in through nox -s ide-smoke", "flaky: executes for visibility but has a temporarily non-blocking result", "legacy: predates the unit-test conventions of WORKFLOW-LOCAL.md; the file only loses tests, see scripts/validate_unittests_follow_conventions.py", + "own_host_patches: the test sets up its host itself; the suite's amd64 Linux host defaults stay off", ] [tool.coverage.run] diff --git a/devcapsule-src/tests/configuration/test_contract.py b/devcapsule-src/tests/configuration/test_contract.py index 74d1b78c..e1eeb182 100644 --- a/devcapsule-src/tests/configuration/test_contract.py +++ b/devcapsule-src/tests/configuration/test_contract.py @@ -44,6 +44,7 @@ replace_answers, ) from devcapsule.launch.pycharm import DockerMode, PycharmRunOptions, build_run_config +from devcapsule.platforms import Platform from tests.test_pycharm import base_env pytestmark = pytest.mark.legacy @@ -299,7 +300,7 @@ def test_launcher_environment_cannot_override_explicit_sudo_denial(checkout, tmp # ProjectRunCommand passes enable_sudo=False for the persisted denial. # Check the NEXT boundary, rather than replacing it with a launch mock. config = build_run_config( - PycharmRunOptions(project=project, docker_mode=DockerMode.none, enable_sudo=False), env, + PycharmRunOptions(project=project, capsule_platform=Platform.LINUX_AMD64, docker_mode=DockerMode.none, enable_sudo=False), env, ) assert config.enable_sudo is False diff --git a/devcapsule-src/tests/configuration/test_invariants.py b/devcapsule-src/tests/configuration/test_invariants.py index b598cc45..923859f2 100644 --- a/devcapsule-src/tests/configuration/test_invariants.py +++ b/devcapsule-src/tests/configuration/test_invariants.py @@ -43,6 +43,7 @@ initialize_project, ) from devcapsule.launch.pycharm import DockerMode, PycharmRunOptions, build_run_config +from devcapsule.platforms import Platform from tests.test_pycharm import base_env from tests.test_upgrade_recovery import checkout, invoke, install_external_fakes @@ -232,7 +233,7 @@ def test_project_launch_cannot_inherit_unbound_paths_or_credentials(checkout, tm 'DOCKER_MODE': 'invalid', }) config = build_run_config(PycharmRunOptions( - project=project, inherit_legacy_configuration=False, + project=project, capsule_platform=Platform.LINUX_AMD64, inherit_legacy_configuration=False, docker_mode=DockerMode.none, enable_sudo=False, network_mode='bridge', ), env) assert config.persistent_home != unbound and config.ide_config != unbound and config.plugins != unbound @@ -247,7 +248,7 @@ def test_explicit_sudo_decision_overrides_legacy_environment(value, tmp_path): project.mkdir() env = base_env(tmp_path) env['PYCHARM_ENABLE_SUDO'] = '1' - config = build_run_config(PycharmRunOptions(project=project, docker_mode=DockerMode.none, + config = build_run_config(PycharmRunOptions(project=project, capsule_platform=Platform.LINUX_AMD64, docker_mode=DockerMode.none, enable_sudo=value), env) assert config.enable_sudo is (True if value is None else value) diff --git a/devcapsule-src/tests/conftest.py b/devcapsule-src/tests/conftest.py index 5c50403e..3fc4067b 100644 --- a/devcapsule-src/tests/conftest.py +++ b/devcapsule-src/tests/conftest.py @@ -2,10 +2,44 @@ import os from pathlib import Path +import shutil +import sys +import tempfile import pytest +def pytest_configure(config: pytest.Config) -> None: + """Give a macOS run a short temporary root. + + macOS limits a Unix socket path to 104 bytes, and its per-user temporary + directory under /var/folders leaves too little of that for the sockets + the tests bind beneath tmp_path. An explicit --basetemp still wins. + """ + + if sys.platform == "darwin" and not config.option.basetemp: + config.option.basetemp = tempfile.mkdtemp(prefix="dcpt-", dir="/tmp") + config.stash[_SHORT_BASETEMP] = Path(config.option.basetemp) + + +_SHORT_BASETEMP = pytest.StashKey[Path]() + + +def pytest_sessionfinish(session: pytest.Session, exitstatus: int) -> None: + """Remove the short temporary root this session created, once every test passed. + + pytest prunes its own default roots, but not one it was given, so a root + per session would otherwise outlive every run and fill the disk; a failed + session keeps its root, and its path, for inspection. + """ + + created = session.config.stash.get(_SHORT_BASETEMP, None) + if created is not None and exitstatus == 0: + shutil.rmtree(created, ignore_errors=True) + elif created is not None: + print(f"\nTemporary files of this failed session: {created}") + + @pytest.fixture(autouse=True) def host_launch_by_default(monkeypatch: pytest.MonkeyPatch) -> None: """Make launcher tests behave as if they run on the host. @@ -24,6 +58,28 @@ def host_launch_by_default(monkeypatch: pytest.MonkeyPatch) -> None: ) +@pytest.fixture(autouse=True) +def amd64_linux_host_by_default(request: pytest.FixtureRequest, monkeypatch: pytest.MonkeyPatch) -> None: + """Make host-dependent tests run on an amd64 Linux host with an amd64 Docker daemon. + + The launcher takes macOS paths for the host-browser bridge, host + networking, the Docker socket's group and the sudo policy, and an Apple + Silicon host runs linux-arm64 capsules natively. Tests of the CI's host + would otherwise depend on where they run; tests of another host patch it + themselves and carry ``own_host_patches``. Docker-marked tests keep the + real host. + """ + + if any(request.node.get_closest_marker(marker) for marker in (*DOCKER_MARKERS, "own_host_patches")): + return + from devcapsule.platforms import HostOs + + monkeypatch.setattr(HostOs, "current", classmethod(lambda cls: HostOs.LINUX)) + monkeypatch.setattr("devcapsule.platforms.host_architecture", lambda: "amd64") + for module in ("environment_realization", "images.build", "launch.pycharm._launcher"): + monkeypatch.setattr(f"devcapsule.{module}.daemon_architecture", lambda: "amd64") + + DOCKER_MARKERS = ("e2e", "base_build_e2e", "recursive_e2e", "contributor_e2e", "ide_smoke") diff --git a/devcapsule-src/tests/e2e/ide_session.py b/devcapsule-src/tests/e2e/ide_session.py index fa461d38..9e2df352 100644 --- a/devcapsule-src/tests/e2e/ide_session.py +++ b/devcapsule-src/tests/e2e/ide_session.py @@ -60,9 +60,17 @@ class IdeSurface: ) CREATOR = "e2e@devcapsule.test" -DESKTOP_READY = re.compile(r"Contained display is ready; open it in a browser: (http://127\.0\.0\.1:\d+/vnc\.html\?\S+)") +DESKTOP_READY = re.compile( + r"Contained display is ready; (?:open it in a browser: (http://127\.0\.0\.1:\d+/vnc\.html\?\S+)" + r"|opened in your browser\.|opened through the host-browser bridge\.)" +) """The launcher's readiness line. It also announces the URL earlier, before the -bridge listens; only the ready line means the page answers.""" +bridge listens; only the ready line means the page answers. Inside a capsule +without a bridge the ready line names the URL. Elsewhere the launcher opens it, +which the smoke's ``BROWSER=true`` makes a no-op, and the URL is the earlier +announcement's.""" +DESKTOP_ANNOUNCED = re.compile(r"Contained display: (http://127\.0\.0\.1:\d+/vnc\.html\?\S+)") +"""The launcher's first mention of the desktop URL, printed before it answers.""" CONSOLE_READY = re.compile(r"Web console is ready; open it in a browser: (http://127\.0\.0\.1:\d+/\?token=\S+)") """The launcher's console readiness line; printed once the console answered its port.""" LAUNCH_TIMEOUT = 600.0 @@ -171,8 +179,11 @@ def initialize_project(executable: Path, surface: IdeSurface, workspace: Path, e """ environment = dict(os.environ, BROWSER="true") # webbrowser runs `true URL`: no tab opens built_base = os.environ.get("DEVCAPSULE_E2E_BASE_IMAGE") + # DEVCAPSULE_SMOKE_PLATFORM creates the project for that capsule platform, + # such as linux-arm64 on Apple Silicon, whose new projects are linux-amd64. + platform = os.environ.get("DEVCAPSULE_SMOKE_PLATFORM") init = subprocess.run( - [str(executable), "project", "init", + [str(executable), "project", "init", *(["--platform", platform] if platform else []), *(flag for need in surface.needs for flag in ("--ide" if need.endswith("-ide") else "--require", need)), "--creator", CREATOR, "--slug", slug, "--authorize", "base-image", "default", @@ -241,7 +252,7 @@ def launched_ide(executable: Path, surface: IdeSurface, workspace: Path, def wait_for_desktop_url(launcher: subprocess.Popen[bytes], launcher_log: Path) -> str: """Return the desktop URL once the launcher prints it; fail if it exits first.""" - return _wait_for_ready_url(launcher, launcher_log, DESKTOP_READY, "desktop") + return _wait_for_ready_url(launcher, launcher_log, DESKTOP_READY, "desktop", announced=DESKTOP_ANNOUNCED) def wait_for_console_url(launcher: subprocess.Popen[bytes], launcher_log: Path) -> str: @@ -250,13 +261,18 @@ def wait_for_console_url(launcher: subprocess.Popen[bytes], launcher_log: Path) def _wait_for_ready_url(launcher: subprocess.Popen[bytes], launcher_log: Path, pattern: re.Pattern[str], - what: str, timeout: float = LAUNCH_TIMEOUT) -> str: + what: str, timeout: float = LAUNCH_TIMEOUT, + announced: re.Pattern[str] | None = None) -> str: deadline = time.monotonic() + timeout while True: text = launcher_log.read_text(encoding="utf-8", errors="replace") found = pattern.search(text) - if found: + if found and found.group(1): return found.group(1) + if found and announced and (earlier := announced.search(text, 0, found.start())): + return earlier.group(1) + if found: + raise AssertionError(f"the {what} became ready without a URL announced before it:\n{text[-4000:]}") if launcher.poll() is not None: raise AssertionError(f"launcher exited {launcher.returncode} before publishing a {what} URL:\n{text}") if time.monotonic() > deadline: diff --git a/devcapsule-src/tests/e2e/test_ide_comes_alive.py b/devcapsule-src/tests/e2e/test_ide_comes_alive.py index 01a65a59..989c26fe 100644 --- a/devcapsule-src/tests/e2e/test_ide_comes_alive.py +++ b/devcapsule-src/tests/e2e/test_ide_comes_alive.py @@ -18,6 +18,8 @@ import pytest +from devcapsule.platforms import HostOs +from devcapsule.runtime_artifact import RUNTIME_PEX_ENV, executable_platform from tests.e2e.ide_session import SURFACES, IdeSurface, capture_desktop, command, desktop_page_answers, ide_session, launched_ide, stop_session, wait_for_console_url, wait_for_ide_window pytestmark = pytest.mark.legacy @@ -79,7 +81,8 @@ def test_ide_comes_alive(surface: IdeSurface, built_pex: Path, tmp_path: Path, e identity = command("docker", "inspect", "--format", '{{json .Id}} {{json .Image}}', session.container) (evidence / "container-identity.txt").write_text(identity.stdout, encoding="utf-8") child_digest = command("docker", "exec", session.container, "sha256sum", "/opt/devcapsule/bin/devcapsule.pex").stdout.split()[0] - assert child_digest == hashlib.sha256(built_pex.read_bytes()).hexdigest(), "Child runtime differs from selected executable" + assert child_digest == hashlib.sha256(delivered_runtime(built_pex, session.workspace).read_bytes()).hexdigest(), \ + "Child runtime differs from the runtime the selected executable delivers" if surface.name == "eclipse": package = command("docker", "exec", "--user", f"{os.getuid()}:{os.getgid()}", session.container, "python3", "-c", @@ -154,6 +157,22 @@ def test_ide_comes_alive(surface: IdeSurface, built_pex: Path, tmp_path: Path, e }, indent=2) + "\n", encoding="utf-8") +def delivered_runtime(executable: Path, workspace: Path) -> Path: + """The executable a launch from ``executable`` should deliver into the capsule. + + Derived independently of the launcher's choice: itself when it is a Linux + executable; for a macOS launcher, the Linux runtime its environment names, + from a directory the one for the platform of the one lock the smoke's init wrote. + """ + if executable_platform(executable) is not HostOs.DARWIN: + return executable + named = Path(os.environ[RUNTIME_PEX_ENV]).expanduser().resolve() + if named.is_dir(): + (lock,) = (workspace / ".devcapsule").glob("devcapsule.*.lock") + return named / f"devcapsule-{lock.name.split('.')[1]}" + return named + + def editor_font_size(container: str) -> float: probe = command("docker", "exec", container, "python3", "-c", "import xml.etree.ElementTree as E; " diff --git a/devcapsule-src/tests/test_base_image.py b/devcapsule-src/tests/test_base_image.py index c17f548f..238311c9 100644 --- a/devcapsule-src/tests/test_base_image.py +++ b/devcapsule-src/tests/test_base_image.py @@ -20,6 +20,7 @@ from devcapsule.compat import CliError from devcapsule.images.build import render_build_context from devcapsule.images.tooling import MAVEN_VERSION, TEMURIN_VERSION +from devcapsule.platforms import Platform pytestmark = pytest.mark.legacy @@ -177,12 +178,13 @@ def test_base_image_build_forwards_host_network_to_buildx(tmp_path: Path) -> Non builder = Mock() source_verifier = Mock() - build_base_image(options, builder=builder, network="host", source_verifier=source_verifier) + build_base_image(options, builder=builder, platform=Platform.LINUX_AMD64, network="host", + source_verifier=source_verifier) source_verifier.assert_called_once() builder.build.assert_called_once() assert builder.build.call_args.args[0].image == "test-base:latest" - assert builder.build.call_args.kwargs == {"network": "host"} + assert builder.build.call_args.kwargs == {"network": "host", "platform": Platform.LINUX_AMD64} def test_base_image_rejects_unreachable_revision_before_buildx(tmp_path: Path) -> None: @@ -192,7 +194,7 @@ def test_base_image_rejects_unreachable_revision_before_buildx(tmp_path: Path) - source_verifier = Mock(side_effect=CliError("source revision is not publicly reachable")) with pytest.raises(CliError, match="not publicly reachable"): - build_base_image(options, builder=builder, source_verifier=source_verifier) + build_base_image(options, builder=builder, platform=Platform.LINUX_AMD64, source_verifier=source_verifier) builder.build.assert_not_called() @@ -239,7 +241,8 @@ def test_console_base_rejects_missing_source_before_starting_the_builder(tmp_pat pex = pex_fixture(tmp_path / "devcapsule.pex", revision="unknown", public=False) builder = Mock() with pytest.raises(CliError, match="--allow-local-source cannot supply the console source"): - build_base_image(BaseImageBuildOptions(pex, allow_local_source=True), builder=builder) + build_base_image(BaseImageBuildOptions(pex, allow_local_source=True), builder=builder, + platform=Platform.LINUX_AMD64) builder.build.assert_not_called() @@ -251,6 +254,7 @@ def test_base_image_local_source_escape_hatch_skips_public_verification(tmp_path build_base_image( BaseImageBuildOptions(pex, allow_local_source=True), builder=builder, + platform=Platform.LINUX_AMD64, source_verifier=source_verifier, ) diff --git a/devcapsule-src/tests/test_cli.py b/devcapsule-src/tests/test_cli.py index 8da2fa6b..e342f180 100644 --- a/devcapsule-src/tests/test_cli.py +++ b/devcapsule-src/tests/test_cli.py @@ -16,6 +16,7 @@ from devcapsule.launch.pycharm._image_build import PycharmImageBuildOptions from devcapsule.environment_realization import RealizedEnvironment from devcapsule.images.metadata import LocalImageRecord +from devcapsule.platforms import Platform from devcapsule.materialization import ImageDetails, parse_locked_environment from devcapsule.configuration.file_formats import ( canonical_digest, @@ -370,7 +371,7 @@ def test_images_build_base_maps_cli_options(tmp_path: Path, capsys) -> None: assert options.source_revision == "revision-1" assert options.allow_local_source is False assert options.recipe == "ubuntu-24.04" - assert build.call_args.kwargs == {"network": "host"} + assert build.call_args.kwargs == {"network": "host", "platform": Platform.LINUX_AMD64} output = capsys.readouterr().out assert "Image ID: sha256:abc123" in output assert "Runtime: supplied by the launcher" in output diff --git a/devcapsule-src/tests/test_environment_realization.py b/devcapsule-src/tests/test_environment_realization.py index c5cec44b..19494ae6 100644 --- a/devcapsule-src/tests/test_environment_realization.py +++ b/devcapsule-src/tests/test_environment_realization.py @@ -11,6 +11,7 @@ CLAUDE_CODE_TERMS_URL, ) from devcapsule.environment_realization import ensure_local_image, realize_environment +from devcapsule.platforms import Platform from devcapsule.materialization import ImageDetails from devcapsule.runtime_command import RuntimeCommand from devcapsule.configuration.storage import ( @@ -276,15 +277,29 @@ def test_ensure_local_image_pulls_only_when_missing(already_local: bool) -> None return_value=details, ) as require, ): - assert ensure_local_image(BASE_REFERENCE) == details + assert ensure_local_image(BASE_REFERENCE, Platform.LINUX_AMD64) == details if already_local: pull.assert_not_called() else: - pull.assert_called_once_with(BASE_REFERENCE) + pull.assert_called_once_with(BASE_REFERENCE, platform=None) require.assert_called_once_with(BASE_REFERENCE) +@pytest.mark.parametrize("daemon,expected", [("arm64", "linux/amd64"), ("amd64", None)]) +def test_a_daemon_of_another_architecture_pulls_the_lock_platform(daemon: str, expected: str | None) -> None: + # Docker Desktop on Apple Silicon is an arm64 daemon; a linux-amd64 lock's + # base must not resolve to its arm64 variant. + with ( + patch("devcapsule.environment_realization.daemon_architecture", return_value=daemon), + patch("devcapsule.environment_realization.docker.image.exists", return_value=False), + patch("devcapsule.environment_realization.docker.image.pull") as pull, + patch("devcapsule.environment_realization.required_local_image", return_value=base_image()), + ): + ensure_local_image(BASE_REFERENCE, Platform.LINUX_AMD64) + pull.assert_called_once_with(BASE_REFERENCE, platform=expected) + + def test_realize_environment_allows_explicit_managed_base_override(tmp_path: Path) -> None: selected = resolved_project(tmp_path, authorized=False) local_reference = "local/devcapsule-base:test" @@ -398,7 +413,7 @@ def test_realizer_uses_resolved_shipped_command_choice(tmp_path, monkeypatch, co selected.resolution["runtime"]["devcapsule-command"] = command materialize = Mock(return_value=(CANONICAL_IMAGE, True)) monkeypatch.setattr("devcapsule.environment_realization.ensure_materialized_surface", materialize) - monkeypatch.setattr("devcapsule.environment_realization.runtime_artifact", lambda: tmp_path / "runtime.pex") + monkeypatch.setattr("devcapsule.environment_realization.runtime_artifact", lambda _platform: tmp_path / "runtime.pex") realize_environment(selected, root=tmp_path / "cache", obtain_image=lambda _ref: base_image(), require_image=lambda _ref: completed_image(), build=Mock(), inspect_image=Mock()) assert materialize.call_args.kwargs["runtime_command"] is RuntimeCommand(command or "devcapsule") diff --git a/devcapsule-src/tests/test_host_open.py b/devcapsule-src/tests/test_host_open.py index 7d623d59..31707e76 100644 --- a/devcapsule-src/tests/test_host_open.py +++ b/devcapsule-src/tests/test_host_open.py @@ -5,12 +5,14 @@ from pathlib import Path import socket import subprocess +import sys from types import SimpleNamespace from unittest.mock import Mock, patch import pytest from devcapsule import cli +from devcapsule.platforms import HostOs from devcapsule.host_open import ( HOST_OPEN_SOCKET_ENV, MAX_URL_BYTES, @@ -24,6 +26,7 @@ pytestmark = pytest.mark.legacy +@pytest.mark.skipif(sys.platform != "linux", reason="the broker reads SO_PEERCRED, which only Linux has") def test_broker_forwards_one_exact_url_without_shell_parsing(tmp_path: Path) -> None: path = tmp_path / "host-open.sock" run = Mock(return_value=SimpleNamespace(returncode=0)) @@ -89,6 +92,7 @@ def test_broker_rejects_malformed_protocol_without_invoking_opener(tmp_path: Pat run.assert_not_called() +@pytest.mark.skipif(sys.platform != "linux", reason="the broker reads SO_PEERCRED, which only Linux has") def test_broker_applies_a_bounded_request_rate(tmp_path: Path) -> None: path = tmp_path / "host-open.sock" run = Mock(return_value=SimpleNamespace(returncode=0)) @@ -120,6 +124,7 @@ def test_broker_rejects_a_different_peer_uid(tmp_path: Path) -> None: run.assert_not_called() +@pytest.mark.skipif(sys.platform != "linux", reason="the broker reads SO_PEERCRED, which only Linux has") def test_broker_reports_host_opener_timeout(tmp_path: Path) -> None: path = tmp_path / "host-open.sock" run = Mock(side_effect=subprocess.TimeoutExpired(["xdg-open"], 0.1)) @@ -132,6 +137,7 @@ def test_broker_reports_host_opener_timeout(tmp_path: Path) -> None: open_host_url("https://example.test", socket_path=path) +@pytest.mark.skipif(sys.platform != "linux", reason="the broker reads SO_PEERCRED, which only Linux has") def test_client_reports_missing_and_failed_brokers_without_exposing_url(tmp_path: Path) -> None: url = "https://example.test/path?secret=do-not-report" with pytest.raises(HostOpenError, match="absent") as missing: @@ -169,12 +175,14 @@ def test_bridge_reuses_only_an_explicit_inherited_socket(tmp_path: Path) -> None assert disabled is None +@pytest.mark.skipif(sys.platform != "linux", reason="the broker reads SO_PEERCRED, which only Linux has") def test_physical_host_bridge_owns_and_cleans_broker(tmp_path: Path) -> None: run = Mock(return_value=SimpleNamespace(returncode=0)) env = {"XDG_RUNTIME_DIR": str(tmp_path), "DISPLAY": ":1"} with ( patch("devcapsule.host_open.in_container", return_value=False), + patch("devcapsule.host_open.HostOs.current", return_value=HostOs.LINUX), patch("devcapsule.host_open.subprocess.run", run), host_open_bridge(env, enabled=True, opener=("test-opener",)) as path, ): @@ -186,6 +194,20 @@ def test_physical_host_bridge_owns_and_cleans_broker(tmp_path: Path) -> None: assert not root.exists() +def test_macos_host_bridge_is_unavailable_with_a_warning(tmp_path: Path, capsys) -> None: + # Docker Desktop refuses a host Unix socket as a bind source, so the + # optional integration degrades instead of failing the launch. + with ( + patch("devcapsule.host_open.in_container", return_value=False), + patch("devcapsule.host_open.HostOs.current", return_value=HostOs.DARWIN), + host_open_bridge({"XDG_RUNTIME_DIR": str(tmp_path)}, enabled=True) as path, + ): + assert path is None + assert "host-browser is unavailable on macOS" in capsys.readouterr().err + assert list(tmp_path.iterdir()) == [] + + +@pytest.mark.skipif(sys.platform != "linux", reason="the broker reads SO_PEERCRED, which only Linux has") def test_hidden_host_open_command_uses_authorized_environment(tmp_path: Path, capsys) -> None: assert "host-open" not in cli.command_names() assert "host-open" in cli.command_names(include_hidden=True) diff --git a/devcapsule-src/tests/test_image_build.py b/devcapsule-src/tests/test_image_build.py index f52260a3..51d1466b 100644 --- a/devcapsule-src/tests/test_image_build.py +++ b/devcapsule-src/tests/test_image_build.py @@ -12,6 +12,7 @@ parse_pycharm_build_options, ) from devcapsule.images.build import BuildxImageBuilder, ImageBuildSpec, normalize_pycharm_source, render_build_context +from devcapsule.platforms import Platform pytestmark = pytest.mark.legacy @@ -137,9 +138,9 @@ def test_buildx_builder_reuses_one_context_root_and_attaches_named_contexts(tmp_ spec = ImageBuildSpec(image="result:test", base_image="sha256:base", components=(DirectoryComponent(tree, "/opt/tree"),)) with patch("devcapsule.images.build.docker.build") as build: - BuildxImageBuilder(context_root=context_root).build(spec, network="none") + BuildxImageBuilder(context_root=context_root).build(spec, platform=Platform.LINUX_AMD64, network="none") (context_root / "stale-from-last-build").write_text("x") - BuildxImageBuilder(context_root=context_root).build(spec, network="host") + BuildxImageBuilder(context_root=context_root).build(spec, platform=Platform.LINUX_AMD64, network="host") first, second = build.call_args_list assert first.args[0] == context_root and second.args[0] == context_root assert first.kwargs["build_contexts"] == {"copy-dir-0": str(tree)} @@ -152,7 +153,8 @@ def test_buildx_builder_reuses_one_context_root_and_attaches_named_contexts(tmp_ def test_buildx_builder_without_a_root_uses_a_temporary_context(tmp_path: Path) -> None: with patch("devcapsule.images.build.docker.build") as build: - BuildxImageBuilder().build(ImageBuildSpec(image="result:test", base_image="sha256:base"), network="none") + BuildxImageBuilder().build(ImageBuildSpec(image="result:test", base_image="sha256:base"), + platform=Platform.LINUX_AMD64, network="none") used_context = build.call_args.args[0] assert used_context.name.startswith("devcapsule-buildx-context-") and not used_context.exists() assert build.call_args.kwargs["build_contexts"] == {} @@ -195,9 +197,9 @@ def test_reused_context_preserves_sources_and_recovers_after_build_failure(tmp_p spec = ImageBuildSpec("result:test", "base:test", (DirectoryComponent(source, "/opt/tool"),)) with patch("devcapsule.images.build.docker.build", side_effect=OSError("build failed")): with pytest.raises(CliError, match="build failed"): - BuildxImageBuilder(context).build(spec) + BuildxImageBuilder(context).build(spec, platform=Platform.LINUX_AMD64) with patch("devcapsule.images.build.docker.build") as build: - BuildxImageBuilder(context).build(spec) + BuildxImageBuilder(context).build(spec, platform=Platform.LINUX_AMD64) assert build.call_count == 1 assert (source / "payload").read_text() == "keep" assert sorted(path.name for path in context.iterdir()) == ["Dockerfile"] diff --git a/devcapsule-src/tests/test_linux_arm64_matrix.py b/devcapsule-src/tests/test_linux_arm64_matrix.py new file mode 100644 index 00000000..8f342328 --- /dev/null +++ b/devcapsule-src/tests/test_linux_arm64_matrix.py @@ -0,0 +1,127 @@ +"""The linux-arm64 matrix pins the linux-amd64 versions with arm64 artifacts only.""" + +from __future__ import annotations + +import json +import re +from typing import Any + +import pytest + +from devcapsule.components.linux_arm64_pins import COMPONENT_LOCK_TABLES +from devcapsule.configuration.capability_selection import pins_for_platform +from devcapsule.configuration.file_formats import ProjectConfigurationError +from devcapsule.platforms import Platform +from devcapsule.resolution_matrix import MATRICES + +AMD64 = MATRICES[Platform.LINUX_AMD64] +ARM64 = MATRICES[Platform.LINUX_ARM64] +AMD64_MARKERS = re.compile(r"x86[_-]64|[_-]x64|amd64|linux64/|cft/[^/]+/linux64|ffmpeg-linux\.zip") + + +def _artifacts(value: Any) -> list[dict[str, Any]]: + """Every table at any depth that names a download by ``url``.""" + if isinstance(value, dict): + own = [value] if "url" in value else [] + return own + [found for item in value.values() for found in _artifacts(item)] + if isinstance(value, (list, tuple)): + return [found for item in value for found in _artifacts(item)] + return [] + + +def test_every_arm64_pin_is_a_version_amd64_pins_and_the_matrix_serves() -> None: + # The converse, an amd64 version without an arm64 pin, fails at import. + for (component, version), table in COMPONENT_LOCK_TABLES.items(): + assert AMD64.component_pin(component, version) is not None, (component, version) + assert ARM64.component_pin(component, version) == table, (component, version) + + +def test_no_arm64_pin_names_an_amd64_artifact() -> None: + for key, table in COMPONENT_LOCK_TABLES.items(): + for artifact in _artifacts(table): + assert not AMD64_MARKERS.search(str(artifact["url"])), (key, artifact["url"]) + + +def test_every_arm64_artifact_is_hash_pinned() -> None: + for key, table in COMPONENT_LOCK_TABLES.items(): + for artifact in _artifacts(table): + assert re.fullmatch(r"[0-9a-f]{64}", str(artifact.get("sha256"))), (key, artifact) + + +def test_arm64_artifacts_are_not_amd64_bytes() -> None: + # Architecture-independent files are shared: Codex's JavaScript package, + # pure-Python wheels, and Ubuntu _all packages. Everything else differs. + shared = {artifact["sha256"] for (component, version) in COMPONENT_LOCK_TABLES + for artifact in _artifacts(AMD64.component_pin(component, version))} + for (component, version), table in COMPONENT_LOCK_TABLES.items(): + for artifact in _artifacts(table): + url = str(artifact["url"]) + independent = url.endswith(("_all.deb", "-none-any.whl")) or url.endswith(f"codex-{version}.tgz") + if not independent: + assert artifact["sha256"] not in shared, (component, version, url) + + +def test_platform_fields_and_artifact_keys_name_arm64() -> None: + for (component, _version), table in COMPONENT_LOCK_TABLES.items(): + if "platform" in table: + assert table["platform"] == "linux-arm64", component + if component in {"codex", "claude-code", "antigravity-cli"}: + assert list(table["artifacts"]) == ["linux-arm64"], component + assert "linux-amd64" not in json.dumps(list(COMPONENT_LOCK_TABLES.values())) + + +def test_eclipse_native_closure_is_arm64_or_architecture_independent() -> None: + packages = ARM64.component_pin("eclipse", "2026-09-R")["native-packages"] + # The same packages, plus what arm64's own dependencies add. + assert set(packages) - set(AMD64.component_pin("eclipse", "2026-09-R")["native-packages"]) == {"libunwind8"} + for name, entry in packages.items(): + # Architecture-independent packages keep the amd64 closure's bytes. + if entry["url"].endswith("_all.deb"): + continue + assert entry["url"].startswith("https://ports.ubuntu.com/ubuntu-ports/pool/"), name + assert entry["url"].endswith("_arm64.deb"), name + + +def test_arm64_base_is_its_own_unpublished_recipe_11_pin() -> None: + (base,) = ARM64.base_images() + assert base.contract.recipe == 11 + assert base.reference.startswith("docker.io/mycodespaceai/devcapsule-base@sha256:") + assert base.reference not in {image.reference for image in AMD64.base_images()} + assert ARM64.base_unpublished(base.reference) + assert not any(AMD64.base_unpublished(image.reference) for image in AMD64.base_images()) + + +@pytest.mark.parametrize("ide", ["frontend-ide", "vscode-ide", "eclipse-ide", "python-ide", "java-ide"]) +def test_each_ide_smoked_natively_on_arm64_resolves_verified(ide: str) -> None: + assert ARM64.resolve([ide]).unverified == () + + +def _personal_pins(platform: str, component: str, matrix: Any) -> dict[str, Any]: + pin = matrix.component_pins()[component] + return {"platform": platform, "components": {component: dict(pin), "interactive-surface": component}} + + +def test_personal_pins_take_the_artifacts_of_the_platform_they_run_on() -> None: + # A checkout chose codium while the project ran as linux-amd64; the + # project then gained an arm64 lock. Its pins must deliver arm64 bytes. + amd64 = _personal_pins("linux-amd64", "codium", AMD64) + on_arm64 = pins_for_platform(amd64, ARM64) + assert on_arm64["platform"] == "linux-arm64" + assert on_arm64["components"]["codium"] == dict(ARM64.component_pins()["codium"]) + assert on_arm64["components"]["interactive-surface"] == "codium" + assert amd64["components"]["codium"] == dict(AMD64.component_pins()["codium"]) # input unchanged + # A table that is another platform's embedded pin is that platform's, + # whatever the label says, so a mislabelled record takes arm64's bytes too. + mislabelled = dict(amd64, platform="linux-arm64") + assert pins_for_platform(mislabelled, ARM64)["components"]["codium"] == dict(ARM64.component_pins()["codium"]) + # Pins written for the platform they run on are records, returned as they stand. + recorded = {**amd64, "components": {**amd64["components"], "codium": {**amd64["components"]["codium"], "note": "kept"}}} + assert pins_for_platform(recorded, AMD64) == recorded + assert pins_for_platform(on_arm64, AMD64)["components"]["codium"] == dict(AMD64.component_pins()["codium"]) + + +def test_a_personal_version_the_other_platform_lacks_asks_for_a_new_choice() -> None: + channel = {"platform": "linux-amd64", "components": {"codex": {"version": "9.9.9", "url": "https://example.test/x64.tgz"}}} + with pytest.raises(ProjectConfigurationError, match="codex 9.9.9 is pinned for linux-amd64.*linux-arm64 has no pin"): + pins_for_platform(channel, ARM64) + assert pins_for_platform(channel, AMD64)["components"]["codex"]["version"] == "9.9.9" diff --git a/devcapsule-src/tests/test_owner_commands.py b/devcapsule-src/tests/test_owner_commands.py index 27125725..5bdea0f4 100644 --- a/devcapsule-src/tests/test_owner_commands.py +++ b/devcapsule-src/tests/test_owner_commands.py @@ -11,13 +11,14 @@ from devcapsule import cli from devcapsule.configuration import owner_commands -from devcapsule.configuration.capability_commands import paths +from devcapsule.configuration.capability_commands import paths, recover from devcapsule.configuration.file_formats import ProjectConfigurationError, render_toml from devcapsule.configuration.manifest_document import render_manifest from devcapsule.configuration.storage import load_toml from devcapsule.configuration.storage import checkout_record_paths from devcapsule.platforms import Platform from devcapsule.resolution_matrix import MATRICES +from tests.test_platforms import on_host pytestmark = pytest.mark.legacy @@ -423,3 +424,63 @@ def test_sources_survive_other_owner_edits_and_do_not_change_pending_decisions(p {"kind": "source", "name": "notes", "effective": "no toolchain declared", "source": "project: upstream, read-only", "pending": "", "command": ""}, {"kind": "source", "name": "vendor/robot.code", "effective": "java 17", "source": "project: fork of https://example.test/up", "pending": "", "command": ""}, ] + + +@pytest.fixture +def two_platform_project(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """A project on Apple Silicon with a linux-amd64 lock and a linux-arm64 lock added beside it.""" + monkeypatch.setenv("XDG_CONFIG_HOME", str(tmp_path / "config")) + monkeypatch.setenv("HOME", str(tmp_path / "home")) + on_host(monkeypatch, "Darwin", "arm64") + root = tmp_path / "project" + root.mkdir() + assert cli.main(["project", "--path", str(root), "init", "--platform", "linux-amd64", "--require", "python", + "--creator", "mailto:owner@example.test"]) == 0 + assert cli.main(["project", "--path", str(root), "init", "--platform", "linux-arm64"]) == 0 + return root + + +def _locked_components(root: Path) -> dict[str, set[str]]: + return {path.name: set(load_toml(path).get("components", {})) + for path in sorted((root / ".devcapsule").glob("devcapsule.*.lock"))} + + +@pytest.mark.own_host_patches +def test_an_owner_edit_regenerates_every_platform_lock(two_platform_project: Path, capsys) -> None: + # Edited on the Mac, the committed linux-amd64 lock follows the manifest too. + assert cli.main(["project", "--path", str(two_platform_project), "config", "require", "dotnet", "--unverified"]) == 0 + assert "the manifest and its locks are updated" in capsys.readouterr().out + assert _locked_components(two_platform_project) == { + "devcapsule.linux-amd64.lock": {"dotnet-sdk"}, "devcapsule.linux-arm64.lock": {"dotnet-sdk"}, + } + + +@pytest.mark.own_host_patches +def test_an_edit_one_lock_cannot_follow_changes_nothing(two_platform_project: Path) -> None: + # The .NET SDK has no linux-arm64 validation yet; linux-amd64 alone would accept it. + before = {path: path.read_bytes() for path in (two_platform_project / ".devcapsule").iterdir() if path.is_file()} + with pytest.raises(ProjectConfigurationError, match="linux-arm64 lock cannot follow this edit"): + owner_commands.require_tools(two_platform_project, ["dotnet"]) + assert {path: path.read_bytes() for path in before} == before + + +@pytest.mark.own_host_patches +@pytest.mark.parametrize("committed", [False, True]) +def test_an_interrupted_edit_recovers_every_lock_to_one_side(two_platform_project: Path, committed: bool) -> None: + devcapsule = two_platform_project / ".devcapsule" + manifest, locks = paths(two_platform_project)[0], sorted(devcapsule.glob("devcapsule.*.lock")) + before = {path: path.read_text() for path in (manifest, *locks)} + owner_commands.require_tools(two_platform_project, ["dotnet"], allow_unverified=True) + after = {path: path.read_text() for path in (manifest, *locks)} + assert all(before[path] != after[path] for path in locks) + journal = {"before-manifest": before[manifest], "after-manifest": after[manifest], + "before-locks": {path.name.split(".")[1]: before[path] for path in locks}, + "after-locks": {path.name.split(".")[1]: after[path] for path in locks}} + paths(two_platform_project)[2].write_text(render_toml(journal)) + if not committed: + manifest.write_text(before[manifest]) + locks[0].write_text(before[locks[0]]) # Interrupted between the two lock writes. + recover(two_platform_project) + expected = after if committed else before + assert {path: path.read_text() for path in (manifest, *locks)} == expected + assert not paths(two_platform_project)[2].exists() diff --git a/devcapsule-src/tests/test_platforms.py b/devcapsule-src/tests/test_platforms.py index d47a77c0..6011c41c 100644 --- a/devcapsule-src/tests/test_platforms.py +++ b/devcapsule-src/tests/test_platforms.py @@ -6,7 +6,8 @@ import pytest -from devcapsule.platforms import Platform, UnsupportedPlatformError, XdgHomes +from devcapsule import platforms +from devcapsule.platforms import HostOs, Platform, UnsupportedPlatformError, XdgHomes pytestmark = pytest.mark.legacy @@ -16,6 +17,7 @@ def test_platform_values_are_the_wire_format() -> None: # StrEnum: the member formats as its value wherever locks and filenames # interpolate it. assert f"devcapsule.{Platform.LINUX_AMD64}.lock" == "devcapsule.linux-amd64.lock" + assert Platform.LINUX_ARM64.value == "linux-arm64" def test_parse_accepts_every_member_and_only_members() -> None: @@ -36,6 +38,79 @@ def test_current_returns_a_member_on_a_supported_host() -> None: assert Platform.parse(current.value) is current +def on_host(monkeypatch: pytest.MonkeyPatch, system: str, machine: str) -> None: + """Make this host the one ``platform.system()`` and ``platform.machine()`` would name.""" + monkeypatch.setattr(platforms.host_platform, "system", lambda: system) + monkeypatch.setattr(platforms.host_platform, "machine", lambda: machine) + + +@pytest.mark.own_host_patches +@pytest.mark.parametrize("system,machine,runnable,created", [ + ("Linux", "x86_64", ("linux-amd64",), "linux-amd64"), + ("Linux", "aarch64", ("linux-arm64",), "linux-arm64"), + ("Darwin", "x86_64", ("linux-amd64",), "linux-amd64"), + ("Darwin", "arm64", ("linux-arm64", "linux-amd64"), "linux-arm64"), +]) +def test_each_host_runs_its_platforms_and_creates_projects_for_one(monkeypatch, system, machine, runnable, created) -> None: + # A Mac runs Linux capsules. Apple Silicon runs linux-arm64 natively and + # linux-amd64 under emulation; a new project defaults to the native one. + on_host(monkeypatch, system, machine) + assert tuple(str(platform) for platform in Platform.runnable()) == runnable + assert Platform.current() is Platform.parse(created) + + +def test_emulation_is_the_daemons() -> None: + # Docker Desktop on Apple Silicon is an arm64 daemon, whatever runs the launcher. + assert Platform.LINUX_AMD64.emulated_docker_platform("arm64") == "linux/amd64" + assert Platform.LINUX_ARM64.emulated_docker_platform("arm64") is None + assert Platform.LINUX_AMD64.docker_platform_args("arm64") == ["--platform=linux/amd64"] + assert Platform.LINUX_AMD64.docker_platform_args("amd64") == [] + assert Platform.LINUX_ARM64.docker_platform_args("arm64") == [] + + +@pytest.mark.own_host_patches +@pytest.mark.parametrize("system,machine,message", [ + ("Linux", "riscv64", "supported platforms: linux-amd64, linux-arm64"), + ("Darwin", "ppc", "supported platforms: linux-amd64, linux-arm64"), + ("Windows", "AMD64", "supported: linux, darwin"), +]) +def test_current_refuses_a_host_without_a_capsule_platform(monkeypatch, system, machine, message) -> None: + on_host(monkeypatch, system, machine) + with pytest.raises(UnsupportedPlatformError, match=message): + Platform.current() + + +def test_an_elf_machine_names_its_platform() -> None: + assert Platform.of_elf_machine(0x3E) is Platform.LINUX_AMD64 + assert Platform.of_elf_machine(0xB7) is Platform.LINUX_ARM64 + assert Platform.of_elf_machine(0xF3) is None + + +def test_docker_desktop_facts_are_the_macos_hosts() -> None: + assert HostOs.DARWIN.docker_socket_gid == 0 and HostOs.LINUX.docker_socket_gid is None + assert not HostOs.DARWIN.host_network_reachable and HostOs.LINUX.host_network_reachable + assert not HostOs.DARWIN.binds_unix_sockets and HostOs.LINUX.binds_unix_sockets + assert not HostOs.DARWIN.chown_reaches_bind_mounts and HostOs.LINUX.chown_reaches_bind_mounts + + +def test_docker_spelling_and_architecture() -> None: + assert Platform.LINUX_AMD64.docker_platform == "linux/amd64" + assert Platform.LINUX_AMD64.architecture == "amd64" + assert Platform.LINUX_ARM64.docker_platform == "linux/arm64" + assert Platform.LINUX_ARM64.architecture == "arm64" + + +@pytest.mark.own_host_patches +def test_host_os_names_linux_and_macos_only(monkeypatch) -> None: + on_host(monkeypatch, "Darwin", "arm64") + assert HostOs.current() is HostOs.DARWIN + on_host(monkeypatch, "Linux", "x86_64") + assert HostOs.current() is HostOs.LINUX + on_host(monkeypatch, "Windows", "AMD64") + with pytest.raises(UnsupportedPlatformError, match="'windows'"): + HostOs.current() + + def test_xdg_homes_honor_explicit_overrides() -> None: homes = XdgHomes.from_environment( { diff --git a/devcapsule-src/tests/test_project_platform_selection.py b/devcapsule-src/tests/test_project_platform_selection.py new file mode 100644 index 00000000..a1c2d61c --- /dev/null +++ b/devcapsule-src/tests/test_project_platform_selection.py @@ -0,0 +1,130 @@ +"""The capsule platform a project runs as: the first one this host runs that it has a lock for.""" + +from __future__ import annotations + +import io +from pathlib import Path + +import pytest + +from devcapsule import cli +from devcapsule.configuration.file_formats import ProjectConfigurationError +from devcapsule.configuration.operations import InitializeRequest, initialize_project +from devcapsule.configuration.storage import load_toml, project_platform +from devcapsule.platforms import Platform +from devcapsule.resolution_matrix import MATRICES +from tests.test_platforms import on_host + + +def _project(root: Path, *locked: str) -> Path: + (root / ".devcapsule").mkdir(parents=True) + (root / ".devcapsule" / "devcapsule.toml").write_text("") + for platform in locked: + (root / ".devcapsule" / f"devcapsule.{platform}.lock").write_text("") + return root + + +@pytest.mark.own_host_patches +@pytest.mark.parametrize("locked,expected", [ + ((), "linux-arm64"), + (("linux-amd64",), "linux-amd64"), + (("linux-arm64",), "linux-arm64"), + (("linux-amd64", "linux-arm64"), "linux-arm64"), +]) +def test_apple_silicon_runs_a_project_as_the_platform_it_is_locked_for( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, locked: tuple[str, ...], expected: str, +) -> None: + # Natively when the project has an arm64 lock, else its amd64 lock under + # emulation; a project without a lock is created for the native platform. + on_host(monkeypatch, "Darwin", "arm64") + assert project_platform(_project(tmp_path / "project", *locked)) is Platform.parse(expected) + + +@pytest.mark.own_host_patches +def test_a_host_runs_a_project_only_as_a_platform_it_runs(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + # An amd64 Linux host cannot run an arm64-only project; it keeps its own + # platform, and the missing lock is reported where the lock is read. + on_host(monkeypatch, "Linux", "x86_64") + assert project_platform(_project(tmp_path / "project", "linux-arm64")) is Platform.LINUX_AMD64 + + +@pytest.mark.own_host_patches +def test_init_platform_adds_that_lock_and_the_project_runs_as_it(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + on_host(monkeypatch, "Darwin", "arm64") + project = tmp_path / "project" + project.mkdir() + initialize_project(InitializeRequest(directory=project, creator="owner@example.test", + platform=Platform.LINUX_AMD64, interactive=False)) + initialize_project(InitializeRequest(directory=project, platform=Platform.LINUX_ARM64, interactive=False)) + locks = sorted(path.name for path in (project / ".devcapsule").glob("*.lock")) + assert locks == ["devcapsule.linux-amd64.lock", "devcapsule.linux-arm64.lock"] + arm64_lock = load_toml(project / ".devcapsule" / "devcapsule.linux-arm64.lock") + assert MATRICES[Platform.LINUX_ARM64].base_unpublished(arm64_lock["base"]["reference"]) + assert project_platform(project) is Platform.LINUX_ARM64 + + +@pytest.mark.own_host_patches +def test_init_refuses_a_lock_the_project_would_not_run_as(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + # Init completes the checkout for the lock it writes; with an arm64 lock + # the project runs as arm64 here, so an amd64 lock cannot be added here. + on_host(monkeypatch, "Darwin", "arm64") + project = _project(tmp_path / "project", "linux-arm64") + with pytest.raises(ProjectConfigurationError, match="runs as linux-arm64"): + initialize_project(InitializeRequest(directory=project, platform=Platform.LINUX_AMD64, interactive=False)) + assert not (project / ".devcapsule" / "devcapsule.linux-amd64.lock").exists() + + +def test_init_platform_refuses_a_platform_this_host_cannot_run( + tmp_path: Path, capsys: pytest.CaptureFixture[str], +) -> None: + # The suite's host runs linux-amd64 only. + project = _project(tmp_path / "project", "linux-amd64") + assert cli.main(["project", "--path", str(project), "init", "--platform", "linux-arm64"]) != 0 + assert "this host runs linux-amd64 capsules, not linux-arm64" in capsys.readouterr().err + assert cli.main(["project", "--path", str(project), "init", "--platform", "windows-amd64"]) != 0 + assert "Unsupported platform 'windows-amd64'" in capsys.readouterr().err + + +def _init_answering(project: Path, answers: str) -> str: + """Run an interactive init of ``project``, answering each question in turn; return what it asked.""" + asked = io.StringIO() + initialize_project( + InitializeRequest(directory=project, creator="owner@example.test", interactive=True), + input_stream=io.StringIO(answers), output_stream=asked, + ) + return asked.getvalue() + + +@pytest.mark.own_host_patches +@pytest.mark.parametrize("answer,expected", [("", "linux-arm64"), ("linux-amd64", "linux-amd64")]) +def test_init_asks_a_host_of_two_platforms_the_native_one_by_default( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, answer: str, expected: str, +) -> None: + on_host(monkeypatch, "Darwin", "arm64") + project = tmp_path / "project" + project.mkdir() + asked = _init_answering(project, f"{answer}\n" + "\n" * 8) + assert ("Capsule platform for this project: linux-arm64 (native; its base is built locally) " + "or linux-amd64 (emulated) [linux-arm64]:") in asked + assert [path.name for path in (project / ".devcapsule").glob("*.lock")] == [f"devcapsule.{expected}.lock"] + + +@pytest.mark.own_host_patches +def test_init_keeps_the_platform_of_a_lock_the_host_runs_without_asking( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, +) -> None: + on_host(monkeypatch, "Darwin", "arm64") + project = tmp_path / "project" + project.mkdir() + initialize_project(InitializeRequest(directory=project, creator="owner@example.test", + platform=Platform.LINUX_AMD64, interactive=False)) + assert "Capsule platform" not in _init_answering(project, "\n" * 8) + assert [path.name for path in (project / ".devcapsule").glob("*.lock")] == ["devcapsule.linux-amd64.lock"] + + +def test_a_host_of_one_platform_is_not_asked(tmp_path: Path) -> None: + project = tmp_path / "project" + project.mkdir() + assert "Capsule platform" not in _init_answering(project, "\n" * 8) + assert (project / ".devcapsule" / "devcapsule.linux-amd64.lock").is_file() + diff --git a/devcapsule-src/tests/test_pycharm.py b/devcapsule-src/tests/test_pycharm.py index 269414ce..036c22e0 100644 --- a/devcapsule-src/tests/test_pycharm.py +++ b/devcapsule-src/tests/test_pycharm.py @@ -1,5 +1,6 @@ from __future__ import annotations +from datetime import datetime, timedelta, timezone from pathlib import Path import socket import time @@ -19,6 +20,7 @@ build_run_config, ) from devcapsule.container_runtime.contract import INFRASTRUCTURE_FAILURE_EXIT_CODE +from devcapsule.platforms import HostOs, Platform from devcapsule.launch.pycharm._launcher import ( AttachedResult, display_disclosure, @@ -36,6 +38,7 @@ ensure_sudoers_policy_ownership, prepare_temp_runtime_files, print_storage_summary, + remove_abandoned_sudoers_volumes, run_pycharm, write_user_files, ) @@ -81,6 +84,7 @@ def test_external_runtime_plan_is_readable_and_mounted_read_only(tmp_path: Path) config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -145,6 +149,7 @@ def test_plan_driven_surface_mounts_declared_slots_instead_of_pycharm_paths( config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -195,6 +200,7 @@ def test_interactive_state_mounts_require_a_runtime_plan(tmp_path: Path) -> None build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -211,6 +217,7 @@ def test_interactive_state_mounts_must_belong_to_the_plan_component(tmp_path: Pa build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -234,6 +241,7 @@ def test_no_sudo_launches_keep_full_hardening_on_every_surface( config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -275,6 +283,7 @@ def test_sudo_launches_grant_no_capabilities_beyond_dockers_default( config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -311,6 +320,7 @@ def test_plan_driven_storage_summary_names_the_surface_and_its_slots( config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -340,6 +350,7 @@ def test_detached_lifecycle_changes_only_the_docker_process_flags(tmp_path: Path config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -371,6 +382,7 @@ def test_jcef_disclosure_is_printed_for_component_policy( config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -402,6 +414,7 @@ def test_host_browser_bridge_is_explicit_in_plan_mount_environment_and_summary( config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -447,6 +460,7 @@ def test_host_browser_bridge_can_be_disabled_despite_inherited_environment( config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, docker_mode=DockerMode.none, enable_host_browser=False, ), @@ -466,6 +480,7 @@ def test_physical_host_launcher_owns_browser_broker_for_docker_lifetime( with ( patch("devcapsule.host_open.in_container", return_value=False), + patch("devcapsule.host_open.HostOs.current", return_value=HostOs.LINUX), patch("devcapsule.launch.pycharm._launcher.shutil.which", return_value=None), patch("devcapsule.launch.pycharm._launcher.run_attached") as run, ): @@ -473,6 +488,7 @@ def test_physical_host_launcher_owns_browser_broker_for_docker_lifetime( assert run_pycharm( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, docker_mode=DockerMode.none, enable_host_browser=True, ), @@ -513,6 +529,7 @@ def test_selected_codex_state_and_explicit_secret_are_delivered(tmp_path: Path) config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -561,6 +578,7 @@ def test_authorized_runtime_effects_build_the_complete_docker_plan(tmp_path: Pat config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", persistent_home=home, ide_config=config_dir, @@ -589,6 +607,9 @@ def test_authorized_runtime_effects_build_the_complete_docker_plan(tmp_path: Pat assert sudoers_file.read_text(encoding="utf-8") == SUDOERS_POLICY assert sudoers_file.stat().st_mode & 0o777 == 0o440 args = build_docker_args(config, files, env) + files.sudoers_volume = "capsule-sudoers" + volume_args = build_docker_args(config, files, env) + files.sudoers_volume = None files.sudoers_file = None with pytest.raises(PycharmRunError, match="shadow and sudoers files"): build_docker_args(config, files, env) @@ -601,6 +622,9 @@ def test_authorized_runtime_effects_build_the_complete_docker_plan(tmp_path: Pat assert ["--user", "1000:1000"] == args[args.index("--user") : args.index("--user") + 2] assert "ENABLE_SUDO=1" in args assert f"type=bind,src={sudoers_file},dst={SUDOERS_POLICY_PATH},ro" in args + # On macOS the policy comes from its volume, not the Mac's file. + assert f"type=volume,src=capsule-sudoers,volume-subpath=policy,dst={SUDOERS_POLICY_PATH},ro" in volume_args + assert f"type=bind,src={sudoers_file},dst={SUDOERS_POLICY_PATH},ro" not in volume_args assert not sudoers_file.exists() assert not sudoers_directory.exists() assert "DOCKER_HOST=unix:///run/host-docker.sock" in args @@ -621,6 +645,72 @@ def test_authorized_runtime_effects_build_the_complete_docker_plan(tmp_path: Pat assert "SYS_ADMIN" not in args +@pytest.mark.parametrize("host_os,expected_gid", [(HostOs.LINUX, "file"), (HostOs.DARWIN, 0)]) +def test_host_docker_group_follows_where_the_socket_is_owned(tmp_path: Path, host_os, expected_gid) -> None: + # On Linux the capsule joins the socket file's group. Docker Desktop shows + # its socket inside the VM as root:root 0660, so a Mac host joins group 0 + # and writes no second group for it. + project = tmp_path / "project" + project.mkdir() + env = base_env(tmp_path) + docker_socket = tmp_path / "docker.sock" + docker_socket.touch() + gid = docker_socket.stat().st_gid if expected_gid == "file" else expected_gid + host_user = HostUser(uid=501, gid=20, name="developer", group_name="staff") + with ( + patch("devcapsule.launch.pycharm._launcher.current_host_user", return_value=host_user), + patch("devcapsule.launch.pycharm._launcher.is_socket", return_value=True), + patch("devcapsule.launch.pycharm._launcher.HostOs.current", return_value=host_os), + ): + config = build_run_config( + PycharmRunOptions( + project=project, + capsule_platform=Platform.LINUX_AMD64, + project_mount="/workspace/project", + docker_mode=DockerMode.host, + host_docker_socket=docker_socket, + network_mode="bridge", + runtime_plan=external_runtime_plan(), + use_image_process=True, + ), + env, + ) + with patch("devcapsule.launch.pycharm._launcher.write_xauthority"): + files = prepare_temp_runtime_files(config, env) + try: + args = build_docker_args(config, files, env) + groups = files.group_file.read_text(encoding="utf-8") + finally: + cleanup_temp_runtime_files(files) + + assert config.host_docker_gid == gid + assert ["--group-add", str(gid)] == args[args.index("--group-add") : args.index("--group-add") + 2] + assert ("host-docker:x:" in groups) is (gid not in (0, host_user.gid)) + + +@pytest.mark.parametrize("host_os,effective", [(HostOs.LINUX, "host"), (HostOs.DARWIN, "bridge")]) +def test_host_networking_falls_back_to_bridge_on_macos(tmp_path: Path, capsys, host_os, effective) -> None: + # Docker Desktop's host network is its VM's; the contained desktop would + # listen where the Mac cannot reach it. + project = tmp_path / "project" + project.mkdir() + with patch("devcapsule.launch.pycharm._launcher.HostOs.current", return_value=host_os): + config = build_run_config( + PycharmRunOptions( + project=project, + capsule_platform=Platform.LINUX_AMD64, + project_mount="/workspace/project", + docker_mode=DockerMode.none, + network_mode="host", + runtime_plan=external_runtime_plan(), + use_image_process=True, + ), + base_env(tmp_path), + ) + assert config.network_mode == effective + assert ("host networking is unavailable on macOS" in capsys.readouterr().err) is (effective == "bridge") + + def test_unauthorized_runtime_effects_keep_safe_docker_defaults(tmp_path: Path) -> None: project = tmp_path / "project" project.mkdir() @@ -635,6 +725,7 @@ def test_unauthorized_runtime_effects_keep_safe_docker_defaults(tmp_path: Path) config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -675,7 +766,8 @@ def test_sudoers_ownership_helper_is_constrained(tmp_path: Path) -> None: ) config = cast( PycharmRunConfig, - SimpleNamespace(enable_sudo=True, image="devcapsule-local-pycharm:canonical"), + SimpleNamespace(enable_sudo=True, image="devcapsule-local-pycharm:canonical", + capsule_platform=Platform.LINUX_AMD64, daemon_architecture="amd64"), ) with ( @@ -709,7 +801,8 @@ def test_sudoers_ownership_helper_failure_is_actionable(tmp_path: Path) -> None: ) config = cast( PycharmRunConfig, - SimpleNamespace(enable_sudo=True, image="devcapsule-local-pycharm:canonical"), + SimpleNamespace(enable_sudo=True, image="devcapsule-local-pycharm:canonical", + capsule_platform=Platform.LINUX_AMD64, daemon_architecture="amd64"), ) with ( @@ -721,6 +814,93 @@ def test_sudoers_ownership_helper_failure_is_actionable(tmp_path: Path) -> None: ensure_sudoers_policy_ownership(config, files, {}) +def _macos_sudo_files(tmp_path: Path) -> tuple[PycharmRunConfig, TempRuntimeFiles]: + policy = tmp_path / "sudoers" + policy.write_text(SUDOERS_POLICY, encoding="utf-8") + files = TempRuntimeFiles( + xauth_file=tmp_path / "xauth", + passwd_file=tmp_path / "passwd", + group_file=tmp_path / "group", + sudoers_file=policy, + ) + config = cast( + PycharmRunConfig, + SimpleNamespace(enable_sudo=True, image="devcapsule-local-pycharm:canonical", name="capsule", + capsule_platform=Platform.LINUX_AMD64, daemon_architecture="arm64"), + ) + return config, files + + +def test_macos_sudo_policy_is_written_into_a_volume_by_root( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, +) -> None: + # Docker Desktop shares Mac files as the Mac user: no chown makes a + # bind-mounted policy root's, so the policy lives in a volume. + monkeypatch.setattr(HostOs, "current", classmethod(lambda cls: HostOs.DARWIN)) + config, files = _macos_sudo_files(tmp_path) + + with ( + patch("devcapsule.launch.pycharm._launcher.os.chown") as chown, + patch("devcapsule.launch.pycharm._launcher.subprocess.run") as run, + ): + run.return_value = SimpleNamespace(returncode=0, stderr="", stdout="") + ensure_sudoers_policy_ownership(config, files, {"PATH": "/usr/bin"}) + + chown.assert_not_called() + assert files.sudoers_volume == "capsule-sudoers" + listed, create, write = (call.args[0] for call in run.call_args_list) + assert listed == ["docker", "volume", "ls", "--quiet", "--filter", + "label=devcapsule.purpose=development-sudo-policy"] + assert create == ["docker", "volume", "create", "--label", "devcapsule.purpose=development-sudo-policy", + "capsule-sudoers"] + assert write[:4] == ["docker", "run", "--rm", "-i"] + assert "--platform=linux/amd64" in write + assert ["--network", "none"] == write[write.index("--network") : write.index("--network") + 2] + assert ["--cap-drop", "ALL"] == write[write.index("--cap-drop") : write.index("--cap-drop") + 2] + assert "--cap-add" not in write + assert ["--user", "0:0"] == write[write.index("--user") : write.index("--user") + 2] + assert {"--read-only", "--pull=never", "no-new-privileges"} <= set(write) + assert "type=volume,src=capsule-sudoers,dst=/policy" in write + assert write[-1].startswith("umask 0337;") + assert write[-1].endswith("/policy/policy") + assert run.call_args_list[2].kwargs["input"] == SUDOERS_POLICY + + +def test_macos_sudo_policy_failure_removes_its_volume( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.setattr(HostOs, "current", classmethod(lambda cls: HostOs.DARWIN)) + config, files = _macos_sudo_files(tmp_path) + + with patch("devcapsule.launch.pycharm._launcher.subprocess.run") as run: + run.side_effect = [ + SimpleNamespace(returncode=0, stderr="", stdout=""), + SimpleNamespace(returncode=0, stderr=""), + SimpleNamespace(returncode=125, stderr="no such image"), + SimpleNamespace(returncode=0, stderr=""), + ] + with pytest.raises(PycharmRunError, match="no such image"): + ensure_sudoers_policy_ownership(config, files, {}) + + assert run.call_args_list[-1].args[0] == ["docker", "volume", "rm", "--force", "capsule-sudoers"] + assert files.sudoers_volume is None + + +def test_only_an_old_policy_volume_counts_as_abandoned() -> None: + # A concurrent launch's fresh volume is unused until its capsule starts. + old = (datetime.now(timezone.utc) - timedelta(days=2)).isoformat() + fresh = datetime.now(timezone.utc).isoformat() + with patch("devcapsule.launch.pycharm._launcher.subprocess.run") as run: + run.side_effect = [ + SimpleNamespace(returncode=0, stdout="killed-sudoers\nconcurrent-sudoers\n"), + SimpleNamespace(returncode=0, stdout=f"killed-sudoers {old}\nconcurrent-sudoers {fresh}\n"), + SimpleNamespace(returncode=0), + ] + remove_abandoned_sudoers_volumes({}) + + assert run.call_args_list[-1].args[0] == ["docker", "volume", "rm", "killed-sudoers"] + + def test_sudo_banner_is_printed_after_complete_policy_preparation( tmp_path: Path, capsys: pytest.CaptureFixture[str], @@ -733,8 +913,10 @@ def test_sudo_banner_is_printed_after_complete_policy_preparation( with ( patch("devcapsule.launch.pycharm._launcher.shutil.which", return_value=None), patch( - "devcapsule.launch.pycharm._launcher.ensure_sudoers_policy_ownership" + "devcapsule.launch.pycharm._launcher.ensure_sudoers_policy_ownership", + side_effect=lambda _config, files, _env: setattr(files, "sudoers_volume", "capsule-sudoers"), ) as ensure, + patch("devcapsule.launch.pycharm._launcher.remove_sudoers_policy_volume") as remove, patch("devcapsule.launch.pycharm._launcher.run_attached") as run, ): run.return_value = AttachedResult(0, interrupted=False, stopped=True) @@ -742,6 +924,7 @@ def test_sudo_banner_is_printed_after_complete_policy_preparation( run_pycharm( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, image="devcapsule-local-pycharm:canonical", docker_mode=DockerMode.none, network_mode="bridge", @@ -755,7 +938,9 @@ def test_sudo_banner_is_printed_after_complete_policy_preparation( ensure.assert_called_once() assert "DEVELOPMENT SUDO IS ENABLED" in capsys.readouterr().err command = run.call_args.args[0] - assert any(SUDOERS_POLICY_PATH in argument for argument in command) + assert any(SUDOERS_POLICY_PATH in argument and "src=capsule-sudoers" in argument for argument in command) + # The policy volume outlives nothing but the session. + assert remove.call_args.args[0] == "capsule-sudoers" def test_sudo_policy_preparation_failure_cleans_files_without_enabled_banner( @@ -782,6 +967,7 @@ def test_sudo_policy_preparation_failure_cleans_files_without_enabled_banner( run_pycharm( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, image="devcapsule-local-pycharm:canonical", docker_mode=DockerMode.none, network_mode="bridge", @@ -814,6 +1000,7 @@ def test_bound_secret_must_exist_on_host(tmp_path: Path) -> None: build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, docker_mode=DockerMode.none, secret_environment=("OPENAI_API_KEY",), ), @@ -836,6 +1023,7 @@ def test_image_process_uses_oci_command_and_cleans_all_temporary_files(tmp_path: run_pycharm( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", image=image, docker_mode=DockerMode.none, @@ -885,6 +1073,7 @@ def test_launch_preparation_failure_cleans_runtime_and_identity_files(tmp_path: run_pycharm( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", image="devcapsule-local-pycharm:canonical", docker_mode=DockerMode.none, @@ -958,6 +1147,7 @@ def test_ide_config_option_implies_custom_config_mode(tmp_path: Path) -> None: config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, ide_config=custom_config, docker_mode=DockerMode.none, ), @@ -977,6 +1167,7 @@ def test_explicit_shared_config_mode_ignores_env_ide_config_dir(tmp_path: Path) config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, config_mode=IdeConfigMode.shared, docker_mode=DockerMode.none, ), @@ -996,7 +1187,7 @@ def test_env_ide_config_dir_implies_custom_when_mode_is_unset(tmp_path: Path) -> env = base_env(tmp_path) env["PYCHARM_IDE_CONFIG_DIR"] = str(tmp_path / "env-config") - config = build_run_config(PycharmRunOptions(project=project, docker_mode=DockerMode.none), env) + config = build_run_config(PycharmRunOptions(project=project, capsule_platform=Platform.LINUX_AMD64, docker_mode=DockerMode.none), env) assert config.ide_config_mode == "custom" assert config.ide_config == (tmp_path / "env-config").resolve() @@ -1011,6 +1202,7 @@ def test_profile_sets_global_settings_and_plugins_defaults(tmp_path: Path) -> No config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, profile="codex", docker_mode=DockerMode.none, ), @@ -1030,6 +1222,7 @@ def test_project_state_root_mirrors_project_path_under_root_parent(tmp_path: Pat config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_state_root=workspace / ".state", docker_mode=DockerMode.none, ), @@ -1046,6 +1239,7 @@ def test_explicit_project_state_overrides_project_state_root(tmp_path: Path) -> config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_state=tmp_path / "explicit-state", project_state_root=tmp_path / ".state", docker_mode=DockerMode.none, @@ -1066,6 +1260,7 @@ def test_dogfood_legacy_roots_map_to_persistent_home_and_component_slots(tmp_pat config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, global_settings=global_settings, plugins=plugins, project_state=project_state, @@ -1089,6 +1284,7 @@ def test_persistent_home_defaults_to_checkout_scoped_xdg_data(tmp_path: Path) -> config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, docker_mode=DockerMode.none, ), base_env(tmp_path), @@ -1136,6 +1332,7 @@ def contained_config(tmp_path: Path, network_mode: str = "bridge", **overrides: env.pop("DISPLAY") # the contained display needs no host X session at all options: dict[str, object] = dict( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode=network_mode, @@ -1223,11 +1420,12 @@ def test_host_x11_passthrough_still_needs_a_host_display_and_binds_it(tmp_path: env = base_env(tmp_path) env.pop("DISPLAY") with pytest.raises(PycharmRunError, match="DISPLAY is not set"): - build_run_config(PycharmRunOptions(project=project, project_mount="/workspace/project"), env) + build_run_config(PycharmRunOptions(project=project, capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project"), env) env = base_env(tmp_path) config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -1266,6 +1464,7 @@ def test_display_disclosure_states_the_transport(tmp_path: Path) -> None: build_run_config( PycharmRunOptions( project=tmp_path / "project", + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -1307,6 +1506,7 @@ def fake_docker_run(command: list[str], env: dict[str, str], **_kwargs: object) exit_code = run_pycharm( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -1343,7 +1543,7 @@ def test_launcher_mounts_live_configuration_directory_and_private_snapshot_read_ env = base_env(tmp_path) env["XDG_RUNTIME_DIR"] = str(tmp_path / "runtime") config = build_run_config(PycharmRunOptions( - project=project, project_mount="/workspace/project", docker_mode=DockerMode.none, + project=project, capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, runtime_plan=external_runtime_plan(), use_image_process=True, launch_configuration=LaunchConfiguration(records, document), ), env) @@ -1371,7 +1571,7 @@ def test_launcher_mounts_live_configuration_directory_and_private_snapshot_read_ assert not files.launch_context_file.exists() and not files.configuration_directory.exists() listing = {"schema-version": 2, "rows": [], "pending": [], "notes": []} config = build_run_config(PycharmRunOptions( - project=project, project_mount="/workspace/project", docker_mode=DockerMode.none, + project=project, capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, runtime_plan=external_runtime_plan(), use_image_process=True, launch_configuration=LaunchConfiguration(records, document, listing), ), env) @@ -1580,7 +1780,7 @@ def test_console_launcher_cleans_up_after_command_reporting_or_docker_failure( runtime = tmp_path / "runtime" env = {**base_env(tmp_path), "XDG_RUNTIME_DIR": str(runtime)} report = Mock() - options = PycharmRunOptions(project=config.project, command_report=report if report_only else None) + options = PycharmRunOptions(project=config.project, capsule_platform=Platform.LINUX_AMD64, command_report=report if report_only else None) with ( patch("devcapsule.launch.pycharm._launcher.build_run_config", return_value=config), patch("devcapsule.launch.pycharm._launcher.write_user_files"), @@ -1616,7 +1816,7 @@ def test_web_console_is_absent_without_a_runtime_plan(tmp_path: Path) -> None: project = tmp_path / "project" project.mkdir() config = build_run_config( - PycharmRunOptions(project=project, project_mount="/workspace/project", docker_mode=DockerMode.none), + PycharmRunOptions(project=project, capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none), base_env(tmp_path), ) assert config.console_host_port is None and config.console_token == "" @@ -1677,7 +1877,7 @@ def ready(port: int, url: str, opener: object, stop: Event, **kwargs: object) -> patch("devcapsule.launch.pycharm._launcher.default_opener", return_value=default_console_opener) as default, ): assert run_pycharm(PycharmRunOptions( - project=project, project_mount="/workspace/project", docker_mode=DockerMode.none, + project=project, capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", runtime_plan=external_runtime_plan(), use_image_process=True, display_transport=display_transport, open_display_url=display_opener, open_console_url=console_opener if custom_console_opener else None, @@ -1732,6 +1932,7 @@ def session_config(tmp_path: Path, **environment: str) -> tuple[PycharmRunConfig config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="bridge", @@ -1841,7 +2042,7 @@ def test_unusable_session_directory_does_not_add_a_mount_or_stop_launch(tmp_path patch.object(launcher, "watch_display_ready"), patch.object(launcher, "run_attached", return_value=AttachedResult(7, interrupted=False, stopped=True)) as docker, ): - assert run_pycharm(PycharmRunOptions(project=config.project), env) == 7 + assert run_pycharm(PycharmRunOptions(project=config.project, capsule_platform=Platform.LINUX_AMD64), env) == 7 command = docker.call_args.args[0] assert not any(SESSION_LOG_DESTINATION in arg for arg in command) assert "cannot use session logs" in capsys.readouterr().err @@ -1923,7 +2124,7 @@ def wait(self, timeout=None): patch.object(launcher, "watch_display_ready"), patch.object(launcher.subprocess, "Popen", Client), ): - assert run_pycharm(PycharmRunOptions(project=config.project), env) == 0 + assert run_pycharm(PycharmRunOptions(project=config.project, capsule_platform=Platform.LINUX_AMD64), env) == 0 # Docker receives the literal arguments, in order, before the image. [command] = commands image = command.index(config.image) @@ -1968,7 +2169,7 @@ def wait(self, timeout=None): patch.object(launcher, "watch_display_ready", side_effect=lambda port, url, opener, stop, **kw: stops.append(stop)), patch.object(launcher.subprocess, "Popen", return_value=Client()), ): - assert run_pycharm(PycharmRunOptions(project=config.project), env) == launcher.INTERRUPTED_EXIT_CODE + assert run_pycharm(PycharmRunOptions(project=config.project, capsule_platform=Platform.LINUX_AMD64), env) == launcher.INTERRUPTED_EXIT_CODE assert stopped_before_return assert stops and all(stop.is_set() for stop in stops) assert list((tmp_path / "runtime").iterdir()) == [] @@ -2252,7 +2453,7 @@ def test_the_launcher_records_the_sessions_end_in_its_manifest(tmp_path: Path) - patch.object(launcher, "watch_display_ready"), patch.object(launcher, "run_attached", return_value=AttachedResult(7, interrupted=False, stopped=True)), ): - assert run_pycharm(PycharmRunOptions(project=config.project), env) == 7 + assert run_pycharm(PycharmRunOptions(project=config.project, capsule_platform=Platform.LINUX_AMD64), env) == 7 manifest = json.loads((config.session_log_directory / "session.json").read_text(encoding="utf-8")) assert manifest["session"] == config.session_id and manifest["exit-code"] == 7 assert manifest["ended-at"].endswith("Z") and logs.session_ended(config.session_log_directory) @@ -2263,7 +2464,7 @@ def test_the_launcher_records_the_sessions_end_in_its_manifest(tmp_path: Path) - patch.object(launcher, "watch_display_ready"), patch.object(launcher, "run_attached", return_value=AttachedResult(143, interrupted=True, stopped=True)), ): - assert run_pycharm(PycharmRunOptions(project=config.project), env) == launcher.INTERRUPTED_EXIT_CODE + assert run_pycharm(PycharmRunOptions(project=config.project, capsule_platform=Platform.LINUX_AMD64), env) == launcher.INTERRUPTED_EXIT_CODE manifest = json.loads((config.session_log_directory / "session.json").read_text(encoding="utf-8")) assert manifest["exit-code"] == launcher.INTERRUPTED_EXIT_CODE and manifest["ended-at"] finally: @@ -2290,7 +2491,7 @@ def test_interrupted_launch_with_unstopped_capsule_leaves_session_manifest_unend return_value=AttachedResult(launcher.INTERRUPTED_EXIT_CODE, interrupted=True, stopped=False), ), ): - assert run_pycharm(PycharmRunOptions(project=config.project), env) == launcher.INTERRUPTED_EXIT_CODE + assert run_pycharm(PycharmRunOptions(project=config.project, capsule_platform=Platform.LINUX_AMD64), env) == launcher.INTERRUPTED_EXIT_CODE manifest = json.loads((config.session_log_directory / "session.json").read_text(encoding="utf-8")) assert "ended-at" not in manifest and "exit-code" not in manifest assert not logs.session_ended(config.session_log_directory) diff --git a/devcapsule-src/tests/test_recursive_orchestrator.py b/devcapsule-src/tests/test_recursive_orchestrator.py index 58898f20..9c4026ad 100644 --- a/devcapsule-src/tests/test_recursive_orchestrator.py +++ b/devcapsule-src/tests/test_recursive_orchestrator.py @@ -3,6 +3,7 @@ from dataclasses import dataclass from pathlib import Path import socket +import sys import pytest @@ -152,6 +153,7 @@ def test_normal_project_launch_derives_only_a_downgradable_readiness_marker( assert unauthorized == {RECURSIVE_E2E_ENABLED_ENV: "0"} +@pytest.mark.skipif(sys.platform != "linux", reason="the recursive dogfood flow is Linux-only") def test_public_dry_run_is_unique_redacted_and_cleans_owned_staging( recursive_fixture: RecursiveFixture, ) -> None: @@ -180,6 +182,7 @@ def test_public_dry_run_is_unique_redacted_and_cleans_owned_staging( assert not (workspace / second.run_id).exists() +@pytest.mark.skipif(sys.platform != "linux", reason="the recursive dogfood flow is Linux-only") def test_public_dry_run_cleans_after_preparation_failure( recursive_fixture: RecursiveFixture, ) -> None: @@ -203,6 +206,7 @@ def test_public_dry_run_cleans_after_preparation_failure( assert not run_root.exists() +@pytest.mark.skipif(sys.platform != "linux", reason="the recursive dogfood flow is Linux-only") def test_keep_on_failure_preserves_only_the_owned_run( recursive_fixture: RecursiveFixture, ) -> None: @@ -228,6 +232,7 @@ def test_keep_on_failure_preserves_only_the_owned_run( assert run_root.is_dir() +@pytest.mark.skipif(sys.platform != "linux", reason="the recursive dogfood flow is Linux-only") def test_contained_display_dry_run_forwards_no_host_x11( recursive_fixture: RecursiveFixture, tmp_path: Path ) -> None: diff --git a/devcapsule-src/tests/test_recursive_successor_plan.py b/devcapsule-src/tests/test_recursive_successor_plan.py index 9e2d0ac3..2c1532f3 100644 --- a/devcapsule-src/tests/test_recursive_successor_plan.py +++ b/devcapsule-src/tests/test_recursive_successor_plan.py @@ -10,6 +10,7 @@ from devcapsule.components.pycharm import runtime_template as pycharm_runtime_template from devcapsule.launch.pycharm import DockerMode, PycharmRunOptions +from devcapsule.platforms import Platform from devcapsule.launch.pycharm._launcher import ( TempRuntimeFiles, build_docker_args, @@ -184,6 +185,7 @@ def test_plan_models_every_flag_the_real_launcher_emits(tmp_path: Path, enable_s config = build_run_config( PycharmRunOptions( project=project, + capsule_platform=Platform.LINUX_AMD64, project_mount="/workspace/project", docker_mode=DockerMode.none, network_mode="host", @@ -566,3 +568,21 @@ def test_duplicate_mount_destination_is_rejected() -> None: inspection["Mounts"].append(dict(inspection["Mounts"][0])) assert "declared more than once" in rejected(inspection) + + +@pytest.mark.parametrize("spelling", [["--platform=linux/amd64"], ["--platform", "linux/amd64"]]) +def test_an_emulated_launch_plans_its_platform_and_the_successor_must_run_it(spelling: list[str]) -> None: + # The launcher names the platform when the daemon emulates it; both of + # Docker's spellings parse, and a successor that ran another is refused. + plan = build_plan([*docker_args(), *spelling]) + assert plan.platform == "linux/amd64" + inspection = matching_inspection(plan) + inspection["ImageManifestDescriptor"] = {"platform": {"os": "linux", "architecture": "amd64"}} + compare_inspection(plan, inspection) + inspection["ImageManifestDescriptor"] = {"platform": {"os": "linux", "architecture": "arm64"}} + with pytest.raises(SuccessorPlanError, match="successor platform does not match the planned --platform"): + compare_inspection(plan, inspection) + # The independent inspection reloads the retained plan, platform included; + # a native plan's retained form is unchanged. + assert ExpectedSuccessorPlan.from_mapping(plan.to_mapping(show_host_paths=True)) == plan + assert "platform" not in build_plan(docker_args()).to_mapping() diff --git a/devcapsule-src/tests/test_runtime_artifact.py b/devcapsule-src/tests/test_runtime_artifact.py index 4eda6d3a..e8857ac7 100644 --- a/devcapsule-src/tests/test_runtime_artifact.py +++ b/devcapsule-src/tests/test_runtime_artifact.py @@ -7,18 +7,50 @@ from devcapsule.compat import CliError from devcapsule.materialization import ArtifactSpec, formation_descriptor, formation_identity -from devcapsule.runtime_artifact import runtime_artifact +from devcapsule.build_info import BuildInfo +from devcapsule.platforms import HostOs, Platform +from devcapsule.runtime_artifact import executable_platform, runtime_artifact from devcapsule.runtime_command import RuntimeCommand from tests.test_base_image import pex_fixture +from tests.test_platforms import on_host pytestmark = pytest.mark.legacy +AMD64 = Platform.LINUX_AMD64 +ARM64 = Platform.LINUX_ARM64 +MACH_O_ARM64 = b"\xcf\xfa\xed\xfe" + bytes(16) +ELF_AMD64 = b"\x7fELF\x02\x01\x01" + bytes(11) + (0x3E).to_bytes(2, "little") +ELF_ARM64 = b"\x7fELF\x02\x01\x01" + bytes(11) + (0xB7).to_bytes(2, "little") +ELF_RISCV = b"\x7fELF\x02\x01\x01" + bytes(11) + (0xF3).to_bytes(2, "little") + + +def pex_with_native_header(path: Path, header: bytes, *, revision: str = "a" * 40) -> Path: + """A PEX fixture whose first bytes are a native executable's header. + + A scie is a native launcher with the PEX appended; the build information + is read from the archive, so the header and the archive coexist. + """ + + pex = pex_fixture(path, revision=revision) + pex.write_bytes(header + pex.read_bytes()) + return pex + + +@pytest.fixture(autouse=True) +def launcher_of_the_fixtures_build(monkeypatch: pytest.MonkeyPatch) -> None: + """The running launcher is the build the fixtures record: 0.1.0 at revision a…a.""" + + monkeypatch.setattr("devcapsule.runtime_artifact.current_build_info", lambda: BuildInfo( + schema_version=2, version="0.1.0", source_repository="https://github.com/example/devcapsule", + source_revision="a" * 40, source_url="unknown")) + + def test_packaged_launcher_supplies_itself_despite_source_override(tmp_path: Path, monkeypatch) -> None: launcher = pex_fixture(tmp_path / "launcher.pex") monkeypatch.setenv("SCIE", str(launcher)) monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(tmp_path / "different.pex")) - assert runtime_artifact() == launcher + assert runtime_artifact(AMD64) == launcher def test_source_launcher_requires_explicit_built_runtime(tmp_path: Path, monkeypatch) -> None: @@ -26,10 +58,84 @@ def test_source_launcher_requires_explicit_built_runtime(tmp_path: Path, monkeyp monkeypatch.delenv("PEX", raising=False) monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(tmp_path / "missing.pex")) with pytest.raises(CliError, match="DEVCAPSULE_RUNTIME_PEX"): - runtime_artifact() + runtime_artifact(AMD64) launcher = pex_fixture(tmp_path / "runtime.pex") monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(launcher)) - assert runtime_artifact() == launcher + assert runtime_artifact(AMD64) == launcher + + +def test_executable_platform_reads_native_headers(tmp_path: Path) -> None: + assert executable_platform(pex_with_native_header(tmp_path / "mac", MACH_O_ARM64)) is HostOs.DARWIN + assert executable_platform(pex_with_native_header(tmp_path / "amd64", ELF_AMD64)) is AMD64 + assert executable_platform(pex_with_native_header(tmp_path / "arm64", ELF_ARM64)) is ARM64 + assert executable_platform(pex_with_native_header(tmp_path / "riscv", ELF_RISCV)) is HostOs.LINUX + assert executable_platform(pex_fixture(tmp_path / "plain.pex")) is None + assert executable_platform(tmp_path / "absent") is None + + +@pytest.mark.own_host_patches +def test_macos_launcher_delivers_the_named_linux_runtime(tmp_path: Path, monkeypatch) -> None: + on_host(monkeypatch, "Darwin", "arm64") + monkeypatch.setenv("SCIE", str(pex_with_native_header(tmp_path / "devcapsule-macos", MACH_O_ARM64))) + linux = pex_with_native_header(tmp_path / "devcapsule-linux", ELF_AMD64) + monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(linux)) + assert runtime_artifact(AMD64) == linux + + +@pytest.mark.own_host_patches +def test_macos_launcher_without_a_linux_runtime_says_what_to_set(tmp_path: Path, monkeypatch) -> None: + on_host(monkeypatch, "Darwin", "arm64") + monkeypatch.setenv("SCIE", str(pex_with_native_header(tmp_path / "devcapsule-macos", MACH_O_ARM64))) + monkeypatch.delenv("DEVCAPSULE_RUNTIME_PEX", raising=False) + with pytest.raises(CliError, match="a macOS executable and cannot run inside linux-amd64 capsules.*DEVCAPSULE_RUNTIME_PEX"): + runtime_artifact(AMD64) + monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(pex_with_native_header(tmp_path / "wrong-arch", ELF_ARM64))) + with pytest.raises(CliError, match="a linux-arm64 executable"): + runtime_artifact(AMD64) + + +@pytest.mark.own_host_patches +def test_a_plain_pex_launcher_on_a_mac_is_not_delivered(tmp_path: Path, monkeypatch) -> None: + # A PEX bundles native wheels for the machine it was built on. + on_host(monkeypatch, "Darwin", "arm64") + monkeypatch.delenv("SCIE", raising=False) + monkeypatch.setenv("PEX", str(pex_fixture(tmp_path / "launcher.pex"))) + linux = pex_with_native_header(tmp_path / "devcapsule-linux", ELF_AMD64) + monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(linux)) + assert runtime_artifact(AMD64) == linux + monkeypatch.delenv("DEVCAPSULE_RUNTIME_PEX") + with pytest.raises(CliError, match="a plain PEX built for another machine"): + runtime_artifact(AMD64) + + +@pytest.mark.own_host_patches +def test_a_runtime_of_another_build_is_refused(tmp_path: Path, monkeypatch) -> None: + # D-0009: the runtime runs the plan this launcher writes. A runtime left + # from an earlier build, after the launcher was updated, is refused. + on_host(monkeypatch, "Darwin", "arm64") + monkeypatch.setenv("SCIE", str(pex_with_native_header(tmp_path / "devcapsule-macos", MACH_O_ARM64))) + stale = pex_with_native_header(tmp_path / "devcapsule-linux", ELF_AMD64, revision="b" * 40) + monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(stale)) + with pytest.raises(CliError, match="is DevCapsule 0.1.0 from b+, but this launcher is 0.1.0 from a+"): + runtime_artifact(AMD64) + + +@pytest.mark.own_host_patches +def test_a_runtime_directory_serves_each_capsule_platform(tmp_path: Path, monkeypatch) -> None: + # Apple Silicon runs both platforms; one setting names both runtimes, and + # each launch delivers the one for its lock's platform. + on_host(monkeypatch, "Darwin", "arm64") + monkeypatch.setenv("SCIE", str(pex_with_native_header(tmp_path / "devcapsule-macos", MACH_O_ARM64))) + runtimes = tmp_path / "runtimes" + runtimes.mkdir() + amd64 = pex_with_native_header(runtimes / "devcapsule-linux-amd64", ELF_AMD64) + arm64 = pex_with_native_header(runtimes / "devcapsule-linux-arm64", ELF_ARM64) + monkeypatch.setenv("DEVCAPSULE_RUNTIME_PEX", str(runtimes)) + assert runtime_artifact(ARM64) == arm64 + assert runtime_artifact(AMD64) == amd64 + amd64.unlink() + with pytest.raises(CliError, match="holds no devcapsule-linux-amd64"): + runtime_artifact(AMD64) def test_runtime_update_changes_formation_identity_without_changing_components() -> None: diff --git a/devcapsule-src/tests/test_supervisor.py b/devcapsule-src/tests/test_supervisor.py index 0bb50b36..3ae83194 100644 --- a/devcapsule-src/tests/test_supervisor.py +++ b/devcapsule-src/tests/test_supervisor.py @@ -30,6 +30,10 @@ pytestmark = pytest.mark.legacy +# The supervisor is the capsule's entry process: it runs only inside Linux +# capsules and relies on Linux signal primitives (sigwaitinfo). +pytestmark = pytest.mark.skipif(sys.platform != "linux", reason="container-side code; capsules are Linux") + DRIVER = Path(__file__).parent / "resources" / "supervisor_driver.py" REPO_ROOT = Path(__file__).resolve().parents[1] diff --git a/devcapsule-src/tests/test_version_sets.py b/devcapsule-src/tests/test_version_sets.py index 16b1a733..c0723c0b 100644 --- a/devcapsule-src/tests/test_version_sets.py +++ b/devcapsule-src/tests/test_version_sets.py @@ -152,8 +152,8 @@ def open_payload(url, **kwargs): selected = ExecutionConfiguration.load(root).project runtime = tmp_path / "runtime.pex" runtime.write_bytes(b"fixture-runtime") - monkeypatch.setattr("devcapsule.environment_realization.runtime_artifact", lambda: runtime) - monkeypatch.setattr("devcapsule.runtime_artifact.runtime_artifact", lambda: runtime) + monkeypatch.setattr("devcapsule.environment_realization.runtime_artifact", lambda _platform: runtime) + monkeypatch.setattr("devcapsule.runtime_artifact.runtime_artifact", lambda _platform: runtime) monkeypatch.setattr("devcapsule.materialization.read_pex_build_info", lambda _: SimpleNamespace( build_mnemonic="fixture", source_repository="fixture", source_revision="fixture", source_url="fixture")) base = ImageDetails(lock["base"]["reference"], "sha256:" + "a" * 64, { @@ -163,7 +163,7 @@ def open_payload(url, **kwargs): state = SimpleNamespace(fail_build=False, exit_code=0, during_run=None, launched=[], built=[], images=images, root=root, record=selected.checkout_path, resolution=selected.resolution_path, component=component, lock=path, original=path.read_bytes(), metadata=metadata, base=base) - def require(reference): + def require(reference, platform=None): if reference not in images: raise CliError("local image missing") return images[reference] diff --git a/docs/getting-started/install.md b/docs/getting-started/install.md index 7a298168..31342016 100644 --- a/docs/getting-started/install.md +++ b/docs/getting-started/install.md @@ -1,7 +1,7 @@ --- description: Check prerequisites, download the DevCapsule executable for this release, verify its SHA-256 checksum, and install it under ~/.local/bin. weight: 1 -updated: 2026-10-30 +updated: 2026-11-03 --- # Install DevCapsule @@ -9,7 +9,9 @@ DevCapsule ships as a single executable for **Linux (`x86_64` / Intel-AMD 64-bit)**. It requires a web browser and a working Docker daemon on the same machine. It does not run natively on macOS, Windows, or ARM. **On Windows, read [Windows via WSL2](../platforms/windows-wsl2.md) first** and run every command -below inside your WSL2 Linux terminal. +below inside your WSL2 Linux terminal. On an Apple Silicon Mac, read +[macOS](../platforms/macos.md) for the experimental launcher you build from +source; the steps below do not apply. ## Check your computer diff --git a/docs/platforms/macos.md b/docs/platforms/macos.md index 7e92b680..c89837fd 100644 --- a/docs/platforms/macos.md +++ b/docs/platforms/macos.md @@ -1,24 +1,133 @@ --- -description: Why DevCapsule does not run on macOS yet, and what a native macOS port would require. +description: An experimental launcher you build from source runs on Apple Silicon Macs against Docker Desktop, with capsules emulated or native. weight: 3 -updated: 2026-10-30 +updated: 2026-11-04 --- # macOS -DevCapsule {{tag}} does not run on macOS (`darwin-arm64` or `darwin-amd64`). +DevCapsule {{tag}} ships no macOS executable. The release executable is for +64-bit Intel/AMD Linux and does not run on macOS. -Today, both the release executable and the locked component catalog target -`linux-amd64` only, and the launcher expects a Linux Docker host sharing a -native Linux filesystem with the checkout. Supporting macOS properly will -require: +The launcher in the source tree runs natively on **Apple Silicon Macs** +(`darwin-arm64`) against **Docker Desktop**, built by you. This is +experimental and not a supported platform in {{tag}}. Intel Macs are untested. -- Native macOS launcher builds and `linux-arm64` / `linux-amd64` container - resolution rules for Apple Silicon and Intel Macs. -- End-to-end verification of Docker Desktop (or Lima/Colima) bind-mount - semantics, permissions, and loopback port forwarding. -- Contained-desktop and [web console](../sessions/web-console.md) validation in - macOS browsers. +A project runs as one of two capsule platforms: -None of that work is in {{tag}}; when macOS support begins, the supported -platforms in `.devcapsule/devcapsule..lock` and [Install](../getting-started/install.md) -will announce it explicitly. +- **`linux-arm64`, natively, the default.** Faster, but its base image is not + published yet: you build it on your Mac once and select it for each checkout. +- **`linux-amd64`, under emulation.** The same environments Linux users get, + with the published base, run by Docker Desktop under x86-64 emulation. + +`devcapsule project init` asks which one a new project is for; Enter takes +`linux-arm64`, and `--platform PLATFORM` answers without the question. A +project runs as `linux-arm64` when it has a `linux-arm64` lock, and otherwise +as `linux-amd64`. + +## What differs from Linux + +- **Two executables, or three.** On Linux the launcher copies itself into the + capsule. A macOS executable cannot run there, so you build the Linux runtime + for each capsule platform you use and put them in one directory that + `DEVCAPSULE_RUNTIME_PEX` names. Each launch delivers the one for its + project's platform and refuses one of another DevCapsule version. Rebuild + them whenever you rebuild the launcher: runtimes of the same version from + another commit are not told apart. +- **Emulation.** Under `linux-amd64`, the first launch of an IDE builds its + image under emulation: about ten minutes for PyCharm on a 14-core Mac. Later + launches reuse the image. The IDE runs slower than on a native host. +- **Networking.** Docker Desktop's host network belongs to its virtual machine, + out of your Mac's reach. A project authorized for `network host` launches on + bridge networking instead, with a warning. The contained desktop and the + [web console](../sessions/web-console.md) are published to `127.0.0.1` as on + Linux, and the capsule reaches services on your Mac as `host.docker.internal`. +- **No host-browser bridge.** Docker Desktop cannot share a Mac socket with a + container, so links the IDE opens cannot reach your Mac's browser. The launch + warns and continues. The contained desktop still opens in your browser. +- **Docker inside the capsule.** With `docker-daemon` authorized, the capsule's + Docker client reaches Docker Desktop's daemon, whose engine is `arm64` on + either capsule platform: an image you run from inside the capsule is the ARM + variant unless you pass `--platform linux/amd64`. +- **File sharing.** Keep projects under a directory Docker Desktop shares, by + default `/Users`, `/Volumes`, `/private`, `/tmp` and `/var/folders`. Files the + capsule writes there belong to your user. + +## Build and install + +You need Docker Desktop with several gigabytes free under **Settings → +Resources → Disk usage limit**, and about as much free on the Mac itself, plus +Git and Python 3.12. From a clean checkout of a published commit: + +```bash +git clone https://github.com/ccozianu/devcapsule.git +cd devcapsule/devcapsule-src +python3.12 -m venv .venv +.venv/bin/pip install -r dev-requirements.txt +.venv/bin/pip install -e . --no-deps +revision="$(git rev-parse HEAD)" +DEVCAPSULE_SCIE_PLATFORM=macos-aarch64 PYTHON=.venv/bin/python \ + scripts/build-pex.sh --source-revision "$revision" --allow-unpublished-revision \ + --output dist/devcapsule-macos-arm64 +for target in amd64:linux-x86_64 arm64:linux-aarch64; do + git -C .. archive HEAD | docker run --rm -i --platform "linux/${target%%:*}" \ + -v "$PWD/dist:/out" python:3.12-slim sh -c " + mkdir /build && tar -x -C /build && cd /build/devcapsule-src && + python -m venv /venv && /venv/bin/pip install -q -r dev-requirements.txt && + /venv/bin/pip install -q -e . --no-deps && + DEVCAPSULE_SCIE_PLATFORM=${target#*:} PYTHON=/venv/bin/python scripts/build-pex.sh \ + --allow-local-source --output /out/devcapsule-linux-${target%%:*}" +done +``` + +This writes the macOS launcher and a Linux runtime for each capsule platform, +all from the same commit. `--allow-unpublished-revision` skips only the check +that the commit is a branch tip; the commit itself must be public, since a base +build installs the web console from it. Install them: + +```bash +mkdir -p ~/.local/bin ~/.local/lib/devcapsule +install -m 0755 dist/devcapsule-macos-arm64 ~/.local/bin/devcapsule +install -m 0755 dist/devcapsule-linux-amd64 dist/devcapsule-linux-arm64 ~/.local/lib/devcapsule/ +export DEVCAPSULE_RUNTIME_PEX="$HOME/.local/lib/devcapsule" +~/.local/bin/devcapsule version +``` + +Add the `export` line, and `$HOME/.local/bin` on your `PATH`, to your shell +profile. Then build the arm64 base below, or continue with +[your first session](../getting-started/first-session.md) and answer +`linux-amd64` when init asks for the platform. + +## Run a project natively as `linux-arm64` + +Build the arm64 base once, natively, with the launcher you installed and the +revision it was built from (`devcapsule version` shows it): + +```bash +devcapsule images build --type base --recipe ubuntu-24.04 --platform linux-arm64 \ + --tag devcapsule-base-local:arm64 --source-revision "$revision" +``` + +Then, in each project, create the arm64 lock, or add it beside an existing +`linux-amd64` one, and select your local base for the checkout: + +```bash +devcapsule project init --platform linux-arm64 --authorize base-image default +devcapsule project config authorize base-image devcapsule-base-local:arm64 +devcapsule project run +``` + +VSCodium, VS Code, Eclipse, PyCharm, IntelliJ IDEA and Rider have each started +natively on this base. A component no arm64 run has validated yet, such as the +.NET SDK or Playwright, needs `--unverified` on `init` or on the `config` +command that adds it; the command says so. The arm64 lock names a base that is +not published yet, so do not commit it to a shared repository: anyone without +a local arm64 base cannot run it. A change to the project's requirements made +on your Mac updates both locks. + +To run the project under emulation again, delete +`.devcapsule/devcapsule.linux-arm64.lock` and select the project's base for +your checkout again with `devcapsule project config authorize base-image default`. + +If something fails on your Mac, please [open an issue](https://github.com/ccozianu/devcapsule/issues/new) +with your macOS and Docker Desktop versions and the error output, omitting +private session URLs and tokens. diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 078cbca1..04cb7fa6 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -1,7 +1,7 @@ --- description: The command tree of the devcapsule launcher, with one line of intent per command and usage for init, run and config. weight: 1 -updated: 2026-10-30 +updated: 2026-11-04 --- # Command reference @@ -55,7 +55,8 @@ home instead and disables this fallback to launcher settings. ```text devcapsule project init [--require NAME[>=VERSION]]... [--offer CAPABILITY]... [--ide IDE] [--with CAPABILITY]... [--name NAME] [--slug SLUG] [--creator CREATOR] [--project-mount PATH] - [--regenerate] [--unverified] [--set NAME VALUE]... [--authorize NAME VALUE]... [--bind NAME VALUE]... + [--platform PLATFORM] [--regenerate] [--unverified] + [--set NAME VALUE]... [--authorize NAME VALUE]... [--bind NAME VALUE]... ``` `--require` and `--offer` write shared project capabilities into @@ -72,6 +73,15 @@ IDE is selected, `init` can ask for base or vendor consent; non-interactive scripts can pass `--authorize NAME VALUE`. `--set`, `--bind`, and `--authorize` also accept one optional justification argument after the value. +A project has one lock per capsule platform it runs as, such as +`devcapsule.linux-amd64.lock`. A project runs as the first platform this host +runs that it has a lock for; an Apple Silicon Mac runs `linux-arm64` before +`linux-amd64`. On a host that runs more than one, init asks which platform a +new project is for, the native one by default. `--platform PLATFORM` answers +without the question, and adds that platform's lock to a project locked for +another, so on the Mac `devcapsule project init --platform linux-arm64` lets an +emulated project run natively. + ### `wrap` ```text @@ -194,7 +204,7 @@ Shared project contract (for repository owners; writes schema 2): Every shared-contract command accepts `--preview` to validate and print the resulting change without writing files. Capability changes and provider -recommendations update the manifest and platform lock together. Capability +recommendations update the manifest and every platform lock together. Capability commands accept `--unverified`; `recommend` does not. Environment declarations and non-provider recommendations update only `.devcapsule/devcapsule.toml`. @@ -248,7 +258,7 @@ Each command below follows `devcapsule`: | Command | Intent | |---|---| | `version [--json]` | Print the launcher version, package version, and source revision provenance. | -| `images build --type {base,environment} ...` | Build a curated base image or materialize a project environment image. | +| `images build --type {base,environment} ...` | Build a curated base image, for the capsule platform `--platform` names, or materialize a project environment image. | | `images list [--include-legacy]` | List DevCapsule base and environment images in the local Docker store. | | `bootstrap [--refresh-workflow-definition] project` | Install or refresh DevCapsule workflow files and initialize project status records. | | `workflow brief` | Print the active workstream summary, live claims, waiting mail, sync distance, and validation commands. | diff --git a/engineering-docs/development/e2e-tests.md b/engineering-docs/development/e2e-tests.md index cdba9525..80595fbe 100644 --- a/engineering-docs/development/e2e-tests.md +++ b/engineering-docs/development/e2e-tests.md @@ -76,6 +76,8 @@ to installing `xprop` in every image. | `DEVCAPSULE_EARLY_EXIT_E2E_IMAGE`, `DEVCAPSULE_CONTRIBUTOR_E2E_IMAGE`, `DEVCAPSULE_PEX_CLEAN_MACHINE_IMAGE` | the sessions | Disposable images the tests build or reuse | | `DEVCAPSULE_E2E_BUILD_NETWORK` | `-- --build-network` | The network mode of image builds | | `DEVCAPSULE_SMOKE_SCENARIO` | `-- --scenario` | Catalog ID or JSON path of a recorded IDE scenario to replay | +| `DEVCAPSULE_SMOKE_PLATFORM` | `-- --platform` | The capsule platform the IDE smoke creates each project for, such as `linux-arm64` | +| `DEVCAPSULE_RUNTIME_PEX` | you, for a macOS launcher | The Linux runtime it delivers, or a directory of `devcapsule-` runtimes | | `PLAYWRIGHT_BROWSERS_PATH` | you, optionally | Where `ide-smoke -- --display` finds or installs Chromium | ## Evidence diff --git a/engineering-docs/development/unit-tests.md b/engineering-docs/development/unit-tests.md index 9126e306..c552869b 100644 --- a/engineering-docs/development/unit-tests.md +++ b/engineering-docs/development/unit-tests.md @@ -26,7 +26,8 @@ with a terminal report of missing lines, and the default selection `-m "not integration and not e2e"`. So a bare `pytest` under `tests/` is the unit suite; the heavier suites are chosen by marker, never by accident. The `flaky` marker runs a test for visibility with a temporarily non-blocking -result; the other markers belong to the heavier suites. +result; `own_host_patches` marks a unit test that patches the host itself +(see Fixtures); the other markers belong to the heavier suites. On a pull request the hosted workflow also writes both coverage figures, changed lines covered and the whole package, into the pull request's @@ -122,11 +123,16 @@ so prefer the existing laws and an acceptance step for what the user sees. ## Fixtures -`tests/conftest.py` holds three fixtures and nothing else: +`tests/conftest.py` holds the suite's shared fixtures and hooks: - `host_launch_by_default`, autouse: makes launcher tests behave as if they run on the host, since the launcher translates bind sources when it finds itself inside a capsule and the suite is often run from one. +- `amd64_linux_host_by_default`, autouse: makes the host an amd64 Linux + machine with an amd64 Docker daemon, as on the CI, since the launcher takes + other paths on macOS and Apple Silicon runs linux-arm64 capsules. A test of + another host patches the host itself and carries `own_host_patches`; + Docker-marked tests keep the real host. - `private_user_directories`, autouse: moves `HOME` and every XDG directory under the test's `tmp_path` for non-Docker tests. Inside a capsule this is not yet a private home: a home with no settings of its own reads the @@ -136,6 +142,10 @@ so prefer the existing laws and an acceptance step for what the user sees. set, otherwise `dist/devcapsule-local.pex`; it fails with the Nox session to run when the file is missing. The integration and end-to-end suites use it; the unit suite does not need an artifact. +- `pytest_configure` and `pytest_sessionfinish`: on macOS, a short temporary + root under `/tmp`, because a Unix socket path there is limited to 104 + bytes. A session whose tests all passed removes it; a failed one keeps it + and prints its path. Everything else is local to its module: fakes for Docker and vendor services, temporary XDG roots, and the `tmp_path` pytest provides. diff --git a/engineering-docs/requirements/devcapsule/r-runtime-001-launcher-runtime-identity.md b/engineering-docs/requirements/devcapsule/r-runtime-001-launcher-runtime-identity.md index bbb111cc..8e6fd616 100644 --- a/engineering-docs/requirements/devcapsule/r-runtime-001-launcher-runtime-identity.md +++ b/engineering-docs/requirements/devcapsule/r-runtime-001-launcher-runtime-identity.md @@ -50,10 +50,21 @@ dependency cycle and keeps runtime-plan production and execution in one build. - Source-form launchers can explicitly select a built PEX using `DEVCAPSULE_RUNTIME_PEX`; running that PEX directly gives the ordinary identical-executable path. Source Python files are not themselves a PEX. -- The packaged launcher currently takes precedence over - `DEVCAPSULE_RUNTIME_PEX`. That variable is a source-development selection, - not a general packaged-launcher runtime override. No new override interface - is implemented by recording this decision. +- A packaged launcher that runs on the capsule platform takes precedence + over `DEVCAPSULE_RUNTIME_PEX`. That variable is otherwise a source-development + selection, not a general packaged-launcher runtime override. +- Since the `linux-arm64` workstream (2026-11-04, unreleased): a packaged + launcher that cannot run inside the capsule, a macOS executable launching + Linux capsules, delivers the runtime `DEVCAPSULE_RUNTIME_PEX` names instead, + and stops with that remedy when it is unset. The variable may name a + directory holding one runtime per capsule platform as + `devcapsule-`; the lock's platform selects. This is the explicit + user choice D-0009 admits, not a new default: the outside and inside bytes + differ, so the delivered runtime must be the launcher's build, the same + version, and the same source revision wherever both record one. A local + build without a recorded revision can be matched by version only. A + release shipping macOS launchers would reopen D-0009, whose *Reopen If* + names this case. ## Verification @@ -74,3 +85,9 @@ These fixture-image checks do not stand in for actual IDE-terminal acceptance. Future changes must preserve these properties. Any additional override interface needs its own explicit-selection and default-regression coverage; this record does not claim that interface has been validated. + +The macOS delivery path is covered by `tests/test_runtime_artifact.py`: the +named runtime and the directory form, refusal of a foreign or missing runtime, +of a plain PEX on a non-Linux host, and of a runtime from another build. The +IDE smoke compares the capsule's runtime digest with the runtime the workspace's +lock selects. Neither is a release validation of macOS launchers. diff --git a/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md b/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md new file mode 100644 index 00000000..111e9139 --- /dev/null +++ b/engineering-docs/wip/2026-11-03-macos-launcher/2026-11-03-design-macos-host-port.md @@ -0,0 +1,158 @@ +# Design: The Launcher On macOS + +Workstream: `macos-launcher`. Written 2026-11-03 from a read of the launcher and probes on +an Apple Silicon Mac with Docker Desktop 29.3.1. Statements marked *to verify* are read from +code or recalled Docker Desktop behaviour, not yet observed on this host. + +## 1. The two platforms + +Today one value plays two roles. `Platform.current()` (`devcapsule/platforms.py`) derives a +key from the host's `system` and `machine`, and every caller uses it to name and select the +platform lock and the resolution matrix: + +- `configuration/storage.py`: the lock path `devcapsule..lock`, and the refusal of a + version set recorded for another platform; +- `configuration/derivation.py`, `configuration/operations.py`, + `configuration/owner_commands.py`, `version_sets.py`: `MATRICES[Platform.current()]`. + +None of those callers asks about the host. They ask which platform the capsules run as. On +Linux the two coincide. On a Mac the host is `darwin-arm64` and the capsules are Linux. + +Design: + +1. `Platform` keeps its meaning, the wire format of a lock: a capsule platform. Its members + stay append-only; no `darwin-*` member is added, because no lock is ever for macOS. +2. `Platform.current()` keeps its name and its callers, and its contract becomes *the + platform this host runs capsules as*. On Linux it is the host's own platform, as now. On + macOS it is `linux-amd64` on both architectures: Intel natively, Apple Silicon under + Docker's emulation. When `linux-arm64` exists, Apple Silicon maps to it and emulation + becomes the fallback, not the rule. The mapping lives in `platforms.py` alone. +3. A new `HostOs` (`linux`, `darwin`) names the host's operating system for the host-side + mechanisms of section 2. Feature code asks `HostOs.current()`, never `sys.platform`. +4. `Platform.docker_platform` gives Docker's spelling (`linux/amd64`), and + `Platform.emulated()` says whether this host's CPU differs from the capsule platform's. + When it does, every `docker run` and every buildx build the launcher composes carries + `--platform`, so that Docker does not pick the host's architecture for a multi-platform + image or warn on a single-platform one. + +Rejected: renaming `current()` to `for_capsules()`. The name is accurate once the contract is +written down, and the rename touches 27 call and test sites for no behaviour. + +## 2. Host mechanisms that assume Linux + +Found by reading the launch path from `run_pycharm` (`launch/pycharm/_launcher.py`) outward. + +| Mechanism | Where | On macOS | Plan | +|---|---|---|---| +| Lock and matrix key | section 1 | refuses `darwin-arm64` | slice 1 | +| Docker `--platform` | `_launcher.py` `docker run`; `images/build.py` buildx | Docker picks arm64 or warns | slice 1 | +| Host-browser bridge | `host_open.py` | peer check used `SO_PEERCRED`, Linux only; the broker socket is a host Unix socket bind-mounted into the capsule, which Docker Desktop refuses (section 2.1) | slice 2: unavailable on macOS with a warning, per R-CONFIG-001; the peer check also reads macOS `LOCAL_PEERCRED`; a TCP transport is a later decision. The contained desktop's own opener is Python's `webbrowser`, which already uses `open` | +| Host Docker socket | `_launcher.py` `host_docker_socket`, `write_user_files` | both socket paths bind; `--group-add` took the host file's gid (`staff`), but the socket is `0:0 0660` inside (section 2.1) | slice 3: on macOS the capsule user joins group 0; the capsule already holds the daemon, root on the VM, so the root group adds no authority beyond it; confirmed in the live launch (section 2.2) | +| Container identity | `current_host_user`, `write_user_files` | 501:20; the generated `/etc/group` avoids the image's `dialout`; ownership round-trips (sections 2.1, 2.2) | none | +| Container detection | `host_daemon.py` | a marker file, not `/proc`; already correct (section 2.1) | none | +| Runtime artifact | `runtime_artifact.py` | the launcher delivers its own executable into the capsule; a macOS binary cannot run there | slice 1: deliver `DEVCAPSULE_RUNTIME_PEX` when the launcher's own binary is foreign, refuse with the remedy otherwise; how a release ships the Linux runtime beside the macOS launcher is slice 4 | +| Executable | `scripts/build-pex.sh` | `DEVCAPSULE_SCIE_PLATFORM=macos-aarch64` | slice 4 | +| Development sudo | `ensure_sudoers_policy_ownership`, `build_docker_args` | the policy must be root's; a bind-mounted Mac file stays the Mac user's, and the helper's `chown` is refused even to root (found by the human, 2026-11-04) | fixed: on macOS root writes the policy into a Docker volume named after the container, mounted read-only from it and removed with the session; `sudo -n id -u` prints 0 in a live launch | +| Ports | `--publish 127.0.0.1:` | forwarded to the Mac's loopback (section 2.1) | none | +| Host networking | `_launcher.py` `build_run_config` | a listener on loopback under `--network host` is unreachable from the Mac; a bridge publish to `127.0.0.1` answers (section 2.1) | slice 3: on macOS an authorized `network host` launches on bridge with a warning; the Mac's services are `host.docker.internal`. This is the concrete reason and fallback *Local Launch Networking* asks to record, and a product default for the owner to confirm | + +### 2.1 Observed on this Mac, 2026-11-03 + +Docker Desktop 29.3.1, macOS 27.0, arm64, `desktop-linux` context. + +- **Host Unix sockets.** A host socket as a bind source fails with `invalid mount config for + type "bind": bind source path does not exist: /socket_mnt/`, for a socket in the + session scratch, in `$TMPDIR` (`/var/folders/…/T/devcapsule-host-open.*/broker.sock`, where + the broker would bind it, 90 bytes) and in `~/.cache`. Docker Desktop rewrites the source to + `/socket_mnt/…` and does not find it. The host-browser bridge cannot reach a capsule. +- **The Docker socket** is the exception: `/var/run/docker.sock` on the Mac is a symlink to + `~/.docker/run/docker.sock`, and binding either path gives the capsule a socket owned + `0:0`, mode `0660`. The host file's group means nothing inside. +- **Ownership.** A capsule process running as `501:20` writes files that the Mac shows as + owned by uid 501; the group is the parent directory's, as macOS assigns it. +- **Identity.** The launcher writes the capsule's `/etc/passwd` and `/etc/group` whole, so the + host's `staff` (20) does not collide with the image's `dialout` (20). +- **`/proc`.** `host_daemon.in_container()` tests a marker file; nothing on the host path reads + `/proc`. The row above is void. +- **Networking.** A `linux/amd64` HTTP server bound to `127.0.0.1` under `--network host` is + unreachable from the Mac (`curl` HTTP 000); the same server on bridge, published to + `127.0.0.1:18766`, answers HTTP 200. `host.docker.internal` resolves inside a capsule. +- **Executables.** `build-pex.sh` with `DEVCAPSULE_SCIE_PLATFORM=macos-aarch64` builds natively + here: a 31 MB Mach-O arm64 scie, CPython 3.12.14, that answers `version --json` and + `project config show`. The `linux-x86_64` runtime, 40 MB, builds inside a `linux/amd64` + `python:3.12-slim` container from an export of the commit. With both, `runtime_artifact()` + delivers the Linux one, and without `DEVCAPSULE_RUNTIME_PEX` it refuses with the remedy. +- **Disk.** Docker Desktop's VM disk was 32 GB with 5.9 GB free; the base is 0.89 GB + compressed. The human raised it to 96 GB before the live launch. + +### 2.2 The live launch, 2026-11-03 + +The macOS launcher and the `linux-x86_64` runtime, both built from 4e42d073, against Docker +Desktop on this Mac. Evidence under `devcapsule-src/dist/e2e-evidence/` (not committed). + +- **`nox -s ide-smoke -- --surface pycharm` passes.** The first run took 10 minutes: it built + the PyCharm formation under emulation, the build's last step alone 63 s. It failed only in + the harness, which matched the ready line a capsule prints, not the one a host prints; see + below. The second run, from the built image, passed in 13 s: desktop page 200, console 200 + with its token and 403 without, a `jetbrains-pycharm` window, the capsule's runtime equal to + the Linux runtime delivered, container removed, project records removed. +- **The launch**, as the launcher printed it: the host-networking warning, then bridge, the + desktop on container port 6080 published to `127.0.0.1:`, the console on 6081 the + same way. `docker inspect` shows network mode `bridge`, platform `linux`, both ports bound + to `127.0.0.1`. +- **In a Mac browser.** The desktop URL opened in Chrome on this Mac shows the capsule's + desktop with PyCharm's User Agreement, the first window of a fresh configuration, 16 s + after launch: splash at 10 s, the agreement at 16 s. Nobody accepted it, so the project + window was not reached. +- **Identity and ownership.** `uname -m` in the capsule is `x86_64`. PyCharm's JVM runs as + uid 501, gid 20. A file the IDE user writes in the project is uid 501 on the Mac, group + the parent directory's; a host file reads as `501 20` inside. The workspace was under + `/private/tmp`, which Docker Desktop shares by default. +- **Docker socket.** With `--authorize docker-daemon host-socket`, the container gets + `--group-add 0`, PyCharm's process carries groups `0 20`, and the IDE user's + `docker version` reaches Docker Desktop's daemon (server 29.3.1, `arm64`) and `docker ps` + lists the host's containers. The daemon is the VM's: an image the IDE user runs without + `--platform` is `arm64`, unlike the capsule itself. +- **Host-browser bridge.** Not exercised: unavailable on macOS with its warning (section 2.1). +- **The harness.** The smoke's readiness pattern required the ready line that names the URL, + which a launcher inside a capsule prints. A launcher outside a container opens the URL + instead, a no-op under the smoke's `BROWSER=true`, and prints the ready line without it; + on any host but a capsule the smoke waited out its 600 s. The harness now takes either + ready line and the URL from the launcher's earlier announcement. The child-runtime + digest check compares with the runtime the launcher delivers. +- **Test suite.** Before slice 1: 411 failed and 461 errors of 2,743. After the platform + split: 30 failed and 24 errors, in four groups: the supervisor (container-side, + `signal.sigwaitinfo` is Linux only); Unix socket paths beyond macOS's 104 bytes under the + per-user temporary directory; the recursive dogfood, which validates a host X11 socket + directory; the host-browser bridge. + +## 3. Slices + +1. **Capsule platform.** Section 1, `--platform` on run and build, `/proc` absence. Done when + `project config show` and `project run --print-command` answer on this Mac, and the unit + tests pass here and keep passing on Linux. +2. **Browser bridge.** Unavailable-with-warning on macOS; `open` as the opener for when a + transport exists. +3. **Live launch.** A real `project run` of this repository on this Mac: Docker socket, + identity, ownership of written files, ports, the contained desktop in a Mac browser. + Findings decide the code; this slice is mostly evidence. +4. **Executable and documentation.** A `macos-aarch64` scie, smoke-tested here; the macOS + platform page and the install page rewritten for what works; release automation is the + owner's call. Done 2026-11-03: the page describes the experimental from-source launcher, + its differences from Linux, and a build of both executables whose commands were run on + this Mac; the install page sends Mac readers there. + +## 4. What this does not decide + +- Whether macOS becomes a *supported* platform in a release. That is the owner's ruling; + this workstream produces the evidence for it. +- [D-0009](../../decisions/product/d-0009-launcher-delivers-identical-runtime.md), one + executable outside and inside. A macOS launcher cannot deliver its own bytes, so on macOS + the delivered runtime is the user's explicit choice in `DEVCAPSULE_RUNTIME_PEX`, which + D-0009's third point admits. As a product default it would not hold: D-0009's *Reopen If* + names this case, a supported execution environment that cannot run the launcher inside its + container. A macOS release needs the owner to reopen it and say how identity holds, for + example a macOS launcher that carries the exact Linux runtime bytes and delivers them, or + one that fetches the release's Linux executable against a digest it pins. +- Signing and notarization of a macOS executable. +- `linux-arm64` capsules. diff --git a/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md new file mode 100644 index 00000000..7d3eb985 --- /dev/null +++ b/engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md @@ -0,0 +1,134 @@ +# Workstream Current Status: macOS Launcher + +Mnemonic: `macos-launcher` + +Start date: 2026-11-03 + +State: active; slices 1 to 4 done 2026-11-03; waits on the SSH key to publish, and on the owner's rulings + +Branch association: `ws-macos-launcher/host-port`, forked from local `main` at b3f10b7d + +Integration target: `main` + +Delivery method: pull request, merge commit; the human opens and merges it + +Requirements: R-PRODUCT-002 (host boundaries stay explicit on the new host); R-CONFIG-001 + +## Goal And Scope + +Run the DevCapsule launcher natively on an Apple Silicon Mac (`darwin-arm64`) against +Docker Desktop, with capsules running as `linux-amd64` under Docker's x86 emulation. + +In scope: the launcher on macOS, the existing `linux-amd64` lock, pins and base image used +unchanged, the host-side mechanisms that assume Linux, a `macos-aarch64` executable, and the +macOS platform page. + +Out of scope, by the human's direction of 2026-11-03 ("do stage 2 no need for stage 1"): +`linux-arm64` capsules, their resolution matrix, pins and base image. Native-speed +capsules on Apple Silicon need that work later; this workstream leaves room for it. + +## Current State + +**2026-11-03, opened.** The human asked whether DevCapsule builds for ARM, then for native +macOS on Apple Silicon, and directed the macOS host port without ARM capsules. Evidence on +the human's Mac (macOS 27.0, arm64, Docker Desktop 29.3.1, `desktop-linux` context at +`~/.docker/run/docker.sock`, 14 CPUs, 47 GiB): + +- `docker run --platform linux/amd64 ubuntu:24.04 uname -m` prints `x86_64`: emulation works. +- The locked base `devcapsule-base@sha256:8c97d308…` is a single-platform manifest. +- The development CLI runs (`version --json` answers), and every project command stops at + `Unsupported platform 'darwin-arm64'; supported platforms: linux-amd64.` + +The work plan, one slice each, is in the +[design note](2026-11-03-design-macos-host-port.md). + +**2026-11-03, slices 1 and 2.** `Platform.current()` now means the platform this host runs +capsules as (`linux-amd64` on a Mac), with `HostOs` for host mechanisms. Every `docker run`, +buildx build and base pull carries `--platform linux/amd64` when the host emulates it, and a +native Linux launch composes the same command as before. A launcher whose own binary cannot +run in the capsule delivers `DEVCAPSULE_RUNTIME_PEX` instead, or refuses with that remedy. +The host-browser bridge is unavailable on macOS with a warning; its peer check reads +`LOCAL_PEERCRED`, and two macOS socket races in its protocol are fixed (an early rejection +closed with unread input; `shutdown` after the broker closed). The capsule user joins group 0 +for Docker Desktop's socket. `project config show` answers on this Mac. + +Validation on this Mac: the unit suite, 2,725 passed, 19 skipped (the container-side +supervisor and the X11-dependent recursive dogfood, by marker), from 411 failed and 461 errors +before; the host-open module 20 runs clean; `mypy --platform linux` clean over 269 files. `nox` +cannot run its sessions here, having no `python3.12` on `PATH`, so the commands were run from +`.venv` directly. The unit suite on Linux is still to run, inside an amd64 container. + +**2026-11-03, slice 3 so far.** Host networking falls back to bridge on macOS with a warning: +under `--network host` the contained desktop and console would listen on the VM's loopback, +out of the Mac's reach. The unit suite pins the host OS to Linux by default, as it pins the +host-not-container default, and the macOS tests opt in; `real_host_os` opts a test out. + +Both executables exist: `devcapsule-src/dist/devcapsule-macos-arm64`, built natively, and the +`linux-x86_64` runtime, built in an amd64 container from an export of 062782bf. In that +container the unit suite ran on Linux: 2,738 passed, 1 failed, +`test_recursive_host.py::test_staging_cleans_up_after_preparation_failure`, which expects a +`chmod 0500` directory to refuse a write and runs as root there; not caused by this change. +On this Mac after the fallback: 2,727 passed, 19 skipped; `mypy --platform linux` clean. + +**2026-11-03, slice 3, the live launch.** The human raised Docker Desktop's disk to 96 GB. +Both executables rebuilt from 4e42d073. `nox -s ide-smoke -- --surface pycharm` passes on +this Mac with the macOS launcher delivering the Linux runtime: the first run built the +PyCharm formation under emulation in 10 minutes and failed only in the harness, the second +passed in 13 s. Two probe launches added what the smoke does not check: the capsule's desktop +open in Chrome on this Mac, PyCharm at its User Agreement 16 s after launch; `x86_64` in the +capsule; files written as uid 501 on both sides; with `docker-daemon` authorized, the IDE +user reaches Docker Desktop's daemon through group 0. The record is section 2.2 of the +design note. The User Agreement was left unaccepted; accepting it is the human's. + +The harness had matched only the ready line a capsule prints; a launcher outside a container +prints another, so the smoke could not pass on any host but a capsule. It now takes either. +Its runtime digest check compares with the runtime the launcher delivers. + +**2026-11-03, slice 4.** `docs/platforms/macos.md` now describes the experimental launcher +built from source on Apple Silicon: no macOS executable in the release, the two executables, +emulation, the bridge fallback, no host-browser bridge, Docker access reaching an `arm64` +daemon, file sharing, and a build of both executables. The page's commands were run on this +Mac, the Linux build from `git archive` of the commit into a `linux/amd64` container. The +install page sends Mac readers there. `nox -s docs-contract` skipped itself: the website +submodule is not initialized here, and fetching it needs the SSH key. + +Reading for slice 4 found that a supported macOS release reopens +[D-0009](../../decisions/product/d-0009-launcher-delivers-identical-runtime.md): the +launcher cannot deliver its own bytes there. Today's path is the explicit user override +D-0009 admits. The design note's section 4 states the question for the owner. + +**2026-11-04, development sudo.** The human's first own launch on this Mac, a project with +development sudo, stopped at "Permission denied": the sudo policy must be root's, and Docker +Desktop shares a Mac file as the Mac user, which even root in the helper container cannot +change. Neither smoke nor probe had enabled sudo. On macOS root now writes the policy into a +Docker volume named after the container, mounted read-only and removed when the session +ends (e5bd8acd); the Linux helper gained `--platform` when emulating. A live VSCodium launch +with development sudo: the policy `0 0 0440` in the capsule, `sudo -n id -u` as the IDE user +prints 0, the volume gone afterwards. Unit suite on this Mac: 2,729 passed, 19 skipped. The +fixed pair is installed in the human's `~/.local`. + +## Next Resumable Task + +None in the plan; the slices are done. Publish the workstream and push the branch once the +SSH key is loaded, give the human the compare link, and wait for the owner's rulings below. + +## Open Threads + +- The SSH agent on this Mac holds no identity, so `git fetch`, `workflow claim` and + `workflow publish` fail with `Permission denied (publickey)`. The human loads the key; + until then the registration is local only and `main` could not be re-verified. +- No `python3.12` exists on this Mac. `.venv` was created with `uv venv --python 3.12` + (CPython 3.12.12) instead of `python3.12 -m venv`; the executable still embeds 3.12.14. +- Adding a supported host platform is a product commitment. The repository owner has not + yet ruled on it; the human started the work. It includes reopening D-0009 (design note, + section 4) and how a release would ship the Linux runtime beside the macOS launcher. +- The macOS default for an authorized `network host`, bridge with a warning, was put to the + human on 2026-11-03 with refusal as the alternative; no answer yet, so the warning stands. +- The PyCharm User Agreement in the probe sessions was left unaccepted, so the project + window was not reached on this Mac. + +## Document Index + +- [2026-11-03-design-macos-host-port.md](2026-11-03-design-macos-host-port.md): the host + assumptions found, the design of the platform split, and the slice plan. Open before + each slice. diff --git a/engineering-docs/wip/2026-11-03-macos-launcher/intake-dispositions.md b/engineering-docs/wip/2026-11-03-macos-launcher/intake-dispositions.md new file mode 100644 index 00000000..e11d9eb8 --- /dev/null +++ b/engineering-docs/wip/2026-11-03-macos-launcher/intake-dispositions.md @@ -0,0 +1,4 @@ +# Intake Decisions: `macos-launcher` + +Record each removed intake item, its decision, reasoning, and resulting +status file requirement or task here. diff --git a/engineering-docs/wip/2026-11-03-macos-launcher/intake/README.md b/engineering-docs/wip/2026-11-03-macos-launcher/intake/README.md new file mode 100644 index 00000000..34538b70 --- /dev/null +++ b/engineering-docs/wip/2026-11-03-macos-launcher/intake/README.md @@ -0,0 +1,9 @@ +# Intake: `macos-launcher` + +Work other workstreams have delivered to `macos-launcher` and that it has not +yet decided. Senders add one file per item, named +`YYYY-MM-DD--.md`, and deliver it by mail on the +coordination branch, never through `main` or by editing this directory. Each +item is acknowledged into the status file or forwarded as `WORKFLOW.md` +permits, and every decision is recorded in `intake-dispositions.md` before the +item is removed. diff --git a/engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md b/engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md new file mode 100644 index 00000000..d8b8c027 --- /dev/null +++ b/engineering-docs/wip/2026-11-04-linux-arm64/2026-11-04-design-linux-arm64.md @@ -0,0 +1,114 @@ +# Design: linux-arm64 Capsules + +Workstream: `linux-arm64`. Written 2026-11-04 from two read-only surveys of the launcher and +the base recipe, on top of `macos-launcher`'s platform split. File references are to +`devcapsule-src/`. + +## 1. What exists + +- **Resolution is already per platform.** `MATRICES` (`resolution_matrix.py`) maps each + `Platform` to one matrix of base pins, component pins, verified edges, capabilities and + materialization recipes. Locks are `devcapsule..lock`; a project may commit one + per platform, and a checkout holds one version set. +- **The base recipe is arch-neutral.** `images/base.py` renders a Dockerfile from + `ubuntu:24.04` and Ubuntu packages, which exist for arm64; Node and Temurin choose their + archive by `dpkg --print-architecture`, and Temurin's arm64 digest is already pinned. +- **Discovery already speaks arm64.** `components/discovery.py` and `npm_channel.py` map + `linux-arm64` for JetBrains, Eclipse, VSCodium, VS Code, Claude Code and npm packages. +- **What is amd64-only.** `Platform` has one member; every component pin is an amd64 + archive; Eclipse refuses other platforms (`components/eclipse.py`); Codex's npm alias map + names only `linux-amd64` (`components/codex.py`); the JetBrains runtime check names + `libskiko-linux-x64.so` (`assets/*/check-runtime-deps.sh`); `build-pex.sh` defaults to a + `linux-x86_64` scie; the published base is a single-platform amd64 manifest. + +## 2. Decisions taken by the human, 2026-11-04 + +1. Every component at once, not a first slice. +2. The arm64 base is built locally and approved per checkout; the owner publishes later. +3. On Apple Silicon, a project runs as `linux-arm64` when it has a `linux-arm64` lock, and + otherwise as `linux-amd64` under emulation. Amended the same day, after the human's + first init of a new project: on a host that runs several platforms, init asks which a + new project is for, the native one by default ("depending on the current os just ask", + "default arm64 on arm"). +4. A new workstream on the macOS branch. + +## 3. Design + +1. **`Platform.LINUX_ARM64`.** Host detection stays in `platforms.py`: `runnable()`, every + platform a host runs, native first (Apple Silicon: arm64, then amd64 under emulation), + and `current()`, the host's native platform, a new project's default, per section 2's + amended ruling; init asks on a host of several platforms. Which platform a + project runs as is the configuration's question: `configuration.storage.project_platform`, + read-only, answers the first runnable platform the project has a lock for, else + `current()`. `project init --platform` adds a platform's lock, and the project then runs + as it; `images build --platform` names the platform of a base build. A checkout so runs + one platform until its project's locks change. An owner's edit of the shared contract + regenerates every lock the project carries, so a committed lock never falls behind the + manifest (R-COMPAT-001). Below the configuration, the platform is the lock's, a required + parameter of the base pull, the image build, the runtime choice and the launch (D-0006's + consequence). Emulation is judged against the Docker daemon's architecture, asked once + per launch, not the launcher's CPU. +2. **The runtime.** An arm64 capsule needs a `linux-aarch64` executable. A launcher whose own + binary runs on the capsule platform delivers itself (R-RUNTIME-001). Otherwise the user + names the runtime: `DEVCAPSULE_RUNTIME_PEX` names one file, or a directory holding + `devcapsule-linux-amd64` and `devcapsule-linux-arm64`. A runtime that is not the + launcher's own bytes must be its build: the same version, and the same revision where + both record one. +3. **The base.** Built natively on this Mac by `devcapsule images build --type base`, and + approved per checkout by the documented local-base path. Whether the `linux-arm64` matrix + can resolve without a published base pin is slice 2's first question. +4. **Pins.** The same versions as the amd64 matrix, each with its arm64 archive's URL and a + sha256 computed from the downloaded bytes, never copied from a page alone. Eclipse's + native closure comes from `ports.ubuntu.com`. Playwright's Chromium has no + Chrome-for-Testing arm64 build; its arm64 source is slice 3's open question, and + `browser-automation` is unavailable on arm64 with a warning (R-CONFIG-001) if none passes. + +## 4. Slices + +1. **Second platform.** `LINUX_ARM64`, the per-project selection, the runtime directory, the + `linux-aarch64` scie, the Eclipse, Codex and skiko hard-codes, tests. Done when the unit + suite passes here and on Linux, and `project init` on this Mac writes an arm64 lock + against a stub matrix. +2. **Base.** Build the arm64 base here; settle how the arm64 matrix and a local base meet. +3. **Pins.** Every component, by script: download, hash, record. +4. **Live and documentation.** `nox -s ide-smoke` for every surface natively on this Mac; + the Linux, macOS and install pages. + +## 5. Reviews, 2026-11-04 + +A review of all changes since `main`, run by a separate agent, rejected the +branch with seven high findings: a Mac's new projects defaulting to the unpublished arm64 +base; `images build` ignoring the project's locks; the platform read from a process-wide +selection below the configuration instead of from the lock; no version check on the +delivered runtime; personal pins re-derived on their own platform; a leaking sudo-policy +volume; a hash test that skipped three components. Fourteen medium and three low findings +followed. The human said "fix all"; each is fixed as described in section 3, except that a +lock naming an unpublished base is reported, not refused, since the human's ruling has the +checkout select a local base for it. + +A second review of all changes since `main`, and a review of the branch against the +codebase's own conventions, followed the fixes. The first rejected the branch again with +three high findings: an owner's edit on a Mac rewrote only the lock of the platform the +project ran as, leaving the committed `linux-amd64` lock behind its manifest; a checkout's +one base-image answer applied to whichever lock `project --platform` named, so switching +platforms per command broke the launch; and each macOS launch's sweep of sudo-policy +volumes could remove a concurrent launch's fresh one. Nine medium and three low findings +followed. The second found seventeen departures, the first that `--platform` was kept in a +module global, process-wide state the codebase nowhere else has. The fixes are section 3's +design: `project --platform` is gone, owner edits regenerate every lock under one journal +that still reads the single-lock form an older launcher wrote, the sweep removes only +unused policy volumes older than a day, and the retained successor plan carries its +planned `--platform`. Comments, messages, tests and the generated pin module now follow the +surrounding code. + +Left as it is: a checkout keeps one base-image answer. A developer who deletes the arm64 +lock to return to emulation answers the base again (`docs/platforms/macos.md`). + +## 6. What this does not decide + +- Publishing an arm64 or multi-platform base, or arm64 release executables: the owner's. +- Whether ARM Linux becomes a supported release platform: the owner's. +- D-0006 says host detection is `current()` and parsing a lock is `parse()`, with + deliberately no third way to obtain a `Platform`. `runnable()`, a host's list, and + `of_elf_machine()`, an executable's platform, are two more; amending D-0006 to name them + is the owner's. diff --git a/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md new file mode 100644 index 00000000..5c1e0a31 --- /dev/null +++ b/engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md @@ -0,0 +1,140 @@ +# Workstream Current Status: linux-arm64 Capsules + +Mnemonic: `linux-arm64` + +Start date: 2026-11-04 + +State: active; slices 1 to 4 done 2026-11-04, six IDEs running natively; waits on publication and the owner + +Branch association: `ws-linux-arm64/capsules`, forked from local `main` at b3f10b7d, with +`ws-macos-launcher/host-port` merged in as its foundation + +Integration target: `main`, after `macos-launcher`'s pull request + +Delivery method: pull request, merge commit; the human opens and merges it + +Requirements: R-RUNTIME-001 (the runtime delivered matches the capsule); R-COMPAT-001 +(projects locked for `linux-amd64` keep working); R-CONFIG-001 + +## Goal And Scope + +Run capsules as `linux-arm64` as well as `linux-amd64`: natively on Apple Silicon Macs +through the `macos-launcher` work, and on ARM Linux hosts. + +In scope, by the human's answers of 2026-11-04: + +- every component the `linux-amd64` matrix offers, pinned for `linux-arm64` in one + workstream: PyCharm, IntelliJ IDEA, Rider and the .NET SDK, Eclipse and its native + package closure, VSCodium, VS Code, the Codex, Claude Code and Antigravity CLIs, and + Playwright; +- an arm64 base image built locally from the existing recipe and approved per checkout as a + local base; publishing it, and pinning its digest, is the owner's; +- on an Apple Silicon Mac, a project runs as `linux-arm64` when it has a `linux-arm64` lock, + and otherwise as `linux-amd64` under emulation, as today; +- a `linux-aarch64` runtime executable for arm64 capsules. + +Out of scope: publishing images or executables; release automation. + +## Current State + +**2026-11-04, opened.** The human, running capsules on the Mac under emulation, directed +native ARM capsules: "ok we need linux arm64", then answered four questions: everything at +once; build the base locally now; arm64 when locked, else amd64; a new workstream built on +the macOS branch. Two read-only surveys mapped the work; the plan is in the +[design note](2026-11-04-design-linux-arm64.md). + +**2026-11-04, slices 1 to 3.** `Platform.LINUX_ARM64`; a project runs as the first platform +the host runs that it has a lock for, and `project --platform` names one (f1bba6c9, +7a32f1c6). `DEVCAPSULE_RUNTIME_PEX` may name a directory of runtimes. The arm64 base was +built natively here from a launcher at public `main` b3f10b7d and pinned at its local index +digest, unpublished (7a32f1c6). Every component is pinned for arm64 at the amd64 versions, +102 artifacts, 15.6 GB hashed from their bytes (87e71b19), and the Eclipse closure gained +`libunwind8`, which arm64's GStreamer needs (b12e2516). The human's own switch of +`~/hello-devcapsule` to arm64 found that a checkout's personal pins kept the amd64 VSCodium; +personal pins are now re-pinned per platform (52fb1a1a). + +Native arm64 smokes on this Mac with the local base: VSCodium passed in 58 s with its main +window; VS Code passed. Eclipse failed at `libunwind8`, since fixed; PyCharm, Rider and +IntelliJ failed on Docker I/O errors when the Mac's disk filled. The disk filled because +these runs built five formations at once on top of a 20 GB artifact cache, and Docker +Desktop's VM failed with it; a restart recovered it. With the human's consent the test +images and build cache were removed, and the reruns go one surface at a time, checking free +space first and removing each image after. The macOS test session now removes its temporary +root; each run had left one in `/tmp`. + +Found for `maintenance`, not fixed here: the amd64 Eclipse closure's `libsoup-3.0-0` and +`libsoup-3.0-common` 3.4.4-5ubuntu0.8 URLs return 404, since Ubuntu removed that version from +every noble pocket on 2026-10-08, so Eclipse cannot be acquired afresh on amd64. + +**2026-11-04, review and fixes.** The human had all changes since `main` reviewed by a +separate agent: rejected, seven high, fourteen medium, three low findings. The human said "fix all"; the design note's section 5 records them, and +section 3 the design they produced. In short: a Mac creates `linux-amd64` projects again and +runs a project as the first platform it has a lock for; the configuration answers which +(`storage.project_platform`), and below it the lock's platform is passed down explicitly; a +delivered runtime must be the launcher's build; the unpublished arm64 base is reported at +init and refused at launch without a local base; Docker Desktop's conventions are `HostOs` +properties; emulation is the daemon's. Unit suite on this Mac: 2,749 passed, 25 skipped. +The human's `~/hello-devcapsule` checkout holds amd64 VSCodium pins labelled arm64, written +by the bug since fixed; re-choosing its IDE through DevCapsule commands repairs it. + +**2026-11-04, after the fixes.** A personal pin is remapped when its table is exactly +another platform's embedded pin, whatever its label, so the human's checkout healed without +a command (4d2aeb28): their `~/hello-devcapsule` capsule runs natively, `aarch64`, an arm64 +VSCodium. The unit suite in Linux containers: amd64, emulated, 2,770 passed, 4 failed; arm64, +native, 2,757 passed, 17 failed. Both fail three recursive-orchestrator tests that need a host +X11 directory, as `main` does in a headless container now that the skip is Linux-only. The +amd64 run fails a console test whose 0.3 s timeout emulated Python misses; it passes natively. +The arm64 run fails 14 console-channel tests whose real `devcapsule` subprocess sees the +arm64 host while the suite's fixtures are amd64: a limit of the suite on ARM Linux, not CI. +The IDE smoke takes `DEVCAPSULE_SMOKE_PLATFORM` to create its project for a platform. + +**2026-11-04, native smokes and a second review.** One at a time, each after a check of free +space and followed by removing its images, every IDE passed natively as `linux-arm64` with +the local base: VSCodium, VS Code, Eclipse, PyCharm, Rider and IntelliJ IDEA, runs +20261104T120158Z-e8f78f to 20261104T121007Z-289109. The JetBrains IDEs were seen at their +splash window. The six are the `linux-arm64` matrix's first verified edges; the .NET SDK and +Playwright rode along unexercised and stay unverified. A second review of all changes since +`main`, and a review against the codebase's own conventions, followed; the design note's +section 5 records both, and 9d285a5a their fixes: `project --platform` and its module-global +selection are gone and `project init --platform` adds a lock; an owner's edit regenerates +every lock; the lock's platform is required below the configuration; the sudo-policy sweep +spares a concurrent launch's volume; comments, messages and tests follow the surrounding +code. Unit suite on this Mac: 2,760 passed, 25 skipped; in a `linux/amd64` container, 2,779 +passed and 5 failed, the same five recursive tests that fail on `main` there, for want of a +host X11 directory, a non-root user and a Docker group. From 9d285a5a, VSCodium passed +natively (20261104T130201Z-392364, created with `init --platform linux-arm64` and verified +without `--unverified`) and under emulation (20261104T130042Z-36f56a). + +**2026-11-04, init asks for the platform.** The human's first init of a new project on the +Mac chose `linux-amd64` without a question; they ruled that init ask on a host of several +platforms, the native one by default ("depending on the current os just ask", "default +arm64 on arm"). `Platform.current()` is now the native platform, and init asks through its +elicitor before the identity questions: Enter, or a run without a terminal, takes it. A +project with a lock the host runs is not asked, and a Linux host never is. + +## Next Resumable Task + +Publish this workstream and `macos-launcher` once the SSH agent holds a key, and hand both +pull requests to the human, the macOS one first. The owner's rulings and the arm64 base's +publication follow. + +## Open Threads + +- The SSH agent on this Mac holds no identity, so the workstream cannot be published or + pushed; registration is local, as is `macos-launcher`'s. +- This branch builds on `ws-macos-launcher/host-port`, which is not yet on `main`. The + registration commit is forked from `main` as topic 7.5 requires, and the macOS branch is + merged after it; this pull request follows that one. +- An arm64 base on `docker.io/mycodespaceai`, and its digest in the matrix, need the owner. + The pin holds the digest a push of this Mac's exact image keeps; a rebuild must re-pin. +- The libsoup 404 on amd64 goes to `maintenance` by intake mail once the coordination branch + can be pushed. +- Docker access from an arm64 capsule reaches an arm64 daemon either way; on amd64 capsules + under emulation it also defaults to arm64 images (`docs/platforms/macos.md`). +- A checkout keeps one base-image answer. Deleting the arm64 lock to return to emulation + needs the base answered again, as the macOS page says. + +## Document Index + +- [2026-11-04-design-linux-arm64.md](2026-11-04-design-linux-arm64.md): what the second + platform needs, the decisions, and the slices. Open before each slice. diff --git a/engineering-docs/wip/2026-11-04-linux-arm64/intake-dispositions.md b/engineering-docs/wip/2026-11-04-linux-arm64/intake-dispositions.md new file mode 100644 index 00000000..c67569ac --- /dev/null +++ b/engineering-docs/wip/2026-11-04-linux-arm64/intake-dispositions.md @@ -0,0 +1,4 @@ +# Intake Decisions: `linux-arm64` + +Record each removed intake item, its decision, reasoning, and resulting +status file requirement or task here. diff --git a/engineering-docs/wip/2026-11-04-linux-arm64/intake/README.md b/engineering-docs/wip/2026-11-04-linux-arm64/intake/README.md new file mode 100644 index 00000000..69ba7553 --- /dev/null +++ b/engineering-docs/wip/2026-11-04-linux-arm64/intake/README.md @@ -0,0 +1,9 @@ +# Intake: `linux-arm64` + +Work other workstreams have delivered to `linux-arm64` and that it has not +yet decided. Senders add one file per item, named +`YYYY-MM-DD--.md`, and deliver it by mail on the +coordination branch, never through `main` or by editing this directory. Each +item is acknowledged into the status file or forwarded as `WORKFLOW.md` +permits, and every decision is recorded in `intake-dispositions.md` before the +item is removed. diff --git a/index.md b/index.md index 5698399e..2e2516bd 100644 --- a/index.md +++ b/index.md @@ -178,6 +178,8 @@ status file; internal WIP/archive documents use the local index in that status. - [Maintenance current status](engineering-docs/wip/2026-09-18-maintenance/CURRENT-STATUS.md) - [Capsule web console current status](engineering-docs/wip/2026-10-09-capsule-webconsole/CURRENT-STATUS.md) - [Wrapped projects current status](engineering-docs/wip/2026-11-02-wrapped-projects/CURRENT-STATUS.md) +- [macOS launcher current status](engineering-docs/wip/2026-11-03-macos-launcher/CURRENT-STATUS.md) +- [linux-arm64 capsules current status](engineering-docs/wip/2026-11-04-linux-arm64/CURRENT-STATUS.md) - [Sample demo projects current status](engineering-docs/wip/2026-08-14-sample-projects/CURRENT-STATUS.md) - [Contained display current status](engineering-docs/wip/2026-08-19-contained-display/CURRENT-STATUS.md)