From 81c7b92a600765ee8a5fb9183e3b96118a442104 Mon Sep 17 00:00:00 2001 From: Costin Cozianu Date: Mon, 16 Nov 2026 03:28:01 +0000 Subject: [PATCH 01/43] Register GitHub authentication proxy workstream Co-authored-by: GPT-6 Codex --- CURRENT-STATUS.md | 1 + .../2026-11-16-github-auth/CURRENT-STATUS.md | 51 +++++++++++++++++++ .../intake-dispositions.md | 3 ++ .../2026-11-16-github-auth/intake/README.md | 3 ++ index.md | 1 + 5 files changed, 59 insertions(+) create mode 100644 engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md create mode 100644 engineering-docs/wip/2026-11-16-github-auth/intake-dispositions.md create mode 100644 engineering-docs/wip/2026-11-16-github-auth/intake/README.md diff --git a/CURRENT-STATUS.md b/CURRENT-STATUS.md index 33733d6e..0d55f14f 100644 --- a/CURRENT-STATUS.md +++ b/CURRENT-STATUS.md @@ -9,6 +9,7 @@ coordination branch. Detailed state belongs in each linked status file. | Name | Started | Goal | State | Branch association | Status file | |---|---|---|---|---|---| +| `github-auth` | 2026-11-16 | GitHub SSH authentication through a launcher-owned host proxy | active; implementation and Gemini Hydrogen Pair Review | `ws-github-auth/ssh-proxy` | [workstream status](engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md) | | `workflow-improvements` | 2026-08-09 | Improve the multiple-stream workflow from concrete dogfood findings during the recursive E2E cycle | active 2026-10-30; Pair Review Loop naming implemented and packaging verified for 0.3.0; mainline and release integration pending | `ws-workflow-improvements/v1` | [workstream status](engineering-docs/wip/2026-08-09-workflow-improvements/CURRENT-STATUS.md) | | `project-management` | 2026-08-09 | Maintain project-wide priorities, sequencing, dependencies, and lifecycle coordination | paused 2026-10-11 at owner direction; consent and forced configuration before RC work; remaining diagnostics/image scope and release cut decisions retained | `ws-project-management/coordination` | [workstream status](engineering-docs/wip/2026-08-09-project-management/CURRENT-STATUS.md) | | `maintenance` | 2026-09-18 | Own the defects no open workstream covers, on `main` and on maintained release lines, and drive maintenance releases | active; releasing 0.3.1; cut 2026-11-07 from `main` at `31e8fec7`, `main` reopened at `0.3.2.dev0` (#8); candidate 2 (`v0.3.1-rc2` on `be0140ca`, propagated to `main` via PR #22 `aabace3c`) published and verified: Eclipse libsoup (#10, closed on rc1), launcher-delivered `devcapsule_webconsole` (`D-0017`, #18/#16, closed on rc2), Codex npm abbreviated metadata (#20, closed on rc2 + `component-status` `0bc31c64`), and Codex `0.167.0` + Claude Code `2.1.321` pins (`embedded-33`, #21); rulings open: executables for the Mac, the compatibility feed's home | `release-0.3.1`, the workstream's selection from the cut to the final tag; `release-0.3.0` closed and retained; earlier slice branches merged and closed | [workstream status](engineering-docs/wip/2026-09-18-maintenance/CURRENT-STATUS.md) | diff --git a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md new file mode 100644 index 00000000..2286b890 --- /dev/null +++ b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md @@ -0,0 +1,51 @@ +# Workstream Current Status: GitHub Authentication + +Mnemonic: `github-auth` + +Start date: 2026-11-16 + +State: active; first implementation slice. + +Branch association: `ws-github-auth/ssh-proxy`, from canonical `upstream/main` at `1942f396`. + +Integration target: `main` + +Delivery method: pull request with Pair Review; the human merges. + +Requirements: R-PRODUCT-002 (explicit host boundaries), R-PRODUCT-003 (durable memory). + +## Goal and accepted scope + +GitHub Git operations inside a capsule use the host's existing SSH authentication +through a proxy owned by the outside launcher. Private keys remain on the host. +The owner selected the new `github-auth` workstream and Git over SSH first on +2026-11-16 through the console decision. HTTPS and `gh` API authentication are +outside this slice. Review partner: Antigravity, `gemini-4-hydrogen-high`. + +## Current state + +A clean checkout at `/home/devcapsule/worktrees/github-auth` preserves the previous +checkout, its maintenance records commit, and its independent website gitlink. +The branch starts at fetched canonical main: no synchronization is required now. +No open bug is owned by this newly registered workstream. Mail and intake are empty. +The new feature belongs here, not in the maintenance release. + +## Planned next step + +Implement a constrained GitHub Git transport broker, capsule adapter, explicit +host authorization, and lifecycle integration. Validate the transport and refusal +paths, then run Pair Review with Gemini 4 Hydrogen High. + +## Open Threads + +- Awaiting the human: none for workstream or first delivery scope; both accepted. +- Engineering choices: initial transport must account for Linux/rootless Docker + permissions; Docker Desktop cannot bind ordinary host Unix sockets. +- GitHub CLI is absent at the policy-prescribed path. PR creation/merging remains + owner-operated under WORKFLOW-LOCAL.md; prepare reviewed commits and compare links. +- Deliberately not preserved: no credentials or console decision tokens in Git. + +## Workstream document index + +- [Intake](intake/README.md) +- [Intake decisions](intake-dispositions.md) diff --git a/engineering-docs/wip/2026-11-16-github-auth/intake-dispositions.md b/engineering-docs/wip/2026-11-16-github-auth/intake-dispositions.md new file mode 100644 index 00000000..0e175eff --- /dev/null +++ b/engineering-docs/wip/2026-11-16-github-auth/intake-dispositions.md @@ -0,0 +1,3 @@ +# Intake decisions + +No items received. diff --git a/engineering-docs/wip/2026-11-16-github-auth/intake/README.md b/engineering-docs/wip/2026-11-16-github-auth/intake/README.md new file mode 100644 index 00000000..42353b5f --- /dev/null +++ b/engineering-docs/wip/2026-11-16-github-auth/intake/README.md @@ -0,0 +1,3 @@ +# GitHub authentication intake + +No items pending. Deliver items through workflow mail. diff --git a/index.md b/index.md index 8c020aaf..5d2f5e21 100644 --- a/index.md +++ b/index.md @@ -186,6 +186,7 @@ status file; internal WIP/archive documents use the local index in that status. - [Project management current status](engineering-docs/wip/2026-08-09-project-management/CURRENT-STATUS.md) - [Workflow improvements current status](engineering-docs/wip/2026-08-09-workflow-improvements/CURRENT-STATUS.md) - [Component upgrades current status](engineering-docs/wip/2026-09-21-component-upgrades/CURRENT-STATUS.md) +- [GitHub authentication current status](engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md) - [Maintenance current status](engineering-docs/wip/2026-09-18-maintenance/CURRENT-STATUS.md) - [Capsule web console current status](engineering-docs/wip/2026-10-09-capsule-webconsole/CURRENT-STATUS.md) - [Wrapped projects current status](engineering-docs/wip/2026-11-02-wrapped-projects/CURRENT-STATUS.md) From 52d574a9bb6406dcc25299c0db31d582f134db11 Mon Sep 17 00:00:00 2001 From: Costin Cozianu Date: Mon, 16 Nov 2026 03:37:33 +0000 Subject: [PATCH 02/43] Add launcher-owned GitHub SSH transport proxy Keep host credentials outside the capsule; gate Git transfer access through github-ssh authorization. Include binary transport, Git process and admission tests. Co-authored-by: GPT-6 Codex --- .../devcapsule/commands/github_ssh.py | 37 +++ devcapsule-src/devcapsule/commands/project.py | 3 +- .../devcapsule/configuration/authorization.py | 5 + .../configuration/console_channel.py | 1 + .../devcapsule/configuration/listing.py | 2 +- .../devcapsule/configuration/resolution.py | 2 +- .../devcapsule/configuration/review.py | 4 +- devcapsule-src/devcapsule/github_proxy.py | 262 ++++++++++++++++++ devcapsule-src/devcapsule/github_transport.py | 102 +++++++ .../devcapsule/launch/pycharm/_launcher.py | 37 ++- .../tests/integration/test_github_proxy.py | 161 +++++++++++ .../configuration/test_authorization.py | 8 + .../devcapsule/configuration/test_review.py | 20 ++ .../unit/devcapsule/test_github_transport.py | 68 +++++ docs/containment/the-boundary.md | 5 +- docs/everyday-development/git.md | 44 ++- .../2026-11-16-github-auth/CURRENT-STATUS.md | 21 ++ .../evidence/review-brief.md | 87 ++++++ 18 files changed, 858 insertions(+), 11 deletions(-) create mode 100644 devcapsule-src/devcapsule/commands/github_ssh.py create mode 100644 devcapsule-src/devcapsule/github_proxy.py create mode 100644 devcapsule-src/devcapsule/github_transport.py create mode 100644 devcapsule-src/tests/integration/test_github_proxy.py create mode 100644 devcapsule-src/tests/unit/devcapsule/configuration/test_review.py create mode 100644 devcapsule-src/tests/unit/devcapsule/test_github_transport.py create mode 100644 engineering-docs/wip/2026-11-16-github-auth/evidence/review-brief.md diff --git a/devcapsule-src/devcapsule/commands/github_ssh.py b/devcapsule-src/devcapsule/commands/github_ssh.py new file mode 100644 index 00000000..e36fd1fc --- /dev/null +++ b/devcapsule-src/devcapsule/commands/github_ssh.py @@ -0,0 +1,37 @@ +"""Git's capsule-side SSH adapter for the authorized launcher proxy.""" +from __future__ import annotations + +import argparse +import os +from pathlib import Path +import subprocess +import sys + +from devcapsule.commands.framework import Command +from devcapsule.compat import CliError +from devcapsule.github_transport import GitHubTransportError, parse_git_ssh +from devcapsule.github_proxy import SOCKET_ENV, relay_git + + +class GitHubSshCommand(Command): + name = "github-ssh" + hidden = True + help = "Git SSH transport through the authorized host GitHub proxy." + passthrough_dest = "ssh_arguments" + + @classmethod + def run(cls, arguments: argparse.Namespace, context: object | None) -> int: + try: + request = parse_git_ssh(arguments.ssh_arguments) + if request is None: + return subprocess.run(["ssh", *arguments.ssh_arguments], check=False).returncode + path = os.environ.get(SOCKET_ENV) + if not path: + raise GitHubTransportError("No authorized GitHub proxy. Enable github-ssh on the host launcher.") + return relay_git(Path(path), request, stdin=sys.stdin.buffer, + stdout=sys.stdout.buffer, stderr=sys.stderr.buffer) + except (OSError, GitHubTransportError) as error: + raise CliError(f"GitHub proxy: {error} Host keys must be trusted and encrypted keys unlocked on the host.") from error + + +COMMAND = GitHubSshCommand diff --git a/devcapsule-src/devcapsule/commands/project.py b/devcapsule-src/devcapsule/commands/project.py index 48253ed6..6b0b2541 100644 --- a/devcapsule-src/devcapsule/commands/project.py +++ b/devcapsule-src/devcapsule/commands/project.py @@ -2164,7 +2164,7 @@ def subcommands(cls) -> Mapping[str, type[Command] | type[Group]]: # The authorization nodes whose run-once answers feed the launch plan; every # other authorization (base-image, acquisitions) is inherently persistent. -_RUN_ONCE_AUTHORIZATIONS = ("docker-daemon", "network", "development-sudo", "host-browser", "host-x11") +_RUN_ONCE_AUTHORIZATIONS = ("docker-daemon", "network", "development-sudo", "host-browser", "host-x11", "github-ssh") class ProjectRunCommand(Command): @@ -2468,6 +2468,7 @@ def prepare_display(base: ImageDetails) -> None: extra_docker_args=["--pull=never", *saved_docker_args, *prepared.environment_tokens, *docker_options], project_state=None, enable_host_browser=selected_host_browser, + enable_github_ssh=host_access.github_ssh, display_transport=display_transport, web_console=web_console, capsule_platform=lock_platform(lock), diff --git a/devcapsule-src/devcapsule/configuration/authorization.py b/devcapsule-src/devcapsule/configuration/authorization.py index 3927bfaf..a7b54a9d 100644 --- a/devcapsule-src/devcapsule/configuration/authorization.py +++ b/devcapsule-src/devcapsule/configuration/authorization.py @@ -80,6 +80,11 @@ True, "Enable the declared development-sudo behavior inside the capsule.", ), + "github-ssh": ( + True, + "Allow GitHub Git fetch and push through the outside launcher using the host SSH identity. " + "All repositories accessible to that identity are accessible; private keys stay on the host.", + ), "host-browser": ( True, "Open HTTP(S) links from the capsule in the physical host's default browser " diff --git a/devcapsule-src/devcapsule/configuration/console_channel.py b/devcapsule-src/devcapsule/configuration/console_channel.py index b66436ff..eef5dc4e 100644 --- a/devcapsule-src/devcapsule/configuration/console_channel.py +++ b/devcapsule-src/devcapsule/configuration/console_channel.py @@ -73,6 +73,7 @@ DEFERRED_SUBVERBS = {("trust", "accept"), ("docker-args", "set")} SECURE_AUTHORIZATIONS = { "network": {"bridge", "none"}, "docker-daemon": {"none"}, + "github-ssh": {"false", "none"}, "development-sudo": {"false", "none"}, "host-browser": {"false", "none"}, "host-x11": {"false", "none"}, } READ_ONLY_VERBS = {"show", "check", "console"} diff --git a/devcapsule-src/devcapsule/configuration/listing.py b/devcapsule-src/devcapsule/configuration/listing.py index df7bd6dc..07b81f8f 100644 --- a/devcapsule-src/devcapsule/configuration/listing.py +++ b/devcapsule-src/devcapsule/configuration/listing.py @@ -61,7 +61,7 @@ LISTING_SCHEMA_VERSION = 2 -HOST_ACCESS_NODES = ("network", "docker-daemon", "development-sudo", "host-browser", "host-x11") +HOST_ACCESS_NODES = ("network", "docker-daemon", "development-sudo", "host-browser", "host-x11", "github-ssh") @dataclass(frozen=True) diff --git a/devcapsule-src/devcapsule/configuration/resolution.py b/devcapsule-src/devcapsule/configuration/resolution.py index 25e97671..bfa143dc 100644 --- a/devcapsule-src/devcapsule/configuration/resolution.py +++ b/devcapsule-src/devcapsule/configuration/resolution.py @@ -210,7 +210,7 @@ def base_reference(self) -> str | None: @property def host_access(self) -> HostAccess: authorizations = self._document.get("authorization", {}) - host_names = {"docker-daemon", "network", "development-sudo", "host-browser", "host-x11"} + host_names = {"docker-daemon", "network", "development-sudo", "host-browser", "host-x11", "github-ssh"} return HostAccess().overlay(self._document.get("host", {})).overlay({ name: value for name, value in authorizations.items() if name in host_names }) diff --git a/devcapsule-src/devcapsule/configuration/review.py b/devcapsule-src/devcapsule/configuration/review.py index 4d1e6ec5..2a14a488 100644 --- a/devcapsule-src/devcapsule/configuration/review.py +++ b/devcapsule-src/devcapsule/configuration/review.py @@ -27,6 +27,7 @@ class HostAccess: docker_daemon: str = "none" network: str = "bridge" development_sudo: bool = False + github_ssh: bool = False host_browser: bool = False host_x11: bool | None = None @@ -35,6 +36,7 @@ def overlay(self, answers: Mapping[str, Any]) -> HostAccess: "docker-daemon": ("docker_daemon", ("none", "host-socket")), "network": ("network", ("bridge", "host")), "development-sudo": ("development_sudo", (False, True)), + "github-ssh": ("github_ssh", (False, True)), "host-browser": ("host_browser", (False, True)), "host-x11": ("host_x11", (False, True)), } @@ -127,7 +129,7 @@ def resolved_authorizations(self) -> dict[str, AuthorizationScalar]: def effective_host(self, overrides: Mapping[str, Any] | None = None) -> HostAccess: answers = { name: value for name, value in self.resolved_authorizations().items() - if name in {"docker-daemon", "network", "development-sudo", "host-browser", "host-x11"} + if name in {"docker-daemon", "network", "development-sudo", "host-browser", "host-x11", "github-ssh"} } return self.legacy_host.overlay(answers).overlay(overrides or {}) diff --git a/devcapsule-src/devcapsule/github_proxy.py b/devcapsule-src/devcapsule/github_proxy.py new file mode 100644 index 00000000..83d2ba40 --- /dev/null +++ b/devcapsule-src/devcapsule/github_proxy.py @@ -0,0 +1,262 @@ +"""Launcher-owned GitHub transport broker and capsule client. + +``GitHubBroker`` owns a private Unix socket and bounded concurrent SSH children. +``relay_git`` connects Git's byte streams to it. Only the launcher creates the +broker; neither side reads private keys. The wire begins with a bounded JSON +request. Client input then streams unchanged, with a write-half close for EOF. +Server replies are frames: O stdout, E stderr, X unsigned exit status. +""" +from __future__ import annotations + +from collections.abc import Iterator, Mapping +from contextlib import contextmanager +import os +from pathlib import Path +import selectors +import shutil +import signal +import socket +import struct +import subprocess +import tempfile +import threading +import time +from typing import BinaryIO + +from devcapsule.github_transport import GitHubRequest, GitHubTransportError, decode_request + +__all__ = ["GitHubBroker", "relay_git", "github_ssh_bridge", "SOCKET_ENV", "SOCKET_DESTINATION", + "GIT_SSH_COMMAND", "MAX_REQUEST", "MAX_FRAME"] + +SOCKET_ENV = "DEVCAPSULE_GITHUB_SOCKET" +SOCKET_DESTINATION = Path("/run/devcapsule-github.sock") +GIT_SSH_COMMAND = "/opt/devcapsule/bin/devcapsule.pex github-ssh --" +MAX_REQUEST = 4096 +MAX_FRAME = 65536 + + +def _read_exact(connection: socket.socket, length: int) -> bytes: + chunks = bytearray() + while len(chunks) < length: + chunk = connection.recv(length - len(chunks)) + if not chunk: + raise GitHubTransportError("GitHub proxy connection ended unexpectedly.") + chunks.extend(chunk) + return bytes(chunks) + + +def _send_frame(connection: socket.socket, kind: bytes, payload: bytes) -> None: + connection.sendall(kind + struct.pack("!I", len(payload)) + payload) + + +class GitHubBroker: + """One session's GitHub bridge. Close cancels active children and removes its socket. + + ``ssh`` and ``environment`` come from the launcher, never from a request. + Explicit deadlines bound idle handshakes, blocked consumers and Git operations. + """ + + def __init__(self, path: Path, *, ssh: str, environment: Mapping[str, str], + operation_timeout: float = 3600, io_timeout: float = 10) -> None: + self.path = path + self.ssh = ssh + self.environment = dict(environment) + self.operation_timeout = operation_timeout + self.io_timeout = io_timeout + self._stop = threading.Event() + self._lock = threading.Lock() + self._connections: set[socket.socket] = set() + self._threads: list[threading.Thread] = [] + self._slots = threading.BoundedSemaphore(8) + self._server: socket.socket | None = None + self._listener: threading.Thread | None = None + self._bound = False + + def __enter__(self) -> GitHubBroker: + server = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + try: + server.bind(str(self.path)) # Never unlink or replace someone else's socket. + self._bound = True + self.path.chmod(0o600) + server.listen(8) + server.settimeout(0.2) + except BaseException: + server.close() + if self._bound: + self.path.unlink(missing_ok=True) + raise + self._server = server + self._listener = threading.Thread(target=self._serve, daemon=True, name="github-proxy") + self._listener.start() + return self + + def __exit__(self, *_args: object) -> None: + self._stop.set() + if self._server is not None: + self._server.close() + if self._listener is not None: + self._listener.join() + with self._lock: + for connection in self._connections: + try: + connection.shutdown(socket.SHUT_RDWR) + except OSError: + pass + for thread in self._threads: + thread.join() + if self._bound: + self.path.unlink(missing_ok=True) + + def _serve(self) -> None: + assert self._server is not None + while not self._stop.is_set(): + try: + connection, _ = self._server.accept() + except TimeoutError: + continue + except OSError: + break + if not self._slots.acquire(blocking=False): + connection.close() + continue + with self._lock: + self._connections.add(connection) + thread = threading.Thread(target=self._handle, args=(connection,), daemon=True) + # Retain only active workers; completed Thread objects need not accumulate. + self._threads = [item for item in self._threads if item.is_alive()] + self._threads.append(thread) + thread.start() + + def _handle(self, connection: socket.socket) -> None: + try: + with connection: + connection.settimeout(self.io_timeout) + length = struct.unpack("!I", _read_exact(connection, 4))[0] + if not 0 < length <= MAX_REQUEST: + raise GitHubTransportError("GitHub request exceeds its size limit.") + request = decode_request(_read_exact(connection, length)) + self._transfer(connection, request) + except (OSError, GitHubTransportError, subprocess.SubprocessError): + # An invalid or disconnected caller gets EOF, never a host command. + pass + finally: + with self._lock: + self._connections.discard(connection) + self._slots.release() + + def _transfer(self, connection: socket.socket, request: GitHubRequest) -> None: + # Timeout sockets use O_NONBLOCK. Child stdin must be a blocking stream. + # dup shares file flags, so restore blocking and use send timeouts via + # a separate connection wrapper only while transmitting output below. + connection.setblocking(True) + with subprocess.Popen(request.ssh_command(self.ssh), stdin=connection.fileno(), + stdout=subprocess.PIPE, stderr=subprocess.PIPE, + env=self.environment, cwd="/", start_new_session=True) as child: + assert child.stdout is not None and child.stderr is not None + started = time.monotonic() + try: + with selectors.DefaultSelector() as selector: + selector.register(child.stdout, selectors.EVENT_READ, b"O") + selector.register(child.stderr, selectors.EVENT_READ, b"E") + while selector.get_map(): + if self._stop.is_set() or time.monotonic() - started > self.operation_timeout: + raise GitHubTransportError("GitHub transfer stopped or timed out.") + for key, _ in selector.select(0.2): + data = os.read(key.fd, MAX_FRAME) + if not data: + selector.unregister(key.fileobj) + else: + self._send(connection, key.data, data) + while child.poll() is None: + if self._stop.wait(0.1) or time.monotonic() - started > self.operation_timeout: + raise GitHubTransportError("GitHub transfer stopped or timed out.") + self._send(connection, b"X", struct.pack("!I", child.returncode if child.returncode >= 0 else 255)) + finally: + # SSH may have started a host-configured ProxyCommand. Reap the + # whole private process group even after the leader exits. + try: + os.killpg(child.pid, signal.SIGKILL) + except ProcessLookupError: + pass + child.wait() + + def _send(self, connection: socket.socket, kind: bytes, payload: bytes) -> None: + # MSG_DONTWAIT avoids altering the shared file flags of child stdin. + data = memoryview(kind + struct.pack("!I", len(payload)) + payload) + deadline = time.monotonic() + self.io_timeout + with selectors.DefaultSelector() as selector: + selector.register(connection, selectors.EVENT_WRITE) + while data: + if self._stop.is_set() or time.monotonic() > deadline: + raise GitHubTransportError("GitHub proxy output stalled.") + if not selector.select(0.2): + continue + try: + count = connection.send(data, socket.MSG_DONTWAIT) + except BlockingIOError: + continue + if count == 0: + raise GitHubTransportError("GitHub proxy output disconnected.") + data = data[count:] + + +def relay_git(path: Path, request: GitHubRequest, *, stdin: BinaryIO, + stdout: BinaryIO, stderr: BinaryIO) -> int: + """Relay binary Git data, separate diagnostics and the host SSH exit status.""" + payload = request.encode() + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as connection: + connection.settimeout(10) + connection.connect(str(path)) + connection.sendall(struct.pack("!I", len(payload)) + payload) + connection.settimeout(None) + + def send_input() -> None: + try: + while chunk := os.read(stdin.fileno(), MAX_FRAME): + connection.sendall(chunk) + connection.shutdown(socket.SHUT_WR) + except (OSError, ValueError): + pass # The server can finish before Git closes stdin. + + threading.Thread(target=send_input, daemon=True, name="github-git-input").start() + while True: + kind = _read_exact(connection, 1) + length = struct.unpack("!I", _read_exact(connection, 4))[0] + if length > MAX_FRAME: + raise GitHubTransportError("GitHub proxy frame exceeds its size limit.") + data = _read_exact(connection, length) + if kind == b"X" and length == 4: + return min(struct.unpack("!I", data)[0], 255) + if kind not in {b"O", b"E"}: + raise GitHubTransportError("Unknown GitHub proxy response.") + output = stdout if kind == b"O" else stderr + output.write(data) + output.flush() + + +@contextmanager +def github_ssh_bridge(environment: Mapping[str, str], *, enabled: bool, + containerized: bool, binds_unix_sockets: bool) -> Iterator[Path | None]: + """Create an authorized physical-host broker, or reuse an inherited one. + + Docker Desktop needs a separate transport. An explicit enable there fails + with an explanation instead of silently mounting keys or exposing an agent. + """ + if not enabled: + yield None + return + if containerized: + inherited = environment.get(SOCKET_ENV) + if inherited != str(SOCKET_DESTINATION) or not SOCKET_DESTINATION.is_socket(): + raise GitHubTransportError("GitHub proxy unavailable: authorize github-ssh on the outside launcher first.") + yield SOCKET_DESTINATION + return + if not binds_unix_sockets: + raise GitHubTransportError("github-ssh currently requires Linux/WSL with native Docker; Docker Desktop transport is not implemented.") + executable = shutil.which("ssh", path=environment.get("PATH", os.defpath)) + if executable is None: + raise GitHubTransportError("github-ssh requires OpenSSH on the host.") + with tempfile.TemporaryDirectory(prefix="devcapsule-github-", dir="/tmp") as directory: + path = Path(directory) / "ssh.sock" + with GitHubBroker(path, ssh=executable, environment=environment): + yield path diff --git a/devcapsule-src/devcapsule/github_transport.py b/devcapsule-src/devcapsule/github_transport.py new file mode 100644 index 00000000..b53d5210 --- /dev/null +++ b/devcapsule-src/devcapsule/github_transport.py @@ -0,0 +1,102 @@ +"""GitHub-only Git SSH requests: validate values before any host process starts. + +The capsule selects a Git service and repository, never a host command or SSH +option. Host OpenSSH owns key selection, host verification and authentication. +""" +from __future__ import annotations + +from dataclasses import dataclass +import json +import re +import shlex +from typing import Sequence + +__all__ = ["GitHubRequest", "GitHubTransportError", "parse_git_ssh", "decode_request"] + + +class GitHubTransportError(ValueError): + """A request cannot cross the authorized GitHub boundary.""" + + +@dataclass(frozen=True) +class GitHubRequest: + service: str + repository: str + + def validate(self) -> None: + """Require a GitHub repository and one of Git's two transfer services.""" + if self.service not in {"git-upload-pack", "git-receive-pack"}: + raise GitHubTransportError("Only Git fetch and push services are allowed.") + if (len(self.repository) > 256 or + not re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9-]*/[A-Za-z0-9_.-]+", self.repository) or + self.repository.split("/")[-1] in {".", ".."}): + raise GitHubTransportError("Expected a GitHub OWNER/REPOSITORY path.") + + def encode(self) -> bytes: + """Serialize a validated request; no host paths or credentials cross.""" + self.validate() + return json.dumps({"version": 1, "service": self.service, + "repository": self.repository}, separators=(",", ":")).encode() + + def ssh_command(self, executable: str) -> tuple[str, ...]: + """Use host SSH configuration with noninteractive, verified authentication.""" + self.validate() + return (executable, "-T", "-oBatchMode=yes", "-oStrictHostKeyChecking=yes", + "-oClearAllForwardings=yes", "-oForwardAgent=no", "-oPermitLocalCommand=no", + "-oControlMaster=no", "-oControlPath=none", "-oRemoteCommand=none", + "git@github.com", f"{self.service} '{self.repository}'") + + +def decode_request(payload: bytes) -> GitHubRequest: + """Validate the entire wire request, including field types and version.""" + try: + value = json.loads(payload) + except (ValueError, UnicodeError) as error: + raise GitHubTransportError("Invalid GitHub request JSON.") from error + if (not isinstance(value, dict) or set(value) != {"version", "service", "repository"} or + type(value["version"]) is not int or value["version"] != 1 or + not isinstance(value["service"], str) or not isinstance(value["repository"], str)): + raise GitHubTransportError("Unsupported GitHub request.") + request = GitHubRequest(value["service"], value["repository"]) + request.validate() + return request + + +def parse_git_ssh(arguments: Sequence[str]) -> GitHubRequest | None: + """Parse Git's OpenSSH invocation; other hosts use the capsule's own SSH. + + Git's protocol-v2 SendEnv request is accepted, but the broker deliberately + negotiates protocol v0 rather than accepting arbitrary client environment. + An explicit port is accepted only when it is GitHub's standard SSH port. + Host SSH configuration can select ssh.github.com:443 as usual. + """ + args = list(arguments) + port: str | None = None + while args and args[0].startswith("-"): + option = args.pop(0) + if option in {"-o", "-p"} and args: + value = args.pop(0) + if option == "-p": + port = value + elif value != "SendEnv=GIT_PROTOCOL": + raise GitHubTransportError("Unsupported SSH option for the GitHub proxy.") + elif option == "-oSendEnv=GIT_PROTOCOL": + continue + else: + raise GitHubTransportError("Unsupported SSH option for the GitHub proxy.") + if not args: + raise GitHubTransportError("Missing SSH destination.") + destination = args.pop(0) + if destination not in {"github.com", "git@github.com"}: + return None + if port not in {None, "22"} or len(args) != 1: + raise GitHubTransportError("Expected GitHub's SSH destination and one Git service.") + try: + command = shlex.split(args[0]) + except ValueError as error: + raise GitHubTransportError("Malformed Git service command.") from error + if len(command) != 2: + raise GitHubTransportError("Expected a Git service and repository only.") + request = GitHubRequest(command[0], command[1].removeprefix("/")) + request.validate() + return request diff --git a/devcapsule-src/devcapsule/launch/pycharm/_launcher.py b/devcapsule-src/devcapsule/launch/pycharm/_launcher.py index 97bfdb90..283c5ac6 100644 --- a/devcapsule-src/devcapsule/launch/pycharm/_launcher.py +++ b/devcapsule-src/devcapsule/launch/pycharm/_launcher.py @@ -61,12 +61,18 @@ ) from ...host_daemon import ( current_container, + in_container, daemon_architecture, docker_socket, is_host_backed, requires_translation, translate_bind_sources, ) +from ...github_proxy import ( + SOCKET_ENV as GITHUB_SOCKET_ENV, SOCKET_DESTINATION as GITHUB_SOCKET_DESTINATION, + GIT_SSH_COMMAND, github_ssh_bridge, +) +from ...github_transport import GitHubTransportError from ...host_open import ( HOST_OPEN_BROWSER, HOST_OPEN_INTEGRATION, @@ -228,6 +234,7 @@ class PycharmRunOptions: additional_environment: dict[str, str] = field(default_factory=dict) secret_environment: tuple[str, ...] = () extra_docker_args: list[str] = field(default_factory=list) + enable_github_ssh: bool = False enable_host_browser: bool = False # How the surface gets a display. ``host-x11`` is the historical # passthrough (host X socket and session credential bound into the @@ -284,6 +291,7 @@ class PycharmRunConfig: daemon_architecture: str cpus: int | None = None launch_configuration: LaunchConfiguration | None = None + github_socket: Path | None = None host_browser_socket: Path | None = None display_transport: str = HOST_X11_DISPLAY_TRANSPORT # Contained display only: the host loopback port Docker publishes (or the @@ -331,11 +339,18 @@ def run_pycharm(options: PycharmRunOptions, env: Mapping[str, str] | None = None with host_open_bridge( runtime_env, enabled=options.enable_host_browser, - ) as browser_socket: + ) as browser_socket, github_ssh_bridge( + runtime_env, enabled=options.enable_github_ssh, + containerized=in_container(), binds_unix_sockets=HostOs.current().binds_unix_sockets, + ) as github_socket: if browser_socket is None: runtime_env.pop(HOST_OPEN_SOCKET_ENV, None) else: runtime_env[HOST_OPEN_SOCKET_ENV] = str(browser_socket) + if github_socket is None: + runtime_env.pop(GITHUB_SOCKET_ENV, None) + else: + runtime_env[GITHUB_SOCKET_ENV] = str(github_socket) config = build_run_config(options, runtime_env) begin_session_logs(config, runtime_env) prepare_home_mount_points(config) @@ -427,7 +442,7 @@ def run_pycharm(options: PycharmRunOptions, env: Mapping[str, str] | None = None finally: remove_sudoers_policy_volume(files.sudoers_volume, runtime_env) cleanup_temp_runtime_files(files) - except HostOpenError as exc: + except (HostOpenError, GitHubTransportError) as exc: raise PycharmRunError(str(exc)) from exc @@ -1045,6 +1060,14 @@ def build_run_config(options: PycharmRunOptions, env: Mapping[str, str]) -> Pych raise PycharmRunError("The authorized host-browser socket is absent or unsafe.") host_browser_socket = candidate + github_socket: Path | None = None + configured_github_socket = env.get(GITHUB_SOCKET_ENV) + if options.enable_github_ssh and configured_github_socket: + candidate = Path(configured_github_socket) + if not candidate.is_absolute() or ".." in candidate.parts or not is_socket(candidate): + raise PycharmRunError("The authorized GitHub socket is absent or unsafe.") + github_socket = candidate + selected_runtime_plan = options.runtime_plan if host_browser_socket is not None and selected_runtime_plan is not None: selected_runtime_plan = selected_runtime_plan.with_host_integration( @@ -1120,6 +1143,9 @@ def build_run_config(options: PycharmRunOptions, env: Mapping[str, str]) -> Pych raise PycharmRunError( "Component runtime metadata conflicts with the host-browser integration." ) + if github_socket is not None and any(name in fixed_environment or name in options.additional_environment + for name in (GITHUB_SOCKET_ENV, "GIT_SSH_COMMAND", "GIT_SSH_VARIANT")): + raise PycharmRunError("Component environment conflicts with the GitHub SSH integration.") additional_environment: list[tuple[str, str]] = [] for name, value in sorted(options.additional_environment.items()): if not name.isascii() or not name.isidentifier() or name.upper() != name: @@ -1187,6 +1213,7 @@ def build_run_config(options: PycharmRunOptions, env: Mapping[str, str]) -> Pych launch_configuration=options.launch_configuration, use_image_process=options.use_image_process, host_browser_socket=host_browser_socket, + github_socket=github_socket, display_transport=options.display_transport, display_host_port=display_host_port, display_token=display_token, @@ -1417,6 +1444,10 @@ def build_docker_args( ), ] ) + if config.github_socket is not None: + args.extend(["--env", f"{GITHUB_SOCKET_ENV}={GITHUB_SOCKET_DESTINATION}", + "--env", f"GIT_SSH_COMMAND={GIT_SSH_COMMAND}", "--env", "GIT_SSH_VARIANT=ssh", + "--mount", f"type=bind,src={config.github_socket},dst={GITHUB_SOCKET_DESTINATION},ro"]) for name, value in config.additional_environment: args.extend(["--env", f"{name}={value}"]) for name in config.secret_environment: @@ -2261,6 +2292,8 @@ def print_storage_summary(config: PycharmRunConfig) -> None: Host browser integration: Enabled through a URL-only HTTP(S) broker. Any process running as the capsule user can ask the physical host to navigate its default browser.""" + if config.github_socket is not None: + host_browser_disclosure += "\n GitHub SSH: host-authenticated Git fetch/push proxy (all repositories your host identity can access)." host_browser_disclosure += display_disclosure(config) + console_disclosure(config) if config.interactive_state_mounts: component_id = ( diff --git a/devcapsule-src/tests/integration/test_github_proxy.py b/devcapsule-src/tests/integration/test_github_proxy.py new file mode 100644 index 00000000..a8abf31b --- /dev/null +++ b/devcapsule-src/tests/integration/test_github_proxy.py @@ -0,0 +1,161 @@ +"""GitHub proxy process and socket integration; no GitHub account or network is used. + +A fixture SSH executable stands in for the remote SSH service. Git itself and +its upload/receive services are real, as are the proxy, pipes and command line. +""" +from __future__ import annotations + +from contextlib import contextmanager +import os +from pathlib import Path +import shlex +import socket +import struct +import subprocess +import sys +import tempfile +from collections.abc import Iterator + +import pytest + +from devcapsule.github_proxy import GitHubBroker, MAX_REQUEST, SOCKET_ENV + +pytestmark = pytest.mark.integration +SOURCE = Path(__file__).resolve().parents[2] + + +def _environment(root: Path) -> dict[str, str]: + return {'PATH': os.defpath, 'HOME': str(root), 'PYTHONPATH': str(SOURCE), + 'DEVCAPSULE_CONFIG_HOME': str(root / 'config'), 'GIT_CONFIG_NOSYSTEM': '1', + 'GIT_CONFIG_GLOBAL': '/dev/null', 'GIT_AUTHOR_NAME': 'Proxy test', + 'GIT_COMMITTER_NAME': 'Proxy test', 'GIT_AUTHOR_EMAIL': 'test@example.test', + 'GIT_COMMITTER_EMAIL': 'test@example.test', 'GIT_TERMINAL_PROMPT': '0'} + + +def _program(root: Path, body: str) -> Path: + path = root / 'ssh-fixture' + path.write_text(f'#!{sys.executable}\n' + body) + path.chmod(0o700) + return path + + +@contextmanager +def _broker(root: Path, body: str, **kwargs: float) -> Iterator[tuple[Path, dict[str, str]]]: + env = _environment(root) + # A short explicit socket path works under either Unix sockaddr path limit. + with tempfile.TemporaryDirectory(prefix='gh-test-', dir='/tmp') as temporary: + path = Path(temporary) / 's' + with GitHubBroker(path, ssh=str(_program(root, body)), environment=env, **kwargs): + yield path, env + assert not path.exists() + + +def _client(path: Path, env: dict[str, str], payload: bytes = b'') -> subprocess.CompletedProcess[bytes]: + return subprocess.run([sys.executable, '-m', 'devcapsule', 'github-ssh', '--', + 'git@github.com', "git-upload-pack 'acme/repo.git'"], + env={**env, SOCKET_ENV: str(path)}, input=payload, + capture_output=True, timeout=15) + + +def test_binary_streams_and_nonzero_exit_are_preserved(tmp_path: Path) -> None: + """Large binary input, stdout, stderr and failure status survive the proxy without mixing.""" + payload = bytes(range(256)) * 2048 + program = "import sys\ndata = sys.stdin.buffer.read()\nsys.stdout.buffer.write(data)\nsys.stderr.buffer.write(b'diagnostic\\n')\nsys.exit(7)\n" + with _broker(tmp_path, program) as (path, env): + result = _client(path, env, payload) + assert result.returncode == 7 + assert result.stdout == payload + assert result.stderr == b'diagnostic\n' + + +def test_real_git_clone_fetch_and_push_through_capsule_adapter(tmp_path: Path) -> None: + """Real Git clone, fetch and push transfer repository objects through the broker.""" + env = _environment(tmp_path) + bare = tmp_path / 'remote.git' + subprocess.run(['git', 'init', '--bare', '--initial-branch=main', str(bare)], env=env, check=True, capture_output=True) + # This stand-in uses only the validated service; the repository is fixed by the fixture. + program = f"import os, shlex, sys\nservice = shlex.split(sys.argv[-1])[0]\nos.execv('/usr/bin/git', ['git', service.removeprefix('git-'), {str(bare)!r}])\n" + with _broker(tmp_path, program) as (path, _): + git_env = {**env, SOCKET_ENV: str(path), 'GIT_SSH_VARIANT': 'ssh', + 'GIT_SSH_COMMAND': shlex.join([sys.executable, '-m', 'devcapsule', 'github-ssh', '--'])} + clone = tmp_path / 'clone' + def git(*args: str) -> subprocess.CompletedProcess[bytes]: + return subprocess.run(['git', *args], env=git_env, capture_output=True, check=True, timeout=20) + git('-c', 'init.defaultBranch=main', 'clone', 'git@github.com:acme/repo.git', str(clone)) + (clone / 'binary').write_bytes(bytes(range(256)) * 1024) + git('-C', str(clone), 'add', 'binary') + git('-C', str(clone), 'commit', '-m', 'Fixture commit') + git('-C', str(clone), 'push', 'origin', 'main') + second = tmp_path / 'second' + git('clone', 'ssh://git@github.com/acme/repo.git', str(second)) + git('-C', str(second), 'fetch', 'origin') + assert (second / 'binary').read_bytes() == (clone / 'binary').read_bytes() + assert git('--git-dir', str(bare), 'rev-parse', 'main').stdout == git('-C', str(clone), 'rev-parse', 'HEAD').stdout + + +@pytest.mark.parametrize('payload', [b'{"version":1,"service":"sh","repository":"acme/repo"}', + b'{"version":1,"service":"git-upload-pack","repository":"a/b; id"}', + b'{' ]) +def test_invalid_requests_never_start_host_ssh(tmp_path: Path, payload: bytes) -> None: + """Host admission refuses hostile wire bytes even when the capsule adapter is bypassed.""" + marker = tmp_path / 'started' + with _broker(tmp_path, f"from pathlib import Path\nPath({str(marker)!r}).touch()\n") as (path, _): + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as connection: + connection.settimeout(2) + connection.connect(str(path)) + connection.sendall(struct.pack('!I', len(payload)) + payload) + assert connection.recv(1) == b'' + assert not marker.exists() + + +def test_oversized_handshake_is_refused_without_reading_body(tmp_path: Path) -> None: + """An excessive length prefix cannot allocate a body or spawn SSH.""" + with _broker(tmp_path, 'raise AssertionError("must not execute")\n') as (path, _): + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as connection: + connection.settimeout(2) + connection.connect(str(path)) + connection.sendall(struct.pack('!I', MAX_REQUEST + 1)) + assert connection.recv(1) == b'' + + +def test_operation_deadline_stops_a_stalled_ssh_process(tmp_path: Path) -> None: + """A stalled host SSH process is killed and the capsule receives failure within the bound.""" + with _broker(tmp_path, 'import time\ntime.sleep(60)\n', operation_timeout=0.2) as (path, env): + result = _client(path, env) + assert result.returncode != 0 + assert b'connection ended' in result.stderr + + +def test_broker_shutdown_closes_incomplete_requests(tmp_path: Path) -> None: + """Stopping the launcher interrupts a client that never completed its handshake.""" + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as connection: + connection.settimeout(2) + with _broker(tmp_path, 'raise AssertionError("must not execute")\n') as (path, _): + connection.connect(str(path)) + connection.sendall(b'\x00') + try: + assert connection.recv(1) == b'' + except ConnectionResetError: + pass # Closing with unread input may report reset instead of EOF. + + +@pytest.mark.parametrize('enabled', [False, True]) +def test_launcher_exposes_the_socket_only_with_the_grant(tmp_path: Path, enabled: bool) -> None: + """The Docker plan carries the Git adapter and broker mount only after explicit authorization.""" + from devcapsule.launch.pycharm._launcher import PycharmRunOptions, DockerMode, TempRuntimeFiles, build_run_config, build_docker_args + from devcapsule.platforms import Platform + from devcapsule.github_proxy import GIT_SSH_COMMAND, SOCKET_DESTINATION + + project = tmp_path / 'project' + project.mkdir() + with _broker(tmp_path, 'raise AssertionError("no transfer expected")\n') as (path, env): + env = {**env, 'DISPLAY': ':1', 'PYCHARM_GIT_IDENTITY_FROM_HOST': '0', SOCKET_ENV: str(path)} + config = build_run_config(PycharmRunOptions(project=project, capsule_platform=Platform.LINUX_AMD64, + docker_mode=DockerMode.none, enable_sudo=False, network_mode='bridge', + enable_github_ssh=enabled), env) + files = TempRuntimeFiles(xauth_file=tmp_path / 'xauth', passwd_file=tmp_path / 'passwd', group_file=tmp_path / 'group') + args = build_docker_args(config, files, env) + assert (f'GIT_SSH_COMMAND={GIT_SSH_COMMAND}' in args) is enabled + assert (f'{SOCKET_ENV}={SOCKET_DESTINATION}' in args) is enabled + assert (f'type=bind,src={path},dst={SOCKET_DESTINATION},ro' in args) is enabled + assert not any('.ssh' in arg or 'SSH_AUTH_SOCK=' in arg for arg in args) diff --git a/devcapsule-src/tests/unit/devcapsule/configuration/test_authorization.py b/devcapsule-src/tests/unit/devcapsule/configuration/test_authorization.py index 92f49bda..7a8e5c0f 100644 --- a/devcapsule-src/tests/unit/devcapsule/configuration/test_authorization.py +++ b/devcapsule-src/tests/unit/devcapsule/configuration/test_authorization.py @@ -67,3 +67,11 @@ def test_a_lock_without_a_base_declares_no_base_image_decision() -> None: """Precondition: a lock with no base table. Action: build the declarations. Postcondition: there is no base-image declaration to answer.""" assert "base-image" not in authorization_declarations({}, {"components": {}}) + + +def test_github_access_is_available_but_never_recommended_by_omission() -> None: + """A project without recommendations exposes the optional GitHub decision without granting it.""" + declaration = authorization_declarations({}, {"components": {}})['github-ssh'] + assert declaration.recommended_value is True + assert declaration.project_recommended is False + assert declaration.required is False diff --git a/devcapsule-src/tests/unit/devcapsule/configuration/test_review.py b/devcapsule-src/tests/unit/devcapsule/configuration/test_review.py new file mode 100644 index 00000000..638d9e26 --- /dev/null +++ b/devcapsule-src/tests/unit/devcapsule/configuration/test_review.py @@ -0,0 +1,20 @@ +"""HostAccess value contracts for the GitHub transport grant.""" +from devcapsule.configuration.review import HostAccess + + +def test_github_access_is_denied_without_an_explicit_answer() -> None: + """Default host access never exposes host GitHub authentication.""" + assert HostAccess().github_ssh is False + + +def test_a_recorded_denial_overrides_an_earlier_grant() -> None: + """Overlaying a denial withdraws GitHub access without changing other host choices.""" + access = HostAccess(github_ssh=True, host_browser=True) + result = access.overlay({'github-ssh': False}) + assert result.github_ssh is False + assert result.host_browser is True + + +def test_a_run_once_answer_enables_github_access() -> None: + """An explicit boolean grant becomes the launcher's effective GitHub access.""" + assert HostAccess().overlay({'github-ssh': True}).github_ssh is True diff --git a/devcapsule-src/tests/unit/devcapsule/test_github_transport.py b/devcapsule-src/tests/unit/devcapsule/test_github_transport.py new file mode 100644 index 00000000..695e8997 --- /dev/null +++ b/devcapsule-src/tests/unit/devcapsule/test_github_transport.py @@ -0,0 +1,68 @@ +"""Contracts of GitHub request admission, independent of host SSH or credentials.""" +import json + +import pytest + +from devcapsule.github_transport import GitHubRequest, GitHubTransportError, decode_request, parse_git_ssh + + +@pytest.mark.parametrize('command,service', [("git-upload-pack 'acme/widgets.git'", 'git-upload-pack'), + ("git-receive-pack '/acme/widgets'", 'git-receive-pack')]) +def test_git_arguments_select_only_the_service_and_repository(command: str, service: str) -> None: + """Git's destination and v2 request admit a GitHub transfer with no SSH option forwarding.""" + request = parse_git_ssh(['-o', 'SendEnv=GIT_PROTOCOL', 'git@github.com', command]) + assert request == GitHubRequest(service, 'acme/widgets.git' if service == 'git-upload-pack' else 'acme/widgets') + + +@pytest.mark.parametrize('command', ["sh 'acme/repo'", "git-upload-pack 'acme/repo'; id", "git-upload-pack 'a/../b'", + "git-upload-pack '-oProxyCommand=whoami'", "git-upload-pack 'acme/x\nwhoami'", + "git-upload-pack 'acme/..'", "git-upload-pack 'acme/$(whoami)'", + "git-upload-pack 'acme/repo", "git-upload-pack 'acme/repo' extra"]) +def test_untrusted_git_commands_are_refused(command: str) -> None: + """A capsule cannot select a shell command, traversal or option-shaped repository.""" + with pytest.raises(GitHubTransportError): + parse_git_ssh(['git@github.com', command]) + + +@pytest.mark.parametrize('arguments', [['-o', 'ProxyCommand=id', 'git@github.com', "git-upload-pack 'acme/repo'"], + ['-p', '2222', 'git@github.com', "git-upload-pack 'acme/repo'"], + ['-A', 'git@github.com'], []]) +def test_capsule_ssh_options_cannot_change_host_execution(arguments: list[str]) -> None: + """Unsupported options and destinations never become host SSH arguments.""" + with pytest.raises(GitHubTransportError): + parse_git_ssh(arguments) + + +def test_other_git_hosts_are_not_forwarded_to_the_host() -> None: + """A different destination selects the capsule's ordinary SSH fallback.""" + assert parse_git_ssh(['git@example.test', "git-upload-pack 'acme/repo'"]) is None + + +@pytest.mark.parametrize('document', [{}, {'version': True, 'service': 'git-upload-pack', 'repository': 'a/b'}, + {'version': 1, 'service': ['git-upload-pack'], 'repository': 'a/b'}, + {'version': 1, 'service': 'git-upload-pack', 'repository': 'a/b', 'host': 'evil'}, + {'version': 1, 'service': 'sh', 'repository': 'a/b'}, + {'version': 1, 'service': 'git-upload-pack', 'repository': 'a/../b'}]) +def test_wire_validation_does_not_trust_the_client_parser(document: object) -> None: + """The host independently rejects malformed, extended and hostile requests.""" + with pytest.raises(GitHubTransportError): + decode_request(json.dumps(document).encode()) + + +def test_host_command_keeps_verification_and_disables_forwarding() -> None: + """The host command preserves SSH identity selection but exposes no general SSH service.""" + command = GitHubRequest('git-upload-pack', 'acme/repo.git').ssh_command('/usr/bin/ssh') + assert command[0] == '/usr/bin/ssh' + assert command[-2:] == ('git@github.com', "git-upload-pack 'acme/repo.git'") + assert '-oStrictHostKeyChecking=yes' in command + assert '-oBatchMode=yes' in command + assert '-oForwardAgent=no' in command + assert '-oClearAllForwardings=yes' in command + assert '-oPermitLocalCommand=no' in command + assert '-oControlPath=none' in command + + +def test_valid_wire_request_roundtrips() -> None: + """Encoding a valid request carries exactly the service and repository contract.""" + request = GitHubRequest('git-receive-pack', 'acme/widgets.git') + assert decode_request(request.encode()) == request diff --git a/docs/containment/the-boundary.md b/docs/containment/the-boundary.md index be0275c9..aba695d5 100644 --- a/docs/containment/the-boundary.md +++ b/docs/containment/the-boundary.md @@ -2,7 +2,7 @@ description: What a capsule and its coding agents can reach on your computer by default, what stays isolated, and which permissions you can grant or withdraw. role: containment weight: 1 -updated: 2026-10-30 +updated: 2026-11-16 --- # The boundary @@ -59,13 +59,14 @@ access](granting-and-withdrawing.md) for the commands. | `network host` | Shares your host network namespace: host `localhost` services and capsule ports become one namespace | Your app needs direct access to host-bound services or dynamic local ports ([Services and ports](../your-project/services-and-ports.md)) | | `docker-daemon host-socket` | Mounts `/var/run/docker.sock` into the capsule | Your build or tests start containers; this grants broad control of the daemon and its host bind mounts | | `development-sudo true` | Grants passwordless `sudo` **inside the capsule container** | You need to install temporary OS packages inside a session | +| `github-ssh true` | Lets capsule Git fetch and push through host OpenSSH to every GitHub repository the host identity can access; private keys stay outside | You want GitHub SSH remotes to work inside the capsule ([Git](../everyday-development/git.md)); Linux/WSL with native Docker only | | `host-browser true` | Forwards `http://` and `https://` URLs opened inside the capsule to your host browser through a URL-only broker | OAuth sign-ins and external documentation links are more convenient on your host browser | | `host-x11 true` | Runs the IDE directly on your host X11 display instead of the contained desktop | Almost never—X11 lets clients read keystrokes, clipboard, and windows across your entire host display session; manifests cannot recommend it | | `vscode-download`, `claude-code-download`, `antigravity-download` (`true`) | Permits DevCapsule to download that vendor's pinned artifact under its license terms (when not already covered by home channel trust) | You selected that IDE or agent and accept its vendor terms ([Vendor software and trust](../configuration/software-approvals.md)) | | `base-image` (`default` or ``) | Approves the lock's pinned base digest for this checkout or selects an inspected [local base image](../configuration/local-base.md) | Settled once via home `devcapsule-base` trust or per-checkout consent | Authorizations take effect on the **next** launch; they do not change a running -container. The five host-access nodes can also be granted for one run with +container. The host-access nodes can also be granted for one run with `project run --authorize ...`. Base and vendor consents require saved approval or home channel trust. A Git repository cannot grant itself an authorization. diff --git a/docs/everyday-development/git.md b/docs/everyday-development/git.md index 0604144b..a7c6b4c9 100644 --- a/docs/everyday-development/git.md +++ b/docs/everyday-development/git.md @@ -1,7 +1,7 @@ --- description: Committing, pushing and pulling from inside the capsule, and how credentials reach Git without your host keys entering the capsule. weight: 2 -updated: 2026-10-30 +updated: 2026-11-16 --- # Git @@ -57,7 +57,45 @@ This avoids giving the capsule remote credentials. Review commits before pushing: processes inside the capsule can change the working tree and local Git history. -### 2. Forward your host SSH agent for a session +### 2. Use the launcher’s GitHub SSH proxy + +On Linux or WSL with native Docker, authorize the proxy once from your host: + +```bash +devcapsule project config authorize github-ssh true +devcapsule project run +``` + +For a single run, use `devcapsule project run --authorize github-ssh true`. +Inside the capsule, ordinary `git clone`, `git fetch`, `git pull`, and `git push` +with `git@github.com:OWNER/REPO.git` or `ssh://git@github.com/OWNER/REPO.git` use +OpenSSH on your host. The host's SSH configuration selects the identity and +verifies GitHub's host key. Neither private keys nor the host's agent socket are +mounted into the capsule. Other Git SSH hosts use the capsule's own SSH. + +The host must already trust GitHub's host key and be able to authenticate +without a prompt. Unlock encrypted keys on the host with `ssh-add` before +launching. Hardware keys may still require their normal touch. The proxy never +asks the capsule for a host key passphrase. GitHub connection failures and SSH +diagnostics appear in the capsule's Git output. + +The grant lets every process in this capsule fetch and push **all GitHub +repositories accessible to that host identity**. It is not a per-repository +grant. The proxy accepts only Git transfer commands, not general host commands. +It lives with the outside launcher; stopping that launcher closes the proxy +and cancels active transfers. A later launch creates a new proxy. + +Withdraw the saved grant with `devcapsule project config deny github-ssh` and +relaunch. Configuration changes apply to the next launch, not the running one. + +This first implementation does not provide HTTPS credentials, `gh` API login, +SSH commit signing, arbitrary GitHub SSH aliases, or a general `ssh -T` command. +Git LFS's separate authentication protocol is not supported. Docker Desktop on +macOS needs another transport and reports that limitation when enabled. +An inner capsule launch can reuse an authorized outer proxy; it cannot create +one that reads credentials on the physical host. + +### 3. Forward your host SSH agent for a session When you want `git push` and `git pull` over SSH inside the capsule without copying private key files into the container, run this from your host terminal. @@ -80,7 +118,7 @@ Keep two details in mind: agent to sign challenges while the session is open. Load only the key you need or require per-use confirmation (`ssh-add -c`) on your host. -### 3. Use a repository-scoped credential per checkout +### 4. Use a repository-scoped credential per checkout If a workflow inside the capsule must push or call forge APIs unattended, create a **fine-grained HTTPS personal access token** or **single-repository SSH deploy diff --git a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md index 2286b890..44a9325e 100644 --- a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md @@ -6,6 +6,8 @@ Start date: 2026-11-16 State: active; first implementation slice. +Definition read: WORKFLOW.md@89de30622697, WORKFLOW-LOCAL.md@d92bdcbb0a01 + Branch association: `ws-github-auth/ssh-proxy`, from canonical `upstream/main` at `1942f396`. Integration target: `main` @@ -30,6 +32,24 @@ The branch starts at fetched canonical main: no synchronization is required now. No open bug is owned by this newly registered workstream. Mail and intake are empty. The new feature belongs here, not in the maintenance release. +## Implementation and evidence + +The initial Linux/WSL implementation includes the GitHub-only request validator, +launcher-owned Unix socket broker, capsule Git adapter, optional `github-ssh` +authorization, launch mount/environment wiring and user documentation. It does not +copy private keys or forward the host agent. The socket grants all GitHub Git +repositories available to the host identity. Docker Desktop transport is deferred +and explicitly refused. GitHub aliases, Git LFS authentication, SSH signing and +HTTPS/API credentials are not implemented in this slice. + +Focused validation: 42 tests pass, including real Git clone/fetch/push through a +fixture SSH process, binary streaming, refusal, deadline, shutdown and authorized +Docker arguments. Initial mypy over the touched import graph passed (157 files). +Full build gate is running. Its first attempt failed in pytest setup because +`/opt` is read-only and `/opt/devcapsule-gate` does not exist. Scratch now uses +`/home/devcapsule/.cache/devcapsule-gate/github-auth` per the local rule allowing +another directory with space. No Docker containers or real credential files changed. + ## Planned next step Implement a constrained GitHub Git transport broker, capsule adapter, explicit @@ -47,5 +67,6 @@ paths, then run Pair Review with Gemini 4 Hydrogen High. ## Workstream document index +- [Review brief](evidence/review-brief.md) - [Intake](intake/README.md) - [Intake decisions](intake-dispositions.md) diff --git a/engineering-docs/wip/2026-11-16-github-auth/evidence/review-brief.md b/engineering-docs/wip/2026-11-16-github-auth/evidence/review-brief.md new file mode 100644 index 00000000..a5d2f5d8 --- /dev/null +++ b/engineering-docs/wip/2026-11-16-github-auth/evidence/review-brief.md @@ -0,0 +1,87 @@ +# Review brief: GitHub SSH proxy, Gemini 4 Hydrogen High + +Implementor: Codex, GPT-6. Review partner: Antigravity, gemini-4-hydrogen-high. +The owner requested this feature with Pair Review and selected the github-auth +workstream and Git over SSH first. Review correctness, clarity and meaningful +coverage of the implementation against upstream/main. + +## Checkout and first reads + +Work only in /home/devcapsule/worktrees/github-auth-review on +ws-github-auth/ssh-proxy-review. Do not edit the implementor checkout. +Read AGENTS.md, WORKFLOW.md topic 9.11, WORKFLOW-LOCAL.md, this brief, +engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md, then +`git diff upstream/main...HEAD`. Read the contracts in github_transport.py and +github_proxy.py before judging implementations. This brief is your only context. + +## Scope and questions + +Ordinary GitHub clone/fetch/push from a Linux/WSL capsule uses host-side OpenSSH +through a private launcher-owned Unix socket. No private keys or host agent socket +enter the capsule. Only fixed GitHub upload-pack/receive-pack requests are allowed. +The authorization is optional and denied by default; grant through host config or +run-once. All repositories accessible by the host identity are within the grant. +Host key checking remains strict; keys must authenticate noninteractively on the +host. Other Git hosts fall back to the capsule SSH. Mac Docker Desktop, HTTPS, gh, +Git LFS auth, SSH signing, arbitrary aliases are outside the first slice. + +Review in order: +1. Correctness: injection/destination escape, request validation independent of + the client, socket permission boundary, lifecycle cleanup, cancellation, + bounded memory/concurrency, deadlocks/half-close/exit status, Docker environment + integration and denial. Never weaken host-key checking to make a test pass. +2. Clarity: remove misleading guarantees or unnecessary machinery. +3. Coverage: new tests must establish independent contracts, especially auth + gating, real binary streaming, failure/exit handling and process cleanup. +4. Follow Unit Tests rules 1-4 in WORKFLOW-LOCAL.md. New unit tests live in the + module mirror, state their contract in a docstring, and construct inputs. + For each, identify possible failures not caused by a product bug. Do not edit + tests in legacy files. Add integration tests for subprocess/network effects. +5. Prefer explicit host inputs and pure decisions. No monkeypatch/unittest.mock + in new tests. Code runs no invented cryptography; authentication is OpenSSH. + +Keep corrections within this feature. Do not repair unrelated legacy tests or +expand to macOS, HTTPS, repository allowlists or generic SSH forwarding. + +## Validation + +Create your own environment exactly as the local workflow says, from devcapsule-src: +python3.12 -m venv .venv +.venv/bin/python -m pip install -r dev-requirements.txt +.venv/bin/python -m pip install -e . --no-deps + +Use scratch under /home/devcapsule/.cache/devcapsule-gate/github-auth-review; +/opt is read-only. The parent cache directory exists. +Initialize website with `git submodule update --init website` for the docs gate. +Run focused tests: +.venv/bin/python -m pytest -q --no-cov -o addopts='' tests/unit/devcapsule/test_github_transport.py tests/unit/devcapsule/configuration/test_authorization.py tests/unit/devcapsule/configuration/test_review.py tests/integration/test_github_proxy.py + +Run full gate synchronously, not a background tool job: +PYTEST_ADDOPTS='--basetemp=/home/devcapsule/.cache/devcapsule-gate/github-auth-review' DEVCAPSULE_DIFF_BASE=upstream/main .venv/bin/python -m nox -s build + +The implementor's focused run passed 42 tests. Its full gate may expose existing +mainline legacy test failures in this capsule. Report exact failures; do not fix +unrelated code or disable tests. Do not build images or launch Docker capsules. + +## Delivery and forbidden actions + +The GitHub CLI is ABSENT at /opt/xtras/miniconda3/envs/gh/bin/gh. Local policy +requires the human to create PRs in this case. Do not use another credential or +GitHub interface. There is not yet a PRn; PR-mediated Pair Review completion is +pending owner delivery. Perform the local code review now, preserve your findings +and corrections, and report this delivery blocker accurately. Do not claim a +completed PR review loop. Push your review branch over ordinary upstream SSH +only after checks; no push to main. Provide a compare URL if changes exist. + +Do not edit status files, intake/dispositions, evidence, root CURRENT-STATUS.md, +index.md, or run workflow publish. Do not change website gitlinks or configuration +manifests. Do not merge, rebase, force-push, create releases, inspect private keys, +read credential contents, use real GitHub repositories in tests, or run containers. +Do not modify the implementor's files or state. Do not add broad test-only seams. + +Commit corrections with the existing human author and this trailer: +Co-authored-by: Gemini 4 Hydrogen High Antigravity + +Report findings (what/where/fix/evidence), things considered and left alone, +unimplemented proposals, exact validation counts and limits. If no changes, say +so. Print REVIEW_COMPLETE as your last line, or REVIEW_BLOCKED with the reason. From f1dbb3387b2152293d77fce9ea2bc87fe880b579 Mon Sep 17 00:00:00 2001 From: Costin Cozianu Date: Mon, 16 Nov 2026 03:41:22 +0000 Subject: [PATCH 03/43] Refurbish node registry tests for the GitHub authorization Preserve existing registry contracts in the module mirror and cover the optional github-ssh node. All 20 cases pass; no test-convention breaches. Co-authored-by: GPT-6 Codex --- .../tests/configuration/test_nodes.py | 170 ------------------ .../devcapsule/configuration/test_nodes.py | 166 +++++++++++++++++ 2 files changed, 166 insertions(+), 170 deletions(-) delete mode 100644 devcapsule-src/tests/configuration/test_nodes.py create mode 100644 devcapsule-src/tests/unit/devcapsule/configuration/test_nodes.py diff --git a/devcapsule-src/tests/configuration/test_nodes.py b/devcapsule-src/tests/configuration/test_nodes.py deleted file mode 100644 index 4a1ac13d..00000000 --- a/devcapsule-src/tests/configuration/test_nodes.py +++ /dev/null @@ -1,170 +0,0 @@ -"""Focused tests for the configuration-node registry.""" - -from __future__ import annotations - -import pytest - -from devcapsule.configuration.nodes import ( - CARRIER_FAMILY_AUTHORIZE, - CARRIER_FAMILY_BIND, - CARRIER_FAMILY_SET, - PROVIDER_HOST_DIRECTORY, - PROVIDER_HOST_ENVIRONMENT, - build_node_registry, -) -from devcapsule.configuration.file_formats import ( - ProjectConfigurationError, -) - -pytestmark = pytest.mark.legacy - - -def example_manifest() -> dict: - return { - "devcapsule-schema-version": 1, - "capabilities": {"need": ["python", "python-ide", "codex-agent"]}, - "project": { - "name": "Example", - "slug": "example", - "creator": "https://github.com/example", - "mount": "/workspace/example", - }, - "configuration": { - "values": { - "runtime.memory-limit": { - "type": "memory-size", - "runtime-effect": "docker.memory-limit", - "description": "Hard memory limit for the project container.", - } - } - }, - "host": { - "docker": { - "mode": { - "recommended": { - "value": "host-socket", - "justification": "Required for the full test suite.", - } - } - } - }, - } - - -def example_lock() -> dict: - return { - "devcapsule-lock-format-version": 1, - "platform": "linux-amd64", - "base": { - "reference": "docker.io/mycodespaceai/devcapsule-base@sha256:" + "6" * 64, - }, - "components": { - "interactive-surface": "pycharm", - "pycharm": { - "version": "2026.2.0.1", - "variant": "professional", - "delivery-policy": "local-materialization", - "url": "https://example.test/pycharm.tar.gz", - "sha256": "a" * 64, - }, - "codex": { - "version": "0.145.0", - "delivery-policy": "local-materialization", - }, - }, - } - - -def test_registry_assigns_each_declaration_kind_to_its_family() -> None: - registry = build_node_registry(example_manifest(), example_lock()) - - value = registry.node("runtime.memory-limit") - assert value.family == CARRIER_FAMILY_SET - assert value.required is False - - home = registry.node("home") - assert home.family == CARRIER_FAMILY_BIND - assert home.providers == (PROVIDER_HOST_DIRECTORY,) - - slot = registry.node("pycharm/system") - assert slot.family == CARRIER_FAMILY_BIND - - secret = registry.node("codex/openai-api-key") - assert secret.family == CARRIER_FAMILY_BIND - assert secret.providers == (PROVIDER_HOST_ENVIRONMENT,) - assert secret.required is False - - base = registry.node("base-image") - assert base.family == CARRIER_FAMILY_AUTHORIZE - assert base.required is True - assert base.accepts_justification is False - - docker = registry.node("docker-daemon") - assert docker.family == CARRIER_FAMILY_AUTHORIZE - assert docker.required is False - assert docker.accepts_justification is True - assert docker.description == "Required for the full test suite." - - -def test_registry_names_are_sorted_and_family_filtered() -> None: - registry = build_node_registry(example_manifest(), example_lock()) - names = registry.names() - assert names == tuple(sorted(names)) - authorize = registry.family(CARRIER_FAMILY_AUTHORIZE) - # docker-daemon comes from the manifest recommendation; network, - # development-sudo, host-browser and host-x11 exist as - # home-capability defaults, network since 2026-10-22. - assert {node.name for node in authorize} == { - "base-image", - "docker-daemon", - "network", - "development-sudo", - "host-browser", - "host-x11", - } - - -def test_unknown_node_lists_the_declared_vocabulary() -> None: - registry = build_node_registry(example_manifest(), example_lock()) - with pytest.raises(ProjectConfigurationError) as failure: - registry.node("does-not-exist") - assert "'does-not-exist'" in str(failure.value) - assert "runtime.memory-limit" in str(failure.value) - - -def test_wrong_carrier_family_names_the_right_spelling() -> None: - registry = build_node_registry(example_manifest(), example_lock()) - with pytest.raises(ProjectConfigurationError, match="answered with --bind, not --set"): - registry.answerable("home", CARRIER_FAMILY_SET) - - -def test_bind_values_split_on_the_first_colon_only() -> None: - registry = build_node_registry(example_manifest(), example_lock()) - provider, value = registry.split_bind_value("home", "host-directory:/state/x:y") - assert provider == PROVIDER_HOST_DIRECTORY - assert value == "/state/x:y" - - provider, value = registry.split_bind_value( - "codex/openai-api-key", "host-environment:OPENAI_API_KEY" - ) - assert provider == PROVIDER_HOST_ENVIRONMENT - assert value == "OPENAI_API_KEY" - - -def test_bind_value_rejects_wrong_or_missing_provider() -> None: - registry = build_node_registry(example_manifest(), example_lock()) - with pytest.raises(ProjectConfigurationError, match="host-environment:VALUE"): - registry.split_bind_value("codex/openai-api-key", "host-directory:/somewhere") - with pytest.raises(ProjectConfigurationError, match="host-directory:VALUE"): - registry.split_bind_value("home", "/no/provider/prefix") - - -def test_a_name_declared_twice_fails_at_construction() -> None: - manifest = example_manifest() - # A project-declared value colliding with the component slot name 'home'. - manifest["configuration"]["values"]["home"] = { - "type": "string", - "description": "Colliding declaration.", - } - with pytest.raises(ProjectConfigurationError, match="declared twice"): - build_node_registry(manifest, example_lock()) diff --git a/devcapsule-src/tests/unit/devcapsule/configuration/test_nodes.py b/devcapsule-src/tests/unit/devcapsule/configuration/test_nodes.py new file mode 100644 index 00000000..33ba049e --- /dev/null +++ b/devcapsule-src/tests/unit/devcapsule/configuration/test_nodes.py @@ -0,0 +1,166 @@ +"""The node registry is built from values and gives every input one address.""" +from typing import Any + +import pytest + +from devcapsule.configuration.nodes import ( + CARRIER_FAMILY_AUTHORIZE, CARRIER_FAMILY_BIND, CARRIER_FAMILY_SET, + PROVIDER_HOST_DIRECTORY, PROVIDER_HOST_ENVIRONMENT, build_node_registry, +) +from devcapsule.configuration.file_formats import ProjectConfigurationError + +def example_manifest() -> dict[str, Any]: + return { + "devcapsule-schema-version": 1, + "capabilities": {"need": ["python", "python-ide", "codex-agent"]}, + "project": { + "name": "Example", + "slug": "example", + "creator": "https://github.com/example", + "mount": "/workspace/example", + }, + "configuration": { + "values": { + "runtime.memory-limit": { + "type": "memory-size", + "runtime-effect": "docker.memory-limit", + "description": "Hard memory limit for the project container.", + } + } + }, + "host": { + "docker": { + "mode": { + "recommended": { + "value": "host-socket", + "justification": "Required for the full test suite.", + } + } + } + }, + } + + +def example_lock() -> dict[str, Any]: + return { + "devcapsule-lock-format-version": 1, + "platform": "linux-amd64", + "base": { + "reference": "docker.io/mycodespaceai/devcapsule-base@sha256:" + "6" * 64, + }, + "components": { + "interactive-surface": "pycharm", + "pycharm": { + "version": "2026.2.0.1", + "variant": "professional", + "delivery-policy": "local-materialization", + "url": "https://example.test/pycharm.tar.gz", + "sha256": "a" * 64, + }, + "codex": { + "version": "0.145.0", + "delivery-policy": "local-materialization", + }, + }, + } + + + +@pytest.mark.parametrize('name,family,required', [ + ('runtime.memory-limit', CARRIER_FAMILY_SET, False), + ('home', CARRIER_FAMILY_BIND, False), + ('pycharm/system', CARRIER_FAMILY_BIND, False), + ('codex/openai-api-key', CARRIER_FAMILY_BIND, False), + ('base-image', CARRIER_FAMILY_AUTHORIZE, True), + ('docker-daemon', CARRIER_FAMILY_AUTHORIZE, False), + ('github-ssh', CARRIER_FAMILY_AUTHORIZE, False), +]) +def test_each_declaration_has_its_carrier(name: str, family: str, required: bool) -> None: + """The registry maps each declared value, binding and authorization to its input family.""" + registry = build_node_registry(example_manifest(), example_lock()) + node = registry.node(name) + assert node.family == family + assert node.required is required + + +def test_registry_names_are_sorted() -> None: + """The registry presents deterministic sorted names independent of declaration order.""" + registry = build_node_registry(example_manifest(), example_lock()) + names = registry.names() + assert names == tuple(sorted(names)) + + +def test_authorization_family_includes_optional_github_access() -> None: + """The authorization family contains every default host boundary and the locked base.""" + registry = build_node_registry(example_manifest(), example_lock()) + nodes = registry.family(CARRIER_FAMILY_AUTHORIZE) + assert {node.name for node in nodes} == { + 'base-image', 'docker-daemon', 'network', 'development-sudo', 'host-browser', 'host-x11', 'github-ssh', + } + + +@pytest.mark.parametrize('name,providers', [('home', (PROVIDER_HOST_DIRECTORY,)), + ('codex/openai-api-key', (PROVIDER_HOST_ENVIRONMENT,))]) +def test_binding_declaration_exposes_its_allowed_providers(name: str, providers: tuple[str, ...]) -> None: + """A directory binding and an environment secret accept their own providers only.""" + registry = build_node_registry(example_manifest(), example_lock()) + assert registry.node(name).providers == providers + + +def test_base_consent_cannot_be_justified_by_project_advice() -> None: + """The base-image node requires an answer without project-authored justification.""" + registry = build_node_registry(example_manifest(), example_lock()) + assert registry.node('base-image').accepts_justification is False + + +def test_host_recommendation_carries_its_reason() -> None: + """A project recommendation exposes its own justification on the host-access node.""" + registry = build_node_registry(example_manifest(), example_lock()) + node = registry.node('docker-daemon') + assert node.accepts_justification is True + assert node.description == 'Required for the full test suite.' + + +def test_unknown_node_lists_the_declared_vocabulary() -> None: + """Unknown names are refused with the missing name and known vocabulary.""" + registry = build_node_registry(example_manifest(), example_lock()) + with pytest.raises(ProjectConfigurationError) as failure: + registry.node('does-not-exist') + assert "'does-not-exist'" in str(failure.value) + assert 'runtime.memory-limit' in str(failure.value) + + +def test_wrong_carrier_family_names_the_right_spelling() -> None: + """A directory cannot be answered as a scalar; the diagnostic names the proper carrier.""" + registry = build_node_registry(example_manifest(), example_lock()) + with pytest.raises(ProjectConfigurationError, match='answered with --bind, not --set'): + registry.answerable('home', CARRIER_FAMILY_SET) + + +@pytest.mark.parametrize('name,binding,expected', [ + ('home', 'host-directory:/state/x:y', (PROVIDER_HOST_DIRECTORY, '/state/x:y')), + ('codex/openai-api-key', 'host-environment:KEY', (PROVIDER_HOST_ENVIRONMENT, 'KEY')), +]) +def test_bind_values_split_on_the_first_colon_only(name: str, binding: str, expected: tuple[str, str]) -> None: + """Provider parsing retains colons inside a directory value and accepts environment names.""" + registry = build_node_registry(example_manifest(), example_lock()) + assert registry.split_bind_value(name, binding) == expected + + +@pytest.mark.parametrize('name,binding,message', [ + ('codex/openai-api-key', 'host-directory:/somewhere', 'host-environment:VALUE'), + ('home', '/no/provider/prefix', 'host-directory:VALUE'), +]) +def test_bind_value_rejects_wrong_or_missing_provider(name: str, binding: str, message: str) -> None: + """A binding with a missing or incompatible provider is refused with the expected form.""" + registry = build_node_registry(example_manifest(), example_lock()) + with pytest.raises(ProjectConfigurationError, match=message): + registry.split_bind_value(name, binding) + + +def test_a_name_declared_twice_fails_at_construction() -> None: + """A scalar declaration cannot reuse the name of a component binding.""" + manifest = example_manifest() + manifest['configuration']['values']['home'] = {'type': 'string', 'description': 'Collision.'} + with pytest.raises(ProjectConfigurationError, match='declared twice'): + build_node_registry(manifest, example_lock()) From 4ba1725f30eb4a6e7cb6f893a82ce03b2f77e543 Mon Sep 17 00:00:00 2001 From: Costin Cozianu Date: Mon, 16 Nov 2026 03:41:56 +0000 Subject: [PATCH 04/43] Keep console authorization contract aligned with GitHub access Treat github-ssh denial as a restriction on both sides of the configuration channel. Co-authored-by: GPT-6 Codex --- devcapsule-webconsole/CONFIGURATION-CHANNEL.md | 2 +- devcapsule-webconsole/devcapsule_webconsole/configuration.py | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/devcapsule-webconsole/CONFIGURATION-CHANNEL.md b/devcapsule-webconsole/CONFIGURATION-CHANNEL.md index c312b93b..75fa971d 100644 --- a/devcapsule-webconsole/CONFIGURATION-CHANNEL.md +++ b/devcapsule-webconsole/CONFIGURATION-CHANNEL.md @@ -42,7 +42,7 @@ names the project. reaches: `ide`, `add`, `omit`, `restore`, `use`, `set`, `deny`, literal `env set NAME VALUE`, `env unset`, `unbind`, `trust withdraw`, `docker-args clear`. Explicit secure authorizations also apply: `network bridge|none`, `docker-daemon - none`, and `development-sudo|host-browser|host-x11 false|none`. + none`, and `development-sudo|host-browser|host-x11|github-ssh false|none`. - **Deferred** when the verb widens what the next launch reaches: `authorize`, `trust accept`, `bind`, `forget`, `docker-args set`, and `env set` forms other than an explicit literal value. `forget` can restore diff --git a/devcapsule-webconsole/devcapsule_webconsole/configuration.py b/devcapsule-webconsole/devcapsule_webconsole/configuration.py index c71801c7..f6997073 100644 --- a/devcapsule-webconsole/devcapsule_webconsole/configuration.py +++ b/devcapsule-webconsole/devcapsule_webconsole/configuration.py @@ -46,6 +46,7 @@ VERBS = ("ide", "add", "omit", "restore", "use", "set", "deny", "forget", "unbind", "env", "trust", "docker-args", "authorize", "bind") SECURE_AUTHORIZATIONS = { "network": {"bridge", "none"}, "docker-daemon": {"none"}, + "github-ssh": {"false", "none"}, "development-sudo": {"false", "none"}, "host-browser": {"false", "none"}, "host-x11": {"false", "none"}, } WIDENING = ("authorize", "bind", "forget", "trust accept", "docker-args set") From 96a867fea9246589f2e8bec445a7a0769146c809 Mon Sep 17 00:00:00 2001 From: Costin Cozianu Date: Mon, 16 Nov 2026 03:44:52 +0000 Subject: [PATCH 05/43] Record proxy validation and pending review evidence Co-authored-by: GPT-6 Codex --- .../2026-11-16-github-auth/CURRENT-STATUS.md | 28 ++++++++++++++++--- 1 file changed, 24 insertions(+), 4 deletions(-) diff --git a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md index 44a9325e..3a5fa54c 100644 --- a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md @@ -45,22 +45,42 @@ HTTPS/API credentials are not implemented in this slice. Focused validation: 42 tests pass, including real Git clone/fetch/push through a fixture SSH process, binary streaming, refusal, deadline, shutdown and authorized Docker arguments. Initial mypy over the touched import graph passed (157 files). -Full build gate is running. Its first attempt failed in pytest setup because +The full build gate reached the unit suite: 20 failed, 2917 passed, 1 skipped, +14 xfailed and 7 xpassed. A selected rerun on an archive of unchanged +`upstream/main` reproduced 19 of those failures; the added authorization exposed +one exact-vocabulary legacy test. That file was refurbished into its proper +module mirror, preserving its contracts in 20 passing cases. The original gate +attempt failed in pytest setup because `/opt` is read-only and `/opt/devcapsule-gate` does not exist. Scratch now uses `/home/devcapsule/.cache/devcapsule-gate/github-auth` per the local rule allowing another directory with space. No Docker containers or real credential files changed. +Additional checks: CLI smoke, PEX build and smoke, and documentation contract +pass. The console gate identified a mirrored authorization-table omission; after +alignment, all 458 console tests pass. A read-only live `git ls-remote` through +both the source adapter and the built PEX returned GitHub HEAD `1942f396`. +That used the current capsule's already-installed SSH credentials on the broker +side; it is not physical-host/capsule acceptance evidence. + +The branch is pushed to `upstream/ws-github-auth/ssh-proxy`. Gemini Hydrogen High +is reviewing in `/home/devcapsule/worktrees/github-auth-review`. The first invocation +received no brief through redirected stdin. The same conversation resumed with an +explicit path to this committed brief; flags/model stayed unchanged. No credentials +were examined to repair this CLI input issue. + ## Planned next step -Implement a constrained GitHub Git transport broker, capsule adapter, explicit -host authorization, and lifecycle integration. Validate the transport and refusal -paths, then run Pair Review with Gemini 4 Hydrogen High. +Assess Gemini's review diff, incorporate accepted corrections, repeat affected +checks, and prepare owner-operated PR delivery. Full gate remains blocked by the +19 demonstrated mainline/environment failures; no test was disabled or reclassified +to hide them. ## Open Threads - Awaiting the human: none for workstream or first delivery scope; both accepted. - Engineering choices: initial transport must account for Linux/rootless Docker permissions; Docker Desktop cannot bind ordinary host Unix sockets. +- Physical-host launch acceptance and Docker Desktop transport remain open. - GitHub CLI is absent at the policy-prescribed path. PR creation/merging remains owner-operated under WORKFLOW-LOCAL.md; prepare reviewed commits and compare links. - Deliberately not preserved: no credentials or console decision tokens in Git. From 388f62a34fe373882201f9118faaf34cec61680b Mon Sep 17 00:00:00 2001 From: Costin Cozianu Date: Mon, 16 Nov 2026 05:57:28 +0000 Subject: [PATCH 06/43] Review corrections for GitHub SSH proxy Remove unused frame helper, separate capsule SSH fallback errors from proxy failures, clarify launcher disclosure/CLI help and public surface, and add unit/integration coverage for bridge gating and process-group cleanup. Co-authored-by: Gemini 4 Hydrogen High Antigravity --- .../devcapsule/commands/github_ssh.py | 10 +- devcapsule-src/devcapsule/commands/project.py | 2 +- .../devcapsule/configuration/review.py | 2 + devcapsule-src/devcapsule/github_proxy.py | 4 - .../devcapsule/launch/pycharm/_launcher.py | 8 +- .../tests/integration/test_github_proxy.py | 154 +++++++++++++++++- .../unit/devcapsule/test_github_proxy.py | 42 +++++ .../unit/devcapsule/test_github_transport.py | 37 ++++- 8 files changed, 239 insertions(+), 20 deletions(-) create mode 100644 devcapsule-src/tests/unit/devcapsule/test_github_proxy.py diff --git a/devcapsule-src/devcapsule/commands/github_ssh.py b/devcapsule-src/devcapsule/commands/github_ssh.py index e36fd1fc..443c0e90 100644 --- a/devcapsule-src/devcapsule/commands/github_ssh.py +++ b/devcapsule-src/devcapsule/commands/github_ssh.py @@ -23,8 +23,16 @@ class GitHubSshCommand(Command): def run(cls, arguments: argparse.Namespace, context: object | None) -> int: try: request = parse_git_ssh(arguments.ssh_arguments) - if request is None: + except GitHubTransportError as error: + raise CliError( + f"GitHub proxy: {error} Host keys must be trusted and encrypted keys unlocked on the host." + ) from error + if request is None: + try: return subprocess.run(["ssh", *arguments.ssh_arguments], check=False).returncode + except OSError as error: + raise CliError(f"Capsule SSH: {error}") from error + try: path = os.environ.get(SOCKET_ENV) if not path: raise GitHubTransportError("No authorized GitHub proxy. Enable github-ssh on the host launcher.") diff --git a/devcapsule-src/devcapsule/commands/project.py b/devcapsule-src/devcapsule/commands/project.py index 6b0b2541..b35cfa96 100644 --- a/devcapsule-src/devcapsule/commands/project.py +++ b/devcapsule-src/devcapsule/commands/project.py @@ -1276,7 +1276,7 @@ def run(cls, arguments: argparse.Namespace, context: object | None) -> int: class ConfigDenyCommand(Command): name = "deny" capsule_access: ClassVar[CapsuleAccess] = CapsuleAccess.MUTATES - help = "Deny a host access for this checkout, recorded: network, docker-daemon, development-sudo, host-browser, host-x11." + help = "Deny a host access for this checkout, recorded: network, docker-daemon, development-sudo, host-browser, host-x11, github-ssh." @classmethod def configure(cls, parser: argparse.ArgumentParser) -> None: diff --git a/devcapsule-src/devcapsule/configuration/review.py b/devcapsule-src/devcapsule/configuration/review.py index 2a14a488..823b411e 100644 --- a/devcapsule-src/devcapsule/configuration/review.py +++ b/devcapsule-src/devcapsule/configuration/review.py @@ -20,6 +20,8 @@ from .trust import HomeTrust from .values import RuntimeEffects, resolve_configuration_values +__all__ = ["HostAccess", "ConfigurationReview", "review_configuration"] + @dataclass(frozen=True) class HostAccess: diff --git a/devcapsule-src/devcapsule/github_proxy.py b/devcapsule-src/devcapsule/github_proxy.py index 83d2ba40..9939b28c 100644 --- a/devcapsule-src/devcapsule/github_proxy.py +++ b/devcapsule-src/devcapsule/github_proxy.py @@ -45,10 +45,6 @@ def _read_exact(connection: socket.socket, length: int) -> bytes: return bytes(chunks) -def _send_frame(connection: socket.socket, kind: bytes, payload: bytes) -> None: - connection.sendall(kind + struct.pack("!I", len(payload)) + payload) - - class GitHubBroker: """One session's GitHub bridge. Close cancels active children and removes its socket. diff --git a/devcapsule-src/devcapsule/launch/pycharm/_launcher.py b/devcapsule-src/devcapsule/launch/pycharm/_launcher.py index 283c5ac6..0c327684 100644 --- a/devcapsule-src/devcapsule/launch/pycharm/_launcher.py +++ b/devcapsule-src/devcapsule/launch/pycharm/_launcher.py @@ -726,6 +726,8 @@ def describe_run_command(command: list[str], config: PycharmRunConfig, ) if config.host_browser_socket is not None: comments.append("Host-browser socket requires a live broker; a broker owned by this invocation stops on return.") + if config.github_socket is not None: + comments.append("GitHub SSH socket requires a live broker; a broker owned by this invocation stops on return.") if config.display_transport == CONTAINED_DISPLAY_TRANSPORT: comments.append("Contained display needs its temporary token file; the chosen host port is not reserved for later use.") if config.console_host_port is not None: @@ -2293,7 +2295,11 @@ def print_storage_summary(config: PycharmRunConfig) -> None: Enabled through a URL-only HTTP(S) broker. Any process running as the capsule user can ask the physical host to navigate its default browser.""" if config.github_socket is not None: - host_browser_disclosure += "\n GitHub SSH: host-authenticated Git fetch/push proxy (all repositories your host identity can access)." + host_browser_disclosure += """ + +GitHub SSH integration: + Host-authenticated Git fetch/push proxy (all repositories your host identity + can access); private keys stay on the host.""" host_browser_disclosure += display_disclosure(config) + console_disclosure(config) if config.interactive_state_mounts: component_id = ( diff --git a/devcapsule-src/tests/integration/test_github_proxy.py b/devcapsule-src/tests/integration/test_github_proxy.py index a8abf31b..f4e5a08f 100644 --- a/devcapsule-src/tests/integration/test_github_proxy.py +++ b/devcapsule-src/tests/integration/test_github_proxy.py @@ -8,13 +8,18 @@ from contextlib import contextmanager import os from pathlib import Path +import selectors import shlex +import shutil import socket +import stat import struct import subprocess import sys import tempfile +import time from collections.abc import Iterator +from typing import IO import pytest @@ -72,9 +77,11 @@ def test_real_git_clone_fetch_and_push_through_capsule_adapter(tmp_path: Path) - """Real Git clone, fetch and push transfer repository objects through the broker.""" env = _environment(tmp_path) bare = tmp_path / 'remote.git' - subprocess.run(['git', 'init', '--bare', '--initial-branch=main', str(bare)], env=env, check=True, capture_output=True) + git_bin = shutil.which('git') + assert git_bin is not None + subprocess.run([git_bin, 'init', '--bare', '--initial-branch=main', str(bare)], env=env, check=True, capture_output=True) # This stand-in uses only the validated service; the repository is fixed by the fixture. - program = f"import os, shlex, sys\nservice = shlex.split(sys.argv[-1])[0]\nos.execv('/usr/bin/git', ['git', service.removeprefix('git-'), {str(bare)!r}])\n" + program = f"import os, shlex, sys\nservice = shlex.split(sys.argv[-1])[0]\nos.execv({git_bin!r}, ['git', service.removeprefix('git-'), {str(bare)!r}])\n" with _broker(tmp_path, program) as (path, _): git_env = {**env, SOCKET_ENV: str(path), 'GIT_SSH_VARIANT': 'ssh', 'GIT_SSH_COMMAND': shlex.join([sys.executable, '-m', 'devcapsule', 'github-ssh', '--'])} @@ -118,12 +125,131 @@ def test_oversized_handshake_is_refused_without_reading_body(tmp_path: Path) -> assert connection.recv(1) == b'' +def _process_running(pid: int) -> bool: + try: + os.kill(pid, 0) + except ProcessLookupError: + return False + # An orphaned grandchild cannot be waitpid'd by the broker and stays a + # zombie until PID 1 reaps it; treat 'Z' as terminated, not running. + try: + status = Path(f'/proc/{pid}/status').read_text(encoding='utf-8') + except OSError: + return False + for line in status.splitlines(): + if line.startswith('State:') and line.split(':', 1)[1].lstrip().startswith('Z'): + return False + return True + + +def _wait_pids_stopped(pids: list[int], timeout: float = 2.0) -> None: + deadline = time.monotonic() + timeout + remaining = list(pids) + while remaining and time.monotonic() < deadline: + remaining = [pid for pid in remaining if _process_running(pid)] + if remaining: + time.sleep(0.02) + assert not remaining, f"Processes still running after cleanup: {remaining}" + + +def _read_ready_line(stream: IO[bytes], expected: bytes, timeout: float = 5.0) -> None: + with selectors.DefaultSelector() as selector: + selector.register(stream, selectors.EVENT_READ) + if not selector.select(timeout): + raise TimeoutError(f"Timed out waiting {timeout}s for ready signal {expected!r}") + assert stream.readline() == expected + + def test_operation_deadline_stops_a_stalled_ssh_process(tmp_path: Path) -> None: - """A stalled host SSH process is killed and the capsule receives failure within the bound.""" - with _broker(tmp_path, 'import time\ntime.sleep(60)\n', operation_timeout=0.2) as (path, env): + """A stalled host SSH process and its descendant are terminated (no running process remains) and the capsule fails.""" + pid_file = tmp_path / 'pids' + program = ( + "import os, subprocess, sys, time\n" + "from pathlib import Path\n" + "grandchild = subprocess.Popen([sys.executable, '-c', 'import time; time.sleep(60)'])\n" + f"Path({str(pid_file)!r}).write_text(f'{{os.getpid()}}\\n{{grandchild.pid}}\\n')\n" + "time.sleep(60)\n" + ) + with _broker(tmp_path, program, operation_timeout=0.2) as (path, env): result = _client(path, env) assert result.returncode != 0 assert b'connection ended' in result.stderr + pids = [int(line) for line in pid_file.read_text().splitlines() if line] + assert len(pids) == 2 + _wait_pids_stopped(pids) + + +def test_broker_shutdown_cancels_active_ssh_process_group(tmp_path: Path) -> None: + """Closing the broker terminates an active host SSH transfer so no process in its group remains running.""" + started = tmp_path / 'started' + program = ( + "import os, subprocess, sys, time\n" + "from pathlib import Path\n" + "grandchild = subprocess.Popen([sys.executable, '-c', 'import time; time.sleep(60)'])\n" + f"Path({str(started)!r}).write_text(f'{{os.getpid()}}\\n{{grandchild.pid}}\\n')\n" + "sys.stdout.buffer.write(b'ready\\n')\n" + "sys.stdout.buffer.flush()\n" + "time.sleep(60)\n" + ) + client: subprocess.Popen[bytes] | None = None + try: + with _broker(tmp_path, program, operation_timeout=30) as (path, env): + assert stat.S_IMODE(path.stat().st_mode) == 0o600 + client = subprocess.Popen( + [sys.executable, '-m', 'devcapsule', 'github-ssh', '--', + 'git@github.com', "git-upload-pack 'acme/repo.git'"], + env={**env, SOCKET_ENV: str(path)}, + stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, + ) + assert client.stdout is not None + _read_ready_line(client.stdout, b'ready\n', timeout=5.0) + returncode = client.wait(timeout=5) + client.communicate(timeout=5) + assert returncode != 0 + finally: + if client is not None and client.poll() is None: + client.kill() + client.communicate(timeout=5) + pids = [int(line) for line in started.read_text().splitlines() if line] + assert len(pids) == 2 + _wait_pids_stopped(pids) + + +def test_capsule_adapter_refuses_when_proxy_is_not_authorized(tmp_path: Path) -> None: + """Without an authorized proxy socket in the environment, GitHub SSH fails rather than running SSH.""" + result = subprocess.run( + [sys.executable, '-m', 'devcapsule', 'github-ssh', '--', + 'git@github.com', "git-upload-pack 'acme/repo.git'"], + env=_environment(tmp_path), + input=b'', capture_output=True, timeout=15, + ) + assert result.returncode != 0 + assert b'No authorized GitHub proxy' in result.stderr + + +def test_capsule_adapter_falls_back_to_capsule_ssh_for_other_hosts(tmp_path: Path) -> None: + """Non-GitHub destinations execute the capsule's own ssh binary without touching the proxy.""" + bin_dir = tmp_path / 'bin' + bin_dir.mkdir() + ssh_stub = bin_dir / 'ssh' + args_file = tmp_path / 'ssh_args' + ssh_stub.write_text( + f"#!{sys.executable}\n" + "import json, sys\n" + "from pathlib import Path\n" + f"Path({str(args_file)!r}).write_text(json.dumps(sys.argv[1:]))\n" + "sys.exit(4)\n" + ) + ssh_stub.chmod(0o700) + env = _environment(tmp_path) + env['PATH'] = f"{bin_dir}:{env['PATH']}" + result = subprocess.run( + [sys.executable, '-m', 'devcapsule', 'github-ssh', '--', + '-p', '2222', 'git@example.test', "git-upload-pack 'acme/repo.git'"], + env=env, input=b'', capture_output=True, timeout=15, + ) + assert result.returncode == 4 + assert args_file.read_text() == '["-p", "2222", "git@example.test", "git-upload-pack \'acme/repo.git\'"]' def test_broker_shutdown_closes_incomplete_requests(tmp_path: Path) -> None: @@ -159,3 +285,23 @@ def test_launcher_exposes_the_socket_only_with_the_grant(tmp_path: Path, enabled assert (f'{SOCKET_ENV}={SOCKET_DESTINATION}' in args) is enabled assert (f'type=bind,src={path},dst={SOCKET_DESTINATION},ro' in args) is enabled assert not any('.ssh' in arg or 'SSH_AUTH_SOCK=' in arg for arg in args) + + +def test_launcher_rejects_environment_conflicts_with_github_ssh(tmp_path: Path) -> None: + """Additional environment cannot override GitHub SSH transport variables when enabled.""" + from devcapsule.launch.pycharm._launcher import PycharmRunError, PycharmRunOptions, DockerMode, build_run_config + from devcapsule.platforms import Platform + + project = tmp_path / 'project' + project.mkdir() + with _broker(tmp_path, 'raise AssertionError("no transfer expected")\n') as (path, env): + env = {**env, 'DISPLAY': ':1', 'PYCHARM_GIT_IDENTITY_FROM_HOST': '0', SOCKET_ENV: str(path)} + with pytest.raises(PycharmRunError, match="GitHub SSH"): + build_run_config( + PycharmRunOptions( + project=project, capsule_platform=Platform.LINUX_AMD64, + docker_mode=DockerMode.none, enable_sudo=False, network_mode='bridge', + enable_github_ssh=True, additional_environment={'GIT_SSH_COMMAND': 'ssh'}, + ), + env, + ) diff --git a/devcapsule-src/tests/unit/devcapsule/test_github_proxy.py b/devcapsule-src/tests/unit/devcapsule/test_github_proxy.py new file mode 100644 index 00000000..7a09c104 --- /dev/null +++ b/devcapsule-src/tests/unit/devcapsule/test_github_proxy.py @@ -0,0 +1,42 @@ +"""Unit tests of ``devcapsule.github_proxy``: pure gating decisions of ``github_ssh_bridge``. + +No sockets are bound and no subprocesses start; every outcome follows from the +explicit arguments passed to ``github_ssh_bridge``. +""" +from __future__ import annotations + +from pathlib import Path + +import pytest + +from devcapsule.github_proxy import SOCKET_DESTINATION, SOCKET_ENV, github_ssh_bridge +from devcapsule.github_transport import GitHubTransportError + + +def test_disabled_bridge_yields_none_regardless_of_host_capabilities() -> None: + """When github-ssh is denied, the bridge yields no socket even on unsupported or containerized hosts.""" + with github_ssh_bridge({}, enabled=False, containerized=True, binds_unix_sockets=False) as path: + assert path is None + + +def test_docker_desktop_bridge_is_refused_when_enabled() -> None: + """Enabling github-ssh on a host that cannot bind Unix sockets fails loudly instead of degrading.""" + with pytest.raises(GitHubTransportError, match="Docker Desktop"): + with github_ssh_bridge({}, enabled=True, containerized=False, binds_unix_sockets=False): + pass + + +def test_containerized_bridge_requires_an_inherited_canonical_socket() -> None: + """An inner capsule cannot enable github-ssh without the launcher's canonical socket variable.""" + for environment in ({}, {SOCKET_ENV: "/tmp/forged.sock"}): + with pytest.raises(GitHubTransportError, match="outside launcher"): + with github_ssh_bridge(environment, enabled=True, containerized=True, binds_unix_sockets=True): + pass + assert str(SOCKET_DESTINATION) != "/tmp/forged.sock" + + +def test_host_bridge_requires_openssh_on_path(tmp_path: Path) -> None: + """Enabling github-ssh on the host fails clearly when OpenSSH is absent from an empty PATH directory.""" + with pytest.raises(GitHubTransportError, match="OpenSSH"): + with github_ssh_bridge({"PATH": str(tmp_path)}, enabled=True, containerized=False, binds_unix_sockets=True): + pass diff --git a/devcapsule-src/tests/unit/devcapsule/test_github_transport.py b/devcapsule-src/tests/unit/devcapsule/test_github_transport.py index 695e8997..30560161 100644 --- a/devcapsule-src/tests/unit/devcapsule/test_github_transport.py +++ b/devcapsule-src/tests/unit/devcapsule/test_github_transport.py @@ -6,12 +6,15 @@ from devcapsule.github_transport import GitHubRequest, GitHubTransportError, decode_request, parse_git_ssh -@pytest.mark.parametrize('command,service', [("git-upload-pack 'acme/widgets.git'", 'git-upload-pack'), - ("git-receive-pack '/acme/widgets'", 'git-receive-pack')]) -def test_git_arguments_select_only_the_service_and_repository(command: str, service: str) -> None: - """Git's destination and v2 request admit a GitHub transfer with no SSH option forwarding.""" - request = parse_git_ssh(['-o', 'SendEnv=GIT_PROTOCOL', 'git@github.com', command]) - assert request == GitHubRequest(service, 'acme/widgets.git' if service == 'git-upload-pack' else 'acme/widgets') +@pytest.mark.parametrize('arguments,expected', [ + (['-o', 'SendEnv=GIT_PROTOCOL', 'git@github.com', "git-upload-pack 'acme/widgets.git'"], + GitHubRequest('git-upload-pack', 'acme/widgets.git')), + (['-oSendEnv=GIT_PROTOCOL', '-p', '22', 'github.com', "git-receive-pack '/acme/widgets'"], + GitHubRequest('git-receive-pack', 'acme/widgets')), +]) +def test_git_arguments_select_only_the_service_and_repository(arguments: list[str], expected: GitHubRequest) -> None: + """Git's destination, standard port 22, and v2 request admit a GitHub transfer with no SSH option forwarding.""" + assert parse_git_ssh(arguments) == expected @pytest.mark.parametrize('command', ["sh 'acme/repo'", "git-upload-pack 'acme/repo'; id", "git-upload-pack 'a/../b'", @@ -33,9 +36,13 @@ def test_capsule_ssh_options_cannot_change_host_execution(arguments: list[str]) parse_git_ssh(arguments) -def test_other_git_hosts_are_not_forwarded_to_the_host() -> None: - """A different destination selects the capsule's ordinary SSH fallback.""" - assert parse_git_ssh(['git@example.test', "git-upload-pack 'acme/repo'"]) is None +@pytest.mark.parametrize('arguments', [ + ['git@example.test', "git-upload-pack 'acme/repo'"], + ['-p', '2222', 'git@example.test', "git-upload-pack 'acme/repo'"], +]) +def test_other_git_hosts_are_not_forwarded_to_the_host(arguments: list[str]) -> None: + """A different destination, including with a custom port, selects the capsule's ordinary SSH fallback.""" + assert parse_git_ssh(arguments) is None @pytest.mark.parametrize('document', [{}, {'version': True, 'service': 'git-upload-pack', 'repository': 'a/b'}, @@ -62,6 +69,18 @@ def test_host_command_keeps_verification_and_disables_forwarding() -> None: assert '-oControlPath=none' in command +@pytest.mark.parametrize('payload', [ + b'{', + b'\xff', + b'[]', + json.dumps({'version': 1, 'service': 'git-upload-pack', 'repository': 'a/' + 'b' * 255}).encode(), +]) +def test_wire_validation_rejects_malformed_and_oversized_repository_payloads(payload: bytes) -> None: + """Malformed JSON, non-object frames and overlong repository paths are refused on the wire.""" + with pytest.raises(GitHubTransportError): + decode_request(payload) + + def test_valid_wire_request_roundtrips() -> None: """Encoding a valid request carries exactly the service and repository contract.""" request = GitHubRequest('git-receive-pack', 'acme/widgets.git') From 890fa91d356610165d582b0d021ec018354ffad3 Mon Sep 17 00:00:00 2001 From: Costin Cozianu Date: Mon, 16 Nov 2026 06:00:00 +0000 Subject: [PATCH 07/43] Record Hydrogen review follow-up Co-authored-by: GPT-6 Codex --- .../2026-11-16-github-auth/CURRENT-STATUS.md | 1 + .../evidence/round-1.md | 42 +++++++++++++++++++ 2 files changed, 43 insertions(+) create mode 100644 engineering-docs/wip/2026-11-16-github-auth/evidence/round-1.md diff --git a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md index 3a5fa54c..9ec99585 100644 --- a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md @@ -88,5 +88,6 @@ to hide them. ## Workstream document index - [Review brief](evidence/review-brief.md) +- [Review follow-up 1](evidence/round-1.md) - [Intake](intake/README.md) - [Intake decisions](intake-dispositions.md) diff --git a/engineering-docs/wip/2026-11-16-github-auth/evidence/round-1.md b/engineering-docs/wip/2026-11-16-github-auth/evidence/round-1.md new file mode 100644 index 00000000..c2dc01cf --- /dev/null +++ b/engineering-docs/wip/2026-11-16-github-auth/evidence/round-1.md @@ -0,0 +1,42 @@ +# Review follow-up 1: finish the bounded local review + +Continue the same Gemini Hydrogen review in /home/devcapsule/worktrees/github-auth-review. +Your full build has FINISHED: step 130 returned 20 failed, 2926 passed, 1 skipped, +14 xfailed and 7 xpassed. No pytest or nox process remains. Your CLI stayed idle +after that completion, so the implementor interrupted it and resumed this turn. +Do not wait for the completed job. Do not repeat the full gate: its failures are known. + +The implementor read your diff and accepts its production-code changes and added +coverage in principle. Fix these test issues before committing: +1. test_host_bridge_requires_openssh_on_path passes PATH="". shutil.which searches + the current directory in that case; the test depends on no file named ssh being + present there. Use tmp_path as an explicitly empty PATH directory instead. +2. test_broker_shutdown_cancels_active_ssh_process_group calls stdout.readline with + no timeout. Bound the ready handshake and ensure client cleanup in finally even + if readiness fails. Avoid a test that hangs forever when the product is broken. +3. An orphaned grandchild may be a zombie until the host's reaper runs. The broker + cannot waitpid a grandchild. Verify that no descendant is running, without + asserting timely reaping by an unrelated PID 1. State the test's actual claim. + +Run focused tests INCLUDING your new unit test_github_proxy.py and integration +file and mypy. If new focused tests fail, fix those failures within this scope. +Commit your corrections. Do not merge, reset, rebase or touch the parent checkout. + +Parent updates already committed separately (do not duplicate or cherry-pick): +- f1dbb338 refurbishes tests/configuration/test_nodes.py into its correct mirror, + 20 passing cases; this resolves the new github-ssh name expectation. +- 4ba1725f aligns webconsole/configuration.py SECURE_AUTHORIZATIONS and its contract; + all 458 console tests now pass. +- The remaining 19 failures are reproduced on unchanged upstream/main, by selecting + all 20 failing test IDs in an archive of the base (19 fail, the registry test passes). +- Parent CLI smoke, PEX build/smoke and docs contract pass. Both source and PEX Git + adapters passed read-only authenticated GitHub ls-remote using the capsule's existing + credential on the broker side; not a physical-host launch test. + +Your branch can be pushed to upstream as review preparation, with limitations in +its report. No PR exists because local policy makes creation owner-operated without +its prescribed gh installation. Do not use another GitHub interface or credential. + +Return the findings, corrections, focused results, known full-gate failures and +unimplemented proposals. End with REVIEW_COMPLETE or REVIEW_BLOCKED. Do not begin +another broad investigation; the remaining task is these test corrections and delivery. From 712466ab9547ec7c9153977f725fc6cfe26344f6 Mon Sep 17 00:00:00 2001 From: Costin Cozianu Date: Mon, 16 Nov 2026 06:02:40 +0000 Subject: [PATCH 08/43] Record accepted local review and delivery blockers Co-authored-by: GPT-6 Codex --- CURRENT-STATUS.md | 2 +- .../development/review-partner-loop.md | 1 + .../2026-11-16-github-auth/CURRENT-STATUS.md | 33 +++--- .../evidence/delivery.md | 63 +++++++++++ .../evidence/review-assessment.md | 100 ++++++++++++++++++ 5 files changed, 186 insertions(+), 13 deletions(-) create mode 100644 engineering-docs/wip/2026-11-16-github-auth/evidence/delivery.md create mode 100644 engineering-docs/wip/2026-11-16-github-auth/evidence/review-assessment.md diff --git a/CURRENT-STATUS.md b/CURRENT-STATUS.md index 0d55f14f..877c36e0 100644 --- a/CURRENT-STATUS.md +++ b/CURRENT-STATUS.md @@ -9,7 +9,7 @@ coordination branch. Detailed state belongs in each linked status file. | Name | Started | Goal | State | Branch association | Status file | |---|---|---|---|---|---| -| `github-auth` | 2026-11-16 | GitHub SSH authentication through a launcher-owned host proxy | active; implementation and Gemini Hydrogen Pair Review | `ws-github-auth/ssh-proxy` | [workstream status](engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md) | +| `github-auth` | 2026-11-16 | GitHub SSH authentication through a launcher-owned host proxy | paused; implementation/local review prepared; gate, host acceptance and PR delivery pending | `ws-github-auth/ssh-proxy` | [workstream status](engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md) | | `workflow-improvements` | 2026-08-09 | Improve the multiple-stream workflow from concrete dogfood findings during the recursive E2E cycle | active 2026-10-30; Pair Review Loop naming implemented and packaging verified for 0.3.0; mainline and release integration pending | `ws-workflow-improvements/v1` | [workstream status](engineering-docs/wip/2026-08-09-workflow-improvements/CURRENT-STATUS.md) | | `project-management` | 2026-08-09 | Maintain project-wide priorities, sequencing, dependencies, and lifecycle coordination | paused 2026-10-11 at owner direction; consent and forced configuration before RC work; remaining diagnostics/image scope and release cut decisions retained | `ws-project-management/coordination` | [workstream status](engineering-docs/wip/2026-08-09-project-management/CURRENT-STATUS.md) | | `maintenance` | 2026-09-18 | Own the defects no open workstream covers, on `main` and on maintained release lines, and drive maintenance releases | active; releasing 0.3.1; cut 2026-11-07 from `main` at `31e8fec7`, `main` reopened at `0.3.2.dev0` (#8); candidate 2 (`v0.3.1-rc2` on `be0140ca`, propagated to `main` via PR #22 `aabace3c`) published and verified: Eclipse libsoup (#10, closed on rc1), launcher-delivered `devcapsule_webconsole` (`D-0017`, #18/#16, closed on rc2), Codex npm abbreviated metadata (#20, closed on rc2 + `component-status` `0bc31c64`), and Codex `0.167.0` + Claude Code `2.1.321` pins (`embedded-33`, #21); rulings open: executables for the Mac, the compatibility feed's home | `release-0.3.1`, the workstream's selection from the cut to the final tag; `release-0.3.0` closed and retained; earlier slice branches merged and closed | [workstream status](engineering-docs/wip/2026-09-18-maintenance/CURRENT-STATUS.md) | diff --git a/engineering-docs/development/review-partner-loop.md b/engineering-docs/development/review-partner-loop.md index 09c54df5..32cd4347 100644 --- a/engineering-docs/development/review-partner-loop.md +++ b/engineering-docs/development/review-partner-loop.md @@ -299,6 +299,7 @@ when the workstream pauses, and say so in its open threads. | Date | Workstream | Implementor | Review partner | Slices and reviews | Outcome | |---|---|---|---|---|---| +| 2026-11-16 | github-auth, launcher-owned GitHub SSH proxy | GPT-6 Codex | Antigravity, Gemini 4 Hydrogen High (`gemini-4-hydrogen-high`) | Local preparation; PRs pending | Round 1 accepted locally at `388f62a3`; diagnostics/disclosure and process/authorization coverage corrected; combined candidate 75 focused tests, mypy and conventions passed. Full gate blocked by 19 failures reproduced on unchanged main; physical-host acceptance pending. Reviewer stream stalled after completed full gate; same conversation resumed and completed with verified commit/push. Formal PR-mediated loop remains open because prescribed gh is absent. [Assessment](../wip/2026-11-16-github-auth/evidence/review-assessment.md) records limits and unimplemented proposals. | | 2026-11-08 / 2026-11-09 | maintenance, launcher-delivered web console (`D-0017`, 0.3.1 candidate 2 blocker, stacked on #18) | Antigravity, Gemini 4 Argon (High) pair | Claude Code, Claude Fable 5.1 pair | #16/#17 | accepted at `e23e22c7` without a round, merged at `9919f4ce`, and rebased onto PR #18 (`ws-maintenance/webconsole-legacy-tests`) per owner decision `stacked-delivery-branch = force-push-pr-16`; the reviewer removed the dead per-start `import psutil` subprocess probe (`WEBCONSOLE_RECIPE` 11 settles compatibility on the host) and unified console package precedence in `console_package_path`, fixed `extract_webconsole_bundle` to mask `external_attr >> 16` with `S_IFMT` so typeless regular zip entries (`ZipFile.writestr`) extract, pinned `match=` on every bundle refusal test (99% bundle coverage), tightened `DEVCAPSULE_DEV_WEBCONSOLE` to `1`/`true`/`0`/`false`, and aligned `D-0017` and the bug record; stacked gate 2904 unit tests, 40 integration (3 skipped), 458 webconsole, 332 mypy files, 104 legacy / 44 conforming files with 0 breaches, 249 contract pages, plus live base-Python HTTP check; remaining unimplemented proposals recorded in `maintenance` status; brief committed under `evidence/2026-11-08-webconsole-delivery/review-brief.md` | | 2026-11-09 | maintenance, whole-file legacy test retirement and 4-file refurbishment (`0.3.1` prerequisite) | Antigravity, Gemini 4 Argon (High) pair | Claude Code, Claude Fable 5.1 pair | #18/#19 | accepted at `4aac5dfc` without a round, merged at `a1aad17f`; completed `__all__` in `console.py`, `entrypoint.py` and `materialization.py`, dropped a dead `subprocess` import in `project.py`, stated basename pairing and mirror test deletion refusal across rule 5, the companion, the developer guide and the checker with 4 new checker tests, proved `entrypoint.run` writes session variables only into the supplied `environ`, and restored interpreter-path matching in `test_console_tools.py`; hosted CI on `e301367d` caught `/ide-config` state-slot paths in `test_entrypoint.py` (masked inside the PyCharm capsule), resolved by confining `make_plan` to `tmp_path` and injecting `runtime_root`, `caller_uid`, and `configure_logs` across `filesystem.py`, `identity.py`, and `entrypoint.run` (`test_filesystem.py` added, implementing PR #19's third proposal with zero global state); gate 2842 unit, 36 integration (1 skipped), 458 webconsole, 330 mypy files, 104 legacy / 43 conforming files with 0 breaches, 249 contract pages; two remaining proposals recorded in `maintenance` status; brief committed under `evidence/2026-11-08-webconsole-legacy-tests/review-brief.md` | | 2026-11-08 | maintenance, the web console's Files page (slice 2 of 0.3.1) | Antigravity, Gemini 4 Argon (High) pair | Claude Code, Claude Fable 5.1 pair | #12/#13 | accepted at `49bb2e8e` without a round, merged at `83949a30`; the first loop with the pairs reversed at the owner's word, the reviewer's pair writing the implementor brief from the owner's order; the reviewer removed a raw-route branch that detected its own test and repointed the legacy permission test at the streaming open, and smoked the routes against a standalone console; gate on the merged branch 458 console tests, 2825 unit tests, 0 breaches, 249 contract pages; both briefs committed under the workstream's evidence; the Antigravity stream-json form relayed one event per tool step, proving that form | diff --git a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md index 9ec99585..37dee8c1 100644 --- a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md @@ -4,7 +4,7 @@ Mnemonic: `github-auth` Start date: 2026-11-16 -State: active; first implementation slice. +State: paused; implementation and local review prepared, gate and PR delivery pending. Definition read: WORKFLOW.md@89de30622697, WORKFLOW-LOCAL.md@d92bdcbb0a01 @@ -62,28 +62,34 @@ both the source adapter and the built PEX returned GitHub HEAD `1942f396`. That used the current capsule's already-installed SSH credentials on the broker side; it is not physical-host/capsule acceptance evidence. -The branch is pushed to `upstream/ws-github-auth/ssh-proxy`. Gemini Hydrogen High -is reviewing in `/home/devcapsule/worktrees/github-auth-review`. The first invocation -received no brief through redirected stdin. The same conversation resumed with an -explicit path to this committed brief; flags/model stayed unchanged. No credentials -were examined to repair this CLI input issue. +The implementation and review branches are pushed to canonical upstream. +Gemini Hydrogen completed local round 1 at `388f62a3`. Its corrections remain +on the separate review branch for PR delivery. A temporary combined merge passed +75 focused cases, type checking and conventions (zero breaches), then was aborted. +Git connectivity integrity and authenticated fetch/push checks pass. The original +checkout remains untouched. Review evidence and prepared PR text are indexed below. ## Planned next step -Assess Gemini's review diff, incorporate accepted corrections, repeat affected -checks, and prepare owner-operated PR delivery. Full gate remains blocked by the -19 demonstrated mainline/environment failures; no test was disabled or reclassified -to hide them. +Resolve the 19 reproduced baseline/environment failures in an appropriate gate +environment, then complete owner-operated implementation/review PR delivery and +physical-host acceptance. Do not claim a green full gate or mainline integration. +At resume, fetch upstream, check live workflow state and the review SHA, merge +through the prescribed PR process, and validate the actual combined tree. ## Open Threads -- Awaiting the human: none for workstream or first delivery scope; both accepted. +- Awaiting the human: suitable gate environment and owner-operated GitHub PR delivery. + Workstream selection and Git-over-SSH scope are already accepted. - Engineering choices: initial transport must account for Linux/rootless Docker permissions; Docker Desktop cannot bind ordinary host Unix sockets. - Physical-host launch acceptance and Docker Desktop transport remain open. - GitHub CLI is absent at the policy-prescribed path. PR creation/merging remains owner-operated under WORKFLOW-LOCAL.md; prepare reviewed commits and compare links. -- Deliberately not preserved: no credentials or console decision tokens in Git. +- Reviewer worktree removed at pause; pushed `ws-github-auth/ssh-proxy-review` + preserves its commit. Review corrections are not yet merged into implementation. +- Unimplemented review proposals and exact baseline failure IDs are in the assessment. +- Deliberately not preserved: scratch logs/venvs, credentials or console decision tokens in Git. ## Workstream document index @@ -91,3 +97,6 @@ to hide them. - [Review follow-up 1](evidence/round-1.md) - [Intake](intake/README.md) - [Intake decisions](intake-dispositions.md) + +- [Review assessment](evidence/review-assessment.md) +- [Prepared PR delivery](evidence/delivery.md) diff --git a/engineering-docs/wip/2026-11-16-github-auth/evidence/delivery.md b/engineering-docs/wip/2026-11-16-github-auth/evidence/delivery.md new file mode 100644 index 00000000..5c4767d6 --- /dev/null +++ b/engineering-docs/wip/2026-11-16-github-auth/evidence/delivery.md @@ -0,0 +1,63 @@ +# Prepared delivery + +No PR has been opened. WORKFLOW-LOCAL.md, GitHub Integration rules 4–6, +requires a passing unit gate before PR opening/updating, and assigns GitHub +operations to the human when its prescribed gh installation is absent. The +current environment has both blockers. Preserve both branches until the gate +is resolved in a suitable environment and the owner performs PR delivery. + +## Implementation PR draft + +Base: `main`. Head: `ws-github-auth/ssh-proxy`. + +Title: Add opt-in host GitHub SSH authentication for capsules + +Body: + +Git operations currently require installing SSH credentials inside each capsule. +This adds an opt-in `github-ssh` authorization: the outside launcher owns a Unix +socket broker that runs host OpenSSH for validated GitHub upload/receive requests, +while capsule Git uses the shipped PEX adapter. Private keys remain on the host. +Linux/WSL with native Docker is supported first; Docker Desktop is refused. +Host trust and unlocked encrypted keys are prerequisites. The capability grants +Git access to all repositories available to that host identity. + +Validation: combined local candidate passed 75 focused tests, type checking and +convention checks; source/PEX authenticated read-only GitHub probe passed; local +Git clone/fetch/push, smoke/PEX/docs and 458 console tests passed. Full gate remains +blocked by 19 failures also reproduced on unchanged main. Physical-host launch +acceptance is outstanding. See the workstream review assessment for exact limits. + +Pair: GPT-6 Codex with Antigravity Gemini 4 Hydrogen High +(`gemini-4-hydrogen-high`). Local round 1 accepted at `388f62a3`; formal review PR +must be merged into this branch before mainline delivery. + +## Review PR draft + +Base: `ws-github-auth/ssh-proxy`. Head: `ws-github-auth/ssh-proxy-review`. + +Title: Review GitHub SSH proxy diagnostics and lifecycle coverage + +Body: + +Gemini Hydrogen review separates capsule SSH failures from broker failures, +clarifies disclosure and dry-run lifetime, removes an unused frame helper, +declares the public review API and strengthens parser, authorization and process +cleanup coverage. Round 1 addresses test environment independence and cleanup. +The implementor accepted `388f62a3` locally after a conflict-free temporary merge +passed 75 focused cases. The reviewer branch alone lacks the parent's later node +registry refurbishment and console-table fix; validate the merged candidate. +Full gate and owner-operated PR delivery remain outstanding. + +## Resume sequence + +1. Fetch canonical upstream and recheck workflow synchronization and live state. +2. Resolve the known full-gate environment failures without disabling tests; + run the required gate on the combined candidate before PR opening/updating. +3. The owner creates the implementation and review PRs, then merges the review + into the implementation branch. Record the formal assessment at its SHA. +4. Fetch that merged branch, run required validation and physical-host acceptance: + authorize `github-ssh` once, launch with the rebuilt outside PEX, and exercise + GitHub reads plus a push to an explicitly chosen disposable repository. +5. The owner merges the implementation PR into main; fetch and verify main's tree + before recording integration. No release or installation has been performed. diff --git a/engineering-docs/wip/2026-11-16-github-auth/evidence/review-assessment.md b/engineering-docs/wip/2026-11-16-github-auth/evidence/review-assessment.md new file mode 100644 index 00000000..ecf6b43e --- /dev/null +++ b/engineering-docs/wip/2026-11-16-github-auth/evidence/review-assessment.md @@ -0,0 +1,100 @@ +# GitHub SSH proxy review assessment + +Date: 2026-11-16. Implementor: GPT-6 Codex. Reviewer: Antigravity, +`gemini-4-hydrogen-high` (Gemini 4 Hydrogen High). + +## Disposition + +Round 1 accepted locally at `388f62a34fe373882201f9118faaf34cec61680b`. +The reviewer branch is pushed as `ws-github-auth/ssh-proxy-review`. +No review PR or implementation PR exists yet. This is local review preparation, +not completion of the PR-mediated Pair Review Loop or mainline integration. + +The implementor inspected the diff and accepted the unused helper removal, +separate capsule-SSH error attribution, public API declaration, launcher dry-run +warning and disclosure layout, deny-help update, and expanded hostile-input, +permission, cancellation, process-group and authorization coverage. +Round 1 replaced an ambient-current-directory PATH check, bounded the readiness +handshake, guaranteed client cleanup, and distinguished dead orphan zombies from +running descendants. The process tests depend on Linux /proc and scheduling +within their explicit timeouts; they do not prove prompt reaping by PID 1. + +The reviewer ran 55 focused cases successfully. A temporary, uncommitted merge of +`388f62a3` with implementation `890fa91d` was conflict-free and passed **75 focused +tests**, mypy on the nine touched modules/test files, and the conventions checker +(103 legacy files, 1380 legacy tests, 50 conforming files, zero breaches). +`git diff --check` and `git fsck --connectivity-only --no-dangling` passed. +The temporary merge was aborted deliberately: reviewer corrections remain on +their separate pushed branch for the owner-operated review PR. + +## Build evidence and limits + +The implementor's full build reached pytest: 20 failed, 2917 passed, 1 skipped, +76 deselected, 14 xfailed, 7 xpassed. The reviewer's independent build found the +same failure set: 20 failed, 2926 passed, 1 skipped, 80 deselected, 14 xfailed, +7 xpassed. The new node-name expectation was repaired on the implementation +branch by refurbishing the complete legacy file into the unit mirror (20 passing +cases); the reviewer was told not to duplicate this change. The other 19 failures +reproduced by selecting all 20 failed IDs on an archive of unchanged canonical +main `1942f396`: 19 failed, 1 passed. They concern inherited capsule trust, +permissions/ownership and recursive-launch environment; they remain unresolved. +A full gate on the combined final tree has not passed. + +CLI smoke, PEX build/smoke and documentation contract passed separately. After +aligning the console authorization mirror, all 458 webconsole tests and its types +passed. Both source and built PEX adapters completed authenticated GitHub +`ls-remote` through the broker, returning mainline HEAD `1942f396`. The broker in +that probe used the current capsule's existing SSH setup. This is **not** evidence +of physical-host Docker launch acceptance. No real GitHub write was tested; local +fixture repositories exercised real Git clone/fetch/push. No containers were +launched and no key files were read by the agent, copied or changed. + +## Remaining proposals + +- Docker Desktop needs a separate transport design; this slice explicitly refuses + hosts unable to bind Unix sockets. Linux/WSL with native Docker is first. +- GitHub aliases, HTTPS/gh, Git LFS authentication, signing, and arbitrary SSH flags + (including Git's -4/-6 switches) are outside this adapter's current contract. +- Additional reference cross-links and a tighter operation-deadline cap inside + the bounded send loop were considered; the existing send timeout is 10 seconds + and cancellation checks remain active. These are recorded follow-ups, not + claims of implemented behavior. +- The inherited canonical socket positive path is not covered by a pure unit + test; physical-host and recursive-launch acceptance remain required. + +## Reviewer execution + +Conversation `6280de13-6b45-4215-8e97-af6b466fbd61` used streaming JSON output. +The first redirected-stdin invocation failed to convey the brief; continuation +with an explicit brief path proceeded. The reviewer's CLI stalled after its full +gate had already returned. The implementor interrupted and continued that same +conversation with the committed round brief and a bounded print timeout. The +final response reported REVIEW_COMPLETE, the final shell marker was +ANTIGRAVITY_EXIT=0, and the commit and remote SHA were independently verified. +The aggregate JSON result retained an ERROR/idle-timeout from the earlier turn; +we therefore rely on observed commits/checks, not that aggregate status alone. +Scratch logs and virtual environments are not durable evidence. Briefs, the +assessment and validation counts are retained here. The reviewer worktree is +removed at pause; its pushed branch preserves all changes. + +## Baseline failure IDs + +- `tests/configuration/test_contract.py::test_second_checkout_cannot_inherit_the_first_checkouts_authority` +- `tests/configuration/test_contract.py::test_config_list_is_data_only_and_show_carries_the_review` +- `tests/test_capability_policy.py::test_cli_check_and_capability_mutation` +- `tests/test_host_daemon.py::test_unmountable_staging_fails_loudly` +- `tests/test_owner_commands.py::test_tool_edit_does_not_change_checkout_answers_when_not_ready[True]` +- `tests/test_recursive_orchestrator.py::test_public_dry_run_is_unique_redacted_and_cleans_owned_staging` +- `tests/test_recursive_orchestrator.py::test_contained_display_dry_run_forwards_no_host_x11` +- `tests/test_run_flow.py::test_a_second_checkout_is_registered_under_its_directory_name` +- `tests/test_run_flow.py::test_interactive_questions_record_answers_where_the_config_family_would` +- `tests/test_run_flow.py::test_save_with_pending_consent_writes_nothing[False-False]` +- `tests/test_run_flow.py::test_save_with_pending_consent_writes_nothing[False-True]` +- `tests/test_run_flow.py::test_save_with_pending_consent_writes_nothing[True-False]` +- `tests/test_run_flow.py::test_save_with_pending_consent_writes_nothing[True-True]` +- `tests/test_run_flow.py::test_partial_terminal_review_saves_no_answers[1\n]` +- `tests/test_run_flow.py::test_partial_terminal_review_saves_no_answers[1\nn\n]` +- `tests/test_runtime_entrypoint.py::test_entrypoint_rejects_unknown_adapters` +- `tests/test_trust.py::test_a_real_run_launches_on_the_homes_trust_alone` +- `tests/test_trust.py::test_trust_write_is_private_atomic_and_keeps_previous_record_on_failure` +- `tests/test_trust.py::test_terminal_acceptance_stays_staged_if_a_later_answer_is_missing` From f6890bb918c179581c04622004217f5bc0bb590d Mon Sep 17 00:00:00 2001 From: Costin Cozianu Date: Mon, 16 Nov 2026 08:44:41 +0000 Subject: [PATCH 09/43] Prepare local PEX host relaunch and Codex resume handoff Co-authored-by: GPT-6 Codex --- CURRENT-STATUS.md | 2 +- .../2026-11-16-github-auth/CURRENT-STATUS.md | 79 +++++++++++++++++-- 2 files changed, 72 insertions(+), 9 deletions(-) diff --git a/CURRENT-STATUS.md b/CURRENT-STATUS.md index 877c36e0..a2a5630a 100644 --- a/CURRENT-STATUS.md +++ b/CURRENT-STATUS.md @@ -9,7 +9,7 @@ coordination branch. Detailed state belongs in each linked status file. | Name | Started | Goal | State | Branch association | Status file | |---|---|---|---|---|---| -| `github-auth` | 2026-11-16 | GitHub SSH authentication through a launcher-owned host proxy | paused; implementation/local review prepared; gate, host acceptance and PR delivery pending | `ws-github-auth/ssh-proxy` | [workstream status](engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md) | +| `github-auth` | 2026-11-16 | GitHub SSH authentication through a launcher-owned host proxy | paused for owner host relaunch; PEX built; recorded recursive authentication E2E next | `ws-github-auth/ssh-proxy` | [workstream status](engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md) | | `workflow-improvements` | 2026-08-09 | Improve the multiple-stream workflow from concrete dogfood findings during the recursive E2E cycle | active 2026-10-30; Pair Review Loop naming implemented and packaging verified for 0.3.0; mainline and release integration pending | `ws-workflow-improvements/v1` | [workstream status](engineering-docs/wip/2026-08-09-workflow-improvements/CURRENT-STATUS.md) | | `project-management` | 2026-08-09 | Maintain project-wide priorities, sequencing, dependencies, and lifecycle coordination | paused 2026-10-11 at owner direction; consent and forced configuration before RC work; remaining diagnostics/image scope and release cut decisions retained | `ws-project-management/coordination` | [workstream status](engineering-docs/wip/2026-08-09-project-management/CURRENT-STATUS.md) | | `maintenance` | 2026-09-18 | Own the defects no open workstream covers, on `main` and on maintained release lines, and drive maintenance releases | active; releasing 0.3.1; cut 2026-11-07 from `main` at `31e8fec7`, `main` reopened at `0.3.2.dev0` (#8); candidate 2 (`v0.3.1-rc2` on `be0140ca`, propagated to `main` via PR #22 `aabace3c`) published and verified: Eclipse libsoup (#10, closed on rc1), launcher-delivered `devcapsule_webconsole` (`D-0017`, #18/#16, closed on rc2), Codex npm abbreviated metadata (#20, closed on rc2 + `component-status` `0bc31c64`), and Codex `0.167.0` + Claude Code `2.1.321` pins (`embedded-33`, #21); rulings open: executables for the Mac, the compatibility feed's home | `release-0.3.1`, the workstream's selection from the cut to the final tag; `release-0.3.0` closed and retained; earlier slice branches merged and closed | [workstream status](engineering-docs/wip/2026-09-18-maintenance/CURRENT-STATUS.md) | diff --git a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md index 37dee8c1..5e4328d6 100644 --- a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md @@ -4,7 +4,7 @@ Mnemonic: `github-auth` Start date: 2026-11-16 -State: paused; implementation and local review prepared, gate and PR delivery pending. +State: paused for owner host relaunch; local PEX built, recursive authentication E2E next. Definition read: WORKFLOW.md@89de30622697, WORKFLOW-LOCAL.md@d92bdcbb0a01 @@ -71,16 +71,79 @@ checkout remains untouched. Review evidence and prepared PR text are indexed bel ## Planned next step -Resolve the 19 reproduced baseline/environment failures in an appropriate gate -environment, then complete owner-operated implementation/review PR delivery and -physical-host acceptance. Do not claim a green full gate or mainline integration. -At resume, fetch upstream, check live workflow state and the review SHA, merge -through the prescribed PR process, and validate the actual combined tree. +Owner chose on 2026-11-16 to exit this capsule, relaunch the same checkout from +the physical host with the new local PEX and github-ssh enabled, then resume Codex. +This supersedes the proposed SSH-to-host harness adaptation. After resumption, +verify the inherited broker and executable bytes, then write and execute the +recorded E2E scenario using a fresh project and explicit github-ssh authorization. +Observe the Git checkout independently and retain WebM/screenshots/action evidence. +Recursive proxy inheritance is still unproved; do not report success until observed. +The 19 reproduced full-gate failures and formal PR delivery remain open separately. + +## Host relaunch handoff + +The owner explicitly chose a development-build self-hosting experiment. The normal +host launcher is not replaced and remains the recovery route; no saved project +configuration is changed by the run-once command. Reuse the same checkout and its +existing persistent state. The original checkout still has only its pre-existing +website gitlink modification. Main remains unchanged; synchronization can wait. + +Build: `.venv/bin/python -m nox -s pex` passed, including shell syntax, packaging +and executable smoke. Built from clean `712466ab9547ec7c9153977f725fc6cfe26344f6`. +The local PEX reports `v0.3.2.dev0-local-linux-x86_64` with revision `unknown` by +local-build policy; the clean Git revision and checksum here identify its inputs +and bytes independently. It contains the committed implementation, including the +node-registry and console-table fixes. Review corrections at `388f62a3` remain on +the separate review branch and are not included; their PR merge is still pending. + +SHA-256: `2f9470c6ca5efeaccd9844bf6feb1494d2e95564d054051a2c005afbc35d78ed`. + +Capsule artifact: `/home/devcapsule/worktrees/github-auth/devcapsule-src/dist/devcapsule-local.pex`. +Host artifact (from Docker mount inspection): +`/home/costin/.local/share/devcapsule/projects/by-path/fe326e03c822-devcapsule/home/worktrees/github-auth/devcapsule-src/dist/devcapsule-local.pex`. +Checksum and build metadata are saved beside it, outside Git. + +After exiting Codex and stopping the current capsule normally, run on the host: + +```sh +cd /opt/costin3/myProjects/devcapsule +/home/costin/.local/share/devcapsule/projects/by-path/fe326e03c822-devcapsule/home/worktrees/github-auth/devcapsule-src/dist/devcapsule-local.pex project run \ + --no-update-check \ + --authorize github-ssh true \ + --authorize network host \ + --authorize docker-daemon host-socket +``` + +Keep that outside launcher running throughout the resumed session: it owns the +broker. Host OpenSSH must already authenticate noninteractively to GitHub and +trust its host key. A key without a passphrase needs no SSH agent. + +Inside the relaunched capsule: + +```sh +cd /workspace/301e4208ef81-ChatGPT_Codex +codex resume 01a1e25b-6b76-72d0-b311-614d6adca912 +``` + +The session file exists under ~/.codex/sessions and Docker inspection shows +~/.codex backed by the persistent Codex component home. Installed CLI help +supports explicit session IDs. Conversation/history is resumable; process state +and outstanding tool sessions are not. The feature checkout remains +`/home/devcapsule/worktrees/github-auth`, on `ws-github-auth/ssh-proxy`. + +Resume checks before E2E: read this record, fetch upstream, inspect live workflow +state, claim the slice, check the PEX checksum, verify +`DEVCAPSULE_GITHUB_SOCKET=/run/devcapsule-github.sock` and that it is a socket, +then perform an authenticated GitHub read through the adapter. Fresh E2E homes +must have no copied key; the recording harness must authorize github-ssh for its +fresh project. Do not use this current capsule's manually installed key as proof +of the proxy. No E2E container was launched during handoff preparation. ## Open Threads -- Awaiting the human: suitable gate environment and owner-operated GitHub PR delivery. - Workstream selection and Git-over-SSH scope are already accepted. +- Awaiting the human: exit/relaunch/resume using the command above. + Workstream selection, Git-over-SSH scope and the relaunch approach are accepted. + Full-gate environment and owner-operated GitHub PR delivery also remain open. - Engineering choices: initial transport must account for Linux/rootless Docker permissions; Docker Desktop cannot bind ordinary host Unix sockets. - Physical-host launch acceptance and Docker Desktop transport remain open. From d6fd93e0ba6c18915ace45b9dfef2881068d7b39 Mon Sep 17 00:00:00 2001 From: Costin Cozianu Date: Mon, 16 Nov 2026 09:08:56 +0000 Subject: [PATCH 10/43] Propagate rootless Docker identity fix to GitHub proxy branch Apply the code and tests from release-0.3.1 commit a1a54eba3098eb3c80fd78ef3e483d1bb5314466. Adapt CapsuleUser behavior to this branch's foreground_command and prepare_filesystem APIs; retain GitHub authorization and verify both rootless and rootful proxy plans. Release and maintenance records remain with their owner. Co-authored-by: Antigravity Co-authored-by: GPT-6 Codex --- CURRENT-STATUS.md | 2 +- .../assets/docker4pycharm/entrypoint.sh | 4 +- .../devcapsule/assets/pycharm/entrypoint.sh | 4 +- devcapsule-src/devcapsule/commands/project.py | 9 +- .../devcapsule/container_runtime/display.py | 2 +- .../container_runtime/filesystem.py | 2 +- .../devcapsule/container_runtime/identity.py | 5 +- devcapsule-src/devcapsule/host_daemon.py | 42 ++++++++ .../devcapsule/launch/pycharm/_launcher.py | 96 +++++++++++++------ devcapsule-src/devcapsule/logs.py | 4 +- .../devcapsule/recursive_successor.py | 13 ++- devcapsule-src/tests/conftest.py | 2 + .../tests/integration/test_github_proxy.py | 8 +- .../container_runtime/test_filesystem.py | 19 ++++ .../container_runtime/test_identity.py | 32 +++++++ .../launch/pycharm/test_launcher.py | 84 ++++++++++++++++ .../2026-11-16-github-auth/CURRENT-STATUS.md | 30 +++++- 17 files changed, 310 insertions(+), 48 deletions(-) create mode 100644 devcapsule-src/tests/unit/devcapsule/container_runtime/test_identity.py diff --git a/CURRENT-STATUS.md b/CURRENT-STATUS.md index a2a5630a..66ff89e8 100644 --- a/CURRENT-STATUS.md +++ b/CURRENT-STATUS.md @@ -9,7 +9,7 @@ coordination branch. Detailed state belongs in each linked status file. | Name | Started | Goal | State | Branch association | Status file | |---|---|---|---|---|---| -| `github-auth` | 2026-11-16 | GitHub SSH authentication through a launcher-owned host proxy | paused for owner host relaunch; PEX built; recorded recursive authentication E2E next | `ws-github-auth/ssh-proxy` | [workstream status](engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md) | +| `github-auth` | 2026-11-16 | GitHub SSH authentication through a launcher-owned host proxy | active; propagating rootless Docker fix and rebuilding local PEX | `ws-github-auth/ssh-proxy` | [workstream status](engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md) | | `workflow-improvements` | 2026-08-09 | Improve the multiple-stream workflow from concrete dogfood findings during the recursive E2E cycle | active 2026-10-30; Pair Review Loop naming implemented and packaging verified for 0.3.0; mainline and release integration pending | `ws-workflow-improvements/v1` | [workstream status](engineering-docs/wip/2026-08-09-workflow-improvements/CURRENT-STATUS.md) | | `project-management` | 2026-08-09 | Maintain project-wide priorities, sequencing, dependencies, and lifecycle coordination | paused 2026-10-11 at owner direction; consent and forced configuration before RC work; remaining diagnostics/image scope and release cut decisions retained | `ws-project-management/coordination` | [workstream status](engineering-docs/wip/2026-08-09-project-management/CURRENT-STATUS.md) | | `maintenance` | 2026-09-18 | Own the defects no open workstream covers, on `main` and on maintained release lines, and drive maintenance releases | active; releasing 0.3.1; cut 2026-11-07 from `main` at `31e8fec7`, `main` reopened at `0.3.2.dev0` (#8); candidate 2 (`v0.3.1-rc2` on `be0140ca`, propagated to `main` via PR #22 `aabace3c`) published and verified: Eclipse libsoup (#10, closed on rc1), launcher-delivered `devcapsule_webconsole` (`D-0017`, #18/#16, closed on rc2), Codex npm abbreviated metadata (#20, closed on rc2 + `component-status` `0bc31c64`), and Codex `0.167.0` + Claude Code `2.1.321` pins (`embedded-33`, #21); rulings open: executables for the Mac, the compatibility feed's home | `release-0.3.1`, the workstream's selection from the cut to the final tag; `release-0.3.0` closed and retained; earlier slice branches merged and closed | [workstream status](engineering-docs/wip/2026-09-18-maintenance/CURRENT-STATUS.md) | diff --git a/devcapsule-src/devcapsule/assets/docker4pycharm/entrypoint.sh b/devcapsule-src/devcapsule/assets/docker4pycharm/entrypoint.sh index 06316beb..a3856caf 100755 --- a/devcapsule-src/devcapsule/assets/docker4pycharm/entrypoint.sh +++ b/devcapsule-src/devcapsule/assets/docker4pycharm/entrypoint.sh @@ -17,7 +17,7 @@ export XDG_CACHE_HOME="${XDG_CACHE_HOME:-$IDE_PROJECT_STATE_PATH/home/.cache}" export XDG_DATA_HOME="${XDG_DATA_HOME:-$HOME/.local/share}" RUN_AS_IDE_USER=() -if [ "$(id -u)" -eq 0 ] && [ -n "$IDE_UID" ] && [ -n "$IDE_GID" ]; then +if [ "$(id -u)" -eq 0 ] && [ -n "$IDE_UID" ] && [ -n "$IDE_GID" ] && { [ "$IDE_UID" != "0" ] || [ "$IDE_GID" != "0" ]; }; then if [ "$ENABLE_SUDO" = "1" ] && getent passwd "$IDE_USER" >/dev/null 2>&1; then RUN_AS_IDE_USER=(gosu "$IDE_USER") else @@ -38,7 +38,7 @@ install_ide_file() { local dst="$2" local mode="$3" - if [ "$(id -u)" -eq 0 ] && [ -n "$IDE_UID" ] && [ -n "$IDE_GID" ]; then + if [ "$(id -u)" -eq 0 ] && [ -n "$IDE_UID" ] && [ -n "$IDE_GID" ] && { [ "$IDE_UID" != "0" ] || [ "$IDE_GID" != "0" ]; }; then install -o "$IDE_UID" -g "$IDE_GID" -m "$mode" "$src" "$dst" else install -m "$mode" "$src" "$dst" diff --git a/devcapsule-src/devcapsule/assets/pycharm/entrypoint.sh b/devcapsule-src/devcapsule/assets/pycharm/entrypoint.sh index d992407c..0dd43273 100644 --- a/devcapsule-src/devcapsule/assets/pycharm/entrypoint.sh +++ b/devcapsule-src/devcapsule/assets/pycharm/entrypoint.sh @@ -17,7 +17,7 @@ export XDG_CACHE_HOME="${XDG_CACHE_HOME:-$IDE_PROJECT_STATE_PATH/home/.cache}" export XDG_DATA_HOME="${XDG_DATA_HOME:-$HOME/.local/share}" RUN_AS_IDE_USER=() -if [ "$(id -u)" -eq 0 ] && [ -n "$IDE_UID" ] && [ -n "$IDE_GID" ]; then +if [ "$(id -u)" -eq 0 ] && [ -n "$IDE_UID" ] && [ -n "$IDE_GID" ] && { [ "$IDE_UID" != "0" ] || [ "$IDE_GID" != "0" ]; }; then if [ "$ENABLE_SUDO" = "1" ] && getent passwd "$IDE_USER" >/dev/null 2>&1; then RUN_AS_IDE_USER=(gosu "$IDE_USER") else @@ -38,7 +38,7 @@ install_ide_file() { local dst="$2" local mode="$3" - if [ "$(id -u)" -eq 0 ] && [ -n "$IDE_UID" ] && [ -n "$IDE_GID" ]; then + if [ "$(id -u)" -eq 0 ] && [ -n "$IDE_UID" ] && [ -n "$IDE_GID" ] && { [ "$IDE_UID" != "0" ] || [ "$IDE_GID" != "0" ]; }; then install -o "$IDE_UID" -g "$IDE_GID" -m "$mode" "$src" "$dst" else install -m "$mode" "$src" "$dst" diff --git a/devcapsule-src/devcapsule/commands/project.py b/devcapsule-src/devcapsule/commands/project.py index 6b0b2541..caf4e162 100644 --- a/devcapsule-src/devcapsule/commands/project.py +++ b/devcapsule-src/devcapsule/commands/project.py @@ -50,6 +50,7 @@ from devcapsule.configuration.approvals import render_software_approvals, select_software_approvals from devcapsule import notifications from devcapsule import version_sets +from devcapsule.host_daemon import daemon_is_rootless from devcapsule import runtime_configuration from devcapsule.runtime_configuration import CapsuleAccess from devcapsule.compat import CliError, DecisionPending @@ -2323,6 +2324,7 @@ def prepare_display(base: ImageDetails) -> None: image_labels: Mapping[str, str] = {} realized = None degraded_download = False + rootless_docker = daemon_is_rootless() if isinstance(lock.get("base"), dict) and isinstance(lock.get("materialization"), dict): while True: try: @@ -2346,7 +2348,11 @@ def prepare_display(base: ImageDetails) -> None: degraded_download = True image = realized.image.reference image_labels = realized.image.labels - checkout_runtime_plan = project_runtime_plan(selected, realized.locked) + checkout_runtime_plan = ( + project_runtime_plan(selected, realized.locked, uid=0, gid=0, user="root") + if rootless_docker + else project_runtime_plan(selected, realized.locked) + ) use_image_process = True action = "Materialized" if realized.created else "Reused" print(f"{action} canonical environment: {image}") @@ -2472,6 +2478,7 @@ def prepare_display(base: ImageDetails) -> None: display_transport=display_transport, web_console=web_console, capsule_platform=lock_platform(lock), + rootless_docker=rootless_docker, ) ) if command_report is not None: diff --git a/devcapsule-src/devcapsule/container_runtime/display.py b/devcapsule-src/devcapsule/container_runtime/display.py index ea53a9aa..7adb14e0 100644 --- a/devcapsule-src/devcapsule/container_runtime/display.py +++ b/devcapsule-src/devcapsule/container_runtime/display.py @@ -313,7 +313,7 @@ def field(value: bytes) -> bytes: def _own(path: Path, plan: RuntimePlan) -> None: - if os.geteuid() == 0: + if os.geteuid() == 0 and (plan.identity.uid != 0 or plan.identity.gid != 0): os.chown(path, plan.identity.uid, plan.identity.gid) diff --git a/devcapsule-src/devcapsule/container_runtime/filesystem.py b/devcapsule-src/devcapsule/container_runtime/filesystem.py index 920cd6d1..38cff2ca 100644 --- a/devcapsule-src/devcapsule/container_runtime/filesystem.py +++ b/devcapsule-src/devcapsule/container_runtime/filesystem.py @@ -53,7 +53,7 @@ def prepare_filesystem( missing.append(cursor) cursor = cursor.parent path.mkdir(parents=True, exist_ok=True) - if effective_uid == 0: + if effective_uid == 0 and (identity.uid != 0 or identity.gid != 0): for created in reversed(missing): apply_chown(created, identity.uid, identity.gid) Path(plan.environment["XDG_RUNTIME_DIR"]).chmod(0o700) diff --git a/devcapsule-src/devcapsule/container_runtime/identity.py b/devcapsule-src/devcapsule/container_runtime/identity.py index b6077bad..fb8c772f 100644 --- a/devcapsule-src/devcapsule/container_runtime/identity.py +++ b/devcapsule-src/devcapsule/container_runtime/identity.py @@ -18,12 +18,13 @@ def foreground_command( """Wrap ``command`` so it runs as the capsule user when the caller is root. Named for its first use; every supervised child, infrastructure included, - is wrapped the same way, so no display or IDE process ever runs as root. + is wrapped the same way. A rootless Docker target of 0:0 already has the + correct identity; invoking gosu there would require dropped capabilities. """ if not command: raise ValueError("foreground command must not be empty") effective_uid = os.geteuid() if caller_uid is None else caller_uid - if effective_uid == 0: + if effective_uid == 0 and (identity.uid != 0 or identity.gid != 0): return ("gosu", f"{identity.uid}:{identity.gid}", *command) return command diff --git a/devcapsule-src/devcapsule/host_daemon.py b/devcapsule-src/devcapsule/host_daemon.py index ff9844e4..afb80343 100644 --- a/devcapsule-src/devcapsule/host_daemon.py +++ b/devcapsule-src/devcapsule/host_daemon.py @@ -231,3 +231,45 @@ def daemon_architecture() -> str: return host_architecture() reported = completed.stdout.strip() return architecture_key(reported) if completed.returncode == 0 and reported else host_architecture() + + +def security_options_indicate_rootless(security_options: Sequence[str] | str) -> bool: + """Return whether Docker daemon ``SecurityOptions`` report rootless mode.""" + + raw_items: Sequence[str] = ( + security_options.splitlines() if isinstance(security_options, str) else security_options + ) + for item in raw_items: + for token in item.split(","): + if token.strip() in {"rootless", "name=rootless"}: + return True + return False + + +def daemon_is_rootless() -> bool: + """Whether the Docker daemon runs in rootless mode. + + Under rootless Docker, host UID/GID are mapped to ``0:0`` inside the + container user namespace, whereas container UID/GID equal to the host IDs + map to subordinate IDs that cannot access ``0600``/``0700`` bind mounts. + Returns ``False`` when the daemon does not answer or runs rootful. + """ + try: + completed = subprocess.run( + ["docker", "info", "--format", "{{json .SecurityOptions}}"], + capture_output=True, text=True, timeout=30, check=False, + ) + except (OSError, subprocess.TimeoutExpired): + return False + reported = completed.stdout.strip() + if completed.returncode != 0 or not reported or reported == "null": + return False + try: + parsed = json.loads(reported) + except ValueError: + return security_options_indicate_rootless(reported) + if not isinstance(parsed, list): + return False + return security_options_indicate_rootless( + [item for item in parsed if isinstance(item, str)] + ) diff --git a/devcapsule-src/devcapsule/launch/pycharm/_launcher.py b/devcapsule-src/devcapsule/launch/pycharm/_launcher.py index 283c5ac6..4be254ee 100644 --- a/devcapsule-src/devcapsule/launch/pycharm/_launcher.py +++ b/devcapsule-src/devcapsule/launch/pycharm/_launcher.py @@ -249,6 +249,8 @@ class PycharmRunOptions: # caller (commands/project.py); without it the launcher announces the # absence once and plans no console: no port, no token, no wait. web_console: WebConsoleAvailability = WebConsoleAvailability(True, None) + # True only when the host Docker daemon reports rootless mode. + rootless_docker: bool = False @dataclass @@ -320,6 +322,7 @@ class PycharmRunConfig: libgl_always_software: str = "1" mesa_loader_driver_override: str = "llvmpipe" libgl_dri3_disable: str = "1" + rootless_docker: bool = False HOST_NETWORK_UNAVAILABLE_ON_MACOS = ( @@ -1006,7 +1009,10 @@ def build_run_config(options: PycharmRunOptions, env: Mapping[str, str]) -> Pych # daemon already holds root on its virtual machine, so the group adds # no authority. Elsewhere the host socket file's group applies. socket_gid = HostOs.current().docker_socket_gid - host_docker_gid = socket_gid if socket_gid is not None else host_docker_socket.stat().st_gid + raw_socket_gid = socket_gid if socket_gid is not None else host_docker_socket.stat().st_gid + host_docker_gid = ( + 0 if options.rootless_docker and raw_socket_gid == host_user.gid else raw_socket_gid + ) if enable_sudo: while ide_sudo_gid in {0, host_user.gid, host_docker_gid}: @@ -1238,6 +1244,7 @@ def build_run_config(options: PycharmRunOptions, env: Mapping[str, str]) -> Pych env.get("MESA_LOADER_DRIVER_OVERRIDE", "llvmpipe"), ), libgl_dri3_disable=env.get("PYCHARM_LIBGL_DRI3_DISABLE", env.get("LIBGL_DRI3_DISABLE", "1")), + rootless_docker=options.rootless_docker, ) @@ -1353,6 +1360,10 @@ def build_docker_args( # launcher is not a case this product has; the recursive dogfood holds # its successor's launcher open exactly as a developer holds `project run`. host_user = current_host_user() + rootless = bool(getattr(config, "rootless_docker", False)) + container_uid, container_gid, container_user = ( + (0, 0, "root") if rootless else (host_user.uid, host_user.gid, host_user.name) + ) args = [ "--rm", "-i", @@ -1385,11 +1396,11 @@ def build_docker_args( "--env", "IDE_PROJECT_STATE_PATH=/ide-project-state", "--env", - f"IDE_UID={host_user.uid}", + f"IDE_UID={container_uid}", "--env", - f"IDE_GID={host_user.gid}", + f"IDE_GID={container_gid}", "--env", - f"IDE_USER={host_user.name}", + f"IDE_USER={container_user}", "--env", f"LIBGL_ALWAYS_SOFTWARE={config.libgl_always_software}", "--env", @@ -1620,12 +1631,16 @@ def _display_mount_args(config: PycharmRunConfig, files: TempRuntimeFiles) -> li def append_docker_mode_args(args: list[str], config: PycharmRunConfig, host_user: HostUser) -> None: + rootless = bool(getattr(config, "rootless_docker", False)) + container_uid, container_gid = ( + (0, 0) if rootless else (host_user.uid, host_user.gid) + ) if config.docker_mode == "host": print_host_docker_warning(config) args.extend( [ "--user", - f"{host_user.uid}:{host_user.gid}", + f"{container_uid}:{container_gid}", "--group-add", str(config.host_docker_gid), "--env", @@ -1662,7 +1677,7 @@ def append_docker_mode_args(args: list[str], config: PycharmRunConfig, host_user args.extend( [ "--user", - f"{host_user.uid}:{host_user.gid}", + f"{container_uid}:{container_gid}", "--env", "ENABLE_DIND=0", "--env", @@ -1870,37 +1885,54 @@ def write_xauthority(xauth_file: Path, env: Mapping[str, str]) -> None: def write_user_files(config: PycharmRunConfig, files: TempRuntimeFiles) -> None: - host_user = current_host_user() - files.passwd_file.write_text( - "\n".join( - [ - "root:x:0:0:root:/root:/bin/bash", - f"{host_user.name}:x:{host_user.uid}:{host_user.gid}:PyCharm Docker User:/home/devcapsule:/bin/bash", - "", - ] + if bool(getattr(config, "rootless_docker", False)): + files.passwd_file.write_text( + "root:x:0:0:PyCharm Docker User:/home/devcapsule:/bin/bash\n" ) - ) - group_lines = [ - "root:x:0:", - f"{host_user.group_name}:x:{host_user.gid}:", - ] - if config.host_docker_gid not in (None, 0, host_user.gid): - group_lines.append(f"host-docker:x:{config.host_docker_gid}:") - if config.enable_sudo: - group_lines.append(f"ide-sudo:x:{config.ide_sudo_gid}:{host_user.name}") - files.shadow_file = make_temp(files.group_file.parent, "pycharm-docker-shadow.") - shadow_last_change = int(time.time()) // 86400 - files.shadow_file.write_text( + group_lines = ["root:x:0:"] + if config.host_docker_gid not in (None, 0): + group_lines.append(f"host-docker:x:{config.host_docker_gid}:") + if config.enable_sudo: + group_lines.append(f"ide-sudo:x:{config.ide_sudo_gid}:root") + files.shadow_file = make_temp(files.group_file.parent, "pycharm-docker-shadow.") + shadow_last_change = int(time.time()) // 86400 + files.shadow_file.write_text( + f"root:*:{shadow_last_change}:0:99999:7:::\n" + ) + files.shadow_file.chmod(0o600) + files.group_file.write_text("\n".join([*group_lines, ""])) + else: + host_user = current_host_user() + files.passwd_file.write_text( "\n".join( [ - f"root:*:{shadow_last_change}:0:99999:7:::", - f"{host_user.name}:*:{shadow_last_change}:0:99999:7:::", + "root:x:0:0:root:/root:/bin/bash", + f"{host_user.name}:x:{host_user.uid}:{host_user.gid}:PyCharm Docker User:/home/devcapsule:/bin/bash", "", ] ) ) - files.shadow_file.chmod(0o600) - files.group_file.write_text("\n".join([*group_lines, ""])) + group_lines = [ + "root:x:0:", + f"{host_user.group_name}:x:{host_user.gid}:", + ] + if config.host_docker_gid not in (None, 0, host_user.gid): + group_lines.append(f"host-docker:x:{config.host_docker_gid}:") + if config.enable_sudo: + group_lines.append(f"ide-sudo:x:{config.ide_sudo_gid}:{host_user.name}") + files.shadow_file = make_temp(files.group_file.parent, "pycharm-docker-shadow.") + shadow_last_change = int(time.time()) // 86400 + files.shadow_file.write_text( + "\n".join( + [ + f"root:*:{shadow_last_change}:0:99999:7:::", + f"{host_user.name}:*:{shadow_last_change}:0:99999:7:::", + "", + ] + ) + ) + files.shadow_file.chmod(0o600) + files.group_file.write_text("\n".join([*group_lines, ""])) files.passwd_file.chmod(0o644) files.group_file.chmod(0o644) @@ -1913,6 +1945,10 @@ def ensure_sudoers_policy_ownership( policy = files.sudoers_file if not config.enable_sudo or policy is None: raise PycharmRunError("Development sudo requires a generated sudoers policy.") + if bool(getattr(config, "rootless_docker", False)): + # Rootless Docker maps files owned by the host user directly to 0:0 + # inside the container while host-side st_uid remains the host UID. + return if not HostOs.current().chown_reaches_bind_mounts: files.sudoers_volume = prepare_sudoers_policy_volume(config, policy, env) return diff --git a/devcapsule-src/devcapsule/logs.py b/devcapsule-src/devcapsule/logs.py index ec41310d..af369f3f 100644 --- a/devcapsule-src/devcapsule/logs.py +++ b/devcapsule-src/devcapsule/logs.py @@ -282,9 +282,9 @@ def events_logger(component: str) -> logging.Logger: def own(path: Path, owner: Owner) -> None: - """Give ``path`` to ``owner`` when this process is root; a no-op otherwise.""" + """Give ``path`` to ``owner`` when this process is root and ownership differs.""" - if owner is not None and os.geteuid() == 0: + if owner is not None and os.geteuid() == 0 and owner != (os.geteuid(), os.getegid()): os.chown(path, owner[0], owner[1]) diff --git a/devcapsule-src/devcapsule/recursive_successor.py b/devcapsule-src/devcapsule/recursive_successor.py index b07c9ee3..bfaa0772 100644 --- a/devcapsule-src/devcapsule/recursive_successor.py +++ b/devcapsule-src/devcapsule/recursive_successor.py @@ -13,6 +13,7 @@ from typing import Any, Callable, Mapping, Sequence from devcapsule.build_info import current_build_info +from devcapsule.host_daemon import daemon_is_rootless from devcapsule.launch.pycharm import DockerMode, PycharmRunOptions from devcapsule.launch.pycharm._launcher import ( PycharmRunError, @@ -154,7 +155,12 @@ def launch_successor( selected = fresh_resolved_project(checkout) realized = realize_environment(selected) - runtime_plan = project_runtime_plan(selected, realized.locked) + rootless_docker = daemon_is_rootless() + runtime_plan = ( + project_runtime_plan(selected, realized.locked, uid=0, gid=0, user="root") + if rootless_docker + else project_runtime_plan(selected, realized.locked) + ) source_revision = current_build_info().source_revision name = successor_container_name(run_id) options = _resolved_run_options( @@ -165,6 +171,7 @@ def launch_successor( run_id=run_id, source_revision=source_revision, image_labels=realized.image.labels, + rootless_docker=rootless_docker, ) config = build_run_config(options, env) successor_display_url: str | None = None @@ -189,7 +196,7 @@ def launch_successor( args = build_docker_args(config, files, launch_env) context = _launch_context(report, config.persistent_home, args) host_args = _translate_bind_sources(args, context) - if files.sudoers_file is not None: + if files.sudoers_file is not None and not rootless_docker: _make_root_owned(files.sudoers_file, launch_env) plan = _expected_plan(host_args, files, realized) _write_expected_plan(run_root, plan) @@ -473,6 +480,7 @@ def _resolved_run_options( run_id: str, source_revision: str, image_labels: Mapping[str, str] | None = None, + rootless_docker: bool = False, ) -> PycharmRunOptions: runtime = selected.resolution.get("runtime", {}) state_root = selected.resolution.get("state", {}) @@ -544,6 +552,7 @@ def _resolved_run_options( "--label", f"devcapsule.e2e.role={SUCCESSOR_ROLE}", ], + rootless_docker=rootless_docker, ) diff --git a/devcapsule-src/tests/conftest.py b/devcapsule-src/tests/conftest.py index 1350f921..005cbf82 100644 --- a/devcapsule-src/tests/conftest.py +++ b/devcapsule-src/tests/conftest.py @@ -78,6 +78,8 @@ def amd64_linux_host_by_default(request: pytest.FixtureRequest, monkeypatch: pyt monkeypatch.setattr("devcapsule.platforms.host_architecture", lambda: "amd64") for module in ("environment_realization", "images.build", "launch.pycharm._launcher"): monkeypatch.setattr(f"devcapsule.{module}.daemon_architecture", lambda: "amd64") + for module in ("commands.project", "recursive_successor"): + monkeypatch.setattr(f"devcapsule.{module}.daemon_is_rootless", lambda: False) DOCKER_MARKERS = ("e2e", "base_build_e2e", "recursive_e2e", "contributor_e2e", "ide_smoke", "docker") diff --git a/devcapsule-src/tests/integration/test_github_proxy.py b/devcapsule-src/tests/integration/test_github_proxy.py index a8abf31b..c98f1c07 100644 --- a/devcapsule-src/tests/integration/test_github_proxy.py +++ b/devcapsule-src/tests/integration/test_github_proxy.py @@ -140,8 +140,9 @@ def test_broker_shutdown_closes_incomplete_requests(tmp_path: Path) -> None: @pytest.mark.parametrize('enabled', [False, True]) -def test_launcher_exposes_the_socket_only_with_the_grant(tmp_path: Path, enabled: bool) -> None: - """The Docker plan carries the Git adapter and broker mount only after explicit authorization.""" +@pytest.mark.parametrize('rootless', [False, True]) +def test_launcher_exposes_the_socket_only_with_the_grant(tmp_path: Path, enabled: bool, rootless: bool) -> None: + """Rootless and rootful plans carry the adapter/socket only with authorization and use the corresponding identity.""" from devcapsule.launch.pycharm._launcher import PycharmRunOptions, DockerMode, TempRuntimeFiles, build_run_config, build_docker_args from devcapsule.platforms import Platform from devcapsule.github_proxy import GIT_SSH_COMMAND, SOCKET_DESTINATION @@ -152,10 +153,11 @@ def test_launcher_exposes_the_socket_only_with_the_grant(tmp_path: Path, enabled env = {**env, 'DISPLAY': ':1', 'PYCHARM_GIT_IDENTITY_FROM_HOST': '0', SOCKET_ENV: str(path)} config = build_run_config(PycharmRunOptions(project=project, capsule_platform=Platform.LINUX_AMD64, docker_mode=DockerMode.none, enable_sudo=False, network_mode='bridge', - enable_github_ssh=enabled), env) + enable_github_ssh=enabled, rootless_docker=rootless), env) files = TempRuntimeFiles(xauth_file=tmp_path / 'xauth', passwd_file=tmp_path / 'passwd', group_file=tmp_path / 'group') args = build_docker_args(config, files, env) assert (f'GIT_SSH_COMMAND={GIT_SSH_COMMAND}' in args) is enabled assert (f'{SOCKET_ENV}={SOCKET_DESTINATION}' in args) is enabled assert (f'type=bind,src={path},dst={SOCKET_DESTINATION},ro' in args) is enabled + assert args[args.index('--user') + 1] == ('0:0' if rootless else f'{os.getuid()}:{os.getgid()}') assert not any('.ssh' in arg or 'SSH_AUTH_SOCK=' in arg for arg in args) diff --git a/devcapsule-src/tests/unit/devcapsule/container_runtime/test_filesystem.py b/devcapsule-src/tests/unit/devcapsule/container_runtime/test_filesystem.py index 3be60b0f..ab1d06af 100644 --- a/devcapsule-src/tests/unit/devcapsule/container_runtime/test_filesystem.py +++ b/devcapsule-src/tests/unit/devcapsule/container_runtime/test_filesystem.py @@ -92,3 +92,22 @@ def test_prepare_filesystem_assigns_ownership_only_to_newly_created_ancestors_wh (tmp_path / "runtime", 1000, 1001), (runtime_dir, 1000, 1001), ] + + +def test_mapped_root_creates_directories_without_redundant_chown(tmp_path: Path) -> None: + """Rootless 0:0 preparation creates private directories without requiring chown capabilities.""" + home = tmp_path / "home" + runtime_dir = tmp_path / "runtime" + plan = FilesystemPlan( + directories=(str(home), str(home / ".ssh"), str(runtime_dir)), + environment={"HOME": str(home), "XDG_RUNTIME_DIR": str(runtime_dir)}, + ) + chowned: list[tuple[Path, int, int]] = [] + prepare_filesystem( + plan, Identity(0, 0, "root"), caller_uid=0, + chown=lambda path, uid, gid: chowned.append((path, uid, gid)), + ) + assert chowned == [] + assert all(Path(directory).is_dir() for directory in plan.directories) + assert stat.S_IMODE((home / ".ssh").stat().st_mode) == 0o700 + assert stat.S_IMODE(runtime_dir.stat().st_mode) == 0o700 diff --git a/devcapsule-src/tests/unit/devcapsule/container_runtime/test_identity.py b/devcapsule-src/tests/unit/devcapsule/container_runtime/test_identity.py new file mode 100644 index 00000000..b63a130d --- /dev/null +++ b/devcapsule-src/tests/unit/devcapsule/container_runtime/test_identity.py @@ -0,0 +1,32 @@ +"""Unit tests of container_runtime.identity: privilege switching for explicit identities.""" + +import pytest + +from devcapsule.container_runtime.contract import Identity +from devcapsule.container_runtime.identity import foreground_command + + +@pytest.mark.parametrize("caller_uid", [0, 1000]) +def test_mapped_root_command_needs_no_privilege_switch(caller_uid: int) -> None: + """A target of 0:0 needs no gosu wrapper, including a rootless Docker root caller.""" + command = ("ide", "--wait") + assert foreground_command(command, Identity(0, 0, "root"), caller_uid=caller_uid) == command + + +@pytest.mark.parametrize("uid,gid", [(1000, 1001), (0, 1001), (1000, 0)]) +def test_root_caller_switches_when_either_target_id_differs(uid: int, gid: int) -> None: + """Root still invokes gosu for any target other than 0:0, preserving rootful behavior.""" + assert foreground_command(("ide",), Identity(uid, gid, "developer"), caller_uid=0) == ( + "gosu", f"{uid}:{gid}", "ide", + ) + + +def test_non_root_caller_keeps_the_command() -> None: + """A non-root caller does not invoke a privilege switch.""" + assert foreground_command(("ide",), Identity(1000, 1001, "developer"), caller_uid=1000) == ("ide",) + + +def test_empty_command_is_refused_for_mapped_root() -> None: + """Skipping gosu does not make an empty foreground command valid.""" + with pytest.raises(ValueError, match="must not be empty"): + foreground_command((), Identity(0, 0, "root"), caller_uid=0) diff --git a/devcapsule-src/tests/unit/devcapsule/launch/pycharm/test_launcher.py b/devcapsule-src/tests/unit/devcapsule/launch/pycharm/test_launcher.py index b1581f3c..35d7192c 100644 --- a/devcapsule-src/tests/unit/devcapsule/launch/pycharm/test_launcher.py +++ b/devcapsule-src/tests/unit/devcapsule/launch/pycharm/test_launcher.py @@ -9,6 +9,7 @@ from dataclasses import replace from datetime import datetime, timedelta, timezone import json +import os from pathlib import Path import socket from threading import Event @@ -1883,3 +1884,86 @@ def test_build_run_config_omits_checkout_console_source_under_a_built_launcher_u assert opted_in_config.runtime_plan is not None and opted_in_config.runtime_plan.console is not None assert opted_in_config.runtime_plan.console.source_path == "/workspace/project/devcapsule-webconsole" + +def test_rootless_docker_runs_container_as_mapped_root_while_rootful_keeps_host_uid( + tmp_path: Path, +) -> None: + """Rootless Docker maps host mounts to ``0:0`` and drops ``--user``/``IDE_*`` to ``0:0``/``root``; non-rootless preserves host identity.""" + env = base_env(tmp_path) + env["XDG_RUNTIME_DIR"] = str(tmp_path / "runtime") + rootful_config = contained_config(tmp_path) + rootless_config = contained_config(tmp_path, rootless_docker=True) + assert rootful_config.rootless_docker is False + assert rootless_config.rootless_docker is True + + rootful_files = prepare_temp_runtime_files(rootful_config, env) + rootless_files = prepare_temp_runtime_files(rootless_config, env) + try: + rootful_args = build_docker_args(rootful_config, rootful_files, env) + rootless_args = build_docker_args(rootless_config, rootless_files, env) + + host_uid, host_gid = os.getuid(), os.getgid() + assert rootful_args[rootful_args.index("--user") + 1] == f"{host_uid}:{host_gid}" + assert f"IDE_UID={host_uid}" in rootful_args + assert f"IDE_GID={host_gid}" in rootful_args + + assert rootless_args[rootless_args.index("--user") + 1] == "0:0" + assert "IDE_UID=0" in rootless_args + assert "IDE_GID=0" in rootless_args + assert "IDE_USER=root" in rootless_args + assert rootless_files.passwd_file.read_text(encoding="utf-8") == ( + "root:x:0:0:PyCharm Docker User:/home/devcapsule:/bin/bash\n" + ) + assert rootless_files.group_file.read_text(encoding="utf-8") == "root:x:0:\n" + finally: + cleanup_temp_runtime_files(rootful_files) + cleanup_temp_runtime_files(rootless_files) + + +def test_rootless_host_docker_socket_owned_by_host_user_maps_to_container_root_group( + tmp_path: Path, +) -> None: + """In rootless ``host`` Docker mode, a daemon socket owned by the host user's GID maps to container GID ``0``; rootful keeps its host GID.""" + sock_path = tmp_path / "docker.sock" + listener = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + listener.bind(str(sock_path)) + try: + rootful_config = contained_config( + tmp_path, docker_mode=DockerMode.host, host_docker_socket=sock_path, rootless_docker=False + ) + rootless_config = contained_config( + tmp_path, docker_mode=DockerMode.host, host_docker_socket=sock_path, rootless_docker=True + ) + assert rootful_config.host_docker_gid == os.getgid() + assert rootless_config.host_docker_gid == 0 + + env = base_env(tmp_path) + env["XDG_RUNTIME_DIR"] = str(tmp_path / "runtime") + files = prepare_temp_runtime_files(rootless_config, env) + try: + args = build_docker_args(rootless_config, files, env) + assert args[args.index("--user") + 1] == "0:0" + assert args[args.index("--group-add") + 1] == "0" + assert files.group_file.read_text(encoding="utf-8") == "root:x:0:\n" + finally: + cleanup_temp_runtime_files(files) + finally: + listener.close() + + +def test_rootless_sudoers_policy_ownership_accepts_host_user_file_without_chown( + tmp_path: Path, +) -> None: + """Under rootless Docker, the host-owned sudoers policy is already container-root-owned so preparation skips ``chown``.""" + env = base_env(tmp_path) + env["XDG_RUNTIME_DIR"] = str(tmp_path / "runtime") + config = contained_config(tmp_path, enable_sudo=True, rootless_docker=True) + files = prepare_temp_runtime_files(config, env) + try: + ensure_sudoers_policy_ownership(config, files, env) + assert files.sudoers_volume is None + assert "ide-sudo:x:" in files.group_file.read_text(encoding="utf-8") + assert files.shadow_file is not None + assert files.shadow_file.read_text(encoding="utf-8").startswith("root:*:") + finally: + cleanup_temp_runtime_files(files) diff --git a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md index 5e4328d6..70154149 100644 --- a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md @@ -4,7 +4,7 @@ Mnemonic: `github-auth` Start date: 2026-11-16 -State: paused for owner host relaunch; local PEX built, recursive authentication E2E next. +State: active; propagating the released rootless Docker fix and rebuilding the host-relaunch PEX. Definition read: WORKFLOW.md@89de30622697, WORKFLOW-LOCAL.md@d92bdcbb0a01 @@ -80,6 +80,34 @@ Observe the Git checkout independently and retain WebM/screenshots/action eviden Recursive proxy inheritance is still unproved; do not report success until observed. The 19 reproduced full-gate failures and formal PR delivery remain open separately. +## Rootless Docker fix propagation + +The owner's first host relaunch failed with an unwritable PEX cache and permission +denied on `/etc/devcapsule/home/settings.toml`. Fetched canonical `release-0.3.1` +and identified `a1a54eba3098eb3c80fd78ef3e483d1bb5314466`, merged in PR #38 and +recorded as owner-verified on the target host by `8bd56eb3`. Neither current main +nor this feature branch included it. Main has no new commits to synchronize. + +Applied that fix's deliverable/test patch, without release version changes or +another workstream's records, under the owner's explicit propagation request and +WORKFLOW-LOCAL.md's release-fix propagation exception. The branch predates the +release's CapsuleUser refactor: adapted its `command` correction to the existing +`foreground_command`, and its `owner_changes` correction to `prepare_filesystem`. +Rootless launches use container 0:0 (the host user's mapped identity), map the +Docker socket group and home accordingly, and avoid redundant gosu/chown under +dropped capabilities. Rootful behavior is preserved. The available Docker daemon +reports `name=rootless`; no container was launched for this propagation. + +Checks so far: 151 focused tests passed, then all 12 GitHub proxy integration +cases passed after extending socket-grant coverage to both daemon modes. The +full gate's type check passed over 340 files; conventions found zero breaches. +New pure identity tests and filesystem preparation tests cover the adaptation. +Full gate: 19 failed, 2942 passed, 1 skipped, 78 deselected, 14 xfailed, +7 xpassed. All 19 failure IDs match the previously reproduced mainline/environment +baseline exactly; no new failure IDs. Packaging/smoke is run separately because +the full gate stops at pytest. The rebuilt artifact identity follows below. +Mainline integration and the separate Gemini review branch remain pending. + ## Host relaunch handoff The owner explicitly chose a development-build self-hosting experiment. The normal From d679656d8a47ac50adf6ecc122175b504129a04c Mon Sep 17 00:00:00 2001 From: Costin Cozianu Date: Mon, 16 Nov 2026 09:09:55 +0000 Subject: [PATCH 11/43] Record rebuilt rootless-compatible PEX and retry handoff Co-authored-by: GPT-6 Codex --- CURRENT-STATUS.md | 2 +- .../2026-11-16-github-auth/CURRENT-STATUS.md | 17 +++++++++++------ 2 files changed, 12 insertions(+), 7 deletions(-) diff --git a/CURRENT-STATUS.md b/CURRENT-STATUS.md index 66ff89e8..5cc8e442 100644 --- a/CURRENT-STATUS.md +++ b/CURRENT-STATUS.md @@ -9,7 +9,7 @@ coordination branch. Detailed state belongs in each linked status file. | Name | Started | Goal | State | Branch association | Status file | |---|---|---|---|---|---| -| `github-auth` | 2026-11-16 | GitHub SSH authentication through a launcher-owned host proxy | active; propagating rootless Docker fix and rebuilding local PEX | `ws-github-auth/ssh-proxy` | [workstream status](engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md) | +| `github-auth` | 2026-11-16 | GitHub SSH authentication through a launcher-owned host proxy | paused for owner retry; rootless Docker fix included in rebuilt PEX | `ws-github-auth/ssh-proxy` | [workstream status](engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md) | | `workflow-improvements` | 2026-08-09 | Improve the multiple-stream workflow from concrete dogfood findings during the recursive E2E cycle | active 2026-10-30; Pair Review Loop naming implemented and packaging verified for 0.3.0; mainline and release integration pending | `ws-workflow-improvements/v1` | [workstream status](engineering-docs/wip/2026-08-09-workflow-improvements/CURRENT-STATUS.md) | | `project-management` | 2026-08-09 | Maintain project-wide priorities, sequencing, dependencies, and lifecycle coordination | paused 2026-10-11 at owner direction; consent and forced configuration before RC work; remaining diagnostics/image scope and release cut decisions retained | `ws-project-management/coordination` | [workstream status](engineering-docs/wip/2026-08-09-project-management/CURRENT-STATUS.md) | | `maintenance` | 2026-09-18 | Own the defects no open workstream covers, on `main` and on maintained release lines, and drive maintenance releases | active; releasing 0.3.1; cut 2026-11-07 from `main` at `31e8fec7`, `main` reopened at `0.3.2.dev0` (#8); candidate 2 (`v0.3.1-rc2` on `be0140ca`, propagated to `main` via PR #22 `aabace3c`) published and verified: Eclipse libsoup (#10, closed on rc1), launcher-delivered `devcapsule_webconsole` (`D-0017`, #18/#16, closed on rc2), Codex npm abbreviated metadata (#20, closed on rc2 + `component-status` `0bc31c64`), and Codex `0.167.0` + Claude Code `2.1.321` pins (`embedded-33`, #21); rulings open: executables for the Mac, the compatibility feed's home | `release-0.3.1`, the workstream's selection from the cut to the final tag; `release-0.3.0` closed and retained; earlier slice branches merged and closed | [workstream status](engineering-docs/wip/2026-09-18-maintenance/CURRENT-STATUS.md) | diff --git a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md index 70154149..09b71168 100644 --- a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md @@ -4,7 +4,7 @@ Mnemonic: `github-auth` Start date: 2026-11-16 -State: active; propagating the released rootless Docker fix and rebuilding the host-relaunch PEX. +State: paused for owner retry; rootless Docker fix propagated and local PEX rebuilt. Definition read: WORKFLOW.md@89de30622697, WORKFLOW-LOCAL.md@d92bdcbb0a01 @@ -105,7 +105,10 @@ New pure identity tests and filesystem preparation tests cover the adaptation. Full gate: 19 failed, 2942 passed, 1 skipped, 78 deselected, 14 xfailed, 7 xpassed. All 19 failure IDs match the previously reproduced mainline/environment baseline exactly; no new failure IDs. Packaging/smoke is run separately because -the full gate stops at pytest. The rebuilt artifact identity follows below. +the full gate stops at pytest. The separate PEX build and executable smoke passed (14 seconds). The imported +rootless detector reports True against the available daemon. Fix commit: `d6fd93e0`. +The rebuilt artifact identity follows below. Host relaunch itself still needs +the owner to retry; no container or persistent configuration was changed here. Mainline integration and the separate Gemini review branch remain pending. ## Host relaunch handoff @@ -117,21 +120,22 @@ existing persistent state. The original checkout still has only its pre-existing website gitlink modification. Main remains unchanged; synchronization can wait. Build: `.venv/bin/python -m nox -s pex` passed, including shell syntax, packaging -and executable smoke. Built from clean `712466ab9547ec7c9153977f725fc6cfe26344f6`. +and executable smoke. Built from clean `d6fd93e0ba6c18915ace45b9dfef2881068d7b39` with the rootless Docker fix. The local PEX reports `v0.3.2.dev0-local-linux-x86_64` with revision `unknown` by local-build policy; the clean Git revision and checksum here identify its inputs and bytes independently. It contains the committed implementation, including the node-registry and console-table fixes. Review corrections at `388f62a3` remain on the separate review branch and are not included; their PR merge is still pending. -SHA-256: `2f9470c6ca5efeaccd9844bf6feb1494d2e95564d054051a2c005afbc35d78ed`. +SHA-256: `33c3c7bc7fe6ff6772eedee06424c95afcf4aebddca20ab99f9a31a7c103d35b`. Capsule artifact: `/home/devcapsule/worktrees/github-auth/devcapsule-src/dist/devcapsule-local.pex`. Host artifact (from Docker mount inspection): `/home/costin/.local/share/devcapsule/projects/by-path/fe326e03c822-devcapsule/home/worktrees/github-auth/devcapsule-src/dist/devcapsule-local.pex`. Checksum and build metadata are saved beside it, outside Git. -After exiting Codex and stopping the current capsule normally, run on the host: +Retry the same command with the replacement file. After exiting Codex and stopping +the current capsule normally, run on the host: ```sh cd /opt/costin3/myProjects/devcapsule @@ -169,7 +173,8 @@ of the proxy. No E2E container was launched during handoff preparation. ## Open Threads -- Awaiting the human: exit/relaunch/resume using the command above. +- Awaiting the human: retry exit/relaunch/resume using the rebuilt PEX above. + The first relaunch failed on the rootless identity defect, now propagated. Workstream selection, Git-over-SSH scope and the relaunch approach are accepted. Full-gate environment and owner-operated GitHub PR delivery also remain open. - Engineering choices: initial transport must account for Linux/rootless Docker From 7ec3c8ce3830315bbc145777e8641a2ef9a7e177 Mon Sep 17 00:00:00 2001 From: Costin Cozianu Date: Mon, 16 Nov 2026 09:18:39 +0000 Subject: [PATCH 12/43] docs: record successful host proxy relaunch preflight --- CURRENT-STATUS.md | 2 +- .../2026-11-16-github-auth/CURRENT-STATUS.md | 51 +++++++++++++++---- 2 files changed, 41 insertions(+), 12 deletions(-) diff --git a/CURRENT-STATUS.md b/CURRENT-STATUS.md index 5cc8e442..20fd4edb 100644 --- a/CURRENT-STATUS.md +++ b/CURRENT-STATUS.md @@ -9,7 +9,7 @@ coordination branch. Detailed state belongs in each linked status file. | Name | Started | Goal | State | Branch association | Status file | |---|---|---|---|---|---| -| `github-auth` | 2026-11-16 | GitHub SSH authentication through a launcher-owned host proxy | paused for owner retry; rootless Docker fix included in rebuilt PEX | `ws-github-auth/ssh-proxy` | [workstream status](engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md) | +| `github-auth` | 2026-11-16 | GitHub SSH authentication through a launcher-owned host proxy | paused after successful host relaunch/proxy preflight; recorded E2E preparation next | `ws-github-auth/ssh-proxy` | [workstream status](engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md) | | `workflow-improvements` | 2026-08-09 | Improve the multiple-stream workflow from concrete dogfood findings during the recursive E2E cycle | active 2026-10-30; Pair Review Loop naming implemented and packaging verified for 0.3.0; mainline and release integration pending | `ws-workflow-improvements/v1` | [workstream status](engineering-docs/wip/2026-08-09-workflow-improvements/CURRENT-STATUS.md) | | `project-management` | 2026-08-09 | Maintain project-wide priorities, sequencing, dependencies, and lifecycle coordination | paused 2026-10-11 at owner direction; consent and forced configuration before RC work; remaining diagnostics/image scope and release cut decisions retained | `ws-project-management/coordination` | [workstream status](engineering-docs/wip/2026-08-09-project-management/CURRENT-STATUS.md) | | `maintenance` | 2026-09-18 | Own the defects no open workstream covers, on `main` and on maintained release lines, and drive maintenance releases | active; releasing 0.3.1; cut 2026-11-07 from `main` at `31e8fec7`, `main` reopened at `0.3.2.dev0` (#8); candidate 2 (`v0.3.1-rc2` on `be0140ca`, propagated to `main` via PR #22 `aabace3c`) published and verified: Eclipse libsoup (#10, closed on rc1), launcher-delivered `devcapsule_webconsole` (`D-0017`, #18/#16, closed on rc2), Codex npm abbreviated metadata (#20, closed on rc2 + `component-status` `0bc31c64`), and Codex `0.167.0` + Claude Code `2.1.321` pins (`embedded-33`, #21); rulings open: executables for the Mac, the compatibility feed's home | `release-0.3.1`, the workstream's selection from the cut to the final tag; `release-0.3.0` closed and retained; earlier slice branches merged and closed | [workstream status](engineering-docs/wip/2026-09-18-maintenance/CURRENT-STATUS.md) | diff --git a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md index 09b71168..2a8505ed 100644 --- a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md @@ -4,7 +4,7 @@ Mnemonic: `github-auth` Start date: 2026-11-16 -State: paused for owner retry; rootless Docker fix propagated and local PEX rebuilt. +State: paused after successful host relaunch preflight; recorded E2E preparation is next. Definition read: WORKFLOW.md@89de30622697, WORKFLOW-LOCAL.md@d92bdcbb0a01 @@ -73,9 +73,9 @@ checkout remains untouched. Review evidence and prepared PR text are indexed bel Owner chose on 2026-11-16 to exit this capsule, relaunch the same checkout from the physical host with the new local PEX and github-ssh enabled, then resume Codex. -This supersedes the proposed SSH-to-host harness adaptation. After resumption, -verify the inherited broker and executable bytes, then write and execute the -recorded E2E scenario using a fresh project and explicit github-ssh authorization. +This supersedes the proposed SSH-to-host harness adaptation. The resumed-session +preflight below verified the broker and executable bytes. Next, write and execute +the recorded E2E scenario using a fresh project and explicit github-ssh authorization. Observe the Git checkout independently and retain WebM/screenshots/action evidence. Recursive proxy inheritance is still unproved; do not report success until observed. The 19 reproduced full-gate failures and formal PR delivery remain open separately. @@ -107,8 +107,8 @@ Full gate: 19 failed, 2942 passed, 1 skipped, 78 deselected, 14 xfailed, baseline exactly; no new failure IDs. Packaging/smoke is run separately because the full gate stops at pytest. The separate PEX build and executable smoke passed (14 seconds). The imported rootless detector reports True against the available daemon. Fix commit: `d6fd93e0`. -The rebuilt artifact identity follows below. Host relaunch itself still needs -the owner to retry; no container or persistent configuration was changed here. +The rebuilt artifact identity follows below. The owner subsequently relaunched +successfully; the resumed-session observations are recorded below. Mainline integration and the separate Gemini review branch remain pending. ## Host relaunch handoff @@ -171,15 +171,44 @@ must have no copied key; the recording harness must authorize github-ssh for its fresh project. Do not use this current capsule's manually installed key as proof of the proxy. No E2E container was launched during handoff preparation. +## Resumed-session readiness, 2026-11-16 + +The owner resumed inside the rebuilt capsule. Running `/opt/devcapsule/bin/devcapsule.pex` +and the saved local PEX both match the SHA-256 above. Runtime identity is 0:0 and +Docker reports `name=rootless`. The canonical proxy socket exists, and Git's SSH +command points to the shipped `github-ssh` adapter. + +A read-only `git ls-remote git@github.com:mycodespace-ai/devcapsule.git HEAD` +returned `1942f396dd11ffabb64784eb7279f283181e32dd`. Repeating it from a fresh empty +HOME with system/global Git configuration disabled and the adapter explicitly +selected also passed. Pointing that invocation at a missing socket failed with +exit 128. This demonstrates proxy authentication independently of the manually +installed capsule key; it does not yet prove a new child capsule's inheritance. + +Docker mount inspection resolves the inherited socket to the outside launcher's +host socket, and resolves both the persistent E2E workspace root and PEX directory +to host-backed paths. Docker access, Git, pytest and nox are available. No new +container was started for this preflight. Canonical main was fetched: this branch +is ten commits ahead and zero behind, so no synchronization is needed. Mail and +intake remain empty. + +Recording preparation remains: author the GitHub showcase scenario and its Markdown +procedure, explicitly grant github-ssh in its fresh project, and prepare Playwright. +This capsule has neither local Playwright nor FFmpeg nor `/opt/playwright`; the +existing recorder supports Chromium/FFmpeg in a child with `browser-automation`. +The scenario must request that component, verify a real SSH Git checkout from +outside the recorded UI, retain video/screenshots/action evidence, and clean up. +No nested E2E or recording has run yet. No additional host credentials or restart +are indicated by the preflight. The outside launcher must stay running. + ## Open Threads -- Awaiting the human: retry exit/relaunch/resume using the rebuilt PEX above. - The first relaunch failed on the rootless identity defect, now propagated. - Workstream selection, Git-over-SSH scope and the relaunch approach are accepted. - Full-gate environment and owner-operated GitHub PR delivery also remain open. +- Host relaunch and authenticated proxy preflight passed. The next task is the + fresh-project recorded E2E; recursive inheritance and recording remain unproved. + Full-gate environment and owner-operated GitHub PR delivery remain open. - Engineering choices: initial transport must account for Linux/rootless Docker permissions; Docker Desktop cannot bind ordinary host Unix sockets. -- Physical-host launch acceptance and Docker Desktop transport remain open. +- Physical-host relaunch is observed working; Docker Desktop transport remains deferred. - GitHub CLI is absent at the policy-prescribed path. PR creation/merging remains owner-operated under WORKFLOW-LOCAL.md; prepare reviewed commits and compare links. - Reviewer worktree removed at pause; pushed `ws-github-auth/ssh-proxy-review` From e45866b2d420cd8f71cd6d1c311d02cdaaa9d7df Mon Sep 17 00:00:00 2001 From: Costin Cozianu Date: Mon, 16 Nov 2026 09:31:34 +0000 Subject: [PATCH 13/43] test: record GitHub proxy clone in a fresh capsule Allow scenario-specific launch needs and consents in the interactive recorder, bound its interactive phase, and retain the procedure and successful rootless proxy E2E evidence. Co-authored-by: GPT-6 Astra Codex --- CURRENT-STATUS.md | 2 +- devcapsule-src/tests/e2e/scenario_recorder.py | 12 +- .../2026-11-16-github-auth/CURRENT-STATUS.md | 51 ++++-- .../evidence/showcase.md | 147 ++++++++++++++++++ 4 files changed, 199 insertions(+), 13 deletions(-) create mode 100644 engineering-docs/wip/2026-11-16-github-auth/evidence/showcase.md diff --git a/CURRENT-STATUS.md b/CURRENT-STATUS.md index 20fd4edb..36cc4715 100644 --- a/CURRENT-STATUS.md +++ b/CURRENT-STATUS.md @@ -9,7 +9,7 @@ coordination branch. Detailed state belongs in each linked status file. | Name | Started | Goal | State | Branch association | Status file | |---|---|---|---|---|---| -| `github-auth` | 2026-11-16 | GitHub SSH authentication through a launcher-owned host proxy | paused after successful host relaunch/proxy preflight; recorded E2E preparation next | `ws-github-auth/ssh-proxy` | [workstream status](engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md) | +| `github-auth` | 2026-11-16 | GitHub SSH authentication through a launcher-owned host proxy | paused after successful recorded proxy E2E; review integration and baseline gate disposition pending | `ws-github-auth/ssh-proxy` | [workstream status](engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md) | | `workflow-improvements` | 2026-08-09 | Improve the multiple-stream workflow from concrete dogfood findings during the recursive E2E cycle | active 2026-10-30; Pair Review Loop naming implemented and packaging verified for 0.3.0; mainline and release integration pending | `ws-workflow-improvements/v1` | [workstream status](engineering-docs/wip/2026-08-09-workflow-improvements/CURRENT-STATUS.md) | | `project-management` | 2026-08-09 | Maintain project-wide priorities, sequencing, dependencies, and lifecycle coordination | paused 2026-10-11 at owner direction; consent and forced configuration before RC work; remaining diagnostics/image scope and release cut decisions retained | `ws-project-management/coordination` | [workstream status](engineering-docs/wip/2026-08-09-project-management/CURRENT-STATUS.md) | | `maintenance` | 2026-09-18 | Own the defects no open workstream covers, on `main` and on maintained release lines, and drive maintenance releases | active; releasing 0.3.1; cut 2026-11-07 from `main` at `31e8fec7`, `main` reopened at `0.3.2.dev0` (#8); candidate 2 (`v0.3.1-rc2` on `be0140ca`, propagated to `main` via PR #22 `aabace3c`) published and verified: Eclipse libsoup (#10, closed on rc1), launcher-delivered `devcapsule_webconsole` (`D-0017`, #18/#16, closed on rc2), Codex npm abbreviated metadata (#20, closed on rc2 + `component-status` `0bc31c64`), and Codex `0.167.0` + Claude Code `2.1.321` pins (`embedded-33`, #21); rulings open: executables for the Mac, the compatibility feed's home | `release-0.3.1`, the workstream's selection from the cut to the final tag; `release-0.3.0` closed and retained; earlier slice branches merged and closed | [workstream status](engineering-docs/wip/2026-09-18-maintenance/CURRENT-STATUS.md) | diff --git a/devcapsule-src/tests/e2e/scenario_recorder.py b/devcapsule-src/tests/e2e/scenario_recorder.py index a8dd8352..c1df19ef 100644 --- a/devcapsule-src/tests/e2e/scenario_recorder.py +++ b/devcapsule-src/tests/e2e/scenario_recorder.py @@ -599,18 +599,25 @@ def serve_interactive_controller( executable: Path, spec: ScenarioSpec, evidence: Path, + *, + surface: IdeSurface | None = None, + timeout: float = 1800.0, ) -> None: """Start an interactive E2E IDE session and HTTP controller for live agent takeover. ``controller.json`` under ``evidence`` names the port once the IDE is ready; a stale copy from an earlier run is removed first, so a waiting ``ctl`` never dials a dead port. + A caller may supply scenario-specific needs and consents through ``surface``. + The interactive phase has an overall deadline, including time between commands. """ from playwright.sync_api import sync_playwright # type: ignore[import-not-found,unused-ignore] evidence.mkdir(parents=True, exist_ok=True) (evidence / "controller.json").unlink(missing_ok=True) os.environ["DEVCAPSULE_SMOKE_COMPONENT_BROWSER"] = "1" - surface: IdeSurface = next(s for s in SURFACES if s.name == spec.surface) + surface = surface or next(s for s in SURFACES if s.name == spec.surface) + if surface.name != spec.surface: + raise DriverError("Scenario and launch surface must agree") recorder = BurstFrameRecorder(evidence / "video-frames") action_log: list[dict[str, Any]] = [] step_counter = 0 @@ -657,8 +664,11 @@ def serve_interactive_controller( print(f"CONTROLLER_READY port={port} project_path={session.project_path}", flush=True) stopped = False + deadline = time.monotonic() + timeout try: while not stopped: + if time.monotonic() >= deadline: + raise DriverError("Interactive scenario deadline expired") item = queue.take() if item is None: time.sleep(0.05) diff --git a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md index 2a8505ed..b7d2ee11 100644 --- a/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md +++ b/engineering-docs/wip/2026-11-16-github-auth/CURRENT-STATUS.md @@ -4,7 +4,7 @@ Mnemonic: `github-auth` Start date: 2026-11-16 -State: paused after successful host relaunch preflight; recorded E2E preparation is next. +State: paused after successful recorded GitHub SSH proxy E2E; review integration and baseline gate disposition remain open. Definition read: WORKFLOW.md@89de30622697, WORKFLOW-LOCAL.md@d92bdcbb0a01 @@ -71,14 +71,12 @@ checkout remains untouched. Review evidence and prepared PR text are indexed bel ## Planned next step -Owner chose on 2026-11-16 to exit this capsule, relaunch the same checkout from -the physical host with the new local PEX and github-ssh enabled, then resume Codex. -This supersedes the proposed SSH-to-host harness adaptation. The resumed-session -preflight below verified the broker and executable bytes. Next, write and execute -the recorded E2E scenario using a fresh project and explicit github-ssh authorization. -Observe the Git checkout independently and retain WebM/screenshots/action evidence. -Recursive proxy inheritance is still unproved; do not report success until observed. -The 19 reproduced full-gate failures and formal PR delivery remain open separately. +The owner-authorized host relaunch and recorded fresh-capsule E2E are complete. +The next delivery step is to resolve the known gate baseline and have the owner +merge the separate Gemini review corrections through the prepared PR process. +Validate the combined branch before integration; the recorded PEX does not contain +those corrections. The engineering showcase and procedure are indexed below. +No further restart or host credential setup is needed for this completed test. ## Rootless Docker fix propagation @@ -201,10 +199,40 @@ outside the recorded UI, retain video/screenshots/action evidence, and clean up. No nested E2E or recording has run yet. No additional host credentials or restart are indicated by the preflight. The outside launcher must stay running. +## Recorded E2E acceptance, 2026-11-16 + +One fresh VSCodium capsule, with browser-automation and explicit github-ssh +consent, was launched by the saved PEX from this capsule. The outer physical-host +launcher remained alive. Child and parent Docker inspection showed the same host +broker source, mounted read-only at the canonical socket. The child had no SSH +files or agent; its delivered runtime checksum matched the tested PEX. + +Through the actual recorded terminal, Git cloned the canonical repository over +SSH. Independent Git inspection of the host-backed checkout confirmed expected +HEAD `1942f396dd11ffabb64784eb7279f283181e32dd`, the SSH origin, a clean tree and +the README. A missing-socket read failed with exit 128. Ordinary proxy access +succeeded again afterward. This proves recursive proxy inheritance on this +rootless Linux host. No GitHub write was made. + +Evidence: `devcapsule-src/dist/e2e-evidence/github-auth/20261116T092248Z-33c3c7bc/`. +The 38.17-second WebM, full desktop capture, poster, action trace, `run.json`, +`verification.json` and `result.json` are retained outside Git. The test-owned +container, fresh workspace and checkout records are independently confirmed gone. +The controller gained optional per-scenario surface needs/consents and a bounded +interactive phase; the product PEX was not changed. All 62 existing recorder tests +passed. See the procedure for the on-screen empty-directory check correction and +a minor final command/output overlap, both retained in the original recording. + +Full gate for the recorder change: 19 failed, 2942 passed, 1 skipped, 78 deselected, +14 xfailed and 7 xpassed. All 19 failure IDs exactly match the recorded baseline. +Types passed over 340 files; the corrected convention check found zero breaches. The initial attempt +used the stale default origin/main comparison and stopped on three unrelated +convention differences. The corrected run used DEVCAPSULE_DIFF_BASE=upstream/main. Its log is retained at +`~/.cache/devcapsule-gate/github-auth-showcase-build-upstream.log`. + ## Open Threads -- Host relaunch and authenticated proxy preflight passed. The next task is the - fresh-project recorded E2E; recursive inheritance and recording remain unproved. +- Host relaunch, recursive inheritance and recorded fresh-project E2E passed. Full-gate environment and owner-operated GitHub PR delivery remain open. - Engineering choices: initial transport must account for Linux/rootless Docker permissions; Docker Desktop cannot bind ordinary host Unix sockets. @@ -218,6 +246,7 @@ are indicated by the preflight. The outside launcher must stay running. ## Workstream document index +- [Recorded E2E procedure](evidence/showcase.md) - [Review brief](evidence/review-brief.md) - [Review follow-up 1](evidence/round-1.md) - [Intake](intake/README.md) diff --git a/engineering-docs/wip/2026-11-16-github-auth/evidence/showcase.md b/engineering-docs/wip/2026-11-16-github-auth/evidence/showcase.md new file mode 100644 index 00000000..49fff04b --- /dev/null +++ b/engineering-docs/wip/2026-11-16-github-auth/evidence/showcase.md @@ -0,0 +1,147 @@ +# GitHub SSH proxy: recorded fresh-capsule clone + +## Purpose and acceptance + +Show a user cloning a GitHub repository over SSH inside a fresh capsule. The +outside launcher uses the host identity. No SSH key or agent is supplied to the +child. Public GitHub repositories also require authentication on their SSH URL. +This scenario reads the canonical DevCapsule repository; it does not push. + +Pass requires a real clone, the expected independently read commit, the unchanged +SSH origin, a clean working tree, no child SSH files or agent, and the canonical +broker socket mounted from the same host source as the parent. A second Git read +with an absent broker socket must fail. Keep the recording and independent +observations, and remove the fresh container, workspace and checkout state. +A video alone does not pass the test. + +## Preconditions + +Keep the outside launcher alive with `github-ssh` enabled. The host must already +trust GitHub and authenticate noninteractively. An unencrypted default SSH key +needs no host agent. The parent needs host Docker and host networking. + +From `devcapsule-src`, use the development virtualenv and the exact PEX under test. +The recorded run uses SHA-256 +`33c3c7bc7fe6ff6772eedee06424c95afcf4aebddca20ab99f9a31a7c103d35b`, built from +`d6fd93e0`. Do not rebuild between the launch and the evidence capture. + +```sh +.venv/bin/python -m pip install 'playwright==1.63.0' +export DEVCAPSULE_EXPECTED_SHA256=33c3c7bc7fe6ff6772eedee06424c95afcf4aebddca20ab99f9a31a7c103d35b +``` + +## Start the recording controller + +Save the following as a temporary Python script and run it with +`PYTHONPATH=. .venv/bin/python