Skip to content

Recovery media: package /boot/efi and root repair tools in the initrd (issue #17) - #55

Merged
aien-dev merged 15 commits into
mainfrom
copilot/bootable-recovery-media-machine-1
Sep 24, 2026
Merged

aien-dev merged 15 commits into
mainfrom
copilot/bootable-recovery-media-machine-1

Conversation

Copilot AI commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Reworked into a small, rule-compliant PR at the maintainer's request (rebase on main, don't replace scripts/build_recovery_media.sh or scripts/build_standalone_recovery_initrd.sh, keep only what's still missing for issue #17). Verified locally:

$ bash scripts/verify_recovery_tools.sh /tmp/aienos-recovery-standalone-initrd.img
...
-- required tools for /boot/efi repair --
PASS  fsck.vfat present
PASS  mkfs.vfat present
PASS  fsck.ext4 present
PASS  efibootmgr present

-- required tools for boot-entry restore --
PASS  efibootmgr present (boot entries)
PASS  chroot present
...
RECOVERY_TOOLS: PASS (initrd ships mount, EFI repair, and boot-entry restore capability)

$ bash scripts/verify_all.sh
...
--- [Recovery Media Tooling Manifest] ---
...
RECOVERY_TOOLS: PASS (initrd ships mount, EFI repair, and boot-entry restore capability)
RC=0

Full verify_all.sh run: all steps PASS, including Gate 1 QEMU zero-disk recovery, Gate 4 security suite (swTPM + 3-run soak + fault injection), and the new Recovery Media Tooling Manifest step. No FAIL anywhere. rc=0.

What issue #17 still needed

The recovery initrd from #53 already boots Machine 1 with Secure Boot on, mounts the driver stack for the NVMe/USB/keyboard, and returns to Linux unattended (Gate 1 selftest evidence). What it didn't ship yet were the tools issue #17's "done when" clause needs to actually repair /boot/efi and check the mounted root: fsck.vfat, mkfs.vfat, fsck.ext4, chroot.

What changed (10 files, +429/-8)

  • scripts/build_standalone_recovery_initrd.sh: minimal, additive patch (+7/-0) to the existing BINARIES array -- adds fsck.vfat, mkfs.vfat, fsck.ext4, chroot with their shared libraries, following the script's existing copy pattern. No restructuring.
  • scripts/verify_recovery_tools.sh (new): host-only, no real devices, no root. Extracts the built initrd and checks that it ships the mount / EFI-repair / boot-entry-restore tooling and init hooks. Wired into scripts/verify_all.sh (step 9) since it needs only gzip/cpio.
  • scripts/collect_recovery_boot_evidence.sh (new): evidence collector for the attended recovery boot on Machine 1. Read-only -- it inspects mount state, filesystem presence, and firmware variables the kernel already exposes read-only; it never writes to the ESP, the NVMe root, or BootOrder.
  • docs/RECOVERY_MEDIA_MACHINE1.md (new): operator runbook rewritten to match main's actual mechanics post-fix(recovery): package drivers and put the shell on the screen #53 -- the AIENOSRECOV stick name (not the retired ATLAS_RECOV), the real script names, and the actual shell-based repair steps (no invented rescue-menu system). States plainly that the stick must be rebuilt with the new tools and the attended boot re-run before issue Bootable recovery media for Machine 1 #17 can close.
  • evidence/recovery_boot_machine1.md (new): honest PENDING ATTENDED EXERCISE template. No "merged"/"verified" language and no hardware-boot claim -- the physical repair boot has not happened.
  • Small pointer edits (ROADMAP.md, docs/MILESTONES.md, docs/NATIVE_BOOT_ONE_TIME.md, docs/TRUST-1-IMPLEMENTATION-PLAN.md) linking to the new runbook and correcting stale status text.

What was dropped from the earlier draft, and why

  • The ~250-line rewrite of scripts/build_standalone_recovery_initrd.sh's init/menu system (custom aienos-efi-repair helper, interactive rescue menu, LVM activation). Out of scope per the maintainer's request; main already has a working shell-based flow from fix(recovery): package drivers and put the shell on the screen #53, and the operator has full tool access from the plain shell without a bespoke menu layer.
  • The edit to evidence/gate1_zero_disk_recovery_receipt.json -- an existing evidence file that must not change.
  • The change to scripts/build_recovery_media.sh -- not needed for the tooling gap this PR closes.
  • Deletions of unrelated files (docs/HARDWARE_TEST_RIG.md, ADRs 0010/0011, crates/aienos-kernel/src/usb/hid.rs, etc.) that were artifacts of the draft branch predating several since-merged PRs (docs(adr): formalize two-stage EL2 bootstrap to EL1h kernel transition #52-chore(agents): standing instructions and setup steps for Copilot #60). This branch is now rebased cleanly on current main (comparing main...copilot/bootable-recovery-media-machine-1 shows exactly these 10 files, ahead 5 / behind 0).

What is NOT claimed

No hardware boot happened as part of this change. The AIENOSRECOV stick as last written (#53) predates these repair tools. The operator must rebuild the stick with scripts/build_recovery_media.sh and re-run the attended boot in docs/RECOVERY_MEDIA_MACHINE1.md before issue #17 can close -- that is exactly what evidence/recovery_boot_machine1.md is a pending template for.

🤖 Generated with Claude Code

SECURITY_IMPACT: RECOVERY

Copilot AI linked an issue Sep 24, 2026 that may be closed by this pull request
aien-dev and others added 3 commits September 24, 2026 07:56
…#53)

* fix(recovery): package drivers and put the shell on the screen

The recovery stick booted to a shell Machine 1 could not use:
- The Ubuntu kernel builds xHCI platform, USB HID, USB storage, NVMe and
  dm-crypt as modules, and the initrd carried none, so no keyboard, no USB
  and no NVMe. Package 16 signed modules (decompressed, signatures kept)
  and load them from init.
- GRUB listed console=tty0 before console=ttyAMA0, so /dev/console and the
  shell were on the serial port, which Machine 1 cannot reach. tty0 is now
  last; the report is mirrored to serial for emulator tests.
- Ubuntu's signed GRUB reads /EFI/ubuntu/grub.cfg first; write both copies.
- Report Secure Boot state (efivarfs mounted read-only) and kernel lockdown.
- qemu_verify_recovery_media.sh adds a boot with a scratch USB stick, NVMe
  drive and USB keyboard and requires every module to load.

SECURITY_IMPACT: RECOVERY

Co-authored-by: Drake Stapleton <drake@aienos.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(recovery): unattended one-shot self-test and fuller report

- Ship blkid from /sbin: /bin/blkid does not exist on the build host, so it
  was silently skipped (the Gate 1 receipt lists it anyway). A missing
  recovery binary is now a build error.
- Find the stick with blkid -p: with no udev, blkid -L and lsblk see no
  labels. Report lists every partition's label, type and UUID.
- Report adds input device names, kernel, clock, TPM PCR 0 and 7 (sysfs)
  and the firmware boot entries (efivarfs stays read-only).
- Self-test: if aienos-selftest-once is on the stick, remount it rw, delete
  the flag, save the report to aienos-evidence/, and restart after 60 s
  unless Enter is pressed. Ordinary recovery boots never write.

Verified: Secure Boot QEMU (AAVMF .ms, KVM) with the flag set restarts by
itself in 78 s, flag removed, report saved; verify_all.sh passes.

SECURITY_IMPACT: RECOVERY

Co-authored-by: Drake Stapleton <drake@aienos.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Drake <dhgmonkey94@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…tore) + evidence tooling + operator runbook for Machine 1 recovery media (issue #17)

Co-authored-by: aien-dev <330927100+aien-dev@users.noreply.github.com>
…ntry restore, shellcheck fixes; validated end-to-end

Co-authored-by: aien-dev <330927100+aien-dev@users.noreply.github.com>
@aien-dev

Copy link
Copy Markdown
Owner

@copilot heads-up: #53 (merged, 0379b2f) already builds and verifies the Machine 1 recovery media: signed shim/GRUB/kernel, 16 signed kernel modules (xHCI platform, HID, USB storage, NVMe, dm-crypt), shell on tty0, and an unattended one-shot self-test. Please rebase on main and do not replace scripts/build_recovery_media.sh or scripts/build_standalone_recovery_initrd.sh; limit this PR to anything still missing for issue #17.

Copilot AI changed the title [WIP] Add bootable recovery media for Machine 1 Recovery media tooling: rescue initrd, evidence collector, and operator runbook for Machine 1 Sep 24, 2026
Copilot AI requested a review from aien-dev September 24, 2026 13:06
Unattended one-shot self-test from AIENOSRECOV with Secure Boot on:
firmware accepted the shim chain (BootCurrent 0004), 16/16 signed modules
loaded, keyboard, stick and NVMe found, PCR 0 and 7 equal the Gate 0
baseline, report saved to the stick, machine returned to Linux by itself
with BootNext consumed and storage unlocked.

Gate 1 is PARTIAL: items 9-14 and the test-artifact round trip are
pending. M0 stays PARTIAL: the sealed Config A bundle keeps
recovery_procedure documented_only until restore is exercised.

Records the first-attempt Secure Boot violation (a firmware entry booting
vmlinuz directly, now deleted) and an existing TPM_RC_PCR_CHANGED unseal
race on normal Linux boots.

SECURITY_IMPACT: NONE

Co-authored-by: Drake <dhgmonkey94@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
aien-dev and others added 10 commits September 24, 2026 08:18
- .github/copilot-instructions.md: finish line (verify_all.sh output in the
  PR, skipped steps named, SECURITY_IMPACT line, PRs under ~600 lines), hard
  rules (Rust only, no Python, no systemd, no hardware claims, protected
  files), the ADR 0009 ordering for M3, and ADR numbering against open PRs.
- .github/workflows/copilot-setup-steps.yml: preinstalls the Rust targets,
  QEMU, AAVMF and swtpm so the agent can run verify_all.sh. workflow_dispatch
  only; it does not re-enable CI.

SECURITY_IMPACT: NONE

Co-authored-by: Drake <dhgmonkey94@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…gnostics in the report (#56)

* Initial plan

* fix(boot): share-open PciRootBridgeIo and record GB10 discovery diagnostics

Co-authored-by: aien-dev <330927100+aien-dev@users.noreply.github.com>

* test(accel): unit-test discovery diagnostics; docs+evidence for GB10 discovery fix

Co-authored-by: aien-dev <330927100+aien-dev@users.noreply.github.com>

* docs(gb10): keep evidence untouched; state ACCESS_DENIED as hypothesis

Revert the edit to evidence/m2_first_boot_2026-09-24.md (existing evidence
files are not edited) and reword GB10_NATIVE_STATUS.md so the exclusive-open
ACCESS_DENIED explanation is the leading hypothesis, unconfirmed on hardware
until the next native boot's gb10_pci_root_open lines.

Co-authored-by: Drake <drake@aienos.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: aien-dev <330927100+aien-dev@users.noreply.github.com>
Co-authored-by: Drake <dhgmonkey94@gmail.com>
Co-authored-by: Drake <drake@aienos.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…#62)

Record the operator's 2026-09-24 decisions: AIEN Seed / Forge / Generation;
agent as untrusted proposer; admission separate from permission; no native
admission before M3 enforcement; capability artifact vs admission records;
knowledge shareable, trust not transferable; rollback-first repair;
compatibility islands as visible debt; owner-priority compute budget;
bounded execution state with permanent provenance. Insert SEED-0A before M3
and SEED-0B after it in the roadmap critical path.

Co-authored-by: Drake <dhgmonkey94@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Publishes contact channels and support plans for the AIENOS project
per the funding.json v1.1.0 spec. Approved by the founder.
…rd.usb, part 1) (#58)

* Initial plan

* kernel: add xHCI controller and HID boot-protocol keyboard drivers

- usb/mod.rs: PCI class-code recognizer for xHCI (0x0c0330) with 64-bit BAR extraction
- usb/xhci.rs: polled xHCI driver (reset, command/event rings, Enable Slot,
  Address Device, SET_PROTOCOL boot, Configure Endpoint, bounded interrupt-IN
  report polling, DMA cache maintenance)
- usb/hid.rs: pure boot-protocol report decoder (make-only, US keymap,
  modifiers, arrows as ANSI) + fixed-capacity TextSink; host unit tests
- arch/aarch64.rs: clean/invalidate cache-range helpers for DMA buffers

Co-authored-by: aien-dev <330927100+aien-dev@users.noreply.github.com>

* Apply remaining changes

Co-authored-by: aien-dev <330927100+aien-dev@users.noreply.github.com>

* kernel(usb): cut to the host-tested HID boot-protocol decoder

Resize per review: keep only the pure-Rust HID boot-protocol decoder and
keymap (usb/hid.rs, 7 unit tests) under ~600 lines. The polled xHCI driver,
the xHCI PCI recognizer, the aarch64 cache helpers and the handoff wiring
move to a follow-up; the full work is preserved on branch
seed0a/xhci-keyboard-driver. SEED-0A experiment #1 (input.keyboard.usb,
ADR 0012).

Co-authored-by: Drake <drake@aienos.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: aien-dev <330927100+aien-dev@users.noreply.github.com>
Co-authored-by: Drake <dhgmonkey94@gmail.com>
Co-authored-by: Drake <drake@aienos.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…ent design record (#54)

* Initial plan

* docs(adr): add ADR 0010 fabric machine identity and capability advertisement design record

Co-authored-by: aien-dev <330927100+aien-dev@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: aien-dev <330927100+aien-dev@users.noreply.github.com>
Co-authored-by: Drake <dhgmonkey94@gmail.com>
Co-authored-by: Drake <drake@aienos.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…MI capture, HID input) (#57)

* Initial plan

* docs(rig): propose Machine 1 hardware test rig design and ADR 0010

Co-authored-by: aien-dev <330927100+aien-dev@users.noreply.github.com>

* docs(adr): renumber the hardware test rig ADR to 0011

ADR 0010 stays with the fabric machine identity ADR (#54, opened first).
Rename the file, its title, and the links from HARDWARE_TEST_RIG.md,
MILESTONES.md and the ADR index.

Co-authored-by: Drake <drake@aienos.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: aien-dev <330927100+aien-dev@users.noreply.github.com>
Co-authored-by: Drake <dhgmonkey94@gmail.com>
Co-authored-by: Drake <drake@aienos.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…oling/evidence checks (issue #17)

The standalone recovery initrd (#53) already boots on Machine 1, mounts
the NVMe root's driver stack, and returns to Linux unattended, but it
was missing the tools issue #17's "done when" clause needs to actually
repair /boot/efi and the mounted root: fsck.vfat, mkfs.vfat, fsck.ext4,
and chroot. This is a minimal, additive patch to
build_standalone_recovery_initrd.sh's existing BINARIES list -- no
restructuring of that script or of build_recovery_media.sh.

Adds:
- scripts/verify_recovery_tools.sh: host-only, no-root, no-real-devices
  check that the built initrd ships the mount / EFI repair / boot-entry
  restore tooling. Wired into scripts/verify_all.sh.
- scripts/collect_recovery_boot_evidence.sh: read-only evidence
  collector for the attended recovery boot. Never writes to the ESP,
  the NVMe root, or BootOrder.
- docs/RECOVERY_MEDIA_MACHINE1.md: runbook matching main's actual
  mechanics after #53 (AIENOSRECOV stick name, shell-based repair
  steps, no invented menu system). States plainly that the stick must
  be rebuilt and the attended boot re-run before issue #17 closes.
- evidence/recovery_boot_machine1.md: honest PENDING template. No
  hardware-boot claim.
- Small pointers in ROADMAP.md, docs/MILESTONES.md,
  docs/NATIVE_BOOT_ONE_TIME.md, docs/TRUST-1-IMPLEMENTATION-PLAN.md to
  the new runbook.

Dropped from the earlier draft of this PR: the rewrite of
build_standalone_recovery_initrd.sh's init/menu system (out of scope,
main already has a working shell-based flow from #53), the edit to
the existing evidence/gate1_zero_disk_recovery_receipt.json (must not
change), and the change to build_recovery_media.sh (not needed for
this).

Verified: `bash scripts/verify_recovery_tools.sh` PASS against a built
initrd; `bash scripts/verify_all.sh` exits 0, including the new
Recovery Media Tooling Manifest step.

Co-authored-by: Drake <drake@aienos.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@aien-dev aien-dev changed the title Recovery media tooling: rescue initrd, evidence collector, and operator runbook for Machine 1 Recovery media: package /boot/efi and root repair tools in the initrd (issue #17) Sep 24, 2026
@aien-dev
aien-dev marked this pull request as ready for review September 24, 2026 14:36
@aien-dev
aien-dev merged commit fdff1db into main Sep 24, 2026
@aien-dev
aien-dev deleted the copilot/bootable-recovery-media-machine-1 branch September 24, 2026 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bootable recovery media for Machine 1

3 participants