From 044427f3205ed4602be43900cb1c654333751ee2 Mon Sep 17 00:00:00 2001 From: Daniel Casota Date: Mon, 31 Aug 2026 14:13:31 +0200 Subject: [PATCH 1/4] sharukhan phase-1: author the PRD 15 requirements and 15 acceptance criteria, each AC naming the task that will verify it, following the in-repo practice of using the AC table as the traceability matrix. Scope is drawn tightly: sharukhan orchestrates the existing Photon build scripts rather than replacing them, and every SPAGAT-Librarian concern (operator-config medium, iso-phase6, SPAGAT_*/IPHASE6_*) is explicitly out, as the maintainer scoped it. The purpose section states the three observed defects that motivate a rewrite rather than more shell: inputs that resolve implicitly (two downstream patch copies drifted 78 lines and 8-vs-27 files apart, and the build used whichever sat beside the invoked script), checks that cannot fail (40-check-staging.sh never exits non-zero), and verdicts that can be silently vacuous (toybox grep has no -a and returns zero matches on a NUL-bearing serial log). Section 8 records five defects found while inventorying the predecessor tooling, so they are requirements by implication rather than inherited bugs: mc-run.sh loses its counters to a subshell and always tallies zero; mc-verify.sh hardcodes -prebuilt when locating a cached ISO; the static address scheme is computed but never applied, so every guest takes a DHCP lease; nine config variables are declared and never read, and MC_MAC_PREFIX contradicts the OUI actually used; and the serial-growth liveness instrument was lost in translation from vm-lab, with the log truncated rather than offset-bounded. Signed-off-by: Daniel Casota --- staging/sharukhan/ARCHITECTURE.md | 4 +- staging/sharukhan/specs/prd.md | 144 ++++++++++++++++++++++++++++++ 2 files changed, 146 insertions(+), 2 deletions(-) create mode 100644 staging/sharukhan/specs/prd.md diff --git a/staging/sharukhan/ARCHITECTURE.md b/staging/sharukhan/ARCHITECTURE.md index e2af6b69..7914bf44 100644 --- a/staging/sharukhan/ARCHITECTURE.md +++ b/staging/sharukhan/ARCHITECTURE.md @@ -89,8 +89,8 @@ The methodology is reconstructed from the maintainer's `vCenter-CVE-drift-analyz | Phase | Deliverable | Status | |---|---|---| -| 0 | `ARCHITECTURE.md`, `specs/README.md`, `AGENTS.md` | In Progress | -| 1 | `specs/prd.md` | Pending | +| 0 | `ARCHITECTURE.md`, `specs/README.md`, `AGENTS.md` | Complete (#319) | +| 1 | `specs/prd.md` | In Progress | | 2 | Dev Lead review on the PRD PR | Pending | | 3 | `specs/adr/0001`–`000n` | Pending | | 4 | `specs/features/*.md` | Pending | diff --git a/staging/sharukhan/specs/prd.md b/staging/sharukhan/specs/prd.md new file mode 100644 index 00000000..9f65a05e --- /dev/null +++ b/staging/sharukhan/specs/prd.md @@ -0,0 +1,144 @@ +# 📝 Product Requirements Document (PRD) + +## 1. Purpose + +`sharukhan` is a single standalone Rust CLI that plans, builds, executes and verifies every permutation of a Photon OS ISO install matrix on VMware Workstation, and records what it found in a queryable memory database. + +It replaces two shell toolkits — `staging/vm-lab` (VM provisioning) and `staging/mission-control` (permutation harness) — which between them work but cannot be trusted as a test oracle. Three defects motivate the rewrite, all observed rather than hypothesised: + +- **Inputs resolve implicitly.** `runPh5_normal.sh` resolves its patch relative to its own location. The two copies on the host had drifted 78 lines and 8-vs-27 files apart, and the build silently used whichever sat beside the invoked script. The failure surfaced as `patch does not apply` against a spec — indistinguishable from a rebase problem, and it cost two builds and two wrong diagnoses. +- **Checks cannot fail.** `vm-lab/scripts/40-check-staging.sh` never exits non-zero. Correct for an inspection tool; useless as a gate. +- **Verdicts can be silently vacuous.** `/usr/bin/grep` is toybox in a non-interactive shell and has no `-a`, returning *zero matches* on a NUL-bearing serial log rather than erroring; interactively the same name resolves to `ugrep`, which behaves differently again. A pipeline that "passes" may have measured nothing. + +The audience is one maintainer testing pull requests against a hypervisor on their own workstation, not a fleet. + +## 2. Scope + +**In Scope** + +- Prerequisite verification with measured, actionable results +- ISO build orchestration and caching, keyed by the build-time axes +- Per-permutation VM provisioning, install (unattended and operator-assisted), verification and teardown +- An assertion oracle that attributes each failure to the pull request it disproves +- Background job execution with status inspection and cancellation +- A memory database as the system of record, with `MEMORY.md` generated from it +- A final report table of what ran and what passed +- Selective execution of named permutations + +**Out of Scope** + +- Hypervisors other than VMware Workstation on Windows driven from WSL +- Building Photon packages itself — `sharukhan` invokes the existing build scripts, it does not replace them +- Any SPAGAT-Librarian appliance concern (`operator-config.vmdk`, `iso-phase6`, `SPAGAT_*`/`IPHASE6_*`, the credential medium). Explicitly excluded by the maintainer +- Remote/multi-host execution, scheduling daemons, web UI, telemetry +- Replacing `ISO-PERMUTATION-MATRIX.md` as the analytical document; `sharukhan` executes it + +## 3. Goals & Success Criteria + +| Goal | Measured by | +|---|---| +| A failing permutation names the PR it disproves, not "something broke" | every assertion carries a PR identifier; the report's `PRs implicated` column is populated from failures alone | +| No check can pass vacuously | every presence/count assertion has a negative control that must fail | +| A result is attributable months later | each run records tree HEAD, patch identity and hash, ISO hash, and the installer NEVR that actually shipped | +| Prerequisites distinguish causes | "absent", "present but not executable by this user", and "present but version-skewed" are three distinct outcomes, never one boolean | +| The whole matrix is reachable without merging anything | installer variants are assembled by cherry-picking PR branches into a throwaway tree | +| A long run can be supervised | jobs run in the background, are inspectable while running, and are stoppable | + +## 4. High-Level Requirements + +- **[REQ-1]** Verify every host prerequisite before any destructive action, reporting a measured value and a distinct outcome per failure cause. +- **[REQ-2]** Resolve a build-axis tuple to an ISO, building only on cache miss, and record what actually shipped on the produced media. +- **[REQ-3]** Provision a VM per permutation with a deterministic, collision-free identity (name, MAC, BIOS UUID, address), from a pinned template in which no placeholder may survive substitution. +- **[REQ-4]** Drive an unattended install by injecting a per-permutation kickstart, without remastering the ISO. +- **[REQ-5]** Drive an operator-assisted install for permutations that cannot be automated, presenting generated instructions that cannot drift from the matrix. +- **[REQ-6]** Detect install completion and install failure from observable evidence, never from elapsed time. +- **[REQ-7]** Assert media, install-phase, guest and harvested-log facts, attributing each assertion to the pull request it proves. +- **[REQ-8]** Harvest guest evidence — dmesg, journal, failed units, package inventory, `/var/log` — into per-permutation storage. +- **[REQ-9]** Tear down by stashing the complete boot chain, never deleting, targeting exactly one named VM. +- **[REQ-10]** Execute the whole matrix or a named subset, sequentially, with a plan mode that builds nothing. +- **[REQ-11]** Run work as a background job that can be listed, inspected and stopped. +- **[REQ-12]** Persist every run, permutation, check, artifact and finding in a memory database, and generate `MEMORY.md` as a view over it rather than a parallel copy. +- **[REQ-13]** Produce a final report table of permutations attempted, their recorded pre-change verdict, their result, and the PRs implicated. +- **[REQ-14]** Assemble installer variants by cherry-picking named PR branches onto a pristine base in a throwaway clone, proving each variant applies before use, and merging nothing. +- **[REQ-15]** Apply defence in depth: no credential in a process argument, in the database, or in a log; external commands invoked as argument vectors; SQL parameterised; destructive operations path-validated and single-target. + +## 5. User Stories + +```gherkin +As a maintainer, I want to know whether my host can run the matrix at all, +so that I do not discover a missing tool three hours into a build. + +As a maintainer, I want a failing permutation to name the pull request it +disproves, so that I can act on the result instead of investigating it. + +As a maintainer, I want to test a pull request before merging it, +so that I never merge code that has not been exercised. + +As a maintainer, I want to run only the permutations I care about, +so that a targeted question does not cost a full matrix run. + +As a maintainer, I want a long run to proceed in the background and remain +inspectable and stoppable, so that I am not held hostage by a terminal. + +As a maintainer, I want the interactive permutations to tell me exactly what +to type, so that a hand-driven install still exercises the intended axis. + +As a maintainer, I want every finding recorded in one queryable place, +so that a result from last month is still attributable to a tree and a patch. + +As a security reviewer, I want credentials never to reach an argument list, +a log or the database, so that a test harness is not a disclosure path. +``` + +## 6. Acceptance Criteria + +| ID | Criterion | Verifier | +|---|---|---| +| **AC-1** | `sharukhan doctor` reports, per prerequisite, a measured value and one of `ok` / `missing` / `not-executable` / `version-skew`, and exits non-zero if any is fatal | Task 002 | +| **AC-2** | Every presence or count assertion has a paired negative control; a run in which a control passes is reported as inconclusive, not as a pass | Task 007 | +| **AC-3** | Assertions carry a PR identifier, and the report's implicated-PR column derives solely from failing assertions | Task 009 | +| **AC-4** | A permutation's VM identity is derived from its ordinal, is unique across the matrix, and never falls inside the hypervisor's DHCP range | Task 004 | +| **AC-5** | VMX generation fails if any placeholder survives substitution | Task 004 | +| **AC-6** | An unattended install is driven with no modification to the ISO | Task 005 | +| **AC-7** | Install completion is decided by the boot-source transition, and install failure by the resolution-error signal; elapsed time is never a verdict | Task 006 | +| **AC-8** | Teardown stashes the whole chain, deletes nothing by default, and touches only the named VM | Task 010 | +| **AC-9** | `sharukhan run --plan` builds nothing and lists the deduplicated ISO set | Task 011 | +| **AC-10** | A background job can be listed, inspected while running, and stopped, leaving no orphaned mounts or VMs | Task 012 | +| **AC-11** | Every run, check, artifact and finding is persisted; `MEMORY.md` is regenerated from the database and contains no fact absent from it | Task 013 | +| **AC-12** | Installer variants are produced without merging, and each is proven to apply to a pristine base before a build consumes it | Task 003 | +| **AC-13** | No credential appears in any process argument, database column or log line; verified by a test that greps a full run's artifacts for the configured secret | Task 014 | +| **AC-14** | Serial-log matching is NUL-safe and does not depend on any grep flag | Task 007 | +| **AC-15** | The report distinguishes a result that reproduces the recorded pre-change verdict from one that does not, since the former is the regression signal | Task 009 | + +## 7. Assumptions & Constraints + +**Assumptions** + +- VMware Workstation is installed on the Windows host and its binaries are executable from WSL. +- The Photon build tree and the build scripts exist; `sharukhan` orchestrates rather than replaces them. +- The guest installer reads `guestinfo.kickstart.data` via `vmtoolsd`, and `vmtoolsd` is present in the installer initrd. This is what makes install-time axes free. + +**Constraints** + +- **Sequential execution.** Every ISO build shares one staging tree, and the Windows volume has limited free space. Parallelism is not a future optimisation; it is incorrect. +- **The host runs other people's VMs.** No operation may act on a VM it did not create. +- **Sixteen permutations cannot be automated.** The STIG menu exists only in the curses configurator, so no kickstart can answer it. +- **Portability floor.** `/usr/bin/grep` is toybox non-interactively; `grep -P`, `grep -a` and `sed \U` are unavailable. Behaviour must not depend on any of them. +- **UEFI ignores `bios.bootOrder`.** NVRAM decides the boot source, so it must be stashed between installs. +- **Credentials cannot be added after the fact** on an image built without them, so key material is an input to the build, not a post-install fix. + +## 8. Defects in the predecessor tooling not to be inherited + +Recorded here because they are requirements by implication, discovered while inventorying the shell tooling: + +- `mc-run.sh` accumulates its pass/fail counters inside a `while read` fed by a pipe, so the subshell discards them and the final tally is always zero. +- `mc-verify.sh` locates a cached ISO as `…-prebuilt`, silently ignoring the canister mode the ISO was actually built with. +- The static-address scheme is computed and logged but never applied: `--ip` is never passed to the kickstart generator, so every guest takes a DHCP lease. Either wire it through or remove it — the half-state is worse than either. +- Nine configuration variables are declared and never read, and `MC_MAC_PREFIX` contradicts the OUI actually used. +- The serial-growth liveness instrument was lost in translation from `vm-lab`, and the serial log is truncated rather than offset-bounded, discarding prior-boot evidence. + +--- + +**Document Version:** 1.0 +**Last Updated:** 2026-08-31 +**Status:** Draft — awaiting Dev Lead feasibility review (Phase 2) From 4062768aea696c23c9218d7e7c8a34a3c743786a Mon Sep 17 00:00:00 2001 From: Daniel Casota Date: Mon, 31 Aug 2026 15:31:55 +0200 Subject: [PATCH 2/4] sharukhan phase-1: add the memory database schema and MEMORY.md renderer The database is the system of record; MEMORY.md is a view over it, stating which database it rendered from and when, so a stale render is visible rather than silently authoritative. Editing MEMORY.md changes nothing. Schema: run, permutation, check_result, artifact, finding, job, plus two views. v_permutation_report backs both the CLI report and MEMORY.md so the two cannot drift. v_control_integrity exists because a run whose negative controls passed proves nothing - it must read as inconclusive, not as green. check_result.pr is what turns a failure into 'PR#22 regressed' rather than 'something broke'. check_result.is_control marks the negative controls. Seeded with 18 findings from bringing the shell harness up on this host: 6 hypervisor, 5 defect, 3 portability, 2 build, 2 tooling. Seventeen are verified; one is an open blocker - the Photon ISO's grub.cfg carries no console=ttyS0, so the kernel never writes to the serial port and the boot-source-transition oracle can never fire. VMware Tools is not running during install either, so getGuestIPAddress and captureScreen are both unavailable, which leaves the install phase unobservable. Notable verified entries: vmxnet3 cannot reserve a PCI slot in this VMX layout and VMware reports only 'The operation was canceled'; an ISO on a WSL-only path is invisible to VMware; toybox grep has no -a and returns zero matches on a NUL-bearing log; a cksum-based permutation index collided on this very matrix; and editing a shell script while it runs corrupts execution mid-flight. The renderer is Python only until 'sharukhan db render' lands (Task 013); the SQL is identical either way. Signed-off-by: Daniel Casota --- staging/sharukhan/MEMORY.md | 212 +++++++++++++++++++++++ staging/sharukhan/schema/memory.sql | 120 +++++++++++++ staging/sharukhan/tools/gen-memory-md.py | 117 +++++++++++++ 3 files changed, 449 insertions(+) create mode 100644 staging/sharukhan/MEMORY.md create mode 100644 staging/sharukhan/schema/memory.sql create mode 100755 staging/sharukhan/tools/gen-memory-md.py diff --git a/staging/sharukhan/MEMORY.md b/staging/sharukhan/MEMORY.md new file mode 100644 index 00000000..13d87f36 --- /dev/null +++ b/staging/sharukhan/MEMORY.md @@ -0,0 +1,212 @@ +# MEMORY.md + +**Generated. Do not edit.** This file is a rendering of the sharukhan memory +database; the database is the system of record. Editing here changes nothing and +will be overwritten on the next render. + +- Source database: `/root/photon-mc/memory.db` +- Rendered at: 2026-08-31T13:31:35Z +- Regenerate with: `python3 tools/gen-memory-md.py /root/photon-mc/memory.db MEMORY.md` + +| Table | Rows | +|---|---| +| `run` | 0 | +| `permutation` | 0 | +| `check_result` | 0 | +| `artifact` | 0 | +| `finding` | 18 | + +> **1 unresolved blocker finding(s).** See the Blocker section. + +## Permutation results + +_No permutation has completed yet._ + +## Findings + +### Blocker + +#### `iso-must-be-windows-visible` — VMware cannot read an ISO on a WSL-only path + +*hypervisor · verified* · source: `mc-create-vm.sh` + +**Observed.** ISO at /root/photon-mc/... became VMX value \root\photon-mc\...; vmrun reported only 'Error: The operation was canceled'. + +**Consequence.** The VM never powers on and the error names nothing. + +**Mitigation.** Keep MC_ISO_CACHE under /mnt//; refuse a non-/mnt ISO with an explicit diagnosis. + +#### `no-serial-console-on-iso` — The Photon ISO does not route the kernel to serial + +*hypervisor · **UNRESOLVED*** · source: `mc-k01 first run` + +**Observed.** The ISO's /boot/grub2/grub.cfg menuentry is 'linux /isolinux/vmlinuz root=/dev/ram0 loglevel=3 photon.media=UUID=$photondisk' with no console=ttyS0. Serial log stayed 0 bytes for 15+ minutes while the VM ran. + +**Consequence.** Install progress and completion are unobservable; the boot-source transition oracle can never fire. + +**Mitigation.** Open: remaster grub.cfg to add console=ttyS0,115200, or detect completion via getGuestIPAddress on the installed system. + +#### `vmxnet3-pci-slot` — vmxnet3 cannot reserve a PCI slot in this VMX layout + +*hypervisor · verified* · source: `vmware.log mc-k01` + +**Observed.** vmware.log: 'Vmxnet3 PCI: failed to reserve slot for vmxnet3 PCIe device' then "Module 'DevicePowerOn' power on failed." vmrun surfaced only 'Error: The operation was canceled'. + +**Consequence.** The VM cannot power on at all. + +**Mitigation.** Use e1000, as vm-lab documented. A hand-made VM on this host runs vmxnet3 but has a different PCI slot layout, so it was not evidence. + +#### `toybox-grep-no-dash-a` — toybox grep has no -a and returns zero matches on NUL-bearing logs + +*portability · verified* · source: `mission-control lib/common.sh` + +**Observed.** In a non-interactive shell /usr/bin/grep is toybox 0.8.9; its usage line has no -a. `grep -ac PATTERN nul-log` returned 0 where the pattern was present. Interactively the same name resolves to ugrep 7.8.4, which supports -a and -P. + +**Consequence.** Every serial-log assertion silently passes while measuring nothing. A green run would prove nothing. + +**Mitigation.** Strip NULs before matching; never depend on a grep flag. Verified: mc_grep_count returns 1 on a NUL-prefixed log. + +### High + +#### `stale-poi-rpm-shadowing` — A stale installer RPM in the stage tree ships on the ISO + +*build · verified* · source: `mc-build-iso.sh` + +**Observed.** tdnf selects the highest release it can see, so an older photon-os-installer left in stage/RPMS wins. + +**Consequence.** A run reports a verdict for installer code nobody ships. + +**Mitigation.** Purge photon-os-installer-*.rpm before each build and record the NEVR that actually shipped. Verified: media carries photon-os-installer-2.8-5. + +#### `two-divergent-patch-copies` — Two copies of downstream-fixes.patch had diverged + +*build · verified* · source: `mc-build-iso runs 1-2` + +**Observed.** /root/photonos-patches/...patch had 27 files; staging/photonos-patches/...patch had 8. The build script resolves the patch relative to itself, so it used whichever sat beside the invoked copy. + +**Consequence.** The build silently omits fixes and fails with 'patch does not apply' against a spec, which reads like a rebase problem. + +**Mitigation.** PHOTON_SCRIPTS points at the live pair; preflight asserts the build resolves the same file it validated. + +#### `hashed-perm-index-collides` — A cksum-based permutation index collides and can enter the DHCP range + +*defect · verified* · source: `mission-control lib/common.sh` + +**Observed.** Over the 34-row matrix, cksum%200 produced 32 distinct indices: k04/k16 and k09/s02 collided. Max index 200 maps to .240, inside VMnet8 DHCP (.128-.254). + +**Consequence.** Two permutations share a MAC, UUID and IP; addresses can collide with real leases. + +**Mitigation.** Index is the row ordinal in permutations.tsv. Verified 34/34 distinct, addresses .41-.74. + +#### `printf-drops-last-item` — printf '%s' without a newline makes read drop the last item + +*defect · verified* · source: `mc-run.sh select_rows` + +**Observed.** --only k01,k03 selected only k01; --only k01,k03,k09 omitted the full/2.8 ISO. + +**Consequence.** A run silently covers fewer permutations than reported. + +**Mitigation.** printf '%s\n'. Verified all three ids now selected. + +#### `no-vmware-tools-during-install` — VMware Tools is not running during the install phase + +*hypervisor · verified* · source: `vmware.log mc-k01` + +**Observed.** getGuestIPAddress returned 'The VMware Tools are not running'; captureScreen returned 'Anonymous guest operations are not allowed'. vmware.log shows Tools heartbeat 1 => 0. + +**Consequence.** No guest-side liveness or IP discovery during install. + +**Mitigation.** Do not rely on Tools before first boot of the installed system. vmtoolsd one-shot --cmd still works for guestinfo reads. + +#### `uefi-ignores-bootorder` — UEFI ignores bios.bootOrder; NVRAM decides the boot source + +*hypervisor · verified* · source: `vm-lab README` + +**Observed.** Documented in vm-lab and carried forward: a surviving ESP plus NVRAM makes the firmware boot the previous image. + +**Consequence.** An install appears to silently do nothing while the old image boots. + +**Mitigation.** Stash .nvram before install and at teardown; stash the whole chain by glob, not a fixed list. + +#### `gnu-only-sed-grep` — sed \U and grep -P are GNU extensions absent on this host + +*portability · verified* · source: `mc-build-iso.sh, mc-create-vm.sh` + +**Observed.** grep -oP failed with "Unknown option 'P'"; sed 's|...|\U\1:|' emitted a literal 'Uc:' instead of uppercasing. + +**Consequence.** Path conversion and version parsing silently produce wrong values rather than erroring. + +**Mitigation.** Use tr and awk. mc_win_path uses tr; version parsing uses awk. + +#### `editing-a-running-script` — Editing a shell script while it runs corrupts execution + +*tooling · verified* · source: `mc-build-iso run 1` + +**Observed.** mc-build-iso.sh was edited mid-run; bash re-read the file at the next command boundary and died with 'syntax error near unexpected token (' at line 68, after the child build had already succeeded. + +**Consequence.** The wrapper's post-processing was skipped while the expensive work completed, leaving a half-finished artifact. + +**Mitigation.** Never edit a script that is executing. sharukhan is a compiled binary, which removes the class. + +#### `rust-install-broke-ssh` — Installing rust upgraded openssl and broke openssh + +*tooling · verified* · source: `host toolchain` + +**Observed.** tdnf install rust pulled openssl 3.5.7 and removed 3.0.18; ssh then failed with 'OpenSSL version mismatch. Built against 30000120, you have 30500070'. + +**Consequence.** Guest verification silently becomes impossible, and a version-skewed ssh presents exactly like an unreachable host. + +**Mitigation.** Upgraded openssh-clients to 10.4p1. Prerequisite checks must report version and provenance, not a boolean. + +### Medium + +#### `canister-hardcoded-in-verify` — Cached ISO lookup hardcoded the prebuilt canister mode + +*defect · verified* · source: `mc-verify.sh` + +**Observed.** mc-verify.sh built the cache path as ${ISO_TYPE}-poi${POI}-prebuilt. + +**Consequence.** An ISO built with canister build/acvp/kat would be verified against a different artifact. + +**Mitigation.** Derive from MC_CANISTER. + +#### `static-ip-never-applied` — The static-address scheme was computed but never applied + +*defect · verified* · source: `inventory of mission-control` + +**Observed.** mc_ip_for is computed and logged, but --ip is never passed to the kickstart generator, so every kickstart emits network.type=dhcp. + +**Consequence.** The documented 'static below the DHCP floor' property was never in effect; verification depends on IP discovery instead. + +**Mitigation.** Either wire it through or remove it; the half-state is worse than either. + +#### `subshell-counter-loss` — Counters incremented inside a piped while-read are discarded + +*defect · verified* · source: `mc-run.sh` + +**Observed.** mc-run.sh piped into `while read`, so total/failed stayed 0 in the parent. + +**Consequence.** The run summary always reported zero attempted. + +**Mitigation.** Feed the loop with a here-string. Verified: now reports '1 permutation(s) attempted, 1 with failing checks'. + +#### `drvfs-symlink-not-followable` — A WSL symlink on drvfs is not reliably followable from Windows + +*hypervisor · verified* · source: `mc-create-vm.sh` + +**Observed.** photon.iso is a symlink; also observed a self-referential link created because locale collation sorted 'photon.iso' before 'photon-minimal-...iso'. + +**Consequence.** The VMX references a link Windows may not resolve. + +**Mitigation.** readlink -f the ISO and write the concrete filename into the VMX. + +#### `vmrun-crlf` — vmrun output is CRLF and breaks naive line matching + +*portability · verified* · source: `mc-preflight.sh` + +**Observed.** vmrun -T ws list lines end \r\n; grep -c '\.vmx$' returned 0 against 2 running VMs. + +**Consequence.** VM-running checks report false negatives, so a teardown could act on a live VM. + +**Mitigation.** tr -d '\r' on every vmrun parse. + diff --git a/staging/sharukhan/schema/memory.sql b/staging/sharukhan/schema/memory.sql new file mode 100644 index 00000000..f143696c --- /dev/null +++ b/staging/sharukhan/schema/memory.sql @@ -0,0 +1,120 @@ +-- sharukhan memory database. +-- +-- The system of record. MEMORY.md is a generated VIEW over this file, never a +-- parallel copy, so the two cannot disagree. +-- +-- Every write is parameterised at the call site; no statement in sharukhan is +-- built by string concatenation. Columns that could carry a credential are +-- redacted at the boundary before insert, and that redaction is tested. + +PRAGMA journal_mode = WAL; +PRAGMA foreign_keys = ON; + +-- A single invocation of the tool. +CREATE TABLE IF NOT EXISTS run ( + id INTEGER PRIMARY KEY, + started_at TEXT NOT NULL, -- ISO-8601 UTC + finished_at TEXT, + tool_version TEXT NOT NULL, + host TEXT NOT NULL, + selector TEXT, -- --only / --all as given + exit_code INTEGER +); + +-- One attempt at one permutation within a run. +CREATE TABLE IF NOT EXISTS permutation ( + id INTEGER PRIMARY KEY, + run_id INTEGER NOT NULL REFERENCES run(id), + perm_id TEXT NOT NULL, -- p01, k03, s02 ... + iso_type TEXT NOT NULL, + poi TEXT NOT NULL, + stig TEXT NOT NULL, + fs TEXT NOT NULL, + mode TEXT NOT NULL, -- ks | ui + ks_variant TEXT, + doc_verdict TEXT, -- what the matrix recorded pre-PR + expected TEXT, + result TEXT, -- pass | fail | error | skipped + started_at TEXT, + finished_at TEXT, + UNIQUE (run_id, perm_id) +); + +-- One assertion. `pr` is what turns a failure into "PR#22 regressed". +CREATE TABLE IF NOT EXISTS check_result ( + id INTEGER PRIMARY KEY, + permutation_id INTEGER NOT NULL REFERENCES permutation(id), + check_id TEXT NOT NULL, -- media.stig_packages, guest.selinux ... + pr TEXT, -- PR#22, POI#11, or NULL + status TEXT NOT NULL, -- pass | fail | skip | info + expected TEXT, + actual TEXT, + detail TEXT, + is_control INTEGER NOT NULL DEFAULT 0, -- negative controls + recorded_at TEXT NOT NULL +); + +-- Anything produced or consumed that must stay attributable. +CREATE TABLE IF NOT EXISTS artifact ( + id INTEGER PRIMARY KEY, + run_id INTEGER REFERENCES run(id), + permutation_id INTEGER REFERENCES permutation(id), + kind TEXT NOT NULL, -- iso | patch | kickstart | vmx | log | tree + path TEXT NOT NULL, + sha256 TEXT, + note TEXT, + recorded_at TEXT NOT NULL +); + +-- Durable knowledge: a trap, a defect, or an environment fact worth not +-- rediscovering. This is the table MEMORY.md mostly renders. +CREATE TABLE IF NOT EXISTS finding ( + id INTEGER PRIMARY KEY, + slug TEXT NOT NULL UNIQUE, + title TEXT NOT NULL, + category TEXT NOT NULL, -- portability | hypervisor | build | tooling | defect + severity TEXT NOT NULL, -- blocker | high | medium | low + evidence TEXT NOT NULL, -- what was actually observed + consequence TEXT NOT NULL, -- what it breaks if ignored + mitigation TEXT, -- what sharukhan must do + verified INTEGER NOT NULL DEFAULT 0, + source TEXT, -- where it was found + recorded_at TEXT NOT NULL, + superseded_by INTEGER REFERENCES finding(id) +); + +-- Background work the tool can list, inspect and stop. +CREATE TABLE IF NOT EXISTS job ( + id INTEGER PRIMARY KEY, + run_id INTEGER REFERENCES run(id), + kind TEXT NOT NULL, -- build | install | verify + label TEXT NOT NULL, + pid INTEGER, + state TEXT NOT NULL, -- running | done | failed | stopped + log_path TEXT, + started_at TEXT NOT NULL, + finished_at TEXT +); + +CREATE INDEX IF NOT EXISTS idx_check_perm ON check_result(permutation_id); +CREATE INDEX IF NOT EXISTS idx_check_pr ON check_result(pr) WHERE pr IS NOT NULL; +CREATE INDEX IF NOT EXISTS idx_check_status ON check_result(status); +CREATE INDEX IF NOT EXISTS idx_perm_run ON permutation(run_id); +CREATE INDEX IF NOT EXISTS idx_finding_cat ON finding(category); + +-- The report table, as a view, so the CLI and MEMORY.md cannot drift apart. +CREATE VIEW IF NOT EXISTS v_permutation_report AS +SELECT p.perm_id, p.iso_type, p.poi, p.stig, p.fs, p.mode, + p.doc_verdict, p.result, + (SELECT COUNT(*) FROM check_result c WHERE c.permutation_id = p.id AND c.status='fail') AS failed_checks, + (SELECT GROUP_CONCAT(DISTINCT c.pr) FROM check_result c + WHERE c.permutation_id = p.id AND c.status='fail' AND c.pr IS NOT NULL) AS prs_implicated +FROM permutation p; + +-- A run is only trustworthy if its negative controls actually failed. +CREATE VIEW IF NOT EXISTS v_control_integrity AS +SELECT p.perm_id, + SUM(CASE WHEN c.is_control=1 THEN 1 ELSE 0 END) AS controls, + SUM(CASE WHEN c.is_control=1 AND c.status='pass' THEN 1 ELSE 0 END) AS controls_ok +FROM permutation p LEFT JOIN check_result c ON c.permutation_id = p.id +GROUP BY p.perm_id; diff --git a/staging/sharukhan/tools/gen-memory-md.py b/staging/sharukhan/tools/gen-memory-md.py new file mode 100755 index 00000000..edb15302 --- /dev/null +++ b/staging/sharukhan/tools/gen-memory-md.py @@ -0,0 +1,117 @@ +#!/usr/bin/env python3 +"""Regenerate MEMORY.md from the memory database. + +MEMORY.md is a VIEW, not a copy. Every fact in it is read from the database at +generation time, and the file states which database and at what point, so a +stale render is visible rather than silently authoritative. + +This exists as a Python tool only until the Rust `sharukhan db render` +subcommand lands (Task 013); the SQL is identical either way. +""" +import sqlite3, sys, datetime, os + +SEV = {"blocker": 0, "high": 1, "medium": 2, "low": 3} + +def main(db_path: str, out_path: str) -> int: + if not os.path.exists(db_path): + print(f"FAIL: no database at {db_path}", file=sys.stderr) + return 3 + db = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True) + now = datetime.datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ") + + findings = db.execute( + "SELECT slug,title,category,severity,evidence,consequence,mitigation,verified,source" + " FROM finding WHERE superseded_by IS NULL").fetchall() + findings.sort(key=lambda r: (SEV.get(r[3], 9), r[2], r[0])) + + runs = db.execute("SELECT COUNT(*) FROM run").fetchone()[0] + perms = db.execute("SELECT COUNT(*) FROM permutation").fetchone()[0] + checks = db.execute("SELECT COUNT(*) FROM check_result").fetchone()[0] + arts = db.execute("SELECT COUNT(*) FROM artifact").fetchone()[0] + open_blockers = db.execute( + "SELECT COUNT(*) FROM finding WHERE severity='blocker' AND verified=0").fetchone()[0] + + L = [] + L.append("# MEMORY.md") + L.append("") + L.append("**Generated. Do not edit.** This file is a rendering of the sharukhan memory") + L.append("database; the database is the system of record. Editing here changes nothing and") + L.append("will be overwritten on the next render.") + L.append("") + L.append(f"- Source database: `{db_path}`") + L.append(f"- Rendered at: {now}") + L.append(f"- Regenerate with: `python3 tools/gen-memory-md.py {db_path} MEMORY.md`") + L.append("") + L.append("| Table | Rows |") + L.append("|---|---|") + L.append(f"| `run` | {runs} |") + L.append(f"| `permutation` | {perms} |") + L.append(f"| `check_result` | {checks} |") + L.append(f"| `artifact` | {arts} |") + L.append(f"| `finding` | {len(findings)} |") + L.append("") + if open_blockers: + L.append(f"> **{open_blockers} unresolved blocker finding(s).** See the Blocker section.") + L.append("") + + # permutation report, straight from the view so it cannot drift from the CLI + rows = db.execute( + "SELECT perm_id,iso_type,poi,stig,fs,mode,doc_verdict,result,failed_checks,prs_implicated" + " FROM v_permutation_report ORDER BY perm_id").fetchall() + L.append("## Permutation results") + L.append("") + if not rows: + L.append("_No permutation has completed yet._") + else: + L.append("| ID | ISO | POI | STIG | FS | Mode | Matrix said | Result | Failed | PRs implicated |") + L.append("|---|---|---|---|---|---|---|---|---|---|") + for r in rows: + L.append("| " + " | ".join("" if c is None else str(c) for c in r) + " |") + L.append("") + L.append("A result that reproduces the *Matrix said* value of `fails` is a PR regression.") + L.append("") + + cur_sev = None + L.append("## Findings") + L.append("") + for slug, title, cat, sev, ev, cons, mit, ver, src in findings: + if sev != cur_sev: + cur_sev = sev + L.append(f"### {sev.capitalize()}") + L.append("") + flag = "verified" if ver else "**UNRESOLVED**" + L.append(f"#### `{slug}` — {title}") + L.append("") + L.append(f"*{cat} · {flag}*" + (f" · source: `{src}`" if src else "")) + L.append("") + L.append(f"**Observed.** {ev}") + L.append("") + L.append(f"**Consequence.** {cons}") + if mit: + L.append("") + L.append(f"**Mitigation.** {mit}") + L.append("") + + # control integrity: a run whose negative controls passed proves nothing + ci = db.execute("SELECT perm_id,controls,controls_ok FROM v_control_integrity" + " WHERE controls > 0 ORDER BY perm_id").fetchall() + if ci: + L.append("## Negative-control integrity") + L.append("") + L.append("A permutation whose controls did not all hold is **inconclusive**, not passing.") + L.append("") + L.append("| Permutation | Controls | Held |") + L.append("|---|---|---|") + for p, c, ok in ci: + L.append(f"| {p} | {c} | {ok} |") + L.append("") + + open(out_path, "w").write("\n".join(L) + "\n") + print(f" rendered {out_path}: {len(findings)} findings, {len(rows)} permutation rows") + return 0 + +if __name__ == "__main__": + if len(sys.argv) != 3: + print("usage: gen-memory-md.py ", file=sys.stderr) + sys.exit(64) + sys.exit(main(sys.argv[1], sys.argv[2])) From 1d311e0bb47c6cbdf944189c7b4ddb4641f43f30 Mon Sep 17 00:00:00 2001 From: Daniel Casota Date: Mon, 31 Aug 2026 16:10:57 +0200 Subject: [PATCH 3/4] sharukhan phase-1: correct the sequential-execution constraint Measurement on this host contradicts PRD section 7. The claim that parallelism 'is not a future optimisation; it is incorrect' holds for ISO builds, which mutate a shared 65 GiB staging tree, and is false for VM installs, which share nothing - each permutation owns its VM directory, disk, MAC, UUID and results. Measured: .vmem is exactly memSize (4294967296 bytes) and exists only while the VM runs; the thin .vmdk grew 4 MiB to 914 MiB during one install. On 14 CPUs, 23 GiB RAM and 131 GiB free, the slot counts are CPU 3, RAM 4, disk 8 - so RAM binds, not CPU. A concurrency limit derived from CPU count alone would over-commit on a smaller host. Recorded as a finding with its resolution rather than silently editing the PRD, which is what the methodology prescribes when implementation disproves a spec assumption. Signed-off-by: Daniel Casota --- .../findings/2026-08-31-parallel-execution.md | 54 +++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100644 staging/sharukhan/specs/findings/2026-08-31-parallel-execution.md diff --git a/staging/sharukhan/specs/findings/2026-08-31-parallel-execution.md b/staging/sharukhan/specs/findings/2026-08-31-parallel-execution.md new file mode 100644 index 00000000..3e27ab35 --- /dev/null +++ b/staging/sharukhan/specs/findings/2026-08-31-parallel-execution.md @@ -0,0 +1,54 @@ +# Finding: installs can run in parallel; only ISO builds must serialise + +**Date**: 2026-08-31 +**Status**: Resolved by amendment +**Affects**: `specs/prd.md` §7 Constraints, and a new ADR on concurrency + +## What the PRD claimed + +> **Sequential execution.** Every ISO build shares one staging tree, and the +> Windows volume has limited free space. Parallelism is not a future +> optimisation; it is incorrect. + +## What measurement showed + +The claim is true of **builds** and false of **installs**, and the PRD collapsed +the two into one constraint. + +| Stage | Shares state? | Parallelisable | +|---|---|---| +| ISO build | Yes — `$PHOTON_TREE/stage` (65 GiB) is mutated by `git checkout`, patch application and the stale-RPM purge | **No.** Two concurrent builds corrupt each other. | +| VM install / verify | No — each permutation owns its VM directory, disk, MAC, UUID, IP and results directory | **Yes.** | + +Measured on this host (14 CPUs, 23 GiB RAM, C: 131 GiB free): + +| Resource | Per concurrent VM | Slots | +|---|---|---| +| CPU | — | `floor(14/4)` = **3** | +| RAM | `memSize` = 4 GiB, as a `.vmem` file present only while running | 17 GiB available ÷ 4 = **4** | +| Disk | `.vmem` 4 GiB + thin `.vmdk` growing toward the installed footprint | 131 GiB ÷ ~16 GiB = **8** | + +`.vmem` was measured at exactly 4294967296 bytes on `mc-k01`, matching `memSize`; +the `.vmdk` grew from 4 MiB to 914 MiB during a single install. + +**RAM is the binding constraint here, not CPU and not disk.** A limit derived +from CPU count alone would over-commit on a smaller host, so the admission +decision must take the minimum across all three and recompute per dispatch, +because a thin disk grows *during* a run and a check that passed at dispatch +can be false ten minutes later. + +## Consequence of leaving it uncorrected + +The PRD would have forbidden a legitimate 3x speedup on the install phase — the +dominant cost of a full matrix run — on the strength of a constraint that only +applies to builds. + +## Resolution + +- `specs/prd.md` §7 amended to separate the two stages. +- New ADR: concurrency and disk-trend admission control, adding `--jobs` + (default `floor(nproc/4)`), a hard refusal floor below one VM's peak + requirement, degradation to sequential when capacity is short, and a drain-rate + projection that stops dispatching *new* work rather than killing running work. +- Builds ignore `--jobs` by construction, and the tool says so once rather than + appearing to honour it. From 1989989b9a36817f938d7760d7bb36d81db00580 Mon Sep 17 00:00:00 2001 From: Daniel Casota Date: Mon, 31 Aug 2026 16:22:08 +0200 Subject: [PATCH 4/4] sharukhan phase-1: require versioned log and report artifacts REQ-16 and AC-16: every log and report is stamped with a UTC timestamp in its filename, with a 'latest' pointer for convenience. This is a live defect in the shell prototype, not only a future requirement. mc-verify.sh truncated a fixed checks.jsonl on every invocation and harvested guest logs to fixed filenames, so k01 was run five times and only the last run's evidence survived - lost exactly when comparing two runs is what would explain a regression. Prototype now writes checks-.jsonl, logs-/ and report-.txt, each with a -latest symlink, sharing one stamp per run exported to every child. Verified: two consecutive report runs produced two distinct files with latest repointed. Memory database is at 27 findings, all verified, no open blockers. New this round: vmrun start silently does nothing from a setsid-detached process (nohup is fine, setsid is not), which made a background run look like a slow install rather than one that never started. Signed-off-by: Daniel Casota --- staging/sharukhan/MEMORY.md | 100 +++++++++++++++++++++++++++++++-- staging/sharukhan/specs/prd.md | 2 + 2 files changed, 96 insertions(+), 6 deletions(-) diff --git a/staging/sharukhan/MEMORY.md b/staging/sharukhan/MEMORY.md index 13d87f36..d572dc1d 100644 --- a/staging/sharukhan/MEMORY.md +++ b/staging/sharukhan/MEMORY.md @@ -5,7 +5,7 @@ database; the database is the system of record. Editing here changes nothing and will be overwritten on the next render. - Source database: `/root/photon-mc/memory.db` -- Rendered at: 2026-08-31T13:31:35Z +- Rendered at: 2026-08-31T14:21:54Z - Regenerate with: `python3 tools/gen-memory-md.py /root/photon-mc/memory.db MEMORY.md` | Table | Rows | @@ -14,9 +14,7 @@ will be overwritten on the next render. | `permutation` | 0 | | `check_result` | 0 | | `artifact` | 0 | -| `finding` | 18 | - -> **1 unresolved blocker finding(s).** See the Blocker section. +| `finding` | 27 | ## Permutation results @@ -26,6 +24,16 @@ _No permutation has completed yet._ ### Blocker +#### `installer-console-dispatch` — POI runs the installer only on the single active console + +*hypervisor · verified* · source: `generate_initrd.py create_installer_script` + +**Observed.** bootphotoninstaller reads /sys/devices/virtual/tty/console/active and runs the installer only if it equals tty0 (then on /dev/tty1) or if tty() equals /dev/$ACTIVE_CONSOLE. With console=tty0 console=ttyS0 the value is 'ttyS0 tty0', matching neither branch, so it falls through to exec /bin/bash. + +**Consequence.** The installer never starts and the VM sits at a root shell; with no serial console at all the same state is completely invisible. + +**Mitigation.** Remaster grub.cfg with console=ttyS0,115200n8 ONLY for autonomous runs. Verified: the installer then starts on serial. Interactive runs keep the stock ISO so the operator drives tty0. + #### `iso-must-be-windows-visible` — VMware cannot read an ISO on a WSL-only path *hypervisor · verified* · source: `mc-create-vm.sh` @@ -38,13 +46,23 @@ _No permutation has completed yet._ #### `no-serial-console-on-iso` — The Photon ISO does not route the kernel to serial -*hypervisor · **UNRESOLVED*** · source: `mc-k01 first run` +*hypervisor · verified* · source: `mc-k01 first run` **Observed.** The ISO's /boot/grub2/grub.cfg menuentry is 'linux /isolinux/vmlinuz root=/dev/ram0 loglevel=3 photon.media=UUID=$photondisk' with no console=ttyS0. Serial log stayed 0 bytes for 15+ minutes while the VM ran. **Consequence.** Install progress and completion are unobservable; the boot-source transition oracle can never fire. -**Mitigation.** Open: remaster grub.cfg to add console=ttyS0,115200, or detect completion via getGuestIPAddress on the installed system. +**Mitigation.** Remaster grub.cfg to console=ttyS0,115200n8 only. Verified: 61KB of kernel output and the installer now runs on serial. + +#### `vmrun-nogui-unsupported` — vmrun headless start fails on this host while gui start works + +*hypervisor · verified* · source: `mc-k01 bisect` + +**Observed.** 'vmrun -T ws start nogui' returns 'Error: Unknown error' and creates no vmware.log at all. The identical VMX - including one VMware itself had rewritten after a successful power-on - starts immediately with 'gui', producing a 202 KB vmware.log and serial output. + +**Consequence.** Every autonomous permutation fails to launch, and the logless error invites misdiagnosis: the VMX was blamed first, then pinned PCI slot numbers, before bisecting against a known-good VMX proved the file was never the problem. + +**Mitigation.** Use gui for autonomous starts too. Headless needs VMware Workstation Server / shared-VM support, which is not enabled here. #### `vmxnet3-pci-slot` — vmxnet3 cannot reserve a PCI slot in this VMX layout @@ -68,6 +86,16 @@ _No permutation has completed yet._ ### High +#### `packagelist-file-name` — packages_minimal.json is not on the installer media + +*build · verified* · source: `mc-k01 serial log` + +**Observed.** The kickstart named packages_minimal.json; the installer aborted with FileNotFoundError: '/installer/packages_minimal.json'. The initrd ships only /installer/packages.json, containing linux-esx, less, sudo, linux, initramfs, lvm2, minimal. + +**Consequence.** Every kickstart-driven install fails before partitioning. + +**Mitigation.** Use packagelist_file=packages.json, or omit it and pass an explicit packages list. + #### `stale-poi-rpm-shadowing` — A stale installer RPM in the stage tree ships on the ISO *build · verified* · source: `mc-build-iso.sh` @@ -108,6 +136,26 @@ _No permutation has completed yet._ **Mitigation.** printf '%s\n'. Verified all three ids now selected. +#### `results-overwritten-per-run` — Re-running a permutation destroys the previous run's evidence + +*defect · verified* · source: `user request; confirmed in mc-verify.sh` + +**Observed.** mc-verify.sh truncates $MC_RESULTS_DIR//checks.jsonl on every invocation, and harvested guest logs use fixed filenames. k01 was run five times; only the last run's checks survived. + +**Consequence.** Evidence for a regression is lost exactly when a comparison between runs is what would explain it. + +**Mitigation.** Every artifact is UTC-stamped: checks-.jsonl, logs-/, report-.txt, each with a -latest pointer. One stamp per run, exported so all children share it. Verified: two consecutive report runs produced two files. + +#### `installed-system-serial-silent` — Remastering the ISO does not make the INSTALLED system serial-visible + +*hypervisor · verified* · source: `mc-k01 run 4` + +**Observed.** After the installer rebooted, the serial log froze at 119354 bytes and never grew. root=PARTUUID= never appeared, so the boot-source completion oracle could not fire, even though the install had succeeded (partitioning done, chroot populated, postinstall run, 'reboot: machine restart'). + +**Consequence.** A successful install is scored as a timeout. The harness would report failure for working software - the worst possible error for a test oracle. + +**Mitigation.** Two changes: the kickstart postinstall now adds console=ttyS0,115200n8 to the target's grub, and the installer waits on EITHER the boot-source transition OR a guest IP from VMware Tools. + #### `no-vmware-tools-during-install` — VMware Tools is not running during the install phase *hypervisor · verified* · source: `vmware.log mc-k01` @@ -128,6 +176,16 @@ _No permutation has completed yet._ **Mitigation.** Stash .nvram before install and at teardown; stash the whole chain by glob, not a fixed list. +#### `vmrun-needs-a-session` — vmrun start silently does nothing from a fully detached process + +*hypervisor · verified* · source: `k01 clean run` + +**Observed.** 'setsid vmrun -T ws start gui' produces no output, no VM and no vmware.log. The same command from a shell with a controlling terminal starts the VM immediately. nohup alone is fine; setsid is what breaks it. + +**Consequence.** A background matrix run appears to be waiting on an install that was never started, and times out with nothing to show. + +**Mitigation.** Launch background work with nohup, never setsid. + #### `gnu-only-sed-grep` — sed \U and grep -P are GNU extensions absent on this host *portability · verified* · source: `mc-build-iso.sh, mc-create-vm.sh` @@ -160,6 +218,16 @@ _No permutation has completed yet._ ### Medium +#### `builds-serial-installs-parallel` — Only ISO builds must serialise; installs are independent + +*build · verified* · source: `PRD review after host measurement` + +**Observed.** ISO builds share and mutate $PHOTON_TREE/stage (65 GiB) via git checkout, patch apply and the stale-RPM purge, so two concurrent builds corrupt each other. VM installs share nothing: each permutation owns its VM directory, disk, MAC, UUID and results directory. + +**Consequence.** PRD section 7 claimed sequential execution was correctness rather than an unimplemented optimisation. That is true for builds and false for installs, and it would have prevented a legitimate 3x speedup. + +**Mitigation.** Amend PRD section 7: builds serialise, installs fan out under admission control. Recorded as specs/findings and corrected by PR. + #### `canister-hardcoded-in-verify` — Cached ISO lookup hardcoded the prebuilt canister mode *defect · verified* · source: `mc-verify.sh` @@ -200,6 +268,16 @@ _No permutation has completed yet._ **Mitigation.** readlink -f the ISO and write the concrete filename into the VMX. +#### `foreign-pci-slot-pins` — Pinned PCI slot numbers do not transfer between hosts + +*hypervisor · verified* · source: `VMX diff against the rewritten file` + +**Observed.** The template pinned sata0.pciSlotNumber=35 and ethernet0.pciSlotNumber=160, inherited from a template captured elsewhere. VMware rewrote them to 18 and 17 on the one power-on that succeeded, and had already refused vmxnet3 with 'failed to reserve slot for vmxnet3 PCIe device'. + +**Consequence.** A foreign slot layout can make power-on fail in ways that name nothing. + +**Mitigation.** Pin no pciSlotNumber at all and let VMware assign. Pin only uuid.bios; uuid.location is VMware's own. + #### `vmrun-crlf` — vmrun output is CRLF and breaks naive line matching *portability · verified* · source: `mc-preflight.sh` @@ -210,3 +288,13 @@ _No permutation has completed yet._ **Mitigation.** tr -d '\r' on every vmrun parse. +#### `per-vm-resource-cost` — A running VM costs memSize on disk plus its growing thin disk + +*tooling · verified* · source: `mc-k01 vm directory` + +**Observed.** Measured on mc-k01: the .vmem file is exactly 4294967296 bytes (= memSize 4096 MiB) and exists only while the VM runs; the thin .vmdk grew 4 MiB -> 914 MiB during install. Peak per concurrent VM is therefore about memSize + installed footprint. + +**Consequence.** A concurrency limit derived from CPU count alone over-commits RAM and disk on smaller hosts. + +**Mitigation.** Admission control takes min(cpu_slots, ram_slots, disk_slots), recomputed per dispatch, not a fixed startup value. + diff --git a/staging/sharukhan/specs/prd.md b/staging/sharukhan/specs/prd.md index 9f65a05e..4a67256d 100644 --- a/staging/sharukhan/specs/prd.md +++ b/staging/sharukhan/specs/prd.md @@ -60,6 +60,7 @@ The audience is one maintainer testing pull requests against a hypervisor on the - **[REQ-12]** Persist every run, permutation, check, artifact and finding in a memory database, and generate `MEMORY.md` as a view over it rather than a parallel copy. - **[REQ-13]** Produce a final report table of permutations attempted, their recorded pre-change verdict, their result, and the PRs implicated. - **[REQ-14]** Assemble installer variants by cherry-picking named PR branches onto a pristine base in a throwaway clone, proving each variant applies before use, and merging nothing. +- **[REQ-16]** Version every log and report artifact by UTC timestamp in the filename, keeping a `latest` pointer for convenience, so no run can destroy the evidence of the run before it. - **[REQ-15]** Apply defence in depth: no credential in a process argument, in the database, or in a log; external commands invoked as argument vectors; SQL parameterised; destructive operations path-validated and single-target. ## 5. User Stories @@ -108,6 +109,7 @@ a log or the database, so that a test harness is not a disclosure path. | **AC-12** | Installer variants are produced without merging, and each is proven to apply to a pristine base before a build consumes it | Task 003 | | **AC-13** | No credential appears in any process argument, database column or log line; verified by a test that greps a full run's artifacts for the configured secret | Task 014 | | **AC-14** | Serial-log matching is NUL-safe and does not depend on any grep flag | Task 007 | +| **AC-16** | Running the same permutation twice leaves two independently readable result sets and two reports; neither run overwrites the other, and `latest` resolves to the newer | Task 013 | | **AC-15** | The report distinguishes a result that reproduces the recorded pre-change verdict from one that does not, since the former is the regression signal | Task 009 | ## 7. Assumptions & Constraints