From 9685ac391b7fee16aaaf8936f1a0c06ad33f3436 Mon Sep 17 00:00:00 2001 From: Sean Shaffer Date: Sun, 8 Nov 2026 22:37:03 -0600 Subject: [PATCH 01/12] docs: plan selected 1.x roadmap features Record the approved scope, staged acceptance criteria, and format/boot asset references before implementation. Preserve the existing USB-read work as separate uncommitted changes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- docs/references.md | 3 + tasks/plan.md | 91 ++++++++++++++ tasks/todo.md | 307 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 401 insertions(+) create mode 100644 tasks/plan.md create mode 100644 tasks/todo.md diff --git a/docs/references.md b/docs/references.md index 057bcd2..db5cd21 100644 --- a/docs/references.md +++ b/docs/references.md @@ -38,6 +38,9 @@ Primary sources to check before implementing or changing behavior (per the dev c - UEFI specifications: https://uefi.org/specifications · ATA (T13): https://www.t13.org - Boot loaders: GRUB https://www.gnu.org/software/grub · Syslinux https://www.syslinux.org · rEFInd https://www.rodsbooks.com/refind +- GRUB loopback booting: https://www.gnu.org/software/grub/manual/grub/html_node/Loopback-booting.html — GRUB documents that loopback compatibility depends on each OS being able to discover its root from the loopback image; it is not universal. +- Microsoft VHD format specification: https://www.microsoft.com/en-us/download/details.aspx?id=23850 +- XDVDFS/XISO implementation reference: https://github.com/XboxDev/extract-xiso — format behavior reference only; no code or game content is copied. - Virtualization: QEMU https://www.qemu.org · VirtualBox https://www.virtualbox.org · VMware https://www.vmware.com · Hyper-V https://learn.microsoft.com/windows-server/virtualization/hyper-v · Evaluation Center https://www.microsoft.com/evalcenter - Partitioning / FAT: GNU Parted https://www.gnu.org/software/parted · gdisk https://www.rodsbooks.com/gdisk · mtools https://www.gnu.org/software/mtools - Installers / init: Anaconda https://www.fedoraproject.org/wiki/Anaconda · systemd https://systemd.io diff --git a/tasks/plan.md b/tasks/plan.md new file mode 100644 index 0000000..8767447 --- /dev/null +++ b/tasks/plan.md @@ -0,0 +1,91 @@ +# Implementation Plan: IsoFoundry 1.x Roadmap Features + +## Overview + +Complete the four explicitly selected 1.x roadmap features: multiboot USB, FAT/VHD image editing, Xbox XISO, and Spanish UI. Keep existing product behavior and the in-progress USB-to-image GUI changes intact. Deliver each feature through the appropriate GUI and CLI surfaces, with fixtures, focused tests, documentation, and software-only boot verification. Do not claim physical-hardware verification. + +## Scope and assumptions + +- Multiboot USB supports multiple Linux ISO loopback entries and Windows/WinPE ISO or WIM boot entries on BIOS and UEFI. The app manages installation, adding/removing payloads, and per-entry options. Secure Boot support must use distributable signed components or document a clearly bounded limitation; users must not be told to disable Secure Boot as the default path. +- FAT image editing supports FAT12/16/32 floppy and partition images. VHD support means legacy fixed and dynamic VHD containers containing supported FAT volumes. VHDX support is not included unless existing formats or implementation evidence require it. +- Xbox XISO means XDVDFS filesystem support for opening, listing, extracting, and creating images. No executable patching, decryption, or copy-protection circumvention. +- Spanish UI covers all user-facing GUI strings, including dialogs, menus, errors and task windows. CLI output and technical documentation remain English unless needed to explain language selection. +- Physical hardware tests are excluded. Unit/integration fixtures and QEMU boot checks remain required where applicable. + +## Architecture decisions + +- Extend `if-image`'s filesystem/source abstractions for FAT and XDVDFS rather than introducing parallel app-only parsers. Keep format writers pure Rust and platform-independent where practical. +- VHD parsing/writing is a container layer over byte-addressable storage; validate bounds, checksums, block maps and metadata before exposing a filesystem. +- Multiboot USB writes continue through the existing guarded USB device/helper path. Disk-image/VHDX test targets stay behind the current explicit development gate. +- Preserve the app's offline-by-default behavior. Any bootloader assets must have documented provenance and redistribution terms; no runtime download is required for ordinary use. +- Use a Spanish resource catalog with English fallback, explicit language selection, and completeness tests. Avoid scattering translated literals through Win32 window procedures. + +## Task List + +### Phase 0: Baseline and format investigation +- [ ] Task 1: Establish preserved baseline and validate third-party boot assets. + +### Checkpoint: Baseline +- [ ] Existing working-tree changes are preserved and baseline workspace checks pass. +- [ ] XISO, FAT/VHD, and multiboot format/asset decisions are supported by repository evidence and documented references. + +### Phase 1: Xbox XISO +- [ ] Task 2: Read and validate XDVDFS volumes. +- [ ] Task 3: Expose XISO browse and extraction in CLI and GUI. +- [ ] Task 4: Create XISO images and add end-to-end coverage. + +### Checkpoint: XISO +- [ ] Fixture images list, extract and recreate files with byte-for-byte verification. +- [ ] CLI and GUI workflows are documented and tested without physical hardware. + +### Phase 2: FAT and VHD image editing +- [ ] Task 5: Read FAT12/16/32 volumes and locate supported partitions. +- [ ] Task 6: Browse and extract FAT images through existing image workflows. +- [ ] Task 7: Edit FAT images and save safely as a new image. +- [ ] Task 8: Read and write fixed/dynamic VHD containers with validated metadata. +- [ ] Task 9: Integrate VHD-contained FAT editing and verify round trips. + +### Checkpoint: FAT/VHD +- [ ] FAT12/16/32 fixture images and fixed/dynamic VHD fixtures browse, extract, edit and reopen correctly. +- [ ] Original images remain unchanged; malformed and oversized inputs fail cleanly. + +### Phase 3: Multiboot USB +- [ ] Task 10: Prove the bootloader/asset path and reproducible BIOS/UEFI QEMU test setup. +- [ ] Task 11: Install a multiboot layout through the guarded USB engine. +- [ ] Task 12: Manage ISO/WIM payloads and persistent per-entry boot options. +- [ ] Task 13: Integrate multiboot management in the Windows task window and CLI. +- [ ] Task 14: Verify boot scenarios and document supported payloads and Secure Boot status. + +### Checkpoint: Multiboot +- [ ] VHDX-backed install/manage operations pass safety tests; supported BIOS and UEFI scenarios boot under QEMU. +- [ ] No destructive physical-device operation is used for testing; Secure Boot claims match the verified asset chain. + +### Phase 4: Spanish UI +- [ ] Task 15: Add locale/resource loading and language selection with English fallback. +- [ ] Task 16: Localize shared menus, task catalog, and core dialogs. +- [ ] Task 17: Localize task windows and operational messages in bounded groups. +- [ ] Task 18: Enforce catalog completeness and verify the Spanish UI workflow. + +### Checkpoint: Spanish UI +- [ ] Every user-facing GUI string has an English source and Spanish translation. +- [ ] English remains the default/fallback; both locales render through the same GUI paths. + +### Checkpoint: Complete +- [ ] All four selected feature acceptance criteria are met. +- [ ] Workspace formatting, tests, Clippy, and application build pass. +- [ ] README, roadmap, limitations, and changelog accurately describe support and verification. + +## Risks and mitigations + +| Risk | Impact | Mitigation | +|---|---|---| +| Secure Boot-compatible multiboot assets may have restrictive distribution terms or key-enrollment constraints. | High | Validate provenance/license and the actual trust chain before choosing the bootloader; do not make unsupported Secure Boot claims. | +| Dynamic VHD and FAT partition edge cases can cause corruption or out-of-bounds reads. | High | Keep parsers read-only until validation is complete; test malformed metadata, bounds, and atomic new-file writes. | +| Booting arbitrary Linux ISOs by loopback is not universal. | High | Define a tested compatibility set and expose/document per-entry options rather than claiming every ISO works. | +| XDVDFS and FAT name/metadata semantics differ from ISO editing. | Medium | Preserve raw names and metadata where feasible; add adversarial fixtures and explicit unsupported-feature errors. | +| UI strings are spread across many Win32 task windows. | High | Centralize string lookup and migrate windows in small groups with a coverage test preventing untranslated literals. | +| Existing USB-to-image changes are uncommitted in this worktree. | High | Treat them and `.serena/` as user/worktree state; do not reset, overwrite, or clean them up. | + +## Open questions + +- None blocking at planning time. The plan uses the supported-format and Secure Boot assumptions above; revise only if format research disproves them. diff --git a/tasks/todo.md b/tasks/todo.md new file mode 100644 index 0000000..8c2aece --- /dev/null +++ b/tasks/todo.md @@ -0,0 +1,307 @@ +# IsoFoundry 1.x Roadmap Work + +## Task 1: Establishing the preserved baseline and boot-asset constraints — Done + +**Description:** Verify the current workspace state and baseline checks without disturbing the existing USB-to-image work. Inspect XISO/FAT/VHD format references and identify lawful, reproducible multiboot boot assets and their Secure Boot properties. + +**Acceptance criteria:** +- [x] Existing tracked and untracked changes are recorded and preserved. +- [x] Bootloader candidates, asset provenance/license obligations, and Secure Boot requirements are recorded in the feature docs or task implementation notes. +- [x] No unsupported format or boot compatibility claim is made. + +**Verification:** +- [x] Run `cargo fmt --check`, `cargo test --workspace`, workspace Clippy with warnings denied, and `cargo build -p isofoundry`. +- [x] Record candidate references and terms before adding binary assets; final multiboot asset selection remains gated by Task 10. + +**Dependencies:** None +**Files likely touched:** `docs/roadmap.md`, `docs/references.md` (only after evidence is established) +**Estimated scope:** Small + +## Task 2: Reading and validating XDVDFS volumes — In progress + +**Description:** Add a pure-Rust XDVDFS reader behind the image engine's filesystem/source abstractions, including bounded directory traversal and file extent reads. + +**Acceptance criteria:** +- [ ] Valid XISO fixtures expose volume metadata, directory entries, and file contents. +- [ ] Cycles, invalid extents, truncated images, and unreasonable directory sizes return errors without panic or unbounded allocation. +- [ ] Filesystem name/label and read-only status are exposed to callers. + +**Verification:** +- [ ] Add unit tests for valid, nested, empty, and malformed XDVDFS fixtures. +- [ ] Run the focused `if-image` tests. + +**Dependencies:** Task 1 +**Files likely touched:** `crates/if-image/src/xiso.rs`, `crates/if-image/src/fs.rs`, `crates/if-image/src/lib.rs`, `crates/if-image/tests/read.rs` +**Estimated scope:** Medium + +## Task 3: Exposing XISO browse and extraction in CLI and GUI + +**Description:** Route XISO images through image-info, browse, and extract flows while keeping XISO-specific metadata visible and preserving existing formats. + +**Acceptance criteria:** +- [ ] `image-info` recognizes XISO and reports its filesystem. +- [ ] CLI extraction and the existing browse/extract task window can read XISO files. +- [ ] Unsupported operations fail with an explicit format-specific message. + +**Verification:** +- [ ] Add CLI and integration tests that browse and extract nested XISO fixtures. +- [ ] Build the app and check GUI routing through existing image-task patterns. + +**Dependencies:** Task 2 +**Files likely touched:** `crates/isofoundry/src/cli_image.rs`, `crates/isofoundry/src/win/tasks/browse.rs`, `crates/if-image/tests/read.rs` +**Estimated scope:** Medium + +## Task 4: Creating XISO images and end-to-end coverage + +**Description:** Add an XDVDFS writer and an explicit XISO create path that produces compatible images without touching game executables or protected content. + +**Acceptance criteria:** +- [ ] A source tree can be written as XISO and reopened by the new reader. +- [ ] File/directory extents, alignment, names, and size limits are validated. +- [ ] Existing ISO creation and editing output remain unchanged. + +**Verification:** +- [ ] Round-trip tests compare every source file byte-for-byte and exercise invalid names/limits. +- [ ] Document CLI and GUI availability, filesystem scope, and exclusions. + +**Dependencies:** Task 2 +**Files likely touched:** `crates/if-image/src/xiso_write.rs`, `crates/if-image/src/lib.rs`, `crates/isofoundry/src/cli_image.rs`, focused tests/docs +**Estimated scope:** Medium + +## Task 5: Reading FAT volumes and locating supported partitions + +**Description:** Implement bounded FAT12/16/32 volume parsing and identify supported superfloppy or MBR-partitioned FAT volumes in raw disk images. + +**Acceptance criteria:** +- [ ] FAT12/16/32 fixtures expose directory trees, long names, file extents, and labels where available. +- [ ] Partition/BPB geometry and all cluster chains are bounds-checked; unsupported/exFAT layouts are rejected clearly. +- [ ] Malformed loops and inconsistent chains terminate safely. + +**Verification:** +- [ ] Add fixture tests across FAT types, nested directories, long names, fragmented files, and corrupt chains. +- [ ] Run focused `if-image` tests and malformed-input drills. + +**Dependencies:** Task 1 +**Files likely touched:** `crates/if-image/src/fat.rs`, `crates/if-image/src/fs.rs`, `crates/if-image/src/lib.rs`, `crates/if-image/tests/read.rs` +**Estimated scope:** Medium + +## Task 6: Browsing and extracting FAT images + +**Description:** Connect raw FAT floppy and partition images to the existing image-info, browse, and extraction surfaces. + +**Acceptance criteria:** +- [ ] CLI image-info and extraction can open supported raw FAT images. +- [ ] Existing GUI browse/extract flow handles FAT paths with the same progress and cancellation behavior. +- [ ] ISO/Joliet/UDF selection and behavior remain unchanged. + +**Verification:** +- [ ] Add CLI/integration coverage for FAT images and readback after extraction. +- [ ] Build app and run all `if-image` integration tests. + +**Dependencies:** Task 5 +**Files likely touched:** `crates/if-image/src/container.rs`, `crates/if-image/src/fs.rs`, `crates/isofoundry/src/cli_image.rs`, `crates/isofoundry/src/win/tasks/browse.rs` +**Estimated scope:** Medium + +## Task 7: Editing FAT images and saving safely + +**Description:** Extend edit-image operations to supported FAT volumes and rebuild to a separate output image while preserving boot/partition metadata that the implementation can safely retain. + +**Acceptance criteria:** +- [ ] Add, remove, rename, create-folder, and label operations work for FAT images. +- [ ] The source is never modified; existing-destination and source/destination identity rules are enforced. +- [ ] Unsupported boot sectors, partitions, or metadata are reported rather than silently discarded. + +**Verification:** +- [ ] Add edit round-trip tests for all supported FAT types, fragmentation, collisions, cancellation, and source preservation. +- [ ] Exercise the existing CLI and GUI edit paths against raw FAT fixtures. + +**Dependencies:** Task 5, Task 6 +**Files likely touched:** `crates/if-image/src/fat_write.rs`, `crates/if-image/src/edit.rs`, `crates/isofoundry/src/cli_image.rs`, `crates/isofoundry/src/win/tasks/edit.rs`, focused tests +**Estimated scope:** Medium + +## Task 8: Reading and writing fixed/dynamic VHD containers + +**Description:** Add a container adapter for legacy VHD fixed and dynamic disks, validating footer/header checksums, BAT entries, block bitmaps, and virtual-size bounds. + +**Acceptance criteria:** +- [ ] Fixed and dynamic VHD fixtures expose logical sector reads correctly. +- [ ] Invalid checksums, overlapping/out-of-range blocks, and malformed BATs fail cleanly. +- [ ] Writes produce a new VHD file and never mutate the input. + +**Verification:** +- [ ] Add fixed/dynamic read-write round-trip tests plus malformed-container tests. +- [ ] Compare generated fixtures with an independent VHD parser/tool where available. + +**Dependencies:** Task 1 +**Files likely touched:** `crates/if-image/src/vhd.rs`, `crates/if-image/src/lib.rs`, tests +**Estimated scope:** Medium + +## Task 9: Integrating VHD-contained FAT editing + +**Description:** Open VHD containers through the FAT image path and save edited virtual disks while retaining container type and safe metadata. + +**Acceptance criteria:** +- [ ] Fixed and dynamic VHD images containing supported FAT volumes browse, extract, and edit through app workflows. +- [ ] Successful edits reopen in IsoFoundry and an independent VHD reader. +- [ ] All failures preserve both source and any pre-existing destination. + +**Verification:** +- [ ] Add end-to-end VHD/FAT edit tests, including malformed image and cancel cases. +- [ ] Run workspace tests and app build. + +**Dependencies:** Task 7, Task 8 +**Files likely touched:** `crates/if-image/src/container.rs`, `crates/if-image/src/edit.rs`, CLI/GUI adapters, tests +**Estimated scope:** Medium + +## Task 10: Proving the multiboot boot chain and QEMU verification path + +**Description:** Select bootloader assets and define the minimal supported boot profiles, asset licensing/provenance, BIOS/UEFI layout, and reproducible QEMU software-test matrix before any destructive writer changes. + +**Acceptance criteria:** +- [ ] Selected boot assets have documented origin, version, license, and required notices. +- [ ] Linux ISO loopback and Windows/WinPE ISO/WIM boot approaches are demonstrated in QEMU or bounded as unsupported. +- [ ] Secure Boot claims are backed by the actual signed chain; otherwise the limitation is explicit. + +**Verification:** +- [ ] Build the boot layout in a disposable disk-image fixture and boot supported profiles in QEMU. +- [ ] Review dependency/license changes before integrating assets. + +**Dependencies:** Task 1 +**Files likely touched:** `docs/roadmap.md`, `docs/references.md`, test scripts/fixtures, possible `CREDITS.md` +**Estimated scope:** Medium + +## Task 11: Installing a multiboot layout through the guarded USB engine + +**Description:** Add an install path that uses existing target assessment, helper IPC, confirmation, and development image-target gates to partition/format a multiboot device and place boot assets. + +**Acceptance criteria:** +- [ ] Target safety checks apply to every GUI, CLI, and helper write route. +- [ ] Disk-image test targets require the existing explicit development flag. +- [ ] Install cancellation/errors do not report success and leave the target state diagnosable. + +**Verification:** +- [ ] Add file-backed device tests for partitioning, formatting, asset placement, cancellation, and safety denials. +- [ ] Run helper protocol and USB writer suites; no physical disk is used. + +**Dependencies:** Task 10 +**Files likely touched:** `crates/usbwriter/src/multiboot.rs`, `crates/usbwriter/src/lib.rs`, CLI/helper route files, tests +**Estimated scope:** Medium + +## Task 12: Managing ISO/WIM payloads and persistent boot options + +**Description:** Add validated add/remove/list operations for supported ISO/WIM payloads and a versioned per-entry configuration that survives content updates. + +**Acceptance criteria:** +- [ ] Multiple payloads are added and removed without rewriting unrelated payload data. +- [ ] Names/paths are safe and collisions are handled explicitly. +- [ ] Per-entry boot options persist across refresh/install updates and invalid config fails visibly. + +**Verification:** +- [ ] Add config round-trip, path-safety, collision, and payload manifest tests. +- [ ] Boot at least one Linux ISO and one supported Windows/WinPE path under QEMU. + +**Dependencies:** Task 10, Task 11 +**Files likely touched:** `crates/usbwriter/src/multiboot.rs`, `crates/if-core/src/settings.rs` or a dedicated config module, tests +**Estimated scope:** Medium + +## Task 13: Integrating multiboot management in the app and CLI + +**Description:** Expose multiboot install/manage operations through a dedicated task window and documented CLI commands using existing progress, cancellation, confirmation, and helper patterns. + +**Acceptance criteria:** +- [ ] Users can inspect a target, install/update the boot menu, add/remove payloads, and review per-entry options. +- [ ] CLI supports scripted list/add/remove/update flows with stable exit behavior. +- [ ] Dangerous operations display the target identity and require the established confirmation flow. + +**Verification:** +- [ ] Add CLI parser/behavior tests and GUI adapter tests where the repository supports them. +- [ ] Build the Windows app/helper and exercise complete workflows against a disposable disk image. + +**Dependencies:** Task 11, Task 12 +**Files likely touched:** `crates/isofoundry/src/cli_usb.rs`, `crates/isofoundry/src/win/tasks`, `crates/isofoundry/src/win/gui.rs`, help/docs +**Estimated scope:** Medium + +## Task 14: Verifying and documenting multiboot scenarios + +**Description:** Complete QEMU scenarios for supported BIOS/UEFI combinations and document compatibility boundaries, Secure Boot behavior, and recovery/update semantics. + +**Acceptance criteria:** +- [ ] Each claimed boot profile has an automated or repeatable QEMU test. +- [ ] Unsupported ISO/WIM layouts and Secure Boot cases are explicitly described. +- [ ] Physical hardware remains labeled unverified and is not required for completion. + +**Verification:** +- [ ] Run the documented QEMU matrix and USB file-target suite. +- [ ] Review README, LIMITATIONS, TESTING, and roadmap statements against results. + +**Dependencies:** Task 13 +**Files likely touched:** `docs/TESTING.md`, `docs/LIMITATIONS.md`, `docs/roadmap.md`, `README.md`, `CHANGELOG.md` +**Estimated scope:** Medium + +## Task 15: Adding locale resources and language selection + +**Description:** Add a resource-based string lookup with English and Spanish catalogs, English fallback, and an explicit language setting integrated with existing settings. + +**Acceptance criteria:** +- [ ] GUI strings resolve through a stable typed/keyed lookup instead of per-window literals. +- [ ] Users can choose English or Spanish; invalid/missing keys fall back to English and are detectable in tests. +- [ ] Default behavior remains English unless existing product language-selection conventions establish a locale-based default. + +**Verification:** +- [ ] Add locale selection, fallback, settings migration, and missing-key tests. +- [ ] Confirm existing task/window strings remain English before the translation sweep. + +**Dependencies:** Tasks 3, 6, 9, 13 +**Files likely touched:** `crates/if-core/src/strings.rs`, settings, resource files, `Cargo.toml`/`Cargo.lock` if needed +**Estimated scope:** Medium + +## Task 16: Localizing shared menus, task catalog, and core dialogs + +**Description:** Move the main window, task catalog, menu, common control, and shared confirmation/error strings to the resource catalogs and provide Spanish translations. + +**Acceptance criteria:** +- [ ] Main window, task grid, menus, shared dialogs and common status text are localized. +- [ ] Accessibility names/tooltips are localized along with visible labels. +- [ ] English and Spanish catalogs have matching keys. + +**Verification:** +- [ ] Add catalog parity tests and exercise menu/task selection in both locales. +- [ ] Build the Windows app and inspect key dialog text through UI automation or accessibility tree. + +**Dependencies:** Task 15 +**Files likely touched:** `crates/if-core/src/strings.rs`, main GUI, task catalog, resource files, tests +**Estimated scope:** Medium + +## Task 17: Localizing task windows in bounded groups + +**Description:** Migrate all task-window labels, prompts, progress/status messages, errors, and confirmation text to keyed resources in small batches. + +**Acceptance criteria:** +- [ ] Every existing and newly added task window has Spanish-visible user-facing text. +- [ ] Technical identifiers, paths, and media names remain intact and safely formatted. +- [ ] No task window silently falls back due to a missing Spanish key. + +**Verification:** +- [ ] Migrate and test no more than a few windows per change batch. +- [ ] Run string coverage tests after each batch; finish with both-locale workflow smoke tests. + +**Dependencies:** Task 16 +**Files likely touched:** task-window modules in groups, resource catalogs, focused tests +**Estimated scope:** Medium per batch + +## Task 18: Enforcing localization completeness and final acceptance + +**Description:** Add a repository test that enumerates all GUI string keys and ensures Spanish coverage, then align release docs and perform a final full validation. + +**Acceptance criteria:** +- [ ] All catalog keys required by GUI code are present in both locales. +- [ ] README, roadmap, limitations, changelog, and compatibility statements reflect actual support and test evidence. +- [ ] No physical-hardware verification is claimed. + +**Verification:** +- [ ] Run `cargo fmt --check`, `cargo test --workspace`, `cargo clippy --workspace --all-targets -- -D warnings`, and `cargo build -p isofoundry`. +- [ ] Run the QEMU matrix and all software-only format/USB integration tests. + +**Dependencies:** Tasks 4, 9, 14, 17 +**Files likely touched:** localization tests, README/docs, changelog, relevant feature tests +**Estimated scope:** Medium From c1402c74552aecb7f2f45996b8d863d9b16c4ef2 Mon Sep 17 00:00:00 2001 From: Sean Shaffer Date: Sun, 8 Nov 2026 22:43:58 -0600 Subject: [PATCH 02/12] feat: read XDVDFS image files Add bounded XDVDFS probing, directory listing, and file extent reads so Xbox images open through the shared image filesystem API. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- crates/if-image/src/fs.rs | 13 ++- crates/if-image/src/lib.rs | 1 + crates/if-image/src/xiso.rs | 157 ++++++++++++++++++++++++++++++++++ crates/if-image/tests/xiso.rs | 112 ++++++++++++++++++++++++ 4 files changed, 280 insertions(+), 3 deletions(-) create mode 100644 crates/if-image/src/xiso.rs create mode 100644 crates/if-image/tests/xiso.rs diff --git a/crates/if-image/src/fs.rs b/crates/if-image/src/fs.rs index e1dd3a9..208e1a9 100644 --- a/crates/if-image/src/fs.rs +++ b/crates/if-image/src/fs.rs @@ -33,6 +33,7 @@ pub struct Entry { #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] pub enum FsKind { + Xdvdfs, Iso9660, Joliet, Udf, @@ -41,6 +42,7 @@ pub enum FsKind { impl FsKind { pub fn name(self) -> &'static str { match self { + FsKind::Xdvdfs => "XDVDFS", FsKind::Iso9660 => "ISO 9660", FsKind::Joliet => "Joliet", FsKind::Udf => "UDF", @@ -137,14 +139,18 @@ pub struct Volume { } impl Volume { - /// Opens the best file system on `src`: UDF (large files, Unicode), - /// then Joliet, then plain ISO 9660. `prefer` picks one explicitly. + /// Opens the best file system on `src`: XDVDFS, UDF (large files, + /// Unicode), Joliet, then plain ISO 9660. `prefer` picks one explicitly. pub fn open(mut src: Box, prefer: Option) -> Result { + let xdvdfs = crate::xiso::Xdvdfs::probe(src.as_mut())?; let iso = crate::iso9660::scan(src.as_mut()).ok(); let udf_result = crate::udf::Udf::open(src.as_mut()); let udf_error = udf_result.as_ref().err().map(|e| e.to_string()).filter(|m| m != "no UDF anchor"); let udf = udf_result.ok(); let mut found = Vec::new(); + if xdvdfs.is_some() { + found.push(FsKind::Xdvdfs); + } if udf.is_some() { found.push(FsKind::Udf); } @@ -159,7 +165,7 @@ impl Volume { if found.is_empty() { return match udf_error { Some(m) => bad(format!("UDF file system found but not readable: {m}")), - None => bad("no ISO 9660, Joliet or UDF file system found"), + None => bad("no XDVDFS, ISO 9660, Joliet or UDF file system found"), }; } let want = prefer.unwrap_or(found[0]); @@ -169,6 +175,7 @@ impl Volume { let boot = iso.as_ref().and_then(|s| s.boot_catalog).and_then(|lba| crate::eltorito::read(src.as_mut(), lba).ok()); let fs: Box = match want { + FsKind::Xdvdfs => Box::new(xdvdfs.expect("checked")), FsKind::Udf => Box::new(udf.expect("checked")), FsKind::Joliet => Box::new(iso.and_then(|s| s.joliet).expect("checked")), FsKind::Iso9660 => Box::new(iso.and_then(|s| s.primary).expect("checked")), diff --git a/crates/if-image/src/lib.rs b/crates/if-image/src/lib.rs index f3f613f..651f7f6 100644 --- a/crates/if-image/src/lib.rs +++ b/crates/if-image/src/lib.rs @@ -25,6 +25,7 @@ pub mod nrg; pub mod source; pub mod udf; pub mod write; +pub mod xiso; pub const SECTOR: usize = 2048; diff --git a/crates/if-image/src/xiso.rs b/crates/if-image/src/xiso.rs new file mode 100644 index 0000000..8c354a5 --- /dev/null +++ b/crates/if-image/src/xiso.rs @@ -0,0 +1,157 @@ +//! Read-only XDVDFS filesystem support for Xbox XISO images. + +use crate::fs::{read_data, Data, Entry, Extent, FileSystem, FsKind}; +use crate::source::SectorSource; +use crate::{bad, Result, SECTOR}; +use std::collections::HashSet; + +const DESCRIPTOR_SECTOR: u64 = 32; +const SIGNATURE: &[u8; 20] = b"MICROSOFT*XBOX*MEDIA"; +const ROOT_SECTOR_OFFSET: usize = 20; +const ROOT_SIZE_OFFSET: usize = 24; +const SECOND_SIGNATURE_OFFSET: usize = 0x7ec; +const ATTRIBUTE_DIRECTORY: u8 = 0x10; +const MAX_DIRECTORY_BYTES: u64 = 64 << 20; +const MAX_DIRECTORY_ENTRIES: usize = 100_000; + +pub struct Xdvdfs { + root: Entry, +} + +impl Xdvdfs { + /// Returns `None` when the descriptor signature is absent; a recognized + /// but malformed descriptor is an error rather than a silent fallback. + pub fn probe(src: &mut dyn SectorSource) -> Result> { + if src.sectors() <= DESCRIPTOR_SECTOR { + return Ok(None); + } + + let mut header = [0; SECTOR]; + src.read_sectors(DESCRIPTOR_SECTOR, &mut header)?; + if &header[..SIGNATURE.len()] != SIGNATURE { + return Ok(None); + } + if &header[SECOND_SIGNATURE_OFFSET..SECOND_SIGNATURE_OFFSET + SIGNATURE.len()] != SIGNATURE { + return bad("invalid XDVDFS volume descriptor signature"); + } + + let root_sector = u32::from_le_bytes(header[ROOT_SECTOR_OFFSET..ROOT_SECTOR_OFFSET + 4].try_into().unwrap()); + let root_size = u32::from_le_bytes(header[ROOT_SIZE_OFFSET..ROOT_SIZE_OFFSET + 4].try_into().unwrap()); + validate_extent(src, root_sector, root_size, true)?; + + Ok(Some(Self { + root: Entry { + name: String::new(), + dir: true, + size: u64::from(root_size), + mtime: None, + hidden: false, + data: Data::Extents(vec![Extent { + lba: u64::from(root_sector), + len: u64::from(root_size), + sparse: false, + }]), + }, + })) + } + + fn list_directory(&self, src: &mut dyn SectorSource, dir: &Entry) -> Result> { + if !dir.dir { + return bad(format!("{} is not a folder", dir.name)); + } + if dir.size > MAX_DIRECTORY_BYTES { + return bad("XDVDFS directory is larger than 64 MB"); + } + let bytes_len = + usize::try_from(dir.size).map_err(|_| crate::Error::Format("XDVDFS directory is too large".into()))?; + let mut bytes = vec![0; bytes_len]; + read_data(src, &dir.data, dir.size, 0, &mut bytes)?; + + let mut entries = Vec::new(); + let mut names = HashSet::new(); + let mut offset = 0usize; + while offset + 2 <= bytes.len() { + if bytes[offset] == 0xff && bytes[offset + 1] == 0xff { + offset = (offset / SECTOR + 1) * SECTOR; + continue; + } + if offset + 14 > bytes.len() { + return bad("truncated XDVDFS directory entry"); + } + + let file_sector = u32::from_le_bytes(bytes[offset + 4..offset + 8].try_into().unwrap()); + let file_size = u32::from_le_bytes(bytes[offset + 8..offset + 12].try_into().unwrap()); + let attributes = bytes[offset + 12]; + let name_len = usize::from(bytes[offset + 13]); + let name_end = offset + .checked_add(14 + name_len) + .ok_or_else(|| crate::Error::Format("XDVDFS directory entry size overflow".into()))?; + if name_len == 0 || name_end > bytes.len() { + return bad("invalid XDVDFS filename length"); + } + let name = String::from_utf8_lossy(&bytes[offset + 14..name_end]).into_owned(); + if name == "." || name == ".." || name.contains(['/', '\\']) || name.chars().any(char::is_control) { + return bad(format!("invalid XDVDFS filename: {name:?}")); + } + if !names.insert(name.to_lowercase()) { + return bad(format!("duplicate XDVDFS filename: {name}")); + } + if entries.len() == MAX_DIRECTORY_ENTRIES { + return bad("XDVDFS directory contains too many entries"); + } + + let is_dir = attributes & ATTRIBUTE_DIRECTORY != 0; + validate_extent(src, file_sector, file_size, is_dir)?; + entries.push(Entry { + name, + dir: is_dir, + size: u64::from(file_size), + mtime: None, + hidden: attributes & 0x02 != 0, + data: Data::Extents(vec![Extent { + lba: u64::from(file_sector), + len: u64::from(file_size), + sparse: false, + }]), + }); + + let record_len = (14 + name_len + 3) & !3; + offset = offset + .checked_add(record_len) + .ok_or_else(|| crate::Error::Format("XDVDFS directory offset overflow".into()))?; + } + + if offset < bytes.len() && bytes[offset..].iter().any(|b| *b != 0xff) { + return bad("invalid trailing bytes in XDVDFS directory"); + } + Ok(entries) + } +} + +fn validate_extent(src: &dyn SectorSource, sector: u32, size: u32, directory: bool) -> Result<()> { + let start = u64::from(sector); + let sectors = u64::from(size).div_ceil(SECTOR as u64); + let end = start.checked_add(sectors).ok_or_else(|| crate::Error::Format("XDVDFS extent overflow".into()))?; + if end > src.sectors() || (directory && u64::from(size) > MAX_DIRECTORY_BYTES) { + return bad("XDVDFS extent is outside the image"); + } + Ok(()) +} + +impl FileSystem for Xdvdfs { + fn kind(&self) -> FsKind { + FsKind::Xdvdfs + } + + fn label(&self) -> String { + String::new() + } + + fn root(&self) -> Entry { + self.root.clone() + } + + fn list(&self, src: &mut dyn SectorSource, dir: &Entry) -> Result> { + self.list_directory(src, dir) + } +} diff --git a/crates/if-image/tests/xiso.rs b/crates/if-image/tests/xiso.rs new file mode 100644 index 0000000..9bd8c15 --- /dev/null +++ b/crates/if-image/tests/xiso.rs @@ -0,0 +1,112 @@ +use if_image::fs::{FsKind, Volume}; +use if_image::source::MemSource; + +const SECTOR: usize = if_image::SECTOR; + +fn xiso_with_one_file() -> MemSource { + let mut image = vec![0u8; 64 * SECTOR]; + let header = 32 * SECTOR; + image[header..header + 20].copy_from_slice(b"MICROSOFT*XBOX*MEDIA"); + image[header + 20..header + 24].copy_from_slice(&40u32.to_le_bytes()); + image[header + 24..header + 28].copy_from_slice(&(SECTOR as u32).to_le_bytes()); + image[header + 0x7ec..header + 0x7ec + 20].copy_from_slice(b"MICROSOFT*XBOX*MEDIA"); + + let root = 40 * SECTOR; + image[root..root + SECTOR].fill(0xff); + image[root..root + 4].fill(0); + image[root + 4..root + 8].copy_from_slice(&41u32.to_le_bytes()); + image[root + 8..root + 12].copy_from_slice(&5u32.to_le_bytes()); + image[root + 12] = 0x20; + image[root + 13] = 9; + image[root + 14..root + 23].copy_from_slice(b"hello.txt"); + image[41 * SECTOR..41 * SECTOR + 5].copy_from_slice(b"hello"); + MemSource(image) +} + +#[test] +fn opens_xdvdfs_and_reads_file_extents() { + let mut volume = Volume::open(Box::new(xiso_with_one_file()), None).unwrap(); + + assert_eq!(volume.fs.kind(), FsKind::Xdvdfs); + assert_eq!(volume.found, vec![FsKind::Xdvdfs]); + let file = volume.lookup("hello.txt").unwrap(); + assert_eq!(file.size, 5); + + let mut contents = [0; 5]; + volume.read(&file, 0, &mut contents).unwrap(); + assert_eq!(&contents, b"hello"); +} + +#[test] +fn rejects_xdvdfs_root_extent_outside_image() { + let mut source = xiso_with_one_file(); + let root_sector_offset = 32 * SECTOR + 20; + source.0[root_sector_offset..root_sector_offset + 4].copy_from_slice(&u32::MAX.to_le_bytes()); + + assert!(Volume::open(Box::new(source), None).is_err()); +} + +#[test] +fn rejects_invalid_xdvdfs_filename() { + let mut source = xiso_with_one_file(); + let filename_offset = 40 * SECTOR + 14; + source.0[filename_offset] = b'/'; + let mut volume = Volume::open(Box::new(source), None).unwrap(); + + assert!(volume.list(&volume.root()).is_err()); +} + +#[test] +fn rejects_invalid_xdvdfs_descriptor_signature() { + let mut source = xiso_with_one_file(); + source.0[32 * SECTOR + 0x7ec] = b'X'; + + assert!(Volume::open(Box::new(source), None).is_err()); +} + +#[test] +fn reads_files_in_nested_xdvdfs_directories() { + let mut source = xiso_with_one_file(); + let root = 40 * SECTOR; + source.0[root + 17..root + SECTOR].fill(0xff); + source.0[root + 4..root + 8].copy_from_slice(&42u32.to_le_bytes()); + source.0[root + 8..root + 12].copy_from_slice(&(SECTOR as u32).to_le_bytes()); + source.0[root + 12] = 0x10; + source.0[root + 13] = 3; + source.0[root + 14..root + 17].copy_from_slice(b"sub"); + + let child = 42 * SECTOR; + source.0[child..child + SECTOR].fill(0xff); + source.0[child..child + 4].fill(0); + source.0[child + 4..child + 8].copy_from_slice(&43u32.to_le_bytes()); + source.0[child + 8..child + 12].copy_from_slice(&5u32.to_le_bytes()); + source.0[child + 12] = 0x20; + source.0[child + 13] = 9; + source.0[child + 14..child + 23].copy_from_slice(b"child.txt"); + source.0[43 * SECTOR..43 * SECTOR + 5].copy_from_slice(b"child"); + + let mut volume = Volume::open(Box::new(source), None).unwrap(); + let file = volume.lookup("sub/child.txt").unwrap(); + let mut contents = [0; 5]; + + volume.read(&file, 0, &mut contents).unwrap(); + + assert_eq!(&contents, b"child"); +} + +#[test] +fn lists_empty_xdvdfs_directory() { + let mut source = xiso_with_one_file(); + let root = 40 * SECTOR; + source.0[root + 17..root + SECTOR].fill(0xff); + source.0[root + 4..root + 8].copy_from_slice(&42u32.to_le_bytes()); + source.0[root + 8..root + 12].copy_from_slice(&0u32.to_le_bytes()); + source.0[root + 12] = 0x10; + source.0[root + 13] = 3; + source.0[root + 14..root + 17].copy_from_slice(b"sub"); + + let mut volume = Volume::open(Box::new(source), None).unwrap(); + let directory = volume.lookup("sub").unwrap(); + + assert!(volume.list(&directory).unwrap().is_empty()); +} From 2c1e7942e23bb3eec6364bc9f75ef93bc2b97eef Mon Sep 17 00:00:00 2001 From: Sean Shaffer Date: Sun, 8 Nov 2026 22:46:16 -0600 Subject: [PATCH 03/12] fix: keep XDVDFS images read-only Reject image-edit operations that would silently rebuild an Xbox filesystem as ISO 9660, and record the validated reader milestone. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- crates/if-image/src/edit.rs | 3 +++ crates/if-image/tests/xiso.rs | 45 +++++++++++++++++++++++++++++++++++ tasks/todo.md | 14 +++++------ 3 files changed, 55 insertions(+), 7 deletions(-) diff --git a/crates/if-image/src/edit.rs b/crates/if-image/src/edit.rs index 48d5a6d..e5a9603 100644 --- a/crates/if-image/src/edit.rs +++ b/crates/if-image/src/edit.rs @@ -62,6 +62,9 @@ impl Project { pub fn open(image: &Path) -> Result { let (_, src) = container::open(image)?; let mut v = Volume::open(src, None)?; + if v.fs.kind() == FsKind::Xdvdfs { + return bad("XDVDFS images are read-only and cannot be edited"); + } let mut warnings = Vec::new(); let label = v.fs.label(); let joliet = v.found.contains(&FsKind::Joliet) || v.found.contains(&FsKind::Udf); diff --git a/crates/if-image/tests/xiso.rs b/crates/if-image/tests/xiso.rs index 9bd8c15..29644e6 100644 --- a/crates/if-image/tests/xiso.rs +++ b/crates/if-image/tests/xiso.rs @@ -46,6 +46,15 @@ fn rejects_xdvdfs_root_extent_outside_image() { assert!(Volume::open(Box::new(source), None).is_err()); } +#[test] +fn rejects_unreasonable_xdvdfs_directory_size() { + let mut source = xiso_with_one_file(); + let root_size_offset = 32 * SECTOR + 24; + source.0[root_size_offset..root_size_offset + 4].copy_from_slice(&(64 * 1024 * 1024u32 + 1).to_le_bytes()); + + assert!(Volume::open(Box::new(source), None).is_err()); +} + #[test] fn rejects_invalid_xdvdfs_filename() { let mut source = xiso_with_one_file(); @@ -110,3 +119,39 @@ fn lists_empty_xdvdfs_directory() { assert!(volume.list(&directory).unwrap().is_empty()); } + +#[test] +fn bounds_recursive_walks_through_cyclic_xdvdfs_directories() { + let mut source = xiso_with_one_file(); + let root = 40 * SECTOR; + source.0[root + 17..root + SECTOR].fill(0xff); + source.0[root + 4..root + 8].copy_from_slice(&42u32.to_le_bytes()); + source.0[root + 8..root + 12].copy_from_slice(&(SECTOR as u32).to_le_bytes()); + source.0[root + 12] = 0x10; + source.0[root + 13] = 3; + source.0[root + 14..root + 17].copy_from_slice(b"sub"); + + let child = 42 * SECTOR; + source.0[child..child + SECTOR].fill(0xff); + source.0[child..child + 4].fill(0); + source.0[child + 4..child + 8].copy_from_slice(&40u32.to_le_bytes()); + source.0[child + 8..child + 12].copy_from_slice(&(SECTOR as u32).to_le_bytes()); + source.0[child + 12] = 0x10; + source.0[child + 13] = 4; + source.0[child + 14..child + 18].copy_from_slice(b"back"); + + let mut volume = Volume::open(Box::new(source), None).unwrap(); + let root = volume.root(); + + assert!(volume.walk(&root).is_err()); +} + +#[test] +fn refuses_to_edit_xdvdfs_images() { + let path = std::env::temp_dir().join(format!("if-xiso-edit-{}.iso", std::process::id())); + std::fs::write(&path, xiso_with_one_file().0).unwrap(); + let result = if_image::edit::Project::open(&path); + std::fs::remove_file(path).unwrap(); + + assert!(matches!(result, Err(if_image::Error::Format(message)) if message.contains("read-only"))); +} diff --git a/tasks/todo.md b/tasks/todo.md index 8c2aece..c7fb26a 100644 --- a/tasks/todo.md +++ b/tasks/todo.md @@ -17,24 +17,24 @@ **Files likely touched:** `docs/roadmap.md`, `docs/references.md` (only after evidence is established) **Estimated scope:** Small -## Task 2: Reading and validating XDVDFS volumes — In progress +## Task 2: Reading and validating XDVDFS volumes — Done **Description:** Add a pure-Rust XDVDFS reader behind the image engine's filesystem/source abstractions, including bounded directory traversal and file extent reads. **Acceptance criteria:** -- [ ] Valid XISO fixtures expose volume metadata, directory entries, and file contents. -- [ ] Cycles, invalid extents, truncated images, and unreasonable directory sizes return errors without panic or unbounded allocation. -- [ ] Filesystem name/label and read-only status are exposed to callers. +- [x] Valid XISO fixtures expose filesystem identity, directory entries, and file contents. +- [x] Invalid extents, truncated images, and unreasonable directory sizes are rejected before unbounded reads or allocation; recursive walking is bounded by the shared filesystem depth limit. +- [x] Filesystem name/label are available through the shared filesystem API, and editing XDVDFS is explicitly rejected as read-only. **Verification:** -- [ ] Add unit tests for valid, nested, empty, and malformed XDVDFS fixtures. -- [ ] Run the focused `if-image` tests. +- [x] Add integration tests for valid, nested, empty, and malformed XDVDFS fixtures. +- [x] Run focused and complete `if-image` test suites, formatting, and Clippy. **Dependencies:** Task 1 **Files likely touched:** `crates/if-image/src/xiso.rs`, `crates/if-image/src/fs.rs`, `crates/if-image/src/lib.rs`, `crates/if-image/tests/read.rs` **Estimated scope:** Medium -## Task 3: Exposing XISO browse and extraction in CLI and GUI +## Task 3: Exposing XISO browse and extraction in CLI and GUI — In progress **Description:** Route XISO images through image-info, browse, and extract flows while keeping XISO-specific metadata visible and preserving existing formats. From a63d7b4621ba3c86332eaa4b4721465f3c2a4763 Mon Sep 17 00:00:00 2001 From: Sean Shaffer Date: Sun, 8 Nov 2026 22:48:31 -0600 Subject: [PATCH 04/12] feat: expose XISO browsing and extraction Teach image-info and extract to select XDVDFS, and record the shared browse/extract path verified by the app build. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- crates/isofoundry/src/cli_image.rs | 63 +++++++++++++++++++++++++++++- tasks/todo.md | 14 +++---- 2 files changed, 68 insertions(+), 9 deletions(-) diff --git a/crates/isofoundry/src/cli_image.rs b/crates/isofoundry/src/cli_image.rs index c37d8ba..08924c8 100644 --- a/crates/isofoundry/src/cli_image.rs +++ b/crates/isofoundry/src/cli_image.rs @@ -19,7 +19,7 @@ pub const USAGE: &str = " create-image SRC... --out FILE.iso [--label L] [--fs Convert ISO, BIN/CUE, CCD, NRG, MDS, CHD (format from --to or extension)\n\ edit-image IMAGE --out NEW [--add DIR=PATH] [--remove P] [--rename P=NAME]\n\ [--mkdir P] [--label L] Change an image and save it as a new file (options repeat, run in order)\n\ - extract IMAGE --out DIR [--path P] [--fs udf|joliet|iso9660]\n\ + extract IMAGE --out DIR [--path P] [--fs udf|joliet|iso9660|xdvdfs]\n\ Copy files out of an image (all, or the file/folder P)\n\ checksum FILE [--algo crc32,md5,sha1,sha256] [--dat OUT.dat]\n\ Hash a file (default: all four); optionally write a .dat\n\ @@ -77,7 +77,8 @@ fn fs_kind(s: &str) -> Result { "udf" => Ok(FsKind::Udf), "joliet" => Ok(FsKind::Joliet), "iso9660" | "iso" => Ok(FsKind::Iso9660), - o => Err(format!("unknown file system {o} (udf, joliet, iso9660)")), + "xdvdfs" | "xiso" => Ok(FsKind::Xdvdfs), + o => Err(format!("unknown file system {o} (udf, joliet, iso9660, xdvdfs)")), } } @@ -574,6 +575,40 @@ mod tests { p } + fn xiso_fixture() -> PathBuf { + let d = tmp("xiso"); + std::fs::create_dir_all(&d).unwrap(); + let mut image = vec![0u8; 64 * if_image::SECTOR]; + let header = 32 * if_image::SECTOR; + image[header..header + 20].copy_from_slice(b"MICROSOFT*XBOX*MEDIA"); + image[header + 20..header + 24].copy_from_slice(&40u32.to_le_bytes()); + image[header + 24..header + 28].copy_from_slice(&(if_image::SECTOR as u32).to_le_bytes()); + image[header + 0x7ec..header + 0x7ec + 20].copy_from_slice(b"MICROSOFT*XBOX*MEDIA"); + + let root = 40 * if_image::SECTOR; + image[root..root + if_image::SECTOR].fill(0xff); + image[root..root + 4].fill(0); + image[root + 4..root + 8].copy_from_slice(&42u32.to_le_bytes()); + image[root + 8..root + 12].copy_from_slice(&(if_image::SECTOR as u32).to_le_bytes()); + image[root + 12] = 0x10; + image[root + 13] = 3; + image[root + 14..root + 17].copy_from_slice(b"sub"); + + let child = 42 * if_image::SECTOR; + image[child..child + if_image::SECTOR].fill(0xff); + image[child..child + 4].fill(0); + image[child + 4..child + 8].copy_from_slice(&43u32.to_le_bytes()); + image[child + 8..child + 12].copy_from_slice(&5u32.to_le_bytes()); + image[child + 12] = 0x20; + image[child + 13] = 9; + image[child + 14..child + 23].copy_from_slice(b"hello.txt"); + image[43 * if_image::SECTOR..43 * if_image::SECTOR + 5].copy_from_slice(b"hello"); + + let path = d.join("game.iso"); + std::fs::write(&path, image).unwrap(); + path + } + #[test] fn info_json() { let (code, out, _) = call(&["image-info", &fixture("boot.iso"), "--json"]); @@ -598,6 +633,30 @@ mod tests { assert_eq!(code, 11, "{err}"); } + #[test] + fn image_info_and_extract_support_xiso() { + let image = xiso_fixture(); + let (code, out, err) = call(&["image-info", image.to_str().unwrap(), "--json"]); + assert_eq!(code, 0, "{err}"); + let info: serde_json::Value = serde_json::from_str(out.trim()).unwrap(); + assert_eq!(info["file_systems"], json!(["XDVDFS"])); + + let dest = image.parent().unwrap().join("extracted"); + let (code, _, err) = call(&[ + "extract", + image.to_str().unwrap(), + "--out", + dest.to_str().unwrap(), + "--path", + "sub", + "--fs", + "xdvdfs", + ]); + assert_eq!(code, 0, "{err}"); + assert_eq!(std::fs::read(dest.join("hello.txt")).unwrap(), b"hello"); + std::fs::remove_dir_all(image.parent().unwrap()).unwrap(); + } + #[test] fn checksum_dat_and_verify() { let d = tmp("dat"); diff --git a/tasks/todo.md b/tasks/todo.md index c7fb26a..1d45369 100644 --- a/tasks/todo.md +++ b/tasks/todo.md @@ -34,24 +34,24 @@ **Files likely touched:** `crates/if-image/src/xiso.rs`, `crates/if-image/src/fs.rs`, `crates/if-image/src/lib.rs`, `crates/if-image/tests/read.rs` **Estimated scope:** Medium -## Task 3: Exposing XISO browse and extraction in CLI and GUI — In progress +## Task 3: Exposing XISO browse and extraction in CLI and GUI — Done **Description:** Route XISO images through image-info, browse, and extract flows while keeping XISO-specific metadata visible and preserving existing formats. **Acceptance criteria:** -- [ ] `image-info` recognizes XISO and reports its filesystem. -- [ ] CLI extraction and the existing browse/extract task window can read XISO files. -- [ ] Unsupported operations fail with an explicit format-specific message. +- [x] `image-info` recognizes XISO and reports its filesystem. +- [x] CLI extraction and the existing browse/extract task window use the shared XDVDFS reader. +- [x] Unsupported image-edit operations fail with an explicit read-only message. **Verification:** -- [ ] Add CLI and integration tests that browse and extract nested XISO fixtures. -- [ ] Build the app and check GUI routing through existing image-task patterns. +- [x] Add CLI and integration tests that browse and extract nested XISO fixtures. +- [x] Build the app and verify that the existing GUI browse/extract task routes through the shared image API. **Dependencies:** Task 2 **Files likely touched:** `crates/isofoundry/src/cli_image.rs`, `crates/isofoundry/src/win/tasks/browse.rs`, `crates/if-image/tests/read.rs` **Estimated scope:** Medium -## Task 4: Creating XISO images and end-to-end coverage +## Task 4: Creating XISO images and end-to-end coverage — In progress **Description:** Add an XDVDFS writer and an explicit XISO create path that produces compatible images without touching game executables or protected content. From 38f40e32f405971205682704fe684b38d897142c Mon Sep 17 00:00:00 2001 From: Sean Shaffer Date: Sun, 8 Nov 2026 23:04:07 -0600 Subject: [PATCH 05/12] feat: add XISO creation support Add XDVDFS image creation and CLI routing with round-trip coverage, while keeping existing XISO edits explicitly read-only. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- CHANGELOG.md | 4 + README.md | 10 +- crates/if-image/src/write/mod.rs | 3 + crates/if-image/src/write/xiso.rs | 337 ++++++++++++++++++++++++++++ crates/if-image/tests/xiso_write.rs | 130 +++++++++++ crates/isofoundry/src/cli_image.rs | 79 ++++++- docs/LIMITATIONS.md | 1 + docs/roadmap.md | 2 +- tasks/plan.md | 10 +- tasks/todo.md | 14 +- 10 files changed, 567 insertions(+), 23 deletions(-) create mode 100644 crates/if-image/src/write/xiso.rs create mode 100644 crates/if-image/tests/xiso_write.rs diff --git a/CHANGELOG.md b/CHANGELOG.md index 1624714..4745906 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,10 @@ All notable changes are listed here. Versions follow [Semantic Versioning](https ## [Unreleased] +### Added + +- XDVDFS/Xbox XISO support: identify, browse, and extract existing images, and create new XISOs with `create-image --fs xdvdfs` (or `xiso`). Existing XISOs are read-only; ISO-specific labels and boot-entry options are not supported. + ## [1.0.1] - 2026-11-08 Patch release: one robustness fix from the panic-path review, plus documentation. No feature changes. diff --git a/README.md b/README.md index dda3e60..8200e11 100644 --- a/README.md +++ b/README.md @@ -33,17 +33,17 @@ IsoFoundry is a single free Windows program for working with CDs, DVDs, Blu-rays These jobs run from the task windows and the command line: -- **`create-image`** — build an ISO from files and folders: ISO 9660 + Joliet + UDF 1.02 in one image sharing a single copy of the data, files over 4 GB, El Torito boot entries for BIOS (optional boot information table) and UEFI. Written to a `.part` file and renamed only when complete, so a cancelled build never leaves a half-written image. -- **`image-info`** — an image's container format, tracks, file systems, label and boot entries. -- **`extract`** — copy files out of an image (everything, or one file/folder), renaming names Windows cannot store, and keeping nothing half-written on cancel. -- **`edit-image`** — add, remove, rename, create folders, change the label, saved as a new image; untouched files are copied straight from the source, and BIOS/UEFI boot entries are kept. +- **`create-image`** — build an ISO from files and folders: ISO 9660 + Joliet + UDF 1.02 in one image sharing a single copy of the data, files over 4 GB, El Torito boot entries for BIOS (optional boot information table) and UEFI. `--fs xdvdfs` (or `xiso`) creates an Xbox XISO. Written to a `.part` file and renamed only when complete, so a cancelled build never leaves a half-written image. +- **`image-info`** — an image's container format, tracks, file systems, label and boot entries; recognizes XDVDFS/XISO. +- **`extract`** — copy files out of an image (everything, or one file/folder), renaming names Windows cannot store, and keeping nothing half-written on cancel. XISO browsing and extraction also work in the GUI browse task. +- **`edit-image`** — add, remove, rename, create folders, change the label, saved as a new image; untouched files are copied straight from the source, and BIOS/UEFI boot entries are kept. XDVDFS/XISO images are read-only. - **`convert-image`** — any readable image to ISO, BIN/CUE, CloneCD (CCD/IMG/SUB) or CHD. - **`checksum`** — CRC32, MD5, SHA-1 and SHA-256 in one pass; `--dat` also writes a Logiqx-style `.dat` hash file. - **`verify --dat`** — look a file's hashes up in a known-good `.dat` list (for example a redump list) and report match or no match. - **`compare`** — byte-for-byte comparison of two files. - **`mount` / `unmount`** — mount an image read-only with Windows' built-in virtual disk support; other formats are converted to a temporary ISO first, which is removed on unmount. -Formats: ISO 9660 (with Rock Ridge names), Joliet, UDF 1.02–2.60 (physical, sparable and metadata partitions), multi-extent files, El Torito boot catalogs (BIOS and UEFI). CD images: BIN/CUE (single and multi-file; MODE1, MODE2 and audio tracks), CloneCD CCD/IMG/SUB, sheetless raw `.bin`/`.img`, and MAME CHD v5 — reading `zlib`/`cdzl` discs and writing CD (`cdzl`) and DVD (`zlib`) CHDs whose SHA-1s match chdman 0.264 byte for byte on the test images. Nero NRG and Alcohol MDS/MDF are read-only and marked unverified. Raw sector ECC/EDC generation and checks, and Q-subchannel generation, are included. Known format limits are listed in [docs/LIMITATIONS.md](docs/LIMITATIONS.md). +Formats: ISO 9660 (with Rock Ridge names), Joliet, UDF 1.02–2.60 (physical, sparable and metadata partitions), XDVDFS/Xbox XISO (browse, extract and create), multi-extent files, El Torito boot catalogs (BIOS and UEFI). CD images: BIN/CUE (single and multi-file; MODE1, MODE2 and audio tracks), CloneCD CCD/IMG/SUB, sheetless raw `.bin`/`.img`, and MAME CHD v5 — reading `zlib`/`cdzl` discs and writing CD (`cdzl`) and DVD (`zlib`) CHDs whose SHA-1s match chdman 0.264 byte for byte on the test images. Nero NRG and Alcohol MDS/MDF are read-only and marked unverified. Raw sector ECC/EDC generation and checks, and Q-subchannel generation, are included. Known format limits are listed in [docs/LIMITATIONS.md](docs/LIMITATIONS.md). ## USB tools diff --git a/crates/if-image/src/write/mod.rs b/crates/if-image/src/write/mod.rs index 199afea..e7eeb1e 100644 --- a/crates/if-image/src/write/mod.rs +++ b/crates/if-image/src/write/mod.rs @@ -8,6 +8,7 @@ mod names; pub(crate) mod udfw; +mod xiso; use crate::{bad, Error, Result, SECTOR}; use std::collections::HashSet; @@ -200,6 +201,8 @@ pub struct Summary { pub dirs: u64, } +pub use xiso::build_xiso; + struct Plan { nodes: Vec

, opts: Options, diff --git a/crates/if-image/src/write/xiso.rs b/crates/if-image/src/write/xiso.rs new file mode 100644 index 0000000..b9f685e --- /dev/null +++ b/crates/if-image/src/write/xiso.rs @@ -0,0 +1,337 @@ +use super::{reader, Error, Kind, Node, Out, Source, Summary}; +use crate::{bad, Result, SECTOR}; +use std::collections::HashSet; +use std::fs::File; +use std::io::{BufWriter, Read, Write}; +use std::path::{Path, PathBuf}; +use std::time::{SystemTime, UNIX_EPOCH}; + +const HEADER_SECTOR: u64 = 32; +const ROOT_SECTOR: u64 = 0x108; +const DIRECTORY_LIMIT: u64 = 64 << 20; +const MAX_NODES: usize = 100_000; +const MAX_DEPTH: usize = 64; +const SECTOR_SIZE: u64 = SECTOR as u64; +const IMAGE_ALIGNMENT_SECTORS: u64 = 32; + +struct Plan { + root: Directory, + sectors: u64, + files: u64, + directories: u64, +} + +struct Directory { + entries: Vec, + order: Vec, + table: Vec, + allocated_size: u64, + sector: u32, +} + +struct Entry { + name: Vec, + size: u32, + source: Option, + directory: Option>, + sector: u32, + offset: u64, + left: Option, + right: Option, +} + +fn validate_name(name: &str) -> Result> { + let bytes = name.as_bytes(); + if bytes.is_empty() + || bytes.len() > u8::MAX as usize + || name == "." + || name == ".." + || name.contains(['/', '\\']) + || name.chars().any(char::is_control) + { + return bad(format!("invalid XDVDFS filename: {name:?}")); + } + Ok(bytes.to_vec()) +} + +fn tree_order(indices: &[usize], entries: &mut [Entry], order: &mut Vec) -> Option { + if indices.is_empty() { + return None; + } + let middle = indices.len() / 2; + let index = indices[middle]; + order.push(index); + let left = tree_order(&indices[..middle], entries, order); + let right = tree_order(&indices[middle + 1..], entries, order); + entries[index].left = left; + entries[index].right = right; + Some(index) +} + +fn plan_directory( + nodes: &[Node], + depth: usize, + count: &mut usize, + files: &mut u64, + directories: &mut u64, +) -> Result { + if depth > MAX_DEPTH { + return bad("folders nested deeper than 64 levels"); + } + let mut names = HashSet::new(); + let mut entries = Vec::with_capacity(nodes.len()); + for node in nodes { + *count += 1; + if *count > MAX_NODES { + return bad("XDVDFS contains too many files and folders"); + } + let name = validate_name(&node.name)?; + if !names.insert(node.name.to_ascii_lowercase()) { + return bad(format!("duplicate XDVDFS filename: {}", node.name)); + } + let entry = match &node.kind { + Kind::Dir(children) => { + *directories += 1; + let directory = plan_directory(children, depth + 1, count, files, directories)?; + let stored_size = directory.allocated_size; + Entry { + name, + size: u32::try_from(stored_size) + .map_err(|_| Error::Format("XDVDFS directory is too large".into()))?, + source: None, + directory: Some(Box::new(directory)), + sector: 0, + offset: 0, + left: None, + right: None, + } + } + Kind::File { source, size } => { + *files += 1; + Entry { + name, + size: u32::try_from(*size).map_err(|_| Error::Format("XDVDFS files cannot exceed 4 GiB".into()))?, + source: Some(source.clone()), + directory: None, + sector: 0, + offset: 0, + left: None, + right: None, + } + } + }; + entries.push(entry); + } + + let mut indices: Vec = (0..entries.len()).collect(); + indices.sort_by(|&a, &b| entries[a].name.to_ascii_uppercase().cmp(&entries[b].name.to_ascii_uppercase())); + let mut order = Vec::with_capacity(entries.len()); + tree_order(&indices, &mut entries, &mut order); + + let mut end = 0u64; + for &index in &order { + let record_size = (14 + entries[index].name.len() as u64 + 3) & !3; + if end % SECTOR_SIZE + record_size > SECTOR_SIZE { + end = end.div_ceil(SECTOR_SIZE) * SECTOR_SIZE; + } + if end / 4 > u16::MAX as u64 { + return bad("XDVDFS directory exceeds its 16-bit entry-offset limit"); + } + entries[index].offset = end; + end = end.checked_add(record_size).ok_or_else(|| Error::Format("XDVDFS directory size overflow".into()))?; + } + if end > DIRECTORY_LIMIT { + return bad("XDVDFS directory is larger than 64 MB"); + } + for entry in &entries { + for child in [entry.left, entry.right].into_iter().flatten() { + if entries[child].offset / 4 > u16::MAX as u64 { + return bad("XDVDFS directory exceeds its 16-bit entry-offset limit"); + } + } + } + + let allocated_size = if end == 0 { SECTOR_SIZE } else { end.div_ceil(SECTOR_SIZE) * SECTOR_SIZE }; + Ok(Directory { entries, order, table: vec![0xff; end as usize], allocated_size, sector: 0 }) +} + +fn serialize_tables(dir: &mut Directory) { + for entry in &mut dir.entries { + if let Some(child) = entry.directory.as_mut() { + serialize_tables(child); + } + } + let mut table = vec![0xff; dir.table.len()]; + for &index in &dir.order { + let entry = &dir.entries[index]; + let offset = entry.offset as usize; + let left = entry.left.map(|i| (dir.entries[i].offset / 4) as u16).unwrap_or(0); + let right = entry.right.map(|i| (dir.entries[i].offset / 4) as u16).unwrap_or(0); + table[offset..offset + 2].copy_from_slice(&left.to_le_bytes()); + table[offset + 2..offset + 4].copy_from_slice(&right.to_le_bytes()); + table[offset + 4..offset + 8].copy_from_slice(&entry.sector.to_le_bytes()); + table[offset + 8..offset + 12].copy_from_slice(&entry.size.to_le_bytes()); + table[offset + 12] = if entry.directory.is_some() { 0x10 } else { 0x20 }; + table[offset + 13] = entry.name.len() as u8; + table[offset + 14..offset + 14 + entry.name.len()].copy_from_slice(&entry.name); + } + dir.table = table; +} + +fn assign_sectors(dir: &mut Directory, next: &mut u64) -> Result<()> { + dir.sector = + u32::try_from(*next).map_err(|_| Error::Format("XDVDFS image exceeds 32-bit sector offsets".into()))?; + *next = next + .checked_add(dir.allocated_size / SECTOR_SIZE) + .ok_or_else(|| Error::Format("XDVDFS image size overflow".into()))?; + + for entry in &mut dir.entries { + if entry.directory.is_none() { + entry.sector = + u32::try_from(*next).map_err(|_| Error::Format("XDVDFS image exceeds 32-bit sector offsets".into()))?; + *next = next + .checked_add(u64::from(entry.size).div_ceil(SECTOR_SIZE)) + .ok_or_else(|| Error::Format("XDVDFS image size overflow".into()))?; + } + } + for entry in &mut dir.entries { + if let Some(child) = entry.directory.as_mut() { + assign_sectors(child, next)?; + entry.sector = child.sector; + entry.size = child.allocated_size as u32; + } + } + Ok(()) +} + +fn put_xdvdfs_data(out: &mut Out, lba: u64, data: &[u8], allocated_size: u64) -> Result<()> { + out.zero(lba)?; + out.w.write_all(data)?; + let padding = allocated_size - data.len() as u64; + let pad = [0xff; SECTOR]; + let mut remaining = padding; + while remaining > 0 { + let size = remaining.min(SECTOR_SIZE) as usize; + out.w.write_all(&pad[..size])?; + remaining -= size as u64; + } + out.at += allocated_size / SECTOR_SIZE; + out.tick() +} + +fn write_file(out: &mut Out, entry: &Entry) -> Result<()> { + let size = u64::from(entry.size); + out.zero(u64::from(entry.sector))?; + if size == 0 { + return Ok(()); + } + + let source = entry.source.as_ref().expect("file source"); + let mut input = reader(source, size)?; + let mut buffer = vec![0; 1 << 20]; + let mut remaining = size; + while remaining > 0 { + let want = remaining.min(buffer.len() as u64) as usize; + let mut read = 0; + while read < want { + let n = input.read(&mut buffer[read..want])?; + if n == 0 { + return bad("file data became shorter while the XISO was being built"); + } + read += n; + } + out.w.write_all(&buffer[..read])?; + remaining -= read as u64; + out.at += read as u64 / SECTOR_SIZE; + out.tick()?; + } + let padding = (SECTOR_SIZE - size % SECTOR_SIZE) % SECTOR_SIZE; + if padding > 0 { + out.w.write_all(&[0xff; SECTOR][..padding as usize])?; + out.at += 1; + out.tick()?; + } + Ok(()) +} + +fn write_directory(out: &mut Out, dir: &Directory) -> Result<()> { + put_xdvdfs_data(out, u64::from(dir.sector), &dir.table, dir.allocated_size)?; + for entry in &dir.entries { + if entry.directory.is_none() { + write_file(out, entry)?; + } + } + for entry in &dir.entries { + if let Some(child) = &entry.directory { + write_directory(out, child)?; + } + } + Ok(()) +} + +fn plan(root: &Node) -> Result { + let Kind::Dir(children) = &root.kind else { + return bad("an XDVDFS image must be built from a folder"); + }; + let mut files = 0; + let mut directories = 1; + let mut count = 0; + let mut root = plan_directory(children, 0, &mut count, &mut files, &mut directories)?; + let mut next = ROOT_SECTOR; + assign_sectors(&mut root, &mut next)?; + serialize_tables(&mut root); + let sectors = next.div_ceil(IMAGE_ALIGNMENT_SECTORS) * IMAGE_ALIGNMENT_SECTORS; + Ok(Plan { root, sectors, files, directories }) +} + +/// Builds a standalone XDVDFS/XISO image, without modifying or patching file contents. +pub fn build_xiso(root: &Node, dest: &Path, progress: impl FnMut(u64, u64) -> bool + 'static) -> Result

{ + let plan = plan(root)?; + let mut part = dest.as_os_str().to_owned(); + part.push(".part"); + let part = PathBuf::from(part); + let file = File::create(&part)?; + let result = (|| -> Result<()> { + let mut out = Out { + w: BufWriter::with_capacity(1 << 20, &file), + at: 0, + total: plan.sectors, + progress: Box::new(progress), + }; + let mut header = vec![0; SECTOR]; + header[..20].copy_from_slice(b"MICROSOFT*XBOX*MEDIA"); + header[20..24].copy_from_slice(&plan.root.sector.to_le_bytes()); + header[24..28].copy_from_slice(&(plan.root.table.len() as u32).to_le_bytes()); + let elapsed = SystemTime::now().duration_since(UNIX_EPOCH).unwrap_or_default(); + let filetime = 116_444_736_000_000_000u64 + .saturating_add(elapsed.as_secs().saturating_mul(10_000_000)) + .saturating_add(u64::from(elapsed.subsec_nanos() / 100)); + header[28..36].copy_from_slice(&filetime.to_le_bytes()); + header[0x7ec..0x7ec + 20].copy_from_slice(b"MICROSOFT*XBOX*MEDIA"); + out.put(HEADER_SECTOR, &header)?; + write_directory(&mut out, &plan.root)?; + out.zero(plan.sectors)?; + out.w.flush()?; + drop(out); + file.sync_all()?; + Ok(()) + })(); + drop(file); + if let Err(error) = result { + let _ = std::fs::remove_file(&part); + return Err(error); + } + if let Err(error) = std::fs::rename(&part, dest) { + if let Err(cleanup) = std::fs::remove_file(&part) { + if cleanup.kind() != std::io::ErrorKind::NotFound { + return bad(format!( + "could not publish XISO to {} ({error}) or remove temporary file {} ({cleanup})", + dest.display(), + part.display() + )); + } + } + return Err(error.into()); + } + Ok(Summary { sectors: plan.sectors, files: plan.files, dirs: plan.directories }) +} diff --git a/crates/if-image/tests/xiso_write.rs b/crates/if-image/tests/xiso_write.rs new file mode 100644 index 0000000..efc1e2a --- /dev/null +++ b/crates/if-image/tests/xiso_write.rs @@ -0,0 +1,130 @@ +use if_image::fs::{FsKind, Volume}; +use if_image::source::IsoFile; +use if_image::write::{build_xiso, Kind, Node, Source}; +use std::path::{Path, PathBuf}; + +fn output(name: &str) -> PathBuf { + std::env::temp_dir().join(format!("if-image-xiso-{name}-{}.iso", std::process::id())) +} + +fn tree() -> Node { + let files = + (0..31).map(|i| Node::bytes(&format!("file-{i:02}.txt"), 0, format!("data-{i}").into_bytes())).collect(); + Node::dir( + "", + 0, + vec![ + Node::bytes("readme.txt", 0, b"hello XISO\n".to_vec()), + Node::dir("nested", 0, vec![Node::dir("empty", 0, vec![]), Node::dir("files", 0, files)]), + ], + ) +} + +fn read_file(volume: &mut Volume, path: &str) -> Vec { + let entry = volume.lookup(path).unwrap(); + let mut bytes = vec![0; entry.size as usize]; + volume.read(&entry, 0, &mut bytes).unwrap(); + bytes +} + +fn table_names(image: &[u8], sector: u32, size: u32) -> Vec { + let start = sector as usize * 2048; + let table = &image[start..start + size as usize]; + let mut pending = vec![0usize]; + let mut visited = std::collections::HashSet::new(); + let mut names = Vec::new(); + while let Some(offset) = pending.pop() { + assert!(visited.insert(offset), "directory tree contains a cycle"); + assert!(offset + 14 <= table.len(), "entry offset is outside the directory table"); + let left = u16::from_le_bytes(table[offset..offset + 2].try_into().unwrap()); + let right = u16::from_le_bytes(table[offset + 2..offset + 4].try_into().unwrap()); + let len = usize::from(table[offset + 13]); + assert!(offset + 14 + len <= table.len(), "filename is outside the directory table"); + names.push(String::from_utf8(table[offset + 14..offset + 14 + len].to_vec()).unwrap()); + if right != 0 { + pending.push(usize::from(right) * 4); + } + if left != 0 { + pending.push(usize::from(left) * 4); + } + } + names +} + +#[test] +fn creates_xdvdfs_image_with_nested_files_and_empty_directories() { + let path = output("roundtrip"); + let summary = build_xiso(&tree(), &path, |_, _| true).unwrap(); + assert_eq!(std::fs::metadata(&path).unwrap().len(), summary.sectors * 2048); + assert_eq!(summary.sectors % 32, 0); + + let mut volume = Volume::open(Box::new(IsoFile::open(&path).unwrap()), None).unwrap(); + assert_eq!(volume.fs.kind(), FsKind::Xdvdfs); + assert_eq!(read_file(&mut volume, "readme.txt"), b"hello XISO\n"); + for i in 0..31 { + assert_eq!(read_file(&mut volume, &format!("nested/files/file-{i:02}.txt")), format!("data-{i}").as_bytes()); + } + let empty = volume.lookup("nested/empty").unwrap(); + assert!(empty.dir); + assert!(volume.list(&empty).unwrap().is_empty()); + + let bytes = std::fs::read(&path).unwrap(); + let root_sector = u32::from_le_bytes(bytes[32 * 2048 + 20..32 * 2048 + 24].try_into().unwrap()); + let root_size = u32::from_le_bytes(bytes[32 * 2048 + 24..32 * 2048 + 28].try_into().unwrap()); + let root_names = table_names(&bytes, root_sector, root_size); + assert_eq!(root_names.len(), 2); + assert!(root_names.iter().any(|name| name == "nested")); + assert!(root_names.iter().any(|name| name == "readme.txt")); + + std::fs::remove_file(path).unwrap(); +} + +#[test] +fn rejects_xdvdfs_names_that_cannot_be_encoded() { + let path = output("invalid-name"); + let source = Node::dir("", 0, vec![Node::bytes("bad/name", 0, vec![])]); + + assert!(build_xiso(&source, &path, |_, _| true).is_err()); + assert!(!Path::new(&path).exists()); +} + +#[test] +fn rejects_files_larger_than_xdvdfs_size_fields() { + let path = output("oversized-file"); + let source = Node::dir( + "", + 0, + vec![Node { + name: "large.bin".into(), + mtime: 0, + kind: Kind::File { source: Source::Bytes(Vec::new()), size: u64::from(u32::MAX) + 1 }, + }], + ); + + assert!(build_xiso(&source, &path, |_, _| true).is_err()); + assert!(!path.exists()); +} + +#[test] +fn rejects_directories_that_exceed_xdvdfs_pointer_range() { + let path = output("oversized-directory"); + let files = (0..13_500).map(|i| Node::bytes(&format!("f{i:05}"), 0, Vec::new())).collect(); + let source = Node::dir("", 0, files); + + assert!(build_xiso(&source, &path, |_, _| true).is_err()); + assert!(!path.exists()); +} + +#[test] +fn removes_partial_output_when_final_rename_fails() { + let path = output("rename-failure"); + std::fs::create_dir(&path).unwrap(); + std::fs::write(path.join("keep.txt"), b"keep").unwrap(); + let part = PathBuf::from(format!("{}.part", path.display())); + + assert!(build_xiso(&tree(), &path, |_, _| true).is_err()); + assert_eq!(std::fs::read(path.join("keep.txt")).unwrap(), b"keep"); + assert!(!part.exists()); + + std::fs::remove_dir_all(path).unwrap(); +} diff --git a/crates/isofoundry/src/cli_image.rs b/crates/isofoundry/src/cli_image.rs index 08924c8..1ec163e 100644 --- a/crates/isofoundry/src/cli_image.rs +++ b/crates/isofoundry/src/cli_image.rs @@ -11,7 +11,7 @@ use serde_json::json; use std::io::Read; use std::path::{Path, PathBuf}; -pub const USAGE: &str = " create-image SRC... --out FILE.iso [--label L] [--fs iso9660,joliet,udf]\n\ +pub const USAGE: &str = " create-image SRC... --out FILE.iso [--label L] [--fs iso9660,joliet,udf|xdvdfs]\n\ [--boot-bios IMG [--boot-load-sectors N] [--boot-info-table]] [--boot-efi IMG]\n\ Build an ISO from files and folders (one folder = its contents)\n\ image-info IMAGE Format, tracks, file systems, label and boot entries\n\ @@ -149,7 +149,7 @@ pub fn source_tree(sources: &[PathBuf]) -> Result) -> R { - use if_image::write::{build, BootImage, Options}; + use if_image::write::{build, build_xiso, BootImage, Options}; let a = parse(tail, &["out", "label", "fs", "boot-bios", "boot-efi", "boot-load-sectors"]).map_err(usage_err)?; let mut info_table = false; let pos: Vec = a @@ -167,16 +167,34 @@ fn create(tail: &[String], g: &Global, o: &mut Out<'_>) -> R { .collect(); let dest = PathBuf::from(a.get("out").ok_or_else(|| usage_err("missing --out FILE.iso"))?); let mut opts = Options::default(); + let mut xdvdfs = false; if let Some(fs) = a.get("fs") { - let set: Vec = fs.split(',').map(|x| x.trim().to_ascii_lowercase()).collect(); + let set: Vec = fs + .split(',') + .map(|x| x.trim().to_ascii_lowercase()) + .map(|x| if x == "xiso" { "xdvdfs".to_string() } else { x }) + .collect(); for x in &set { - if !["iso9660", "joliet", "udf"].contains(&x.as_str()) { + if !["iso9660", "joliet", "udf", "xdvdfs"].contains(&x.as_str()) { return Err(usage_err(format!("unknown file system {x}"))); } } + xdvdfs = set.iter().any(|x| x == "xdvdfs"); + if xdvdfs && set.len() != 1 { + return Err(usage_err("XDVDFS cannot be combined with ISO 9660, Joliet or UDF")); + } opts.joliet = set.iter().any(|x| x == "joliet"); opts.udf = set.iter().any(|x| x == "udf"); } + if xdvdfs + && (a.get("label").is_some() + || a.get("boot-bios").is_some() + || a.get("boot-efi").is_some() + || a.get("boot-load-sectors").is_some() + || info_table) + { + return Err(usage_err("XDVDFS does not support volume labels or El Torito boot options")); + } opts.label = a.get("label").map(str::to_string).unwrap_or_else(|| { match pos.as_slice() { [one] => one.file_name().map(|n| n.to_string_lossy().into_owned()), @@ -202,7 +220,8 @@ fn create(tail: &[String], g: &Global, o: &mut Out<'_>) -> R { o.ok_text(&format!("would build {} from {} source(s)", dest.display(), pos.len())); return Ok(()); } - let s = build(&root, opts, &dest, |_, _| true).map_err(image_err)?; + let s = if xdvdfs { build_xiso(&root, &dest, |_, _| true) } else { build(&root, opts, &dest, |_, _| true) } + .map_err(image_err)?; if o.json { o.ok_text( &json!({ "ok": true, "path": dest.display().to_string(), "bytes": s.sectors * 2048, @@ -657,6 +676,56 @@ mod tests { std::fs::remove_dir_all(image.parent().unwrap()).unwrap(); } + #[test] + fn create_image_can_build_xdvdfs() { + let d = tmp("create-xiso"); + std::fs::create_dir_all(d.join("nested")).unwrap(); + std::fs::write(d.join("nested").join("payload.bin"), b"Xbox").unwrap(); + let image = d.join("game.iso"); + let (code, out, err) = + call(&["create-image", d.to_str().unwrap(), "--out", image.to_str().unwrap(), "--fs", "xdvdfs"]); + assert_eq!(code, 0, "{out}{err}"); + + let (code, out, err) = call(&["image-info", image.to_str().unwrap(), "--json"]); + assert_eq!(code, 0, "{err}"); + let info: serde_json::Value = serde_json::from_str(out.trim()).unwrap(); + assert_eq!(info["file_systems"], json!(["XDVDFS"])); + + let dest = d.join("extracted"); + let (code, _, err) = + call(&["extract", image.to_str().unwrap(), "--out", dest.to_str().unwrap(), "--fs", "xiso"]); + assert_eq!(code, 0, "{err}"); + assert_eq!(std::fs::read(dest.join("nested").join("payload.bin")).unwrap(), b"Xbox"); + std::fs::remove_dir_all(d).unwrap(); + } + + #[test] + fn xdvdfs_create_rejects_unsupported_iso_options() { + let d = tmp("xiso-options"); + std::fs::create_dir_all(&d).unwrap(); + std::fs::write(d.join("payload.bin"), b"Xbox").unwrap(); + let output = d.join("game.iso"); + let (code, _, err) = + call(&["create-image", d.to_str().unwrap(), "--out", output.to_str().unwrap(), "--fs", "xdvdfs,joliet"]); + assert_eq!(code, 2, "{err}"); + assert!(err.contains("cannot be combined")); + + let (code, _, err) = call(&[ + "create-image", + d.to_str().unwrap(), + "--out", + output.to_str().unwrap(), + "--fs", + "xdvdfs", + "--label", + "GAMES", + ]); + assert_eq!(code, 2, "{err}"); + assert!(err.contains("does not support volume labels")); + assert!(!output.exists()); + std::fs::remove_dir_all(d).unwrap(); + } + #[test] fn checksum_dat_and_verify() { let d = tmp("dat"); diff --git a/docs/LIMITATIONS.md b/docs/LIMITATIONS.md index 7d80a7b..6841a23 100644 --- a/docs/LIMITATIONS.md +++ b/docs/LIMITATIONS.md @@ -22,6 +22,7 @@ Kept current every release. Items move out only with evidence. - UDF metadata partitions (Blu-ray) are implemented but not yet tested against a real disc image. - CHD: reads `zlib` and `cdzl` hunks only. CHDs compressed with chdman's defaults (LZMA, FLAC, Huffman) need `chdman copy -c cdzl` (CD) or `-c zlib` (DVD) first. Parent (delta) CHDs are not supported. IsoFoundry writes `cdzl`/`zlib`, so its CHDs are a few percent larger than chdman's defaults. - NRG and MDS/MDF are read only and unverified; first session only. Multi-session images and CD-Text are not converted. +- XDVDFS/Xbox XISO images can be browsed and extracted, and the CLI can create new XISOs. Editing existing XISOs, volume labels, and El Torito boot entries are unsupported. File sizes and data sectors use 32-bit fields; directory tree links use 16-bit dword offsets. Game executable patching and copy-protection circumvention are out of scope. - Converting a multi-track CD to ISO keeps only the first data track. - Editing rewrites the image as ISO 9660 + Joliet + UDF 1.02. Hidden flags, Rock Ridge permissions and UDF 2.x extras of the source are not kept. Images that boot with floppy or hard-disk emulation can't be edited yet. Boot images that are also visible files are stored twice. - Mounting uses `AttachVirtualDisk`; whether it works without administrator rights depends on Windows policy and is untested outside CI (where the account is an administrator). A mount lasts until `unmount` or restart. diff --git a/docs/roadmap.md b/docs/roadmap.md index 05070cf..1ff6f38 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -38,7 +38,7 @@ Out of scope: anything for copy-protected discs (SecuROM, SafeDisc, LibCrypt and - [x] **Auto-action on disc insert** (idea from [Automatic Ripping Machine](https://github.com/automatic-ripping-machine/automatic-ripping-machine), MIT): optional mode that detects a newly inserted disc and identifies it (audio / data). As built (M6, D-051) it opens the chosen job's window rather than starting the job itself — the output location is a per-job choice; encrypted video discs are not ripped. - [ ] **Edit image files** (reference: [TotalImage](https://github.com/TotalImage/TotalImage), MIT): open an image, browse it, extract, add or remove files and save. ISO9660/Joliet/UDF first (M1), then FAT floppy/HDD images and VHD (1.x). - [x] **Preservation dumping UI** (reference: [Redumper GUI](https://github.com/Deterous/Redumper-GUI), GPL-3.0, Rust): pick drive and system, show progress and the log, produce redump-style output. GPL code is not copied. (M5) -- [ ] Optional: open/extract/create Xbox XISO (XDVDFS) images (reference: [extract-xiso](https://github.com/XboxDev/extract-xiso)). Filesystem only; no patching of game executables. (1.x) +- [x] Open, extract, and create Xbox XISO (XDVDFS) images (reference: [extract-xiso](https://github.com/XboxDev/extract-xiso)). The CLI creates XISOs; the GUI browse task opens and extracts them. Existing XISOs are read-only. Filesystem only; no patching of game executables. (1.x) Not used, because they bypass console copy protection: PS3-Quick-Disc-Decryptor (decrypts PS3 disc images with keys), ftpdump (dumps games from modified PS4 consoles). diff --git a/tasks/plan.md b/tasks/plan.md index 8767447..ec4a6b3 100644 --- a/tasks/plan.md +++ b/tasks/plan.md @@ -30,13 +30,13 @@ Complete the four explicitly selected 1.x roadmap features: multiboot USB, FAT/V - [ ] XISO, FAT/VHD, and multiboot format/asset decisions are supported by repository evidence and documented references. ### Phase 1: Xbox XISO -- [ ] Task 2: Read and validate XDVDFS volumes. -- [ ] Task 3: Expose XISO browse and extraction in CLI and GUI. -- [ ] Task 4: Create XISO images and add end-to-end coverage. +- [x] Task 2: Read and validate XDVDFS volumes. +- [x] Task 3: Expose XISO browse and extraction in CLI and GUI. +- [x] Task 4: Create XISO images and add end-to-end coverage. ### Checkpoint: XISO -- [ ] Fixture images list, extract and recreate files with byte-for-byte verification. -- [ ] CLI and GUI workflows are documented and tested without physical hardware. +- [x] Fixture images list, extract and recreate files with byte-for-byte verification. +- [x] CLI and GUI workflows are documented and tested without physical hardware. ### Phase 2: FAT and VHD image editing - [ ] Task 5: Read FAT12/16/32 volumes and locate supported partitions. diff --git a/tasks/todo.md b/tasks/todo.md index 1d45369..8cb9778 100644 --- a/tasks/todo.md +++ b/tasks/todo.md @@ -51,21 +51,21 @@ **Files likely touched:** `crates/isofoundry/src/cli_image.rs`, `crates/isofoundry/src/win/tasks/browse.rs`, `crates/if-image/tests/read.rs` **Estimated scope:** Medium -## Task 4: Creating XISO images and end-to-end coverage — In progress +## Task 4: Creating XISO images and end-to-end coverage — Done **Description:** Add an XDVDFS writer and an explicit XISO create path that produces compatible images without touching game executables or protected content. **Acceptance criteria:** -- [ ] A source tree can be written as XISO and reopened by the new reader. -- [ ] File/directory extents, alignment, names, and size limits are validated. -- [ ] Existing ISO creation and editing output remain unchanged. +- [x] A source tree can be written as XISO and reopened by the new reader. +- [x] File/directory extents, alignment, names, and size limits are validated. +- [x] Existing ISO creation and editing output remain unchanged. **Verification:** -- [ ] Round-trip tests compare every source file byte-for-byte and exercise invalid names/limits. -- [ ] Document CLI and GUI availability, filesystem scope, and exclusions. +- [x] Round-trip tests compare every source file byte-for-byte and exercise invalid names/limits. +- [x] Document CLI and GUI availability, filesystem scope, and exclusions. **Dependencies:** Task 2 -**Files likely touched:** `crates/if-image/src/xiso_write.rs`, `crates/if-image/src/lib.rs`, `crates/isofoundry/src/cli_image.rs`, focused tests/docs +**Files likely touched:** `crates/if-image/src/write/xiso.rs`, `crates/if-image/src/write/mod.rs`, `crates/isofoundry/src/cli_image.rs`, focused tests/docs **Estimated scope:** Medium ## Task 5: Reading FAT volumes and locating supported partitions From 238604dcce05d7dc8af295e887c6180c7e327eb3 Mon Sep 17 00:00:00 2001 From: Sean Shaffer Date: Sun, 8 Nov 2026 23:20:45 -0600 Subject: [PATCH 06/12] feat: add FAT image browsing and extraction Recognize FAT12/16/32 superfloppies and primary MBR partitions so raw disk images can use the existing image-info, browse, and extract flows. Keep FAT edits explicitly read-only until writing is implemented.\n\nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- crates/if-image/src/edit.rs | 3 + crates/if-image/src/fat.rs | 623 +++++++++++++++++++++++++++++ crates/if-image/src/fs.rs | 40 +- crates/if-image/src/lib.rs | 1 + crates/if-image/tests/fat.rs | 285 +++++++++++++ crates/isofoundry/src/cli_image.rs | 58 +++ docs/ARCHITECTURE.md | 2 +- docs/roadmap.md | 2 +- tasks/plan.md | 4 +- tasks/todo.md | 24 +- 10 files changed, 1025 insertions(+), 17 deletions(-) create mode 100644 crates/if-image/src/fat.rs create mode 100644 crates/if-image/tests/fat.rs diff --git a/crates/if-image/src/edit.rs b/crates/if-image/src/edit.rs index e5a9603..fd43a4f 100644 --- a/crates/if-image/src/edit.rs +++ b/crates/if-image/src/edit.rs @@ -65,6 +65,9 @@ impl Project { if v.fs.kind() == FsKind::Xdvdfs { return bad("XDVDFS images are read-only and cannot be edited"); } + if v.fs.kind() == FsKind::Fat { + return bad("FAT images are read-only and cannot be edited yet"); + } let mut warnings = Vec::new(); let label = v.fs.label(); let joliet = v.found.contains(&FsKind::Joliet) || v.found.contains(&FsKind::Udf); diff --git a/crates/if-image/src/fat.rs b/crates/if-image/src/fat.rs new file mode 100644 index 0000000..cedf2bd --- /dev/null +++ b/crates/if-image/src/fat.rs @@ -0,0 +1,623 @@ +//! Read-only FAT12/16/32 support for raw floppy and MBR-partitioned images. + +use crate::fs::{read_data, ByteExtent, Data, Entry, FileSystem, FsKind}; +use crate::source::{read_bytes, SectorSource}; +use crate::{bad, Result, SECTOR}; +use std::cell::RefCell; +use std::collections::{HashMap, HashSet}; + +const MAX_DIRECTORY_BYTES: u64 = 64 << 20; +const MAX_DIRECTORY_ENTRIES: usize = 100_000; +const FAT_CACHE_PAGES: usize = 64; + +#[derive(Clone, Copy)] +enum FatType { + Fat12, + Fat16, + Fat32, +} + +pub struct Fat { + kind: FatType, + bytes_per_sector: u64, + sectors_per_cluster: u64, + cluster_count: u32, + fat_offset: u64, + fat_size: u64, + data_offset: u64, + volume_end: u64, + root: Entry, + label: String, + fat_cache: RefCell>>, +} + +impl Fat { + /// Finds a FAT superfloppy or the first supported FAT partition in an MBR. + pub fn probe(src: &mut dyn SectorSource) -> Result> { + let image_bytes = src + .sectors() + .checked_mul(SECTOR as u64) + .ok_or_else(|| crate::Error::Format("image size overflow".into()))?; + if image_bytes < 512 { + return Ok(None); + } + let mbr = read_bytes(src, 0, 0, 512)?; + let has_partition_table = mbr[510..512] == [0x55, 0xaa] + && (0..4).any(|index| { + let entry = &mbr[446 + index * 16..462 + index * 16]; + matches!(entry[0], 0x00 | 0x80) + && entry[4] != 0 + && entry[8..12].iter().any(|byte| *byte != 0) + && entry[12..16].iter().any(|byte| *byte != 0) + }); + if !has_partition_table { + if let Some(fat) = Self::at(src, 0, image_bytes)? { + return Ok(Some(fat)); + } + } + if mbr[510..512] != [0x55, 0xaa] { + return Ok(None); + } + for index in 0..4 { + let entry = &mbr[446 + index * 16..462 + index * 16]; + if !is_fat_partition(entry[4]) { + continue; + } + let start = u64::from(u32::from_le_bytes(entry[8..12].try_into().unwrap())) + .checked_mul(512) + .ok_or_else(|| crate::Error::Format("partition offset overflow".into()))?; + let length = u64::from(u32::from_le_bytes(entry[12..16].try_into().unwrap())) + .checked_mul(512) + .ok_or_else(|| crate::Error::Format("partition size overflow".into()))?; + if length == 0 || start.checked_add(length).is_none_or(|end| end > image_bytes) { + return bad("FAT partition extends beyond the image"); + } + return Self::at(src, start, length) + .and_then(|fat| fat.ok_or_else(|| crate::Error::Format("invalid FAT partition boot sector".into()))) + .map(Some); + } + Ok(None) + } + + fn at(src: &mut dyn SectorSource, base: u64, available: u64) -> Result> { + if available < 512 { + return Ok(None); + } + let boot = read_bytes(src, 0, base, 512)?; + if &boot[3..11] == b"EXFAT " { + return bad("exFAT volumes are not supported"); + } + if !matches!(boot[0], 0xeb | 0xe9) || &boot[3..11] == b"NTFS " { + return Ok(None); + } + let bytes_per_sector = u64::from(u16::from_le_bytes([boot[11], boot[12]])); + if !matches!(bytes_per_sector, 512 | 1024 | 2048 | 4096) { + return bad("unsupported FAT bytes-per-sector value"); + } + let full_boot = read_bytes(src, 0, base, bytes_per_sector as usize)?; + if full_boot[bytes_per_sector as usize - 2..] != [0x55, 0xaa] { + return Ok(None); + } + + let sectors_per_cluster = u64::from(boot[13]); + let reserved_sectors = u64::from(u16::from_le_bytes([boot[14], boot[15]])); + let fat_count = u64::from(boot[16]); + let root_entries = u64::from(u16::from_le_bytes([boot[17], boot[18]])); + let total16 = u64::from(u16::from_le_bytes([boot[19], boot[20]])); + let fat16 = u64::from(u16::from_le_bytes([boot[22], boot[23]])); + let total32 = u64::from(u32::from_le_bytes(boot[32..36].try_into().unwrap())); + let fat32 = u64::from(u32::from_le_bytes(boot[36..40].try_into().unwrap())); + let total_sectors = if total16 != 0 { total16 } else { total32 }; + let fat_sectors = if fat16 != 0 { fat16 } else { fat32 }; + if !sectors_per_cluster.is_power_of_two() + || sectors_per_cluster > 128 + || reserved_sectors == 0 + || !(1..=2).contains(&fat_count) + || total_sectors == 0 + || fat_sectors == 0 + { + return bad("invalid FAT volume geometry"); + } + + let root_dir_sectors = root_entries + .checked_mul(32) + .and_then(|n| n.checked_add(bytes_per_sector - 1)) + .ok_or_else(|| crate::Error::Format("FAT root directory size overflow".into()))? + / bytes_per_sector; + let first_data_sector = reserved_sectors + .checked_add( + fat_count + .checked_mul(fat_sectors) + .ok_or_else(|| crate::Error::Format("FAT region size overflow".into()))?, + ) + .and_then(|n| n.checked_add(root_dir_sectors)) + .ok_or_else(|| crate::Error::Format("FAT data offset overflow".into()))?; + if first_data_sector >= total_sectors { + return bad("FAT data region is outside the volume"); + } + let cluster_count = (total_sectors - first_data_sector) / sectors_per_cluster; + let cluster_count = u32::try_from(cluster_count) + .map_err(|_| crate::Error::Format("FAT volume has too many clusters".into()))?; + let kind = if cluster_count < 4085 { + FatType::Fat12 + } else if cluster_count < 65_525 { + FatType::Fat16 + } else { + FatType::Fat32 + }; + if cluster_count > 0x0fff_ffed { + return bad("FAT volume exceeds the supported cluster-number range"); + } + if matches!(kind, FatType::Fat32) && root_entries != 0 { + return bad("invalid FAT32 root directory geometry"); + } + if !matches!(kind, FatType::Fat32) && (root_entries == 0 || fat16 == 0) { + return bad("invalid FAT12/16 root directory geometry"); + } + + let entries = u64::from(cluster_count) + .checked_add(2) + .ok_or_else(|| crate::Error::Format("FAT entry count overflow".into()))?; + let required_fat_bytes = match kind { + FatType::Fat12 => entries.checked_mul(3).and_then(|n| n.checked_add(1)).map(|n| n / 2), + FatType::Fat16 => entries.checked_mul(2), + FatType::Fat32 => entries.checked_mul(4), + } + .ok_or_else(|| crate::Error::Format("FAT size overflow".into()))?; + let fat_size = fat_sectors + .checked_mul(bytes_per_sector) + .ok_or_else(|| crate::Error::Format("FAT size overflow".into()))?; + if fat_size < required_fat_bytes { + return bad("FAT table is too small for the volume"); + } + + let volume_size = total_sectors + .checked_mul(bytes_per_sector) + .ok_or_else(|| crate::Error::Format("FAT volume size overflow".into()))?; + if volume_size > available + || base.checked_add(volume_size).is_none_or(|end| end > src.sectors() * SECTOR as u64) + { + return bad("FAT volume extends beyond the image"); + } + let fat_offset = base + .checked_add( + reserved_sectors + .checked_mul(bytes_per_sector) + .ok_or_else(|| crate::Error::Format("FAT offset overflow".into()))?, + ) + .ok_or_else(|| crate::Error::Format("FAT offset overflow".into()))?; + let data_offset = base + .checked_add( + first_data_sector + .checked_mul(bytes_per_sector) + .ok_or_else(|| crate::Error::Format("FAT data offset overflow".into()))?, + ) + .ok_or_else(|| crate::Error::Format("FAT data offset overflow".into()))?; + let volume_end = base + volume_size; + + let label_offset: usize = if matches!(kind, FatType::Fat32) { 71 } else { 43 }; + let label = if full_boot.get(label_offset.wrapping_sub(5)) == Some(&0x29) { + String::from_utf8_lossy(&full_boot[label_offset..label_offset + 11]).trim().to_string() + } else { + String::new() + }; + let mut fat = Self { + kind, + bytes_per_sector, + sectors_per_cluster, + cluster_count, + fat_offset, + fat_size, + data_offset, + volume_end, + root: Entry { + name: String::new(), + dir: true, + size: 0, + mtime: None, + hidden: false, + data: Data::ByteExtents(Vec::new()), + }, + label, + fat_cache: RefCell::new(HashMap::new()), + }; + + let (root_data, root_size) = if matches!(kind, FatType::Fat32) { + let cluster = u32::from_le_bytes(full_boot[44..48].try_into().unwrap()) & 0x0fff_ffff; + if cluster < 2 || cluster > cluster_count + 1 { + return bad("invalid FAT32 root directory cluster"); + } + let extents = fat.cluster_extents(src, cluster, None)?; + let size = extents.iter().map(|e| e.len).sum(); + if size > MAX_DIRECTORY_BYTES { + return bad("FAT root directory is larger than 64 MB"); + } + (extents, size) + } else { + let root_sector = reserved_sectors + fat_count * fat_sectors; + let size = root_dir_sectors * bytes_per_sector; + (vec![ByteExtent { offset: base + root_sector * bytes_per_sector, len: size }], size) + }; + fat.root.size = root_size; + fat.root.data = Data::ByteExtents(root_data); + Ok(Some(fat)) + } + + fn fat_bytes(&self, src: &mut dyn SectorSource, offset: u64, len: usize) -> Result> { + let mut out = vec![0; len]; + let mut done = 0usize; + while done < len { + let position = offset + done as u64; + let absolute = self + .fat_offset + .checked_add(position) + .ok_or_else(|| crate::Error::Format("FAT offset overflow".into()))?; + let page = absolute / SECTOR as u64; + let in_page = (absolute % SECTOR as u64) as usize; + let count = (SECTOR - in_page).min(len - done); + if !self.fat_cache.borrow().contains_key(&page) { + let bytes = read_bytes(src, 0, page * SECTOR as u64, SECTOR)?; + let mut cache = self.fat_cache.borrow_mut(); + if cache.len() >= FAT_CACHE_PAGES { + cache.clear(); + } + cache.insert(page, bytes); + } + let cache = self.fat_cache.borrow(); + let bytes = cache.get(&page).expect("FAT cache page was inserted"); + out[done..done + count].copy_from_slice(&bytes[in_page..in_page + count]); + done += count; + } + Ok(out) + } + + fn next_cluster(&self, src: &mut dyn SectorSource, cluster: u32) -> Result> { + let cluster = u64::from(cluster); + let (offset, width, value, eoc, bad_value) = match self.kind { + FatType::Fat12 => { + let offset = cluster + cluster / 2; + if offset + 2 > self.fat_size { + return bad("FAT cluster entry is outside the allocation table"); + } + let bytes = self.fat_bytes(src, offset, 2)?; + let pair = u16::from_le_bytes([bytes[0], bytes[1]]); + let value = if cluster & 1 == 0 { pair & 0x0fff } else { pair >> 4 }; + (offset, 2, u32::from(value), 0x0ff8, 0x0ff7) + } + FatType::Fat16 => { + let offset = cluster * 2; + if offset + 2 > self.fat_size { + return bad("FAT cluster entry is outside the allocation table"); + } + let bytes = self.fat_bytes(src, offset, 2)?; + (offset, 2, u32::from(u16::from_le_bytes([bytes[0], bytes[1]])), 0xfff8, 0xfff7) + } + FatType::Fat32 => { + let offset = cluster * 4; + if offset + 4 > self.fat_size { + return bad("FAT cluster entry is outside the allocation table"); + } + let bytes = self.fat_bytes(src, offset, 4)?; + (offset, 4, u32::from_le_bytes(bytes[..4].try_into().unwrap()) & 0x0fff_ffff, 0x0fff_fff8, 0x0fff_fff7) + } + }; + if offset + width > self.fat_size { + return bad("FAT cluster entry is outside the allocation table"); + } + if value >= eoc { + return Ok(None); + } + if value == 0 || value == 1 || value == bad_value || value > self.cluster_count + 1 { + return bad("invalid cluster in FAT allocation chain"); + } + Ok(Some(value)) + } + + fn cluster_extents( + &self, + src: &mut dyn SectorSource, + first: u32, + expected_size: Option, + ) -> Result> { + if first < 2 || first > self.cluster_count + 1 { + return bad("invalid first cluster in FAT directory entry"); + } + let cluster_bytes = self.bytes_per_sector * self.sectors_per_cluster; + let mut extents = Vec::new(); + let mut current = first; + let mut run_start = first; + let mut run_count = 0u64; + let mut total = 0u64; + let mut slow = Some(first); + let mut fast = Some(first); + + loop { + if current < 2 || current > self.cluster_count + 1 { + return bad("FAT chain references a cluster outside the volume"); + } + if run_count == 0 { + run_start = current; + } else if current != run_start + run_count as u32 { + self.push_cluster_extent(&mut extents, run_start, run_count, cluster_bytes)?; + run_start = current; + run_count = 0; + } + run_count += 1; + total = total + .checked_add(cluster_bytes) + .ok_or_else(|| crate::Error::Format("FAT chain size overflow".into()))?; + if let Some(size) = expected_size { + if total >= size { + break; + } + } else if total > MAX_DIRECTORY_BYTES { + return bad("FAT directory is larger than 64 MB"); + } + + let Some(next) = self.next_cluster(src, current)? else { break }; + current = next; + slow = match slow { + Some(cluster) => self.next_cluster(src, cluster)?, + None => None, + }; + fast = match fast { + Some(cluster) => self.next_cluster(src, cluster)?, + None => None, + }; + fast = match fast { + Some(cluster) => self.next_cluster(src, cluster)?, + None => None, + }; + if slow.is_some() && slow == fast { + return bad("cycle in FAT allocation chain"); + } + } + if let Some(size) = expected_size { + if total < size { + return bad("FAT allocation chain is shorter than the file size"); + } + } + if run_count > 0 { + self.push_cluster_extent(&mut extents, run_start, run_count, cluster_bytes)?; + } + if let Some(size) = expected_size { + let mut remaining = size; + extents.retain_mut(|extent| { + extent.len = extent.len.min(remaining); + remaining -= extent.len; + extent.len > 0 + }); + } + Ok(extents) + } + + fn push_cluster_extent( + &self, + extents: &mut Vec, + first: u32, + clusters: u64, + cluster_bytes: u64, + ) -> Result<()> { + let offset = self + .data_offset + .checked_add( + u64::from(first - 2) + .checked_mul(cluster_bytes) + .ok_or_else(|| crate::Error::Format("FAT cluster offset overflow".into()))?, + ) + .ok_or_else(|| crate::Error::Format("FAT cluster offset overflow".into()))?; + let len = clusters + .checked_mul(cluster_bytes) + .ok_or_else(|| crate::Error::Format("FAT extent size overflow".into()))?; + if offset.checked_add(len).is_none_or(|end| end > self.volume_end) { + return bad("FAT cluster extent is outside the volume"); + } + extents.push(ByteExtent { offset, len }); + Ok(()) + } + + fn list_directory(&self, src: &mut dyn SectorSource, dir: &Entry) -> Result> { + if !dir.dir { + return bad(format!("{} is not a folder", dir.name)); + } + if dir.size > MAX_DIRECTORY_BYTES { + return bad("FAT directory is larger than 64 MB"); + } + let size = usize::try_from(dir.size).map_err(|_| crate::Error::Format("FAT directory is too large".into()))?; + let mut bytes = vec![0; size]; + read_data(src, &dir.data, dir.size, 0, &mut bytes)?; + + let mut entries = Vec::new(); + let mut names = HashSet::new(); + let mut long_name = LongName::default(); + for record in bytes.chunks_exact(32) { + if record[0] == 0 { + break; + } + if record[0] == 0xe5 { + long_name.clear(); + continue; + } + if record[11] == 0x0f { + long_name.push(record); + continue; + } + let attributes = record[11]; + if attributes & 0x08 != 0 { + long_name.clear(); + continue; + } + let short_name = short_name(record)?; + let name = long_name.take(&record[..11]).unwrap_or(short_name); + if name == "." || name == ".." { + continue; + } + if name.is_empty() + || name.contains(['/', '\\']) + || name.chars().any(char::is_control) + || !names.insert(name.to_lowercase()) + { + return bad(format!("invalid or duplicate FAT filename: {name:?}")); + } + if entries.len() == MAX_DIRECTORY_ENTRIES { + return bad("FAT directory contains too many entries"); + } + let first_cluster = (u32::from(u16::from_le_bytes([record[20], record[21]])) << 16) + | u32::from(u16::from_le_bytes([record[26], record[27]])); + let first_cluster = + if matches!(self.kind, FatType::Fat32) { first_cluster & 0x0fff_ffff } else { first_cluster }; + let file_size = u64::from(u32::from_le_bytes(record[28..32].try_into().unwrap())); + let is_dir = attributes & 0x10 != 0; + let data = if is_dir { + let extents = self.cluster_extents(src, first_cluster, None)?; + let size = extents.iter().map(|e| e.len).sum(); + (Data::ByteExtents(extents), size) + } else if file_size == 0 { + (Data::ByteExtents(Vec::new()), 0) + } else { + (Data::ByteExtents(self.cluster_extents(src, first_cluster, Some(file_size))?), file_size) + }; + entries.push(Entry { + name, + dir: is_dir, + size: data.1, + mtime: None, + hidden: attributes & 0x02 != 0, + data: data.0, + }); + long_name.clear(); + } + Ok(entries) + } +} + +fn is_fat_partition(kind: u8) -> bool { + matches!(kind, 0x01 | 0x04 | 0x06 | 0x0b | 0x0c | 0x0e | 0x11 | 0x14 | 0x16 | 0x1b | 0x1c | 0x1e) +} + +fn short_name(record: &[u8]) -> Result { + let mut base = record[..8].to_vec(); + let extension = &record[8..11]; + if base.first() == Some(&0x05) { + base[0] = 0xe5; + } + let trim = |bytes: &[u8]| { + let end = bytes.iter().rposition(|b| *b != b' ').map_or(0, |i| i + 1); + String::from_utf8_lossy(&bytes[..end]).into_owned() + }; + let mut name = trim(&base); + let mut ext = trim(extension); + if record[12] & 0x08 != 0 { + name.make_ascii_lowercase(); + } + if record[12] & 0x10 != 0 { + ext.make_ascii_lowercase(); + } + if !ext.is_empty() { + name.push('.'); + name.push_str(&ext); + } + if name.is_empty() { + return bad("empty FAT short filename"); + } + Ok(name) +} + +#[derive(Default)] +struct LongName { + checksum: Option, + parts: Vec>>, + next_sequence: usize, + invalid: bool, +} + +impl LongName { + fn clear(&mut self) { + self.checksum = None; + self.parts.clear(); + self.next_sequence = 0; + self.invalid = false; + } + + fn push(&mut self, record: &[u8]) { + let sequence = usize::from(record[0] & 0x1f); + let is_last = record[0] & 0x40 != 0; + if record[0] & 0xa0 != 0 || record[12] != 0 || record[26..28] != [0, 0] { + self.invalid = true; + return; + } + if is_last { + self.clear(); + if sequence == 0 || sequence > 20 { + self.invalid = true; + return; + } + self.checksum = Some(record[13]); + self.parts.resize(sequence, None); + self.next_sequence = sequence; + } + if sequence == 0 + || sequence > self.parts.len() + || self.checksum != Some(record[13]) + || sequence != self.next_sequence + || self.parts[sequence - 1].is_some() + { + self.invalid = true; + return; + } + let mut units = Vec::with_capacity(13); + for range in [1..11, 14..26, 28..32] { + for pair in record[range].chunks_exact(2) { + units.push(u16::from_le_bytes([pair[0], pair[1]])); + } + } + self.parts[sequence - 1] = Some(units); + self.next_sequence -= 1; + } + + fn take(&mut self, short: &[u8]) -> Option { + let expected = short_checksum(short); + let mut units = Vec::new(); + let valid = !self.invalid + && self.checksum == Some(expected) + && self.next_sequence == 0 + && !self.parts.is_empty() + && self.parts.iter().all(Option::is_some); + if valid { + for part in self.parts.iter().flatten() { + for unit in part { + if *unit == 0 || *unit == 0xffff { + break; + } + units.push(*unit); + } + } + } + self.clear(); + String::from_utf16(&units).ok().filter(|name| !name.is_empty()) + } +} + +fn short_checksum(short: &[u8]) -> u8 { + short + .iter() + .fold(0u8, |sum, byte| (if sum & 1 != 0 { 0x80u8 } else { 0u8 }).wrapping_add(sum >> 1).wrapping_add(*byte)) +} + +impl FileSystem for Fat { + fn kind(&self) -> FsKind { + FsKind::Fat + } + + fn label(&self) -> String { + self.label.clone() + } + + fn root(&self) -> Entry { + self.root.clone() + } + + fn list(&self, src: &mut dyn SectorSource, dir: &Entry) -> Result> { + self.list_directory(src, dir) + } +} diff --git a/crates/if-image/src/fs.rs b/crates/if-image/src/fs.rs index 208e1a9..c641b05 100644 --- a/crates/if-image/src/fs.rs +++ b/crates/if-image/src/fs.rs @@ -13,9 +13,16 @@ pub struct Extent { pub sparse: bool, } +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct ByteExtent { + pub offset: u64, + pub len: u64, +} + #[derive(Debug, Clone, PartialEq, Eq)] pub enum Data { Extents(Vec), + ByteExtents(Vec), /// Small files stored inside the UDF file entry. Inline(Vec), } @@ -34,6 +41,7 @@ pub struct Entry { #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] pub enum FsKind { Xdvdfs, + Fat, Iso9660, Joliet, Udf, @@ -43,6 +51,7 @@ impl FsKind { pub fn name(self) -> &'static str { match self { FsKind::Xdvdfs => "XDVDFS", + FsKind::Fat => "FAT", FsKind::Iso9660 => "ISO 9660", FsKind::Joliet => "Joliet", FsKind::Udf => "UDF", @@ -71,6 +80,30 @@ pub fn read_data(src: &mut dyn SectorSource, data: &Data, size: u64, offset: u64 buf[n..want].fill(0); Ok(want) } + Data::ByteExtents(exts) => { + let mut done = 0usize; + let mut base = 0u64; + for extent in exts { + if done == want { + break; + } + let pos = offset + done as u64; + if pos >= base + extent.len { + base += extent.len; + continue; + } + let in_extent = pos - base; + let count = ((extent.len - in_extent) as usize).min(want - done); + let bytes = crate::source::read_bytes(src, 0, extent.offset + in_extent, count)?; + buf[done..done + count].copy_from_slice(&bytes); + done += count; + base += extent.len; + } + if done < want { + buf[done..want].fill(0); + } + Ok(want) + } Data::Extents(exts) => { let mut done = 0usize; let mut base = 0u64; @@ -143,6 +176,7 @@ impl Volume { /// Unicode), Joliet, then plain ISO 9660. `prefer` picks one explicitly. pub fn open(mut src: Box, prefer: Option) -> Result { let xdvdfs = crate::xiso::Xdvdfs::probe(src.as_mut())?; + let fat = crate::fat::Fat::probe(src.as_mut())?; let iso = crate::iso9660::scan(src.as_mut()).ok(); let udf_result = crate::udf::Udf::open(src.as_mut()); let udf_error = udf_result.as_ref().err().map(|e| e.to_string()).filter(|m| m != "no UDF anchor"); @@ -151,6 +185,9 @@ impl Volume { if xdvdfs.is_some() { found.push(FsKind::Xdvdfs); } + if fat.is_some() { + found.push(FsKind::Fat); + } if udf.is_some() { found.push(FsKind::Udf); } @@ -165,7 +202,7 @@ impl Volume { if found.is_empty() { return match udf_error { Some(m) => bad(format!("UDF file system found but not readable: {m}")), - None => bad("no XDVDFS, ISO 9660, Joliet or UDF file system found"), + None => bad("no XDVDFS, FAT, ISO 9660, Joliet or UDF file system found"), }; } let want = prefer.unwrap_or(found[0]); @@ -176,6 +213,7 @@ impl Volume { iso.as_ref().and_then(|s| s.boot_catalog).and_then(|lba| crate::eltorito::read(src.as_mut(), lba).ok()); let fs: Box = match want { FsKind::Xdvdfs => Box::new(xdvdfs.expect("checked")), + FsKind::Fat => Box::new(fat.expect("checked")), FsKind::Udf => Box::new(udf.expect("checked")), FsKind::Joliet => Box::new(iso.and_then(|s| s.joliet).expect("checked")), FsKind::Iso9660 => Box::new(iso.and_then(|s| s.primary).expect("checked")), diff --git a/crates/if-image/src/lib.rs b/crates/if-image/src/lib.rs index 651f7f6..7e87561 100644 --- a/crates/if-image/src/lib.rs +++ b/crates/if-image/src/lib.rs @@ -18,6 +18,7 @@ pub mod cue; pub mod disc; pub mod edit; pub mod eltorito; +pub mod fat; pub mod fs; pub mod iso9660; pub mod mds; diff --git a/crates/if-image/tests/fat.rs b/crates/if-image/tests/fat.rs new file mode 100644 index 0000000..be03e56 --- /dev/null +++ b/crates/if-image/tests/fat.rs @@ -0,0 +1,285 @@ +use if_image::fs::Volume; +use if_image::source::MemSource; + +#[derive(Clone, Copy)] +enum FatType { + Fat12, + Fat16, + Fat32, +} + +struct Fixture { + image: Vec, + kind: FatType, + cluster_count: u32, +} + +fn set_fat_entry(fat: &mut [u8], kind: FatType, cluster: u32, value: u32) { + match kind { + FatType::Fat12 => { + let offset = cluster as usize + cluster as usize / 2; + let mut pair = u16::from_le_bytes([fat[offset], fat[offset + 1]]); + pair = if cluster & 1 == 0 { + (pair & 0xf000) | (value as u16 & 0x0fff) + } else { + (pair & 0x000f) | ((value as u16 & 0x0fff) << 4) + }; + fat[offset..offset + 2].copy_from_slice(&pair.to_le_bytes()); + } + FatType::Fat16 => { + let offset = cluster as usize * 2; + fat[offset..offset + 2].copy_from_slice(&(value as u16).to_le_bytes()); + } + FatType::Fat32 => { + let offset = cluster as usize * 4; + fat[offset..offset + 4].copy_from_slice(&value.to_le_bytes()); + } + } +} + +fn checksum(short: &[u8; 11]) -> u8 { + short + .iter() + .fold(0u8, |sum, byte| (if sum & 1 != 0 { 0x80u8 } else { 0u8 }).wrapping_add(sum >> 1).wrapping_add(*byte)) +} + +fn fat_fixture(kind: FatType) -> Fixture { + let (total_sectors, reserved, fat_sectors, root_entries) = match kind { + FatType::Fat12 => (2880usize, 1usize, 9usize, 32usize), + FatType::Fat16 => (8192, 1, 32, 32), + FatType::Fat32 => (66_594, 32, 513, 0), + }; + let fats = 2usize; + let root_sectors = (root_entries * 32).div_ceil(512); + let first_data_sector = reserved + fats * fat_sectors + root_sectors; + let cluster_count = (total_sectors - first_data_sector) as u32; + let mut image = vec![0; (total_sectors * 512).div_ceil(2048) * 2048]; + let boot = &mut image[..512]; + boot[..3].copy_from_slice(&[0xeb, 0x3c, 0x90]); + boot[3..11].copy_from_slice(b"ISOFNDRY"); + boot[11..13].copy_from_slice(&512u16.to_le_bytes()); + boot[13] = 1; + boot[14..16].copy_from_slice(&(reserved as u16).to_le_bytes()); + boot[16] = fats as u8; + boot[17..19].copy_from_slice(&(root_entries as u16).to_le_bytes()); + if total_sectors < 65_536 { + boot[19..21].copy_from_slice(&(total_sectors as u16).to_le_bytes()); + } else { + boot[32..36].copy_from_slice(&(total_sectors as u32).to_le_bytes()); + } + boot[21] = 0xf8; + if matches!(kind, FatType::Fat32) { + boot[36..40].copy_from_slice(&(fat_sectors as u32).to_le_bytes()); + boot[44..48].copy_from_slice(&2u32.to_le_bytes()); + boot[66] = 0x29; + boot[71..82].copy_from_slice(b"FAT TEST "); + } else { + boot[22..24].copy_from_slice(&(fat_sectors as u16).to_le_bytes()); + boot[38] = 0x29; + boot[43..54].copy_from_slice(b"FAT TEST "); + } + boot[510..512].copy_from_slice(&[0x55, 0xaa]); + + for fat_index in 0..fats { + let start = (reserved + fat_index * fat_sectors) * 512; + let fat = &mut image[start..start + fat_sectors * 512]; + match kind { + FatType::Fat12 => { + fat[..3].copy_from_slice(&[0xf8, 0xff, 0xff]); + set_fat_entry(fat, kind, 3, 5); + set_fat_entry(fat, kind, 5, 0xfff); + set_fat_entry(fat, kind, 7, 0xfff); + set_fat_entry(fat, kind, 8, 0xfff); + } + FatType::Fat16 => { + set_fat_entry(fat, kind, 0, 0xfff8); + set_fat_entry(fat, kind, 1, 0xffff); + set_fat_entry(fat, kind, 3, 5); + set_fat_entry(fat, kind, 5, 0xffff); + set_fat_entry(fat, kind, 7, 0xffff); + set_fat_entry(fat, kind, 8, 0xffff); + } + FatType::Fat32 => { + set_fat_entry(fat, kind, 0, 0x0fff_fff8); + set_fat_entry(fat, kind, 1, 0xffff_ffff); + set_fat_entry(fat, kind, 2, 0x0fff_ffff); + set_fat_entry(fat, kind, 3, 5); + set_fat_entry(fat, kind, 5, 0x0fff_ffff); + set_fat_entry(fat, kind, 7, 0x0fff_ffff); + set_fat_entry(fat, kind, 8, 0x0fff_ffff); + } + } + } + + let root_offset = + if matches!(kind, FatType::Fat32) { first_data_sector * 512 } else { (reserved + fats * fat_sectors) * 512 }; + let root = + &mut image[root_offset..root_offset + if matches!(kind, FatType::Fat32) { 512 } else { root_sectors * 512 }]; + let short: [u8; 11] = *b"AVERYL~1TXT"; + let name: Vec = "A very long filename.txt".encode_utf16().collect(); + let lfn_count = name.len().div_ceil(13); + for sequence in (1..=lfn_count).rev() { + let start = (lfn_count - sequence) * 32; + let lfn = &mut root[start..start + 32]; + lfn[0] = sequence as u8 | if sequence == lfn_count { 0x40 } else { 0 }; + lfn[11] = 0x0f; + lfn[13] = checksum(&short); + let mut units = [0xffffu16; 13]; + let unit_start = (sequence - 1) * 13; + let count = (name.len() - unit_start).min(13); + units[..count].copy_from_slice(&name[unit_start..unit_start + count]); + if count < 13 { + units[count] = 0; + } + for (index, unit) in units.iter().enumerate() { + let offset = match index { + 0..=4 => 1 + index * 2, + 5..=10 => 14 + (index - 5) * 2, + _ => 28 + (index - 11) * 2, + }; + lfn[offset..offset + 2].copy_from_slice(&unit.to_le_bytes()); + } + } + let entry_offset = lfn_count * 32; + let entry = &mut root[entry_offset..entry_offset + 32]; + entry[..11].copy_from_slice(&short); + entry[11] = 0x20; + entry[26..28].copy_from_slice(&3u16.to_le_bytes()); + entry[28..32].copy_from_slice(&600u32.to_le_bytes()); + let nested = &mut root[entry_offset + 32..entry_offset + 64]; + nested[..11].copy_from_slice(b"SUBDIR "); + nested[11] = 0x10; + nested[26..28].copy_from_slice(&7u16.to_le_bytes()); + + let first_file_sector = first_data_sector + (3 - 2); + let second_file_sector = first_data_sector + (5 - 2); + image[first_file_sector * 512..first_file_sector * 512 + 512].fill(b'a'); + image[second_file_sector * 512..second_file_sector * 512 + 88].fill(b'b'); + let subdir_sector = first_data_sector + (7 - 2); + let subdir = &mut image[subdir_sector * 512..(subdir_sector + 1) * 512]; + subdir[..11].copy_from_slice(b"NESTED TXT"); + subdir[11] = 0x20; + subdir[26..28].copy_from_slice(&8u16.to_le_bytes()); + subdir[28..32].copy_from_slice(&3u32.to_le_bytes()); + let nested_file_sector = first_data_sector + (8 - 2); + image[nested_file_sector * 512..nested_file_sector * 512 + 3].copy_from_slice(b"sub"); + Fixture { image, kind, cluster_count } +} + +fn partitioned(fixture: Fixture) -> Vec { + let start_sector = 1u32; + let length_sectors = (fixture.image.len() / 512) as u32; + let mut disk = vec![0; (512 + fixture.image.len()).div_ceil(2048) * 2048]; + let entry = &mut disk[446..462]; + entry[4] = match fixture.kind { + FatType::Fat12 => 0x01, + FatType::Fat16 => 0x16, + FatType::Fat32 => 0x0c, + }; + entry[8..12].copy_from_slice(&start_sector.to_le_bytes()); + entry[12..16].copy_from_slice(&length_sectors.to_le_bytes()); + disk[510..512].copy_from_slice(&[0x55, 0xaa]); + let start = start_sector as usize * 512; + disk[start..start + fixture.image.len()].copy_from_slice(&fixture.image); + disk +} + +fn read_long_file(mut image: Vec) -> Vec { + let mut volume = Volume::open(Box::new(MemSource(std::mem::take(&mut image))), None).unwrap(); + assert_eq!(volume.fs.kind().name(), "FAT"); + assert_eq!(volume.fs.label(), "FAT TEST"); + let file = volume.lookup("A very long filename.txt").unwrap(); + let mut bytes = vec![0; file.size as usize]; + volume.read(&file, 0, &mut bytes).unwrap(); + bytes +} + +#[test] +fn reads_fat12_fat16_and_fat32_long_names_and_fragmented_files() { + for kind in [FatType::Fat12, FatType::Fat16, FatType::Fat32] { + let fixture = fat_fixture(kind); + let mut volume = Volume::open(Box::new(MemSource(fixture.image)), None).unwrap(); + assert_eq!(volume.fs.kind().name(), "FAT"); + assert_eq!(volume.fs.label(), "FAT TEST"); + let file = volume.lookup("A very long filename.txt").unwrap(); + let mut bytes = vec![0; file.size as usize]; + volume.read(&file, 0, &mut bytes).unwrap(); + assert_eq!(&bytes[..512], vec![b'a'; 512]); + assert_eq!(&bytes[512..], vec![b'b'; 88]); + let nested = volume.lookup("SUBDIR/NESTED.TXT").unwrap(); + let mut nested_bytes = vec![0; nested.size as usize]; + volume.read(&nested, 0, &mut nested_bytes).unwrap(); + assert_eq!(nested_bytes, b"sub"); + } +} + +#[test] +fn reads_a_fat_partition_at_a_non_2048_byte_aligned_mbr_offset() { + let disk = partitioned(fat_fixture(FatType::Fat16)); + let bytes = read_long_file(disk); + assert_eq!(&bytes[..512], vec![b'a'; 512]); + assert_eq!(&bytes[512..], vec![b'b'; 88]); +} + +#[test] +fn rejects_fat_file_chains_that_point_outside_the_volume() { + let mut fixture = fat_fixture(FatType::Fat16); + let first_fat = 512; + let invalid = fixture.cluster_count + 2; + set_fat_entry(&mut fixture.image[first_fat..], fixture.kind, 3, invalid); + + let mut volume = Volume::open(Box::new(MemSource(fixture.image)), None).unwrap(); + assert!(volume.lookup("Long File.txt").is_err()); +} + +#[test] +fn rejects_cyclic_fat_file_chains() { + let mut fixture = fat_fixture(FatType::Fat16); + let first_fat = 512; + set_fat_entry(&mut fixture.image[first_fat..], fixture.kind, 3, 5); + set_fat_entry(&mut fixture.image[first_fat..], fixture.kind, 5, 3); + let root = (1 + 2 * 32) * 512; + fixture.image[root + 64 + 28..root + 64 + 32].copy_from_slice(&1200u32.to_le_bytes()); + + let mut volume = Volume::open(Box::new(MemSource(fixture.image)), None).unwrap(); + assert!(volume.lookup("A very long filename.txt").is_err()); +} + +#[test] +fn rejects_fat_volumes_with_invalid_cluster_geometry() { + let mut fixture = fat_fixture(FatType::Fat12); + fixture.image[13] = 3; + + assert!(Volume::open(Box::new(MemSource(fixture.image)), None).is_err()); +} + +#[test] +fn rejects_fat_mbr_partitions_outside_the_image() { + let mut disk = vec![0; 2048]; + disk[450] = 0x0c; + disk[454..458].copy_from_slice(&2u32.to_le_bytes()); + disk[458..462].copy_from_slice(&8u32.to_le_bytes()); + disk[510..512].copy_from_slice(&[0x55, 0xaa]); + + assert!(Volume::open(Box::new(MemSource(disk)), None).is_err()); +} + +#[test] +fn reports_exfat_as_unsupported() { + let mut image = vec![0; 2048]; + image[..3].copy_from_slice(&[0xeb, 0x76, 0x90]); + image[3..11].copy_from_slice(b"EXFAT "); + + let error = Volume::open(Box::new(MemSource(image)), None).err().unwrap().to_string(); + assert!(error.contains("exFAT volumes are not supported")); +} + +#[test] +fn refuses_to_edit_fat_images_until_fat_writing_is_supported() { + let path = std::env::temp_dir().join(format!("if-image-fat-readonly-{}.img", std::process::id())); + std::fs::write(&path, fat_fixture(FatType::Fat16).image).unwrap(); + + let error = if_image::edit::Project::open(&path).err().unwrap().to_string(); + assert!(error.contains("FAT images are read-only")); + std::fs::remove_file(path).unwrap(); +} diff --git a/crates/isofoundry/src/cli_image.rs b/crates/isofoundry/src/cli_image.rs index 1ec163e..46d1495 100644 --- a/crates/isofoundry/src/cli_image.rs +++ b/crates/isofoundry/src/cli_image.rs @@ -74,6 +74,7 @@ pub fn open_volume(path: &Path, prefer: Option) -> Result Result { match s.to_ascii_lowercase().as_str() { + "fat" => Ok(FsKind::Fat), "udf" => Ok(FsKind::Udf), "joliet" => Ok(FsKind::Joliet), "iso9660" | "iso" => Ok(FsKind::Iso9660), @@ -628,6 +629,38 @@ mod tests { path } + fn fat12_fixture() -> PathBuf { + let d = tmp("fat12"); + std::fs::create_dir_all(&d).unwrap(); + let mut image = vec![0; 2880 * 512]; + let boot = &mut image[..512]; + boot[..3].copy_from_slice(&[0xeb, 0x3c, 0x90]); + boot[3..11].copy_from_slice(b"ISOFNDRY"); + boot[11..13].copy_from_slice(&512u16.to_le_bytes()); + boot[13] = 1; + boot[14..16].copy_from_slice(&1u16.to_le_bytes()); + boot[16] = 2; + boot[17..19].copy_from_slice(&32u16.to_le_bytes()); + boot[19..21].copy_from_slice(&2880u16.to_le_bytes()); + boot[21] = 0xf0; + boot[22..24].copy_from_slice(&9u16.to_le_bytes()); + boot[38] = 0x29; + boot[43..54].copy_from_slice(b"FAT CLI "); + boot[510..512].copy_from_slice(&[0x55, 0xaa]); + for start in [512usize, 10 * 512] { + image[start..start + 5].copy_from_slice(&[0xf0, 0xff, 0xff, 0xff, 0x0f]); + } + let root = &mut image[19 * 512..21 * 512]; + root[..11].copy_from_slice(b"HELLO TXT"); + root[11] = 0x20; + root[26..28].copy_from_slice(&2u16.to_le_bytes()); + root[28..32].copy_from_slice(&5u32.to_le_bytes()); + image[21 * 512..21 * 512 + 5].copy_from_slice(b"hello"); + let path = d.join("fat.img"); + std::fs::write(&path, image).unwrap(); + path + } + #[test] fn info_json() { let (code, out, _) = call(&["image-info", &fixture("boot.iso"), "--json"]); @@ -676,6 +709,31 @@ mod tests { std::fs::remove_dir_all(image.parent().unwrap()).unwrap(); } + #[test] + fn image_info_and_extract_support_fat() { + let image = fat12_fixture(); + let (code, out, err) = call(&["image-info", image.to_str().unwrap(), "--json"]); + assert_eq!(code, 0, "{err}"); + let info: serde_json::Value = serde_json::from_str(out.trim()).unwrap(); + assert_eq!(info["file_systems"], json!(["FAT"])); + assert_eq!(info["label"], "FAT CLI"); + + let dest = image.parent().unwrap().join("extracted"); + let (code, _, err) = call(&[ + "extract", + image.to_str().unwrap(), + "--out", + dest.to_str().unwrap(), + "--path", + "hello.txt", + "--fs", + "fat", + ]); + assert_eq!(code, 0, "{err}"); + assert_eq!(std::fs::read(dest.join("hello.txt")).unwrap(), b"hello"); + std::fs::remove_dir_all(image.parent().unwrap()).unwrap(); + } + #[test] fn create_image_can_build_xdvdfs() { let d = tmp("create-xiso"); diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index ab931a8..c0d5ff1 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -69,7 +69,7 @@ Every job follows: **plan → preview → confirm (proportional) → journal → ## 5. Image engine - Writers: ISO9660 (levels 1–3, Rock Ridge off), Joliet, UDF 1.02/1.50/2.01/2.50/2.60, hybrid ISO+UDF, El Torito (BIOS + UEFI). -- Readers: above + BIN/CUE (multi-track, audio), CCD/IMG/SUB, MDS/MDF (v1 only, reverse-engineered: Unverified), NRG v1/v2, CDI, DMG (UDIF raw/zlib, HFS+ read-only browse), WIM (browse via wimlib or own reader: decide M1). +- Readers: above + FAT12/16/32 (raw superfloppy or primary MBR partition, browse/extract), XDVDFS (read-only browse/extract), BIN/CUE (multi-track, audio), CCD/IMG/SUB, MDS/MDF (v1 only, reverse-engineered: Unverified), NRG v1/v2, CDI, DMG (UDIF raw/zlib, HFS+ read-only browse), WIM (browse via wimlib or own reader: decide M1). - Edit image: load tree, add/remove/rename, rebuild to new file (never in place). - Convert: any readable → ISO / BIN-CUE / CCD / IMG / CHD. diff --git a/docs/roadmap.md b/docs/roadmap.md index 1ff6f38..1d90e09 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -36,7 +36,7 @@ Out of scope: anything for copy-protected discs (SecuROM, SafeDisc, LibCrypt and ### Automation and image tools - [x] **Auto-action on disc insert** (idea from [Automatic Ripping Machine](https://github.com/automatic-ripping-machine/automatic-ripping-machine), MIT): optional mode that detects a newly inserted disc and identifies it (audio / data). As built (M6, D-051) it opens the chosen job's window rather than starting the job itself — the output location is a per-job choice; encrypted video discs are not ripped. -- [ ] **Edit image files** (reference: [TotalImage](https://github.com/TotalImage/TotalImage), MIT): open an image, browse it, extract, add or remove files and save. ISO9660/Joliet/UDF first (M1), then FAT floppy/HDD images and VHD (1.x). +- [ ] **Edit image files** (reference: [TotalImage](https://github.com/TotalImage/TotalImage), MIT): ISO9660/Joliet/UDF editing is available; FAT12/16/32 raw floppy and primary MBR images can now be browsed and extracted. FAT editing and VHD support remain in progress. (1.x) - [x] **Preservation dumping UI** (reference: [Redumper GUI](https://github.com/Deterous/Redumper-GUI), GPL-3.0, Rust): pick drive and system, show progress and the log, produce redump-style output. GPL code is not copied. (M5) - [x] Open, extract, and create Xbox XISO (XDVDFS) images (reference: [extract-xiso](https://github.com/XboxDev/extract-xiso)). The CLI creates XISOs; the GUI browse task opens and extracts them. Existing XISOs are read-only. Filesystem only; no patching of game executables. (1.x) diff --git a/tasks/plan.md b/tasks/plan.md index ec4a6b3..72c222e 100644 --- a/tasks/plan.md +++ b/tasks/plan.md @@ -39,8 +39,8 @@ Complete the four explicitly selected 1.x roadmap features: multiboot USB, FAT/V - [x] CLI and GUI workflows are documented and tested without physical hardware. ### Phase 2: FAT and VHD image editing -- [ ] Task 5: Read FAT12/16/32 volumes and locate supported partitions. -- [ ] Task 6: Browse and extract FAT images through existing image workflows. +- [x] Task 5: Read FAT12/16/32 volumes and locate supported partitions. +- [x] Task 6: Browse and extract FAT images through existing image workflows. - [ ] Task 7: Edit FAT images and save safely as a new image. - [ ] Task 8: Read and write fixed/dynamic VHD containers with validated metadata. - [ ] Task 9: Integrate VHD-contained FAT editing and verify round trips. diff --git a/tasks/todo.md b/tasks/todo.md index 8cb9778..ea2a6a1 100644 --- a/tasks/todo.md +++ b/tasks/todo.md @@ -68,35 +68,35 @@ **Files likely touched:** `crates/if-image/src/write/xiso.rs`, `crates/if-image/src/write/mod.rs`, `crates/isofoundry/src/cli_image.rs`, focused tests/docs **Estimated scope:** Medium -## Task 5: Reading FAT volumes and locating supported partitions +## Task 5: Reading FAT volumes and locating supported partitions — Done **Description:** Implement bounded FAT12/16/32 volume parsing and identify supported superfloppy or MBR-partitioned FAT volumes in raw disk images. **Acceptance criteria:** -- [ ] FAT12/16/32 fixtures expose directory trees, long names, file extents, and labels where available. -- [ ] Partition/BPB geometry and all cluster chains are bounds-checked; unsupported/exFAT layouts are rejected clearly. -- [ ] Malformed loops and inconsistent chains terminate safely. +- [x] FAT12/16/32 fixtures expose directory trees, long names, file extents, and labels where available. +- [x] Partition/BPB geometry and all cluster chains are bounds-checked; unsupported/exFAT layouts are rejected clearly. +- [x] Malformed loops and inconsistent chains terminate safely. **Verification:** -- [ ] Add fixture tests across FAT types, nested directories, long names, fragmented files, and corrupt chains. -- [ ] Run focused `if-image` tests and malformed-input drills. +- [x] Add fixture tests across FAT types, nested directories, long names, fragmented files, and corrupt chains. +- [x] Run focused `if-image` tests and malformed-input drills. **Dependencies:** Task 1 **Files likely touched:** `crates/if-image/src/fat.rs`, `crates/if-image/src/fs.rs`, `crates/if-image/src/lib.rs`, `crates/if-image/tests/read.rs` **Estimated scope:** Medium -## Task 6: Browsing and extracting FAT images +## Task 6: Browsing and extracting FAT images — Done **Description:** Connect raw FAT floppy and partition images to the existing image-info, browse, and extraction surfaces. **Acceptance criteria:** -- [ ] CLI image-info and extraction can open supported raw FAT images. -- [ ] Existing GUI browse/extract flow handles FAT paths with the same progress and cancellation behavior. -- [ ] ISO/Joliet/UDF selection and behavior remain unchanged. +- [x] CLI image-info and extraction can open supported raw FAT images. +- [x] Existing GUI browse/extract flow handles FAT paths with the same progress and cancellation behavior. +- [x] ISO/Joliet/UDF selection and behavior remain unchanged. **Verification:** -- [ ] Add CLI/integration coverage for FAT images and readback after extraction. -- [ ] Build app and run all `if-image` integration tests. +- [x] Add CLI/integration coverage for FAT images and readback after extraction. +- [x] Build app and run all `if-image` integration tests. **Dependencies:** Task 5 **Files likely touched:** `crates/if-image/src/container.rs`, `crates/if-image/src/fs.rs`, `crates/isofoundry/src/cli_image.rs`, `crates/isofoundry/src/win/tasks/browse.rs` From 5699c30d9293e90c1cc969bde21ff5ef8f8e7183 Mon Sep 17 00:00:00 2001 From: Sean Shaffer Date: Mon, 9 Nov 2026 00:02:33 -0600 Subject: [PATCH 07/12] feat: add bounded VHD image reading Expose fixed and dynamic VHD disks as validated sector sources, including sparse block semantics and exact virtual lengths. Add malformed-container fixtures to guard the bounds checks. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- crates/if-image/src/lib.rs | 1 + crates/if-image/src/source.rs | 18 ++- crates/if-image/src/vhd.rs | 271 ++++++++++++++++++++++++++++++++++ crates/if-image/tests/vhd.rs | 207 ++++++++++++++++++++++++++ 4 files changed, 496 insertions(+), 1 deletion(-) create mode 100644 crates/if-image/src/vhd.rs create mode 100644 crates/if-image/tests/vhd.rs diff --git a/crates/if-image/src/lib.rs b/crates/if-image/src/lib.rs index 7e87561..e632e7c 100644 --- a/crates/if-image/src/lib.rs +++ b/crates/if-image/src/lib.rs @@ -25,6 +25,7 @@ pub mod mds; pub mod nrg; pub mod source; pub mod udf; +pub mod vhd; pub mod write; pub mod xiso; diff --git a/crates/if-image/src/source.rs b/crates/if-image/src/source.rs index 93f46af..d09c0d4 100644 --- a/crates/if-image/src/source.rs +++ b/crates/if-image/src/source.rs @@ -8,6 +8,10 @@ use std::path::Path; pub trait SectorSource { /// Number of 2048-byte sectors. fn sectors(&self) -> u64; + /// Exact virtual byte length; defaults to the complete logical sectors. + fn byte_len(&self) -> u64 { + self.sectors().saturating_mul(SECTOR as u64) + } /// Reads `buf.len() / 2048` sectors starting at `lba`. `buf.len()` is a /// multiple of 2048. Reading past the end is an error. fn read_sectors(&mut self, lba: u64, buf: &mut [u8]) -> Result<()>; @@ -17,6 +21,9 @@ impl SectorSource for Box { fn sectors(&self) -> u64 { (**self).sectors() } + fn byte_len(&self) -> u64 { + (**self).byte_len() + } fn read_sectors(&mut self, lba: u64, buf: &mut [u8]) -> Result<()> { (**self).read_sectors(lba, buf) } @@ -25,6 +32,7 @@ impl SectorSource for Box { /// A plain `.iso` (or any file of 2048-byte sectors). pub struct IsoFile { f: File, + bytes: u64, sectors: u64, } @@ -32,7 +40,7 @@ impl IsoFile { pub fn open(path: &Path) -> Result { let f = File::open(path)?; let len = f.metadata()?.len(); - Ok(Self { f, sectors: len.div_ceil(SECTOR as u64) }) + Ok(Self { f, bytes: len, sectors: len.div_ceil(SECTOR as u64) }) } } @@ -41,6 +49,10 @@ impl SectorSource for IsoFile { self.sectors } + fn byte_len(&self) -> u64 { + self.bytes + } + fn read_sectors(&mut self, lba: u64, buf: &mut [u8]) -> Result<()> { let n = (buf.len() / SECTOR) as u64; if lba + n > self.sectors { @@ -69,6 +81,10 @@ impl SectorSource for MemSource { (self.0.len() / SECTOR) as u64 } + fn byte_len(&self) -> u64 { + self.0.len() as u64 + } + fn read_sectors(&mut self, lba: u64, buf: &mut [u8]) -> Result<()> { let s = lba as usize * SECTOR; let e = s + buf.len(); diff --git a/crates/if-image/src/vhd.rs b/crates/if-image/src/vhd.rs new file mode 100644 index 0000000..4b662cb --- /dev/null +++ b/crates/if-image/src/vhd.rs @@ -0,0 +1,271 @@ +//! Read-only support for fixed and dynamic VHD 1.0 containers. + +use crate::source::SectorSource; +use crate::{bad, Result, SECTOR}; +use std::fs::File; +use std::io::{Read, Seek, SeekFrom}; +use std::path::Path; + +const VHD_SECTOR: u64 = 512; +const FOOTER_SIZE: u64 = 512; +const MAX_VIRTUAL_SIZE: u64 = 2 * 1024 * 1024 * 1024 * 1024; +const MAX_BAT_ENTRIES: u32 = 1_048_576; +const MAX_BLOCK_SIZE: u64 = 256 * 1024 * 1024; +const FOOTER_COOKIE: &[u8; 8] = b"conectix"; +const HEADER_COOKIE: &[u8; 8] = b"cxsparse"; + +enum Layout { + Fixed, + Dynamic { block_size: u64, bitmap_size: u64, blocks: Vec }, +} + +pub struct VhdFile { + file: File, + bytes: u64, + sectors: u64, + layout: Layout, +} + +impl VhdFile { + pub fn open(path: &Path) -> Result { + let mut file = File::open(path)?; + let file_len = file.metadata()?.len(); + if file_len < FOOTER_SIZE || file_len % VHD_SECTOR != 0 { + return bad("VHD file is truncated or not sector-aligned"); + } + + let footer_offset = file_len - FOOTER_SIZE; + let mut footer = [0; FOOTER_SIZE as usize]; + read_at(&mut file, footer_offset, &mut footer)?; + if &footer[..8] != FOOTER_COOKIE { + return bad("VHD footer cookie is invalid"); + } + verify_checksum(&mut footer, 64, "VHD footer")?; + if be32(&footer, 12) != 0x0001_0000 { + return bad("VHD footer version is unsupported"); + } + + let bytes = be64(&footer, 48); + if bytes == 0 || bytes > MAX_VIRTUAL_SIZE || bytes % VHD_SECTOR != 0 { + return bad("VHD virtual size is invalid or unsupported"); + } + let layout = match be32(&footer, 60) { + 2 => { + if be64(&footer, 16) != u64::MAX || bytes.checked_add(FOOTER_SIZE) != Some(file_len) { + return bad("fixed VHD size or data offset is invalid"); + } + Layout::Fixed + } + 3 => { + let header_offset = be64(&footer, 16); + let (block_size, bitmap_size, blocks) = parse_dynamic(&mut file, header_offset, footer_offset, bytes)?; + Layout::Dynamic { block_size, bitmap_size, blocks } + } + _ => return bad("only fixed and dynamic VHD containers are supported"), + }; + + Ok(Self { file, bytes, sectors: bytes.div_ceil(SECTOR as u64), layout }) + } + + fn read_bytes(&mut self, offset: u64, out: &mut [u8]) -> Result<()> { + let end = + offset.checked_add(out.len() as u64).ok_or_else(|| crate::Error::Format("VHD read overflow".into()))?; + if end > self.bytes { + return bad("read past end of VHD virtual disk"); + } + let file = &mut self.file; + match &self.layout { + Layout::Fixed => read_at(file, offset, out), + Layout::Dynamic { block_size, bitmap_size, blocks } => { + read_dynamic(file, offset, out, *block_size, *bitmap_size, blocks) + } + } + } +} + +impl SectorSource for VhdFile { + fn sectors(&self) -> u64 { + self.sectors + } + + fn byte_len(&self) -> u64 { + self.bytes + } + + fn read_sectors(&mut self, lba: u64, buf: &mut [u8]) -> Result<()> { + if buf.len() % SECTOR != 0 { + return bad("sector read buffer is not 2048-byte aligned"); + } + let start = + lba.checked_mul(SECTOR as u64).ok_or_else(|| crate::Error::Format("VHD read offset overflow".into()))?; + let requested_end = start + .checked_add(buf.len() as u64) + .ok_or_else(|| crate::Error::Format("VHD read length overflow".into()))?; + if requested_end > self.sectors.saturating_mul(SECTOR as u64) { + return bad("read past end of VHD virtual disk"); + } + + buf.fill(0); + if start < self.bytes { + let len = (self.bytes - start).min(buf.len() as u64) as usize; + self.read_bytes(start, &mut buf[..len])?; + } + Ok(()) + } +} + +fn read_at(file: &mut File, offset: u64, out: &mut [u8]) -> Result<()> { + file.seek(SeekFrom::Start(offset))?; + file.read_exact(out)?; + Ok(()) +} + +fn read_dynamic( + file: &mut File, + mut offset: u64, + out: &mut [u8], + block_size: u64, + bitmap_size: u64, + blocks: &[u32], +) -> Result<()> { + let mut done = 0usize; + while done < out.len() { + let block_index = (offset / block_size) as usize; + let block_offset = offset % block_size; + let len = (block_size - block_offset).min((out.len() - done) as u64) as usize; + let entry = + *blocks.get(block_index).ok_or_else(|| crate::Error::Format("VHD block map is incomplete".into()))?; + if entry == u32::MAX { + out[done..done + len].fill(0); + } else { + let block_start = u64::from(entry) * VHD_SECTOR; + let mut bitmap = vec![0; bitmap_size as usize]; + read_at(file, block_start, &mut bitmap)?; + + let end_byte = block_offset + len as u64; + let first_sector = block_offset / VHD_SECTOR; + let end_sector = end_byte.div_ceil(VHD_SECTOR); + let mut sector = first_sector; + while sector < end_sector { + let bit = bitmap[(sector / 8) as usize] & (0x80 >> (sector % 8)) != 0; + let mut run_end = sector + 1; + while run_end < end_sector && (bitmap[(run_end / 8) as usize] & (0x80 >> (run_end % 8)) != 0) == bit { + run_end += 1; + } + let run_start = sector * VHD_SECTOR; + let run_finish = (run_end * VHD_SECTOR).min(end_byte); + let out_start = done + (run_start - block_offset) as usize; + let run_len = (run_finish - run_start) as usize; + if bit { + let data_offset = block_start + .checked_add(bitmap_size) + .and_then(|base| base.checked_add(run_start)) + .ok_or_else(|| crate::Error::Format("VHD block offset overflow".into()))?; + read_at(file, data_offset, &mut out[out_start..out_start + run_len])?; + } else { + out[out_start..out_start + run_len].fill(0); + } + sector = run_end; + } + } + offset = + offset.checked_add(len as u64).ok_or_else(|| crate::Error::Format("VHD read offset overflow".into()))?; + done += len; + } + Ok(()) +} + +fn parse_dynamic(file: &mut File, header_offset: u64, footer_offset: u64, bytes: u64) -> Result<(u64, u64, Vec)> { + let header_end = + header_offset.checked_add(1024).ok_or_else(|| crate::Error::Format("VHD header offset overflow".into()))?; + if header_offset % VHD_SECTOR != 0 || header_offset < VHD_SECTOR || header_end > footer_offset { + return bad("dynamic VHD header is outside the file"); + } + + let mut header = [0; 1024]; + read_at(file, header_offset, &mut header)?; + if &header[..8] != HEADER_COOKIE { + return bad("dynamic VHD header cookie is invalid"); + } + verify_checksum(&mut header, 36, "dynamic VHD header")?; + if be64(&header, 8) != u64::MAX || be32(&header, 24) != 0x0001_0000 { + return bad("dynamic VHD header version or data offset is unsupported"); + } + + let table_offset = be64(&header, 16); + let max_entries = be32(&header, 28); + let block_size = u64::from(be32(&header, 32)); + if max_entries == 0 + || max_entries > MAX_BAT_ENTRIES + || !(VHD_SECTOR..=MAX_BLOCK_SIZE).contains(&block_size) + || !block_size.is_power_of_two() + || block_size % VHD_SECTOR != 0 + { + return bad("dynamic VHD block map or block size is unsupported"); + } + let needed_entries = bytes.div_ceil(block_size); + if needed_entries > u64::from(max_entries) { + return bad("dynamic VHD block map does not cover the virtual disk"); + } + + let table_bytes = u64::from(max_entries) + .checked_mul(4) + .ok_or_else(|| crate::Error::Format("VHD block map size overflow".into()))?; + let table_end = table_offset + .checked_add(table_bytes) + .ok_or_else(|| crate::Error::Format("VHD block map offset overflow".into()))?; + if table_offset % VHD_SECTOR != 0 || table_offset < header_end || table_end > footer_offset { + return bad("dynamic VHD block map is outside the file"); + } + + let needed_bytes = needed_entries + .checked_mul(4) + .and_then(|value| usize::try_from(value).ok()) + .ok_or_else(|| crate::Error::Format("VHD block map is too large".into()))?; + let mut raw_entries = vec![0; needed_bytes]; + read_at(file, table_offset, &mut raw_entries)?; + let blocks: Vec = + raw_entries.chunks_exact(4).map(|entry| u32::from_be_bytes(entry.try_into().unwrap())).collect(); + + let sectors_per_block = block_size / VHD_SECTOR; + let bitmap_size = sectors_per_block.div_ceil(8).div_ceil(VHD_SECTOR) * VHD_SECTOR; + let mut ranges = Vec::new(); + for &entry in &blocks { + if entry == u32::MAX { + continue; + } + let start = u64::from(entry) * VHD_SECTOR; + let end = start + .checked_add(bitmap_size) + .and_then(|value| value.checked_add(block_size)) + .ok_or_else(|| crate::Error::Format("VHD block extent overflow".into()))?; + if start < table_end || end > footer_offset { + return bad("dynamic VHD block extent is outside the data area"); + } + ranges.push((start, end)); + } + ranges.sort_unstable(); + if ranges.windows(2).any(|pair| pair[0].1 > pair[1].0) { + return bad("dynamic VHD blocks overlap"); + } + + Ok((block_size, bitmap_size, blocks)) +} + +fn verify_checksum(block: &mut [u8], offset: usize, name: &str) -> Result<()> { + let expected = be32(block, offset); + block[offset..offset + 4].fill(0); + let sum: u32 = block.iter().map(|byte| u32::from(*byte)).sum(); + if !sum != expected { + return bad(format!("{name} checksum is invalid")); + } + Ok(()) +} + +fn be32(bytes: &[u8], offset: usize) -> u32 { + u32::from_be_bytes(bytes[offset..offset + 4].try_into().unwrap()) +} + +fn be64(bytes: &[u8], offset: usize) -> u64 { + u64::from_be_bytes(bytes[offset..offset + 8].try_into().unwrap()) +} diff --git a/crates/if-image/tests/vhd.rs b/crates/if-image/tests/vhd.rs new file mode 100644 index 0000000..b7a9d12 --- /dev/null +++ b/crates/if-image/tests/vhd.rs @@ -0,0 +1,207 @@ +use if_image::source::SectorSource; +use if_image::vhd::VhdFile; +use std::path::PathBuf; +use std::sync::atomic::{AtomicU64, Ordering}; + +const SECTOR: usize = 2048; +const FOOTER_SIZE: usize = 512; +static NEXT_FILE: AtomicU64 = AtomicU64::new(0); + +struct TempVhd(PathBuf); + +impl TempVhd { + fn new(bytes: &[u8]) -> Self { + let id = NEXT_FILE.fetch_add(1, Ordering::Relaxed); + let path = std::env::temp_dir().join(format!("if-image-vhd-{}-{id}.vhd", std::process::id())); + std::fs::write(&path, bytes).unwrap(); + Self(path) + } + + fn path(&self) -> &std::path::Path { + &self.0 + } +} + +impl Drop for TempVhd { + fn drop(&mut self) { + let _ = std::fs::remove_file(&self.0); + } +} + +fn checksum(block: &mut [u8], offset: usize) { + block[offset..offset + 4].fill(0); + let sum: u32 = block.iter().map(|byte| u32::from(*byte)).sum(); + block[offset..offset + 4].copy_from_slice(&(!sum).to_be_bytes()); +} + +fn footer(virtual_size: u64, disk_type: u32, data_offset: u64) -> [u8; FOOTER_SIZE] { + let mut footer = [0; FOOTER_SIZE]; + footer[..8].copy_from_slice(b"conectix"); + footer[8..12].copy_from_slice(&2u32.to_be_bytes()); + footer[12..16].copy_from_slice(&0x0001_0000u32.to_be_bytes()); + footer[16..24].copy_from_slice(&data_offset.to_be_bytes()); + footer[40..48].copy_from_slice(&virtual_size.to_be_bytes()); + footer[48..56].copy_from_slice(&virtual_size.to_be_bytes()); + footer[60..64].copy_from_slice(&disk_type.to_be_bytes()); + footer[68..84].copy_from_slice(&[0x5a; 16]); + checksum(&mut footer, 64); + footer +} + +fn fixed_vhd(payload: &[u8]) -> Vec { + let mut bytes = payload.to_vec(); + bytes.extend_from_slice(&footer(payload.len() as u64, 2, u64::MAX)); + bytes +} + +fn dynamic_vhd() -> Vec { + let virtual_size = 8192u64; + let mut bytes = vec![0; 2048 + 512 + 4096]; + + let mut header = [0; 1024]; + header[..8].copy_from_slice(b"cxsparse"); + header[8..16].copy_from_slice(&u64::MAX.to_be_bytes()); + header[16..24].copy_from_slice(&1536u64.to_be_bytes()); + header[24..28].copy_from_slice(&0x0001_0000u32.to_be_bytes()); + header[28..32].copy_from_slice(&2u32.to_be_bytes()); + header[32..36].copy_from_slice(&4096u32.to_be_bytes()); + checksum(&mut header, 36); + bytes[512..1536].copy_from_slice(&header); + + bytes[1536..1540].copy_from_slice(&4u32.to_be_bytes()); + bytes[1540..1544].copy_from_slice(&u32::MAX.to_be_bytes()); + bytes[2048] = 0xff; + for (index, byte) in bytes[2560..6656].iter_mut().enumerate() { + *byte = (index % 251) as u8; + } + bytes.extend_from_slice(&footer(virtual_size, 3, 512)); + bytes +} + +#[test] +fn reads_fixed_vhd_payload_as_logical_sectors() { + let payload: Vec = (0..4096).map(|byte| (byte % 239) as u8).collect(); + let file = TempVhd::new(&fixed_vhd(&payload)); + let mut vhd = VhdFile::open(file.path()).unwrap(); + + assert_eq!(vhd.sectors(), 2); + let mut out = vec![0; SECTOR]; + vhd.read_sectors(1, &mut out).unwrap(); + assert_eq!(out, payload[SECTOR..]); +} + +#[test] +fn reports_exact_virtual_size_and_pads_the_last_sector() { + let payload: Vec = (0..4608).map(|byte| (byte % 239) as u8).collect(); + let file = TempVhd::new(&fixed_vhd(&payload)); + let mut vhd = VhdFile::open(file.path()).unwrap(); + + assert_eq!(vhd.byte_len(), payload.len() as u64); + assert_eq!(vhd.sectors(), 3); + let mut out = vec![0xff; SECTOR]; + vhd.read_sectors(2, &mut out).unwrap(); + assert_eq!(&out[..512], &payload[4096..]); + assert!(out[512..].iter().all(|byte| *byte == 0)); + assert!(vhd.read_sectors(3, &mut out).is_err()); +} + +#[test] +fn reads_allocated_and_sparse_dynamic_vhd_blocks() { + let file = TempVhd::new(&dynamic_vhd()); + let mut vhd = VhdFile::open(file.path()).unwrap(); + let mut out = vec![0; 4 * SECTOR]; + + vhd.read_sectors(0, &mut out).unwrap(); + assert!(out[..4096].iter().enumerate().all(|(index, byte)| *byte == (index % 251) as u8)); + assert!(out[4096..].iter().all(|byte| *byte == 0)); +} + +#[test] +fn reads_unallocated_sectors_as_zero_inside_an_allocated_dynamic_block() { + let mut bytes = dynamic_vhd(); + bytes[2048] = 0xf0; + let file = TempVhd::new(&bytes); + let mut vhd = VhdFile::open(file.path()).unwrap(); + let mut out = vec![0xff; 4 * SECTOR]; + + vhd.read_sectors(0, &mut out).unwrap(); + assert!(out[..2048].iter().enumerate().all(|(index, byte)| *byte == (index % 251) as u8)); + assert!(out[2048..].iter().all(|byte| *byte == 0)); +} + +#[test] +fn rejects_a_vhd_with_an_invalid_footer_checksum() { + let mut bytes = fixed_vhd(&vec![0x31; 4096]); + let footer_start = bytes.len() - FOOTER_SIZE; + bytes[footer_start + 10] ^= 1; + let file = TempVhd::new(&bytes); + + assert!(VhdFile::open(file.path()).is_err()); +} + +#[test] +fn rejects_a_truncated_vhd_footer() { + let file = TempVhd::new(&[0; 511]); + + assert!(VhdFile::open(file.path()).is_err()); +} + +#[test] +fn rejects_a_dynamic_vhd_bat_entry_outside_the_file() { + let mut bytes = dynamic_vhd(); + bytes[1536..1540].copy_from_slice(&u32::MAX.wrapping_sub(1).to_be_bytes()); + let file = TempVhd::new(&bytes); + + assert!(VhdFile::open(file.path()).is_err()); +} + +#[test] +fn rejects_overlapping_dynamic_vhd_blocks() { + let mut bytes = dynamic_vhd(); + bytes[1540..1544].copy_from_slice(&4u32.to_be_bytes()); + let file = TempVhd::new(&bytes); + + assert!(VhdFile::open(file.path()).is_err()); +} + +#[test] +fn rejects_a_dynamic_vhd_with_an_invalid_header_checksum() { + let mut bytes = dynamic_vhd(); + bytes[512 + 100] ^= 1; + let file = TempVhd::new(&bytes); + + assert!(VhdFile::open(file.path()).is_err()); +} + +#[test] +fn rejects_a_dynamic_vhd_with_a_zero_block_size() { + let mut bytes = dynamic_vhd(); + bytes[512 + 32..512 + 36].fill(0); + checksum(&mut bytes[512..1536], 36); + let file = TempVhd::new(&bytes); + + assert!(VhdFile::open(file.path()).is_err()); +} + +#[test] +fn rejects_a_dynamic_vhd_with_an_excessive_block_map() { + let mut bytes = dynamic_vhd(); + bytes[512 + 28..512 + 32].copy_from_slice(&1_048_577u32.to_be_bytes()); + checksum(&mut bytes[512..1536], 36); + let file = TempVhd::new(&bytes); + + assert!(VhdFile::open(file.path()).is_err()); +} + +#[test] +fn rejects_differencing_vhds() { + let mut bytes = fixed_vhd(&vec![0x31; 4096]); + let footer_start = bytes.len() - FOOTER_SIZE; + bytes[footer_start + 60..footer_start + 64].copy_from_slice(&4u32.to_be_bytes()); + let mut footer = bytes[footer_start..].to_vec(); + checksum(&mut footer, 64); + bytes[footer_start..].copy_from_slice(&footer); + let file = TempVhd::new(&bytes); + + assert!(VhdFile::open(file.path()).is_err()); +} From 8d54bb7958cf821a7ef0ec674788b19fe22a3798 Mon Sep 17 00:00:00 2001 From: Sean Shaffer Date: Mon, 9 Nov 2026 00:05:17 -0600 Subject: [PATCH 08/12] feat: add safe fixed VHD writing Write 512-byte-aligned virtual disks to a new fixed VHD using validated metadata, exact source lengths, and cancellation-safe temporary output. Preserve existing destinations and round-trip the final partial logical sector. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- crates/if-image/src/lib.rs | 1 + crates/if-image/src/source.rs | 21 ++++- crates/if-image/src/vhd.rs | 4 +- crates/if-image/src/vhd_write.rs | 156 +++++++++++++++++++++++++++++++ crates/if-image/tests/vhd.rs | 60 +++++++++++- 5 files changed, 234 insertions(+), 8 deletions(-) create mode 100644 crates/if-image/src/vhd_write.rs diff --git a/crates/if-image/src/lib.rs b/crates/if-image/src/lib.rs index e632e7c..356a64a 100644 --- a/crates/if-image/src/lib.rs +++ b/crates/if-image/src/lib.rs @@ -26,6 +26,7 @@ pub mod nrg; pub mod source; pub mod udf; pub mod vhd; +pub mod vhd_write; pub mod write; pub mod xiso; diff --git a/crates/if-image/src/source.rs b/crates/if-image/src/source.rs index d09c0d4..936b8cb 100644 --- a/crates/if-image/src/source.rs +++ b/crates/if-image/src/source.rs @@ -78,7 +78,7 @@ pub struct MemSource(pub Vec); impl SectorSource for MemSource { fn sectors(&self) -> u64 { - (self.0.len() / SECTOR) as u64 + self.0.len().div_ceil(SECTOR) as u64 } fn byte_len(&self) -> u64 { @@ -86,12 +86,23 @@ impl SectorSource for MemSource { } fn read_sectors(&mut self, lba: u64, buf: &mut [u8]) -> Result<()> { - let s = lba as usize * SECTOR; - let e = s + buf.len(); - if e > self.0.len() { + if buf.len() % SECTOR != 0 { + return crate::bad("sector read buffer is not 2048-byte aligned"); + } + let count = (buf.len() / SECTOR) as u64; + if lba.checked_add(count).is_none_or(|end| end > self.sectors()) { return crate::bad("read past end of image"); } - buf.copy_from_slice(&self.0[s..e]); + let start = lba + .checked_mul(SECTOR as u64) + .and_then(|offset| usize::try_from(offset).ok()) + .ok_or_else(|| crate::Error::Format("image read offset overflow".into()))?; + start.checked_add(buf.len()).ok_or_else(|| crate::Error::Format("image read length overflow".into()))?; + buf.fill(0); + if start < self.0.len() { + let len = (self.0.len() - start).min(buf.len()); + buf[..len].copy_from_slice(&self.0[start..start + len]); + } Ok(()) } } diff --git a/crates/if-image/src/vhd.rs b/crates/if-image/src/vhd.rs index 4b662cb..2912abe 100644 --- a/crates/if-image/src/vhd.rs +++ b/crates/if-image/src/vhd.rs @@ -8,8 +8,8 @@ use std::path::Path; const VHD_SECTOR: u64 = 512; const FOOTER_SIZE: u64 = 512; -const MAX_VIRTUAL_SIZE: u64 = 2 * 1024 * 1024 * 1024 * 1024; -const MAX_BAT_ENTRIES: u32 = 1_048_576; +pub(crate) const MAX_VIRTUAL_SIZE: u64 = 2 * 1024 * 1024 * 1024 * 1024; +pub(crate) const MAX_BAT_ENTRIES: u32 = 1_048_576; const MAX_BLOCK_SIZE: u64 = 256 * 1024 * 1024; const FOOTER_COOKIE: &[u8; 8] = b"conectix"; const HEADER_COOKIE: &[u8; 8] = b"cxsparse"; diff --git a/crates/if-image/src/vhd_write.rs b/crates/if-image/src/vhd_write.rs new file mode 100644 index 0000000..ff9fa9a --- /dev/null +++ b/crates/if-image/src/vhd_write.rs @@ -0,0 +1,156 @@ +//! Safe writers for fixed and dynamic VHD 1.0 containers. + +use crate::source::SectorSource; +use crate::vhd::MAX_VIRTUAL_SIZE; +use crate::{bad, Error, Result, SECTOR}; +use std::fs::{self, File, OpenOptions}; +use std::io::Write; +use std::path::{Path, PathBuf}; +use std::sync::atomic::{AtomicU64, Ordering}; +use std::time::{SystemTime, UNIX_EPOCH}; + +const VHD_SECTOR: u64 = 512; +const FOOTER_SIZE: u64 = 512; +const COPY_BUFFER: usize = 1 << 20; +static NEXT_UUID: AtomicU64 = AtomicU64::new(0); + +fn validate_source(source: &dyn SectorSource) -> Result { + let bytes = source.byte_len(); + if bytes == 0 || bytes > MAX_VIRTUAL_SIZE || bytes % VHD_SECTOR != 0 { + return bad("VHD source size must be a nonzero, 512-byte-aligned virtual disk no larger than 2 TiB"); + } + if source.sectors() < bytes.div_ceil(SECTOR as u64) { + return bad("image source does not expose all of its virtual bytes"); + } + Ok(bytes) +} + +fn read_source(source: &mut dyn SectorSource, offset: u64, out: &mut [u8]) -> Result<()> { + let sector_count = out.len().div_ceil(SECTOR); + let mut sectors = vec![0; sector_count * SECTOR]; + source.read_sectors(offset / SECTOR as u64, &mut sectors)?; + out.copy_from_slice(§ors[..out.len()]); + Ok(()) +} + +fn part_path(dest: &Path) -> PathBuf { + let mut name = dest.as_os_str().to_owned(); + name.push(".part"); + PathBuf::from(name) +} + +fn publish_new(part: &Path, dest: &Path) -> Result<()> { + if let Err(error) = fs::hard_link(part, dest) { + if let Err(cleanup) = fs::remove_file(part) { + if cleanup.kind() != std::io::ErrorKind::NotFound { + return bad(format!( + "could not publish VHD to {} ({error}) or remove temporary file {} ({cleanup})", + dest.display(), + part.display() + )); + } + } + return Err(error.into()); + } + if let Err(error) = fs::remove_file(part) { + return bad(format!( + "VHD was written to {} but temporary file {} could not be removed ({error})", + dest.display(), + part.display() + )); + } + Ok(()) +} + +fn write_atomic(dest: &Path, write: impl FnOnce(&mut File) -> Result<()>) -> Result<()> { + if dest.exists() { + return bad(format!("destination already exists: {}", dest.display())); + } + let part = part_path(dest); + let mut file = OpenOptions::new().write(true).read(true).create_new(true).open(&part)?; + let result = write(&mut file).and_then(|()| file.sync_all().map_err(Error::from)); + drop(file); + if let Err(error) = result { + if let Err(cleanup) = fs::remove_file(&part) { + if cleanup.kind() != std::io::ErrorKind::NotFound { + return bad(format!( + "could not write VHD to {} ({error}) or remove temporary file {} ({cleanup})", + dest.display(), + part.display() + )); + } + } + return Err(error); + } + publish_new(&part, dest) +} + +fn make_uuid() -> [u8; 16] { + let mut id = [0; 16]; + let ticks = SystemTime::now().duration_since(UNIX_EPOCH).unwrap_or_default().as_nanos(); + id[..8].copy_from_slice(&(ticks as u64).to_be_bytes()); + id[8..12].copy_from_slice(&std::process::id().to_be_bytes()); + id[12..].copy_from_slice(&(NEXT_UUID.fetch_add(1, Ordering::Relaxed) as u32).to_be_bytes()); + id[6] = (id[6] & 0x0f) | 0x40; + id[8] = (id[8] & 0x3f) | 0x80; + id +} + +fn footer(bytes: u64, disk_type: u32, data_offset: u64) -> [u8; FOOTER_SIZE as usize] { + let mut footer = [0; FOOTER_SIZE as usize]; + footer[..8].copy_from_slice(b"conectix"); + footer[8..12].copy_from_slice(&2u32.to_be_bytes()); + footer[12..16].copy_from_slice(&0x0001_0000u32.to_be_bytes()); + footer[16..24].copy_from_slice(&data_offset.to_be_bytes()); + let timestamp = SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap_or_default() + .as_secs() + .saturating_sub(946_684_800) + .min(u64::from(u32::MAX)) as u32; + footer[24..28].copy_from_slice(×tamp.to_be_bytes()); + footer[28..32].copy_from_slice(b"IFND"); + footer[32..36].copy_from_slice(&0x0001_0000u32.to_be_bytes()); + footer[36..40].copy_from_slice(b"Wi2k"); + footer[40..48].copy_from_slice(&bytes.to_be_bytes()); + footer[48..56].copy_from_slice(&bytes.to_be_bytes()); + let sectors = bytes / VHD_SECTOR; + let cylinders = (sectors / (16 * 63)).clamp(1, u64::from(u16::MAX)) as u16; + footer[56..58].copy_from_slice(&cylinders.to_be_bytes()); + footer[58] = 16; + footer[59] = 63; + footer[60..64].copy_from_slice(&disk_type.to_be_bytes()); + footer[68..84].copy_from_slice(&make_uuid()); + set_checksum(&mut footer, 64); + footer +} + +fn set_checksum(block: &mut [u8], offset: usize) { + block[offset..offset + 4].fill(0); + let sum: u32 = block.iter().map(|byte| u32::from(*byte)).sum(); + block[offset..offset + 4].copy_from_slice(&(!sum).to_be_bytes()); +} + +/// Writes a fixed VHD to a new destination. `progress(done, total)` returns false to cancel. +pub fn write_fixed( + source: &mut dyn SectorSource, + dest: &Path, + mut progress: impl FnMut(u64, u64) -> bool, +) -> Result<()> { + let bytes = validate_source(source)?; + write_atomic(dest, |file| { + let mut offset = 0u64; + let mut buffer = vec![0; COPY_BUFFER]; + while offset < bytes { + let len = (bytes - offset).min(buffer.len() as u64) as usize; + read_source(source, offset, &mut buffer[..len])?; + file.write_all(&buffer[..len])?; + offset += len as u64; + if !progress(offset, bytes) { + return Err(Error::Cancelled); + } + } + file.write_all(&footer(bytes, 2, u64::MAX))?; + Ok(()) + }) +} diff --git a/crates/if-image/tests/vhd.rs b/crates/if-image/tests/vhd.rs index b7a9d12..f1284e1 100644 --- a/crates/if-image/tests/vhd.rs +++ b/crates/if-image/tests/vhd.rs @@ -1,5 +1,6 @@ use if_image::source::SectorSource; use if_image::vhd::VhdFile; +use if_image::vhd_write::write_fixed; use std::path::PathBuf; use std::sync::atomic::{AtomicU64, Ordering}; @@ -10,9 +11,14 @@ static NEXT_FILE: AtomicU64 = AtomicU64::new(0); struct TempVhd(PathBuf); impl TempVhd { - fn new(bytes: &[u8]) -> Self { + fn absent() -> Self { let id = NEXT_FILE.fetch_add(1, Ordering::Relaxed); let path = std::env::temp_dir().join(format!("if-image-vhd-{}-{id}.vhd", std::process::id())); + Self(path) + } + + fn new(bytes: &[u8]) -> Self { + let path = Self::absent().0.clone(); std::fs::write(&path, bytes).unwrap(); Self(path) } @@ -25,6 +31,9 @@ impl TempVhd { impl Drop for TempVhd { fn drop(&mut self) { let _ = std::fs::remove_file(&self.0); + let mut part = self.0.as_os_str().to_owned(); + part.push(".part"); + let _ = std::fs::remove_file(PathBuf::from(part)); } } @@ -205,3 +214,52 @@ fn rejects_differencing_vhds() { assert!(VhdFile::open(file.path()).is_err()); } + +#[test] +fn writes_fixed_vhd_that_reopens_with_exact_contents() { + let bytes: Vec = (0..4608).map(|byte| (byte % 239) as u8).collect(); + let mut source = if_image::source::MemSource(bytes.clone()); + let dest = TempVhd::absent(); + + write_fixed(&mut source, dest.path(), |_, _| true).unwrap(); + + let mut vhd = VhdFile::open(dest.path()).unwrap(); + let mut output = vec![0xff; 3 * SECTOR]; + vhd.read_sectors(0, &mut output).unwrap(); + assert_eq!(vhd.byte_len(), bytes.len() as u64); + assert_eq!(&output[..bytes.len()], bytes); + assert!(output[bytes.len()..].iter().all(|byte| *byte == 0)); +} + +#[test] +fn refuses_to_overwrite_a_vhd_destination() { + let dest = TempVhd::new(b"keep this file"); + let mut source = if_image::source::MemSource(vec![0x31; 4096]); + + assert!(write_fixed(&mut source, dest.path(), |_, _| true).is_err()); + + assert_eq!(std::fs::read(dest.path()).unwrap(), b"keep this file"); +} + +#[test] +fn removes_partial_vhd_output_when_cancelled() { + let dest = TempVhd::absent(); + let mut source = if_image::source::MemSource(vec![0x31; 4096]); + + let result = write_fixed(&mut source, dest.path(), |_, _| false); + + assert!(matches!(result, Err(if_image::Error::Cancelled))); + assert!(!dest.path().exists()); + let mut part = dest.path().as_os_str().to_owned(); + part.push(".part"); + assert!(!PathBuf::from(part).exists()); +} + +#[test] +fn rejects_non_512_byte_virtual_sizes_without_creating_output() { + let dest = TempVhd::absent(); + let mut source = if_image::source::MemSource(vec![0; 513]); + + assert!(write_fixed(&mut source, dest.path(), |_, _| true).is_err()); + assert!(!dest.path().exists()); +} From 1047a814bb5a3cca32326c5aea30621f0805efc4 Mon Sep 17 00:00:00 2001 From: Sean Shaffer Date: Mon, 9 Nov 2026 00:08:44 -0600 Subject: [PATCH 09/12] feat: add sparse dynamic VHD writing Write dynamic VHD block maps and allocated blocks from a sector source, leaving zero-only blocks sparse. Reopen through the reader and verify byte accuracy, cancellation cleanup, and source-error cleanup. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- crates/if-image/src/vhd_write.rs | 113 +++++++++++++++++++++++++++---- crates/if-image/tests/vhd.rs | 70 ++++++++++++++++++- 2 files changed, 170 insertions(+), 13 deletions(-) diff --git a/crates/if-image/src/vhd_write.rs b/crates/if-image/src/vhd_write.rs index ff9fa9a..727bfc4 100644 --- a/crates/if-image/src/vhd_write.rs +++ b/crates/if-image/src/vhd_write.rs @@ -1,16 +1,19 @@ //! Safe writers for fixed and dynamic VHD 1.0 containers. use crate::source::SectorSource; -use crate::vhd::MAX_VIRTUAL_SIZE; +use crate::vhd::{MAX_BAT_ENTRIES, MAX_VIRTUAL_SIZE}; use crate::{bad, Error, Result, SECTOR}; use std::fs::{self, File, OpenOptions}; -use std::io::Write; +use std::io::{Seek, SeekFrom, Write}; use std::path::{Path, PathBuf}; use std::sync::atomic::{AtomicU64, Ordering}; use std::time::{SystemTime, UNIX_EPOCH}; const VHD_SECTOR: u64 = 512; const FOOTER_SIZE: u64 = 512; +const HEADER_OFFSET: u64 = 512; +const BAT_OFFSET: u64 = 1536; +const BLOCK_SIZE: u64 = 2 * 1024 * 1024; const COPY_BUFFER: usize = 1 << 20; static NEXT_UUID: AtomicU64 = AtomicU64::new(0); @@ -39,18 +42,22 @@ fn part_path(dest: &Path) -> PathBuf { PathBuf::from(name) } +fn cleanup_after_publish_failure(part: &Path, dest: &Path, error: std::io::Error) -> Result<()> { + if let Err(cleanup) = fs::remove_file(part) { + if cleanup.kind() != std::io::ErrorKind::NotFound { + return bad(format!( + "could not publish VHD to {} ({error}) or remove temporary file {} ({cleanup})", + dest.display(), + part.display() + )); + } + } + Err(error.into()) +} + fn publish_new(part: &Path, dest: &Path) -> Result<()> { if let Err(error) = fs::hard_link(part, dest) { - if let Err(cleanup) = fs::remove_file(part) { - if cleanup.kind() != std::io::ErrorKind::NotFound { - return bad(format!( - "could not publish VHD to {} ({error}) or remove temporary file {} ({cleanup})", - dest.display(), - part.display() - )); - } - } - return Err(error.into()); + return cleanup_after_publish_failure(part, dest, error); } if let Err(error) = fs::remove_file(part) { return bad(format!( @@ -154,3 +161,85 @@ pub fn write_fixed( Ok(()) }) } + +/// Writes a sparse dynamic VHD to a new destination. `progress(done, total)` returns false to cancel. +pub fn write_dynamic( + source: &mut dyn SectorSource, + dest: &Path, + mut progress: impl FnMut(u64, u64) -> bool, +) -> Result<()> { + let bytes = validate_source(source)?; + let entries = bytes.div_ceil(BLOCK_SIZE); + if entries == 0 || entries > u64::from(MAX_BAT_ENTRIES) { + return bad("VHD source requires an unsupported number of dynamic blocks"); + } + let table_bytes = entries.checked_mul(4).ok_or_else(|| Error::Format("VHD block map size overflow".into()))?; + let table_padded = table_bytes + .checked_add(VHD_SECTOR - 1) + .map(|size| size / VHD_SECTOR * VHD_SECTOR) + .ok_or_else(|| Error::Format("VHD block map size overflow".into()))?; + let table_len = usize::try_from(table_bytes).map_err(|_| Error::Format("VHD block map is too large".into()))?; + let pad_len = usize::try_from(table_padded - table_bytes) + .map_err(|_| Error::Format("VHD block map padding is too large".into()))?; + let sectors_per_block = BLOCK_SIZE / VHD_SECTOR; + let bitmap_size = sectors_per_block.div_ceil(8).div_ceil(VHD_SECTOR) * VHD_SECTOR; + let bitmap_len = usize::try_from(bitmap_size).map_err(|_| Error::Format("VHD block bitmap is too large".into()))?; + + write_atomic(dest, |file| { + file.write_all(&[0; HEADER_OFFSET as usize])?; + + let mut header = [0; 1024]; + header[..8].copy_from_slice(b"cxsparse"); + header[8..16].copy_from_slice(&u64::MAX.to_be_bytes()); + header[16..24].copy_from_slice(&BAT_OFFSET.to_be_bytes()); + header[24..28].copy_from_slice(&0x0001_0000u32.to_be_bytes()); + header[28..32].copy_from_slice(&(entries as u32).to_be_bytes()); + header[32..36].copy_from_slice(&(BLOCK_SIZE as u32).to_be_bytes()); + set_checksum(&mut header, 36); + file.seek(SeekFrom::Start(HEADER_OFFSET))?; + file.write_all(&header)?; + file.seek(SeekFrom::Start(BAT_OFFSET))?; + file.write_all(&vec![0xff; table_len])?; + if pad_len != 0 { + file.write_all(&vec![0; pad_len])?; + } + + let mut offset = 0u64; + let mut block_index = 0u64; + while offset < bytes { + let len = (bytes - offset).min(BLOCK_SIZE) as usize; + let mut block = vec![0; BLOCK_SIZE as usize]; + read_source(source, offset, &mut block[..len])?; + + let mut bitmap = vec![0; bitmap_len]; + let mut allocated = false; + for sector in 0..sectors_per_block as usize { + let start = sector * VHD_SECTOR as usize; + if block[start..start + VHD_SECTOR as usize].iter().any(|byte| *byte != 0) { + bitmap[sector / 8] |= 0x80 >> (sector % 8); + allocated = true; + } + } + + if allocated { + let block_offset = file.stream_position()?; + let table_sector = u32::try_from(block_offset / VHD_SECTOR) + .map_err(|_| Error::Format("dynamic VHD exceeds the BAT sector-offset limit".into()))?; + file.write_all(&bitmap)?; + file.write_all(&block)?; + let end = file.stream_position()?; + file.seek(SeekFrom::Start(BAT_OFFSET + block_index * 4))?; + file.write_all(&table_sector.to_be_bytes())?; + file.seek(SeekFrom::Start(end))?; + } + + offset += len as u64; + block_index += 1; + if !progress(offset, bytes) { + return Err(Error::Cancelled); + } + } + file.write_all(&footer(bytes, 3, HEADER_OFFSET))?; + Ok(()) + }) +} diff --git a/crates/if-image/tests/vhd.rs b/crates/if-image/tests/vhd.rs index f1284e1..53a8709 100644 --- a/crates/if-image/tests/vhd.rs +++ b/crates/if-image/tests/vhd.rs @@ -1,6 +1,6 @@ use if_image::source::SectorSource; use if_image::vhd::VhdFile; -use if_image::vhd_write::write_fixed; +use if_image::vhd_write::{write_dynamic, write_fixed}; use std::path::PathBuf; use std::sync::atomic::{AtomicU64, Ordering}; @@ -37,6 +37,28 @@ impl Drop for TempVhd { } } +struct FailingSource { + sectors: u64, +} + +impl SectorSource for FailingSource { + fn sectors(&self) -> u64 { + self.sectors + } + + fn byte_len(&self) -> u64 { + self.sectors * SECTOR as u64 + } + + fn read_sectors(&mut self, lba: u64, buf: &mut [u8]) -> if_image::Result<()> { + if lba != 0 { + return Err(if_image::Error::Format("injected source read error".into())); + } + buf.fill(0x44); + Ok(()) + } +} + fn checksum(block: &mut [u8], offset: usize) { block[offset..offset + 4].fill(0); let sum: u32 = block.iter().map(|byte| u32::from(*byte)).sum(); @@ -231,6 +253,26 @@ fn writes_fixed_vhd_that_reopens_with_exact_contents() { assert!(output[bytes.len()..].iter().all(|byte| *byte == 0)); } +#[test] +fn writes_sparse_dynamic_vhd_that_reopens_with_exact_contents() { + let mut bytes = vec![0; 2 * 1024 * 1024 + 4096]; + bytes[512..1024].fill(0x5a); + let mut source = if_image::source::MemSource(bytes.clone()); + let dest = TempVhd::absent(); + + write_dynamic(&mut source, dest.path(), |_, _| true).unwrap(); + + let mut vhd = VhdFile::open(dest.path()).unwrap(); + let mut output = vec![0xff; bytes.len().div_ceil(SECTOR) * SECTOR]; + vhd.read_sectors(0, &mut output).unwrap(); + assert_eq!(vhd.byte_len(), bytes.len() as u64); + assert_eq!(&output[..bytes.len()], bytes); + assert!(output[bytes.len()..].iter().all(|byte| *byte == 0)); + let encoded = std::fs::read(dest.path()).unwrap(); + assert_ne!(&encoded[1536..1540], &u32::MAX.to_be_bytes()); + assert_eq!(&encoded[1540..1544], &u32::MAX.to_be_bytes()); +} + #[test] fn refuses_to_overwrite_a_vhd_destination() { let dest = TempVhd::new(b"keep this file"); @@ -255,6 +297,32 @@ fn removes_partial_vhd_output_when_cancelled() { assert!(!PathBuf::from(part).exists()); } +#[test] +fn removes_partial_dynamic_vhd_when_cancelled() { + let dest = TempVhd::absent(); + let mut source = if_image::source::MemSource(vec![0x31; 2 * 1024 * 1024]); + + let result = write_dynamic(&mut source, dest.path(), |_, _| false); + + assert!(matches!(result, Err(if_image::Error::Cancelled))); + assert!(!dest.path().exists()); + let mut part = dest.path().as_os_str().to_owned(); + part.push(".part"); + assert!(!PathBuf::from(part).exists()); +} + +#[test] +fn removes_partial_vhd_output_when_the_source_fails() { + let dest = TempVhd::absent(); + let mut source = FailingSource { sectors: 513 }; + + assert!(write_fixed(&mut source, dest.path(), |_, _| true).is_err()); + assert!(!dest.path().exists()); + let mut part = dest.path().as_os_str().to_owned(); + part.push(".part"); + assert!(!PathBuf::from(part).exists()); +} + #[test] fn rejects_non_512_byte_virtual_sizes_without_creating_output() { let dest = TempVhd::absent(); From 81d04665cf0e432076509387fabc1529c4f4de4e Mon Sep 17 00:00:00 2001 From: Sean Shaffer Date: Mon, 9 Nov 2026 00:24:05 -0600 Subject: [PATCH 10/12] feat: add USB-to-image task window Expose a source-read-only disk imaging flow for physical disks and image files, including optional limits, progress, and cancellable file reads. Keep physical disk reads under the existing helper path. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- crates/isofoundry/src/cli_usb.rs | 49 ++++ crates/isofoundry/src/layout.rs | 17 +- crates/isofoundry/src/win/gui.rs | 13 +- crates/isofoundry/src/win/tasks/mod.rs | 1 + crates/isofoundry/src/win/tasks/usbread.rs | 309 +++++++++++++++++++++ crates/usbwriter/src/raw.rs | 17 ++ 6 files changed, 397 insertions(+), 9 deletions(-) create mode 100644 crates/isofoundry/src/win/tasks/usbread.rs diff --git a/crates/isofoundry/src/cli_usb.rs b/crates/isofoundry/src/cli_usb.rs index f2654b4..708eca8 100644 --- a/crates/isofoundry/src/cli_usb.rs +++ b/crates/isofoundry/src/cli_usb.rs @@ -227,6 +227,11 @@ pub(crate) fn run_on_file_with( Ok((0, n, notes)) } UsbJobKind::ReadImage { out, limit_bytes } => { + if let (Ok(source), Ok(destination)) = (path.canonicalize(), Path::new(out).canonicalize()) { + if source == destination { + return fail(Exit::Usage, "the output image must differ from the disk-image source"); + } + } let dev = FileDevice::open(path, false).map_err(map)?; let n = usbwriter::raw::device_to_image(&dev, Path::new(out), *limit_bytes, progress, cancel).map_err(map)?; @@ -304,3 +309,47 @@ fn windows_to_usb(tail: &[String], g: &Global, platform: &dyn Platform, o: &mut let (files, bytes, notes) = execute(&target, &kind, g, platform, o)?; Ok(report(o, "windows-to-usb", files, bytes, ¬es)) } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn usb_read_from_disk_image_honors_the_optional_limit() { + let dir = std::env::temp_dir().join(format!("isofoundry-usb-read-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let source = dir.join("source.img"); + let output = dir.join("backup.img"); + std::fs::write(&source, b"0123456789abcdef").unwrap(); + let kind = UsbJobKind::ReadImage { out: output.to_string_lossy().into_owned(), limit_bytes: Some(6) }; + let mut stdout = std::io::sink(); + let mut stderr = std::io::sink(); + let mut out = Out { out: &mut stdout, err: &mut stderr, json: false }; + + let (_, bytes, _) = + run_on_file_with(&source, &kind, &mut out, &Progress::default(), &CancelToken::new()).unwrap(); + + assert_eq!(bytes, 6); + assert_eq!(std::fs::read(&output).unwrap(), b"012345"); + let _ = std::fs::remove_dir_all(&dir); + } + + #[test] + fn usb_read_does_not_overwrite_its_disk_image_source() { + let dir = std::env::temp_dir().join(format!("isofoundry-usb-read-source-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let source = dir.join("source.img"); + let original = b"keep this disk image intact"; + std::fs::write(&source, original).unwrap(); + let kind = UsbJobKind::ReadImage { out: source.to_string_lossy().into_owned(), limit_bytes: Some(6) }; + let mut stdout = std::io::sink(); + let mut stderr = std::io::sink(); + let mut out = Out { out: &mut stdout, err: &mut stderr, json: false }; + + assert!(run_on_file_with(&source, &kind, &mut out, &Progress::default(), &CancelToken::new()).is_err()); + assert_eq!(std::fs::read(&source).unwrap(), original); + let _ = std::fs::remove_dir_all(&dir); + } +} diff --git a/crates/isofoundry/src/layout.rs b/crates/isofoundry/src/layout.rs index 5c6ef9e..a8b6c07 100644 --- a/crates/isofoundry/src/layout.rs +++ b/crates/isofoundry/src/layout.rs @@ -118,11 +118,12 @@ pub fn compute(client_w: i32, client_h: i32, dpi: u32) -> Layout { let mut cells = Vec::new(); for (r, row) in GRID.iter().enumerate() { + let span_row = row[0].is_some() && row[1].is_none(); for (c, slot) in row.iter().enumerate() { let Some(task) = slot else { continue }; - let left = inner_l + c as i32 * (col_w + col_gap); + let left = if span_row { inner_l } else { inner_l + c as i32 * (col_w + col_gap) }; // The right column absorbs the odd pixel so both edges line up. - let width = if c == 1 { inner_r - left } else { col_w }; + let width = if span_row || c == 1 { inner_r - left } else { col_w }; let top = first + r as i32 * row_step; let cell = if task.variants.is_empty() { Cell { row: r, col: c, main: Rect::new(left, top, width, row_h), arrow: None } @@ -199,6 +200,18 @@ mod tests { assert!(big.footer.height() > small.footer.height()); } + #[test] + fn lone_bottom_left_task_stretches_across_the_grid() { + let layout = compute(816, 640, 96); + let bottom = layout.cells.iter().find(|cell| cell.row == GRID.len() - 1).unwrap(); + let inner_left = layout.group.left + px(GROUP_PAD_X, 96); + let inner_right = layout.group.right - px(GROUP_PAD_X, 96); + + assert_eq!(bottom.col, 0); + assert_eq!(bottom.main.left, inner_left); + assert_eq!(bottom.arrow.unwrap().right, inner_right); + } + #[test] fn below_minimum_is_clamped() { let l = compute(10, 10, 96); diff --git a/crates/isofoundry/src/win/gui.rs b/crates/isofoundry/src/win/gui.rs index 34bb73e..0611125 100644 --- a/crates/isofoundry/src/win/gui.rs +++ b/crates/isofoundry/src/win/gui.rs @@ -958,9 +958,9 @@ unsafe extern "system" fn button_subclass( enum Action { None, - Info(String), NotYet(String, &'static str, &'static str), OpenTask(&'static str), + OpenUsbRead, VariantMenu(usize), Exit, OpenData, @@ -1071,9 +1071,7 @@ fn on_command(hwnd: HWND, id: u16) { x if x >= ID_VARIANT => match variant_list().get((x - ID_VARIANT) as usize) { Some((_, v)) if v.id == "backup-restore" => Action::OpenTask("backup-restore"), Some((_, v)) if v.id == "usb-image-to-usb" => Action::OpenTask("bootable-usb"), - Some((_, v)) if v.id == "usb-usb-to-image" => Action::Info( - "Copy a USB stick to an image file from the command line:\n\nisofoundry usb read --disk N --out FILE\n\nA window for it is planned after 1.0; the copy itself works today.".to_string(), - ), + Some((_, v)) if v.id == "usb-usb-to-image" => Action::OpenUsbRead, Some((_, v)) => Action::NotYet( strings::variant_label(v.id).trim_end_matches('…').to_string(), v.milestone.label(), @@ -1089,9 +1087,6 @@ fn on_command(hwnd: HWND, id: u16) { fn run_action(hwnd: HWND, action: Action) { match action { Action::None => {} - Action::Info(text) => { - message(hwnd, &text, MB_ICONINFORMATION); - } Action::Exit => { // SAFETY: closing our own window. unsafe { PostMessageW(hwnd, WM_CLOSE, 0, 0) }; @@ -1128,6 +1123,10 @@ fn run_action(hwnd: HWND, action: Action) { _ => {} } } + Action::OpenUsbRead => { + with_app(|a| a.log.info("task opened: Save USB drive to image file")); + super::tasks::usbread::open(hwnd); + } Action::VariantMenu(i) => variant_menu(hwnd, i), Action::OpenData => { let root = with_app(|a| a.dirs.root.clone()); diff --git a/crates/isofoundry/src/win/tasks/mod.rs b/crates/isofoundry/src/win/tasks/mod.rs index 2b3bb86..b12f28b 100644 --- a/crates/isofoundry/src/win/tasks/mod.rs +++ b/crates/isofoundry/src/win/tasks/mod.rs @@ -21,5 +21,6 @@ pub mod erasewin; pub mod recoverwin; pub mod ripaudio; pub mod surfacetest; +pub mod usbread; pub mod usbtools; pub mod winusb; diff --git a/crates/isofoundry/src/win/tasks/usbread.rs b/crates/isofoundry/src/win/tasks/usbread.rs new file mode 100644 index 0000000..8c7ab68 --- /dev/null +++ b/crates/isofoundry/src/win/tasks/usbread.rs @@ -0,0 +1,309 @@ +//! Save a USB drive to an image file (M2). The source is read-only; +//! physical disks are read through the elevated helper, while disk-image +//! files use the same in-process route as the CLI. + +use super::super::gui; +use super::super::taskwin::*; +use super::usbtools::*; +use if_core::ipc::UsbJobKind; +use std::sync::Arc; +use windows_sys::Win32::Foundation::{HWND, LPARAM, LRESULT, WPARAM}; +use windows_sys::Win32::UI::WindowsAndMessaging::*; + +const CLASS: &str = "IsoFoundryTaskUsbRead"; + +const ID_OUTPUT: u16 = 1; +const ID_BROWSE: u16 = 2; +const ID_LIMIT: u16 = 3; +const ID_LIST: u16 = 4; +const ID_REFRESH: u16 = 5; +const ID_FILE: u16 = 6; +const ID_READ: u16 = 7; +const ID_CANCEL: u16 = 8; +const ID_OUTLOG: u16 = 9; + +const EN_CHANGE: u32 = 0x0300; +const LBN_SELCHANGE: u32 = 1; + +struct State { + output: HWND, + limit: HWND, + list: HWND, + read: HWND, + cancel: HWND, + bar: HWND, + status: HWND, + outlog: HWND, + pane: UsbPane, + shared: Option>, + running: bool, + cancellable: bool, + close_when_done: bool, +} + +pub fn open(owner: HWND) { + register_class(CLASS, Some(wndproc)); + let state = Box::new(State { + output: std::ptr::null_mut(), + limit: std::ptr::null_mut(), + list: std::ptr::null_mut(), + read: std::ptr::null_mut(), + cancel: std::ptr::null_mut(), + bar: std::ptr::null_mut(), + status: std::ptr::null_mut(), + outlog: std::ptr::null_mut(), + pane: UsbPane::default(), + shared: None, + running: false, + cancellable: false, + close_when_done: false, + }); + create_window(owner, CLASS, "Save USB drive to image file", 664, 500, Box::into_raw(state).cast()); +} + +fn st(hwnd: HWND) -> &'static mut State { + // SAFETY: our pointer from WM_NCCREATE; window thread only. + unsafe { &mut *state_ptr::(hwnd) } +} + +fn parsed_limit(text: &str) -> Result, &'static str> { + let text = text.trim(); + if text.is_empty() { + return Ok(None); + } + let bytes = text.parse::().map_err(|_| "Read limit must be a positive number of bytes.")?; + if bytes == 0 { + return Err("Read limit must be greater than zero."); + } + Ok(Some(bytes)) +} + +fn refresh_enabled(s: &mut State) { + if s.running { + return; + } + let has_output = !get_text(s.output).trim().is_empty(); + let has_source = s.pane.selection(s.list).is_some(); + let valid_limit = parsed_limit(&get_text(s.limit)).is_ok(); + set_enabled(s.read, has_output && has_source && valid_limit); +} + +fn start(hwnd: HWND) { + let s = st(hwnd); + if s.running { + return; + } + let output = get_text(s.output); + if output.trim().is_empty() { + append_line(s.outlog, "Choose an output image file first."); + return; + } + let limit_bytes = match parsed_limit(&get_text(s.limit)) { + Ok(limit) => limit, + Err(message) => { + append_line(s.outlog, message); + return; + } + }; + let Some(choice) = s.pane.selection(s.list) else { + append_line(s.outlog, "Pick a source disk (Refresh disks, or a disk-image file)."); + return; + }; + + let cancellable = matches!(&choice, Choice::File(_, _)); + let kind = UsbJobKind::ReadImage { out: output, limit_bytes }; + let shared = JobShared::new(); + s.shared = Some(shared.clone()); + s.running = true; + s.cancellable = cancellable; + set_enabled(s.read, false); + set_enabled(s.cancel, cancellable); + progress_show(s.bar, None); + set_text( + s.status, + if cancellable { + "Reading disk-image file…" + } else { + "Reading physical disk… cancellation is unavailable while the helper is running." + }, + ); + append_line(s.outlog, "Reading source; its contents will not be changed."); + run_job(hwnd, &shared, move |sh| { + let (_, bytes, notes) = run_choice(&choice, kind, &sh)?; + let mut msg = format!("Done: {bytes} bytes saved."); + for note in notes { + msg.push_str(&format!("\nNote: {note}")); + } + Ok(msg) + }); +} + +unsafe extern "system" fn wndproc(hwnd: HWND, msg: u32, wp: WPARAM, lp: LPARAM) -> LRESULT { + match msg { + WM_NCCREATE => { + attach::(hwnd, lp); + DefWindowProcW(hwnd, msg, wp, lp) + } + WM_CREATE => { + let s = st(hwnd); + label(hwnd, 12, 14, 640, "Source disk (read-only):"); + s.list = listbox(hwnd, ID_LIST, 12, 36, 640, 92); + button(hwnd, ID_REFRESH, 12, 136, 200, "Refresh disks (admin)…"); + button(hwnd, ID_FILE, 220, 136, 230, "Read from disk-image file…"); + label(hwnd, 12, 172, 90, "Output image:"); + s.output = edit(hwnd, ID_OUTPUT, 108, 170, 420, "", false); + button(hwnd, ID_BROWSE, 536, 169, 112, "Browse…"); + label(hwnd, 12, 206, 640, "Read limit in bytes (optional; blank reads the whole source):"); + s.limit = edit(hwnd, ID_LIMIT, 12, 228, 220, "", false); + label(hwnd, 244, 230, 400, "Leave blank to create a full-size image."); + s.bar = progress_bar(hwnd, 12, 270, 640); + s.status = label( + hwnd, + 12, + 292, + 640, + "The source is read-only. Disk-image reads can be cancelled; physical-disk reads run to completion.", + ); + s.outlog = multiline(hwnd, ID_OUTLOG, 12, 318, 640, 96); + s.read = button(hwnd, ID_READ, 428, 426, 104, "Read"); + s.cancel = button(hwnd, ID_CANCEL, 540, 426, 108, "Cancel"); + set_enabled(s.cancel, false); + set_enabled(s.read, false); + progress_show(s.bar, Some(0.0)); + 0 + } + WM_COMMAND => { + let id = (wp & 0xFFFF) as u16; + let code = (wp >> 16) as u32; + match id { + ID_BROWSE => { + if let Some(path) = pick_save( + hwnd, + "Save USB image", + "Disk images|*.img;*.bin|All files|*.*", + "img", + "usb-backup.img", + ) { + set_text(st(hwnd).output, &path.to_string_lossy()); + } + refresh_enabled(st(hwnd)); + } + ID_REFRESH => { + let outlog = st(hwnd).outlog; + append_line(outlog, "Asking the helper for the disk list (approve the admin prompt)…"); + match refresh_disks() { + Ok(disks) => { + let s = st(hwnd); + s.pane.disks = disks; + s.pane.fill(s.list); + append_line(s.outlog, "Disk list updated."); + } + Err(message) => append_line(st(hwnd).outlog, &format!("Could not list disks: {message}")), + } + refresh_enabled(st(hwnd)); + } + ID_FILE => { + if let Some(path) = + pick_open(hwnd, "Source disk image", "Disk images|*.img;*.vhdx;*.vhd|All files|*.*") + { + let path = path.to_string_lossy().into_owned(); + match file_info(&path) { + Ok(info) => { + let s = st(hwnd); + let index = s.pane.disks.len(); + s.pane.file = Some((path, info)); + s.pane.fill(s.list); + // SAFETY: our own list box on the window thread. + unsafe { select_row(s.list, index) }; + } + Err(message) => append_line(st(hwnd).outlog, &message), + } + } + refresh_enabled(st(hwnd)); + } + ID_READ => start(hwnd), + ID_CANCEL => { + if let Some(shared) = &st(hwnd).shared { + shared.cancel.cancel(); + } + } + ID_LIST | ID_OUTPUT | ID_LIMIT if code == LBN_SELCHANGE || code == EN_CHANGE => { + refresh_enabled(st(hwnd)); + } + _ => {} + } + 0 + } + WM_TIMER if wp == TIMER_JOB => { + let s = st(hwnd); + if let Some(shared) = &s.shared { + let (done, total) = shared.snapshot(); + progress_show(s.bar, shared.fraction()); + set_text(s.status, &progress_text(done, total)); + } + 0 + } + WM_JOB_DONE => { + stop_timer(hwnd); + let s = st(hwnd); + s.running = false; + s.cancellable = false; + set_enabled(s.cancel, false); + progress_show(s.bar, Some(1.0)); + if let Some(shared) = &s.shared { + if let Some(result) = shared.take_result() { + match result { + Ok(text) => { + for line in text.lines() { + append_line(s.outlog, line); + } + set_text(s.status, "Done."); + gui::note_log("usb-read-image: done", false); + } + Err(message) => { + append_line(s.outlog, &format!("Failed: {message}")); + set_text(s.status, "Failed."); + gui::note_log(&format!("usb-read-image: {message}"), true); + } + } + } + } + refresh_enabled(s); + if s.close_when_done { + // SAFETY: closing our own window after the job ended. + unsafe { DestroyWindow(hwnd) }; + return 0; + } + 0 + } + WM_CLOSE => { + let s = st(hwnd); + if s.running { + if s.cancellable { + if let Some(shared) = &s.shared { + shared.cancel.cancel(); + } + set_text(s.status, "Cancelling…"); + } else { + set_text(s.status, "Waiting for the physical disk read to finish before closing…"); + } + s.close_when_done = true; + 0 + } else { + // SAFETY: closing our own window. + unsafe { DestroyWindow(hwnd) }; + 0 + } + } + WM_NCDESTROY => { + let pointer = state_ptr::(hwnd); + if !pointer.is_null() { + // SAFETY: created with Box::into_raw in `open`. + unsafe { drop(Box::from_raw(pointer)) }; + } + DefWindowProcW(hwnd, msg, wp, lp) + } + WM_CTLCOLOREDIT | WM_CTLCOLORSTATIC | WM_CTLCOLORLISTBOX | WM_CTLCOLORBTN => ctl_color(wp), + _ => DefWindowProcW(hwnd, msg, wp, lp), + } +} diff --git a/crates/usbwriter/src/raw.rs b/crates/usbwriter/src/raw.rs index a680844..60b6704 100644 --- a/crates/usbwriter/src/raw.rs +++ b/crates/usbwriter/src/raw.rs @@ -142,6 +142,23 @@ mod tests { let _ = std::fs::remove_dir_all(&dir); } + #[test] + fn device_to_image_replaces_an_existing_output() { + let dir = std::env::temp_dir().join(format!("usbw-replace-{}", std::process::id())); + std::fs::create_dir_all(&dir).unwrap(); + let device_path = dir.join("device.img"); + let mut dev = FileDevice::create(&device_path, 4096).unwrap(); + dev.write_at(0, b"new image").unwrap(); + + let out = dir.join("backup.img"); + std::fs::write(&out, b"old image").unwrap(); + device_to_image(&dev, &out, Some(9), &Progress::default(), &CancelToken::new()).unwrap(); + + assert_eq!(std::fs::read(&out).unwrap(), b"new image"); + assert!(!dir.join("backup.img.part").exists()); + let _ = std::fs::remove_dir_all(&dir); + } + #[test] fn image_too_large_refused() { let dir = std::env::temp_dir().join(format!("usbw-raw2-{}", std::process::id())); From 5a6723aeaff05574c08b647da59b44cb1953fab0 Mon Sep 17 00:00:00 2001 From: Sean Shaffer Date: Mon, 9 Nov 2026 00:25:14 -0600 Subject: [PATCH 11/12] feat: add FAT and VHD image workflows Make raw FAT12/16/32 and supported fixed/dynamic VHD images usable through the shared browse, extract, and edit paths. Keep edits isolated to new output images and document preserved metadata and unsupported layouts. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- CAPABILITY-MAP.md | 24 + CHANGELOG.md | 2 + README.md | 12 +- SPEC-image-foundation.md | 85 +++ crates/if-image/src/container.rs | 4 + crates/if-image/src/edit.rs | 158 ++++- crates/if-image/src/fat.rs | 85 ++- crates/if-image/src/fat_write.rs | 674 ++++++++++++++++++++++ crates/if-image/src/lib.rs | 1 + crates/if-image/src/vhd.rs | 4 + crates/if-image/src/write/mod.rs | 2 +- crates/if-image/tests/fat.rs | 206 ++++++- crates/isofoundry/src/cli_image.rs | 110 +++- crates/isofoundry/src/win/tasks/browse.rs | 2 +- crates/isofoundry/src/win/tasks/edit.rs | 21 +- docs/ARCHITECTURE.md | 6 +- docs/LIMITATIONS.md | 6 +- docs/PARITY.md | 2 +- docs/roadmap.md | 2 +- tasks/plan.md | 121 ++-- tasks/todo.md | 130 +++-- 21 files changed, 1514 insertions(+), 143 deletions(-) create mode 100644 CAPABILITY-MAP.md create mode 100644 SPEC-image-foundation.md create mode 100644 crates/if-image/src/fat_write.rs diff --git a/CAPABILITY-MAP.md b/CAPABILITY-MAP.md new file mode 100644 index 0000000..b876241 --- /dev/null +++ b/CAPABILITY-MAP.md @@ -0,0 +1,24 @@ +# Capability Map: IsoFoundry + +This map expands the existing 1.x roadmap to cover the user's disc, image, archive, and USB feature catalog. It is an index and build-order proposal, not a promise that every proprietary format, device, or hardware workflow can be supported. Existing features and in-progress work remain in place. + +| Module id | Responsibility | Depends on | +|---|---|---| +| `image-foundation` | Shared image I/O, filesystem/container support, safe editing, integrity primitives, and VHD support. Builds on ISO/UDF, XISO, and FAT work already in the repository. | — | +| `image-operations` | Create images from files or readable media; browse and extract; edit; convert; compare and verify; sector-level inspection/editing where safe and format-defined. | `image-foundation` | +| `archive-formats` | Add practical compressed, split, encrypted, and additional image/container formats with explicit capability and compatibility reporting. | `image-foundation` | +| `usb-creation` | Format and create bootable USB media, BIOS/UEFI layouts, multiboot menus, write verification, and protections against selecting system or otherwise unsafe disks. | `image-foundation` | +| `optical-media` | Read and write supported optical media, including data-disc authoring, image-to-disc, disc-to-image, duplication, rewritable-media erase, and applicable write/media diagnostics. | `image-foundation` | +| `media-conversion` | Extract audio from unprotected media and convert supported user-provided or readable audio/video content using legally distributable codecs. | `image-foundation`, `optical-media` | +| `virtual-mounting` | Mount, browse, and unmount supported images through operating-system virtual-drive facilities, including multiple mounts where the OS supports them. | `image-foundation` | +| `workflow-surfaces` | Consistent GUI and CLI workflows, scripting and batch queues, portable operation, localization (including Spanish), and user-facing capability/limitation documentation. | `image-operations`, `archive-formats`, `usb-creation`, `optical-media`, `media-conversion`, `virtual-mounting` | + +Build order: `image-foundation` → `image-operations`, `archive-formats`, `usb-creation`, `optical-media`, and `virtual-mounting` → `media-conversion` → `workflow-surfaces`. The existing FAT editor and USB-to-image work remain part of the current progress; the previously approved VHD, multiboot USB, and Spanish UI roadmap work is retained. + +## Boundaries and delivery rules + +- Do not implement copy-protection circumvention or claim support for encrypted commercial media. +- Treat formatting, disc erasure, and raw writes as destructive operations: require clear target identification, proportionate confirmation, cancellation/error handling, and safe output behavior. +- Hardware testing is not a completion gate. Verify with fixtures, simulated/image-backed devices, and software integration tests; document hardware-dependent behavior as unverified where appropriate. +- Prefer formats and operations with reliable specifications, existing project architecture, and redistributable dependencies. Report unsupported formats or OS-dependent behavior rather than claiming universal compatibility. +- Deliver one end-to-end vertical slice at a time, with focused tests and user documentation. A module is not complete merely because one code path exists. diff --git a/CHANGELOG.md b/CHANGELOG.md index 4745906..52c655b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,8 @@ All notable changes are listed here. Versions follow [Semantic Versioning](https ### Added - XDVDFS/Xbox XISO support: identify, browse, and extract existing images, and create new XISOs with `create-image --fs xdvdfs` (or `xiso`). Existing XISOs are read-only; ISO-specific labels and boot-entry options are not supported. +- FAT12/16/32 support for raw floppy and primary MBR-partitioned images: identify, browse, extract, and edit volumes while retaining boot/partition sectors. Fixed and dynamic VHD 1.0 containers with FAT volumes are also supported for browse, extract, and edit; new fixed or sparse dynamic VHDs can be written. Per-file timestamps and attributes reset on edit; GPT/extended partitions, VHDX, differencing VHDs, and exFAT are not supported. +- **Save USB drive to image file** now has a task window with physical-disk and disk-image sources, optional byte limits, progress, and cancellable disk-image reads. Physical-disk reads continue until the helper finishes; the source is read-only and real USB hardware remains unverified. ## [1.0.1] - 2026-11-08 diff --git a/README.md b/README.md index 8200e11..44ad70a 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ IsoFoundry is a single free Windows program for working with CDs, DVDs, Blu-rays - **0.4.0** — M2 USB and M3 audio together (0.3.0 was never released separately): Create bootable USB, Install Windows to USB, and batch audio conversion with tags and ReplayGain. USB is tested end-to-end against disk-image files; writing a physical disk and booting from it are still to be confirmed. MP3 needs `libmp3lame.dll` at runtime; AAC uses Windows Media Foundation; both encode paths are **Unverified** on real hardware. - **0.2.0** — M1 image toolkit (create, browse, extract, edit, convert, mount, checksum, verify) in task windows and on the command line. - **0.1.0** — M0 shell: main window, command line and elevated helper. -- **Next** — 1.x: multiboot USB, FAT/VHD image editing, Xbox XISO, Spanish UI. See the [roadmap](#roadmap). +- **Next** — 1.x: multiboot USB and Spanish UI. FAT/VHD image editing and Xbox XISO support are available in the current development branch; see the [roadmap](#roadmap). ## What the program does today @@ -34,20 +34,20 @@ IsoFoundry is a single free Windows program for working with CDs, DVDs, Blu-rays These jobs run from the task windows and the command line: - **`create-image`** — build an ISO from files and folders: ISO 9660 + Joliet + UDF 1.02 in one image sharing a single copy of the data, files over 4 GB, El Torito boot entries for BIOS (optional boot information table) and UEFI. `--fs xdvdfs` (or `xiso`) creates an Xbox XISO. Written to a `.part` file and renamed only when complete, so a cancelled build never leaves a half-written image. -- **`image-info`** — an image's container format, tracks, file systems, label and boot entries; recognizes XDVDFS/XISO. -- **`extract`** — copy files out of an image (everything, or one file/folder), renaming names Windows cannot store, and keeping nothing half-written on cancel. XISO browsing and extraction also work in the GUI browse task. -- **`edit-image`** — add, remove, rename, create folders, change the label, saved as a new image; untouched files are copied straight from the source, and BIOS/UEFI boot entries are kept. XDVDFS/XISO images are read-only. +- **`image-info`** — an image's container format, tracks, file systems, label and boot entries; recognizes XDVDFS/XISO, FAT, and fixed/dynamic VHD containers. +- **`extract`** — copy files out of an image (everything, or one file/folder), renaming names Windows cannot store, and keeping nothing half-written on cancel. XISO browsing and extraction also work in the GUI browse task; raw FAT12/16/32 and fixed/dynamic VHDs containing supported FAT volumes are supported too. +- **`edit-image`** — add, remove, rename, create folders, and change the label, saved as a new image. FAT12/16/32 floppy and primary MBR volumes can be edited while retaining their image size and boot/partition sectors; fixed/dynamic VHD inputs are saved back as the same VHD type. Per-file timestamps and attributes reset. XDVDFS/XISO images remain read-only. - **`convert-image`** — any readable image to ISO, BIN/CUE, CloneCD (CCD/IMG/SUB) or CHD. - **`checksum`** — CRC32, MD5, SHA-1 and SHA-256 in one pass; `--dat` also writes a Logiqx-style `.dat` hash file. - **`verify --dat`** — look a file's hashes up in a known-good `.dat` list (for example a redump list) and report match or no match. - **`compare`** — byte-for-byte comparison of two files. - **`mount` / `unmount`** — mount an image read-only with Windows' built-in virtual disk support; other formats are converted to a temporary ISO first, which is removed on unmount. -Formats: ISO 9660 (with Rock Ridge names), Joliet, UDF 1.02–2.60 (physical, sparable and metadata partitions), XDVDFS/Xbox XISO (browse, extract and create), multi-extent files, El Torito boot catalogs (BIOS and UEFI). CD images: BIN/CUE (single and multi-file; MODE1, MODE2 and audio tracks), CloneCD CCD/IMG/SUB, sheetless raw `.bin`/`.img`, and MAME CHD v5 — reading `zlib`/`cdzl` discs and writing CD (`cdzl`) and DVD (`zlib`) CHDs whose SHA-1s match chdman 0.264 byte for byte on the test images. Nero NRG and Alcohol MDS/MDF are read-only and marked unverified. Raw sector ECC/EDC generation and checks, and Q-subchannel generation, are included. Known format limits are listed in [docs/LIMITATIONS.md](docs/LIMITATIONS.md). +Formats: ISO 9660 (with Rock Ridge names), Joliet, UDF 1.02–2.60 (physical, sparable and metadata partitions), XDVDFS/Xbox XISO (browse, extract and create), FAT12/16/32 raw floppy or primary MBR partitions, and fixed/dynamic VHD 1.0 containers with FAT12/16/32 (browse, extract and edit); multi-extent files, El Torito boot catalogs (BIOS and UEFI). CD images: BIN/CUE (single and multi-file; MODE1, MODE2 and audio tracks), CloneCD CCD/IMG/SUB, sheetless raw `.bin`/`.img`, and MAME CHD v5 — reading `zlib`/`cdzl` discs and writing CD (`cdzl`) and DVD (`zlib`) CHDs whose SHA-1s match chdman 0.264 byte for byte on the test images. Nero NRG and Alcohol MDS/MDF are read-only and marked unverified. Raw sector ECC/EDC generation and checks, and Q-subchannel generation, are included. Known format limits are listed in [docs/LIMITATIONS.md](docs/LIMITATIONS.md). ## USB tools -- **`bootable-usb`** — turn an ISO into a bootable USB drive: partition it (MBR or GPT), format FAT32 and copy the image's files, or copy the image byte-for-byte in raw mode with an optional SHA-256 verify afterwards. `usb read` copies a drive back to an image file. +- **`bootable-usb`** — turn an ISO into a bootable USB drive: partition it (MBR or GPT), format FAT32 and copy the image's files, or copy the image byte-for-byte in raw mode with an optional SHA-256 verify afterwards. Its **Save USB drive to image file** variant reads a physical disk or disk-image file byte-for-byte, with an optional byte limit; it is also available as `usb read`. - **`install-windows-usb`** — the same write with Windows installer layouts: MBR+FAT32 (default), GPT+FAT32, or GPT+NTFS for install images over the FAT32 file limit. The boot files are Microsoft's own, copied unchanged from the ISO, so Secure Boot keeps working. Writing is guarded: the system disk, the boot/pagefile disk and fixed disks are refused, and a target with mounted volumes or over 64 GB needs a typed ERASE confirmation. The engine is tested against disk-image files (the VHDX route); physical disks are **Unverified** until exercised on real hardware. diff --git a/SPEC-image-foundation.md b/SPEC-image-foundation.md new file mode 100644 index 0000000..7213c2a --- /dev/null +++ b/SPEC-image-foundation.md @@ -0,0 +1,85 @@ +# Spec: Image Foundation — VHD Support + +## Objective + +Complete the next `image-foundation` increment: support fixed and dynamic VHD containers for FAT12/16/32 image workflows, including inspect, browse, extract, and safe edit-to-new-image. This extends the existing ISO/XISO/FAT image engine and the already implemented FAT editor. It does not replace or discard existing USB-to-image work. + +The primary user is someone managing legacy disk images who needs to inspect and edit supported FAT volumes inside VHD files without writing to the original. Success means the application can recognize valid supported VHDs, expose the contained FAT filesystem through existing workflows, and safely save edits as a separate VHD that reopens with byte-correct file contents. + +## Assumptions + +1. The accepted container scope is fixed and dynamic VHD (VHD 1.0), with FAT12/16/32 volumes; differencing VHD chains, VHDX image editing, and filesystems other than supported FAT remain unsupported. +2. The existing GUI and CLI image workflows should use the shared `if-image` implementation rather than gain separate parsers. +3. Source images must remain unchanged; edits write to a new destination and reject an existing destination using the established project behavior. +4. Physical hardware tests are not required. VHD fixture tests and software-backed integration tests are the verification gate; unsupported host-dependent behavior must be documented. +5. No new dependency is needed unless repository evidence shows the existing Rust implementation cannot safely meet the format requirements; adding one requires approval. + +## Tech Stack + +- Rust 2021, minimum supported Rust 1.82. +- Workspace crates: `if-image` for format parsing and editing; `isofoundry` for CLI/GUI integration. +- Current `if-image` dependencies include `miniz_oxide` and `sha1`. Preserve the small-dependency policy. +- Windows desktop application with Win32 UI and CLI; format logic remains platform-independent where practical. + +## Commands + +- Format: `cargo fmt --check` +- Image tests: `cargo test -p if-image` +- App/CLI tests: `cargo test -p isofoundry` +- Lint: `cargo clippy -p if-image -p isofoundry --all-targets -- -D warnings` +- Build: `cargo build -p isofoundry` + +## Project Structure + +- `crates/if-image/src/` — image container, sector source, filesystem, and edit implementations. +- `crates/if-image/tests/` — generated image fixtures and format/edit round-trip tests. +- `crates/isofoundry/src/cli_image.rs` — command-line image workflows. +- `crates/isofoundry/src/win/tasks/` — GUI browse, extract, and edit tasks. +- `README.md`, `docs/ARCHITECTURE.md`, and `docs/LIMITATIONS.md` — supported behavior and limits. +- `tasks/plan.md` and `tasks/todo.md` — staged project work; preserve completed items and unrelated working-tree changes. + +## Code Style + +Follow the existing checked, fallible parser style and return the shared `if_image::Result` rather than silently accepting malformed metadata. For example, the current FAT probe uses a trait-object sector source and an optional match result: + +```rust +pub fn probe(src: &mut dyn SectorSource) -> Result> { + let image_bytes = src + .sectors() + .checked_mul(SECTOR as u64) + .ok_or_else(|| crate::Error::Format("image size overflow".into()))?; + // Validate format metadata and bounds before exposing a filesystem. + Ok(None) +} +``` + +Use checked arithmetic for offsets and lengths, explicit format errors for unsupported layouts, and the existing source/extent abstractions. Avoid unchecked casts, broad error swallowing, and edits in place. + +## Testing Strategy + +- Generate small fixed and dynamic VHD fixtures in Rust tests rather than depend on physical media. +- Test valid footer/header checksums, metadata bounds, fixed payloads, dynamic block maps, sparse/unallocated blocks, and FAT12/16/32 payload layouts. +- Test malformed/truncated headers, invalid offsets and block maps, overflow, unsupported differencing images, and corrupt checksums. +- Bound supported images to 2 TiB virtual size, 1,048,576 dynamic BAT entries, and 256 MiB dynamic blocks. +- Verify browse/extract contents and edit round trips; verify the source bytes are unchanged and destination collisions/cancellation fail safely. +- Exercise the shared CLI path and ensure the GUI selects an appropriate VHD output extension while using the shared implementation. +- Run the commands above. Do not require hardware tests; document real-host limitations separately. + +## Boundaries + +- **Always:** Preserve source images; use bounded parsing and checked arithmetic; write edits atomically to a separate file; report unsupported formats explicitly; add focused tests and update user documentation. +- **Ask first:** VHDX editing, differencing chains, support for additional filesystems, new runtime dependencies, or any change to the user's existing USB-to-image work. +- **Never:** Modify a source image in place; claim universal VHD compatibility; implement copy-protection circumvention; silently ignore invalid metadata. + +## Success Criteria + +- Valid fixed and dynamic VHDs containing supported FAT volumes are recognized and can be browsed and extracted through existing image workflows. +- FAT editing inside supported VHDs produces a valid separate VHD that reopens with correct edited contents and retained required container/partition metadata. +- Invalid, oversized, unsupported, or malformed VHD metadata is rejected with a clear error before unsafe allocation or I/O. +- The source remains byte-for-byte unchanged, an existing destination is not overwritten, and cancellation leaves no success-shaped partial output. +- Focused tests, formatting, Clippy, and the application build pass. +- README and limitations document supported VHD variants and explicit exclusions. + +## Open Questions + +- None. The fixed/dynamic VHD, FAT-only, separate-output scope follows the approved capability map and existing 1.x roadmap. diff --git a/crates/if-image/src/container.rs b/crates/if-image/src/container.rs index be59869..335b0aa 100644 --- a/crates/if-image/src/container.rs +++ b/crates/if-image/src/container.rs @@ -18,6 +18,7 @@ pub enum Format { Nrg, Mds, Chd, + Vhd, /// A sheetless raw `.bin`/`.img` of 2352-byte sectors. RawBin, } @@ -31,6 +32,7 @@ impl Format { Format::Nrg => "Nero NRG", Format::Mds => "Alcohol MDS/MDF", Format::Chd => "MAME CHD", + Format::Vhd => "Virtual Hard Disk (VHD)", Format::RawBin => "raw 2352-byte sectors", } } @@ -83,6 +85,7 @@ pub fn detect(path: &Path) -> Result<(Format, PathBuf)> { None => return bad("an .mdf needs its .mds descriptor next to it"), }, "chd" => (Format::Chd, p), + "vhd" => (Format::Vhd, p), "bin" | "img" => { if let Some(c) = sibling(path, "cue") { (Format::BinCue, c) @@ -127,6 +130,7 @@ pub fn open_image(path: &Path) -> Result<(Format, Image)> { let disc = |d: Disc| Image::Cd(Box::new(DiscReader::new(d))); let img = match f { Format::Iso => Image::Sectors(Box::new(IsoFile::open(&p)?)), + Format::Vhd => Image::Sectors(Box::new(crate::vhd::VhdFile::open(&p)?)), Format::BinCue => disc(cue::read(&p)?), Format::CloneCd => disc(ccd::read(&p)?), Format::Nrg => disc(nrg::read(&p)?), diff --git a/crates/if-image/src/edit.rs b/crates/if-image/src/edit.rs index fd43a4f..9ddf45c 100644 --- a/crates/if-image/src/edit.rs +++ b/crates/if-image/src/edit.rs @@ -7,10 +7,53 @@ use crate::container; use crate::eltorito::{Emulation, Platform}; use crate::fs::{Data, Entry, FsKind, Volume}; use crate::write::{self, BootImage, ImageData, Kind, Node, Options, SharedSource, Source, Summary}; -use crate::{bad, Result, SECTOR}; +use crate::{bad, Error, Result, SECTOR}; use std::cell::RefCell; +use std::fs::{self, OpenOptions}; +use std::io::Write; use std::path::{Path, PathBuf}; use std::rc::Rc; +use std::sync::atomic::{AtomicU64, Ordering}; + +static NEXT_TEMP: AtomicU64 = AtomicU64::new(0); + +struct TempImage(PathBuf); + +impl TempImage { + fn new(label: &str) -> Result { + loop { + let id = NEXT_TEMP.fetch_add(1, Ordering::Relaxed); + let path = std::env::temp_dir().join(format!("isofoundry-{label}-{}-{id}.img", std::process::id())); + match OpenOptions::new().write(true).create_new(true).open(&path) { + Ok(_) => return Ok(Self(path)), + Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => continue, + Err(error) => return Err(error.into()), + } + } + } + + fn path(&self) -> &Path { + &self.0 + } + + fn clear(&self) -> Result<()> { + match fs::remove_file(&self.0) { + Ok(()) => Ok(()), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(()), + Err(error) => Err(error.into()), + } + } + + fn remove(self) -> Result<()> { + self.clear() + } +} + +impl Drop for TempImage { + fn drop(&mut self) { + let _ = fs::remove_file(&self.0); + } +} /// One change. Paths are inside the image, `/`-separated, case-insensitive. #[derive(Debug, Clone, PartialEq, Eq)] @@ -36,6 +79,10 @@ pub struct Project { pub root: Node, pub opts: Options, pub warnings: Vec, + source_path: PathBuf, + source_format: container::Format, + vhd_dynamic: bool, + fat_layout: Option, _src: SharedSource, } @@ -60,15 +107,30 @@ fn children(n: &mut Node) -> Result<&mut Vec> { impl Project { pub fn open(image: &Path) -> Result { - let (_, src) = container::open(image)?; + let (source_format, src) = container::open(image)?; + let vhd_dynamic = if source_format == container::Format::Vhd { + crate::vhd::VhdFile::open(image)?.is_dynamic() + } else { + false + }; let mut v = Volume::open(src, None)?; if v.fs.kind() == FsKind::Xdvdfs { return bad("XDVDFS images are read-only and cannot be edited"); } - if v.fs.kind() == FsKind::Fat { - return bad("FAT images are read-only and cannot be edited yet"); - } let mut warnings = Vec::new(); + let fat_layout = if v.fs.kind() == FsKind::Fat { + warnings.push( + "FAT edits preserve volume geometry and boot/partition sectors; per-file timestamps and attributes are reset" + .into(), + ); + Some( + crate::fat::Fat::probe(v.src.as_mut())? + .ok_or_else(|| crate::Error::Format("FAT volume disappeared while opening the image".into()))? + .layout(), + ) + } else { + None + }; let label = v.fs.label(); let joliet = v.found.contains(&FsKind::Joliet) || v.found.contains(&FsKind::Udf); let udf = v.found.contains(&FsKind::Udf); @@ -121,7 +183,16 @@ impl Project { } } } - Ok(Project { root, opts, warnings, _src: shared }) + Ok(Project { + root, + opts, + warnings, + source_path: image.to_path_buf(), + source_format, + vhd_dynamic, + fat_layout, + _src: shared, + }) } fn dir_mut(&mut self, path: &str, create: bool) -> Result<&mut Vec> { @@ -184,7 +255,80 @@ impl Project { /// Writes the edited image to `dest` (never the source). pub fn save(self, dest: &Path, progress: impl FnMut(u64, u64) -> bool + 'static) -> Result { - write::build(&self.root, self.opts, dest, progress) + if let Some(layout) = self.fat_layout { + if self.source_format == container::Format::Vhd { + return self.save_vhd(layout, dest, progress); + } + crate::fat_write::build(&self.root, &self.opts.label, layout, &self.source_path, dest, progress) + } else { + write::build(&self.root, self.opts, dest, progress) + } + } + + fn save_vhd( + self, + layout: crate::fat::FatLayout, + dest: &Path, + mut progress: impl FnMut(u64, u64) -> bool + 'static, + ) -> Result { + if dest.exists() { + return bad(format!("destination already exists: {}", dest.display())); + } + let bytes = self._src.borrow().byte_len(); + let total = bytes.saturating_mul(3).max(1); + let raw_source = TempImage::new("source")?; + let raw_output = TempImage::new("edited")?; + let result = (|| -> Result { + { + let mut src = self._src.borrow_mut(); + let mut file = OpenOptions::new().write(true).truncate(true).open(raw_source.path())?; + let mut offset = 0u64; + let mut buffer = vec![0; 1 << 20]; + while offset < bytes { + let len = (bytes - offset).min(buffer.len() as u64) as usize; + let read_len = len.div_ceil(SECTOR) * SECTOR; + src.read_sectors(offset / SECTOR as u64, &mut buffer[..read_len])?; + file.write_all(&buffer[..len])?; + offset += len as u64; + if !progress(offset, total) { + return Err(Error::Cancelled); + } + } + file.sync_all()?; + } + raw_output.clear()?; + let summary = crate::fat_write::build( + &self.root, + &self.opts.label, + layout, + raw_source.path(), + raw_output.path(), + |done, stage_total| { + let scaled = if stage_total == 0 { + 0 + } else { + ((u128::from(done.min(stage_total)) * u128::from(bytes)) / u128::from(stage_total)) as u64 + }; + progress(bytes.saturating_add(scaled), total) + }, + )?; + let mut raw = crate::source::IsoFile::open(raw_output.path())?; + let write_progress = + |done: u64, _stage_total: u64| progress(bytes.saturating_mul(2).saturating_add(done), total); + if self.vhd_dynamic { + crate::vhd_write::write_dynamic(&mut raw, dest, write_progress)?; + } else { + crate::vhd_write::write_fixed(&mut raw, dest, write_progress)?; + } + Ok(summary) + })(); + let cleanup = raw_source.remove().and_then(|()| raw_output.remove()); + match (result, cleanup) { + (Err(error), Err(cleanup)) => bad(format!("{error}; additionally, {cleanup}")), + (Err(error), _) => Err(error), + (Ok(_), Err(cleanup)) => Err(cleanup), + (Ok(summary), Ok(())) => Ok(summary), + } } } diff --git a/crates/if-image/src/fat.rs b/crates/if-image/src/fat.rs index cedf2bd..724a5cd 100644 --- a/crates/if-image/src/fat.rs +++ b/crates/if-image/src/fat.rs @@ -11,21 +11,49 @@ const MAX_DIRECTORY_ENTRIES: usize = 100_000; const FAT_CACHE_PAGES: usize = 64; #[derive(Clone, Copy)] -enum FatType { +pub(crate) enum FatType { Fat12, Fat16, Fat32, } +#[derive(Clone, Copy)] +pub(crate) struct FatLayout { + pub kind: FatType, + pub base: u64, + pub bytes_per_sector: u64, + pub sectors_per_cluster: u64, + pub reserved_sectors: u64, + pub fat_count: u64, + pub root_entries: u64, + pub total_sectors: u64, + pub cluster_count: u32, + pub fat_offset: u64, + pub fat_size: u64, + pub data_offset: u64, + pub backup_boot_sector: u16, + pub fs_info_sector: u16, + pub media_descriptor: u8, +} + pub struct Fat { kind: FatType, + base: u64, bytes_per_sector: u64, sectors_per_cluster: u64, + reserved_sectors: u64, + fat_count: u64, + root_entries: u64, + total_sectors: u64, cluster_count: u32, fat_offset: u64, fat_size: u64, data_offset: u64, volume_end: u64, + root_cluster: u32, + backup_boot_sector: u16, + fs_info_sector: u16, + media_descriptor: u8, root: Entry, label: String, fat_cache: RefCell>>, @@ -34,10 +62,7 @@ pub struct Fat { impl Fat { /// Finds a FAT superfloppy or the first supported FAT partition in an MBR. pub fn probe(src: &mut dyn SectorSource) -> Result> { - let image_bytes = src - .sectors() - .checked_mul(SECTOR as u64) - .ok_or_else(|| crate::Error::Format("image size overflow".into()))?; + let image_bytes = src.byte_len(); if image_bytes < 512 { return Ok(None); } @@ -174,9 +199,7 @@ impl Fat { let volume_size = total_sectors .checked_mul(bytes_per_sector) .ok_or_else(|| crate::Error::Format("FAT volume size overflow".into()))?; - if volume_size > available - || base.checked_add(volume_size).is_none_or(|end| end > src.sectors() * SECTOR as u64) - { + if volume_size > available || base.checked_add(volume_size).is_none_or(|end| end > src.byte_len()) { return bad("FAT volume extends beyond the image"); } let fat_offset = base @@ -201,15 +224,39 @@ impl Fat { } else { String::new() }; + let root_cluster = if matches!(kind, FatType::Fat32) { + u32::from_le_bytes(full_boot[44..48].try_into().unwrap()) & 0x0fff_ffff + } else { + 0 + }; + let backup_boot_sector = if matches!(kind, FatType::Fat32) { + u16::from_le_bytes(full_boot[50..52].try_into().unwrap()) + } else { + u16::MAX + }; + let fs_info_sector = if matches!(kind, FatType::Fat32) { + u16::from_le_bytes(full_boot[48..50].try_into().unwrap()) + } else { + u16::MAX + }; let mut fat = Self { kind, + base, bytes_per_sector, sectors_per_cluster, + reserved_sectors, + fat_count, + root_entries, + total_sectors, cluster_count, fat_offset, fat_size, data_offset, volume_end, + root_cluster, + backup_boot_sector, + fs_info_sector, + media_descriptor: boot[21], root: Entry { name: String::new(), dir: true, @@ -223,7 +270,7 @@ impl Fat { }; let (root_data, root_size) = if matches!(kind, FatType::Fat32) { - let cluster = u32::from_le_bytes(full_boot[44..48].try_into().unwrap()) & 0x0fff_ffff; + let cluster = fat.root_cluster; if cluster < 2 || cluster > cluster_count + 1 { return bad("invalid FAT32 root directory cluster"); } @@ -243,6 +290,26 @@ impl Fat { Ok(Some(fat)) } + pub(crate) fn layout(&self) -> FatLayout { + FatLayout { + kind: self.kind, + base: self.base, + bytes_per_sector: self.bytes_per_sector, + sectors_per_cluster: self.sectors_per_cluster, + reserved_sectors: self.reserved_sectors, + fat_count: self.fat_count, + root_entries: self.root_entries, + total_sectors: self.total_sectors, + cluster_count: self.cluster_count, + fat_offset: self.fat_offset, + fat_size: self.fat_size, + data_offset: self.data_offset, + backup_boot_sector: self.backup_boot_sector, + fs_info_sector: self.fs_info_sector, + media_descriptor: self.media_descriptor, + } + } + fn fat_bytes(&self, src: &mut dyn SectorSource, offset: u64, len: usize) -> Result> { let mut out = vec![0; len]; let mut done = 0usize; diff --git a/crates/if-image/src/fat_write.rs b/crates/if-image/src/fat_write.rs new file mode 100644 index 0000000..6dd401b --- /dev/null +++ b/crates/if-image/src/fat_write.rs @@ -0,0 +1,674 @@ +use crate::fat::{FatLayout, FatType}; +use crate::write::{self, Kind, Node, Source, Summary}; +use crate::{bad, Error, Result}; +use std::collections::HashSet; +use std::fs::{File, OpenOptions}; +use std::io::{Read, Seek, SeekFrom, Write}; +use std::path::{Path, PathBuf}; + +const COPY_BUFFER: usize = 1 << 20; +const MAX_EDIT_CLUSTERS: u64 = 4_000_000; +const MAX_FAT_CLUSTERS: u32 = 16_000_000; +const MAX_FAT_TABLE_BYTES: u64 = 64 << 20; + +struct Progress { + callback: F, + done: u64, + total: u64, +} + +impl bool> Progress { + fn tick(&mut self, amount: u64) -> Result<()> { + self.done = self.done.saturating_add(amount); + if !(self.callback)(self.done, self.total) { + return Err(Error::Cancelled); + } + Ok(()) + } +} + +struct Planned { + name: String, + short: [u8; 11], + file: Option<(Source, u64)>, + children: Vec, + clusters: Vec, +} + +impl Planned { + fn is_dir(&self) -> bool { + self.file.is_none() + } +} + +fn validate_name(name: &str) -> Result> { + let units: Vec = name.encode_utf16().collect(); + if units.is_empty() + || units.len() > 255 + || name.ends_with(['.', ' ']) + || name.chars().any(|c| c.is_control() || matches!(c, '/' | '\\' | ':' | '*' | '?' | '"' | '<' | '>' | '|')) + { + return bad(format!("FAT cannot store this filename: {name:?}")); + } + Ok(units) +} + +fn alias(name: &str, used: &mut HashSet<[u8; 11]>) -> Result<[u8; 11]> { + const ALLOWED: &str = "$%'-_@~`!(){}^#&"; + let (stem, extension) = name.rsplit_once('.').unwrap_or((name, "")); + let clean = |part: &str| { + part.chars() + .filter_map(|c| { + let c = c.to_ascii_uppercase(); + (c.is_ascii_alphanumeric() || ALLOWED.contains(c)).then_some(c as u8) + }) + .collect::>() + }; + let stem = clean(stem); + let extension = clean(extension); + let stem = if stem.is_empty() { b"FILE".to_vec() } else { stem }; + for suffix in 1..=999_999u32 { + let tail = format!("~{suffix}"); + let mut candidate = [b' '; 11]; + let stem_len = (8 - tail.len()).min(stem.len()); + candidate[..stem_len].copy_from_slice(&stem[..stem_len]); + candidate[stem_len..stem_len + tail.len()].copy_from_slice(tail.as_bytes()); + candidate[8..8 + extension.len().min(3)].copy_from_slice(&extension[..extension.len().min(3)]); + if used.insert(candidate) { + return Ok(candidate); + } + } + bad(format!("too many FAT filename collisions in {name:?}")) +} + +fn plan(node: &Node, root: bool, depth: u32) -> Result { + if depth > 64 { + return bad("folders nested deeper than 64 levels cannot be written to FAT"); + } + let mut used = HashSet::new(); + let mut planned = + Planned { name: node.name.clone(), short: [b' '; 11], file: None, children: Vec::new(), clusters: Vec::new() }; + match &node.kind { + Kind::File { source, size } => { + validate_name(&node.name)?; + if *size > u64::from(u32::MAX) { + return bad(format!("FAT cannot store files larger than 4 GiB: {}", node.name)); + } + planned.file = Some((source.clone(), *size)); + } + Kind::Dir(children) => { + if children.len() > 100_000 { + return bad(format!("{} contains too many entries for FAT editing", node.name)); + } + for child in children { + validate_name(&child.name)?; + let short = alias(&child.name, &mut used)?; + let mut child_plan = plan(child, false, depth + 1)?; + child_plan.short = short; + planned.children.push(child_plan); + } + if root { + planned.name.clear(); + } + } + } + Ok(planned) +} + +fn count_files(node: &Planned) -> (u64, u64, u64) { + if let Some((_, size)) = &node.file { + (1, 0, *size) + } else { + node.children.iter().fold((0, 1, 0), |(files, dirs, bytes), child| { + let (f, d, b) = count_files(child); + (files + f, dirs + d, bytes + b) + }) + } +} + +fn cluster_bytes(layout: FatLayout) -> Result { + layout + .bytes_per_sector + .checked_mul(layout.sectors_per_cluster) + .ok_or_else(|| Error::Format("FAT cluster size overflow".into())) +} + +fn read_fat_entry(data: &[u8], layout: FatLayout, cluster: u32) -> Result { + let (offset, width) = match layout.kind { + FatType::Fat12 => (u64::from(cluster) + u64::from(cluster) / 2, 2), + FatType::Fat16 => (u64::from(cluster) * 2, 2), + FatType::Fat32 => (u64::from(cluster) * 4, 4), + }; + if offset + width > layout.fat_size { + return bad("FAT table is too small for the volume"); + } + let start = usize::try_from(offset).map_err(|_| Error::Format("FAT offset exceeds address space".into()))?; + let end = start + width as usize; + let bytes = data.get(start..end).ok_or_else(|| Error::Format("FAT table is too small for the volume".into()))?; + let value = match layout.kind { + FatType::Fat12 => { + let word = u16::from_le_bytes([bytes[0], bytes[1]]); + u32::from(if cluster & 1 == 0 { word & 0x0fff } else { word >> 4 }) + } + FatType::Fat16 => u32::from(u16::from_le_bytes([bytes[0], bytes[1]])), + FatType::Fat32 => u32::from_le_bytes(bytes.try_into().unwrap()) & 0x0fff_ffff, + }; + Ok(value) +} + +fn special_value(kind: FatType, value: u32) -> bool { + match kind { + FatType::Fat12 => (0x0ff0..=0x0ff7).contains(&value), + FatType::Fat16 => (0xfff0..=0xfff7).contains(&value), + FatType::Fat32 => (0x0fff_fff0..=0x0fff_fff7).contains(&value), + } +} + +fn set_fat_entry(data: &mut [u8], layout: FatLayout, cluster: u32, value: u32) -> Result<()> { + let (offset, bytes) = match layout.kind { + FatType::Fat12 => { + let offset = u64::from(cluster) + u64::from(cluster) / 2; + let start = + usize::try_from(offset).map_err(|_| Error::Format("FAT offset exceeds address space".into()))?; + let old = u16::from_le_bytes( + data.get(start..start + 2) + .ok_or_else(|| Error::Format("FAT table is too small for the volume".into()))? + .try_into() + .unwrap(), + ); + let new = if cluster & 1 == 0 { + (old & 0xf000) | (value as u16 & 0x0fff) + } else { + (old & 0x000f) | ((value as u16 & 0x0fff) << 4) + }; + (start, new.to_le_bytes().to_vec()) + } + FatType::Fat16 => (cluster as usize * 2, (value as u16).to_le_bytes().to_vec()), + FatType::Fat32 => (cluster as usize * 4, (value & 0x0fff_ffff).to_le_bytes().to_vec()), + }; + let end = offset + bytes.len(); + let target = + data.get_mut(offset..end).ok_or_else(|| Error::Format("FAT table is too small for the volume".into()))?; + target.copy_from_slice(&bytes); + Ok(()) +} + +struct Allocator { + layout: FatLayout, + next: u32, + blocked: Vec, + fat_data: Vec, + allocated: u64, +} + +impl Allocator { + fn new(file: &mut File, layout: FatLayout) -> Result { + if layout.fat_size > MAX_FAT_TABLE_BYTES { + return bad("FAT tables larger than 64 MB cannot be edited safely yet"); + } + let fat_len = + usize::try_from(layout.fat_size).map_err(|_| Error::Format("FAT table exceeds address space".into()))?; + let mut old_fat = vec![0; fat_len]; + file.seek(SeekFrom::Start(layout.fat_offset))?; + file.read_exact(&mut old_fat)?; + let blocked_len = usize::try_from(layout.cluster_count) + .map_err(|_| Error::Format("FAT cluster count exceeds address space".into()))? + .saturating_add(2); + let mut blocked = vec![false; blocked_len]; + let mut fat_data = vec![0; fat_len]; + for cluster in 2..=layout.cluster_count + 1 { + let value = read_fat_entry(&old_fat, layout, cluster)?; + if special_value(layout.kind, value) { + blocked[cluster as usize] = true; + set_fat_entry(&mut fat_data, layout, cluster, value)?; + } + } + let reserved = match layout.kind { + FatType::Fat12 => [layout.media_descriptor as u32, 0x0fff], + FatType::Fat16 => [0xff00 | u32::from(layout.media_descriptor), 0xffff], + FatType::Fat32 => [0x0fff_ff00 | u32::from(layout.media_descriptor), 0x0fff_ffff], + }; + set_fat_entry(&mut fat_data, layout, 0, reserved[0])?; + set_fat_entry(&mut fat_data, layout, 1, reserved[1])?; + Ok(Self { layout, next: 2, blocked, fat_data, allocated: 0 }) + } + + fn allocate(&mut self, count: u64) -> Result> { + if self.allocated.saturating_add(count) > MAX_EDIT_CLUSTERS { + return bad("editing this volume would require too many clusters in memory"); + } + let count = usize::try_from(count).map_err(|_| Error::Format("too many FAT clusters requested".into()))?; + let mut clusters = Vec::with_capacity(count); + while clusters.len() < count { + if self.next > self.layout.cluster_count + 1 { + return bad("the edited files do not fit in the FAT volume"); + } + let cluster = self.next; + self.next += 1; + if self.blocked[cluster as usize] { + continue; + } + if let Some(previous) = clusters.last().copied() { + set_fat_entry(&mut self.fat_data, self.layout, previous, cluster)?; + } + clusters.push(cluster); + } + if let Some(last) = clusters.last().copied() { + set_fat_entry(&mut self.fat_data, self.layout, last, end_marker(self.layout.kind))?; + } + self.allocated += count as u64; + Ok(clusters) + } +} + +fn end_marker(kind: FatType) -> u32 { + match kind { + FatType::Fat12 => 0x0fff, + FatType::Fat16 => 0xffff, + FatType::Fat32 => 0x0fff_ffff, + } +} + +fn directory_records(node: &Planned, root: bool) -> Result { + let mut records = u64::from(!root) * 2; + for child in &node.children { + let units = validate_name(&child.name)?; + records += (units.len().div_ceil(13) + 1) as u64; + } + if root { + records += 1; + } + Ok(records) +} + +fn allocate_tree(node: &mut Planned, root: bool, layout: FatLayout, allocator: &mut Allocator) -> Result<()> { + let cb = cluster_bytes(layout)?; + if node.is_dir() && (!root || matches!(layout.kind, FatType::Fat32)) { + let bytes = directory_records(node, root)? + .checked_mul(32) + .ok_or_else(|| Error::Format("FAT directory size overflow".into()))?; + let count = bytes.div_ceil(cb).max(1); + node.clusters = allocator.allocate(count)?; + } else if let Some((_, size)) = &node.file { + if *size != 0 { + node.clusters = allocator.allocate(size.div_ceil(cb))?; + } + } + for child in &mut node.children { + allocate_tree(child, false, layout, allocator)?; + } + Ok(()) +} + +fn lfn_records(name: &str, short: &[u8; 11]) -> Result> { + let units = validate_name(name)?; + let slots = units.len().div_ceil(13); + let checksum = short + .iter() + .fold(0u8, |sum, byte| (if sum & 1 != 0 { 0x80u8 } else { 0u8 }).wrapping_add(sum >> 1).wrapping_add(*byte)); + let mut output = Vec::with_capacity(slots + 1); + for sequence in (1..=slots).rev() { + let mut record = [0xffu8; 32]; + record[0] = sequence as u8 | if sequence == slots { 0x40 } else { 0 }; + record[11] = 0x0f; + record[12] = 0; + record[13] = checksum; + record[26..28].fill(0); + for index in 0..13 { + let position = (sequence - 1) * 13 + index; + let unit = if position < units.len() { + units[position] + } else if position == units.len() { + 0 + } else { + 0xffff + }; + let offset = match index { + 0..=4 => 1 + index * 2, + 5..=10 => 14 + (index - 5) * 2, + _ => 28 + (index - 11) * 2, + }; + record[offset..offset + 2].copy_from_slice(&unit.to_le_bytes()); + } + output.push(record); + } + Ok(output) +} + +fn short_entry(node: &Planned) -> [u8; 32] { + let mut record = [0u8; 32]; + record[..11].copy_from_slice(&node.short); + record[11] = if node.is_dir() { 0x10 } else { 0x20 }; + let cluster = node.clusters.first().copied().unwrap_or(0); + record[20..22].copy_from_slice(&((cluster >> 16) as u16).to_le_bytes()); + record[26..28].copy_from_slice(&(cluster as u16).to_le_bytes()); + let size = node.file.as_ref().map(|(_, size)| *size).unwrap_or(0); + record[28..32].copy_from_slice(&(size as u32).to_le_bytes()); + record +} + +fn dot_record(name: &[u8; 11], cluster: u32) -> [u8; 32] { + let mut record = [0u8; 32]; + record[..11].copy_from_slice(name); + record[11] = 0x10; + record[20..22].copy_from_slice(&((cluster >> 16) as u16).to_le_bytes()); + record[26..28].copy_from_slice(&(cluster as u16).to_le_bytes()); + record +} + +fn label_record(label: &str) -> Result<[u8; 32]> { + let mut record = [0u8; 32]; + let mut label_bytes = [b' '; 11]; + let label = label.trim().to_ascii_uppercase(); + if label.len() > 11 + || !label.is_ascii() + || label.chars().any(|c| { + c.is_control() + || matches!( + c, + '"' | '*' | '+' | ',' | '.' | '/' | ':' | ';' | '<' | '=' | '>' | '?' | '[' | '\\' | ']' | '|' + ) + }) + { + return bad("FAT volume labels must be at most 11 ASCII characters and cannot contain path characters"); + } + label_bytes[..label.len()].copy_from_slice(label.as_bytes()); + record[..11].copy_from_slice(&label_bytes); + record[11] = 0x08; + Ok(record) +} + +fn build_directory(node: &Planned, root: bool, parent_cluster: u32, label: &str) -> Result> { + let mut output = Vec::new(); + if !root { + output.extend_from_slice(&dot_record(b". ", node.clusters[0])); + output.extend_from_slice(&dot_record(b".. ", parent_cluster)); + } else { + output.extend_from_slice(&label_record(label)?); + } + for child in &node.children { + for record in lfn_records(&child.name, &child.short)? { + output.extend_from_slice(&record); + } + if child.is_dir() { + let mut record = [0u8; 32]; + record[..11].copy_from_slice(&child.short); + record[11] = 0x10; + let cluster = child.clusters[0]; + record[20..22].copy_from_slice(&((cluster >> 16) as u16).to_le_bytes()); + record[26..28].copy_from_slice(&(cluster as u16).to_le_bytes()); + output.extend_from_slice(&record); + } else { + output.extend_from_slice(&short_entry(child)); + } + } + Ok(output) +} + +fn write_file_data( + output: &mut File, + layout: FatLayout, + node: &Planned, + progress: &mut Progress bool>, +) -> Result<()> { + let Some((source, size)) = &node.file else { return Ok(()) }; + if *size == 0 { + return Ok(()); + } + let cb = usize::try_from(cluster_bytes(layout)?) + .map_err(|_| Error::Format("FAT cluster size exceeds address space".into()))?; + let mut reader = write::reader(source, *size)?; + let mut remaining = *size; + for cluster in &node.clusters { + let count = remaining.min(cb as u64) as usize; + let mut data = vec![0u8; cb]; + reader.read_exact(&mut data[..count])?; + let offset = layout.data_offset + u64::from(*cluster - 2) * cb as u64; + output.seek(SeekFrom::Start(offset))?; + output.write_all(&data)?; + remaining -= count as u64; + progress.tick(count as u64)?; + } + Ok(()) +} + +fn write_tree_data( + output: &mut File, + layout: FatLayout, + node: &Planned, + root: bool, + parent_cluster: u32, + label: &str, + progress: &mut Progress bool>, +) -> Result<()> { + if node.file.is_some() { + return write_file_data(output, layout, node, progress); + } + for child in &node.children { + write_tree_data(output, layout, child, false, node.clusters.first().copied().unwrap_or(0), label, progress)?; + } + let bytes = build_directory(node, root, parent_cluster, label)?; + let offset = if root && !matches!(layout.kind, FatType::Fat32) { + layout.fat_offset + layout.fat_size * layout.fat_count + } else { + layout.data_offset + u64::from(node.clusters[0] - 2) * cluster_bytes(layout)? + }; + let capacity = if root && !matches!(layout.kind, FatType::Fat32) { + layout.root_entries * 32 + } else { + node.clusters.len() as u64 * cluster_bytes(layout)? + }; + if bytes.len() as u64 > capacity { + return bad(format!("{} exceeds its FAT directory capacity", node.name)); + } + output.seek(SeekFrom::Start(offset))?; + output.write_all(&bytes)?; + let remaining = capacity - bytes.len() as u64; + write_zeroes(output, remaining)?; + progress.tick(0) +} + +fn write_zeroes(output: &mut File, mut count: u64) -> Result<()> { + let zeroes = [0u8; 64 * 1024]; + while count != 0 { + let n = count.min(zeroes.len() as u64) as usize; + output.write_all(&zeroes[..n])?; + count -= n as u64; + } + Ok(()) +} + +fn copy_original( + src: &Path, + part: &Path, + created: &mut bool, + progress: &mut Progress bool>, +) -> Result<()> { + let mut input = File::open(src)?; + let mut output = OpenOptions::new().write(true).create_new(true).open(part)?; + *created = true; + let mut buffer = vec![0u8; COPY_BUFFER]; + loop { + let n = input.read(&mut buffer)?; + if n == 0 { + break; + } + output.write_all(&buffer[..n])?; + progress.tick(n as u64)?; + } + output.flush()?; + Ok(()) +} + +fn check_fat_mirrors(file: &mut File, layout: FatLayout) -> Result<()> { + if layout.fat_count < 2 { + return Ok(()); + } + let mut left = vec![0; COPY_BUFFER]; + let mut right = vec![0; COPY_BUFFER]; + let mut offset = 0; + while offset < layout.fat_size { + let n = (layout.fat_size - offset).min(COPY_BUFFER as u64) as usize; + file.seek(SeekFrom::Start(layout.fat_offset + offset))?; + file.read_exact(&mut left[..n])?; + file.seek(SeekFrom::Start(layout.fat_offset + layout.fat_size + offset))?; + file.read_exact(&mut right[..n])?; + if left[..n] != right[..n] { + return bad("FAT copies differ; editing this volume is unsafe"); + } + offset += n as u64; + } + Ok(()) +} + +fn write_fat_copy( + output: &mut File, + layout: FatLayout, + progress: &mut Progress bool>, +) -> Result<()> { + if layout.fat_count == 1 { + return Ok(()); + } + let second = layout.fat_offset + layout.fat_size; + let mut buffer = vec![0u8; COPY_BUFFER]; + let mut offset = 0; + while offset < layout.fat_size { + let n = (layout.fat_size - offset).min(COPY_BUFFER as u64) as usize; + output.seek(SeekFrom::Start(layout.fat_offset + offset))?; + output.read_exact(&mut buffer[..n])?; + output.seek(SeekFrom::Start(second + offset))?; + output.write_all(&buffer[..n])?; + offset += n as u64; + } + progress.tick(layout.fat_size * layout.fat_count) +} + +fn write_fat_table(output: &mut File, layout: FatLayout, data: &[u8]) -> Result<()> { + output.seek(SeekFrom::Start(layout.fat_offset))?; + output.write_all(data)?; + Ok(()) +} + +fn update_boot_metadata(output: &mut File, layout: FatLayout, label: &str, root_cluster: u32) -> Result<()> { + let label = label.trim().to_ascii_uppercase(); + let label_offset = if matches!(layout.kind, FatType::Fat32) { 71 } else { 43 }; + let signature_offset = label_offset - 5; + let write_boot = |file: &mut File, sector: u64| -> Result<()> { + let base = layout.base + sector * layout.bytes_per_sector; + file.seek(SeekFrom::Start(base + signature_offset))?; + let mut signature = [0u8; 1]; + file.read_exact(&mut signature)?; + if signature[0] == 0x29 { + file.seek(SeekFrom::Start(base + label_offset))?; + let mut field = [b' '; 11]; + field[..label.len()].copy_from_slice(label.as_bytes()); + file.write_all(&field)?; + } else if !label.is_empty() { + return bad("this FAT volume has no extended boot signature for a safe volume-label update"); + } + Ok(()) + }; + write_boot(output, 0)?; + if matches!(layout.kind, FatType::Fat32) + && layout.backup_boot_sector != 0 + && layout.backup_boot_sector != u16::MAX + && u64::from(layout.backup_boot_sector) < layout.reserved_sectors + { + write_boot(output, u64::from(layout.backup_boot_sector))?; + output + .seek(SeekFrom::Start(layout.base + 44 + u64::from(layout.backup_boot_sector) * layout.bytes_per_sector))?; + output.write_all(&root_cluster.to_le_bytes())?; + } + if matches!(layout.kind, FatType::Fat32) { + output.seek(SeekFrom::Start(layout.base + 44))?; + output.write_all(&root_cluster.to_le_bytes())?; + if layout.fs_info_sector != 0 + && layout.fs_info_sector != u16::MAX + && u64::from(layout.fs_info_sector) < layout.reserved_sectors + { + update_fs_info(output, layout.base + u64::from(layout.fs_info_sector) * layout.bytes_per_sector)?; + let backup_info = u64::from(layout.backup_boot_sector) + .checked_add(u64::from(layout.fs_info_sector)) + .filter(|sector| *sector < layout.reserved_sectors); + if let Some(sector) = backup_info { + update_fs_info(output, layout.base + sector * layout.bytes_per_sector)?; + } + } + } + Ok(()) +} + +fn update_fs_info(output: &mut File, offset: u64) -> Result<()> { + let mut signatures = [0u8; 512]; + output.seek(SeekFrom::Start(offset))?; + output.read_exact(&mut signatures)?; + if signatures[..4] == 0x4161_5252u32.to_le_bytes() + && signatures[484..488] == 0x6141_7272u32.to_le_bytes() + && signatures[508..512] == 0xaa55_0000u32.to_le_bytes() + { + output.seek(SeekFrom::Start(offset + 488))?; + output.write_all(&u32::MAX.to_le_bytes())?; + output.write_all(&u32::MAX.to_le_bytes())?; + } + Ok(()) +} + +fn part_path(dest: &Path) -> PathBuf { + let mut name = dest.as_os_str().to_owned(); + name.push(".part"); + PathBuf::from(name) +} + +pub(crate) fn build bool>( + root: &Node, + label: &str, + layout: FatLayout, + source: &Path, + dest: &Path, + mut callback: F, +) -> Result { + if dest.exists() { + return bad(format!("destination already exists: {}", dest.display())); + } + let image_bytes = std::fs::metadata(source)?.len(); + if layout.cluster_count > MAX_FAT_CLUSTERS { + return bad("FAT volumes with more than 16 million clusters cannot be edited safely yet"); + } + let volume_end = layout + .total_sectors + .checked_mul(layout.bytes_per_sector) + .and_then(|size| layout.base.checked_add(size)) + .ok_or_else(|| Error::Format("FAT volume size overflow".into()))?; + if volume_end > image_bytes { + return bad("FAT volume extends beyond the source image"); + } + let mut planned = plan(root, true, 0)?; + let (files, dirs, file_bytes) = count_files(&planned); + let total = image_bytes.saturating_mul(3).saturating_add(file_bytes).max(1); + let mut progress = Progress { callback: &mut callback, done: 0, total }; + let part = part_path(dest); + let mut created_part = false; + let result = (|| -> Result { + copy_original(source, &part, &mut created_part, &mut progress)?; + let mut output = OpenOptions::new().read(true).write(true).open(&part)?; + check_fat_mirrors(&mut output, layout)?; + let mut allocator = Allocator::new(&mut output, layout)?; + allocate_tree(&mut planned, true, layout, &mut allocator)?; + let root_cluster = planned.clusters.first().copied().unwrap_or(0); + write_tree_data(&mut output, layout, &planned, true, 0, label, &mut progress)?; + write_fat_table(&mut output, layout, &allocator.fat_data)?; + write_fat_copy(&mut output, layout, &mut progress)?; + update_boot_metadata(&mut output, layout, label, root_cluster)?; + output.sync_all()?; + drop(output); + if dest.exists() { + return bad(format!("destination already exists: {}", dest.display())); + } + std::fs::rename(&part, dest)?; + Ok(Summary { sectors: image_bytes.div_ceil(crate::SECTOR as u64), files, dirs }) + })(); + if result.is_err() && created_part { + let _ = std::fs::remove_file(&part); + } + result +} diff --git a/crates/if-image/src/lib.rs b/crates/if-image/src/lib.rs index 356a64a..9746cde 100644 --- a/crates/if-image/src/lib.rs +++ b/crates/if-image/src/lib.rs @@ -19,6 +19,7 @@ pub mod disc; pub mod edit; pub mod eltorito; pub mod fat; +mod fat_write; pub mod fs; pub mod iso9660; pub mod mds; diff --git a/crates/if-image/src/vhd.rs b/crates/if-image/src/vhd.rs index 2912abe..561118e 100644 --- a/crates/if-image/src/vhd.rs +++ b/crates/if-image/src/vhd.rs @@ -67,6 +67,10 @@ impl VhdFile { Ok(Self { file, bytes, sectors: bytes.div_ceil(SECTOR as u64), layout }) } + pub fn is_dynamic(&self) -> bool { + matches!(self.layout, Layout::Dynamic { .. }) + } + fn read_bytes(&mut self, offset: u64, out: &mut [u8]) -> Result<()> { let end = offset.checked_add(out.len() as u64).ok_or_else(|| crate::Error::Format("VHD read overflow".into()))?; diff --git a/crates/if-image/src/write/mod.rs b/crates/if-image/src/write/mod.rs index e7eeb1e..b309a0b 100644 --- a/crates/if-image/src/write/mod.rs +++ b/crates/if-image/src/write/mod.rs @@ -68,7 +68,7 @@ impl Read for ImageReader { } } -fn reader(src: &Source, size: u64) -> Result> { +pub(crate) fn reader(src: &Source, size: u64) -> Result> { Ok(match src { Source::File(p) => Box::new(File::open(p).map_err(|e| Error::Format(format!("{}: {e}", p.display())))?), Source::Bytes(b) => Box::new(std::io::Cursor::new(b.clone())), diff --git a/crates/if-image/tests/fat.rs b/crates/if-image/tests/fat.rs index be03e56..9660760 100644 --- a/crates/if-image/tests/fat.rs +++ b/crates/if-image/tests/fat.rs @@ -213,6 +213,103 @@ fn reads_fat12_fat16_and_fat32_long_names_and_fragmented_files() { } } +#[test] +fn opens_fixed_and_dynamic_vhds_as_fat_volumes() { + let dir = std::env::temp_dir().join(format!("if-image-vhd-fat-{}", std::process::id())); + std::fs::create_dir_all(&dir).unwrap(); + let source_image = fat_fixture(FatType::Fat16).image; + + for (index, dynamic) in [false, true].into_iter().enumerate() { + let path = dir.join(format!("fat-{index}.vhd")); + let mut source = MemSource(source_image.clone()); + if dynamic { + if_image::vhd_write::write_dynamic(&mut source, &path, |_, _| true).unwrap(); + } else { + if_image::vhd_write::write_fixed(&mut source, &path, |_, _| true).unwrap(); + } + + let (format, source) = if_image::container::open(&path).unwrap(); + assert_eq!(format, if_image::container::Format::Vhd); + let mut volume = Volume::open(source, None).unwrap(); + assert_eq!(volume.fs.kind().name(), "FAT"); + assert_eq!(volume.fs.label(), "FAT TEST"); + let file = volume.lookup("A very long filename.txt").unwrap(); + let mut contents = vec![0; file.size as usize]; + volume.read(&file, 0, &mut contents).unwrap(); + assert_eq!(&contents[..512], vec![b'a'; 512]); + assert_eq!(&contents[512..], vec![b'b'; 88]); + + std::fs::remove_file(path).unwrap(); + } + + std::fs::remove_dir(dir).unwrap(); +} + +#[test] +fn edits_fixed_and_dynamic_vhds_as_new_vhd_images() { + let dir = std::env::temp_dir().join(format!("if-image-vhd-edit-{}", std::process::id())); + std::fs::create_dir_all(&dir).unwrap(); + let original = fat_fixture(FatType::Fat16).image; + let added_file = dir.join("added.txt"); + std::fs::write(&added_file, b"added through VHD editing").unwrap(); + + for (index, dynamic) in [false, true].into_iter().enumerate() { + let source_path = dir.join(format!("source-{index}.vhd")); + let output_path = dir.join(format!("edited-{index}.vhd")); + let mut source = MemSource(original.clone()); + if dynamic { + if_image::vhd_write::write_dynamic(&mut source, &source_path, |_, _| true).unwrap(); + } else { + if_image::vhd_write::write_fixed(&mut source, &source_path, |_, _| true).unwrap(); + } + let original_vhd = std::fs::read(&source_path).unwrap(); + + let mut project = if_image::edit::Project::open(&source_path).unwrap(); + project + .apply(&if_image::edit::Edit::Rename { path: "A very long filename.txt".into(), to: "renamed.txt".into() }) + .unwrap(); + project.apply(&if_image::edit::Edit::Remove("SUBDIR/NESTED.TXT".into())).unwrap(); + project.apply(&if_image::edit::Edit::MkDir("created".into())).unwrap(); + project.apply(&if_image::edit::Edit::Add { dir: String::new(), from: added_file.clone() }).unwrap(); + project.apply(&if_image::edit::Edit::Label("EDITED".into())).unwrap(); + project.save(&output_path, |_, _| true).unwrap(); + + assert_eq!(if_image::vhd::VhdFile::open(&output_path).unwrap().is_dynamic(), dynamic); + let (format, source) = if_image::container::open(&output_path).unwrap(); + assert_eq!(format, if_image::container::Format::Vhd); + let mut volume = Volume::open(source, None).unwrap(); + assert_eq!(volume.fs.label(), "EDITED"); + assert!(volume.lookup("renamed.txt").is_ok()); + assert!(volume.lookup("created").unwrap().dir); + assert!(volume.lookup("SUBDIR/NESTED.TXT").is_err()); + let added = volume.lookup("added.txt").unwrap(); + let mut added_bytes = vec![0; added.size as usize]; + volume.read(&added, 0, &mut added_bytes).unwrap(); + assert_eq!(added_bytes, b"added through VHD editing"); + assert_eq!(std::fs::read(&source_path).unwrap(), original_vhd); + + let cancelled = dir.join(format!("cancelled-{index}.vhd")); + let mut project = if_image::edit::Project::open(&source_path).unwrap(); + project.apply(&if_image::edit::Edit::Label("CANCEL".into())).unwrap(); + assert!(matches!(project.save(&cancelled, |_, _| false), Err(if_image::Error::Cancelled))); + assert!(!cancelled.exists()); + + let existing = dir.join(format!("existing-{index}.vhd")); + std::fs::write(&existing, b"preserve").unwrap(); + let mut project = if_image::edit::Project::open(&source_path).unwrap(); + project.apply(&if_image::edit::Edit::Label("COLLISION".into())).unwrap(); + assert!(project.save(&existing, |_, _| true).is_err()); + assert_eq!(std::fs::read(&existing).unwrap(), b"preserve"); + + std::fs::remove_file(source_path).unwrap(); + std::fs::remove_file(output_path).unwrap(); + std::fs::remove_file(existing).unwrap(); + } + + std::fs::remove_file(added_file).unwrap(); + std::fs::remove_dir(dir).unwrap(); +} + #[test] fn reads_a_fat_partition_at_a_non_2048_byte_aligned_mbr_offset() { let disk = partitioned(fat_fixture(FatType::Fat16)); @@ -275,11 +372,108 @@ fn reports_exfat_as_unsupported() { } #[test] -fn refuses_to_edit_fat_images_until_fat_writing_is_supported() { - let path = std::env::temp_dir().join(format!("if-image-fat-readonly-{}.img", std::process::id())); - std::fs::write(&path, fat_fixture(FatType::Fat16).image).unwrap(); +fn edits_a_fat_image_without_modifying_the_source() { + use if_image::edit::{Edit, Project}; + + let dir = std::env::temp_dir().join(format!("if-image-fat-edit-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + std::fs::write(dir.join("added.txt"), b"new file").unwrap(); + std::fs::write(dir.join("long filename alpha.txt"), b"alpha").unwrap(); + std::fs::write(dir.join("long filename beta.txt"), b"beta").unwrap(); + std::fs::write(dir.join("duplicate.txt"), b"first").unwrap(); + std::fs::create_dir(dir.join("replacement")).unwrap(); + std::fs::write(dir.join("replacement").join("duplicate.txt"), b"second").unwrap(); + + for (index, kind) in [FatType::Fat12, FatType::Fat16, FatType::Fat32].into_iter().enumerate() { + let fixture = fat_fixture(kind); + let path = dir.join(format!("source-{index}.img")); + let output = dir.join(format!("edited-{index}.img")); + std::fs::write(&path, &fixture.image).unwrap(); + + let mut project = Project::open(&path).unwrap(); + project.apply(&Edit::Rename { path: "A very long filename.txt".into(), to: "renamed.txt".into() }).unwrap(); + project.apply(&Edit::Remove("SUBDIR/NESTED.TXT".into())).unwrap(); + project.apply(&Edit::MkDir("New/Nested".into())).unwrap(); + project.apply(&Edit::Add { dir: "New/Nested".into(), from: dir.join("added.txt") }).unwrap(); + project.apply(&Edit::Add { dir: "".into(), from: dir.join("long filename alpha.txt") }).unwrap(); + project.apply(&Edit::Add { dir: "".into(), from: dir.join("long filename beta.txt") }).unwrap(); + project.apply(&Edit::Add { dir: "".into(), from: dir.join("duplicate.txt") }).unwrap(); + project.apply(&Edit::Add { dir: "".into(), from: dir.join("replacement").join("duplicate.txt") }).unwrap(); + project.apply(&Edit::Label("EDITED".into())).unwrap(); + project.save(&output, |_, _| true).unwrap(); + + assert_eq!(std::fs::read(&path).unwrap(), fixture.image); + let mut volume = Volume::open(Box::new(MemSource(std::fs::read(&output).unwrap())), None).unwrap(); + assert_eq!(volume.fs.label(), "EDITED"); + let renamed = volume.lookup("renamed.txt").unwrap(); + let mut bytes = vec![0; renamed.size as usize]; + volume.read(&renamed, 0, &mut bytes).unwrap(); + assert_eq!(&bytes[..512], vec![b'a'; 512]); + assert_eq!(&bytes[512..], vec![b'b'; 88]); + assert!(volume.lookup("SUBDIR/NESTED.TXT").is_err()); + assert_eq!(volume.lookup("New/Nested/added.txt").unwrap().size, 8); + for (name, expected) in [ + ("long filename alpha.txt", b"alpha".as_slice()), + ("long filename beta.txt", b"beta".as_slice()), + ("duplicate.txt", b"second".as_slice()), + ] { + let entry = volume.lookup(name).unwrap(); + let mut actual = vec![0; entry.size as usize]; + volume.read(&entry, 0, &mut actual).unwrap(); + assert_eq!(actual, expected); + } + } + + std::fs::remove_dir_all(dir).unwrap(); +} + +#[test] +fn edits_a_partitioned_fat_volume_and_preserves_the_mbr() { + use if_image::edit::{Edit, Project}; + + let dir = std::env::temp_dir().join(format!("if-image-fat-mbr-edit-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let source_bytes = partitioned(fat_fixture(FatType::Fat16)); + let source = dir.join("source.img"); + let output = dir.join("edited.img"); + std::fs::write(&source, &source_bytes).unwrap(); + + let mut project = Project::open(&source).unwrap(); + project.apply(&Edit::Rename { path: "A very long filename.txt".into(), to: "changed.txt".into() }).unwrap(); + project.save(&output, |_, _| true).unwrap(); + + let result = std::fs::read(&output).unwrap(); + assert_eq!(&result[..512], &source_bytes[..512]); + let mut volume = Volume::open(Box::new(MemSource(result)), None).unwrap(); + assert!(volume.lookup("changed.txt").is_ok()); + assert_eq!(std::fs::read(&source).unwrap(), source_bytes); + std::fs::remove_dir_all(dir).unwrap(); +} + +#[test] +fn fat_edit_refuses_existing_destinations_and_cleans_up_on_cancel() { + use if_image::edit::Project; + + let dir = std::env::temp_dir().join(format!("if-image-fat-safety-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).unwrap(); + let source = dir.join("source.img"); + std::fs::write(&source, fat_fixture(FatType::Fat16).image).unwrap(); + let existing = dir.join("existing.img"); + std::fs::write(&existing, b"preserve").unwrap(); + + let error = match Project::open(&source).unwrap().save(&existing, |_, _| true) { + Ok(_) => panic!("editing should not overwrite an existing destination"), + Err(error) => error, + }; + assert!(error.to_string().contains("destination already exists")); + assert_eq!(std::fs::read(&existing).unwrap(), b"preserve"); - let error = if_image::edit::Project::open(&path).err().unwrap().to_string(); - assert!(error.contains("FAT images are read-only")); - std::fs::remove_file(path).unwrap(); + let cancelled = dir.join("cancelled.img"); + assert!(Project::open(&source).unwrap().save(&cancelled, |_, _| false).is_err()); + assert!(!cancelled.exists()); + assert!(!dir.join("cancelled.img.part").exists()); + std::fs::remove_dir_all(dir).unwrap(); } diff --git a/crates/isofoundry/src/cli_image.rs b/crates/isofoundry/src/cli_image.rs index 46d1495..0f90495 100644 --- a/crates/isofoundry/src/cli_image.rs +++ b/crates/isofoundry/src/cli_image.rs @@ -629,8 +629,8 @@ mod tests { path } - fn fat12_fixture() -> PathBuf { - let d = tmp("fat12"); + fn fat12_fixture(name: &str) -> PathBuf { + let d = tmp(name); std::fs::create_dir_all(&d).unwrap(); let mut image = vec![0; 2880 * 512]; let boot = &mut image[..512]; @@ -711,7 +711,7 @@ mod tests { #[test] fn image_info_and_extract_support_fat() { - let image = fat12_fixture(); + let image = fat12_fixture("fat12-read"); let (code, out, err) = call(&["image-info", image.to_str().unwrap(), "--json"]); assert_eq!(code, 0, "{err}"); let info: serde_json::Value = serde_json::from_str(out.trim()).unwrap(); @@ -734,6 +734,110 @@ mod tests { std::fs::remove_dir_all(image.parent().unwrap()).unwrap(); } + #[test] + fn image_cli_supports_fixed_and_dynamic_vhds() { + let image = fat12_fixture("vhd-cli"); + let raw = std::fs::read(&image).unwrap(); + let dir = image.parent().unwrap(); + + for (index, dynamic) in [false, true].into_iter().enumerate() { + let source = dir.join(format!("source-{index}.vhd")); + let mut raw_source = if_image::source::MemSource(raw.clone()); + if dynamic { + if_image::vhd_write::write_dynamic(&mut raw_source, &source, |_, _| true).unwrap(); + } else { + if_image::vhd_write::write_fixed(&mut raw_source, &source, |_, _| true).unwrap(); + } + let original = std::fs::read(&source).unwrap(); + + let (code, out, err) = call(&["image-info", source.to_str().unwrap(), "--json"]); + assert_eq!(code, 0, "{out}{err}"); + let info: serde_json::Value = serde_json::from_str(out.trim()).unwrap(); + assert_eq!(info["file_systems"], json!(["FAT"])); + assert_eq!(info["label"], "FAT CLI"); + + let extracted = dir.join(format!("extracted-{index}")); + let (code, out, err) = call(&[ + "extract", + source.to_str().unwrap(), + "--out", + extracted.to_str().unwrap(), + "--path", + "hello.txt", + "--fs", + "fat", + ]); + assert_eq!(code, 0, "{out}{err}"); + assert_eq!(std::fs::read(extracted.join("hello.txt")).unwrap(), b"hello"); + + let edited = dir.join(format!("edited-{index}.vhd")); + let (code, out, err) = call(&[ + "edit-image", + source.to_str().unwrap(), + "--out", + edited.to_str().unwrap(), + "--rename", + "hello.txt=renamed.txt", + "--label", + "EDITED", + ]); + assert_eq!(code, 0, "{out}{err}"); + assert_eq!(std::fs::read(&source).unwrap(), original); + + let edited_files = dir.join(format!("edited-files-{index}")); + let (code, out, err) = call(&[ + "extract", + edited.to_str().unwrap(), + "--out", + edited_files.to_str().unwrap(), + "--path", + "renamed.txt", + "--fs", + "fat", + ]); + assert_eq!(code, 0, "{out}{err}"); + assert_eq!(std::fs::read(edited_files.join("renamed.txt")).unwrap(), b"hello"); + + std::fs::remove_dir_all(extracted).unwrap(); + std::fs::remove_dir_all(edited_files).unwrap(); + std::fs::remove_file(source).unwrap(); + std::fs::remove_file(edited).unwrap(); + } + + std::fs::remove_dir_all(dir).unwrap(); + } + + #[test] + fn edit_image_supports_fat_and_keeps_the_source() { + let image = fat12_fixture("fat12-edit"); + let output = image.parent().unwrap().join("edited.img"); + let before = std::fs::read(&image).unwrap(); + let (code, out, err) = call(&[ + "edit-image", + image.to_str().unwrap(), + "--out", + output.to_str().unwrap(), + "--rename", + "hello.txt=renamed.txt", + "--label", + "EDITED", + ]); + assert_eq!(code, 0, "{out}{err}"); + assert_eq!(std::fs::read(&image).unwrap(), before); + + let (code, out, err) = call(&["image-info", output.to_str().unwrap(), "--json"]); + assert_eq!(code, 0, "{err}"); + let info: serde_json::Value = serde_json::from_str(out.trim()).unwrap(); + assert_eq!(info["label"], "EDITED"); + + let extracted = image.parent().unwrap().join("edited-files"); + let (code, _, err) = + call(&["extract", output.to_str().unwrap(), "--out", extracted.to_str().unwrap(), "--path", "renamed.txt"]); + assert_eq!(code, 0, "{err}"); + assert_eq!(std::fs::read(extracted.join("renamed.txt")).unwrap(), b"hello"); + std::fs::remove_dir_all(image.parent().unwrap()).unwrap(); + } + #[test] fn create_image_can_build_xdvdfs() { let d = tmp("create-xiso"); diff --git a/crates/isofoundry/src/win/tasks/browse.rs b/crates/isofoundry/src/win/tasks/browse.rs index ae1ba2c..54fcdbf 100644 --- a/crates/isofoundry/src/win/tasks/browse.rs +++ b/crates/isofoundry/src/win/tasks/browse.rs @@ -87,7 +87,7 @@ unsafe extern "system" fn wndproc(hwnd: HWND, msg: u32, wp: WPARAM, lp: LPARAM) if let Some(p) = pick_open( hwnd, "Open an image", - "Disc images|*.iso;*.cue;*.ccd;*.chd;*.nrg;*.mds;*.bin;*.img|All files|*.*", + "Disc images|*.iso;*.cue;*.ccd;*.chd;*.nrg;*.mds;*.bin;*.img;*.vhd|All files|*.*", ) { load(hwnd, p); } diff --git a/crates/isofoundry/src/win/tasks/edit.rs b/crates/isofoundry/src/win/tasks/edit.rs index 6e358a8..b5787e8 100644 --- a/crates/isofoundry/src/win/tasks/edit.rs +++ b/crates/isofoundry/src/win/tasks/edit.rs @@ -6,6 +6,7 @@ use super::super::gui; use super::super::taskwin::*; use if_image::edit::Edit; +use if_image::fs::FsKind; use std::path::PathBuf; use std::sync::Arc; use windows_sys::Win32::Foundation::{HWND, LPARAM, LRESULT, WPARAM}; @@ -44,6 +45,8 @@ struct State { /// (path inside the image, is folder) in list order. entries: Vec<(String, bool)>, ops: Vec, + fat: bool, + vhd: bool, shared: Option>, running: bool, close_when_done: bool, @@ -65,6 +68,8 @@ pub fn open(owner: HWND) { image: None, entries: Vec::new(), ops: Vec::new(), + fat: false, + vhd: false, shared: None, running: false, close_when_done: false, @@ -134,7 +139,7 @@ unsafe extern "system" fn wndproc(hwnd: HWND, msg: u32, wp: WPARAM, lp: LPARAM) if let Some(p) = pick_open( hwnd, "Open an image to edit", - "Disc images|*.iso;*.cue;*.ccd;*.chd;*.nrg;*.mds;*.bin;*.img|All files|*.*", + "Disc images|*.iso;*.cue;*.ccd;*.chd;*.nrg;*.mds;*.bin;*.img;*.vhd|All files|*.*", ) { load(hwnd, p); } @@ -297,6 +302,8 @@ fn load(hwnd: HWND, path: PathBuf) { return; } }; + let fat = v.fs.kind() == FsKind::Fat; + let vhd = path.extension().is_some_and(|ext| ext.eq_ignore_ascii_case("vhd")); let root = v.root(); match v.walk(&root) { Ok(items) => { @@ -312,6 +319,8 @@ fn load(hwnd: HWND, path: PathBuf) { let lbl = v.fs.label(); set_text(s.label_edit, &lbl); s.image = Some(path); + s.fat = fat; + s.vhd = vhd; s.ops.clear(); list_clear(s.ops_list); set_text(s.status, "Stage changes, then Save as… to write a new image."); @@ -333,9 +342,15 @@ fn save(hwnd: HWND) { set_text(s.status, "No changes staged yet."); return; } - let initial = image.with_file_name("edited.iso"); + let (initial, filter, extension) = if s.vhd { + (image.with_file_name("edited.vhd"), "Virtual hard disks (*.vhd)|*.vhd", "vhd") + } else if s.fat { + (image.with_file_name("edited.img"), "Raw disk images (*.img)|*.img", "img") + } else { + (image.with_file_name("edited.iso"), "ISO images (*.iso)|*.iso", "iso") + }; let name = initial.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default(); - let Some(dest) = pick_save(hwnd, "Save edited image as", "ISO images (*.iso)|*.iso", "iso", &name) else { + let Some(dest) = pick_save(hwnd, "Save edited image as", filter, extension, &name) else { return; }; let ops = s.ops.clone(); diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index c0d5ff1..d2c669d 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -25,7 +25,7 @@ The helper exits when its job finishes. It accepts only typed job messages (serd | `isofoundry-helper` | elevated binary | windows | | `if-core` | job model, journal, settings, errors (stable codes) | serde, thiserror | | `if-drive` | `OpticalDrive` trait; IMAPI2 backend; MMC/SPTI backend; fake backend | windows | -| `if-image` | read/write ISO9660/Joliet/UDF, BIN/CUE, CCD, raw IMG, CHD; readers for MDS, NRG, CDI | — | +| `if-image` | read/write ISO9660/Joliet/UDF, BIN/CUE, CCD, raw IMG, CHD; readers for MDS, NRG, CDI; FAT and fixed/dynamic VHD | — | | `if-audio` | decode (Symphonia, MF), encode (FLAC, LAME dyn, Opus, Vorbis, MF AAC), CD-DA ripping, CUE, CD-Text | symphonia | | `if-backup` | ZIP64 spanning, manifest, AES-256 | zip | | `usbwriter` | shared with WinISOPilot: enumerate, lock, partition (MBR/GPT), write image, boot files | windows | @@ -68,9 +68,9 @@ Every job follows: **plan → preview → confirm (proportional) → journal → ## 5. Image engine -- Writers: ISO9660 (levels 1–3, Rock Ridge off), Joliet, UDF 1.02/1.50/2.01/2.50/2.60, hybrid ISO+UDF, El Torito (BIOS + UEFI). +- Writers: ISO9660 (levels 1–3, Rock Ridge off), Joliet, UDF 1.02/1.50/2.01/2.50/2.60, hybrid ISO+UDF, El Torito (BIOS + UEFI), FAT12/16/32 raw-volume editing to a separate image. - Readers: above + FAT12/16/32 (raw superfloppy or primary MBR partition, browse/extract), XDVDFS (read-only browse/extract), BIN/CUE (multi-track, audio), CCD/IMG/SUB, MDS/MDF (v1 only, reverse-engineered: Unverified), NRG v1/v2, CDI, DMG (UDIF raw/zlib, HFS+ read-only browse), WIM (browse via wimlib or own reader: decide M1). -- Edit image: load tree, add/remove/rename, rebuild to new file (never in place). +- Edit image: load tree, add/remove/rename, rebuild ISO images or rewrite supported FAT volumes to a new file (never in place). - Convert: any readable → ISO / BIN-CUE / CCD / IMG / CHD. ## 6. Audio diff --git a/docs/LIMITATIONS.md b/docs/LIMITATIONS.md index 6841a23..b5769b7 100644 --- a/docs/LIMITATIONS.md +++ b/docs/LIMITATIONS.md @@ -23,6 +23,7 @@ Kept current every release. Items move out only with evidence. - CHD: reads `zlib` and `cdzl` hunks only. CHDs compressed with chdman's defaults (LZMA, FLAC, Huffman) need `chdman copy -c cdzl` (CD) or `-c zlib` (DVD) first. Parent (delta) CHDs are not supported. IsoFoundry writes `cdzl`/`zlib`, so its CHDs are a few percent larger than chdman's defaults. - NRG and MDS/MDF are read only and unverified; first session only. Multi-session images and CD-Text are not converted. - XDVDFS/Xbox XISO images can be browsed and extracted, and the CLI can create new XISOs. Editing existing XISOs, volume labels, and El Torito boot entries are unsupported. File sizes and data sectors use 32-bit fields; directory tree links use 16-bit dword offsets. Game executable patching and copy-protection circumvention are out of scope. +- Raw FAT12/16/32 floppy images and the first supported primary MBR FAT partition can be browsed, extracted, and edited to a new image. Fixed and dynamic VHD 1.0 containers expose the same FAT workflows and are saved back in their original VHD type. Editing preserves volume geometry and boot/partition sectors, but resets per-file timestamps and attributes. Edits are limited to 16 million clusters, 64 MB FAT tables, and 4 million allocated clusters. GPT and extended/logical partitions, VHDX, differencing VHD chains, and exFAT are unsupported. VHD editing temporarily materializes the virtual disk and therefore needs free space for the raw working copy and output. - Converting a multi-track CD to ISO keeps only the first data track. - Editing rewrites the image as ISO 9660 + Joliet + UDF 1.02. Hidden flags, Rock Ridge permissions and UDF 2.x extras of the source are not kept. Images that boot with floppy or hard-disk emulation can't be edited yet. Boot images that are also visible files are stored twice. - Mounting uses `AttachVirtualDisk`; whether it works without administrator rights depends on Windows policy and is untested outside CI (where the account is an administrator). A mount lasts until `unmount` or restart. @@ -69,8 +70,9 @@ Kept current every release. Items move out only with evidence. - Several jobs at once means several task windows; there is no central job queue. -- Copying a USB stick to an image has no window yet — it works - from the command line (`usb read`). +- The USB-to-image window reads physical disks through the elevated + helper and disk-image files through the in-process route. Physical USB + hardware remains unverified; see the USB notes above. ## Audio (M3) - Opus and Ogg Vorbis **encoding** is not offered: there is no permissive pure-Rust encoder for either format and the C libraries cannot be built by this project's toolchain. Decoding both works. diff --git a/docs/PARITY.md b/docs/PARITY.md index b6ab678..2c5d1de 100644 --- a/docs/PARITY.md +++ b/docs/PARITY.md @@ -28,7 +28,7 @@ limit · Missing = not there yet. | Surface / quality scan | Have | M5 | | Recover files from a damaged disc | Have | M5 | | Spanning backup / restore to discs | Have | M6 | -| USB stick → image window | Missing | Works from the command line (`usb read`); window planned after 1.0 | +| USB stick → image | Have | Task-grid variant and CLI; physical USB hardware remains unverified | | Blu-ray video (BDMV) authoring | Missing | Post-1.0 candidate | | Copy-protection handling | Never | Out of scope by design | diff --git a/docs/roadmap.md b/docs/roadmap.md index 1d90e09..60cf459 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -36,7 +36,7 @@ Out of scope: anything for copy-protected discs (SecuROM, SafeDisc, LibCrypt and ### Automation and image tools - [x] **Auto-action on disc insert** (idea from [Automatic Ripping Machine](https://github.com/automatic-ripping-machine/automatic-ripping-machine), MIT): optional mode that detects a newly inserted disc and identifies it (audio / data). As built (M6, D-051) it opens the chosen job's window rather than starting the job itself — the output location is a per-job choice; encrypted video discs are not ripped. -- [ ] **Edit image files** (reference: [TotalImage](https://github.com/TotalImage/TotalImage), MIT): ISO9660/Joliet/UDF editing is available; FAT12/16/32 raw floppy and primary MBR images can now be browsed and extracted. FAT editing and VHD support remain in progress. (1.x) +- [ ] **Edit image files** (reference: [TotalImage](https://github.com/TotalImage/TotalImage), MIT): ISO9660/Joliet/UDF and FAT12/16/32 raw floppy/primary MBR images can be edited to a new file. VHD support remains in progress. (1.x) - [x] **Preservation dumping UI** (reference: [Redumper GUI](https://github.com/Deterous/Redumper-GUI), GPL-3.0, Rust): pick drive and system, show progress and the log, produce redump-style output. GPL code is not copied. (M5) - [x] Open, extract, and create Xbox XISO (XDVDFS) images (reference: [extract-xiso](https://github.com/XboxDev/extract-xiso)). The CLI creates XISOs; the GUI browse task opens and extracts them. Existing XISOs are read-only. Filesystem only; no patching of game executables. (1.x) diff --git a/tasks/plan.md b/tasks/plan.md index 72c222e..8eb64b9 100644 --- a/tasks/plan.md +++ b/tasks/plan.md @@ -1,91 +1,92 @@ -# Implementation Plan: IsoFoundry 1.x Roadmap Features +# Implementation Plan: IsoFoundry Capability Expansion ## Overview -Complete the four explicitly selected 1.x roadmap features: multiboot USB, FAT/VHD image editing, Xbox XISO, and Spanish UI. Keep existing product behavior and the in-progress USB-to-image GUI changes intact. Deliver each feature through the appropriate GUI and CLI surfaces, with fixtures, focused tests, documentation, and software-only boot verification. Do not claim physical-hardware verification. +Deliver the user's requested disc, image, archive, and USB capabilities in tested vertical slices, guided by the approved [capability map](../CAPABILITY-MAP.md). Preserve completed XISO/FAT work and the in-progress USB-to-image changes. The next implementation slice is fixed/dynamic VHD support for FAT12/16/32. The expanded catalog is staged; no feature is called complete until its user-facing path, focused tests, documentation, and software-only verification are in place. ## Scope and assumptions -- Multiboot USB supports multiple Linux ISO loopback entries and Windows/WinPE ISO or WIM boot entries on BIOS and UEFI. The app manages installation, adding/removing payloads, and per-entry options. Secure Boot support must use distributable signed components or document a clearly bounded limitation; users must not be told to disable Secure Boot as the default path. -- FAT image editing supports FAT12/16/32 floppy and partition images. VHD support means legacy fixed and dynamic VHD containers containing supported FAT volumes. VHDX support is not included unless existing formats or implementation evidence require it. -- Xbox XISO means XDVDFS filesystem support for opening, listing, extracting, and creating images. No executable patching, decryption, or copy-protection circumvention. -- Spanish UI covers all user-facing GUI strings, including dialogs, menus, errors and task windows. CLI output and technical documentation remain English unless needed to explain language selection. -- Physical hardware tests are excluded. Unit/integration fixtures and QEMU boot checks remain required where applicable. +- The first `image-foundation` module spec is approved: fixed and dynamic VHD 1.0 containing FAT12/16/32, using the existing image engine and GUI/CLI paths. VHDX editing, differencing chains, and other filesystems are outside this module. +- The previously approved XISO, FAT/VHD, multiboot USB, and Spanish UI work remains in scope. +- Hardware tests are not required. Use fixtures, simulated or image-backed devices, and QEMU where applicable; clearly mark hardware-dependent behavior unverified. +- Do not implement copy-protection circumvention or claim support for encrypted commercial media. +- Formatting, disc erasure, and raw writes require explicit target identification, proportionate confirmation, cancellation/error handling, and safe output behavior. +- New dependencies, broad format support, and OS-dependent operations require evidence and a per-module spec before implementation. ## Architecture decisions -- Extend `if-image`'s filesystem/source abstractions for FAT and XDVDFS rather than introducing parallel app-only parsers. Keep format writers pure Rust and platform-independent where practical. -- VHD parsing/writing is a container layer over byte-addressable storage; validate bounds, checksums, block maps and metadata before exposing a filesystem. -- Multiboot USB writes continue through the existing guarded USB device/helper path. Disk-image/VHDX test targets stay behind the current explicit development gate. -- Preserve the app's offline-by-default behavior. Any bootloader assets must have documented provenance and redistribution terms; no runtime download is required for ordinary use. -- Use a Spanish resource catalog with English fallback, explicit language selection, and completeness tests. Avoid scattering translated literals through Win32 window procedures. +- Keep parsing, filesystem integration, and image editing in the platform-independent `if-image` crate; GUI and CLI use the shared implementation. +- Validate all VHD offsets, checksums, block maps, lengths, and arithmetic before exposing a filesystem or allocating based on untrusted metadata. +- Always write edits to a separate destination; never modify a source image in place or overwrite an existing destination. +- Keep USB writes behind the existing guarded USB/helper path. Preserve the existing USB-to-image worktree changes and `.serena/` data. +- The approved capability map defines module boundaries and dependency order. Each remaining module receives its own spec and task breakdown before implementation. ## Task List -### Phase 0: Baseline and format investigation -- [ ] Task 1: Establish preserved baseline and validate third-party boot assets. - -### Checkpoint: Baseline -- [ ] Existing working-tree changes are preserved and baseline workspace checks pass. -- [ ] XISO, FAT/VHD, and multiboot format/asset decisions are supported by repository evidence and documented references. +### Phase 0: Preserved baseline +- [x] Task 1: Record the existing workspace state and establish repository/asset constraints. ### Phase 1: Xbox XISO - [x] Task 2: Read and validate XDVDFS volumes. - [x] Task 3: Expose XISO browse and extraction in CLI and GUI. - [x] Task 4: Create XISO images and add end-to-end coverage. -### Checkpoint: XISO -- [x] Fixture images list, extract and recreate files with byte-for-byte verification. -- [x] CLI and GUI workflows are documented and tested without physical hardware. - -### Phase 2: FAT and VHD image editing +### Phase 2: Image foundation — FAT and VHD - [x] Task 5: Read FAT12/16/32 volumes and locate supported partitions. - [x] Task 6: Browse and extract FAT images through existing image workflows. -- [ ] Task 7: Edit FAT images and save safely as a new image. -- [ ] Task 8: Read and write fixed/dynamic VHD containers with validated metadata. -- [ ] Task 9: Integrate VHD-contained FAT editing and verify round trips. - -### Checkpoint: FAT/VHD -- [ ] FAT12/16/32 fixture images and fixed/dynamic VHD fixtures browse, extract, edit and reopen correctly. -- [ ] Original images remain unchanged; malformed and oversized inputs fail cleanly. - -### Phase 3: Multiboot USB -- [ ] Task 10: Prove the bootloader/asset path and reproducible BIOS/UEFI QEMU test setup. -- [ ] Task 11: Install a multiboot layout through the guarded USB engine. -- [ ] Task 12: Manage ISO/WIM payloads and persistent per-entry boot options. -- [ ] Task 13: Integrate multiboot management in the Windows task window and CLI. -- [ ] Task 14: Verify boot scenarios and document supported payloads and Secure Boot status. - -### Checkpoint: Multiboot -- [ ] VHDX-backed install/manage operations pass safety tests; supported BIOS and UEFI scenarios boot under QEMU. -- [ ] No destructive physical-device operation is used for testing; Secure Boot claims match the verified asset chain. - -### Phase 4: Spanish UI -- [ ] Task 15: Add locale/resource loading and language selection with English fallback. -- [ ] Task 16: Localize shared menus, task catalog, and core dialogs. -- [ ] Task 17: Localize task windows and operational messages in bounded groups. -- [ ] Task 18: Enforce catalog completeness and verify the Spanish UI workflow. - -### Checkpoint: Spanish UI -- [ ] Every user-facing GUI string has an English source and Spanish translation. -- [ ] English remains the default/fallback; both locales render through the same GUI paths. +- [x] Task 7: Edit FAT images and save safely as a new image. +- [x] Task 8: Parse fixed/dynamic VHD containers and expose bounded logical reads. +- [x] Task 9: Write fixed/dynamic VHD containers safely to new files. +- [x] Task 10: Browse and extract FAT volumes inside supported VHDs. +- [x] Task 11: Edit FAT volumes inside VHDs through CLI and GUI. +- [x] Task 12: Document VHD support and limits. + +### Checkpoint: Image foundation +- [x] Fixed/dynamic VHD fixtures with supported FAT volumes browse, extract, edit, and reopen correctly. +- [x] Malformed/unsupported inputs fail cleanly; source images remain unchanged and destination collisions are preserved. +- [x] Focused `if-image` and app tests, Clippy, formatting, and app build pass. +- [x] README and limitations document VHD variants and exclusions. + +### Phase 3: USB creation and multiboot +- [ ] Task 13: Prove the multiboot boot chain, asset terms, and QEMU test path. +- [ ] Task 14: Install multiboot layouts through the guarded USB engine. +- [ ] Task 15: Manage ISO/WIM payloads and persistent boot options. +- [ ] Task 16: Integrate multiboot management in the Windows app and CLI. +- [ ] Task 17: Verify and document supported multiboot scenarios. + +### Checkpoint: USB creation +- [ ] File-backed target install/manage flows pass safety, cancellation, and recovery tests. +- [ ] Supported BIOS/UEFI scenarios boot in QEMU; hardware behavior remains explicitly unverified. +- [ ] Secure Boot claims reflect the verified boot chain and asset terms. + +### Phase 4: Remaining image and media capabilities + +After image foundation and USB creation, specify and implement the approved-map modules in dependency order: `image-operations`, `archive-formats`, `optical-media`, and `virtual-mounting`; then `media-conversion` (which depends on optical media). This scope includes practical image create/extract/edit/convert/verify operations, useful supported compressed/container formats, supported optical read/write workflows, virtual mounting, and unprotected-media extraction/conversion. Each module gets its own reviewed spec and focused task list before implementation; format breadth and OS-dependent behavior are evidence-gated. + +### Phase 5: Workflow surfaces and localization +- [ ] Task 18: Add locale resources and language selection with English fallback. +- [ ] Task 19: Localize shared menus, task catalog, and core dialogs. +- [ ] Task 20: Localize task windows in bounded groups. +- [ ] Task 21: Enforce localization completeness and final acceptance. + +CLI/scripting, batch queues, portable operation, diagnostics, and capability/limitation documentation are part of `workflow-surfaces`. Their scope and dependencies will be specified after the underlying feature modules exist, so the UI/catalog coverage is complete rather than partial. ### Checkpoint: Complete -- [ ] All four selected feature acceptance criteria are met. +- [ ] Every approved module is end-to-end implemented, focused-tested, documented, and accurately bounded. - [ ] Workspace formatting, tests, Clippy, and application build pass. -- [ ] README, roadmap, limitations, and changelog accurately describe support and verification. +- [ ] No physical-hardware verification is claimed. ## Risks and mitigations | Risk | Impact | Mitigation | |---|---|---| -| Secure Boot-compatible multiboot assets may have restrictive distribution terms or key-enrollment constraints. | High | Validate provenance/license and the actual trust chain before choosing the bootloader; do not make unsupported Secure Boot claims. | -| Dynamic VHD and FAT partition edge cases can cause corruption or out-of-bounds reads. | High | Keep parsers read-only until validation is complete; test malformed metadata, bounds, and atomic new-file writes. | -| Booting arbitrary Linux ISOs by loopback is not universal. | High | Define a tested compatibility set and expose/document per-entry options rather than claiming every ISO works. | -| XDVDFS and FAT name/metadata semantics differ from ISO editing. | Medium | Preserve raw names and metadata where feasible; add adversarial fixtures and explicit unsupported-feature errors. | -| UI strings are spread across many Win32 task windows. | High | Centralize string lookup and migrate windows in small groups with a coverage test preventing untranslated literals. | -| Existing USB-to-image changes are uncommitted in this worktree. | High | Treat them and `.serena/` as user/worktree state; do not reset, overwrite, or clean them up. | +| Dynamic VHD metadata or FAT layout edge cases can cause unsafe reads or corruption. | High | Use bounded parsing and checked arithmetic; test malformed metadata; write only to a separate atomic output. | +| Multiboot assets may have restrictive distribution terms or key-enrollment constraints. | High | Validate provenance/license and the actual trust chain before choosing assets; avoid unsupported Secure Boot claims. | +| Loopback boot support varies among Linux and Windows images. | High | Define a tested compatibility set and document unsupported payloads rather than promise universal booting. | +| Destructive disk or optical operations can erase user data. | High | Preserve existing target safety checks and require clear target identity and proportionate confirmation on every route. | +| The feature catalog includes proprietary formats, protected media, hardware-specific operations, and platform-specific APIs. | High | Exclude copy-protection circumvention; implement only formats and operations with reliable evidence; document host/hardware limits. | +| Existing USB-to-image changes and `.serena/` are uncommitted. | High | Preserve them; do not reset, overwrite, or stage them with unrelated feature commits. | ## Open questions -- None blocking at planning time. The plan uses the supported-format and Secure Boot assumptions above; revise only if format research disproves them. +- No blocking questions for the approved VHD module. Per-module specifications remain approval-gated as described in `CAPABILITY-MAP.md`. diff --git a/tasks/todo.md b/tasks/todo.md index ea2a6a1..32100c5 100644 --- a/tasks/todo.md +++ b/tasks/todo.md @@ -102,58 +102,108 @@ **Files likely touched:** `crates/if-image/src/container.rs`, `crates/if-image/src/fs.rs`, `crates/isofoundry/src/cli_image.rs`, `crates/isofoundry/src/win/tasks/browse.rs` **Estimated scope:** Medium -## Task 7: Editing FAT images and saving safely +## Task 7: Editing FAT images and saving safely — Done **Description:** Extend edit-image operations to supported FAT volumes and rebuild to a separate output image while preserving boot/partition metadata that the implementation can safely retain. **Acceptance criteria:** -- [ ] Add, remove, rename, create-folder, and label operations work for FAT images. -- [ ] The source is never modified; existing-destination and source/destination identity rules are enforced. -- [ ] Unsupported boot sectors, partitions, or metadata are reported rather than silently discarded. +- [x] Add, remove, rename, create-folder, and label operations work for FAT images. +- [x] The source is never modified; existing-destination and source/destination identity rules are enforced. +- [x] Unsupported boot sectors, partitions, or metadata are reported rather than silently discarded. **Verification:** -- [ ] Add edit round-trip tests for all supported FAT types, fragmentation, collisions, cancellation, and source preservation. -- [ ] Exercise the existing CLI and GUI edit paths against raw FAT fixtures. +- [x] Add edit round-trip tests for all supported FAT types, fragmentation, collisions, cancellation, and source preservation. +- [x] Exercise the existing CLI edit path against raw FAT fixtures; the GUI uses the same edit API and selects a raw-image output extension for FAT. **Dependencies:** Task 5, Task 6 **Files likely touched:** `crates/if-image/src/fat_write.rs`, `crates/if-image/src/edit.rs`, `crates/isofoundry/src/cli_image.rs`, `crates/isofoundry/src/win/tasks/edit.rs`, focused tests **Estimated scope:** Medium -## Task 8: Reading and writing fixed/dynamic VHD containers +## Task 8: Parsing fixed and dynamic VHD containers — Done -**Description:** Add a container adapter for legacy VHD fixed and dynamic disks, validating footer/header checksums, BAT entries, block bitmaps, and virtual-size bounds. +**Description:** Add bounded, read-only support for fixed and dynamic VHD 1.0 images as logical byte/sector sources, validating metadata before exposing payload data. **Acceptance criteria:** -- [ ] Fixed and dynamic VHD fixtures expose logical sector reads correctly. -- [ ] Invalid checksums, overlapping/out-of-range blocks, and malformed BATs fail cleanly. -- [ ] Writes produce a new VHD file and never mutate the input. +- [x] Fixed and dynamic VHD fixtures expose logical reads, including sparse/unallocated dynamic blocks. +- [x] Footer/header checksums, offsets, sizes, BAT entries, and block maps are validated with checked arithmetic. +- [x] Truncated, malformed, out-of-range, overlapping, and differencing VHDs fail explicitly without unsafe allocation or I/O. **Verification:** -- [ ] Add fixed/dynamic read-write round-trip tests plus malformed-container tests. -- [ ] Compare generated fixtures with an independent VHD parser/tool where available. +- [x] Add fixture tests for fixed/dynamic reads and malformed metadata. +- [x] Check for an independent VHD parser; `qemu-img` is unavailable in this environment, so verification uses generated fixtures. **Dependencies:** Task 1 -**Files likely touched:** `crates/if-image/src/vhd.rs`, `crates/if-image/src/lib.rs`, tests +**Files likely touched:** `crates/if-image/src/vhd.rs`, `crates/if-image/src/source.rs`, `crates/if-image/src/lib.rs`, `crates/if-image/tests/vhd.rs` **Estimated scope:** Medium -## Task 9: Integrating VHD-contained FAT editing +## Task 9: Writing fixed and dynamic VHD containers safely -**Description:** Open VHD containers through the FAT image path and save edited virtual disks while retaining container type and safe metadata. +**Description:** Add fixed and dynamic VHD writers that produce validated new containers without changing source files or overwriting existing destinations. **Acceptance criteria:** -- [ ] Fixed and dynamic VHD images containing supported FAT volumes browse, extract, and edit through app workflows. -- [ ] Successful edits reopen in IsoFoundry and an independent VHD reader. -- [ ] All failures preserve both source and any pre-existing destination. +- [x] Fixed and dynamic writers produce images that the VHD reader reopens with byte-correct logical contents. +- [x] Footer/header metadata, checksums, block maps, and virtual sizes are valid and bounded. +- [x] Cancellation and write errors do not leave a success-shaped partial destination. **Verification:** -- [ ] Add end-to-end VHD/FAT edit tests, including malformed image and cancel cases. -- [ ] Run workspace tests and app build. +- [x] Add writer round-trip, destination-collision, cancellation, and malformed-output tests. +- [x] Use an independent VHD parser for generated fixtures when available; `qemu-img` is unavailable in this environment. -**Dependencies:** Task 7, Task 8 -**Files likely touched:** `crates/if-image/src/container.rs`, `crates/if-image/src/edit.rs`, CLI/GUI adapters, tests +**Dependencies:** Task 8 +**Files likely touched:** `crates/if-image/src/vhd.rs`, `crates/if-image/src/vhd_write.rs`, `crates/if-image/tests/vhd.rs` **Estimated scope:** Medium -## Task 10: Proving the multiboot boot chain and QEMU verification path +## Task 10: Browsing and extracting FAT volumes inside VHDs + +**Description:** Route supported fixed/dynamic VHD images through existing image-info, browse, and extraction workflows. + +**Acceptance criteria:** +- [x] Supported VHDs containing FAT12/16/32 volumes are recognized, browsed, and extracted by the shared image engine. +- [x] Existing ISO/XISO/raw FAT behavior remains unchanged. +- [x] Unsupported containers and VHDs without supported FAT volumes fail with actionable errors. + +**Verification:** +- [x] Add CLI and integration tests for image-info and byte-correct extraction from fixed/dynamic VHD fixtures. +- [x] Confirm the GUI browse/extract task uses the shared container path and retains cancellation behavior. + +**Dependencies:** Task 6, Task 8 +**Files likely touched:** `crates/if-image/src/container.rs`, `crates/isofoundry/src/cli_image.rs`, `crates/if-image/tests/vhd.rs` +**Estimated scope:** Medium + +## Task 11: Editing FAT volumes inside VHDs + +**Description:** Integrate the VHD reader/writer with the existing FAT edit API and expose safe new-VHD saves through the CLI and GUI. + +**Acceptance criteria:** +- [x] Add, remove, rename, create-folder, and label edits work for supported FAT volumes in fixed/dynamic VHDs. +- [x] Edited VHDs reopen with correct contents; source bytes and pre-existing destinations remain unchanged. +- [x] GUI and CLI use the shared implementation and report unsupported layouts and cancellation explicitly. + +**Verification:** +- [x] Add end-to-end edit round trips for fixed/dynamic VHDs, including collision and cancellation cases. +- [x] Run focused `if-image` and app tests and build the Windows application. + +**Dependencies:** Task 7, Task 9, Task 10 +**Files likely touched:** `crates/if-image/src/edit.rs`, `crates/isofoundry/src/cli_image.rs`, `crates/isofoundry/src/win/tasks/edit.rs`, VHD integration tests +**Estimated scope:** Medium + +## Task 12: Documenting VHD support and limits + +**Description:** Document supported VHD variants, FAT-only behavior, safe editing semantics, and explicit exclusions. + +**Acceptance criteria:** +- [x] README and architecture docs describe supported fixed/dynamic VHD workflows accurately. +- [x] Limitations identify unsupported VHDX, differencing chains, and unsupported filesystems. +- [x] Changelog states user-visible behavior without claiming hardware validation. + +**Verification:** +- [x] Review all VHD claims against passing fixture and integration tests. + +**Dependencies:** Task 11 +**Files likely touched:** `README.md`, `CHANGELOG.md`, `docs/ARCHITECTURE.md`, `docs/LIMITATIONS.md` +**Estimated scope:** Small + +## Task 13: Proving the multiboot boot chain and QEMU verification path **Description:** Select bootloader assets and define the minimal supported boot profiles, asset licensing/provenance, BIOS/UEFI layout, and reproducible QEMU software-test matrix before any destructive writer changes. @@ -170,7 +220,7 @@ **Files likely touched:** `docs/roadmap.md`, `docs/references.md`, test scripts/fixtures, possible `CREDITS.md` **Estimated scope:** Medium -## Task 11: Installing a multiboot layout through the guarded USB engine +## Task 14: Installing a multiboot layout through the guarded USB engine **Description:** Add an install path that uses existing target assessment, helper IPC, confirmation, and development image-target gates to partition/format a multiboot device and place boot assets. @@ -183,11 +233,11 @@ - [ ] Add file-backed device tests for partitioning, formatting, asset placement, cancellation, and safety denials. - [ ] Run helper protocol and USB writer suites; no physical disk is used. -**Dependencies:** Task 10 +**Dependencies:** Task 13 **Files likely touched:** `crates/usbwriter/src/multiboot.rs`, `crates/usbwriter/src/lib.rs`, CLI/helper route files, tests **Estimated scope:** Medium -## Task 12: Managing ISO/WIM payloads and persistent boot options +## Task 15: Managing ISO/WIM payloads and persistent boot options **Description:** Add validated add/remove/list operations for supported ISO/WIM payloads and a versioned per-entry configuration that survives content updates. @@ -200,11 +250,11 @@ - [ ] Add config round-trip, path-safety, collision, and payload manifest tests. - [ ] Boot at least one Linux ISO and one supported Windows/WinPE path under QEMU. -**Dependencies:** Task 10, Task 11 +**Dependencies:** Task 13, Task 14 **Files likely touched:** `crates/usbwriter/src/multiboot.rs`, `crates/if-core/src/settings.rs` or a dedicated config module, tests **Estimated scope:** Medium -## Task 13: Integrating multiboot management in the app and CLI +## Task 16: Integrating multiboot management in the app and CLI **Description:** Expose multiboot install/manage operations through a dedicated task window and documented CLI commands using existing progress, cancellation, confirmation, and helper patterns. @@ -217,11 +267,11 @@ - [ ] Add CLI parser/behavior tests and GUI adapter tests where the repository supports them. - [ ] Build the Windows app/helper and exercise complete workflows against a disposable disk image. -**Dependencies:** Task 11, Task 12 +**Dependencies:** Task 14, Task 15 **Files likely touched:** `crates/isofoundry/src/cli_usb.rs`, `crates/isofoundry/src/win/tasks`, `crates/isofoundry/src/win/gui.rs`, help/docs **Estimated scope:** Medium -## Task 14: Verifying and documenting multiboot scenarios +## Task 17: Verifying and documenting multiboot scenarios **Description:** Complete QEMU scenarios for supported BIOS/UEFI combinations and document compatibility boundaries, Secure Boot behavior, and recovery/update semantics. @@ -234,11 +284,11 @@ - [ ] Run the documented QEMU matrix and USB file-target suite. - [ ] Review README, LIMITATIONS, TESTING, and roadmap statements against results. -**Dependencies:** Task 13 +**Dependencies:** Task 16 **Files likely touched:** `docs/TESTING.md`, `docs/LIMITATIONS.md`, `docs/roadmap.md`, `README.md`, `CHANGELOG.md` **Estimated scope:** Medium -## Task 15: Adding locale resources and language selection +## Task 18: Adding locale resources and language selection **Description:** Add a resource-based string lookup with English and Spanish catalogs, English fallback, and an explicit language setting integrated with existing settings. @@ -251,11 +301,11 @@ - [ ] Add locale selection, fallback, settings migration, and missing-key tests. - [ ] Confirm existing task/window strings remain English before the translation sweep. -**Dependencies:** Tasks 3, 6, 9, 13 +**Dependencies:** Tasks 3, 6, 11, 16; add dependencies for approved catalog modules before starting localization. **Files likely touched:** `crates/if-core/src/strings.rs`, settings, resource files, `Cargo.toml`/`Cargo.lock` if needed **Estimated scope:** Medium -## Task 16: Localizing shared menus, task catalog, and core dialogs +## Task 19: Localizing shared menus, task catalog, and core dialogs **Description:** Move the main window, task catalog, menu, common control, and shared confirmation/error strings to the resource catalogs and provide Spanish translations. @@ -268,11 +318,11 @@ - [ ] Add catalog parity tests and exercise menu/task selection in both locales. - [ ] Build the Windows app and inspect key dialog text through UI automation or accessibility tree. -**Dependencies:** Task 15 +**Dependencies:** Task 18 **Files likely touched:** `crates/if-core/src/strings.rs`, main GUI, task catalog, resource files, tests **Estimated scope:** Medium -## Task 17: Localizing task windows in bounded groups +## Task 20: Localizing task windows in bounded groups **Description:** Migrate all task-window labels, prompts, progress/status messages, errors, and confirmation text to keyed resources in small batches. @@ -285,11 +335,11 @@ - [ ] Migrate and test no more than a few windows per change batch. - [ ] Run string coverage tests after each batch; finish with both-locale workflow smoke tests. -**Dependencies:** Task 16 +**Dependencies:** Task 19 **Files likely touched:** task-window modules in groups, resource catalogs, focused tests **Estimated scope:** Medium per batch -## Task 18: Enforcing localization completeness and final acceptance +## Task 21: Enforcing localization completeness and final acceptance **Description:** Add a repository test that enumerates all GUI string keys and ensures Spanish coverage, then align release docs and perform a final full validation. @@ -302,6 +352,6 @@ - [ ] Run `cargo fmt --check`, `cargo test --workspace`, `cargo clippy --workspace --all-targets -- -D warnings`, and `cargo build -p isofoundry`. - [ ] Run the QEMU matrix and all software-only format/USB integration tests. -**Dependencies:** Tasks 4, 9, 14, 17 +**Dependencies:** Tasks 4, 11, 12, 17, 20; add dependencies for approved catalog modules before final acceptance. **Files likely touched:** localization tests, README/docs, changelog, relevant feature tests **Estimated scope:** Medium From ec879ce32a121f34f994d3aad7b00f734af4f656 Mon Sep 17 00:00:00 2001 From: Sean Shaffer Date: Mon, 9 Nov 2026 00:32:34 -0600 Subject: [PATCH 12/12] docs: align roadmap and feature descriptions Separate the released 1.0.1 scope from current development features, remove stale roadmap status, and make capability descriptions match tested behavior and documented limitations. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- DESCRIPTION.md | 10 +++++----- README.md | 6 +++--- SPEC-image-foundation.md | 2 ++ docs/ARCHITECTURE.md | 18 +++++++++--------- docs/DECISIONS.md | 10 ++++++++++ docs/PARITY.md | 5 +++-- docs/roadmap.md | 8 +++++--- tasks/plan.md | 6 +++--- 8 files changed, 40 insertions(+), 25 deletions(-) diff --git a/DESCRIPTION.md b/DESCRIPTION.md index ae546f8..de13a84 100644 --- a/DESCRIPTION.md +++ b/DESCRIPTION.md @@ -1,11 +1,11 @@ # Description -IsoFoundry is a free, open-source Windows tool for burning discs, making and checking disc images, and writing bootable USB drives. It aims to match AnyBurn Pro's task-grid simplicity while adding features from other burning and imaging tools, and later the Windows ISO customization from WinISOPilot. +IsoFoundry is a free, open-source Windows tool for supported disc, image, audio, backup, and USB workflows. It uses a task grid and command-line tools. Windows ISO customization is a future roadmap item. -**Short (≤ 80 chars):** Open source disc burning, imaging and bootable USB suite for Windows. +**Short (≤ 80 chars):** Open-source Windows suite for discs, images, and bootable USB. -**Medium:** IsoFoundry burns CDs, DVDs and Blu-rays, rips and burns audio, creates and converts disc images, and builds bootable USB drives, all from one compact task grid or the command line. +**Medium:** IsoFoundry handles supported CD, DVD, and Blu-ray workflows; creates, checks, and edits supported images; and creates or reads bootable USB images from one task grid and command line. -**Long:** IsoFoundry is a free, open-source (MIT) suite for Windows 10 and 11. Pick a task from a single window: burn images or files to any optical disc, copy discs, rip audio CDs securely to FLAC, MP3 or Opus, burn audio CDs with CD-Text, create, edit, browse, convert and mount disc images, verify checksums, back up large folders across several discs, test disc surfaces, and create bootable or Windows-to-Go USB drives. Every task is also available from the command line. It works offline, collects no telemetry, and asks for administrator rights only for the tasks that need them. +**Long:** IsoFoundry is a free, open-source (MIT) suite for Windows 10 and 11. Its task grid and command-line tools cover supported optical-disc operations, audio conversion and ripping, backups, image creation and conversion, checksum verification, and bootable USB workflows. Image support includes ISO/UDF, Xbox XISO, raw FAT12/16/32, and fixed or dynamic VHDs containing supported FAT volumes. The USB-to-image workflow can read a physical disk or disk-image file to an image file. Audio encoding supports WAV, FLAC, MP3, and AAC/M4A; Opus and Vorbis are decode-only. Physical optical and USB hardware paths remain unverified; supported formats and other limits are documented in the README and limitations guide. The app works offline, collects no telemetry, and requests administrator rights only for tasks that need them. -**Keywords:** burn, iso, cd, dvd, blu-ray, rip, flac, bootable usb, disc image, bin cue, checksum, open source +**Keywords:** burn, iso, xiso, fat, vhd, cd, dvd, blu-ray, rip, flac, bootable usb, usb imaging, disc image, bin cue, checksum, open source diff --git a/README.md b/README.md index 44ad70a..d324aa5 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ IsoFoundry is a single free Windows program for working with CDs, DVDs, Blu-rays - **0.4.0** — M2 USB and M3 audio together (0.3.0 was never released separately): Create bootable USB, Install Windows to USB, and batch audio conversion with tags and ReplayGain. USB is tested end-to-end against disk-image files; writing a physical disk and booting from it are still to be confirmed. MP3 needs `libmp3lame.dll` at runtime; AAC uses Windows Media Foundation; both encode paths are **Unverified** on real hardware. - **0.2.0** — M1 image toolkit (create, browse, extract, edit, convert, mount, checksum, verify) in task windows and on the command line. - **0.1.0** — M0 shell: main window, command line and elevated helper. -- **Next** — 1.x: multiboot USB and Spanish UI. FAT/VHD image editing and Xbox XISO support are available in the current development branch; see the [roadmap](#roadmap). +- **Next** — 1.x: multiboot USB, Spanish UI, and a native Windows 11 context menu. FAT/VHD image editing, Xbox XISO support, and the USB-to-image task window are implemented after 1.0.1; see the [roadmap](#roadmap). ## What the program does today @@ -200,13 +200,13 @@ The 0.x milestones shipped as public pre-releases; **1.0** (2026-11-08) is the f |---|---|---| | M0 Shell | 0.1.0 ✅ | Window, task grid, CLI skeleton, settings, logs, elevated helper, packaging | | M1 Image tools | 0.2.0 | Create / browse / extract / edit / convert / mount images, checksums + `.dat`, verify — released in 0.2.0 | -| M2 USB | 0.3.0 | Bootable USB (GPT/MBR, UEFI/BIOS), Windows installer layouts, install Windows to USB, USB ↔ image | +| M2 USB | 0.4.0 | Bootable USB (GPT/MBR, UEFI/BIOS), Windows installer layouts, install Windows to USB, USB ↔ image | | M3 Audio convert | 0.4.0 | Decode/encode pipeline, tags, batch convert, ReplayGain | | M4 Burning (IMAPI2) | 0.5.0 | Burn image and files, multisession, erase, verify after burn | | M5 Disc & ripping (MMC) | 0.5.0 | Disc → image (raw + subchannel), disc → disc, secure audio ripping, audio CD burning, surface/quality scans, recovery | | M6 Integration | 0.9.0 | Spanning backup, auto-action on disc insert, Explorer verbs, opt-in update check, MusicBrainz, completion actions | | RC / 1.0 | 0.9.0 / 1.0 | Audit pass, failure drills, accessibility pass, measured budgets, release | -| 1.x | — | Multiboot USB, FAT/VHD image editing, Xbox XISO, Spanish UI, native Windows 11 context menu | +| 1.x | — | Multiboot USB, Spanish UI, native Windows 11 context menu | | 2.0 | — | Windows ISO customization integrated from [WinISOPilot](https://github.com/SeanShaffer88/WinISOPilot) | Full detail: [docs/roadmap.md](docs/roadmap.md). diff --git a/SPEC-image-foundation.md b/SPEC-image-foundation.md index 7213c2a..75cbb8b 100644 --- a/SPEC-image-foundation.md +++ b/SPEC-image-foundation.md @@ -1,5 +1,7 @@ # Spec: Image Foundation — VHD Support +**Status:** Implemented and documented in the current source. + ## Objective Complete the next `image-foundation` increment: support fixed and dynamic VHD containers for FAT12/16/32 image workflows, including inspect, browse, extract, and safe edit-to-new-image. This extends the existing ISO/XISO/FAT image engine and the already implemented FAT editor. It does not replace or discard existing USB-to-image work. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index d2c669d..c176f7a 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -1,6 +1,6 @@ # Architecture -Status: plan, pre-code. Every capability below is **Unverified** until a milestone marks it otherwise. +Status: implementation overview. Hardware-dependent paths remain **Unverified**; see [LIMITATIONS.md](LIMITATIONS.md) for per-feature boundaries. ## 1. Process model @@ -25,8 +25,8 @@ The helper exits when its job finishes. It accepts only typed job messages (serd | `isofoundry-helper` | elevated binary | windows | | `if-core` | job model, journal, settings, errors (stable codes) | serde, thiserror | | `if-drive` | `OpticalDrive` trait; IMAPI2 backend; MMC/SPTI backend; fake backend | windows | -| `if-image` | read/write ISO9660/Joliet/UDF, BIN/CUE, CCD, raw IMG, CHD; readers for MDS, NRG, CDI; FAT and fixed/dynamic VHD | — | -| `if-audio` | decode (Symphonia, MF), encode (FLAC, LAME dyn, Opus, Vorbis, MF AAC), CD-DA ripping, CUE, CD-Text | symphonia | +| `if-image` | read/write ISO9660/Joliet/UDF and XDVDFS; read BIN/CUE, CCD, CHD, MDS and NRG; browse/edit supported FAT volumes in raw images and fixed/dynamic VHDs | — | +| `if-audio` | decode (Symphonia, MF), encode (WAV, FLAC, LAME dyn, MF AAC), CD-DA ripping, CUE, CD-Text | symphonia | | `if-backup` | ZIP64 spanning, manifest, AES-256 | zip | | `usbwriter` | shared with WinISOPilot: enumerate, lock, partition (MBR/GPT), write image, boot files | windows | | `if-hash` | CRC32, MD5, SHA-1, SHA-256 streaming | — | @@ -68,21 +68,21 @@ Every job follows: **plan → preview → confirm (proportional) → journal → ## 5. Image engine -- Writers: ISO9660 (levels 1–3, Rock Ridge off), Joliet, UDF 1.02/1.50/2.01/2.50/2.60, hybrid ISO+UDF, El Torito (BIOS + UEFI), FAT12/16/32 raw-volume editing to a separate image. -- Readers: above + FAT12/16/32 (raw superfloppy or primary MBR partition, browse/extract), XDVDFS (read-only browse/extract), BIN/CUE (multi-track, audio), CCD/IMG/SUB, MDS/MDF (v1 only, reverse-engineered: Unverified), NRG v1/v2, CDI, DMG (UDIF raw/zlib, HFS+ read-only browse), WIM (browse via wimlib or own reader: decide M1). -- Edit image: load tree, add/remove/rename, rebuild ISO images or rewrite supported FAT volumes to a new file (never in place). +- Writers: ISO9660 (levels 1–3, Rock Ridge off), Joliet, UDF 1.02/1.50/2.01/2.50/2.60, hybrid ISO+UDF, El Torito (BIOS + UEFI), XDVDFS, and supported FAT12/16/32 raw-volume editing to a separate image. +- Readers: ISO/UDF, FAT12/16/32 (raw superfloppy or supported primary MBR partition, browse/extract), fixed/dynamic VHD 1.0 with supported FAT volumes, XDVDFS (browse/extract), BIN/CUE (multi-track, audio), CCD/IMG/SUB, MDS/MDF and NRG (read-only; unverified). +- Edit image: load tree, add/remove/rename, rebuild supported ISO images or rewrite supported FAT volumes to a new file (never in place). VHD edits preserve the fixed/dynamic container type. - Convert: any readable → ISO / BIN-CUE / CCD / IMG / CHD. ## 6. Audio - Rip: MMC READ CD with C2 pointers when supported, multiple read passes, AccurateRip v1/v2 checksum (opt-in lookup), offsets from a local table, CUE + per-track or single file, CD-Text read. - Burn: DAO with CD-Text and custom gaps via MMC; TAO fallback via IMAPI2. ReplayGain/normalize optional. -- Convert: batch decode/encode with tag copy. +- Convert: batch decode/encode with tag copy. WAV, FLAC, MP3, and AAC/M4A encoding are supported; Opus/Vorbis encoding is not. ## 7. UI - Pure Win32 + Common Controls v6 (D-043): owner-drawn task buttons painted from `docs/design-tokens.md`, native menus, file dialogs and message boxes; dark title bar via `DWMWA_USE_IMMERSIVE_DARK_MODE`. -- Main window: task grid (see mockup). Each task opens a modal-less task window; multiple tasks can queue. +- Main window: task grid (see mockup). Each task opens a modal-less task window; several task windows can run independently, with no central job queue. - Accessibility: real `BUTTON` controls (owner-drawn) so UI Automation sees names and roles, tooltips on every control, full keyboard navigation, visible focus ring, no color-only states, text scaling via Windows text size, high-contrast mode switches to system colors. - Per-monitor DPI v2 manifest. @@ -96,7 +96,7 @@ isofoundry copy-to-image --drive E: --out x.iso [--raw] [--subchannel] isofoundry copy-disc --from E: --to F: [--on-the-fly] isofoundry rip --drive E: --format flac --out [--cue] [--accuraterip] isofoundry burn-audio ... --drive E: [--cd-text] -isofoundry convert-audio ... --to opus --out +isofoundry convert-audio ... --to flac --out isofoundry convert-image --to iso --out isofoundry create-image ... --out x.iso [--fs udf] [--boot ] isofoundry extract --out diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md index 8867cf4..c3deedc 100644 --- a/docs/DECISIONS.md +++ b/docs/DECISIONS.md @@ -91,3 +91,13 @@ Format: `D-NNN — decision — why — status`. Newest at the bottom. Changing release notes: promotion asserts the feature set is complete and gated (simulated drive + CI), not that real-hardware field runs have happened. + +## D-054 — Post-1.0.1 implementation status (2026-11-09) + +- The development source adds XISO creation/browsing, raw FAT and + fixed/dynamic VHD workflows, and a USB-to-image task window. These + additions are not part of the 1.0.1 release; hardware status remains + **Unverified** as stated in `LIMITATIONS.md`. +- D-018 records the intended audio format scope, not the shipped + encoders. Opus and Vorbis encoding are not implemented; WAV, FLAC, + MP3, and AAC/M4A are the available encoders. diff --git a/docs/PARITY.md b/docs/PARITY.md index 2c5d1de..75d9567 100644 --- a/docs/PARITY.md +++ b/docs/PARITY.md @@ -32,5 +32,6 @@ limit · Missing = not there yet. | Blu-ray video (BDMV) authoring | Missing | Post-1.0 candidate | | Copy-protection handling | Never | Out of scope by design | -Update rule: a row moves only when the behaviour is in a release -and covered by a test or a recorded field check. +Update rule: rows describe the current source build and must be +covered by a test or recorded field check. Release and real-hardware +status are called out separately in the notes. diff --git a/docs/roadmap.md b/docs/roadmap.md index 60cf459..fa34bc7 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,6 +1,8 @@ # Roadmap -**Status: in development since 2026-10-31 (WinISOPilot 1.0.0 shipped).** Progress: M0 done (0.1.0). M1 done on `main` (reading, extraction, hashing, ISO creation, CD formats, CHD, conversion, editing, mounting and the GUI task windows); ships as 0.2.0. M2 USB built on `main` (usbwriter crate, both task windows, CLI, helper protocol v2); ships as 0.3.0 — physical-disk runs and QEMU boot checks still owed. M3 audio conversion built on `main` (if-audio crate: Symphonia decode, WAV/FLAC encode, MP3 via runtime LAME, AAC via Media Foundation, tags, ReplayGain, batch window + CLI); ships as 0.4.0 — the Media Foundation AAC path and LAME-on-Windows path are still to be exercised on a real machine. M4 burning built on `main` (if-drive crate: drive trait, fake drive with digest verify, IMAPI2 backend with hand-declared bindings; burn-image / burn-files with multisession append / burn-mp3 / burn-video / erase / drive-info windows + CLI, all runnable against `--fake`); shipped as 0.5.0 — no burn has run on real hardware, per this milestone's fake-drive test gate. M5 MMC ripping/copying built on `main` (SPTI raw reader with READ TOC / READ CD, rip/copy/scan engines in if-drive, fake drive read side with a damage map; copy-to-image / copy-disc / rip / burn-audio / surface-test / recover windows + CLI); shipped as 0.5.0 alongside M4 — no rip has run on real hardware either, per the same fake-drive gate. M6 integration built on `main` (spanning backup/restore with a catalog on every span, auto-action on disc insert, CLI completion scripts, Explorer verbs, opt-in update check, MusicBrainz disc lookup, and completion actions — sound/notification/shutdown); ships grouped with the RC pass as 0.9.0 (0.6.0 was never released separately). RC pass done on `main` (full audit, failure drills, accessibility pass, measured budgets, multi-drive burn from the command line); ships as 0.9.0. **1.0 released 2026-11-08** as the stable promotion of the 0.9.0 feature set (no feature changes; the release workflow marks it a full release, not a pre-release). **1.0.1 (2026-11-08)** is a patch release: the CHD header bound (AUDIT B-11) plus docs. +**Status: in development since 2026-10-31 (WinISOPilot 1.0.0 shipped).** Progress: M0 done (0.1.0). M1 done on `main` (reading, extraction, hashing, ISO creation, CD formats, CHD, conversion, editing, mounting and the GUI task windows); ships as 0.2.0. M2 USB built on `main` (usbwriter crate, both task windows, CLI, helper protocol v2); shipped grouped with M3 as 0.4.0 (0.3.0 was held) — physical-disk runs and QEMU boot checks still owed. M3 audio conversion built on `main` (if-audio crate: Symphonia decode, WAV/FLAC encode, MP3 via runtime LAME, AAC via Media Foundation, tags, ReplayGain, batch window + CLI); shipped as 0.4.0 — the Media Foundation AAC path and LAME-on-Windows path are still to be exercised on a real machine. M4 burning built on `main` (if-drive crate: drive trait, fake drive with digest verify, IMAPI2 backend with hand-declared bindings; burn-image / burn-files with multisession append / burn-mp3 / burn-video / erase / drive-info windows + CLI, all runnable against `--fake`); shipped as 0.5.0 — no burn has run on real hardware, per this milestone's fake-drive test gate. M5 MMC ripping/copying built on `main` (SPTI raw reader with READ TOC / READ CD, rip/copy/scan engines in if-drive, fake drive read side with a damage map; copy-to-image / copy-disc / rip / burn-audio / surface-test / recover windows + CLI); shipped as 0.5.0 alongside M4 — no rip has run on real hardware either, per the same fake-drive gate. M6 integration built on `main` (spanning backup/restore with a catalog on every span, auto-action on disc insert, CLI completion scripts, Explorer verbs, opt-in update check, MusicBrainz disc lookup, and completion actions — sound/notification/shutdown); ships grouped with the RC pass as 0.9.0 (0.6.0 was never released separately). RC pass done on `main` (full audit, failure drills, accessibility pass, measured budgets, multi-drive burn from the command line); ships as 0.9.0. **1.0 released 2026-11-08** as the stable promotion of the 0.9.0 feature set (no feature changes; the release workflow marks it a full release, not a pre-release). **1.0.1 (2026-11-08)** is a patch release: the CHD header bound (AUDIT B-11) plus docs. + +The current source also includes Xbox XISO creation and browsing, FAT/VHD image workflows, and a USB-to-image task window. These are development additions after 1.0.1, not part of that release. Real optical and USB hardware behavior remains unverified. All milestones ship as public **0.x pre-releases**. **1.0** = every 1.0 row done, tested, documented. Evidence label per feature: **Verified** (tested on real target), **Partially verified** (fake drive / VHDX / VM only), **Unverified**. @@ -20,7 +22,7 @@ Task grid modeled on AnyBurn Pro, with features drawn from Nero, Ashampoo Burnin | M6 Integration | 0.9 | Spanning backup/restore, auto-action on disc insert, CLI completion, Explorer verbs, opt-in update check, MusicBrainz opt-in, completion actions (sound/toast/shutdown) | Yes | | RC | 0.9 | Full audit pass, failure drills, accessibility pass, perf budgets measured, docs + screenshots | — | | 1.0 | 1.0 | Release | — | -| 1.x | — | Spanish UI, native Win11 context menu (sparse package), multiboot USB, FAT/VHD image editing, Xbox XISO, more quality-scan vendors, MDS/NRG write if demand | — | +| 1.x | — | Spanish UI, native Win11 context menu (sparse package), multiboot USB, more quality-scan vendors, MDS/NRG write if demand | — | ### Disc imaging (ideas from [DiscImageCreator](https://github.com/saramibreak/DiscImageCreator)) @@ -36,7 +38,7 @@ Out of scope: anything for copy-protected discs (SecuROM, SafeDisc, LibCrypt and ### Automation and image tools - [x] **Auto-action on disc insert** (idea from [Automatic Ripping Machine](https://github.com/automatic-ripping-machine/automatic-ripping-machine), MIT): optional mode that detects a newly inserted disc and identifies it (audio / data). As built (M6, D-051) it opens the chosen job's window rather than starting the job itself — the output location is a per-job choice; encrypted video discs are not ripped. -- [ ] **Edit image files** (reference: [TotalImage](https://github.com/TotalImage/TotalImage), MIT): ISO9660/Joliet/UDF and FAT12/16/32 raw floppy/primary MBR images can be edited to a new file. VHD support remains in progress. (1.x) +- [x] **Edit image files** (reference: [TotalImage](https://github.com/TotalImage/TotalImage), MIT): ISO9660/Joliet/UDF and FAT12/16/32 raw floppy/primary MBR images can be edited to a new file; supported FAT volumes in fixed/dynamic VHDs retain their container type. FAT edit limits and metadata resets are listed in [LIMITATIONS.md](LIMITATIONS.md). (development addition after 1.0.1) - [x] **Preservation dumping UI** (reference: [Redumper GUI](https://github.com/Deterous/Redumper-GUI), GPL-3.0, Rust): pick drive and system, show progress and the log, produce redump-style output. GPL code is not copied. (M5) - [x] Open, extract, and create Xbox XISO (XDVDFS) images (reference: [extract-xiso](https://github.com/XboxDev/extract-xiso)). The CLI creates XISOs; the GUI browse task opens and extracts them. Existing XISOs are read-only. Filesystem only; no patching of game executables. (1.x) diff --git a/tasks/plan.md b/tasks/plan.md index 8eb64b9..4d65b7e 100644 --- a/tasks/plan.md +++ b/tasks/plan.md @@ -2,7 +2,7 @@ ## Overview -Deliver the user's requested disc, image, archive, and USB capabilities in tested vertical slices, guided by the approved [capability map](../CAPABILITY-MAP.md). Preserve completed XISO/FAT work and the in-progress USB-to-image changes. The next implementation slice is fixed/dynamic VHD support for FAT12/16/32. The expanded catalog is staged; no feature is called complete until its user-facing path, focused tests, documentation, and software-only verification are in place. +Deliver the user's requested disc, image, archive, and USB capabilities in tested vertical slices, guided by the approved [capability map](../CAPABILITY-MAP.md). The XISO, FAT/VHD image-foundation, and USB-to-image slices are implemented and verified in the current source. Continue with the remaining multiboot, image/media, and localization modules; no feature is complete until its user-facing path, focused tests, documentation, and software-only verification are in place. ## Scope and assumptions @@ -18,7 +18,7 @@ Deliver the user's requested disc, image, archive, and USB capabilities in teste - Keep parsing, filesystem integration, and image editing in the platform-independent `if-image` crate; GUI and CLI use the shared implementation. - Validate all VHD offsets, checksums, block maps, lengths, and arithmetic before exposing a filesystem or allocating based on untrusted metadata. - Always write edits to a separate destination; never modify a source image in place or overwrite an existing destination. -- Keep USB writes behind the existing guarded USB/helper path. Preserve the existing USB-to-image worktree changes and `.serena/` data. +- Keep USB writes behind the existing guarded USB/helper path. Preserve the untracked `.serena/` session metadata; do not stage it with feature commits. - The approved capability map defines module boundaries and dependency order. Each remaining module receives its own spec and task breakdown before implementation. ## Task List @@ -85,7 +85,7 @@ CLI/scripting, batch queues, portable operation, diagnostics, and capability/lim | Loopback boot support varies among Linux and Windows images. | High | Define a tested compatibility set and document unsupported payloads rather than promise universal booting. | | Destructive disk or optical operations can erase user data. | High | Preserve existing target safety checks and require clear target identity and proportionate confirmation on every route. | | The feature catalog includes proprietary formats, protected media, hardware-specific operations, and platform-specific APIs. | High | Exclude copy-protection circumvention; implement only formats and operations with reliable evidence; document host/hardware limits. | -| Existing USB-to-image changes and `.serena/` are uncommitted. | High | Preserve them; do not reset, overwrite, or stage them with unrelated feature commits. | +| `.serena/` session metadata is untracked. | Low | Preserve it; do not stage it with feature commits. | ## Open questions