Skip to content

T04 P1: Windows restore-journal deletion must follow durable directory rename #73

Open
Open
T04 P1: Windows restore-journal deletion must follow durable directory rename#73

Description

@eybersjp
eybersjp
opened on Oct 12, 2026

Status — blocking T04 Windows recovery correctness

Independent review of PR #70 source c3f3fd951d0d0f8f5ff6882b7271a75977f26062 identified a P1 at #70 (comment).

On Windows, syncRestoreParent() currently returns without a filesystem durability operation. During recoverLegacyRestore(), renameSync(displaced, root) can restore the original root before the code deletes the only restore journal. Power loss may persist the journal deletion without guaranteeing persistence of the preceding directory rename. T04-H47 deliberately exercises POSIX fsync and skips Windows; this does not prove Windows durability.

This problem is not the hostile same-account mutation risk deferred in Issue #72. The Product Owner deferred that different threat model only; do not silently treat Windows crash durability as approved out-of-scope.

Required engineering solution

  1. Establish the exact supported Windows filesystems and any Windows APIs providing a real durable same-volume directory rename and journal unlink ordering; use credible OS documentation and supported native interop if required. Microsoft documents MoveFileExW(MOVEFILE_WRITE_THROUGH), but its published guarantee explicitly addresses copy/delete moves; do not assume this alone proves same-volume NTFS directory rename durability without further evidence.
  2. Introduce an isolated, audited Windows helper only if it actually guarantees the required ordering. Protect path safety, ACLs, original tree, rollback witnesses and error propagation. Avoid hardcoded PowerShell assumptions, injection, unsigned executable distribution or silent fallback to fs.renameSync.
  3. If no verified native method is available, fail closed without deleting the recovery journal for uncertain Windows rollback/commit state, provide an evidence-preserving operator recovery process, and explicitly amend the Windows support/acceptance contract through the proper approval gate rather than faking durability.
  4. Add deterministic Windows-specific fault injection at each rename, flush and journal-unlink boundary; preserve original directories/journals on failure. Reconcile H25/H28/H47 and avoid skipped Windows durability fixtures.
  5. Run npm run release:validate and required T04 security suites twice on Windows/Ubuntu at the same reviewed head. Request fresh independent code, architecture and security source-bound review and resolve the PR fix(v0.12 T04): crash-safe event append, process locks and restore confinement #70 P1 only after evidence.

Source references

No permission to waive this in-scope P1, merge PR #70, advance T05 or publish v0.12.0. This issue exists to make the blocking Windows implementation concrete and independently verifiable.

Activity

  1. eybersjp commented on Oct 12, 2026

    @eybersjp
    eybersjp
    on Oct 12, 2026 – with ChatGPT Codex Connector
    OwnerAuthor

    Candidate Windows-native correction — independent durability review still required

    Native MoveFileExW(MOVEFILE_WRITE_THROUGH) recovery helper and regression H48 have been added to PR #70. Source code uses a bundled Windows PowerShell/Win32 P/Invoke helper only when restoring a displaced original during recovery or in-process rollback; helper failure throws and retains the recovery journal. A real Windows GitHub runner executed the actual API path.

    Verified GitHub Actions: https://github.com/eybersjp/development-kit/actions/runs/38321401970

    • Windows validation SUCCESS; T04-H48 PASS twice (normal focused suite + release:validate).
    • Ubuntu validation SUCCESS; POSIX T04-H47 remains the directory-fsync ordering fixture.
    • Independent acceptance job FAILED/PENDING because no authentic clean source-bound role receipts exist.

    Durability qualification: Passing H48 proves the Windows native call is executed and recovery ordering is functional, not a controlled real-power-loss durability experiment. Microsoft documents the native flag, but its explicit copy/delete flush guarantee does not settle every same-volume NTFS namespace scenario. Existing normal-stage promotion and initial journal publication still require independent evaluation. Do not close this P1 without an approved guarantee/fail-closed design for every supported Windows transaction boundary.

    The Product Owner's separate same-account attacker deferral in Issue #72 is not a waiver of this Windows crash-durability requirement.

  2. eybersjp commented on Oct 13, 2026

    @eybersjp
    eybersjp
    on Oct 13, 2026 – with ChatGPT Codex Connector
    OwnerAuthor

    Antigravity local candidate (13 October 2026) — safe Windows fail-closed behavior pending support decision

    The engineering agent reports uncommitted local T04 changes aimed at failing closed before Windows restore mutates the target project because current published OS API evidence does not establish durable publication of the entire restore journal / directory-rename / acknowledgement transaction.

    Important status: PR #70 still points to 3f48fc910ca03caab4e80f7d9f97d803ee3b7f5a; these local changes have not yet been independently inspected, committed or verified by GitHub CI. Existing npm run release:validate at this older remote head is not evidence for the candidate.

    The canonical v0.12 specification retains backward compatibility (INV-120-006) and requires DKF-120-AC-015 supported legacy migration to be idempotent, fail-closed and recoverable before cutover. An unconditional Windows restore/recovery disable may violate these if not explicitly limited and approved.

    Required before claiming T04 accepted:

    1. Precisely scope which Windows APIs fail closed (restoreLegacyBackup, recoverLegacyRestore); do NOT conflate backup restore with supported ongoing State Engine append, migration, ledger loads or read-only backup inspection.
    2. Document how operators can inspect and safely recover previously interrupted Windows restores without deleting recovery journals, displaced originals or staging witnesses.
    3. Capture formal Product Owner acceptance of any regression/restriction to supported Windows backup restoration, plus independent security/architecture review; Issue Future hardening: OS-enforced isolation for hostile same-account filesystem processes #72's same-UID threat-model deferral does not authorize this separate change.
    4. Add adversarial tests that assert the preflight fails before any destructive target mutation, that ordinary Windows runtime/migration remains working, and that journal/manual recovery evidence is never automatically deleted on refusal.
    5. Push focused source + tests + design/contract changes only after local verification; then run full exact-head Windows/Ubuntu CI, inspect skipped tests and request genuine reviewer roles. T04 acceptance remains PENDING.

    Microsoft MoveFileExW describes WRITE_THROUGH and its specific copy/delete flush guarantee: https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-movefileexw. It must not be presented as complete crash-durability certification for the whole multi-step transaction.

    This note documents a candidate technical direction, not a Product Owner decision to drop Windows restore support.

  3. eybersjp commented on Oct 13, 2026

    @eybersjp
    eybersjp
    on Oct 13, 2026 – with ChatGPT Codex Connector
    OwnerAuthor

    BINDING PRODUCT OWNER DECISION — v0.12.0 Windows restore support (13 October 2026)

    Decision: RETAIN FULL WINDOWS LEGACY-BACKUP RESTORE SUPPORT in DKF v0.12.0.

    The Product Owner explicitly rejected the proposed temporary removal/disablement of Windows restore. T04 remains BLOCKED until the complete Windows restore transaction (journal creation/publication, original displacement, candidate promotion, journal acknowledgement, retained-original cleanup, and interrupted recovery) is demonstrably crash-durable, tested, independently reviewed, and deterministically accepted. Continue all safe corrective engineering.

    Implementation constraints:

    1. Do not commit an unconditional process.platform === 'win32' throw that permanently disables restoreLegacyBackup() or recoverLegacyRestore(); do not silently narrow Windows platform support, or mislabel loss of restore support as an accepted safety solution.
    2. Fail closed and preserve the original directory, stage and journal on any unprovable transition. Such targeted error handling is permitted, but the supported Windows restoration capability must ultimately be complete and verifiable.
    3. Keep ordinary Windows State Engine operations, legacy migration and recovery functionality compatible.
    4. Establish an actual Windows filesystem persistence contract supported by authoritative OS APIs and controlled crash/fault tests. MoveFileExW(MOVEFILE_WRITE_THROUGH) is a candidate primitive, not by itself evidence of durable ordering for the entire journal transaction. Do not make unsupported durability claims.
    5. Add deterministic Windows-specific before/after fault tests for journal publication, both renames, acknowledgement and interrupted/manual recovery. Verify original content, ACL/metadata limits, transaction witnesses, and non-destructive rollback.
    6. A source change requires fresh exact-SHA Windows/Ubuntu validation, independently authenticated code/architecture/security reviews, zero in-scope unresolved findings, and the real ACCEPTED receipt. No T05, main merge or npm release until T04 accepted.
    7. The separate same-account malicious-concurrent-process deferral in Issue Future hardening: OS-enforced isolation for hostile same-account filesystem processes #72 remains authorized and does not waive crash consistency. The new full-Windows-support decision supersedes any uncommitted Antigravity local proposal to prohibit Windows restore.

    Engineering status: PR #70 remote branch currently at 3f48fc910ca03caab4e80f7d9f97d803ee3b7f5a; the new local candidate changes discussed in chat have not yet been pushed. Do not reset newer legitimate local work blindly; inspect and revise only the Windows-disable portion to match this decision. The independent-review evidence remains pending.

  4. eybersjp commented on Oct 13, 2026

    @eybersjp
    eybersjp
    on Oct 13, 2026 – with ChatGPT Codex Connector
    OwnerAuthor

    T04 Windows full-support engineering contract — non-destructive next implementation increment

    Binding authority: Product Owner approved retaining full Windows restore/recovery support for v0.12 on 13 October 2026. See preceding decision comment. A Windows-wide restoreLegacyBackup or recoverLegacyRestore disable is rejected. This note scopes research/tests for the existing runtime, not a proven implementation or acceptance.

    Verified OS API facts / limits

    Required transaction state model

    Record an explicit, authoritative transaction state: PREPARED (validated staged tree and durable discoverable recovery intent) → ORIGINAL_DISPLACED → CANDIDATE_PUBLISHED → PROMOTION_ACKNOWLEDGED → CLEANUP_SAFE. Interruption at every boundary must have an unambiguous witness. Absence of a journal must not be interpreted as NO_TRANSACTION if independently persisted displacement/staging witnesses exist.

    For each state transition, engineering must define: (a) atomic/ordered filesystem mutation; (b) durable identity and content witness; (c) exact Win32 API/flags/handle permissions; (d) documented durability guarantee and supported local filesystem/host; (e) injected failure behavior; (f) restart recovery outcome; (g) evidence that neither original root nor recovery assets can be prematurely lost.

    No clean-up of a displaced original before promotion and transaction intent are durably proven. If any operation is ambiguous, do not advance the state or acknowledge completion: retain journal, stage and original directory, and return a typed error with explicit operator recovery instructions. Preserve normal successful Windows restoration, not just error paths.

    Ordered acceptance fixtures to add (Windows, not POSIX-only skips)

    Fixture Inject interruption at Required invariant
    W73-01 before / after creation of recovery intent no destructive root rename unless intent is durably discoverable
    W73-02 after original→displaced root or original recoverable by authenticated witness, even if journal directory entry is uncertain
    W73-03 after staged→root recovery distinguishes published candidate, original and ambiguous state; never destroys the last good copy
    W73-04 before / after promotion acknowledgement no journal removal or false success unless required namespace ordering is proven
    W73-05 before / after removal of displaced original original retained until all recovery witnesses and candidate are durable
    W73-06 interrupted rollback and repeat recovery idempotent: retry recovers or fails closed without destroying recoverable assets
    W73-07 invalid or tampered witness no target mutation, no unsafe heuristic promotion, typed failure
    W73-08 Windows host with no supported persistence primitive fail safely for that attempted transition, preserving original; do not disable the entire product's restore API as the final v0.12 answer
    W73-09 supported, previously existing Windows restore journal explicit recovery remains available; upgrade/migration does not strand existing artifacts
    W73-10 normal success (old/new roots and nested metadata) functional restore, retention guarantees, ACL compatibility as specified, integrity and old API behavior

    Fault-injection unit tests verify call ordering and failure safety. Independently demonstrate persisted behavior using reproducible virtual-disk crash/power-cut/reboot experiments on the supported Windows/filesystem matrix where feasible. Clearly separate simulated exceptions/process death from actual storage-power-loss evidence. Hardware/cache limitations must be documented; do not claim universal guarantees beyond the OS contract.

    Safe work executable immediately

    1. Before committing unpushed Antigravity changes, inspect the local diff. Remove/revise only any blanket Windows disabling; retain verified review-parser hardening and non-destructive fixes. Preserve local edits, no reset/force-push.
    2. Add isolated transaction-transition boundary tests and baseline red tests for journal publication and promotion. Do not dilute assertions or turn tests into skips.
    3. Research and prototype a complete persistence design (native write-through metadata, reliably discoverable recovery intent, and explicit recovery witnesses). Compare alternatives including durable pre-existing log, native APIs and OS privilege requirements; evaluate whether they satisfy full support without administrative elevation. Do not select a technique solely by name.
    4. Run focused Windows migration + restore tests, npm run state-engine-v2:test, npm run release:validate and equivalent Ubuntu CI at the final SHA; produce actual test logs and metadata. Review independent source findings before closing any P1.
    5. On independently reviewed solution, update scope/architecture document and migration contract; re-run all three authentic risk-3 reviews and deterministic acceptance. Do not advance to T05 until actual ACCEPTED.

    Independent review infrastructure

    Codex PR thread has previously indicated missing repository environment configuration. This is a separate blocker: test success alone cannot create authentic three-role approval. Avoid duplicate @codex review requests until its environment is verified.

    Status: Windows crash-durability implementation OPEN, not accepted. Support restriction not authorized; deferred same-account attacker protection remains confined to Issue #72.

  5. chatgpt-codex-connector commented on Oct 13, 2026

    @chatgpt-codex-connector
    chatgpt-codex-connectorBot
    on Oct 13, 2026 – with ChatGPT Codex Connector

    To use Codex here, create an environment for this repo.

  6. eybersjp
    added a commit that references this issue on Oct 13, 2026

    fix(t04): establish uniform directory flush barrier for Windows resto…

    747009c

  7. eybersjp commented on Oct 13, 2026

    @eybersjp
    eybersjp
    on Oct 13, 2026 – with ChatGPT Codex Connector
    OwnerAuthor

    T04 post-CI engineering audit — 747009c6c85c2a8add32707785e073ceb256911c

    Independently checked GitHub Actions https://github.com/eybersjp/development-kit/actions/runs/38358728977: Windows SUCCESS, Ubuntu SUCCESS, risk-3 independent acceptance FAILURE because reviewedCommit:null and reviews:[]. CI passing is not source approval.

    Three currently blocking source/provenance issues:

    1. P1 runtime State Engine, not merely legacy restore: reviewer fix(v0.12 T04): crash-safe event append, process locks and restore confinement #70 (comment) identifies runtime/orchestration/state-engine-v2.mjs atomicWrite(paths.pending,...): after fs.renameSync(tempPath,destination), atomicWrite() explicitly skips fs.fsyncSync(dirFd) on Windows; the next operation appends physically to the canonical ledger. If the pending journal's directory entry has not persisted, a power loss can leave an unrepairable truncated ledger. Issue T04 P1: Windows restore-journal deletion must follow durable directory rename #73's new syncRestoreParent() applies to legacy restoration only, so it does not fix this runtime P1. Preserve the pending recovery marker and solve the complete ledger-journal durable ordering before claiming T04 acceptance.

    2. Windows directory durability mechanism remains insufficiently documented/verified: restore code opens directories with fs.openSync(parent,'r+') and calls fs.fsyncSync(fd). A passing Windows runner proves the operation completes there and that tests observe call ordering, not that namespace updates survive sudden power loss across the supported filesystems. The source cites docs/04-architecture/dkf-t04-windows-restore-durability-protocol.md, but that document does not exist on this commit; reviewer fix(v0.12 T04): crash-safe event append, process locks and restore confinement #70 (comment) confirms the documentation gap. Microsoft docs: FlushFileBuffers https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-flushfilebuffers ; CreateFile directory handles https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilew . Require precise justified supported OS/filesystem/handle semantics and fault testing; no silent downgrading of full Windows restore support.

    3. Self-incompatible source-bound review requests: three requests fix(v0.12 T04): crash-safe event append, process locks and restore confinement #70 (comment) , fix(v0.12 T04): crash-safe event append, process locks and restore confinement #70 (comment) , fix(v0.12 T04): crash-safe event append, process locks and restore confinement #70 (comment) each contain the identical 40-character SHA twice (once in Commit: and once at the end). requireCanonicalReviewRequestCommit() demands hashes.length === 1. These requests are therefore structurally ineligible as clean-review receipt sources even if Codex later reacts +1. For the NEXT immutable source review, request each role with exactly one canonical Commit: <SHA> field and no other full or abbreviated hash, and first validate the request with the actual production helper/test. Do not edit already-posted review requests to make them pass; the gate intentionally rejects edited requests.

    Additional P1 review response issue: #70 (comment) shows Verdict: PASS and Findings: none are still accepted alongside contradictory unstructured defect prose. Use a closed structured review response schema with no trailing arbitrary prose, with regression tests. Review source metadata and role must match, source SHA must be immutable, and no findings or unresolved PR threads may remain.

    Retain the PO-approved full Windows restore/recovery scope. Same-UID malicious concurrency is a separate deferred Issue #72 and does not waive ordinary crash consistency. Do not close Issue #73, merge PR #70, start T05, or publish v0.12.0 until all source-level P1/P2 findings, exact-source CI, authentic three-role reviews, deterministic ACCEPTED and necessary formal authorizations exist.

  8. eybersjp commented on Oct 13, 2026

    @eybersjp
    eybersjp
    on Oct 13, 2026 – with ChatGPT Codex Connector
    OwnerAuthor

    T04 autonomous remediation completed — e7fb4b4

    All four corrective areas have source-level changes on PR #70:

    1. Windows canonical pending-journal publication — runtime/orchestration/state-engine-v2.mjs now applies a write-capable Windows parent-directory fsync/FlushFileBuffers barrier after publishing pending-commit.json, before any physical ledger append; also flushes namespace after materialized publication/journal cleanup. If the barrier fails, mutation is refused and the recovery marker remains. T04-H49 tests injected namespace flush failure and unchanged ledger bytes; T04-H50 proves the flush precedes canonical append.
    2. Durability evidence contract — Added docs/04-architecture/dkf-t04-windows-restore-durability-protocol.md, indexed in docs/SUMMARY.md, with complete restoration/recovery and runtime state-journal ordering, primary Windows API links, fault-injection requirements and explicit limits on power-loss claims. Strengthened Windows H48 to verify actual parent-directory inode before accepting an fsync witness.
    3. Source-bound review requests — Added formatCanonicalT04ReviewRequest() and T04-A08; verified single full-SHA/source role contract. Re-requested three exact-head independent reviews using one SHA per request:
    4. Closed response grammar — requireCanonicalGithubReviewVerdict() rejects extra text and contradiction-bearing trailers, including trailing newline. T04-A07 verifies only exact four-field role/source/PASS/zero-findings responses may pass; no prose blacklist is used.

    Verified CI: https://github.com/eybersjp/development-kit/actions/runs/38360350050

    • Ubuntu validation SUCCESS.
    • Windows validation SUCCESS.
    • H49/H50/A07/A08 PASS on Windows in focused suite and release:validate; H49/H50 Windows-specific fixtures appropriately skipped on Ubuntu.
    • T04 independent risk-3 acceptance FAILURE/PENDING: authentic three-role receipts are still empty, and previously raised review threads remain unresolved (10 at last check).

    Open source/acceptance requirements: Independent reviewers must assess the true Windows crash/power-loss durability claims. A successful directory flush syscall and fault-injection call-order tests are NOT proof of all supported hardware/filesystem power-loss guarantees. Full Windows backup restoration and recovery remains a binding PO requirement, and Issue #73 must stay open pending actual authoritative evidence and reviews. Do not change supported Windows scope, self-approve, merge PR #70, start T05 or publish release. Deferred hostile same-account isolation Issue #72 is unrelated.

    This is implemented, cross-platform CI verified, and independently requested—NOT ACCEPTED.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions