Windows blockchain replay bug

[david-profrontsolutions]

On a Mainnet Windows install of Daedalus daedalus-11.0.0-85389-mainnet-eefb68880-x86_64-windows.exe, after a successful sync, app close, and restart with no errors showing, the app began replaying the blockchain. The installer built off latest Mithril Partial Sync work but I believe still exists in the Master v11 app.

Steps

1. Installing the application.
2. Choosing a Chain storage location on an external SSD
3. Choosing the latest Mithril Snapshot to sync to.
4. Successfully syncing via the Mithril Snapshot and the remaining Cardano Node blocks.
5. Selected to close the application
6. Took 10 minutes or more on the Cardano Node state page, to then stop Cardano Node and close.
7. Opening the Daedalus Application and seeing the Cardano Node status screen with blockchain replay starting from scratch.

Evidence

Logs

Available for review, in comment below. Possible main source of issue and resolution identified in comment below.

Ledge State

Saw that there were two ledger states, one with 642 MB state and 2 GB tables, and another around the time of closure or opening with only a 600 MB state file.

Expected

After a successful sync and restart, the Daedalus app connects to Cardano Node, recognizes the latest ledger state, and begins syncing again.

[david-profrontsolutions]

TL;DR

The Mithril snapshot sync worked perfectly, and the node never replayed the ledger from scratch. What you saw on relaunch was cardano-node's unclean-shutdown recovery: a full integrity validation of all 8,865 immutable chunks (~235 GB) on the external SSD, which takes roughly 40–90 minutes at the observed pace and which Daedalus renders identically to a from-zero sync. The root defect is the shutdown: cardano-node hung after being asked to stop, Daedalus/cardano-launcher force-killed it after the 5-minute timeout, and the kill prevented the node from writing its clean-shutdown marker.

What actually happened (from the logs)

logs.zip

Details

Restore and first run were textbook. mithril-client downloaded snapshot epoch 640 (~235 GB, ~50 min) to E:\Daedalus Backup\Mainnet\chain\db with the ancillary ledger state included and verified, and the snapshot-converter produced the LSM ledger DB. At 15:06:41 the node logged Replaying ledger from snapshot DiskSnapshot {dsNumber = 191462372}, opened the ledger DB 22 seconds later, and was at the live tip 2m16s after process start. So your step-5 hypothesis (new ledger state created instead of using the Mithril one) is refuted — the Mithril ledger state was used, and the node never wrote a snapshot of its own (zero TookSnapshot events; it only ran ~2 misnapshot interval).

The shutdown is where it broke. You closed the app at 15:08:34; the node logged Received shutdown request immediately — but then kept running and adopting blocks for at least 4.5 more minutes (last block at 15:13:01), never closed the ChainDB, and the log ends mid-stream. The code side explains the rest: CardanoNode.stop() gives the node a 300-second graceful window (NODE_SHUTDOWN_TIMEOUT in source/main/config.ts:ard-kills it (SIGKILL/TerminateProcess), withfurther wait/kill escalation stages in CardanoNode.ts that can add 5–10 more minutes — matching your "10+ minutes to close."Because the process was killed rather than exiti and shutdown-time ledger snapshot were neverwritten.
Relaunch at 15:17:05: ChainDB is not clean. Validating all immutable chunks → Validating chunk no. 0 out of 8864. This is ImmutableDB verification, not ledger replay — the Mithril ledger snapshot was still on disk and was never invalidated. Startargs and database path were byte-identical to th junction was fine.

One aggravator worth knowing: at 15:20:27 (8.28% validated) a "Restart cardano node" was triggered from the UI, which killed the node mid-validation — and validation restarts from chunk 0 after every interruption. The final app quit at 15:21:28 did the same. On a slow external drive this becomes the user restarts, progress resets, forever. Ifyou had left it alone for ~40–90 minutes, it would have finished validation and resumed from the Mithril ledger snapshot without redownloading anything.

What should be fixed

Details

1. Windows shutdown hang / kill timeout (root defect). The node acknowledged shutdown-ipc but didn't exit within 5 minutes; Daedalus then guaranteed an unclean DB. This is 4 in.agent/plans/mithril-partial-sync/research/19-ux-refinement-state-and-gaps.md:397. Options: raise/adapt the timeout for Windows mainnet, or investigate the node-side hang (it was still adopting blocks after TrServerStopped — possibly the V2LSM
   backend closing on a USB drive).
2. UX: chunk validation is indistinguishable from a fresh sync. The status screen should say something like "Verifying existing blockchain after an unexpected shutdown (~1 hour) — your synced data is intact, do not restart."
3. Disable/guard the "Restart cardano node" actiit's actively harmful since it resets progress to zero.

The code audit also surfaced two latent hazards that did not fire here but are worth tickets: a leftover
mithril-bootstrap.lock (whose removal failure ises a fully-synced chain on the next launch, andlaunching with the external SSD unplugged deletes the chain junction and silently falls back to an empty local directory,
permanently forgetting the custom location.

[david-profrontsolutions]

I can trigger the clean file removal issue on Windows Mainnet Cardano node. Where deleting the clean file at the base of the chain folder triggers a blockchain reverfication. However for Pre-prod on Windows and Linux if the clean file isn't there, the node or app deletes all the immutable files and starts over via the first block on genesis sync. Not sure if a weird quirk or some other logic path I'm not understanding.