Server folder index with heavy tombstone history crash-loops the app ~1s after connect; app burns a new device identity on each crash

[tr3lane]

Environment: VaultSync 1.8.2 (iOS 18, iPhone 12) ↔ Syncthing v2.1.2 server (Windows), LAN sync (also reproduces over relays).

Bug 1: server folder index with heavy deletion/tombstone history crash-loops the app ~1 second after connect

Symptom: the app connects (TLS completes, hello exchanged), then hard-resets the connection ~1s later — right at index exchange. Server log signature, repeating every ~10s:

Established secure connection (... TLS1.3-TLS_CHACHA20_POLY1305_SHA256 ...)
New device connection (... remote.client=syncthing remote.version=unknown-dev ...)
Lost device connection (... error="reading length: read tcp ...: wsarecv: An existing connection
  was forcibly closed by the remote host.")

Reproduces on every transport (TCP both dial directions, QUIC, relay) and on LAN with full connectivity — so it is not a network issue.

Evidence chain:

• The folder (≈9,200 files / 1.1 GB, mostly small markdown) synced fine on first pairing — 543 MiB transferred at full LAN speed.
• After ~4 days of server-side churn (heavy renames/deletes — i.e. a tombstone-rich index), every connection died at the ~1s mark, permanently.
• Two fresh app reinstalls reproduced it identically → not app-local state.
• Control test: a fresh 3-file folder shared from the same server to the same phone on the same network synced to 100% and the connection held → fault isolated to the original folder's served index.
• A server-side scan of the live index (/rest/db/browse, all entries) found no pathological names/mtimes/types — the poison is in the non-browsable layer (deletion tombstones / version vectors).
• Fix that proves the mechanism: rebuilding the folder's index server-side (remove folder from config → restart → re-add same config → rescan; files untouched) fixed it instantly. Reproduced the fix on a second affected folder (622 files / 318 MB) same day.

Looks like the app's index-exchange parser aborts (and the app hard-crashes) on some tombstone/version-vector record shape that desktop Syncthing peers accept without complaint.

Possibly related: earlier, offering a 2nd folder mid-initial-sync triggered an identical RST loop — may be the same parser abort with a different trigger.

Bug 2: the app generates a NEW device identity after each hard crash

During the crash-loop above, the app burned through 5 device identities in one day (each crash → new device ID → appears server-side as a brand-new pending device). This makes diagnosis very confusing (looks like repeated reinstalls), orphans stale device entries on the server, and resets sync completion tracking each cycle. Expected: device identity should survive a crash.

---

Happy to provide full timestamped server-side logs for any of the above.

[psimaker]

This is an exceptionally well-isolated report — the control-folder test and the index-rebuild-fixes-it proof narrow Bug 1 down about as far as it can be narrowed from the server side. Thank you. Status on both:

Bug 2 (new device identity after each crash) — confirmed and fixed

You found a real design flaw, and your expectation is correct: identity must survive a crash. The embedded engine loaded its device certificate through an upstream helper that silently generates a fresh key pair whenever loading fails — so a transient read failure or a corrupt file after a hard crash minted a brand-new device ID instead of reporting an error. That matches your observation exactly (each crash cycle → new pending device server-side, orphaned entries, reset completion tracking).

The fix ships in the next app update: the engine now creates an identity only on a confirmed first launch (both files verifiably absent). Corrupt, partial, or unreadable identity state fails the engine start with a visible error, never rewrites the stored files, and the original identity loads again once the transient condition clears. Regression tests cover each of those states.

Bug 1 (index exchange crash-loop) — plausible, needs your crash logs to pin down

Your evidence chain points at the app's index-decode path aborting on something in the served index's tombstone/version-vector records (the remote.version=unknown-dev in your server log is our embedded engine — that label is expected). A hard app crash ~1 s after connect is consistent with a panic in the embedded engine, which takes the whole app down. What would let us pin it precisely:

1. iOS crash logs: Settings → Privacy & Security → Analytics & Improvements → Analytics Data → files named VaultSync-….ips from a crash-loop day. If this is an engine panic, the log contains the exact panic message and stack — that alone will likely identify the record shape that kills the parser.
2. The timestamped server logs you offered, for one crash window (a few seconds around one Established secure connection → Lost device connection pair is enough).
3. If at all possible before you rebuild any further affected folder: a copy of that folder's server-side index database (it contains file metadata — names/sizes/hashes — so only if you're comfortable sharing it privately). A poisoned index would let us reproduce the abort in our Go test rig directly.

The "offering a 2nd folder mid-initial-sync" observation is a useful second trigger candidate — please mention in the crash log which scenario each file belongs to if you can tell.

The Bug 2 fix also decouples the failure modes: even while Bug 1 is unresolved, a crash loop will no longer burn identities or spawn phantom devices on your server.