fix(compile): persist default Deno.openKv() path in compiled binaries#34576
fix(compile): persist default Deno.openKv() path in compiled binaries#34576bartlomieju wants to merge 3 commits into
Conversation
A compiled binary built with `deno compile` ran with an empty storage key resolver and no origin data folder, so `Deno.openKv()` without an explicit path silently fell back to an in-memory database and never persisted data. `localStorage` was affected by the same mechanism, and a compile-time `--location` did not help because the standalone runtime ignored it for storage purposes. Resolve a real storage key (mirroring `deno run`: `--location` if set, otherwise the main module) together with the `DENO_DIR`'s `location_data` folder so default KV and localStorage persist to disk. When the deno dir cannot be resolved, both stay empty and the in-memory fallback is kept. Closes #24318
lunadogbot
left a comment
There was a problem hiding this comment.
Overall
LGTM with two small things worth addressing. This is a clean, well-scoped bug fix for #24318. The root-cause analysis is correct, and the fix mirrors deno run's storage-key resolution faithfully (verified against cli/args/mod.rs:873-881). Small surface area (+66/-2), clean regression test, and the in-memory fallback is preserved as a safety net.
What I checked
- Main-module URL stability across runs. In
cli/rt/binary.rs:69-83,root_pathfor a non-self-extracting binary istemp_dir/deno-compile-<exe-name>, and for self-extracting it's a hash-derived extraction dir.main_module = root_dir_url.join(metadata.entrypoint_key)(run.rs:946). Both inputs are stable for a given binary, soStorageKeyResolver::new_use_main_module()produces a stable key across runs. ✅ - Mirrors
deno run. Same priority order (--location→ main module).deno runhas an extra middle tier fordeno.json, which is correctly omitted here because compiled binaries don't have one. - Coupling. Gating the storage key on
origin_data_folder_path.is_some()is the right call — it avoids the unwrap thatLibMainWorkerdoes on the data folder path when a key is present.
Suggestions
-
Cross-binary collision worth a comment or follow-up. For non-self-extracting builds,
root_pathis derived from just the exe filename (temp_dir/deno-compile-<name>atbinary.rs:82). Two unrelated compiled binaries both named e.g.outorappwill resolve to the same storage key and share KV/localStorage. This isn't introduced by this PR — it's a property of the existingroot_pathderivation — but it becomes user-visible now that data actually persists. Worth at least a// NOTE:near the storage_key block, or filing a follow-up. Could mitigate later by mixingmetadata.entrypoint_keycontent hash (or the self-extracting hash when present) into the key. -
Test should also cover
localStorage. The PR body explicitly says "the same mechanism also prevented localStorage from persisting" — but only KV is covered bytests/specs/compile/kv_default_path. Adding a parallellocalStorage_default_pathspec (or a single spec exercising both) would lock in the second half of the fix and prevent a future regression in just one path. Cheap to add — same shape as the KV test.
Nits
cli/rt/run.rs:1234-1243—DenoDirProvider::new(...)is constructed fresh here. Confirmed nothing inrun()already holds one, so this is fine, but ifDenoDirOptions { maybe_initial_cwd: None, maybe_custom_root: None }ever needsDENO_DIRenv handling tweaks (e.g.--node-modules-dirinteractions), this is the spot. Not blocking.- The two new comment blocks (lines ~1227-1232 and ~1245-1248) are good — they explain the why and the coupling, which is exactly what's non-obvious here. Keep them.
Test plan I'd want to see
./out setthen./out getacross two cold processes ✅ (covered)- Same flow with
localStorage❌ (missing) - Compile with
--location https://example.comand verify that key takes precedence ❌ (nice-to-have)
Address review: document that non-self-extracting binaries derive the storage key from just the executable file name, so two compiled binaries sharing a name share the same default KV database.
|
Thanks for the thorough review. Addressed in 57b8814:
The |
Two differently-named binaries compiled with the same --location share the default Deno.openKv() database, proving the location-derived storage key takes precedence over the main-module key.
|
Added the |
|
Took a look at #33775 to check whether it covers the localStorage gap here — partially, but with a couple of wrinkles worth surfacing before treating it as covered: #33775 doesn't test cross-run persistenceIts localStorage.setItem("a", "A");
console.log(localStorage.getItem("a"));One process. That proves The two PRs conflict on
|
| This PR (#34576) | #33775 | |
|---|---|---|
| Folder | location_data (matches deno run) |
New origin_data (compile-only) |
| Gating | Set if DENO_DIR resolves | Set only if --location is passed |
Default Deno.openKv() |
Persists (falls back to main module key) | Stays in-memory |
Whichever lands second has to reconcile. If #33775 lands as-written after this one, default Deno.openKv() without --location regresses back to in-memory.
Why a localStorage spec here is still worth it
A cross-run localStorage spec in this PR (same shape as kv_default_path — set then exit, then get) would:
- Lock down the second half of the fix described in the PR body while feat(compile): support WebCache API/LocalStorage API #33775 is still iterating.
- Force the reconciliation to be explicit when feat(compile): support WebCache API/LocalStorage API #33775 rebases — the test would fail under feat(compile): support WebCache API/LocalStorage API #33775's
--location-only gating, surfacing the regression before merge.
Cheap to add and the coverage isn't redundant with #33775. Not blocking, but I'd lean toward including it.
|
Superseded by #34618, which takes a broader approach: instead of routing default |
Fixes #24318.
In a
deno compile-d binary,Deno.openKv()called without an explicitpath silently fell back to an in-memory database, so data never persisted
across runs. The same mechanism also prevented
localStoragefrompersisting, and passing
--locationat compile time did not help.The root cause was in the standalone runtime (
cli/rt/run.rs), whichconstructed the worker factory with
StorageKeyResolver::empty()andorigin_data_folder_path: None. With no storage key, the worker resolvesorigin_storage_dirtoNone, which is handed to the KV SQLite handler asits default storage dir. In
ext/kv/sqlite.rsthe "no path, no defaultdir" case opens an in-memory connection, so writes never reach disk. This
is unlike
deno run, which builds a real storage key (location, thendeno.json URL, then the main module) and points the data folder at the
DENO_DIR'slocation_datadirectory.This change makes the standalone runtime resolve both pieces. It derives the
storage key the same way
deno rundoes (prefer an explicit--location,otherwise fall back to the main module, which is stable for a given compiled
binary) and reuses the
DENO_DIR'slocation_datafolder so default KV andlocalStoragepersist to disk. The two are tied together: the storage keyis only set when a data folder can be resolved, so if the deno dir is
unavailable both stay empty and the previous in-memory behavior is kept
(this also avoids unwrapping a missing data folder path in the worker).
A spec test under
tests/specs/compile/kv_default_pathcompiles a scriptthat uses
Deno.openKv()without a path, runs the binary once to write avalue and again to read it back, asserting the value survives across
processes.