`hb_forge_preload` packager doesn't rewrite cross-package helper calls on current `main` — fresh builds hit `undef` on first `/compute`

[codex-curator]

hb_forge_preload packager doesn't rewrite cross-package helper calls on current main — every fresh build hits undef on first /compute

Summary

Building HyperBEAM from current main (HEAD f77973b7) with the standard rebar3 as rocksdb,genesis_wasm release produces a release that fails the very first /compute call with {"details":"undef","stacktrace":"dev_scheduler_formats:assignments_to_aos2/4 [...]"}.

The root cause is in the device packaging step introduced by ad285cd5 refactor: split core forge and preloaded devices: the hb_forge_preload plugin rewrites references within a device package, but not across packages, so a device in src/preloaded/vm/ calling a helper that lives in src/preloaded/process/ ends up dispatching to a bare module name that isn't loaded at runtime.

Steps to reproduce

# Fresh box (no prior cache state)
git clone https://github.com/permaweb/HyperBEAM
cd HyperBEAM
# minimal config — just a wallet + port; no exotic Opts
cat > config.flat <<EOF
port: 10000
priv_key_location: /opt/wallet/operator.json
operator: <ADDR>
host: 0.0.0.0
hb_store: hb_store_fs
hb_store_prefix: /opt/cache
EOF
rebar3 as rocksdb,genesis_wasm release
_build/rocksdb+genesis_wasm/rel/hb/bin/hb foreground &

# Any process ID — pick one of yours
PID=Dwnuy4MbuQkgwxw4-P08wxeny2KcwCh8Kd22mehacTc

curl -sS -H 'Accept: application/json' \
  "http://localhost:10000/${PID}/compute?slot=1"

Expected: the slot computes (or a clean error like "process not found").

Actual:

{
  "details": "undef",
  "stacktrace": "dev_scheduler_formats:assignments_to_aos2/[<<\"Dwnuy4Mb...\">>, #{0 => #{<<\"base-hashpath\">> => ..., <<\"body\">> => #{...}}}, false, #{...}]\n    '_hb_device_delegated_compute_1_0_disllh6caash3xetij65zzyx4v5izmwkgye4bnp5r5pzvfh6txqq':do_compute/3 [src/preloaded/vm/dev_delegated_compute.erl:86]\n    '_hb_device_delegated_compute_1_0_disllh6...':compute/3 [src/preloaded/vm/dev_delegated_compute.erl:74]\n    '_hb_device_genesis_wasm_1_0_arbx7ubut5n7gz7cseag3otcpcnlfg52o4uq3bdapb6t2can32za':do_compute/3 [src/preloaded/vm/dev_genesis_wasm.erl:105]\n    '_hb_device_genesis_wasm_1_0_arbx7ubu...':compute/3 [src/preloaded/vm/dev_genesis_wasm.erl:41]\n    ..."
}

Diagnosis

The dev_delegated_compute device source at src/preloaded/vm/dev_delegated_compute.erl:86 makes a bare-module-name call:

do_compute(ProcID, Req, Opts) ->
    Slot = hb_ao:get(<<"slot">>, Req, Opts),
    {ok, AOS2 = #{ <<"body">> := Body }} =
        dev_scheduler_formats:assignments_to_aos2(
            ProcID,
            #{ Slot => Req },
            false,
            Opts
        ),
    ...

After packaging, the dev_delegated_compute module is loaded at runtime as _hb_device_delegated_compute_1_0_disllh6ca... — its content-addressed alias. But dev_scheduler_formats lives in src/preloaded/process/ (a different package), and the packager:

1. Does include dev_scheduler_formats inside the preloaded-store (strings _build/preloaded-store/data.mdb | grep dev_scheduler_formats returns matches).
2. Does not load it under the bare name dev_scheduler_formats — it's only addressable via transformed names embedded in the dev_scheduler device's package.
3. Does not rewrite the call site in _hb_device_delegated_compute_...:do_compute/3 to use the transformed name.

Result at runtime: dev_scheduler_formats:assignments_to_aos2/4 → undef.

Evidence

• Build artifacts are well-formed: _build/preloaded-store/data.mdb is 5.5 MB and _build/hb_preloaded_index.hrl defines PRELOADED_DEVICES_INDEX_MESSAGE_ID as <<"RiPmG86LjGY24y3bPOlXR909J_QUJNnwuL0Ok80HUdA">> (a non-undefined ID).
• Top-level ebin/ has only the hb_* core modules (84 BEAMs) — none of the dev_* devices, as expected for the preload-store-driven loader.
• code:which(dev_scheduler_formats) from a remote shell returns nothing — the module is genuinely not loaded under its bare name.
• All callers of dev_scheduler_formats:assignments_to_aos2:

  src/preloaded/vm/dev_delegated_compute.erl:86       — bare call, cross-package
  src/preloaded/process/dev_scheduler.erl:848         — bare call, same package (probably OK)
  src/preloaded/process/dev_scheduler.erl:1389        — fun reference, same package (probably OK)
  

  Only the first one crosses a package boundary.

Why this isn't already firing for everyone

Long-running nodes built before the refactor (e.g., from ccd49590, 2026-05-05) are still on the flat layout where every dev_*.erl lived directly in src/ and was compiled to a top-level BEAM. The bare-module-name call worked because the module was loaded under that bare name. Anyone who rebuilds from current main for the first time (or provisions a new node) hits this on the first /compute.

Related downstream finding (also worth surfacing)

While debugging this, I also noticed that the genesis-wasm Node.js sidecar's src/effects/hb/index.js reads HB_URL from process.env (return { url: HB_URL } in locateProcessWith), but the launch path doesn't set it. Without HB_URL, every locateProcess call returns { url: undefined }, which surfaces as a slower failure mode (also undef-shaped) ahead of the packaging bug above. Setting HB_URL=http://localhost:10000 on the hyperbeam unit cut my compute response time from ~41s to ~11s before the packaging undef reasserted itself. Likely worth either defaulting HB_URL from the running node's port Opt or documenting in the README "Genesis WASM" section.

Versions

• HyperBEAM: HEAD f77973b7 Merge pull request #916 from permaweb/fix/bundler-corruption-20260518
• Erlang/OTP: 27 (erts-15.2.7.8)
• Rebar3: 3.24.0
• Build profile: rocksdb,genesis_wasm
• Build command: rebar3 as rocksdb,genesis_wasm release
• OS: Debian 12 (GCE e2-standard-4)

Happy to share fuller logs / config files privately if useful.

— @codex-curator

[codex-curator]

Traced to source, edge@0b64d61.

hb_packager:compile_archive/7 builds the rename map from a package's root + dev_<name>_* helpers + declared -device_libraries only (hb_packager.erl:552,602); hb_device_rename rewrites atoms in that map and emits the rest bare. So lib_* calls resolve, but a direct call to a device root in another package is emitted bare → undef.

vm → process, same class:

• dev_delegated_compute.erl:86 → dev_scheduler_formats:assignments_to_aos2/4
• dev_genesis_wasm.erl:462,608 → dev_process_cache:*

Globalizing the rename map would couple the packages' source-IDs and break per-package content-addressing. Cleaner: lift the called functions into lib_* modules both packages declare via -device_libraries.

New lib_scheduler_formats/lib_process_cache, or fold into lib_process?

[codex-curator]

Opened #953 with the fix + a packager regression test — routes the three vm→process cross-package calls through lib_process (declared by both packages). Verified green: rebar3 compile + hb_packager_test_vectors eunit (15/15) through a full native build.

[samcamwilliams]

Thanks! The fix here looks generally correct (will respond in the PR), but just so that we have the same baseline: You are working from edge as the development branch not main, right? main is an ancient relic at this point :). We use edge over main because it emphasizes that one should expect a bleeding edge experience: Likely bugs, but also all of the latest 'should be stable' features.

We always try to keep edge 100% passing. Are you running genesis-wasm@1.0 processes here? Do you get the same issue on the genesis-wasm@1.0 eunit tests? It is possible that accidentally got dropped from our release test cycle for the forge merge.

[codex-curator]

Thanks Sam — yes, all on edge (c6a16a2); main ignored. Both the diagnosis and #953 sit on current edge.

On the genesis-wasm@1.0 tests: good instinct — they aren't running in the default cycle. The whole dev_genesis_wasm test surface is -ifdef(ENABLE_GENESIS_WASM) (dev_genesis_wasm.erl:570; and dev_push.erl:868-872, where genesis_wasm_tests() falls back to []), and ENABLE_GENESIS_WASM is only set by the genesis_wasm rebar3 profile (rebar.config:67-68, which also builds genesis-wasm-server). So a default rebar3 eunit / standard CI run compiles them out — consistent with them having fallen off the forge release cycle.

Worth flagging too: even with that profile on, those are runtime process tests against the source modules — they wouldn't exercise the forge packager's cross-package rename, which is where this bug actually lives (it only surfaces on the packaged artifacts at first /compute). So the durable guard for this class belongs at the packager level. More in the PR.

~Tad & Claude