Port onto @anyone-protocol/ao-client (D15/D16/D17) - #5
Open
jim-toth wants to merge 6 commits into
Open
Conversation
Reference port for W4. Replaces @permaweb/aoconnect 0.0.59 and the local
send-aos-message.ts wrapper with the shared client: writes are ANS-104 DataItems
signed with the controller key and pushed to our own HyperBEAM node, reads are
plain unsigned GETs against the process's own device.
D17 — explicit endpoints. CU_URL/GATEWAY_URL/GRAPHQL_URL are gone, replaced by a
single HB_URL with NO default (nodeUrlFromEnv fails closed). Two of the three
pointed at ar-io.net, a third party sitting in the critical path. The nginx edge
whitelists /~meta@1.0 and ^/{contract-pid}, which covers both the
~process@1.0/now/... reads and the ~process@1.0/push writes, so the public
hostname serves all three operations.
Reads are reworked to the D4 design: the View-State dryrun becomes a `dump` view
read. Measured on a node holding the real migrated registry (7932 verified /
2940 claimable / 1088 hardware) before choosing:
- Read latency is dominated by materializing `now` and scales with STATE size,
not response size — every read of this process costs ~2.7s whether it returns
226 bytes or 1MB. The same contract with empty state answers in ~30ms.
- The `fingerprints` view returns HTTP 414 between 100 and 200 ids (~8KB of
URL), so a bulk check would have to chunk at ~150.
So chunking is far more expensive, not less: ~7000 fingerprints is ~47 requests
x 2.7s ~= 127s against a single 2.9s `dump`. The comment on getOperatorRegistryState
records this so it does not get "optimized" into 47 round-trips later.
Two traps found and avoided while porting:
- Base-addressed whole-map reads (now/state/verified/serialize~json@1.0) are
NOT usable by consumers: they leak HyperBEAM message metadata (`device`,
`commitments`) as extra map entries and coerce booleans to the STRING
"true". The lua views return clean, correctly-typed JSON. Note the native
contract's own comment claims whole maps "come FREE from base-addressing" —
they do not, in this shape.
- Lua still serializes an EMPTY table as a JSON array, so the legacynet
normalization loop is still load-bearing. A seeded-empty process dumps
{"claimable":[],"verified":[],...}.
The state interface moves to the native D26 field names (claimable/verified/
blocked/registrationCredits/verifiedHardware) with the legacy mapping documented
on it, including that addresses are now EIP-55 rather than legacynet ALLCAPS.
Deletes the vendored src/util/arbundles-lite (an old copy of Irys arbundles) in
favour of the maintained @dha-team/arbundles that ao-client peer-depends on, plus
create-ethereum-data-item-signer.ts which only existed to feed aoconnect.
Contract rejections are now actually detected. A rejected write returns HTTP 200
with the reason only in its own slot output, so the old code reported success for
writes that changed nothing; ao-client confirms the slot and raises
AoContractError, which the service reports as a business outcome rather than an
outage.
Integration spec: 5/5 green against a local node holding the real 1MB migrated
registry, including a duplicate-credit case that asserts the rejection path
(contract reason "Registration Credit already added!" surfaced from the slot).
Typecheck is net -1 error against baseline and introduces none.
Declared as a direct dependency but never imported — the Arweave interaction all goes through the bundling service, which used the Turbo SDK. No effect on `npm audit`: arweave stays in the tree transitively via @ardrive/turbo-sdk and @dha-team/arbundles. This is declaration hygiene, so the manifest says what the service actually uses.
ArDrive left Arweave, and the SDK was doing very little here — sign a data item,
POST it — while costing a great deal.
What it dragged in, all of it now gone from the tree:
* @permaweb/aoconnect 0.0.57, bundled inside the SDK. Older than any pin this
migration removed, and it carries third-party default endpoints. So the
"aoconnect removed from every service" claim was true of our code and false
of the dependency tree.
* The entire Solana payment stack — @solana/spl-token, @solana/buffer-layout-utils,
bigint-buffer — inherited purely to upload to Arweave.
`npm audit` drops from 66 to 53.
The wire format is the one publish-module.ts already proved: POST the raw signed
item to `<bundler>/~bundler@1.0/tx`. That path is served both by up.arweave.net
— it is HyperBEAM's own default `bundler_ans104` target, see dev_arweave.erl
post_tx/4 — and by our own node. So moving to self-hosted bundling later is a
BUNDLER_NODE change, not a rewrite; the commented-out line above it in each HCL
records the two prerequisites (edge-allow /~bundler@1.0/tx, faff-allow-list the
signer).
BUNDLER_GATEWAY and BUNDLER_NETWORK are dropped — both existed only to configure
Turbo.
@noble/curves is now declared, which is unrelated to bundling but was exposed by
this change. hardware-verification.service.ts imports it and never declared it;
it resolved to a hoisted copy that the Turbo SDK happened to provide. Removing
the SDK left only ethers' nested 1.2.0, which lacks the `nist.js` subpath, so the
build broke. The pin is ^1.9.x because that line has both subpaths the code uses
— `nist.js` (v2 path) and `abstract/utils` (v1 path, dropped in v2). This was a
latent break waiting on any unrelated dependency change.
Non-breaking advisory fixes only; no direct dependency changed, lockfile only. Production vulnerabilities 33 -> 16 (high 11 -> 3). Tracked with `npm audit --omit=dev`: the all-inclusive headline moves the wrong way because the tree now resolves fully and the growth is dev-only tooling (jest, eslint, the Nest CLI) that never ships. Build clean.
Clears all 3 production criticals. These came from @dha-team/arbundles, which pins the EOL @ethersproject v5 line; arbundles 1.0.4 is already latest and the advisory range is `*`, so there was no upgrade to take — npm's suggested "fix" is a downgrade to 1.0.3. But the advisories are version-exact and both packages already ship fixes: elliptic critical hits <=6.6.0, patched in 6.6.1 secp256k1 high hits exactly 5.0.0, patched in 5.0.1 Both were installed twice — the patched version hoisted at the root, and a vulnerable copy nested under arbundles that its ethersproject chain resolved to. An override collapses them onto the patched ones, so this needs no change to arbundles or to ethers (our own code is already on v6.17). Verified rather than assumed: these are the ECDSA signing primitives, so a clean build proves nothing. The integration specs were run against a real node holding real migrated state, exercising ANS-104 signed writes end to end — operator registry 5/5 (registration credits, operator certificates) and relay rewards 3/3 (settles a round, reads the snapshot back from the settle slot). One low remains in elliptic with range <=6.6.1, i.e. unfixed at latest.
Clears the last production highs. @dha-team/arbundles pulls @ethersproject/providers 5.7.2, which pins ws 7.4.6 — inside the advisory range 7.0.0 - 7.5.10 for two DoS reports. Same nested-vs-hoisted shape as elliptic and secp256k1: a safe ws 8.21.0 sits hoisted for ethers v6 while the vulnerable 7.4.6 sits nested. The override is SCOPED to @ethersproject/providers so ethers v6 keeps ws 8; forcing ws 7 globally would break it. 7.5.13 is the patched 7.x line, so ethersproject stays on a compatible major. This also clears the aggregate advisories on @dha-team/arbundles and @ethersproject/providers themselves, which were flagged only by way of their vulnerable leaves. With elliptic, secp256k1 and now ws all patched, nothing is left — so the cluster was never actually blocked on arbundles leaving ethersproject v5, as previously recorded. Worth stating plainly: these are DoS bugs in WebSocket frame handling, and nothing on this path opens a WebSocket — arbundles pulls @ethersproject/providers for wallet and provider types, and no WebSocketProvider is ever constructed. This is noise reduction, not risk reduction. Verified against a real node holding real migrated state: the operator-registry integration spec passes 5/5 including ANS-104 signed writes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reference port for W4. Replaces @permaweb/aoconnect 0.0.59 and the local send-aos-message.ts wrapper with the shared client: writes are ANS-104 DataItems signed with the controller key and pushed to our own HyperBEAM node, reads are plain unsigned GETs against the process's own device.
D17 — explicit endpoints. CU_URL/GATEWAY_URL/GRAPHQL_URL are gone, replaced by a single HB_URL with NO default (nodeUrlFromEnv fails closed). Two of the three pointed at ar-io.net, a third party sitting in the critical path. The nginx edge whitelists /~meta@1.0 and ^/{contract-pid}, which covers both the ~process@1.0/now/... reads and the ~process@1.0/push writes, so the public hostname serves all three operations.
Reads are reworked to the D4 design: the View-State dryrun becomes a
dumpview read. Measured on a node holding the real migrated registry (7932 verified / 2940 claimable / 1088 hardware) before choosing:nowand scales with STATE size, not response size — every read of this process costs ~2.7s whether it returns 226 bytes or 1MB. The same contract with empty state answers in ~30ms.fingerprintsview returns HTTP 414 between 100 and 200 ids (~8KB of URL), so a bulk check would have to chunk at ~150.So chunking is far more expensive, not less: ~7000 fingerprints is ~47 requests x 2.7s ~= 127s against a single 2.9s
dump. The comment on getOperatorRegistryState records this so it does not get "optimized" into 47 round-trips later.Two traps found and avoided while porting:
device,commitments) as extra map entries and coerce booleans to the STRING "true". The lua views return clean, correctly-typed JSON. Note the native contract's own comment claims whole maps "come FREE from base-addressing" — they do not, in this shape.The state interface moves to the native D26 field names (claimable/verified/ blocked/registrationCredits/verifiedHardware) with the legacy mapping documented on it, including that addresses are now EIP-55 rather than legacynet ALLCAPS.
Deletes the vendored src/util/arbundles-lite (an old copy of Irys arbundles) in favour of the maintained @dha-team/arbundles that ao-client peer-depends on, plus create-ethereum-data-item-signer.ts which only existed to feed aoconnect.
Contract rejections are now actually detected. A rejected write returns HTTP 200 with the reason only in its own slot output, so the old code reported success for writes that changed nothing; ao-client confirms the slot and raises AoContractError, which the service reports as a business outcome rather than an outage.
Integration spec: 5/5 green against a local node holding the real 1MB migrated registry, including a duplicate-credit case that asserts the rejection path (contract reason "Registration Credit already added!" surfaced from the slot). Typecheck is net -1 error against baseline and introduces none.