From 96e888612ab21133809c766b173f25dab21bbcff Mon Sep 17 00:00:00 2001 From: Ty J Everett Date: Mon, 10 Aug 2026 18:37:10 -0700 Subject: [PATCH 1/7] fix(wallet): make managed change starvation-resistant --- docs/packages/wallet/wallet-toolbox-client.md | 11 +- docs/packages/wallet/wallet-toolbox-mobile.md | 11 +- docs/packages/wallet/wallet-toolbox.md | 16 +- docs/reference/package-api-migrations.md | 82 +-- docs/reference/service-operations.md | 2 +- docs/reference/service-resource-profiles.md | 106 +-- governance/package-release-notes.json | 14 +- governance/service-operations.json | 3 + infra/wallet-infra/.env.example | 5 + infra/wallet-infra/README.md | 39 ++ infra/wallet-infra/docker-compose.yml | 3 + infra/wallet-infra/example.env.yaml | 3 + .../guides/kube_samples/README.md | 10 + .../guides/kube_samples/wallet-configmap.yaml | 3 + infra/wallet-infra/src/index.ts | 50 +- packages/wallet/wallet-toolbox/CHANGELOG.md | 22 +- packages/wallet/wallet-toolbox/README.md | 29 +- .../docs/managed-change-liquidity.md | 238 +++++++ packages/wallet/wallet-toolbox/src/Setup.ts | 4 + .../wallet/wallet-toolbox/src/SetupClient.ts | 4 + .../wallet/wallet-toolbox/src/SetupWallet.ts | 3 + .../src/WalletPermissionsManager.ts | 13 +- ...ssionsManager.permissionSettlement.test.ts | 64 +- .../src/monitor/tasks/TaskReviewUtxos.ts | 37 ++ .../src/sdk/ActionBatch.interfaces.ts | 9 +- .../signer/actionBatch/ActionBatchPlanner.ts | 33 +- .../wallet-toolbox/src/storage/StorageIdb.ts | 28 +- .../wallet-toolbox/src/storage/StorageKnex.ts | 9 +- .../src/storage/StorageProvider.ts | 32 +- .../src/storage/StorageReaderWriter.ts | 8 +- .../src/storage/__test/StorageIdb.test.ts | 42 +- .../__test/TaskReviewUtxosLiquidity.test.ts | 61 ++ .../src/storage/__test/actionBatch.test.ts | 52 ++ .../__test/createActionPerformance.test.ts | 170 ++++- .../__tests/adminFormatting.test.ts | 10 +- .../src/storage/adminServer/adminServer.ts | 23 +- .../src/storage/adminServer/adminUi.ts | 6 +- .../wallet-toolbox/src/storage/index.all.ts | 1 + .../src/storage/index.client.ts | 1 + .../src/storage/index.mobile.ts | 1 + .../GenerateChange/generateChangeSdk.test.ts | 163 +++++ .../__test/managedChangePolicy.test.ts | 67 ++ .../src/storage/methods/actionBatch.ts | 627 ++++++++++-------- .../storage/methods/availableManagedChange.ts | 31 +- .../src/storage/methods/createAction.ts | 477 ++++++++----- .../src/storage/methods/generateChange.ts | 96 ++- .../storage/methods/managedChangePolicy.ts | 86 +++ .../src/storage/portable/index.ts | 5 +- .../remoting/__test/StorageClient.test.ts | 10 +- .../src/storage/schema/KnexMigrations.ts | 31 +- .../schema/entities/EntityOutputBasket.ts | 7 +- .../__tests/OutputBasketTests.test.ts | 27 + .../test/storage/KnexMigrations.test.ts | 35 + .../test/storage/portable.test.ts | 33 + .../test/wallet/action/createAction.test.ts | 2 +- .../test/wallet/action/createAction2.test.ts | 34 +- scripts/patch-coverage.mjs | 6 +- scripts/patch-coverage.test.mjs | 6 + 58 files changed, 2314 insertions(+), 687 deletions(-) create mode 100644 packages/wallet/wallet-toolbox/docs/managed-change-liquidity.md create mode 100644 packages/wallet/wallet-toolbox/src/storage/__test/TaskReviewUtxosLiquidity.test.ts create mode 100644 packages/wallet/wallet-toolbox/src/storage/methods/__test/managedChangePolicy.test.ts create mode 100644 packages/wallet/wallet-toolbox/src/storage/methods/managedChangePolicy.ts diff --git a/docs/packages/wallet/wallet-toolbox-client.md b/docs/packages/wallet/wallet-toolbox-client.md index db7b29c67..70833d088 100644 --- a/docs/packages/wallet/wallet-toolbox-client.md +++ b/docs/packages/wallet/wallet-toolbox-client.md @@ -25,11 +25,12 @@ can resume a soft-expired workspace using its exact persisted inputs. `endpointURL` after production bundlers minify class names, so backup selection and make-primary flows remain stable. Immediate browser actions can chain wallet-managed change from a delayed -parent; the child broadcast includes the parent BEEF so background delivery -cannot temporarily strand the wallet balance. -Durable permission grants finish broadcasting their internal token transaction -before the waiting browser request resumes, keeping the grant's funding change -available to the following wallet action. +parent only after completed and unproven liquidity is exhausted or exact +serialized-cost comparison proves that a pathological settled plan is larger. +Pending funds are never hidden. IndexedDB wallets migrate exact untouched +144-output / 32-satoshi defaults to a progressive 5,000-satoshi preference. +Durable permission tokens retain delayed broadcast so permission approval does +not inherit network latency. Opt-in remote-storage timing spans retain trace and parent-span correlation in the telemetry sink without adding headers to authenticated requests. Browser authentication accepts one verified matching UMP token as an existing diff --git a/docs/packages/wallet/wallet-toolbox-mobile.md b/docs/packages/wallet/wallet-toolbox-mobile.md index a1ef72d15..44180717e 100644 --- a/docs/packages/wallet/wallet-toolbox-mobile.md +++ b/docs/packages/wallet/wallet-toolbox-mobile.md @@ -25,11 +25,12 @@ can resume a soft-expired workspace using its exact persisted inputs. `endpointURL` after production bundlers minify class names, so backup selection and make-primary flows remain stable. Immediate mobile actions can chain wallet-managed change from a delayed parent; -the child broadcast includes the parent BEEF so background delivery cannot -temporarily strand the wallet balance. -Durable permission grants finish broadcasting their internal token transaction -before the waiting mobile request resumes, keeping the grant's funding change -available to the following wallet action. +the wallet first exhausts completed and unproven liquidity and uses exact +serialized-cost comparison only for pathological settled plans. Pending funds +are never hidden. New and migrated wallets progressively prefer useful +5,000-satoshi liquidity units without gathering inputs merely to create them. +Durable permission tokens retain delayed broadcast so permission approval does +not inherit network latency. Opt-in remote-storage timing spans retain trace and parent-span correlation in the telemetry sink without adding headers to authenticated requests. Mobile authentication accepts one verified matching UMP token as an existing diff --git a/docs/packages/wallet/wallet-toolbox.md b/docs/packages/wallet/wallet-toolbox.md index 22b2f67f7..f48afa844 100644 --- a/docs/packages/wallet/wallet-toolbox.md +++ b/docs/packages/wallet/wallet-toolbox.md @@ -25,14 +25,15 @@ workspaces can resume an expired soft lease by reacquiring only their exact persisted inputs under the provider's advertised reservation bound. Immediate actions may use wallet-managed change from a transaction awaiting -background broadcast. The child broadcast recursively carries the delayed -parent BEEF, preventing queued work from temporarily hiding most of the -wallet's spendable balance. +background broadcast, but only after completed and unproven liquidity is +exhausted or an over-16-input settled plan is larger by exact serialized +transaction-plus-BEEF cost. Queued funds are never hidden. -Durable permission grants finish broadcasting their internal token transaction -before the waiting application request resumes. A broadcast failure rejects the -grant, so the application can surface the existing error and safely retry -without planning against temporarily reserved funding inputs. +Durable permission grants retain delayed broadcast, avoiding network latency in +the permission path. New and existing wallets progressively target 144 useful +5,000-satoshi change outputs, create no more than eight outputs per action, and +migrate no more than four fee-positive legacy fragments per action. Optional +shaping cannot make a formerly fundable action fail. Completed `createAction` and `signAction` results expose Atomic BEEF as a numeric array at the public wallet boundary. The historical shape survives @@ -208,3 +209,4 @@ See `packages/wallet/wallet-toolbox-examples/src/p2pkh.ts`, `brc29.ts`, `pushdro - [Wallet domain overview](./index.md) - [Wallet toolbox examples](./wallet-toolbox-examples.md) - [Conformance vectors](../../conformance/vectors.md#wallet-brc-100) +- [Managed-change liquidity policy](https://github.com/bsv-blockchain/ts-stack/blob/main/packages/wallet/wallet-toolbox/docs/managed-change-liquidity.md) diff --git a/docs/reference/package-api-migrations.md b/docs/reference/package-api-migrations.md index a87674518..d11e825ca 100644 --- a/docs/reference/package-api-migrations.md +++ b/docs/reference/package-api-migrations.md @@ -3,8 +3,8 @@ id: package-api-migrations title: 'Package API, Declarations, and Migration Ledger' kind: reference version: '1.0.0' -last_updated: '2026-08-06' -last_verified: '2026-08-06' +last_updated: '2026-08-10' +last_verified: '2026-08-10' review_cadence_days: 30 status: stable tags: [reference, packages, api, declarations, migrations, release-notes] @@ -23,39 +23,39 @@ and clean-consumer tests remain the executable type authority. ## Current release boundary -| Package | npm baseline | Source | Candidate | API | Migration | -| --------------------------------- | ------------ | ------- | --------- | --------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `@bsv/402-pay` | `0.2.1` | `0.2.4` | patch | [API and usage](../packages/middleware/402-pay.md) | No consumer migration is required; client and server exports, payment protocol behavior, and runtime defaults are unchanged. | -| `@bsv/air-gap` | `0.0.0` | `0.1.1` | minor | [API and usage](../packages/helpers/air-gap.md) | No consumer migration is required; this is the first published release of a new package with no prior public API. The experimental pre-release framing that circulated on the unmerged feature branch is not accepted by the v1 decoder. | -| `@bsv/amountinator` | `2.1.1` | `2.1.4` | patch | [API and usage](../packages/helpers/amountinator.md) | No consumer migration is required; this is a backward-compatible patch candidate. | -| `@bsv/auth` | `0.1.1` | `0.1.3` | patch | [API and usage](../packages/middleware/auth.md) | No consumer migration is required; authentication APIs, wire behavior, and runtime defaults are unchanged. | -| `@bsv/auth-express-middleware` | `2.2.0` | `2.2.1` | patch | [API and usage](../packages/middleware/auth-express-middleware.md) | No consumer migration is required; valid request, response, authentication, and error contracts are unchanged. | -| `@bsv/authsocket` | `2.1.1` | `2.1.5` | patch | [API and usage](../packages/messaging/authsocket.md) | Valid traffic and the wire contract are unchanged. A socket is now disconnected when its authentication processing exceeds the concurrency limit or a callback fails; use onError for diagnostics and maxPendingAuthMessages to tune the default limit of 32. | -| `@bsv/authsocket-client` | `2.1.1` | `2.1.4` | patch | [API and usage](../packages/messaging/authsocket-client.md) | Valid traffic and supported imports are unchanged. The client now disconnects from a server that causes authentication failure or exceeds the concurrency limit; use onError for diagnostics and maxPendingAuthMessages to tune the default limit of 32. | -| `@bsv/btms` | `1.1.1` | `1.1.4` | patch | [API and usage](../packages/wallet/btms.md) | No consumer migration is required; token and lookup wire contracts are unchanged. | -| `@bsv/btms-permission-module` | `1.1.1` | `1.1.3` | patch | [API and usage](../packages/wallet/btms-permission-module.md) | No consumer migration is required; permission-module APIs and token semantics are unchanged. | -| `@bsv/did` | `0.2.1` | `0.2.4` | patch | [API and usage](../packages/helpers/did.md) | No consumer migration is required; DID APIs, encodings, credential behavior, and supported import forms are unchanged. | -| `@bsv/did-client` | `1.2.1` | `1.2.3` | patch | [API and usage](../packages/helpers/did-client.md) | No consumer migration is required; DID client APIs, encodings, and supported imports are unchanged. | -| `@bsv/fund-wallet` | `1.4.1` | `1.4.3` | patch | [API and usage](../packages/helpers/fund-wallet.md) | No consumer migration is required; wallet funding APIs and transaction behavior are unchanged. | -| `@bsv/gasp` | `1.3.1` | `1.3.5` | patch | [API and usage](../packages/overlays/gasp.md) | No consumer migration is required; existing constructor calls, imports, synchronization behavior, and wire semantics are unchanged. | -| `@bsv/message-box-client` | `2.2.2` | `2.3.0` | minor | [API and usage](../packages/messaging/message-box-client.md) | Existing listMessages and listMessagesLite calls continue to fetch all available messages. Applications that need an aggregate memory ceiling should set limit and/or maxPages; no additional BRC-105 approval callback is required because AuthFetch uses wallet permissions. | -| `@bsv/overlay` | `2.2.1` | `2.3.0` | minor | [API and usage](../packages/overlays/overlay.md) | Existing Engine constructor calls remain valid and default to 1,000 lookup formulas. Pass -1 as the final maxLookupResults argument only when a custom lookup service and deployment enforce an equivalent bound. | -| `@bsv/overlay-discovery-services` | `2.1.1` | `2.1.6` | patch | [API and usage](../packages/overlays/overlay-discovery-services.md) | No consumer migration is required; discovery records and public network behavior are unchanged. | -| `@bsv/overlay-express` | `2.5.0` | `2.5.1` | patch | [API and usage](../packages/overlays/overlay-express.md) | No consumer migration is required. Omit OVERLAY_CORS_ALLOWED_HEADERS for additive compatibility, or set an exact comma-separated list to retain a strict browser request-header policy. Origin modes, authentication, authorization, and endpoint validation are unchanged. | -| `@bsv/overlay-topics` | `1.6.1` | `1.6.8` | patch | [API and usage](../packages/overlays/overlay-topics.md) | No consumer migration is required; topic IDs, lookup contracts, and persisted formats are unchanged. | -| `@bsv/paymail` | `2.4.2` | `2.4.6` | patch | [API and usage](../packages/messaging/paymail.md) | Existing Paymail client APIs and protocol semantics are retained. Consumers provide one Express 4.18 or 5 runtime and matching type graph; browser bundles continue to exclude the server router implementation. | -| `@bsv/payment-express-middleware` | `2.1.1` | `2.1.5` | patch | [API and usage](../packages/middleware/payment-express-middleware.md) | No consumer migration is required; legacy x-bsv-payment JSON behavior remains supported, and Express 4 and 5 applications use their own peer-provided Express installation. | -| `@bsv/sdk` | `2.3.1` | `2.3.2` | patch | [API and usage](../packages/sdk/bsv-sdk.md) | No consumer migration is required. Existing HTTPWalletWire constructor calls, injected HTTP clients, Wallet Wire messages, and substrate selection order remain compatible. | -| `@bsv/simple` | `0.4.1` | `0.4.8` | patch | [API and usage](../packages/helpers/simple.md) | No consumer migration is required; the browser and server entry points remain compatible. | -| `@bsv/templates` | `1.9.1` | `1.9.6` | patch | [API and usage](../packages/helpers/templates.md) | No consumer migration is required; template APIs, supported imports, and generated script semantics are unchanged. | -| `@bsv/teranode-listener` | `1.1.1` | `1.1.4` | patch | [API and usage](../packages/network/teranode-listener.md) | No consumer migration is required; listener APIs, topics, and network configuration are unchanged. | -| `@bsv/verifast` | `0.3.0` | `0.3.4` | patch | [API and usage](../packages/sdk/verifast.md) | No consumer migration is required; exports, verification behavior, worker protocols, package paths, and runtime defaults are unchanged. | -| `@bsv/wallet-helper` | `0.1.1` | `0.1.6` | patch | [API and usage](../packages/helpers/wallet-helper.md) | No consumer migration is required; fluent builder APIs and transaction semantics are unchanged. | -| `@bsv/wallet-relay` | `0.2.2` | `0.3.4` | minor | [API and usage](../packages/wallet/wallet-relay.md) | QRPairingCode now renders a native button and accepts button wrapper attributes. Existing className, style, data, and ARIA props continue to work; update div-specific wrapper selectors or explicitly typed div event handlers. Express integrations now use the host application's matching Express runtime and type graph. | -| `@bsv/wallet-toolbox` | `2.6.5` | `2.6.6` | patch | [API and usage](../packages/wallet/wallet-toolbox.md) | No consumer migration is required. Unrelated actions continue through the legacy path instead of joining or committing an open workspace. Providers may optionally advertise resume support and a maximum reservation count; built-in providers default to 256 outputs and operators can select another positive value or -1 for unlimited operation. | -| `@bsv/wallet-toolbox-client` | `2.6.5` | `2.6.6` | patch | [API and usage](../packages/wallet/wallet-toolbox-client.md) | No browser consumer migration is required. Related noSend chains retain local batching; unrelated actions remain on their ordinary storage path. | -| `@bsv/wallet-toolbox-mobile` | `2.6.5` | `2.6.6` | patch | [API and usage](../packages/wallet/wallet-toolbox-mobile.md) | No mobile consumer migration is required. Related noSend chains retain local batching; unrelated actions remain on their ordinary storage path. | -| `create-bsv-app` | `1.0.2` | `1.0.4` | patch | [API and usage](../packages/helpers/create-bsv-app.md) | No consumer migration is required; generated application structure and CLI behavior are unchanged. | +| Package | npm baseline | Source | Candidate | API | Migration | +| --------------------------------- | ------------ | ------- | --------- | --------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `@bsv/402-pay` | `0.2.1` | `0.2.4` | patch | [API and usage](../packages/middleware/402-pay.md) | No consumer migration is required; client and server exports, payment protocol behavior, and runtime defaults are unchanged. | +| `@bsv/air-gap` | `0.0.0` | `0.1.1` | minor | [API and usage](../packages/helpers/air-gap.md) | No consumer migration is required; this is the first published release of a new package with no prior public API. The experimental pre-release framing that circulated on the unmerged feature branch is not accepted by the v1 decoder. | +| `@bsv/amountinator` | `2.1.1` | `2.1.4` | patch | [API and usage](../packages/helpers/amountinator.md) | No consumer migration is required; this is a backward-compatible patch candidate. | +| `@bsv/auth` | `0.1.1` | `0.1.3` | patch | [API and usage](../packages/middleware/auth.md) | No consumer migration is required; authentication APIs, wire behavior, and runtime defaults are unchanged. | +| `@bsv/auth-express-middleware` | `2.2.0` | `2.2.1` | patch | [API and usage](../packages/middleware/auth-express-middleware.md) | No consumer migration is required; valid request, response, authentication, and error contracts are unchanged. | +| `@bsv/authsocket` | `2.1.1` | `2.1.5` | patch | [API and usage](../packages/messaging/authsocket.md) | Valid traffic and the wire contract are unchanged. A socket is now disconnected when its authentication processing exceeds the concurrency limit or a callback fails; use onError for diagnostics and maxPendingAuthMessages to tune the default limit of 32. | +| `@bsv/authsocket-client` | `2.1.1` | `2.1.4` | patch | [API and usage](../packages/messaging/authsocket-client.md) | Valid traffic and supported imports are unchanged. The client now disconnects from a server that causes authentication failure or exceeds the concurrency limit; use onError for diagnostics and maxPendingAuthMessages to tune the default limit of 32. | +| `@bsv/btms` | `1.1.1` | `1.1.4` | patch | [API and usage](../packages/wallet/btms.md) | No consumer migration is required; token and lookup wire contracts are unchanged. | +| `@bsv/btms-permission-module` | `1.1.1` | `1.1.3` | patch | [API and usage](../packages/wallet/btms-permission-module.md) | No consumer migration is required; permission-module APIs and token semantics are unchanged. | +| `@bsv/did` | `0.2.1` | `0.2.4` | patch | [API and usage](../packages/helpers/did.md) | No consumer migration is required; DID APIs, encodings, credential behavior, and supported import forms are unchanged. | +| `@bsv/did-client` | `1.2.1` | `1.2.3` | patch | [API and usage](../packages/helpers/did-client.md) | No consumer migration is required; DID client APIs, encodings, and supported imports are unchanged. | +| `@bsv/fund-wallet` | `1.4.1` | `1.4.3` | patch | [API and usage](../packages/helpers/fund-wallet.md) | No consumer migration is required; wallet funding APIs and transaction behavior are unchanged. | +| `@bsv/gasp` | `1.3.1` | `1.3.5` | patch | [API and usage](../packages/overlays/gasp.md) | No consumer migration is required; existing constructor calls, imports, synchronization behavior, and wire semantics are unchanged. | +| `@bsv/message-box-client` | `2.2.2` | `2.3.0` | minor | [API and usage](../packages/messaging/message-box-client.md) | Existing listMessages and listMessagesLite calls continue to fetch all available messages. Applications that need an aggregate memory ceiling should set limit and/or maxPages; no additional BRC-105 approval callback is required because AuthFetch uses wallet permissions. | +| `@bsv/overlay` | `2.2.1` | `2.3.0` | minor | [API and usage](../packages/overlays/overlay.md) | Existing Engine constructor calls remain valid and default to 1,000 lookup formulas. Pass -1 as the final maxLookupResults argument only when a custom lookup service and deployment enforce an equivalent bound. | +| `@bsv/overlay-discovery-services` | `2.1.1` | `2.1.6` | patch | [API and usage](../packages/overlays/overlay-discovery-services.md) | No consumer migration is required; discovery records and public network behavior are unchanged. | +| `@bsv/overlay-express` | `2.5.0` | `2.5.1` | patch | [API and usage](../packages/overlays/overlay-express.md) | No consumer migration is required. Omit OVERLAY_CORS_ALLOWED_HEADERS for additive compatibility, or set an exact comma-separated list to retain a strict browser request-header policy. Origin modes, authentication, authorization, and endpoint validation are unchanged. | +| `@bsv/overlay-topics` | `1.6.1` | `1.6.8` | patch | [API and usage](../packages/overlays/overlay-topics.md) | No consumer migration is required; topic IDs, lookup contracts, and persisted formats are unchanged. | +| `@bsv/paymail` | `2.4.2` | `2.4.6` | patch | [API and usage](../packages/messaging/paymail.md) | Existing Paymail client APIs and protocol semantics are retained. Consumers provide one Express 4.18 or 5 runtime and matching type graph; browser bundles continue to exclude the server router implementation. | +| `@bsv/payment-express-middleware` | `2.1.1` | `2.1.5` | patch | [API and usage](../packages/middleware/payment-express-middleware.md) | No consumer migration is required; legacy x-bsv-payment JSON behavior remains supported, and Express 4 and 5 applications use their own peer-provided Express installation. | +| `@bsv/sdk` | `2.3.1` | `2.3.2` | patch | [API and usage](../packages/sdk/bsv-sdk.md) | No consumer migration is required. Existing HTTPWalletWire constructor calls, injected HTTP clients, Wallet Wire messages, and substrate selection order remain compatible. | +| `@bsv/simple` | `0.4.1` | `0.4.8` | patch | [API and usage](../packages/helpers/simple.md) | No consumer migration is required; the browser and server entry points remain compatible. | +| `@bsv/templates` | `1.9.1` | `1.9.6` | patch | [API and usage](../packages/helpers/templates.md) | No consumer migration is required; template APIs, supported imports, and generated script semantics are unchanged. | +| `@bsv/teranode-listener` | `1.1.1` | `1.1.4` | patch | [API and usage](../packages/network/teranode-listener.md) | No consumer migration is required; listener APIs, topics, and network configuration are unchanged. | +| `@bsv/verifast` | `0.3.0` | `0.3.4` | patch | [API and usage](../packages/sdk/verifast.md) | No consumer migration is required; exports, verification behavior, worker protocols, package paths, and runtime defaults are unchanged. | +| `@bsv/wallet-helper` | `0.1.1` | `0.1.6` | patch | [API and usage](../packages/helpers/wallet-helper.md) | No consumer migration is required; fluent builder APIs and transaction semantics are unchanged. | +| `@bsv/wallet-relay` | `0.2.2` | `0.3.4` | minor | [API and usage](../packages/wallet/wallet-relay.md) | QRPairingCode now renders a native button and accepts button wrapper attributes. Existing className, style, data, and ARIA props continue to work; update div-specific wrapper selectors or explicitly typed div event handlers. Express integrations now use the host application's matching Express runtime and type graph. | +| `@bsv/wallet-toolbox` | `2.6.5` | `2.6.6` | patch | [API and usage](../packages/wallet/wallet-toolbox.md) | No consumer code or BRC-100 migration is required. Exact untouched default baskets at 144 outputs / 32 satoshis advance to a 5,000-satoshi preference; custom basket values remain unchanged and funds migrate only through future authorized actions. Same-tier compatibility planning and retained pending fallback ensure the policy adds no new funding refusal. Operators may tune all work limits, including explicit -1 unlimited modes. | +| `@bsv/wallet-toolbox-client` | `2.6.5` | `2.6.6` | patch | [API and usage](../packages/wallet/wallet-toolbox-client.md) | No browser consumer code migration is required. Exact untouched IndexedDB defaults advance from 32 to 5,000 satoshis and then migrate progressively through ordinary authorized actions; custom basket values and remote-storage wire contracts remain unchanged. | +| `@bsv/wallet-toolbox-mobile` | `2.6.5` | `2.6.6` | patch | [API and usage](../packages/wallet/wallet-toolbox-mobile.md) | No mobile consumer code migration is required. Wallets progressively adopt useful change values through future authorized actions; custom basket values, BRC-100 methods, and remote-storage wire contracts remain unchanged. | +| `create-bsv-app` | `1.0.2` | `1.0.4` | patch | [API and usage](../packages/helpers/create-bsv-app.md) | No consumer migration is required; generated application structure and CLI behavior are unchanged. | `none` means the source manifest matches the recorded npm baseline. Any other value is an unpublished candidate. Publication, tags, releases, registry @@ -477,8 +477,8 @@ CLI entry points: `{"wallet-relay":"./bin/init.mjs"}`. - Package documentation: [docs/packages/wallet/wallet-toolbox.md](../packages/wallet/wallet-toolbox.md) - Source: [packages/wallet/wallet-toolbox](https://github.com/bsv-blockchain/ts-stack/tree/main/packages/wallet/wallet-toolbox) -- Release note: Isolates action batches by explicit transaction-graph membership, adds bounded exact-input lease resume, and carries structured lifecycle errors across local and remote storage. -- Migration: No consumer migration is required. Unrelated actions continue through the legacy path instead of joining or committing an open workspace. Providers may optionally advertise resume support and a maximum reservation count; built-in providers default to 256 outputs and operators can select another positive value or -1 for unlimited operation. +- Release note: Isolates and resumes action batches, then adds progressive managed-change liquidity: 144 useful 5,000-satoshi units, bounded surplus-only fanout and fragment migration, settled-first parent selection, exact BEEF-cost comparison for pathological plans, last-resort pending funding, delayed permission persistence, and read-only Monitor reporting. +- Migration: No consumer code or BRC-100 migration is required. Exact untouched default baskets at 144 outputs / 32 satoshis advance to a 5,000-satoshi preference; custom basket values remain unchanged and funds migrate only through future authorized actions. Same-tier compatibility planning and retained pending fallback ensure the policy adds no new funding refusal. Operators may tune all work limits, including explicit -1 unlimited modes. | Public subpath | Runtime target(s) | Declaration target(s) | | ---------------- | ---------------------------------------------------- | -------------------------- | @@ -491,8 +491,8 @@ CLI entry points: `{"wallet-relay":"./bin/init.mjs"}`. - Package documentation: [docs/packages/wallet/wallet-toolbox-client.md](../packages/wallet/wallet-toolbox-client.md) - Source: [packages/wallet/wallet-toolbox/client](https://github.com/bsv-blockchain/ts-stack/tree/main/packages/wallet/wallet-toolbox/client) -- Release note: Carries the lockstep browser build with explicit action-batch isolation and bounded lease recovery. -- Migration: No browser consumer migration is required. Related noSend chains retain local batching; unrelated actions remain on their ordinary storage path. +- Release note: Carries the lockstep browser build with isolated resumable action batches and progressive IndexedDB managed-change liquidity, settled-first ancestry, exact BEEF-cost comparison, and delayed permission persistence. +- Migration: No browser consumer code migration is required. Exact untouched IndexedDB defaults advance from 32 to 5,000 satoshis and then migrate progressively through ordinary authorized actions; custom basket values and remote-storage wire contracts remain unchanged. | Public subpath | Runtime target(s) | Declaration target(s) | | ---------------- | -------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | @@ -503,8 +503,8 @@ CLI entry points: `{"wallet-relay":"./bin/init.mjs"}`. - Package documentation: [docs/packages/wallet/wallet-toolbox-mobile.md](../packages/wallet/wallet-toolbox-mobile.md) - Source: [packages/wallet/wallet-toolbox/mobile](https://github.com/bsv-blockchain/ts-stack/tree/main/packages/wallet/wallet-toolbox/mobile) -- Release note: Carries the lockstep mobile build with explicit action-batch isolation and bounded lease recovery. -- Migration: No mobile consumer migration is required. Related noSend chains retain local batching; unrelated actions remain on their ordinary storage path. +- Release note: Carries the lockstep mobile build with isolated resumable action batches, progressive managed-change liquidity, settled-first ancestry, exact BEEF-cost comparison, and delayed permission persistence. +- Migration: No mobile consumer code migration is required. Wallets progressively adopt useful change values through future authorized actions; custom basket values, BRC-100 methods, and remote-storage wire contracts remain unchanged. | Public subpath | Runtime target(s) | Declaration target(s) | | ---------------- | -------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | diff --git a/docs/reference/service-operations.md b/docs/reference/service-operations.md index 0070af409..7219f8cc4 100644 --- a/docs/reference/service-operations.md +++ b/docs/reference/service-operations.md @@ -287,7 +287,7 @@ Incident handling follows this evidence-preserving sequence: ### wallet-infra - Configuration: required `BSV_NETWORK`, `KNEX_DB_CONNECTION`, `SERVER_PRIVATE_KEY`; optional - `COMMISSION_FEE`, `COMMISSION_PUBLIC_KEY`, `ENABLE_NGINX`, `FEE_MODEL`, `HTTP_PORT`, `TAAL_API_KEY`, `WALLET_STORAGE_BIND_HOST`, `WALLET_STORAGE_MONITOR_START_TASKS`, `WALLET_STORAGE_MONITOR_STARTUP_TASK_MODE`, `WALLET_STORAGE_MONITOR_ADMIN_ENABLED`, `WALLET_STORAGE_MONITOR_ADMIN_HOST`, `WALLET_STORAGE_MONITOR_ADMIN_PORT`, `WALLET_STORAGE_MONITOR_ADMIN_ALLOWED_ORIGINS`, `WALLET_STORAGE_MONITOR_ADMIN_PRIVATE_KEY`, `WALLET_STORAGE_ADMIN_IDENTITY_KEYS`, `WALLET_STORAGE_TRUST_PROXY_HOPS`; secret-bearing + `COMMISSION_FEE`, `COMMISSION_PUBLIC_KEY`, `ENABLE_NGINX`, `FEE_MODEL`, `HTTP_PORT`, `TAAL_API_KEY`, `WALLET_STORAGE_BIND_HOST`, `WALLET_STORAGE_MONITOR_START_TASKS`, `WALLET_STORAGE_MONITOR_STARTUP_TASK_MODE`, `WALLET_STORAGE_MONITOR_ADMIN_ENABLED`, `WALLET_STORAGE_MONITOR_ADMIN_HOST`, `WALLET_STORAGE_MONITOR_ADMIN_PORT`, `WALLET_STORAGE_MONITOR_ADMIN_ALLOWED_ORIGINS`, `WALLET_STORAGE_MONITOR_ADMIN_PRIVATE_KEY`, `WALLET_STORAGE_ADMIN_IDENTITY_KEYS`, `WALLET_STORAGE_MANAGED_CHANGE_MAX_OUTPUTS_PER_ACTION`, `WALLET_STORAGE_MANAGED_CHANGE_MIGRATION_INPUTS_PER_ACTION`, `WALLET_STORAGE_MANAGED_CHANGE_PENDING_COMPARISON_INPUTS`, `WALLET_STORAGE_TRUST_PROXY_HOPS`; secret-bearing `KNEX_DB_CONNECTION`, `OTEL_EXPORTER_OTLP_HEADERS`, `SERVER_PRIVATE_KEY`, `TAAL_API_KEY`, `WALLET_STORAGE_MONITOR_ADMIN_PRIVATE_KEY`. - Telemetry: ESM bootstrap `src/telemetry.ts`, logger diff --git a/docs/reference/service-resource-profiles.md b/docs/reference/service-resource-profiles.md index 04896498f..3512c2ed0 100644 --- a/docs/reference/service-resource-profiles.md +++ b/docs/reference/service-resource-profiles.md @@ -3,8 +3,8 @@ id: service-resource-profiles title: 'Service Resource Profiles, Scaling, and Message Box Economics' kind: reference version: '1.0.0' -last_updated: '2026-08-04' -last_verified: '2026-08-04' +last_updated: '2026-08-10' +last_verified: '2026-08-10' review_cadence_days: 30 status: stable tags: [reference, infrastructure, resource-safety, scaling, message-box, brc-105] @@ -36,18 +36,18 @@ integers because disabling them can strand sockets or database waiters. The common service prefixes are `CHAINTRACKS`, `MESSAGE_BOX`, `OVERLAY`, `UHRP`, `WAB`, and `WALLET_STORAGE`. -| Common control | Meaning | -| --- | --- | -| `_MAX_BODY_BYTES` | Default materialized request body ceiling. JSON or binary routes may use a more specific prefix such as `UHRP_JSON` or `WALLET_STORAGE_BINARY`. | -| `_MAX_RESPONSE_BYTES` | Materialized response ceiling; a response above it receives `413 ERR_RESPONSE_TOO_LARGE`. | -| `_MAX_CONCURRENT_REQUESTS` | In-flight application requests per process; saturation receives `503 ERR_SERVER_BUSY`. | -| `_MAX_CONNECTIONS` | Open TCP/WebSocket connections per process. | -| `_REQUEST_TIMEOUT_MS` | Complete-request timeout. | -| `_HEADERS_TIMEOUT_MS` | Header receive timeout. | -| `_KEEP_ALIVE_TIMEOUT_MS` | Idle keep-alive timeout. | -| `_SOCKET_TIMEOUT_MS` | Socket inactivity timeout. | -| `_MAX_REQUESTS_PER_SOCKET` | Requests accepted before connection recycling. | -| `_MAX` / `_WINDOW_MS` | Route-class rate limit and window. Rate maxima accept `-1`/`unlimited`; windows remain finite. | +| Common control | Meaning | +| ---------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | +| `_MAX_BODY_BYTES` | Default materialized request body ceiling. JSON or binary routes may use a more specific prefix such as `UHRP_JSON` or `WALLET_STORAGE_BINARY`. | +| `_MAX_RESPONSE_BYTES` | Materialized response ceiling; a response above it receives `413 ERR_RESPONSE_TOO_LARGE`. | +| `_MAX_CONCURRENT_REQUESTS` | In-flight application requests per process; saturation receives `503 ERR_SERVER_BUSY`. | +| `_MAX_CONNECTIONS` | Open TCP/WebSocket connections per process. | +| `_REQUEST_TIMEOUT_MS` | Complete-request timeout. | +| `_HEADERS_TIMEOUT_MS` | Header receive timeout. | +| `_KEEP_ALIVE_TIMEOUT_MS` | Idle keep-alive timeout. | +| `_SOCKET_TIMEOUT_MS` | Socket inactivity timeout. | +| `_MAX_REQUESTS_PER_SOCKET` | Requests accepted before connection recycling. | +| `_MAX` / `_WINDOW_MS` | Route-class rate limit and window. Rate maxima accept `-1`/`unlimited`; windows remain finite. | Every API role exposes `/healthz` in addition to its existing health/readiness contract. Two or more initial slashes are normalized for compatibility, so @@ -59,14 +59,14 @@ The table shows the dominant list/range maximum, response ceiling, and per-process request concurrency. Route-specific defaults follow in the next section. -| Service | Small | Standard | High-throughput | -| --- | --- | --- | --- | -| Chaintracks | 500 headers, 1 MiB, 32 | 1,000 headers, 4 MiB, 64 | 5,000 headers, 32 MiB, 256 | -| Message Box | 500 messages, 4 MiB, 8 | 1,000 messages, 8 MiB, 24 | 5,000 messages, 32 MiB, 96 | -| Overlay Express | 500 lookup results, 4 MiB, 8 | 1,000 results, 8 MiB, 24 | 5,000 results, 32 MiB, 96 | -| UHRP Basic / Cloud | 500 records, 1 MiB, 16 | 1,000 records, 4 MiB, 64 | 5,000 records, 16 MiB, 250 | -| WAB | single-record APIs, 1 MiB, 64 | single-record APIs, 2 MiB, 128 | single-record APIs, 8 MiB, 256 | -| Wallet Storage API | 500 RPC rows, 4 MiB, 8 | 1,000 rows, 8 MiB, 24 | 5,000 rows, 32 MiB, 96 | +| Service | Small | Standard | High-throughput | +| ------------------ | ----------------------------- | ------------------------------ | ------------------------------ | +| Chaintracks | 500 headers, 1 MiB, 32 | 1,000 headers, 4 MiB, 64 | 5,000 headers, 32 MiB, 256 | +| Message Box | 500 messages, 4 MiB, 8 | 1,000 messages, 8 MiB, 24 | 5,000 messages, 32 MiB, 96 | +| Overlay Express | 500 lookup results, 4 MiB, 8 | 1,000 results, 8 MiB, 24 | 5,000 results, 32 MiB, 96 | +| UHRP Basic / Cloud | 500 records, 1 MiB, 16 | 1,000 records, 4 MiB, 64 | 5,000 records, 16 MiB, 250 | +| WAB | single-record APIs, 1 MiB, 64 | single-record APIs, 2 MiB, 128 | single-record APIs, 8 MiB, 256 | +| Wallet Storage API | 500 RPC rows, 4 MiB, 8 | 1,000 rows, 8 MiB, 24 | 5,000 rows, 32 MiB, 96 | Start with 512 MiB for `small`, 1 GiB for `standard`, and 8 GiB for `high-throughput`. High-throughput is not a promise that every configured @@ -75,15 +75,15 @@ concurrency when response sizes approach their byte ceiling. ## Service-specific controls -| Service | Controls and defaults in `standard` | -| --- | --- | -| Chaintracks | `CHAINTRACKS_HEADERS_DEFAULT_LIMIT=1000`, `CHAINTRACKS_HEADERS_MAX_LIMIT=1000`; the static CDN streams files and has its own `CHAINTRACKS_CDN_*` connection policy. | -| Message Box | `MAX_MESSAGE_BODY_BYTES=1048576`, `MAX_RECIPIENTS=100`, `LIST_DEFAULT_LIMIT=1000`, `LIST_MAX_LIMIT=1000`, `LIST_MAX_OFFSET=100000`, `LIST_MAX_RESPONSE_BYTES=8388608`, inbox/sender quotas of 10,000 messages and 1 GiB, `MAX_ACKNOWLEDGMENT_IDS=1000`, device/permission page maximum 100, notification fan-out 100, and `RETENTION_DAYS=30`. `MESSAGE_LIST_BATCH_SIZE` remains a compatibility fallback. | -| Message Box maintenance/state | `AUTH_SESSION_TTL_MS=86400000`, `PAYMENT_REPLAY_TTL_DAYS=365`, `RETENTION_CLEANUP_INTERVAL_MS=900000`, `RETENTION_CLEANUP_BATCH_SIZE=1000`, `DB_POOL_MIN=0`, `DB_POOL_MAX=7`, and `DB_IDLE_TIMEOUT_MS=15000`. Auth sessions, quota locks, and payment replay records are shared in MySQL. | -| Overlay Express | `OVERLAY_MAX_LOOKUP_RESULTS=1000`, `MAX_BASM_TXIDS=1000`, `MAX_BASM_ANCHOR_RANGE=1000`, admin default/max pages 50/200, `JANITOR_BATCH_SIZE=250`, and `JANITOR_MAX_REPORT_RESULTS=1000`. Janitor scans every record through a cursor while retaining only the configured report detail. | -| UHRP Basic / Cloud | list default/max 200/1,000 and max offset 100,000; `MAX_FILE_BYTES=11000000000`, `MAX_RETENTION_MINUTES=525600`, JSON body 256 KiB, and upload body 64 MiB. Basic also bounds its MIME LRU at 10,000 entries. Downloads and uploads remain streamed. | -| WAB | 256 KiB JSON, 2 MiB response, MySQL pool min/max 2/10, and separate pre-auth, authentication, user, deletion, faucet, and share rate policies. Account-deletion state is database-backed. | -| Wallet Storage | RPC default/max rows 1,000/1,000, max request array items 1,000,000, 8 MiB RPC response, 8 MiB JSON, 8 MiB binary, and MySQL pool min/max 2/10 with configurable create/acquire/idle/reap/retry timeouts. Use `WALLET_INFRA_ROLE=api` for HTTP replicas and one `monitor` replica for background work. The optional monitor operator UI/API uses its own bounded `WALLET_ADMIN_*` edge policy and must remain on that singleton's private operator listener. | +| Service | Controls and defaults in `standard` | +| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Chaintracks | `CHAINTRACKS_HEADERS_DEFAULT_LIMIT=1000`, `CHAINTRACKS_HEADERS_MAX_LIMIT=1000`; the static CDN streams files and has its own `CHAINTRACKS_CDN_*` connection policy. | +| Message Box | `MAX_MESSAGE_BODY_BYTES=1048576`, `MAX_RECIPIENTS=100`, `LIST_DEFAULT_LIMIT=1000`, `LIST_MAX_LIMIT=1000`, `LIST_MAX_OFFSET=100000`, `LIST_MAX_RESPONSE_BYTES=8388608`, inbox/sender quotas of 10,000 messages and 1 GiB, `MAX_ACKNOWLEDGMENT_IDS=1000`, device/permission page maximum 100, notification fan-out 100, and `RETENTION_DAYS=30`. `MESSAGE_LIST_BATCH_SIZE` remains a compatibility fallback. | +| Message Box maintenance/state | `AUTH_SESSION_TTL_MS=86400000`, `PAYMENT_REPLAY_TTL_DAYS=365`, `RETENTION_CLEANUP_INTERVAL_MS=900000`, `RETENTION_CLEANUP_BATCH_SIZE=1000`, `DB_POOL_MIN=0`, `DB_POOL_MAX=7`, and `DB_IDLE_TIMEOUT_MS=15000`. Auth sessions, quota locks, and payment replay records are shared in MySQL. | +| Overlay Express | `OVERLAY_MAX_LOOKUP_RESULTS=1000`, `MAX_BASM_TXIDS=1000`, `MAX_BASM_ANCHOR_RANGE=1000`, admin default/max pages 50/200, `JANITOR_BATCH_SIZE=250`, and `JANITOR_MAX_REPORT_RESULTS=1000`. Janitor scans every record through a cursor while retaining only the configured report detail. | +| UHRP Basic / Cloud | list default/max 200/1,000 and max offset 100,000; `MAX_FILE_BYTES=11000000000`, `MAX_RETENTION_MINUTES=525600`, JSON body 256 KiB, and upload body 64 MiB. Basic also bounds its MIME LRU at 10,000 entries. Downloads and uploads remain streamed. | +| WAB | 256 KiB JSON, 2 MiB response, MySQL pool min/max 2/10, and separate pre-auth, authentication, user, deletion, faucet, and share rate policies. Account-deletion state is database-backed. | +| Wallet Storage | RPC default/max rows 1,000/1,000, max request array items 1,000,000, 8 MiB RPC response, 8 MiB JSON, 8 MiB binary, and MySQL pool min/max 2/10 with configurable create/acquire/idle/reap/retry timeouts. Managed-change shaping defaults to 8 new outputs, 4 fee-positive migration inputs, and exact pending comparison above 16 inputs per action; each `WALLET_STORAGE_MANAGED_CHANGE_*` work budget accepts `-1`. Use `WALLET_INFRA_ROLE=api` for HTTP replicas and one `monitor` replica for background work. The optional monitor operator UI/API uses its own bounded `WALLET_ADMIN_*` edge policy and must remain on that singleton's private operator listener. | All names above are appended to the service prefix where it is omitted in the table. For example, Message Box `MAX_RECIPIENTS` means @@ -105,14 +105,14 @@ three-copy concurrency model consumes more than 80% of profile memory. The 2026-08-04 Apple arm64 run produced these `standard` results: -| Service | Representative maximum page | Measured RSS increase | Three-copy concurrency model | -| --- | ---: | ---: | ---: | -| Chaintracks | 157 KiB | 1.2 MiB | 29 MiB | -| Message Box | 2,001 KiB | 11.9 MiB | 141 MiB | -| Overlay Express | 4,001 KiB | 22.3 MiB | 281 MiB | -| UHRP Basic / Cloud | 1,001 KiB | 5.7 MiB | 188 MiB | -| WAB | 2 KiB | 0.1 MiB | 0.8 MiB | -| Wallet Storage | 4,001 KiB | 21.9 MiB | 281 MiB | +| Service | Representative maximum page | Measured RSS increase | Three-copy concurrency model | +| ------------------ | --------------------------: | --------------------: | ---------------------------: | +| Chaintracks | 157 KiB | 1.2 MiB | 29 MiB | +| Message Box | 2,001 KiB | 11.9 MiB | 141 MiB | +| Overlay Express | 4,001 KiB | 22.3 MiB | 281 MiB | +| UHRP Basic / Cloud | 1,001 KiB | 5.7 MiB | 188 MiB | +| WAB | 2 KiB | 0.1 MiB | 0.8 MiB | +| Wallet Storage | 4,001 KiB | 21.9 MiB | 281 MiB | The model uses fixed representative record sizes (160 B headers, 2 KiB messages, 4 KiB overlay/wallet rows, and 1 KiB UHRP metadata). Real records, @@ -178,15 +178,15 @@ HTTP sends also bound push work with `MESSAGE_BOX_FCM_SEND_CONCURRENCY`; this prevents a valid multi-recipient send from multiplying recipient and device fan-out into an unbounded promise set. -| Variable | Default satoshis | Meaning | -| --- | ---: | --- | -| `MESSAGE_BOX_PRICE_BASE_SATOSHIS` | 50 | Fixed authenticated request component. | -| `MESSAGE_BOX_PRICE_PER_RECIPIENT_SATOSHIS` | 5 | Send fan-out component per recipient. | -| `MESSAGE_BOX_PRICE_PER_KIB_SATOSHIS` | 5 | UTF-8 message body component, rounded up by KiB. | -| `MESSAGE_BOX_PRICE_STORAGE_MIB_MONTH_SATOSHIS` | 1,000 | Retained payload component. | -| `MESSAGE_BOX_PRICE_LIST_PAGE_SATOSHIS` | 5 | Listing page component in addition to the base. | -| `MESSAGE_BOX_PRICE_UNLIMITED_RETENTION_MONTHS` | 12 | Up-front storage horizon when an operator explicitly configures unlimited retention. | -| `MESSAGE_BOX_ROUTE_PRICES_JSON` | `{}` | Absolute route-to-satoshi overrides; `0` makes a protected route free. | +| Variable | Default satoshis | Meaning | +| ---------------------------------------------- | ---------------: | ------------------------------------------------------------------------------------ | +| `MESSAGE_BOX_PRICE_BASE_SATOSHIS` | 50 | Fixed authenticated request component. | +| `MESSAGE_BOX_PRICE_PER_RECIPIENT_SATOSHIS` | 5 | Send fan-out component per recipient. | +| `MESSAGE_BOX_PRICE_PER_KIB_SATOSHIS` | 5 | UTF-8 message body component, rounded up by KiB. | +| `MESSAGE_BOX_PRICE_STORAGE_MIB_MONTH_SATOSHIS` | 1,000 | Retained payload component. | +| `MESSAGE_BOX_PRICE_LIST_PAGE_SATOSHIS` | 5 | Listing page component in addition to the base. | +| `MESSAGE_BOX_PRICE_UNLIMITED_RETENTION_MONTHS` | 12 | Up-front storage horizon when an operator explicitly configures unlimited retention. | +| `MESSAGE_BOX_ROUTE_PRICES_JSON` | `{}` | Absolute route-to-satoshi overrides; `0` makes a protected route free. | Recipient-configured delivery fees remain separate from the operator charge. The operator price is paid and replay-checked before message fan-out; the send @@ -215,11 +215,11 @@ convention: [BSV fee concepts](https://hub.bsvblockchain.org/bsv-skills-center/g The official images now accept the generic runtime settings needed to replace custom Babbage-derived images: -| Workload | Upstream configuration now available | -| --- | --- | -| Message Box | Shared MySQL sessions/replay/quota locks, list/body/inbox/sender/retention limits, DB pool, Firebase and WebSocket controls, BRC-105 pricing, `/healthz`, and legacy `MESSAGE_LIST_BATCH_SIZE`. | -| WAB | DB pool, granular rate/resource policy, shared database deletion flow, `/healthz`, and leading-double-slash compatibility. | -| Wallet Storage | Raw or base64 JSON for `KNEX_DB_CONNECTION` and `FEE_MODEL`; raw or base64 admin keys; API/monitor role split; TAAL, WhatsOnChain, Bitails, Arcade, GorillaPool, and exchange-rate provider settings under `WALLET_STORAGE_*` with legacy aliases; logger level; DB/RPC/resource/payment controls. | +| Workload | Upstream configuration now available | +| -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Message Box | Shared MySQL sessions/replay/quota locks, list/body/inbox/sender/retention limits, DB pool, Firebase and WebSocket controls, BRC-105 pricing, `/healthz`, and legacy `MESSAGE_LIST_BATCH_SIZE`. | +| WAB | DB pool, granular rate/resource policy, shared database deletion flow, `/healthz`, and leading-double-slash compatibility. | +| Wallet Storage | Raw or base64 JSON for `KNEX_DB_CONNECTION` and `FEE_MODEL`; raw or base64 admin keys; API/monitor role split; TAAL, WhatsOnChain, Bitails, Arcade, GorillaPool, and exchange-rate provider settings under `WALLET_STORAGE_*` with legacy aliases; logger level; DB/RPC/resource/payment controls; progressive managed-change output, fragment-migration, and pending-comparison work budgets. | Secrets, DNS, certificates, ingress, replica counts, provider credentials, and cluster-specific shared rate limiting remain downstream. Migration should diff --git a/governance/package-release-notes.json b/governance/package-release-notes.json index 3b064369a..0e18e5259 100644 --- a/governance/package-release-notes.json +++ b/governance/package-release-notes.json @@ -1,6 +1,6 @@ { "schemaVersion": 1, - "lastReviewed": "2026-08-06", + "lastReviewed": "2026-08-10", "owner": "ts-stack-maintainers", "entries": [ { @@ -196,22 +196,22 @@ "name": "@bsv/wallet-toolbox", "publishedVersion": "2.6.5", "releaseType": "patch", - "summary": "Isolates action batches by explicit transaction-graph membership, adds bounded exact-input lease resume, and carries structured lifecycle errors across local and remote storage.", - "migration": "No consumer migration is required. Unrelated actions continue through the legacy path instead of joining or committing an open workspace. Providers may optionally advertise resume support and a maximum reservation count; built-in providers default to 256 outputs and operators can select another positive value or -1 for unlimited operation." + "summary": "Isolates and resumes action batches, then adds progressive managed-change liquidity: 144 useful 5,000-satoshi units, bounded surplus-only fanout and fragment migration, settled-first parent selection, exact BEEF-cost comparison for pathological plans, last-resort pending funding, delayed permission persistence, and read-only Monitor reporting.", + "migration": "No consumer code or BRC-100 migration is required. Exact untouched default baskets at 144 outputs / 32 satoshis advance to a 5,000-satoshi preference; custom basket values remain unchanged and funds migrate only through future authorized actions. Same-tier compatibility planning and retained pending fallback ensure the policy adds no new funding refusal. Operators may tune all work limits, including explicit -1 unlimited modes." }, { "name": "@bsv/wallet-toolbox-client", "publishedVersion": "2.6.5", "releaseType": "patch", - "summary": "Carries the lockstep browser build with explicit action-batch isolation and bounded lease recovery.", - "migration": "No browser consumer migration is required. Related noSend chains retain local batching; unrelated actions remain on their ordinary storage path." + "summary": "Carries the lockstep browser build with isolated resumable action batches and progressive IndexedDB managed-change liquidity, settled-first ancestry, exact BEEF-cost comparison, and delayed permission persistence.", + "migration": "No browser consumer code migration is required. Exact untouched IndexedDB defaults advance from 32 to 5,000 satoshis and then migrate progressively through ordinary authorized actions; custom basket values and remote-storage wire contracts remain unchanged." }, { "name": "@bsv/wallet-toolbox-mobile", "publishedVersion": "2.6.5", "releaseType": "patch", - "summary": "Carries the lockstep mobile build with explicit action-batch isolation and bounded lease recovery.", - "migration": "No mobile consumer migration is required. Related noSend chains retain local batching; unrelated actions remain on their ordinary storage path." + "summary": "Carries the lockstep mobile build with isolated resumable action batches, progressive managed-change liquidity, settled-first ancestry, exact BEEF-cost comparison, and delayed permission persistence.", + "migration": "No mobile consumer code migration is required. Wallets progressively adopt useful change values through future authorized actions; custom basket values, BRC-100 methods, and remote-storage wire contracts remain unchanged." }, { "name": "create-bsv-app", diff --git a/governance/service-operations.json b/governance/service-operations.json index 064522f5b..3aa29286c 100644 --- a/governance/service-operations.json +++ b/governance/service-operations.json @@ -490,6 +490,9 @@ "WALLET_STORAGE_MONITOR_ADMIN_ALLOWED_ORIGINS", "WALLET_STORAGE_MONITOR_ADMIN_PRIVATE_KEY", "WALLET_STORAGE_ADMIN_IDENTITY_KEYS", + "WALLET_STORAGE_MANAGED_CHANGE_MAX_OUTPUTS_PER_ACTION", + "WALLET_STORAGE_MANAGED_CHANGE_MIGRATION_INPUTS_PER_ACTION", + "WALLET_STORAGE_MANAGED_CHANGE_PENDING_COMPARISON_INPUTS", "WALLET_STORAGE_TRUST_PROXY_HOPS" ], "secrets": [ diff --git a/infra/wallet-infra/.env.example b/infra/wallet-infra/.env.example index 6c8d993e9..5d19fa0ee 100644 --- a/infra/wallet-infra/.env.example +++ b/infra/wallet-infra/.env.example @@ -67,6 +67,11 @@ WALLET_STORAGE_RPC_DEFAULT_LIST_LIMIT=1000 WALLET_STORAGE_RPC_MAX_LIST_LIMIT=1000 WALLET_STORAGE_RPC_MAX_ARRAY_ITEMS=1000000 WALLET_STORAGE_RPC_MAX_RESPONSE_BYTES=8388608 +# Managed-change liquidity work budgets. Each accepts -1 for unlimited; see +# README.md before selecting an unlimited value. +WALLET_STORAGE_MANAGED_CHANGE_MAX_OUTPUTS_PER_ACTION=8 +WALLET_STORAGE_MANAGED_CHANGE_MIGRATION_INPUTS_PER_ACTION=4 +WALLET_STORAGE_MANAGED_CHANGE_PENDING_COMPARISON_INPUTS=16 WALLET_STORAGE_REQUEST_TIMEOUT_MS=120000 WALLET_STORAGE_HEADERS_TIMEOUT_MS=15000 WALLET_STORAGE_KEEP_ALIVE_TIMEOUT_MS=5000 diff --git a/infra/wallet-infra/README.md b/infra/wallet-infra/README.md index 4f81f0b57..308847b9c 100644 --- a/infra/wallet-infra/README.md +++ b/infra/wallet-infra/README.md @@ -62,6 +62,45 @@ production-shaped rows under the real memory limit, reduce concurrency when needed, and migrate clients to bounded pages or the balance special operation before restoring the standard 1,000-row maximum. +### Managed-change liquidity + +The official image applies Wallet Toolbox's progressive managed-change policy +without requiring a custom build. New and exact untouched legacy default +baskets target 144 independently useful outputs with a preferred value of +5,000 satoshis. The value is a liquidity preference, not a dust limit: a valid +smaller remainder is retained and no action is refused merely because its +change cannot reach 5,000 satoshis. + +Three environment settings bound the optional work performed by one +user-authorized action: + +| Setting | Default | Meaning | +| ----------------------------------------------------------- | ------: | --------------------------------------------------------------------------------------------------- | +| `WALLET_STORAGE_MANAGED_CHANGE_MAX_OUTPUTS_PER_ACTION` | `8` | Maximum fanout while shaping real surplus. | +| `WALLET_STORAGE_MANAGED_CHANGE_MIGRATION_INPUTS_PER_ACTION` | `4` | Maximum fee-positive legacy fragments consumed only to improve the pool. | +| `WALLET_STORAGE_MANAGED_CHANGE_PENDING_COMPARISON_INPUTS` | `16` | Settled-input count that triggers exact transaction-plus-BEEF comparison with pending alternatives. | + +Each value accepts `-1`. For the first two settings, `-1` removes the +per-action work bound while the available funds and basket target remain +natural bounds. For pending comparison, `-1` disables the optional comparison; +it does not hide pending funds when they are required to fund an action. +Unlimited fanout or migration can create large transactions and ancestry +payloads, so use it only after production-shaped measurement. + +Funding always prefers completed parents, then unproven parents, then sending +parents. Each tier retains the former funding shape as a compatibility fallback +before widening to less-preferred ancestry, so these preferences cannot add a +new insufficient-funds result. Action-batch reservations follow the same +ordering. The Monitor's read-only managed-change report makes pool health and +last-resort pending liquidity observable without signing or consolidating on +the user's behalf; select **managed-change liquidity (read only)** in the +authenticated Monitor admin UI's UTXO review. + +See the Wallet Toolbox +[managed-change liquidity guide](../../packages/wallet/wallet-toolbox/docs/managed-change-liquidity.md) +for the complete algorithm, migration predicate, fee model, action-batch +behavior, rollout checks, and direct-library configuration. + ### Monitor task profile and Arcade events `WALLET_INFRA_ROLE=all` or `monitor` starts monitor work by default. Set diff --git a/infra/wallet-infra/docker-compose.yml b/infra/wallet-infra/docker-compose.yml index 268af70e9..796ce1262 100644 --- a/infra/wallet-infra/docker-compose.yml +++ b/infra/wallet-infra/docker-compose.yml @@ -29,6 +29,9 @@ services: WALLET_STORAGE_FRAME_OPTIONS: ${WALLET_STORAGE_FRAME_OPTIONS:-} WALLET_STORAGE_PERMISSIONS_POLICY: ${WALLET_STORAGE_PERMISSIONS_POLICY:-} WALLET_STORAGE_STRICT_TRANSPORT_SECURITY: ${WALLET_STORAGE_STRICT_TRANSPORT_SECURITY:-} + WALLET_STORAGE_MANAGED_CHANGE_MAX_OUTPUTS_PER_ACTION: ${WALLET_STORAGE_MANAGED_CHANGE_MAX_OUTPUTS_PER_ACTION:-8} + WALLET_STORAGE_MANAGED_CHANGE_MIGRATION_INPUTS_PER_ACTION: ${WALLET_STORAGE_MANAGED_CHANGE_MIGRATION_INPUTS_PER_ACTION:-4} + WALLET_STORAGE_MANAGED_CHANGE_PENDING_COMPARISON_INPUTS: ${WALLET_STORAGE_MANAGED_CHANGE_PENDING_COMPARISON_INPUTS:-16} # Fill in the DB credentials as a JSON object or direct URL KNEX_DB_CONNECTION: '{"host":"mysql","user":"root","password":"rootPass","database":"wallet_storage","port":3306}' # OpenTelemetry — point at any OTLP/HTTP collector. Unset => console exporters. diff --git a/infra/wallet-infra/example.env.yaml b/infra/wallet-infra/example.env.yaml index 06487e55f..81c140651 100644 --- a/infra/wallet-infra/example.env.yaml +++ b/infra/wallet-infra/example.env.yaml @@ -6,6 +6,9 @@ WALLET_STORAGE_TRUST_PROXY_HOPS: '1' WALLET_STORAGE_MONITOR_START_TASKS: 'true' WALLET_STORAGE_MONITOR_STARTUP_TASK_MODE: 'default' WALLET_STORAGE_MONITOR_ADMIN_ENABLED: 'false' +WALLET_STORAGE_MANAGED_CHANGE_MAX_OUTPUTS_PER_ACTION: '8' +WALLET_STORAGE_MANAGED_CHANGE_MIGRATION_INPUTS_PER_ACTION: '4' +WALLET_STORAGE_MANAGED_CHANGE_PENDING_COMPARISON_INPUTS: '16' SERVER_PRIVATE_KEY: '' KNEX_DB_CONNECTION: '{"host": "", "user": "wallet_admin", "password": "", "database": "wallet_storage", "port": 3306}' diff --git a/infra/wallet-infra/guides/kube_samples/README.md b/infra/wallet-infra/guides/kube_samples/README.md index 90d465355..3b997003e 100644 --- a/infra/wallet-infra/guides/kube_samples/README.md +++ b/infra/wallet-infra/guides/kube_samples/README.md @@ -39,6 +39,16 @@ multi-user provider should set leader, and supply matching Arcade URL/callback-token configuration when SSE status delivery is enabled. +The ConfigMap retains the standard managed-change work budgets: at most eight +new change outputs and four fee-positive legacy migration inputs per action, +with exact pending-plan comparison beginning above 16 settled inputs. These are +CPU, fee, transaction-size, and BEEF-ancestry work bounds rather than balance +or spendability limits. Operators can tune the three +`WALLET_STORAGE_MANAGED_CHANGE_*` values after measuring action input counts, +serialized BEEF size, fee, and broadcast outcomes. Every setting accepts `-1`, +but an unlimited value can make one request consume substantially more CPU and +memory and should not be combined casually with increased API concurrency. + The optional monitor operator service is disabled in this sample. If enabled, run it only on the singleton `all` or `monitor` pod, mount its private key and allowed identity keys from the secret manager, use a port distinct from the diff --git a/infra/wallet-infra/guides/kube_samples/wallet-configmap.yaml b/infra/wallet-infra/guides/kube_samples/wallet-configmap.yaml index 133daae4c..afc310e23 100644 --- a/infra/wallet-infra/guides/kube_samples/wallet-configmap.yaml +++ b/infra/wallet-infra/guides/kube_samples/wallet-configmap.yaml @@ -13,3 +13,6 @@ data: WALLET_STORAGE_MONITOR_START_TASKS: 'true' WALLET_STORAGE_MONITOR_STARTUP_TASK_MODE: 'default' WALLET_STORAGE_MONITOR_ADMIN_ENABLED: 'false' + WALLET_STORAGE_MANAGED_CHANGE_MAX_OUTPUTS_PER_ACTION: '8' + WALLET_STORAGE_MANAGED_CHANGE_MIGRATION_INPUTS_PER_ACTION: '4' + WALLET_STORAGE_MANAGED_CHANGE_PENDING_COMPARISON_INPUTS: '16' diff --git a/infra/wallet-infra/src/index.ts b/infra/wallet-infra/src/index.ts index 4e2da7b78..5d4a1a996 100644 --- a/infra/wallet-infra/src/index.ts +++ b/infra/wallet-infra/src/index.ts @@ -127,6 +127,28 @@ function readNonNegativeInteger(name: string, fallback: number): number { return value } +function readManagedChangeLimit( + name: string, + fallback: number, + minimum: number +): number { + const raw = process.env[name] + if (raw == null || raw.trim() === '') return fallback + if (raw === '-1') return -1 + if (!/^\d+$/.test(raw)) { + throw new Error( + `${name} must be ${minimum === 0 ? 'a non-negative' : 'a positive'} integer or -1` + ) + } + const value = Number(raw) + if (!Number.isSafeInteger(value) || value < minimum) { + throw new Error( + `${name} must be ${minimum === 0 ? 'a non-negative' : 'a positive'} safe integer or -1` + ) + } + return value +} + function readPort(name: string, fallback: number): number { const value = readPositiveInteger(name, fallback) if (value > 65_535) throw new Error(`${name} must be between 1 and 65535`) @@ -590,13 +612,35 @@ async function setupWalletStorageAndMonitor(): Promise { const rootKey = PrivateKey.fromHex(SERVER_PRIVATE_KEY) const storageIdentityKey = rootKey.toPublicKey().toString() - const activeStorage = new StorageKnex({ + // Keep this object separate from the constructor call so the coordinated + // image can still compile against the previously published toolbox during + // source review. The protected release refreshes the lock to the package + // candidate that consumes managedChangePolicy before publishing the image. + const storageOptions = { chain, knex, commissionSatoshis, commissionPubKeyHex: COMMISSION_PUBLIC_KEY || undefined, - feeModel: JSON.parse(decodeJsonSetting('FEE_MODEL', String(FEE_MODEL))) - }) + feeModel: JSON.parse(decodeJsonSetting('FEE_MODEL', String(FEE_MODEL))), + managedChangePolicy: { + maxOutputsPerAction: readManagedChangeLimit( + 'WALLET_STORAGE_MANAGED_CHANGE_MAX_OUTPUTS_PER_ACTION', + 8, + 1 + ), + migrationInputsPerAction: readManagedChangeLimit( + 'WALLET_STORAGE_MANAGED_CHANGE_MIGRATION_INPUTS_PER_ACTION', + 4, + 0 + ), + pendingComparisonInputs: readManagedChangeLimit( + 'WALLET_STORAGE_MANAGED_CHANGE_PENDING_COMPARISON_INPUTS', + 16, + 1 + ) + } + } + const activeStorage = new StorageKnex(storageOptions) await activeStorage.migrate(databaseName, storageIdentityKey) const settings = await activeStorage.makeAvailable() diff --git a/packages/wallet/wallet-toolbox/CHANGELOG.md b/packages/wallet/wallet-toolbox/CHANGELOG.md index a0cd13677..c9507951a 100644 --- a/packages/wallet/wallet-toolbox/CHANGELOG.md +++ b/packages/wallet/wallet-toolbox/CHANGELOG.md @@ -6,6 +6,20 @@ attention to changes that materially alter behavior or extend functionality. ## wallet-toolbox (unreleased) +- Replace 32-satoshi default-basket fragments with a progressive liquidity + policy targeting 144 useful 5,000-satoshi outputs. New actions create at most + eight outputs from real surplus and migrate at most four fee-positive legacy + fragments, while a same-tier compatibility plan guarantees that optional + shaping cannot refuse an action the former planner could fund. +- Prefer completed, then unproven, then sending parents. Plans above 16 inputs + compare exact transaction-plus-BEEF bytes before accepting pending ancestry; + pending change remains an unconditional last-resort funding source. Align + action-batch reservation/planning and add a read-only Monitor liquidity + report. All work limits are configurable and accept `-1` for explicit + operator-selected unlimited behavior. +- Restore delayed broadcast for durable permission-token persistence. Permission + grants no longer inherit network-broadcast latency; the managed-change policy + handles queued ancestry without hiding it or preferring it over settled funds. - Isolate each in-memory action batch by explicit staged-output or `sendWith` membership, so unrelated immediate actions and `noSend` roots cannot be captured by or commit a workspace. Add an exact-input resume protocol for @@ -24,14 +38,6 @@ attention to changes that materially alter behavior or extend functionality. `constructor.name === 'StorageClient'`. Production minifiers rename classes, so the name check left remote stores with `endpointURL: undefined` while sync still worked; clients that select a backup by URL (make primary) failed. -- Allow immediate actions to chain wallet-managed change from transactions - awaiting background broadcast when settled change is insufficient. The child - broadcast recursively includes the delayed parent BEEF, preventing a large - funding output from making the wallet appear temporarily unfunded while - preserving settled-change preference and delayed-broadcast semantics. -- Finish broadcasting durable permission-token grants before resuming the - waiting application request, preventing the grant transaction from briefly - reserving the wallet's funding inputs out from under the resumed action. - Let Storage Server operators select an explicit listener host while retaining the historical omitted-host behavior for existing callers. The official Wallet Infrastructure image uses this to bind direct-mode traffic on IPv4 diff --git a/packages/wallet/wallet-toolbox/README.md b/packages/wallet/wallet-toolbox/README.md index e14cbe81e..11c71eece 100644 --- a/packages/wallet/wallet-toolbox/README.md +++ b/packages/wallet/wallet-toolbox/README.md @@ -26,18 +26,16 @@ BSV Desktop and BSV Browser are the BSV Association reference wallet application | **MockChain** | In-memory blockchain for testing — mock mining, UTXO tracking, and merkle proof generation without a network | | **Entropy** | `EntropyCollector` gathers mouse/touch entropy for high-quality randomness in browser environments | -Durable permission grants finish broadcasting their internal token transaction -before the waiting application request resumes. This keeps the grant atomic from -the caller's perspective and makes its funding change immediately reusable by a -following wallet action. If broadcasting is unavailable, the grant rejects and -the application can safely surface the error and retry; ephemeral one-time grants -remain off-chain. - -Immediate actions can fund from wallet-managed change created by a transaction -that is still awaiting background broadcast when settled change is insufficient. -The wallet prefers settled change, then recursively includes the delayed parent -in the child BEEF only when needed, so queued work cannot temporarily strand the -wallet's balance behind a large reserved input. +Durable permission grants queue their internal token transaction for delayed +broadcast, so permission approval does not inherit network-broadcast latency. +The funding planner prefers settled change and uses queued permission ancestry +only as a last resort, keeping the application path fast without hiding funds. + +Immediate actions prefer completed, then unproven, then sending change. A +pathological settled plan is compared with pending alternatives by exact +serialized BEEF plus transaction bytes; queued ancestry is used only when it is +necessary or smaller. Pending change is never withheld, so queued +work cannot strand the balance behind a large reserved input. ### Packages @@ -128,6 +126,11 @@ See [Managed change, sweeping, and recovery](./docs/managed-change-policy.md) for the default-basket invariant, automatic funding policy, and supported `internalizeAction` repair paths. +See [Managed-change liquidity policy](./docs/managed-change-liquidity.md) for +the 144-output / 5,000-satoshi defaults, gradual legacy-wallet migration, +pending-parent policy, exact BEEF comparison, operator tuning, action-batch +alignment, monitoring, and rollout guidance. + See [In-memory action batch planning](./docs/action-batch-planning.md) for capability-negotiated `noSend` planning, compact manifests, compressed binary pack transport, atomic commit, compatibility behavior, and retained benchmarks. @@ -145,7 +148,7 @@ The planner uses the same exact / least-over / largest-under selection policy as the historical allocator, but proves economic sufficiency before writing a transaction and claims every selected input in one database transaction. Knex storage automatically adds a composite funding-selection index on migration; -IndexedDB schema version 3 adds corresponding user/basket and outpoint indexes +IndexedDB schema version 4 adds corresponding user/basket and outpoint indexes and resolves transaction-status eligibility in one indexed pass. The retained fragmented-funding benchmark is runnable with: diff --git a/packages/wallet/wallet-toolbox/docs/managed-change-liquidity.md b/packages/wallet/wallet-toolbox/docs/managed-change-liquidity.md new file mode 100644 index 000000000..2ff52aca6 --- /dev/null +++ b/packages/wallet/wallet-toolbox/docs/managed-change-liquidity.md @@ -0,0 +1,238 @@ +# Managed-change liquidity policy + +Wallet-managed change is both the wallet's balance and its concurrency pool. +An output that is technically spendable but too small to carry a useful action +at the current fee rate does not provide useful liquidity. Conversely, creating +the entire pool in one transaction makes every child carry a large common BEEF +ancestor and creates unnecessary linkage. + +This policy keeps `createAction` available to existing callers while gradually +moving new and existing wallets toward useful, parallel funding units. + +## Invariants + +The implementation treats these as non-negotiable: + +1. A policy preference cannot make an action fail if the former planner could + fund it. Each parent-status tier retries the compatibility funding shape + before the planner widens to less-preferred ancestry. +2. `completed` parents are preferred, then `unproven`, then `sending`. + Pending outputs are never withheld: they remain a last-resort source and + are selected when settled liquidity cannot fund the action. +3. A large settled-input plan may be compared with pending alternatives, but + only after the configurable comparison threshold. The comparison uses the + actual serialized BEEF bytes plus the planned transaction bytes; input + count and satoshi value are not used as BEEF-size proxies. +4. Pool growth consumes only surplus already present after the requested + action and its exact incremental fee are funded. The wallet does not gather + another input merely to manufacture change outputs. +5. An output below the preferred value is permitted when it is the only + fundable remainder. The preferred value is not a dust rule and cannot turn + a valid payment into `WERR_INSUFFICIENT_FUNDS`. +6. Legacy fragments migrate only on a caller-authorized `createAction`, only + when their value exceeds their marginal input fee, and only within the + configured per-action budget. +7. Action-batch workspaces reserve disjoint outputs and use the same change + values and per-action shaping limits as the legacy `createAction` path. +8. Permission-token persistence retains delayed broadcast so a permission + grant does not inherit network-broadcast latency. The funding policy makes + queued change available only after preferred alternatives are exhausted. + +## Defaults + +| Setting | Default | Purpose | +| ---------------------------------- | -------------: | ---------------------------------------------------------------------------------------------------------------- | +| Default-basket target | 144 outputs | Supports many independently planned actions without requiring one large fanout transaction. | +| Preferred output value | 5,000 satoshis | Keeps a liquidity unit useful at fee rates materially above the historical 32-satoshi era. | +| New outputs per action | 8 | Builds the pool progressively and bounds any one transaction's fanout and descendant BEEF footprint. | +| Legacy migration inputs per action | 4 | Retires old fragments progressively without recreating 178-input permission transactions. | +| Pending-comparison threshold | 16 inputs | Keeps the common settled path fast; above this point the planner measures alternatives by exact serialized cost. | + +At the Wallet Toolbox default of 100 satoshis/kB, a 148-byte managed input adds +about 15 satoshis of fee and a minimal one-input/one-output transaction costs +about 20 satoshis. A 5,000-satoshi preferred unit is therefore roughly 250 +minimal-spend fees at that rate. Even at 1,000 satoshis/kB it remains roughly +26 minimal-spend fees. The value is intentionally a liquidity target, not a +consensus or economic-dust boundary. + +A completely filled default pool represents 720,000 satoshis. Wallets with a +smaller balance do not attempt to manufacture that reserve. They retain fewer +outputs, and a remainder below 5,000 satoshis is kept when that is the only +available shape. + +## Funding and ancestry selection + +For each action, storage loads unreserved managed change once and plans in this +order: + +1. completed parents; +2. completed plus unproven parents; +3. completed, unproven, and sending parents. + +Within each tier, the new surplus-only shape runs first. If that shape reports +insufficient funds, the same tier is immediately retried with the former +funding algorithm and the allocator's economic floor for its first remainder. +This is deliberately at least as permissive as the historical 32-satoshi +basket. An inability to create a preferred 5,000-satoshi change output is not +evidence that settled funds cannot pay the requested output. + +The first successful plan is accepted immediately when it uses no more than 16 +managed inputs. Above that threshold, later status tiers are also planned and +the wallet compares: + +```text +serialized cost = planned transaction bytes + exact input BEEF bytes +``` + +The smallest measured plan wins. If proof retrieval needed only for comparison +is unavailable, that alternative receives an infinite comparison cost; the +already fundable baseline remains available. This optimization can therefore +improve latency and BEEF size but cannot become a new availability dependency. + +Using a `sending` parent necessarily extends the unconfirmed BEEF chain and a +failed ancestor can invalidate its descendants. That is why it is last in the +normal order. It remains supported because refusing a fundable user action is +worse than reluctantly extending the chain after all safer liquidity is +exhausted. + +## Progressive pool shaping + +After compulsory funding succeeds, the planner may consume up to four +undersized outputs. A fragment is skipped when spending it would cost at least +its value. Optional migration never supplies a missing satoshi for the caller's +requested outputs and never runs when the basket is already at its target. + +The resulting surplus is split into at most eight outputs and only when every +new output can meet the preferred value after paying the exact added output +fee. Otherwise the wallet keeps one output. Excess is distributed through the +existing randomized change algorithm, and ordinary output randomization still +applies. The policy therefore preserves the existing privacy intent and +non-uniform values whenever surplus exists instead of replacing it with a +fixed deterministic denomination scheme. When the available value is exactly +the sum of the preferred minima, equal minima are unavoidable; the wallet does +not create a smaller output merely to force cosmetic variance. + +The target is based on healthy outputs (those at or above the basket's +preferred value), not raw output count. Consuming a legacy fragment and +creating a useful output moves the pool forward; merely retaining another +32-satoshi fragment does not make the pool appear healthy. + +## Existing-wallet migration + +The SQL and IndexedDB providers recognize only the exact historical untouched +default: + +```text +basket name = default +target count = 144 +minimum value = 32 +``` + +Those baskets advance to a 5,000-satoshi preferred value during the normal +provider migration. Other basket names, custom target counts, and custom +minimum values are unchanged. The migration does not consolidate, sign, or +broadcast a transaction. Future `createAction` calls progressively consume at +most the configured number of fee-positive fragments and create useful change +only from real surplus. + +The SQL data migration is intentionally one-way. Rolling code back does not +rewrite a migrated preference to 32 or fragment funds. Older code can still +read and honor the 5,000-satoshi basket value. + +## Operator configuration + +The defaults are suitable for a general wallet. Local Knex and IndexedDB +operators can tune the work budgets without rebuilding the package: + +```ts +const setup = await Setup.createWalletKnex({ + ...args, + managedChangePolicy: { + maxOutputsPerAction: 8, + migrationInputsPerAction: 4, + pendingComparisonInputs: 16 + } +}) +``` + +The same `managedChangePolicy` option is accepted by `StorageKnex`, +`StorageIdb`, `Setup.createStorageKnex`, and `SetupClient.createStorageIdb`. +The basket target and preferred value remain user-wallet settings and can be +changed through `wallet.setWalletChangeParams(count, satoshis)`. + +Each policy limit accepts `-1`, with deliberately different meanings: + +- `maxOutputsPerAction: -1` makes the basket target the only fanout bound; +- `migrationInputsPerAction: -1` permits all fee-positive legacy fragments in + one authorized action; +- `pendingComparisonInputs: -1` disables optional pending-plan comparison, but + pending funds remain available when earlier tiers are insufficient. + +Unlimited modes can create large transactions or BEEF payloads and should be +used only by operators that have measured their workload. They do not remove +the wallet's economic-dust check, transaction validity checks, action-batch +reservation limit, or available-funding bound. + +The official `wallet-infra` image exposes the same settings as validated +environment values: + +```dotenv +WALLET_STORAGE_MANAGED_CHANGE_MAX_OUTPUTS_PER_ACTION=8 +WALLET_STORAGE_MANAGED_CHANGE_MIGRATION_INPUTS_PER_ACTION=4 +WALLET_STORAGE_MANAGED_CHANGE_PENDING_COMPARISON_INPUTS=16 +``` + +This lets a hosted Wallet Storage provider choose the policy without rebuilding +the image. Invalid, unsafe-integer, or out-of-range values fail startup instead +of silently falling back. The API and singleton Monitor roles must use the same +values so planning, reservation, and operator reporting describe one policy. + +## Action batches and concurrent workspaces + +An action-batch begin response carries the effective output and migration +limits when the provider supports them. Older and third-party providers can +omit the optional policy; clients then use the same defaults. Initial and +extended reservation selection prefers completed parents, then unproven, then +sending, while still reserving enough last-resort liquidity to avoid wedging a +workspace. + +Workspaces remain isolated by explicit transaction-graph membership and their +reservations remain disjoint. Pool-shaping state is local to each plan; there +is no global in-memory "current batch" whose change can be consumed by an +unrelated action. + +## Monitoring and rollout + +`TaskReviewUtxos.reviewManagedChangeByIdentityKey(identityKey)` is a read-only +operator report. It returns: + +- total and target managed-change counts; +- healthy and undersized counts; +- active action-batch reservations; +- completed, unproven, and sending parent counts; +- total satoshis and the preferred minimum. + +Monitor has no signing authority and does not perform consolidation. This +keeps migration tied to normal user-authorized actions and makes rollout +observable without creating surprise transactions. The official Monitor admin +UI exposes the same report as the **managed-change liquidity (read only)** UTXO +review mode. + +Recommended rollout checks are: + +1. record the report before upgrade; +2. migrate the storage provider and confirm only exact legacy defaults changed; +3. exercise a small immediate action, a delayed permission action, and two + concurrent action-batch workspaces; +4. confirm undersized count declines gradually and sending-parent use remains + exceptional; +5. watch action input count, serialized BEEF bytes, fee, and broadcast failure + rates before changing defaults or selecting an unlimited mode. + +## Compatibility surface + +No BRC-100 method, Wallet Wire method, Storage Server RPC method, or persisted +transaction format changes. The action-batch policy field is optional. Existing +custom basket settings remain authoritative. A same-tier compatibility plan, +followed by the retained pending-parent fallback, ensures the new preferences +do not add a refusal where the previous wallet could create an action. diff --git a/packages/wallet/wallet-toolbox/src/Setup.ts b/packages/wallet/wallet-toolbox/src/Setup.ts index ebfa6101c..72f5f65b5 100644 --- a/packages/wallet/wallet-toolbox/src/Setup.ts +++ b/packages/wallet/wallet-toolbox/src/Setup.ts @@ -28,6 +28,7 @@ import { Wallet } from './Wallet' import { StorageClient } from './storage/remoting/StorageClient' import { StorageKnex } from './storage/StorageKnex' import { WalletStorageProvider } from './sdk/WalletStorage.interfaces' +import type { ManagedChangePolicyOptions } from './storage/methods/managedChangePolicy' // To rely on your own headers service, uncomment the following line: // import { BHServiceClient } from './services/chaintracker' @@ -391,6 +392,7 @@ DEV_KEYS = '{ commissionPubKeyHex: undefined, feeModel: { model: 'sat/kb', value: 100 }, actionBatchMaxReservedOutputs: args.actionBatchMaxReservedOutputs, + managedChangePolicy: args.managedChangePolicy, scriptVerifier: args.scriptVerifier }) await storage.migrate(args.databaseName, randomBytesHex(33)) @@ -490,6 +492,8 @@ export interface SetupWalletArgs { * storage validation. This does not alter the BRC-100 interface. */ scriptVerifier?: SpendVerifierInterface + /** Optional operator tuning for wallet-managed liquidity shaping. */ + managedChangePolicy?: ManagedChangePolicyOptions } /** diff --git a/packages/wallet/wallet-toolbox/src/SetupClient.ts b/packages/wallet/wallet-toolbox/src/SetupClient.ts index aea03b6e5..5868bee7e 100644 --- a/packages/wallet/wallet-toolbox/src/SetupClient.ts +++ b/packages/wallet/wallet-toolbox/src/SetupClient.ts @@ -25,6 +25,7 @@ import { Wallet } from './Wallet' import { Chain } from './sdk/types' import { randomBytesHex } from './utility/utilityHelpers' import { StorageClient } from './storage/remoting/StorageClient' +import type { ManagedChangePolicyOptions } from './storage/methods/managedChangePolicy' /** * The 'Setup` class provides static setup functions to construct BRC-100 compatible @@ -284,6 +285,7 @@ export abstract class SetupClient { commissionSatoshis: 0, commissionPubKeyHex: undefined, feeModel: { model: 'sat/kb', value: 100 }, + managedChangePolicy: args.managedChangePolicy, scriptVerifier: args.scriptVerifier }) await storage.migrate(args.databaseName, randomBytesHex(33)) @@ -297,6 +299,8 @@ export abstract class SetupClient { */ export interface SetupWalletIdbArgs extends SetupClientWalletArgs { databaseName: string + /** Optional operator tuning for wallet-managed liquidity shaping. */ + managedChangePolicy?: ManagedChangePolicyOptions } /** diff --git a/packages/wallet/wallet-toolbox/src/SetupWallet.ts b/packages/wallet/wallet-toolbox/src/SetupWallet.ts index 3e78ac145..e55bb1eb7 100644 --- a/packages/wallet/wallet-toolbox/src/SetupWallet.ts +++ b/packages/wallet/wallet-toolbox/src/SetupWallet.ts @@ -1,5 +1,6 @@ import { PrivateKey, PublicKey, KeyDeriverApi } from '@bsv/sdk' import type { SpendVerifierInterface } from '@bsv/sdk' +import type { ManagedChangePolicyOptions } from './storage/methods/managedChangePolicy' import { Wallet } from './Wallet' import { Chain } from './sdk/types' @@ -100,6 +101,8 @@ export interface SetupClientWalletArgs { * storage validation. This does not alter the BRC-100 interface. */ scriptVerifier?: SpendVerifierInterface + /** Optional operator tuning for local wallet-managed liquidity shaping. */ + managedChangePolicy?: ManagedChangePolicyOptions } /** diff --git a/packages/wallet/wallet-toolbox/src/WalletPermissionsManager.ts b/packages/wallet/wallet-toolbox/src/WalletPermissionsManager.ts index 0c9bc8467..311ebdeb0 100644 --- a/packages/wallet/wallet-toolbox/src/WalletPermissionsManager.ts +++ b/packages/wallet/wallet-toolbox/src/WalletPermissionsManager.ts @@ -3065,10 +3065,10 @@ export class WalletPermissionsManager implements WalletInterface { } ], options: { - // The application request resumes as soon as this durable grant - // returns. Finish broadcasting first so the grant transaction's - // funding change is reusable by that request. - acceptDelayedBroadcast: false + // Permission persistence must not inherit network-broadcast latency. + // The managed-change planner keeps the queued output available as a + // last-resort funding source for the resumed request. + acceptDelayedBroadcast: true } }, this.adminOriginator @@ -3167,14 +3167,11 @@ export class WalletPermissionsManager implements WalletInterface { 8, async c => await this.buildPermissionOutput(c.request, c.expiry, c.amount) ) - // Grouped callers resume immediately after this method returns. Delayed - // broadcast can leave the grant transaction's funding inputs reserved - // while the resumed request is being planned. await this.createAction( { description: `Grant ${built.length} permissions`, outputs: built.map(b => b.output), - options: { acceptDelayedBroadcast: false } + options: { acceptDelayedBroadcast: true } }, this.adminOriginator ) diff --git a/packages/wallet/wallet-toolbox/src/__tests__/WalletPermissionsManager.permissionSettlement.test.ts b/packages/wallet/wallet-toolbox/src/__tests__/WalletPermissionsManager.permissionSettlement.test.ts index ce8ccd03a..fbbf10041 100644 --- a/packages/wallet/wallet-toolbox/src/__tests__/WalletPermissionsManager.permissionSettlement.test.ts +++ b/packages/wallet/wallet-toolbox/src/__tests__/WalletPermissionsManager.permissionSettlement.test.ts @@ -1,12 +1,32 @@ +import { LockingScript, PushDrop } from '@bsv/sdk' import { WalletPermissionsManager } from '../WalletPermissionsManager' describe('WalletPermissionsManager permission settlement', () => { - it('finishes broadcasting grouped permission tokens before the grant returns', async () => { - let finishBroadcast: (() => void) | undefined - const broadcastFinished = new Promise(resolve => { - finishBroadcast = resolve - }) - const createAction = jest.fn(async () => await broadcastFinished) + afterEach(() => jest.restoreAllMocks()) + + it('queues a single durable permission token without inheriting broadcast latency', async () => { + const createAction = jest.fn(async () => ({ txid: 'single-permission-token' })) + const manager = Object.create(WalletPermissionsManager.prototype) as WalletPermissionsManager + const internals = manager as any + internals.adminOriginator = 'admin.com' + internals.underlying = {} + internals.createAction = createAction + internals.buildPushdropFields = jest.fn().mockResolvedValue([]) + internals.buildTagsForRequest = jest.fn().mockReturnValue([]) + jest.spyOn(PushDrop.prototype, 'lock').mockResolvedValue(LockingScript.fromHex('51')) + + await internals.createPermissionOnChain({ type: 'basket', originator: 'todo.example', basket: 'todo tokens' }, 0) + + expect(createAction).toHaveBeenCalledWith( + expect.objectContaining({ + options: { acceptDelayedBroadcast: true } + }), + 'admin.com' + ) + }) + + it('queues grouped permission tokens without inheriting network-broadcast latency', async () => { + const createAction = jest.fn(async () => ({ txid: 'permission-token-transaction' })) const manager = Object.create(WalletPermissionsManager.prototype) as WalletPermissionsManager const internals = manager as any internals.adminOriginator = 'admin.com' @@ -22,32 +42,22 @@ describe('WalletPermissionsManager permission settlement', () => { } })) - let settled = false - const grant = internals - .createPermissionTokensBestEffort( - [ - { - request: { type: 'basket', originator: 'todo.example', basket: 'todo tokens' }, - expiry: 0 - } - ], - true - ) - .then(() => { - settled = true - }) + const granted = await internals.createPermissionTokensBestEffort( + [ + { + request: { type: 'basket', originator: 'todo.example', basket: 'todo tokens' }, + expiry: 0 + } + ], + true + ) - await new Promise(resolve => setImmediate(resolve)) - expect(settled).toBe(false) + expect(granted).toHaveLength(1) expect(createAction).toHaveBeenCalledWith( expect.objectContaining({ - options: { acceptDelayedBroadcast: false } + options: { acceptDelayedBroadcast: true } }), expect.any(String) ) - - finishBroadcast?.() - await grant - expect(settled).toBe(true) }) }) diff --git a/packages/wallet/wallet-toolbox/src/monitor/tasks/TaskReviewUtxos.ts b/packages/wallet/wallet-toolbox/src/monitor/tasks/TaskReviewUtxos.ts index b6c8fa817..95e2a79c4 100644 --- a/packages/wallet/wallet-toolbox/src/monitor/tasks/TaskReviewUtxos.ts +++ b/packages/wallet/wallet-toolbox/src/monitor/tasks/TaskReviewUtxos.ts @@ -1,6 +1,8 @@ import { Validation, WalletOutput } from '@bsv/sdk' import { specOpInvalidChange } from '../../sdk' +import { isAutoSpendableChangeOutput, managedChangeOutputFields } from '../../storage/methods/managedChange' import { TableUser } from '../../storage/schema/tables' +import { verifyOne } from '../../utility/utilityHelpers' import { Monitor } from '../Monitor' import { WalletMonitorTask } from './WalletMonitorTask' @@ -71,6 +73,41 @@ export class TaskReviewUtxos extends WalletMonitorTask { }) } + /** + * Report managed-change liquidity without changing it. Monitor deliberately + * has no signing authority; progressive migration occurs only during a + * caller-authorized createAction. + */ + async reviewManagedChangeByIdentityKey (identityKey: string): Promise { + return await this.storage.runAsStorageProvider(async sp => { + const user = (await sp.findUsers({ partial: { identityKey } }))[0] + if (user == null) return `identityKey ${identityKey} was not found\n` + const basket = verifyOne(await sp.findOutputBaskets({ partial: { userId: user.userId, name: 'default' } })) + const outputs = (await sp.findOutputs({ + partial: { userId: user.userId, basketId: basket.basketId, spendable: true, ...managedChangeOutputFields }, + txStatus: ['completed', 'unproven', 'sending'], + noScript: true + })).filter(isAutoSpendableChangeOutput) + const reserved = new Set(await sp.findReservedActionBatchOutputIds(outputs.map(output => output.outputId))) + const statuses = await sp.findTransactionStatusesByIds( + user.userId, + outputs.map(output => output.transactionId) + ) + const preferred = Math.max(1, basket.minimumDesiredUTXOValue) + const healthy = outputs.filter(output => output.satoshis >= preferred) + const undersized = outputs.filter(output => output.satoshis < preferred) + const countStatus = (status: 'completed' | 'unproven' | 'sending'): number => + outputs.filter(output => statuses.get(output.transactionId) === status).length + const satoshis = outputs.reduce((sum, output) => sum + output.satoshis, 0) + return ( + `userId ${user.userId}: managed change ${outputs.length}/${basket.numberOfDesiredUTXOs}, ` + + `healthy ${healthy.length}, undersized ${undersized.length}, reserved ${reserved.size}, ` + + `completed ${countStatus('completed')}, unproven ${countStatus('unproven')}, ` + + `sending ${countStatus('sending')}, satoshis ${satoshis}, preferred minimum ${preferred}\n` + ) + }) + } + private toUserLog ( user: TableUser, outputs: WalletOutput[], diff --git a/packages/wallet/wallet-toolbox/src/sdk/ActionBatch.interfaces.ts b/packages/wallet/wallet-toolbox/src/sdk/ActionBatch.interfaces.ts index 46b87c2ef..cb2ec5f52 100644 --- a/packages/wallet/wallet-toolbox/src/sdk/ActionBatch.interfaces.ts +++ b/packages/wallet/wallet-toolbox/src/sdk/ActionBatch.interfaces.ts @@ -67,6 +67,11 @@ export interface BeginActionBatchResult { commissionSatoshis: number commissionPubKeyHex?: string availableChangeCount: number + /** Internal planner policy; absent on older providers, which use defaults. */ + managedChangePolicy?: { + maxOutputsPerAction: number + migrationInputsPerAction: number + } reservedOutputs: ActionBatchFundingOutput[] explicitOutputs: ActionBatchFundingOutput[] inputBeef?: number[] | Uint8Array @@ -76,7 +81,7 @@ export interface ExtendActionBatchArgs { batchId: string targetSatoshis: number requestedOutputs: number - explicitOutpoints: Array<{ txid: string, vout: number }> + explicitOutpoints: Array<{ txid: string; vout: number }> includeSourceTransactions: boolean } @@ -94,7 +99,7 @@ export interface RenewActionBatchResult { export interface ResumeActionBatchArgs { batchId: string /** Exact persisted outputs still held by the client workspace. */ - outpoints: Array<{ txid: string, vout: number }> + outpoints: Array<{ txid: string; vout: number }> } export interface ResumeActionBatchResult extends RenewActionBatchResult {} diff --git a/packages/wallet/wallet-toolbox/src/signer/actionBatch/ActionBatchPlanner.ts b/packages/wallet/wallet-toolbox/src/signer/actionBatch/ActionBatchPlanner.ts index 6ad6107cf..04c8cc3b2 100644 --- a/packages/wallet/wallet-toolbox/src/signer/actionBatch/ActionBatchPlanner.ts +++ b/packages/wallet/wallet-toolbox/src/signer/actionBatch/ActionBatchPlanner.ts @@ -6,7 +6,7 @@ import { StorageCreateTransactionSdkOutput, StorageProvidedBy } from '../../sdk/WalletStorage.interfaces' -import { WERR_INTERNAL, WERR_INVALID_PARAMETER } from '../../sdk/WERR_errors' +import { WERR_INSUFFICIENT_FUNDS, WERR_INTERNAL, WERR_INVALID_PARAMETER } from '../../sdk/WERR_errors' import { randomBytesBase64, verifyTruthy } from '../../utility/utilityHelpers' import { asArray, asString } from '../../utility/utilityHelpers.noBuffer' import { beefForTxids } from '../../utility/beefForTxids' @@ -17,6 +17,7 @@ import { maxPossibleSatoshis } from '../../storage/methods/generateChange' import { randomizeOutputVouts, repeatableRandom, selectCanonicalChange } from '../../storage/methods/actionPlanning' +import { validateManagedChangePolicy } from '../../storage/methods/managedChangePolicy' export interface ActionBatchPlannedAction { dcr: StorageCreateActionResult @@ -224,6 +225,8 @@ async function planFunding( const allocated = new Map() const noSend = [...noSendChange] const changeBasket = state.begin.changeBasket + const policy = validateManagedChangePolicy(state.begin.managedChangePolicy) + const preferredSatoshis = Math.max(1, changeBasket.minimumDesiredUTXOValue) const params = { fixedInputs: explicit.map((output, index) => ({ satoshis: output.satoshis, @@ -239,11 +242,14 @@ async function planFunding( : []) ], feeModel: state.begin.feeModel, - changeInitialSatoshis: Math.max(1, changeBasket.minimumDesiredUTXOValue), - changeFirstSatoshis: Math.max(1, Math.round(changeBasket.minimumDesiredUTXOValue / 4)), + changeInitialSatoshis: preferredSatoshis, + changeFirstSatoshis: preferredSatoshis, changeLockingScriptLength: 25, changeUnlockingScriptLength: 107, targetNetCount: changeBasket.numberOfDesiredUTXOs - state.estimatedChangeCount, + maxChangeOutputs: policy.maxOutputsPerAction, + surplusPoolShaping: true, + maxMigrationInputs: policy.migrationInputsPerAction, randomVals: args.randomVals } const allocate = async ( @@ -266,7 +272,26 @@ async function planFunding( allocated.delete(outputId) if (noSendChange.includes(output)) noSend.push(output) } - const result = await generateChangeSdk(params, allocate, release, args.logger) + let result + try { + result = await generateChangeSdk(params, allocate, release, args.logger) + } catch (error) { + if (!(error instanceof WERR_INSUFFICIENT_FUNDS)) throw error + // Match legacy createAction's one-way availability guarantee. A workspace + // must not fail solely because the preferred pool shape cannot be made + // from its already reserved inputs. + result = await generateChangeSdk( + { + ...params, + changeFirstSatoshis: 1, + surplusPoolShaping: false, + maxMigrationInputs: 0 + }, + allocate, + release, + args.logger + ) + } return { allocated: result.allocatedChangeInputs.map(input => verifyTruthy(allocated.get(input.outputId))), changeSatoshis: result.changeOutputs.map(output => output.satoshis), diff --git a/packages/wallet/wallet-toolbox/src/storage/StorageIdb.ts b/packages/wallet/wallet-toolbox/src/storage/StorageIdb.ts index 248cfa846..f6e2fba8f 100644 --- a/packages/wallet/wallet-toolbox/src/storage/StorageIdb.ts +++ b/packages/wallet/wallet-toolbox/src/storage/StorageIdb.ts @@ -74,6 +74,10 @@ import { WERR_INTERNAL, WERR_INVALID_OPERATION, WERR_INVALID_PARAMETER, WERR_UNA import { EntityTimeStamp, TransactionStatus } from '../sdk/types' import { isAutoSpendableChangeOutput, managedChangeOutputFields } from './methods/managedChange' import { selectCanonicalChange } from './methods/actionPlanning' +import { + DEFAULT_MANAGED_CHANGE_MINIMUM_SATOSHIS, + isLegacyManagedChangeBasketDefault +} from './methods/managedChangePolicy' export interface StorageIdbOptions extends StorageProviderOptions {} @@ -121,6 +125,7 @@ async function scanCursor( export class StorageIdb extends StorageProvider implements WalletStorageProvider { dbName: string db?: IDBPDatabase + private managedChangeDefaultsMigrated = false constructor(options: StorageIdbOptions) { super(options) @@ -164,6 +169,7 @@ export class StorageIdb extends StorageProvider implements WalletStorageProvider async verifyDB(storageName?: string, storageIdentityKey?: string): Promise> { if (this.db != null) return this.db this.db = await this.initDB(storageName, storageIdentityKey) + await this.migrateManagedChangeDefaults(this.db) this._settings = (await this.db.getAll('settings'))[0] this.whenLastAccess = new Date() return this.db @@ -207,7 +213,7 @@ export class StorageIdb extends StorageProvider implements WalletStorageProvider async initDB(storageName?: string, storageIdentityKey?: string): Promise> { const chain = this.chain const maxOutputScript = 1024 - const db = await openDB(this.dbName, 3, { + const db = await openDB(this.dbName, 4, { upgrade(db, _oldVersion, _newVersion, transaction) { upgradeAllStoresV1(db) upgradeActionBatchStoresV2(db) @@ -239,6 +245,25 @@ export class StorageIdb extends StorageProvider implements WalletStorageProvider return db } + private async migrateManagedChangeDefaults (db: IDBPDatabase): Promise { + if (this.managedChangeDefaultsMigrated) return + const trx = db.transaction('output_baskets', 'readwrite') + let cursor = await trx.store.openCursor() + while (cursor != null) { + const basket = cursor.value + if (isLegacyManagedChangeBasketDefault(basket)) { + await cursor.update({ + ...basket, + minimumDesiredUTXOValue: DEFAULT_MANAGED_CHANGE_MINIMUM_SATOSHIS, + updated_at: new Date() + }) + } + cursor = await cursor.continue() + } + await trx.done + this.managedChangeDefaultsMigrated = true + } + // // StorageProvider abstract methods // @@ -1225,6 +1250,7 @@ export class StorageIdb extends StorageProvider implements WalletStorageProvider } this.db = undefined this._settings = undefined + this.managedChangeDefaultsMigrated = false } allStores: string[] = [ diff --git a/packages/wallet/wallet-toolbox/src/storage/StorageKnex.ts b/packages/wallet/wallet-toolbox/src/storage/StorageKnex.ts index 0715a817f..cab11752d 100644 --- a/packages/wallet/wallet-toolbox/src/storage/StorageKnex.ts +++ b/packages/wallet/wallet-toolbox/src/storage/StorageKnex.ts @@ -1556,7 +1556,14 @@ export class StorageKnex extends StorageProvider implements WalletStorageProvide .where('ab.hardExpiresAt', '>', now) }) .whereIn('t.status', statuses) - .select('o.outputId', 'o.transactionId', 'o.satoshis', 'o.txid', 'o.vout') + .select( + 'o.outputId', + 'o.transactionId', + 'o.satoshis', + 'o.txid', + 'o.vout', + 't.status as transactionStatus' + ) } override async findOutputsByIds (outputIds: number[], trx?: TrxToken): Promise> { diff --git a/packages/wallet/wallet-toolbox/src/storage/StorageProvider.ts b/packages/wallet/wallet-toolbox/src/storage/StorageProvider.ts index a57106b83..bb08078aa 100644 --- a/packages/wallet/wallet-toolbox/src/storage/StorageProvider.ts +++ b/packages/wallet/wallet-toolbox/src/storage/StorageProvider.ts @@ -112,6 +112,12 @@ import { putActionBatchPack as putBatchPack } from './methods/actionBatchBlobs' import { availableManagedChange, ManagedChangeInputCandidate } from './methods/availableManagedChange' +import { + ManagedChangePolicy, + ManagedChangePolicyOptions, + defaultManagedChangePolicy, + validateManagedChangePolicy +} from './methods/managedChangePolicy' export abstract class StorageProvider extends StorageReaderWriter implements WalletStorageProvider { isDirty = false @@ -121,6 +127,7 @@ export abstract class StorageProvider extends StorageReaderWriter implements Wal commissionPubKeyHex?: PubKeyHex maxRecursionDepth?: number readonly actionBatchMaxReservedOutputs: number + readonly managedChangePolicy: ManagedChangePolicy readonly scriptVerifier?: SpendVerifierInterface static defaultOptions(): { @@ -128,12 +135,14 @@ export abstract class StorageProvider extends StorageReaderWriter implements Wal commissionSatoshis: number commissionPubKeyHex: undefined actionBatchMaxReservedOutputs: number + managedChangePolicy: ManagedChangePolicy } { const opts = { feeModel: { model: 'sat/kb' as const, value: 100 }, commissionSatoshis: 0, commissionPubKeyHex: undefined, - actionBatchMaxReservedOutputs: 256 + actionBatchMaxReservedOutputs: 256, + managedChangePolicy: defaultManagedChangePolicy() } return opts } @@ -161,6 +170,7 @@ export abstract class StorageProvider extends StorageReaderWriter implements Wal ) } this.actionBatchMaxReservedOutputs = maxReservedOutputs + this.managedChangePolicy = validateManagedChangePolicy(options.managedChangePolicy) this.scriptVerifier = options.scriptVerifier } @@ -215,13 +225,21 @@ export abstract class StorageProvider extends StorageReaderWriter implements Wal excludeSending: boolean, trx?: TrxToken ): Promise { - return (await this.findAvailableManagedChangeInputs(userId, basketId, excludeSending, trx)) - .map(({ outputId, transactionId, satoshis, txid, vout }) => ({ + const outputs = await this.findAvailableManagedChangeInputs(userId, basketId, excludeSending, trx) + // Keep prototype-level/custom provider fallbacks working even when a + // minimal older implementation has not exposed the additive batch status + // lookup. createAction resolves missing metadata before tier planning. + const findStatuses = this.findTransactionStatusesByIds?.bind(this) + const statuses = findStatuses == null + ? new Map() + : await findStatuses(userId, outputs.map(output => output.transactionId), trx) + return outputs.map(({ outputId, transactionId, satoshis, txid, vout }) => ({ outputId, transactionId, satoshis, txid, - vout + vout, + transactionStatus: statuses.get(transactionId) })) } @@ -1631,6 +1649,12 @@ export interface StorageProviderOptions extends StorageReaderWriterOptions { * Defaults to 256; -1 disables this cumulative provider limit. */ actionBatchMaxReservedOutputs?: number + /** + * Optional wallet-managed liquidity tuning. Values are soft shaping and + * comparison budgets; none can prevent an otherwise fundable action. Each + * limit accepts -1 for an explicit operator-selected unlimited mode. + */ + managedChangePolicy?: ManagedChangePolicyOptions } export function validateStorageFeeModel(v?: StorageFeeModel): StorageFeeModel { diff --git a/packages/wallet/wallet-toolbox/src/storage/StorageReaderWriter.ts b/packages/wallet/wallet-toolbox/src/storage/StorageReaderWriter.ts index 46ddbd009..f780351a1 100644 --- a/packages/wallet/wallet-toolbox/src/storage/StorageReaderWriter.ts +++ b/packages/wallet/wallet-toolbox/src/storage/StorageReaderWriter.ts @@ -28,6 +28,10 @@ import { TrxToken } from '../sdk/WalletStorage.interfaces' import { createSyncMap } from './schema/entities/EntityBase' +import { + DEFAULT_MANAGED_CHANGE_MINIMUM_SATOSHIS, + DEFAULT_MANAGED_CHANGE_TARGET_UTXOS +} from './methods/managedChangePolicy' export abstract class StorageReaderWriter extends StorageReader { abstract dropAllData (): Promise @@ -173,8 +177,8 @@ export abstract class StorageReaderWriter extends StorageReader { basketId: 0, userId: user.userId, name: 'default', - numberOfDesiredUTXOs: 144, - minimumDesiredUTXOValue: 32, + numberOfDesiredUTXOs: DEFAULT_MANAGED_CHANGE_TARGET_UTXOS, + minimumDesiredUTXOValue: DEFAULT_MANAGED_CHANGE_MINIMUM_SATOSHIS, isDeleted: false }) break diff --git a/packages/wallet/wallet-toolbox/src/storage/__test/StorageIdb.test.ts b/packages/wallet/wallet-toolbox/src/storage/__test/StorageIdb.test.ts index 2f3bfc6ff..c4aaa4847 100644 --- a/packages/wallet/wallet-toolbox/src/storage/__test/StorageIdb.test.ts +++ b/packages/wallet/wallet-toolbox/src/storage/__test/StorageIdb.test.ts @@ -27,7 +27,7 @@ describe('StorageIdb tests', () => { try { const r = await storage.migrate(`storageIdbTest-${Date.now()}`, '42'.repeat(32)) const db = storage.db - expect(r).toBe('3') + expect(r).toBe('4') expect(db).toBeTruthy() expect(db?.transaction('outputs').objectStore('outputs').indexNames.contains('userId_basketId')).toBe(true) expect(db?.transaction('outputs').objectStore('outputs').indexNames.contains('txid_vout_userId')).toBe(true) @@ -56,7 +56,7 @@ describe('StorageIdb tests', () => { try { const upgraded = await storage.initDB('version 2 upgrade test', '42'.repeat(32)) - expect(upgraded.version).toBe(3) + expect(upgraded.version).toBe(4) expect(upgraded.transaction('outputs').objectStore('outputs') .indexNames.contains('userId_basketId')).toBe(true) expect(upgraded.transaction('outputs').objectStore('outputs') @@ -108,6 +108,44 @@ describe('StorageIdb tests', () => { } }) + test('migrates only untouched legacy managed-change defaults', async () => { + const options: StorageProviderOptions = StorageProvider.createStorageBaseOptions('main') + const storage = new StorageIdb(options) + storage.dbName = `storageIdbManagedChangeUpgrade-${randomUUID()}` + await storage.migrate('managed change migration', '42'.repeat(32)) + const userId = await insertUser(storage) + const customMinimumUserId = await insertUser(storage, '03'.repeat(33)) + const customTargetUserId = await insertUser(storage, '04'.repeat(33)) + const now = new Date() + const insert = async (basketUserId: number, name: string, target: number, minimum: number): Promise => await storage.insertOutputBasket({ + basketId: 0, + userId: basketUserId, + name, + numberOfDesiredUTXOs: target, + minimumDesiredUTXOValue: minimum, + isDeleted: false, + created_at: now, + updated_at: now + }) + const untouchedId = await insert(userId, 'default', 144, 32) + const customizedMinimumId = await insert(customMinimumUserId, 'default', 144, 64) + const customizedTargetId = await insert(customTargetUserId, 'default', 100, 32) + await storage.destroy() + + const reopened = new StorageIdb(options) + reopened.dbName = storage.dbName + try { + await reopened.migrate('managed change migration', '42'.repeat(32)) + const baskets = await reopened.findOutputBaskets({ partial: {} }) + const byId = new Map(baskets.map(basket => [basket.basketId, basket])) + expect(byId.get(untouchedId)?.minimumDesiredUTXOValue).toBe(5_000) + expect(byId.get(customizedMinimumId)?.minimumDesiredUTXOValue).toBe(64) + expect(byId.get(customizedTargetId)?.minimumDesiredUTXOValue).toBe(32) + } finally { + await resetStorage(reopened) + } + }) + test('batches user-scoped transaction statuses and indexed outpoint reads', async () => { const storage = await makeStorage() try { diff --git a/packages/wallet/wallet-toolbox/src/storage/__test/TaskReviewUtxosLiquidity.test.ts b/packages/wallet/wallet-toolbox/src/storage/__test/TaskReviewUtxosLiquidity.test.ts new file mode 100644 index 000000000..79f63b2dc --- /dev/null +++ b/packages/wallet/wallet-toolbox/src/storage/__test/TaskReviewUtxosLiquidity.test.ts @@ -0,0 +1,61 @@ +import { TaskReviewUtxos } from '../../monitor/tasks/TaskReviewUtxos' +import { managedChangeOutputFields } from '../methods/managedChange' + +describe('TaskReviewUtxos managed-change liquidity report', () => { + test('reports value, reservation, and parent-status health without mutation', async () => { + const now = new Date() + const output = (outputId: number, transactionId: number, satoshis: number): any => ({ + outputId, + transactionId, + userId: 1, + basketId: 7, + satoshis, + txid: outputId.toString(16).padStart(64, '0'), + vout: 0, + spendable: true, + spentBy: undefined, + derivationPrefix: 'prefix', + derivationSuffix: `suffix-${outputId}`, + created_at: now, + updated_at: now, + ...managedChangeOutputFields + }) + const outputs = [output(11, 101, 5_000), output(12, 102, 4_999), output(13, 103, 8_000)] + const provider = { + findUsers: jest.fn().mockResolvedValue([{ + userId: 1, + identityKey: 'key-1', + activeStorage: 'storage-key', + created_at: now, + updated_at: now + }]), + findOutputBaskets: jest.fn().mockResolvedValue([{ + basketId: 7, + userId: 1, + name: 'default', + numberOfDesiredUTXOs: 144, + minimumDesiredUTXOValue: 5_000 + }]), + findOutputs: jest.fn().mockResolvedValue(outputs), + findReservedActionBatchOutputIds: jest.fn().mockResolvedValue([13]), + findTransactionStatusesByIds: jest.fn().mockResolvedValue(new Map([ + [101, 'completed'], + [102, 'unproven'], + [103, 'sending'] + ])) + } + const runAsStorageProvider = jest.fn(async (fn: any) => await fn(provider)) + const task = new TaskReviewUtxos({ storage: { runAsStorageProvider } } as any) + + const log = await task.reviewManagedChangeByIdentityKey('key-1') + + expect(provider.findOutputs).toHaveBeenCalledWith(expect.objectContaining({ + txStatus: ['completed', 'unproven', 'sending'], + noScript: true + })) + expect(log).toBe( + 'userId 1: managed change 3/144, healthy 2, undersized 1, reserved 1, ' + + 'completed 1, unproven 1, sending 1, satoshis 17999, preferred minimum 5000\n' + ) + }) +}) diff --git a/packages/wallet/wallet-toolbox/src/storage/__test/actionBatch.test.ts b/packages/wallet/wallet-toolbox/src/storage/__test/actionBatch.test.ts index c1954984f..f0b00711b 100644 --- a/packages/wallet/wallet-toolbox/src/storage/__test/actionBatch.test.ts +++ b/packages/wallet/wallet-toolbox/src/storage/__test/actionBatch.test.ts @@ -101,6 +101,58 @@ describe('action batch reservations', () => { expect(getActionBatchCapabilities(32, { resume: true }).actionBatch?.resume).toBe(true) }) + test('begin advertises the effective managed-change shaping policy to the workspace planner', async () => { + ctx.activeStorage.managedChangePolicy.maxOutputsPerAction = 2 + ctx.activeStorage.managedChangePolicy.migrationInputsPerAction = 1 + ctx.activeStorage.managedChangePolicy.pendingComparisonInputs = -1 + + const begun = await ctx.storage.beginActionBatch({ + batchId: 'managed-change-policy', + firstAction: firstAction() + }) + + expect(begun.managedChangePolicy).toEqual({ + maxOutputsPerAction: 2, + migrationInputsPerAction: 1 + }) + await ctx.storage.abortActionBatch(begun.batchId) + }) + + test('reservations prefer completed ancestry but retain sending liquidity as a last resort', async () => { + const basket = (await ctx.activeStorage.findOutputBaskets({ + partial: { userId: ctx.userId, name: 'default' } + }))[0] + const available = (await ctx.activeStorage.findAvailableManagedChangeInputs(ctx.userId, basket.basketId, false)) + .sort((a, b) => b.satoshis - a.satoshis) + const completed = available[0] + const sending = available.find(output => output.transactionId !== completed.transactionId) + expect(completed).toBeDefined() + expect(sending).toBeDefined() + for (const output of available) { + if (output.outputId !== completed.outputId && output.outputId !== sending!.outputId) { + await ctx.activeStorage.updateOutput(output.outputId, { spendable: false }) + } + } + await ctx.activeStorage.updateTransaction(completed.transactionId, { status: 'completed' }) + await ctx.activeStorage.updateTransaction(sending!.transactionId, { status: 'sending' }) + setReservationLimit(ctx, 1) + + const preferred = await ctx.storage.beginActionBatch({ + batchId: 'completed-reservation-preference', + firstAction: firstAction() + }) + expect(preferred.reservedOutputs.map(output => output.outputId)).toEqual([completed.outputId]) + await ctx.storage.abortActionBatch(preferred.batchId) + + await ctx.activeStorage.updateOutput(completed.outputId, { spendable: false }) + const fallback = await ctx.storage.beginActionBatch({ + batchId: 'sending-reservation-fallback', + firstAction: firstAction() + }) + expect(fallback.reservedOutputs.map(output => output.outputId)).toEqual([sending!.outputId]) + await ctx.storage.abortActionBatch(fallback.batchId) + }) + test.each([ { limit: -1, expectedMaximum: 8 }, { limit: 1, expectedMaximum: 1 }, diff --git a/packages/wallet/wallet-toolbox/src/storage/__test/createActionPerformance.test.ts b/packages/wallet/wallet-toolbox/src/storage/__test/createActionPerformance.test.ts index fc60edb2a..af6a9a53f 100644 --- a/packages/wallet/wallet-toolbox/src/storage/__test/createActionPerformance.test.ts +++ b/packages/wallet/wallet-toolbox/src/storage/__test/createActionPerformance.test.ts @@ -110,6 +110,114 @@ describe('createAction funding performance', () => { expect(result.inputs[0].sourceTxid).toBe(candidate.txid) }) + test('prefers settled liquidity even when a pending output is a closer fit', async () => { + const candidates = await replaceFundingCandidatesAcrossSources([ + { satoshis: 7_000, status: 'completed' }, + { satoshis: 5_000, status: 'sending' } + ]) + + const result = await ctx.activeStorage.createAction( + { userId: ctx.userId }, + actionArgs(1_000) + ) + + expect(result.inputs.map(input => input.sourceTxid)).toContain(candidates[0].txid) + expect(result.inputs.map(input => input.sourceTxid)).not.toContain(candidates[1].txid) + }) + + test('uses pending liquidity only after settled and unproven tiers cannot fund the action', async () => { + const candidates = await replaceFundingCandidatesAcrossSources([ + { satoshis: 500, status: 'completed' }, + { satoshis: 10_000, status: 'sending' } + ]) + + const result = await ctx.activeStorage.createAction( + { userId: ctx.userId }, + actionArgs(5_000) + ) + + expect(result.inputs.map(input => input.sourceTxid)).toContain(candidates[1].txid) + }) + + test('resolves ancestry for an older custom provider that omits additive status metadata', async () => { + const candidates = await replaceFundingCandidatesAcrossSources([ + { satoshis: 7_000, status: 'completed' }, + { satoshis: 5_000, status: 'sending' } + ]) + const original = ctx.activeStorage.findAvailableManagedChangeInputCandidates.bind(ctx.activeStorage) + jest.spyOn(ctx.activeStorage, 'findAvailableManagedChangeInputCandidates') + .mockImplementation(async (...args) => (await original(...args)).map(candidate => { + const { transactionStatus: _transactionStatus, ...legacyCandidate } = candidate + return legacyCandidate + })) + const statusLookup = jest.spyOn(ctx.activeStorage, 'findTransactionStatusesByIds') + + const result = await ctx.activeStorage.createAction( + { userId: ctx.userId }, + actionArgs(1_000) + ) + + expect(statusLookup).toHaveBeenCalled() + expect(result.inputs.map(input => input.sourceTxid)).toContain(candidates[0].txid) + expect(result.inputs.map(input => input.sourceTxid)).not.toContain(candidates[1].txid) + }) + + test('compares exact serialized cost before choosing pending liquidity for a pathological settled plan', async () => { + const candidates = await replaceFundingCandidatesAcrossSources([ + { satoshis: 4_000, status: 'completed' }, + { satoshis: 4_000, status: 'completed', source: 0 }, + { satoshis: 11_000, status: 'sending', source: 1 } + ]) + ctx.activeStorage.managedChangePolicy.pendingComparisonInputs = 1 + + const result = await ctx.activeStorage.createAction( + { userId: ctx.userId }, + actionArgs(5_000) + ) + + expect(result.inputs).toHaveLength(1) + expect(result.inputs[0].sourceTxid).toBe(candidates[2].txid) + }) + + test('operator can disable pending comparison without disabling last-resort pending funding', async () => { + const candidates = await replaceFundingCandidatesAcrossSources([ + { satoshis: 4_000, status: 'completed' }, + { satoshis: 4_000, status: 'completed', source: 0 }, + { satoshis: 11_000, status: 'sending', source: 1 } + ]) + ctx.activeStorage.managedChangePolicy.pendingComparisonInputs = -1 + const basket = (await ctx.activeStorage.findOutputBaskets({ + partial: { userId: ctx.userId, name: 'default' } + }))[0] + const available = await ctx.activeStorage.findAvailableManagedChangeInputCandidates( + ctx.userId, + basket.basketId, + false + ) + expect(available.map(candidate => candidate.transactionStatus)).toEqual([ + 'completed', + 'completed', + 'sending' + ]) + + const settled = await ctx.activeStorage.createAction( + { userId: ctx.userId }, + actionArgs(5_000) + ) + expect(settled.inputs).toHaveLength(2) + expect(settled.inputs.some(input => input.sourceTxid === candidates[2].txid)).toBe(false) + + const fallbackCandidates = await replaceFundingCandidatesAcrossSources([ + { satoshis: 500, status: 'completed' }, + { satoshis: 10_000, status: 'sending' } + ]) + const fallback = await ctx.activeStorage.createAction( + { userId: ctx.userId }, + actionArgs(5_000) + ) + expect(fallback.inputs.some(input => input.sourceTxid === fallbackCandidates[1].txid)).toBe(true) + }) + test('reports funding and BEEF phases with bounded cardinality attributes', async () => { await replaceFundingCandidates(1, 5_000) const events: TelemetryEvent[] = [] @@ -135,7 +243,10 @@ describe('createAction funding performance', () => { }) expect(byName.get('wallet.storage.create_action.funding_candidates')?.attributes).toMatchObject({ 'funding.candidate_count': 1, - 'funding.candidate_satoshis': 5_000 + 'funding.candidate_satoshis': 5_000, + 'funding.completed_candidate_count': 1, + 'funding.unproven_candidate_count': 0, + 'funding.sending_candidate_count': 0 }) expect(byName.get('wallet.storage.create_action.funding_claim')?.attributes).toMatchObject({ 'funding.claim_retry_count': 0, @@ -263,4 +374,61 @@ describe('createAction funding performance', () => { await ctx.activeStorage.insertOutput(output) } } + + async function replaceFundingCandidatesAcrossSources ( + specs: Array<{ satoshis: number, status: 'completed' | 'unproven' | 'sending', source?: number }> + ): Promise> { + const basket = (await ctx.activeStorage.findOutputBaskets({ + partial: { userId: ctx.userId, name: 'default' } + }))[0] as TableOutputBasket + await ctx.activeStorage.updateOutputBasket(basket.basketId, { + numberOfDesiredUTXOs: 0, + minimumDesiredUTXOValue: 5_000 + }) + const existing = await ctx.activeStorage.findOutputs({ + partial: { userId: ctx.userId, basketId: basket.basketId }, + noScript: true + }) + for (const output of existing) { + if (output.spendable) await ctx.activeStorage.updateOutput(output.outputId, { spendable: false }) + } + const sources = await ctx.activeStorage.findTransactions({ + partial: { userId: ctx.userId }, + status: ['completed'], + noRawTx: true + }) as TableTransaction[] + expect(sources.length).toBeGreaterThanOrEqual(2) + const lockingScript = [0x76, 0xa9, 0x14, ...Array(20).fill(0x11), 0x88, 0xac] + const selectedSources = specs.map((spec, index) => sources[spec.source ?? index]) + for (let index = 0; index < specs.length; index++) { + const spec = specs[index] + const source = selectedSources[index] + const now = new Date() + await ctx.activeStorage.insertOutput({ + outputId: 0, + userId: ctx.userId, + transactionId: source.transactionId, + basketId: basket.basketId, + spendable: true, + spentBy: undefined, + satoshis: spec.satoshis, + vout: 30_000 + existing.length + index, + txid: source.txid, + lockingScript, + scriptLength: lockingScript.length, + derivationPrefix: 'funding-policy-prefix', + derivationSuffix: `funding-policy-${index}`, + outputDescription: 'funding policy candidate', + ...managedChangeOutputFields, + created_at: now, + updated_at: now + }) + } + for (const [transactionId, status] of new Map( + specs.map((spec, index) => [selectedSources[index].transactionId, spec.status]) + )) { + await ctx.activeStorage.updateTransaction(transactionId, { status }) + } + return selectedSources.map(source => ({ txid: source.txid!, transactionId: source.transactionId })) + } }) diff --git a/packages/wallet/wallet-toolbox/src/storage/adminServer/__tests/adminFormatting.test.ts b/packages/wallet/wallet-toolbox/src/storage/adminServer/__tests/adminFormatting.test.ts index 3c6a87421..cc6f4993e 100644 --- a/packages/wallet/wallet-toolbox/src/storage/adminServer/__tests/adminFormatting.test.ts +++ b/packages/wallet/wallet-toolbox/src/storage/adminServer/__tests/adminFormatting.test.ts @@ -1,4 +1,5 @@ -import { alignLeft, alignRight, asNumber, toAdminStatsLog } from '../adminServer' +import { alignLeft, alignRight, asNumber, normalizeReviewMode, toAdminStatsLog } from '../adminServer' +import { renderAdminPage } from '../adminUi' describe('storage admin diagnostic formatting', () => { test('normalizes numeric and structured values without default object coercion', () => { @@ -22,4 +23,11 @@ describe('storage admin diagnostic formatting', () => { expect(log).toContain('{"service":"monitor"}') expect(log).toContain('users') }) + + test('exposes a read-only managed-change liquidity review mode', () => { + expect(normalizeReviewMode('liquidity')).toBe('liquidity') + expect(normalizeReviewMode('change')).toBe('change') + expect(normalizeReviewMode('unknown')).toBe('all') + expect(renderAdminPage()).toContain('') + }) }) diff --git a/packages/wallet/wallet-toolbox/src/storage/adminServer/adminServer.ts b/packages/wallet/wallet-toolbox/src/storage/adminServer/adminServer.ts index 456d48cf9..66b61e5ad 100644 --- a/packages/wallet/wallet-toolbox/src/storage/adminServer/adminServer.ts +++ b/packages/wallet/wallet-toolbox/src/storage/adminServer/adminServer.ts @@ -322,18 +322,25 @@ async function queryReqReview (context: MonitorAdminContext, query: Record Promise + reviewManagedChangeByIdentityKey?: (identityKey: string) => Promise } { const monitor = context.daemon.setup?.monitor if (monitor == null) throw new Error('Monitor is not available.') const task = [...monitor._tasks, ...monitor._otherTasks].find(item => item.name === 'ReviewUtxos') as - | { reviewByIdentityKey?: (identityKey: string, mode: 'all' | 'change') => Promise } + | { + reviewByIdentityKey?: (identityKey: string, mode: 'all' | 'change') => Promise + reviewManagedChangeByIdentityKey?: (identityKey: string) => Promise + } | undefined if ((task?.reviewByIdentityKey) == null) { @@ -341,7 +348,8 @@ function getReviewUtxosTask (context: MonitorAdminContext): { } return { - reviewByIdentityKey: task.reviewByIdentityKey.bind(task) + reviewByIdentityKey: task.reviewByIdentityKey.bind(task), + reviewManagedChangeByIdentityKey: task.reviewManagedChangeByIdentityKey?.bind(task) } } @@ -422,12 +430,15 @@ async function reviewUtxosByIdentityKey ( context: MonitorAdminContext, requestedBy: string, userInput: string, - mode: 'all' | 'change' + mode: UtxoReviewMode ) { const storage = await getStorage(context) const task = getReviewUtxosTask(context) const identityKey = await resolveIdentityKeyFromInput(context, userInput) - const log = await task.reviewByIdentityKey(identityKey, mode) + const log = mode === 'liquidity' + ? await task.reviewManagedChangeByIdentityKey?.(identityKey) + : await task.reviewByIdentityKey(identityKey, mode) + if (log == null) throw new Error('Managed-change liquidity review is not available in this monitor runtime.') await storage.insertMonitorEvent({ created_at: new Date(), diff --git a/packages/wallet/wallet-toolbox/src/storage/adminServer/adminUi.ts b/packages/wallet/wallet-toolbox/src/storage/adminServer/adminUi.ts index 03652d373..b5cf57ea1 100644 --- a/packages/wallet/wallet-toolbox/src/storage/adminServer/adminUi.ts +++ b/packages/wallet/wallet-toolbox/src/storage/adminServer/adminUi.ts @@ -1,4 +1,4 @@ -export function renderAdminPage (): string { +export function renderAdminPage(): string { return ` @@ -388,6 +388,7 @@ export function renderAdminPage (): string { @@ -892,7 +893,8 @@ export function renderAdminPage (): string { throw new Error('Enter or select an identityKey first.') } byId('utxoIdentityKey').value = identityKey - const mode = byId('utxoMode').value === 'change' ? 'change' : 'all' + const selectedMode = byId('utxoMode').value + const mode = selectedMode === 'change' || selectedMode === 'liquidity' ? selectedMode : 'all' setButtonPending('runUtxoReview', true, 'Running...') byId('utxoReviewLog').textContent = 'Running review...' try { diff --git a/packages/wallet/wallet-toolbox/src/storage/index.all.ts b/packages/wallet/wallet-toolbox/src/storage/index.all.ts index f9897e7b1..a7c272714 100644 --- a/packages/wallet/wallet-toolbox/src/storage/index.all.ts +++ b/packages/wallet/wallet-toolbox/src/storage/index.all.ts @@ -12,6 +12,7 @@ export * from './adminServer/index.all' export * from './methods/ListActionsSpecOp' export * from './methods/ListOutputsSpecOp' export * from './methods/managedChange' +export * from './methods/managedChangePolicy' export * from './schema/tables/index' export * from './schema/entities/index' export * as sync from './sync' diff --git a/packages/wallet/wallet-toolbox/src/storage/index.client.ts b/packages/wallet/wallet-toolbox/src/storage/index.client.ts index 728d83140..89419312b 100644 --- a/packages/wallet/wallet-toolbox/src/storage/index.client.ts +++ b/packages/wallet/wallet-toolbox/src/storage/index.client.ts @@ -9,3 +9,4 @@ export * from './portable' export * from './methods/ListActionsSpecOp' export * from './methods/ListOutputsSpecOp' export * from './methods/managedChange' +export * from './methods/managedChangePolicy' diff --git a/packages/wallet/wallet-toolbox/src/storage/index.mobile.ts b/packages/wallet/wallet-toolbox/src/storage/index.mobile.ts index 77b6f91db..0a709e46a 100644 --- a/packages/wallet/wallet-toolbox/src/storage/index.mobile.ts +++ b/packages/wallet/wallet-toolbox/src/storage/index.mobile.ts @@ -7,3 +7,4 @@ export * from './remoting/StorageMobile' export * from './portable' export * from './methods/ListActionsSpecOp' export * from './methods/ListOutputsSpecOp' +export * from './methods/managedChangePolicy' diff --git a/packages/wallet/wallet-toolbox/src/storage/methods/__test/GenerateChange/generateChangeSdk.test.ts b/packages/wallet/wallet-toolbox/src/storage/methods/__test/GenerateChange/generateChangeSdk.test.ts index 9917824b4..11bb60903 100644 --- a/packages/wallet/wallet-toolbox/src/storage/methods/__test/GenerateChange/generateChangeSdk.test.ts +++ b/packages/wallet/wallet-toolbox/src/storage/methods/__test/GenerateChange/generateChangeSdk.test.ts @@ -1216,6 +1216,169 @@ describe('generateChange tests', () => { expectTransactionSize(params, r) }) + test('10c surplus shaping never gathers inputs merely to reach the pool target', async () => { + const params: GenerateChangeSdkParams = { + ...defParams, + fixedOutputs: [{ satoshis: 1_000, lockingScriptLength: 25 }], + changeFirstSatoshis: 5_000, + changeInitialSatoshis: 5_000, + targetNetCount: 144, + maxChangeOutputs: 8, + surplusPoolShaping: true, + maxMigrationInputs: 0 + } + const availableChange: GenerateChangeSdkChangeInput[] = [ + { satoshis: 10_000, outputId: 1 }, + { satoshis: 10_000, outputId: 2 } + ] + const { allocateChangeInput, releaseChangeInput } = generateChangeSdkMakeStorage(availableChange) + + const r = await generateChangeSdk(params, allocateChangeInput, releaseChangeInput) + + expect(r.allocatedChangeInputs.map(input => input.outputId)).toEqual([1]) + expect(r.changeOutputs).toHaveLength(1) + expect(r.changeOutputs[0].satoshis).toBeGreaterThanOrEqual(5_000) + expectTransactionSize(params, r) + }) + + test('10d surplus shaping splits large liquidity into bounded independently useful outputs', async () => { + const params: GenerateChangeSdkParams = { + ...defParams, + fixedOutputs: [{ satoshis: 1_000, lockingScriptLength: 25 }], + changeFirstSatoshis: 5_000, + changeInitialSatoshis: 5_000, + targetNetCount: 144, + maxChangeOutputs: 8, + surplusPoolShaping: true, + maxMigrationInputs: 0 + } + const { allocateChangeInput, releaseChangeInput } = generateChangeSdkMakeStorage([ + { satoshis: 100_000, outputId: 1 } + ]) + + const r = await generateChangeSdk(params, allocateChangeInput, releaseChangeInput) + + expect(r.allocatedChangeInputs).toHaveLength(1) + expect(r.changeOutputs).toHaveLength(8) + expect(r.changeOutputs.every(output => output.satoshis >= 5_000)).toBe(true) + expectTransactionSize(params, r) + }) + + test('10e a small remainder is retained when the preferred minimum cannot be met', async () => { + const params: GenerateChangeSdkParams = { + ...defParams, + fixedOutputs: [{ satoshis: 1_000, lockingScriptLength: 25 }], + changeFirstSatoshis: 5_000, + changeInitialSatoshis: 5_000, + targetNetCount: 144, + maxChangeOutputs: 8, + surplusPoolShaping: true, + maxMigrationInputs: 4 + } + const { allocateChangeInput, releaseChangeInput } = generateChangeSdkMakeStorage([{ satoshis: 2_000, outputId: 1 }]) + + const r = await generateChangeSdk(params, allocateChangeInput, releaseChangeInput) + + expect(r.allocatedChangeInputs).toHaveLength(1) + expect(r.changeOutputs).toHaveLength(1) + expect(r.changeOutputs[0].satoshis).toBeLessThan(5_000) + expectTransactionSize(params, r) + }) + + test('10f legacy fragments migrate gradually within the configured input budget', async () => { + const params: GenerateChangeSdkParams = { + ...defParams, + fixedOutputs: [{ satoshis: 9_000, lockingScriptLength: 25 }], + changeFirstSatoshis: 5_000, + changeInitialSatoshis: 5_000, + targetNetCount: 144, + maxChangeOutputs: 8, + surplusPoolShaping: true, + maxMigrationInputs: 4 + } + const availableChange: GenerateChangeSdkChangeInput[] = [ + { satoshis: 20_000, outputId: 1 }, + ...Array.from({ length: 10 }, (_, index) => ({ satoshis: 1_000, outputId: index + 2 })) + ] + const { allocateChangeInput, releaseChangeInput } = generateChangeSdkMakeStorage(availableChange) + + const r = await generateChangeSdk(params, allocateChangeInput, releaseChangeInput) + + expect(r.allocatedChangeInputs).toHaveLength(5) + expect(r.allocatedChangeInputs.filter(input => input.satoshis < 5_000)).toHaveLength(4) + expect(r.changeOutputs.every(output => output.satoshis >= 5_000)).toBe(true) + expectTransactionSize(params, r) + }) + + test('10g migration rejects fragments whose value does not cover their marginal input fee', async () => { + const params: GenerateChangeSdkParams = { + ...defParams, + fixedOutputs: [{ satoshis: 9_000, lockingScriptLength: 25 }], + feeModel: { model: 'sat/kb', value: 500 }, + changeFirstSatoshis: 5_000, + changeInitialSatoshis: 5_000, + targetNetCount: 144, + maxChangeOutputs: 8, + surplusPoolShaping: true, + maxMigrationInputs: 4 + } + const { allocateChangeInput, releaseChangeInput } = generateChangeSdkMakeStorage([ + { satoshis: 50, outputId: 2 }, + { satoshis: 20_000, outputId: 1 } + ]) + + const r = await generateChangeSdk(params, allocateChangeInput, releaseChangeInput) + + expect(r.allocatedChangeInputs.map(input => input.outputId)).toEqual([1]) + expectTransactionSize(params, r) + }) + + test('10h targetNetCount zero disables pool growth and migration', async () => { + const params: GenerateChangeSdkParams = { + ...defParams, + fixedOutputs: [{ satoshis: 9_000, lockingScriptLength: 25 }], + changeFirstSatoshis: 5_000, + changeInitialSatoshis: 5_000, + targetNetCount: 0, + maxChangeOutputs: 8, + surplusPoolShaping: true, + maxMigrationInputs: 4 + } + const { allocateChangeInput, releaseChangeInput } = generateChangeSdkMakeStorage([ + { satoshis: 1_000, outputId: 2 }, + { satoshis: 20_000, outputId: 1 } + ]) + + const r = await generateChangeSdk(params, allocateChangeInput, releaseChangeInput) + + expect(r.allocatedChangeInputs.map(input => input.outputId)).toEqual([1]) + expect(r.changeOutputs).toHaveLength(1) + expectTransactionSize(params, r) + }) + + test('10i explicit unlimited limits remain bounded by available value and inputs', async () => { + const params: GenerateChangeSdkParams = { + ...defParams, + fixedOutputs: [{ satoshis: 1_000, lockingScriptLength: 25 }], + changeFirstSatoshis: 5_000, + changeInitialSatoshis: 5_000, + targetNetCount: 3, + maxChangeOutputs: -1, + surplusPoolShaping: true, + maxMigrationInputs: -1 + } + const { allocateChangeInput, releaseChangeInput } = generateChangeSdkMakeStorage([ + { satoshis: 1_000, outputId: 2 }, + { satoshis: 30_000, outputId: 1 } + ]) + + const r = await generateChangeSdk(params, allocateChangeInput, releaseChangeInput) + + expect(r.allocatedChangeInputs).toHaveLength(2) + expect(r.changeOutputs.length - r.allocatedChangeInputs.length).toBeLessThanOrEqual(3) + expectTransactionSize(params, r) + }) + test('11 emits correlated allocation and generate-change spans without values', async () => { const events: any[] = [] const telemetry = new Telemetry({ diff --git a/packages/wallet/wallet-toolbox/src/storage/methods/__test/managedChangePolicy.test.ts b/packages/wallet/wallet-toolbox/src/storage/methods/__test/managedChangePolicy.test.ts new file mode 100644 index 000000000..1d70dba54 --- /dev/null +++ b/packages/wallet/wallet-toolbox/src/storage/methods/__test/managedChangePolicy.test.ts @@ -0,0 +1,67 @@ +import { + DEFAULT_MANAGED_CHANGE_MAX_OUTPUTS_PER_ACTION, + DEFAULT_MANAGED_CHANGE_MIGRATION_INPUTS_PER_ACTION, + DEFAULT_MANAGED_CHANGE_MINIMUM_SATOSHIS, + DEFAULT_MANAGED_CHANGE_PENDING_COMPARISON_INPUTS, + DEFAULT_MANAGED_CHANGE_TARGET_UTXOS, + defaultManagedChangePolicy, + isLegacyManagedChangeBasketDefault, + upgradeLegacyManagedChangeBasketDefault, + validateManagedChangePolicy +} from '../managedChangePolicy' + +describe('managed change policy', () => { + test('defaults provide parallel liquidity without creating dust', () => { + expect(DEFAULT_MANAGED_CHANGE_TARGET_UTXOS).toBe(144) + expect(DEFAULT_MANAGED_CHANGE_MINIMUM_SATOSHIS).toBe(5_000) + expect(defaultManagedChangePolicy()).toEqual({ + maxOutputsPerAction: DEFAULT_MANAGED_CHANGE_MAX_OUTPUTS_PER_ACTION, + migrationInputsPerAction: DEFAULT_MANAGED_CHANGE_MIGRATION_INPUTS_PER_ACTION, + pendingComparisonInputs: DEFAULT_MANAGED_CHANGE_PENDING_COMPARISON_INPUTS + }) + }) + + test('partial overrides preserve unspecified defaults', () => { + expect(validateManagedChangePolicy({ maxOutputsPerAction: 3 })).toEqual({ + ...defaultManagedChangePolicy(), + maxOutputsPerAction: 3 + }) + }) + + test('normalizes only the exact legacy basket default for migration, sync, and restore', () => { + const legacy = { name: 'default', numberOfDesiredUTXOs: 144, minimumDesiredUTXOValue: 32, marker: 'kept' } + expect(isLegacyManagedChangeBasketDefault(legacy)).toBe(true) + expect(upgradeLegacyManagedChangeBasketDefault(legacy)).toEqual({ + ...legacy, + minimumDesiredUTXOValue: 5_000 + }) + expect(upgradeLegacyManagedChangeBasketDefault({ ...legacy, minimumDesiredUTXOValue: 64 })) + .toEqual({ ...legacy, minimumDesiredUTXOValue: 64 }) + expect(upgradeLegacyManagedChangeBasketDefault({ ...legacy, numberOfDesiredUTXOs: 100 })) + .toEqual({ ...legacy, numberOfDesiredUTXOs: 100 }) + expect(upgradeLegacyManagedChangeBasketDefault({ ...legacy, name: 'application' })) + .toEqual({ ...legacy, name: 'application' }) + }) + + test('every operator limit supports explicit unlimited mode', () => { + expect(validateManagedChangePolicy({ + maxOutputsPerAction: -1, + migrationInputsPerAction: -1, + pendingComparisonInputs: -1 + })).toEqual({ + maxOutputsPerAction: -1, + migrationInputsPerAction: -1, + pendingComparisonInputs: -1 + }) + }) + + test.each([ + { maxOutputsPerAction: 0 }, + { migrationInputsPerAction: -2 }, + { pendingComparisonInputs: 0 }, + { maxOutputsPerAction: 1.5 }, + { migrationInputsPerAction: Number.MAX_SAFE_INTEGER + 1 } + ])('rejects unsafe or out-of-range policy $policy', policy => { + expect(() => validateManagedChangePolicy(policy)).toThrow('managedChangePolicy') + }) +}) diff --git a/packages/wallet/wallet-toolbox/src/storage/methods/actionBatch.ts b/packages/wallet/wallet-toolbox/src/storage/methods/actionBatch.ts index de31a27d5..2d71ba854 100644 --- a/packages/wallet/wallet-toolbox/src/storage/methods/actionBatch.ts +++ b/packages/wallet/wallet-toolbox/src/storage/methods/actionBatch.ts @@ -1,8 +1,4 @@ -import { - Beef, - Script, - Validation -} from '@bsv/sdk' +import { Beef, Script, Validation } from '@bsv/sdk' import { AbortActionBatchResult, ActionBatchCommitAction, @@ -67,15 +63,14 @@ const INITIAL_EXTRA_OUTPUTS = 3 export const ACTION_BATCH_MAX_RESERVATION_EXTENSION_OUTPUTS = 64 export const ACTION_BATCH_MAX_RESERVED_OUTPUTS = 256 -function isValidOutpoint (outpoint: { txid: string, vout: number }): boolean { +function isValidOutpoint(outpoint: { txid: string; vout: number }): boolean { // Storage keys and canonical transaction IDs are lowercase. Rejecting an // alternate spelling here is preferable to validating it and then silently // missing the case-sensitive lookup below. - return /^[0-9a-f]{64}$/.test(outpoint.txid) && - Number.isSafeInteger(outpoint.vout) && outpoint.vout >= 0 + return /^[0-9a-f]{64}$/.test(outpoint.txid) && Number.isSafeInteger(outpoint.vout) && outpoint.vout >= 0 } -export function getActionBatchCapabilities ( +export function getActionBatchCapabilities( maxReservedOutputs = ACTION_BATCH_MAX_RESERVED_OUTPUTS, supportsResume = false ): StorageCapabilities { @@ -106,13 +101,15 @@ export function getActionBatchCapabilities ( } } -function activeExpiry (batch: TableActionBatch, now = new Date()): boolean { - return batch.status !== 'active' && batch.status !== 'prepared' || +function activeExpiry(batch: TableActionBatch, now = new Date()): boolean { + return ( + (batch.status !== 'active' && batch.status !== 'prepared') || batch.expiresAt.getTime() <= now.getTime() || batch.hardExpiresAt.getTime() <= now.getTime() + ) } -function actionBatchErrorState ( +function actionBatchErrorState( batch: TableActionBatch | undefined, now = new Date() ): ActionBatchErrorState | undefined { @@ -125,12 +122,14 @@ function actionBatchErrorState ( return undefined } -function canExpire (batch: TableActionBatch, now = new Date()): boolean { - return (batch.status === 'active' || batch.status === 'prepared') && +function canExpire(batch: TableActionBatch, now = new Date()): boolean { + return ( + (batch.status === 'active' || batch.status === 'prepared') && (batch.expiresAt.getTime() <= now.getTime() || batch.hardExpiresAt.getTime() <= now.getTime()) + ) } -async function releaseBatchState ( +async function releaseBatchState( storage: StorageProvider, batch: TableActionBatch, status: 'aborted' | 'committed' | 'expired', @@ -138,14 +137,18 @@ async function releaseBatchState ( ): Promise { await storage.deleteActionBatchOutputReservations(batch.actionBatchId, trx) await storage.deleteActionBatchBlobRecords(batch.actionBatchId, trx) - await storage.updateActionBatch(batch.actionBatchId, { - status, - manifest: undefined, - uploadDigests: undefined - }, trx) + await storage.updateActionBatch( + batch.actionBatchId, + { + status, + manifest: undefined, + uploadDigests: undefined + }, + trx + ) } -export async function cleanupExpiredActionBatches (storage: StorageProvider): Promise { +export async function cleanupExpiredActionBatches(storage: StorageProvider): Promise { const now = new Date() const expired = await storage.findExpiredActionBatches(now) let released = 0 @@ -160,23 +163,29 @@ export async function cleanupExpiredActionBatches (storage: StorageProvider): Pr return released } -function sourceOutputFromBeef ( +function sourceOutputFromBeef( beef: Beef, - outpoint: { txid: string, vout: number } -): { satoshis: number, lockingScript: Script } | undefined { + outpoint: { txid: string; vout: number } +): { satoshis: number; lockingScript: Script } | undefined { const tx = beef.findTxid(outpoint.txid)?.tx const output = tx?.outputs[outpoint.vout] if (output == null) return undefined - return { satoshis: Validation.validateSatoshis(output.satoshis, 'source output satoshis'), lockingScript: output.lockingScript } + return { + satoshis: Validation.validateSatoshis(output.satoshis, 'source output satoshis'), + lockingScript: output.lockingScript + } } -async function resolveExplicitOutputs ( +async function resolveExplicitOutputs( storage: StorageProvider, userId: number, args: Validation.ValidCreateActionArgs, allowDeferredProofs: boolean -): Promise<{ outputs: TableOutput[], inputSatoshis: number }> { - const byOutpoint = await storage.findOutputsByOutpoints(userId, args.inputs.map(input => input.outpoint)) +): Promise<{ outputs: TableOutput[]; inputSatoshis: number }> { + const byOutpoint = await storage.findOutputsByOutpoints( + userId, + args.inputs.map(input => input.outpoint) + ) const beef = args.inputBEEF == null ? new Beef() : Beef.fromBinary(args.inputBEEF) const outputs: TableOutput[] = [] let inputSatoshis = 0 @@ -198,11 +207,11 @@ async function resolveExplicitOutputs ( return { outputs, inputSatoshis } } -async function resolveNoSendChangeOutputs ( +async function resolveNoSendChangeOutputs( storage: StorageProvider, userId: number, args: Validation.ValidCreateActionArgs -): Promise<{ outputs: TableOutput[], inputSatoshis: number }> { +): Promise<{ outputs: TableOutput[]; inputSatoshis: number }> { const outpoints = args.options.noSendChange const byOutpoint = await storage.findOutputsByOutpoints(userId, outpoints) const outputs: TableOutput[] = [] @@ -220,7 +229,7 @@ async function resolveNoSendChangeOutputs ( return { outputs, inputSatoshis } } -function estimateFirstActionTarget ( +function estimateFirstActionTarget( storage: StorageProvider, args: Validation.ValidCreateActionArgs, inputSatoshis: number, @@ -232,12 +241,28 @@ function estimateFirstActionTarget ( if (storage.commissionSatoshis > 0) outputLengths.push(25) outputLengths.push(25) const fee = validateStorageFeeModel(storage.feeModel).value ?? 0 - const minFee = Math.ceil(transactionSize([...inputLengths, 107], outputLengths) * fee / 1000) + const minFee = Math.ceil((transactionSize([...inputLengths, 107], outputLengths) * fee) / 1000) return Math.max(1, outputSatoshis + minFee - inputSatoshis) } -function chooseReservationPool ( - candidates: TableOutput[], +type ReservationCandidate = TableOutput & { transactionStatus: TransactionStatus } + +function selectReservationChange( + candidates: T[], + targetSatoshis: number +): T | undefined { + for (const status of ['completed', 'unproven', 'sending'] as const) { + const selected = selectCanonicalChange( + candidates.filter(candidate => candidate.transactionStatus === status), + targetSatoshis + ) + if (selected != null) return selected + } + return undefined +} + +function chooseReservationPool( + candidates: T[], targetSatoshis: number, limit: number, extras: number, @@ -246,20 +271,16 @@ function chooseReservationPool ( firstChangeCost: number marginalInputFee: number } -): TableOutput[] { - const remaining = candidates - .filter(output => output.satoshis > planningCosts.marginalInputFee) - const chosen: TableOutput[] = [] +): T[] { + const remaining = candidates.filter(output => output.satoshis > planningCosts.marginalInputFee) + const chosen: T[] = [] // A reservation is useful to generateChangeSdk only after it also covers // the marginal funding-input fee and leaves an economically viable first // change output. Without this buffer, a tiny target repeatedly selects dust // that satisfies the nominal deficit but can never close the real plan. let deficit = targetSatoshis + planningCosts.firstChangeCost while (deficit > 0 && chosen.length < limit) { - const output = selectCanonicalChange( - remaining, - deficit + planningCosts.marginalInputFee - ) + const output = selectReservationChange(remaining, deficit + planningCosts.marginalInputFee) if (output == null) break chosen.push(output) remaining.splice(remaining.indexOf(output), 1) @@ -267,7 +288,7 @@ function chooseReservationPool ( } const desiredCount = fillLimit ? limit : Math.min(limit, chosen.length + extras) while (remaining.length > 0 && chosen.length < desiredCount) { - const output = selectCanonicalChange(remaining, targetSatoshis) + const output = selectReservationChange(remaining, targetSatoshis) if (output == null) break chosen.push(output) remaining.splice(remaining.indexOf(output), 1) @@ -275,27 +296,32 @@ function chooseReservationPool ( return chosen } -function reservationPlanningCosts ( +function reservationOutput(candidate: ReservationCandidate): TableOutput { + const { transactionStatus: _transactionStatus, ...output } = candidate + return output +} + +function reservationPlanningCosts( storage: StorageProvider, - basket: TableOutputBasket -): { firstChangeCost: number, marginalInputFee: number } { + _basket: TableOutputBasket +): { firstChangeCost: number; marginalInputFee: number } { const satsPerKb = validateStorageFeeModel(storage.feeModel).value ?? 0 const minimumSpendSize = transactionSize([107], [25]) - const minimumSpendFee = Math.ceil(minimumSpendSize * satsPerKb / 1000) + const minimumSpendFee = Math.ceil((minimumSpendSize * satsPerKb) / 1000) const dustFloor = Math.max(1, minimumSpendFee * 2) const marginalInputSize = transactionSize([107], []) - transactionSize([], []) const marginalOutputSize = transactionSize([], [25]) - transactionSize([], []) - const desiredFirstChange = Math.max( - dustFloor, - Math.max(1, Math.round(basket.minimumDesiredUTXOValue / 4)) - ) + // Reserving inputs is compulsory-funding work, not pool shaping. Requiring + // the preferred liquidity value here could consume reservation capacity or + // reject a batch solely to manufacture optional change. + const desiredFirstChange = dustFloor return { - firstChangeCost: desiredFirstChange + Math.ceil(marginalOutputSize * satsPerKb / 1000), - marginalInputFee: Math.ceil(marginalInputSize * satsPerKb / 1000) + firstChangeCost: desiredFirstChange + Math.ceil((marginalOutputSize * satsPerKb) / 1000), + marginalInputFee: Math.ceil((marginalInputSize * satsPerKb) / 1000) } } -async function reserveOutputs ( +async function reserveOutputs( storage: StorageProvider, batch: TableActionBatch, outputs: TableOutput[], @@ -308,35 +334,38 @@ async function reserveOutputs ( if (output.txid == null) throw new WERR_INVALID_OPERATION('action batch output is missing its txid') return { txid: output.txid, vout: output.vout } }) - const current = await storage.findOutputsByOutpointsForUpdate( - batch.userId, - outpoints, - transaction - ) + const current = await storage.findOutputsByOutpointsForUpdate(batch.userId, outpoints, transaction) for (const output of unique) { const stored = output.txid == null ? undefined : current[`${output.txid}.${output.vout}`] if (stored == null || !stored.spendable || stored.spentBy != null) { throw new WERR_INVALID_OPERATION('one or more action batch outputs are no longer spendable') } } - const conflicts = await storage.findReservedActionBatchOutputIds(unique.map(output => output.outputId), transaction) - if (conflicts.length > 0) throw new WERR_INVALID_OPERATION('one or more action batch outputs were concurrently reserved') - await storage.reserveActionBatchOutputs(unique.map(output => ({ - actionBatchId: batch.actionBatchId, - outputId: output.outputId, - created_at: now, - updated_at: now - })), transaction) + const conflicts = await storage.findReservedActionBatchOutputIds( + unique.map(output => output.outputId), + transaction + ) + if (conflicts.length > 0) + throw new WERR_INVALID_OPERATION('one or more action batch outputs were concurrently reserved') + await storage.reserveActionBatchOutputs( + unique.map(output => ({ + actionBatchId: batch.actionBatchId, + outputId: output.outputId, + created_at: now, + updated_at: now + })), + transaction + ) } if (trx != null) await reserve(trx) else await storage.transaction(reserve) } -async function makeFundingResult ( +async function makeFundingResult( storage: StorageProvider, args: Validation.ValidCreateActionArgs, outputs: TableOutput[] -): Promise<{ outputs: ActionBatchFundingOutput[], beef?: Uint8Array }> { +): Promise<{ outputs: ActionBatchFundingOutput[]; beef?: Uint8Array }> { const beef = new Beef() const result: ActionBatchFundingOutput[] = [] for (const output of outputs) { @@ -346,17 +375,19 @@ async function makeFundingResult ( copy.sourceTransaction = await storage.getRawTxOfKnownValidTransaction(output.txid) } if (output.txid != null && beef.findTxid(output.txid) == null) { - beef.mergeBeef(await storage.getBeefForTransaction(output.txid, { - knownTxids: args.options.knownTxids, - ignoreServices: true - })) + beef.mergeBeef( + await storage.getBeefForTransaction(output.txid, { + knownTxids: args.options.knownTxids, + ignoreServices: true + }) + ) } result.push(copy) } return { outputs: result, beef: beef.toUint8Array() } } -function newBatch (userId: number, batchId: string): TableActionBatch { +function newBatch(userId: number, batchId: string): TableActionBatch { const now = new Date() return { actionBatchId: 0, @@ -370,61 +401,63 @@ function newBatch (userId: number, batchId: string): TableActionBatch { } } -export async function beginActionBatch ( +export async function beginActionBatch( storage: StorageProvider, auth: AuthId, args: BeginActionBatchArgs ): Promise { const userId = verifyId(auth.userId) await cleanupExpiredActionBatches(storage) - if (await storage.findActionBatch(userId, args.batchId) != null) { + if ((await storage.findActionBatch(userId, args.batchId)) != null) { throw new WERR_INVALID_PARAMETER('batchId', 'unique') } const outputScriptLengths = args.firstActionOutputScriptLengths - if (outputScriptLengths != null && ( - outputScriptLengths.length !== args.firstAction.outputs.length || - outputScriptLengths.some(length => !Number.isSafeInteger(length) || length < 0) || - args.firstAction.outputs.some((output, index) => - output.lockingScript.length > 0 && - output.lockingScript.length / 2 !== outputScriptLengths[index] - ) - )) { + if ( + outputScriptLengths != null && + (outputScriptLengths.length !== args.firstAction.outputs.length || + outputScriptLengths.some(length => !Number.isSafeInteger(length) || length < 0) || + args.firstAction.outputs.some( + (output, index) => + output.lockingScript.length > 0 && output.lockingScript.length / 2 !== outputScriptLengths[index] + )) + ) { throw new WERR_INVALID_PARAMETER( 'firstActionOutputScriptLengths', 'valid byte lengths aligned with firstAction outputs' ) } const changeBasket = verifyOne(await storage.findOutputBaskets({ partial: { userId, name: 'default' } })) - const explicit = await resolveExplicitOutputs( - storage, - userId, - args.firstAction, - outputScriptLengths != null - ) + const explicit = await resolveExplicitOutputs(storage, userId, args.firstAction, outputScriptLengths != null) const noSendChange = await resolveNoSendChangeOutputs(storage, userId, args.firstAction) const fixedOutputIds = new Set([...explicit.outputs, ...noSendChange.outputs].map(output => output.outputId)) - const available = (await storage.findAvailableManagedChangeInputs( - userId, changeBasket.basketId, !args.firstAction.isDelayed - )).filter(output => !fixedOutputIds.has(output.outputId)) + const availableOutputs = ( + await storage.findAvailableManagedChangeInputs(userId, changeBasket.basketId, false) + ).filter(output => !fixedOutputIds.has(output.outputId)) + const availableStatuses = await storage.findTransactionStatusesByIds( + userId, + availableOutputs.map(output => output.transactionId) + ) + const available: ReservationCandidate[] = availableOutputs.map(output => ({ + ...output, + // An orphaned/missing status is never promoted ahead of known settled + // ancestry. Retain it only in the final last-resort reservation tier. + transactionStatus: availableStatuses.get(output.transactionId) ?? 'sending' + })) const target = estimateFirstActionTarget( storage, args.firstAction, explicit.inputSatoshis + noSendChange.inputSatoshis, outputScriptLengths ) - const fixedOutputs = [...new Map( - [...explicit.outputs, ...noSendChange.outputs].map(output => [output.outputId, output]) - ).values()] + const fixedOutputs = [ + ...new Map([...explicit.outputs, ...noSendChange.outputs].map(output => [output.outputId, output])).values() + ] const maxReservedOutputs = storage.actionBatchMaxReservedOutputs if (maxReservedOutputs >= 0 && fixedOutputs.length > maxReservedOutputs) { - throw new WERR_INVALID_PARAMETER( - 'firstAction', - `no more than ${maxReservedOutputs} persisted inputs` - ) + throw new WERR_INVALID_PARAMETER('firstAction', `no more than ${maxReservedOutputs} persisted inputs`) } - const initialCapacity = maxReservedOutputs < 0 - ? INITIAL_RESERVATION_LIMIT - : Math.min(INITIAL_RESERVATION_LIMIT, maxReservedOutputs) + const initialCapacity = + maxReservedOutputs < 0 ? INITIAL_RESERVATION_LIMIT : Math.min(INITIAL_RESERVATION_LIMIT, maxReservedOutputs) const requiredCapacity = Math.max(0, initialCapacity - fixedOutputs.length) const funding = chooseReservationPool( available, @@ -434,14 +467,15 @@ export async function beginActionBatch ( false, reservationPlanningCosts(storage, changeBasket) ) + const fundingOutputs = funding.map(reservationOutput) const batch = newBatch(userId, args.batchId) await storage.transaction(async trx => { await storage.insertActionBatch(batch, trx) - await reserveOutputs(storage, batch, [...fixedOutputs, ...funding], trx) + await reserveOutputs(storage, batch, [...fixedOutputs, ...fundingOutputs], trx) }) let fundingResult: Awaited> try { - fundingResult = await makeFundingResult(storage, args.firstAction, [...funding, ...fixedOutputs]) + fundingResult = await makeFundingResult(storage, args.firstAction, [...fundingOutputs, ...fixedOutputs]) } catch (error) { await storage.transaction(async trx => await releaseBatchState(storage, batch, 'aborted', trx)) throw error @@ -455,21 +489,29 @@ export async function beginActionBatch ( feeModel: validateStorageFeeModel(storage.feeModel), commissionSatoshis: storage.commissionSatoshis, commissionPubKeyHex: storage.commissionPubKeyHex, - availableChangeCount: available.length, - reservedOutputs: fundingResult.outputs.filter(output => funding.some(candidate => candidate.outputId === output.outputId)), + availableChangeCount: available.filter( + output => output.satoshis >= Math.max(1, changeBasket.minimumDesiredUTXOValue) + ).length, + managedChangePolicy: { + maxOutputsPerAction: storage.managedChangePolicy.maxOutputsPerAction, + migrationInputsPerAction: storage.managedChangePolicy.migrationInputsPerAction + }, + reservedOutputs: fundingResult.outputs.filter(output => + fundingOutputs.some(candidate => candidate.outputId === output.outputId) + ), explicitOutputs: fundingResult.outputs.filter(output => explicitIds.has(output.outputId)), inputBeef: fundingResult.beef } } -function requireLiveBatch (batch: TableActionBatch | undefined, batchId?: string): TableActionBatch { +function requireLiveBatch(batch: TableActionBatch | undefined, batchId?: string): TableActionBatch { const state = actionBatchErrorState(batch) if (state != null) throw new WERR_ACTION_BATCH_STATE(state, batchId ?? batch?.batchId) if (batch == null) throw new WERR_ACTION_BATCH_STATE('missing', batchId) return batch } -export async function extendActionBatch ( +export async function extendActionBatch( storage: StorageProvider, auth: AuthId, args: ExtendActionBatchArgs @@ -486,27 +528,30 @@ export async function extendActionBatch ( throw new WERR_INVALID_PARAMETER('targetSatoshis', 'non-negative safe integer') } const maxReservedOutputs = storage.actionBatchMaxReservedOutputs - if ((maxReservedOutputs >= 0 && args.explicitOutpoints.length > maxReservedOutputs) || + if ( + (maxReservedOutputs >= 0 && args.explicitOutpoints.length > maxReservedOutputs) || new Set(args.explicitOutpoints.map(outpoint => `${outpoint.txid}.${outpoint.vout}`)).size !== args.explicitOutpoints.length || - args.explicitOutpoints.some(outpoint => !isValidOutpoint(outpoint))) { + args.explicitOutpoints.some(outpoint => !isValidOutpoint(outpoint)) + ) { throw new WERR_INVALID_PARAMETER( 'explicitOutpoints', `at most ${maxReservedOutputs < 0 ? 'the configured request capacity' : maxReservedOutputs} valid outpoints` ) } - const remainingCapacity = maxReservedOutputs < 0 - ? Number.MAX_SAFE_INTEGER - : maxReservedOutputs - alreadyReserved.length + const remainingCapacity = + maxReservedOutputs < 0 ? Number.MAX_SAFE_INTEGER : maxReservedOutputs - alreadyReserved.length if (remainingCapacity <= 0 && (args.requestedOutputs > 0 || args.explicitOutpoints.length > 0)) { - throw new WERR_INVALID_OPERATION( - `action batch already holds the maximum of ${maxReservedOutputs} outputs` - ) + throw new WERR_INVALID_OPERATION(`action batch already holds the maximum of ${maxReservedOutputs} outputs`) } const explicitByOutpoint = await storage.findOutputsByOutpoints(userId, args.explicitOutpoints) - const explicit = [...new Map(Object.values(explicitByOutpoint) - .filter(output => !alreadyReserved.includes(output.outputId)) - .map(output => [output.outputId, output])).values()] + const explicit = [ + ...new Map( + Object.values(explicitByOutpoint) + .filter(output => !alreadyReserved.includes(output.outputId)) + .map(output => [output.outputId, output]) + ).values() + ] if (explicit.length > remainingCapacity) { throw new WERR_INVALID_PARAMETER( 'explicitOutpoints', @@ -514,8 +559,17 @@ export async function extendActionBatch ( ) } const explicitIds = new Set(explicit.map(output => output.outputId)) - const available = (await storage.findAvailableManagedChangeInputs(userId, basket.basketId, false)) - .filter(output => !explicitIds.has(output.outputId)) + const availableOutputs = (await storage.findAvailableManagedChangeInputs(userId, basket.basketId, false)).filter( + output => !explicitIds.has(output.outputId) + ) + const availableStatuses = await storage.findTransactionStatusesByIds( + userId, + availableOutputs.map(output => output.transactionId) + ) + const available: ReservationCandidate[] = availableOutputs.map(output => ({ + ...output, + transactionStatus: availableStatuses.get(output.transactionId) ?? 'sending' + })) const requestedCount = Math.min( args.requestedOutputs, ACTION_BATCH_MAX_RESERVATION_EXTENSION_OUTPUTS, @@ -529,53 +583,63 @@ export async function extendActionBatch ( true, reservationPlanningCosts(storage, basket) ) + const fundingOutputs = funding.map(reservationOutput) const fundingShape = argsToFundingShape(args.includeSourceTransactions) - const fundingResult = await makeFundingResult(storage, fundingShape, [...funding, ...explicit]) - const expiresAt = new Date(Math.min( - batch.hardExpiresAt.getTime(), - Date.now() + ACTION_BATCH_LEASE_MS - )) + const fundingResult = await makeFundingResult(storage, fundingShape, [...fundingOutputs, ...explicit]) + const expiresAt = new Date(Math.min(batch.hardExpiresAt.getTime(), Date.now() + ACTION_BATCH_LEASE_MS)) await storage.transaction(async trx => { - const current = requireLiveBatch( - await storage.findActionBatchForUpdate(userId, args.batchId, trx), - args.batchId - ) - const additions = [...new Map( - [...funding, ...explicit].map(output => [output.outputId, output]) - ).values()] + const current = requireLiveBatch(await storage.findActionBatchForUpdate(userId, args.batchId, trx), args.batchId) + const additions = [...new Map([...fundingOutputs, ...explicit].map(output => [output.outputId, output])).values()] const currentReserved = new Set(await storage.findActionBatchOutputIds(current.actionBatchId, trx)) const newAdditions = additions.filter(output => !currentReserved.has(output.outputId)) if (maxReservedOutputs >= 0 && currentReserved.size + newAdditions.length > maxReservedOutputs) { - throw new WERR_INVALID_OPERATION( - `action batch cannot reserve more than ${maxReservedOutputs} outputs` - ) + throw new WERR_INVALID_OPERATION(`action batch cannot reserve more than ${maxReservedOutputs} outputs`) } await reserveOutputs(storage, current, newAdditions, trx) await storage.updateActionBatch(current.actionBatchId, { expiresAt }, trx) }) return { expiresAt: expiresAt.toISOString(), - reservedOutputs: fundingResult.outputs.filter(output => funding.some(candidate => candidate.outputId === output.outputId)), - explicitOutputs: fundingResult.outputs.filter(output => explicit.some(candidate => candidate.outputId === output.outputId)), + reservedOutputs: fundingResult.outputs.filter(output => + fundingOutputs.some(candidate => candidate.outputId === output.outputId) + ), + explicitOutputs: fundingResult.outputs.filter(output => + explicit.some(candidate => candidate.outputId === output.outputId) + ), inputBeef: fundingResult.beef } } -function argsToFundingShape (includeSourceTransactions: boolean): Validation.ValidCreateActionArgs { +function argsToFundingShape(includeSourceTransactions: boolean): Validation.ValidCreateActionArgs { return { - inputs: [], outputs: [], labels: [], description: 'action batch extension', version: 1, lockTime: 0, + inputs: [], + outputs: [], + labels: [], + description: 'action batch extension', + version: 1, + lockTime: 0, options: { - acceptDelayedBroadcast: true, returnTXIDOnly: false, noSend: true, sendWith: [], signAndProcess: true, - knownTxids: [], noSendChange: [], randomizeOutputs: true + acceptDelayedBroadcast: true, + returnTXIDOnly: false, + noSend: true, + sendWith: [], + signAndProcess: true, + knownTxids: [], + noSendChange: [], + randomizeOutputs: true }, - isSendWith: false, isNewTx: true, isRemixChange: false, isNoSend: true, isDelayed: true, + isSendWith: false, + isNewTx: true, + isRemixChange: false, + isNoSend: true, + isDelayed: true, isTestWerrReviewActions: false, isSignAction: includeSourceTransactions, includeAllSourceTransactions: includeSourceTransactions } } -export async function renewActionBatch ( +export async function renewActionBatch( storage: StorageProvider, auth: AuthId, batchId: string @@ -589,12 +653,14 @@ export async function renewActionBatch ( }) } -function validateResumeOutpoints (storage: StorageProvider, args: ResumeActionBatchArgs): void { +function validateResumeOutpoints(storage: StorageProvider, args: ResumeActionBatchArgs): void { const unique = new Set(args.outpoints.map(outpoint => `${outpoint.txid}.${outpoint.vout}`)) const maxReservedOutputs = storage.actionBatchMaxReservedOutputs - if ((maxReservedOutputs >= 0 && args.outpoints.length > maxReservedOutputs) || + if ( + (maxReservedOutputs >= 0 && args.outpoints.length > maxReservedOutputs) || unique.size !== args.outpoints.length || - args.outpoints.some(outpoint => !isValidOutpoint(outpoint))) { + args.outpoints.some(outpoint => !isValidOutpoint(outpoint)) + ) { throw new WERR_INVALID_PARAMETER( 'outpoints', `at most ${maxReservedOutputs < 0 ? 'the configured request capacity' : maxReservedOutputs} unique valid outpoints` @@ -607,7 +673,7 @@ function validateResumeOutpoints (storage: StorageProvider, args: ResumeActionBa * workspace. This is deliberately explicit: another action may not become a * member merely because it happens to use the same Wallet instance. */ -export async function resumeActionBatch ( +export async function resumeActionBatch( storage: StorageProvider, auth: AuthId, args: ResumeActionBatchArgs @@ -623,15 +689,15 @@ export async function resumeActionBatch ( } if (batch == null) throw new WERR_ACTION_BATCH_STATE('missing', args.batchId) - const expiresAt = new Date(Math.min( - batch.hardExpiresAt.getTime(), - Date.now() + ACTION_BATCH_LEASE_MS - )) + const expiresAt = new Date(Math.min(batch.hardExpiresAt.getTime(), Date.now() + ACTION_BATCH_LEASE_MS)) if (state == null) { const reserved = new Set(await storage.findActionBatchOutputIds(batch.actionBatchId, trx)) const current = await storage.findOutputsByOutpointsForUpdate(userId, args.outpoints, trx) - if (Object.values(current).some(output => !reserved.has(output.outputId)) || - Object.keys(current).length !== args.outpoints.length || reserved.size !== args.outpoints.length) { + if ( + Object.values(current).some(output => !reserved.has(output.outputId)) || + Object.keys(current).length !== args.outpoints.length || + reserved.size !== args.outpoints.length + ) { throw new WERR_ACTION_BATCH_STATE('conflicted', args.batchId) } await storage.updateActionBatch(batch.actionBatchId, { expiresAt }, trx) @@ -651,24 +717,31 @@ export async function resumeActionBatch ( ) if (conflicts.length > 0) throw new WERR_ACTION_BATCH_STATE('conflicted', args.batchId) const now = new Date() - await storage.reserveActionBatchOutputs(exactOutputs.map(output => ({ - actionBatchId: batch.actionBatchId, - outputId: output.outputId, - created_at: now, - updated_at: now - })), trx) - await storage.updateActionBatch(batch.actionBatchId, { - status: 'active', - expiresAt, - manifest: undefined, - manifestDigest: undefined, - uploadDigests: undefined - }, trx) + await storage.reserveActionBatchOutputs( + exactOutputs.map(output => ({ + actionBatchId: batch.actionBatchId, + outputId: output.outputId, + created_at: now, + updated_at: now + })), + trx + ) + await storage.updateActionBatch( + batch.actionBatchId, + { + status: 'active', + expiresAt, + manifest: undefined, + manifestDigest: undefined, + uploadDigests: undefined + }, + trx + ) return { expiresAt: expiresAt.toISOString() } }) } -async function reacquireManifestInputs ( +async function reacquireManifestInputs( storage: StorageProvider, userId: number, batch: TableActionBatch, @@ -679,13 +752,21 @@ async function reacquireManifestInputs ( throw new WERR_ACTION_BATCH_STATE('hard-expired', batch.batchId) } const stagedTxids = new Set(validated.actions.map(({ action }) => action.txid)) - const outpoints = [...new Map(validated.actions.flatMap(({ action }) => action.plan.inputs) - .filter(input => !stagedTxids.has(input.sourceTxid)) - .map(input => [`${input.sourceTxid}.${input.sourceVout}`, { - txid: input.sourceTxid, - vout: input.sourceVout, - providedBy: input.providedBy - }])).values()] + const outpoints = [ + ...new Map( + validated.actions + .flatMap(({ action }) => action.plan.inputs) + .filter(input => !stagedTxids.has(input.sourceTxid)) + .map(input => [ + `${input.sourceTxid}.${input.sourceVout}`, + { + txid: input.sourceTxid, + vout: input.sourceVout, + providedBy: input.providedBy + } + ]) + ).values() + ] const stored = await storage.findOutputsByOutpointsForUpdate(userId, outpoints, trx) await storage.deleteActionBatchOutputReservations(batch.actionBatchId, trx) @@ -704,30 +785,40 @@ async function reacquireManifestInputs ( } outputs.push(output) } - const conflicts = await storage.findReservedActionBatchOutputIds(outputs.map(output => output.outputId), trx) + const conflicts = await storage.findReservedActionBatchOutputIds( + outputs.map(output => output.outputId), + trx + ) if (conflicts.length > 0) { throw new WERR_INVALID_OPERATION('one or more expired action batch inputs were reserved elsewhere') } const now = new Date() - await storage.reserveActionBatchOutputs(outputs.map(output => ({ - actionBatchId: batch.actionBatchId, - outputId: output.outputId, - created_at: now, - updated_at: now - })), trx) - await storage.updateActionBatch(batch.actionBatchId, { - status: 'active', - expiresAt: new Date(Math.min(batch.hardExpiresAt.getTime(), now.getTime() + ACTION_BATCH_LEASE_MS)) - }, trx) + await storage.reserveActionBatchOutputs( + outputs.map(output => ({ + actionBatchId: batch.actionBatchId, + outputId: output.outputId, + created_at: now, + updated_at: now + })), + trx + ) + await storage.updateActionBatch( + batch.actionBatchId, + { + status: 'active', + expiresAt: new Date(Math.min(batch.hardExpiresAt.getTime(), now.getTime() + ACTION_BATCH_LEASE_MS)) + }, + trx + ) } -function transactionStatuses (action: ActionBatchCommitAction): { tx: TransactionStatus, req: ProvenTxReqStatus } { +function transactionStatuses(action: ActionBatchCommitAction): { tx: TransactionStatus; req: ProvenTxReqStatus } { if (action.metadata.isNoSend) return { tx: 'nosend', req: 'nosend' } if (action.metadata.isDelayed) return { tx: 'unprocessed', req: 'unsent' } return { tx: 'unprocessed', req: 'unprocessed' } } -async function persistLabels ( +async function persistLabels( storage: StorageProvider, transactionId: number, labels: string[], @@ -741,7 +832,7 @@ async function persistLabels ( } } -function outputBasketId ( +function outputBasketId( isChange: boolean, basket: string | undefined, baskets: Record @@ -750,7 +841,7 @@ function outputBasketId ( return basket == null ? undefined : baskets[basket].basketId } -async function persistOutputTags ( +async function persistOutputTags( storage: StorageProvider, outputId: number, tagNames: string[], @@ -763,7 +854,7 @@ async function persistOutputTags ( } } -async function persistOutputCommission ( +async function persistOutputCommission( storage: StorageProvider, row: TableOutput, action: ActionBatchCommitAction, @@ -787,7 +878,7 @@ async function persistOutputCommission ( await storage.insertCommission(commission, trx) } -async function persistOutputs ( +async function persistOutputs( storage: StorageProvider, userId: number, transactionId: number, @@ -802,12 +893,14 @@ async function persistOutputs ( for (const planned of action.plan.outputs) { const output = tx.outputs[planned.vout] const isChange = planned.providedBy === 'storage' && planned.purpose === 'change' - const isCommission = planned.providedBy === 'storage' && + const isCommission = + planned.providedBy === 'storage' && (planned.purpose === 'storage-commission' || planned.purpose === 'service-charge') const offset = offsets.outputs[planned.vout] - const lockingScript = offset.length <= storage.getSettings().maxOutputScript || isCommission - ? output.lockingScript.toBinary() - : undefined + const lockingScript = + offset.length <= storage.getSettings().maxOutputScript || isCommission + ? output.lockingScript.toBinary() + : undefined const now = new Date() const row: TableOutput = { outputId: 0, @@ -840,7 +933,7 @@ async function persistOutputs ( return rows } -async function persistAction ( +async function persistAction( ...[ storage, userId, @@ -896,51 +989,44 @@ async function persistAction ( let output = stagedByOutpoint.get(outpoint) output ??= storedByOutpoint[outpoint] if (output == null) continue - if (output.spentBy != null || !output.spendable) throw new WERR_INVALID_OPERATION(`input ${outpoint} is no longer spendable`) + if (output.spentBy != null || !output.spendable) + throw new WERR_INVALID_OPERATION(`input ${outpoint} is no longer spendable`) if (!stagedByOutpoint.has(outpoint) && !reservedOutputIds.has(output.outputId)) { throw new WERR_INVALID_OPERATION(`input ${outpoint} was not reserved by this action batch`) } - await storage.updateOutput(output.outputId, { - spendable: false, - spentBy: transaction.transactionId, - spendingDescription: action.metadata.inputs[input.vin]?.inputDescription - }, trx) + await storage.updateOutput( + output.outputId, + { + spendable: false, + spentBy: transaction.transactionId, + spendingDescription: action.metadata.inputs[input.vin]?.inputDescription + }, + trx + ) output.spendable = false output.spentBy = transaction.transactionId } - const outputRows = await persistOutputs( - storage, - userId, - transaction.transactionId, - validated, - baskets, - tags, - trx - ) + const outputRows = await persistOutputs(storage, userId, transaction.transactionId, validated, baskets, tags, trx) for (const output of outputRows) stagedByOutpoint.set(`${action.txid}.${output.vout}`, output) - const req = EntityProvenTxReq.fromTxid( - action.txid, - rawTx, - validated.externalInputBeef - ) + const req = EntityProvenTxReq.fromTxid(action.txid, rawTx, validated.externalInputBeef) req.status = statuses.req req.addNotifyTransactionId(transaction.transactionId) return await req.insertOrMerge(storage, trx) } -async function persistManifestAtomically ( +async function persistManifestAtomically( storage: StorageProvider, userId: number, batch: TableActionBatch, manifest: ActionBatchManifest, - validated: { actions: ValidatedBatchAction[], dependencyBeef: Uint8Array, beef: Beef } + validated: { actions: ValidatedBatchAction[]; dependencyBeef: Uint8Array; beef: Beef } ): Promise<{ - batch: TableActionBatch - alreadyCommitted: boolean - share?: GetReqsAndBeefResult - }> { + batch: TableActionBatch + alreadyCommitted: boolean + share?: GetReqsAndBeefResult +}> { return await storage.transaction(async trx => { const current = await storage.findActionBatchForUpdate(userId, batch.batchId, trx) if (current == null) throw new WERR_ACTION_BATCH_STATE('missing', batch.batchId) @@ -965,39 +1051,34 @@ async function persistManifestAtomically ( } const reservedOutputIds = new Set(await storage.findActionBatchOutputIds(current.actionBatchId, trx)) const stagedTxids = new Set(validated.actions.map(({ action }) => action.txid)) - const inputOutpoints = [...new Map(validated.actions.flatMap(({ action }) => action.plan.inputs) - .filter(input => !stagedTxids.has(input.sourceTxid)) - .map(input => [`${input.sourceTxid}.${input.sourceVout}`, { - txid: input.sourceTxid, - vout: input.sourceVout - }])).values()] - const storedByOutpoint = await storage.findOutputsByOutpointsForUpdate( - userId, - inputOutpoints, - trx - ) + const inputOutpoints = [ + ...new Map( + validated.actions + .flatMap(({ action }) => action.plan.inputs) + .filter(input => !stagedTxids.has(input.sourceTxid)) + .map(input => [ + `${input.sourceTxid}.${input.sourceVout}`, + { + txid: input.sourceTxid, + vout: input.sourceVout + } + ]) + ).values() + ] + const storedByOutpoint = await storage.findOutputsByOutpointsForUpdate(userId, inputOutpoints, trx) const allBasketNames = validated.actions.flatMap(({ action }) => - action.plan.outputs.flatMap(output => output.basket == null ? [] : [output.basket]) - ) - if (validated.actions.some(({ action }) => - action.plan.outputs.some(output => output.purpose === 'change') - )) allBasketNames.push('default') - const baskets = await storage.findOrInsertOutputBasketsBulk( - userId, - [...new Set(allBasketNames)], - trx + action.plan.outputs.flatMap(output => (output.basket == null ? [] : [output.basket])) ) + if (validated.actions.some(({ action }) => action.plan.outputs.some(output => output.purpose === 'change'))) + allBasketNames.push('default') + const baskets = await storage.findOrInsertOutputBasketsBulk(userId, [...new Set(allBasketNames)], trx) const tags = await storage.findOrInsertOutputTagsBulk( userId, - [...new Set(validated.actions.flatMap(({ action }) => - action.plan.outputs.flatMap(output => output.tags) - ))], + [...new Set(validated.actions.flatMap(({ action }) => action.plan.outputs.flatMap(output => output.tags)))], trx ) const labelNames = [...new Set(validated.actions.flatMap(({ action }) => action.metadata.labels))] - const labelsByName = new Map(Object.entries( - await storage.findOrInsertTxLabelsBulk(userId, labelNames, trx) - )) + const labelsByName = new Map(Object.entries(await storage.findOrInsertTxLabelsBulk(userId, labelNames, trx))) const stagedByOutpoint = new Map() const reqsByTxid = new Map() for (const action of validated.actions) { @@ -1017,10 +1098,14 @@ async function persistManifestAtomically ( } await storage.deleteActionBatchOutputReservations(current.actionBatchId, trx) await storage.deleteActionBatchBlobRecords(current.actionBatchId, trx) - await storage.updateActionBatch(current.actionBatchId, { - status: 'committed', - manifestDigest: manifest.digest - }, trx) + await storage.updateActionBatch( + current.actionBatchId, + { + status: 'committed', + manifestDigest: manifest.digest + }, + trx + ) current.status = 'committed' current.manifestDigest = manifest.digest const details = manifest.sendWith.map(txid => { @@ -1049,7 +1134,7 @@ async function persistManifestAtomically ( }) } -async function completeCommittedBatch ( +async function completeCommittedBatch( storage: StorageProvider, userId: number, batch: TableActionBatch, @@ -1069,13 +1154,7 @@ async function completeCommittedBatch ( log: saved.log } } - const { swr, ndr } = await shareReqsWithWorld( - storage, - userId, - manifest.sendWith, - manifest.isDelayed, - share - ) + const { swr, ndr } = await shareReqsWithWorld(storage, userId, manifest.sendWith, manifest.isDelayed, share) const result: CommitActionBatchResult = { batchId: manifest.batchId, manifestDigest: manifest.digest, @@ -1088,7 +1167,7 @@ async function completeCommittedBatch ( return result } -async function commitActionBatchOnce ( +async function commitActionBatchOnce( storage: StorageProvider, userId: number, manifest: ActionBatchManifest @@ -1096,7 +1175,8 @@ async function commitActionBatchOnce ( const batch = await storage.findActionBatch(userId, manifest.batchId) if (batch == null) throw new WERR_ACTION_BATCH_STATE('missing', manifest.batchId) if (batch.status === 'committed') { - if (batch.manifestDigest !== manifest.digest) throw new WERR_INVALID_OPERATION('batch committed with another manifest') + if (batch.manifestDigest !== manifest.digest) + throw new WERR_INVALID_OPERATION('batch committed with another manifest') return await completeCommittedBatch(storage, userId, batch, manifest, true) } if (batch.status === 'aborted') throw new WERR_ACTION_BATCH_STATE('aborted', manifest.batchId) @@ -1127,7 +1207,7 @@ interface ActiveBatchCommit { const activeBatchCommits = new WeakMap>() -export async function commitActionBatch ( +export async function commitActionBatch( storage: StorageProvider, auth: AuthId, manifest: ActionBatchManifest @@ -1149,13 +1229,14 @@ export async function commitActionBatch ( } return await active.promise } - const promise = commitActionBatchOnce(storage, userId, manifest) - .finally(() => { commits?.delete(key) }) + const promise = commitActionBatchOnce(storage, userId, manifest).finally(() => { + commits?.delete(key) + }) commits.set(key, { digest: manifest.digest, promise }) return await promise } -export async function commitActionBatchByDigest ( +export async function commitActionBatchByDigest( storage: StorageProvider, auth: AuthId, args: CommitActionBatchByDigestArgs @@ -1166,14 +1247,18 @@ export async function commitActionBatchByDigest ( throw new WERR_INVALID_OPERATION('prepared action batch manifest was not found') } const manifest = JSON.parse(batch.manifest) as ActionBatchManifest - if (manifest.format !== 2 || manifest.batchId !== args.batchId || manifest.digest !== args.digest || - !verifyActionBatchManifestDigest(manifest)) { + if ( + manifest.format !== 2 || + manifest.batchId !== args.batchId || + manifest.digest !== args.digest || + !verifyActionBatchManifestDigest(manifest) + ) { throw new WERR_INVALID_OPERATION('prepared action batch manifest is invalid') } return await commitActionBatch(storage, auth, manifest) } -export async function abortActionBatch ( +export async function abortActionBatch( storage: StorageProvider, auth: AuthId, batchId: string diff --git a/packages/wallet/wallet-toolbox/src/storage/methods/availableManagedChange.ts b/packages/wallet/wallet-toolbox/src/storage/methods/availableManagedChange.ts index 4bcf7d052..9817387ee 100644 --- a/packages/wallet/wallet-toolbox/src/storage/methods/availableManagedChange.ts +++ b/packages/wallet/wallet-toolbox/src/storage/methods/availableManagedChange.ts @@ -7,14 +7,20 @@ import { isAutoSpendableChangeOutput, managedChangeOutputFields } from './manage export type ManagedChangeInputCandidate = Pick< TableOutput, 'outputId' | 'transactionId' | 'satoshis' | 'txid' | 'vout' -> +> & { + /** + * Additive ancestry metadata. Older custom providers may omit it; the + * planner resolves a missing value through the provider's transaction API. + */ + transactionStatus?: TransactionStatus +} /** * Return the exact set of wallet-managed outputs currently eligible for * automatic funding. Keeping this predicate shared prevents the planner, * allocator, action-batch reservations, and availability count from drifting. */ -export async function availableManagedChange ( +export async function availableManagedChange( storage: StorageProvider, userId: number, basketId: number, @@ -23,13 +29,20 @@ export async function availableManagedChange ( ): Promise { const statuses: TransactionStatus[] = ['completed', 'unproven'] if (!excludeSending) statuses.push('sending') - const outputs = (await storage.findOutputs({ - partial: { userId, basketId, spendable: true, ...managedChangeOutputFields }, - txStatus: statuses, - noScript: true, - trx - })).filter(isAutoSpendableChangeOutput) + const outputs = ( + await storage.findOutputs({ + partial: { userId, basketId, spendable: true, ...managedChangeOutputFields }, + txStatus: statuses, + noScript: true, + trx + }) + ).filter(isAutoSpendableChangeOutput) if (outputs.length === 0) return outputs - const reserved = new Set(await storage.findReservedActionBatchOutputIds(outputs.map(output => output.outputId), trx)) + const reserved = new Set( + await storage.findReservedActionBatchOutputIds( + outputs.map(output => output.outputId), + trx + ) + ) return outputs.filter(output => !reserved.has(output.outputId)) } diff --git a/packages/wallet/wallet-toolbox/src/storage/methods/createAction.ts b/packages/wallet/wallet-toolbox/src/storage/methods/createAction.ts index 8e362cc61..3a8dbc4db 100644 --- a/packages/wallet/wallet-toolbox/src/storage/methods/createAction.ts +++ b/packages/wallet/wallet-toolbox/src/storage/methods/createAction.ts @@ -1,4 +1,12 @@ -import { Beef, OriginatorDomainNameStringUnder250Bytes, Random, Script, TelemetrySpan, Utils, Validation } from '@bsv/sdk' +import { + Beef, + OriginatorDomainNameStringUnder250Bytes, + Random, + Script, + TelemetrySpan, + Utils, + Validation +} from '@bsv/sdk' import { generateChangeSdk, GenerateChangeSdkChangeInput, @@ -118,7 +126,7 @@ async function createActionCore( * - Create result inputs with source locking scripts * - Create result outputs with new locking scripts. * - Create and return result. - */ + */ const userId = auth.userId! const validated = await traceStorageStep( @@ -162,7 +170,7 @@ async function createActionCore( const feeModel = validateStorageFeeModel(storage.feeModel) logger?.log(`validated fee model ${JSON.stringify(feeModel)}`) - const initialFundingPlan = await prepareFundingPlanWithSendingFallback( + const initialFundingPlan = await prepareFundingPlanWithLiquidityPolicy( storage, [userId, vargs, xinputs, xoutputs, changeBasket, noSendChangeIn, feeModel], parent @@ -196,14 +204,7 @@ async function createActionCore( 'action.storage_beef_bytes': storageBeefBytes.length }, async span => { - const transaction = await createNewTxRecord( - storage, - userId, - vargs, - storageBeefBytes, - initialSatoshis, - trx - ) + const transaction = await createNewTxRecord(storage, userId, vargs, storageBeefBytes, initialSatoshis, trx) span?.end({ attributes: { 'action.transaction_record_created': true } }) return transaction } @@ -229,7 +230,8 @@ async function createActionCore( logger?.log('adjusted change outputs to max possible') } - const satoshis = funded.changeOutputs.reduce((sum, output) => sum + output.satoshis, 0) - + const satoshis = + funded.changeOutputs.reduce((sum, output) => sum + output.satoshis, 0) - funded.allocatedChange.reduce((sum, output) => sum + output.satoshis, 0) if (satoshis !== initialSatoshis) { await storage.updateTransaction(newTx.transactionId, { satoshis }, trx) @@ -551,13 +553,16 @@ async function persistNewOutput( o.outputId = await storage.insertOutput(o, trx) for (const tagName of new Set(tags)) { const tag = txTags[tagName] - await storage.insertOutputTagMap({ - outputId: verifyId(o.outputId), - outputTagId: verifyId(tag.outputTagId), - created_at: new Date(), - updated_at: new Date(), - isDeleted: false - }, trx) + await storage.insertOutputTagMap( + { + outputId: verifyId(o.outputId), + outputTagId: verifyId(tag.outputTagId), + created_at: new Date(), + updated_at: new Date(), + isDeleted: false + }, + trx + ) } return describeNewOutput(o, tags, txBaskets) } @@ -587,17 +592,20 @@ async function createNewOutputs( const lockingScript = asArray(xo.lockingScript) if (xo.purpose === 'service-charge') { const now = new Date() - await storage.insertCommission({ - userId, - transactionId: ctx.transactionId, - lockingScript, - satoshis: xo.satoshis, - isRedeemed: false, - keyOffset: verifyTruthy(xo.keyOffset), - created_at: now, - updated_at: now, - commissionId: 0 - }, trx) + await storage.insertCommission( + { + userId, + transactionId: ctx.transactionId, + lockingScript, + satoshis: xo.satoshis, + isRedeemed: false, + keyOffset: verifyTruthy(xo.keyOffset), + created_at: now, + updated_at: now, + commissionId: 0 + }, + trx + ) const o = makeDefaultOutput(userId, ctx.transactionId, xo.satoshis, xo.vout) o.lockingScript = lockingScript o.providedBy = 'storage' @@ -634,14 +642,18 @@ async function createNewOutputs( // one multi-row statement. Tagged rows retain the established id-dependent // insertion path below. const untagged = newOutputs.filter(output => output.tags.length === 0) - await storage.insertOutputs(untagged.map(output => output.o), trx) + await storage.insertOutputs( + untagged.map(output => output.o), + trx + ) const outputs: StorageCreateTransactionSdkOutput[] = [] const changeVouts: number[] = [] for (const { o, tags } of newOutputs) { - const { changeVout, ro } = tags.length === 0 - ? describeNewOutput(o, tags, txBaskets) - : await persistNewOutput(storage, o, tags, txTags, txBaskets, trx) + const { changeVout, ro } = + tags.length === 0 + ? describeNewOutput(o, tags, txBaskets) + : await persistNewOutput(storage, o, tags, txTags, txBaskets, trx) if (changeVout !== undefined) changeVouts.push(changeVout) outputs.push(ro) } @@ -986,27 +998,35 @@ async function validateNoSendChange( return r } +type ResolvedManagedChangeInputCandidate = ManagedChangeInputCandidate & { + transactionStatus: TransactionStatus +} + interface PreparedFundingPlan { params: GenerateChangeSdkParams result: GenerateChangeSdkResult - selected: ManagedChangeInputCandidate[] + selected: ResolvedManagedChangeInputCandidate[] availableChangeCount: number - excludeSending: boolean + eligibleStatuses: TransactionStatus[] + policyTier: 'completed' | 'unproven' | 'sending' | 'compatibility' + compatibilityFallback: boolean } type FundingClaimRequest = readonly [ userId: number, basketId: number, - excludeSending: boolean, + eligibleStatuses: TransactionStatus[], transactionId: number, noSendChangeIn: TableOutput[], plan: PreparedFundingPlan, trx?: TrxToken ] -function fundingPlanSatoshis (plan: PreparedFundingPlan): number { - return plan.result.changeOutputs.reduce((sum, output) => sum + output.satoshis, 0) - +function fundingPlanSatoshis(plan: PreparedFundingPlan): number { + return ( + plan.result.changeOutputs.reduce((sum, output) => sum + output.satoshis, 0) - plan.selected.reduce((sum, output) => sum + output.satoshis, 0) + ) } type FundingPlanBaseContext = readonly [ @@ -1019,34 +1039,27 @@ type FundingPlanBaseContext = readonly [ feeModel: StorageFeeModel ] -type FundingPlanContext = readonly [ - ...FundingPlanBaseContext, - excludeSending: boolean, - parent?: TelemetrySpan, - trx?: TrxToken -] - type FundingClaim = | { - outputs: TableOutput[] - sourceTransactionCount: number - hydratedScriptCount: number - scriptSourceTransactionCount: number - conflict?: undefined - } - | { outputs?: undefined, conflict: 'candidate' | 'noSendChange' } + outputs: TableOutput[] + sourceTransactionCount: number + hydratedScriptCount: number + scriptSourceTransactionCount: number + conflict?: undefined + } + | { outputs?: undefined; conflict: 'candidate' | 'noSendChange' } type LockedFundingClaim = - | { outputs: TableOutput[], sourceTransactionCount: number, conflict?: undefined } - | { outputs?: undefined, sourceTransactionCount?: undefined, conflict: 'candidate' | 'noSendChange' } + | { outputs: TableOutput[]; sourceTransactionCount: number; conflict?: undefined } + | { outputs?: undefined; sourceTransactionCount?: undefined; conflict: 'candidate' | 'noSendChange' } class FundingClaimConflict extends Error { - constructor (readonly conflict: 'candidate' | 'noSendChange') { + constructor(readonly conflict: 'candidate' | 'noSendChange') { super('createAction funding claim changed concurrently') } } -async function traceStorageStep ( +async function traceStorageStep( storage: StorageProvider, name: string, parent: TelemetrySpan | undefined, @@ -1061,14 +1074,17 @@ async function traceStorageStep ( ) } -function makeFundingParams ( +function makeFundingParams( + storage: StorageProvider, vargs: Validation.ValidCreateActionArgs, xinputs: XValidCreateActionInput[], xoutputs: XValidCreateActionOutput[], changeBasket: TableOutputBasket, feeModel: StorageFeeModel, - availableChangeCount: number + healthyChangeCount: number, + compatibilityFallback = false ): GenerateChangeSdkParams { + const preferredSatoshis = Math.max(1, changeBasket.minimumDesiredUTXOValue) return { fixedInputs: xinputs.map(input => ({ satoshis: input.satoshis, @@ -1079,46 +1095,64 @@ function makeFundingParams ( lockingScriptLength: output.lockingScript.length / 2 })), feeModel, - changeInitialSatoshis: Math.max(1, changeBasket.minimumDesiredUTXOValue), - changeFirstSatoshis: Math.max(1, Math.round(changeBasket.minimumDesiredUTXOValue / 4)), + changeInitialSatoshis: preferredSatoshis, + // The final compatibility shape uses the allocator's economic floor. This + // remains at least as permissive as every historical basket preference, + // including an exact 144/32 basket that has just migrated to 144/5000. + changeFirstSatoshis: compatibilityFallback ? 1 : preferredSatoshis, changeLockingScriptLength: 25, changeUnlockingScriptLength: 107, - targetNetCount: changeBasket.numberOfDesiredUTXOs - availableChangeCount, + targetNetCount: changeBasket.numberOfDesiredUTXOs - healthyChangeCount, + maxChangeOutputs: storage.managedChangePolicy.maxOutputsPerAction, + surplusPoolShaping: !compatibilityFallback, + maxMigrationInputs: compatibilityFallback ? 0 : storage.managedChangePolicy.migrationInputsPerAction, randomVals: vargs.randomVals } } -async function prepareFundingPlan ( +async function buildFundingPlan( storage: StorageProvider, - context: FundingPlanContext + context: FundingPlanBaseContext, + candidates: ResolvedManagedChangeInputCandidate[], + eligibleStatuses: TransactionStatus[], + policyTier: PreparedFundingPlan['policyTier'], + compatibilityFallback: boolean, + parent?: TelemetrySpan ): Promise { - const [userId, vargs, xinputs, xoutputs, changeBasket, noSendChangeIn, feeModel, excludeSending, parent, trx] = context - const candidates = await traceStorageStep( + const [userId, vargs, xinputs, xoutputs, changeBasket, noSendChangeIn, feeModel] = context + const noSendIds = new Set(noSendChangeIn.map(output => output.outputId)) + const available = candidates.filter( + output => !noSendIds.has(output.outputId) && eligibleStatuses.includes(output.transactionStatus) + ) + const preferredSatoshis = Math.max(1, changeBasket.minimumDesiredUTXOValue) + const healthyChangeCount = compatibilityFallback + // Preserve the legacy target-count input exactly, including noSendChange + // that is removed from the allocator below but still belongs to this tier. + ? candidates.filter(output => eligibleStatuses.includes(output.transactionStatus)).length + : candidates.filter(output => output.satoshis >= preferredSatoshis).length + const params = makeFundingParams( storage, - 'wallet.storage.create_action.funding_candidates', - parent, - { 'funding.exclude_sending': excludeSending }, - async span => { - const outputs = await storage.findAvailableManagedChangeInputCandidates( - userId, - changeBasket.basketId, - excludeSending, - trx - ) - span?.end({ - attributes: { - 'funding.candidate_count': outputs.length, - 'funding.candidate_satoshis': outputs.reduce((sum, output) => sum + output.satoshis, 0) - } - }) - return outputs - } + vargs, + xinputs, + xoutputs, + changeBasket, + feeModel, + healthyChangeCount, + compatibilityFallback ) - const noSendIds = new Set(noSendChangeIn.map(output => output.outputId)) - const available = candidates.filter(output => !noSendIds.has(output.outputId)) - // Preserve the legacy target-net-count input: noSendChange was included in - // countChangeInputs before it was consumed by the allocator. - const params = makeFundingParams(vargs, xinputs, xoutputs, changeBasket, feeModel, candidates.length) + const noSendStatuses = await storage.findTransactionStatusesByIds( + userId, + noSendChangeIn.map(output => output.transactionId) + ) + const plannedNoSend: ResolvedManagedChangeInputCandidate[] = noSendChangeIn.map(output => ({ + outputId: output.outputId, + transactionId: output.transactionId, + satoshis: output.satoshis, + txid: output.txid, + vout: output.vout, + transactionStatus: noSendStatuses.get(output.transactionId) ?? 'nosend' + })) + const claimStatuses = [...new Set([...eligibleStatuses, ...plannedNoSend.map(output => output.transactionStatus)])] return await traceStorageStep( storage, @@ -1126,18 +1160,20 @@ async function prepareFundingPlan ( parent, { 'funding.candidate_count': available.length, - 'funding.no_send_change_count': noSendChangeIn.length + 'funding.no_send_change_count': noSendChangeIn.length, + 'funding.policy_tier': policyTier, + 'funding.compatibility_fallback': compatibilityFallback }, async span => { - const allocated = new Map() + const allocated = new Map() const availableSelector = new CanonicalChangeSelector(available) - const noSend = [...noSendChangeIn] - const noSendById = new Map(noSendChangeIn.map(output => [output.outputId, output])) + const noSend = [...plannedNoSend] + const noSendById = new Map(plannedNoSend.map(output => [output.outputId, output])) const allocate = async ( targetSatoshis: number, exactSatoshis?: number ): Promise => { - let output: ManagedChangeInputCandidate | undefined = noSend.pop() + let output: ResolvedManagedChangeInputCandidate | undefined = noSend.pop() output ??= availableSelector.take(targetSatoshis, exactSatoshis) if (output == null) return undefined allocated.set(output.outputId, output) @@ -1166,82 +1202,181 @@ async function prepareFundingPlan ( result, selected, availableChangeCount: candidates.length, - excludeSending + eligibleStatuses: claimStatuses, + policyTier, + compatibilityFallback } } ) } -async function prepareFundingPlanWithSendingFallback ( +async function fundingPlanSerializedCost( + storage: StorageProvider, + plan: PreparedFundingPlan, + knownTxids: string[] +): Promise { + const txids = [...new Set(plan.selected.map(candidate => verifyTruthy(candidate.txid)))] + if (txids.length === 0) return plan.result.size + try { + const beef = await storage.getBeefForTransactions(txids, { + knownTxids, + ignoreStorage: false, + ignoreServices: true, + ignoreNewProven: false + }) + return plan.result.size + beef.toUint8Array().length + } catch { + return Number.MAX_SAFE_INTEGER + } +} + +async function prepareFundingPlanWithLiquidityPolicy( storage: StorageProvider, context: FundingPlanBaseContext, parent?: TelemetrySpan, trx?: TrxToken ): Promise { - const excludeSending = !context[1].isDelayed - try { - return await prepareFundingPlan(storage, [...context, excludeSending, parent, trx]) - } catch (error) { - if (!excludeSending || !(error instanceof WERR_INSUFFICIENT_FUNDS)) throw error - // A transaction waiting for background broadcast already has a complete - // raw transaction and input BEEF in storage. Its wallet-managed change is - // safe to chain: the immediate broadcast path recursively merges that - // ancestor into the child BEEF. Prefer settled change, then admit queued - // change only when excluding it would report an underfunded wallet. - return await prepareFundingPlan(storage, [...context, false, parent, trx]) + const [userId, vargs, , , changeBasket] = context + const rawCandidates = await traceStorageStep( + storage, + 'wallet.storage.create_action.funding_candidates', + parent, + { 'funding.include_pending': true }, + async span => { + const outputs = await storage.findAvailableManagedChangeInputCandidates(userId, changeBasket.basketId, false, trx) + span?.end({ + attributes: { + 'funding.candidate_count': outputs.length, + 'funding.candidate_satoshis': outputs.reduce((sum, output) => sum + output.satoshis, 0), + 'funding.completed_candidate_count': outputs.filter(output => output.transactionStatus === 'completed') + .length, + 'funding.unproven_candidate_count': outputs.filter(output => output.transactionStatus === 'unproven').length, + 'funding.sending_candidate_count': outputs.filter(output => output.transactionStatus === 'sending').length + } + }) + return outputs + } + ) + const missingStatusIds = rawCandidates + .filter(output => output.transactionStatus == null) + .map(output => output.transactionId) + const missingStatuses = await storage.findTransactionStatusesByIds(userId, missingStatusIds, trx) + const candidates: ResolvedManagedChangeInputCandidate[] = rawCandidates.map(output => ({ + ...output, + transactionStatus: output.transactionStatus ?? missingStatuses.get(output.transactionId) as TransactionStatus + })) + if (candidates.some(output => output.transactionStatus == null)) { + throw new WERR_INTERNAL('managed change candidate is missing its source transaction status') + } + + const tiers: Array<{ + policyTier: Exclude + statuses: TransactionStatus[] + }> = [ + { policyTier: 'completed', statuses: ['completed'] }, + { policyTier: 'unproven', statuses: ['completed', 'unproven'] }, + { policyTier: 'sending', statuses: ['completed', 'unproven', 'sending'] } + ] + const successful: PreparedFundingPlan[] = [] + let fundingError: unknown + for (const tier of tiers) { + let plan: PreparedFundingPlan | undefined + try { + plan = await buildFundingPlan(storage, context, candidates, tier.statuses, tier.policyTier, false, parent) + } catch (error) { + if (!(error instanceof WERR_INSUFFICIENT_FUNDS)) throw error + fundingError = error + // Pool shaping is optional. Before widening ancestry to unproven or + // sending parents, retry the same status tier with the former funding + // shape. This is the one-way compatibility guarantee: a preferred + // minimum can never manufacture starvation or force a pending chain. + try { + plan = await buildFundingPlan(storage, context, candidates, tier.statuses, 'compatibility', true, parent) + } catch (compatibilityError) { + if (!(compatibilityError instanceof WERR_INSUFFICIENT_FUNDS)) throw compatibilityError + fundingError = compatibilityError + } + } + if (plan != null) { + successful.push(plan) + if ( + successful.length === 1 && + (storage.managedChangePolicy.pendingComparisonInputs === -1 || + plan.selected.length <= storage.managedChangePolicy.pendingComparisonInputs) + ) + return plan + } } + + if (successful.length > 0) { + const baseline = successful[0] + if (successful.length === 1) return baseline + let chosen = baseline + let chosenCost = await fundingPlanSerializedCost(storage, baseline, vargs.options.knownTxids) + for (const alternative of successful.slice(1)) { + const cost = await fundingPlanSerializedCost(storage, alternative, vargs.options.knownTxids) + if (cost < chosenCost) { + chosen = alternative + chosenCost = cost + } + } + return chosen + } + + if (fundingError != null) throw fundingError + throw new WERR_INTERNAL('funding policy produced neither a plan nor an error') } -async function claimFundingPlan ( - storage: StorageProvider, - request: FundingClaimRequest -): Promise { - const [userId, basketId, excludeSending, transactionId, noSendChangeIn, plan, trx] = request +async function claimFundingPlan(storage: StorageProvider, request: FundingClaimRequest): Promise { + const [userId, basketId, eligibleStatuses, transactionId, noSendChangeIn, plan, trx] = request if (plan.selected.length === 0) { return { outputs: [], sourceTransactionCount: 0, hydratedScriptCount: 0, scriptSourceTransactionCount: 0 } } const noSendIds = new Set(noSendChangeIn.map(output => output.outputId)) - const statuses: TransactionStatus[] = ['completed', 'unproven'] - if (!excludeSending) statuses.push('sending') - - const claim: LockedFundingClaim = await storage.transaction(async claimTrx => { - const currentById = await storage.findFundingOutputsForUpdate( - userId, - plan.selected.map(output => output.outputId), - statuses, - claimTrx - ) - const transactionIds = [...new Set(Object.values(currentById).map(output => output.transactionId))] - const claimed: TableOutput[] = [] - for (const planned of plan.selected) { - const current = currentById[planned.outputId] - if ( - current?.outputId !== planned.outputId || - current?.satoshis !== planned.satoshis || - current?.basketId !== basketId || - !isAutoSpendableChangeOutput(current) || - current?.txid !== planned.txid || - current?.vout !== planned.vout - ) { - return { conflict: noSendIds.has(planned.outputId) ? 'noSendChange' : 'candidate' } as const + const claim: LockedFundingClaim = await storage + .transaction(async claimTrx => { + const currentById = await storage.findFundingOutputsForUpdate( + userId, + plan.selected.map(output => output.outputId), + eligibleStatuses, + claimTrx + ) + const transactionIds = [...new Set(Object.values(currentById).map(output => output.transactionId))] + const claimed: TableOutput[] = [] + for (const planned of plan.selected) { + const current = currentById[planned.outputId] + if ( + current?.outputId !== planned.outputId || + current?.satoshis !== planned.satoshis || + current?.basketId !== basketId || + !isAutoSpendableChangeOutput(current) || + current?.txid !== planned.txid || + current?.vout !== planned.vout + ) { + return { conflict: noSendIds.has(planned.outputId) ? 'noSendChange' : 'candidate' } as const + } + claimed.push(current) } - claimed.push(current) - } - const updated = await storage.markChangeInputsSpent(claimed.map(output => output.outputId), transactionId, claimTrx) - if (updated !== claimed.length) { - throw new FundingClaimConflict( - claimed.some(output => noSendIds.has(output.outputId)) ? 'noSendChange' : 'candidate' + const updated = await storage.markChangeInputsSpent( + claimed.map(output => output.outputId), + transactionId, + claimTrx ) - } - for (const output of claimed) { - output.spendable = false - output.spentBy = transactionId - } - return { outputs: claimed, sourceTransactionCount: transactionIds.length } - }, trx).catch(error => { - if (error instanceof FundingClaimConflict) return { conflict: error.conflict } as const - throw error - }) + if (updated !== claimed.length) { + throw new FundingClaimConflict( + claimed.some(output => noSendIds.has(output.outputId)) ? 'noSendChange' : 'candidate' + ) + } + for (const output of claimed) { + output.spendable = false + output.spentBy = transactionId + } + return { outputs: claimed, sourceTransactionCount: transactionIds.length } + }, trx) + .catch(error => { + if (error instanceof FundingClaimConflict) return { conflict: error.conflict } as const + throw error + }) if (claim.outputs == null) return claim const hydration = await hydrateFundingInputScripts(storage, claim.outputs, trx) return { @@ -1251,16 +1386,20 @@ async function claimFundingPlan ( } } -async function hydrateFundingInputScripts ( +async function hydrateFundingInputScripts( storage: StorageProvider, outputs: TableOutput[], trx?: TrxToken -): Promise<{ hydratedScriptCount: number, scriptSourceTransactionCount: number }> { - const missing = outputs.filter(output => - output.lockingScript?.length !== output.scriptLength && - output.scriptLength != null && output.scriptLength > 0 && - output.scriptOffset != null && output.scriptOffset > 0 && - output.txid != null && output.txid !== '' +): Promise<{ hydratedScriptCount: number; scriptSourceTransactionCount: number }> { + const missing = outputs.filter( + output => + output.lockingScript?.length !== output.scriptLength && + output.scriptLength != null && + output.scriptLength > 0 && + output.scriptOffset != null && + output.scriptOffset > 0 && + output.txid != null && + output.txid !== '' ) if (missing.length === 0) return { hydratedScriptCount: 0, scriptSourceTransactionCount: 0 } @@ -1328,7 +1467,7 @@ async function fundNewTransactionSdk( const claim = await claimFundingPlan(storage, [ userId, ctx.changeBasket.basketId, - plan.excludeSending, + plan.eligibleStatuses, ctx.transactionId, ctx.noSendChangeIn, plan, @@ -1350,17 +1489,9 @@ async function fundNewTransactionSdk( throw new WERR_INVALID_PARAMETER('noSendChange', 'outputs that remain spendable during action planning') } retryCount++ - plan = await prepareFundingPlanWithSendingFallback( + plan = await prepareFundingPlanWithLiquidityPolicy( storage, - [ - userId, - vargs, - ctx.xinputs, - ctx.xoutputs, - ctx.changeBasket, - ctx.noSendChangeIn, - ctx.feeModel - ], + [userId, vargs, ctx.xinputs, ctx.xoutputs, ctx.changeBasket, ctx.noSendChangeIn, ctx.feeModel], parent, trx ) @@ -1467,7 +1598,7 @@ function trimInputBeef(beef: Beef, vargs: Validation.ValidCreateActionArgs): Uin return beef.toUint8Array() } -function makeKnownTxidLookup (knownTxids: string[]): (txid: string) => boolean { +function makeKnownTxidLookup(knownTxids: string[]): (txid: string) => boolean { let lookups = 0 let indexed: Set | undefined return txid => { @@ -1488,7 +1619,7 @@ interface AllocatedChangeBeefPrefetchResult { txids: string[] } -function missingAllocatedChangeTxids ( +function missingAllocatedChangeTxids( allocatedChange: Array<{ txid?: string }>, beef: Beef, knownTxids: string[] @@ -1503,7 +1634,7 @@ function missingAllocatedChangeTxids ( ) } -function startAllocatedChangeBeefPrefetch ( +function startAllocatedChangeBeefPrefetch( storage: StorageProvider, vargs: Validation.ValidCreateActionArgs, allocatedChange: ManagedChangeInputCandidate[], @@ -1547,7 +1678,7 @@ function startAllocatedChangeBeefPrefetch ( ) } -function sameTxids (left: readonly string[], right: readonly string[]): boolean { +function sameTxids(left: readonly string[], right: readonly string[]): boolean { if (left.length !== right.length) return false const expected = new Set(left) return right.every(txid => expected.has(txid)) diff --git a/packages/wallet/wallet-toolbox/src/storage/methods/generateChange.ts b/packages/wallet/wallet-toolbox/src/storage/methods/generateChange.ts index 87af8dbb6..fc07cf473 100644 --- a/packages/wallet/wallet-toolbox/src/storage/methods/generateChange.ts +++ b/packages/wallet/wallet-toolbox/src/storage/methods/generateChange.ts @@ -229,7 +229,11 @@ async function generateChangeSdkCore( * Applies the per-transaction limit so that the UTXO pool grows * gradually rather than all at once. */ - const maxChangeOutputs = params.maxChangeOutputs ?? maxChangeOutputsPerTransaction + const maxChangeOutputs = + params.maxChangeOutputs === -1 + ? Number.MAX_SAFE_INTEGER + : (params.maxChangeOutputs ?? maxChangeOutputsPerTransaction) + const surplusPoolShaping = params.surplusPoolShaping === true const randomVals = [...(params.randomVals || [])] const nextRandomVal = (): number => { @@ -294,7 +298,8 @@ async function generateChangeSdkCore( const size = (addedChangeInputs?: number, addedChangeOutputs?: number): number => { const inputCount = fixedInputs.length + r.allocatedChangeInputs.length + (addedChangeInputs || 0) const outputCount = fixedOutputs.length + r.changeOutputs.length + (addedChangeOutputs || 0) - return 4 + + return ( + 4 + varUintSize(inputCount) + fixedInputSize + (r.allocatedChangeInputs.length + (addedChangeInputs || 0)) * changeInputSize + @@ -302,6 +307,7 @@ async function generateChangeSdkCore( fixedOutputSize + (r.changeOutputs.length + (addedChangeOutputs || 0)) * changeOutputSize + 4 + ) } /** @@ -344,6 +350,7 @@ async function generateChangeSdkCore( } const addOutputToBalanceNewInput = (): boolean => { + if (surplusPoolShaping) return false if (!hasTargetNetCount) return false // Also respect the absolute cap on change output count. if (r.changeOutputs.length >= maxChangeOutputs) return false @@ -362,6 +369,7 @@ async function generateChangeSdkCore( } const addDesiredChangeOutputs = (): void => { + if (surplusPoolShaping) return // They may be removed if it turns out we can't fund them. Respect the // per-transaction cap and ensure each output meets the dust floor. while ( @@ -388,7 +396,12 @@ async function generateChangeSdkCore( const canAdd = (ao === 1 || r.changeOutputs.length === 0) && r.changeOutputs.length < maxChangeOutputs if (!canAdd) return const cap = r.changeOutputs.length === 0 ? params.changeFirstSatoshis : params.changeInitialSatoshis - const satoshis = Math.min(feeExcess(), Math.max(dustFloor, cap)) + // Account for the exact serialized fee of the output before assigning + // its value. Otherwise the output consumes the whole pre-output + // excess, leaves the plan short by its own marginal fee, and can make + // an otherwise fundable small-remainder transaction look starved. + const outputFunding = surplusPoolShaping ? feeExcess(0, 1) : feeExcess() + const satoshis = Math.min(outputFunding, Math.max(dustFloor, cap)) if (satoshis >= dustFloor) { r.changeOutputs.push({ satoshis, lockingScriptLength: params.changeLockingScriptLength }) } @@ -489,11 +502,66 @@ async function generateChangeSdkCore( throw new WERR_INSUFFICIENT_FUNDS(totalSatoshisNeeded, moreSatoshisNeeded) } + /** + * Progressively retire economically useful legacy fragments without ever + * making them necessary for the requested action. The target of zero asks + * canonical allocators for their smallest remaining output. A candidate at + * or above the preferred value is not legacy migration material and is + * immediately released. + */ + if (surplusPoolShaping && r.changeOutputs.length > 0 && targetNetCount > netChangeCount()) { + const migrationLimit = + params.maxMigrationInputs === -1 ? Number.MAX_SAFE_INTEGER : (params.maxMigrationInputs ?? 0) + for (let migrated = 0; migrated < migrationLimit; migrated++) { + const marginalInputFee = feeTarget(1) - feeTarget() + const candidate = await allocateChangeInput(0) + if (candidate == null) break + if (candidate.satoshis >= params.changeInitialSatoshis || candidate.satoshis <= marginalInputFee) { + await releaseChangeInput(candidate.outputId) + break + } + r.allocatedChangeInputs.push(candidate) + allocatedFunding += candidate.satoshis + feeExcessNow = feeExcess() + } + } + /** * Distribute the excess fees across the changeOutputs added. */ feeExcessNow = distributeExcessFees(r.changeOutputs, params.changeInitialSatoshis, feeExcessNow, rand) + /** + * Pool growth is funded only from the surplus already present in the + * transaction. Splitting one change output increases the serialized fee; + * that exact delta is deducted before assigning the new outputs. If the + * preferred minimum cannot be met, the transaction retains one smaller + * output instead of gathering more inputs or refusing an otherwise valid + * action. + */ + if (surplusPoolShaping && r.changeOutputs.length === 1 && targetNetCount > netChangeCount()) { + const original = r.changeOutputs[0] + const originalSatoshis = original.satoshis + const desiredOutputs = Math.min(maxChangeOutputs, Math.max(1, targetNetCount + r.allocatedChangeInputs.length)) + for (let count = desiredOutputs; count > 1; count--) { + const addedOutputs = count - 1 + const addedFee = feeTarget(0, addedOutputs) - feeTarget() + const distributable = originalSatoshis - addedFee + if (distributable < count * params.changeInitialSatoshis) continue + r.changeOutputs = Array.from({ length: count }, () => ({ + satoshis: params.changeInitialSatoshis, + lockingScriptLength: params.changeLockingScriptLength + })) + distributeExcessFees( + r.changeOutputs, + params.changeInitialSatoshis, + distributable - count * params.changeInitialSatoshis, + rand + ) + break + } + } + /** * Remove any change outputs that ended up below the dust floor after distribution. * Consolidates removed satoshis into the largest remaining output. @@ -594,7 +662,8 @@ export interface GenerateChangeSdkParams { /** * Maximum number of change outputs to create in this transaction. - * Defaults to `maxChangeOutputsPerTransaction` (8). + * Defaults to `maxChangeOutputsPerTransaction` (8). Set to -1 only when an + * operator deliberately wants the basket target to be the sole bound. * * Callers may override this to allow more outputs in special cases (e.g. * consolidation transactions) or fewer outputs when a compact transaction @@ -602,6 +671,19 @@ export interface GenerateChangeSdkParams { */ maxChangeOutputs?: number + /** + * When true, targetNetCount shapes only genuine post-funding surplus. The + * planner will not add inputs merely to reach the desired pool count. + */ + surplusPoolShaping?: boolean + + /** + * Soft bound on undersized, fee-positive inputs consumed after compulsory + * funding to migrate an old wallet gradually. Set to -1 for an intentionally + * unbounded migration pass. Ignored unless surplusPoolShaping is true. + */ + maxMigrationInputs?: number + randomVals?: number[] noLogging?: boolean log?: string @@ -662,6 +744,12 @@ export function validateGenerateChangeSdkParams( if (params.feeModel.model !== 'sat/kb') throw new WERR_INVALID_PARAMETER('feeModel.model', "'sat/kb'") Validation.validateOptionalInteger(params.targetNetCount, 'targetNetCount') + if (params.maxChangeOutputs !== -1) { + Validation.validateOptionalInteger(params.maxChangeOutputs, 'maxChangeOutputs', 1) + } + if (params.maxMigrationInputs !== -1) { + Validation.validateOptionalInteger(params.maxMigrationInputs, 'maxMigrationInputs', 0) + } Validation.validateSatoshis(params.changeFirstSatoshis, 'changeFirstSatoshis', 1) Validation.validateSatoshis(params.changeInitialSatoshis, 'changeInitialSatoshis', 1) diff --git a/packages/wallet/wallet-toolbox/src/storage/methods/managedChangePolicy.ts b/packages/wallet/wallet-toolbox/src/storage/methods/managedChangePolicy.ts new file mode 100644 index 000000000..a3f6e2fd8 --- /dev/null +++ b/packages/wallet/wallet-toolbox/src/storage/methods/managedChangePolicy.ts @@ -0,0 +1,86 @@ +import { WERR_INVALID_PARAMETER } from '../../sdk/WERR_errors' + +/** Historical default retained only to identify untouched wallet baskets. */ +export const LEGACY_MANAGED_CHANGE_MINIMUM_SATOSHIS = 32 + +/** + * Default liquidity policy for wallet-managed change. + * + * The preferred minimum is deliberately much larger than the dust threshold. + * Dust answers "can this output ever be spent economically?"; this value + * answers "is this output useful as an independently selectable liquidity + * unit at contemporary fee rates?". + */ +export const DEFAULT_MANAGED_CHANGE_TARGET_UTXOS = 144 +export const DEFAULT_MANAGED_CHANGE_MINIMUM_SATOSHIS = 5_000 +export const DEFAULT_MANAGED_CHANGE_MAX_OUTPUTS_PER_ACTION = 8 +export const DEFAULT_MANAGED_CHANGE_MIGRATION_INPUTS_PER_ACTION = 4 +export const DEFAULT_MANAGED_CHANGE_PENDING_COMPARISON_INPUTS = 16 + +export interface ManagedChangePolicy { + /** Maximum change outputs created by one action while growing the pool; -1 is unlimited. */ + maxOutputsPerAction: number + /** Maximum undersized, fee-positive inputs consumed only to improve the pool; -1 is unlimited. */ + migrationInputsPerAction: number + /** + * A completed-only plan above this input count is compared with pending + * alternatives using exact BEEF bytes. This is a comparison trigger, never + * a funding limit. -1 disables pending comparison until settled funding is + * actually insufficient. + */ + pendingComparisonInputs: number +} + +export type ManagedChangePolicyOptions = Partial + +export interface ManagedChangeBasketDefaults { + name: string + numberOfDesiredUTXOs: number + minimumDesiredUTXOValue: number +} + +/** True only for the exact historical default that is safe to auto-upgrade. */ +export function isLegacyManagedChangeBasketDefault ( + basket: ManagedChangeBasketDefaults +): boolean { + return basket.name === 'default' && + basket.numberOfDesiredUTXOs === DEFAULT_MANAGED_CHANGE_TARGET_UTXOS && + basket.minimumDesiredUTXOValue === LEGACY_MANAGED_CHANGE_MINIMUM_SATOSHIS +} + +/** + * Normalize a legacy default while retaining every other field and every + * operator-selected non-default value. Used by migrations, sync, and restore. + */ +export function upgradeLegacyManagedChangeBasketDefault ( + basket: T +): T { + if (!isLegacyManagedChangeBasketDefault(basket)) return basket + return { ...basket, minimumDesiredUTXOValue: DEFAULT_MANAGED_CHANGE_MINIMUM_SATOSHIS } +} + +export function defaultManagedChangePolicy (): ManagedChangePolicy { + return { + maxOutputsPerAction: DEFAULT_MANAGED_CHANGE_MAX_OUTPUTS_PER_ACTION, + migrationInputsPerAction: DEFAULT_MANAGED_CHANGE_MIGRATION_INPUTS_PER_ACTION, + pendingComparisonInputs: DEFAULT_MANAGED_CHANGE_PENDING_COMPARISON_INPUTS + } +} + +export function validateManagedChangePolicy ( + options?: ManagedChangePolicyOptions +): ManagedChangePolicy { + const policy = { ...defaultManagedChangePolicy(), ...options } + const validateLimit = (value: number, name: keyof ManagedChangePolicy, minimum: number): void => { + if (value !== -1 && (!Number.isSafeInteger(value) || value < minimum)) { + throw new WERR_INVALID_PARAMETER( + `managedChangePolicy.${name}`, + `${minimum === 0 ? 'a non-negative' : 'a positive'} safe integer or -1 for unlimited` + ) + } + } + validateLimit(policy.maxOutputsPerAction, 'maxOutputsPerAction', 1) + validateLimit(policy.migrationInputsPerAction, 'migrationInputsPerAction', 0) + validateLimit(policy.pendingComparisonInputs, 'pendingComparisonInputs', 1) + return policy +} diff --git a/packages/wallet/wallet-toolbox/src/storage/portable/index.ts b/packages/wallet/wallet-toolbox/src/storage/portable/index.ts index 9658cc6c1..f43403f71 100644 --- a/packages/wallet/wallet-toolbox/src/storage/portable/index.ts +++ b/packages/wallet/wallet-toolbox/src/storage/portable/index.ts @@ -26,6 +26,7 @@ import { import { createSyncMap, SyncMap } from '../schema/entities/EntityBase' import * as sdk from '../../sdk' import { verifyOne, verifyOneOrNone, verifyTruthy } from '../../utility/utilityHelpers' +import { upgradeLegacyManagedChangeBasketDefault } from '../methods/managedChangePolicy' type JsonPrimitive = string | number | boolean type JsonValue = JsonPrimitive | JsonValue[] | { [key: string]: JsonValue } @@ -425,7 +426,9 @@ async function restoreBRC38 (storage: StorageProvider, data: DecodedBRC38): Prom await storage.transaction(async trx => { await storage.insertUser({ ...data.user }, trx) for (const row of data.provenTxs) await storage.insertProvenTx({ ...row }, trx) - for (const row of data.outputBaskets) await storage.insertOutputBasket({ ...row }, trx) + for (const row of data.outputBaskets) { + await storage.insertOutputBasket(upgradeLegacyManagedChangeBasketDefault({ ...row }), trx) + } for (const row of data.outputTags) await storage.insertOutputTag({ ...row }, trx) for (const row of data.txLabels) await storage.insertTxLabel({ ...row }, trx) for (const row of data.transactions) await storage.insertTransaction({ ...row }, trx) diff --git a/packages/wallet/wallet-toolbox/src/storage/remoting/__test/StorageClient.test.ts b/packages/wallet/wallet-toolbox/src/storage/remoting/__test/StorageClient.test.ts index 1ca53e1c3..6408cb8f3 100644 --- a/packages/wallet/wallet-toolbox/src/storage/remoting/__test/StorageClient.test.ts +++ b/packages/wallet/wallet-toolbox/src/storage/remoting/__test/StorageClient.test.ts @@ -108,7 +108,7 @@ describe('StorageClient tests', () => { } const cr = await wallet.createAction(createArgs) - expect(cr.txid).toBe('4f428a93c43c2d120204ecdc06f7916be8a5f4542cc8839a0fd79bd1b44582f3') + expect(cr.txid).toBe('14f715d111f2ddc1783fad0213509a6626fd73fa55f58b4132d3438e26fd2c5d') const sent = await wallet.createAction({ description: 'commit repeatable action batch', options: { sendWith: [cr.txid!] } @@ -382,6 +382,12 @@ describe('StorageClient tests', () => { }) expect(spendable.filter(output => output.satoshis < 100).length).toBeGreaterThanOrEqual(50) expect(spendable.some(output => output.satoshis >= 1000)).toBe(true) + // Keep this as an extension-path fixture rather than allowing one large + // seed to carry the whole workspace. The remaining fragmented pool is + // still sufficient, but must be acquired progressively over Wallet Wire. + for (const output of spendable.filter(output => output.satoshis >= 1000)) { + await server.setup.activeStorage.updateOutput(output.outputId, { spendable: false }) + } client.wallet.randomVals = [0.1, 0.2, 0.3, 0.7, 0.8, 0.9] const extend = jest.spyOn(client.storage, 'extendActionBatch') @@ -407,7 +413,7 @@ describe('StorageClient tests', () => { description: 'commit remote fragmented batch sequence', options: { sendWith: txids, acceptDelayedBroadcast: false } }) - expect(committed.sendWithResults).toHaveLength(16) + expect(committed.sendWithResults).toHaveLength(txids.length) }) }) diff --git a/packages/wallet/wallet-toolbox/src/storage/schema/KnexMigrations.ts b/packages/wallet/wallet-toolbox/src/storage/schema/KnexMigrations.ts index a795ff0cd..a974e4b73 100644 --- a/packages/wallet/wallet-toolbox/src/storage/schema/KnexMigrations.ts +++ b/packages/wallet/wallet-toolbox/src/storage/schema/KnexMigrations.ts @@ -5,11 +5,17 @@ import { Chain } from '../../sdk/types' import { StorageKnex } from '../StorageKnex' import { WalletError } from '../../sdk/WalletError' import { WERR_NOT_IMPLEMENTED } from '../../sdk/WERR_errors' +import { + DEFAULT_MANAGED_CHANGE_MINIMUM_SATOSHIS, + DEFAULT_MANAGED_CHANGE_TARGET_UTXOS, + LEGACY_MANAGED_CHANGE_MINIMUM_SATOSHIS +} from '../methods/managedChangePolicy' export const AUTH_SESSION_MIGRATION = '2026-07-14-001 add shared auth sessions' export const MONITOR_CREATED_AT_INDEX_MIGRATION = '2026-07-14-002 add monitor created index' export const CREATE_ACTION_FUNDING_INDEX_MIGRATION = '2026-08-02-001 add createAction funding selection index' export const PAYMENT_REPLAY_MIGRATION = '2026-08-04-001 add payment replay claims' +export const MANAGED_CHANGE_POLICY_MIGRATION = '2026-08-10-001 upgrade managed change liquidity defaults' interface Migration { up: (knex: Knex) => Promise @@ -146,6 +152,27 @@ export class KnexMigrations implements MigrationSource { } } + migrations[MANAGED_CHANGE_POLICY_MIGRATION] = { + async up(knex) { + // Only the exact historical defaults identify an untouched basket. + // Operator-selected non-default values remain authoritative. + await knex('output_baskets') + .where({ + name: 'default', + numberOfDesiredUTXOs: DEFAULT_MANAGED_CHANGE_TARGET_UTXOS, + minimumDesiredUTXOValue: LEGACY_MANAGED_CHANGE_MINIMUM_SATOSHIS + }) + .update({ + minimumDesiredUTXOValue: DEFAULT_MANAGED_CHANGE_MINIMUM_SATOSHIS, + updated_at: knex.fn.now() + }) + }, + async down() { + // Intentionally irreversible. Restoring 32-satoshi liquidity units on + // rollback would actively re-fragment wallets that already migrated. + } + } + migrations['2026-07-15-001 add action batch reservations and blobs'] = { async up(knex) { const dbtype = await determineDBType(knex) @@ -494,8 +521,8 @@ export class KnexMigrations implements MigrationSource { table.increments('basketId') table.integer('userId').unsigned().references('userId').inTable('users').notNullable() table.string('name', 300).notNullable() - table.integer('numberOfDesiredUTXOs', 6).defaultTo(6).notNullable() - table.integer('minimumDesiredUTXOValue', 15).defaultTo(10000).notNullable() + table.integer('numberOfDesiredUTXOs', 6).defaultTo(DEFAULT_MANAGED_CHANGE_TARGET_UTXOS).notNullable() + table.integer('minimumDesiredUTXOValue', 15).defaultTo(DEFAULT_MANAGED_CHANGE_MINIMUM_SATOSHIS).notNullable() table.boolean('isDeleted').notNullable().defaultTo(false) table.unique(['name', 'userId']) }) diff --git a/packages/wallet/wallet-toolbox/src/storage/schema/entities/EntityOutputBasket.ts b/packages/wallet/wallet-toolbox/src/storage/schema/entities/EntityOutputBasket.ts index bbe6ae142..3d3148450 100644 --- a/packages/wallet/wallet-toolbox/src/storage/schema/entities/EntityOutputBasket.ts +++ b/packages/wallet/wallet-toolbox/src/storage/schema/entities/EntityOutputBasket.ts @@ -1,6 +1,7 @@ import { TrxToken } from '../../../sdk/WalletStorage.interfaces' import { verifyId, verifyOneOrNone } from '../../../utility/utilityHelpers' import { TableOutputBasket } from '../tables/TableOutputBasket' +import { upgradeLegacyManagedChangeBasketDefault } from '../../methods/managedChangePolicy' import { EntityBase, EntityStorage, SyncMap } from './EntityBase' export class EntityOutputBasket extends EntityBase { @@ -144,6 +145,7 @@ export class EntityOutputBasket extends EntityBase { this.userId = userId this.name ||= 'default' this.basketId = 0 + this.api = upgradeLegacyManagedChangeBasketDefault(this.api) this.basketId = await storage.insertOutputBasket(this.toApi(), trx) } @@ -158,8 +160,9 @@ export class EntityOutputBasket extends EntityBase { let wasMerged = false if (ei.updated_at > this.updated_at) { // basket name is its identity, should not change - this.minimumDesiredUTXOValue = ei.minimumDesiredUTXOValue - this.numberOfDesiredUTXOs = ei.numberOfDesiredUTXOs + const incoming = upgradeLegacyManagedChangeBasketDefault(ei) + this.minimumDesiredUTXOValue = incoming.minimumDesiredUTXOValue + this.numberOfDesiredUTXOs = incoming.numberOfDesiredUTXOs this.isDeleted = ei.isDeleted this.updated_at = new Date(Math.max(ei.updated_at.getTime(), this.updated_at.getTime())) await storage.updateOutputBasket(this.id, this.toApi(), trx) diff --git a/packages/wallet/wallet-toolbox/src/storage/schema/entities/__tests/OutputBasketTests.test.ts b/packages/wallet/wallet-toolbox/src/storage/schema/entities/__tests/OutputBasketTests.test.ts index 1170bd0bb..bc466ffd3 100644 --- a/packages/wallet/wallet-toolbox/src/storage/schema/entities/__tests/OutputBasketTests.test.ts +++ b/packages/wallet/wallet-toolbox/src/storage/schema/entities/__tests/OutputBasketTests.test.ts @@ -149,6 +149,33 @@ describe('OutputBasket class method tests', () => { expect(updatedRecord[0].isDeleted).toBe(false) }) + test('sync cannot reintroduce the exact legacy 32-satoshi default', async () => { + const ctx = ctxs[0] + const current = ( + await ctx.activeStorage.findOutputBaskets({ + partial: { userId: ctx.userId, name: 'default' } + }) + )[0] + expect(current).toBeDefined() + const entity = new EntityOutputBasket({ ...current, minimumDesiredUTXOValue: 5_000 }) + const incoming: TableOutputBasket = { + ...current, + updated_at: new Date(current.updated_at.getTime() + 1_000), + numberOfDesiredUTXOs: 144, + minimumDesiredUTXOValue: 32 + } + + await expect(entity.mergeExisting(ctx.activeStorage, undefined, incoming, createSyncMap())).resolves.toBe(true) + + expect(entity.minimumDesiredUTXOValue).toBe(5_000) + const stored = ( + await ctx.activeStorage.findOutputBaskets({ + partial: { basketId: current.basketId } + }) + )[0] + expect(stored.minimumDesiredUTXOValue).toBe(5_000) + }) + test('equals identifies matching entities with and without SyncMap', async () => { const ctx = ctxs[0] diff --git a/packages/wallet/wallet-toolbox/test/storage/KnexMigrations.test.ts b/packages/wallet/wallet-toolbox/test/storage/KnexMigrations.test.ts index 94abc9a43..dbc62b520 100644 --- a/packages/wallet/wallet-toolbox/test/storage/KnexMigrations.test.ts +++ b/packages/wallet/wallet-toolbox/test/storage/KnexMigrations.test.ts @@ -3,6 +3,7 @@ import { AUTH_SESSION_MIGRATION, CREATE_ACTION_FUNDING_INDEX_MIGRATION, KnexMigrations, + MANAGED_CHANGE_POLICY_MIGRATION, MONITOR_CREATED_AT_INDEX_MIGRATION, StorageKnex, wait @@ -205,6 +206,40 @@ describe('KnexMigrations tests', () => { } }) + test('5a upgrades only exact untouched managed-change defaults', async () => { + const localSQLiteFile = await _tu.newTmpFile('migratemanagedchange.sqlite', false, false, false) + const knex = _tu.createLocalSQLite(localSQLiteFile) + + try { + await knex.schema.createTable('output_baskets', table => { + table.increments('basketId') + table.integer('userId').notNullable() + table.string('name').notNullable() + table.integer('numberOfDesiredUTXOs').notNullable() + table.bigInteger('minimumDesiredUTXOValue').notNullable() + table.timestamp('updated_at').notNullable().defaultTo(knex.fn.now()) + }) + await knex('output_baskets').insert([ + { userId: 1, name: 'default', numberOfDesiredUTXOs: 144, minimumDesiredUTXOValue: 32 }, + { userId: 2, name: 'default', numberOfDesiredUTXOs: 144, minimumDesiredUTXOValue: 64 }, + { userId: 3, name: 'default', numberOfDesiredUTXOs: 100, minimumDesiredUTXOValue: 32 }, + { userId: 4, name: 'application basket', numberOfDesiredUTXOs: 144, minimumDesiredUTXOValue: 32 } + ]) + const source = new KnexMigrations('test', 'managed change migration test', '1'.repeat(64), 1000) + const migration = await source.getMigration(MANAGED_CHANGE_POLICY_MIGRATION) + + await migration.up(knex) + + const rows = await knex('output_baskets').orderBy('userId') + expect(rows.map(row => Number(row.minimumDesiredUTXOValue))).toEqual([5_000, 64, 32, 32]) + await migration.down?.(knex) + const afterDown = await knex('output_baskets').orderBy('userId') + expect(afterDown.map(row => Number(row.minimumDesiredUTXOValue))).toEqual([5_000, 64, 32, 32]) + } finally { + await knex.destroy() + } + }) + test.each([ { migrationName: '2026-02-27-001 add listOutputs path indexes', diff --git a/packages/wallet/wallet-toolbox/test/storage/portable.test.ts b/packages/wallet/wallet-toolbox/test/storage/portable.test.ts index 9decb5422..881d85239 100644 --- a/packages/wallet/wallet-toolbox/test/storage/portable.test.ts +++ b/packages/wallet/wallet-toolbox/test/storage/portable.test.ts @@ -113,6 +113,39 @@ describe('BRC-38/39 portable wallet data', () => { await expect(importBRC38(target, document, { mode: 'restore' })).rejects.toThrow(/empty target storage/) }) + test('normalizes an exact legacy managed-change default during BRC-38 restore', async () => { + const document = minimalDocument() + document.tables.outputBaskets.push( + { + created_at: iso, + updated_at: iso, + basketId: 1, + userId: document.user.userId, + name: 'default', + numberOfDesiredUTXOs: 144, + minimumDesiredUTXOValue: 32, + isDeleted: false + }, + { + created_at: iso, + updated_at: iso, + basketId: 2, + userId: document.user.userId, + name: 'custom legacy-shaped basket', + numberOfDesiredUTXOs: 144, + minimumDesiredUTXOValue: 32, + isDeleted: false + } + ) + const target = await createEmptyStorage('portable_restore_managed_change') + + await importBRC38(target, document, { mode: 'restore' }) + + const baskets = await target.findOutputBaskets({ partial: { userId: document.user.userId as number } }) + expect(baskets.find(basket => basket.name === 'default')?.minimumDesiredUTXOValue).toBe(5_000) + expect(baskets.find(basket => basket.name === 'custom legacy-shaped basket')?.minimumDesiredUTXOValue).toBe(32) + }) + test('merges BRC-38 into non-empty SQLite storage with ID and sync-map remapping', async () => { const rootKeyHex = '3'.repeat(64) const source = await createPortableSource('portable_merge_source', rootKeyHex) diff --git a/packages/wallet/wallet-toolbox/test/wallet/action/createAction.test.ts b/packages/wallet/wallet-toolbox/test/wallet/action/createAction.test.ts index 88c547eea..b9bb47536 100644 --- a/packages/wallet/wallet-toolbox/test/wallet/action/createAction.test.ts +++ b/packages/wallet/wallet-toolbox/test/wallet/action/createAction.test.ts @@ -100,7 +100,7 @@ describe('createAction test', () => { } const cr = await wallet.createAction(createArgs) - expect(cr.txid).toBe('4f428a93c43c2d120204ecdc06f7916be8a5f4542cc8839a0fd79bd1b44582f3') + expect(cr.txid).toBe('14f715d111f2ddc1783fad0213509a6626fd73fa55f58b4132d3438e26fd2c5d') } }) diff --git a/packages/wallet/wallet-toolbox/test/wallet/action/createAction2.test.ts b/packages/wallet/wallet-toolbox/test/wallet/action/createAction2.test.ts index f118e5593..915cda2a3 100644 --- a/packages/wallet/wallet-toolbox/test/wallet/action/createAction2.test.ts +++ b/packages/wallet/wallet-toolbox/test/wallet/action/createAction2.test.ts @@ -83,18 +83,18 @@ describe('createAction2 nosend transactions', () => { includeLabels: true }) const rl1 = toLogString(fundingResult.tx!, actionsResult) - expect(rl1.log).toBe(`transactions:3 - txid:30bdac0f5c6491f130820517802ff57e20e5a50c08b5c65e6976627fb82ae930 version:1 lockTime:0 sats:-4 status:nosend + expect(rl1.log).toBe(`transactions:2 + txid:0bf6453843c29d6df1b9e0549587c696d2aaa8340f18be056c4711d853b8369a version:1 lockTime:0 sats:-4 status:nosend${' '} outgoing:true desc:'Funding transaction' labels:['funding transaction for createaction','this is an extra long test label that should be truncated at 80 chars when it is...'] inputs: 1 - 0: sourceTXID:a3a8fe7f541c1383ff7b975af49b27284ae720af5f2705d8409baaf519190d26.2 sats:913 - lock:(50)76a914f7238871139f4926cbd592a03a737981e558245d88ac - unlock:(214)483045022100cfef1f6d781af99a1de14efd6f24f2a14234a26097012f27121eb36f4e330c1d0220... seq:4294967295 + 0: sourceTXID:527ffe88f70d5b7de2b8b5ba9966b9c755e7da4de749d4fcd27140a03145a11d.0 sats:995${' '} + lock:(50)76a914ab2b66432503a3681fc5af1502207ca458c8752d88ac${' '} + unlock:(214)483045022100f8ea8705c0c6253032481f194f5ccb43d3b751650c51d603b7ee19f910abf37b0220... seq:4294967295 outputs: 2 0: sats:3 lock:(48)76a914abcdef0123456789abcdef0123456789abcdef88ac index:0 spendable:true basket:'funding basket' desc:'Funding Output' tags:['funding transaction output','test tag'] - 1: sats:909 lock:(50)76a9145947e66cdd43c70fb1780116b79e6f7d96e30e0888ac index:1 spendable:true basket:'default'`) + 1: sats:991 lock:(50)76a9145947e66cdd43c70fb1780116b79e6f7d96e30e0888ac index:1 spendable:true basket:'default'`) } }) @@ -174,20 +174,20 @@ describe('createAction2 nosend transactions', () => { includeLabels: true }) const rl1 = toLogString(fundingResult.tx!, actionsResult) - expect(rl1.log).toBe(`transactions:3 - txid:b3848f2cabf5887ec679ca60347a29f6ecad425fda738700265c2f9d22c18ab5 version:1 lockTime:0 sats:-12 status:nosend + expect(rl1.log).toBe(`transactions:2 + txid:a03789724ac10b32365b84cbcbee24ec5ef0a1a8d465112603a4e2219f4c952e version:1 lockTime:0 sats:-12 status:nosend${' '} outgoing:true desc:'Funding transaction with multiple outputs' labels:['funding transaction for createaction','this is the extra label'] inputs: 1 - 0: sourceTXID:a3a8fe7f541c1383ff7b975af49b27284ae720af5f2705d8409baaf519190d26.2 sats:913 - lock:(50)76a914f7238871139f4926cbd592a03a737981e558245d88ac - unlock:(212)473044022079020cc8ea5ee6b3610806286e41567147d4b4b07d16bc1341311e00ce7647b0022034... seq:4294967295 + 0: sourceTXID:527ffe88f70d5b7de2b8b5ba9966b9c755e7da4de749d4fcd27140a03145a11d.0 sats:995${' '} + lock:(50)76a914ab2b66432503a3681fc5af1502207ca458c8752d88ac${' '} + unlock:(214)483045022100ffa66a41bf8c3f7ffc9699c067ab0248ba7de3835afd48739af4107d1d8dea030220... seq:4294967295 outputs: 3 0: sats:5 lock:(48)76a914abcdef0123456789abcdef0123456789abcdef88ac index:0 spendable:true basket:'funding basket' desc:'Funding output' tags:['funding transaction for createaction','test tag'] 1: sats:6 lock:(48)76a914fedcba9876543210fedcba9876543210fedcba88ac index:1 spendable:true basket:'extra basket' desc:'Extra Output' tags:['extra transaction output','extra test tag'] - 2: sats:901 lock:(50)76a9145947e66cdd43c70fb1780116b79e6f7d96e30e0888ac index:2 spendable:true basket:'default'`) + 2: sats:983 lock:(50)76a9145947e66cdd43c70fb1780116b79e6f7d96e30e0888ac index:2 spendable:true basket:'default'`) } }) @@ -330,16 +330,16 @@ describe('createAction2 nosend transactions', () => { .replace(/,'reference \n\s+[0-9a-f]+'/, '') .replace(/[ \t]+$/gm, '') expect(stableLog).toBe(`transactions:2 - txid:471c15662c437ce5765d467eabfdd67adfea5c569b2da520b2d8a09fbb868370 version:1 lockTime:0 sats:-5 status:nosend + txid:d6e2d06ad92ba812da1bc89e493f620f95234d4947997b396cd4a50ba9ef4c1b version:1 lockTime:0 sats:-5 status:nosend outgoing:true desc:'Check knownTxids and returnTXIDOnly' labels:['custom options test'] inputs: 1 - 0: sourceTXID:63a159d422ba29db52b728b05c30cd181300984bf1ea4dc2354f1d1870fbe016.1 sats:996 - lock:(50)76a914fbc3ac7e96362b6f0d7bedc217568901488e5ad788ac - unlock:(212)47304402201d0f8f1802d33937454e372bc6978ddbad22c95a4442a6c14b23d06f60f96f7d02205e... seq:4294967295 + 0: sourceTXID:63dc5420a3e898dd16163c48ed6c338e6a59832b7c3bf9d9d227725ca5bffdf1.17 sats:1001 + lock:(50)76a9141a32c1c07dd4f9c632ce6b43dd28c8b27a37d81588ac + unlock:(214)4830450221009b7a811b7aa80ec95cffba3f37c8689e61d5939a9ae7eb8a969cb5f4ff7436040220... seq:4294967295 outputs: 2 0: sats:4 lock:(48)76a914abcdef0123456789abcdef0123456789abcdef88ac index:0 spendable:true desc:'returnTXIDOnly false test' - 1: sats:991 lock:(50)76a9145947e66cdd43c70fb1780116b79e6f7d96e30e0888ac index:1 spendable:true basket:'default'`) + 1: sats:996 lock:(50)76a9145947e66cdd43c70fb1780116b79e6f7d96e30e0888ac index:1 spendable:true basket:'default'`) } }) diff --git a/scripts/patch-coverage.mjs b/scripts/patch-coverage.mjs index 5b4c301f6..7a24dadaf 100644 --- a/scripts/patch-coverage.mjs +++ b/scripts/patch-coverage.mjs @@ -39,7 +39,11 @@ const EXCLUDED_SOURCE_PATTERNS = [ // is a CLI that reads `process.argv` and branches on it, and excluding it by // shape would quietly drop real code out of this gate. /packages\/overlays\/topics\/src\/index\.ts$/, - /packages\/overlays\/topics\/src\/uoradpp\/types\.ts$/ + /packages\/overlays\/topics\/src\/uoradpp\/types\.ts$/, + // SetupWallet is declarations only, while the mobile storage entry point is + // a pure re-export barrel. Neither emits executable statements for LCOV. + /packages\/wallet\/wallet-toolbox\/src\/SetupWallet\.ts$/, + /packages\/wallet\/wallet-toolbox\/src\/storage\/index\.mobile\.ts$/ ] function normalizedPath(value) { diff --git a/scripts/patch-coverage.test.mjs b/scripts/patch-coverage.test.mjs index d81b47da9..273a2e347 100644 --- a/scripts/patch-coverage.test.mjs +++ b/scripts/patch-coverage.test.mjs @@ -97,6 +97,12 @@ diff --git a/packages/wallet/wallet-toolbox/src/storage/schema/StorageIdbSchema. diff --git a/packages/wallet/wallet-toolbox/src/sdk/WalletStorage.interfaces.ts b/packages/wallet/wallet-toolbox/src/sdk/WalletStorage.interfaces.ts +++ b/packages/wallet/wallet-toolbox/src/sdk/WalletStorage.interfaces.ts @@ -0,0 +1,12 @@ +diff --git a/packages/wallet/wallet-toolbox/src/SetupWallet.ts b/packages/wallet/wallet-toolbox/src/SetupWallet.ts ++++ b/packages/wallet/wallet-toolbox/src/SetupWallet.ts +@@ -0,0 +1,12 @@ +diff --git a/packages/wallet/wallet-toolbox/src/storage/index.mobile.ts b/packages/wallet/wallet-toolbox/src/storage/index.mobile.ts ++++ b/packages/wallet/wallet-toolbox/src/storage/index.mobile.ts +@@ -0,0 +1,12 @@ diff --git a/packages/helpers/example/src/index.ts b/packages/helpers/example/src/index.ts +++ b/packages/helpers/example/src/index.ts @@ -0,0 +1 @@ From ed5443664ca614553909d23a1c55b8727dfcb9d7 Mon Sep 17 00:00:00 2001 From: Ty J Everett Date: Mon, 10 Aug 2026 18:48:52 -0700 Subject: [PATCH 2/7] refactor(wallet): simplify liquidity planning --- .../src/storage/methods/createAction.ts | 170 +++++++++++------- .../src/storage/methods/generateChange.ts | 124 +++++++++---- 2 files changed, 194 insertions(+), 100 deletions(-) diff --git a/packages/wallet/wallet-toolbox/src/storage/methods/createAction.ts b/packages/wallet/wallet-toolbox/src/storage/methods/createAction.ts index 3a8dbc4db..afbec86d6 100644 --- a/packages/wallet/wallet-toolbox/src/storage/methods/createAction.ts +++ b/packages/wallet/wallet-toolbox/src/storage/methods/createAction.ts @@ -1074,16 +1074,19 @@ async function traceStorageStep( ) } -function makeFundingParams( - storage: StorageProvider, - vargs: Validation.ValidCreateActionArgs, - xinputs: XValidCreateActionInput[], - xoutputs: XValidCreateActionOutput[], - changeBasket: TableOutputBasket, - feeModel: StorageFeeModel, - healthyChangeCount: number, - compatibilityFallback = false -): GenerateChangeSdkParams { +interface MakeFundingParamsArgs { + storage: StorageProvider + vargs: Validation.ValidCreateActionArgs + xinputs: XValidCreateActionInput[] + xoutputs: XValidCreateActionOutput[] + changeBasket: TableOutputBasket + feeModel: StorageFeeModel + healthyChangeCount: number + compatibilityFallback: boolean +} + +function makeFundingParams(args: MakeFundingParamsArgs): GenerateChangeSdkParams { + const { storage, vargs, xinputs, xoutputs, changeBasket, feeModel, healthyChangeCount, compatibilityFallback } = args const preferredSatoshis = Math.max(1, changeBasket.minimumDesiredUTXOValue) return { fixedInputs: xinputs.map(input => ({ @@ -1126,11 +1129,11 @@ async function buildFundingPlan( ) const preferredSatoshis = Math.max(1, changeBasket.minimumDesiredUTXOValue) const healthyChangeCount = compatibilityFallback - // Preserve the legacy target-count input exactly, including noSendChange - // that is removed from the allocator below but still belongs to this tier. - ? candidates.filter(output => eligibleStatuses.includes(output.transactionStatus)).length + ? // Preserve the legacy target-count input exactly, including noSendChange + // that is removed from the allocator below but still belongs to this tier. + candidates.filter(output => eligibleStatuses.includes(output.transactionStatus)).length : candidates.filter(output => output.satoshis >= preferredSatoshis).length - const params = makeFundingParams( + const params = makeFundingParams({ storage, vargs, xinputs, @@ -1139,7 +1142,7 @@ async function buildFundingPlan( feeModel, healthyChangeCount, compatibilityFallback - ) + }) const noSendStatuses = await storage.findTransactionStatusesByIds( userId, noSendChangeIn.map(output => output.transactionId) @@ -1230,20 +1233,31 @@ async function fundingPlanSerializedCost( } } -async function prepareFundingPlanWithLiquidityPolicy( +interface FundingTier { + policyTier: Exclude + statuses: TransactionStatus[] +} + +const FUNDING_TIERS: FundingTier[] = [ + { policyTier: 'completed', statuses: ['completed'] }, + { policyTier: 'unproven', statuses: ['completed', 'unproven'] }, + { policyTier: 'sending', statuses: ['completed', 'unproven', 'sending'] } +] + +async function resolveFundingCandidates( storage: StorageProvider, - context: FundingPlanBaseContext, + userId: number, + basketId: number, parent?: TelemetrySpan, trx?: TrxToken -): Promise { - const [userId, vargs, , , changeBasket] = context +): Promise { const rawCandidates = await traceStorageStep( storage, 'wallet.storage.create_action.funding_candidates', parent, { 'funding.include_pending': true }, async span => { - const outputs = await storage.findAvailableManagedChangeInputCandidates(userId, changeBasket.basketId, false, trx) + const outputs = await storage.findAvailableManagedChangeInputCandidates(userId, basketId, false, trx) span?.end({ attributes: { 'funding.candidate_count': outputs.length, @@ -1263,64 +1277,88 @@ async function prepareFundingPlanWithLiquidityPolicy( const missingStatuses = await storage.findTransactionStatusesByIds(userId, missingStatusIds, trx) const candidates: ResolvedManagedChangeInputCandidate[] = rawCandidates.map(output => ({ ...output, - transactionStatus: output.transactionStatus ?? missingStatuses.get(output.transactionId) as TransactionStatus + transactionStatus: output.transactionStatus ?? (missingStatuses.get(output.transactionId) as TransactionStatus) })) if (candidates.some(output => output.transactionStatus == null)) { throw new WERR_INTERNAL('managed change candidate is missing its source transaction status') } + return candidates +} - const tiers: Array<{ - policyTier: Exclude - statuses: TransactionStatus[] - }> = [ - { policyTier: 'completed', statuses: ['completed'] }, - { policyTier: 'unproven', statuses: ['completed', 'unproven'] }, - { policyTier: 'sending', statuses: ['completed', 'unproven', 'sending'] } - ] - const successful: PreparedFundingPlan[] = [] - let fundingError: unknown - for (const tier of tiers) { - let plan: PreparedFundingPlan | undefined - try { - plan = await buildFundingPlan(storage, context, candidates, tier.statuses, tier.policyTier, false, parent) - } catch (error) { - if (!(error instanceof WERR_INSUFFICIENT_FUNDS)) throw error - fundingError = error - // Pool shaping is optional. Before widening ancestry to unproven or - // sending parents, retry the same status tier with the former funding - // shape. This is the one-way compatibility guarantee: a preferred - // minimum can never manufacture starvation or force a pending chain. - try { - plan = await buildFundingPlan(storage, context, candidates, tier.statuses, 'compatibility', true, parent) - } catch (compatibilityError) { - if (!(compatibilityError instanceof WERR_INSUFFICIENT_FUNDS)) throw compatibilityError - fundingError = compatibilityError - } +async function buildFundingPlanForTier( + storage: StorageProvider, + context: FundingPlanBaseContext, + candidates: ResolvedManagedChangeInputCandidate[], + tier: FundingTier, + parent?: TelemetrySpan +): Promise<{ plan?: PreparedFundingPlan; fundingError?: WERR_INSUFFICIENT_FUNDS }> { + try { + return { + plan: await buildFundingPlan(storage, context, candidates, tier.statuses, tier.policyTier, false, parent) + } + } catch (error) { + if (!(error instanceof WERR_INSUFFICIENT_FUNDS)) throw error + } + + // Pool shaping is optional. Before widening ancestry to unproven or sending + // parents, retry the same status tier with the former funding shape. This is + // the one-way compatibility guarantee: a preferred minimum can never + // manufacture starvation or force a pending chain. + try { + return { + plan: await buildFundingPlan(storage, context, candidates, tier.statuses, 'compatibility', true, parent) + } + } catch (error) { + if (!(error instanceof WERR_INSUFFICIENT_FUNDS)) throw error + return { fundingError: error } + } +} + +function firstPlanNeedsNoPendingComparison(storage: StorageProvider, plan: PreparedFundingPlan): boolean { + const threshold = storage.managedChangePolicy.pendingComparisonInputs + return threshold === -1 || plan.selected.length <= threshold +} + +async function chooseLowestSerializedFundingPlan( + storage: StorageProvider, + plans: PreparedFundingPlan[], + knownTxids: string[] +): Promise { + let chosen = plans[0] + let chosenCost = await fundingPlanSerializedCost(storage, chosen, knownTxids) + for (const alternative of plans.slice(1)) { + const cost = await fundingPlanSerializedCost(storage, alternative, knownTxids) + if (cost < chosenCost) { + chosen = alternative + chosenCost = cost } + } + return chosen +} + +async function prepareFundingPlanWithLiquidityPolicy( + storage: StorageProvider, + context: FundingPlanBaseContext, + parent?: TelemetrySpan, + trx?: TrxToken +): Promise { + const [userId, vargs, , , changeBasket] = context + const candidates = await resolveFundingCandidates(storage, userId, changeBasket.basketId, parent, trx) + const successful: PreparedFundingPlan[] = [] + let fundingError: WERR_INSUFFICIENT_FUNDS | undefined + for (const tier of FUNDING_TIERS) { + const attempted = await buildFundingPlanForTier(storage, context, candidates, tier, parent) + fundingError = attempted.fundingError ?? fundingError + const plan = attempted.plan if (plan != null) { successful.push(plan) - if ( - successful.length === 1 && - (storage.managedChangePolicy.pendingComparisonInputs === -1 || - plan.selected.length <= storage.managedChangePolicy.pendingComparisonInputs) - ) - return plan + if (successful.length === 1 && firstPlanNeedsNoPendingComparison(storage, plan)) return plan } } if (successful.length > 0) { - const baseline = successful[0] - if (successful.length === 1) return baseline - let chosen = baseline - let chosenCost = await fundingPlanSerializedCost(storage, baseline, vargs.options.knownTxids) - for (const alternative of successful.slice(1)) { - const cost = await fundingPlanSerializedCost(storage, alternative, vargs.options.knownTxids) - if (cost < chosenCost) { - chosen = alternative - chosenCost = cost - } - } - return chosen + if (successful.length === 1) return successful[0] + return await chooseLowestSerializedFundingPlan(storage, successful, vargs.options.knownTxids) } if (fundingError != null) throw fundingError diff --git a/packages/wallet/wallet-toolbox/src/storage/methods/generateChange.ts b/packages/wallet/wallet-toolbox/src/storage/methods/generateChange.ts index fc07cf473..afddce5a7 100644 --- a/packages/wallet/wallet-toolbox/src/storage/methods/generateChange.ts +++ b/packages/wallet/wallet-toolbox/src/storage/methods/generateChange.ts @@ -101,6 +101,77 @@ function removeDustOutputs(changeOutputs: GenerateChangeSdkChangeOutput[], dustF } } +interface LegacyChangeMigrationRequest { + params: GenerateChangeSdkParams + result: GenerateChangeSdkResult + targetNetCount: number + netChangeCount: () => number + feeTarget: (addedChangeInputs?: number, addedChangeOutputs?: number) => number + allocateChangeInput: ( + targetSatoshis: number, + exactSatoshis?: number + ) => Promise + releaseChangeInput: (outputId: number) => Promise + recordAllocatedInput: (candidate: GenerateChangeSdkChangeInput) => void +} + +async function migrateLegacyChangeInputs(request: LegacyChangeMigrationRequest): Promise { + const { params, result } = request + if (!params.surplusPoolShaping || result.changeOutputs.length === 0) return + if (request.targetNetCount <= request.netChangeCount()) return + + const migrationLimit = params.maxMigrationInputs === -1 ? Number.MAX_SAFE_INTEGER : (params.maxMigrationInputs ?? 0) + for (let migrated = 0; migrated < migrationLimit; migrated++) { + const marginalInputFee = request.feeTarget(1) - request.feeTarget() + const candidate = await request.allocateChangeInput(0) + if (candidate == null) break + if (candidate.satoshis >= params.changeInitialSatoshis || candidate.satoshis <= marginalInputFee) { + await request.releaseChangeInput(candidate.outputId) + break + } + request.recordAllocatedInput(candidate) + } +} + +interface SurplusChangeShapingRequest { + params: GenerateChangeSdkParams + result: GenerateChangeSdkResult + targetNetCount: number + netChangeCount: () => number + maxChangeOutputs: number + feeTarget: (addedChangeInputs?: number, addedChangeOutputs?: number) => number + rand: (min: number, max: number) => number +} + +function shapeSurplusChangeOutputs(request: SurplusChangeShapingRequest): void { + const { params, result } = request + if (!params.surplusPoolShaping || result.changeOutputs.length !== 1) return + if (request.targetNetCount <= request.netChangeCount()) return + + const originalSatoshis = result.changeOutputs[0].satoshis + const desiredOutputs = Math.min( + request.maxChangeOutputs, + Math.max(1, request.targetNetCount + result.allocatedChangeInputs.length) + ) + for (let count = desiredOutputs; count > 1; count--) { + const addedOutputs = count - 1 + const addedFee = request.feeTarget(0, addedOutputs) - request.feeTarget() + const distributable = originalSatoshis - addedFee + if (distributable < count * params.changeInitialSatoshis) continue + result.changeOutputs = Array.from({ length: count }, () => ({ + satoshis: params.changeInitialSatoshis, + lockingScriptLength: params.changeLockingScriptLength + })) + distributeExcessFees( + result.changeOutputs, + params.changeInitialSatoshis, + distributable - count * params.changeInitialSatoshis, + request.rand + ) + break + } +} + /** * Simplifications: * - only support one change type with fixed length scripts. @@ -509,22 +580,20 @@ async function generateChangeSdkCore( * or above the preferred value is not legacy migration material and is * immediately released. */ - if (surplusPoolShaping && r.changeOutputs.length > 0 && targetNetCount > netChangeCount()) { - const migrationLimit = - params.maxMigrationInputs === -1 ? Number.MAX_SAFE_INTEGER : (params.maxMigrationInputs ?? 0) - for (let migrated = 0; migrated < migrationLimit; migrated++) { - const marginalInputFee = feeTarget(1) - feeTarget() - const candidate = await allocateChangeInput(0) - if (candidate == null) break - if (candidate.satoshis >= params.changeInitialSatoshis || candidate.satoshis <= marginalInputFee) { - await releaseChangeInput(candidate.outputId) - break - } + await migrateLegacyChangeInputs({ + params, + result: r, + targetNetCount, + netChangeCount, + feeTarget, + allocateChangeInput, + releaseChangeInput, + recordAllocatedInput: candidate => { r.allocatedChangeInputs.push(candidate) allocatedFunding += candidate.satoshis feeExcessNow = feeExcess() } - } + }) /** * Distribute the excess fees across the changeOutputs added. @@ -539,28 +608,15 @@ async function generateChangeSdkCore( * output instead of gathering more inputs or refusing an otherwise valid * action. */ - if (surplusPoolShaping && r.changeOutputs.length === 1 && targetNetCount > netChangeCount()) { - const original = r.changeOutputs[0] - const originalSatoshis = original.satoshis - const desiredOutputs = Math.min(maxChangeOutputs, Math.max(1, targetNetCount + r.allocatedChangeInputs.length)) - for (let count = desiredOutputs; count > 1; count--) { - const addedOutputs = count - 1 - const addedFee = feeTarget(0, addedOutputs) - feeTarget() - const distributable = originalSatoshis - addedFee - if (distributable < count * params.changeInitialSatoshis) continue - r.changeOutputs = Array.from({ length: count }, () => ({ - satoshis: params.changeInitialSatoshis, - lockingScriptLength: params.changeLockingScriptLength - })) - distributeExcessFees( - r.changeOutputs, - params.changeInitialSatoshis, - distributable - count * params.changeInitialSatoshis, - rand - ) - break - } - } + shapeSurplusChangeOutputs({ + params, + result: r, + targetNetCount, + netChangeCount, + maxChangeOutputs, + feeTarget, + rand + }) /** * Remove any change outputs that ended up below the dust floor after distribution. From cfccab9cb6a420accf29e6d3f61c93d7cc85d64e Mon Sep 17 00:00:00 2001 From: Ty J Everett Date: Mon, 10 Aug 2026 18:59:35 -0700 Subject: [PATCH 3/7] chore(wallet): record platform size budgets --- packages/wallet/wallet-toolbox/CHANGELOG.md | 16 +++++++++------- .../wallet-toolbox/client/platform-budget.json | 12 ++++++------ .../wallet-toolbox/mobile/platform-budget.json | 2 +- 3 files changed, 16 insertions(+), 14 deletions(-) diff --git a/packages/wallet/wallet-toolbox/CHANGELOG.md b/packages/wallet/wallet-toolbox/CHANGELOG.md index c9507951a..ed87f5050 100644 --- a/packages/wallet/wallet-toolbox/CHANGELOG.md +++ b/packages/wallet/wallet-toolbox/CHANGELOG.md @@ -26,13 +26,15 @@ attention to changes that materially alter behavior or extend functionality. expired leases, structured lifecycle errors, and a provider-enforced cumulative reservation limit that defaults to 256 outputs and can be configured, including `-1` for operator-selected unlimited operation. -- Keep the resumable lifecycle's browser cost bounded: the retained platform - contract measures 1,548,179 raw / 364,550 gzip / 285,697 Brotli bytes with - Vite and 1,209,217 raw / 331,318 gzip / 267,001 Brotli bytes with esbuild; - every browser ceiling remains unchanged. Mobile measures 1,609,633 Metro - bytes and 3,253,366 raw Hermes bytes; only the Hermes raw ceiling advances, - by less than 0.15%, while every compressed and Metro ceiling remains - unchanged. +- Keep the combined action-batch and managed-liquidity browser/mobile cost + bounded and measured from exact packed artifacts. The browser contract now + measures 1,557,196 raw / 365,526 gzip / 287,329 Brotli bytes with Vite and + 1,216,272 raw / 334,453 gzip / 268,547 Brotli bytes with esbuild. Linux CI + observed 1,558,352 Vite raw bytes; the reviewed ceilings advance by at most + 1.0% and retain narrow headroom. Mobile measures 1,616,960 Metro bytes and + 3,266,887 raw Hermes bytes locally; Linux CI observed 3,271,396 raw Hermes + bytes. The Metro and compressed-mobile ceilings remain unchanged, while the + Hermes raw ceiling advances by 0.61% from the pre-liquidity-policy value. - Fix `WalletStorageManager.getStoreEndpointURL` / `getStores().endpointURL` to duck-type provider `endpointUrl` instead of matching `constructor.name === 'StorageClient'`. Production minifiers rename classes, diff --git a/packages/wallet/wallet-toolbox/client/platform-budget.json b/packages/wallet/wallet-toolbox/client/platform-budget.json index 54cc8a702..e9d7f3933 100644 --- a/packages/wallet/wallet-toolbox/client/platform-budget.json +++ b/packages/wallet/wallet-toolbox/client/platform-budget.json @@ -2,14 +2,14 @@ "profile": "browser", "maximumBytes": { "vite": { - "raw": 1550000, - "gzip": 365000, - "brotli": 286000 + "raw": 1560000, + "gzip": 367000, + "brotli": 289000 }, "esbuild": { - "raw": 1210000, - "gzip": 333000, - "brotli": 268000 + "raw": 1218000, + "gzip": 336000, + "brotli": 270000 } } } diff --git a/packages/wallet/wallet-toolbox/mobile/platform-budget.json b/packages/wallet/wallet-toolbox/mobile/platform-budget.json index 9c0dd4651..35af4e170 100644 --- a/packages/wallet/wallet-toolbox/mobile/platform-budget.json +++ b/packages/wallet/wallet-toolbox/mobile/platform-budget.json @@ -7,7 +7,7 @@ "brotli": 360000 }, "hermes": { - "raw": 3254000, + "raw": 3274000, "gzip": 1325000, "brotli": 1035000 } From 60186d37bb7bd300389103a095f5853fc4893310 Mon Sep 17 00:00:00 2001 From: Ty J Everett Date: Mon, 10 Aug 2026 19:12:45 -0700 Subject: [PATCH 4/7] test(wallet): cover funding compatibility fallbacks --- .../StorageProviderBatchFallbacks.test.ts | 25 ++++++++++++ .../__test/createActionPerformance.test.ts | 39 +++++++++++++++++++ 2 files changed, 64 insertions(+) diff --git a/packages/wallet/wallet-toolbox/src/storage/__test/StorageProviderBatchFallbacks.test.ts b/packages/wallet/wallet-toolbox/src/storage/__test/StorageProviderBatchFallbacks.test.ts index 77550f0be..cc148b48a 100644 --- a/packages/wallet/wallet-toolbox/src/storage/__test/StorageProviderBatchFallbacks.test.ts +++ b/packages/wallet/wallet-toolbox/src/storage/__test/StorageProviderBatchFallbacks.test.ts @@ -46,6 +46,31 @@ describe('StorageProvider batch fallbacks', () => { expect(findAvailableManagedChangeInputs).toHaveBeenCalledWith(1, 2, true, trx) }) + test('enriches projected funding candidates when a custom provider exposes status lookup', async () => { + const rows = [{ + outputId: 11, + transactionId: 22, + satoshis: 33, + txid: '44'.repeat(32), + vout: 5 + }] as TableOutput[] + const provider = { + findAvailableManagedChangeInputs: jest.fn(async () => rows), + findTransactionStatusesByIds: jest.fn(async () => new Map([[22, 'sending']])) + } + + const result = await StorageProvider.prototype.findAvailableManagedChangeInputCandidates.call( + provider, + 1, + 2, + false, + trx + ) + + expect(result).toEqual([{ ...rows[0], transactionStatus: 'sending' }]) + expect(provider.findTransactionStatusesByIds).toHaveBeenCalledWith(1, [22], trx) + }) + test('filters locked funding rows by owner, reservation, and source status', async () => { const eligible = { outputId: 1, userId: 7, transactionId: 11 } const reserved = { outputId: 2, userId: 7, transactionId: 12 } diff --git a/packages/wallet/wallet-toolbox/src/storage/__test/createActionPerformance.test.ts b/packages/wallet/wallet-toolbox/src/storage/__test/createActionPerformance.test.ts index af6a9a53f..1f0adb63f 100644 --- a/packages/wallet/wallet-toolbox/src/storage/__test/createActionPerformance.test.ts +++ b/packages/wallet/wallet-toolbox/src/storage/__test/createActionPerformance.test.ts @@ -179,6 +179,45 @@ describe('createAction funding performance', () => { expect(result.inputs[0].sourceTxid).toBe(candidates[2].txid) }) + test('keeps the settled baseline when exact BEEF cost comparison cannot load proofs', async () => { + const candidates = await replaceFundingCandidatesAcrossSources([ + { satoshis: 4_000, status: 'completed' }, + { satoshis: 4_000, status: 'completed', source: 0 }, + { satoshis: 11_000, status: 'sending', source: 1 } + ]) + ctx.activeStorage.managedChangePolicy.pendingComparisonInputs = 1 + jest.spyOn(ctx.activeStorage, 'getBeefForTransactions') + .mockRejectedValue(new Error('proof service unavailable')) + + const result = await ctx.activeStorage.createAction( + { userId: ctx.userId }, + actionArgs(5_000) + ) + + expect(result.inputs).toHaveLength(2) + expect(result.inputs.every(input => input.sourceTxid === candidates[0].txid)).toBe(true) + expect(result.inputs.some(input => input.sourceTxid === candidates[2].txid)).toBe(false) + }) + + test('rejects a custom-provider candidate whose ancestry cannot be resolved', async () => { + await replaceFundingCandidates(1, 5_000) + const original = ctx.activeStorage.findAvailableManagedChangeInputCandidates.bind(ctx.activeStorage) + jest.spyOn(ctx.activeStorage, 'findAvailableManagedChangeInputCandidates') + .mockImplementation(async (...args) => (await original(...args)).map(candidate => ({ + ...candidate, + transactionStatus: undefined + }))) + jest.spyOn(ctx.activeStorage, 'findTransactionStatusesByIds').mockResolvedValue(new Map()) + + await expect(ctx.activeStorage.createAction( + { userId: ctx.userId }, + actionArgs(1_000) + )).rejects.toMatchObject({ + code: 'WERR_INTERNAL', + message: expect.stringContaining('missing its source transaction status') + }) + }) + test('operator can disable pending comparison without disabling last-resort pending funding', async () => { const candidates = await replaceFundingCandidatesAcrossSources([ { satoshis: 4_000, status: 'completed' }, From 8379367a984ad992970aef36297315a5959154b0 Mon Sep 17 00:00:00 2001 From: Ty J Everett Date: Mon, 10 Aug 2026 20:54:39 -0700 Subject: [PATCH 5/7] fix(wallet): shape fixed-input surplus safely --- docs/reference/package-api-migrations.md | 2 +- governance/package-release-notes.json | 2 +- infra/wallet-infra/README.md | 10 ++ packages/wallet/wallet-toolbox/CHANGELOG.md | 5 +- .../docs/managed-change-liquidity.md | 15 ++- .../GenerateChange/generateChangeSdk.test.ts | 99 +++++++++++++++++++ .../src/storage/methods/generateChange.ts | 64 ++++++++++-- .../src/storage/schema/KnexMigrations.ts | 6 +- .../test/storage/KnexMigrations.test.ts | 6 ++ 9 files changed, 198 insertions(+), 11 deletions(-) diff --git a/docs/reference/package-api-migrations.md b/docs/reference/package-api-migrations.md index d11e825ca..967ea17c9 100644 --- a/docs/reference/package-api-migrations.md +++ b/docs/reference/package-api-migrations.md @@ -477,7 +477,7 @@ CLI entry points: `{"wallet-relay":"./bin/init.mjs"}`. - Package documentation: [docs/packages/wallet/wallet-toolbox.md](../packages/wallet/wallet-toolbox.md) - Source: [packages/wallet/wallet-toolbox](https://github.com/bsv-blockchain/ts-stack/tree/main/packages/wallet/wallet-toolbox) -- Release note: Isolates and resumes action batches, then adds progressive managed-change liquidity: 144 useful 5,000-satoshi units, bounded surplus-only fanout and fragment migration, settled-first parent selection, exact BEEF-cost comparison for pathological plans, last-resort pending funding, delayed permission persistence, and read-only Monitor reporting. +- Release note: Isolates and resumes action batches, then adds progressive managed-change liquidity: 144 useful 5,000-satoshi units, bounded surplus-only fanout and fragment migration (including actions already funded by explicit inputs), settled-first parent selection, exact BEEF-cost comparison for pathological plans, last-resort pending funding, sync-visible SQL metadata, delayed permission persistence, and read-only Monitor reporting. - Migration: No consumer code or BRC-100 migration is required. Exact untouched default baskets at 144 outputs / 32 satoshis advance to a 5,000-satoshi preference; custom basket values remain unchanged and funds migrate only through future authorized actions. Same-tier compatibility planning and retained pending fallback ensure the policy adds no new funding refusal. Operators may tune all work limits, including explicit -1 unlimited modes. | Public subpath | Runtime target(s) | Declaration target(s) | diff --git a/governance/package-release-notes.json b/governance/package-release-notes.json index 0e18e5259..c8faf8ef4 100644 --- a/governance/package-release-notes.json +++ b/governance/package-release-notes.json @@ -196,7 +196,7 @@ "name": "@bsv/wallet-toolbox", "publishedVersion": "2.6.5", "releaseType": "patch", - "summary": "Isolates and resumes action batches, then adds progressive managed-change liquidity: 144 useful 5,000-satoshi units, bounded surplus-only fanout and fragment migration, settled-first parent selection, exact BEEF-cost comparison for pathological plans, last-resort pending funding, delayed permission persistence, and read-only Monitor reporting.", + "summary": "Isolates and resumes action batches, then adds progressive managed-change liquidity: 144 useful 5,000-satoshi units, bounded surplus-only fanout and fragment migration (including actions already funded by explicit inputs), settled-first parent selection, exact BEEF-cost comparison for pathological plans, last-resort pending funding, sync-visible SQL metadata, delayed permission persistence, and read-only Monitor reporting.", "migration": "No consumer code or BRC-100 migration is required. Exact untouched default baskets at 144 outputs / 32 satoshis advance to a 5,000-satoshi preference; custom basket values remain unchanged and funds migrate only through future authorized actions. Same-tier compatibility planning and retained pending fallback ensure the policy adds no new funding refusal. Operators may tune all work limits, including explicit -1 unlimited modes." }, { diff --git a/infra/wallet-infra/README.md b/infra/wallet-infra/README.md index 308847b9c..9f3ddb303 100644 --- a/infra/wallet-infra/README.md +++ b/infra/wallet-infra/README.md @@ -87,6 +87,16 @@ it does not hide pending funds when they are required to fund an action. Unlimited fanout or migration can create large transactions and ancestry payloads, so use it only after production-shaped measurement. +Release sequencing matters: this source candidate intentionally continues to +lock the standalone image build to the currently published Wallet Toolbox +2.6.5 while the package and image are reviewed together. The variables above +are parsed and validated in that build, but the 2.6.5 runtime does not consume +the policy object. They become effective only after the protected release +publishes Wallet Toolbox 2.6.6, version synchronization refreshes the +standalone package and lock, and an official image is built from that synced +commit. Verify the image's package provenance reports 2.6.6 or newer before +depending on these settings operationally. + Funding always prefers completed parents, then unproven parents, then sending parents. Each tier retains the former funding shape as a compatibility fallback before widening to less-preferred ancestry, so these preferences cannot add a diff --git a/packages/wallet/wallet-toolbox/CHANGELOG.md b/packages/wallet/wallet-toolbox/CHANGELOG.md index ed87f5050..516565ea1 100644 --- a/packages/wallet/wallet-toolbox/CHANGELOG.md +++ b/packages/wallet/wallet-toolbox/CHANGELOG.md @@ -10,7 +10,10 @@ attention to changes that materially alter behavior or extend functionality. policy targeting 144 useful 5,000-satoshi outputs. New actions create at most eight outputs from real surplus and migrate at most four fee-positive legacy fragments, while a same-tier compatibility plan guarantees that optional - shaping cannot refuse an action the former planner could fund. + shaping cannot refuse an action the former planner could fund. Explicitly + funded actions materialize change from their existing surplus without + gathering pool inputs, and SQLite policy migrations use sync-compatible UTC + ISO timestamps. - Prefer completed, then unproven, then sending parents. Plans above 16 inputs compare exact transaction-plus-BEEF bytes before accepting pending ancestry; pending change remains an unconditional last-resort funding source. Align diff --git a/packages/wallet/wallet-toolbox/docs/managed-change-liquidity.md b/packages/wallet/wallet-toolbox/docs/managed-change-liquidity.md index 2ff52aca6..9b9947cb2 100644 --- a/packages/wallet/wallet-toolbox/docs/managed-change-liquidity.md +++ b/packages/wallet/wallet-toolbox/docs/managed-change-liquidity.md @@ -97,6 +97,16 @@ exhausted. ## Progressive pool shaping +Explicit or fixed inputs can already cover the requested outputs and fee. In +that case the planner materializes the first change output directly from that +existing surplus before considering optional fragment migration. It does not +call the managed-change allocator merely to create pool outputs. This keeps +consolidations and externally funded actions on the same shaping policy while +preserving the invariant that pool growth never gathers compulsory inputs. +When the surplus cannot pay both the marginal output fee and the economic dust +floor, the bounded remainder stays in the transaction fee instead of causing a +compatibility retry to gather another input solely to manufacture change. + After compulsory funding succeeds, the planner may consume up to four undersized outputs. A fragment is skipped when spending it would cost at least its value. Optional migration never supplies a missing satoshi for the caller's @@ -137,7 +147,10 @@ only from real surplus. The SQL data migration is intentionally one-way. Rolling code back does not rewrite a migrated preference to 32 or fragment funds. Older code can still -read and honor the 5,000-satoshi basket value. +read and honor the 5,000-satoshi basket value. SQLite writes the migrated +row's timestamp in UTC ISO form, matching incremental-sync query values and +keeping that metadata immediately sync-visible. MySQL retains its native +millisecond timestamp expression. ## Operator configuration diff --git a/packages/wallet/wallet-toolbox/src/storage/methods/__test/GenerateChange/generateChangeSdk.test.ts b/packages/wallet/wallet-toolbox/src/storage/methods/__test/GenerateChange/generateChangeSdk.test.ts index 11bb60903..87760cba0 100644 --- a/packages/wallet/wallet-toolbox/src/storage/methods/__test/GenerateChange/generateChangeSdk.test.ts +++ b/packages/wallet/wallet-toolbox/src/storage/methods/__test/GenerateChange/generateChangeSdk.test.ts @@ -1379,6 +1379,105 @@ describe('generateChange tests', () => { expectTransactionSize(params, r) }) + test('10j fixed-input surplus is shaped without gathering wallet inputs', async () => { + const params: GenerateChangeSdkParams = { + ...defParams, + fixedInputs: [{ satoshis: 7_000, unlockingScriptLength: 107 }], + fixedOutputs: [{ satoshis: 1_000, lockingScriptLength: 25 }], + changeFirstSatoshis: 5_000, + changeInitialSatoshis: 5_000, + targetNetCount: 144, + maxChangeOutputs: 8, + surplusPoolShaping: true, + maxMigrationInputs: 0 + } + const storage = generateChangeSdkMakeStorage( + Array.from({ length: 10 }, (_, index) => ({ satoshis: 10_000, outputId: index + 1 })) + ) + const allocateChangeInput = jest.fn(storage.allocateChangeInput) + + const r = await generateChangeSdk(params, allocateChangeInput, storage.releaseChangeInput) + + expect(allocateChangeInput).not.toHaveBeenCalled() + expect(r.allocatedChangeInputs).toHaveLength(0) + expect(r.changeOutputs).toEqual([{ satoshis: 5_999, lockingScriptLength: 25 }]) + expectTransactionSize(params, r) + }) + + test('10k fixed-input surplus keeps optional fragment retirement within its migration budget', async () => { + const params: GenerateChangeSdkParams = { + ...defParams, + fixedInputs: [{ satoshis: 7_000, unlockingScriptLength: 107 }], + fixedOutputs: [{ satoshis: 1_000, lockingScriptLength: 25 }], + changeFirstSatoshis: 5_000, + changeInitialSatoshis: 5_000, + targetNetCount: 144, + maxChangeOutputs: 8, + surplusPoolShaping: true, + maxMigrationInputs: 4 + } + const storage = generateChangeSdkMakeStorage( + Array.from({ length: 10 }, (_, index) => ({ satoshis: 1_000, outputId: index + 1 })) + ) + const allocateChangeInput = jest.fn(storage.allocateChangeInput) + + const r = await generateChangeSdk(params, allocateChangeInput, storage.releaseChangeInput) + + expect(allocateChangeInput).toHaveBeenCalledTimes(4) + expect(r.allocatedChangeInputs).toHaveLength(4) + expect(r.allocatedChangeInputs.every(input => input.satoshis < 5_000)).toBe(true) + expectTransactionSize(params, r) + }) + + test('10l large fixed-input surplus receives bounded multi-output shaping', async () => { + const params: GenerateChangeSdkParams = { + ...defParams, + fixedInputs: [{ satoshis: 100_000, unlockingScriptLength: 107 }], + fixedOutputs: [{ satoshis: 1_000, lockingScriptLength: 25 }], + changeFirstSatoshis: 5_000, + changeInitialSatoshis: 5_000, + targetNetCount: 144, + maxChangeOutputs: 8, + surplusPoolShaping: true, + maxMigrationInputs: 0 + } + const storage = generateChangeSdkMakeStorage([{ satoshis: 50_000, outputId: 1 }]) + const allocateChangeInput = jest.fn(storage.allocateChangeInput) + + const r = await generateChangeSdk(params, allocateChangeInput, storage.releaseChangeInput) + + expect(allocateChangeInput).not.toHaveBeenCalled() + expect(r.allocatedChangeInputs).toHaveLength(0) + expect(r.changeOutputs).toHaveLength(8) + expect(r.changeOutputs.every(output => output.satoshis >= 5_000)).toBe(true) + expectTransactionSize(params, r) + }) + + test('10m sub-dust fixed-input surplus never gathers another input solely to manufacture change', async () => { + const params: GenerateChangeSdkParams = { + ...defParams, + fixedInputs: [{ satoshis: 1_000, unlockingScriptLength: 107 }], + fixedOutputs: [{ satoshis: 900, lockingScriptLength: 25 }], + feeModel: { model: 'sat/kb', value: 200 }, + changeFirstSatoshis: 5_000, + changeInitialSatoshis: 5_000, + targetNetCount: 144, + maxChangeOutputs: 8, + surplusPoolShaping: true, + maxMigrationInputs: 0 + } + const storage = generateChangeSdkMakeStorage([{ satoshis: 10_000, outputId: 1 }]) + const allocateChangeInput = jest.fn(storage.allocateChangeInput) + + const r = await generateChangeSdk(params, allocateChangeInput, storage.releaseChangeInput) + + expect(allocateChangeInput).not.toHaveBeenCalled() + expect(r.allocatedChangeInputs).toHaveLength(0) + expect(r.changeOutputs).toHaveLength(0) + expect(r.fee).toBe(100) + expectTransactionSize(params, r) + }) + test('11 emits correlated allocation and generate-change spans without values', async () => { const events: any[] = [] const telemetry = new Telemetry({ diff --git a/packages/wallet/wallet-toolbox/src/storage/methods/generateChange.ts b/packages/wallet/wallet-toolbox/src/storage/methods/generateChange.ts index afddce5a7..cbb8df01d 100644 --- a/packages/wallet/wallet-toolbox/src/storage/methods/generateChange.ts +++ b/packages/wallet/wallet-toolbox/src/storage/methods/generateChange.ts @@ -143,6 +143,33 @@ interface SurplusChangeShapingRequest { rand: (min: number, max: number) => number } +interface SurplusChangeMaterializationRequest { + params: GenerateChangeSdkParams + result: GenerateChangeSdkResult + dustFloor: number + feeExcess: (addedChangeInputs?: number, addedChangeOutputs?: number) => number +} + +/** + * Materialize the first managed-change output from surplus that is already in + * the transaction. This is especially important for explicit/fixed inputs: + * their value can fully fund an action before the allocator loop runs, but the + * shaping policy must still capture the remainder without gathering another + * wallet-managed input. + */ +function materializeSurplusChangeOutput(request: SurplusChangeMaterializationRequest): void { + const { params, result } = request + if (!params.surplusPoolShaping || result.changeOutputs.length > 0) return + + const availableAfterOutputFee = request.feeExcess(0, 1) + if (availableAfterOutputFee < request.dustFloor) return + result.changeOutputs.push({ + satoshis: Math.min(availableAfterOutputFee, Math.max(request.dustFloor, params.changeFirstSatoshis)), + lockingScriptLength: params.changeLockingScriptLength + }) + request.feeExcess() +} + function shapeSurplusChangeOutputs(request: SurplusChangeShapingRequest): void { const { params, result } = request if (!params.surplusPoolShaping || result.changeOutputs.length !== 1) return @@ -552,6 +579,14 @@ async function generateChangeSdkCore( } } + /** + * The action may already be funded entirely by explicit/fixed inputs. In + * that case the allocator loop never runs, so capture the existing surplus + * here before the no-change compatibility guard. This operation cannot + * allocate an input; bounded legacy migration remains a separate step. + */ + materializeSurplusChangeOutput({ params, result: r, dustFloor, feeExcess }) + /** * Trigger an account funding event if we don't have enough to cover this transaction. */ @@ -566,11 +601,18 @@ async function generateChangeSdkCore( * If needed, seek funding to avoid overspending on fees without a change output to recapture it. */ if (r.changeOutputs.length === 0 && feeExcessNow > 0) { - const minimumChange = Math.max(dustFloor, params.changeFirstSatoshis) - const totalSatoshisNeeded = spending() + feeTarget(0, 1) + minimumChange - const moreSatoshisNeeded = Math.max(1, totalSatoshisNeeded - funding()) - await releaseAllocatedChangeInputs() - throw new WERR_INSUFFICIENT_FUNDS(totalSatoshisNeeded, moreSatoshisNeeded) + const hasOnlyUnreturnableShapingSurplus = surplusPoolShaping && feeExcess(0, 1) < dustFloor + if (!hasOnlyUnreturnableShapingSurplus) { + const minimumChange = Math.max(dustFloor, params.changeFirstSatoshis) + const totalSatoshisNeeded = spending() + feeTarget(0, 1) + minimumChange + const moreSatoshisNeeded = Math.max(1, totalSatoshisNeeded - funding()) + await releaseAllocatedChangeInputs() + throw new WERR_INSUFFICIENT_FUNDS(totalSatoshisNeeded, moreSatoshisNeeded) + } + // The remaining value cannot pay both the marginal output fee and the + // economic dust floor. Leave that bounded remainder in the miner fee; + // gathering another input solely to manufacture change would violate + // surplus-only shaping and make the action less efficient. } /** @@ -662,7 +704,17 @@ export function validateGenerateChangeSdkResult( ok = false } const feeRequired = Math.ceil(((r.size || 0) / 1000) * (r.satsPerKb || 0)) - if (feeRequired !== r.fee) { + const minSpendTxSize = transactionSize([params.changeUnlockingScriptLength], [params.changeLockingScriptLength]) + const dustFloor = Math.max(1, Math.ceil((minSpendTxSize / 1000) * (r.satsPerKb || 0)) * 2) + const feeWithChangeOutput = Math.ceil( + (((r.size || 0) + transactionOutputSize(params.changeLockingScriptLength)) / 1000) * (r.satsPerKb || 0) + ) + const isBoundedUnreturnableShapingSurplus = + params.surplusPoolShaping === true && + r.changeOutputs.length === 0 && + r.fee > feeRequired && + r.fee - feeWithChangeOutput < dustFloor + if (feeRequired !== r.fee && !isBoundedUnreturnableShapingSurplus) { log += `required fee error ${feeRequired} !== ${r.fee};` ok = false } diff --git a/packages/wallet/wallet-toolbox/src/storage/schema/KnexMigrations.ts b/packages/wallet/wallet-toolbox/src/storage/schema/KnexMigrations.ts index a974e4b73..5a4ef9e79 100644 --- a/packages/wallet/wallet-toolbox/src/storage/schema/KnexMigrations.ts +++ b/packages/wallet/wallet-toolbox/src/storage/schema/KnexMigrations.ts @@ -156,6 +156,10 @@ export class KnexMigrations implements MigrationSource { async up(knex) { // Only the exact historical defaults identify an untouched basket. // Operator-selected non-default values remain authoritative. + // SQLite sync predicates compare ISO timestamp text, while MySQL uses + // native timestamp values. Preserve that provider-specific contract so + // the migrated row remains visible to incremental sync immediately. + const updatedAt = (await determineDBType(knex)) === 'SQLite' ? new Date().toISOString() : knex.fn.now(3) await knex('output_baskets') .where({ name: 'default', @@ -164,7 +168,7 @@ export class KnexMigrations implements MigrationSource { }) .update({ minimumDesiredUTXOValue: DEFAULT_MANAGED_CHANGE_MINIMUM_SATOSHIS, - updated_at: knex.fn.now() + updated_at: updatedAt }) }, async down() { diff --git a/packages/wallet/wallet-toolbox/test/storage/KnexMigrations.test.ts b/packages/wallet/wallet-toolbox/test/storage/KnexMigrations.test.ts index dbc62b520..aa7aa7632 100644 --- a/packages/wallet/wallet-toolbox/test/storage/KnexMigrations.test.ts +++ b/packages/wallet/wallet-toolbox/test/storage/KnexMigrations.test.ts @@ -227,11 +227,17 @@ describe('KnexMigrations tests', () => { ]) const source = new KnexMigrations('test', 'managed change migration test', '1'.repeat(64), 1000) const migration = await source.getMigration(MANAGED_CHANGE_POLICY_MIGRATION) + const incrementalSyncSince = new Date().toISOString() await migration.up(knex) const rows = await knex('output_baskets').orderBy('userId') expect(rows.map(row => Number(row.minimumDesiredUTXOValue))).toEqual([5_000, 64, 32, 32]) + expect(rows[0].updated_at).toMatch(/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z$/) + await expect(knex('output_baskets') + .where('updated_at', '>=', incrementalSyncSince) + .orderBy('userId')) + .resolves.toMatchObject([{ userId: 1, minimumDesiredUTXOValue: 5_000 }]) await migration.down?.(knex) const afterDown = await knex('output_baskets').orderBy('userId') expect(afterDown.map(row => Number(row.minimumDesiredUTXOValue))).toEqual([5_000, 64, 32, 32]) From 9dea045df41e9a86ef2fe87c3b66e10647eaadb9 Mon Sep 17 00:00:00 2001 From: Ty J Everett Date: Mon, 10 Aug 2026 21:05:45 -0700 Subject: [PATCH 6/7] refactor(wallet): isolate change recapture policy --- .../src/storage/methods/generateChange.ts | 56 ++++++++++++++----- 1 file changed, 42 insertions(+), 14 deletions(-) diff --git a/packages/wallet/wallet-toolbox/src/storage/methods/generateChange.ts b/packages/wallet/wallet-toolbox/src/storage/methods/generateChange.ts index cbb8df01d..7e89a2741 100644 --- a/packages/wallet/wallet-toolbox/src/storage/methods/generateChange.ts +++ b/packages/wallet/wallet-toolbox/src/storage/methods/generateChange.ts @@ -150,6 +150,13 @@ interface SurplusChangeMaterializationRequest { feeExcess: (addedChangeInputs?: number, addedChangeOutputs?: number) => number } +interface ChangeRecaptureRequest extends SurplusChangeMaterializationRequest { + releaseAllocatedChangeInputs: () => Promise + funding: () => number + spending: () => number + feeTarget: (addedChangeInputs?: number, addedChangeOutputs?: number) => number +} + /** * Materialize the first managed-change output from surplus that is already in * the transaction. This is especially important for explicit/fixed inputs: @@ -170,6 +177,28 @@ function materializeSurplusChangeOutput(request: SurplusChangeMaterializationReq request.feeExcess() } +/** + * Preserve the historical compatibility retry when another input can make a + * viable change output, except when surplus-only shaping has nothing economic + * to return. In that case the bounded remainder stays in the miner fee instead + * of manufacturing change from an additional wallet input. + */ +async function requireViableChangeOrRetainBoundedFee(request: ChangeRecaptureRequest): Promise { + const { params, result } = request + const feeExcessNow = request.feeExcess() + if (result.changeOutputs.length > 0 || feeExcessNow <= 0) return + + const hasOnlyUnreturnableShapingSurplus = + params.surplusPoolShaping === true && request.feeExcess(0, 1) < request.dustFloor + if (hasOnlyUnreturnableShapingSurplus) return + + const minimumChange = Math.max(request.dustFloor, params.changeFirstSatoshis) + const totalSatoshisNeeded = request.spending() + request.feeTarget(0, 1) + minimumChange + const moreSatoshisNeeded = Math.max(1, totalSatoshisNeeded - request.funding()) + await request.releaseAllocatedChangeInputs() + throw new WERR_INSUFFICIENT_FUNDS(totalSatoshisNeeded, moreSatoshisNeeded) +} + function shapeSurplusChangeOutputs(request: SurplusChangeShapingRequest): void { const { params, result } = request if (!params.surplusPoolShaping || result.changeOutputs.length !== 1) return @@ -599,21 +628,20 @@ async function generateChangeSdkCore( /** * If needed, seek funding to avoid overspending on fees without a change output to recapture it. + * An economically unreturnable shaping remainder stays in the miner fee; + * gathering another input solely to manufacture change would violate + * surplus-only shaping and make the action less efficient. */ - if (r.changeOutputs.length === 0 && feeExcessNow > 0) { - const hasOnlyUnreturnableShapingSurplus = surplusPoolShaping && feeExcess(0, 1) < dustFloor - if (!hasOnlyUnreturnableShapingSurplus) { - const minimumChange = Math.max(dustFloor, params.changeFirstSatoshis) - const totalSatoshisNeeded = spending() + feeTarget(0, 1) + minimumChange - const moreSatoshisNeeded = Math.max(1, totalSatoshisNeeded - funding()) - await releaseAllocatedChangeInputs() - throw new WERR_INSUFFICIENT_FUNDS(totalSatoshisNeeded, moreSatoshisNeeded) - } - // The remaining value cannot pay both the marginal output fee and the - // economic dust floor. Leave that bounded remainder in the miner fee; - // gathering another input solely to manufacture change would violate - // surplus-only shaping and make the action less efficient. - } + await requireViableChangeOrRetainBoundedFee({ + params, + result: r, + dustFloor, + feeExcess, + feeTarget, + funding, + spending, + releaseAllocatedChangeInputs + }) /** * Progressively retire economically useful legacy fragments without ever From 7390ce94ae00ccb796743c9f83cfe256ebdb9795 Mon Sep 17 00:00:00 2001 From: Ty J Everett Date: Mon, 10 Aug 2026 21:14:28 -0700 Subject: [PATCH 7/7] chore(wallet): record exact browser size budget --- packages/wallet/wallet-toolbox/CHANGELOG.md | 5 +++-- packages/wallet/wallet-toolbox/client/platform-budget.json | 2 +- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/packages/wallet/wallet-toolbox/CHANGELOG.md b/packages/wallet/wallet-toolbox/CHANGELOG.md index 5ffd19ba7..b6e702305 100644 --- a/packages/wallet/wallet-toolbox/CHANGELOG.md +++ b/packages/wallet/wallet-toolbox/CHANGELOG.md @@ -37,8 +37,9 @@ attention to changes that materially alter behavior or extend functionality. bounded and measured from exact packed artifacts. The browser contract now measures 1,557,196 raw / 365,526 gzip / 287,329 Brotli bytes with Vite and 1,216,272 raw / 334,453 gzip / 268,547 Brotli bytes with esbuild. Linux CI - observed 1,558,352 Vite raw bytes; the reviewed ceilings advance by at most - 1.0% and retain narrow headroom. Mobile measures 1,616,960 Metro bytes and + observed 1,558,352 Vite raw bytes and 1,218,452 esbuild raw bytes; the + reviewed ceilings advance by at most 1.0% and retain narrow headroom. Mobile + measures 1,616,960 Metro bytes and 3,266,887 raw Hermes bytes locally; Linux CI observed 3,271,396 raw Hermes bytes. The Metro and compressed-mobile ceilings remain unchanged, while the Hermes raw ceiling advances by 0.61% from the pre-liquidity-policy value. diff --git a/packages/wallet/wallet-toolbox/client/platform-budget.json b/packages/wallet/wallet-toolbox/client/platform-budget.json index e9d7f3933..6290b9156 100644 --- a/packages/wallet/wallet-toolbox/client/platform-budget.json +++ b/packages/wallet/wallet-toolbox/client/platform-budget.json @@ -7,7 +7,7 @@ "brotli": 289000 }, "esbuild": { - "raw": 1218000, + "raw": 1220000, "gzip": 336000, "brotli": 270000 }