feat(external-signer): add Visibility arg to file_prepare_upload

[Nic-dorman]

Summary

Adds an external-signer pathway for public file uploads. Calling Client::file_prepare_upload_with_visibility(path, Visibility::Public) bundles the serialized DataMap into the existing payment batch (wave-batch or merkle) as one additional chunk. After finalize_upload / finalize_upload_merkle, FileUploadResult.data_map_address carries the chunk address of the published DataMap — a single network address the uploader can share so anyone can retrieve the file.

This unblocks public uploads in the Autonomi desktop GUI (ant-gui). The GUI currently has its Public upload option disabled in the UI (WithAutonomi/ant-ui#12, merged) because no external-signer pathway exists for publishing the data map. Once this lands and ant-gui bumps the git dep, a follow-up PR on that repo will thread the visibility flag through the Tauri commands and re-enable the Public button.

Why bundling, not a separate call

Publishing the data map means storing one extra content-addressed chunk. The two obvious alternatives both hit dead ends on the external-signer path:

• client.data_map_store(&data_map) after finalize_upload — what ant-cli does. Internally goes through pay_for_storage, which hard-requires self.require_wallet()? (ant-core/src/data/client/payment.rs:46). ant-gui's Rust client has no wallet — all payments go via WalletConnect / direct-wallet on the frontend.
• Hand-roll chunk storage externally. The internal chunk-storage helpers (store_paid_chunks, chunk_put_to_close_group, merkle_upload_chunks) are all pub(crate), so external callers cannot reproduce the flow.

Bundling the data map chunk into the existing PaymentIntent (wave-batch) or merkle batch reuses the one-signature flow that already works for file chunks. The external signer pays for the data chunks and the data map chunk in a single on-chain transaction, and finalize_upload[_merkle] stores them all in the same wave.

Changes

• Visibility enum (Private default, Public) in data::client::file, re-exported from data::.
• Client::file_prepare_upload_with_visibility(path, visibility). The existing file_prepare_upload(path) delegates with Visibility::Private — signature and behaviour are preserved for existing callers.
• PreparedUpload.data_map_address: Option<[u8; 32]> carries the address between prepare and finalize.
• FileUploadResult.data_map_address: Option<[u8; 32]> is Some(addr) for public uploads. Pair it with data_map_fetch(&addr) + file_download(&data_map, dest) to retrieve the file.
• Both finalize_upload and finalize_upload_merkle propagate data_map_address; no extra network call needed, since the data map chunk is stored alongside the rest of the batch.
• The internal-wallet path (file_upload_with_mode) is unchanged — ant-cli continues to use file_upload + data_map_store for its public flow.

Test plan

• cargo fmt -p ant-core clean
• cargo check --workspace --all-features passes
• cargo clippy --tests -p ant-core --all-features -- -D warnings clean
• New e2e test test_file_prepare_upload_visibility in ant-core/tests/e2e_file.rs verifies:
  • Private prepare leaves data_map_address unset
  • Public prepare records the address, and the recorded address equals compute_address(rmp_serde::to_vec(&data_map))
  • For wave-batch, the Public path adds exactly one extra prepared chunk (the data map) to the batch, at the recorded address
• End-to-end verification on ant-gui lands in a follow-up PR (wiring visibility through Tauri commands and re-enabling the button)

🤖 Generated with Claude Code

[grumbach]

Clean PR, well-scoped. The bundling-into-PaymentIntent approach is the right call given pay_for_storage hard-requires a wallet and all the chunk-store helpers are pub(crate) — reusing the wave-batch / merkle batch gets you one signature for N+1 chunks with zero new network plumbing. Traced the extra chunk from chunk_data.push(bytes) in file_prepare_upload_with_visibility through to store_paid_chunks (wave path) and merkle_upload_chunks (merkle path); the data map chunk is genuinely paid for AND stored in both paths, not paid-for-and-lost. Backward compat is preserved — file_prepare_upload(path) delegates to Visibility::Private, data_map_address lands as None for every existing construction site including both internal-wallet file_upload_* paths. CI green.

What I'd want before merging

1. Test gap: the contract is half-verified. test_file_prepare_upload_visibility checks that the recorded address equals compute_address(rmp_serde::to_vec(&data_map)) and that the wave-batch gained one chunk, but it never actually finalizes, fetches, or round-trips. Storing the address is half the contract; a reader being able to data_map_fetch(&addr) + file_download(&data_map, dest) and get the original bytes back is the other half. Given the whole purpose of this PR is enabling public retrieval from ant-gui, a single e2e that does finalize_upload → data_map_fetch → file_download → assert_eq is the only check that actually proves the feature works on a real testnet. Adding it is cheap — you already have the testnet harness up.

2. Merkle path is completely untested. The test asserts match … WaveBatch and explicitly panics on merkle, but finalize_upload_merkle got the exact same data_map_address propagation and needs its own coverage — a file large enough to cross should_use_merkle would exercise it. Without that, the whole merkle branch of the public-upload code path rides on code review alone.

P2 — polish

• FileUploadResult and PreparedUpload aren't #[non_exhaustive]. Adding data_map_address is a technically-breaking change for any downstream that constructs these via struct literal (rare for a result type but not impossible — tests, mocks). Easy to fix forever by adding #[non_exhaustive] to both in this PR while you're already touching the public API; harder once 0.2 ships.
• Silent AlreadyStored for the data map chunk. prepare_chunk_payment returns Ok(None) on AlreadyStored and the wave loop filters None, so if the serialized DataMap hashes to an address that's already on the network (same file uploaded twice — deterministic self-encryption makes this plausible) the data map chunk gets silently dropped from prepared_chunks while data_map_address stays Some(addr). That's arguably correct behaviour (the address is valid, someone already paid) but worth an info! log so it's not invisible, and worth a one-line doc comment on data_map_address clarifying the "Some means address is retrievable, not that we stored it" semantics.
• Option<[u8; 32]> is the pragmatic choice — compute_address already returns [u8; 32] and data_map_fetch takes &[u8; 32], so a newtype here would just be shimming. Consistent with the rest of the module. No change needed.

Nits

• visibility={visibility:?} in the debug log is fine; consider visibility.as_str() or Display if this ever reaches a user-facing log.
• PR description's test-plan checkbox for the gui follow-up is unchecked — fine as a tracking item but worth a linked issue so it doesn't get lost.

Core change is right. LGTM after (1), with (2) strongly recommended.

[Nic-dorman]

Addressed in 414c38e.

Blockers

1. End-to-end round-trip test (wave-batch) — done. test_public_upload_round_trip_wave_batch prepares a file as Visibility::Public, signs payments via the testnet wallet (wallet.pay_for_quotes), calls finalize_upload, then retrieves the file using only data_map_fetch(&data_map_address) + file_download, and asserts the downloaded bytes equal the original. This is the half of the contract the original test didn't cover.

2. Merkle-path round-trip — attempted but removed. I wrote a full round-trip test that prepares a 280 MB file (> should_use_merkle threshold), simulates external signing via wallet.pay_for_merkle_tree, and finalizes via finalize_upload_merkle. Every step up to the pay_for_merkle_tree call succeeds; the call itself reverts with PaymentVaultV2::WrongPoolCount(expected=16, got=8). This reproduces with the untouched internal pay_for_merkle_batch code path too — it's a mismatch between prepare_merkle_batch_external's pool-commitment count and the contract's expected count for the tree depth, not something this PR introduces or touches. Rather than block the PR on an upstream investigation, I've removed the failing test. The data_map_address propagation in finalize_upload_merkle is a one-line field assignment (let data_map_address = prepared.data_map_address; → Ok(FileUploadResult { …, data_map_address })), so the risk surface there is essentially zero. Happy to file a follow-up issue to investigate the merkle-pool math and land the test then.

P2

• #[non_exhaustive] on FileUploadResult and PreparedUpload — done. Forecloses the same "adding a field is a breaking change" concern for the next field.
• Silent AlreadyStored on the DataMap chunk — done. When prepare_chunk_payment returns Ok(None) for the DataMap chunk (its serialized bytes hashed to something already on the network), we now emit an info! explaining that the address is still valid/retrievable but no storage payment was made. Doc comments on data_map_address (both structs) now spell out the "Some = retrievable, not necessarily we stored it" semantics.

Nits

• visibility={visibility:?} → Display — left as-is. The log is debug!-level and internal-only; switching to a Display impl would be a new trait impl + ceremony for output that only trace consumers ever see. Easy to change later if a user-facing log appears.
• GUI follow-up tracking — tracked separately; not in this PR's scope.