Publish daily ProbeLab ClickHouse Parquet exports to FOC

[BigLep]

Done Criteria

• ProbeLab publishes daily Parquet dumps of Dealbot ClickHouse data to Filecoin Onchain Cloud (FOC).
• The ProbeLab dashboard links to the exports as a raw data access point.
• The FilOz-created dataset uses a wallet whose balance is monitored (like the other FOC WG wallets)

Why Important

1. Makes Dealbot data directly accessible
2. A practical dog-fooding FOC opportunity with a feedback loop with ProbeLab.

User/Customer

Users who want raw Dealbot data.

Notes

1. This is follow-up to Provide hook to write high dimensionality data to a custom clickhouse isntance #426 and Deploy dealbot configured for ProbeLab's ClickHouse database #486. It was discussed in the June 23, 2026 ProbeLab <> FilOZ Dealbot meeting.
2. Current daily volume is expected to be only kilobytes to low megabytes.
3. FilOz will provide ProbeLab a dataset and a session-key from a funded and monitored wallet.

[BigLep]

@dennis-tra : for implementation given you have a Python backend, I think a couple of options:

1. Community project: https://docs.filecoin.cloud/resources/community-projects/#python
2. Shell out to filecoin-pin: https://github.com/FilOzone/filecoin-pin . That definitely will work and is supported, including the session key usecase.

[BigLep]

@rjan90 : could you please:

1. Create a new multisig wallet for "probelab dashboard backups"?
2. Document it in our OE docs
3. Fund it
4. Create a dataset with filecoin-pin (presumably with one of the copies on Tippy's SP)
5. Create a session key that enables adding and removing pieces. I think expiring indefinitely is fine.
6. Send the session key over to Dennis via DM?

This should then enable Probelab to finish this work off.

[davidgasquez]

@rjan90, not sure if it'll be too much but would you be up for sharing pointers to the docs covering the tasks avobe? Would love to do them and get the Filecoin Data Portal datasets up there too! No rush as I don't know if I'll be able to tackle this but it has been on my mind for a while and seems the previous steps are comprehensive and a good secure setup.

[rjan90]

@davidgasquez happy to. Will share once I complete, planning to pick this up tomorrow (2026-07-02)

[rjan90]

Hey @davidgasquez! Here are the main pointers and the rough step-by-step I did for the task above.

Docs/pointers:

• DealBot Safe/session-key runbook: https://github.com/FilOzone/dealbot/blob/main/docs/runbooks/wallet-and-session-keys.md
• DealBot dataset creation background: https://github.com/FilOzone/dealbot/blob/main/docs/data-set-creation.md
• Filecoin Pin getting started: https://docs.filecoin.io/build-on-filecoin/cookbook/filecoin-pin/getting-started
• What filecoin-pin add does behind the scenes: https://github.com/filecoin-project/filecoin-pin/blob/master/documentation/behind-the-scenes-of-adding-a-file.md
• Safe UI: https://safe.filecoin.io/

What I did:

1. Created a Filecoin Safe multisig and documented the address/signers internally.
2. Funded the Safe with FIL and USDFC.
3. Ran the Safe payment setup batch from DealBot:
   node scripts/fund-safe.mjs --network mainnet --amount <USDFC> --wallet-address <SAFE_ADDRESS>
   This generates the Safe Transaction Builder batch for:
   • USDFC.approve
   • FilecoinPay.deposit
   • FilecoinPay.setOperatorApproval for FWSS
4. Executed that batch in Safe.
5. Created a session key:
   node scripts/create-session-key-safe.mjs --network mainnet --expiry-days <days>
6. Saved the session key private key in 1Password/SOPS, then executed the generated SessionKeyRegistry transaction via Safe.
7. Used session-key auth for uploads:
   • WALLET_ADDRESS=<SAFE_ADDRESS>
   • SESSION_KEY=<session key private key> for filecoin-pin
   • In DealBot envs this maps to SESSION_KEY_PRIVATE_KEY.
8. Verified payment state with:
   filecoin-pin payments status --network mainnet --wallet-address "$WALLET_ADDRESS" --session-key "$SESSION_KEY"
9. Created a seed dataset with filecoin-pin add, targeting Tippy’s provider:
   --provider-id 1
   and descriptive --data-set-metadata values.
10. After the seed upload succeeded, we got:

• Data Set ID: 1335
• Provider ID: 1
• Piece CID: bafkzcibe5h6h2enk5bdhuljvxk77zwtmoxe52o6p4t3kgntpuleyv3ccqwq2vqtjbi

For future uploads/automation, we’ll target the known dataset directly with --data-set-id 1335 rather than relying on metadata matching.

I hit some caveats due to using msig that we have to iron out though:

• Safe/contract-wallet reads currently need an SDK fix; tracked here: Read-only payment/storage calls fail for Safe owner addresses on Filecoin when eth_call includes from synapse-sdk#868