FM-S08: Proxmox client spike — fallback (reqwest + pinned rustls verifier) chosen

[Andreas-Froyland]

Closes #97 (FM-S08).

Question

Does the typed proxmox-client crate satisfy authentication, UPID task polling, custom TLS trust/pinning, unknown-field tolerance, and cancellation — or does Fleet keep a small reqwest transport with typed provider DTOs?

Decision: fallback chosen — reqwest transport with a pinned-fingerprint rustls verifier

Evidence (live against the integration VM's PVE 9.2.2 host, disposable probe project outside the repo)

Evidence	Result
Authentication	Typed crate's token auth works live (version/cluster/nodes/QEMU/tasks, incl. a real qmreboot UPID round-trip)
TLS trust/pinning	Failed — decisive. Builder surface is accept_invalid_certs(bool) only; no pinning hook, no custom roots. PVE presents its own cluster CA, so system trust fails and the only "working" option is disabling verification (forbidden)
UPID polling	Works; raw String UPID needs Fleet-side parsing; no wait helper (Fleet owns the loop anyway)
Unknown fields	Tolerant (all-Option, envelope unwraps data: null, VmConfig flattens indexed params)
Cancellation	Timeouts only
Coverage	Broad enough for M6's first three epics
Supply chain	3 commits ever, 3 releases in one week (2026-03), 0 stars, 591 downloads, 1 maintainer; drags reqwest 0.13 + aws-lc-rs (second TLS stack + second reqwest major) into a graph standardized on reqwest 0.12 + ring
Fallback proven	A custom rustls::client::danger::ServerCertVerifier SHA-256-pinning the leaf cert connected live and refused a wrong fingerprint at the handshake (is_connect()), on reqwest 0.12 + rustls ring — the TLS stack Fleet already uses

A second candidate, proxmox-api 0.2.0 (schema-generated), was rejected faster: its default client hardcodes danger_accept_invalid_certs(true) and its Debug prints the raw token.

Changes

• docs/research/ecosystem.md — FM-S08 evidence table + decision, Proxmox section refreshed with current versions/commits
• docs/planning/spikes.md — FM-S08 resolution entry with the rejected option named
• docs/planning/initial-issues.md — M4 status corrected to Complete; FM-S08 ledger entry

Non-goals (spike rules)

No production code: the pinning verifier and probe binaries live in a disposable project outside the repository; FM-600 implements the transport.

---

Summary by cubic

Closes #97 with the FM-S08 spike outcome: Fleet will use a small reqwest transport with a pinned-fingerprint rustls verifier instead of the typed proxmox-client crate. The typed crate failed the TLS security gate because it only exposes accept_invalid_certs(bool), and PVE presents its own cluster CA.

Changes

• Documents the live evidence and fallback decision in docs/research/ecosystem.md, including the positive and negative pinning probe against the PVE 9.2 host and the recorded deviation for PVE 8.x.
• Adds the FM-S08 resolution to docs/planning/spikes.md, naming the rejected proxmox-api alternative and the 8.x deviation.
• Updates docs/planning/initial-issues.md to mark M4 as code-complete and add the FM-S08 ledger entry.

No production code or migration steps are included; the pinning verifier is proven in a disposable probe outside this repo, and FM-600 implements the transport.

^{Written for commit 16b0fcf. Summary will update on new commits.}

[cubic-dev-ai]

All reported issues were addressed across 3 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[Andreas-Froyland]

All six findings addressed in 16b0fcf:

• P2 (both majors, ecosystem.md): recorded as an explicit deviation against the FM-000 acceptance criterion — live evidence exists for PVE 9.2.2 only (single PVE host in the integration environment); the 8.x leg is the PVE 8.x API-doc archive (verified genuine 8.x: it lacks the 9.x-only sdn/fabrics endpoints; contains PVEAPIToken and the exitstatus task shape) covering every endpoint the spike exercised; the live 8.x validation moves to the M6 real-cluster suite and the limitation is recorded on epic M6 epic: Proxmox accounts, TLS trust, and discovery #9 and in both ledgers. Note the decisive evidence is TLS behavior, which is client-side and version-independent.
• P2 (M4 status, initial-issues.md): reverted to "Code-complete", matching the M3 convention while the live convergence run is pending; the stale "not on the critical path ... now that M3 is code-complete" phrasing was rewritten past tense.
• P2 (next-step pointer, initial-issues.md): the M4 status line now points at the fallback implementation (FM-600), not the resolved spike.
• P3 (authority for the TLS prohibition, ecosystem.md): re-cited to the spike rules and security.md ("TLS verification/pinning cannot be disabled silently"), not ADR-0005.
• P3 (M4 "Complete" vs pending run): resolved by the status revert above.
• P3 (spikes.md row split): FM-S08's outcome entry now follows the complete FM-S04–FM-S09 register table; FM-S09 is back with its rows.
• P3 ("Fork" typo): → "Fleet owns cancellation by dropping futures".