meta: resolve PlatformIO package aliases and build requested stacks

[zackees]

Context

A platformio.ini can request platform = <platform>@<version> or platform_packages = <package>@<version>. fbuild currently accepts these registry-version forms, emits a warning, and builds with its own pinned packages. crates/fbuild-config/src/platform_packages.rs::ignored_version_pins says fbuild has no PlatformIO registry resolver. The warning fixed the silent failure reported in #1407, but it does not make the requested build reproducible or equivalent to PlatformIO.

The published ESP32-S3 comparison demonstrates the impact: bench/blink/platformio.ini requests espressif32@6.13.0; PlatformIO resolves Arduino-ESP32 2.0.17 and Xtensa GCC 8.4.0, while fbuild ignores that registry pin and locally resolves Arduino-ESP32 3.3.12 and Xtensa GCC 14.2.0. The benchmark comparison is tracked in #1489. Pre-unified pioarduino releases also need per-MCU registry toolchains (#1444).

This parent coordinates an audit of all supported platform families, not an assumption that each is affected. Findings should become native sub-issues only when an agent verifies a concrete ignored pin or package mismatch; clean/unsupported cases should be reported without filing noise.

Proposal

1. Inventory every supported platform family and the platform / platform_packages registry-version paths it accepts. For each family, compare the requested version with the resolved platform, framework, toolchain, and SDK package versions from a real fbuild build or a focused resolution test. Compare with PlatformIO's registry metadata where feasible.
2. File one focused child per independently actionable platform-family failure. Include a minimal platformio.ini, observed warning and resolved versions, the resolution code path, user impact, and a plausible fix boundary. Link existing issues rather than duplicate them.
3. Implement a real PlatformIO registry alias resolver for short and owner-qualified platform and platform_packages specs. Resolve valid exact pins and supported version requirements to the published platform/package archive URL and SHA-256; use the selected platform manifest, board/framework, host architecture, and explicit package overrides to resolve the compatible framework, SDK, and toolchain set. A server-side metadata alias service may cache and normalize PlatformIO registry responses while binaries remain PlatformIO-hosted. Never substitute a default or a different vendor release. Preserve working URL/archive overrides.
4. Make build and benchmark artifacts record requested and resolved package/toolchain versions so cross-version results are visible.

Shared core resolver and unit tests

• Put the generic PlatformIO package-spec parser, normalized request, resolved-payload/lock model, version and precedence rules, and structured errors behind a reusable fbuild-core API. It must not be gated by the current Platform enum or hard-coded ESP/ARM/AVR names: an unknown-but-valid owner/name@version can resolve its payload even if native build dispatch later reports no orchestrator. Keep network/download/extraction adapters in existing fbuild-packages-fetch; every platform orchestrator consumes the same resolved result. Do not add a new crate.
• The core API resolves registry aliases (short and owner-qualified; exact versions and supported ranges), local/remote archives, repository URLs/refs, and local package directories to a normalized source/payload identity. For registry files it records the selected host-specific URL and checksum; for a platform it parses platform.json and applies framework/toolchain/SDK requirements and explicit platform_packages precedence. Unsupported schemes or unavailable versions produce structured errors, never a default substitution.
• Add offline, table-driven core unit tests with fixture registry responses and platform manifests: each source form above; a synthetic platform name absent from the Platform enum; exact/range selection; host architecture; checksum/digest and cache-key identity; dependency and override precedence; malformed/unavailable inputs. Add a representative resolution fixture for every native platform family in its child issue. Unit tests must not depend on the live registry or downloaded compilers; focused integration/build repros remain separate.
• Treat any syntactically valid PlatformIO package path as a core payload-resolution request, independent of build-platform dispatch: short aliases (espressif32@6.13.0), owner-qualified aliases (platformio/espressif32@6.13.0), explicit type/owner/name paths, and registry/archive/repository/local-path sources must normalize to the same immutable payload identity when they identify the same artifact. Core must expose the resolved URL or local source, digest/ref, selected host file, and manifest dependency identities; unsupported native builds are a later, separate error.
• Require a shared, offline core test matrix with one representative registry/platform-manifest fixture for each supported native family (ESP32, ESP8266, AVR/megaAVR, ARM families, CH32V), plus a synthetic unknown family. For each fixture assert the exact platform, framework, SDK and toolchain payloads that apply; host filtering, explicit override precedence, no default substitution, malformed/unavailable-path errors, and stable cache identity. Children add adapter/build tests, but must not duplicate or bypass generic lookup in their orchestrators.

Acceptance criteria

• RED: focused repros/tests for every confirmed child show that a supported registry pin currently yields a successful build with a different resolved package stack (or document the exact related failure).
• GREEN: valid published platform = name@version and platform_packages = name@version aliases for every supported native family resolve the requested platform/package versions, select compatible host-specific dependencies, and build with that stack. Cover exact pins, owner-qualified and short aliases, and supported semver requirements. A missing version, unsupported host, or truly unsupported package fails before compilation; refusing a valid supported alias is not completion.
• Resolved platform, framework, toolchain, and SDK versions are visible in build output or machine-readable metadata; benchmark comparisons validate matching stacks or label/exclude cross-version comparisons.
• URL/archive overrides continue to work. Tests cover cache reuse, host architecture, and platform-specific package layouts; each confirmed child has focused RED → GREEN evidence.
• All supported platform families are inventoried, with positive child issues attached natively to this parent and negative findings summarized in the parent discussion.
• RED → GREEN core unit evidence: a failing test first shows a valid generic registry/path spec currently becomes None, a warning, or a default package; after the shared resolver lands, that same fixture yields the exact normalized payload and dependent package identities. A synthetic unknown platform must resolve payload metadata without claiming native build support.
• Every supported-platform child has a RED failing core resolution assertion before its fix, then a GREEN assertion against the exact expected payload URL/checksum or VCS ref and dependency identities. The common core matrix passes offline on CI, without installed PlatformIO tools or downloaded compilers; family-specific build tests verify the adapter consumes those results.

Required alias contract

• Example: espressif32@6.13.0 / platformio/espressif32@6.13.0 must resolve to PlatformIO's platformio/platform/espressif32 version 6.13.0, not pioarduino stable. The registry metadata supplies the archive URL and SHA-256; the selected platform.json supplies package requirements. The metadata alias service is optional infrastructure, not a license to substitute a different stack.
• Resolve tool and framework files for the actual host/system, apply platform_packages override precedence, verify checksums, and key caches by the resolved package identity and digest. Emit the requested and resolved identities in build/benchmark metadata.
• Early errors are guardrails for unavailable or unsupported inputs. This parent is not complete until known-valid PlatformIO aliases build with their requested compatible stack across supported native families.

Decisions

• Treat ignored registry pins as a correctness/reproducibility problem, priority P1: warnings are useful but cannot fulfill an explicit build-version request.
• Coordinate by platform family so distinct package sources and toolchain layouts remain independently actionable; use one child for closely related boards sharing the same resolver, not one issue per board.
• Do not infer that GCC 14 alone caused the ESP32-S3 timing gap. The verified problem is a mismatched requested/resolved stack; a controlled same-stack benchmark remains required (fix(bench): align ESP32-S3 toolchains and explain cold-build gap #1489).
• Include both platform and platform_packages registry-version forms, but keep URL overrides as a separate already-supported path.
• Core ownership: define the generic spec/resolved-payload contract in fbuild-core, with fetch/extract in fbuild-packages-fetch, because these are the existing shared and package-transport boundaries; platform-specific orchestrators only adapt the resolved build recipe.

Related issues

• A pinned platform version is silently ignored — in platform and in platform_packages #1407 fixed the warning for silently ignored version pins; it did not implement registry resolution.
• fix(bench): align ESP32-S3 toolchains and explain cold-build gap #1489 tracks the ESP32-S3 benchmark mismatch and should be a child of this parent.
• ESP32: provision per-MCU registry toolchains so pre-unified platform pins (pioarduino 51.x) actually build against their pin #1444 tracks legacy pioarduino per-MCU registry toolchains and should be a child of this parent.

Testing gate: core resolves any generic PlatformIO path

• Implement and exercise a public fbuild-core resolver that accepts a PlatformIO path/spec without consulting the native Platform enum or any ESP/ARM/AVR-specific code. A syntactically valid unknown platform/package must resolve its payload metadata; lack of a native compiler adapter is a separate, later error.
• RED → GREEN unit tests first: a valid pinned path that currently becomes a warning, None, or a default must initially fail an exact-payload assertion. The same test must pass after the shared core fix. Each platform child contributes at least one failing-then-passing fixture to this shared core suite, in addition to adapter tests.
• Table-drive offline fixtures for ESP32, ESP8266, AVR and megaAVR, each supported ARM family, CH32V, and a synthetic unknown family. Cover short names, owner-qualified names, explicit type/owner/name paths, exact pins and supported version ranges, registry packages, HTTP(S) archives, repository URLs/refs, and local paths. Equivalent spellings of one artifact must produce the same normalized payload identity.
• For each fixture assert the selected platform/package version, exact host-specific URL and SHA-256 (or immutable VCS ref/local content identity), manifest-derived framework/SDK/toolchain requirements, explicit platform_packages precedence, and a stable cache key that changes when payload content changes. Test multiple host systems and reject unsupported hosts.
• Negative unit cases must reject malformed/ambiguous paths, missing versions, unsupported schemes, checksum mismatches, and incompatible or unavailable dependencies with structured errors before compilation. Never silently use a default or different release for a valid pin.
• Core unit tests run offline in CI with fixture metadata: no live registry, PlatformIO installation, or compiler downloads. Separate family adapter/build tests prove the resolved core payload is actually consumed and requested versus resolved identities are visible. A passing family test that bypasses core does not satisfy this gate.

[zackees]

Audit work order for parallel agents

Shared prompt: Read CLAUDE.md and the issue-filing workflow before investigating. Make two passes: (1) enumerate supported Platform variants, their platform = name@version and platform_packages = package@version paths, existing issues, and URL override behavior; (2) for each owned family, use a minimal version-pinned fixture, resolution code, package manifests, build/daemon logs or an isolated local build to establish the requested vs resolved platform, framework, SDK and compiler. A positive finding requires evidence that an explicit registry pin is accepted but a different/default stack is built (or another concrete pin-related failure). A warning is not equivalent to honoring the pin. Do not infer performance causality from package mismatch alone.

For each distinct independently actionable positive, search for duplicates, file a focused issue as a native child of #1491, and verify its /parent. Include the minimal ini, requested and actual package versions, warning/log or code evidence, why the substitution happens, user impact, RED → GREEN test acceptance, and a Decisions section. Group by shared resolver, not by board. File no issue for a negative or inconclusive result; report those to this parent. Preserve working archive/URL override behavior. Do not edit source or clear shared caches during the audit; use isolated temp projects for builds.

Ownership directories:

• ESP: crates/fbuild-build-esp/src/{esp32,esp8266}, related crates/fbuild-library/src/library/esp*, tests/platform/{esp32s3,esp32h2,esp32p4}. fix(bench): align ESP32-S3 toolchains and explain cold-build gap #1489 and ESP32: provision per-MCU registry toolchains so pre-unified platform pins (pioarduino 51.x) actually build against their pin #1444 are existing native children; do not duplicate them.
• ARM: crates/fbuild-build-arm/src/{rp2040,stm32,sam,nrf52,teensy,nxplpc,apollo3,renesas,silabs}, related library modules and tests/platform fixtures.
• MCU: crates/fbuild-build-mcu/src/{avr,ch32v}, related library modules and tests/platform AVR/CH32V fixtures. Inventory crates/fbuild-core/src/lib.rs and crates/fbuild-config/src/platform_packages.rs for any unowned variant.
• Coordinator: cross-cutting Wasm and final completeness check against every Platform enum variant.

[zackees]

Coordinator inventory note: Platform::Wasm is recognized by Platform::from_platform_str but get_platform_support returns “native orchestrator … not yet implemented — use --platformio” for it (crates/fbuild-build/src/lib.rs). It is outside the successful-native-build substitution audit unless an agent finds a separate acceptance path.

[zackees]

ESP ownership audit complete: filed #1492 for the separate ESP32 platform_packages framework registry-pin path and verified it is natively attached under #1491. Round 2 confirmed the current stable platform cache metadata selects Arduino framework 3.3.12, ESP-IDF 5.5.5 and toolchain 14.2.0+20260121; daemon build logs show successful ESP32 builds with the 3.3.12 framework after warning that the registry platform pin was ignored. Code shows a bare framework name@version is parsed as no override, then pioarduino_framework falls through to stable platform.json. ESP8266's fixture asks for espressif8266@4.2.1, while its fixed framework 3.1.2 and toolchain 3.2.0-gcc10.3 are both documented in code as matching that platform; I found no positive ESP8266 stack-mismatch case and filed no ESP8266 issue. #1489 and #1444 cover the other ESP32 platform-version routes.

[zackees]

Second ESP8266 pass confirmed a positive registry-pin mismatch and filed #1496, natively attached under this parent. Repro: isolated tests/platform/esp8266 fixture with platform=espressif8266@4.0.1; current checkout fbuild exits 0, prints the ignored-pin warning, reports GCC 10.3.0, downloads/installs framework 3.1.2 and emits firmware artifacts. Downloaded 4.0.1 platform.json requires framework ~3.30002.0 (registry resolves 3.30002.0 / Arduino core 3.0.2) and toolchain ~2.100300.0 (2.100300.220621 / GCC 10.3.0). Framework plus bundled SDK generation differs; compiler major/minor happens to match. Repro ini and local daemon log paths are documented in #1496. This supersedes my earlier negative note based only on the current 4.2.1 fixture.

[zackees]

Completed platform coverage (2026-09-26)

Platform has 15 variants. Fourteen have native orchestrators and are covered by the sub-issues below; Wasm is recognized but has no native orchestrator (get_platform_support returns an unsupported error), so it cannot produce this specific successful-build substitution.

Native variants	Child	Evidence level
Espressif32	#1489, #1444, #1492	ESP32-S3 benchmark/local builds and package-resolution code; platform and framework-package pin routes
Espressif8266	#1496	Direct successful local build of espressif8266@4.0.1, resolving Arduino 3.1.2 instead of requested 3.0.2 generation
Apollo3, AtmelSam, NordicNrf52, NxpLpc, RaspberryPi, RenesasRa, SiliconLabs, Ststm32, Teensy	#1493	Confirmed Teensy package-version mismatch plus shared parser and family resolver inventory; other family-specific build repros remain acceptance work
AtmelAvr, AtmelMegaAvr	#1494	AVR MiniCore manifest/board/default package mismatch; same AVR resolver covers megaAVR, whose own build repro remains acceptance work
Ch32v	#1495	CH32V platform metadata and published PlatformIO toolchain output compared with fbuild's fixed toolchain/framework resolver; local RED build remains acceptance work

The common failure is that explicit registry-version pins become warnings or None overrides while package constructors still select fbuild defaults. URL/archive overrides are a separate supported mechanism and must be preserved. The evidence does not isolate GCC version as the cause of the ESP32-S3 timing gap; a matched-stack benchmark is still required in #1489.

All positive findings above are native children of this parent. The AVR JSON-core URL override gap observed during the audit is outside this registry-pin scope and was not filed as a child.

[zackees]

Testing-criteria addendum for the shared resolver:

• Unit-test the public fbuild-core resolution API, not only native platform adapters. Feed it offline fixture metadata and a synthetic owner/platform/name@version whose name is absent from the current Platform enum. Assert that it resolves the exact requested version, payload URL, SHA-256, host selection, and stable cache identity. Native build dispatch may separately report that no orchestrator exists.
• Table-drive generic PlatformIO package paths: short and owner-qualified registry aliases; exact pins and supported ranges; platform, tool, framework, and library package kinds; platform-manifest dependency paths; explicit platform_packages precedence; local directory and archive paths; HTTP(S) archives; Git/repository URLs and refs. A valid path must normalize to a payload/source identity without family-specific special cases.
• Negative cases must assert structured errors for malformed paths, ambiguous short names, missing versions, unsupported hosts, checksum mismatch, and unsupported schemes. No case may silently choose a default or a different vendor package.
• Keep these tests offline and deterministic. Each child issue adds a representative real-family fixture plus an adapter test showing it consumes the same core result and selects the requested build stack; retain separate focused integration builds for actual compiler/layout compatibility.