test: Add Debian 12/13 experiment suite for build#16
Conversation
Problem: - We needed a fair and repeatable way to compare build behavior across Debian 12 and Debian 13 Docker environments. - Previous checks mixed relative verdict logic and did not provide a clean per-image pass/fail matrix. - Experimental artifacts needed to be grouped in a dedicated location with local documentation and ignore rules. Actions: - Added an isolated experiment package under experiments/debian12-13. - Added Debian 12 and Debian 13 baseline Dockerfiles plus a Debian 13 remediated Dockerfile that pins a known-good toolchain workflow. - Implemented run-compare.sh to execute the same build workflow per image and report per-case PASS/FAIL with toolchain versions and exit codes. - Updated the experiment README to describe usage, optional flags, and matrix-style output. - Added a local .gitignore for experiment artifacts and made the script clean the experiment out/ directory before running. Result: - The repository now has a self-contained, fair comparison harness that reports Docker-specific build pass/fail outcomes. - Debian 13 remediation success can be verified as its own independent case without altering main project docs. - Experiment files, docs, and ignores are consolidated in one dedicated directory.
Problem: - The Debian image comparison output made PASS/FAIL states harder to scan quickly in terminal runs. - Case results were printed as plain text, which slowed visual review during repeated experiments. Actions: - Added terminal-aware ANSI coloring in experiments/debian12-13/run-compare.sh. - Rendered PASS in green and FAIL in red for per-case workflow output. - Applied the same coloring in the final result matrix while keeping plain output for non-TTY environments. Result: - Per-image outcomes are now immediately distinguishable in interactive runs. - Logs remain compatible with non-interactive environments without escape-code noise.
|
and use it for actual thumb functions |
|
@m-kozlowski confirmed |
|
What is this necessary/useful for? Debian packaging? Is this directly relevant to the project itself? |
|
This is not really about Debian packaging, and Debian itself is not the root cause. What the experiment shows is that the build result changes with the arm-none-eabi toolchain version: the older toolchain links successfully, while the newer GCC/binutils combination fails at link time with I found this while trying to introduce continuous integration testing to this project. This PR is closer to an issue/reproducer. The likely root cause is that I think the correct order is:
The actual fix will probably require changing quite a few stub entries. |
Problem:
Actions:
Result:
Debian 13 docker toolchain fails to link (dangerous relocation), while Debian 12 succeeds
Summary
On this repository, the same build workflow (
make binaries) behaves differently by Docker base image:This is reproducible with isolated experiment files under
experiments/debian12-13.Background
We needed a fair, per-image comparison (same command, same source tree, isolated output dirs).
A matrix script was added to run identical build steps per image and report PASS/FAIL independently.
Relevant experiment files:
experiments/debian12-13/run-compare.shexperiments/debian12-13/Dockerfile.debian12experiments/debian12-13/Dockerfile.debian13experiments/debian12-13/Dockerfile.debian13-remediatedexperiments/debian12-13/README.mdEnvironment
luckyabsoluter/airbreak-plusmake binaries BUILD=<case-output-dir>Reproduction
Observed Results (per-image)
debian12
PASSarm-none-eabi-gcc (15:12.2.rel1-1) 12.2.1 20221205GNU ld (2.40-2+18+b1) 2.40debian13
FAIL(exit=2)arm-none-eabi-gcc (15:14.2.rel1-1) 14.2.1 20241119GNU ld (2.44-3+23+b1) 2.44common_code.elfdebian13-remediated
PASSgcc 12.2.1,ld 2.40Key Error Logs (Debian 13 baseline)
Notes
This strongly suggests a toolchain compatibility issue (gcc/binutils/newlib set in Debian 13) rather than source or workflow differences, because:
Proposed next steps