Close the release gate, pin the last three deps, and stop truncating the clock - #198
Conversation
…the clock
Four findings from the review, all in production code rather than tests.
THE RELEASE GATE HAD NO GATE. check_version.sh recognised a release tag as /^v[0-9]/
while release.yml triggers on the far wider "v*", so a tag of vtest, v-wip or a bare v
left $tag empty, printed the declared version and exited 0 -- a release built and
published with no version check at all. release.yml names its release ${REF_NAME#v},
so vtest would have shipped as "test".
The discriminator is now the ref TYPE rather than the shape of the name, which is what
it always meant: GITHUB_REF_TYPE=tag. A branch is never read as a version whatever it
is called, which is the false positive the old name-shape guard existed to avoid.
It also gets a --self-test, run in CI on every push rather than only at tag time -- a
broken gate should surface before the release that needs it. The first version of that
self-test was itself vacuous: it passed each tag as an ARGUMENT, and every case still
passed with the bug reintroduced, because the argument path was never the broken one.
It now drives the environment with no argument, exactly as release.yml does, and the
three tags I found empirically are the three it fails on.
DEPENDENCIES. ArduinoJson and espMqttClient carried carets (^7.4.3, ^1.7.3), which is
not a pin: it accepts any later minor. .pio/ is gitignored and there is no lockfile, so
the resolved set lived only in whatever machine built last -- and this builds an OTA
image, where "the version that shipped" has to be reconstructible from the tag. Both
were already resolving to the versions now named, and both are the latest release
upstream, so this pins current behaviour rather than changing it.
THE CLOCK. main.cpp narrowed nowMs() to uint32 on the way into breadcrumbs::tick(),
forty lines below a comment warning that millis() wraps every 49.7 days and that
casting a wrapped value to uint64 does not un-wrap it. The record exists to say "this
life reached this uptime", so the bridges it was built for -- the long-lived ones --
were exactly the ones it misreported, and it misreported them as a plausible smaller
number: a 60-day life read as 10.3 days.
Storage now holds SECONDS in the same uint32, which reaches 136 years and keeps the
struct at sixteen bytes, so the CRC window and the RTC-RAM layout are untouched.
Nothing is lost: tick() only ever wrote once per second.
EXTERNAL BEHAVIOUR CHANGE, flagged deliberately. /api/v1/diagnostics still publishes
previous_uptime_ms in milliseconds, but the value is now always a whole number of
seconds. Accuracy is unchanged -- it was already only sampled once per second -- but
a consumer reading the low digits will see zeros where it used to see noise. The
field also now carries values past 2^32, and the payload is tested for that.
MQTT THROTTLE. PublishThrottle::Sample held its id as a std::string, so every publish
rebuilt the whole vector: one allocation per channel, freed on the next, at least once
a minute per device for the life of the bridge. measurement.h keeps ids as const char*
for exactly that reason -- it records 80-120 short-lived allocations per poll as "a
classic fragmentation hazard" -- and this was the one place downstream that put them
back on the heap. Every id is an inline constexpr constant with static storage, so the
pointer is safe to borrow. The comparison is now an explicit strcmp: it would have
silently become a pointer comparison otherwise, which happens to work for interned
constants and is not something to depend on.
Verified: 1024 native cases, check_layering.sh, the new self-test, and a real
waveshare-rs485-can build with no new warnings.
… no check The previous commit made GITHUB_REF_TYPE=tag the sole signal that this is a release. That is correct for GitHub Actions -- release.yml runs on ubuntu-latest and its env: block adds BOARDS without displacing the defaults, so the variable is there -- but it made a variable that has never been absent into a single point of failure for the gate, and the failure mode is silence. With no ref type at all, a v-shaped name is now checked by the old heuristic instead of waved through. A branch wrongly read as a version is a loud failure on a pull request; a release that skips its version check is a silent one. Given the choice, be loud. Two more self-test cases cover it, and the whole set still fails on the original bug.
Found while reviewing the branch that hardens the tests around json_limits::finish(). home_assistant_discovery.cpp carried a private serialise() that was finish() with the `needed > maxBytes` check taken out: it caught an overflowed document but let a well-formed one grow to whatever the heap allowed. That was the wrong path to be missing it. Discovery payloads are the LARGEST this device publishes -- every entity repeats the whole device block -- and identity strings are unbounded std::strings filled from whatever a device reports over RS485. So the one output with no ceiling was the one that emits the most bytes, built from the least trusted input. It now calls the shared finish() at kMaxPayloadBytes, the same ceiling the state payloads already use. Over that, the entity is dropped rather than published oversized, which is the convention every other payload here follows. Measured before choosing the ceiling: the largest entity this fixture produces is well under half of it, and the test asserts that headroom rather than a bare "it fits" -- the device block grows when a field is added, and a ceiling sat under by a hair would start dropping entities on the next one. The test asserts the bound through buildDiscoveryEntities rather than beside it, with a device reporting an absurd model name. That is what makes the guard testable at all: removing an upper bound is strictly MORE permissive, so a test that only checks real payloads fit cannot detect its absence. Proven -- unbounded, that fixture publishes 14 oversized entities; bounded, none. Verified: 1025 native cases, check_layering.sh, and a waveshare-rs485-can build.
|
Two additions since the description above, both from reviewing this work rather than from the original findings. A fallback when Home Assistant discovery had its own serialiser, without the size check. It now uses the shared Proving it needed care: removing an upper bound is strictly more permissive, so a test that only checks real payloads fit cannot detect its absence. The assertion goes through |
Review found the upgrade path, and it is the worst kind of defect: silent, fleet-wide, and introduced by the fix two commits back. Renaming heartbeatUptimeMs to heartbeatUptimeSeconds changed a field's UNIT without changing the layout. The CRC covers the same twelve bytes it always did, so a record left in RTC RAM by the previous firmware still validates -- and is then read as seconds. RTC_NOINIT survives exactly the reset an OTA performs, so on the first boot into that firmware a bridge that had genuinely been up three days would have published three thousand. Not a plausible-looking error: a thousandfold one, on every device in the fleet, through /api/v1/diagnostics. It self-corrects on the following boot, which is what would have made it hard to believe when someone reported it. storageCrc() now folds a schema tag into the checksum. It is not stored -- it costs no RTC bytes -- and a record written under a different schema simply fails validation and reads as cold. The price is one cold start per device: the boot count restarts and the previous life is reported as unknown, which is the honest answer, because those bytes genuinely cannot be interpreted. Bump the tag whenever a field's meaning, unit or width changes. crc32() becomes crc32Update(), carrying the running value without the final inversion, so the checksum can span the tag and the struct. Also restored a property the same commit had quietly dropped. The old millisecond throttle compared with <, so it could never store a value smaller than the one already there; rewriting it as == gave that up. esp_timer_get_time() does not run backwards, so nothing in the firmware reaches it -- but tick() takes the clock as an argument, so the property can be asserted rather than merely asserted about, and now is. Both are mutation-proven: removing the schema tag fails the upgrade test, and == in place of <= fails the monotonic one. And a comment corrected: Sample::id said every id is one of measurement.h's constants. A few tests pass bare string literals instead. Both have static storage duration so the lifetime argument is unaffected, but the claim as written was not true. Verified: 1027 native cases, check_layering.sh, waveshare-rs485-can builds clean.
Review found a defect I introduced two commits back — worth reading before mergingRenaming
Fixed by folding a schema tag into the CRC. Not stored, so it costs no RTC bytes; a record written under a different schema fails validation and reads as cold. ⚠ Second external behaviour change, on top of the one in the descriptionEvery device will report one cold start when this firmware lands, and its boot count restarts from 1. That is the price of the fix, and it is the honest answer — those bytes genuinely cannot be interpreted. The previous life is reported as unknown rather than as a wrong number. Bump Also in this commit
And a property the same commit had quietly dropped: the old millisecond throttle compared with Both mutation-proven: removing the schema tag fails the upgrade test; |
Third of three PRs from the codebase review. Production code this time, not tests.
The release gate had no gate
check_version.shrecognised a release tag as/^v[0-9]/whilerelease.ymltriggers on the far widerv*. So:GITHUB_REF_NAME=vtest bash tools/check_version.sh # exit 0, nothing checkedvtest,v-wipand a barevall built and published a release with no version check at all — andrelease.ymlnames its release${GITHUB_REF_NAME#v}, sovtestwould have shipped as "test".The discriminator is now the ref type, which is what it always meant. A branch is never read as a version whatever it is called — the false positive the old name-shape guard existed to avoid.
It also gets a
--self-test, run in CI on every push rather than only at tag time; a broken gate should surface before the release that needs it.Dependencies
Three caret ranges (
^7.4.3×2,^1.7.3) — not pins; they accept any later minor..pio/is gitignored and there is no lockfile, so the resolved set lived only in whatever machine built last. This builds an OTA image, where "the version that shipped" has to be reconstructible from the tag.Both were already resolving to the versions now named, and both are the latest release upstream — this pins current behaviour rather than changing it.
The clock
main.cppnarrowednowMs()touint32_ton the way intobreadcrumbs::tick()— forty lines below a comment warning thatmillis()wraps every 49.7 days and that casting a wrapped value touint64does not un-wrap it.The record exists to say "this life reached this uptime", so the bridges it was built for — the long-lived ones — were exactly the ones it misreported, as a plausible smaller number: a 60-day life read as 10.3 days.
Storage now holds seconds in the same
uint32_t— 136 years, struct still sixteen bytes, CRC window and RTC-RAM layout untouched. Nothing is lost:tick()only ever wrote once per second.⚠ External behaviour change
/api/v1/diagnosticsstill publishesprevious_uptime_msin milliseconds, but the value is now always a whole number of seconds. Accuracy is unchanged — it was already only sampled once per second — but a consumer reading the low digits will see zeros where it used to see noise. The field now also carries values past 2³², and the payload is tested for that.MQTT throttle
PublishThrottle::Sampleheld its id as astd::string, so every publish rebuilt the whole vector: one allocation per channel, freed on the next, at least once a minute per device for the life of the bridge.measurement.hkeeps ids asconst char*for exactly that reason — it records 80–120 short-lived allocations per poll as "a classic fragmentation hazard" — and this was the one place downstream that put them back on the heap. Every id is aninline constexprconstant with static storage, so the pointer is safe to borrow.The comparison is now an explicit
strcmp: it would have silently become a pointer comparison otherwise, which happens to work for interned constants and is not something to depend on.Verification
1024 native cases ·
check_layering.sh· the new self-test (mutation-proven against the original bug) · a realwaveshare-rs485-canbuild with no new warnings.