Repository navigation
A month nobody reported reached CRAF'd as observed history — the clip exists and did not fire #297
Description
Important
Correction, 2026-08-25 — the premise below is measured on one column of three, and it is too strong.
July 2026 is not a zero-filled month. It carries 6 non-zero cells of 64,742: lr_ged_sb = 0.0 (the figure below, correct) but lr_ged_ns = 12.0 and lr_ged_os = 3.0. So it is a month reported at ~1% of normal coverage, not a month nobody reported.
That confirms candidate 1: the producer's inferred boundary (C-355) saw those six cells, declared July observed, and the clip kept it — correctly, by its own contract. Nothing in this repository was broken. Candidates 3 and 4 are eliminated from code. Full working in this review; the original body is preserved below unchanged.
CRAF'd received a historical artifact containing a complete, zero-filled month for a month nobody had reported — July 2026, all 64,742 cells, sum(lr_ged_sb) = 0.0 — indistinguishable from a month with genuinely no fatalities. Filed downstream as views-datafactory#476 (now closed); measured on historical_dataset_20260814_203554.parquet.
The producer half is fixed and merged — views-datafactory#478 / ADR-052. Every source's coverage bounds are now published in the store attrs and the consumer manifest, where before only the leading edge was.
This half is ours. We already have the clip: _read_historical_frame in views_postprocessing/crafd/managers/crafd.py reads last_valid_month_id, calls observed_range.fabricated_months, and drops anything above it. It landed in 9799e87 on 2026-08-02, twelve days before the affected artifact was cut.
Three candidates, and one of them is not our bug
1. The clip worked correctly and the producer's boundary was too permissive.
The datafactory cron is monthly, the 21st at 00:00. So the store live on 2026-08-14 was the 2026-07-21 export — and that export's boundary reflected UCDP as of 21 July, when July was at best partially reported.
The producer's boundary is inferred, not declared: a month counts as observed when its slice sums above zero (views-datafactory C-355). Any non-zero July event present on 21 July would have set last_valid_month_id = 559, and the clip would then have kept July — correctly, by its own contract. The zero-filled month would be the producer's fault, not the clip's.
2. It degraded open. The same function catches broadly and warns:
last_valid_month_id could not be read; skipping the observed-range clip (degrade-open, C-26). Any unobserved months above the producer's boundary WILL ship as observed history in this delivery.
3. That artifact came via a different read path than the frame-native one the clip guards — _read_historical_data versus _read_historical_frame.
Candidate 1 was missing from the first version of this issue and may well be the answer. It is also the only one where nothing here is broken.
The decisive number, and why it is probably gone
All three are settled by one fact: what was last_valid_month_id when that artifact was cut?
≥ 559→ the clip behaved correctly; the defect is the producer's inferred boundary (C-355), and this issue closes as not our bug.< 559→ the clip should have dropped July and did not; candidate 2 or 3.
Both places it might be recorded turn out to be empty:
- The previous store does not survive.
export_zarr.pyrenames the live store to.oldand thenrmtrees it in the same run. There is no 2026-07-21 export on disk. - The artifact does not carry it.
delivery/provenance.py::build_provenancerecordslookup_version,region,expected_cell_count,actual_cell_countandunmapped_count— not the boundary it clipped against.
So the specific 2026-08-14 cause may be unrecoverable. Say so if it is, rather than picking the most plausible candidate and writing it down as fact.
Correction to the first version of this issue: there is no run log to check
It said "check the 2026-08-14 run log". This repository configures no logging and has no delivery entrypoint — no logging.basicConfig, no FileHandler, no run script. The manager executes under views-pipeline-core's postprocessor machinery and its logger output goes wherever that configures it.
The realistic place to look is wandb — CrafdManager.__init__ takes wandb_notifications: bool = True. If the degrade-open warning was emitted on 2026-08-14, that is where it survives.
What is worth doing regardless of whether the cause is recoverable
Stamp the observed-range boundary into the delivery provenance record.
This question — was this month observed or fabricated, and against what boundary did you decide? — is exactly what a partner asks after the fact, and right now the artifact cannot answer it. One field in build_provenance makes every future instance answerable from the artifact itself, with no logs and no guessing.
The producer now publishes last_valid_month_ids (per source) alongside last_valid_month_id, so there is more to stamp than there was.
Definition of done
- Check wandb for the 2026-08-14 run and the degrade-open warning
- If found: candidate 2 — decide whether this degrade-open should remain open, given the failure it produces is a partner receiving fabricated history labelled as observed, and record the decision
- If not found: check whether that artifact used
_read_historical_datarather than_read_historical_frame— candidate 3 means the clip guards one read path and not the other - If neither: record that the cause is unrecoverable. Do not attribute it to the most plausible candidate
- Regardless: stamp
last_valid_month_idandlast_valid_month_idsintobuild_provenance, so the next instance is answerable from the artifact - Tell CRAF'd the outcome on views-datafactory#476 (closed, but reopenable)
One producer-side caveat that affects any fix here
Even a working clip currently keeps partial months, because the boundary it trusts is inferred (C-355). The store exported 2026-08-21 declares last_valid_month_id = 560 — August 2026, the month the export ran in.
When C-355 closes, the boundary moves earlier, never later. Anything built here should treat it as a ceiling rather than a frontier.
Cross-ref: views-datafactory#476, #478, ADR-052, C-355, C-352 · register C-22, C-26, C-103
Activity
Polichinel commentedon Aug 24, 2026
Body rewritten before anyone acts on it. Three corrections, one of which changes what to do first.
"Check the 2026-08-14 run log" was wrong — there is no run log. This repo configures no logging and has no delivery entrypoint: no
logging.basicConfig, noFileHandler, no run script. The manager runs under views-pipeline-core's machinery. wandb is the realistic place (wandb_notifications: bool = True), not a file on disk.A third candidate was missing, and it may be the answer: the clip worked correctly. The datafactory cron is monthly on the 21st, so the store live on 2026-08-14 was the 2026-07-21 export, whose boundary reflected UCDP as of 21 July. That boundary is inferred from non-zero sums (views-datafactory C-355) — one stray July event would have set it to 559 and the clip would have kept July by contract. In that case nothing here is broken and this closes as not-our-bug.
The decisive number is probably unrecoverable, and I should have checked that before assigning the work.
export_zarr.pyrmtrees the.oldstore in the same run, so no previous export survives; andbuild_provenancerecordslookup_version, region and cell counts but not the boundary it clipped against. Both places one would look are empty.So the highest-value item is now forward-looking rather than forensic: stamp the boundary into the delivery provenance record. "Was this month observed, and against what boundary did you decide?" is precisely what a partner asks afterwards, and the artifact currently cannot answer it. One field makes every future instance answerable without logs.
The definition of done now includes an explicit option to record the cause as unrecoverable rather than attributing it to the most plausible candidate.
Polichinel commentedon Aug 25, 2026
Review — the month is not zero-filled, the decisive number was never lost, and candidate 1 is confirmed
Measured today against the artifact this issue names. The premise needs correcting before anything is built on it.
July 2026 is not empty. It is 1% reported.
historical_dataset_20260814_203554.parquet, read at both surviving copies (views-models/postprocessors/un_crafd/data/generated/…andviews-crafdapi/apis/un_crafd/cache/appwrite_cache/crafd_bucket/…):lr_ged_sblr_ged_nslr_ged_osSummed: month 559 is
lr_ged_sb = 0.0,lr_ged_ns = 12.0,lr_ged_os = 3.0.The body reports
sum(lr_ged_sb) = 0.0and that figure is correct — but it is one of the three delivered columns, and the issue generalises it to "a complete, zero-filled month for a month nobody had reported". Six cells carry events. What the artifact contains is not a fabricated month; it is a sparsely-reported frontier month — about 1% of the preceding months' coverage.Also worth recording: month 560 (August) is absent. The artifact ends at 559.
Which confirms candidate 1, and settles this as not our bug
The producer's boundary is inferred from non-zero sums (views-datafactory C-355). July's slice was not zero on the 2026-07-21 export — those six cells were there. So
last_valid_month_idwas ≥ 559, and_read_historical_framekept July because the producer declared it observed. The clip did exactly what its contract says.The body anticipated this exactly — "any non-zero July event present on 21 July would have set
last_valid_month_id = 559, and the clip would then have kept July — correctly, by its own contract" — and it is what happened. Candidate 2 (degrade-open) is not needed to explain the artifact, and no wandb lookup is required to reach that.The decisive number was recoverable — from the artifact, not the logs
The body concludes it is "probably gone" because the previous store is
rmtreed andbuild_provenancedoes not stamp the boundary. Both facts are true (verified: nobasicConfig, noFileHandleranywhere in the package;build_provenancetakes exactlylookup_version,region,expected_cell_count,actual_cell_count,unmapped_count).But the boundary is inferable from the delivered data itself, because the producer's rule is a function of that data. That is worth noting as a general technique for the next instance, and as the reason to check the artifact before the logs.
Candidate 3 is impossible — strike it from the definition of done
_read_historical_datadoes not bypass_read_historical_frame; it calls it (crafd.py:212). It is a validating wrapper — queryset importable,data_format: feature_framedeclared — and then the single read path. Verified against the exact code in place on the day:ff0278b(2026-08-14) is identical in this respect.There are not two read paths. There is one, with a gate in front of it.
A fourth candidate was never listed, and it is eliminated only narrowly
Was the clip in the version that ran? Commits are not what deploys; tags are.
9799e87(2026-08-02) brought the clip tocrafd/VIEWS_POSTPROCESSING_PIN="1.1.0"in7e0739con 2026-08-13So the clip was live by one day. A delivery cut on 12 August would have had no clip at all, and this investigation would have had a different answer. Worth recording because the margin, not the outcome, is the finding.
Scope correction: this is not CRAF'd-only
unfao/managers/unfao.pyandcrafd/managers/crafd.pyare line-for-line identical here — same clip, same degrade-open, same wrapper, at the same line numbers (160–213). Whatever is decided, it lands in both, and the partner on the UN-FAO side is external. A fix merged tocrafd/alone would leave the FAO path exactly as it is.What should still be built, and it is now better motivated
Stamping the boundary into
build_provenanceremains right — this review only reached its answer because the delivered columns happened to make the boundary inferable, which is luck, not design. Had July's six cells beenged_sbrather thanns/os, the data would have been mute.And the body's closing caveat is the real story: the boundary is inferred, so six cells in one month declared July "observed". That is C-355, and when it closes the boundary moves earlier, never later. Anything stamped here should be recorded as the ceiling this run clipped against, not as a claim that everything below it is fully reported — because in this artifact, plainly, it is not.
Correct the record with the partner
views-datafactory#476 is closed but reopenable. It, and anything CRAF'd was told, currently carries "a month nobody reported". The accurate statement is "a month reported at 1% of normal coverage, which an inferred boundary could not distinguish from a complete one." The producer-side fix (#478 / ADR-052, per-source coverage bounds) is correct either way and is if anything better justified by the real numbers — but CRAF'd may have acted on the stronger claim, and it is not the one the data supports.
Suggested definition of done, revised
Check wandb for the degrade-open warning— not needed; the artifact answers itCheck whether— impossible, one path_read_historical_datawas used instead of_read_historical_framens/osnon-zero)last_valid_month_id/last_valid_month_idsintobuild_provenance— in both partnersMerge pull request #298 from views-platform/feat/297-stamp-observed-r…
Polichinel commentedon Aug 25, 2026
The stamp has landed — PR #298, merged to
developmentbuild_provenancenow requiresobserved_throughand always emits it:559nullUNREADRequired rather than optional because a caller that forgets it is the failure being fixed. Always emitted because an absent key cannot be told from an artifact built before the field existed — which is the whole of this issue. And the third state is kept distinct deliberately: collapsing
UNREADtonullwould report "clip skipped" for a run whose clip in fact ran, which is the C-103 conflation one layer down.Two places it had to be registered rather than merely added: the essential set in
compact_description(the 255-char fallback would otherwise drop it exactly when descriptions run long — a guard that disappears when needed) and the redaction guard's declared keyset, which is a deliberate widening of what leaves the machine. It is amonth_idinteger: no PII, no credential, and precisely what a partner needs to tell a sparse month from a fabricated one.Landed in both partners. This was filed as a CRAF'd defect, but the managers are line-for-line clones and the UN-FAO side has the external partner. A test asserts they do not diverge.
Guard audit
Four mutations against production code, each caught:
test_the_boundary_is_recorded_before_the_degrade_open_returnobserved_throughfrom the essential compaction settest_the_boundary_survives_the_compaction_fallbackUNREADcollapse tonullinstead of raisingtest_building_provenance_without_reading_the_boundary_refusestest_both_partners_carry_it_identically(+1)507 passing, ruff clean, both managers at 442/450 lines.
Deferred, with a trigger
The per-source
last_valid_month_idsmap is not stamped.datafactory_query.defaultsexposes only the scalarget_last_valid_month_id, and a multi-source map would not fit the 255-char carrier in any case. Trigger: when C-15's structured metadata field lands upstream and the ceiling goes with it, stamp the map alongside the scalar.What is left on this issue
One item, and it is a message rather than code:
Everything else here is either done or struck as based on a misreading (candidates 3 and 4 — see the review above).
The standing caveat, unchanged
The boundary remains inferred, so six cells in one month were enough to declare July observed (views-datafactory C-355). What is now stamped is the ceiling this run clipped against, not a claim that everything below it is fully reported — in this artifact, plainly, it is not. When C-355 closes the boundary moves earlier, never later.
Polichinel commentedon Aug 25, 2026
All definition-of-done items are discharged.
_read_historical_datacalls_read_historical_framerather than bypassing it, and the clip was in the deployed version (1.1.0, pinned 2026-08-13 — one day before the artifact).One thing deliberately left, with a trigger rather than a date: the per-source
last_valid_month_idsmap is not stamped, becausedatafactory_query.defaultsexposes only the scalar and a multi-source map does not fit the 255-char carrier. Trigger: C-15's structured metadata field landing upstream.Reopen if the wandb check is ever wanted for its own sake — it is no longer needed to explain this artifact.