Skip to content

C-103: a missing producer client refuses instead of passing for 'no boundary' — and the Tier 2 premise did not survive verification - #281

Merged
Polichinel merged 3 commits into
developmentfrom
fix/c103-observed-range-clip-fails-loud
Aug 17, 2026
Merged

Polichinel merged 3 commits into
developmentfrom
fix/c103-observed-range-clip-fails-loud

Conversation

@Polichinel

Copy link
Copy Markdown
Collaborator

C-103 was registered yesterday on a premise measured only in this repository's venv. Verifying it changed both the tier and the fix, so both are here.

What verification found

The open question was whether the production launcher supplies datafactory_query. It does, twice over:

  • views-models postprocessors/un_fao/requirements.txt and postprocessors/un_crafd/requirements.txt pin views-datafactory>=1.9.0,<2.0.0 — which is what ships the module. There is no separate distribution: pip download datafactory-query finds nothing; datafactory_query is one of nine packages inside views-datafactory's wheel.
  • More decisively: config_queryset.py imports datafactory_query.defaults at module scope and raises a RuntimeError naming the fix. A missing client makes get_queryset() return None, which launch_config.assert_queryset_was_importable turns into a refusal (C-83) before _read_historical_frame runs.

So the headline scenario — a missing dependency silently shipping fabricated months on the live FAO path — cannot occur. It was already guarded, by a check written for an entirely different reason.

C-103 is re-tiered 2 → 3. The instruction the entry borrowed from C-26 — "do not downgrade on inspection of this repo alone" — was the right one: the answer came from reading the launcher, not from reading here.

What was actually wrong, and is fixed

The except Exception was never only about the import. It made two different events one:

event what it means right response
producer publishes no boundary a normal, older store degrade open — the recorded C-26 decision
client cannot be imported at all a broken environment refuse

Collapsing them ships the unobserved zero-padded tail as observed history on the strength of one WARNING. C-60 is the same shape: a provenance stamp that degraded to "unknown" on a bare except and made every delivery untraceable in the one field it existed to answer.

  • source_metadata raises ProducerClientMissing on ImportError — logged and raised (ADR-008), naming the package, the install command, and what degrading open would have cost.
  • both managers re-raise it ahead of the broad except.
  • the broad degrade-open is deliberately unchanged. A network failure, an auth error, a reshaped .zattrs still degrade open. Narrowing that is a decision about what to tell the partner when the boundary cannot be read — not a refactor — and it is what keeps C-103 open at Tier 3.

Tests

contract/source_metadata.py had no tests at all, which is how "return None like everything else" ever looked reasonable. tests/test_source_metadata.py adds 7, and they are mutation-proven both ways:

mutation result
revert the raise to return None 4 failed
collapse the manager's two branches into one 1 failed (the ordering check)
neither 7 passed

The manager check is a source read rather than a behavioural one on purpose: constructing a manager needs pipeline-core, a path manager and an Appwrite environment (C-40), and the property worth holding is a single line — that the narrow branch precedes the broad one.

Verification

  • ruff check . — clean.
  • Full suite locally: 439 passed, 3 skipped, 37 xfailed. The 25 failures are the C-104 venv drift, unchanged and green in CI.

Correction: the commit message on 08b9222 states "1 skipped, 39 xfailed" — I wrote it before reading the run. The measured figures are 3 and 37, as above. Left uncorrected in git rather than force-pushed.

Polichinel and others added 3 commits August 17, 2026 11:29
…o boundary" (C-103)

C-103 was filed on an unverified premise. Verifying it moved the entry down a
tier and changed what the fix should be, so both are recorded here.

WHAT THE VERIFICATION FOUND. The question was whether production supplies
`datafactory_query`. It does, twice:

  - views-models postprocessors/{un_fao,un_crafd}/requirements.txt both pin
    views-datafactory>=1.9.0,<2.0.0, which is what ships the module. There is no
    separate distribution — `pip download datafactory-query` finds nothing; it is
    one of nine packages in views-datafactory's wheel.
  - config_queryset.py imports datafactory_query.defaults at MODULE SCOPE and
    raises naming the fix. A missing client makes get_queryset() return None,
    which launch_config.assert_queryset_was_importable refuses (C-83) before
    _read_historical_frame runs.

So the headline scenario — a missing dependency silently shipping fabricated
months on the live path — cannot occur. It was already guarded, by a check
written for something else. C-103 drops 2 -> 3.

WHAT IS ACTUALLY WRONG, AND IS FIXED HERE. The `except Exception` was never only
about the import. It made "the client is missing" and "the producer publishes no
boundary" the same event. The second is a normal older store and degrading open
is the recorded C-26 decision; the first is a broken environment, and degrading
open there ships the unobserved zero-padded tail as observed history on one
WARNING. C-60 is the same shape: a provenance stamp that degraded to "unknown"
on a bare except and made every delivery untraceable in the one field it existed
to answer.

  - source_metadata raises ProducerClientMissing on ImportError, logged AND
    raised (ADR-008), with a message naming the package, the install command,
    and what degrading open would have cost.
  - both managers re-raise it ahead of the broad except.
  - the broad degrade-open is deliberately UNCHANGED. Narrowing it is a decision
    about what to tell the partner when the boundary cannot be read, not a
    refactor, and it is what keeps C-103 open.

TESTS. contract/source_metadata.py had NONE — which is how "return None like
everything else" ever looked reasonable. tests/test_source_metadata.py adds 7,
mutation-proven both ways: reverting the raise to `return None` fails 4 of them;
collapsing the manager's two branches into one fails the source check that pins
the ordering. That check is a source read rather than a behavioural one on
purpose — constructing a manager needs pipeline-core, a path manager and an
Appwrite environment (C-40), and the property worth holding is one line.

Suite: 439 passed, 1 skipped, 39 xfailed locally (25 failures are the C-104 venv
drift, unchanged). ruff clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…he CIC

Two findings on the previous commit.

1. The new manager check hardcoded ("unfao", "crafd") in its parametrize, when
   `tests/conftest.py::PARTNER_PACKAGES` exists for exactly this and says why:
   eight guards once named "unfao" literally, and all eight went on passing over
   `crafd/` when it landed. This would have been the ninth. Now parametrized
   over the declared list, like `test_store_construction`.

2. `docs/CICs/UNFAOPostProcessorManager.md:96` described `_read_historical_frame`
   as degrading open "if the boundary cannot be resolved" — true before this
   branch, and now only half true. It records both outcomes: degrade-open when
   the producer publishes no boundary or the read fails, refusal when the client
   cannot be imported. Review date moved to match its own content, per
   `test_doc_accuracy::test_a_cic_review_date_is_not_older_than_its_own_content`.

Also simplified the ordering assertion in the same check. It searched for the
broad `except Exception:` from an offset before the refusal, which was fragile
and obscured that each manager has exactly one. It now asserts that count and
searches forward — re-mutation-proven: removing the narrow branch still fails it,
reordering the two branches still fails it.

Suite: 439 passed, 3 skipped, 37 xfailed. ruff clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nistically

Six findings from /code-review high. The first two matter.

1. MEDIUM — the guard caught `ImportError`, and the failure this environment has
   actually had is not one. views-models `postprocessors/un_fao/requirements.txt`
   records, dated 2026-08-13, that resolving numpy 2.x into the shared
   envs/views-postprocessing prefix makes the import die with
   `ValueError: numpy.dtype size changed ... Expected 96 from C header, got 88`,
   found by the pre-delivery rehearsal. An ImportError-only clause lets that sail
   into the caller's degrade-open and ship the unobserved tail — the exact case
   this guard exists to separate, and the likeliest one. Now catches Exception.

2. MEDIUM — four of the tests proved the guard by ambient accident: they relied
   on `datafactory_query` happening to be absent in this venv, so in the launcher
   prefix (where it IS installed) all four would have failed. That is the
   C-30/C-46 shape a day after fixing it. Both states are now simulated
   deterministically: an `absent_client` fixture, and an `exploding_client` one
   that reproduces the numpy ABI ValueError.

3. LOW — the message asserted a diagnosis. An ImportError raised INSIDE an
   installed package was reported as "not importable ... pip install
   views-datafactory", sending the operator to a fix already applied. It now
   distinguishes `ModuleNotFoundError` naming datafactory_query ("not installed",
   with the install line) from anything else ("present but raised while loading",
   quoting the error, and "Do NOT reinstall").
   `ProducerClientMissing` -> `ProducerClientUnavailable`: one type, two causes,
   and the old name asserted the cause that is not the common one.

4. LOW — `_read_historical_frame`'s docstring still said "degrade-open" flat.
   Corrected in both managers. C-107 is the entry recording that docstrings sit
   outside test_doc_accuracy's corpus, so nothing would have caught it.

5. LOW — C-103's Location was made stale by the previous commit (`:37` is now
   mid-docstring). Re-read, and `pyproject.toml` dropped from it: the entry's own
   body concludes the dependency is the launcher's to declare and that both
   launchers do.

6. LOW — the manager tripwire compared file offsets without tying them to the
   same `try`, and misreported a removed degrade-open as "grew a second broad
   except". Now scoped to `_read_historical_frame` and counts both branches, so
   the message says what actually happened.

Mutation-proven, all three ways: narrowing the catch back to ImportError fails
the ValueError test; dropping the installed-vs-missing classification fails the
message test; removing the degrade-open branch fails the tripwire with the
correct message.

Suite: 440 passed, 3 skipped, 37 xfailed. ruff clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Polichinel

Copy link
Copy Markdown
Collaborator Author

Review round — /review-diff + /code-review high

/review-diff — 2 findings (b0d3fcb)

  1. The new manager check hardcoded ("unfao", "crafd") in its parametrize, when conftest.PARTNER_PACKAGES exists because eight guards once named "unfao" literally and all eight went on passing over crafd/ when it landed. This would have been the ninth.
  2. docs/CICs/UNFAOPostProcessorManager.md:96 described _read_historical_frame as degrading open "if the boundary cannot be resolved" — true before this branch, half true after.

/code-review high — 6 findings (0c9b4e7)

The first one is the important one, and it makes the original fix insufficient.

The guard caught ImportError. The failure this environment has actually had is not one — views-models postprocessors/un_fao/requirements.txt records, dated 2026-08-13:

numpy stays on 1.x … the pandas wheel in envs/views-postprocessing is built against the numpy 1.x C ABI. Importing the delivery manager then dies at … ValueError: numpy.dtype size changed … Expected 96 from C header, got 88. Found 2026-08-13 by the pre-delivery rehearsal, before the FAO run, not during it.

An ImportError-only clause lets that sail straight into the caller's degrade-open and ship the unobserved tail — the exact case this PR exists to separate, and the likeliest one to occur. Now catches Exception.

Second: four of my tests proved the guard by ambient accident — they relied on datafactory_query happening to be absent in this venv, so in the launcher prefix, where it is installed, all four would have failed. That is the C-30/C-46 shape one day after fixing it. Both states are now simulated deterministically (absent_client, and an exploding_client reproducing the ABI ValueError).

The rest: the message asserted a diagnosis it had not established (an ImportError from inside an installed package was reported as "not installed — pip install …", sending the operator to a fix already applied); ProducerClientMissing → ProducerClientUnavailable, because the old name asserted the cause that is not the common one; the _read_historical_frame docstring still said "degrade-open" flat; C-103's Location was made stale by my own previous commit; and the tripwire compared file offsets without tying them to the same try, and misreported a removed degrade-open as "grew a second broad except".

Mutation proof

mutation caught by
narrow the catch back to except ImportError the ValueError test
drop the installed-vs-missing classification the message test
remove the manager's narrow branch the tripwire
reorder the two branches the ordering assertion
remove the degrade-open branch the count assertion, with an honest message

Verification

  • CI: 463 passed, 5 skipped, 37 xfailed, 0 failed.
  • ruff check . clean. Local: 440 passed, 3 skipped, 37 xfailed (25 failures are C-104 venv drift, green in CI).

@Polichinel
Polichinel merged commit 34c1c06 into development Aug 17, 2026
4 checks passed
@Polichinel
Polichinel deleted the fix/c103-observed-range-clip-fails-loud branch August 17, 2026 09:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant