Skip to content

C-94: the findability preflight — ask the store what the consumer asks - #287

Merged
Polichinel merged 3 commits into
developmentfrom
feat/c94-findability-preflight
Aug 19, 2026
Merged

Polichinel merged 3 commits into
developmentfrom
feat/c94-findability-preflight

Conversation

@Polichinel

Copy link
Copy Markdown
Collaborator

The one failure mode where everything reports success and the partner sees nothing: the upload lands, storage is billed, the consumer's endpoint returns empty, and nothing raises anywhere.

It is not hypothetical. Run-0's historical artifact stranded on 2026-07-27 as a file with no metadata document (C-79) and nothing in this repository noticed. Every other mechanism the platform aims at this is a CI-time proxy — we check our label against the registry, the consumer checks theirs — and none of them observes the outcome of a real upload.

What it does

After both legs are uploaded, each manager queries the partner store the way the consumer does — name == product.CONSUMER_DOCUMENT_NAME, per category — and refuses a falsy answer. delivery/findability.py holds the rule; the managers own the query, because they own the port.

Two decisions worth not re-deriving

1. It runs on the existing key. Verified in the Appwrite console 2026-08-18: the live VIEWS Pipeline Core key already carries documents.read / rows.read / buckets.read / files.read. The registry's separate APPWRITE_READ_API_KEY slot would buy no isolation here, because the preflight runs inside the delivery process, which already holds the write key it just uploaded with. C-96's permission is about the operation being read-only, and it is. That slot stays for a preflight that runs outside the delivery, where the isolation would be real.

2. It suppresses pipeline-core's injected name filter. get_latest_file_id delegates to get_predictions_by_metadata, which merges the path manager's model name into every query. Without _build_partner_read_store setting model_path = None, the check would verify the views-models directory name — equal to the declared consumer name only by coincidence (C-77). Verifying the coincidence rather than the contract would leave this green while a rename took the delivery dark, which is the precise failure it exists to see.

Both legs are checked separately: a run whose forecast landed and whose historical did not is invisible in exactly one half, and the historical leg is the one that actually stranded.

Scope — stated, not claimed

I over-claimed this entry earlier in the week and the register had already corrected me, so the module says it plainly. This catches "the delivery ran and the consumer cannot see it." It does not catch:

  • no delivery happened at all — the 2026-08-12 empty bucket was an upstream destructive migration with no run since. A post-upload check observes nothing when there was no upload.
  • stale data served over an emptied bucket — faoapi's warm per-key cache.

Both remain recorded as gaps in C-94. The tier stays at 2 because of them: the Tier 2 rationale was "upload succeeds… nothing raises anywhere", and for that cause something now raises — so the tier now rests on the gaps rather than on the mechanism.

Three of the repo's own guards shaped this

All working exactly as designed, which is worth recording:

  • The manager line budgets (450 file / 300 class) rejected the first draft, which put the preflight in the class. Its failure message prescribed the fix — a module-level function, callable without a manager or an Appwrite environment (C-40 (a)). Now 435/450 and 285/300.
  • test_clone_readiness refused the new module until it was declared in _MACHINERY, where the import-purity checks can see it. Same lesson as conftest.PARTNER_PACKAGES.
  • My own new test refused assert_findable("") — an empty file id is "found something unusable", not a find. get_latest_file_id returns .get("fileId", None) when it finds a document missing that field. Same polarity _ContractStorePort.download learned as C-99. Fixed the code, not the test.

Verification

Mutation-proven — each fails its guard:

mutation caught
drop the call from _save_contract wiring check
un-suppress the injected name filter C-77 check
make the rule always pass 3 invariant tests

Suite: 449 passed, 3 skipped, 37 xfailed. The 26 failures are the C-104 venv drift, unchanged and green in CI. ruff clean. CIC updated with the new failure mode and its two stated gaps.

Polichinel and others added 3 commits August 19, 2026 10:32
…s (C-94)

C-94 is the one failure mode where everything reports success and the partner
sees nothing: the upload lands, storage is billed, and the consumer's endpoint
returns empty. It is not hypothetical — run-0's historical artifact stranded on
2026-07-27 as a file with no metadata document (C-79) and nothing here noticed.
Every other mechanism the platform aims at this is a CI-time proxy; none observes
the outcome of a real upload.

After both legs are uploaded, each manager now queries the partner store the way
the consumer does — `name == product.CONSUMER_DOCUMENT_NAME`, per category — and
refuses a falsy answer. `delivery/findability.py` holds the rule; the managers
own the query, because they own the port.

Two decisions worth not re-deriving:

- It runs on the EXISTING key. Verified in the Appwrite console 2026-08-18: the
  live `VIEWS Pipeline Core` key already carries documents.read / rows.read /
  buckets.read / files.read. The registry's separate APPWRITE_READ_API_KEY slot
  would buy no isolation here, because the preflight runs inside the delivery
  process, which already holds the write key it just uploaded with. That slot
  stays for a preflight that runs OUTSIDE the delivery.
- It queries through a store with pipeline-core's automatic `name == model_name`
  filter SUPPRESSED. `get_latest_file_id` merges the path manager's model name
  into every query, so without that the check would verify the views-models
  directory name — equal to the declared consumer name only by coincidence
  (C-77). Verifying the coincidence would leave this green while a rename took
  the delivery dark, which is the precise failure it exists to see.

Both legs are checked separately: a run whose forecast landed and whose
historical did not is invisible in exactly one half, and the historical leg is
the one that actually stranded.

Scope stated in the module rather than claimed: this catches "the delivery ran
and the consumer cannot see it". It does NOT catch "no delivery happened" (the
2026-08-12 empty bucket, an upstream migration with no run since) or "stale data
served from faoapi's warm cache". Both stay recorded as gaps in C-94, and the
tier stays at 2 because of them — the mechanism addresses the cause the tier
rationale named, and the gaps are what now hold it.

Three of the repo's own guards shaped this, all working as designed:

- the manager line budgets (450 file / 300 class) rejected the first draft, which
  put the preflight in the class. Its message prescribed the fix — a module-level
  function, callable without a manager or an Appwrite environment (C-40 (a)).
  Now 435/450 and 285/300.
- `test_clone_readiness` refused the new module until it was declared in
  `_MACHINERY`, where the import-purity checks can see it.
- my own new test refused `assert_findable("")` — an empty file id is "found
  something unusable", not a find, the same polarity `_ContractStorePort.download`
  learned as C-99. Fixed the code, not the test.

Mutation-proven: dropping the call, un-suppressing the name filter, and making
the rule always pass each fail their guard.

Suite: 449 passed, 3 skipped, 37 xfailed. The 26 failures are the C-104 venv
drift, unchanged. ruff clean. CIC updated with the new failure mode and its two
stated gaps.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…iew-diff

Two findings on the preflight.

1. SUBSTANTIVE. The preflight makes a live network call after the uploads have
   already succeeded. If that call RAISES rather than returning None, the error
   propagated from inside pipeline-core with nothing to say that the delivery
   landed and only the verification failed — so a transient store blip would read
   as "the delivery is invisible" and be quarantined.

   That is the same distinction this repo has now drawn three times: C-99 (an
   unrecognised store result is refused and named, not adapted to), C-103 (a
   missing producer client is not a producer that publishes no boundary), and now
   this. The two conditions call for different operator actions — a delivery that
   cannot be FOUND is quarantined; a delivery that could not be CHECKED may be
   perfectly fine. `findability.unverified()` names the second as
   `FindabilityUnverifiedError`, quoting what stopped the check.

2. Cosmetic: one blank line before the manager class where the file uses two
   everywhere else. Ruff did not catch it because the declared rule set is
   ["E4","E7","E9","F"] and blank lines are E3 — deliberately narrow (see the
   pyproject comment on why the set is pinned), so this is a style question the
   linter does not own.

Budgets after: 438/450 file, 285/300 class. Mutation-proven — collapsing the two
refusals back into one bare query fails the new wiring check.

Suite: 452 passed, 3 skipped, 37 xfailed; the 26 failures are C-104 venv drift.
ruff clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Six findings. The first would have made the guard worthless from the second
delivery onward, which is worse than not having it.

1. HIGH. The query asked "is there ANY document under the consumer's name for
   this category" — which the PREVIOUS delivery already answers yes to.
   Documents accumulate across runs; that is what makes "latest" meaningful to
   the consumer. So: run-2 uploads, `upload_data` reports success, its metadata
   document is never created (the exact C-79 shape), the query returns run-1's
   document, and the preflight logs "passed" while the consumer goes on serving
   run-1. The call-site comment claiming nothing above observes the outcome of an
   upload was true only for the very first delivery.

   `_ContractStorePort.upload` now returns the uploaded `file_id` instead of
   discarding it; the sink carries the manifest's id out (uploaded last, so it is
   the newest `category="forecast"` document); and the check asserts the newest
   document the consumer would find IS the one this run put there. The refusal
   distinguishes "nothing found" from "found the previous run's", because those
   are different operator situations.

2. NOT UPHELD, and the reason is worth more than the finding. The review said
   `unverified()` was unreachable because `get_predictions_by_metadata` swallows
   a failed search and returns []. True of 2.3.0 — what the drifted venv holds.
   FALSE of 3.0.1, which poetry.lock pins and CI installs: there it RAISES
   `MetadataSearchIncomplete`, with a comment giving our own argument back to us
   ("Returning [] here would tell every caller 'no predictions match', which is a
   statement about the shelf rather than about the lookup"; views-pipeline-core
   C-241). Verified by reading 3.0.1 from the sibling checkout rather than the
   installed package. This is C-104's hazard in its most expensive form yet: not
   a wall of red, but a confident and wrong conclusion about production drawn
   from a stale environment. Recorded in C-94.

3. Store construction moved inside the try, so a missing APPWRITE_UNFAO_* var
   after a successful upload reports "unverified" rather than a raw traceback.

4. The CIC and register both named `ProducerClientMissing`; the class is
   `ProducerClientUnavailable`. Renamed yesterday, references left stale — which
   is C-107 (docstrings and docs outside the doc-accuracy scan) landing on me
   within two days of registering it.

5. The structural tests were textual and did not test what their messages
   claimed. The interlock check only asserted the call appeared after the `if`,
   so moving it into the `else` kept it green; the preflight slice ran to
   end-of-file, so any later occurrence in the module satisfied it. Both now
   parse the AST: the call must be inside the `if upload_enabled:` body, and each
   slice is exactly one function. Tightening them immediately exposed that the
   declared name and the two legs are supplied at the CALL SITE, so the
   assertions were looking in the wrong function.

6. `_build_partner_read_store` was absent from the two C-40 structural guards in
   test_store_construction, so neither "is a module-level function" nor "no
   coordinate baked in" covered it. Declared in both.

Manager budgets held by trimming docstrings that restated the register rather
than pointing at it: 435/450 file, 289/300 class.

Mutation-proven: accepting any document (not just this run's) fails the new
run-scoping test; moving the call into the `else` branch now fails the interlock
test, which it would not have before.

Suite: 453 passed, 3 skipped, 37 xfailed. ruff clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Polichinel

Copy link
Copy Markdown
Collaborator Author

Review round — /review-diff + /code-review high

Both reviews found real defects. One of them would have made the guard worthless.

The finding that mattered — HIGH

The read-back asked "is there any document under the consumer's name for this category". The previous delivery already answers that yes. Documents accumulate across runs — that is what makes "latest" meaningful to the consumer.

So from delivery 2 onward the check could never fail:

run-2 uploads → upload_data reports success → its metadata document is never created (the exact C-79 shape) → the query returns run-1's document → the preflight logs "passed" → the consumer goes on serving run-1.

The call-site comment claiming "nothing above observes the OUTCOME of an upload" was true only for the very first delivery ever made.

Fixed by scoping to the run. _ContractStorePort.upload now returns the uploaded file_id instead of discarding it; the sink carries the manifest's id out (uploaded last, so it is the newest category="forecast" document); and the check asserts the newest document the consumer would find is the one this run put there. The refusal distinguishes "nothing found" from "found the previous run's" — different operator situations.

The finding I did not uphold, and why it is the more interesting one

The review said unverified() was unreachable, because get_predictions_by_metadata swallows a failed search and returns [].

That is true of 2.3.0 — what the drifted developer venv holds. It is false of 3.0.1, which poetry.lock pins and CI installs. There it raises MetadataSearchIncomplete, with a comment handing our own argument back to us:

"Returning [] here would tell every caller 'no predictions match', which is a statement about the shelf rather than about the lookup… a false negative to an external counterparty" — views-pipeline-core C-241

Verified by reading 3.0.1 from the sibling checkout rather than the installed package.

This is C-104's hazard in its most expensive form yet. Not a wall of red failures, but a confident and wrong conclusion about production drawn from a stale environment. Recorded in C-94.

The rest

# finding fix
3 store construction sat outside the try, so a missing APPWRITE_UNFAO_* var after a successful upload gave a raw traceback moved inside — reports "unverified"
4 CIC and register both named ProducerClientMissing; the class is ProducerClientUnavailable renamed. C-107 landing on me within two days of registering it
5 the structural tests were textual and did not test what their messages claimed — moving the call into else kept the interlock check green, and the preflight slice ran to end-of-file both parse the AST now; tightening them immediately exposed that the declared name and the two legs are supplied at the call site, so the assertions were looking in the wrong function
6 _build_partner_read_store absent from the two C-40 structural guards declared in both

Verification

Mutation-proven, including one the old tests would have missed:

mutation caught
accept any document, not just this run's run-scoping test
move the call into the else branch interlock test — would have passed before
drop the call entirely wiring test
un-suppress the injected name filter C-77 test
collapse unverified back into one refusal store-error test

CI: 477 passed, 5 skipped, 37 xfailed, 0 failed. Manager budgets held at 435/450 file and 289/300 class, by trimming docstrings that restated the register rather than pointing at it.

@Polichinel
Polichinel merged commit cdc0438 into development Aug 19, 2026
4 checks passed
@Polichinel
Polichinel deleted the feat/c94-findability-preflight branch August 19, 2026 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant