Skip to content

_ContractStorePort.download fails open where upload refuses — C-79's untreated sibling #268

Closed

Description

@Polichinel

The defect

_ContractStorePort.download chains .get() onto an unvalidated store result:

def download(self, file_id: str) -> bytes:
    return (
        self._dsm.download_prediction(file_id).to_dict().get("data", {}).get("file_bytes", None)
    )

When the store returns a result whose data is None, .get("data", {}) yields None —
the default never applies, because the key is present — and the next .get raises
AttributeError: 'NoneType' object has no attribute 'get'.

This is C-79's untreated sibling: the same class, the same file, the same fail-open
polarity, in the method next door.

This is not hypothetical — it cost an evening

First un_crafd delivery attempt, 2026-08-13 (views-crafdapi#44, epic #40). The run
resolved its inbound contract and its historical leg cleanly, then died three minutes into
the shard download:

17:43:56  Contract inbound resolved: run rusty_bucket_forecasting_20260727_095355 (3 targets leased)
17:43:57  historical delivery coverage: 64742 distinct cells, 28421738 rows
17:46:59  ERROR - Error during postprocessor run: 'NoneType' object has no attribute 'get'
  File ".../contract/wire/sink.py", line 81, in deliver_run
    frame, headers = lease.load()
  File ".../contract/wire/source_selection.py", line 69, in <dictcomp>
    name: self.store.download(file_id)
  File ".../crafd/managers/crafd.py", line 40, in download
    self._dsm.download_prediction(file_id).to_dict().get("data", {}).get("file_bytes", None)
AttributeError: 'NoneType' object has no attribute 'get'

What the operator can learn from that message: nothing. It does not say which file_id
failed, that a download failed at all, or why. We spent real time ruling out an OOM kill
(there was one in dmesg, three minutes later, on a different pid) before establishing that
the actual cause was a download returning a result the port did not check. A named refusal
would have ended that in one line.

The proximate trigger was memory pressure — see the companion issue — but the trigger is not
the point. Any failed download produces the same unreadable crash: a yanked file, an
expired key, a rate limit, a network blip.

upload already refuses; download does not

The other method on the same class carries a long comment explaining exactly why it must not
do this, and the fix that C-79 landed on 2026-08-05:

    # **Refuse unless success is explicitly True** (register C-79). The earlier
    # `if success is False` failed OPEN: a result that was None, or lacked the
    # attribute, or carried a non-bool, sailed through as though the upload had worked.
    #
    # The old `to_dict()` fallback is gone with it: dead on the real path, and an
    # unrecognised result should be refused and named, not adapted to silently.
    success = getattr(result, "success", None)
    if success is not True:
        ...raise, naming what it actually got...

"An unrecognised result should be refused and named, not adapted to silently."
download adapts silently — it returns None on any shape it does not recognise, and lets
the caller trip over it later. C-79's own resolution note is the specification for this fix;
it was simply never applied to the second method.

Both partners carry it

download is byte-identical in unfao/managers/unfao.py and crafd/managers/crafd.py —
exactly as C-79 recorded for upload. Verified at origin/development (25ca25c). The FAO
delivery has the same defect on the same leg; it has not fired there yet.

The fix already has a pattern and a home

tests/test_store_port.py exists and was written for C-79: five parametrised tests across
both partners, including test_an_unrecognised_result_is_refused_rather_than_assumed_good
and test_the_refusal_names_what_it_actually_got. None of them covers download.

So this is not new machinery — it is the same polarity change and the same test shapes,
applied to the method that was missed. The fake-store fixture is already there.

C-79's resolution also recorded "the standing excuse never applied here" — the port takes a
store object and calls methods on it, needing neither Appwrite env nor a path manager. That
remains true for download.

Acceptance

  • A download that returns an unrecognised result shape is refused and named —
    naming the file_id — not chained into an AttributeError.
  • A download that reports a failure is refused, carrying the store's own error.
  • Both partners covered (unfao and crafd), as test_store_port.py already does for upload.
  • Mutation-proven: reverting the polarity fails the new tests.
  • The register entry cross-references C-79 as the sibling that was fixed first.

Cross-references

Filed from the views-crafdapi seat. No change was made in this repo. Per this repo's own
convention (tests/test_doc_accuracy.py — "The FILE is the claim; the line number is not"),
this issue names symbols; the traceback line numbers above are quoted evidence from a run,
not a location claim.

Activity

  1. Polichinel commented on Aug 14, 2026

    @Polichinel
    CollaboratorAuthor

    Companion: #269 — _ShardLease.load holds every shard of a target in memory at once. That is what made the download fail on 2026-08-13; this issue is why the failure was unreadable when it did. Independent defects, same crash.

  2. added a commit that references this issue on Aug 14, 2026
  3. Polichinel commented on Aug 14, 2026

    @Polichinel
    CollaboratorAuthor

    Fixed in #270 (ff0278b). Every claim in your report reproduced — thank you for the receipts, the dmesg detail in particular saved time.

    The fix. download now refuses anything that is not non-empty bytes, naming the file_id, that a download failed, and the types it actually got. Empty bytes are refused with the rest: no shard, sidecar or manifest is ever zero-length, so b"" is a failed download wearing a valid type, and returning it only moves the same crash to the parser. It protects all three call sites, not only the one that crashed — two of them are manifest reads in resolve_run.

    Mutation-proven on three: restoring the original one-liner fails 18 of the module's tests, accepting empty bytes fails exactly 2 (one per partner), dropping the file_id from the message fails exactly 2.

    You were right that un_fao carries it. Verified byte-identical before the fix and after.

    Two things your report did not ask for, which fell out of it.

    _ContractStorePort moved to {partner}/store_port.py. The refusal pushed managers/ over epic #148's 450-line bound, and that guard's instruction is to move something out rather than raise the number. Review then caught that the move dodged the budget rather than satisfying it — the counted number fell 441→388 while the partner package grew 441→488, because the new module is a sibling of managers/, not inside it. There is now an outer budget over the whole partner package, and the register says plainly that the first version reported headroom against a guard that had stopped measuring the code.

    The port also stopped naming DatastoreModule in its constructor. A DIP seam whose stated purpose is that nothing downstream sees the client's types should not name one.

    What we found while reading the rest of the port, and did not fix: file_metadata has no caller in the package — latest_file_id runs three times, download three times, upload three times, and the fourth method is reachable only from a test, with contract/store_metadata.py dead behind it. Registered as C-100 rather than deleted: whether it goes or gains a caller is a question for the second prediction store (#97), not for a download bug.

    #269 is next, starting with a measurement rather than a fix — the honest answer to "how much does bounding the shard fetch save" is that we do not know yet, and the historical leg's OOM (#126) turned out to have a completely different root cause than its symptoms suggested. One correction for that issue: the class is TargetLease, not _ShardLease.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions