backlog: amend #1112 -- the concurrent-uploads half is answered, the cross-shard half is what remains - #353
Merged
Merged
Conversation
…cross-shard half is what remains
NO NEW NUMBER. The ASVS lane ran the duplicate check BEFORE drafting and it fired:
#1112's own title already names both halves ("concurrent uploads AND across engine
shards"), so the store-side fix is its second half, not a new item. Filing it fresh
would have burned a number on a duplicate -- the failure section 7 records twice, and
which I made myself today on the glyph item.
VERIFIED AGAINST origin/main, not taken on report:
uploads.py:309 _quota_lock = asyncio.Lock() (PR #325)
uploads.py:488 async with self._quota_lock: ... :496 _scan_metas_sync()
"the check-then-write is not atomic" 0 occurrences -- and there is no line 394
settings.py:439 states shards over one dir enforce ONE budget, measured with a
live positive control
TWO STRIKES, both false in the ALARMING direction, kept rather than deleted:
- the banner's "no lock constructed in the module" and its :394 citation;
- the severity paragraph's "N shards would each apply the budget, multiplying it".
The real remainder is an at-most N-1 file overshoot, each bounded by
max_upload_bytes. An overstated severity is not conservative -- it misdirects
whoever costs the fix.
WHAT KEEPS IT OPEN is one sentence: asyncio.Lock is per-process, and the module says
so itself at :487. Engine sharding is the shipped default scaling axis, nothing
partitions uploads_dir per shard, so N shards hold N independent locks.
The fix has a known home -- ADR 0063 already puts every shard on one unified store --
so "cannot honestly reach pass" is NOT the finding. Acceptance is stated as EXECUTION
with concurrent writers on at least 2 shards, never by reading.
OWNER RULING recorded in the item: ASVS 2.3.4 holds at partial.
CAUGHT MY OWN BLIND CHECK doing this: my first grep for the struck string ran against
messagefoundry/api/uploads.py, which does not exist -- the file is messagefoundry/uploads.py.
It printed 0 and I labelled it "the string is gone". Re-ran against the real path with a
negative control (49 def/class lines) before believing it.
wshallwshall
enabled auto-merge (squash)
August 12, 2026 18:32
test_link_resolution caught it: docs/BACKLOG.md:6000 wrote [ADR 0063](docs/adr/...) which resolves RELATIVE TO docs/, so it became docs/docs/adr/... and did not exist. The convention is unambiguous and I was the outlier: 74 links in this file use the adr/ prefix, exactly one used docs/adr/, and it was mine. Verified after fixing: zero docs/adr/ links remain, and the target file exists. I could NOT run the guard locally -- pytest dies on ModuleNotFoundError: pydantic, because a worktree venv installs fewer extras than CI. That is BACKLOG #1230 demonstrating itself on the PR that amends the ledger, an hour after its scope was ruled. CI is the authority here, which is the whole point of #1230's loud-omission half: a local run that cannot collect must not read as a clean one.
…nto claude/amend-1112-shard-quota
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Docs-only. #1112 stays OPEN -- this is an amendment, not a closure. No new number was allocated, deliberately.
Why no new number
The owner asked for the store-side ASVS 2.3.4 fix to be filed. The ASVS lane ran the duplicate check before drafting and it fired: #1112's own title already reads "a quota that holds across concurrent uploads and across engine shards at once." The store-side fix is its second half.
Filing it fresh would have burned a number on a duplicate -- the failure section 7 records twice, and which I made myself today on the glyph item. Running the check first is the whole reason it did not happen again.
Verified against
origin/main, not taken on reportTwo strikes, both false in the ALARMING direction
:394citation. The lock exists and is held across both the scan and the write -- releasing between them was the race.max_upload_bytes.Both struck rather than deleted. An overstated severity is not the conservative error -- it misdirects whoever costs the fix, and this one overstated by a multiplier.
What actually keeps it open -- one sentence
asyncio.Lockis per-event-loop, therefore per-process, and the module says so itself at:487: "One critical section per process." Engine sharding is the built, shipped, default scaling axis, and nothing partitionsuploads_dirper shard -- so N shards over one directory hold N independent locks.The fix has a known home, so "cannot honestly reach pass" is not the finding here: ADR 0063 already puts every shard on one unified store, so the cross-shard coordination point exists. Move the quota accounting into the store, or take an advisory lock on the directory.
Acceptance is stated as EXECUTION, never as reading: the quota holds across N shards sharing one
uploads_dir, with concurrent writers on at least 2 shards, and the single-process path keeps its current behaviour.Owner ruling, recorded in the item: ASVS 2.3.4 holds at
partial-- V2.3.4 names business-logic locking as the means, and a per-process lock does not span a resource shared across processes.One blind check I caught in myself
My first grep for the struck string ran against
messagefoundry/api/uploads.py-- which does not exist; the file ismessagefoundry/uploads.py. It printed0and I labelled it "the string is gone." Right answer, no evidence. Re-ran against the real path with a negative control (49def/classlines) before believing it.