Skip to content

fix(platform): deny a corpus ref bound to neither document nor thread - #3177

Open
Israeltheminer wants to merge 1 commit into
mainfrom
fix/retrieval-gate-deny-unbound
Open

fix(platform): deny a corpus ref bound to neither document nor thread#3177
Israeltheminer wants to merge 1 commit into
mainfrom
fix/retrieval-gate-deny-unbound

Conversation

@Israeltheminer

Copy link
Copy Markdown
Collaborator

The retrieval gate admits a file row bound to neither a document nor a thread, and serves it to any member of the organization. This denies it.

Replaces #3141, which was written against the gate before #3140 rewrote it. Refs #3142.

Why

Retrieval has two gates. The SQL half admits rows and fails open on purpose — backend/core/knowledge/corpus.ts says so and names decideRetrievable as the decider. That function ends its unbound-file loop with return true.

The shape is not obscure. indexUploadedFile stamps corpus scope only from a bound document, so a row with no document carries no project and no team, and corpus.ts reads exactly that as an org-wide hub row. Both gates then pass it.

0.4 denied it: if (metadata.documentId === undefined) { continue; }.

The premise I had to disprove first

The current posture is deliberate and documented — the module docstring says unbound rows "keep the 0.4 same-org posture DELIBERATELY", on the grounds that "video-link transcripts index without a document, so a blanket quarantine would dark a legitimate lane". There is also a test asserting it.

So I checked the lane before changing it, and the premise does not hold:

  • The synthetic transcript row is inserted with thread_id, so it admits through the thread branch, not the unbound one.
  • A welcome-page paste starts thread-less only because no thread exists yet. The first send stamps it — bindStorageIdsToThread updates exactly the rows WHERE document_id IS NULL AND thread_id IS NULL.
  • Before that send there is no thread for a turn to be scoped to, so nothing can legitimately ask for the transcript. After it, the thread branch admits it.

The claim that this "keeps the 0.4 posture" is also the opposite of what 0.4 did.

The docstring is corrected to say all of that, so the next reader does not re-derive it.

What changed

Two lines of logic. The unbound loop's final return true becomes continue.

The assertion that locked in the old behaviour is replaced, not deleted, with a comment recording what it used to claim and why that changed. Three cases are added: the no-access-scope path denies too (it is the internal-caller path and must not become a way around the rule), the video-link lane still admits once its thread is stamped, and a thread-bound row is not admitted from outside its thread.

What I dropped from #3141

#3141 also added a rag_status = 'completed' guard and a superseded-ref check. Both are unnecessary now: #3140's rewrite queries app.documents by file_ref, so a replaced ref finds no document and is already denied, and the deleted-document and trashed-file cases are handled too.

The remaining rag_status gap is a mid-reindex window where a knowledge entry can answer from its previous version's chunks. That is the organization's own content one version stale, not a cross-member leak, and guarding it would mean joining file_metadata back into a query the refactor deliberately narrowed. Left out, named here rather than filed as a caveat.

Tests

Mutation Went red
unbound branch back to return true 2 cases
thread scope ignored (admit any thread-bound row) 2 cases

Gate: typecheck, oxlint --type-aware, oxfmt --check, lint:sast green; knowledge suite 12/12; branched from origin/main at 58c3de381.

The retrieval gate admitted a file row with no document and no thread to
any member of the organization.

That shape is not obscure: the indexer stamps corpus scope only from a
bound document, so such a row carries no project and no team, and the SQL
half of the two-gate design reads exactly that as an org-wide hub row. The
SQL half fails open by design and names this function as the decider, so
both gates passed the ref. 0.4 denied it (documentId === undefined →
continue).

The previous posture was deliberate and documented, on the grounds that
video-link transcripts index without a document and a blanket quarantine
would dark that lane. The premise does not hold: a transcript row carries
thread_id, so it admits through the thread branch. A welcome-page paste
starts thread-less only because no thread exists yet, and the first send
stamps it via bindStorageIdsToThread, which updates exactly the rows with
no document and no thread. Before that send no turn can be scoped to it.

So the assertion that locked in the old behaviour is replaced rather than
deleted, and a new case proves the video-link lane still admits once its
thread is stamped.

Refs #3142.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant