File evidence needs attributable persistence without becoming a second canonical
model. Add the collection and profile cache records and a compatible schema
migration through the existing storage interface.
Acceptance criteria
existing caches migrate; every shipped storage backend round-trips the new
records; fingerprints capture a binding or result-schema change; no per-file
identity and no content is stored.
Detail
Persist collection references and metadata summaries, aggregate profiles,
sampling and denominator information, binding and result-schema fingerprints,
source and result observation times with available freshness evidence, and the
diagnostic availability and limitations that accompanied the report.
Do not persist per-file match records. The per-file layer is where identities
and paths live, and it has no consumer that justifies keeping it.
An explicit profile command recomputes evidence under the cost guard. A cached
report keeps its observation time and never silently stands in for a new
assessment.
Extend the existing non-canonical cache through the existing storage
abstraction, so the records reach every shipped backend rather than only the
default one.
Cache provenance is deliberately rich enough that drift detection over
collections and processing evidence stays possible later. This release does not
extend maintain or the ER diagram to files, and nothing here should be read as
committing to that.
Constraints
Two constraints govern this work wherever it touches the engine, and an issue
that satisfies its own tests while breaking either is implemented wrong:
- Document content never leaves the warehouse. Every content computation
runs inside a warehouse expression. No document body, excerpt, heading,
extracted value, image, file name, path, signed URL, individual file
identifier, or raw provider error string may reach stdout, stderr, the cache,
or an audit record.
- Nothing in this backlog invokes document processing. Only engine-authored
diagnostic queries over materialized tables execute. New document-processing
charges require a provider-enforced hard monetary ceiling that Dex can verify,
and no estimate, page limit, delayed quota, or user confirmation substitutes
for one.
Dependencies
Depends on #450 and #457.
File evidence needs attributable persistence without becoming a second canonical
model. Add the collection and profile cache records and a compatible schema
migration through the existing storage interface.
Acceptance criteria
existing caches migrate; every shipped storage backend round-trips the new
records; fingerprints capture a binding or result-schema change; no per-file
identity and no content is stored.
Detail
Persist collection references and metadata summaries, aggregate profiles,
sampling and denominator information, binding and result-schema fingerprints,
source and result observation times with available freshness evidence, and the
diagnostic availability and limitations that accompanied the report.
Do not persist per-file match records. The per-file layer is where identities
and paths live, and it has no consumer that justifies keeping it.
An explicit profile command recomputes evidence under the cost guard. A cached
report keeps its observation time and never silently stands in for a new
assessment.
Extend the existing non-canonical cache through the existing storage
abstraction, so the records reach every shipped backend rather than only the
default one.
Cache provenance is deliberately rich enough that drift detection over
collections and processing evidence stays possible later. This release does not
extend
maintainor the ER diagram to files, and nothing here should be read ascommitting to that.
Constraints
Two constraints govern this work wherever it touches the engine, and an issue
that satisfies its own tests while breaking either is implemented wrong:
runs inside a warehouse expression. No document body, excerpt, heading,
extracted value, image, file name, path, signed URL, individual file
identifier, or raw provider error string may reach stdout, stderr, the cache,
or an audit record.
diagnostic queries over materialized tables execute. New document-processing
charges require a provider-enforced hard monetary ceiling that Dex can verify,
and no estimate, page limit, delayed quota, or user confirmation substitutes
for one.
Dependencies
Depends on #450 and #457.