File-derived strings and malformed native payloads create failure paths that
ordinary column profiling never exercises. Add the adversarial and compatibility
fixtures, and treat them as release-blocking rather than as follow-up hardening.
Acceptance criteria
no leakage on any output channel; no inference call and no scope bypass; budget
admission and concurrent-ledger protection hold; existing table exploration and
cache behavior are unchanged.
Detail
Behavioral fixtures: empty collections, missing metadata, unsupported formats,
zero-byte files, results that are missing, malformed, partial, failed, and
successful, duplicate processing attempts, timestamp ties, version mismatches,
unknown versions, results covering only some pages of a document, missing
optional fields against real zero values, deterministic sampling with explicit
elision, and sampling drawn from source files rather than from successful result
rows.
Adversarial fixtures: sensitive file paths, signed URLs, content-bearing
provider errors, malicious strings in status and path columns, malformed
payloads, duplicates, and uncertain provenance. Plant canary content in each and
assert it reaches no output channel, no cache record, and no audit record.
Guardrail fixtures: every charged metadata and results query passes existing
budget admission; concurrent commands retain existing ledger protection; no
native processing function is invoked; a view or function binding that could
execute inference is refused; a malformed mapping cannot inject SQL; a file
binding cannot widen a source scope; a missing capability or prerequisite never
triggers a fallback.
Use synthetic, non-sensitive fixtures for known outcomes and internal data for
usefulness. Run the existing core, storage, envelope, safety, and eval checks
the repository requires, and add the safety-spine entries that assert the
content and processing boundaries.
Constraints
Two constraints govern this work wherever it touches the engine, and an issue
that satisfies its own tests while breaking either is implemented wrong:
- Document content never leaves the warehouse. Every content computation
runs inside a warehouse expression. No document body, excerpt, heading,
extracted value, image, file name, path, signed URL, individual file
identifier, or raw provider error string may reach stdout, stderr, the cache,
or an audit record.
- Nothing in this backlog invokes document processing. Only engine-authored
diagnostic queries over materialized tables execute. New document-processing
charges require a provider-enforced hard monetary ceiling that Dex can verify,
and no estimate, page limit, delayed quota, or user confirmation substitutes
for one.
Dependencies
Develop alongside #452 onward. Release-blocking for every connector.
File-derived strings and malformed native payloads create failure paths that
ordinary column profiling never exercises. Add the adversarial and compatibility
fixtures, and treat them as release-blocking rather than as follow-up hardening.
Acceptance criteria
no leakage on any output channel; no inference call and no scope bypass; budget
admission and concurrent-ledger protection hold; existing table exploration and
cache behavior are unchanged.
Detail
Behavioral fixtures: empty collections, missing metadata, unsupported formats,
zero-byte files, results that are missing, malformed, partial, failed, and
successful, duplicate processing attempts, timestamp ties, version mismatches,
unknown versions, results covering only some pages of a document, missing
optional fields against real zero values, deterministic sampling with explicit
elision, and sampling drawn from source files rather than from successful result
rows.
Adversarial fixtures: sensitive file paths, signed URLs, content-bearing
provider errors, malicious strings in status and path columns, malformed
payloads, duplicates, and uncertain provenance. Plant canary content in each and
assert it reaches no output channel, no cache record, and no audit record.
Guardrail fixtures: every charged metadata and results query passes existing
budget admission; concurrent commands retain existing ledger protection; no
native processing function is invoked; a view or function binding that could
execute inference is refused; a malformed mapping cannot inject SQL; a file
binding cannot widen a source scope; a missing capability or prerequisite never
triggers a fallback.
Use synthetic, non-sensitive fixtures for known outcomes and internal data for
usefulness. Run the existing core, storage, envelope, safety, and eval checks
the repository requires, and add the safety-spine entries that assert the
content and processing boundaries.
Constraints
Two constraints govern this work wherever it touches the engine, and an issue
that satisfies its own tests while breaking either is implemented wrong:
runs inside a warehouse expression. No document body, excerpt, heading,
extracted value, image, file name, path, signed URL, individual file
identifier, or raw provider error string may reach stdout, stderr, the cache,
or an audit record.
diagnostic queries over materialized tables execute. New document-processing
charges require a provider-enforced hard monetary ceiling that Dex can verify,
and no estimate, page limit, delayed quota, or user confirmation substitutes
for one.
Dependencies
Develop alongside #452 onward. Release-blocking for every connector.