Stored Document AI responses are too verbose and too content-bearing for an
agent to read directly. Add the result-format adapter that turns them into fixed
SQL diagnostics.
Acceptance criteria
payload validity, processing status, text yield, reported pages, pages
represented, and supported structure counts are aggregated; an unknown payload
shape degrades explicitly rather than guessing; no call to ML.PROCESS_DOCUMENT
or any other inference function occurs.
Detail
The adapter reads fixed structured paths in a materialized response and emits
diagnostic SQL expressions over them. Available diagnostics are the
text-character count, the reported page count, the pages represented in the
result, and structural-element counts where the shape supports them.
Record the provider schema version where it is available. A shape the adapter
does not recognize yields unavailable diagnostics, not a guessed value, and the
report says which fields were unavailable and why.
Never label a document assessed when only a subset of its pages is represented
in the result. Partial page coverage is its own reported state.
The profile report returns five groups of evidence: the collection with its
selected formats and metadata scope, coverage with sample size and files with,
without, and ambiguously matched results, currency with version-matched,
mismatched, and unknown correspondence, processing with reported status and
diagnostic distributions, and limitations naming unsupported fields, sampling
limits, missing provenance, and what was not assessed.
There is no overall readiness score. Successful parsing, long text, and detected
tables do not establish semantic correctness, and a single number would invite
exactly that reading.
Constraints
Two constraints govern this work wherever it touches the engine, and an issue
that satisfies its own tests while breaking either is implemented wrong:
- Document content never leaves the warehouse. Every content computation
runs inside a warehouse expression. No document body, excerpt, heading,
extracted value, image, file name, path, signed URL, individual file
identifier, or raw provider error string may reach stdout, stderr, the cache,
or an audit record.
- Nothing in this backlog invokes document processing. Only engine-authored
diagnostic queries over materialized tables execute. New document-processing
charges require a provider-enforced hard monetary ceiling that Dex can verify,
and no estimate, page limit, delayed quota, or user confirmation substitutes
for one.
Dependencies
Depends on #452 and #454.
Stored Document AI responses are too verbose and too content-bearing for an
agent to read directly. Add the result-format adapter that turns them into fixed
SQL diagnostics.
Acceptance criteria
payload validity, processing status, text yield, reported pages, pages
represented, and supported structure counts are aggregated; an unknown payload
shape degrades explicitly rather than guessing; no call to
ML.PROCESS_DOCUMENTor any other inference function occurs.
Detail
The adapter reads fixed structured paths in a materialized response and emits
diagnostic SQL expressions over them. Available diagnostics are the
text-character count, the reported page count, the pages represented in the
result, and structural-element counts where the shape supports them.
Record the provider schema version where it is available. A shape the adapter
does not recognize yields unavailable diagnostics, not a guessed value, and the
report says which fields were unavailable and why.
Never label a document assessed when only a subset of its pages is represented
in the result. Partial page coverage is its own reported state.
The profile report returns five groups of evidence: the collection with its
selected formats and metadata scope, coverage with sample size and files with,
without, and ambiguously matched results, currency with version-matched,
mismatched, and unknown correspondence, processing with reported status and
diagnostic distributions, and limitations naming unsupported fields, sampling
limits, missing provenance, and what was not assessed.
There is no overall readiness score. Successful parsing, long text, and detected
tables do not establish semantic correctness, and a single number would invite
exactly that reading.
Constraints
Two constraints govern this work wherever it touches the engine, and an issue
that satisfies its own tests while breaking either is implemented wrong:
runs inside a warehouse expression. No document body, excerpt, heading,
extracted value, image, file name, path, signed URL, individual file
identifier, or raw provider error string may reach stdout, stderr, the cache,
or an audit record.
diagnostic queries over materialized tables execute. New document-processing
charges require a provider-enforced hard monetary ceiling that Dex can verify,
and no estimate, page limit, delayed quota, or user confirmation substitutes
for one.
Dependencies
Depends on #452 and #454.