"Unstructured exploration" reads as a promise to look inside raw documents, and
this release deliberately does not do that. Update the command contract, the
agent instructions, the connector notes, the examples, the changelog, and the
capability documentation so the boundary is stated where a reader meets the
feature.
Acceptance criteria
every example uses existing materialized results; the raw-file limitation is
prominent rather than a footnote; all four unsupported connectors state their
boundary; the BigQuery release carries dogfood evidence and passes the required
repository checks.
Detail
The release inventories files and assesses processing evidence that already
exists. It does not parse documents, run OCR, retry processing, classify
business document types, author extraction schemas, evaluate extraction
correctness, download files, or fall back to a local parser. Documentation that
leaves any of that ambiguous is a credibility cost on a published release.
Document the capability matrix honestly. BigQuery first, Snowflake and
Databricks as follow-ups, and DuckDB, ClickHouse, Postgres, and Redshift as
explicitly unsupported for file profiling with existing table exploration
unaffected.
State that this release performs no document PII screening and clears no
existing PII flag, so an absent finding is not read as a clean result. State
that there is no readiness score and why.
Update the command contract and agent instructions so an agent learns the
prerequisite shape: a binding is required, a missing one refuses, and inventory
is what runs without one.
Do not market this as first-time raw-document inspection. That remains blocked
by the cost requirement, and the tracking issue for it stays open.
Follow-up documentation accompanies the Snowflake and Databricks connectors
rather than landing ahead of them.
Constraints
Two constraints govern this work wherever it touches the engine, and an issue
that satisfies its own tests while breaking either is implemented wrong:
- Document content never leaves the warehouse. Every content computation
runs inside a warehouse expression. No document body, excerpt, heading,
extracted value, image, file name, path, signed URL, individual file
identifier, or raw provider error string may reach stdout, stderr, the cache,
or an audit record.
- Nothing in this backlog invokes document processing. Only engine-authored
diagnostic queries over materialized tables execute. New document-processing
charges require a provider-enforced hard monetary ceiling that Dex can verify,
and no estimate, page limit, delayed quota, or user confirmation substitutes
for one.
Dependencies
Depends on #457, #458, #459, and #462. Follow-up documentation
accompanies #460 and #461.
"Unstructured exploration" reads as a promise to look inside raw documents, and
this release deliberately does not do that. Update the command contract, the
agent instructions, the connector notes, the examples, the changelog, and the
capability documentation so the boundary is stated where a reader meets the
feature.
Acceptance criteria
every example uses existing materialized results; the raw-file limitation is
prominent rather than a footnote; all four unsupported connectors state their
boundary; the BigQuery release carries dogfood evidence and passes the required
repository checks.
Detail
The release inventories files and assesses processing evidence that already
exists. It does not parse documents, run OCR, retry processing, classify
business document types, author extraction schemas, evaluate extraction
correctness, download files, or fall back to a local parser. Documentation that
leaves any of that ambiguous is a credibility cost on a published release.
Document the capability matrix honestly. BigQuery first, Snowflake and
Databricks as follow-ups, and DuckDB, ClickHouse, Postgres, and Redshift as
explicitly unsupported for file profiling with existing table exploration
unaffected.
State that this release performs no document PII screening and clears no
existing PII flag, so an absent finding is not read as a clean result. State
that there is no readiness score and why.
Update the command contract and agent instructions so an agent learns the
prerequisite shape: a binding is required, a missing one refuses, and inventory
is what runs without one.
Do not market this as first-time raw-document inspection. That remains blocked
by the cost requirement, and the tracking issue for it stays open.
Follow-up documentation accompanies the Snowflake and Databricks connectors
rather than landing ahead of them.
Constraints
Two constraints govern this work wherever it touches the engine, and an issue
that satisfies its own tests while breaking either is implemented wrong:
runs inside a warehouse expression. No document body, excerpt, heading,
extracted value, image, file name, path, signed URL, individual file
identifier, or raw provider error string may reach stdout, stderr, the cache,
or an audit record.
diagnostic queries over materialized tables execute. New document-processing
charges require a provider-enforced hard monetary ceiling that Dex can verify,
and no estimate, page limit, delayed quota, or user confirmation substitutes
for one.
Dependencies
Depends on #457, #458, #459, and #462. Follow-up documentation
accompanies #460 and #461.