Snowflake users need the same evidence contract over the resources they already
have. Support existing directory tables as collections and materialized
AI_PARSE_DOCUMENT output as a native result format, alongside mapped
diagnostic columns.
Acceptance criteria
the shared conformance tests pass unchanged; stage metadata is read and never
refreshed; no parsing function is invoked; an unsupported result shape is
reported explicitly; the internal dogfood scenario succeeds.
Detail
Directory tables are an existing stage metadata interface. Dex reads them as
configured. It does not create a stage, does not create a directory table, and
does not call a refresh, because a refresh is a side effect on a customer
resource and it costs money.
The native result adapter reads materialized AI_PARSE_DOCUMENT output and
declares the output shape it supports. An unrecognized shape yields unavailable
diagnostics rather than guessed values, and the provider schema version is
recorded where available.
Reuse the shared source and result-format conformance contracts rather than
writing parallel assertions. The point of a common contract is that a second
connector is cheap to trust.
Repeat the BigQuery dogfood scenario against a Snowflake collection: real files,
existing materialized results, at least one meaningful gap, and a concrete
downstream decision.
Constraints
Two constraints govern this work wherever it touches the engine, and an issue
that satisfies its own tests while breaking either is implemented wrong:
- Document content never leaves the warehouse. Every content computation
runs inside a warehouse expression. No document body, excerpt, heading,
extracted value, image, file name, path, signed URL, individual file
identifier, or raw provider error string may reach stdout, stderr, the cache,
or an audit record.
- Nothing in this backlog invokes document processing. Only engine-authored
diagnostic queries over materialized tables execute. New document-processing
charges require a provider-enforced hard monetary ceiling that Dex can verify,
and no estimate, page limit, delayed quota, or user confirmation substitutes
for one.
Dependencies
Depends on #457, #458, and the BigQuery dogfood findings from #459.
Snowflake users need the same evidence contract over the resources they already
have. Support existing directory tables as collections and materialized
AI_PARSE_DOCUMENToutput as a native result format, alongside mappeddiagnostic columns.
Acceptance criteria
the shared conformance tests pass unchanged; stage metadata is read and never
refreshed; no parsing function is invoked; an unsupported result shape is
reported explicitly; the internal dogfood scenario succeeds.
Detail
Directory tables are an existing stage metadata interface. Dex reads them as
configured. It does not create a stage, does not create a directory table, and
does not call a refresh, because a refresh is a side effect on a customer
resource and it costs money.
The native result adapter reads materialized
AI_PARSE_DOCUMENToutput anddeclares the output shape it supports. An unrecognized shape yields unavailable
diagnostics rather than guessed values, and the provider schema version is
recorded where available.
Reuse the shared source and result-format conformance contracts rather than
writing parallel assertions. The point of a common contract is that a second
connector is cheap to trust.
Repeat the BigQuery dogfood scenario against a Snowflake collection: real files,
existing materialized results, at least one meaningful gap, and a concrete
downstream decision.
Constraints
Two constraints govern this work wherever it touches the engine, and an issue
that satisfies its own tests while breaking either is implemented wrong:
runs inside a warehouse expression. No document body, excerpt, heading,
extracted value, image, file name, path, signed URL, individual file
identifier, or raw provider error string may reach stdout, stderr, the cache,
or an audit record.
diagnostic queries over materialized tables execute. New document-processing
charges require a provider-enforced hard monetary ceiling that Dex can verify,
and no estimate, page limit, delayed quota, or user confirmation substitutes
for one.
Dependencies
Depends on #457, #458, and the BigQuery dogfood findings from #459.