Skip to content

connector create: support snowflake + databricks dialects #1

Closed

Description

@bradfair

Context

ana connector create today enumerates exactly one dialect:

// internal/connector/create.go:12
// v1 only supports the `postgres` dialect; other values are a usage error.
// ...
fs.Var(cli.EnumFlag(&typ, []string{"postgres"}), "type", "connector type (postgres — required)")

Anything else → usage error. Scripted provisioning of Snowflake / Databricks connectors therefore requires hitting the private RPC directly or using the UI.

Why it hurts

Came up mid-flow on a mock-data-forge demo: loaded a cross-source dataset (Snowflake + Databricks), needed two new connectors pointed at the freshly-loaded schemas, hit the CLI wall, had to context-switch to UI to finish the load step. Fine for one demo — painful for the recurring pattern this tool is meant to accelerate.

Other scripted flows likely blocked today:

  • CI/CD provisioning of demo / test instances
  • Bulk tenant onboarding
  • Automated connector rotation (create new → flip references → delete old)
  • Any runbook that wants to stand up a TextQL instance end-to-end from a repo

Proposed scope

Add --type snowflake and --type databricks with the dialect-specific flags each type needs. Rough sketch:

Snowflake

ana connector create --type snowflake --name <n> \
  --locator <acct-locator> --user <u> --database <db> --schema <s> \
  --warehouse <w> --role <r> \
  (--password-stdin | --private-key-file <path>)

Databricks

ana connector create --type databricks --name <n> \
  --host <host> --http-path <path> --catalog <cat> --schema <s> \
  (--pat-stdin | --oauth-m2m-client-id <id> --oauth-m2m-secret-stdin)

Server-side oneof spec blobs (snowflakeMetadata, databricksMetadata) already exist — this is a CLI plumbing issue, not a new backend capability.

Scope notes

  • Secret handling: mirror the existing --password-stdin pattern. No plaintext secrets on argv.
  • Auth variants: Snowflake keypair + Databricks OAuth M2M are the common "service account" forms — worth covering in v2, not v1-of-v2.
  • Validation: ana connector test <id> already exists; creation can pipe into it for a dry-run.
  • Update parity: ana connector update presumably has the same gap — worth pairing.

Activity

  1. bradfair commented on Apr 22, 2026

    @bradfair
    ContributorAuthor

    Substantially resolved via #17 (Snowflake) and #21 (Databricks). Closing in favor of a scoped follow-up for connector update.

    Shipped

    ana connector create snowflake <mode> (PR #17, merged 2026-04-21):

    • password / keypair / oauth-sso / oauth-individual

    ana connector create databricks <mode> (PR #21):

    • access-token / client-credentials / oauth-sso / oauth-individual

    All dialect secrets route through --*-stdin (mirrors --password-stdin); no plaintext on argv. Each leaf is unit-covered and exercised live via opt-in e2e (make e2e) against real TextQL + dedicated Snowflake/Databricks test accounts.

    Shape deviation from the proposed sketch

    Shipped as a subcommand tree (ana connector create <dialect> <mode>) rather than --type <dialect> with per-dialect flags. Functionally equivalent for the scripted-provisioning use case this issue was driven by; auth modes compose cleanly into their own leaves.

    Deferred: connector update dialect parity

    internal/connector/update.go is still postgres-only — this issue's scope notes called it out and we should have paired it in. Credential rotation and scripted config tweaks against Snowflake/Databricks connectors remain blocked on the UI / raw RPC.

    Filing a scoped follow-up and closing this.

  2. bradfair commented on Apr 22, 2026

    @bradfair
    ContributorAuthor

    Follow-up filed as #22.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions