Add Databricks, Trino, DuckDB connectors and Teams/PagerDuty alerts#11
Merged
harishconti merged 1 commit intomainfrom Apr 5, 2026
Merged
Conversation
…0→P2) P0 bugs fixed: - pyproject.toml: add [tool.setuptools.packages.find] to fix flat-layout discovery error that blocked `pip install`; fix deprecated license table format - webhooks.py: fix SlackDispatcher(config) call bug — constructor takes **kwargs not config dict P1 features: - CLI: add `observakit validate-config` (dry-run parse of kit.yml + contracts) - CLI: add `observakit diff` (fetch schema diff, exit 1 when drift detected for CI gates) - CLI: add `--config` flag to all commands, `--output json` on status/check - CLI: enforce exit codes (0=pass, 1=failures, 2=config error) - Backend: add `GET /scheduler/jobs` endpoint listing next run times per job P2 features: - Scheduler: honor per-check `enabled: false` flag in freshness and volume loops - Alerts: add alerts/teams.py (MS Teams Adaptive Cards via Incoming Webhook) - Alerts: add alerts/pagerduty.py (PagerDuty Events API v2 with dedup key + resolve) - Alerts: register teams + pagerduty in get_alert_dispatcher() factory - Alerts: upgrade alerts/slack.py to Block Kit layout (header, context, action button) with severity colour strip and retry on 429/5xx - Alerts: fix webhook.py version string (0.1.7 → 0.1.10) P3 connectors: - connectors/duckdb.py — embedded DuckDB for local/Parquet pipelines - connectors/databricks.py — Databricks SQL Connector (Unity Catalog support) - connectors/trino.py — Trino/Presto federated query connector - connectors/base.py: register all three in get_warehouse_connector() factory - pyproject.toml: add duckdb, databricks, trino optional extras Tests: - tests/test_alerts.py: add 11 new tests covering Teams, PagerDuty, Slack Block Kit; 94 tests total, all passing https://claude.ai/code/session_01SWgQhZuNgDNhVBTUoXJa2x
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds three new warehouse connectors (Databricks, Trino, DuckDB) and two new alert channels (Microsoft Teams, PagerDuty). Enhances the Slack dispatcher with Block Kit formatting and retry logic. Adds CLI commands for config validation and scheduler job inspection. Improves alert routing and per-table enable/disable flags.
Type of change
Key Changes
New Connectors
connectors/databricks.py): Connects to Databricks SQL warehouses via the officialdatabricks-sql-connector. Supports Unity Catalog (three-part names) and legacy Hive metastore. Includes schema introspection viainformation_schema.columnswith fallback toDESCRIBE TABLE.connectors/trino.py): Connects to Trino/Presto clusters. Supports all Trino catalogs (Iceberg, Hive, Delta Lake, etc.). Usestrino-python-clientwith optional basic auth.connectors/duckdb.py): Embedded analytical database connector for local files and in-memory databases. Useful for dbt-core development and reading Parquet/Iceberg files directly.New Alert Channels
alerts/teams.py): Sends alerts via Teams Incoming Webhooks using Adaptive Cards.alerts/pagerduty.py): Sends alerts via PagerDuty Events API v2 with severity mapping, deduplication keys, and incident resolution support.Alert Improvements
alerts/slack.py): Refactored to use Block Kit structured layouts with severity colour strips, context footers, and action buttons. Added retry logic with exponential backoff for HTTP 429 and 5xx errors.alerts/base.py): Updated to support Teams and PagerDuty dispatchers.CLI Enhancements
cmd_validate_config): Newobservakit validate-configcommand to dry-run parse kit.yml and contracts without connecting to a warehouse. Validates required fields, alert routing, and contract syntax.cmd_status,cmd_check): Added--output jsonflag for machine-readable status and check results.backend/scheduler.py,backend/main.py): NewGET /scheduler/jobsendpoint to list all scheduled jobs with next run times.Configuration & Robustness
enabled: true/falsefield to freshness and volume table configs to skip checks without error.connectors/base.pyto register new connectors (DuckDB, Trino, Databricks).pyproject.tomlto include all package directories in setuptools discovery.Testing
tests/test_alerts.py.Related issues
N/A
https://claude.ai/code/session_01SWgQhZuNgDNhVBTUoXJa2x