Integrate dbt Cloud for CI/CD and Model Management

[williaby]

📌 Feature: Integrate dbt Cloud for CI/CD and Model Management

Labels: enhancement, dbt, CI/CD, future-work
Milestone: v0.3.0 – Enhanced Documentation & Data Modeling

---

🧭 Overview

Introduce dbt Cloud into the LedgerBase architecture to support version-controlled SQL models, automated data testing, scheduled deployments, and integrated documentation. This feature will lay the groundwork for scalable, testable financial data modeling aligned with team workflows and enterprise auditability.

---

🎯 Objectives

• Set up dbt Cloud integration with the LedgerBase GitHub repository.
• Configure staging, CI, and production environments.
• Implement dbt testing (not_null, unique, relationships) on critical models.
• Establish CI triggers using state:modified+ for efficient model testing.
• Integrate SQLFluff and dbt Project Evaluator for quality control.
• Enable job-level status checks for pull request validation.

---

📋 Tasks

📦 dbt Cloud Project

• Create a new dbt Cloud project named LedgerBase.
• Connect it to the existing GitHub repository.
• Define dbt_project.yml structure and base model layout (stg/, fct/, dim/, int/).

🏗️ Environment Setup

• Configure separate environments for Development, CI, and Production.
• Use distinct schema suffixes for each environment to avoid table conflicts.
• Secure credentials and define deployment targets.

🔄 GitHub Integration

• Install dbt Cloud GitHub App.
• Configure job triggers via GitHub Actions and dbt Cloud webhooks.
• Enforce CI status checks for pull request merges.

✅ Testing & Quality

• Integrate dbt Project Evaluator to enforce project best practices.
• Add SQLFluff for linting dbt models in CI.
• Ensure dbt test validates core budget, transaction, and vendor models.

---

🔗 References

• dbt Cloud + GitHub Setup
• dbt CI/CD Best Practices
• dbt Project Evaluator
• SQLFluff Docs

---

📝 Notes

This integration is deferred until pre-commit and baseline CI/CD scaffolding are finalized. dbt Cloud implementation will support downstream analytics integrity and audit tracking required for institutional use cases.