Skip to content

refactor(adapters): implement polymorphic PipelineAdapter interface (#145) - #151

Merged
tjirab merged 1 commit into
mainfrom
feat/145-pipeline-adapters
Sep 7, 2026
Merged

refactor(adapters): implement polymorphic PipelineAdapter interface (#145)#151
tjirab merged 1 commit into
mainfrom
feat/145-pipeline-adapters

Conversation

@tjirab

@tjirab tjirab commented Sep 7, 2026

Copy link
Copy Markdown
Owner

🚀 What changed?

  • Introduced PipelineAdapter abstract base class in tff.core.adapter defining unified lifecycle methods: provider_name, is_applicable, load_models, run_checks, apply_metadata_fix, and get_diagnostic_files.
  • Implemented concrete adapters:
    • DBTAdapter in tff.dbt.adapter
    • SQLMeshAdapter in tff.sqlmesh.adapter
    • DataformAdapter in tff.dataform.adapter
  • Updated runner modules (dbt/runner.py, sqlmesh/runner.py, dataform/runner.py) to accept an optional preloaded models: dict[str, ModelRepresentation] | None = None.
  • Refactored tff.core.docs.generate_docs_dashboard to eliminate procedural branching and eliminate duplicate manifest/context model loading by reusing preloaded models in run_checks.
  • Refactored tff.core.autofix.apply_autofixes to dispatch metadata fixes polymorphically via adapter.apply_metadata_fix(...) instead of hardcoded provider checks.
  • Refactored tff.core.cli to use detect_provider and _get_adapter(provider), eliminating provider ladders in info, lint, and health, while introducing _MockRunnerAdapter to preserve complete backwards compatibility for existing tests mocking _get_runner.
  • Added comprehensive unit tests in packages/tff-core/tests/test_adapter.py achieving 100% diff test coverage.

🤔 Why is this change needed?

Resolves issue #145:

  • In tff-core, CLI commands (info, lint, health), docs.py, and autofix.py contained hardcoded if provider == "dbt" ... elif provider == "sqlmesh" ... elif provider == "dataform" procedural ladders. Adding support for a new transformation engine required editing multiple disjoint files across the codebase.
  • In tff docs and tff lint --fix, models were compiled and loaded twice (first explicitly for mapping, and a second time inside the runner check execution), causing unnecessary compilation latency.

🔗 Related Issues (Optional)

Closes #145

✅ Next Steps / Checklist (Optional)

  • Abstract PipelineAdapter base class and provider registry implemented
  • Concrete DBTAdapter, SQLMeshAdapter, and DataformAdapter implemented
  • Runner checks accept pre-loaded models to eliminate duplicate compilation
  • Procedural ladders removed from CLI, docs, and autofix
  • Backwards compatibility maintained for existing tests and mocking patterns
  • 100% diff coverage verified
  • All 267 unit tests pass

@tjirab
tjirab merged commit 26cc533 into main Sep 7, 2026
3 checks passed
@tjirab
tjirab deleted the feat/145-pipeline-adapters branch September 7, 2026 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor(adapters): implement polymorphic PipelineAdapter interface

1 participant