Fork test: PR #1259 - Migrate Configs and Rules to Pydantic#1272
Open
mwojtyczka wants to merge 8 commits into
Open
Fork test: PR #1259 - Migrate Configs and Rules to Pydantic#1272mwojtyczka wants to merge 8 commits into
mwojtyczka wants to merge 8 commits into
Conversation
Migrate storage configs, ChecksValidationStatus and the DQRule family to Pydantic v2 BaseModel. WorkspaceConfig/RunConfig and their nested config tree stay as dataclasses because databricks-labs-blueprint's Installation load/save dispatches on dataclasses.is_dataclass() and __dataclass_fields__. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Revert location/columns type widening that broke mypy (str|None cascaded
into checks_storage.py; widened columns broke manager.py and get_rules).
Empty/None location now rejected in @model_validator(mode='before') so the
DQX-specific InvalidConfigError/InvalidParameterError is still raised before
Pydantic coercion; None elements in columns rejected the same way.
- Declare _expected_rule_type/_alternative_rules as ClassVar on DQRuleTypeMixin
so subclass ClassVar overrides no longer trip mypy.
- Use plain mutable field defaults ({}/[]) instead of Field(default_factory=...)
(Pydantic v2 deep-copies per instance) to clear pylint FieldInfo no-member.
- Use self.__class__ is not X instead of type(self) for exact-type guards.
- Add pydantic to uv.lock via make lock-dependencies (was missing from the
databricks-labs-dqx dependency block).
make lint (black, ruff, mypy, pylint 10/10) and 1134 unit tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ace, aliased ChecksValidationStatus.errors, model_copy in lakebase test)
…ttable field (extra=forbid) and add validating BaseChecksStorageConfig.replace(); use it in lakebase test instead of validation-skipping model_copy
mwojtyczka
requested review from
pratikk-databricks
and removed request for
a team
mwojtyczka
added
do-not-merge
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1272 +/- ##
===========================================
- Coverage 92.56% 73.86% -18.71%
===========================================
Files 102 102
Lines 10075 10105 +30
===========================================
- Hits 9326 7464 -1862
- Misses 749 2641 +1892
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
|
❌ 786/788 passed, 2 failed, 41 skipped, 5h15m58s total ❌ test_load_checks_invalid_storage_config: TypeError: ChecksDummyStorageConfig.__init__() missing 1 required positional argument: 'location' (1.349s)❌ test_save_checks_invalid_storage_config: TypeError: ChecksDummyStorageConfig.__init__() missing 1 required positional argument: 'location' (1.213s)Running from acceptance #4935 |
Contributor
|
✅ 177/177 passed, 2 skipped, 9h16m29s total Running from anomaly #1049 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated sync from fork PR for CI testing.
Original PR: #1259
All tests, including unit and integration tests run on this PR (they are skipped for fork PRs).