[MAINTENANCE] Make scipy an optional extra and drop redundant cryptography pin - #12083
[MAINTENANCE] Make scipy an optional extra and drop redundant cryptography pin#12083dataders wants to merge 1 commit into
Conversation
…ptography pin scipy is 82MB and only used by three files for scipy.stats (KL-divergence and KS-test expectations); it's now installed via `great_expectations[scipy]` instead of every install, with a NotImported-style lazy wrapper in compatibility/scipy.py giving a clear error if those specific expectations are used without it. cryptography is dropped outright: it was already function-local imported for Snowflake key-pair auth, and snowflake-connector-python already brings it in transitively for anyone on the snowflake extra. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
✅ Deploy Preview for niobium-lead-7998 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Thanks for the pull request, @dataders! 👋 This change touches files that usually mean new data source or execution engine support:
Changes like that need an RFC agreed before implementation, so the design discussion happens before you invest in code. Sorry if this arrives after the fact — the check is here so the next contributor finds out at the right moment. To resolve this check, add one of these lines to the pull-request description:
Either answer satisfies the check. A maintainer will pick it up from there. |
|
Thank you for your contribution! Before we can merge this pull request, every committer needs to have signed our Contributor License Agreement (CLA). We could not find a signed CLA for: @dataders. Please sign the Individual Contributor License Agreement, or the Software Grant and Corporate Contributor License Agreement if you are contributing on behalf of your employer (see CLA.md for details). Once resolved, comment |
Summary
scipy(82MB) out of the default install and into a newgreat_expectations[scipy]extra. It's only used by three files forscipy.stats(KL-divergence and KS-test expectations). A lazy-import wrapper ingreat_expectations/compatibility/scipy.py(same pattern as the existingcompatibility/pyspark.py/compatibility/snowflake.py) gives a clear error pointing at thescipyextra if those specific expectations are used without it installed.cryptography(13MB) fromrequirements.txtentirely. It was already function-local imported insqlalchemy_execution_engine.pyfor Snowflake key-pair auth (no eager cost), andsnowflake-connector-pythonalready brings it in transitively for anyone on thesnowflakeextra — the top-level pin was pure redundant weight.Test plan
great_expectationsand all three scipy-dependent modules (expect_column_kl_divergence_to_be_less_than.py,column_bootstrapped_ks_test_p_value.py,column_parameterized_distribution_ks_test_p_value.py) import cleanly with scipy simulated as absent (verified viasys.modules/builtins.__import__patching)tests/integration/data_sources_and_expectations/expectations/test_expect_column_kl_divergence_to_be_less_than.py, 9/9)tests/expectations,tests/execution_engine,tests/datasource/fluentunit sweep: 2887 passed, 0 failedgreat_expectations[scipy]extra confirmed generated correctly viasetup.py::get_extras_require()🤖 Generated with Claude Code