Migrate production container to Python 3.13

[ebhills]

Linked issue

Closes #1131

What changes

• Moves both production container stages to python:3.13-slim-bookworm.
• Adds a production-only constraint with pandas==2.3.3 and numpy==2.4.6. The reusable package contract remains pandas>=2.0,<3.0, so Pandas 3 is still prohibited.
• Removes the NumPy 1.24 source-build path, legacy pip option, and compiler packages.
• Makes Botocore/Pandas package-data cleanup independent of the Python minor version while preserving the S3-only Botocore model.
• Adds a credential-free runtime smoke check covering exact versions, package metadata, imports, NumPy/Pandas/PyArrow behavior, S3 model loading, package-data trimming, and compiler absence.
• Loads and executes the exact pull-request image in CI without publishing it or supplying repository secrets. The PR path also runs both local recipe samples, the credential-safe test selection, pip check, pip freeze, and image-size reporting.
• Preserves the existing post-merge immutable candidate, credentialed in-container tests, and promote-by-digest sequence.

This is a Python 3.13 / Pandas 2.3.3 compatibility bridge. It does not add Pandas 3 support or change the supported Python 3.11-3.13 package range.

How it was verified

CI run 80 passed in full.

Exact Linux pull-request image:

• Clean GitHub-hosted Linux/amd64 build completed with no configured build cache.
• Runtime smoke check passed with Python 3.13.1, Wrangles 1.19.0, NumPy 2.4.6, pandas 2.3.3, PyArrow 25.0.1, Polars 1.33.0, NumExpr 2.14.2, Boto3 1.43.78, and Botocore 1.43.78.
• Installed Wrangles metadata retained pandas<3.0; Pandas 3 rejection, S3-only Botocore data, removed pandas tests, and compiler absence all passed.
• python -m pip check reported no broken requirements; pip freeze was captured in the job log.
• Both local recipe samples passed inside the exact image.
• Credential-safe container suite: 1,507 passed, 112 deselected, 31 warnings. The warning summary contained only Python multiprocessing's multi-threaded fork() deprecation warning; no pandas FutureWarning or DeprecationWarning was reported.
• Candidate image size: 667,373,714 bytes (636.46 MiB).
• Current latest size on the same runner and measurement: 697,999,438 bytes (665.66 MiB).
• The candidate is 30,625,724 bytes (29.21 MiB / 4.39%) smaller.

Other CI and local evidence:

• The full existing CI matrix passed on Ubuntu and Windows with Python 3.11 and 3.13.
• All four package-install checks and schema generation passed.
• Local Python 3.13.1 focused smoke-check tests: 12 passed.
• Local Python 3.13.1 credential-safe suite: 1,507 passed, 112 deselected.
• Local python -m pip check, Python compilation, workflow YAML parsing, shell syntax, line-ending, and diff checks passed.
• The NumPy comparison in [TASK] Migrate the production Wrangles Docker image to Python 3.13 #1131 also tested 2.4.6 and 2.5.2 in isolated Python 3.13 environments; 2.4.6 was selected to preserve the package's Python 3.11 boundary and avoid combining additional deprecation removals with this migration.

Not yet claimed:

• The pull-request image is intentionally not published.
• Post-merge immutable-image publication, credentialed tests inside that pushed candidate, remote recipe execution, digest promotion, and verification of the promoted latest image cannot occur until merge.
• No downstream GHCR consumer or other external live-service behavior is claimed from the credential-safe image checks.

Compatibility and risk

• Public package metadata is unchanged: pandas remains >=2.0,<3.0; Python 3.11 and 3.12 support are not removed.
• The production image deliberately pins pandas 2.3.3 and NumPy 2.4.6. Pandas 3 remains separate work because its breaking behavior has not been addressed.
• Existing versioned GHCR tags and pinned downstream consumers are untouched.
• Linux wheel selection and runtime behavior are now covered by exact-image CI. Remaining production risk is limited to the existing post-merge publish/test/promote path.
• Rollback is a revert of this single migration commit. The workflow only promotes latest after the immutable candidate passes its container tests.

Ready-for-review checklist

• One human delivery owner is assigned
• The linked issue and intended milestone are correct
• The branch is current with main and has no unrelated commits
• Focused tests pass
• New or changed behavior has direct test coverage
• Documentation/configuration is updated where applicable
• The PR contains no unrelated changes
• The PR description reflects the branch's current scope and latest validation
• One primary reviewer is requested only when this PR is ready

[mborodii-prog]

@ebhills Change --only-binary=numpy,pandas to --only-binary=:all: so the wheel-only guarantee covers every dependency the compiler-less image relies on; a future wheel gap then fails the build with a clear "no matching distribution" error instead of silently needing a compiler that no longer exists.

[mborodii-prog]

@ebhills Replace the manual string matching with packaging.requirements.Requirement: filter requirements by Requirement(r).name.lower() == "pandas", then check specifier.contains("2.999") and not specifier.contains("3.0") on the resulting SpecifierSet instead of substring-matching "<3.0".

[mborodii-prog]

@ebhills Add a negative-path test that patches pandas.testing.assert_frame_equal to raise and asserts validate_data_round_trip propagates it; for the S3 test, use pytest-mock to patch socket.create_connection to raise on any call and assert it's never invoked, proving the "no network" claim structurally.

[mborodii-prog]

@ebhills Add _require(Path(botocore_data).exists(), f"Botocore data directory not found: {botocore_data}") immediately before the iterdir() call, matching the module's existing _require-based error convention.

[mborodii-prog]

@ebhills Add --only-binary=:all: to this pip install call, mirroring the fix applied to the Dockerfile's production install, so a missing wheel surfaces as a clear pip resolution error instead of a confusing compiler-not-found failure.