Skip to content

[FLINK-40529][python] Warn on use of deprecated APIs, not at import time - #29061

Draft
deepyaman wants to merge 1 commit into
apache:masterfrom
deepyaman:FLINK-40529
Draft

[FLINK-40529][python] Warn on use of deprecated APIs, not at import time#29061
deepyaman wants to merge 1 commit into
apache:masterfrom
deepyaman:FLINK-40529

Conversation

@deepyaman

Copy link
Copy Markdown
Contributor

What is the purpose of the change

Deprecated in flink-python/pyflink/util/api_stability_decorators.py emitted its DeprecationWarning from __call__, which the decorator syntax invokes in order to apply the decorator. The warning therefore fired at decoration time — that is, at import — and the decorated function/class was returned unwrapped, so:

  • importing pyflink.table emitted a DeprecationWarning for every deprecated API it defines, whether or not the user touches them;
  • actually calling a deprecated API emitted nothing;
  • stacklevel=2 pointed at the decoration site inside PyFlink's own source, not at user code.
$ cd flink-python && python -W error::DeprecationWarning -c "import pyflink.table"
  File ".../pyflink/table/table_schema.py", line 28, in <module>
    @Deprecated(since="2.1.0", detail="""
  File ".../pyflink/util/api_stability_decorators.py", line 141, in __call__
    warnings.warn(msg, category=DeprecationWarning, stacklevel=2)
DeprecationWarning: TableSchema has been deprecated since version 2.1.0. ...

FLINK-37365, which introduced these decorators, describes the intended behaviour as warning "at runtime on their invocation", so this was an oversight. PyFlink supports Python >= 3.9, so warnings.deprecated (PEP 702) is not available; the fix is by hand, mirroring PEP 702's semantics where reasonable.

Brief change log

  • Deprecated applied to a function returns a functools.wraps wrapper that warns when the function is called, with stacklevel=2 so the warning is attributed to the caller.
  • Deprecated applied to a class returns the class itself and wraps __init__ on it, so isinstance checks and subclassing are unaffected. As in PEP 702 only instantiating the deprecated class itself warns, which also avoids warning twice when a deprecated class inherits the __init__ of a deprecated base class.
  • staticmethod/classmethod objects are unwrapped, decorated and re-packaged; properties, ABCs and Enum subclasses fall back to applying the docstring directive alone rather than raising. Decorating any of these previously failed, as did omitting the detail argument.
  • The message format, the DeprecationWarning category, the docstring/Sphinx directives and the __stability_decorators attribute read by PythonAPICompletenessTestCase are unchanged; Experimental, Internal, Public and PublicEvolving are untouched.
  • pyflink/util was not among the modules dev/integration_test.sh runs, so it is added there.

Verifying this change

This change added tests and can be verified as follows:

  • Added pyflink/util/tests/test_api_stability_decorators.py: no warning at decoration; a regression test that imports pyflink.table in a fresh interpreter and asserts no deprecation warning is emitted; warning on function call and on class instantiation; the warning is attributed to the caller's file and line; docstring directives still applied; __stability_decorators still populated; staticmethod/classmethod/property/ABC/Enum cases; subclassing and double-warning guards; the other four decorators still silent and still returning their argument unchanged.
  • Red-green verified: 18 of the 21 new tests fail against the unfixed decorator, and all 21 pass with the fix, on Python 3.9 (the minimum supported) and 3.11.
  • Manually verified that python -W error::DeprecationWarning -c "import pyflink.table" no longer raises, and that calling Table.get_schema warns exactly once, pointing at the calling script.
  • flake8 and mypy as configured in flink-python/tox.ini are clean, and the Sphinx docs build (SPHINXOPTS="-a -W" make html) still succeeds, with Deprecated since version 2.1.0 still rendered on the affected APIs.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Code 2.1.252 (Claude Opus 5)

Deprecated emitted its DeprecationWarning from __call__, which the
decorator syntax invokes in order to apply the decorator. The warning
therefore fired at decoration time -- that is, at import -- for every
deprecated API a module defines, whether or not the user touches it,
while actually calling one emitted nothing, and stacklevel=2 pointed
at the decoration site inside PyFlink's own source rather than at user
code.

Functions now get a functools.wraps wrapper that warns when called.
Classes are returned unchanged, with __init__ wrapped in place so that
isinstance checks and subclassing keep working; as in PEP 702, only
instantiating the deprecated class itself warns, which also avoids
warning twice when a deprecated class inherits the __init__ of a
deprecated base class. staticmethod and classmethod objects are
unwrapped and re-packaged, and properties, ABCs and Enum subclasses
degrade to the docstring directive rather than raising -- decorating
any of these used to fail. A missing detail argument no longer raises
either.

The message format, the DeprecationWarning category, the docstring
directives and the __stability_decorators attribute read by
PythonAPICompletenessTestCase are unchanged, and Experimental,
Internal, Public and PublicEvolving are unaffected.

pyflink/util was not in the list of modules that dev/integration_test.sh
runs, so it is added there for the new tests to run in CI.

Generated-by: Claude Code 2.1.252 (Claude Opus 5)
@flinkbot

flinkbot commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants