diff --git a/CHANGELOG.md b/CHANGELOG.md index c2c404f..e5fa335 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,8 @@ project intends to use Semantic Versioning once its public API reaches stability ## [Unreleased] +## [0.2.0] - 2026-07-20 + ### Added - `testenix pytest [PYTEST_ARGS ...]` compatibility bridge for unchanged pytest suites, preserving @@ -28,6 +30,25 @@ project intends to use Semantic Versioning once its public API reaches stability - Truthful post-commit durability/report warnings, package-aware unittest outcome mapping, Testenix validation-worker containment, and conservative blocking of pytest session fixtures and unittest class-cleanup hooks whose lifecycle cannot be preserved. +- Native `tmp_path` and transactional `monkeypatch` fixtures. The initial monkeypatch contract + covers the object/attribute and dotted-import forms of `setattr`, plus `setenv`, with automatic + per-test rollback. Static module-local helper calls are accepted only when every propagated use + can be proven safe; aliases, dynamic rebinding, unsupported methods, and escaped values remain + blocked. +- Safe conversion of bare `@pytest.mark.asyncio` coroutine tests, simple pytest classes through + fresh-instance wrappers, and statically declared autouse fixtures. Async migration creates and + closes an isolated `asyncio.Runner` per test or case, validates effective pytest-asyncio loop and + debug configuration, and blocks custom event-loop policies or unmarked async semantics. +- Fail-closed class conversion for lifecycle hooks, decorated or inherited classes, annotated + class state, custom constructors, and method defaults that cannot be preserved by wrappers. + +### Changed + +- Migration console output now distinguishes analyzed, validated, generated, and published + candidates. Repeated diagnostics are grouped by code, while JSON audit reports retain every + source- and line-addressed entry. +- The one-affinity-unit `MIG006` warning is emitted only for statically supported check/publication + candidates, not for dry-run or already-blocked migrations. ## [0.1.0] - 2026-07-20 diff --git a/README.md b/README.md index 9b73225..0ccedda 100644 --- a/README.md +++ b/README.md @@ -101,6 +101,11 @@ testenix run tests_testenix For pytest migration, install `testenix[pytest]`; unittest migration uses the standard library. `auto` supports pytest and unittest in separate modules within one selection. +Large unsupported suites get a grouped console summary instead of one wall of repeated lines. +Use `--report-json FILE` or `--report-json -` to retain every individual diagnostic with its source +and line. A blocked run labels safe in-memory mappings as a *statically convertible subset*; only a +report with `status: published` means that Testenix created the requested output. + This is a copy-and-validate transaction, not an in-place rewrite. Testenix fingerprints the sources, generates into private staging, runs the green original suite in a disposable project copy, runs the native candidate with one worker and again in parallel, compares inventory and @@ -111,13 +116,19 @@ rename, Testenix warns without pretending the already published output was rolle paths must be new, inside the project, and disjoint from both source and generated suites. There is no `--force` option, and old tests are never deleted or renamed. -The converter stops on semantics it cannot preserve. Its current pytest subset covers module -functions, one static parametrization, simple local/adjacent-conftest fixtures, static skips, and -plain markers. The unittest adapter preserves per-test lifecycle and assertions by generating -native wrappers around the original `TestCase.run()` protocol; those wrappers locate originals -independently of `cwd` and verify the complete selected-Python-source SHA-256 manifest, so the old -unittest files must remain present. Keep the generated unittest directory at its published path as -well; rerun migration after moving either side. +The converter stops on semantics it cannot preserve. The v0.2 pytest subset covers module +functions, one static parametrization, simple local/adjacent-conftest fixtures, statically declared +autouse fixtures, bare `@pytest.mark.asyncio` coroutine tests through fresh function-scoped loop +wrappers, and simple pytest classes. Native `tmp_path` and a dependency-free `monkeypatch` +implementation cover the common `setattr` and `setenv` forms with automatic per-test rollback, +including calls through statically provable module-local helpers. Complex class lifecycle, async +fixtures, unmarked async tests, configured async loop scopes or debug mode, custom +`event_loop_policy`, and the rest of pytest's built-in fixtures remain blocked. The unittest adapter +preserves per-test lifecycle and assertions by generating native wrappers around the original +`TestCase.run()` protocol; those wrappers locate originals independently of `cwd` and verify the +complete selected-Python-source SHA-256 manifest, so the old unittest files must remain present. +Keep the generated unittest directory at its published path as well; rerun migration after moving +either side. See the full [safe migration guide](https://polishdataengineer.github.io/testenix/guides/migration/) for the support matrix, rollback contract, CI rollout, audit-report schema, and performance @@ -218,7 +229,7 @@ test. ## Where Testenix is deliberately different -The `testenix run` engine is not a native drop-in reimplementation of pytest. Its v0.1 value is a +The `testenix run` engine is not a native drop-in reimplementation of pytest. Its v0.2 value is a smaller, coherent native stack: async tests and async fixtures need no plugin, parallel execution and duration-aware scheduling need no xdist, every retry remains visible, and a worker crash cannot silently erase tests that completed before it. The native runtime has no third-party dependencies. @@ -287,7 +298,8 @@ See the [generated results and chart](https://polishdataengineer.github.io/teste JSON-safe representation when a value itself is not serializable. - Synchronous test and fixture bodies run outside Testenix's internal asyncio loop. APIs restricted to Python's main thread, such as installing signal handlers, are not supported inside those - bodies in v0.1. + bodies in v0.2. Migrated pytest-asyncio wrappers are synchronous from Testenix's perspective and + therefore share this restriction while creating a fresh event loop for each test or case. - On Windows, a script that calls the programmatic `run()`/`run_async()` API must use the standard `if __name__ == "__main__":` multiprocessing guard. The `testenix` CLI handles process startup itself. @@ -300,12 +312,12 @@ See the [generated results and chart](https://polishdataengineer.github.io/teste create 3,000 parallel units; spread independent tests across modules and measure the generated suite before making a project-specific speed claim. - Test impact analysis, result caching, remote workers, and deep pytest-result aggregation are not - part of version 0.1. + part of version 0.2. ## Project status -Testenix is pre-1.0 software. The distribution, import package, CLI, configuration namespace, and -state directory consistently use `testenix`. The project is licensed under MIT and its release +Testenix 0.2.0 is pre-1.0 software. The distribution, import package, CLI, configuration namespace, +and state directory consistently use `testenix`. The project is licensed under MIT and its release workflow uses PyPI Trusted Publishing; the first PyPI release has not been published yet. ## Development diff --git a/docs/architecture.md b/docs/architecture.md index 13dc331..6b0eb04 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -18,7 +18,7 @@ Windows: testenix pytest =========> pytest.console_main -> collector/plugins/exe ``` The bridge is a CLI infrastructure adapter, not a native collection adapter. It does not emit -Testenix events or construct a `RunResult` in version 0.1. +Testenix events or construct a `RunResult` in version 0.2. The migration adapter is separate from that handoff. It statically converts a deliberately small pytest subset or generates SHA-pinned wrappers around the standard unittest protocol. Its @@ -73,7 +73,7 @@ Authoring API -> supervised collection -> inert manifest -> affinity scheduler - - migration analyzers depend on serializable migration contracts, while shadow execution and atomic publication remain application/infrastructure concerns. -## Version 0.1 scope +## Version 0.2 scope - explicit `@test` and `@fixture` authoring API, plus conventional `test_*` discovery; - sync functions, coroutines, generators, and async-generator fixture teardown; @@ -87,9 +87,12 @@ Authoring API -> supervised collection -> inert manifest -> affinity scheduler - - an optional platform-aware pytest handoff for unchanged legacy suites. - conservative pytest/unittest migration with static diagnostics, differential validation, source fingerprints, and create-only publication. +- dependency-free `tmp_path` and reversible `monkeypatch` fixtures, plus native autouse resolution; +- conservative migration of bare pytest-asyncio coroutine markers through isolated fresh-loop + wrappers, plus simple pytest classes. Remote workers, distributed storage, result caching, automatic quarantine, and a stable third-party -plugin SDK are deliberately outside version 0.1. +plugin SDK are deliberately outside version 0.2. ## Fixture scopes and process isolation @@ -98,7 +101,7 @@ between parallel shared workers. Multiple modules assigned to one shard execute process and fixture runtime. A test with an explicit timeout (including a global timeout applied at selection) is instead a single-test isolation unit with a hard process deadline. -Scope therefore has the following concrete meaning in version 0.1: +Scope therefore has the following concrete meaning in version 0.2: | Scope | Lifetime | | --- | --- | diff --git a/docs/guides/fixtures.md b/docs/guides/fixtures.md index 2707854..144cbe1 100644 --- a/docs/guides/fixtures.md +++ b/docs/guides/fixtures.md @@ -62,6 +62,41 @@ def test_empty_repository(repository: Repository) -> None: The dependency graph is validated before execution. Missing fixtures and cycles become collection issues instead of hanging the run. +## Built-in fixtures + +Testenix 0.2 provides two dependency-free, test-scoped built-ins by name: + +```python +from pathlib import Path + + +def test_isolated_file(tmp_path: Path, monkeypatch) -> None: + target = tmp_path / "value.txt" + target.write_text("ok", encoding="utf-8") + monkeypatch.setenv("TESTENIX_EXAMPLE", "enabled") + assert target.read_text(encoding="utf-8") == "ok" +``` + +`tmp_path` is a fresh `pathlib.Path` removed during teardown. `monkeypatch` supports reversible +object/attribute and dotted-import `setattr`, `setenv`, and idempotent `undo`. Changes are restored +in LIFO order even when the test fails. Other pytest monkeypatch operations and pytest built-ins +such as `capsys`, `caplog`, and `request` are not native Testenix fixtures. + +## Autouse fixtures + +Use `autouse=True` when setup and cleanup must apply to every test that can see a fixture: + +```python +@fixture(autouse=True) +def isolated_environment(monkeypatch): + monkeypatch.setenv("APP_ENV", "test") + yield +``` + +Explicitly requesting the same fixture still resolves one cached value for the test. A local +fixture definition overrides a visible imported definition with the same name before Testenix +chooses which fixtures run automatically. + ## Scopes ```python @@ -75,7 +110,7 @@ def worker_resource() -> Resource: return Resource() ``` -| Scope | Lifetime in Testenix 0.1 | +| Scope | Lifetime in Testenix 0.2 | | --- | --- | | `test` | One instance for one concrete test attempt. | | `module` | Shared by normal tests from the module inside one worker. | diff --git a/docs/guides/migration.md b/docs/guides/migration.md index 5efdd10..47bf83c 100644 --- a/docs/guides/migration.md +++ b/docs/guides/migration.md @@ -41,10 +41,12 @@ They must be separate modules; a file that mixes both test models is rejected. The destination defaults to `testenix_migrated`. It must be a new directory inside the project, with an existing real parent. There is deliberately no `--force` and no in-place mode. An integer `--workers` value must be at least 2 so the parallel gate cannot silently repeat the -serial command. A one-module suite still has one schedulable affinity unit, which is disclosed as a -`MIG006` warning; spread tests across independent modules to exercise multiple workers. An -audit-report path must also be new, inside the project, and disjoint from every selected source and -the output suite. Testenix never replaces an existing report. +serial command. During `--check` or publication, a one-module candidate still has one schedulable +affinity unit, which is disclosed as a `MIG006` warning; spread tests across independent modules to +exercise multiple workers. The warning is not shown for `--dry-run` or an already-blocked migration +because no parallel candidate is run in either case. An audit-report path must also be new, inside +the project, and disjoint from every selected source and the output suite. Testenix never replaces +an existing report. ## Transaction and rollback contract @@ -89,13 +91,25 @@ with external effects. ## Pytest conversion contract -The current converter supports the subset below: +The v0.2 converter supports the subset below: - module-level pytest-default `test*` functions and normal Python `assert` statements; +- simple `Test*` classes with a fresh zero-argument instance per test method, including ordinary + helper methods; inheritance, class decorators, custom construction, and pytest class lifecycle + hooks remain outside the safe subset; +- bare `@pytest.mark.asyncio` on `async def` tests. The generated synchronous wrapper runs every + test or parametrized case in a fresh, closed `asyncio.Runner`, matching pytest-asyncio's default + function-scoped loop isolation; - one static `pytest.mark.parametrize` with static names, rows, IDs, and unmarked `pytest.param(..., id=...)` values; -- local fixtures with no parameters, autouse, or `request`, using function or module scope; +- local fixtures using function or module scope, including a statically boolean `autouse=True`; - simple fixtures from an adjacent `conftest.py` in the same directory; +- native `tmp_path`, which supplies a fresh `pathlib.Path` and removes its temporary directory at + test teardown; +- native `monkeypatch.setattr` in object/attribute and dotted-import forms, plus `setenv` and + idempotent `undo`; successful changes are restored in LIFO order during test teardown. + `monkeypatch` may also flow through statically resolved module-local helpers when every use can + be proven to stay inside this supported subset; - static `pytest.mark.skip` and `pytest.mark.skipif`; - plain argument-free custom markers, converted to Testenix tags; - pytest runtime helpers `approx`, `deprecated_call`, `fail`, `raises`, and `warns`. Generated @@ -103,14 +117,21 @@ The current converter supports the subset below: It blocks, with a file and line diagnostic: -- pytest test classes, xfail, runtime skip/xfail/importorskip/exit, xunit lifecycle hooks; -- built-in fixtures such as `tmp_path`, `monkeypatch`, `capsys`, and `request`; -- autouse or parametrized fixtures, fixture overrides, and inherited ancestor-`conftest` fixtures; +- complex pytest test classes, xfail, runtime skip/xfail/importorskip/exit, and xunit lifecycle + hooks; +- built-in fixtures other than `tmp_path` and `monkeypatch`, such as `capsys`, `caplog`, and + `request`; monkeypatch operations outside the documented native subset, imported or dynamically + rebound helpers, and values that escape static analysis are also unsupported; +- dynamically configured autouse fixtures, parametrized fixtures, fixture overrides, and inherited + ancestor-`conftest` fixtures; - session-scoped fixtures, because pytest creates one per run while Testenix session scope is currently worker-local; - stacked, dynamic, indirect, scoped, or per-case-marked parametrization; - `usefixtures`, module-level `pytestmark`, hook functions, plugin registration, and semantic - plugin markers such as asyncio/anyio, timeout, order, repeat, or flaky; + plugin markers such as anyio, configured asyncio, timeout, order, repeat, or flaky; +- unmarked async tests, async fixtures, custom `event_loop_policy`, non-function asyncio loop + scopes, and enabled asyncio debug mode. Testenix checks the effective pytest configuration and + relevant `PYTEST_ADDOPTS` overrides before accepting bare asyncio markers; - decorators and required parameters whose execution meaning cannot be established statically. Any converted pytest file whose name is not already `test_*.py` is renamed in the generated copy @@ -167,6 +188,11 @@ hashes, every source-to-target mapping, per-test outcomes, generated files, line timings and summaries for all validation runs, publication status, and an `originals_modified` flag. It is false for a successful transaction and true when a terminal source recheck detects drift; the flag reports observed state and does not claim that Testenix caused an independent edit. +The console groups repeated diagnostics by code and shows the first location, so large suites do +not produce hundreds of near-identical lines. `--report-json FILE` and `--report-json -` always +retain every individual source- and line-addressed diagnostic. On a blocked transaction, the +console calls any safe in-memory result a *statically convertible subset* rather than implying that +those tests were published. ## Performance with thousands of migrated tests diff --git a/docs/guides/pytest-compatibility.md b/docs/guides/pytest-compatibility.md index 6f227b2..240ec89 100644 --- a/docs/guides/pytest-compatibility.md +++ b/docs/guides/pytest-compatibility.md @@ -51,17 +51,21 @@ native `--workers` option into an xdist option. | Capability | `testenix pytest` | `migrate` then `run` | Direct `testenix run` | | --- | --- | --- | --- | | Plain module-level pytest-default `test*` functions | Yes, through pytest | Yes | `test_` or native `@test` | -| Pytest classes | Yes | Blocked | No | -| Simple local fixture | Yes | Converted | Only native `@fixture` | +| Simple pytest classes | Yes | Fresh-instance wrappers | No direct class collection | +| Complex class lifecycle/inheritance | Yes | Blocked | No | +| Simple local fixture | Yes | Converted, including static autouse | Only native `@fixture` | | Session-scoped pytest fixture | Yes | Blocked: run-global vs worker-local | Native scope is worker-local | -| Built-in/dynamic fixture | Yes | Blocked | No pytest fixture semantics | +| `tmp_path` | Yes | Native built-in | Native built-in | +| `monkeypatch` | Yes | `setattr`/`setenv` subset | Native reversible subset | +| Other built-in/dynamic fixture | Yes | Blocked | No pytest fixture semantics | | Adjacent `conftest.py` fixture | Yes | Simple static subset | No automatic conftest discovery | | Static single `parametrize` | Yes | Converted to cases | Use `@case` or `@cases` | | Skip and plain custom marker | Yes | Converted | Use Testenix decorators/tags | | Pytest xfail | Yes | Blocked due semantic differences | Use native `@xfail` intentionally | | Pytest assertion rewriting | Yes | No | No | | Plugins, hooks, pytest config | Yes | Blocked/not translated | No | -| Async plugin semantics | According to plugins | Blocked | Native async needs no plugin | +| Bare `@pytest.mark.asyncio` coroutine | According to plugin | Fresh function-scoped loop wrapper | Native async needs no plugin | +| Configured async/anyio plugin semantics | According to plugins | Blocked | No plugin semantics | | Testenix worker scheduler/history | No | Yes after migration | Yes | | Testenix retries and lossless results | No | Yes after migration | Yes | | Published native speedups | No | Measure generated suite | Only documented workloads | @@ -116,9 +120,12 @@ explicit equivalents: 5. keep unsupported modules on `testenix pytest` and keep all originals during rollout; 6. benchmark the generated directory before making a native performance claim. -The migrator replaces simple fixtures, parametrization, skip conditions, and plain markers in the -generated copy. It never performs edits in place. Manual rewrites are still necessary for blocked -plugin, hook, built-in fixture, class, xfail, and dynamic behavior. +The migrator replaces simple fixtures, parametrization, skip conditions, plain markers, bare +pytest-asyncio coroutine markers with a fresh closed loop per test or case, and the supported +built-in fixtures in the generated copy. It wraps simple pytest class methods with a fresh instance +per test. It never performs edits in place. +Manual rewrites are still necessary for blocked plugin, hook, complex class lifecycle, xfail, and +dynamic behavior. The current bridge does not convert pytest outcomes into a Testenix `RunResult`. Deeper event and report aggregation is a future compatibility layer and requires explicit pytest hook integration. diff --git a/docs/llms-full.txt b/docs/llms-full.txt index 0201cc0..00276b5 100644 --- a/docs/llms-full.txt +++ b/docs/llms-full.txt @@ -404,17 +404,21 @@ native `--workers` option into an xdist option. | Capability | `testenix pytest` | `migrate` then `run` | Direct `testenix run` | | --- | --- | --- | --- | | Plain module-level pytest-default `test*` functions | Yes, through pytest | Yes | `test_` or native `@test` | -| Pytest classes | Yes | Blocked | No | -| Simple local fixture | Yes | Converted | Only native `@fixture` | +| Simple pytest classes | Yes | Fresh-instance wrappers | No direct class collection | +| Complex class lifecycle/inheritance | Yes | Blocked | No | +| Simple local fixture | Yes | Converted, including static autouse | Only native `@fixture` | | Session-scoped pytest fixture | Yes | Blocked: run-global vs worker-local | Native scope is worker-local | -| Built-in/dynamic fixture | Yes | Blocked | No pytest fixture semantics | +| `tmp_path` | Yes | Native built-in | Native built-in | +| `monkeypatch` | Yes | `setattr`/`setenv` subset | Native reversible subset | +| Other built-in/dynamic fixture | Yes | Blocked | No pytest fixture semantics | | Adjacent `conftest.py` fixture | Yes | Simple static subset | No automatic conftest discovery | | Static single `parametrize` | Yes | Converted to cases | Use `@case` or `@cases` | | Skip and plain custom marker | Yes | Converted | Use Testenix decorators/tags | | Pytest xfail | Yes | Blocked due semantic differences | Use native `@xfail` intentionally | | Pytest assertion rewriting | Yes | No | No | | Plugins, hooks, pytest config | Yes | Blocked/not translated | No | -| Async plugin semantics | According to plugins | Blocked | Native async needs no plugin | +| Bare `@pytest.mark.asyncio` coroutine | According to plugin | Fresh function-scoped loop wrapper | Native async needs no plugin | +| Configured async/anyio plugin semantics | According to plugins | Blocked | No plugin semantics | | Testenix worker scheduler/history | No | Yes after migration | Yes | | Testenix retries and lossless results | No | Yes after migration | Yes | | Published native speedups | No | Measure generated suite | Only documented workloads | @@ -469,9 +473,12 @@ explicit equivalents: 5. keep unsupported modules on `testenix pytest` and keep all originals during rollout; 6. benchmark the generated directory before making a native performance claim. -The migrator replaces simple fixtures, parametrization, skip conditions, and plain markers in the -generated copy. It never performs edits in place. Manual rewrites are still necessary for blocked -plugin, hook, built-in fixture, class, xfail, and dynamic behavior. +The migrator replaces simple fixtures, parametrization, skip conditions, plain markers, bare +pytest-asyncio coroutine markers with a fresh closed loop per test or case, and the supported +built-in fixtures in the generated copy. It wraps simple pytest class methods with a fresh instance +per test. It never performs edits in place. +Manual rewrites are still necessary for blocked plugin, hook, complex class lifecycle, xfail, and +dynamic behavior. The current bridge does not convert pytest outcomes into a Testenix `RunResult`. Deeper event and report aggregation is a future compatibility layer and requires explicit pytest hook integration. @@ -526,10 +533,12 @@ They must be separate modules; a file that mixes both test models is rejected. The destination defaults to `testenix_migrated`. It must be a new directory inside the project, with an existing real parent. There is deliberately no `--force` and no in-place mode. An integer `--workers` value must be at least 2 so the parallel gate cannot silently repeat the -serial command. A one-module suite still has one schedulable affinity unit, which is disclosed as a -`MIG006` warning; spread tests across independent modules to exercise multiple workers. An -audit-report path must also be new, inside the project, and disjoint from every selected source and -the output suite. Testenix never replaces an existing report. +serial command. During `--check` or publication, a one-module candidate still has one schedulable +affinity unit, which is disclosed as a `MIG006` warning; spread tests across independent modules to +exercise multiple workers. The warning is not shown for `--dry-run` or an already-blocked migration +because no parallel candidate is run in either case. An audit-report path must also be new, inside +the project, and disjoint from every selected source and the output suite. Testenix never replaces +an existing report. ## Transaction and rollback contract @@ -574,13 +583,25 @@ with external effects. ## Pytest conversion contract -The current converter supports the subset below: +The v0.2 converter supports the subset below: - module-level pytest-default `test*` functions and normal Python `assert` statements; +- simple `Test*` classes with a fresh zero-argument instance per test method, including ordinary + helper methods; inheritance, class decorators, custom construction, and pytest class lifecycle + hooks remain outside the safe subset; +- bare `@pytest.mark.asyncio` on `async def` tests. The generated synchronous wrapper runs every + test or parametrized case in a fresh, closed `asyncio.Runner`, matching pytest-asyncio's default + function-scoped loop isolation; - one static `pytest.mark.parametrize` with static names, rows, IDs, and unmarked `pytest.param(..., id=...)` values; -- local fixtures with no parameters, autouse, or `request`, using function or module scope; +- local fixtures using function or module scope, including a statically boolean `autouse=True`; - simple fixtures from an adjacent `conftest.py` in the same directory; +- native `tmp_path`, which supplies a fresh `pathlib.Path` and removes its temporary directory at + test teardown; +- native `monkeypatch.setattr` in object/attribute and dotted-import forms, plus `setenv` and + idempotent `undo`; successful changes are restored in LIFO order during test teardown. + `monkeypatch` may also flow through statically resolved module-local helpers when every use can + be proven to stay inside this supported subset; - static `pytest.mark.skip` and `pytest.mark.skipif`; - plain argument-free custom markers, converted to Testenix tags; - pytest runtime helpers `approx`, `deprecated_call`, `fail`, `raises`, and `warns`. Generated @@ -588,14 +609,21 @@ The current converter supports the subset below: It blocks, with a file and line diagnostic: -- pytest test classes, xfail, runtime skip/xfail/importorskip/exit, xunit lifecycle hooks; -- built-in fixtures such as `tmp_path`, `monkeypatch`, `capsys`, and `request`; -- autouse or parametrized fixtures, fixture overrides, and inherited ancestor-`conftest` fixtures; +- complex pytest test classes, xfail, runtime skip/xfail/importorskip/exit, and xunit lifecycle + hooks; +- built-in fixtures other than `tmp_path` and `monkeypatch`, such as `capsys`, `caplog`, and + `request`; monkeypatch operations outside the documented native subset, imported or dynamically + rebound helpers, and values that escape static analysis are also unsupported; +- dynamically configured autouse fixtures, parametrized fixtures, fixture overrides, and inherited + ancestor-`conftest` fixtures; - session-scoped fixtures, because pytest creates one per run while Testenix session scope is currently worker-local; - stacked, dynamic, indirect, scoped, or per-case-marked parametrization; - `usefixtures`, module-level `pytestmark`, hook functions, plugin registration, and semantic - plugin markers such as asyncio/anyio, timeout, order, repeat, or flaky; + plugin markers such as anyio, configured asyncio, timeout, order, repeat, or flaky; +- unmarked async tests, async fixtures, custom `event_loop_policy`, non-function asyncio loop + scopes, and enabled asyncio debug mode. Testenix checks the effective pytest configuration and + relevant `PYTEST_ADDOPTS` overrides before accepting bare asyncio markers; - decorators and required parameters whose execution meaning cannot be established statically. Any converted pytest file whose name is not already `test_*.py` is renamed in the generated copy @@ -652,6 +680,11 @@ hashes, every source-to-target mapping, per-test outcomes, generated files, line timings and summaries for all validation runs, publication status, and an `originals_modified` flag. It is false for a successful transaction and true when a terminal source recheck detects drift; the flag reports observed state and does not claim that Testenix caused an independent edit. +The console groups repeated diagnostics by code and shows the first location, so large suites do +not produce hundreds of near-identical lines. `--report-json FILE` and `--report-json -` always +retain every individual source- and line-addressed diagnostic. On a blocked transaction, the +console calls any safe in-memory result a *statically convertible subset* rather than implying that +those tests were published. ## Performance with thousands of migrated tests @@ -945,6 +978,41 @@ def test_empty_repository(repository: Repository) -> None: The dependency graph is validated before execution. Missing fixtures and cycles become collection issues instead of hanging the run. +## Built-in fixtures + +Testenix 0.2 provides two dependency-free, test-scoped built-ins by name: + +```python +from pathlib import Path + + +def test_isolated_file(tmp_path: Path, monkeypatch) -> None: + target = tmp_path / "value.txt" + target.write_text("ok", encoding="utf-8") + monkeypatch.setenv("TESTENIX_EXAMPLE", "enabled") + assert target.read_text(encoding="utf-8") == "ok" +``` + +`tmp_path` is a fresh `pathlib.Path` removed during teardown. `monkeypatch` supports reversible +object/attribute and dotted-import `setattr`, `setenv`, and idempotent `undo`. Changes are restored +in LIFO order even when the test fails. Other pytest monkeypatch operations and pytest built-ins +such as `capsys`, `caplog`, and `request` are not native Testenix fixtures. + +## Autouse fixtures + +Use `autouse=True` when setup and cleanup must apply to every test that can see a fixture: + +```python +@fixture(autouse=True) +def isolated_environment(monkeypatch): + monkeypatch.setenv("APP_ENV", "test") + yield +``` + +Explicitly requesting the same fixture still resolves one cached value for the test. A local +fixture definition overrides a visible imported definition with the same name before Testenix +chooses which fixtures run automatically. + ## Scopes ```python @@ -958,7 +1026,7 @@ def worker_resource() -> Resource: return Resource() ``` -| Scope | Lifetime in Testenix 0.1 | +| Scope | Lifetime in Testenix 0.2 | | --- | --- | | `test` | One instance for one concrete test attempt. | | `module` | Shared by normal tests from the module inside one worker. | @@ -1276,6 +1344,17 @@ rename leaves the output absent. A report-only failure after publication warns b validated output and successful exit status intact. See [safe migration](https://polishdataengineer.github.io/testenix/guides/migration/) for supported constructs, unittest's SHA-pinned wrapper model, rollback guarantees, and external-side-effect boundaries. +Human-readable output distinguishes an analyzed candidate, a validated candidate, a generated +candidate, a statically convertible subset, and a published conversion. Repeated diagnostics are +grouped by severity and code, with the first source location shown. Use `--report-json FILE` or +`--report-json -` when every individual line-addressed diagnostic is required. The JSON field +`converted_tests` counts source-to-target mappings built in memory; only `status: published` and +`published: true` mean that an output directory was created. + +`MIG006` warns that a candidate has only one schedulable module affinity unit despite a parallel +worker setting. It is emitted only for `--check` or publication after static analysis succeeds, +because dry-run and unsupported transactions never execute the parallel gate. + ## Examples ```console @@ -2108,7 +2187,7 @@ Windows: testenix pytest =========> pytest.console_main -> collector/plugins/exe ``` The bridge is a CLI infrastructure adapter, not a native collection adapter. It does not emit -Testenix events or construct a `RunResult` in version 0.1. +Testenix events or construct a `RunResult` in version 0.2. The migration adapter is separate from that handoff. It statically converts a deliberately small pytest subset or generates SHA-pinned wrappers around the standard unittest protocol. Its @@ -2163,7 +2242,7 @@ Authoring API -> supervised collection -> inert manifest -> affinity scheduler - - migration analyzers depend on serializable migration contracts, while shadow execution and atomic publication remain application/infrastructure concerns. -## Version 0.1 scope +## Version 0.2 scope - explicit `@test` and `@fixture` authoring API, plus conventional `test_*` discovery; - sync functions, coroutines, generators, and async-generator fixture teardown; @@ -2177,9 +2256,12 @@ Authoring API -> supervised collection -> inert manifest -> affinity scheduler - - an optional platform-aware pytest handoff for unchanged legacy suites. - conservative pytest/unittest migration with static diagnostics, differential validation, source fingerprints, and create-only publication. +- dependency-free `tmp_path` and reversible `monkeypatch` fixtures, plus native autouse resolution; +- conservative migration of bare pytest-asyncio coroutine markers through isolated fresh-loop + wrappers, plus simple pytest classes. Remote workers, distributed storage, result caching, automatic quarantine, and a stable third-party -plugin SDK are deliberately outside version 0.1. +plugin SDK are deliberately outside version 0.2. ## Fixture scopes and process isolation @@ -2188,7 +2270,7 @@ between parallel shared workers. Multiple modules assigned to one shard execute process and fixture runtime. A test with an explicit timeout (including a global timeout applied at selection) is instead a single-test isolation unit with a hard process deadline. -Scope therefore has the following concrete meaning in version 0.1: +Scope therefore has the following concrete meaning in version 0.2: | Scope | Lifetime | | --- | --- | @@ -2249,17 +2331,29 @@ Source: docs/roadmap.md with source fingerprints, disposable validation copies, serial/parallel parity, and atomic create-only publication. -## 0.2 — fast feedback +## 0.2 — real-world pytest migration + +- Dependency-free native `tmp_path` and reversible `monkeypatch` fixtures for common + `setattr`/`setenv` usage. +- Static autouse fixtures with native setup and teardown ownership. +- Bare pytest-asyncio coroutine markers translated to isolated fresh-loop wrappers that preserve + the plugin's default function-scoped loop lifecycle. +- Fresh-instance wrappers for simple pytest classes, while complex lifecycle and inheritance stay + fail-closed. +- Compact migration diagnostics on the console with complete line detail retained in JSON. +- Differential validation against a 118-test real-world suite before publication. + +## 0.3 — fast feedback - Dynamic micro-shards and work stealing. - `--last-failed`, watch mode, and failure fingerprints. - Test-impact analysis in shadow mode with an explanation for every selection decision. - Stable assertion-diff protocol and improved plain-assert diagnostics. -## 0.3 — adoption +## 0.4 — adoption - Pytest hook adapter translating collection and outcomes into Testenix events and `RunResult`. -- Expand migration beyond the v0.1 static subset only when new transformations have differential +- Expand migration beyond the v0.2 static subset only when new transformations have differential semantics tests on real projects. - Versioned reporter and selector plugin interfaces. - IDE protocol and machine-readable collection manifest. @@ -2288,6 +2382,8 @@ project intends to use Semantic Versioning once its public API reaches stability ## [Unreleased] +## [0.2.0] - 2026-07-20 + ### Added - `testenix pytest [PYTEST_ARGS ...]` compatibility bridge for unchanged pytest suites, preserving @@ -2311,6 +2407,25 @@ project intends to use Semantic Versioning once its public API reaches stability - Truthful post-commit durability/report warnings, package-aware unittest outcome mapping, Testenix validation-worker containment, and conservative blocking of pytest session fixtures and unittest class-cleanup hooks whose lifecycle cannot be preserved. +- Native `tmp_path` and transactional `monkeypatch` fixtures. The initial monkeypatch contract + covers the object/attribute and dotted-import forms of `setattr`, plus `setenv`, with automatic + per-test rollback. Static module-local helper calls are accepted only when every propagated use + can be proven safe; aliases, dynamic rebinding, unsupported methods, and escaped values remain + blocked. +- Safe conversion of bare `@pytest.mark.asyncio` coroutine tests, simple pytest classes through + fresh-instance wrappers, and statically declared autouse fixtures. Async migration creates and + closes an isolated `asyncio.Runner` per test or case, validates effective pytest-asyncio loop and + debug configuration, and blocks custom event-loop policies or unmarked async semantics. +- Fail-closed class conversion for lifecycle hooks, decorated or inherited classes, annotated + class state, custom constructors, and method defaults that cannot be preserved by wrappers. + +### Changed + +- Migration console output now distinguishes analyzed, validated, generated, and published + candidates. Repeated diagnostics are grouped by code, while JSON audit reports retain every + source- and line-addressed entry. +- The one-affinity-unit `MIG006` warning is emitted only for statically supported check/publication + candidates, not for dry-run or already-blocked migrations. ## [0.1.0] - 2026-07-20 @@ -2648,7 +2763,7 @@ Inside a module, ``test_*`` functions and functions decorated with ## `testenix.fixture` ```text -fixture(function: 'Callable[..., Any] | None' = None, /, *, scope: 'Scope | str' = , name: 'str | None' = None) -> 'Callable[..., Any]' +fixture(function: 'Callable[..., Any] | None' = None, /, *, scope: 'Scope | str' = , name: 'str | None' = None, autouse: 'bool' = False) -> 'Callable[..., Any]' ``` Declare a dependency provider with test, module or session lifetime. diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 3bf1eca..d18a63c 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -99,6 +99,17 @@ rename leaves the output absent. A report-only failure after publication warns b validated output and successful exit status intact. See [safe migration](../guides/migration.md) for supported constructs, unittest's SHA-pinned wrapper model, rollback guarantees, and external-side-effect boundaries. +Human-readable output distinguishes an analyzed candidate, a validated candidate, a generated +candidate, a statically convertible subset, and a published conversion. Repeated diagnostics are +grouped by severity and code, with the first source location shown. Use `--report-json FILE` or +`--report-json -` when every individual line-addressed diagnostic is required. The JSON field +`converted_tests` counts source-to-target mappings built in memory; only `status: published` and +`published: true` mean that an output directory was created. + +`MIG006` warns that a candidate has only one schedulable module affinity unit despite a parallel +worker setting. It is emitted only for `--check` or publication after static analysis succeeds, +because dry-run and unsupported transactions never execute the parallel gate. + ## Examples ```console diff --git a/docs/roadmap.md b/docs/roadmap.md index 5a85cb6..c480d6c 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -14,17 +14,29 @@ with source fingerprints, disposable validation copies, serial/parallel parity, and atomic create-only publication. -## 0.2 — fast feedback +## 0.2 — real-world pytest migration + +- Dependency-free native `tmp_path` and reversible `monkeypatch` fixtures for common + `setattr`/`setenv` usage. +- Static autouse fixtures with native setup and teardown ownership. +- Bare pytest-asyncio coroutine markers translated to isolated fresh-loop wrappers that preserve + the plugin's default function-scoped loop lifecycle. +- Fresh-instance wrappers for simple pytest classes, while complex lifecycle and inheritance stay + fail-closed. +- Compact migration diagnostics on the console with complete line detail retained in JSON. +- Differential validation against a 118-test real-world suite before publication. + +## 0.3 — fast feedback - Dynamic micro-shards and work stealing. - `--last-failed`, watch mode, and failure fingerprints. - Test-impact analysis in shadow mode with an explanation for every selection decision. - Stable assertion-diff protocol and improved plain-assert diagnostics. -## 0.3 — adoption +## 0.4 — adoption - Pytest hook adapter translating collection and outcomes into Testenix events and `RunResult`. -- Expand migration beyond the v0.1 static subset only when new transformations have differential +- Expand migration beyond the v0.2 static subset only when new transformations have differential semantics tests on real projects. - Versioned reporter and selector plugin interfaces. - IDE protocol and machine-readable collection manifest. diff --git a/llms-full.txt b/llms-full.txt index 0201cc0..00276b5 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -404,17 +404,21 @@ native `--workers` option into an xdist option. | Capability | `testenix pytest` | `migrate` then `run` | Direct `testenix run` | | --- | --- | --- | --- | | Plain module-level pytest-default `test*` functions | Yes, through pytest | Yes | `test_` or native `@test` | -| Pytest classes | Yes | Blocked | No | -| Simple local fixture | Yes | Converted | Only native `@fixture` | +| Simple pytest classes | Yes | Fresh-instance wrappers | No direct class collection | +| Complex class lifecycle/inheritance | Yes | Blocked | No | +| Simple local fixture | Yes | Converted, including static autouse | Only native `@fixture` | | Session-scoped pytest fixture | Yes | Blocked: run-global vs worker-local | Native scope is worker-local | -| Built-in/dynamic fixture | Yes | Blocked | No pytest fixture semantics | +| `tmp_path` | Yes | Native built-in | Native built-in | +| `monkeypatch` | Yes | `setattr`/`setenv` subset | Native reversible subset | +| Other built-in/dynamic fixture | Yes | Blocked | No pytest fixture semantics | | Adjacent `conftest.py` fixture | Yes | Simple static subset | No automatic conftest discovery | | Static single `parametrize` | Yes | Converted to cases | Use `@case` or `@cases` | | Skip and plain custom marker | Yes | Converted | Use Testenix decorators/tags | | Pytest xfail | Yes | Blocked due semantic differences | Use native `@xfail` intentionally | | Pytest assertion rewriting | Yes | No | No | | Plugins, hooks, pytest config | Yes | Blocked/not translated | No | -| Async plugin semantics | According to plugins | Blocked | Native async needs no plugin | +| Bare `@pytest.mark.asyncio` coroutine | According to plugin | Fresh function-scoped loop wrapper | Native async needs no plugin | +| Configured async/anyio plugin semantics | According to plugins | Blocked | No plugin semantics | | Testenix worker scheduler/history | No | Yes after migration | Yes | | Testenix retries and lossless results | No | Yes after migration | Yes | | Published native speedups | No | Measure generated suite | Only documented workloads | @@ -469,9 +473,12 @@ explicit equivalents: 5. keep unsupported modules on `testenix pytest` and keep all originals during rollout; 6. benchmark the generated directory before making a native performance claim. -The migrator replaces simple fixtures, parametrization, skip conditions, and plain markers in the -generated copy. It never performs edits in place. Manual rewrites are still necessary for blocked -plugin, hook, built-in fixture, class, xfail, and dynamic behavior. +The migrator replaces simple fixtures, parametrization, skip conditions, plain markers, bare +pytest-asyncio coroutine markers with a fresh closed loop per test or case, and the supported +built-in fixtures in the generated copy. It wraps simple pytest class methods with a fresh instance +per test. It never performs edits in place. +Manual rewrites are still necessary for blocked plugin, hook, complex class lifecycle, xfail, and +dynamic behavior. The current bridge does not convert pytest outcomes into a Testenix `RunResult`. Deeper event and report aggregation is a future compatibility layer and requires explicit pytest hook integration. @@ -526,10 +533,12 @@ They must be separate modules; a file that mixes both test models is rejected. The destination defaults to `testenix_migrated`. It must be a new directory inside the project, with an existing real parent. There is deliberately no `--force` and no in-place mode. An integer `--workers` value must be at least 2 so the parallel gate cannot silently repeat the -serial command. A one-module suite still has one schedulable affinity unit, which is disclosed as a -`MIG006` warning; spread tests across independent modules to exercise multiple workers. An -audit-report path must also be new, inside the project, and disjoint from every selected source and -the output suite. Testenix never replaces an existing report. +serial command. During `--check` or publication, a one-module candidate still has one schedulable +affinity unit, which is disclosed as a `MIG006` warning; spread tests across independent modules to +exercise multiple workers. The warning is not shown for `--dry-run` or an already-blocked migration +because no parallel candidate is run in either case. An audit-report path must also be new, inside +the project, and disjoint from every selected source and the output suite. Testenix never replaces +an existing report. ## Transaction and rollback contract @@ -574,13 +583,25 @@ with external effects. ## Pytest conversion contract -The current converter supports the subset below: +The v0.2 converter supports the subset below: - module-level pytest-default `test*` functions and normal Python `assert` statements; +- simple `Test*` classes with a fresh zero-argument instance per test method, including ordinary + helper methods; inheritance, class decorators, custom construction, and pytest class lifecycle + hooks remain outside the safe subset; +- bare `@pytest.mark.asyncio` on `async def` tests. The generated synchronous wrapper runs every + test or parametrized case in a fresh, closed `asyncio.Runner`, matching pytest-asyncio's default + function-scoped loop isolation; - one static `pytest.mark.parametrize` with static names, rows, IDs, and unmarked `pytest.param(..., id=...)` values; -- local fixtures with no parameters, autouse, or `request`, using function or module scope; +- local fixtures using function or module scope, including a statically boolean `autouse=True`; - simple fixtures from an adjacent `conftest.py` in the same directory; +- native `tmp_path`, which supplies a fresh `pathlib.Path` and removes its temporary directory at + test teardown; +- native `monkeypatch.setattr` in object/attribute and dotted-import forms, plus `setenv` and + idempotent `undo`; successful changes are restored in LIFO order during test teardown. + `monkeypatch` may also flow through statically resolved module-local helpers when every use can + be proven to stay inside this supported subset; - static `pytest.mark.skip` and `pytest.mark.skipif`; - plain argument-free custom markers, converted to Testenix tags; - pytest runtime helpers `approx`, `deprecated_call`, `fail`, `raises`, and `warns`. Generated @@ -588,14 +609,21 @@ The current converter supports the subset below: It blocks, with a file and line diagnostic: -- pytest test classes, xfail, runtime skip/xfail/importorskip/exit, xunit lifecycle hooks; -- built-in fixtures such as `tmp_path`, `monkeypatch`, `capsys`, and `request`; -- autouse or parametrized fixtures, fixture overrides, and inherited ancestor-`conftest` fixtures; +- complex pytest test classes, xfail, runtime skip/xfail/importorskip/exit, and xunit lifecycle + hooks; +- built-in fixtures other than `tmp_path` and `monkeypatch`, such as `capsys`, `caplog`, and + `request`; monkeypatch operations outside the documented native subset, imported or dynamically + rebound helpers, and values that escape static analysis are also unsupported; +- dynamically configured autouse fixtures, parametrized fixtures, fixture overrides, and inherited + ancestor-`conftest` fixtures; - session-scoped fixtures, because pytest creates one per run while Testenix session scope is currently worker-local; - stacked, dynamic, indirect, scoped, or per-case-marked parametrization; - `usefixtures`, module-level `pytestmark`, hook functions, plugin registration, and semantic - plugin markers such as asyncio/anyio, timeout, order, repeat, or flaky; + plugin markers such as anyio, configured asyncio, timeout, order, repeat, or flaky; +- unmarked async tests, async fixtures, custom `event_loop_policy`, non-function asyncio loop + scopes, and enabled asyncio debug mode. Testenix checks the effective pytest configuration and + relevant `PYTEST_ADDOPTS` overrides before accepting bare asyncio markers; - decorators and required parameters whose execution meaning cannot be established statically. Any converted pytest file whose name is not already `test_*.py` is renamed in the generated copy @@ -652,6 +680,11 @@ hashes, every source-to-target mapping, per-test outcomes, generated files, line timings and summaries for all validation runs, publication status, and an `originals_modified` flag. It is false for a successful transaction and true when a terminal source recheck detects drift; the flag reports observed state and does not claim that Testenix caused an independent edit. +The console groups repeated diagnostics by code and shows the first location, so large suites do +not produce hundreds of near-identical lines. `--report-json FILE` and `--report-json -` always +retain every individual source- and line-addressed diagnostic. On a blocked transaction, the +console calls any safe in-memory result a *statically convertible subset* rather than implying that +those tests were published. ## Performance with thousands of migrated tests @@ -945,6 +978,41 @@ def test_empty_repository(repository: Repository) -> None: The dependency graph is validated before execution. Missing fixtures and cycles become collection issues instead of hanging the run. +## Built-in fixtures + +Testenix 0.2 provides two dependency-free, test-scoped built-ins by name: + +```python +from pathlib import Path + + +def test_isolated_file(tmp_path: Path, monkeypatch) -> None: + target = tmp_path / "value.txt" + target.write_text("ok", encoding="utf-8") + monkeypatch.setenv("TESTENIX_EXAMPLE", "enabled") + assert target.read_text(encoding="utf-8") == "ok" +``` + +`tmp_path` is a fresh `pathlib.Path` removed during teardown. `monkeypatch` supports reversible +object/attribute and dotted-import `setattr`, `setenv`, and idempotent `undo`. Changes are restored +in LIFO order even when the test fails. Other pytest monkeypatch operations and pytest built-ins +such as `capsys`, `caplog`, and `request` are not native Testenix fixtures. + +## Autouse fixtures + +Use `autouse=True` when setup and cleanup must apply to every test that can see a fixture: + +```python +@fixture(autouse=True) +def isolated_environment(monkeypatch): + monkeypatch.setenv("APP_ENV", "test") + yield +``` + +Explicitly requesting the same fixture still resolves one cached value for the test. A local +fixture definition overrides a visible imported definition with the same name before Testenix +chooses which fixtures run automatically. + ## Scopes ```python @@ -958,7 +1026,7 @@ def worker_resource() -> Resource: return Resource() ``` -| Scope | Lifetime in Testenix 0.1 | +| Scope | Lifetime in Testenix 0.2 | | --- | --- | | `test` | One instance for one concrete test attempt. | | `module` | Shared by normal tests from the module inside one worker. | @@ -1276,6 +1344,17 @@ rename leaves the output absent. A report-only failure after publication warns b validated output and successful exit status intact. See [safe migration](https://polishdataengineer.github.io/testenix/guides/migration/) for supported constructs, unittest's SHA-pinned wrapper model, rollback guarantees, and external-side-effect boundaries. +Human-readable output distinguishes an analyzed candidate, a validated candidate, a generated +candidate, a statically convertible subset, and a published conversion. Repeated diagnostics are +grouped by severity and code, with the first source location shown. Use `--report-json FILE` or +`--report-json -` when every individual line-addressed diagnostic is required. The JSON field +`converted_tests` counts source-to-target mappings built in memory; only `status: published` and +`published: true` mean that an output directory was created. + +`MIG006` warns that a candidate has only one schedulable module affinity unit despite a parallel +worker setting. It is emitted only for `--check` or publication after static analysis succeeds, +because dry-run and unsupported transactions never execute the parallel gate. + ## Examples ```console @@ -2108,7 +2187,7 @@ Windows: testenix pytest =========> pytest.console_main -> collector/plugins/exe ``` The bridge is a CLI infrastructure adapter, not a native collection adapter. It does not emit -Testenix events or construct a `RunResult` in version 0.1. +Testenix events or construct a `RunResult` in version 0.2. The migration adapter is separate from that handoff. It statically converts a deliberately small pytest subset or generates SHA-pinned wrappers around the standard unittest protocol. Its @@ -2163,7 +2242,7 @@ Authoring API -> supervised collection -> inert manifest -> affinity scheduler - - migration analyzers depend on serializable migration contracts, while shadow execution and atomic publication remain application/infrastructure concerns. -## Version 0.1 scope +## Version 0.2 scope - explicit `@test` and `@fixture` authoring API, plus conventional `test_*` discovery; - sync functions, coroutines, generators, and async-generator fixture teardown; @@ -2177,9 +2256,12 @@ Authoring API -> supervised collection -> inert manifest -> affinity scheduler - - an optional platform-aware pytest handoff for unchanged legacy suites. - conservative pytest/unittest migration with static diagnostics, differential validation, source fingerprints, and create-only publication. +- dependency-free `tmp_path` and reversible `monkeypatch` fixtures, plus native autouse resolution; +- conservative migration of bare pytest-asyncio coroutine markers through isolated fresh-loop + wrappers, plus simple pytest classes. Remote workers, distributed storage, result caching, automatic quarantine, and a stable third-party -plugin SDK are deliberately outside version 0.1. +plugin SDK are deliberately outside version 0.2. ## Fixture scopes and process isolation @@ -2188,7 +2270,7 @@ between parallel shared workers. Multiple modules assigned to one shard execute process and fixture runtime. A test with an explicit timeout (including a global timeout applied at selection) is instead a single-test isolation unit with a hard process deadline. -Scope therefore has the following concrete meaning in version 0.1: +Scope therefore has the following concrete meaning in version 0.2: | Scope | Lifetime | | --- | --- | @@ -2249,17 +2331,29 @@ Source: docs/roadmap.md with source fingerprints, disposable validation copies, serial/parallel parity, and atomic create-only publication. -## 0.2 — fast feedback +## 0.2 — real-world pytest migration + +- Dependency-free native `tmp_path` and reversible `monkeypatch` fixtures for common + `setattr`/`setenv` usage. +- Static autouse fixtures with native setup and teardown ownership. +- Bare pytest-asyncio coroutine markers translated to isolated fresh-loop wrappers that preserve + the plugin's default function-scoped loop lifecycle. +- Fresh-instance wrappers for simple pytest classes, while complex lifecycle and inheritance stay + fail-closed. +- Compact migration diagnostics on the console with complete line detail retained in JSON. +- Differential validation against a 118-test real-world suite before publication. + +## 0.3 — fast feedback - Dynamic micro-shards and work stealing. - `--last-failed`, watch mode, and failure fingerprints. - Test-impact analysis in shadow mode with an explanation for every selection decision. - Stable assertion-diff protocol and improved plain-assert diagnostics. -## 0.3 — adoption +## 0.4 — adoption - Pytest hook adapter translating collection and outcomes into Testenix events and `RunResult`. -- Expand migration beyond the v0.1 static subset only when new transformations have differential +- Expand migration beyond the v0.2 static subset only when new transformations have differential semantics tests on real projects. - Versioned reporter and selector plugin interfaces. - IDE protocol and machine-readable collection manifest. @@ -2288,6 +2382,8 @@ project intends to use Semantic Versioning once its public API reaches stability ## [Unreleased] +## [0.2.0] - 2026-07-20 + ### Added - `testenix pytest [PYTEST_ARGS ...]` compatibility bridge for unchanged pytest suites, preserving @@ -2311,6 +2407,25 @@ project intends to use Semantic Versioning once its public API reaches stability - Truthful post-commit durability/report warnings, package-aware unittest outcome mapping, Testenix validation-worker containment, and conservative blocking of pytest session fixtures and unittest class-cleanup hooks whose lifecycle cannot be preserved. +- Native `tmp_path` and transactional `monkeypatch` fixtures. The initial monkeypatch contract + covers the object/attribute and dotted-import forms of `setattr`, plus `setenv`, with automatic + per-test rollback. Static module-local helper calls are accepted only when every propagated use + can be proven safe; aliases, dynamic rebinding, unsupported methods, and escaped values remain + blocked. +- Safe conversion of bare `@pytest.mark.asyncio` coroutine tests, simple pytest classes through + fresh-instance wrappers, and statically declared autouse fixtures. Async migration creates and + closes an isolated `asyncio.Runner` per test or case, validates effective pytest-asyncio loop and + debug configuration, and blocks custom event-loop policies or unmarked async semantics. +- Fail-closed class conversion for lifecycle hooks, decorated or inherited classes, annotated + class state, custom constructors, and method defaults that cannot be preserved by wrappers. + +### Changed + +- Migration console output now distinguishes analyzed, validated, generated, and published + candidates. Repeated diagnostics are grouped by code, while JSON audit reports retain every + source- and line-addressed entry. +- The one-affinity-unit `MIG006` warning is emitted only for statically supported check/publication + candidates, not for dry-run or already-blocked migrations. ## [0.1.0] - 2026-07-20 @@ -2648,7 +2763,7 @@ Inside a module, ``test_*`` functions and functions decorated with ## `testenix.fixture` ```text -fixture(function: 'Callable[..., Any] | None' = None, /, *, scope: 'Scope | str' = , name: 'str | None' = None) -> 'Callable[..., Any]' +fixture(function: 'Callable[..., Any] | None' = None, /, *, scope: 'Scope | str' = , name: 'str | None' = None, autouse: 'bool' = False) -> 'Callable[..., Any]' ``` Declare a dependency provider with test, module or session lifetime. diff --git a/pyproject.toml b/pyproject.toml index 9fc868c..32ff577 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "uv_build" [project] name = "testenix" -version = "0.1.0" +version = "0.2.0" description = "Fast, async-native, parallel-first testing for Python." readme = "README.md" requires-python = ">=3.11" diff --git a/src/testenix/__init__.py b/src/testenix/__init__.py index 442d1ae..69b5194 100644 --- a/src/testenix/__init__.py +++ b/src/testenix/__init__.py @@ -45,4 +45,4 @@ try: __version__ = version("testenix") except PackageNotFoundError: # Source checkout without installed metadata. - __version__ = "0.1.0" + __version__ = "0.2.0" diff --git a/src/testenix/api.py b/src/testenix/api.py index 0f54e86..1d62e39 100644 --- a/src/testenix/api.py +++ b/src/testenix/api.py @@ -41,6 +41,7 @@ class FixtureMetadata: scope: Scope = Scope.TEST name: str | None = None + autouse: bool = False @dataclass(frozen=True, slots=True) @@ -138,6 +139,7 @@ def fixture( *, scope: Scope | str = Scope.TEST, name: str | None = None, + autouse: bool = False, ) -> Callable[[Callable[P, R]], Callable[P, R]]: ... @@ -147,6 +149,7 @@ def fixture( *, scope: Scope | str = Scope.TEST, name: str | None = None, + autouse: bool = False, ) -> Callable[..., Any]: """Declare a dependency provider with test, module or session lifetime.""" @@ -157,7 +160,9 @@ def fixture( raise ValueError(f"unknown fixture scope {scope!r}; expected one of: {choices}") from error if name is not None and (not isinstance(name, str) or not name.strip()): raise ValueError("fixture name must be a non-empty string") - metadata = FixtureMetadata(scope=normalised_scope, name=name) + if not isinstance(autouse, bool): + raise TypeError("fixture autouse must be a boolean") + metadata = FixtureMetadata(scope=normalised_scope, name=name, autouse=autouse) def decorate(target: F) -> F: if not callable(target): diff --git a/src/testenix/builtin_fixtures.py b/src/testenix/builtin_fixtures.py new file mode 100644 index 0000000..9ab89fe --- /dev/null +++ b/src/testenix/builtin_fixtures.py @@ -0,0 +1,167 @@ +"""Dependency-free built-in fixtures for the native Testenix runtime.""" + +from __future__ import annotations + +import importlib +import inspect +import os +import tempfile +from collections.abc import Iterator +from dataclasses import dataclass +from pathlib import Path + + +class _NotSet: + __slots__ = () + + +_NOT_SET = _NotSet() + + +@dataclass(frozen=True, slots=True) +class _AttributeUndo: + target: object + name: str + previous: object + + +@dataclass(frozen=True, slots=True) +class _EnvironmentUndo: + name: str + previous: object + + +_UndoAction = _AttributeUndo | _EnvironmentUndo + + +def _resolve_dotted_target(import_path: str) -> tuple[object, str]: + """Resolve ``package.module.owner.attribute`` without masking import failures.""" + + parts = import_path.split(".") + if len(parts) < 2 or any(not part for part in parts): + raise TypeError("dotted monkeypatch targets must contain a module and attribute") + + for module_length in range(len(parts) - 1, 0, -1): + module_name = ".".join(parts[:module_length]) + try: + owner: object = importlib.import_module(module_name) + except ModuleNotFoundError as error: + # Trying ``package.module.Class`` as a module is expected to fail. + # A missing dependency raised *inside* an import must not be hidden. + if error.name is not None and ( + module_name == error.name or module_name.startswith(f"{error.name}.") + ): + continue + raise + for component in parts[module_length:-1]: + owner = getattr(owner, component) + return owner, parts[-1] + + raise ModuleNotFoundError(f"cannot import an owner for monkeypatch target {import_path!r}") + + +class MonkeyPatch: + """A small, dependency-free subset of pytest's reversible monkeypatch API. + + Testenix intentionally supports only attribute replacement and environment + variables. Every successful fixture use calls :meth:`undo` during teardown, + and manual calls to ``undo()`` are idempotent. + """ + + def __init__(self) -> None: + self._undo_actions: list[_UndoAction] = [] + + def setattr( + self, + target: object | str, + name: object, + value: object = _NOT_SET, + raising: bool = True, + ) -> None: + """Set an object attribute or a dotted import path and remember its old value.""" + + if not isinstance(raising, bool): + raise TypeError("raising must be a boolean") + if value is _NOT_SET: + if not isinstance(target, str): + raise TypeError("two-argument MonkeyPatch.setattr() requires a dotted import path") + resolved_target, attribute_name = _resolve_dotted_target(target) + replacement = name + else: + if not isinstance(name, str): + raise TypeError("attribute name must be a string") + resolved_target = target + attribute_name = name + replacement = value + + previous = getattr(resolved_target, attribute_name, _NOT_SET) + if previous is _NOT_SET and raising: + raise AttributeError(f"{resolved_target!r} has no attribute {attribute_name!r}") + # Avoid binding descriptors while saving class attributes. This is the + # value that must be restored to reproduce the original class body. + if inspect.isclass(resolved_target): + previous = vars(resolved_target).get(attribute_name, _NOT_SET) + + setattr(resolved_target, attribute_name, replacement) + action = _AttributeUndo(resolved_target, attribute_name, previous) + self._undo_actions.append(action) + + def setenv(self, name: str, value: object, prepend: str | None = None) -> None: + """Set an environment variable and restore its previous value at teardown.""" + + if not isinstance(name, str): + raise TypeError("environment variable name must be a string") + if prepend is not None and not isinstance(prepend, str): + raise TypeError("prepend must be a string or None") + rendered = str(value) + if prepend is not None and name in os.environ: + rendered = f"{rendered}{prepend}{os.environ[name]}" + previous: object = os.environ.get(name, _NOT_SET) + os.environ[name] = rendered + self._undo_actions.append(_EnvironmentUndo(name, previous)) + + def undo(self) -> None: + """Rollback all recorded changes in LIFO order, attempting every action.""" + + actions = self._undo_actions + self._undo_actions = [] + failures: list[BaseException] = [] + for action in reversed(actions): + try: + if isinstance(action, _AttributeUndo): + if action.previous is _NOT_SET: + delattr(action.target, action.name) + else: + setattr(action.target, action.name, action.previous) + elif action.previous is _NOT_SET: + os.environ.pop(action.name, None) + else: + os.environ[action.name] = str(action.previous) + except BaseException as error: + failures.append(error) + + if len(failures) == 1: + raise failures[0] + if failures: + raise BaseExceptionGroup("multiple monkeypatch rollback actions failed", failures) + + +def _tmp_path_fixture() -> Iterator[Path]: + temporary = tempfile.TemporaryDirectory(prefix="testenix-") + try: + yield Path(temporary.name) + finally: + # TemporaryDirectory has platform-specific handling for read-only paths + # while still surfacing open-handle cleanup failures on Windows. + temporary.cleanup() + + +def _monkeypatch_fixture() -> Iterator[MonkeyPatch]: + patcher = MonkeyPatch() + try: + yield patcher + finally: + patcher.undo() + + +__all__ = ["MonkeyPatch"] diff --git a/src/testenix/discovery.py b/src/testenix/discovery.py index 9a8d7d2..cb7b46a 100644 --- a/src/testenix/discovery.py +++ b/src/testenix/discovery.py @@ -280,7 +280,9 @@ def _fixture_dependencies( module_name=module_name, function=function, ) - if definition is not None: + # Built-ins are runtime fallbacks rather than decorated source + # functions, so there is no provider closure to register for them. + if definition is not None and not definition.builtin: dependencies.append(definition) return tuple(dependencies) @@ -338,12 +340,16 @@ def _implicit_case_id(parameters: dict[str, Any], ordinal: int) -> str: def _source_line(function: Any) -> int | None: - code = getattr(function, "__code__", None) + try: + source_function = inspect.unwrap(function) + except ValueError: + source_function = function + code = getattr(source_function, "__code__", None) first_line = getattr(code, "co_firstlineno", None) if isinstance(first_line, int): return first_line try: - return inspect.getsourcelines(function)[1] + return inspect.getsourcelines(source_function)[1] except (OSError, TypeError): return None diff --git a/src/testenix/fixtures.py b/src/testenix/fixtures.py index a075749..7748f0e 100644 --- a/src/testenix/fixtures.py +++ b/src/testenix/fixtures.py @@ -21,6 +21,7 @@ from typing import Annotated, Any, Union, get_args, get_origin, get_type_hints from testenix.api import get_fixture_metadata +from testenix.builtin_fixtures import _monkeypatch_fixture, _tmp_path_fixture from testenix.contracts import Scope @@ -138,6 +139,8 @@ class FixtureDefinition: module_name: str | None = None path: str | None = None provided_type: Any = None + autouse: bool = False + builtin: bool = False @property def key(self) -> str: @@ -149,6 +152,22 @@ def key(self) -> str: return f"{owner}::{self.name}" +_BUILTIN_DEFINITIONS = { + "monkeypatch": FixtureDefinition( + name="monkeypatch", + function=_monkeypatch_fixture, + scope=Scope.TEST, + builtin=True, + ), + "tmp_path": FixtureDefinition( + name="tmp_path", + function=_tmp_path_fixture, + scope=Scope.TEST, + builtin=True, + ), +} + + @dataclass(frozen=True, slots=True) class TeardownFailure: """A single losslessly captured fixture finalization error.""" @@ -217,6 +236,7 @@ def register( module_name=module_name, path=str(path) if path is not None else None, provided_type=_provided_type(function), + autouse=metadata.autouse, ) self.add(definition) return definition @@ -246,6 +266,14 @@ def find_for_parameter( local = [definition for definition in named if definition.module_name == module_name] return local[0] if local else named[0] + # Native built-ins are name-only fallbacks. They deliberately do not + # participate in type lookup (for example, an arbitrary ``Path`` + # parameter must not unexpectedly receive ``tmp_path``), and any user + # fixture with the same visible name wins above. + builtin = _BUILTIN_DEFINITIONS.get(parameter.name) + if builtin is not None: + return builtin + annotation = parameter.annotation if function is not None: annotation = _safe_type_hints(function).get(parameter.name, annotation) @@ -263,6 +291,31 @@ def find_for_parameter( ) return candidates[0] if candidates else None + def autouse_for(self, module_name: str | None) -> tuple[FixtureDefinition, ...]: + """Return effective implicit fixtures for a module in setup order. + + A module-local definition replaces a global definition with the same + name, even when the replacement is not autouse. This keeps normal + fixture override rules intact instead of running both providers. + """ + + effective: dict[str, FixtureDefinition] = {} + for definition in self._definitions: + if definition.module_name is None: + effective.setdefault(definition.name, definition) + for definition in self._definitions: + if definition.module_name == module_name: + effective[definition.name] = definition + return tuple( + sorted( + (definition for definition in effective.values() if definition.autouse), + key=lambda definition: ( + -_SCOPE_RANK[definition.scope], + definition.key, + ), + ) + ) + class FixtureRuntime: """Own fixture caches and finalizers for one execution session. @@ -323,6 +376,9 @@ async def resolve_arguments( names = ", ".join(sorted(unexpected)) raise FixtureError(f"unexpected case parameters for {function.__name__}: {names}") + for autouse_definition in self.registry.autouse_for(effective_module): + await self._resolve(autouse_definition, parent=None) + for parameter in signature.parameters.values(): if parameter.name in kwargs or parameter.kind in ( inspect.Parameter.VAR_POSITIONAL, diff --git a/src/testenix/migration_pytest.py b/src/testenix/migration_pytest.py index 1b54744..603f518 100644 --- a/src/testenix/migration_pytest.py +++ b/src/testenix/migration_pytest.py @@ -13,7 +13,9 @@ from __future__ import annotations import ast +import copy import hashlib +import re from collections import Counter from collections.abc import Iterable, Sequence from dataclasses import dataclass, field @@ -40,6 +42,7 @@ "capsys", "capsysbinary", "doctest_namespace", + "event_loop_policy", "monkeypatch", "pytestconfig", "record_property", @@ -54,6 +57,10 @@ } ) +_SUPPORTED_BUILTIN_FIXTURES = frozenset({"monkeypatch", "tmp_path"}) + +_SUPPORTED_MONKEYPATCH_METHODS = frozenset({"setattr", "setenv", "undo"}) + _ALLOWED_RUNTIME_HELPERS = frozenset( { "pytest.approx", @@ -101,6 +108,10 @@ "test": "_testenix_test", } +_MIGRATION_RUNTIME_ALIASES = { + "isolated_pytest_asyncio": "_testenix_isolated_asyncio", +} + @dataclass(frozen=True, slots=True) class _Aliases: @@ -143,6 +154,7 @@ class _Fixture: effective_name: str node: _Function dependencies: tuple[str, ...] + autouse: bool = False @dataclass(slots=True) @@ -152,8 +164,11 @@ class _Module: aliases: _Aliases diagnostics: list[MigrationDiagnostic] = field(default_factory=list) fixtures: dict[str, _Fixture] = field(default_factory=dict) + test_classes: list[ast.ClassDef] = field(default_factory=list) test_mappings: list[TestMapping] = field(default_factory=list) imports: set[str] = field(default_factory=set) + migration_runtime_imports: set[str] = field(default_factory=set) + uses_bare_asyncio: bool = False @property def source_name(self) -> str: @@ -331,6 +346,7 @@ def convert_pytest_suite( if module.tree is None or module.blocked or not module.fixtures: continue _insert_testenix_import(module.tree, module.imports) + _insert_migration_runtime_import(module.tree, module.migration_runtime_imports) _remove_unused_pytest_imports(module.tree) helper_path = module.source.migration_relative.with_name( f"{helper_names[module.source.project_relative]}.py" @@ -358,6 +374,7 @@ def convert_pytest_suite( if module.tree is None or module.blocked: continue _insert_testenix_import(module.tree, module.imports) + _insert_migration_runtime_import(module.tree, module.migration_runtime_imports) _remove_unused_pytest_imports(module.tree) content = _render(module.tree, module.source_name) if content is None: @@ -490,11 +507,117 @@ def _inspect_module(module: _Module, *, is_conftest: bool) -> None: statement, ) elif isinstance(statement, ast.ClassDef) and _is_test_class(statement): + if _inspect_test_class(module, statement): + module.test_classes.append(statement) + + +def _inspect_test_class(module: _Module, class_node: ast.ClassDef) -> bool: + """Accept only classes whose pytest lifecycle reduces to ``object()`` per test. + + Generated wrappers instantiate the retained class once per native test item. Anything + which can alter construction, inheritance, or pytest's class/method lifecycle is therefore + rejected before a wrapper is emitted. + """ + + safe = True + if class_node.bases or class_node.keywords: + module.error( + "PYT311_CLASS_INHERITANCE", + f"pytest class {class_node.name!r} uses inheritance or a metaclass", + class_node, + ) + safe = False + if class_node.decorator_list: + module.error( + "PYT312_CLASS_DECORATOR", + f"pytest class {class_node.name!r} has a class decorator", + class_node.decorator_list[0], + ) + safe = False + + lifecycle_names = { + "__init__", + "__new__", + "setup_class", + "setup_method", + "teardown_class", + "teardown_method", + } + for member in class_node.body: + bindings = _class_scope_bound_names(member) + if "pytestmark" in bindings: module.error( - "PYT301_CLASS_TEST", - f"pytest class {statement.name!r} requires instance and lifecycle semantics", - statement, + "PYT313_CLASS_MARK", + f"pytest class {class_node.name!r} declares class-level pytestmark", + member, + ) + safe = False + lifecycle_bindings = sorted(bindings.intersection(lifecycle_names)) + if lifecycle_bindings: + module.error( + "PYT314_CLASS_LIFECYCLE", + "pytest class lifecycle binding(s) are not supported: " + + ", ".join(lifecycle_bindings), + member, + ) + safe = False + if not isinstance(member, (ast.FunctionDef, ast.AsyncFunctionDef)): + continue + fixture_decorator = next( + ( + decorator + for decorator in member.decorator_list + if _decorator_canonical(decorator, module.aliases) + in {"pytest.fixture", "pytest_asyncio.fixture"} + ), + None, + ) + if fixture_decorator is not None: + module.error( + "PYT315_CLASS_FIXTURE", + f"fixture method {member.name!r} is not supported inside pytest classes", + fixture_decorator, + ) + safe = False + if not member.name.startswith("test"): + continue + positional = (*member.args.posonlyargs, *member.args.args) + if ( + member.args.posonlyargs + or not positional + or positional[0].arg != "self" + or member.args.vararg is not None + or member.args.kwarg is not None + or bool(member.args.defaults) + or any(default is not None for default in member.args.kw_defaults) + or bool(getattr(member, "type_params", ())) + ): + module.error( + "PYT316_CLASS_SIGNATURE", + f"pytest method {class_node.name}.{member.name} needs a plain self signature " + "without defaults", + member, ) + safe = False + semantic_decorator = next( + ( + decorator + for decorator in member.decorator_list + if _decorator_canonical(decorator, module.aliases) + in {"builtins.classmethod", "builtins.staticmethod", "classmethod", "staticmethod"} + or isinstance(decorator, ast.Name) + and decorator.id in {"classmethod", "staticmethod"} + ), + None, + ) + if semantic_decorator is not None: + module.error( + "PYT316_CLASS_SIGNATURE", + f"pytest method {class_node.name}.{member.name} cannot be static or class-bound", + semantic_decorator, + ) + safe = False + return safe def _convert_fixtures(module: _Module) -> None: @@ -541,7 +664,7 @@ def _convert_fixtures(module: _Module) -> None: ) continue - replacement, effective_name = _convert_fixture_decorator( + replacement, effective_name, autouse = _convert_fixture_decorator( module, fixture_decorator, statement.name ) if replacement is None or effective_name is None: @@ -549,7 +672,13 @@ def _convert_fixtures(module: _Module) -> None: statement.decorator_list = [replacement] module.imports.add("fixture") dependencies = _required_parameters(statement) - fixture = _Fixture(statement.name, effective_name, statement, dependencies) + fixture = _Fixture( + statement.name, + effective_name, + statement, + dependencies, + autouse=autouse, + ) converted.append((statement, fixture)) names = Counter(fixture.effective_name for _, fixture in converted) @@ -580,34 +709,39 @@ def _convert_fixtures(module: _Module) -> None: def _convert_fixture_decorator( module: _Module, decorator: ast.expr, function_name: str -) -> tuple[ast.expr | None, str | None]: +) -> tuple[ast.expr | None, str | None, bool]: if not isinstance(decorator, ast.Call): - return _native_name("fixture"), function_name + return _native_name("fixture"), function_name, False if decorator.args: module.error( "PYT201_FIXTURE_ARGUMENTS", "pytest fixture decorator positional arguments are not supported", decorator, ) - return None, None + return None, None, False keywords = _keyword_map(module, decorator, "fixture") if keywords is None: - return None, None + return None, None, False if "params" in keywords or "ids" in keywords: module.error( "PYT202_FIXTURE_PARAMS", f"parametrized fixture {function_name!r} has no native MVP equivalent", decorator, ) - return None, None - if "autouse" in keywords and not _is_false(keywords["autouse"]): - module.error( - "PYT203_FIXTURE_AUTOUSE", - f"autouse fixture {function_name!r} cannot be made implicit by Testenix", - keywords["autouse"], - ) - return None, None + return None, None, False + + autouse = False + if "autouse" in keywords: + if _is_true(keywords["autouse"]): + autouse = True + elif not _is_false(keywords["autouse"]): + module.error( + "PYT203_FIXTURE_AUTOUSE", + f"autouse for fixture {function_name!r} must be the static boolean True or False", + keywords["autouse"], + ) + return None, None, False allowed = {"autouse", "name", "scope"} unknown = sorted(set(keywords) - allowed) @@ -617,10 +751,12 @@ def _convert_fixture_decorator( "unsupported fixture option(s): " + ", ".join(unknown), decorator, ) - return None, None + return None, None, False effective_name = function_name output_keywords: list[ast.keyword] = [] + if autouse: + output_keywords.append(ast.keyword(arg="autouse", value=ast.Constant(True))) if "name" in keywords: name = _literal_nonempty_string(keywords["name"]) if name is None: @@ -629,7 +765,7 @@ def _convert_fixture_decorator( "fixture name must be a static non-empty string", keywords["name"], ) - return None, None + return None, None, False effective_name = name output_keywords.append(ast.keyword(arg="name", value=ast.Constant(name))) @@ -648,11 +784,11 @@ def _convert_fixture_decorator( detail, keywords["scope"], ) - return None, None + return None, None, False output_keywords.append(ast.keyword(arg="scope", value=ast.Constant(scopes[scope]))) if not output_keywords: - return _native_name("fixture"), effective_name + return _native_name("fixture"), effective_name, False return ( ast.Call( func=_native_name("fixture"), @@ -660,103 +796,514 @@ def _convert_fixture_decorator( keywords=output_keywords, ), effective_name, + autouse, ) def _convert_tests(module: _Module, visible_fixtures: set[str]) -> None: assert module.tree is not None - for statement in module.tree.body: + fixture_functions = {fixture.function_name for fixture in module.fixtures.values()} + for statement in tuple(module.tree.body): if not isinstance(statement, (ast.FunctionDef, ast.AsyncFunctionDef)): continue - if not statement.name.startswith("test") or statement.name in { - fixture.function_name for fixture in module.fixtures.values() - }: + if not statement.name.startswith("test") or statement.name in fixture_functions: continue + asyncio_marker = _bare_asyncio_marker(statement, module.aliases) + module.uses_bare_asyncio |= asyncio_marker is not None + decorators, parameter_names, parameter_cases = _convert_test_decorators(module, statement) + statement.decorator_list = decorators + _validate_test_parameters( + module, + statement, + parameter_names=parameter_names, + visible_fixtures=visible_fixtures, + ) + _validate_supported_builtin_usage( + module, + statement, + parameter_names=parameter_names, + visible_fixtures=visible_fixtures, + ) + _validate_asyncio_event_loop_policy( + module, + visible_fixtures=visible_fixtures, + marker=asyncio_marker, + ) + _append_test_mappings( + module, + source_qualname=statement.name, + target_function=statement.name, + parameter_cases=parameter_cases, + ) - decorators: list[ast.expr] = [] - tags: set[str] = set() - parameter_names: tuple[str, ...] = () - parameter_cases: tuple[tuple[str, str], ...] = () - parametrize_count = 0 - - for decorator in statement.decorator_list: - converted = _convert_test_decorator(module, statement, decorator) - if converted is None: + generated_wrappers: list[_Function] = [] + bound_names = _top_level_bound_names(module.tree) + for class_node in module.test_classes: + for member in class_node.body: + if not isinstance(member, (ast.FunctionDef, ast.AsyncFunctionDef)): continue - if _decorator_canonical(decorator, module.aliases) == "pytest.mark.parametrize": - parametrize_count += 1 - if converted.node is not None: - decorators.append(converted.node) - module.imports.update(converted.imports) - tags.update(converted.tags) - if converted.parameter_names: - parameter_names = converted.parameter_names - parameter_cases = converted.cases - - if parametrize_count > 1: - module.error( - "PYT104_STACKED_PARAMETRIZE", - f"test {statement.name!r} has stacked parametrize decorators", - statement, + if not member.name.startswith("test"): + continue + asyncio_marker = _bare_asyncio_marker(member, module.aliases) + module.uses_bare_asyncio |= asyncio_marker is not None + decorators, parameter_names, parameter_cases = _convert_test_decorators(module, member) + # The retained class is an implementation detail. Native discovery sees only the + # generated module-level wrapper, so foreign runner decorators must not execute on + # the original method when the generated module is imported. + member.decorator_list = [] + _validate_test_parameters( + module, + member, + parameter_names=parameter_names, + visible_fixtures=visible_fixtures, + ignored_parameters=frozenset({"self"}), ) - - if tags: - decorators.insert( - 0, - ast.Call( - func=_native_name("test"), - args=[], - keywords=[ - ast.keyword( - arg="tags", - value=ast.Set(elts=[ast.Constant(tag) for tag in sorted(tags)]), - ) - ], - ), + _validate_supported_builtin_usage( + module, + member, + parameter_names=parameter_names, + visible_fixtures=visible_fixtures, + ignored_parameters=frozenset({"self"}), ) - module.imports.add("test") - elif not statement.name.startswith("test_"): - decorators.insert(0, _native_name("test")) - module.imports.add("test") - statement.decorator_list = decorators - - required = set(_required_parameters(statement)) - set(parameter_names) - for parameter in sorted(required): - if parameter in visible_fixtures: - continue - if parameter in _BUILTIN_FIXTURES: - module.error( - "PYT209_BUILTIN_FIXTURE", - f"pytest built-in fixture {parameter!r} has no native Testenix equivalent", - statement, - ) - else: + _validate_asyncio_event_loop_policy( + module, + visible_fixtures=visible_fixtures, + marker=asyncio_marker, + ) + source_qualname = f"{class_node.name}.{member.name}" + wrapper_name = _class_wrapper_name(module, class_node.name, member.name) + if wrapper_name in bound_names: module.error( - "PYT210_UNKNOWN_FIXTURE", - f"required parameter {parameter!r} is not a statically known fixture or case", - statement, + "PYT317_CLASS_WRAPPER_COLLISION", + f"generated wrapper name {wrapper_name!r} already exists in the module", + member, ) + continue + bound_names.add(wrapper_name) + wrapper = _class_test_wrapper( + class_node, + member, + wrapper_name=wrapper_name, + decorators=decorators, + ) + generated_wrappers.append(wrapper) + _append_test_mappings( + module, + source_qualname=source_qualname, + target_function=wrapper_name, + parameter_cases=parameter_cases, + ) + module.tree.body.extend(generated_wrappers) - target_file = _target_relative_path(module.source).as_posix() - source_base = f"{module.source_name}::{statement.name}" - if parameter_cases: - module.test_mappings.extend( - TestMapping( - source_id=f"{source_base}[{source_case}]", - target_file=target_file, - target_function=statement.name, - case_id=target_case, - ) - for source_case, target_case in parameter_cases + +def _bare_asyncio_marker(function: _Function, aliases: _Aliases) -> ast.expr | None: + return next( + ( + decorator + for decorator in function.decorator_list + if not isinstance(decorator, ast.Call) + and _decorator_canonical(decorator, aliases) == "pytest.mark.asyncio" + ), + None, + ) + + +def _validate_asyncio_event_loop_policy( + module: _Module, + *, + visible_fixtures: set[str], + marker: ast.expr | None, +) -> None: + if marker is None or "event_loop_policy" not in visible_fixtures: + return + module.error( + "PYT509_EVENT_LOOP_POLICY", + ( + "a custom event_loop_policy fixture implicitly changes pytest-asyncio loop " + "creation and cannot be reproduced by native migration" + ), + marker, + ) + + +def _convert_test_decorators( + module: _Module, + function: _Function, +) -> tuple[list[ast.expr], tuple[str, ...], tuple[tuple[str, str], ...]]: + decorators: list[ast.expr] = [] + tags: set[str] = set() + parameter_names: tuple[str, ...] = () + parameter_cases: tuple[tuple[str, str], ...] = () + parametrize_count = 0 + asyncio_count = 0 + async_plugin_count = 0 + + for decorator in function.decorator_list: + canonical = _decorator_canonical(decorator, module.aliases) + converted = _convert_test_decorator(module, function, decorator) + if canonical == "pytest.mark.parametrize": + parametrize_count += 1 + elif canonical == "pytest.mark.asyncio": + asyncio_count += 1 + async_plugin_count += 1 + elif canonical == "pytest.mark.anyio": + async_plugin_count += 1 + if converted is None: + continue + if converted.node is not None: + decorators.append(converted.node) + module.imports.update(converted.imports) + tags.update(converted.tags) + if converted.parameter_names: + parameter_names = converted.parameter_names + parameter_cases = converted.cases + + if parametrize_count > 1: + module.error( + "PYT104_STACKED_PARAMETRIZE", + f"test {function.name!r} has stacked parametrize decorators", + function, + ) + if asyncio_count > 1: + module.error( + "PYT507_DUPLICATE_ASYNCIO_MARKER", + f"test {function.name!r} has more than one asyncio marker", + function, + ) + if isinstance(function, ast.AsyncFunctionDef) and async_plugin_count == 0: + module.error( + "PYT508_UNMARKED_ASYNC_TEST", + ( + f"unmarked async test {function.name!r} depends on pytest asyncio_mode; " + "add a bare @pytest.mark.asyncio before migrating" + ), + function, + ) + + if tags: + decorators.insert( + 0, + ast.Call( + func=_native_name("test"), + args=[], + keywords=[ + ast.keyword( + arg="tags", + value=ast.Set(elts=[ast.Constant(tag) for tag in sorted(tags)]), + ) + ], + ), + ) + module.imports.add("test") + elif not function.name.startswith("test_"): + decorators.insert(0, _native_name("test")) + module.imports.add("test") + return decorators, parameter_names, parameter_cases + + +def _validate_test_parameters( + module: _Module, + function: _Function, + *, + parameter_names: tuple[str, ...], + visible_fixtures: set[str], + ignored_parameters: frozenset[str] = frozenset(), +) -> None: + required = set(_required_parameters(function)) - set(parameter_names) - ignored_parameters + for parameter in sorted(required): + if parameter in visible_fixtures or parameter in _SUPPORTED_BUILTIN_FIXTURES: + continue + if parameter in _BUILTIN_FIXTURES: + module.error( + "PYT209_BUILTIN_FIXTURE", + f"pytest built-in fixture {parameter!r} has no native Testenix equivalent", + function, ) else: - module.test_mappings.append( - TestMapping( - source_id=source_base, - target_file=target_file, - target_function=statement.name, - ) + module.error( + "PYT210_UNKNOWN_FIXTURE", + f"required parameter {parameter!r} is not a statically known fixture or case", + function, + ) + + +def _validate_supported_builtin_usage( + module: _Module, + function: _Function, + *, + parameter_names: tuple[str, ...] = (), + visible_fixtures: set[str], + ignored_parameters: frozenset[str] = frozenset(), +) -> None: + """Reject uses which escape the deliberately small native built-in contract. + + A fixture declared by the project under the name ``monkeypatch`` is ordinary user code and + must not be constrained here. The validation applies only when the parameter resolves to + Testenix's built-in compatibility fixture. + """ + + required = set(_required_parameters(function)) - set(parameter_names) - ignored_parameters + if "monkeypatch" not in required or "monkeypatch" in visible_fixtures: + return + _validate_builtin_monkeypatch_usage(module, function) + + +def _validate_builtin_monkeypatch_usage(module: _Module, function: _Function) -> None: + _validate_monkeypatch_binding( + module, + function, + parameter="monkeypatch", + helpers=_static_module_helpers(module), + visiting=set(), + validated=set(), + ) + + +def _validate_monkeypatch_binding( + module: _Module, + function: _Function, + *, + parameter: str, + helpers: dict[str, _Function], + visiting: set[tuple[int, str]], + validated: set[tuple[int, str]], +) -> None: + key = (id(function), parameter) + if key in validated or key in visiting: + return + visiting.add(key) + + parents: dict[int, ast.AST] = {} + body_nodes: list[ast.AST] = [] + for statement in function.body: + walked = tuple(ast.walk(statement)) + body_nodes.extend(walked) + for ancestor in walked: + for child in ast.iter_child_nodes(ancestor): + parents[id(child)] = ancestor + + for node in body_nodes: + if not isinstance(node, ast.Name) or node.id != parameter: + continue + parent = parents.get(id(node)) + attribute = parent if isinstance(parent, ast.Attribute) and parent.value is node else None + call = None if attribute is None else parents.get(id(attribute)) + if ( + attribute is not None + and isinstance(call, ast.Call) + and call.func is attribute + and attribute.attr in _SUPPORTED_MONKEYPATCH_METHODS + and isinstance(node.ctx, ast.Load) + ): + continue + + forwarded = _forwarded_monkeypatch_parameter( + function, + node, + parent=parent, + parents=parents, + helpers=helpers, + ) + if forwarded is not None and isinstance(node.ctx, ast.Load): + helper, helper_parameter = forwarded + _validate_monkeypatch_binding( + module, + helper, + parameter=helper_parameter, + helpers=helpers, + visiting=visiting, + validated=validated, + ) + continue + + if attribute is not None and isinstance(call, ast.Call) and call.func is attribute: + detail = f"method {attribute.attr!r} is not supported" + else: + detail = "the fixture object is read, rebound, passed, or aliased" + allowed = ", ".join(f"{name}()" for name in sorted(_SUPPORTED_MONKEYPATCH_METHODS)) + module.error( + "PYT214_MONKEYPATCH_USAGE", + f"built-in monkeypatch {detail}; only direct calls to {allowed} are migratable", + attribute or node, + ) + + visiting.remove(key) + validated.add(key) + + +def _static_module_helpers(module: _Module) -> dict[str, _Function]: + if module.tree is None: + return {} + if any( + isinstance(statement, ast.ImportFrom) + and any(alias.name == "*" for alias in statement.names) + for statement in module.tree.body + ): + return {} + + binding_counts: Counter[str] = Counter() + for statement in module.tree.body: + binding_counts.update(_class_scope_bound_names(statement)) + return { + statement.name: statement + for statement in module.tree.body + if isinstance(statement, (ast.FunctionDef, ast.AsyncFunctionDef)) + and not statement.decorator_list + and binding_counts[statement.name] == 1 + } + + +def _forwarded_monkeypatch_parameter( + caller: _Function, + argument: ast.Name, + *, + parent: ast.AST | None, + parents: dict[int, ast.AST], + helpers: dict[str, _Function], +) -> tuple[_Function, str] | None: + call: ast.Call | None + if isinstance(parent, ast.Call) and any(item is argument for item in parent.args): + call = parent + elif isinstance(parent, ast.keyword) and parent.value is argument: + possible_call = parents.get(id(parent)) + call = possible_call if isinstance(possible_call, ast.Call) else None + else: + return None + if call is None or not isinstance(call.func, ast.Name): + return None + if call.func.id in _function_local_bound_names(caller): + return None + helper = helpers.get(call.func.id) + if helper is None: + return None + target = _call_argument_parameter(call, argument, parent=parent, helper=helper) + return None if target is None else (helper, target) + + +def _call_argument_parameter( + call: ast.Call, + argument: ast.Name, + *, + parent: ast.AST | None, + helper: _Function, +) -> str | None: + positional_parameters = (*helper.args.posonlyargs, *helper.args.args) + if parent is call: + position = next( + (index for index, value in enumerate(call.args) if value is argument), + None, + ) + if position is None or any( + isinstance(value, ast.Starred) for value in call.args[: position + 1] + ): + return None + if position >= len(positional_parameters): + return None + return positional_parameters[position].arg + + if not isinstance(parent, ast.keyword) or parent.arg is None: + return None + keyword_parameters = { + argument_node.arg for argument_node in (*helper.args.args, *helper.args.kwonlyargs) + } + return parent.arg if parent.arg in keyword_parameters else None + + +def _function_local_bound_names(function: _Function) -> set[str]: + names = set(_all_parameter_names(function)) + if function.args.vararg is not None: + names.add(function.args.vararg.arg) + if function.args.kwarg is not None: + names.add(function.args.kwarg.arg) + for statement in function.body: + names.update(_class_scope_bound_names(statement)) + return names + + +def _append_test_mappings( + module: _Module, + *, + source_qualname: str, + target_function: str, + parameter_cases: tuple[tuple[str, str], ...], +) -> None: + target_file = _target_relative_path(module.source).as_posix() + source_base = f"{module.source_name}::{source_qualname}" + if parameter_cases: + module.test_mappings.extend( + TestMapping( + source_id=f"{source_base}[{source_case}]", + target_file=target_file, + target_function=target_function, + case_id=target_case, ) + for source_case, target_case in parameter_cases + ) + else: + module.test_mappings.append( + TestMapping( + source_id=source_base, + target_file=target_file, + target_function=target_function, + ) + ) + + +def _class_wrapper_name(module: _Module, class_name: str, method_name: str) -> str: + source_id = f"{module.source_name}::{class_name}.{method_name}" + readable = re.sub(r"[^0-9A-Za-z_]", "_", f"test_{class_name}__{method_name}")[:100] + digest = hashlib.sha256(source_id.encode()).hexdigest()[:10] + return f"{readable}__{digest}" + + +def _class_test_wrapper( + class_node: ast.ClassDef, + method: _Function, + *, + wrapper_name: str, + decorators: list[ast.expr], +) -> _Function: + arguments = copy.deepcopy(method.args) + # _inspect_test_class guarantees a conventional first ``self`` argument. + arguments.args = arguments.args[1:] + arguments.defaults = [] + arguments.kw_defaults = [None for _ in arguments.kwonlyargs] + for argument in (*arguments.posonlyargs, *arguments.args, *arguments.kwonlyargs): + argument.annotation = None + argument.type_comment = None + forwarded = [ + ast.keyword(arg=argument.arg, value=ast.Name(id=argument.arg, ctx=ast.Load())) + for argument in (*arguments.args, *arguments.kwonlyargs) + ] + method_call = ast.Call( + func=ast.Attribute( + value=ast.Call( + func=ast.Name(id=class_node.name, ctx=ast.Load()), + args=[], + keywords=[], + ), + attr=method.name, + ctx=ast.Load(), + ), + args=[], + keywords=forwarded, + ) + is_async = isinstance(method, ast.AsyncFunctionDef) + value: ast.expr = ast.Await(method_call) if is_async else method_call + wrapper_type = ast.AsyncFunctionDef if is_async else ast.FunctionDef + wrapper = wrapper_type( + name=wrapper_name, + args=arguments, + body=[ast.Return(value=value)], + decorator_list=decorators, + returns=None, + type_comment=None, + ) + if "type_params" in wrapper._fields: + # Python 3.12+ added this field. Generic pytest methods are rejected above, so the + # generated wrapper intentionally has no PEP 695 parameters of its own. + wrapper.type_params = [] # type: ignore[union-attr] + return ast.copy_location(wrapper, method) def _convert_test_decorator( @@ -790,10 +1337,27 @@ def _convert_test_decorator( decorator, ) return None - if canonical in {"pytest.mark.asyncio", "pytest.mark.anyio"}: + if canonical == "pytest.mark.asyncio": + if not isinstance(function, ast.AsyncFunctionDef): + module.error( + "PYT502_ASYNC_PLUGIN", + "bare pytest.mark.asyncio is supported only on async def tests", + decorator, + ) + return None + if isinstance(decorator, ast.Call): + module.error( + "PYT502_ASYNC_PLUGIN", + "called or configured pytest.mark.asyncio may change event-loop semantics", + decorator, + ) + return None + module.migration_runtime_imports.add("isolated_pytest_asyncio") + return _ConvertedDecorator(_migration_runtime_name("isolated_pytest_asyncio")) + if canonical == "pytest.mark.anyio": module.error( "PYT502_ASYNC_PLUGIN", - "pytest async plugin lifecycle semantics cannot be translated safely", + "pytest anyio backend and lifecycle semantics cannot be translated safely", decorator, ) return None @@ -1131,7 +1695,7 @@ def _parameter_ids(node: ast.expr | None, count: int) -> tuple[str | None, ...] def _validate_fixture_dependencies(module: _Module, *, available: set[str]) -> None: for fixture in module.fixtures.values(): for dependency in fixture.dependencies: - if dependency in available: + if dependency in available or dependency in _SUPPORTED_BUILTIN_FIXTURES: continue if dependency in _BUILTIN_FIXTURES: module.error( @@ -1145,6 +1709,8 @@ def _validate_fixture_dependencies(module: _Module, *, available: set[str]) -> N f"fixture {fixture.effective_name!r} depends on unknown fixture {dependency!r}", fixture.node, ) + if "monkeypatch" in fixture.dependencies and "monkeypatch" not in available: + _validate_builtin_monkeypatch_usage(module, fixture.node) def _validate_runtime_pytest_calls(module: _Module) -> None: @@ -1238,6 +1804,19 @@ def _diagnose_ancestor_fixture_use( for statement in module.tree.body: if isinstance(statement, (ast.FunctionDef, ast.AsyncFunctionDef)): requested.update(_required_parameters(statement)) + elif isinstance(statement, ast.ClassDef) and statement in module.test_classes: + for member in statement.body: + if isinstance(member, (ast.FunctionDef, ast.AsyncFunctionDef)): + requested.update(_required_parameters(member)) + requested.update( + fixture.effective_name + for ancestor in ancestors + for fixture in ancestor.fixtures.values() + if fixture.autouse + ) + if module.uses_bare_asyncio: + # pytest-asyncio requests this fixture implicitly for every marked test. + requested.add("event_loop_policy") inherited = { name for ancestor in ancestors @@ -1269,7 +1848,8 @@ def _ancestor_conftests( def _conftest_helper_name(source: SourceFile) -> str: digest = hashlib.sha1( - source.project_relative.as_posix().encode(), usedforsecurity=False + f"{source.project_relative.as_posix()}\0{source.sha256.lower()}".encode(), + usedforsecurity=False, ).hexdigest()[:12] return f"_testenix_conftest_{digest}" @@ -1358,6 +1938,22 @@ def _insert_testenix_import(tree: ast.Module, names: Iterable[str]) -> None: ) +def _insert_migration_runtime_import(tree: ast.Module, names: Iterable[str]) -> None: + materialized = tuple(sorted(set(names))) + if materialized: + _insert_statement( + tree, + ast.ImportFrom( + module="testenix.migration_runtime", + names=[ + ast.alias(name=name, asname=_MIGRATION_RUNTIME_ALIASES[name]) + for name in materialized + ], + level=0, + ), + ) + + def _remove_unused_pytest_imports(tree: ast.Module) -> None: """Remove only foreign-runner imports made dead by decorator conversion. @@ -1413,15 +2009,31 @@ def _native_name(name: str) -> ast.Name: return ast.Name(id=_NATIVE_ALIASES[name], ctx=ast.Load()) +def _migration_runtime_name(name: str) -> ast.Name: + return ast.Name(id=_MIGRATION_RUNTIME_ALIASES[name], ctx=ast.Load()) + + def _diagnose_native_import_collisions(module: _Module) -> None: if module.tree is None: return required_aliases = {_NATIVE_ALIASES[name] for name in module.imports} + required_aliases.update( + _MIGRATION_RUNTIME_ALIASES[name] for name in module.migration_runtime_imports + ) collisions = _top_level_bound_names(module.tree).intersection(required_aliases) if collisions: + collision_node = next( + ( + statement + for statement in module.tree.body + if _top_level_statement_bound_names(statement).intersection(collisions) + ), + None, + ) module.error( "PYT008_GENERATED_IMPORT_COLLISION", "source binds reserved generated import name(s): " + ", ".join(sorted(collisions)), + collision_node, ) @@ -1466,19 +2078,22 @@ def _render(tree: ast.Module, source_name: str) -> str | None: def _top_level_bound_names(tree: ast.Module) -> set[str]: - names: set[str] = set() - for statement in tree.body: - if isinstance(statement, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)): - names.add(statement.name) - elif isinstance(statement, (ast.Import, ast.ImportFrom)): - for alias in statement.names: - if alias.name != "*": - names.add(alias.asname or alias.name.split(".")[0]) - elif isinstance(statement, (ast.Assign, ast.AnnAssign)): - targets = statement.targets if isinstance(statement, ast.Assign) else [statement.target] - for target in targets: - names.update(_target_names(target)) - return names + return {name for statement in tree.body for name in _top_level_statement_bound_names(statement)} + + +def _top_level_statement_bound_names(statement: ast.stmt) -> set[str]: + if isinstance(statement, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)): + return {statement.name} + if isinstance(statement, (ast.Import, ast.ImportFrom)): + return { + alias.asname or alias.name.split(".")[0] + for alias in statement.names + if alias.name != "*" + } + if isinstance(statement, (ast.Assign, ast.AnnAssign)): + targets = statement.targets if isinstance(statement, ast.Assign) else [statement.target] + return {name for target in targets for name in _target_names(target)} + return set() def _target_names(target: ast.expr) -> set[str]: @@ -1489,6 +2104,73 @@ def _target_names(target: ast.expr) -> set[str]: return set() +class _ClassScopeBindingCollector(ast.NodeVisitor): + """Collect names bound in a class namespace without entering nested scopes.""" + + def __init__(self) -> None: + self.names: set[str] = set() + + def visit_Name(self, node: ast.Name) -> None: # noqa: N802 - ast visitor protocol + if isinstance(node.ctx, (ast.Store, ast.Del)): + self.names.add(node.id) + + def visit_FunctionDef( # noqa: N802 - ast visitor protocol + self, node: ast.FunctionDef + ) -> None: + self.names.add(node.name) + + def visit_AsyncFunctionDef( # noqa: N802 - ast visitor protocol + self, node: ast.AsyncFunctionDef + ) -> None: + self.names.add(node.name) + + def visit_ClassDef(self, node: ast.ClassDef) -> None: # noqa: N802 - ast visitor protocol + self.names.add(node.name) + + def visit_Lambda(self, node: ast.Lambda) -> None: # noqa: N802 - ast visitor protocol + return + + def visit_Import(self, node: ast.Import) -> None: # noqa: N802 - ast visitor protocol + self.names.update(alias.asname or alias.name.split(".")[0] for alias in node.names) + + def visit_ImportFrom( # noqa: N802 - ast visitor protocol + self, node: ast.ImportFrom + ) -> None: + self.names.update(alias.asname or alias.name for alias in node.names if alias.name != "*") + + def visit_ExceptHandler( # noqa: N802 - ast visitor protocol + self, node: ast.ExceptHandler + ) -> None: + if node.name is not None: + self.names.add(node.name) + for statement in node.body: + self.visit(statement) + + def visit_MatchAs(self, node: ast.MatchAs) -> None: # noqa: N802 - ast visitor protocol + if node.name is not None: + self.names.add(node.name) + if node.pattern is not None: + self.visit(node.pattern) + + def visit_MatchStar(self, node: ast.MatchStar) -> None: # noqa: N802 - ast visitor protocol + if node.name is not None: + self.names.add(node.name) + + def visit_MatchMapping( # noqa: N802 - ast visitor protocol + self, node: ast.MatchMapping + ) -> None: + if node.rest is not None: + self.names.add(node.rest) + for pattern in node.patterns: + self.visit(pattern) + + +def _class_scope_bound_names(statement: ast.stmt) -> set[str]: + collector = _ClassScopeBindingCollector() + collector.visit(statement) + return collector.names + + def _assigned_name(statement: ast.stmt) -> str | None: if ( isinstance(statement, ast.Assign) @@ -1530,6 +2212,10 @@ def _is_false(node: ast.expr) -> bool: return isinstance(node, ast.Constant) and node.value is False +def _is_true(node: ast.expr) -> bool: + return isinstance(node, ast.Constant) and node.value is True + + def _deduplicate_diagnostics( diagnostics: Iterable[MigrationDiagnostic], ) -> tuple[MigrationDiagnostic, ...]: diff --git a/src/testenix/migration_pytest_config.py b/src/testenix/migration_pytest_config.py new file mode 100644 index 0000000..1d9e74c --- /dev/null +++ b/src/testenix/migration_pytest_config.py @@ -0,0 +1,621 @@ +"""Static pytest-asyncio configuration guard for safe source migration. + +The converter can remove a bare ``@pytest.mark.asyncio`` marker only when the +source runner uses function-scoped, non-debug event loops. This module mirrors +pytest's configuration-file discovery without importing pytest or executing a +project plugin. Differential execution in disposable project shadows remains +the authoritative validation gate after this preflight analysis. +""" + +from __future__ import annotations + +import ast +import configparser +import importlib.metadata as importlib_metadata +import os +import re +import shlex +import tomllib +from collections.abc import Mapping, Sequence +from dataclasses import dataclass +from pathlib import Path + +from testenix.migration_models import MigrationDiagnostic, SourceFile + +_PYTEST_8_CONFIG_NAMES = ( + "pytest.ini", + ".pytest.ini", + "pyproject.toml", + "tox.ini", + "setup.cfg", +) +_PYTEST_9_CONFIG_NAMES = ( + "pytest.toml", + ".pytest.toml", + *_PYTEST_8_CONFIG_NAMES, +) +_VALID_LOOP_SCOPES = frozenset({"function", "class", "module", "package", "session"}) +_FALSE_VALUES = frozenset({"0", "f", "false", "n", "no", "off"}) +_TRUE_VALUES = frozenset({"1", "on", "t", "true", "y", "yes"}) +_ASYNCIO_SCOPE_OPTION = "asyncio_default_test_loop_scope" +_ASYNCIO_DEBUG_OPTION = "asyncio_debug" +_OPTIONS_WITH_SEPARATE_VALUE = frozenset( + { + "-k", + "-m", + "-n", + "-p", + "-r", + "-W", + "--assert", + "--asyncio-mode", + "--basetemp", + "--cache-show", + "--capture", + "--code-highlight", + "--color", + "--confcutdir", + "--debug", + "--deselect", + "--dist", + "--doctest-glob", + "--doctest-report", + "--durations", + "--durations-min", + "--ignore", + "--ignore-glob", + "--import-mode", + "--junit-prefix", + "--junit-xml", + "--junitxml", + "--log-cli-date-format", + "--log-cli-format", + "--log-cli-level", + "--log-date-format", + "--log-file", + "--log-file-date-format", + "--log-file-format", + "--log-file-level", + "--log-file-mode", + "--log-format", + "--log-level", + "--log-auto-indent", + "--log-disable", + "--maxfail", + "--maxprocesses", + "--max-warnings", + "--max-worker-restart", + "--numprocesses", + "--pastebin", + "--pdbcls", + "--pythonwarnings", + "--report-chars", + "--rootdir", + "--show-capture", + "--tb", + "--tx", + "--verbosity", + "--xmlpath", + } +) + + +@dataclass(frozen=True, slots=True) +class _ResolvedConfig: + path: Path + values: Mapping[str, object] + native_toml: bool = False + + +class _ConfigError(ValueError): + def __init__(self, path: Path | None, message: str) -> None: + super().__init__(message) + self.path = path + + +@dataclass(frozen=True, slots=True) +class _ArgumentOverrides: + values: Mapping[str, str] + label: str + path: Path | None + asyncio_debug_flag: bool = False + + +@dataclass(frozen=True, slots=True) +class _Aliases: + modules: Mapping[str, str] + symbols: Mapping[str, str] + + @classmethod + def from_tree(cls, tree: ast.Module) -> _Aliases: + modules: dict[str, str] = {} + symbols: dict[str, str] = {} + for statement in tree.body: + if isinstance(statement, ast.Import): + for imported in statement.names: + if imported.name == "pytest": + modules[imported.asname or imported.name] = imported.name + elif isinstance(statement, ast.ImportFrom) and statement.module == "pytest": + for imported in statement.names: + if imported.name != "*": + symbols[imported.asname or imported.name] = f"pytest.{imported.name}" + return cls(modules, symbols) + + def canonical(self, node: ast.AST) -> str | None: + if isinstance(node, ast.Name): + return self.symbols.get(node.id) or self.modules.get(node.id) + if isinstance(node, ast.Attribute): + owner = self.canonical(node.value) + return f"{owner}.{node.attr}" if owner is not None else None + return None + + +def pytest_asyncio_config_diagnostics( + *, + project_root: Path, + source_paths: Sequence[Path], + files: Sequence[SourceFile], + environ: Mapping[str, str] | None = None, + pytest_major: int | None = None, +) -> tuple[MigrationDiagnostic, ...]: + """Return blocking diagnostics for unsupported bare pytest-asyncio settings. + + Configuration is deliberately ignored for suites without a bare marker so + an unrelated project-level pytest option cannot block a synchronous or + unittest-only migration. + """ + + if not any(_contains_bare_asyncio_test(source) for source in files): + return () + + root = project_root.resolve() + environment = os.environ if environ is None else environ + try: + environment_overrides = _environment_overrides( + environment.get("PYTEST_ADDOPTS", ""), + project_root=root, + ) + major = _installed_pytest_major() if pytest_major is None else pytest_major + config = _resolve_config(root, major) + config_overrides = _config_addopts_overrides(config) + scope_value, scope_origin, scope_native, scope_path = _effective_option( + _ASYNCIO_SCOPE_OPTION, + default="function", + config=config, + config_overrides=config_overrides, + environment_overrides=environment_overrides, + ) + debug_value, debug_origin, debug_native, debug_path = _effective_option( + _ASYNCIO_DEBUG_OPTION, + default="false", + config=config, + config_overrides=config_overrides, + environment_overrides=environment_overrides, + ) + scope = _parse_loop_scope( + scope_value, + native_toml=scope_native, + error_path=scope_path, + ) + debug = _parse_bool( + debug_value, + native_toml=debug_native, + error_path=debug_path, + ) + except _ConfigError as error: + return ( + MigrationDiagnostic( + code="PYT509_PYTEST_CONFIG", + message=str(error), + source=_diagnostic_source(root, error.path), + ), + ) + + diagnostics: list[MigrationDiagnostic] = [] + if scope != "function": + diagnostics.append( + MigrationDiagnostic( + code="PYT508_ASYNCIO_CONFIG", + message=( + f"bare pytest.mark.asyncio resolves to {scope!r} event-loop scope via " + f"{scope_origin}; automatic migration preserves only function scope" + ), + source=_origin_source(root, scope_path, scope_origin), + ) + ) + if debug: + diagnostics.append( + MigrationDiagnostic( + code="PYT508_ASYNCIO_CONFIG", + message=( + "pytest-asyncio debug mode is enabled via " + f"{debug_origin}; automatic migration does not preserve debug-loop semantics" + ), + source=_origin_source(root, debug_path, debug_origin), + ) + ) + return tuple(diagnostics) + + +def _contains_bare_asyncio_test(source: SourceFile) -> bool: + try: + tree = ast.parse(source.text, filename=source.project_relative.as_posix()) + except SyntaxError: + return False + aliases = _Aliases.from_tree(tree) + functions: list[ast.AsyncFunctionDef] = [] + for statement in tree.body: + if isinstance(statement, ast.AsyncFunctionDef): + functions.append(statement) + elif isinstance(statement, ast.ClassDef): + functions.extend( + member for member in statement.body if isinstance(member, ast.AsyncFunctionDef) + ) + return any( + function.name.startswith("test") + and any( + not isinstance(decorator, ast.Call) + and aliases.canonical(decorator) == "pytest.mark.asyncio" + for decorator in function.decorator_list + ) + for function in functions + ) + + +def _installed_pytest_major() -> int: + """Return the installed major, defaulting to the oldest supported resolver.""" + + try: + raw_version = importlib_metadata.version("pytest") + except importlib_metadata.PackageNotFoundError: + return 8 + match = re.match(r"\s*(\d+)", raw_version) + if match is None: + return 8 + return 9 if int(match.group(1)) >= 9 else 8 + + +def _resolve_config( + project_root: Path, + pytest_major: int, +) -> _ResolvedConfig | None: + # The baseline runs with cwd=project_root and passes paths below that directory. + # Pytest computes their common ancestor with the invocation directory first, so + # its implicit lookup begins at project_root rather than in a nested test folder. + names = _PYTEST_9_CONFIG_NAMES if pytest_major >= 9 else _PYTEST_8_CONFIG_NAMES + for name in names: + candidate = project_root / name + if not candidate.is_file(): + continue + loaded = _load_candidate(candidate, pytest_major) + if loaded is not None: + return loaded + return None + + +def _load_candidate(path: Path, pytest_major: int) -> _ResolvedConfig | None: + try: + if path.suffix == ".ini": + return _load_ini(path, pytest_major) + if path.suffix == ".cfg": + return _load_setup_cfg(path) + if path.suffix == ".toml": + return _load_toml(path, pytest_major) + except (OSError, UnicodeError, configparser.Error, tomllib.TOMLDecodeError) as error: + raise _ConfigError( + path, f"cannot parse pytest configuration {path.name}: {error}" + ) from error + return None + + +def _read_ini(path: Path) -> configparser.RawConfigParser: + parser = configparser.RawConfigParser(interpolation=None) + with path.open(encoding="utf-8") as source: + parser.read_file(source) + return parser + + +def _load_ini(path: Path, pytest_major: int) -> _ResolvedConfig | None: + parser = _read_ini(path) + if parser.has_section("pytest"): + return _ResolvedConfig(path, dict(parser.items("pytest", raw=True))) + if path.name == "pytest.ini" or (path.name == ".pytest.ini" and pytest_major >= 9): + return _ResolvedConfig(path, {}) + return None + + +def _load_setup_cfg(path: Path) -> _ResolvedConfig | None: + parser = _read_ini(path) + if parser.has_section("tool:pytest"): + return _ResolvedConfig(path, dict(parser.items("tool:pytest", raw=True))) + if parser.has_section("pytest"): + raise _ConfigError( + path, + "setup.cfg uses unsupported [pytest]; pytest requires [tool:pytest]", + ) + return None + + +def _load_toml(path: Path, pytest_major: int) -> _ResolvedConfig | None: + with path.open("rb") as source: + document = tomllib.load(source) + if path.name in {"pytest.toml", ".pytest.toml"}: + if pytest_major < 9: + return None + values = document.get("pytest", {}) + if not isinstance(values, dict): + raise _ConfigError(path, "[pytest] in pytest.toml must be a table") + return _ResolvedConfig(path, values, native_toml=True) + + tool = document.get("tool", {}) + if not isinstance(tool, dict): + return None + pytest_table = tool.get("pytest", {}) + if not isinstance(pytest_table, dict): + raise _ConfigError(path, "[tool.pytest] in pyproject.toml must be a table") + ini_values = pytest_table.get("ini_options") + if ini_values is not None and not isinstance(ini_values, dict): + raise _ConfigError(path, "[tool.pytest.ini_options] must be a table") + + if pytest_major >= 9: + native_values = {key: value for key, value in pytest_table.items() if key != "ini_options"} + if native_values and ini_values: + raise _ConfigError( + path, + "pyproject.toml cannot combine [tool.pytest] options with " + "[tool.pytest.ini_options]", + ) + if native_values: + return _ResolvedConfig(path, native_values, native_toml=True) + if ini_values is not None: + values = { + key: value if isinstance(value, list) else str(value) + for key, value in ini_values.items() + } + return _ResolvedConfig(path, values) + return None + + +def _config_addopts_overrides(config: _ResolvedConfig | None) -> _ArgumentOverrides: + if config is None or "addopts" not in config.values: + return _ArgumentOverrides({}, "pytest config addopts", None) + raw = config.values["addopts"] + label = f"{config.path.name} addopts" + if config.native_toml: + if not isinstance(raw, list) or any(not isinstance(item, str) for item in raw): + raise _ConfigError( + config.path, + f"{label} must be a list of strings in native TOML", + ) + tokens = raw + elif isinstance(raw, str): + try: + tokens = shlex.split(raw) + except ValueError as error: + raise _ConfigError(config.path, f"cannot parse {label}: {error}") from error + elif isinstance(raw, list) and all(isinstance(item, str) for item in raw): + tokens = raw + else: + raise _ConfigError(config.path, f"{label} must be a string or list of strings") + return _argument_overrides(tokens, label=label, path=config.path) + + +def _environment_overrides(raw: str, *, project_root: Path) -> _ArgumentOverrides: + if not raw.strip(): + return _ArgumentOverrides({}, "PYTEST_ADDOPTS", None) + try: + tokens = shlex.split(raw) + except ValueError as error: + raise _ConfigError( + None, + f"cannot parse PYTEST_ADDOPTS while checking pytest-asyncio settings: {error}", + ) from error + + return _argument_overrides( + tokens, + label="PYTEST_ADDOPTS", + path=None, + positional_root=project_root, + ) + + +def _argument_overrides( + tokens: Sequence[str], + *, + label: str, + path: Path | None, + positional_root: Path | None = None, +) -> _ArgumentOverrides: + + overrides: dict[str, str] = {} + debug_flag = False + index = 0 + positional_only = False + while index < len(tokens): + token = tokens[index] + if token == "--" and not positional_only: + positional_only = True + index += 1 + continue + if positional_only: + if positional_root is not None: + _validate_positional_path(token, positional_root) + index += 1 + continue + if ( + token in {"-c", "--config-file"} + or token.startswith("--config-file=") + or (token.startswith("-c") and token != "-c") + ): + raise _ConfigError( + path, + f"{label} selects an explicit config with -c/--config-file; " + "the effective pytest-asyncio settings cannot be proven statically", + ) + if token == "--asyncio-debug" or token.startswith("--asyncio-debug="): + debug_flag = True + + override: str | None = None + if token in {"-o", "--override-ini"}: + if index + 1 >= len(tokens): + raise _ConfigError( + path, + f"{label} contains -o/--override-ini without option=value", + ) + index += 1 + override = tokens[index] + elif token.startswith("--override-ini="): + override = token.partition("=")[2] + elif token.startswith("-o") and token != "-o": + override = token[2:].removeprefix("=") + if override is not None: + if "=" not in override: + raise _ConfigError( + path, + f"{label} contains malformed -o/--override-ini; expected option=value", + ) + key, value = override.split("=", 1) + if key in {_ASYNCIO_SCOPE_OPTION, _ASYNCIO_DEBUG_OPTION}: + overrides[key] = value + elif token in _OPTIONS_WITH_SEPARATE_VALUE and index + 1 < len(tokens): + index += 1 + elif not token.startswith("-") and positional_root is not None: + _validate_positional_path(token, positional_root) + index += 1 + return _ArgumentOverrides( + overrides, + label, + path, + asyncio_debug_flag=debug_flag, + ) + + +def _validate_positional_path(token: str, project_root: Path) -> None: + path_text = token.split("::", 1)[0] + if not path_text: + return + candidate = Path(path_text).expanduser() + if not candidate.is_absolute(): + candidate = project_root / candidate + absolute = Path(os.path.abspath(candidate)) + try: + exists = absolute.exists() + except OSError: + return + if not exists: + return + if absolute == project_root or project_root in absolute.parents: + return + raise _ConfigError( + None, + f"PYTEST_ADDOPTS positional path {token!r} is outside the project root and " + "can change pytest configuration discovery", + ) + + +def _effective_option( + name: str, + *, + default: object, + config: _ResolvedConfig | None, + config_overrides: _ArgumentOverrides, + environment_overrides: _ArgumentOverrides, +) -> tuple[object, str, bool, Path | None]: + if name == _ASYNCIO_DEBUG_OPTION: + if environment_overrides.asyncio_debug_flag: + return ( + True, + f"{environment_overrides.label} --asyncio-debug", + True, + environment_overrides.path, + ) + if config_overrides.asyncio_debug_flag: + return ( + True, + f"{config_overrides.label} --asyncio-debug", + True, + config_overrides.path, + ) + if name in environment_overrides.values: + return ( + environment_overrides.values[name], + f"{environment_overrides.label} -o {name}", + False, + environment_overrides.path, + ) + if name in config_overrides.values: + return ( + config_overrides.values[name], + f"{config_overrides.label} -o {name}", + False, + config_overrides.path, + ) + if config is not None and name in config.values: + return config.values[name], config.path.name, config.native_toml, config.path + return default, "pytest-asyncio default", False, None + + +def _parse_loop_scope( + value: object, + *, + native_toml: bool, + error_path: Path | None, +) -> str: + if not isinstance(value, str): + mode = "native TOML" if native_toml else "pytest configuration" + raise _ConfigError( + error_path, + f"{_ASYNCIO_SCOPE_OPTION} must be a string in {mode}", + ) + if value not in _VALID_LOOP_SCOPES: + raise _ConfigError( + error_path, + f"invalid {_ASYNCIO_SCOPE_OPTION}={value!r}; expected one of " + + ", ".join(sorted(_VALID_LOOP_SCOPES)), + ) + return value + + +def _parse_bool( + value: object, + *, + native_toml: bool, + error_path: Path | None, +) -> bool: + if native_toml: + if isinstance(value, bool): + return value + raise _ConfigError( + error_path, + f"{_ASYNCIO_DEBUG_OPTION} must be a boolean in native TOML", + ) + normalized = str(value).strip().lower() + if normalized in _FALSE_VALUES: + return False + if normalized in _TRUE_VALUES: + return True + raise _ConfigError(error_path, f"invalid {_ASYNCIO_DEBUG_OPTION} value {value!r}") + + +def _diagnostic_source(project_root: Path, path: Path | None) -> str: + if path is None: + return "" + try: + return path.relative_to(project_root).as_posix() + except ValueError: + return str(path) + + +def _origin_source( + project_root: Path, + path: Path | None, + origin: str, +) -> str: + if path is not None: + return _diagnostic_source(project_root, path) + if origin.startswith("PYTEST_ADDOPTS"): + return "" + return "" + + +__all__ = ["pytest_asyncio_config_diagnostics"] diff --git a/src/testenix/migration_runtime.py b/src/testenix/migration_runtime.py index 63ba42f..c2f016a 100644 --- a/src/testenix/migration_runtime.py +++ b/src/testenix/migration_runtime.py @@ -1,24 +1,26 @@ -"""Runtime helpers used by generated unittest migration wrappers. +"""Runtime helpers used by generated migration wrappers. -The migration layer intentionally executes the original ``unittest.TestCase`` -instead of approximating its assertion and per-test lifecycle semantics. A -generated wrapper remains a native Testenix function, while this module turns -the standard-library result callbacks back into exceptions understood by the -native executor. +Pytest coroutine wrappers preserve the default function-scoped event-loop +lifecycle of ``pytest-asyncio``. Unittest wrappers intentionally execute the +original ``unittest.TestCase`` instead of approximating its assertion and +per-test lifecycle semantics. """ from __future__ import annotations +import asyncio +import contextvars import hashlib import json import re import threading import unittest -from collections.abc import Mapping +from collections.abc import Callable, Coroutine, Mapping from dataclasses import dataclass +from functools import wraps from pathlib import Path from types import MappingProxyType, ModuleType, TracebackType -from typing import TypeAlias +from typing import Any, ParamSpec, TypeAlias, TypeVar from testenix.discovery import load_module @@ -28,6 +30,38 @@ ExceptionInfo: TypeAlias = tuple[type[BaseException], BaseException, TracebackType] RawExceptionInfo: TypeAlias = ExceptionInfo | tuple[None, None, None] +_P = ParamSpec("_P") +_R = TypeVar("_R") + + +def isolated_pytest_asyncio( + function: Callable[_P, Coroutine[Any, Any, _R]], + /, +) -> Callable[_P, None]: + """Run one migrated pytest coroutine in a fresh, function-scoped loop. + + The returned callable is deliberately synchronous. The native executor + therefore invokes it outside its orchestration loop, allowing ``Runner`` + to reproduce pytest-asyncio's default one-loop-per-test lifecycle without + changing the behavior of native Testenix coroutine tests. + """ + + @wraps(function) + def wrapper(*args: _P.args, **kwargs: _P.kwargs) -> None: + try: + asyncio.get_running_loop() + except RuntimeError: + pass + else: + raise RuntimeError( + "a migrated pytest asyncio test cannot run inside an active event loop" + ) + + context = contextvars.copy_context() + with asyncio.Runner(debug=False) as runner: + runner.run(function(*args, **kwargs), context=context) + + return wrapper @dataclass(frozen=True, slots=True) @@ -485,6 +519,7 @@ def run_unittest_case( "UnittestMigrationRuntimeError", "UnittestResultProtocolError", "UnittestSourceChangedError", + "isolated_pytest_asyncio", "load_unittest_case", "resolve_unittest_source", "run_unittest_case", diff --git a/src/testenix/migration_service.py b/src/testenix/migration_service.py index 40f84d9..94068d0 100644 --- a/src/testenix/migration_service.py +++ b/src/testenix/migration_service.py @@ -303,9 +303,18 @@ def render_migration_summary(report: MigrationReport) -> str: MigrationStatus.VALIDATION_FAILED: "migration validation failed; no output was published", MigrationStatus.SAFETY_ERROR: "migration rejected an unsafe path or concurrent change", }[report.status] + inventory_label = { + MigrationStatus.ANALYZED: "analyzed candidate", + MigrationStatus.VALIDATED: "validated candidate", + MigrationStatus.PUBLISHED: "converted", + MigrationStatus.UNSUPPORTED: "statically convertible subset", + MigrationStatus.VALIDATION_FAILED: "generated candidate", + MigrationStatus.SAFETY_ERROR: "generated candidate", + }[report.status] lines = [ f"Testenix: {headline}", - f" converted: {report.converted_tests} tests in {len(report.generated_files)} files", + f" {inventory_label}: {report.converted_tests} tests in " + f"{len(report.generated_files)} files", f" originals modified: {'yes' if report.originals_modified else 'no'}", f" output: {report.output}", f" detail: {report.message}", @@ -316,15 +325,52 @@ def render_migration_summary(report: MigrationReport) -> str: lines.append(_summary_line("native serial", report.native_serial)) if report.native_parallel is not None: lines.append(_summary_line("native parallel", report.native_parallel)) - for diagnostic in report.diagnostics: - location = diagnostic.source - if diagnostic.line is not None: - location = f"{location}:{diagnostic.line}" + lines.extend(_diagnostic_summary(report.diagnostics)) + return "\n".join(lines) + + +def _diagnostic_summary( + diagnostics: Sequence[MigrationDiagnostic], +) -> tuple[str, ...]: + """Group repeated console diagnostics without losing JSON report detail.""" + + if not diagnostics: + return () + + counts = Counter(diagnostic.severity for diagnostic in diagnostics) + groups: dict[tuple[DiagnosticSeverity, str], list[MigrationDiagnostic]] = {} + for diagnostic in diagnostics: + groups.setdefault((diagnostic.severity, diagnostic.code), []).append(diagnostic) + + error_count = counts[DiagnosticSeverity.ERROR] + warning_count = counts[DiagnosticSeverity.WARNING] + lines = [ + f" diagnostics: {error_count} error(s), {warning_count} warning(s), {len(groups)} code(s)" + ] + severity_order = { + DiagnosticSeverity.ERROR: 0, + DiagnosticSeverity.WARNING: 1, + } + for (severity, code), members in sorted( + groups.items(), + key=lambda item: (severity_order[item[0][0]], item[0][1]), + ): + first = members[0] + location = first.source + if first.line is not None: + location = f"{location}:{first.line}" + if len(members) == 1: + lines.append(f" {severity.value.upper()} {code} {location}: {first.message}") + continue + source_count = len({diagnostic.source for diagnostic in members}) lines.append( - f" {diagnostic.severity.value.upper()} {diagnostic.code} {location}: " - f"{diagnostic.message}" + f" {severity.value.upper()} {code}: {len(members)} occurrence(s) in " + f"{source_count} file(s); first at {location}: {first.message}" ) - return "\n".join(lines) + + if len(diagnostics) > len(groups): + lines.append(" diagnostic detail: --report-json FILE|- retains every line-addressed entry") + return tuple(lines) def _summary_line(label: str, summary: ValidationSummary) -> str: @@ -397,24 +443,24 @@ def migrate(options: MigrationOptions) -> MigrationReport: output_relative=paths.output.relative_to(paths.project_root), ) bundle = plan.bundle + if plan.resolved_framework in {"pytest", "mixed"}: + from testenix.migration_pytest_config import pytest_asyncio_config_diagnostics + + # This is a static preflight over the exact source invocation. The serial and + # parallel shadow runs below remain the authoritative behavioral gates. + asyncio_config_diagnostics = pytest_asyncio_config_diagnostics( + project_root=paths.project_root, + source_paths=paths.sources, + files=source_files, + ) + if asyncio_config_diagnostics: + bundle = _merge_bundles( + bundle, + ConversionBundle(diagnostics=asyncio_config_diagnostics), + ) source_hashes = {source.project_relative.as_posix(): source.sha256 for source in source_files} generated_files = tuple(artifact.relative_path.as_posix() for artifact in bundle.artifacts) diagnostics = bundle.diagnostics - affinity_units = {mapping.target_file for mapping in bundle.mappings} - if bundle.mappings and len(affinity_units) < 2: - diagnostics = ( - *diagnostics, - MigrationDiagnostic( - code="MIG006", - message=( - "the parallel validation command is configured with at least two workers, " - "but this converted suite has one module affinity unit and therefore " - "executes on one worker" - ), - source="", - severity=DiagnosticSeverity.WARNING, - ), - ) context = _ReportContext( framework=plan.resolved_framework, project_root=paths.project_root, @@ -467,6 +513,21 @@ def migrate(options: MigrationOptions) -> MigrationReport: context=context, ) + affinity_units = {mapping.target_file for mapping in bundle.mappings} + if len(affinity_units) < 2: + diagnostic = MigrationDiagnostic( + code="MIG006", + message=( + "the parallel validation command is configured with at least two workers, " + "but this converted suite has one module affinity unit and therefore " + "executes on one worker" + ), + source="", + severity=DiagnosticSeverity.WARNING, + ) + diagnostics = (*diagnostics, diagnostic) + context = replace(context, diagnostics=diagnostics) + baseline: ValidationSummary | None = None native_serial: ValidationSummary | None = None native_parallel: ValidationSummary | None = None diff --git a/tests/test_builtin_fixtures.py b/tests/test_builtin_fixtures.py new file mode 100644 index 0000000..77f7aaf --- /dev/null +++ b/tests/test_builtin_fixtures.py @@ -0,0 +1,408 @@ +from __future__ import annotations + +import os +from pathlib import Path +from textwrap import dedent +from types import SimpleNamespace + +import pytest + +import testenix.builtin_fixtures as builtin_fixtures +from testenix.api import fixture, get_fixture_metadata +from testenix.builtin_fixtures import MonkeyPatch +from testenix.config import TestenixConfig +from testenix.contracts import Scope, Status +from testenix.discovery import discover +from testenix.executor import execute_tests +from testenix.fixtures import FixtureRegistry +from testenix.runner import run + + +def write_test_module(tmp_path: Path, source: str) -> Path: + path = tmp_path / "test_builtins.py" + path.write_text(dedent(source), encoding="utf-8") + return path + + +def test_tmp_path_is_unique_absolute_and_removed_after_each_test(tmp_path: Path) -> None: + path = write_test_module( + tmp_path, + """ + from pathlib import Path + + SEEN = [] + + def test_first(tmp_path): + assert isinstance(tmp_path, Path) + assert tmp_path.is_absolute() + assert tmp_path.is_dir() + artifact = tmp_path / "read-only.txt" + artifact.write_text("temporary", encoding="utf-8") + artifact.chmod(0o400) + SEEN.append(tmp_path) + + def test_second(tmp_path): + assert not SEEN[0].exists() + assert tmp_path != SEEN[0] + assert tmp_path.is_dir() + SEEN.append(tmp_path) + """, + ) + collection = discover(path) + + results = execute_tests(collection.items) + + assert not collection.issues + assert [result.status for result in results] == [Status.PASS, Status.PASS] + seen = collection.items[0].function.__globals__["SEEN"] + assert len(set(seen)) == 2 + assert all(not temporary_path.exists() for temporary_path in seen) + + +def test_user_fixtures_override_name_only_builtins(tmp_path: Path) -> None: + path = write_test_module( + tmp_path, + """ + from pathlib import Path + from testenix import fixture + + @fixture + def tmp_path(): + return Path("user-path") + + @fixture + def monkeypatch(): + return "user-monkeypatch" + + def test_overrides(tmp_path, monkeypatch): + assert tmp_path == Path("user-path") + assert monkeypatch == "user-monkeypatch" + """, + ) + collection = discover(path) + + results = execute_tests(collection.items) + + assert not collection.issues + assert [definition.name for definition in collection.fixtures] == [ + "tmp_path", + "monkeypatch", + ] + assert results[0].status is Status.PASS + + +def test_monkeypatch_rolls_back_lifo_after_a_test_failure(tmp_path: Path) -> None: + environment_name = f"TESTENIX_BUILTIN_{os.getpid()}_{tmp_path.name}" + os.environ.pop(environment_name, None) + path = write_test_module( + tmp_path, + f""" + import os + import sys + + VALUE = "original" + ENVIRONMENT_NAME = {environment_name!r} + + class Settings: + value = "class-original" + + def test_mutates_then_fails(monkeypatch): + module = sys.modules[__name__] + monkeypatch.setattr(module, "VALUE", "first") + monkeypatch.setattr(module, "VALUE", "second") + monkeypatch.setattr(module, "CREATED", 42, raising=False) + monkeypatch.setattr(f"{{__name__}}.Settings.value", "class-patched") + monkeypatch.setenv(ENVIRONMENT_NAME, "first") + monkeypatch.setenv(ENVIRONMENT_NAME, "second") + assert VALUE == "second" + assert Settings.value == "class-patched" + assert os.environ[ENVIRONMENT_NAME] == "second" + raise AssertionError("intentional failure") + + def test_observes_restored_state(): + assert VALUE == "original" + assert "CREATED" not in globals() + assert Settings.value == "class-original" + assert ENVIRONMENT_NAME not in os.environ + """, + ) + collection = discover(path) + + results = execute_tests(collection.items) + + assert not collection.issues + assert [result.status for result in results] == [Status.FAIL, Status.PASS] + assert environment_name not in os.environ + + +def test_monkeypatch_dependency_rolls_back_after_autouse_setup_failure( + tmp_path: Path, +) -> None: + path = write_test_module( + tmp_path, + """ + import sys + from testenix import fixture + + VALUE = "original" + ATTEMPT = 0 + + @fixture(autouse=True) + def sometimes_broken(monkeypatch): + global ATTEMPT + assert VALUE == "original" + monkeypatch.setattr(sys.modules[__name__], "VALUE", "patched") + ATTEMPT += 1 + if ATTEMPT == 1: + raise RuntimeError("setup failed") + + def test_first(): + raise AssertionError("call must not run") + + def test_second(): + assert VALUE == "patched" + """, + ) + collection = discover(path) + + results = execute_tests(collection.items) + + assert not collection.issues + assert [result.status for result in results] == [Status.ERROR_SETUP, Status.PASS] + assert collection.items[0].function.__globals__["VALUE"] == "original" + + +def test_monkeypatch_raising_prepend_manual_undo_and_created_attribute() -> None: + environment_name = f"TESTENIX_MONKEYPATCH_{os.getpid()}" + os.environ[environment_name] = "old" + target = type("Target", (), {})() + patcher = MonkeyPatch() + try: + with pytest.raises(AttributeError): + patcher.setattr(target, "missing", 1) + + patcher.setattr(target, "missing", 1, raising=False) + patcher.setenv(environment_name, "new", prepend=os.pathsep) + assert target.missing == 1 + assert os.environ[environment_name] == f"new{os.pathsep}old" + + patcher.undo() + patcher.undo() + assert not hasattr(target, "missing") + assert os.environ[environment_name] == "old" + finally: + os.environ.pop(environment_name, None) + + +def test_monkeypatch_undo_attempts_later_actions_after_a_failure() -> None: + environment_name = f"TESTENIX_MONKEYPATCH_FAILURE_{os.getpid()}" + os.environ.pop(environment_name, None) + + class Fragile: + fail_restore = False + value = "original" + + def __setattr__(self, name: str, value: object) -> None: + if name == "value" and value == "original" and self.fail_restore: + raise RuntimeError("restore failed") + object.__setattr__(self, name, value) + + target = Fragile() + patcher = MonkeyPatch() + patcher.setenv(environment_name, "patched") + patcher.setattr(target, "value", "changed") + target.fail_restore = True + + with pytest.raises(RuntimeError, match="restore failed"): + patcher.undo() + + assert environment_name not in os.environ + patcher.undo() + + +def test_monkeypatch_does_not_record_rejected_attribute_mutation() -> None: + assignments: list[object] = [] + + class RejectingTarget: + value = "original" + + def __setattr__(self, name: str, value: object) -> None: + if name == "value": + assignments.append(value) + if value == "rejected": + raise RuntimeError("patch rejected") + object.__setattr__(self, name, value) + + target = RejectingTarget() + patcher = MonkeyPatch() + + with pytest.raises(RuntimeError, match="patch rejected"): + patcher.setattr(target, "value", "rejected") + + patcher.undo() + assert target.value == "original" + assert assignments == ["rejected"] + + +def test_monkeypatch_does_not_record_rejected_environment_mutation( + monkeypatch: pytest.MonkeyPatch, +) -> None: + writes: list[tuple[str, str]] = [] + removals: list[str] = [] + + class RejectingEnvironment: + def __contains__(self, name: object) -> bool: + return False + + def get(self, name: str, default: object = None) -> object: + return default + + def __setitem__(self, name: str, value: str) -> None: + writes.append((name, value)) + raise RuntimeError("environment write rejected") + + def pop(self, name: str, default: object = None) -> object: + removals.append(name) + return default + + fake_environment = RejectingEnvironment() + monkeypatch.setattr( + builtin_fixtures, + "os", + SimpleNamespace(environ=fake_environment), + ) + patcher = MonkeyPatch() + + with pytest.raises(RuntimeError, match="environment write rejected"): + patcher.setenv("TESTENIX_REJECTED_WRITE", "value") + + patcher.undo() + assert writes == [("TESTENIX_REJECTED_WRITE", "value")] + assert removals == [] + + +def test_function_and_module_autouse_run_without_explicit_test_parameters( + tmp_path: Path, +) -> None: + path = write_test_module( + tmp_path, + """ + from testenix import fixture + + EVENTS = [] + + @fixture(scope="module", autouse=True) + def module_state(): + EVENTS.append("module setup") + yield "module" + EVENTS.append("module teardown") + + @fixture(autouse=True) + def function_state(module_state): + EVENTS.append("function setup") + yield module_state + " function" + EVENTS.append("function teardown") + + def test_first(): + EVENTS.append("first") + + def test_second(function_state): + assert function_state == "module function" + EVENTS.append("second") + """, + ) + collection = discover(path) + + results = execute_tests(collection.items) + + assert not collection.issues + assert [result.status for result in results] == [Status.PASS, Status.PASS] + assert collection.items[0].function.__globals__["EVENTS"] == [ + "module setup", + "function setup", + "first", + "function teardown", + "function setup", + "second", + "function teardown", + "module teardown", + ] + + +def test_spawn_runner_executes_builtin_dependencies_of_autouse(tmp_path: Path) -> None: + teardown_marker = tmp_path / "autouse-teardown.txt" + environment_name = f"TESTENIX_SPAWN_{os.getpid()}_{tmp_path.name}" + path = write_test_module( + tmp_path, + f""" + import os + from pathlib import Path + from testenix import fixture + + @fixture(autouse=True) + def isolated_state(monkeypatch, tmp_path): + monkeypatch.setenv({environment_name!r}, "worker-value") + assert tmp_path.is_dir() + yield + assert os.environ[{environment_name!r}] == "worker-value" + assert tmp_path.is_dir() + Path({str(teardown_marker)!r}).write_text("finished", encoding="utf-8") + + def test_uses_implicit_state(): + assert os.environ[{environment_name!r}] == "worker-value" + """, + ) + + result = run((str(path),), TestenixConfig(workers=1, history_path=None)) + + assert result.exit_code == 0 + assert [test.status for test in result.tests] == [Status.PASS] + assert teardown_marker.read_text(encoding="utf-8") == "finished" + assert environment_name not in os.environ + + +def test_local_non_autouse_fixture_suppresses_global_autouse_override( + tmp_path: Path, +) -> None: + events: list[str] = [] + + @fixture(name="state", autouse=True) + def global_state() -> None: + events.append("global autouse ran") + + path = write_test_module( + tmp_path, + """ + from testenix import fixture + + @fixture(name="state") + def local_state(): + return "local" + + def test_override_is_not_implicit(): + pass + """, + ) + collection = discover(path) + registry = FixtureRegistry() + registry.register(global_state) + + results = execute_tests(collection.items, registry=registry) + + assert not collection.issues + assert results[0].status is Status.PASS + assert events == [] + + +def test_fixture_autouse_metadata_and_validation() -> None: + @fixture(scope="module", autouse=True) + def implicit() -> None: + return None + + metadata = get_fixture_metadata(implicit) + + assert metadata is not None + assert metadata.scope is Scope.MODULE + assert metadata.autouse is True + with pytest.raises(TypeError, match="autouse must be a boolean"): + fixture(autouse=1) # type: ignore[arg-type] diff --git a/tests/test_migration_pytest.py b/tests/test_migration_pytest.py index 8bcae12..a517181 100644 --- a/tests/test_migration_pytest.py +++ b/tests/test_migration_pytest.py @@ -3,6 +3,7 @@ import ast import builtins import hashlib +import os from collections import Counter from pathlib import Path from textwrap import dedent @@ -299,6 +300,108 @@ def test_double(factor) -> None: assert execute_tests(collection.items)[0].status is Status.PASS +def test_adjacent_conftest_autouse_fixture_stays_implicit( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + environment_name = "TESTENIX_CONFTEST_AUTOUSE" + monkeypatch.delenv(environment_name, raising=False) + conftest = source_file( + "tests/conftest.py", + f""" + import pytest + + @pytest.fixture(autouse=True) + def environment(monkeypatch): + monkeypatch.setenv({environment_name!r}, "ready") + """, + migration_relative="conftest.py", + ) + test_module = source_file( + "tests/test_conftest_autouse.py", + f""" + import os + + def test_environment_is_ready() -> None: + assert os.environ[{environment_name!r}] == "ready" + """, + migration_relative="test_conftest_autouse.py", + ) + + bundle = convert_pytest_suite((test_module,), (conftest,)) + + assert not bundle.blocking_diagnostics + helper = next( + artifact for artifact in bundle.artifacts if artifact.relative_path.name.startswith("_") + ) + assert "@_testenix_fixture(autouse=True)" in helper.content + collection = materialize_and_discover(tmp_path, bundle) + assert not collection.issues + assert execute_tests(collection.items)[0].status is Status.PASS + assert environment_name not in os.environ + + +@pytest.mark.parametrize( + ("fixture_location", "expected_code"), + [ + ("module", "PYT509_EVENT_LOOP_POLICY"), + ("adjacent", "PYT509_EVENT_LOOP_POLICY"), + ("ancestor", "PYT213_ANCESTOR_CONFTEST"), + ], +) +def test_custom_event_loop_policy_fixture_blocks_asyncio_migration( + fixture_location: str, + expected_code: str, +) -> None: + fixture_text = """ + import pytest + + @pytest.fixture + def event_loop_policy(): + return object() + """ + test_text = """ + import pytest + + @pytest.mark.asyncio + async def test_async_value(): + pass + """ + conftests: tuple[SourceFile, ...] = () + if fixture_location == "module": + selected = source_file( + "tests/test_policy.py", + fixture_text + test_text, + migration_relative="test_policy.py", + ) + else: + selected_path = ( + "tests/test_policy.py" + if fixture_location == "adjacent" + else "tests/unit/test_policy.py" + ) + selected = source_file( + selected_path, + test_text, + migration_relative="test_policy.py", + ) + conftests = ( + source_file( + "tests/conftest.py", + fixture_text, + migration_relative="conftest.py", + ), + ) + + bundle = convert_pytest_suite((selected,), conftests) + + diagnostic = next( + diagnostic for diagnostic in bundle.blocking_diagnostics if diagnostic.code == expected_code + ) + assert "event_loop_policy" in diagnostic.message + assert not bundle.mappings + + def test_skip_skipif_and_plain_marker_have_native_outcomes_and_tags( tmp_path: Path, ) -> None: @@ -346,17 +449,695 @@ def test_conditionally_executed() -> None: } +def test_bare_pytest_asyncio_uses_a_fresh_closed_loop_and_preserves_signature( + tmp_path: Path, +) -> None: + source = source_file( + "tests/test_async.py", + """ + import asyncio + import pytest + + LOOPS = [] + CANCELLED = [] + + @pytest.mark.asyncio + async def test_first_loop(tmp_path) -> None: + assert tmp_path.is_dir() + LOOPS.append(asyncio.get_running_loop()) + + async def linger() -> None: + try: + await asyncio.sleep(60) + finally: + CANCELLED.append(True) + + asyncio.create_task(linger()) + await asyncio.sleep(0) + + @pytest.mark.asyncio + async def test_second_loop(tmp_path) -> None: + current = asyncio.get_running_loop() + assert tmp_path.is_dir() + assert current is not LOOPS[0] + assert LOOPS[0].is_closed() + assert CANCELLED == [True] + """, + migration_relative="test_async.py", + ) + + bundle = convert_pytest_suite((source,)) + + assert not bundle.blocking_diagnostics + content = bundle.artifacts[0].content + assert "pytest.mark.asyncio" not in content + assert "import pytest" not in content + assert ( + "from testenix.migration_runtime import isolated_pytest_asyncio as " + "_testenix_isolated_asyncio" + ) in content + assert content.count("@_testenix_isolated_asyncio") == 2 + collection = materialize_and_discover(tmp_path, bundle) + assert not collection.issues + generated_tree = ast.parse(content) + generated_lines = { + statement.name: min( + statement.lineno, + *(decorator.lineno for decorator in statement.decorator_list), + ) + for statement in generated_tree.body + if isinstance(statement, ast.AsyncFunctionDef) + } + assert { + item.spec.function_name: item.spec.source_line for item in collection.items + } == generated_lines + assert Counter(result.status for result in execute_tests(collection.items)) == {Status.PASS: 2} + + +def test_asyncio_isolation_composes_with_both_case_orders_and_class_wrappers( + tmp_path: Path, +) -> None: + source = source_file( + "tests/test_async_cases.py", + """ + import asyncio + import pytest + + SEEN_LOOPS = [] + + def remember_fresh_loop(): + current = asyncio.get_running_loop() + assert all(previous is not current and previous.is_closed() for previous in SEEN_LOOPS) + SEEN_LOOPS.append(current) + + @pytest.mark.parametrize("value", [1, 2], ids=["one", "two"]) + @pytest.mark.asyncio + async def test_parametrize_outside(value, tmp_path): + remember_fresh_loop() + assert tmp_path.is_dir() + assert value in {1, 2} + + @pytest.mark.asyncio + @pytest.mark.parametrize("value", [3, 4], ids=["three", "four"]) + async def test_asyncio_outside(value): + remember_fresh_loop() + assert value in {3, 4} + + class TestAsyncCases: + @pytest.mark.asyncio + @pytest.mark.parametrize("value", [5, 6], ids=["five", "six"]) + async def test_cases(self, value, tmp_path): + remember_fresh_loop() + assert tmp_path.is_dir() + assert value in {5, 6} + """, + migration_relative="test_async_cases.py", + ) + + bundle = convert_pytest_suite((source,)) + + assert not bundle.blocking_diagnostics + assert len(bundle.mappings) == 6 + assert {mapping.case_id for mapping in bundle.mappings} == { + "one", + "two", + "three", + "four", + "five", + "six", + } + content = bundle.artifacts[0].content + assert "pytest.mark" not in content + assert "import pytest" not in content + assert content.count("@_testenix_isolated_asyncio") == 3 + collection = materialize_and_discover(tmp_path, bundle) + assert not collection.issues + assert Counter(result.status for result in execute_tests(collection.items)) == {Status.PASS: 6} + + +def test_asyncio_isolation_propagates_test_failures(tmp_path: Path) -> None: + source = source_file( + "tests/test_async_failure.py", + """ + import pytest + + @pytest.mark.asyncio + async def test_failure(): + raise AssertionError("isolated failure") + """, + migration_relative="test_async_failure.py", + ) + + bundle = convert_pytest_suite((source,)) + + assert not bundle.blocking_diagnostics + collection = materialize_and_discover(tmp_path, bundle) + assert not collection.issues + result = execute_tests(collection.items)[0] + assert result.status is Status.FAIL + assert result.attempts[0].phases[1].exception_type == "builtins.AssertionError" + assert result.attempts[0].phases[1].message == "isolated failure" + + +def test_supported_builtin_fixtures_run_without_a_pytest_runtime_import( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + environment_name = "TESTENIX_MIGRATED_MONKEYPATCH" + monkeypatch.delenv(environment_name, raising=False) + source = source_file( + "tests/test_builtins.py", + f""" + import os + + def test_native_builtins(tmp_path, monkeypatch) -> None: + destination = tmp_path / "proof.txt" + destination.write_text("native", encoding="utf-8") + monkeypatch.setenv({environment_name!r}, destination.read_text(encoding="utf-8")) + assert os.environ[{environment_name!r}] == "native" + """, + migration_relative="test_builtins.py", + ) + + bundle = convert_pytest_suite((source,)) + + assert not bundle.blocking_diagnostics + assert "def test_native_builtins(tmp_path, monkeypatch" in bundle.artifacts[0].content + assert "pytest" not in bundle.artifacts[0].content + collection = materialize_and_discover(tmp_path, bundle) + assert not collection.issues + assert execute_tests(collection.items)[0].status is Status.PASS + assert environment_name not in os.environ + + +def test_migrated_monkeypatch_allows_only_implemented_direct_calls( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + environment_name = "TESTENIX_MIGRATED_MONKEYPATCH_DIRECT" + monkeypatch.delenv(environment_name, raising=False) + source = source_file( + "tests/test_monkeypatch_direct.py", + f""" + import os + + class Target: + value = "original" + + def test_direct_calls(monkeypatch) -> None: + monkeypatch.setattr(Target, "value", "changed") + monkeypatch.setenv({environment_name!r}, Target.value) + assert Target.value == "changed" + assert os.environ[{environment_name!r}] == "changed" + monkeypatch.undo() + assert Target.value == "original" + assert {environment_name!r} not in os.environ + """, + migration_relative="test_monkeypatch_direct.py", + ) + + bundle = convert_pytest_suite((source,)) + + assert not bundle.blocking_diagnostics + collection = materialize_and_discover(tmp_path, bundle) + assert not collection.issues + assert execute_tests(collection.items)[0].status is Status.PASS + assert environment_name not in os.environ + + +def test_monkeypatch_can_flow_through_static_helpers_positionally_and_by_keyword( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + environment_name = "TESTENIX_MIGRATED_MONKEYPATCH_HELPER" + monkeypatch.delenv(environment_name, raising=False) + source = source_file( + "tests/test_monkeypatch_helpers.py", + f""" + import os + + class Target: + value = "original" + + def _set_target(patch, target, value): + patch.setattr(target, "value", value) + + def _set_environment(*, patch, name, value): + patch.setenv(name, value) + + def _configure(patch, name): + _set_environment(patch=patch, name=name, value=Target.value) + + def test_static_helpers(monkeypatch, tmp_path): + assert tmp_path.is_dir() + _set_target(monkeypatch, Target, "through-helper") + _configure(patch=monkeypatch, name={environment_name!r}) + assert os.environ[{environment_name!r}] == "through-helper" + """, + migration_relative="test_monkeypatch_helpers.py", + ) + + bundle = convert_pytest_suite((source,)) + + assert not bundle.blocking_diagnostics + collection = materialize_and_discover(tmp_path, bundle) + assert not collection.issues + assert execute_tests(collection.items)[0].status is Status.PASS + assert environment_name not in os.environ + + +def test_monkeypatch_helper_analysis_has_a_cycle_guard( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + environment_name = "TESTENIX_MIGRATED_MONKEYPATCH_CYCLE" + monkeypatch.delenv(environment_name, raising=False) + source = source_file( + "tests/test_monkeypatch_cycle.py", + f""" + import os + + def _left(patch, depth): + if depth: + _right(patch, depth - 1) + else: + patch.setenv({environment_name!r}, "left") + + def _right(patch, depth): + if depth: + _left(patch, depth - 1) + else: + patch.setenv({environment_name!r}, "right") + + def test_recursive_helpers(monkeypatch): + _left(monkeypatch, 2) + assert os.environ[{environment_name!r}] == "left" + """, + migration_relative="test_monkeypatch_cycle.py", + ) + + bundle = convert_pytest_suite((source,)) + + assert not bundle.blocking_diagnostics + collection = materialize_and_discover(tmp_path, bundle) + assert not collection.issues + assert execute_tests(collection.items)[0].status is Status.PASS + assert environment_name not in os.environ + + +def test_user_defined_monkeypatch_fixture_is_not_restricted_by_builtin_contract( + tmp_path: Path, +) -> None: + source = source_file( + "tests/test_custom_monkeypatch.py", + """ + import pytest + + class CustomPatch: + def delenv(self): + return "project-owned" + + @pytest.fixture + def monkeypatch(): + return CustomPatch() + + def test_custom_fixture(monkeypatch) -> None: + alias = monkeypatch + assert alias.delenv() == "project-owned" + """, + migration_relative="test_custom_monkeypatch.py", + ) + + bundle = convert_pytest_suite((source,)) + + assert not bundle.blocking_diagnostics + collection = materialize_and_discover(tmp_path, bundle) + assert not collection.issues + assert execute_tests(collection.items)[0].status is Status.PASS + + +def test_autouse_fixture_keeps_implicit_setup_and_builtin_dependencies( + tmp_path: Path, +) -> None: + source = source_file( + "tests/test_autouse.py", + """ + import os + import pytest + + @pytest.fixture(autouse=True) + def isolated_environment(monkeypatch, tmp_path): + marker = tmp_path / "autouse.txt" + marker.write_text("ready", encoding="utf-8") + monkeypatch.setenv("TESTENIX_AUTOUSE_PATH", str(marker)) + yield + + def test_implicit_fixture_ran() -> None: + marker = os.environ["TESTENIX_AUTOUSE_PATH"] + assert marker.endswith("autouse.txt") + + def test_explicit_request_reuses_autouse(isolated_environment) -> None: + assert isolated_environment is None + """, + migration_relative="test_autouse.py", + ) + + bundle = convert_pytest_suite((source,)) + + assert not bundle.blocking_diagnostics + content = bundle.artifacts[0].content + assert "@_testenix_fixture(autouse=True)" in content + assert "import pytest" not in content + collection = materialize_and_discover(tmp_path, bundle) + assert not collection.issues + assert Counter(result.status for result in execute_tests(collection.items)) == {Status.PASS: 2} + assert "TESTENIX_AUTOUSE_PATH" not in os.environ + + +def test_simple_pytest_class_becomes_fresh_instance_wrappers_with_stable_mappings( + tmp_path: Path, +) -> None: + source = source_file( + "tests/test_class.py", + """ + import asyncio + import pytest + + class TestExample: + def _double(self, value): + return value * 2 + + def test_first_instance(self, tmp_path): + assert not hasattr(self, "seen") + self.seen = tmp_path + + def test_second_instance(self): + assert not hasattr(self, "seen") + + @pytest.mark.parametrize("value", [2, 3], ids=["two", "three"]) + def test_cases(self, value): + assert self._double(value) == value + value + + @pytest.mark.asyncio + async def test_async_method(self, monkeypatch): + monkeypatch.setenv("TESTENIX_CLASS_ASYNC", "yes") + await asyncio.sleep(0) + """, + migration_relative="test_class.py", + ) + + first = convert_pytest_suite((source,)) + second = convert_pytest_suite((source,)) + + assert first == second + assert not first.blocking_diagnostics + assert {mapping.source_id for mapping in first.mappings} == { + "tests/test_class.py::TestExample.test_first_instance", + "tests/test_class.py::TestExample.test_second_instance", + "tests/test_class.py::TestExample.test_cases[two]", + "tests/test_class.py::TestExample.test_cases[three]", + "tests/test_class.py::TestExample.test_async_method", + } + assert all( + mapping.target_function.startswith("test_TestExample__") for mapping in first.mappings + ) + content = first.artifacts[0].content + assert "pytest.mark" not in content + assert "import pytest" not in content + collection = materialize_and_discover(tmp_path, first) + assert not collection.issues + assert Counter(result.status for result in execute_tests(collection.items)) == {Status.PASS: 5} + assert "TESTENIX_CLASS_ASYNC" not in os.environ + + +def test_class_wrapper_drops_parameter_and_return_annotations(tmp_path: Path) -> None: + source = source_file( + "tests/test_annotated_class.py", + """ + from pathlib import Path + + class TestExample: + def test_path(self, tmp_path: Path) -> None: + assert tmp_path.is_dir() + """, + migration_relative="test_annotated_class.py", + ) + + bundle = convert_pytest_suite((source,)) + + assert not bundle.blocking_diagnostics + tree = ast.parse(bundle.artifacts[0].content) + wrapper = next( + statement + for statement in tree.body + if isinstance(statement, ast.FunctionDef) + and statement.name.startswith("test_TestExample__") + ) + assert wrapper.returns is None + assert all( + argument.annotation is None + for argument in (*wrapper.args.posonlyargs, *wrapper.args.args, *wrapper.args.kwonlyargs) + ) + collection = materialize_and_discover(tmp_path, bundle) + assert not collection.issues + assert execute_tests(collection.items)[0].status is Status.PASS + + +@pytest.mark.parametrize( + "source", + [ + """ + def test_other_method(monkeypatch): + monkeypatch.delenv("NAME") + """, + """ + def test_alias(monkeypatch): + patcher = monkeypatch + patcher.setenv("NAME", "value") + """, + """ + def consume(value): + return value + def test_passed(monkeypatch): + consume(monkeypatch) + """, + """ + def test_object_read(monkeypatch): + assert monkeypatch + """, + """ + def test_method_read(monkeypatch): + callback = monkeypatch.setenv + callback("NAME", "value") + """, + """ + import pytest + @pytest.fixture + def configured(monkeypatch): + monkeypatch.chdir(".") + def test_fixture(configured): + pass + """, + """ + class TestExample: + def test_method(self, monkeypatch): + monkeypatch.setitem({}, "key", "value") + """, + """ + from project_helpers import configure + def test_imported_helper(monkeypatch): + configure(monkeypatch) + """, + """ + def configure(patch): + patch.setenv("NAME", "value") + original = configure + configure = original + def test_rebound_helper(monkeypatch): + configure(monkeypatch) + """, + """ + def configure(patch): + patch.setenv("NAME", "value") + def test_locally_rebound_helper(monkeypatch): + configure = lambda value: None + configure(monkeypatch) + """, + """ + def configure(patch): + patch = None + def test_rebound_parameter(monkeypatch): + configure(monkeypatch) + """, + """ + def factory(): + return lambda value: None + def test_dynamic_helper(monkeypatch): + factory()(monkeypatch) + """, + ], + ids=[ + "unsupported-method", + "alias", + "passed-object", + "object-read", + "method-read", + "fixture-dependency", + "class-method", + "imported-helper", + "module-rebound-helper", + "locally-rebound-helper", + "helper-parameter-rebound", + "dynamic-helper", + ], +) +def test_builtin_monkeypatch_usage_is_fail_closed(source: str) -> None: + selected = source_file( + "tests/test_monkeypatch_fail_closed.py", + source, + migration_relative="test_monkeypatch_fail_closed.py", + ) + + bundle = convert_pytest_suite((selected,)) + + diagnostic = next( + diagnostic + for diagnostic in bundle.blocking_diagnostics + if diagnostic.code == "PYT214_MONKEYPATCH_USAGE" + ) + assert diagnostic.line is not None + assert not bundle.artifacts + assert not bundle.mappings + + +@pytest.mark.parametrize( + ("source", "expected_code"), + [ + ( + """ + class TestExample: + def test_value(self, value=1): + assert value == 1 + """, + "PYT316_CLASS_SIGNATURE", + ), + ( + """ + class TestExample: + def test_value(self, *, value=1): + assert value == 1 + """, + "PYT316_CLASS_SIGNATURE", + ), + ( + """ + class TestExample: + def helper(self): + pass + setup_method = helper + def test_value(self): + pass + """, + "PYT314_CLASS_LIFECYCLE", + ), + ( + """ + class TestExample: + __init__ = object.__init__ + def test_value(self): + pass + """, + "PYT314_CLASS_LIFECYCLE", + ), + ( + """ + class TestExample: + __new__ = object.__new__ + def test_value(self): + pass + """, + "PYT314_CLASS_LIFECYCLE", + ), + ( + """ + class TestExample: + setup_method = teardown_method = lambda self: None + def test_value(self): + pass + """, + "PYT314_CLASS_LIFECYCLE", + ), + ( + """ + class TestExample: + pytestmark = marker_alias = object() + def test_value(self): + pass + """, + "PYT313_CLASS_MARK", + ), + ], + ids=[ + "positional-default", + "keyword-default", + "aliased-setup", + "aliased-init", + "aliased-new", + "multi-target-lifecycle", + "multi-target-pytestmark", + ], +) +def test_simple_class_conversion_rejects_implicit_semantic_bindings( + source: str, + expected_code: str, +) -> None: + selected = source_file( + "tests/test_class_fail_closed.py", + source, + migration_relative="test_class_fail_closed.py", + ) + + bundle = convert_pytest_suite((selected,)) + + assert expected_code in {diagnostic.code for diagnostic in bundle.blocking_diagnostics} + assert not bundle.artifacts + assert not bundle.mappings + + @pytest.mark.parametrize( ("name", "source", "expected_code"), [ ( - "class", + "class-inheritance", + """ + class Base: + pass + class TestExample(Base): + def test_value(self): + pass + """, + "PYT311_CLASS_INHERITANCE", + ), + ( + "class-lifecycle", """ class TestExample: + def setup_method(self): + self.value = 1 def test_value(self): + assert self.value == 1 + """, + "PYT314_CLASS_LIFECYCLE", + ), + ( + "class-staticmethod", + """ + class TestExample: + @staticmethod + def test_value(): pass """, - "PYT301_CLASS_TEST", + "PYT316_CLASS_SIGNATURE", ), ( "xfail", @@ -371,8 +1152,8 @@ def test_value(): ( "builtin-fixture", """ - def test_value(tmp_path): - assert tmp_path.exists() + def test_value(capsys): + assert capsys """, "PYT209_BUILTIN_FIXTURE", ), @@ -388,7 +1169,8 @@ def test_value(database): "autouse", """ import pytest - @pytest.fixture(autouse=True) + ENABLED = True + @pytest.fixture(autouse=ENABLED) def state(): return object() def test_value(): @@ -432,6 +1214,14 @@ def test_value(value): """, "PYT209_BUILTIN_FIXTURE", ), + ( + "event-loop-policy-request", + """ + def test_value(event_loop_policy): + assert event_loop_policy + """, + "PYT209_BUILTIN_FIXTURE", + ), ( "runtime-skip", """ @@ -519,7 +1309,55 @@ def test_value(): "async-plugin", """ import pytest + @pytest.mark.asyncio() + async def test_value(): + pass + """, + "PYT502_ASYNC_PLUGIN", + ), + ( + "unmarked-async-test", + """ + async def test_value(): + pass + """, + "PYT508_UNMARKED_ASYNC_TEST", + ), + ( + "unmarked-async-class-test", + """ + class TestAsync: + async def test_value(self): + pass + """, + "PYT508_UNMARKED_ASYNC_TEST", + ), + ( + "asyncio-runtime-alias-collision", + """ + import pytest + _testenix_isolated_asyncio = object() + @pytest.mark.asyncio + async def test_value(): + pass + """, + "PYT008_GENERATED_IMPORT_COLLISION", + ), + ( + "asyncio-on-sync-test", + """ + import pytest @pytest.mark.asyncio + def test_value(): + pass + """, + "PYT502_ASYNC_PLUGIN", + ), + ( + "anyio-plugin", + """ + import pytest + @pytest.mark.anyio async def test_value(): pass """, @@ -575,3 +1413,36 @@ def test_renamed_target_collision_blocks_both_sources() -> None: } assert not bundle.artifacts assert not bundle.mappings + + +def test_ancestor_conftest_autouse_remains_blocked_until_package_imports_are_supported() -> None: + ancestor = source_file( + "tests/conftest.py", + """ + import pytest + + @pytest.fixture(autouse=True) + def inherited_state(): + return object() + """, + migration_relative="tests/conftest.py", + ) + nested_test = source_file( + "tests/unit/test_nested.py", + """ + def test_nested() -> None: + pass + """, + migration_relative="unit/test_nested.py", + ) + + bundle = convert_pytest_suite((nested_test,), (ancestor,)) + + diagnostic = next( + diagnostic + for diagnostic in bundle.blocking_diagnostics + if diagnostic.code == "PYT213_ANCESTOR_CONFTEST" + ) + assert "inherited_state" in diagnostic.message + assert all(artifact.relative_path.name.startswith("_") for artifact in bundle.artifacts) + assert not bundle.mappings diff --git a/tests/test_migration_pytest_config.py b/tests/test_migration_pytest_config.py new file mode 100644 index 0000000..7684256 --- /dev/null +++ b/tests/test_migration_pytest_config.py @@ -0,0 +1,399 @@ +from __future__ import annotations + +import hashlib +from collections.abc import Mapping +from pathlib import Path +from textwrap import dedent + +import pytest + +from testenix.migration_models import MigrationDiagnostic, SourceFile +from testenix.migration_pytest_config import pytest_asyncio_config_diagnostics +from testenix.migration_service import MigrationOptions, MigrationStatus, migrate + + +def _project(tmp_path: Path, files: Mapping[str, str]) -> Path: + root = tmp_path / "project" + root.mkdir() + for relative, content in files.items(): + destination = root / relative + destination.parent.mkdir(parents=True, exist_ok=True) + destination.write_text(dedent(content).lstrip(), encoding="utf-8") + return root + + +def _source(root: Path, relative: str = "tests/test_async.py") -> SourceFile: + path = root / relative + payload = path.read_bytes() + return SourceFile( + path=path, + project_relative=Path(relative), + migration_relative=Path(Path(relative).name), + sha256=hashlib.sha256(payload).hexdigest(), + text=payload.decode("utf-8"), + ) + + +def _diagnostics( + root: Path, + *, + pytest_major: int = 9, + environ: Mapping[str, str] | None = None, +) -> tuple[MigrationDiagnostic, ...]: + return pytest_asyncio_config_diagnostics( + project_root=root, + source_paths=(root / "tests",), + files=(_source(root),), + environ={} if environ is None else environ, + pytest_major=pytest_major, + ) + + +_BARE_ASYNC_TEST = """ + import asyncio + import pytest + + @pytest.mark.asyncio + async def test_poll_once() -> None: + await asyncio.sleep(0) +""" + + +def test_solanabot_like_bare_marker_uses_safe_defaults_without_config( + tmp_path: Path, +) -> None: + root = _project(tmp_path, {"tests/test_async.py": _BARE_ASYNC_TEST}) + + assert _diagnostics(root) == () + + +def test_sync_only_inventory_ignores_unrelated_asyncio_configuration( + tmp_path: Path, +) -> None: + root = _project( + tmp_path, + { + "tests/test_async.py": "def test_sync():\n assert True\n", + "pytest.ini": "[pytest]\nasyncio_default_test_loop_scope = session\n", + }, + ) + + assert _diagnostics(root) == () + + +def test_pytest_ini_precedes_pyproject_in_the_same_directory(tmp_path: Path) -> None: + root = _project( + tmp_path, + { + "tests/test_async.py": _BARE_ASYNC_TEST, + "pytest.ini": "[pytest]\nasyncio_default_test_loop_scope = module\n", + "pyproject.toml": """ + [tool.pytest.ini_options] + asyncio_default_test_loop_scope = "function" + """, + }, + ) + + diagnostics = _diagnostics(root, pytest_major=8) + + assert [diagnostic.code for diagnostic in diagnostics] == ["PYT508_ASYNCIO_CONFIG"] + assert diagnostics[0].source == "pytest.ini" + assert "'module'" in diagnostics[0].message + + +def test_tox_ini_precedes_setup_cfg_in_the_same_directory(tmp_path: Path) -> None: + root = _project( + tmp_path, + { + "tests/test_async.py": _BARE_ASYNC_TEST, + "tox.ini": "[pytest]\nasyncio_default_test_loop_scope = function\n", + "setup.cfg": "[tool:pytest]\nasyncio_debug = true\n", + }, + ) + + assert _diagnostics(root, pytest_major=8) == () + + +def test_nested_config_is_ignored_when_baseline_runs_from_project_root( + tmp_path: Path, +) -> None: + root = _project( + tmp_path, + { + "tests/test_async.py": _BARE_ASYNC_TEST, + "tests/setup.cfg": ("[tool:pytest]\nasyncio_default_test_loop_scope = package\n"), + "pytest.ini": "[pytest]\nasyncio_default_test_loop_scope = function\n", + }, + ) + + assert _diagnostics(root, pytest_major=8) == () + + +def test_empty_pytest_ini_masks_later_pyproject_configuration(tmp_path: Path) -> None: + root = _project( + tmp_path, + { + "tests/test_async.py": _BARE_ASYNC_TEST, + "pytest.ini": "", + "pyproject.toml": """ + [tool.pytest.ini_options] + asyncio_default_test_loop_scope = "session" + """, + }, + ) + + assert _diagnostics(root, pytest_major=8) == () + + +def test_pytest_nine_toml_precedence_is_not_applied_to_pytest_eight( + tmp_path: Path, +) -> None: + root = _project( + tmp_path, + { + "tests/test_async.py": _BARE_ASYNC_TEST, + "pytest.toml": "[pytest]\nasyncio_debug = true\n", + "pytest.ini": "[pytest]\nasyncio_debug = false\n", + }, + ) + + pytest_nine = _diagnostics(root, pytest_major=9) + + assert [diagnostic.code for diagnostic in pytest_nine] == ["PYT508_ASYNCIO_CONFIG"] + assert pytest_nine[0].source == "pytest.toml" + assert _diagnostics(root, pytest_major=8) == () + + +def test_alias_marker_is_included_in_async_inventory(tmp_path: Path) -> None: + root = _project( + tmp_path, + { + "tests/test_async.py": """ + import pytest as pt + + @pt.mark.asyncio + async def test_alias() -> None: + pass + """, + "pyproject.toml": """ + [tool.pytest.ini_options] + asyncio_debug = true + """, + }, + ) + + diagnostics = _diagnostics(root, pytest_major=9) + + assert [diagnostic.code for diagnostic in diagnostics] == ["PYT508_ASYNCIO_CONFIG"] + + +def test_malformed_selected_configuration_fails_closed(tmp_path: Path) -> None: + root = _project( + tmp_path, + { + "tests/test_async.py": _BARE_ASYNC_TEST, + "pytest.ini": "this is not an ini section\n", + }, + ) + + diagnostics = _diagnostics(root) + + assert [diagnostic.code for diagnostic in diagnostics] == ["PYT509_PYTEST_CONFIG"] + assert diagnostics[0].source == "pytest.ini" + + +def test_invalid_selected_value_reports_its_config_file(tmp_path: Path) -> None: + root = _project( + tmp_path, + { + "tests/test_async.py": _BARE_ASYNC_TEST, + "pytest.ini": "[pytest]\nasyncio_default_test_loop_scope = process\n", + }, + ) + + diagnostics = _diagnostics(root) + + assert [diagnostic.code for diagnostic in diagnostics] == ["PYT509_PYTEST_CONFIG"] + assert diagnostics[0].source == "pytest.ini" + + +def test_ini_addopts_enabling_asyncio_debug_fails_closed(tmp_path: Path) -> None: + root = _project( + tmp_path, + { + "tests/test_async.py": _BARE_ASYNC_TEST, + "pytest.ini": "[pytest]\naddopts = -q --asyncio-debug\n", + }, + ) + + diagnostics = _diagnostics(root) + + assert [diagnostic.code for diagnostic in diagnostics] == ["PYT508_ASYNCIO_CONFIG"] + assert diagnostics[0].source == "pytest.ini" + assert "pytest.ini addopts --asyncio-debug" in diagnostics[0].message + + +def test_native_toml_addopts_changing_loop_scope_fails_closed(tmp_path: Path) -> None: + root = _project( + tmp_path, + { + "tests/test_async.py": _BARE_ASYNC_TEST, + "pytest.toml": """ + [pytest] + addopts = ["-q", "-o", "asyncio_default_test_loop_scope=module"] + """, + }, + ) + + diagnostics = _diagnostics(root, pytest_major=9) + + assert [diagnostic.code for diagnostic in diagnostics] == ["PYT508_ASYNCIO_CONFIG"] + assert diagnostics[0].source == "pytest.toml" + assert "pytest.toml addopts" in diagnostics[0].message + + +def test_environment_override_takes_precedence_over_config_addopts( + tmp_path: Path, +) -> None: + root = _project( + tmp_path, + { + "tests/test_async.py": _BARE_ASYNC_TEST, + "pytest.ini": ("[pytest]\naddopts = -o asyncio_default_test_loop_scope=session\n"), + }, + ) + + diagnostics = _diagnostics( + root, + environ={"PYTEST_ADDOPTS": "-o asyncio_default_test_loop_scope=function"}, + ) + + assert diagnostics == () + + +def test_outside_environment_path_matches_real_pytest_config_discovery( + tmp_path: Path, +) -> None: + from _pytest.config.findpaths import determine_setup + + root = _project( + tmp_path, + { + "tests/test_async.py": _BARE_ASYNC_TEST, + "pytest.ini": "[pytest]\nasyncio_default_test_loop_scope = function\n", + }, + ) + outside = tmp_path / "empty" + outside.mkdir() + parent_config = tmp_path / "pytest.ini" + parent_config.write_text( + "[pytest]\nasyncio_default_test_loop_scope = session\n", + encoding="utf-8", + ) + + real_setup = determine_setup( + inifile=None, + override_ini=None, + args=[str(outside), str(root / "tests")], + rootdir_cmd_arg=None, + invocation_dir=root, + ) + diagnostics = _diagnostics(root, environ={"PYTEST_ADDOPTS": "../empty"}) + + assert real_setup[1] == parent_config + assert [diagnostic.code for diagnostic in diagnostics] == ["PYT509_PYTEST_CONFIG"] + assert diagnostics[0].source == "" + assert "outside the project root" in diagnostics[0].message + + +def test_in_root_nodeid_and_normal_option_values_do_not_false_positive( + tmp_path: Path, +) -> None: + root = _project(tmp_path, {"tests/test_async.py": _BARE_ASYNC_TEST}) + outside_basetemp = tmp_path / "pytest-temp" + outside_basetemp.mkdir() + + diagnostics = _diagnostics( + root, + environ={ + "PYTEST_ADDOPTS": ( + "-k 'not slow' --tb short --maxfail 1 " + "--basetemp ../pytest-temp tests/test_async.py::test_poll_once" + ) + }, + ) + + assert diagnostics == () + + +@pytest.mark.parametrize("addopts", ["-o", "--override-ini malformed", "-c other.ini"]) +def test_malformed_or_explicit_config_addopts_reports_selected_config( + tmp_path: Path, + addopts: str, +) -> None: + root = _project( + tmp_path, + { + "tests/test_async.py": _BARE_ASYNC_TEST, + "pytest.ini": f"[pytest]\naddopts = {addopts}\n", + }, + ) + + diagnostics = _diagnostics(root) + + assert [diagnostic.code for diagnostic in diagnostics] == ["PYT509_PYTEST_CONFIG"] + assert diagnostics[0].source == "pytest.ini" + + +@pytest.mark.parametrize( + ("addopts", "expected_code"), + [ + ("-q --tb=short", None), + ("-o asyncio_default_test_loop_scope=function -o asyncio_debug=false", None), + ("-o asyncio_default_test_loop_scope=class", "PYT508_ASYNCIO_CONFIG"), + ("-o asyncio_debug=true", "PYT508_ASYNCIO_CONFIG"), + ("--asyncio-debug", "PYT508_ASYNCIO_CONFIG"), + ("-c alternate.ini", "PYT509_PYTEST_CONFIG"), + ("-o", "PYT509_PYTEST_CONFIG"), + ("--override-ini malformed", "PYT509_PYTEST_CONFIG"), + ], +) +def test_relevant_pytest_addopts_overrides_fail_closed( + tmp_path: Path, + addopts: str, + expected_code: str | None, +) -> None: + root = _project(tmp_path, {"tests/test_async.py": _BARE_ASYNC_TEST}) + + diagnostics = _diagnostics(root, environ={"PYTEST_ADDOPTS": addopts}) + + assert ([diagnostic.code for diagnostic in diagnostics] or [None]) == [expected_code] + + +def test_migration_service_applies_guard_before_shadow_validation( + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.delenv("PYTEST_ADDOPTS", raising=False) + root = _project( + tmp_path, + { + "tests/test_async.py": _BARE_ASYNC_TEST, + "pytest.ini": "[pytest]\nasyncio_default_test_loop_scope = session\n", + }, + ) + + report = migrate( + MigrationOptions( + framework="pytest", + sources=(Path("tests"),), + output=Path("converted"), + workers=2, + dry_run=True, + project_root=root, + ) + ) + + assert report.status is MigrationStatus.UNSUPPORTED + assert report.baseline is None + assert "PYT508_ASYNCIO_CONFIG" in {diagnostic.code for diagnostic in report.diagnostics} diff --git a/tests/test_migration_service.py b/tests/test_migration_service.py index cbe8277..95a9162 100644 --- a/tests/test_migration_service.py +++ b/tests/test_migration_service.py @@ -22,6 +22,7 @@ _candidate_problem, _run_process, migrate, + render_migration_summary, ) @@ -264,6 +265,10 @@ def unexpected_process(*args: object, **kwargs: object) -> object: assert report.native_serial is None assert report.native_parallel is None assert not report.published + assert not any(diagnostic.code == "MIG006" for diagnostic in report.diagnostics) + summary = render_migration_summary(report) + assert "analyzed candidate: 1 tests in 1 files" in summary + assert " converted:" not in summary assert not (project / "converted").exists() assert source_bytes(project) == before @@ -290,10 +295,9 @@ def test_unsupported_source_exits_four_without_output(tmp_path: Path) -> None: project = write_project( tmp_path, { - "tests/test_class.py": """ - class TestUnsupported: - def test_method(self) -> None: - pass + "tests/test_capture.py": """ + def test_capture(capsys) -> None: + pass """, }, ) @@ -303,12 +307,45 @@ def test_method(self) -> None: assert report.status is MigrationStatus.UNSUPPORTED assert report.exit_code == 4 - assert any(diagnostic.code == "PYT301_CLASS_TEST" for diagnostic in report.diagnostics) + assert any(diagnostic.code == "PYT209_BUILTIN_FIXTURE" for diagnostic in report.diagnostics) assert not report.published assert not (project / "converted").exists() assert source_bytes(project) == before +def test_unsupported_summary_groups_diagnostics_and_names_partial_work_honestly( + tmp_path: Path, +) -> None: + project = write_project( + tmp_path, + { + "tests/test_plain.py": "def test_plain():\n assert True\n", + "tests/test_capture.py": """ + def test_first(capsys) -> None: + pass + + def test_second(capsys) -> None: + pass + """, + }, + ) + + report = migrate(options(project, "pytest", dry_run=True)) + summary = render_migration_summary(report) + document = report.to_dict() + + assert report.status is MigrationStatus.UNSUPPORTED + assert report.converted_tests == 1 + assert not any(diagnostic.code == "MIG006" for diagnostic in report.diagnostics) + assert "statically convertible subset: 1 tests in 1 files" in summary + assert " converted:" not in summary + assert "PYT209_BUILTIN_FIXTURE: 2 occurrence(s) in 1 file(s)" in summary + assert summary.count("PYT209_BUILTIN_FIXTURE") == 1 + assert "--report-json FILE|- retains every line-addressed entry" in summary + assert len(document["diagnostics"]) == 2 + assert {entry["line"] for entry in document["diagnostics"]} == {1, 4} + + def test_failing_source_baseline_exits_one_without_output(tmp_path: Path) -> None: project = write_project( tmp_path, diff --git a/uv.lock b/uv.lock index afd708d..6745652 100644 --- a/uv.lock +++ b/uv.lock @@ -1351,7 +1351,7 @@ wheels = [ [[package]] name = "testenix" -version = "0.1.0" +version = "0.2.0" source = { editable = "." } [package.optional-dependencies]