Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ project intends to use Semantic Versioning once its public API reaches stability

## [Unreleased]

## [0.2.0] - 2026-07-20

### Added

- `testenix pytest [PYTEST_ARGS ...]` compatibility bridge for unchanged pytest suites, preserving
Expand All @@ -28,6 +30,25 @@ project intends to use Semantic Versioning once its public API reaches stability
- Truthful post-commit durability/report warnings, package-aware unittest outcome mapping,
Testenix validation-worker containment, and conservative blocking of pytest session fixtures and
unittest class-cleanup hooks whose lifecycle cannot be preserved.
- Native `tmp_path` and transactional `monkeypatch` fixtures. The initial monkeypatch contract
covers the object/attribute and dotted-import forms of `setattr`, plus `setenv`, with automatic
per-test rollback. Static module-local helper calls are accepted only when every propagated use
can be proven safe; aliases, dynamic rebinding, unsupported methods, and escaped values remain
blocked.
- Safe conversion of bare `@pytest.mark.asyncio` coroutine tests, simple pytest classes through
fresh-instance wrappers, and statically declared autouse fixtures. Async migration creates and
closes an isolated `asyncio.Runner` per test or case, validates effective pytest-asyncio loop and
debug configuration, and blocks custom event-loop policies or unmarked async semantics.
- Fail-closed class conversion for lifecycle hooks, decorated or inherited classes, annotated
class state, custom constructors, and method defaults that cannot be preserved by wrappers.

### Changed

- Migration console output now distinguishes analyzed, validated, generated, and published
candidates. Repeated diagnostics are grouped by code, while JSON audit reports retain every
source- and line-addressed entry.
- The one-affinity-unit `MIG006` warning is emitted only for statically supported check/publication
candidates, not for dry-run or already-blocked migrations.

## [0.1.0] - 2026-07-20

Expand Down
36 changes: 24 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,11 @@ testenix run tests_testenix
For pytest migration, install `testenix[pytest]`; unittest migration uses the standard library.
`auto` supports pytest and unittest in separate modules within one selection.

Large unsupported suites get a grouped console summary instead of one wall of repeated lines.
Use `--report-json FILE` or `--report-json -` to retain every individual diagnostic with its source
and line. A blocked run labels safe in-memory mappings as a *statically convertible subset*; only a
report with `status: published` means that Testenix created the requested output.

This is a copy-and-validate transaction, not an in-place rewrite. Testenix fingerprints the
sources, generates into private staging, runs the green original suite in a disposable project
copy, runs the native candidate with one worker and again in parallel, compares inventory and
Expand All @@ -111,13 +116,19 @@ rename, Testenix warns without pretending the already published output was rolle
paths must be new, inside the project, and disjoint from both source and generated suites. There is
no `--force` option, and old tests are never deleted or renamed.

The converter stops on semantics it cannot preserve. Its current pytest subset covers module
functions, one static parametrization, simple local/adjacent-conftest fixtures, static skips, and
plain markers. The unittest adapter preserves per-test lifecycle and assertions by generating
native wrappers around the original `TestCase.run()` protocol; those wrappers locate originals
independently of `cwd` and verify the complete selected-Python-source SHA-256 manifest, so the old
unittest files must remain present. Keep the generated unittest directory at its published path as
well; rerun migration after moving either side.
The converter stops on semantics it cannot preserve. The v0.2 pytest subset covers module
functions, one static parametrization, simple local/adjacent-conftest fixtures, statically declared
autouse fixtures, bare `@pytest.mark.asyncio` coroutine tests through fresh function-scoped loop
wrappers, and simple pytest classes. Native `tmp_path` and a dependency-free `monkeypatch`
implementation cover the common `setattr` and `setenv` forms with automatic per-test rollback,
including calls through statically provable module-local helpers. Complex class lifecycle, async
fixtures, unmarked async tests, configured async loop scopes or debug mode, custom
`event_loop_policy`, and the rest of pytest's built-in fixtures remain blocked. The unittest adapter
preserves per-test lifecycle and assertions by generating native wrappers around the original
`TestCase.run()` protocol; those wrappers locate originals independently of `cwd` and verify the
complete selected-Python-source SHA-256 manifest, so the old unittest files must remain present.
Keep the generated unittest directory at its published path as well; rerun migration after moving
either side.

See the full [safe migration guide](https://polishdataengineer.github.io/testenix/guides/migration/)
for the support matrix, rollback contract, CI rollout, audit-report schema, and performance
Expand Down Expand Up @@ -218,7 +229,7 @@ test.

## Where Testenix is deliberately different

The `testenix run` engine is not a native drop-in reimplementation of pytest. Its v0.1 value is a
The `testenix run` engine is not a native drop-in reimplementation of pytest. Its v0.2 value is a
smaller, coherent native stack: async tests and async fixtures need no plugin, parallel execution
and duration-aware scheduling need no xdist, every retry remains visible, and a worker crash cannot
silently erase tests that completed before it. The native runtime has no third-party dependencies.
Expand Down Expand Up @@ -287,7 +298,8 @@ See the [generated results and chart](https://polishdataengineer.github.io/teste
JSON-safe representation when a value itself is not serializable.
- Synchronous test and fixture bodies run outside Testenix's internal asyncio loop. APIs restricted
to Python's main thread, such as installing signal handlers, are not supported inside those
bodies in v0.1.
bodies in v0.2. Migrated pytest-asyncio wrappers are synchronous from Testenix's perspective and
therefore share this restriction while creating a fresh event loop for each test or case.
- On Windows, a script that calls the programmatic `run()`/`run_async()` API must use the standard
`if __name__ == "__main__":` multiprocessing guard. The `testenix` CLI handles process startup
itself.
Expand All @@ -300,12 +312,12 @@ See the [generated results and chart](https://polishdataengineer.github.io/teste
create 3,000 parallel units; spread independent tests across modules and measure the generated
suite before making a project-specific speed claim.
- Test impact analysis, result caching, remote workers, and deep pytest-result aggregation are not
part of version 0.1.
part of version 0.2.

## Project status

Testenix is pre-1.0 software. The distribution, import package, CLI, configuration namespace, and
state directory consistently use `testenix`. The project is licensed under MIT and its release
Testenix 0.2.0 is pre-1.0 software. The distribution, import package, CLI, configuration namespace,
and state directory consistently use `testenix`. The project is licensed under MIT and its release
workflow uses PyPI Trusted Publishing; the first PyPI release has not been published yet.

## Development
Expand Down
11 changes: 7 additions & 4 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Windows: testenix pytest =========> pytest.console_main -> collector/plugins/exe
```

The bridge is a CLI infrastructure adapter, not a native collection adapter. It does not emit
Testenix events or construct a `RunResult` in version 0.1.
Testenix events or construct a `RunResult` in version 0.2.

The migration adapter is separate from that handoff. It statically converts a deliberately small
pytest subset or generates SHA-pinned wrappers around the standard unittest protocol. Its
Expand Down Expand Up @@ -73,7 +73,7 @@ Authoring API -> supervised collection -> inert manifest -> affinity scheduler -
- migration analyzers depend on serializable migration contracts, while shadow execution and
atomic publication remain application/infrastructure concerns.

## Version 0.1 scope
## Version 0.2 scope

- explicit `@test` and `@fixture` authoring API, plus conventional `test_*` discovery;
- sync functions, coroutines, generators, and async-generator fixture teardown;
Expand All @@ -87,9 +87,12 @@ Authoring API -> supervised collection -> inert manifest -> affinity scheduler -
- an optional platform-aware pytest handoff for unchanged legacy suites.
- conservative pytest/unittest migration with static diagnostics, differential validation, source
fingerprints, and create-only publication.
- dependency-free `tmp_path` and reversible `monkeypatch` fixtures, plus native autouse resolution;
- conservative migration of bare pytest-asyncio coroutine markers through isolated fresh-loop
wrappers, plus simple pytest classes.

Remote workers, distributed storage, result caching, automatic quarantine, and a stable third-party
plugin SDK are deliberately outside version 0.1.
plugin SDK are deliberately outside version 0.2.

## Fixture scopes and process isolation

Expand All @@ -98,7 +101,7 @@ between parallel shared workers. Multiple modules assigned to one shard execute
process and fixture runtime. A test with an explicit timeout (including a global timeout applied at
selection) is instead a single-test isolation unit with a hard process deadline.

Scope therefore has the following concrete meaning in version 0.1:
Scope therefore has the following concrete meaning in version 0.2:

| Scope | Lifetime |
| --- | --- |
Expand Down
37 changes: 36 additions & 1 deletion docs/guides/fixtures.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,41 @@ def test_empty_repository(repository: Repository) -> None:
The dependency graph is validated before execution. Missing fixtures and cycles become collection
issues instead of hanging the run.

## Built-in fixtures

Testenix 0.2 provides two dependency-free, test-scoped built-ins by name:

```python
from pathlib import Path


def test_isolated_file(tmp_path: Path, monkeypatch) -> None:
target = tmp_path / "value.txt"
target.write_text("ok", encoding="utf-8")
monkeypatch.setenv("TESTENIX_EXAMPLE", "enabled")
assert target.read_text(encoding="utf-8") == "ok"
```

`tmp_path` is a fresh `pathlib.Path` removed during teardown. `monkeypatch` supports reversible
object/attribute and dotted-import `setattr`, `setenv`, and idempotent `undo`. Changes are restored
in LIFO order even when the test fails. Other pytest monkeypatch operations and pytest built-ins
such as `capsys`, `caplog`, and `request` are not native Testenix fixtures.

## Autouse fixtures

Use `autouse=True` when setup and cleanup must apply to every test that can see a fixture:

```python
@fixture(autouse=True)
def isolated_environment(monkeypatch):
monkeypatch.setenv("APP_ENV", "test")
yield
```

Explicitly requesting the same fixture still resolves one cached value for the test. A local
fixture definition overrides a visible imported definition with the same name before Testenix
chooses which fixtures run automatically.

## Scopes

```python
Expand All @@ -75,7 +110,7 @@ def worker_resource() -> Resource:
return Resource()
```

| Scope | Lifetime in Testenix 0.1 |
| Scope | Lifetime in Testenix 0.2 |
| --- | --- |
| `test` | One instance for one concrete test attempt. |
| `module` | Shared by normal tests from the module inside one worker. |
Expand Down
46 changes: 36 additions & 10 deletions docs/guides/migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,12 @@ They must be separate modules; a file that mixes both test models is rejected.
The destination defaults to `testenix_migrated`. It must be a new directory inside the project,
with an existing real parent. There is deliberately no `--force` and no in-place mode.
An integer `--workers` value must be at least 2 so the parallel gate cannot silently repeat the
serial command. A one-module suite still has one schedulable affinity unit, which is disclosed as a
`MIG006` warning; spread tests across independent modules to exercise multiple workers. An
audit-report path must also be new, inside the project, and disjoint from every selected source and
the output suite. Testenix never replaces an existing report.
serial command. During `--check` or publication, a one-module candidate still has one schedulable
affinity unit, which is disclosed as a `MIG006` warning; spread tests across independent modules to
exercise multiple workers. The warning is not shown for `--dry-run` or an already-blocked migration
because no parallel candidate is run in either case. An audit-report path must also be new, inside
the project, and disjoint from every selected source and the output suite. Testenix never replaces
an existing report.

## Transaction and rollback contract

Expand Down Expand Up @@ -89,28 +91,47 @@ with external effects.

## Pytest conversion contract

The current converter supports the subset below:
The v0.2 converter supports the subset below:

- module-level pytest-default `test*` functions and normal Python `assert` statements;
- simple `Test*` classes with a fresh zero-argument instance per test method, including ordinary
helper methods; inheritance, class decorators, custom construction, and pytest class lifecycle
hooks remain outside the safe subset;
- bare `@pytest.mark.asyncio` on `async def` tests. The generated synchronous wrapper runs every
test or parametrized case in a fresh, closed `asyncio.Runner`, matching pytest-asyncio's default
function-scoped loop isolation;
- one static `pytest.mark.parametrize` with static names, rows, IDs, and unmarked
`pytest.param(..., id=...)` values;
- local fixtures with no parameters, autouse, or `request`, using function or module scope;
- local fixtures using function or module scope, including a statically boolean `autouse=True`;
- simple fixtures from an adjacent `conftest.py` in the same directory;
- native `tmp_path`, which supplies a fresh `pathlib.Path` and removes its temporary directory at
test teardown;
- native `monkeypatch.setattr` in object/attribute and dotted-import forms, plus `setenv` and
idempotent `undo`; successful changes are restored in LIFO order during test teardown.
`monkeypatch` may also flow through statically resolved module-local helpers when every use can
be proven to stay inside this supported subset;
- static `pytest.mark.skip` and `pytest.mark.skipif`;
- plain argument-free custom markers, converted to Testenix tags;
- pytest runtime helpers `approx`, `deprecated_call`, `fail`, `raises`, and `warns`. Generated
modules using these helpers still require pytest at runtime.

It blocks, with a file and line diagnostic:

- pytest test classes, xfail, runtime skip/xfail/importorskip/exit, xunit lifecycle hooks;
- built-in fixtures such as `tmp_path`, `monkeypatch`, `capsys`, and `request`;
- autouse or parametrized fixtures, fixture overrides, and inherited ancestor-`conftest` fixtures;
- complex pytest test classes, xfail, runtime skip/xfail/importorskip/exit, and xunit lifecycle
hooks;
- built-in fixtures other than `tmp_path` and `monkeypatch`, such as `capsys`, `caplog`, and
`request`; monkeypatch operations outside the documented native subset, imported or dynamically
rebound helpers, and values that escape static analysis are also unsupported;
- dynamically configured autouse fixtures, parametrized fixtures, fixture overrides, and inherited
ancestor-`conftest` fixtures;
- session-scoped fixtures, because pytest creates one per run while Testenix session scope is
currently worker-local;
- stacked, dynamic, indirect, scoped, or per-case-marked parametrization;
- `usefixtures`, module-level `pytestmark`, hook functions, plugin registration, and semantic
plugin markers such as asyncio/anyio, timeout, order, repeat, or flaky;
plugin markers such as anyio, configured asyncio, timeout, order, repeat, or flaky;
- unmarked async tests, async fixtures, custom `event_loop_policy`, non-function asyncio loop
scopes, and enabled asyncio debug mode. Testenix checks the effective pytest configuration and
relevant `PYTEST_ADDOPTS` overrides before accepting bare asyncio markers;
- decorators and required parameters whose execution meaning cannot be established statically.

Any converted pytest file whose name is not already `test_*.py` is renamed in the generated copy
Expand Down Expand Up @@ -167,6 +188,11 @@ hashes, every source-to-target mapping, per-test outcomes, generated files, line
timings and summaries for all validation runs, publication status, and an `originals_modified`
flag. It is false for a successful transaction and true when a terminal source recheck detects
drift; the flag reports observed state and does not claim that Testenix caused an independent edit.
The console groups repeated diagnostics by code and shows the first location, so large suites do
not produce hundreds of near-identical lines. `--report-json FILE` and `--report-json -` always
retain every individual source- and line-addressed diagnostic. On a blocked transaction, the
console calls any safe in-memory result a *statically convertible subset* rather than implying that
those tests were published.

## Performance with thousands of migrated tests

Expand Down
Loading
Loading