Skip to content

Stage 1: base preprocessor contract, docs cleanup and license - #1

Merged
akifkulll merged 3 commits into
mainfrom
feat/base-preprocessor
Oct 17, 2026
Merged

akifkulll merged 3 commits into
mainfrom
feat/base-preprocessor

Conversation

@akifkulll

Copy link
Copy Markdown
Collaborator

Closes out stage 1: the shared BasePreprocessor contract, its tests and CI. After this lands, the blocks (imputer, encoder, scaler) and Pipeline can be built in parallel on top of it.

Where the code is

BasePreprocessor, its tests and the CI workflow were pushed to main directly before this PR existed, so they do not show up in this diff. Review them here:

bf783a9...a8c1191: everything the base commit added (src/preprocessing_toolkit/base.py, tests/test_base.py, .github/workflows/ci.yml, pyproject.toml, CONTRIBUTING.md).

That push also turned CI red on main: ruff format --check failed on the README example code, so the tests never ran. This PR fixes that.

What this PR changes

  • Repository hygiene: .claude/settings.json (per-developer local settings) is no longer tracked and is now in .gitignore, together with common Python build and cache directories.
  • License: adds the MIT LICENSE that the README already linked to.
  • One README: README_preprocessing_toolkit_EN.md was an identical copy of README.md and is removed.
  • README content:
    • adds a "v0.1 in development" status note;
    • marks the quick start as the planned API;
    • installs from source with the real repository URL (the package is not on PyPI yet);
    • fixes the broken Octacity link;
    • formats the example so ruff passes.
  • v0.1 scope in the README: the option lists now match the planned v0.1 issues. The encoder offers onehot and ordinal with handle_unknown; the scaler offers standard and minmax. Label encoding is dropped and robust stays on the v0.2 roadmap.

Design decisions in BasePreprocessor

  • Compatible with scikit-learn, but no dependency on it. Runtime dependencies are only numpy and pandas. Compatibility comes from following scikit-learn's conventions, not from importing it:

    • get_params / set_params read the parameters from the __init__ signature;
    • __sklearn_is_fitted__ reports whether the block is fitted;
    • __sklearn_tags__ describes the block as an unsupervised transformer. It imports scikit-learn inside the method, and only scikit-learn ≥ 1.6 calls it. Its docstring notes that it must be updated if scikit-learn changes the Tags structure.

    Tests run blocks through clone, cross_val_score and GridSearchCV. A CI row on Python 3.10 with scikit-learn 1.3 (before the tags API existed) checks that blocks also work without tags.

  • Template method. The public fit / transform / fit_transform methods run the shared checks. Subclasses implement only _fit and _transform. __init__ must store its arguments unchanged and must not use *args / **kwargs; the class raises an error if it does. Everything a block learns in _fit goes into attributes ending in _.

  • Column policy. Input must be a pandas DataFrame without duplicate column names. transform requires exactly the columns seen in fit. Missing and unexpected columns are reported together in one error. Columns in a different order are put back into the fit order.

  • A failed fit leaves the block unfitted. fit clears the fitted state before it starts. If _fit raises, or the input is rejected, the block reports itself as not fitted and transform raises NotFittedError. It never keeps using a half-updated state. NotFittedError subclasses both ValueError and AttributeError, like scikit-learn's.

  • English only. Code, docstrings, error messages, commits and PRs are in English, as CONTRIBUTING.md requires.

Verification

  • pytest: 32 passed (30 test functions; one is parametrized over 3 inputs). Ran on Python 3.13.5 with scikit-learn 1.9.1, pandas 3.0.6 and numpy 2.5.4.
  • ruff check . and ruff format --check . (ruff 0.17.1): clean.
  • I could not run the Python 3.10 + scikit-learn 1.3 row locally. On Windows, scikit-learn failed to load one of its DLLs because the temporary path was too long, so no tests ran. This PR's CI is the first real run of that row.

🤖 Generated with Claude Code

akifkulll and others added 3 commits October 17, 2026 15:01
.claude/settings.json is a per-developer file and is no longer tracked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
README already pointed to a LICENSE file that did not exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Drop README_preprocessing_toolkit_EN.md, an identical copy of README.md.
- Add a v0.1 in-development status note and mark the quick start as the
  planned API.
- Install from source (not on PyPI yet) with the real repository URL and
  fix the broken Octacity link.
- Limit the listed options to the v0.1 scope (no label encoding, no robust
  scaler; robust is already on the v0.2 roadmap).
- Format the quick start example so ruff format --check passes in CI.
- Mention the oldest-dependency CI row in CONTRIBUTING.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@akifkulll
akifkulll merged commit fbc92bf into main Oct 17, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant