Skip to content

feat: add missing value imputer - #10

Merged
akifkulll merged 3 commits into
octacity-org:mainfrom
Venius24:feat-missing-value-imputer
Oct 25, 2026
Merged

akifkulll merged 3 commits into
octacity-org:mainfrom
Venius24:feat-missing-value-imputer

Conversation

@Venius24

Copy link
Copy Markdown
Contributor

Summary

  • Implement MissingValueImputer as a BasePreprocessor with mean, median and most-frequent strategies. Non-numeric columns use the most frequent value; ties choose the smallest value.
  • Learn statistics only in fit, preserve the input DataFrame, support nullable integer columns with fractional means, and export the class from the package.
  • Add README usage and tests for leakage, validation, tie-breaking, DataFrame preservation, scikit-learn cross-validation and grid search.

Closes #2.

Validation

  • ruff check . passed.
  • ruff format --check . passed.
  • pytest -q: 41 passed.
  • git diff --check passed.

@akifkulll

Copy link
Copy Markdown
Collaborator

Hi @Venius24, sorry for the silence. That was our mistake: CI for first-time contributors needs a maintainer to approve it, we missed that, so no checks ran and nobody reviewed your PR. Your implementation looks solid (all 41 tests pass on our side). If you're still interested, we'd like to reopen it and merge after a few small changes:

  1. Nullable boolean columns are treated as numeric, so mean gives 0.5. They should use most_frequent.
  2. With mixed-type object columns, the tie-break min(...) raises TypeError (e.g. ['a', 1]). min(candidates, key=str) avoids that.
  3. In the GridSearchCV test, please pass error_score="raise" so a failing fit can't pass silently.
    Let us know and we'll reopen it. Thanks for contributing!

Venius24 and others added 2 commits October 25, 2026 23:19
# Conflicts:
#	src/preprocessing_toolkit/__init__.py
… ties

- Nullable boolean columns now always use the most frequent value; mean and
  median gave 0.5, which is not a boolean.
- Mode ties on object columns mixing types (e.g. "a" and 1) fell over in
  min(); fall back to ordering by str when values are not comparable, so
  numeric ties still pick the smallest number.
- GridSearchCV test passes error_score="raise" so a failing fit cannot
  pass silently.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Venius24

Copy link
Copy Markdown
Contributor Author

Thanks @akifkulll, no worries, and thanks for the careful review. Yes, I'd like to finish this. The branch is updated, so please reopen:

  1. Nullable booleans: boolean columns are now treated as categories and always use the most frequent value, whatever the strategy. A new test covers mean, median and most_frequent on a boolean column and checks the dtype stays boolean.
  2. Mixed-type ties: I used min(candidates) first and fall back to min(candidates, key=str) only when the values can't be compared. Using key=str always would change numeric ties (10 would beat 9), so this keeps "smallest value" for numbers. There's a test for ["a", 1] in both row orders.
  3. GridSearchCV: now passes error_score="raise".

I also merged the current main (with FeatureScaler) into the branch. __init__.py now exports both blocks, and the README status line names both. I merged rather than rebased so the PR can still be reopened.

Locally: 69 tests pass, ruff check and ruff format --check are clean, and the two new tests fail on the previous commit.

@akifkulll akifkulll reopened this Oct 25, 2026

@akifkulll akifkulll left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

“LGTM, thanks!

@akifkulll
akifkulll merged commit 59c5358 into octacity-org:main Oct 25, 2026
6 checks passed
@akifkulll

Copy link
Copy Markdown
Collaborator

Reopened, and thank you for coming back to this.

I checked the branch locally: all 69 tests pass and ruff is clean. The boolean, mixed-type and GridSearchCV fixes all work as described, and thanks for merging main so both blocks are exported. You're also right about key=str: comparing "10" and "9" as strings would pick the wrong numeric value, so trying plain min first is the better design. Good catch, and thanks for explaining the reasoning.

It was a pleasure working with you on this. PreToolkit will keep growing after v0.1, with new blocks and features in the next versions, and we'd be glad to have you on board again. Feel free to pick up any future issue that interests you, or to open your own ideas.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add MissingValueImputer (mean / median / most_frequent)

2 participants