Skip to content

Never drop or shadow feature_dict groups with auto-filled columns (#81) - #85

Merged
upadhyan merged 4 commits into
mainfrom
fix/81-feature-dict-int-key-collision
Oct 3, 2026
Merged

upadhyan merged 4 commits into
mainfrom
fix/81-feature-dict-int-key-collision

Conversation

@upadhyan

@upadhyan upadhyan commented Oct 3, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #81.

Root cause

_feature_dict_to_features stored user groups and auto-filled singleton columns in one dict keyed by logical name. An int key equal to the index of a column no group lists was overwritten by that column's auto-filled singleton, so the group was silently dropped from training:

configure_feature_dict(2, {0: [1]}).features  # [[0]] -- column 1 gone

A str key like "0" didn't lose data but produced duplicate logical names (("0", "0")), making export_text / plot_tree labels ambiguous.

Fix

Auto-filled column i is named str(i) unless a user key already has that name; then it becomes "<i>_1" (_2, … until free). User keys always keep their names.

configure_feature_dict(6, {0: [2, 3]}).logical_names
# ('0', '0_1', '1', '4', '5')   -- group "0" = [2, 3], column 0 = "0_1"
configure_feature_dict(3, {"0": [1, 2]}).logical_names
# ('0_1', '0')

Keys whose string forms coincide ({0: …, "0": …}) now raise ValueError, since their names would be ambiguous. Ordering is unchanged (int keys and auto-filled columns first, then str keys, each by name). Docstrings for configure_feature_dict and ProcessedFeatures.logical_names updated.

(The first commit raised ValueError on collision; the follow-up commit replaces that with renaming.)

Why no test caught it

  • Every existing int-key test happened to list the key's own column ({0: [0, 1, 2]}, {i: [i]}, …), so the collision never occurred.
  • Several regression tests compare against configure_feature_dict itself, so they share the bug.
  • No test asserted "every column appears in exactly one logical feature".

Tests

  • tests/test_features.py: rename cases (int key, str key, suffix already taken), same-string-form keys raise, fit keeps the group, and every-column-exactly-once + unique-names invariant across int/str/mixed keys.
  • tests/regression/test_sgtlearn_features_py__feature_dict_to_features_1.py: 23 blind tests from test-checker run r-20261002-215910-e7c6 (0 misguided). Three that encoded the interim raise / duplicate-name behaviour were updated to the renaming rule.
  • Full suite: 1276 passed, 2 skipped.

Not in this PR

  • int keys still sort as strings (10 before 9), as documented; changing it would reorder feature_importances_.
  • Tutorial categorical-features.ipynb cell 3 and the ProcessedFeatures docstring are also edited on the Update Stale Docs and Tutorials #77 docs branch; expect a small docstring conflict there.

🤖 Generated with Claude Code

upadhyan and others added 3 commits October 2, 2026 21:56
Fixes #81. _feature_dict_to_features stored user groups and auto-filled
singleton columns in one dict keyed by logical name, so an int key equal
to the index of a column no group lists was overwritten by that column's
singleton (configure_feature_dict(2, {0: [1]}) -> [[0]]), silently
dropping the group from training. Now raise a ValueError naming the
clashing keys. str keys and non-colliding int keys are unchanged.

Existing tests missed it because every int-key case listed the key's own
column, and several regression tests compare against configure_feature_dict
itself. Added collision tests (configure and fit) and an
every-column-exactly-once invariant test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test-checker bugfix run r-20261002-215910-e7c6 (53a5c14 -> e2deb3a):
23 tests written blind from the spec; 2 fail on the buggy revision and
pass on the fix, 21 pass on both, none misguided. Unused numpy import
removed for ruff.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Auto-filled column i is named str(i) unless a feature_dict key already
uses that name, in which case it becomes "<i>_1" ("<i>_2", ...). This
replaces the ValueError from the previous commit: {0: [2, 3]} now keeps
the user's group "0" and names column 0 "0_1". It also fixes the
duplicate-name case for str keys ({"0": [1, 2]} gave ("0", "0")). Keys
whose string forms coincide (0 and "0") now raise ValueError.

Three blind regression tests asserted the old raise/duplicate behaviour;
updated per the maintainer's decision on the new naming rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@upadhyan upadhyan changed the title Reject int feature_dict keys that collide with auto-filled columns (#81) Never drop or shadow feature_dict groups with auto-filled columns (#81) Oct 3, 2026
# Conflicts:
#	.test-checker/ledger.jsonl
#	sgtlearn/_features.py
@upadhyan
upadhyan merged commit ef81fb3 into main Oct 3, 2026
16 checks passed
@upadhyan
upadhyan deleted the fix/81-feature-dict-int-key-collision branch October 3, 2026 03:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feature_dict int key equal to an unlisted column index silently drops that group

1 participant