diff --git a/README.md b/README.md index 352e8d2..0f1d59c 100644 --- a/README.md +++ b/README.md @@ -92,7 +92,7 @@ installs the `sgtlearn` package plus native modules into `.venv`. ```bash pip install . -pip install ".[dev]" # dev extras (pytest, scikit-learn) only if needed +pip install ".[dev]" # dev extras (pytest, ruff, black, mypy, ...) only if needed ``` > **Anaconda users:** Do not bootstrap the venv from an Anaconda Python. @@ -102,27 +102,26 @@ pip install ".[dev]" # dev extras (pytest, scikit-learn) only if needed > non-Anaconda Python — e.g. `uv venv --python 3.12 .venv` (downloads a > hermetic CPython), `pyenv`, or your distro's `python3`. -## Build Workflow (scikit-build + CMake) +## Build Workflow (scikit-build-core + CMake) `pip install .` drives this build path: 1. `pyproject.toml` selects `scikit_build_core.build` as the backend. 2. CMake is configured from `cpp/CMakeLists.txt`. 3. Each file in `cpp/bindings/*.cpp` becomes one pybind11 module target. -4. After each module is built, `pybind11-stubgen` generates a matching `.pyi`. -5. The `.pyi` is generated and installed in the same location as the module `.so`. +4. After each module is built, `pybind11-stubgen` generates a matching `.pyi` (best effort; a missing or failing `pybind11-stubgen` does not break the build). +5. Any generated `.pyi` is installed in the same location as the module `.so`. ## C++ Folder Conventions -- `cpp/include/sgtlearn/`: public headers for the core C++ API. -- `cpp/src/`: internal C++ implementation for the core library. +- `cpp/src/`: C++ headers and implementation for the core library (the public include directory of `sgtlearn_core`). - `cpp/bindings/`: pybind11 binding entrypoints; one `.cpp` file maps to one Python extension module. - `cpp/tests/`: C++ unit tests consumed by the `cpp_tests` executable target. ## CMake Targets - `sgtlearn_core` (static library): shared C++ logic used by Python modules and tests. -- `` (pybind11 module, one per file in `cpp/bindings/`): compiled extension modules installed into the package. +- `` (pybind11 module, one per file in `cpp/bindings/`): compiled extension modules installed as top-level modules next to the `sgtlearn` package. - `cpp_tests` (Catch2 executable): optional C++ test target, controlled by: - `-DSGTLEARN_BUILD_TESTS=ON` (build C++ tests) - `-DSGTLEARN_BUILD_TESTS=OFF` (default for `pip install`; the CMake option itself defaults to `ON`, but `pyproject.toml` overrides this so wheels don't ship test binaries) diff --git a/ROADMAP.md b/ROADMAP.md index 3f7382d..9e07acb 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -14,7 +14,7 @@ - [ ] Exporting to Graphviz, planned for v0.4.0 ([#69](https://github.com/optimal-uoft/sgtlearn/issues/69)) - [x] Feature importances matching scikit-learn's API for SGTs and SGT$_K$ - [x] Adding TAO Refinement -- [x] **Sklearn-style NaN support** (replaces current tail-bin placeholder): split search uses finite values only; each candidate is scored with missing sent left vs right—including an explicit missing-vs-non-missing split—with the winning direction stored per node (ties → right). +- [x] **Sklearn-style NaN support**: split search uses finite values only, and each univariate node learns which child receives missing values. - [x] **NaN routing at predict**: if training saw missing at that split, follow the stored direction; otherwise route to the majority child. ## v0.3.0 @@ -29,7 +29,13 @@ See the implementation specification in [#42](https://github.com/optimal-uoft/sg and the umbrella issue [#27](https://github.com/optimal-uoft/sgtlearn/issues/27). +## v0.3.1 +- [x] Regularized best-first outer growth with `branching_penalty` and strict `max_leaf_nodes` budgets for multiway splits + +## Unreleased +- [x] Text export of fitted trees with `export_text` + ## v1.0.0 -- [ ] Cross-Feature Tree Binning (Similar to [DPDT](https://github.com/KohlerHECTOR/DPDTreeEstimator) +- [ ] Cross-Feature Tree Binning (Similar to [DPDT](https://github.com/KohlerHECTOR/DPDTreeEstimator)) - [ ] Boosting - [ ] Default Optuna Support diff --git a/cpp/README.md b/cpp/README.md index b463011..784901d 100644 --- a/cpp/README.md +++ b/cpp/README.md @@ -4,17 +4,15 @@ This directory contains the C++ core library and pybind11 module definitions use ## Layout Conventions -- `include/sgtlearn/`: public headers for `sgtlearn_core`. -- `src/`: internal implementation for `sgtlearn_core`. +- `src/`: headers (`.h`, `.tpp`) and sources (`.cpp`) for `sgtlearn_core`; `src/` is its public include directory. - `bindings/`: pybind11 module sources; each `*.cpp` file becomes one extension module target. -- `../tests/cpp/`: C++ test files used by `cpp_tests`. +- `tests/`: C++ test files used by `cpp_tests`. | Directory | Content Type | Visibility | Target | |:--|:--|:--|:--| -| `include/sgtlearn/` | Public headers (`.hpp`) | Public | `sgtlearn_core` | -| `src/` | Internal logic (`.cpp`) | Private | `sgtlearn_core` | +| `src/` | Headers (`.h`, `.tpp`) and logic (`.cpp`) | Public | `sgtlearn_core` | | `bindings/` | Python wrappers (`.cpp`) | Private | `` | -| `../tests/cpp/` | Unit tests (`.cpp`) | Private | `cpp_tests` | +| `tests/` | Unit tests (`.cpp`) | Private | `cpp_tests` | ## Targets Defined in `CMakeLists.txt` @@ -33,10 +31,10 @@ This directory contains the C++ core library and pybind11 module definitions use For each pybind11 module target: 1. the extension module (`.so`) is built; -2. `pybind11_stubgen` runs as a post-build step; +2. if `pybind11_stubgen` was importable at configure time, it runs as a non-fatal post-build step; 3. the generated `.pyi` is written to the same output directory as that module. -The install rules place the `.so` and matching `.pyi` together in the same install destination. +The install rules place the `.so` and, when it was generated, the matching `.pyi` together in the same install destination. ## Common Commands @@ -64,7 +62,7 @@ Skip C++ test target: cmake -S cpp -B build -DSGTLEARN_BUILD_TESTS=OFF ``` -Most developers should use `pip install .` from the repository root, which drives this CMake configuration via scikit-build. +Most developers should use `pip install .` from the repository root, which drives this CMake configuration via scikit-build-core (with `-DSGTLEARN_BUILD_TESTS=OFF`). ## CLion Notes diff --git a/cpp/bindings/Discretizers.cpp b/cpp/bindings/Discretizers.cpp index 2c0fd34..569ce94 100644 --- a/cpp/bindings/Discretizers.cpp +++ b/cpp/bindings/Discretizers.cpp @@ -1,6 +1,6 @@ /** * @file Discretizers.cpp - * @brief pybind11 module ``Discretizers``: univariate classification/regression discretizers. + * @brief pybind11 module ``Discretizers``: univariate and categorical classification/regression discretizers. */ #include diff --git a/cpp/bindings/ShapeGeneralizedTrees.cpp b/cpp/bindings/ShapeGeneralizedTrees.cpp index f171db2..70f3ab6 100644 --- a/cpp/bindings/ShapeGeneralizedTrees.cpp +++ b/cpp/bindings/ShapeGeneralizedTrees.cpp @@ -86,7 +86,9 @@ PYBIND11_MODULE(ShapeGeneralizedTrees, m) { py::arg("features"), "Fit the routing tree. X is (n_samples, n_features) float32; y is " "uint class labels, 1-D (n_samples,) or 2-D (n_samples, n_outputs). " - "Optional sample_weight is 1-D float32.") + "Optional sample_weight is 1-D float32. features is a list of dicts " + "with 'type' ('continuous' or 'categorical') and 'indices' (the X " + "columns of each logical feature).") .def("predict", &ClassificationShapeGeneralizedTreePy::predict, py::arg("X"), "Predict class labels for X (shape (n_samples, n_features)). Returns " @@ -114,7 +116,9 @@ PYBIND11_MODULE(ShapeGeneralizedTrees, m) { "feature_importance", &ClassificationShapeGeneralizedTreePy::featureImportance, "Normalized feature importances aligned with the features sequence " - "passed to fit. Available only after training.") + "passed to fit (all zeros if the tree has no split). Computed at fit " + "and not updated by TreeAlternatingOptimization. Available only " + "after training.") .def("tree_export", &ClassificationShapeGeneralizedTreePy::tree_export, "Return a flat snapshot of the fitted tree as a Python dict."); @@ -152,16 +156,19 @@ PYBIND11_MODULE(ShapeGeneralizedTrees, m) { py::arg("pairwise_candidates") = 0, py::arg("pairwise_penalty") = 0.0, py::arg("branching_penalty") = 0.0, - R"(Regression tree: inner bins are round-robin seeded. ``squared_error`` runs -coordinate descent and keeps the map only if branch MSE improves clearly vs the seed; -otherwise the snapshot is restored and the branch objective is rebuilt. -``absolute_error`` / ``mae`` skip coordinate descent by default.)") + R"(Regression tree. For each branch count, ``squared_error`` seeds bins with +weighted k-means over bin means, starts from the inner tree's root split instead +when that is no worse, and refines with coordinate descent, keeping the best +feasible assignment per arity. ``absolute_error`` / ``mae`` score the root split +and round-robin seeds and skip coordinate descent unless ``SGTLEARN_MAE_CD=1``.)") .def("fit", &RegressionShapeGeneralizedTreePy::fit, py::arg("X"), py::arg("y"), py::arg("sample_weight") = py::none(), py::arg("features"), "Fit the routing tree. X is (n_samples, n_features) float32; y is " "float32 targets, 1-D (n_samples,) or 2-D (n_samples, n_outputs). " - "Optional sample_weight is 1-D float32.") + "Optional sample_weight is 1-D float32. features is a list of dicts " + "with 'type' ('continuous' or 'categorical') and 'indices' (the X " + "columns of each logical feature).") .def("predict", &RegressionShapeGeneralizedTreePy::predict, py::arg("X"), "Predict targets for X. Returns (n_samples,) for a single output or " "(n_samples, n_outputs) for multi-output.") @@ -179,7 +186,9 @@ otherwise the snapshot is restored and the branch objective is rebuilt. "feature_importance", &RegressionShapeGeneralizedTreePy::featureImportance, "Normalized feature importances aligned with the features sequence " - "passed to fit. Available only after training.") + "passed to fit (all zeros if the tree has no split). Computed at fit " + "and not updated by TreeAlternatingOptimization. Available only " + "after training.") .def_property_readonly( "leaf_regression_stats", &RegressionShapeGeneralizedTreePy::leafRegressionStats) diff --git a/cpp/bindings/TreeAlternatingOptimization.cpp b/cpp/bindings/TreeAlternatingOptimization.cpp index 40c7044..f3466d7 100644 --- a/cpp/bindings/TreeAlternatingOptimization.cpp +++ b/cpp/bindings/TreeAlternatingOptimization.cpp @@ -167,8 +167,10 @@ PYBIND11_MODULE(TreeAlternatingOptimization, m) { py::arg("lambda_") = 0.0, py::arg("tao_pair_scale") = 1.1, "Refine a fitted ClassificationShapeGeneralizedTree or " "RegressionShapeGeneralizedTree in place. X is " - "(n_samples, n_features) float32; y is 1-D class labels (uint) or " - "float targets matching the tree type. Runs up to n_runs bottom-up " + "(n_samples, n_features) float32; y is class labels (uint) or float " + "targets matching the tree type, 1-D (n_samples,) or 2-D " + "(n_samples, n_outputs) with the fitted tree's number of outputs. " + "Optional sample_weight is 1-D float32. Runs up to n_runs bottom-up " "sweeps. In weighted reward units, lambda_ penalizes single-feature " "routers by lambda_ * nodeSampleCount and pair routers by " "tao_pair_scale * lambda_ * nodeSampleCount; dummy routers are " diff --git a/cpp/src/Estimators/RegressionShapeGeneralizedTree.cpp b/cpp/src/Estimators/RegressionShapeGeneralizedTree.cpp index b962f48..4bf135a 100644 --- a/cpp/src/Estimators/RegressionShapeGeneralizedTree.cpp +++ b/cpp/src/Estimators/RegressionShapeGeneralizedTree.cpp @@ -2,8 +2,10 @@ * @file Estimators/RegressionShapeGeneralizedTree.cpp * @brief Training, child partitioning, and prediction for the regression * shape-generalized tree. Per-node inner fit: discretize -> search - * partition counts k in [2, numPartitions] -> coordinate descent (MSE) - * or round-robin (MAE) -> record best branch. + * partition counts k in [2, numPartitions] -> k-means / root-split seed + * plus coordinate descent (MSE), or root / round-robin seeds without + * coordinate descent unless SGTLEARN_MAE_CD=1 (MAE) -> record best + * branch per arity. */ #include diff --git a/cpp/src/Estimators/RegressionShapeGeneralizedTree.h b/cpp/src/Estimators/RegressionShapeGeneralizedTree.h index 9f24e86..9ce12a5 100644 --- a/cpp/src/Estimators/RegressionShapeGeneralizedTree.h +++ b/cpp/src/Estimators/RegressionShapeGeneralizedTree.h @@ -29,9 +29,9 @@ * - **Outer growth** (`OuterTreeBuilder`): best-first regularized expansion; * per-node split search and child creation via lambdas in ``fit``; commit * step remains a local lambda in ``fit``. - * - **Per-node split search** (``fit`` lambdas): for each - * discretize -> round-robin bin-to-partition seed -> ``coordinateDescent`` on - * ``SquaredError`` only; ``AbsoluteError`` keeps the round-robin map (no CD). + * - **Per-node split search** (``fit`` lambdas): for each candidate feature, + * discretize -> seed bin-to-partition maps -> ``coordinateDescent`` + * (``SquaredError``; ``AbsoluteError`` only when ``SGTLEARN_MAE_CD=1``). * - **Leaf state**: per-leaf mean (squared error) or median (absolute error) * plus optional ``[sum y, sum y^2]`` stats for squared error. * - **Inference**: `predict` walks `childIndices_` using @@ -40,17 +40,21 @@ * At every outer-tree node, for each candidate feature: * 1. **Discretize**: train univariate regression discretizer on the node's * samples (per-bin stats and training column indices). - * 2. **Initial assignment**: round-robin by discretizer bin index - * (``b % numPartitions``); regression does not use k-means seeding. + * 2. **Initial assignment** (for each k in [2, numPartitions]): + * ``SquaredError`` seeds with weighted k-means over bin means and starts + * from the inner tree's root split instead when that is no worse; + * ``AbsoluteError`` scores a round-robin seed (``b % k``) and starts from + * the root split. * 3. **Refinement**: ``SquaredError`` runs coordinate descent on the - * bin-to-partition map; if the post-CD objective clearly worsens vs the - * seed, restore the assignment snapshot and rebuild. ``AbsoluteError`` - * keeps the round-robin seed (no CD). + * bin-to-partition map; ``AbsoluteError`` skips it unless + * ``SGTLEARN_MAE_CD=1``. Every scored assignment is observed and the + * lowest-impurity feasible one is kept per occupied arity. * * The best-scoring feature wins; its inner discretizer + bin->partition - * mapping become the routing rule for that node, producing `numPartitions` - * children. Inner-node fitting matches the Python `BranchingTree` pattern; - * the outer loop uses `OuterTreeBuilder` like Python's heap over + * mapping become the routing rule for that node, producing at most + * `numPartitions` children (empty partitions are compacted away). + * Inner-node fitting matches the Python `BranchingTree` pattern; the outer + * loop uses `OuterTreeBuilder` like Python's heap over * `best_impurity_decrease`. * * Inputs use Armadillo's column-major convention: X has shape diff --git a/docs/api/ensemble.rst b/docs/api/ensemble.rst index 746af0b..564fa5b 100644 --- a/docs/api/ensemble.rst +++ b/docs/api/ensemble.rst @@ -18,7 +18,8 @@ Multi-output ``y`` is supported the same way as for single trees Forests accept ``branching_penalty`` (default ``0.0``) and the Shape²CART options ``pairwise_candidates`` and ``pairwise_penalty`` and forward them to every base estimator. Outer growth uses the same total sample-mass-weighted, -output-averaged score, constant ``alpha``/``lambda``/``gamma`` costs, strict +output-averaged score, constant ``min_impurity_decrease``, +``branching_penalty`` and ``pairwise_penalty`` costs, strict actual-arity leaf budget, fixed-epsilon acceptance, and always-best-first order as single trees. Pair candidates are restricted to each node's ``max_features`` logical-feature subset. See :doc:`estimators` for the full @@ -26,11 +27,11 @@ scoring, candidate-count, pair-screening, categorical and joint missing-routing semantics, and the feature-importance warning for pair nodes. See :doc:`../tutorials/bivariate-branching` for a worked classifier example. -:attr:`~sgtlearn.ensemble.RandomSGForestClassifier.mean_feature_importances_` -and :attr:`~sgtlearn.ensemble.RandomSGForestClassifier.std_feature_importance_` +:attr:`~sgtlearn.RandomSGForestClassifier.mean_feature_importances_` +and :attr:`~sgtlearn.RandomSGForestClassifier.std_feature_importance_` (and the regressor counterparts) summarize per-tree :attr:`~sgtlearn.SGTClassifier.feature_importances_` across the forest, -aligned with the shared :attr:`processed_features_`. +aligned with the shared :attr:`~sgtlearn.RandomSGForestClassifier.processed_features_`. They are unavailable if any base tree has undergone positive-run TAO refinement. RandomSGForestClassifier @@ -40,6 +41,7 @@ RandomSGForestClassifier :members: :inherited-members: :show-inheritance: + :exclude-members: mean_feature_importances_, std_feature_importance_ RandomSGForestRegressor ----------------------- @@ -48,3 +50,4 @@ RandomSGForestRegressor :members: :inherited-members: :show-inheritance: + :exclude-members: mean_feature_importances_, std_feature_importance_ diff --git a/docs/api/estimators.rst b/docs/api/estimators.rst index 2ce0942..e4f3acc 100644 --- a/docs/api/estimators.rst +++ b/docs/api/estimators.rst @@ -5,15 +5,16 @@ Estimators Single Shape Generalized Tree estimators following the scikit-learn API. -Both estimators accept ``tao_n_runs`` and ``tao_lambda``; TAO runs automatically -at the end of :meth:`~sklearn.base.BaseEstimator.fit` when ``tao_n_runs > 0``. +Both estimators accept ``tao_n_runs``, ``tao_lambda``, and ``tao_pair_scale``; +TAO runs automatically at the end of :meth:`SGTClassifier.fit` / +:meth:`SGTRegressor.fit` when ``tao_n_runs > 0`` (the default is ``10``). See :doc:`tao` for behaviour and post-hoc :func:`~sgtlearn.tao.TAO_refine`. -After fitting with TAO disabled, :attr:`~sgtlearn.base.BaseShapeCART.feature_importances_` +After fitting with TAO disabled, :attr:`~sgtlearn.BaseShapeCART.feature_importances_` gives normalized impurity importances over logical features, aligned with -:attr:`~sgtlearn.base.BaseShapeCART.processed_features_`. Forests expose -:attr:`~sgtlearn.ensemble.RandomSGForestClassifier.mean_feature_importances_` -and :attr:`~sgtlearn.ensemble.RandomSGForestClassifier.std_feature_importance_` +:attr:`~sgtlearn.BaseShapeCART.processed_features_`. Forests expose +:attr:`~sgtlearn.RandomSGForestClassifier.mean_feature_importances_` +and :attr:`~sgtlearn.RandomSGForestClassifier.std_feature_importance_` instead (see :doc:`ensemble`). After any positive-run TAO refinement these attributes are unavailable; see :doc:`../tutorials/feature-importance` for permutation importance. @@ -32,11 +33,13 @@ Sklearn-compatible return shapes are preserved at the API boundary: * **Regressor** — :meth:`~SGTRegressor.predict` returns ``(n_samples,)`` or ``(n_samples, n_outputs)``. -* **Classifier** — :meth:`~SGTClassifier.predict` matches ``y``'s rank; +* **Classifier** — :meth:`~SGTClassifier.predict` returns ``(n_samples,)`` + for one output (including a single-column 2-D ``y``) or + ``(n_samples, n_outputs)``; :attr:`~SGTClassifier.classes_` / :attr:`~SGTClassifier.n_classes_` are a - scalar array / int for one output, or a list per output for multi-output; + 1-D array / int for one output, or a list per output for multi-output; :meth:`~SGTClassifier.predict_proba` returns an array or a list of arrays. -* **``class_weight``** — a dict applies to every output, or pass a list of +* **class_weight** — a dict applies to every output, or pass a list of dicts (one per output). Weights multiply into one ``sample_weight`` vector. Regularized outer growth @@ -51,7 +54,8 @@ the score for an actual ``k``-child split is W_p H_p - \sum_c W_c H_c - \alpha - \lambda(k-2) - \gamma I_{pair}. Here ``alpha`` is ``min_impurity_decrease``, ``lambda`` is -``branching_penalty``, and ``gamma`` is ``pairwise_penalty``. These costs are +``branching_penalty`` (not TAO's ``tao_lambda``), and ``gamma`` is +``pairwise_penalty``. These costs are constant and applied once; they are not scaled by sample mass, and scores are not divided by leaf cost. Only finite scores strictly greater than double machine epsilon are eligible. Outer growth is always best-first, including @@ -72,6 +76,7 @@ SGTClassifier :members: :inherited-members: :show-inheritance: + :exclude-members: feature_importances_, processed_features_ SGTRegressor ------------ @@ -80,6 +85,7 @@ SGTRegressor :members: :inherited-members: :show-inheritance: + :exclude-members: feature_importances_, processed_features_ BaseShapeCART ------------- @@ -118,8 +124,9 @@ axis-aligned behaviour exactly. pairs) or a float (the fraction of logical features used to determine that number, rounded up). Candidate pairs are formed only from the logical feature subset selected for the node by ``max_features``. ``pairwise_penalty`` -(default ``0``) is applied only while selecting between univariate and -bivariate growth candidates. Pair screening is independent of regularization: +(default ``0``) is the ``gamma`` cost in the growth score above; it is +subtracted from every fitted pair candidate before acceptance and best-first +ordering. Pair screening is independent of regularization: a feature is eligible when a searched, minimum-leaf-feasible assignment has finite raw weighted impurity decrease greater than double machine epsilon. One lowest-raw-impurity proxy per eligible feature is used for the existing pair @@ -127,9 +134,10 @@ ranking; regularized winners do not choose the proxy. A retained pair is fit with an ordinary axis-aligned CART over the two logical features. Continuous and grouped categorical features are supported. Missing -values are routed jointly per feature, so a finite interval on one axis and a -missing value on the other is a distinct bin (as are the converse and both -missing); a missing branch may continue splitting on the other feature. +values are routed jointly per feature: at each pair-tree split, values missing +on the split axis get their own branch when the node's training samples had +any, otherwise they follow the larger finite child; a missing branch may +continue splitting on the other feature. Multiway outer branching uses the same inner pair tree. Without TAO, pair gains are divided equally between the two logical features. Accessing @@ -141,5 +149,6 @@ Pair-aware TAO is available through ``tao_pair_scale`` (default ``1.1``): it is finite and non-negative, affects only the TAO complexity penalty, and TAO only reconsiders pairs retained during initial screening. After TAO, ``feature_importances_`` is unavailable. The existing ``plot_tree`` API renders -exact Shape²CART routing heatmaps with marginal histograms, partition-changing -threshold labels, categorical labels, and missing margins when present in ``X``. +exact Shape²CART routing heatmaps with partition-changing threshold labels, +categorical labels and missing-value margins; passing ``X`` adds marginal +histograms and keeps a missing margin only on axes with observed missing values. diff --git a/docs/api/index.rst b/docs/api/index.rst index 95df016..8158d9a 100644 --- a/docs/api/index.rst +++ b/docs/api/index.rst @@ -1,7 +1,7 @@ API Reference ============= -The public API is re-exported from the top-level :mod:`sgtlearn` package. +The public API is re-exported from the top-level ``sgtlearn`` package. .. currentmodule:: sgtlearn diff --git a/docs/api/plotting.rst b/docs/api/plotting.rst index 36643e4..2c36070 100644 --- a/docs/api/plotting.rst +++ b/docs/api/plotting.rst @@ -3,7 +3,7 @@ Plotting & Export .. currentmodule:: sgtlearn -Visualize a fitted SGT, or export it to other formats. +Visualize a fitted SGT, or export it as indented text rules. plot_tree --------- diff --git a/docs/api/tao.rst b/docs/api/tao.rst index 76742c4..0428bc9 100644 --- a/docs/api/tao.rst +++ b/docs/api/tao.rst @@ -3,15 +3,20 @@ TAO refinement Tree-Alternating Optimization (TAO) refines an *already fitted* shape-generalized tree or random forest **in place**. It walks internal nodes bottom-up and replaces -routing rules when doing so does not decrease training performance at that node. +routing rules when doing so does not lower the node's penalized training +objective (weighted reward minus the ``tao_lambda`` complexity cost). Tree topology (node structure) is preserved; leaf statistics are refreshed as -routing changes. +routing changes. Univariate candidates are shape functions fit on one input +column at a time, over every column regardless of ``max_features`` or +``feature_dict`` grouping (each one-hot column is treated as numeric); +bivariate candidates are limited to the node's retained pairs. There are two ways to use TAO: -1. **During ``fit``** — pass ``tao_n_runs`` and ``tao_lambda`` to any supported - estimator (see below). TAO runs automatically after the native trainer finishes. -2. **After ``fit``** — call :func:`~sgtlearn.tao.TAO_refine` on a fitted model to +1. **During fit** — pass ``tao_n_runs``, ``tao_lambda``, and ``tao_pair_scale`` + to any supported estimator (see below). TAO runs automatically after the + native trainer finishes. +2. **After fit** — call :func:`~sgtlearn.tao.TAO_refine` on a fitted model to refine it further (or to run TAO when ``tao_n_runs=0`` was used at fit time). Import the post-hoc entry point from the dedicated module (also re-exported on the @@ -28,14 +33,18 @@ All four public tree estimators accept the same three constructor / ``fit``-time TAO knobs: ``tao_n_runs`` : int, default=10 - Number of bottom-up TAO passes to run after the native trainer finishes. - Set to ``0`` to skip TAO entirely during :meth:`~sklearn.base.BaseEstimator.fit`. + Maximum number of bottom-up TAO passes to run after the native trainer + finishes (TAO may stop earlier). + Set to ``0`` to skip TAO entirely during ``fit``. ``tao_lambda`` : float, default=0.0 Per-sample complexity rate (cost-complexity style). At each internal node, a non-constant routing rule must beat the constant dummy rule by more than - ``tao_lambda * n_samples`` in weighted reward units to be accepted. With the - default ``0.0``, weighted training accuracy / loss does not decrease. + ``tao_lambda * n_node`` in weighted reward units to be accepted + (``n_node``: unweighted count of training samples reaching the node). + With the default ``0.0``, weighted training loss does not increase for + regression, and weighted training accuracy (mean over outputs) does not + decrease for classifiers. ``tao_pair_scale`` : float, default=1.1 Finite, non-negative multiplier for the TAO complexity penalty of a @@ -80,7 +89,7 @@ Example — TAO during ``fit``: from sgtlearn import SGTClassifier, RandomSGForestClassifier - # Default: 10 TAO passes after native training (tao_lambda=0.0) + # Default: up to 10 TAO passes after native training (tao_lambda=0.0) tree = SGTClassifier(max_depth=4, random_state=42).fit(X, y) # Disable fit-time TAO @@ -128,15 +137,17 @@ Single tree ``y`` must be in the same label space used for :meth:`~sgtlearn.SGTClassifier.fit`. Pass the **same** ``(X, y)`` used to fit the model (per-sample partitions are not -stored after fit). With ``lambda_=0`` (the default), training accuracy / loss does -not decrease. When ``lambda_ > 0``, non-constant routing rules must improve -weighted training reward by more than ``lambda_ * n_samples`` to beat the constant -dummy rule at each node. +stored after fit). With ``lambda_=0`` (the default), weighted training loss does not +increase for regression, and weighted training accuracy (mean over outputs) +does not decrease for classifiers. When ``lambda_ > 0``, non-constant routing +rules must improve weighted training reward by more than ``lambda_ * n_node`` +to beat the constant dummy rule at each node (``n_node``: unweighted count of +training samples reaching that node). Multi-output ``y`` is supported with the same shapes as the estimators (``(n_samples,)`` or ``(n_samples, n_outputs)``). Classification TAO uses an all-but-worst care set when ``n_outputs > 1`` (matching regression); single-output -classification keeps the all-but-correct (best children) care set. +classification keeps only the correct (best) children in the care set. Random forest ~~~~~~~~~~~~~ @@ -180,7 +191,8 @@ Choosing fit-time vs. post-hoc TAO * - :func:`~sgtlearn.tao.TAO_refine` after ``fit`` - Extra refinement passes, different ``lambda_``, or refining a forest on the full ``(X, y)`` instead of per-tree bootstrap data. Safe to call - multiple times; each call further refines the model in place. + multiple times; each call refines the model in place and stops early + once a sweep makes no change. API reference ------------- diff --git a/docs/conf.py b/docs/conf.py index f5e7d84..78fef49 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -59,7 +59,6 @@ autodoc_default_options = { "members": True, - "inherited-members": False, "show-inheritance": True, "member-order": "bysource", } diff --git a/docs/index.rst b/docs/index.rst index 63d5985..aaa2acf 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -71,5 +71,4 @@ Indices ------- * :ref:`genindex` -* :ref:`modindex` * :ref:`search` diff --git a/docs/installation.rst b/docs/installation.rst index 2e310cc..09b0af6 100644 --- a/docs/installation.rst +++ b/docs/installation.rst @@ -58,13 +58,14 @@ installs the ``sgtlearn`` package plus native modules into ``.venv``. Use a non-Anaconda Python — e.g. ``uv venv --python 3.12 .venv``, ``pyenv``, or your distro's ``python3``. -Build workflow (scikit-build + CMake) -------------------------------------- +Build workflow (scikit-build-core + CMake) +------------------------------------------ ``pip install .`` drives this build path: 1. ``pyproject.toml`` selects ``scikit_build_core.build`` as the backend. 2. CMake is configured from ``cpp/CMakeLists.txt``. 3. Each file in ``cpp/bindings/*.cpp`` becomes one pybind11 module target. -4. After each module is built, ``pybind11-stubgen`` generates a matching ``.pyi``. -5. The ``.pyi`` is installed alongside the module ``.so``. +4. After each module is built, ``pybind11-stubgen`` generates a matching ``.pyi`` + (best effort; a missing or failing stubgen does not break the build). +5. Any generated ``.pyi`` is installed alongside the module ``.so``. diff --git a/docs/quickstart.rst b/docs/quickstart.rst index f75b0f5..1ef65c9 100644 --- a/docs/quickstart.rst +++ b/docs/quickstart.rst @@ -27,8 +27,9 @@ Inner vs. outer depth By default, every SGT node's split is itself a small *inner* tree (the shape function) that carves one feature into bins; the *outer* tree routes samples through those bins. -``num_partitions`` sets the outer branching factor (the :math:`\mathrm{SGT}_K` -arity), and ``inner_max_depth`` controls how rich each shape function may be: +``num_partitions`` sets the outer branching factor (the :math:`K` in +:math:`\mathrm{SGT}_K`; an individual split may produce fewer children), and +``inner_max_depth`` controls how rich each shape function may be: .. code-block:: python diff --git a/docs/roadmap.rst b/docs/roadmap.rst index 2ad4e56..52b8cab 100644 --- a/docs/roadmap.rst +++ b/docs/roadmap.rst @@ -2,7 +2,7 @@ Release Roadmap =============== Which features are implemented today and which are planned. Shape²CART, -pair-aware TAO, and dedicated routing heatmaps are available in v0.3.0. +pair-aware TAO, and dedicated routing heatmaps are available since v0.3.0. v0.1.0 ------ @@ -22,7 +22,8 @@ v0.2.0 `issue #69 `_) - ✅ Feature importances matching scikit-learn's API for SGTs and :math:`\mathrm{SGT}_K` - ✅ TAO refinement -- ✅ Sklearn-style NaN support (replaces current tail-bin placeholder): split search uses finite values only; each candidate is scored with missing sent left vs right—including an explicit missing-vs-non-missing split—with the winning direction stored per node (ties → right). +- ✅ Sklearn-style NaN support: split search uses finite values only, and each + univariate node learns which child receives missing values. - ✅ NaN routing at predict: if training saw missing at that split, follow the stored direction; otherwise route to the majority child. v0.3.0 @@ -41,3 +42,22 @@ v0.3.0 The bivariate work is specified in `issue #42 `_ and tracked under the umbrella `issue #27 `_. + +v0.3.1 +------ + +- ✅ Regularized best-first outer growth with ``branching_penalty`` and strict + ``max_leaf_nodes`` budgets for multiway splits + +Unreleased +---------- + +- ✅ Text export of fitted trees with :func:`~sgtlearn.export_text` + +v1.0.0 +------ + +- ⬜ Cross-feature tree binning (similar to + `DPDT `_) +- ⬜ Boosting +- ⬜ Default Optuna support diff --git a/docs/tutorials/bivariate-branching.ipynb b/docs/tutorials/bivariate-branching.ipynb index f169c66..44924e3 100644 --- a/docs/tutorials/bivariate-branching.ipynb +++ b/docs/tutorials/bivariate-branching.ipynb @@ -46,10 +46,10 @@ "id": "imports-data", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:22.582545Z", - "iopub.status.busy": "2026-09-11T17:30:22.582423Z", - "iopub.status.idle": "2026-09-11T17:30:23.241188Z", - "shell.execute_reply": "2026-09-11T17:30:23.240659Z" + "iopub.execute_input": "2026-10-02T20:36:43.644216Z", + "iopub.status.busy": "2026-10-02T20:36:43.644048Z", + "iopub.status.idle": "2026-10-02T20:36:44.496020Z", + "shell.execute_reply": "2026-10-02T20:36:44.495509Z" } }, "outputs": [], @@ -83,10 +83,10 @@ "id": "fit-models", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:23.242368Z", - "iopub.status.busy": "2026-09-11T17:30:23.242257Z", - "iopub.status.idle": "2026-09-11T17:30:23.298040Z", - "shell.execute_reply": "2026-09-11T17:30:23.297700Z" + "iopub.execute_input": "2026-10-02T20:36:44.497152Z", + "iopub.status.busy": "2026-10-02T20:36:44.497047Z", + "iopub.status.idle": "2026-10-02T20:36:44.557441Z", + "shell.execute_reply": "2026-10-02T20:36:44.557071Z" } }, "outputs": [ @@ -122,10 +122,10 @@ "id": "compare-regions", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:23.299513Z", - "iopub.status.busy": "2026-09-11T17:30:23.299445Z", - "iopub.status.idle": "2026-09-11T17:30:23.427588Z", - "shell.execute_reply": "2026-09-11T17:30:23.427238Z" + "iopub.execute_input": "2026-10-02T20:36:44.558513Z", + "iopub.status.busy": "2026-10-02T20:36:44.558459Z", + "iopub.status.idle": "2026-10-02T20:36:44.691550Z", + "shell.execute_reply": "2026-10-02T20:36:44.691205Z" } }, "outputs": [ @@ -176,10 +176,10 @@ "id": "plot-tree", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:23.428802Z", - "iopub.status.busy": "2026-09-11T17:30:23.428734Z", - "iopub.status.idle": "2026-09-11T17:30:23.493195Z", - "shell.execute_reply": "2026-09-11T17:30:23.492812Z" + "iopub.execute_input": "2026-10-02T20:36:44.692641Z", + "iopub.status.busy": "2026-10-02T20:36:44.692584Z", + "iopub.status.idle": "2026-10-02T20:36:44.749540Z", + "shell.execute_reply": "2026-10-02T20:36:44.749149Z" } }, "outputs": [ diff --git a/docs/tutorials/categorical-features.ipynb b/docs/tutorials/categorical-features.ipynb index dd08ee8..9404336 100644 --- a/docs/tutorials/categorical-features.ipynb +++ b/docs/tutorials/categorical-features.ipynb @@ -48,10 +48,10 @@ "id": "c1a1b003", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:24.231247Z", - "iopub.status.busy": "2026-09-11T17:30:24.231106Z", - "iopub.status.idle": "2026-09-11T17:30:24.931010Z", - "shell.execute_reply": "2026-09-11T17:30:24.930731Z" + "iopub.execute_input": "2026-10-02T20:36:45.384524Z", + "iopub.status.busy": "2026-10-02T20:36:45.384301Z", + "iopub.status.idle": "2026-10-02T20:36:46.037073Z", + "shell.execute_reply": "2026-10-02T20:36:46.036780Z" } }, "outputs": [ @@ -138,10 +138,10 @@ "id": "c1a1b006", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:24.932608Z", - "iopub.status.busy": "2026-09-11T17:30:24.932451Z", - "iopub.status.idle": "2026-09-11T17:30:25.974253Z", - "shell.execute_reply": "2026-09-11T17:30:25.973782Z" + "iopub.execute_input": "2026-10-02T20:36:46.038353Z", + "iopub.status.busy": "2026-10-02T20:36:46.038265Z", + "iopub.status.idle": "2026-10-02T20:36:46.154974Z", + "shell.execute_reply": "2026-10-02T20:36:46.154577Z" } }, "outputs": [ @@ -192,10 +192,10 @@ "id": "c1a1b008", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:25.975405Z", - "iopub.status.busy": "2026-09-11T17:30:25.975347Z", - "iopub.status.idle": "2026-09-11T17:30:26.024116Z", - "shell.execute_reply": "2026-09-11T17:30:26.023814Z" + "iopub.execute_input": "2026-10-02T20:36:46.156026Z", + "iopub.status.busy": "2026-10-02T20:36:46.155968Z", + "iopub.status.idle": "2026-10-02T20:36:46.206712Z", + "shell.execute_reply": "2026-10-02T20:36:46.206406Z" } }, "outputs": [ diff --git a/docs/tutorials/feature-importance.ipynb b/docs/tutorials/feature-importance.ipynb index b819500..9d6f3f0 100644 --- a/docs/tutorials/feature-importance.ipynb +++ b/docs/tutorials/feature-importance.ipynb @@ -33,10 +33,10 @@ "id": "930c9644", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:29.046306Z", - "iopub.status.busy": "2026-09-11T17:30:29.046203Z", - "iopub.status.idle": "2026-09-11T17:30:29.747938Z", - "shell.execute_reply": "2026-09-11T17:30:29.747582Z" + "iopub.execute_input": "2026-10-02T20:36:49.231828Z", + "iopub.status.busy": "2026-10-02T20:36:49.231679Z", + "iopub.status.idle": "2026-10-02T20:36:49.881354Z", + "shell.execute_reply": "2026-10-02T20:36:49.881023Z" } }, "outputs": [ @@ -98,10 +98,10 @@ "id": "1b900e07", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:29.749214Z", - "iopub.status.busy": "2026-09-11T17:30:29.749101Z", - "iopub.status.idle": "2026-09-11T17:30:29.778499Z", - "shell.execute_reply": "2026-09-11T17:30:29.778009Z" + "iopub.execute_input": "2026-10-02T20:36:49.882435Z", + "iopub.status.busy": "2026-10-02T20:36:49.882334Z", + "iopub.status.idle": "2026-10-02T20:36:49.909438Z", + "shell.execute_reply": "2026-10-02T20:36:49.909104Z" } }, "outputs": [ @@ -152,10 +152,10 @@ "id": "0939c04e", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:29.779765Z", - "iopub.status.busy": "2026-09-11T17:30:29.779679Z", - "iopub.status.idle": "2026-09-11T17:30:43.751818Z", - "shell.execute_reply": "2026-09-11T17:30:43.751317Z" + "iopub.execute_input": "2026-10-02T20:36:49.910488Z", + "iopub.status.busy": "2026-10-02T20:36:49.910429Z", + "iopub.status.idle": "2026-10-02T20:36:50.209529Z", + "shell.execute_reply": "2026-10-02T20:36:50.209091Z" } }, "outputs": [ diff --git a/docs/tutorials/forests.ipynb b/docs/tutorials/forests.ipynb index 977ae13..11208c4 100644 --- a/docs/tutorials/forests.ipynb +++ b/docs/tutorials/forests.ipynb @@ -16,10 +16,10 @@ "id": "3005b6e4", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:44.493067Z", - "iopub.status.busy": "2026-09-11T17:30:44.492888Z", - "iopub.status.idle": "2026-09-11T17:30:45.278339Z", - "shell.execute_reply": "2026-09-11T17:30:45.277591Z" + "iopub.execute_input": "2026-10-02T20:36:50.787369Z", + "iopub.status.busy": "2026-10-02T20:36:50.787147Z", + "iopub.status.idle": "2026-10-02T20:36:51.555780Z", + "shell.execute_reply": "2026-10-02T20:36:51.555419Z" } }, "outputs": [ @@ -79,10 +79,10 @@ "id": "8589daec", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:45.279763Z", - "iopub.status.busy": "2026-09-11T17:30:45.279573Z", - "iopub.status.idle": "2026-09-11T17:30:45.477090Z", - "shell.execute_reply": "2026-09-11T17:30:45.476657Z" + "iopub.execute_input": "2026-10-02T20:36:51.557699Z", + "iopub.status.busy": "2026-10-02T20:36:51.557453Z", + "iopub.status.idle": "2026-10-02T20:36:51.745454Z", + "shell.execute_reply": "2026-10-02T20:36:51.745042Z" } }, "outputs": [ @@ -92,13 +92,7 @@ "text": [ "n_estimators= 5: test=0.912\n", "n_estimators= 25: test=0.953\n", - "n_estimators=100: test=0.965\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ + "n_estimators=100: test=0.965\n", "bootstrap 60% rows, log2 feats: test=0.959\n" ] } @@ -135,10 +129,10 @@ "id": "853d5a48", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:45.478271Z", - "iopub.status.busy": "2026-09-11T17:30:45.478189Z", - "iopub.status.idle": "2026-09-11T17:30:45.858649Z", - "shell.execute_reply": "2026-09-11T17:30:45.858355Z" + "iopub.execute_input": "2026-10-02T20:36:51.746475Z", + "iopub.status.busy": "2026-10-02T20:36:51.746408Z", + "iopub.status.idle": "2026-10-02T20:36:52.108129Z", + "shell.execute_reply": "2026-10-02T20:36:52.107816Z" } }, "outputs": [ diff --git a/docs/tutorials/inspecting-trees.ipynb b/docs/tutorials/inspecting-trees.ipynb index 39dd075..9d840cd 100644 --- a/docs/tutorials/inspecting-trees.ipynb +++ b/docs/tutorials/inspecting-trees.ipynb @@ -16,10 +16,10 @@ "id": "82dee33a", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:48.904818Z", - "iopub.status.busy": "2026-09-11T17:30:48.904639Z", - "iopub.status.idle": "2026-09-11T17:30:49.541154Z", - "shell.execute_reply": "2026-09-11T17:30:49.540744Z" + "iopub.execute_input": "2026-10-02T20:36:55.148755Z", + "iopub.status.busy": "2026-10-02T20:36:55.148634Z", + "iopub.status.idle": "2026-10-02T20:36:55.729204Z", + "shell.execute_reply": "2026-10-02T20:36:55.728839Z" } }, "outputs": [ @@ -50,10 +50,10 @@ "id": "340a2dc2", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:49.542308Z", - "iopub.status.busy": "2026-09-11T17:30:49.542172Z", - "iopub.status.idle": "2026-09-11T17:30:49.544094Z", - "shell.execute_reply": "2026-09-11T17:30:49.543797Z" + "iopub.execute_input": "2026-10-02T20:36:55.730315Z", + "iopub.status.busy": "2026-10-02T20:36:55.730218Z", + "iopub.status.idle": "2026-10-02T20:36:55.731902Z", + "shell.execute_reply": "2026-10-02T20:36:55.731623Z" } }, "outputs": [ @@ -141,10 +141,10 @@ "id": "80285e60", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:49.545285Z", - "iopub.status.busy": "2026-09-11T17:30:49.545224Z", - "iopub.status.idle": "2026-09-11T17:30:49.596049Z", - "shell.execute_reply": "2026-09-11T17:30:49.595603Z" + "iopub.execute_input": "2026-10-02T20:36:55.732728Z", + "iopub.status.busy": "2026-10-02T20:36:55.732674Z", + "iopub.status.idle": "2026-10-02T20:36:55.778362Z", + "shell.execute_reply": "2026-10-02T20:36:55.777990Z" } }, "outputs": [ @@ -182,10 +182,10 @@ "id": "fa1c106b", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:49.597122Z", - "iopub.status.busy": "2026-09-11T17:30:49.597067Z", - "iopub.status.idle": "2026-09-11T17:30:49.599372Z", - "shell.execute_reply": "2026-09-11T17:30:49.599055Z" + "iopub.execute_input": "2026-10-02T20:36:55.779370Z", + "iopub.status.busy": "2026-10-02T20:36:55.779319Z", + "iopub.status.idle": "2026-10-02T20:36:55.781608Z", + "shell.execute_reply": "2026-10-02T20:36:55.781260Z" } }, "outputs": [ diff --git a/docs/tutorials/regression.ipynb b/docs/tutorials/regression.ipynb index b5fe661..6856232 100644 --- a/docs/tutorials/regression.ipynb +++ b/docs/tutorials/regression.ipynb @@ -16,10 +16,10 @@ "id": "f3faf212", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:50.317527Z", - "iopub.status.busy": "2026-09-11T17:30:50.317405Z", - "iopub.status.idle": "2026-09-11T17:30:51.028231Z", - "shell.execute_reply": "2026-09-11T17:30:51.027856Z" + "iopub.execute_input": "2026-10-02T20:36:56.365315Z", + "iopub.status.busy": "2026-10-02T20:36:56.365142Z", + "iopub.status.idle": "2026-10-02T20:36:57.019681Z", + "shell.execute_reply": "2026-10-02T20:36:57.019246Z" } }, "outputs": [ @@ -61,10 +61,10 @@ "id": "858de2af", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:51.029436Z", - "iopub.status.busy": "2026-09-11T17:30:51.029311Z", - "iopub.status.idle": "2026-09-11T17:30:51.137858Z", - "shell.execute_reply": "2026-09-11T17:30:51.137461Z" + "iopub.execute_input": "2026-10-02T20:36:57.020883Z", + "iopub.status.busy": "2026-10-02T20:36:57.020767Z", + "iopub.status.idle": "2026-10-02T20:36:57.119660Z", + "shell.execute_reply": "2026-10-02T20:36:57.119281Z" } }, "outputs": [ @@ -80,7 +80,7 @@ "name": "stderr", "output_type": "stream", "text": [ - "/var/folders/l5/dnbhhcmd4153b41c11_2v7p00000gn/T/ipykernel_89475/2183533904.py:2: UserWarning: Coordinate descent is disabled for the MAE objective. Set SGTLEARN_MAE_CD=1 to enable it.\n", + "/var/folders/l5/dnbhhcmd4153b41c11_2v7p00000gn/T/ipykernel_86266/2183533904.py:2: UserWarning: Coordinate descent is disabled for the MAE objective. Set SGTLEARN_MAE_CD=1 to enable it.\n", " r = SGTRegressor(max_depth=4, inner_max_depth=3, criterion=crit, random_state=0).fit(X_tr, y_tr)\n" ] } @@ -107,10 +107,10 @@ "id": "80e71be5", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:51.138914Z", - "iopub.status.busy": "2026-09-11T17:30:51.138857Z", - "iopub.status.idle": "2026-09-11T17:30:51.298894Z", - "shell.execute_reply": "2026-09-11T17:30:51.298513Z" + "iopub.execute_input": "2026-10-02T20:36:57.120770Z", + "iopub.status.busy": "2026-10-02T20:36:57.120714Z", + "iopub.status.idle": "2026-10-02T20:36:57.261857Z", + "shell.execute_reply": "2026-10-02T20:36:57.261406Z" } }, "outputs": [ diff --git a/docs/tutorials/sgt-k.ipynb b/docs/tutorials/sgt-k.ipynb index 301378b..e040fd2 100644 --- a/docs/tutorials/sgt-k.ipynb +++ b/docs/tutorials/sgt-k.ipynb @@ -28,10 +28,10 @@ "id": "80178c34", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:54.287994Z", - "iopub.status.busy": "2026-09-11T17:30:54.287905Z", - "iopub.status.idle": "2026-09-11T17:30:54.957388Z", - "shell.execute_reply": "2026-09-11T17:30:54.956910Z" + "iopub.execute_input": "2026-10-02T20:37:00.297610Z", + "iopub.status.busy": "2026-10-02T20:37:00.297407Z", + "iopub.status.idle": "2026-10-02T20:37:00.916027Z", + "shell.execute_reply": "2026-10-02T20:37:00.915624Z" } }, "outputs": [ @@ -65,10 +65,10 @@ "id": "fa0aff56", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:54.958793Z", - "iopub.status.busy": "2026-09-11T17:30:54.958666Z", - "iopub.status.idle": "2026-09-11T17:30:55.047092Z", - "shell.execute_reply": "2026-09-11T17:30:55.046667Z" + "iopub.execute_input": "2026-10-02T20:37:00.917099Z", + "iopub.status.busy": "2026-10-02T20:37:00.917009Z", + "iopub.status.idle": "2026-10-02T20:37:01.002828Z", + "shell.execute_reply": "2026-10-02T20:37:01.002447Z" } }, "outputs": [ @@ -112,10 +112,10 @@ "id": "c84fcaa0", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:55.048161Z", - "iopub.status.busy": "2026-09-11T17:30:55.048097Z", - "iopub.status.idle": "2026-09-11T17:30:55.055925Z", - "shell.execute_reply": "2026-09-11T17:30:55.055543Z" + "iopub.execute_input": "2026-10-02T20:37:01.003946Z", + "iopub.status.busy": "2026-10-02T20:37:01.003882Z", + "iopub.status.idle": "2026-10-02T20:37:01.011708Z", + "shell.execute_reply": "2026-10-02T20:37:01.011437Z" } }, "outputs": [ diff --git a/docs/tutorials/shape-functions.ipynb b/docs/tutorials/shape-functions.ipynb index 77f8ea8..d23a61f 100644 --- a/docs/tutorials/shape-functions.ipynb +++ b/docs/tutorials/shape-functions.ipynb @@ -29,10 +29,10 @@ "id": "2bf2fb54", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:55.799127Z", - "iopub.status.busy": "2026-09-11T17:30:55.799005Z", - "iopub.status.idle": "2026-09-11T17:30:56.577780Z", - "shell.execute_reply": "2026-09-11T17:30:56.577379Z" + "iopub.execute_input": "2026-10-02T20:37:01.625172Z", + "iopub.status.busy": "2026-10-02T20:37:01.625046Z", + "iopub.status.idle": "2026-10-02T20:37:02.304860Z", + "shell.execute_reply": "2026-10-02T20:37:02.304417Z" } }, "outputs": [ @@ -78,10 +78,10 @@ "id": "06d91910", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:56.579011Z", - "iopub.status.busy": "2026-09-11T17:30:56.578893Z", - "iopub.status.idle": "2026-09-11T17:30:56.585109Z", - "shell.execute_reply": "2026-09-11T17:30:56.584694Z" + "iopub.execute_input": "2026-10-02T20:37:02.306030Z", + "iopub.status.busy": "2026-10-02T20:37:02.305922Z", + "iopub.status.idle": "2026-10-02T20:37:02.311597Z", + "shell.execute_reply": "2026-10-02T20:37:02.311335Z" } }, "outputs": [ diff --git a/docs/tutorials/structure-and-accuracy.ipynb b/docs/tutorials/structure-and-accuracy.ipynb index 8d714c4..c3d36fa 100644 --- a/docs/tutorials/structure-and-accuracy.ipynb +++ b/docs/tutorials/structure-and-accuracy.ipynb @@ -26,10 +26,10 @@ "id": "b4fa8666", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:57.288145Z", - "iopub.status.busy": "2026-09-11T17:30:57.288013Z", - "iopub.status.idle": "2026-09-11T17:30:58.009440Z", - "shell.execute_reply": "2026-09-11T17:30:58.009040Z" + "iopub.execute_input": "2026-10-02T20:37:02.931991Z", + "iopub.status.busy": "2026-10-02T20:37:02.931842Z", + "iopub.status.idle": "2026-10-02T20:37:03.570642Z", + "shell.execute_reply": "2026-10-02T20:37:03.570116Z" } }, "outputs": [], @@ -56,10 +56,10 @@ "id": "b248e1d7", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:58.011010Z", - "iopub.status.busy": "2026-09-11T17:30:58.010901Z", - "iopub.status.idle": "2026-09-11T17:30:58.124625Z", - "shell.execute_reply": "2026-09-11T17:30:58.124309Z" + "iopub.execute_input": "2026-10-02T20:37:03.571851Z", + "iopub.status.busy": "2026-10-02T20:37:03.571748Z", + "iopub.status.idle": "2026-10-02T20:37:03.672937Z", + "shell.execute_reply": "2026-10-02T20:37:03.672611Z" } }, "outputs": [ @@ -115,10 +115,10 @@ "id": "81a0cd41", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:58.126241Z", - "iopub.status.busy": "2026-09-11T17:30:58.126138Z", - "iopub.status.idle": "2026-09-11T17:30:58.155552Z", - "shell.execute_reply": "2026-09-11T17:30:58.155228Z" + "iopub.execute_input": "2026-10-02T20:37:03.674014Z", + "iopub.status.busy": "2026-10-02T20:37:03.673949Z", + "iopub.status.idle": "2026-10-02T20:37:03.700422Z", + "shell.execute_reply": "2026-10-02T20:37:03.700128Z" } }, "outputs": [ @@ -147,10 +147,10 @@ "id": "06903b36", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:58.156698Z", - "iopub.status.busy": "2026-09-11T17:30:58.156630Z", - "iopub.status.idle": "2026-09-11T17:30:58.220198Z", - "shell.execute_reply": "2026-09-11T17:30:58.219828Z" + "iopub.execute_input": "2026-10-02T20:37:03.701567Z", + "iopub.status.busy": "2026-10-02T20:37:03.701514Z", + "iopub.status.idle": "2026-10-02T20:37:03.758957Z", + "shell.execute_reply": "2026-10-02T20:37:03.758554Z" } }, "outputs": [ diff --git a/docs/tutorials/tao.ipynb b/docs/tutorials/tao.ipynb index b8a80b0..72f3b9a 100644 --- a/docs/tutorials/tao.ipynb +++ b/docs/tutorials/tao.ipynb @@ -29,10 +29,10 @@ "id": "dc4c7dc9", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:58.946880Z", - "iopub.status.busy": "2026-09-11T17:30:58.946704Z", - "iopub.status.idle": "2026-09-11T17:30:59.691047Z", - "shell.execute_reply": "2026-09-11T17:30:59.690642Z" + "iopub.execute_input": "2026-10-02T20:37:04.355963Z", + "iopub.status.busy": "2026-10-02T20:37:04.355755Z", + "iopub.status.idle": "2026-10-02T20:37:05.040225Z", + "shell.execute_reply": "2026-10-02T20:37:05.039845Z" } }, "outputs": [ @@ -94,10 +94,10 @@ "id": "53ae5924", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:59.692729Z", - "iopub.status.busy": "2026-09-11T17:30:59.692591Z", - "iopub.status.idle": "2026-09-11T17:30:59.952805Z", - "shell.execute_reply": "2026-09-11T17:30:59.952395Z" + "iopub.execute_input": "2026-10-02T20:37:05.041443Z", + "iopub.status.busy": "2026-10-02T20:37:05.041332Z", + "iopub.status.idle": "2026-10-02T20:37:05.286896Z", + "shell.execute_reply": "2026-10-02T20:37:05.286585Z" } }, "outputs": [ @@ -107,20 +107,8 @@ "text": [ "lambda= 0.000: train=0.863 test=0.697\n", "lambda= 0.003: train=0.863 test=0.697\n", - "lambda= 0.010: train=0.856 test=0.700\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "lambda= 0.030: train=0.821 test=0.737\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ + "lambda= 0.010: train=0.856 test=0.700\n", + "lambda= 0.030: train=0.821 test=0.737\n", "lambda= 0.100: train=0.700 test=0.700\n" ] } @@ -163,10 +151,10 @@ "id": "aad2b1b6", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:30:59.954036Z", - "iopub.status.busy": "2026-09-11T17:30:59.953963Z", - "iopub.status.idle": "2026-09-11T17:31:00.086970Z", - "shell.execute_reply": "2026-09-11T17:31:00.086585Z" + "iopub.execute_input": "2026-10-02T20:37:05.288198Z", + "iopub.status.busy": "2026-10-02T20:37:05.288126Z", + "iopub.status.idle": "2026-10-02T20:37:05.400902Z", + "shell.execute_reply": "2026-10-02T20:37:05.400571Z" } }, "outputs": [ @@ -209,10 +197,10 @@ "id": "b89dadaf", "metadata": { "execution": { - "iopub.execute_input": "2026-09-11T17:31:00.088275Z", - "iopub.status.busy": "2026-09-11T17:31:00.088191Z", - "iopub.status.idle": "2026-09-11T17:31:00.099746Z", - "shell.execute_reply": "2026-09-11T17:31:00.099351Z" + "iopub.execute_input": "2026-10-02T20:37:05.402250Z", + "iopub.status.busy": "2026-10-02T20:37:05.402186Z", + "iopub.status.idle": "2026-10-02T20:37:05.412732Z", + "shell.execute_reply": "2026-10-02T20:37:05.412432Z" } }, "outputs": [ diff --git a/example.py b/example.py index 5a6ccc0..2e600cc 100644 --- a/example.py +++ b/example.py @@ -29,7 +29,7 @@ ax_tree.set_title("Learned SGT") -# left: the checkerboard +# left: the Plus Sign dataset ax_data.scatter( X_train[:, 0], X_train[:, 1], c=y_train, cmap="coolwarm", s=12, edgecolors="none" ) diff --git a/examples/plot_tree_demo.py b/examples/plot_tree_demo.py index cde6d1e..0df0ff1 100644 --- a/examples/plot_tree_demo.py +++ b/examples/plot_tree_demo.py @@ -176,7 +176,7 @@ def save(fig: plt.Figure, name: str) -> Path: # --------------------------------------------------------------------------- -# 7. Multi-way partitioning — ``num_partitions > 2`` fans out K children. +# 7. Multi-way partitioning — ``num_partitions > 2`` fans out up to K children. # --------------------------------------------------------------------------- # The default pastel sequence has 8 colors; partitions beyond that cycle. For # a K-way tree, ``cmap`` should provide at least K distinguishable colors. diff --git a/sgtlearn/__init__.py b/sgtlearn/__init__.py index 328d9b4..c354463 100644 --- a/sgtlearn/__init__.py +++ b/sgtlearn/__init__.py @@ -1,7 +1,8 @@ -"""Shape-generalized tree estimators (classification), text export and plotting. +"""Shape-generalized trees and forests, TAO refinement, text export and plotting. -The heavy lifting lives in optional native extensions (``ShapeGeneralizedTrees``, -``Discretizers``). Import ``SGTClassifier`` from this package for the sklearn-style API. +The heavy lifting lives in compiled native extensions (``ShapeGeneralizedTrees``, +``TreeAlternatingOptimization``). Import the estimators from this package for the +sklearn-style API. """ from sgtlearn import tao diff --git a/sgtlearn/_export.py b/sgtlearn/_export.py index acf6ccb..ffae3c2 100644 --- a/sgtlearn/_export.py +++ b/sgtlearn/_export.py @@ -1,9 +1,10 @@ """Tree visualization for fitted SGT estimators. ``plot_tree`` renders a fitted ``SGTClassifier`` / ``SGTRegressor`` with -matplotlib: every internal node is drawn as a small histogram of the chosen -routing feature with bins colored by the destination child partition, and -every leaf is drawn as a text box with the predicted class / value. +matplotlib: every internal node is drawn as a small panel of its routing +bins colored by the destination child partition (a two-feature heatmap for +pair nodes) plus histograms of ``X`` when it is supplied, and every leaf is +drawn as colored text with the predicted class / value. ``export_text`` returns the same tree as indented routing rules. """ @@ -286,9 +287,8 @@ def _format_region( #: Hand-picked pastel sequence used as the default ``cmap`` for ``plot_tree``. -#: The first two colors (pink, peach) match the reference visualization -#: aesthetic in ``ex_viz/*.png``; the remaining colors come from matplotlib's -#: Pastel1 palette in a deterministic order. +#: The first two colors are pink and peach; the remaining colors come from +#: matplotlib's Pastel1 palette in a deterministic order. _DEFAULT_PALETTE_COLORS = ( "#E8A0BF", # pink "#FAC898", # peach @@ -490,17 +490,19 @@ def _merge_routing_regions( def _route_samples(tree: dict, X) -> dict[int, Any]: - """Route ``X`` through the tree; return ``{node_id: column-indices}``. + """Route ``X`` through the tree; return ``{node_id: row-indices}``. The returned array for each node lists the row indices of ``X`` that reach that node. Leaves' sample sets partition the root's sample set. Routing rule (matches the native router): ``X`` is rounded to float32 - as in ``predict``. At each internal node, look up the routing feature + as in ``predict``. At a univariate node, look up the routing feature column. Non-finite values use ``nan_prediction_partition``; finite values use ``bin = np.searchsorted(thresholds, value, side='left')`` (a value on a threshold takes the lower bin, like ``std::lower_bound``) and - ``children[bin_to_partition[bin]]``. + ``children[bin_to_partition[bin]]``. A categorical node maps the active + one-hot column to its ``bin_categories`` bin (no active column uses + ``nan_prediction_partition``); a pair node replays ``pair_inner_tree``. """ X_arr = np.asarray(X, dtype=np.float32).astype(np.float64) @@ -579,8 +581,8 @@ def _compute_layout_leafcounter( Each visible leaf (or depth-cap-truncated internal node, which renders as a draw-leaf) claims the next integer x in DFS left-to-right order; each visible internal node's x = mean of its drawn children's x. y is set - from depth and rescaled into the band ``[0.03, 0.88]`` (top 12% reserved - for annotations, bottom 3% margin). + from depth and rescaled into the band ``[0.10, 0.88]`` (top 12% reserved + for annotations, bottom 10% margin). """ nodes_by_id = {n["id"]: n for n in tree["nodes"]} root = tree["root_index"] @@ -1107,7 +1109,11 @@ def _draw_internal_panel_pair( n_hist_bins: int, precision: int, ) -> list: - """Draw exported pair-routing cells, including the two missing margins.""" + """Draw exported pair-routing cells and their missing-value margins. + + Without ``X_rows`` both margins are drawn; with it, only axes whose rows + contain missing values get one. + """ cx, cy = center w, h = size left, bottom = cx - w / 2, cy - h / 2 @@ -1368,9 +1374,10 @@ def plot_tree( """Render a fitted SGT estimator with matplotlib. Univariate nodes show their one-dimensional shape function. Bivariate - nodes show the exact two-dimensional routing heatmap; when ``X`` is - supplied, top/right marginal histograms and observed missing-value margins - are included. Continuous axes label only thresholds where the final outer + nodes show the exact two-dimensional routing heatmap with a missing-value + margin on each axis; when ``X`` is supplied, top/right marginal histograms + are added and only axes with observed missing values keep a margin. + Continuous axes label only thresholds where the final outer partition changes. Parameters @@ -1378,23 +1385,31 @@ def plot_tree( estimator : SGTClassifier or SGTRegressor Fitted estimator to render. X : array-like of shape (n_samples, n_features), optional - Data used for node sample counts and bivariate marginal histograms. + Data routed through the tree to draw node histograms (univariate, + categorical and bivariate marginal) and missing-value margins. The + ``n=`` labels always come from the fitted tree, not from ``X``. max_depth : int, optional Maximum outer-tree depth to display. feature_names : list of str, optional Display names for input columns. class_names : list of str, bool, or None, default=None - Class labels shown on classifier leaves. + Class labels shown on classifier leaves, in ``classes_`` order. + ``True`` uses ``classes_``; ``None`` or ``False`` shows the encoded + class index (``0``, ``1``, ...). label : str, default="feature" - Controls field-name labels in node text. + Sample-count annotations: ``"feature"`` shows ``n=`` on internal + nodes, ``"all"`` also adds each leaf's ``n`` (and impurity when + ``impurity=True``), and ``"none"`` hides them. impurity : bool, default=False - Whether to display node impurity. + Whether to display leaf impurity; only used with ``label="all"``. proportion : bool, default=False - Whether to display sample proportions instead of counts. + Currently ignored; node annotations always show sample counts. precision : int, default=2 Decimal precision for displayed values and routing thresholds. - cmap : colormap or color sequence, optional - Colors for classes and routing partitions. + cmap : str, Colormap, or list or tuple of colors, default=pastel palette + Colors for the routing partitions: edges, routing bins and leaf text + are colored by the child position they lead to. A list or tuple is + cycled; a colormap (or its registered name) is sampled evenly. ax : matplotlib.axes.Axes, optional Axes on which to draw. fontsize : int, optional @@ -1402,7 +1417,8 @@ def plot_tree( node_aspect_ratio : float, default=2.5 Width-to-height ratio of node boxes. n_hist_bins : int, default=20 - Number of bins in each bivariate marginal histogram. + Number of histogram bins in each univariate node panel and each + continuous bivariate marginal histogram (used only when ``X`` is given). Returns ------- diff --git a/sgtlearn/_features.py b/sgtlearn/_features.py index 2763551..c100483 100644 --- a/sgtlearn/_features.py +++ b/sgtlearn/_features.py @@ -22,10 +22,14 @@ class ProcessedFeatures: List of ``{"type": "continuous"|"categorical", "indices": [...]}`` dicts in trainer order. Index ``i`` aligns with ``estimator.feature_importances_[i]`` after a fit without TAO. + Without ``feature_dict``, entry ``i`` is column ``i``. With one, entries + are re-sorted (integer keys, including auto-filled columns, before + string keys, each ordered by ``str(key)``, so ``10`` sorts before + ``9``), so map entries back to ``X`` through ``indices``. logical_names Parallel names for ``features``. When ``feature_dict`` is supplied, - these are the stringified keys; omitted columns are filled as - ``\"0\"``, ``\"1\"``, …. Default (no ``feature_dict``) uses + these are the stringified keys; each omitted column ``i`` is named + ``str(i)``. Default (no ``feature_dict``) uses ``\"0\"`` … ``\"n_features-1\"`` even if ``X`` is a pandas DataFrame (DataFrame column names are stored on ``feature_names_in_`` instead). """ diff --git a/sgtlearn/_multioutput.py b/sgtlearn/_multioutput.py index e998d8e..166f888 100644 --- a/sgtlearn/_multioutput.py +++ b/sgtlearn/_multioutput.py @@ -1,7 +1,7 @@ """Helpers so single-output is the multi-output path with ``n_outputs == 1``. Public sklearn shapes still unwrap at API boundaries (1-D ``y`` / ``predict``, -scalar ``classes_`` / ``n_classes_``, ndarray ``predict_proba``). Internally +1-D ``classes_`` and int ``n_classes_``, ndarray ``predict_proba``). Internally targets are always treated as ``(n_samples, n_outputs)``. """ @@ -93,7 +93,7 @@ def unwrap_classifier_public_attrs( n_classes_list: Sequence[int], n_outputs: int, ) -> tuple[Any, Any, Any, Any]: - """Sklearn public forms: scalar attrs when ``n_outputs == 1``, else lists. + """Sklearn public forms: unwrapped when ``n_outputs == 1``, else lists. Returns ``(_le, classes_, n_classes_, n_classes_native)`` where ``n_classes_native`` is what the C++ constructor expects (``int`` or @@ -128,7 +128,7 @@ def label_encoders_as_list(encoders: Any | Sequence[Any], n_outputs: int) -> lis def as_native_float32(values: Any, name: str) -> np.ndarray: - """C-contiguous ``float32`` copy of ``values``; rejects infinity. + """C-contiguous ``float32`` ``values``, copied only if needed; rejects infinity. A finite value beyond the ``float32`` range (e.g. ``1e39``) passes float64 validation but becomes ``inf`` here, so it is rejected too. diff --git a/sgtlearn/base.py b/sgtlearn/base.py index 7001d14..1da2d1f 100644 --- a/sgtlearn/base.py +++ b/sgtlearn/base.py @@ -159,9 +159,9 @@ class _IdentityLabelEncoder(LabelEncoder): Meta-estimators (e.g. :class:`~sgtlearn.ensemble.RandomSGForestClassifier`) fit a full :class:`~sklearn.preprocessing.LabelEncoder` once and train base - trees on integer ``y``. Each base tree holds ``classes_ = np.arange(K)`` so - :meth:`inverse_transform` follows the same sklearn contract as a fitted - :class:`~sklearn.preprocessing.LabelEncoder` without re-encoding labels. + trees on integer ``y``. Each base tree holds the meta-estimator's original + ``classes_``, so :meth:`transform` passes the integer codes through and + :meth:`inverse_transform` maps them back to the original labels. """ def __init__(self, classes_: np.ndarray) -> None: @@ -195,7 +195,10 @@ def feature_importances_(self) -> np.ndarray: Length matches the number of logical features passed to ``fit`` (one-to-one with :attr:`processed_features_`). Available only after - training without TAO refinement. + training with ``tao_n_runs=0`` (the default ``tao_n_runs=10`` runs TAO) + and no later TAO refinement; otherwise raises ``AttributeError``. Warns + when the tree has pair nodes, whose gain is split equally between both + features. """ check_is_fitted(self, attributes=("_est",)) if getattr(self, "_tao_refined_", False): @@ -218,7 +221,7 @@ def processed_features_(self) -> ProcessedFeatures: ``features[i]`` and ``logical_names[i]`` correspond to ``feature_importances_[i]``. ``logical_names`` are the ``feature_dict`` - keys (stringified); omitted columns are filled as ``\"0\"``, ``\"1\"``, … + keys (stringified); each omitted column ``i`` is named ``str(i)``. """ check_is_fitted(self, attributes=("_processed_features",)) return self._processed_features @@ -286,7 +289,7 @@ class SGTClassifier(ClassifierMixin, BaseShapeCART): *shape function*. By default the shape function is a depth-limited inner tree over one logical feature; ``pairwise_candidates > 0`` also lets it use an ordinary axis-aligned CART over two logical features. The outer tree - routes the resulting bins into ``num_partitions`` children. Training is + routes those bins into at most ``num_partitions`` children. Training is performed by the native ShapeCART C++ trainer exposed through ``ClassificationShapeGeneralizedTree``. @@ -295,13 +298,15 @@ class SGTClassifier(ClassifierMixin, BaseShapeCART): Parameters ---------- - criterion : {"gini", "entropy"}, default="gini" - Impurity criterion used at outer-tree splits, forwarded to the native - trainer. + criterion : {"gini", "entropy", "log_loss"}, default="gini" + Impurity criterion used both to grow each node's inner shape-function + tree and to score outer-tree splits (``"log_loss"`` is an alias of + ``"entropy"``). num_partitions : int, default=2 - Number of branches each outer split fans out into (i.e. the arity of - the shape function). ``2`` reproduces standard binary tree; larger - values yield the ``SGT_K`` multi-way variant. + Maximum number of branches each outer split fans out into; a split may + have fewer children. Must be at least ``2``. ``2`` reproduces a + standard binary tree; larger values yield the ``SGT_K`` multi-way + variant. max_depth : int, optional Maximum depth of the *outer* tree. ``None`` (default) means grow until another stopping criterion fires. @@ -332,14 +337,14 @@ class SGTClassifier(ClassifierMixin, BaseShapeCART): random_state : int, optional, default=42 Seed forwarded to the native trainer. ``None`` is treated as ``42``. max_features : int, float, {"sqrt", "log2"} or None, default=None - Number of features sampled (without replacement) when searching for - a split: + Number of logical features sampled (without replacement) when + searching for a split: - - ``None``: use all ``n_features`` columns. - - ``int k >= 1``: use ``min(k, n_features)`` columns. - - ``float c in (0, 1]``: use ``max(1, int(c * n_features))`` columns. - - ``"sqrt"``: use ``max(1, int(sqrt(n_features)))`` columns. - - ``"log2"``: use ``max(1, int(log2(n_features)))`` columns. + - ``None``: use all ``n_logical_features`` features. + - ``int k >= 1``: use ``min(k, n_logical_features)`` features. + - ``float c in (0, 1]``: use ``max(1, int(c * n_logical_features))`` features. + - ``"sqrt"``: use ``max(1, int(sqrt(n_logical_features)))`` features. + - ``"log2"``: use ``max(1, int(log2(n_logical_features)))`` features. String values are case-insensitive in the native binding. pairwise_candidates : int or float, default=0 @@ -351,6 +356,15 @@ class SGTClassifier(ClassifierMixin, BaseShapeCART): sample-weighted, output-averaged impurity improvement. branching_penalty : float, default=0.0 Constant nonnegative cost per additional occupied child beyond two. + tao_n_runs : int, default=10 + Maximum number of Tree Alternating Optimization (TAO) sweeps run at the + end of :meth:`fit` (see :func:`~sgtlearn.tao.TAO_refine`); sweeps stop + early once one makes no change. ``0`` disables TAO; any positive value + makes ``feature_importances_`` unavailable. + tao_lambda : float, default=0.0 + TAO per-sample complexity rate: at each internal node a non-constant + routing rule must beat the constant rule by more than ``tao_lambda`` + times the number of samples reaching the node (weighted reward units). tao_pair_scale : float, default=1.1 Multiplier applied to ``tao_lambda`` for pair routers during TAO. class_weight : dict, list of dict, or None, default=None @@ -371,16 +385,21 @@ class SGTClassifier(ClassifierMixin, BaseShapeCART): Number of target columns (``1`` for a 1-D ``y``). n_features_in_ : int Number of features in ``X`` passed to :meth:`fit`. + feature_names_in_ : ndarray of shape (n_features,) or None + Column names, converted to ``str``, of an ``X`` with a ``columns`` + attribute (e.g. a pandas ``DataFrame``) seen during :meth:`fit`; + ``None`` otherwise. feature_importances_ : ndarray of shape (n_logical_features,) Normalized impurity-based importances from the fitted tree. Index ``i`` corresponds to :attr:`processed_features_` entry ``i`` (same - order as the logical features passed to the native trainer). Available - only when the model has not undergone TAO refinement. + order as the logical features passed to the native trainer). Raises + ``AttributeError`` after TAO refinement, which runs by default + (``tao_n_runs=10``); fit with ``tao_n_runs=0`` to use it. processed_features_ : ProcessedFeatures Resolved logical features used at :meth:`fit`. ``features[i]`` and ``logical_names[i]`` align with ``feature_importances_[i]``. - ``logical_names`` are stringified ``feature_dict`` keys (auto-filled - columns use ``\"0\"``, ``\"1\"``, …). + ``logical_names`` are stringified ``feature_dict`` keys (an + auto-filled column ``i`` is named ``str(i)``). Notes ----- @@ -394,9 +413,11 @@ class SGTClassifier(ClassifierMixin, BaseShapeCART): its training NaNs (or the child with the most training samples when the node saw none). At inference, missing values go to that child regardless of where the largest finite values go. Pair nodes route missing values - through dedicated branches of their pair router. Infinity in ``X`` and - ``sample_weight`` is rejected, as are finite values that overflow - ``float32`` (e.g. ``1e39``), including ``sample_weight * class_weight``. + through their pair router: a pair-tree split sends values missing on its + axis to a dedicated branch when its training samples had any, otherwise + to the larger finite child. Infinity in ``X`` and ``sample_weight`` is + rejected, as are finite values that overflow ``float32`` (e.g. ``1e39``), + including ``sample_weight * class_weight``. References ---------- @@ -406,7 +427,7 @@ class SGTClassifier(ClassifierMixin, BaseShapeCART): See Also -------- SGTRegressor : Regression counterpart. - sgtlearn.ensemble.RandomSGForestClassifier : Bootstrap ensemble over + sgtlearn.RandomSGForestClassifier : Bootstrap ensemble over :class:`SGTClassifier`. Examples @@ -507,10 +528,9 @@ def fit( ``{logical_name: [columns]}`` grouping of columns into logical features. Keys are ``int`` or ``str`` logical names. Values are column indices (``int``) or column names (``str``); names require a - pandas ``DataFrame`` ``X`` or an explicit ``column_names``. A group - of more than one column is treated as a single **categorical** - feature (routed through the one-hot inner discretizer); singletons - are **continuous**. Columns not mentioned default to continuous + pandas ``DataFrame`` ``X``. A group of more than one column is + treated as a single **categorical** feature (routed through the + one-hot inner discretizer); singletons are **continuous**. Columns not mentioned default to continuous singletons. See :func:`~sgtlearn.configure_feature_dict` for the full resolution rules. processed_features : ProcessedFeatures, optional @@ -518,7 +538,9 @@ def fit( ensembles that should resolve features once). check_input : bool, default=True If ``False``, ``X`` and ``y`` are not validated (for callers that - already ran :func:`~sklearn.utils.validation.check_X_y`). + already ran :func:`~sklearn.utils.check_X_y`). The caller must then + pass ``y`` as integer codes ``0 .. n_classes - 1`` and preset + ``classes_`` and ``n_classes_``. """ column_names = _column_names_from_X(X) @@ -722,8 +744,9 @@ def predict_proba(self, X: np.ndarray) -> Any: def tree_export(self) -> dict: """Return a flat dict snapshot of the fitted tree. - See ``sgtlearn._export.plot_tree`` for the canonical consumer. Keys: - ``num_partitions``, ``num_nodes``, ``root_index``, ``num_classes``, + Consumed by :func:`~sgtlearn.plot_tree` and :func:`~sgtlearn.export_text`. + Keys: ``num_partitions``, ``num_nodes``, ``root_index``, ``num_classes`` (one + entry per output), ``num_outputs``, ``classes_per_output``, ``criterion``, ``nodes`` (list of per-node dicts). """ check_is_fitted(self, attributes=("_est", "_le")) @@ -747,12 +770,14 @@ class SGTRegressor(RegressorMixin, BaseShapeCART): Parameters ---------- criterion : {"squared_error", "mse", "absolute_error", "mae"}, default="squared_error" - Loss used at outer-tree splits. ``"mse"`` and ``"mae"`` are accepted - as aliases. + Loss used both to grow each node's inner shape-function tree and to + score outer-tree splits. ``"mse"`` and ``"mae"`` are accepted as + aliases. num_partitions : int, default=2 - Number of branches each outer split fans out into (i.e. the arity of - the shape function). ``2`` reproduces standard binary tree; larger - values yield the ``SGT_K`` multi-way variant. + Maximum number of branches each outer split fans out into; a split may + have fewer children. Must be at least ``2``. ``2`` reproduces a + standard binary tree; larger values yield the ``SGT_K`` multi-way + variant. max_depth : int, optional Maximum depth of the outer tree. ``None`` means grow until another stopping criterion fires. @@ -793,6 +818,15 @@ class SGTRegressor(RegressorMixin, BaseShapeCART): sample-weighted, output-averaged impurity improvement. branching_penalty : float, default=0.0 Constant nonnegative cost per additional occupied child beyond two. + tao_n_runs : int, default=10 + Maximum number of Tree Alternating Optimization (TAO) sweeps run at the + end of :meth:`fit` (see :func:`~sgtlearn.tao.TAO_refine`); sweeps stop + early once one makes no change. ``0`` disables TAO; any positive value + makes ``feature_importances_`` unavailable. + tao_lambda : float, default=0.0 + TAO per-sample complexity rate: at each internal node a non-constant + routing rule must beat the constant rule by more than ``tao_lambda`` + times the number of samples reaching the node (weighted reward units). tao_pair_scale : float, default=1.1 Multiplier applied to ``tao_lambda`` for pair routers during TAO. @@ -802,16 +836,21 @@ class SGTRegressor(RegressorMixin, BaseShapeCART): Number of target columns (``1`` for a 1-D ``y``). n_features_in_ : int Number of features seen during :meth:`fit`. + feature_names_in_ : ndarray of shape (n_features,) or None + Column names, converted to ``str``, of an ``X`` with a ``columns`` + attribute (e.g. a pandas ``DataFrame``) seen during :meth:`fit`; + ``None`` otherwise. feature_importances_ : ndarray of shape (n_logical_features,) Normalized impurity-based importances from the fitted tree. Index ``i`` corresponds to :attr:`processed_features_` entry ``i`` (same - order as the logical features passed to the native trainer). Available - only when the model has not undergone TAO refinement. + order as the logical features passed to the native trainer). Raises + ``AttributeError`` after TAO refinement, which runs by default + (``tao_n_runs=10``); fit with ``tao_n_runs=0`` to use it. processed_features_ : ProcessedFeatures Resolved logical features used at :meth:`fit`. ``features[i]`` and ``logical_names[i]`` align with ``feature_importances_[i]``. - ``logical_names`` are stringified ``feature_dict`` keys (auto-filled - columns use ``\"0\"``, ``\"1\"``, …). + ``logical_names`` are stringified ``feature_dict`` keys (an + auto-filled column ``i`` is named ``str(i)``). Notes ----- @@ -824,14 +863,20 @@ class SGTRegressor(RegressorMixin, BaseShapeCART): that best fits its training NaNs (or the child with the most training samples when the node saw none). At inference, missing values go to that child regardless of where the largest finite values go. Pair nodes route - missing values through dedicated branches of their pair router. Infinity - in ``X``, ``y`` and ``sample_weight`` is rejected, as are finite values - that overflow ``float32`` (e.g. ``1e39``). + missing values through their pair router: a pair-tree split sends values + missing on its axis to a dedicated branch when its training samples had + any, otherwise to the larger finite child. Infinity in ``X``, ``y`` and + ``sample_weight`` is rejected, as are finite values that overflow + ``float32`` (e.g. ``1e39``). - For ``squared_error``/``mse``, the trainer runs coordinate descent after - the round-robin seed and keeps the refined assignment only if branch MSE - improves clearly; otherwise it restores the seed. ``absolute_error``/``mae`` - skips coordinate descent entirely. + For each candidate branch count, ``squared_error``/``mse`` starts coordinate + descent from a weighted k-means clustering of the bin means, or from the + inner tree's root split when the root split's impurity is no higher. + ``absolute_error``/``mae`` scores the root split and round-robin seeds + without coordinate descent unless the ``SGTLEARN_MAE_CD`` environment + variable is set to ``1``; :meth:`fit` warns while it is unset. + Across all scored trials, the lowest-impurity feasible assignment is kept + for each number of occupied branches. References ---------- @@ -841,7 +886,7 @@ class SGTRegressor(RegressorMixin, BaseShapeCART): See Also -------- SGTClassifier : Classification counterpart. - sgtlearn.ensemble.RandomSGForestRegressor : Bootstrap ensemble over + sgtlearn.RandomSGForestRegressor : Bootstrap ensemble over :class:`SGTRegressor`. Examples @@ -928,17 +973,16 @@ def fit( ``{logical_name: [columns]}`` grouping of columns into logical features. Keys are ``int`` or ``str`` logical names. Values are column indices (``int``) or column names (``str``); names require a - pandas ``DataFrame`` ``X`` or an explicit ``column_names``. A group - of more than one column is treated as a single **categorical** - feature (routed through the one-hot inner discretizer); singletons - are **continuous**. Columns not mentioned default to continuous + pandas ``DataFrame`` ``X``. A group of more than one column is + treated as a single **categorical** feature (routed through the + one-hot inner discretizer); singletons are **continuous**. Columns not mentioned default to continuous singletons. See :func:`~sgtlearn.configure_feature_dict` for the full resolution rules. processed_features : ProcessedFeatures, optional Pre-resolved features from :func:`configure_feature_dict`. check_input : bool, default=True If ``False``, ``X`` and ``y`` are not validated (for callers that - already ran :func:`~sklearn.utils.validation.check_X_y`). + already ran :func:`~sklearn.utils.check_X_y`). """ column_names = _column_names_from_X(X) @@ -1038,6 +1082,11 @@ def fit( return self def predict(self, X: np.ndarray) -> np.ndarray: + """Predict regression targets for ``X``. + + Returns an array of shape ``(n_samples,)`` for a single output or + ``(n_samples, n_outputs)`` for multi-output ``y``. + """ check_is_fitted(self, attributes=("_est",)) X = check_array( X, accept_sparse=False, dtype=np.float64, ensure_all_finite="allow-nan" @@ -1054,9 +1103,10 @@ def predict(self, X: np.ndarray) -> np.ndarray: def tree_export(self) -> dict: """Return a flat dict snapshot of the fitted tree. - See ``sgtlearn._export.plot_tree`` for the canonical consumer. Keys: - ``num_partitions``, ``num_nodes``, ``root_index``, ``criterion``, - ``nodes`` (list of per-node dicts). Regression has no ``num_classes``. + Consumed by :func:`~sgtlearn.plot_tree` and :func:`~sgtlearn.export_text`. + Keys: ``num_partitions``, ``num_nodes``, ``root_index``, ``num_outputs``, + ``criterion``, ``nodes`` (list of per-node dicts). Regression has no + ``num_classes``. """ check_is_fitted(self, attributes=("_est",)) if self._est is None: diff --git a/sgtlearn/ensemble/_random_sgforest.py b/sgtlearn/ensemble/_random_sgforest.py index 99876fc..7abb733 100644 --- a/sgtlearn/ensemble/_random_sgforest.py +++ b/sgtlearn/ensemble/_random_sgforest.py @@ -196,7 +196,7 @@ def fit( X : array-like of shape (n_samples, n_features) Training features. A pandas ``DataFrame`` lets ``feature_dict`` reference columns by name. - y : array-like of shape (n_samples,) + y : array-like of shape (n_samples,) or (n_samples, n_outputs) Targets. sample_weight : array-like of shape (n_samples,), optional Per-sample weights. @@ -208,6 +208,11 @@ def fit( processed_features : ProcessedFeatures, optional Pre-resolved features from :func:`~sgtlearn.configure_feature_dict`, used instead of resolving ``feature_dict``. + + Returns + ------- + self : object + The fitted forest. """ _validate_int("n_estimators", self.n_estimators, 1) if not self.bootstrap and self.max_samples is not None: @@ -307,8 +312,9 @@ def mean_feature_importances_(self) -> np.ndarray: """Mean per-logical-feature importances across fitted base trees. Aligned with :attr:`processed_features_` (same order as each tree's - ``feature_importances_``). Available only after :meth:`fit` without - TAO refinement. + ``feature_importances_``). Available only after :meth:`fit` with + ``tao_n_runs=0`` and no later TAO refinement; otherwise raises + ``AttributeError``. """ return self._tree_feature_importances_matrix().mean(axis=0) @@ -318,7 +324,7 @@ def std_feature_importance_(self) -> np.ndarray: Population std (``ddof=0``) over base estimators; aligned with :attr:`mean_feature_importances_`. Available only after :meth:`fit` - without TAO refinement. + with ``tao_n_runs=0`` and no later TAO refinement. """ return self._tree_feature_importances_matrix().std(axis=0) diff --git a/sgtlearn/ensemble/random_sgforest_classifier.py b/sgtlearn/ensemble/random_sgforest_classifier.py index 4661032..edbc2fa 100644 --- a/sgtlearn/ensemble/random_sgforest_classifier.py +++ b/sgtlearn/ensemble/random_sgforest_classifier.py @@ -34,11 +34,13 @@ class RandomSGForestClassifier(ClassifierMixin, RandomSGForest): ---------- n_estimators : int, default=100 Number of trees in the forest. - criterion : {"gini", "entropy"}, default="gini" - Impurity criterion forwarded to each base tree's outer splits. + criterion : {"gini", "entropy", "log_loss"}, default="gini" + Impurity criterion forwarded to each base tree, which uses it for its + inner shape-function trees and outer splits (``"log_loss"`` is an + alias of ``"entropy"``). num_partitions : int, default=2 - Arity of the shape function at each outer split. See - :class:`sgtlearn.SGTClassifier`. + Maximum number of children per outer split (a split may have fewer). + See :class:`sgtlearn.SGTClassifier`. max_depth, max_leaf_nodes, min_samples_leaf, min_impurity_decrease : \ see :class:`sgtlearn.SGTClassifier` Outer-tree stopping criteria forwarded to each base estimator. @@ -61,6 +63,18 @@ class RandomSGForestClassifier(ClassifierMixin, RandomSGForest): sample-weighted, output-averaged impurity improvement. branching_penalty : float, default=0.0 Constant cost per additional occupied child beyond two. + tao_n_runs : int, default=10 + Maximum number of bottom-up TAO sweeps run on each base tree at the + end of its ``fit`` (stopping early once a sweep changes nothing), on + that tree's bootstrap sample (or the full training set when + ``bootstrap=False``). ``0`` disables TAO; any positive value makes + :attr:`mean_feature_importances_` and :attr:`std_feature_importance_` + unavailable. + tao_lambda : float, default=0.0 + Per-sample complexity rate for TAO: at each internal node, a + non-constant routing rule must beat the constant dummy rule by more + than ``tao_lambda`` times the node's sample count, in weighted reward + units. See :func:`~sgtlearn.tao.TAO_refine`. tao_pair_scale : float, default=1.1 Multiplier applied to ``tao_lambda`` for pair routers during TAO. bootstrap : bool, default=True @@ -89,26 +103,36 @@ class RandomSGForestClassifier(ClassifierMixin, RandomSGForest): ---------- estimators_ : list of SGTClassifier The collection of fitted base estimators. - classes_ : ndarray of shape (n_classes,) - Class labels in original training label space. - n_classes_ : int - Number of classes seen during :meth:`fit`. + classes_ : ndarray of shape (n_classes,) or list of ndarray + Class labels in original training label space; for multi-output + ``y``, a list with one array per output. + n_classes_ : int or list of int + Number of classes seen during :meth:`fit` (one entry per output for + multi-output ``y``). n_features_in_ : int Number of features seen during :meth:`fit`. + n_outputs_ : int + Number of target columns (``1`` for a 1-D ``y``). + feature_names_in_ : ndarray of shape (n_features,) or None + Column names, converted to ``str``, of an ``X`` with a ``columns`` + attribute (e.g. a pandas ``DataFrame``) seen during :meth:`fit`; + ``None`` otherwise. mean_feature_importances_ : ndarray of shape (n_logical_features,) Mean of per-tree :attr:`~sgtlearn.SGTClassifier.feature_importances_`, - aligned with :attr:`processed_features_`. Unavailable after TAO. + aligned with :attr:`processed_features_`. Accessing it raises + ``AttributeError`` if any base tree was TAO-refined, which happens by + default (``tao_n_runs=10``); fit with ``tao_n_runs=0`` to use it. std_feature_importance_ : ndarray of shape (n_logical_features,) Population standard deviation of per-tree importances across the - forest (same alignment as :attr:`mean_feature_importances_`). - Unavailable after TAO. + forest (same alignment and TAO restriction as + :attr:`mean_feature_importances_`). processed_features_ : ProcessedFeatures Logical features resolved once and shared by every base tree. See Also -------- sgtlearn.SGTClassifier : Single-tree base estimator. - sgtlearn.ensemble.RandomSGForestRegressor : Regression counterpart. + sgtlearn.RandomSGForestRegressor : Regression counterpart. sklearn.ensemble.RandomForestClassifier : Standard CART forest with the same prediction semantics. @@ -227,6 +251,12 @@ def _make_tree(self, tree_seed: int, tree_kw: dict[str, Any]) -> SGTClassifier: return tree def predict_proba(self, X: np.ndarray) -> Any: + """Class probabilities averaged over trees, aligned with :attr:`classes_`. + + Single output: an array of shape ``(n_samples, n_classes)``. + Multi-output: a list of such arrays, one per output (sklearn + convention). + """ X32 = self._check_predict_X(X) n_outputs = int(getattr(self, "n_outputs_", 1) or 1) n_trees = float(len(self.estimators_)) @@ -251,6 +281,11 @@ def predict_proba(self, X: np.ndarray) -> Any: return accs[0] if n_outputs == 1 else accs def predict(self, X: np.ndarray) -> np.ndarray: + """Predict the class with the highest mean probability for each output. + + Returns shape ``(n_samples,)`` for a single output and + ``(n_samples, n_outputs)`` for multi-output ``y``. + """ proba = self.predict_proba(X) n_outputs = int(getattr(self, "n_outputs_", 1) or 1) classes_list = ( diff --git a/sgtlearn/ensemble/random_sgforest_regressor.py b/sgtlearn/ensemble/random_sgforest_regressor.py index b2ad723..34f99f3 100644 --- a/sgtlearn/ensemble/random_sgforest_regressor.py +++ b/sgtlearn/ensemble/random_sgforest_regressor.py @@ -26,10 +26,14 @@ class RandomSGForestRegressor(RegressorMixin, RandomSGForest): n_estimators : int, default=100 Number of trees in the forest. criterion : {"squared_error", "mse", "absolute_error", "mae"}, default="squared_error" - Loss forwarded to each base tree's outer splits. + Loss forwarded to each base tree, which uses it for its inner + shape-function trees and outer splits. With ``"absolute_error"`` / + ``"mae"``, coordinate descent is disabled unless + ``SGTLEARN_MAE_CD=1`` is set; while it is disabled, :meth:`fit` warns + once. num_partitions : int, default=2 - Arity of the shape function at each outer split. See - :class:`sgtlearn.SGTRegressor`. + Maximum number of children per outer split (a split may have fewer). + See :class:`sgtlearn.SGTRegressor`. max_depth, max_leaf_nodes, min_samples_leaf, min_impurity_decrease : \ see :class:`sgtlearn.SGTRegressor` Outer-tree stopping criteria forwarded to each base estimator. @@ -41,7 +45,7 @@ class RandomSGForestRegressor(RegressorMixin, RandomSGForest): Coordinate-descent controls forwarded to each base estimator. max_features : int, float, {"sqrt", "log2"} or None, default="sqrt" Per-split feature subsampling for each base tree. Defaults to - ``"sqrt"`` to follow ``RandomForestRegressor`` convention. See + ``"sqrt"`` (``RandomForestRegressor`` defaults to ``1.0``). See :class:`sgtlearn.SGTRegressor` for the full semantics. pairwise_candidates : int or float, default=0 Maximum retained feature pairs fitted per node. An integer is an @@ -52,6 +56,18 @@ class RandomSGForestRegressor(RegressorMixin, RandomSGForest): sample-weighted, output-averaged impurity improvement. branching_penalty : float, default=0.0 Constant cost per additional occupied child beyond two. + tao_n_runs : int, default=10 + Maximum number of bottom-up TAO sweeps run on each base tree at the + end of its ``fit`` (stopping early once a sweep changes nothing), on + that tree's bootstrap sample (or the full training set when + ``bootstrap=False``). ``0`` disables TAO; any positive value makes + :attr:`mean_feature_importances_` and :attr:`std_feature_importance_` + unavailable. + tao_lambda : float, default=0.0 + Per-sample complexity rate for TAO: at each internal node, a + non-constant routing rule must beat the constant dummy rule by more + than ``tao_lambda`` times the node's sample count, in weighted reward + units. See :func:`~sgtlearn.tao.TAO_refine`. tao_pair_scale : float, default=1.1 Multiplier applied to ``tao_lambda`` for pair routers during TAO. bootstrap : bool, default=True @@ -78,20 +94,28 @@ class RandomSGForestRegressor(RegressorMixin, RandomSGForest): The collection of fitted base estimators. n_features_in_ : int Number of features seen during :meth:`fit`. + n_outputs_ : int + Number of target columns (``1`` for a 1-D ``y``). + feature_names_in_ : ndarray of shape (n_features,) or None + Column names, converted to ``str``, of an ``X`` with a ``columns`` + attribute (e.g. a pandas ``DataFrame``) seen during :meth:`fit`; + ``None`` otherwise. mean_feature_importances_ : ndarray of shape (n_logical_features,) Mean of per-tree :attr:`~sgtlearn.SGTRegressor.feature_importances_`, - aligned with :attr:`processed_features_`. Unavailable after TAO. + aligned with :attr:`processed_features_`. Accessing it raises + ``AttributeError`` if any base tree was TAO-refined, which happens by + default (``tao_n_runs=10``); fit with ``tao_n_runs=0`` to use it. std_feature_importance_ : ndarray of shape (n_logical_features,) Population standard deviation of per-tree importances across the - forest (same alignment as :attr:`mean_feature_importances_`). - Unavailable after TAO. + forest (same alignment and TAO restriction as + :attr:`mean_feature_importances_`). processed_features_ : ProcessedFeatures Logical features resolved once and shared by every base tree. See Also -------- sgtlearn.SGTRegressor : Single-tree base estimator. - sgtlearn.ensemble.RandomSGForestClassifier : Classification counterpart. + sgtlearn.RandomSGForestClassifier : Classification counterpart. sklearn.ensemble.RandomForestRegressor : Standard CART forest with the same prediction semantics. @@ -186,6 +210,11 @@ def _make_tree(self, tree_seed: int, tree_kw: dict[str, Any]) -> SGTRegressor: return SGTRegressor(**tree_kw, random_state=tree_seed) def predict(self, X: np.ndarray) -> np.ndarray: + """Predict the mean of the per-tree predictions. + + Returns shape ``(n_samples,)`` for a single output and + ``(n_samples, n_outputs)`` for multi-output ``y``. + """ X32 = self._check_predict_X(X) n_outputs = int(getattr(self, "n_outputs_", 1) or 1) acc = np.zeros((X32.shape[0], n_outputs), dtype=np.float64) diff --git a/sgtlearn/tao.py b/sgtlearn/tao.py index 7283f48..d850dc6 100644 --- a/sgtlearn/tao.py +++ b/sgtlearn/tao.py @@ -194,8 +194,6 @@ def TAO_refine( ) -> TaoModel: """Refine a fitted shape-generalized tree or forest in place with TAO. - Accepts a single :class:`~sgtlearn.base.BaseShapeCART` or a - :class:`~sgtlearn.ensemble._random_sgforest.RandomSGForest` ensemble. Forests refine each base estimator independently; pass ``n_jobs`` (or rely on the forest's own ``n_jobs`` when ``n_jobs`` is ``None``) to run those passes in parallel via joblib. @@ -206,11 +204,60 @@ def TAO_refine( ``lambda_`` is a per-sample complexity rate (cost-complexity style). At each internal node, a non-constant routing rule must beat the dummy rule by more - than ``lambda_ * n_samples`` in weighted reward units (equivalently - ``lambda_ * n_samples / n_care`` on the mean care reward). + than ``lambda_ * n_node`` in weighted reward units, where ``n_node`` is the + (unweighted) number of samples reaching the node (equivalently + ``lambda_ * n_node / w_care`` on the mean care reward, where ``w_care`` is + the care set's total weight after any ``class_weight`` is applied). Pair routers pay ``tao_pair_scale`` times this cost; dummy routers pay zero. After any call with ``n_runs > 0``, impurity feature importances are unavailable; use held-out permutation importance instead. + + Parameters + ---------- + model : SGTClassifier, SGTRegressor, RandomSGForestClassifier or \ + RandomSGForestRegressor + A fitted estimator, refined in place. + X : array-like of shape (n_samples, n_features) + Dense training features (NaN allowed) with the ``n_features_in_`` + columns seen at fit. + y : array-like of shape (n_samples,) or (n_samples, n_outputs) + Training targets with the same number of outputs as at fit. Class + labels must come from the fitted ``classes_``. + sample_weight : array-like of shape (n_samples,), default=None + Finite, non-negative per-sample weights with at least one positive + value; ``None`` means uniform weights. For classifiers the model's + ``class_weight`` is multiplied in automatically, so pass raw weights. + n_runs : int, default=10 + Maximum number of bottom-up sweeps (a non-negative int); refinement can + stop early once a sweep makes no change. ``0`` leaves the model + unchanged. + lambda_ : float, default=0.0 + Per-sample complexity rate (see above). + tao_pair_scale : float, default=1.1 + Finite, non-negative multiplier on the complexity cost of pair routers. + check_input : bool, default=True + Validate ``X`` and ``y`` with scikit-learn checks. Set to ``False`` only + when they are already validated dense arrays. + n_jobs : int, default=None + Number of joblib threads used to refine a forest's trees. ``None`` uses + the forest's ``n_jobs``; ignored for a single tree. + + Returns + ------- + model : SGTClassifier, SGTRegressor, RandomSGForestClassifier or \ + RandomSGForestRegressor + The same ``model`` object, refined in place. + + Raises + ------ + sklearn.exceptions.NotFittedError + If ``model`` is not fitted. + TypeError + If ``model`` is not a supported estimator. + ValueError + If ``X`` is sparse, ``X`` / ``y`` / ``sample_weight`` are invalid or do + not match the fitted model, or ``tao_pair_scale`` is negative or not + finite. """ tao_pair_scale = _validate_tao_pair_scale(tao_pair_scale) targets = _tao_targets(model)