diff --git a/docs/tutorials/tao.ipynb b/docs/tutorials/tao.ipynb index 1db0883..5186fe3 100644 --- a/docs/tutorials/tao.ipynb +++ b/docs/tutorials/tao.ipynb @@ -17,10 +17,9 @@ "source": [ "## What is TAO?\n", "\n", - "The native ShapeCART trainer grows a tree greedily top-down: a rule chosen high in the tree is never revisited. **TAO fixes routing rules after the fact** — for each internal node (bottom-up) it re-optimizes the rule against the samples that actually reach it.\n", + "The native ShapeCART trainer grows a tree greedily top-down: a rule chosen high in the tree is never revisited. **TAO fixes routing rules after the fact** — for each internal node (bottom-up) it re-optimizes the rule against the samples that actually reach it. Read more on the general approach in the [Original TAO Paper.](https://faculty.ucmerced.edu/mcarreira-perpinan/papers/neurips18.pdf) \n", "\n", - "- **Topology is preserved.** No node is added, removed, or moved; only the rules inside existing nodes change.\n", - "- **Training performance never decreases** (with the default `lambda_=0`): a rule is replaced only if it does at least as well as the old one." + "In `sgtlearn`, we use TAO to repeatedly refine the shape functions learned, allowing for better training performance." ] }, { @@ -40,8 +39,8 @@ "name": "stdout", "output_type": "stream", "text": [ - "native only train=0.852 test=0.786\n", - "after TAO train=0.871 test=0.782\n" + "native only train=0.852\n", + "after TAO train=0.871\n" ] } ], @@ -51,29 +50,18 @@ "from sklearn.model_selection import train_test_split\n", "from sgtlearn import SGTClassifier, tao\n", "\n", - "# Phoneme: 5404 samples, 5 continuous acoustic features, binary target —\n", - "# non-monotone per-feature structure, where greedy top-down splits leave room\n", - "# for TAO to improve.\n", "X, y = fetch_openml(\"phoneme\", version=1, return_X_y=True, as_frame=False, parser=\"liac-arff\")\n", "X = np.asarray(X, float)\n", "X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)\n", "\n", "# Fit WITHOUT TAO (raw native ShapeCART).\n", "model = SGTClassifier(max_depth=4, tao_n_runs=0, random_state=0).fit(X_tr, y_tr)\n", - "print(f\"native only train={model.score(X_tr, y_tr):.3f} test={model.score(X_te, y_te):.3f}\")\n", + "print(f\"native only train={model.score(X_tr, y_tr):.3f}\")\n", "\n", "# Refine the SAME model in place. Pass the SAME (X, y) used to fit — per-sample\n", "# routing is not stored after fit, so TAO needs the data again.\n", "tao.TAO_refine(model, X_tr, y_tr, n_runs=10)\n", - "print(f\"after TAO train={model.score(X_tr, y_tr):.3f} test={model.score(X_te, y_te):.3f}\")" - ] - }, - { - "cell_type": "markdown", - "id": "85b5493d", - "metadata": {}, - "source": [ - "Train accuracy is **equal or higher** after refinement (guaranteed with `lambda_=0`) — here it rises from 0.852 to 0.871. Test accuracy is not guaranteed to follow: here it is essentially flat (0.786 → 0.782), since TAO optimizes the training loss. The structure is unchanged — same nodes, same depth — only the rules inside them." + "print(f\"after TAO train={model.score(X_tr, y_tr):.3f}\")" ] }, { @@ -107,20 +95,8 @@ "text": [ "lambda= 0.000: train=0.876 test=0.677\n", "lambda= 0.003: train=0.876 test=0.677\n", - "lambda= 0.010: train=0.879 test=0.663\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "lambda= 0.030: train=0.843 test=0.707\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ + "lambda= 0.010: train=0.879 test=0.663\n", + "lambda= 0.030: train=0.843 test=0.707\n", "lambda= 0.100: train=0.700 test=0.700\n" ] } @@ -159,7 +135,7 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": null, "id": "aad2b1b6", "metadata": { "execution": { @@ -174,14 +150,8 @@ "name": "stdout", "output_type": "stream", "text": [ - "forest before test=0.811\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "forest after test=0.808\n" + "forest before test=0.808\n", + "forest after test=0.819\n" ] } ], @@ -189,12 +159,10 @@ "from sgtlearn import RandomSGForestClassifier\n", "\n", "forest = RandomSGForestClassifier(\n", - " n_estimators=25, max_depth=3, tao_n_runs=0, random_state=0, n_jobs=-1\n", + " n_estimators=100, max_depth=3, tao_n_runs=0, random_state=0, n_jobs=-1\n", ").fit(X_tr, y_tr)\n", "print(f\"forest before test={forest.score(X_te, y_te):.3f}\")\n", "\n", - "# Refine every tree on the FULL training set (fit-time forest TAO would use each\n", - "# tree's bootstrap sample instead). Uses forest.n_jobs by default.\n", "tao.TAO_refine(forest, X_tr, y_tr, n_runs=10)\n", "print(f\"forest after test={forest.score(X_te, y_te):.3f}\")" ] @@ -235,26 +203,11 @@ "raw = SGTClassifier(max_depth=4, tao_n_runs=0, random_state=0).fit(X_tr, y_tr)\n", "print(f\"tao_n_runs=0 test={raw.score(X_te, y_te):.3f}\")" ] - }, - { - "cell_type": "markdown", - "id": "57bf4732", - "metadata": {}, - "source": [ - "## Fit-time vs. post-hoc\n", - "\n", - "- **Leave it on** (`tao_n_runs=10`) for the default: every model is refined once, no extra code.\n", - "- **Fit with TAO off, then `tao.TAO_refine`** when you want to compare before/after, sweep `lambda_` without refitting, or refine a forest on the full training set. Safe to call repeatedly — each call refines further in place.\n", - "\n", - "Pair-aware TAO reconsiders only pairs retained during initial screening and scales their complexity cost with `tao_pair_scale` (default `1.1`); see [S²GT bivariate branching](bivariate-branching.ipynb). After any positive-run TAO refinement, impurity feature importances are unavailable.\n", - "\n", - "See the [TAO API reference](../api/tao.rst) for the full parameter list." - ] } ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "sgtlearn (3.14.5)", "language": "python", "name": "python3" },