Skip to content

Revise TAO tutorial #90

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
upadhyan merged 1 commit into from
Oct 16, 2026
Merged

Revise TAO tutorial #90

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 13 additions & 60 deletions docs/tutorials/tao.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,9 @@
"source": [
"## What is TAO?\n",
"\n",
"The native ShapeCART trainer grows a tree greedily top-down: a rule chosen high in the tree is never revisited. **TAO fixes routing rules after the fact** — for each internal node (bottom-up) it re-optimizes the rule against the samples that actually reach it.\n",
"The native ShapeCART trainer grows a tree greedily top-down: a rule chosen high in the tree is never revisited. **TAO fixes routing rules after the fact** — for each internal node (bottom-up) it re-optimizes the rule against the samples that actually reach it. Read more on the general approach in the [Original TAO Paper.](https://faculty.ucmerced.edu/mcarreira-perpinan/papers/neurips18.pdf) \n",
"\n",
"- **Topology is preserved.** No node is added, removed, or moved; only the rules inside existing nodes change.\n",
"- **Training performance never decreases** (with the default `lambda_=0`): a rule is replaced only if it does at least as well as the old one."
"In `sgtlearn`, we use TAO to repeatedly refine the shape functions learned, allowing for better training performance."
]
},
{
Expand All @@ -40,8 +39,8 @@
"name": "stdout",
"output_type": "stream",
"text": [
"native only train=0.852 test=0.786\n",
"after TAO train=0.871 test=0.782\n"
"native only train=0.852\n",
"after TAO train=0.871\n"
]
}
],
Expand All @@ -51,29 +50,18 @@
"from sklearn.model_selection import train_test_split\n",
"from sgtlearn import SGTClassifier, tao\n",
"\n",
"# Phoneme: 5404 samples, 5 continuous acoustic features, binary target —\n",
"# non-monotone per-feature structure, where greedy top-down splits leave room\n",
"# for TAO to improve.\n",
"X, y = fetch_openml(\"phoneme\", version=1, return_X_y=True, as_frame=False, parser=\"liac-arff\")\n",
"X = np.asarray(X, float)\n",
"X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)\n",
"\n",
"# Fit WITHOUT TAO (raw native ShapeCART).\n",
"model = SGTClassifier(max_depth=4, tao_n_runs=0, random_state=0).fit(X_tr, y_tr)\n",
"print(f\"native only train={model.score(X_tr, y_tr):.3f} test={model.score(X_te, y_te):.3f}\")\n",
"print(f\"native only train={model.score(X_tr, y_tr):.3f}\")\n",
"\n",
"# Refine the SAME model in place. Pass the SAME (X, y) used to fit — per-sample\n",
"# routing is not stored after fit, so TAO needs the data again.\n",
"tao.TAO_refine(model, X_tr, y_tr, n_runs=10)\n",
"print(f\"after TAO train={model.score(X_tr, y_tr):.3f} test={model.score(X_te, y_te):.3f}\")"
]
},
{
"cell_type": "markdown",
"id": "85b5493d",
"metadata": {},
"source": [
"Train accuracy is **equal or higher** after refinement (guaranteed with `lambda_=0`) — here it rises from 0.852 to 0.871. Test accuracy is not guaranteed to follow: here it is essentially flat (0.786 → 0.782), since TAO optimizes the training loss. The structure is unchanged — same nodes, same depth — only the rules inside them."
"print(f\"after TAO train={model.score(X_tr, y_tr):.3f}\")"
]
},
{
Expand Down Expand Up @@ -107,20 +95,8 @@
"text": [
"lambda= 0.000: train=0.876 test=0.677\n",
"lambda= 0.003: train=0.876 test=0.677\n",
"lambda= 0.010: train=0.879 test=0.663\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"lambda= 0.030: train=0.843 test=0.707\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"lambda= 0.010: train=0.879 test=0.663\n",
"lambda= 0.030: train=0.843 test=0.707\n",
"lambda= 0.100: train=0.700 test=0.700\n"
]
}
Expand Down Expand Up @@ -159,7 +135,7 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"id": "aad2b1b6",
"metadata": {
"execution": {
Expand All @@ -174,27 +150,19 @@
"name": "stdout",
"output_type": "stream",
"text": [
"forest before test=0.811\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"forest after test=0.808\n"
"forest before test=0.808\n",
"forest after test=0.819\n"
]
}
],
"source": [
"from sgtlearn import RandomSGForestClassifier\n",
"\n",
"forest = RandomSGForestClassifier(\n",
" n_estimators=25, max_depth=3, tao_n_runs=0, random_state=0, n_jobs=-1\n",
" n_estimators=100, max_depth=3, tao_n_runs=0, random_state=0, n_jobs=-1\n",
").fit(X_tr, y_tr)\n",
"print(f\"forest before test={forest.score(X_te, y_te):.3f}\")\n",
"\n",
"# Refine every tree on the FULL training set (fit-time forest TAO would use each\n",
"# tree's bootstrap sample instead). Uses forest.n_jobs by default.\n",
"tao.TAO_refine(forest, X_tr, y_tr, n_runs=10)\n",
"print(f\"forest after test={forest.score(X_te, y_te):.3f}\")"
]
Expand Down Expand Up @@ -235,26 +203,11 @@
"raw = SGTClassifier(max_depth=4, tao_n_runs=0, random_state=0).fit(X_tr, y_tr)\n",
"print(f\"tao_n_runs=0 test={raw.score(X_te, y_te):.3f}\")"
]
},
{
"cell_type": "markdown",
"id": "57bf4732",
"metadata": {},
"source": [
"## Fit-time vs. post-hoc\n",
"\n",
"- **Leave it on** (`tao_n_runs=10`) for the default: every model is refined once, no extra code.\n",
"- **Fit with TAO off, then `tao.TAO_refine`** when you want to compare before/after, sweep `lambda_` without refitting, or refine a forest on the full training set. Safe to call repeatedly — each call refines further in place.\n",
"\n",
"Pair-aware TAO reconsiders only pairs retained during initial screening and scales their complexity cost with `tao_pair_scale` (default `1.1`); see [S²GT bivariate branching](bivariate-branching.ipynb). After any positive-run TAO refinement, impurity feature importances are unavailable.\n",
"\n",
"See the [TAO API reference](../api/tao.rst) for the full parameter list."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "sgtlearn (3.14.5)",
"language": "python",
"name": "python3"
},
Expand Down
Loading