Keep TAO from lowering training accuracy at lambda=0

[upadhyan]

Closes #80.

Problem

At a node with 3+ children, one sample can be correct in several children (two children predicting the same class), so routing is multi-label. TAO split credit 1/numCorrect across those children. Its objective then stopped tracking accuracy, and it could accept rules (including the dummy) that lowered training accuracy. On the issue's repro, accuracy dropped from 0.6608 to 0.4487.

Fix (the four parts in the issue)

1. Split weights for the router fit (ClassificationTaoAdapter::buildCareSet): on the single-output path, each expanded row carries w / |correct children|. childCounts keeps full weight, so dummyChild is still the child that is correct for the most care weight. The multi-output path is unchanged.
2. True-score relabel: childRewards gives 1 to every correct child. TaoObjective::scoreDiscretizer now maps each bin to the argmax of summed careWeight * careRewards (ties go to the lowest index), instead of the argmax of the fitted histogram. Bins with zero care weight, including an unused trailing NaN bin, go to dummyChild. One transform is shared by the relabel and the scoring.
3. Same objective for every rule: the incumbent, the dummy and every candidate are scored on the full-credit rewards through penalizedScore. The acceptance rules are unchanged. At lambda_=0 an accepted change can't lower the care-set score, and non-care samples score the same under every rule.
4. Exact best threshold for K=2 (TaoObjective::bestThresholdCut): an exact prefix-sum scan per feature tries both orientations. NaN care samples go to the better child. Each side must hold at least minLeafSize care samples, and the scan uses the splitter's 1e-7 tie rule. The best cut is turned into a real depth-1 numeric ClassificationDiscretizer, trained on the care samples labelled by side. That split is pure, so it reproduces the cut, and the code checks this and throws if not. The candidate is scored with the new public TaoObjective::scoreBinAssignment and competes in the existing bestSingle comparison. Export, plotting and pickling work unchanged. The pair-candidate loop is untouched.

Behaviour change for regression: TaoObjective is shared, so regression TAO also uses the per-bin argmax of the true reward (-loss), and the K=2 exact cut. All existing regression TAO tests pass. A new test checks that squared and absolute training loss don't rise at lambda_=0.

Tests

New tests in tests/test_tao.py:

• the issue's repro;
• a property test: 4 seeds × num_partitions ∈ {2,3,4} × {no weights, sample_weight, class weights} × {tao_n_runs=0, default}. It checks weighted training accuracy with the effective float32 weights;
• K=2: training accuracy is at least a numpy brute-force best single threshold, given the root's fixed leaf labels;
• regression loss (squared and absolute error).

cpp/tests/test_tao_objective.cpp covers the issue's 3-child bin example (histogram (5,5,8) → child 2, but full credit (10,10,8) → child 0), the threshold scan, and its minLeafSize guard.

Checks that the tests catch the bug:

• With the original C++, 17 of the new tests fail: the repro, 12 property cases and all 4 K=2 cases.
• With only part 4 disabled, exactly the 4 K=2 cases fail.
• With the fix, pytest tests gives 1330 passed, 2 skipped, and every C++ Catch2 test passes (401 assertions in 46 test cases).

Tutorials

I re-executed all 10 tutorials, once with the original TAO code and once with the fix, on the same environment. Only regression.ipynb prints different numbers, and I re-executed and committed it in 5dbfde4. Regression TAO now relabels bins by true loss and also gets the K=2 exact cut, which is why it moves:

regression.ipynb	before	after
cell 1 predictions (last two)	9.77, 14.17	9.78, 14.25
cell 1 R² train / test	0.765 / 0.591	0.767 / 0.594
cell 3 squared_error R² test	0.591	0.594
cell 3 absolute_error R² test	0.553	0.573
cell 5 forest R² test	0.691	0.705

tao.ipynb prints the same numbers: its datasets are binary with two-way splits, where parts 1–3 change nothing. Every plot is byte-identical between the two builds. Their bytes differ from the committed plots only because of the environment: the original build shows the same differences. So I left the other notebooks alone.

docs/tutorials/bivariate-branching.ipynb cell 4 uses tao_n_runs=0, which hid this bug. With TAO enabled on that same data, the univariate model dropped from 0.6608 to 0.4487 training accuracy on main. With this fix it reaches 0.6621.

The guarantee in docs/api/tao.rst ("With the default 0.0, weighted training accuracy / loss does not decrease") now holds, and I left it unchanged.

Heads-up for anyone re-running tutorials from a worktree: the user-level python3 Jupyter kernelspec points at the main checkout's .venv, so nbconvert there silently uses that build.

Note

SGTClassifier does not accept class_weight="balanced". The string is treated as a sequence, which raises a misleading ValueError. The property test therefore uses an explicit dict with the same weights. Adding real support is out of scope here.

Pre-existing bugs found while fuzzing this change. They reproduce on main and are not touched here:

• When the dummy rule wins at a node where every care sample is NaN in feature 0, TAO trains its placeholder router on feature 0 and leaves it untrained. fit() (default tao_n_runs) and TAO_refine then raise "Cannot use discretizer outputs without first training it", and the model is left broken.
• Criterion::absoluteErrorPresorted uses an absolute 1e-12 tolerance for the weighted-median half-weight tie. With total sample weight around 1e-12 or less, the leaf refit picks a non-median value, so absolute_error TAO can raise training loss.

🤖 Generated with Claude Code