Treat tkm as tonne-km in unit reconciliation

[B-Navneshwar]

Closes #18

This change adds tkm as an alias for tonne km in the unit cleaner, so model answers using kg CO2e/tkm can reconcile with corpus units such as kg CO2e per tonne-km.

Verification:

• python verify\check_claims.py --offline — PASS
• python verify\figures.py --check-fresh — PASS
• Targeted reconciliation: (0.107, None)
• Benchmark impact: unscoreable 110 → 106; within 10% 45.7% → 45.5%

Only units.py is changed.

[jeremiahsay]

Thanks for this, and sorry it sat without a review — CI on a fork PR needs a maintainer to approve the run, and nobody had, so it reported no checks at all. That's now approved and the results are in.

First, one of the two failures is ours, not yours. claims-are-backed fails on 28.1%, 42.0%. Those are two figures we changed on the live guide page an hour ago, as part of #31. The gate compares your branch's committed data against the live site as it is right now, so any branch cut before #31 goes red on them regardless of what it contains. Please ignore that pair — a rebase onto main once #31 lands will clear it.

The alias itself is right. I checked it in both directions, and every spelling reconciles:

kg CO2e/tkm            vs kg CO2e/tonne km -> (1.0, None)
g CO2e per tkm         vs kg CO2e/tonne km -> (0.001, None)
kgCO2e/t-km            vs kg CO2e/tonne km -> (1.0, None)
kg CO2e per tonne-km   vs kg CO2e/tonne km -> (1.0, None)
kg CO2e per tonne km   vs kg CO2e/tkm      -> (0.1, None)

And it does what #18 asked: 4 answers recover, all None → a value, no already-scored answer is touched.

model	question	truth unit	before	after
claude-opus-5	freight.air.tonne_km	kg CO2e per tonne-km	unscoreable	0.575
claude-opus-5	freight.rail.tonne_km	kg CO2e per tonne-km	unscoreable	0.0256
claude-opus-5	freight.road_hgv.tonne_km	kg CO2e per tonne-km	unscoreable	0.107
claude-opus-5	freight_detailed.rail.eu.diesel.cars	kg CO2e per tonne-km	unscoreable	0.030

What's missing

1. The committed scorer outputs. This is the real failure — figures-reproduce says:

STALE (committed output no longer matches its script): results/comparison.json, results/paired.json

Changing the scorer changes the results, so the results have to be regenerated in the same commit. After a rebase onto main that's three scripts (absolute.py is new in #31):

python3 compare.py
python3 paired.py
python3 absolute.py

Which lands here — only claude-opus-5 moves:

	scoreable	≤10%	≤50%	>50% off
claude-opus-5	315 → 319	45.7% → 45.5%	77.5% → 77.4%	22.5% → 22.6%

absolute.json: correct 144 → 145, pct_of_all 30.8% → 31.0%. paired.json, Claude unaided: within-10% 37.1% → 37.5%, off-by->50% 25.8% → 25.0%.

Worth saying plainly: this fix makes the headline slightly worse, and that's fine. Four answers that were being thrown away are now scored, and most of them are wrong. That's the honest direction and it's the point of closing a coverage gap.

2. The published figures that quote them. After regenerating, check_claims.py --offline will name them:

paper        2 UNEXPLAINED   37.1%, 45.7%
findings     5 UNEXPLAINED   22.5%, 25.8%, 37.1%, 45.7%, 77.5%
repo-readme  3 UNEXPLAINED   25.8%, 37.1%, 45.7%

Those are FINDINGS.md (the results table, the absolute table, the paired table), README.md (the same two tables) and paper/main.tex (the results table and the abstract's paired sentence). Update the numbers; please don't add allowlist entries for them — the allowlist is for figures no script can produce, and these are exactly what your compare.py run prints.

3. A test. Something in the shape of verify/test_currency_mismatch.py asserting the spellings above reconcile, and that a non-tonne km denominator still doesn't. #31 adds a scorer-tests job that runs every verify/test_*.py, so it'll be picked up automatically.

No rush on any of this, and thanks for finding a real gap — the freight rows were the largest single-domain block left in the unscoreable pile.

[jeremiahsay]

Two things, and an apology for the order they happened in.

I pushed a commit to your branch. You had "allow edits by maintainers" on, and I used it to add the parts that were missing — regenerated results/, the prose figures, and a test — because this PR is on the release path and I wanted it verifiable today. Your commit is untouched and mine sits on top. Please review it and say if you'd rather have done it yourself — I'll drop it and hand you the spec instead, no hard feelings. I should have asked first.

What's in it: compare.py / paired.py / absolute.py regenerated, the figures updated in FINDINGS.md, README.md and paper/main.tex, and verify/test_tkm_alias.py covering tkm in both directions, the existing spellings, scale carry-through, and that a unit merely containing those letters is untouched.

Your test found a bug that wasn't yours. Writing the tkm test I asserted that per km must not reconcile with a per tonne-km truth. It failed — it does reconcile, because the denominator scan stops at km before reaching tonne, so both sides report den_dim "length". Chasing that turned up a live instance of the same class: kWh/yr was being scored against a kWh per m2 per year truth, ten answers, the worst off by a factor of 200. That's now #35, fixed in #36. Your alias widens the per km half latently — per tkm used to be refused as other:tkm — so the test pins current behaviour with a pointer to the issue rather than asserting a fix. Nothing in the corpus is affected by that half.

So this needs one more pass before it merges, and it isn't your doing. #36 changes Claude's numbers, so the figures in my commit will be stale once it lands. The order is:

1. Area is a dimension, so a per-m2 rate is not a per-nothing rate #36 merges (the basis fix)
2. rebase this onto main
3. re-run compare.py, paired.py, absolute.py, then let check_claims.py --offline name the prose figures
4. merge

I'll do step 3 if you'd like, or leave it to you — say which. Either way the alias itself is verified and correct: I checked tkm, t-km and tonne-km in both directions, and it recovers four freight answers with no false positives.

Current CI on this branch: figures-reproduce, scorer-tests, guard and licence-drift all pass. claims-are-backed passes offline and fails only on the live guide page, which shows figures that can only be updated after a merge — also not yours.

Thanks for the fix, and sorry again for pushing before asking.

[B-Navneshwar]

Thanks for explaining everything, and no worries about the push. I’m happy to take care of step 3. I’ll re-run compare.py, paired.py, and absolute.py, then run the offline claims check and report back here.

[B-Navneshwar]

I completed step 3 locally.

compare.py:

claude-opus-5: 319 scored, 45.5% ≤10%, 77.4% ≤50%
gpt-5.5: 256 scored, 58.2% ≤10%, 85.2% ≤50%
gemini-3.6-flash: 312 scored, 42.0% ≤10%, 77.9% ≤50%
grok-4.6: 66 scored, 62.1% ≤10%, 86.4% ≤50%
gemini-3.1-pro-preview: 46 scored, 65.2% ≤10%, 80.4% ≤50%

paired.py:

claude-opus-5: 26.7% correct without tools → 86.7% with GreenCalculus (+60.0 pts)
gpt-5.5: 25.6% → 87.8% (+62.2 pts)

absolute.py:

gpt-5.5: 31.9% of all 467
claude-opus-5: 31.0%
gemini-3.6-flash: 28.1%
grok-4.6: 8.8%
gemini-3.1-pro-preview: 6.4%

python verify\check_claims.py --offline also passes: 142 numeric claims checked, 12 explained by allowlist, and 804 figures recomputed from committed data.

The four HTTP-only guide surfaces (guide:accuracy, guide:licences, guide:by-category, guide:recommendations) were skipped as expected. git status is clean, so there are no additional changes to push.

[jeremiahsay]

#36 has landed, so I've rebased this onto the new main and regenerated against final scorer behaviour. Still not merged — waiting on you.

Your commit is unchanged. I dropped and redid my own commit rather than resolving conflicts through it, so the figures are the ones the alias actually produces now:

	before	after
claude-opus-5 scoreable	310	314
claude-opus-5 within 10%	46.5%	46.2%
claude-opus-5 unscoreable	115	111
absolute: correct / of all 467	144 / 30.8%	145 / 31.0%
paired, unaided within 10%	37.7% (n 61)	38.1% (n 63)
paired, unaided off by >50%	24.6%	23.8%

Within-50% and >50%-off are unchanged at this rounding, and no other model moves.

CI: figures-reproduce, scorer-tests, guard, licence-drift all pass, and claims-are-backed passes its offline step. It fails only on Published pages, on the two figures that change the moment this merges — the live guide can't be updated before then. That one is structural and not yours; I'll update the live page straight after.

What I need from you: a note here saying you're happy with the commit I added, or that you'd rather write it yourself. Either is fine. I won't merge until you've said.

The alias itself is verified: tkm, t-km and tonne-km all reconcile in both directions, four freight answers recover, nothing already scored changes. And thank you again — the test I wrote for it is what surfaced #35, which turned out to be ten answers scored against an incompatible denominator.

[B-Navneshwar]

Thanks for the update and for taking care of the regeneration and test. I'm happy with the commit you added and with the final figures. I'm also happy for you to proceed with the merge.

[jeremiahsay]

Thank you — and that's a more useful reply than a simple ack, because it's an independent reproduction. Every figure you reported matches what the commit contained, exactly:

	yours	committed
claude-opus-5	319 / 45.5% / 77.4%	319 / 45.5% / 77.4%
gpt-5.5	256 / 58.2% / 85.2%	256 / 58.2% / 85.2%
gemini-3.6-flash	312 / 42.0% / 77.9%	312 / 42.0% / 77.9%
grok-4.6	66 / 62.1% / 86.4%	66 / 62.1% / 86.4%
gemini-3.1-pro-preview	46 / 65.2% / 80.4%	46 / 65.2% / 80.4%
absolute, all five	31.9 / 31.0 / 28.1 / 8.8 / 6.4	identical
offline gate	142 claims, 12 allowlisted	identical

Different machine, different OS, same numbers. That's worth more than the check itself — it means compare.py, paired.py and absolute.py are genuinely deterministic over the committed answers rather than reproducing only where they were written.

One thing to flag, because it crossed with your reply: you ran that against main as it was at the time, and #36 merged a couple of hours later. So those figures have moved once more — through no fault of yours or the alias's. #36 gave area a dimension, which stopped ten answers being scored against a per-square-metre truth, and it shifted Claude. The branch is now rebased onto that:

	your run	now
claude-opus-5 scoreable	319	314
claude-opus-5 within 10%	45.5%	46.2%
paired, unaided within 10%	37.5%	38.1%

Nothing about the tkm change itself is different — still four freight answers recovered, still None → a value in every case.

Given you've confirmed you're happy with the commit and verified the method reproduces, I'm merging this now rather than asking you for a third pass on numbers that only moved because of someone else's fix. If you'd rather have re-run it yourself first, say so and I'll treat that as the standing preference for anything future.

The live guide page still shows the pre-merge figures, so claims-are-backed will be red until I update it — that's mine, immediately after the merge.

Thanks again. #18 is closed by this, and the test written for it is what turned up #35.