D1: fix Python crashes on MISO freq_map and 3-D single-trajectory input

[pdlourenco]

Summary

Implements plan item D1 of the review-v2 remediation plan (docs/plans/2026-10-04-review-v2-remediation-plan.md). It fixes three Python crashes on documented input (analysis §5.1 F1, F2; §5.3 U5) and needs no decision. MATLAB needs no change: it cannot represent a trailing singleton, and its MISO freq_map already works.

1. freq_map crashed on any MISO data (one output, several inputs), under both algorithms.
   • The noise spectrum was allocated in the MIMO layout (nf, K, 1, 1) whenever the model is not SISO, but stored as 2-D whenever ny == 1. The store now branches on the allocated shape.
   • MISO now returns response (nf, K, 1, nu) and noise (nf, K, 1, 1), the same layout as freq_bt/freq_btfdr per segment (§6.8).
2. freq_etfe and Welch freq_map crashed on a single trajectory given as (N, n, 1).
   • validate_data now squeezes such y and u to 2-D, once, so every caller takes its 2-D path. That is the behaviour MATLAB has by construction (SPEC §1).
   • Shared helper, all callers audited (CLAUDE.md §3): freq_bt, freq_btfdr, freq_etfe, freq_map.
     • Output shapes are unchanged for every path that already worked, with one exception: an freq_etfe time series given as 3-D gave noise_spectrum (nf, 1) and now gives (nf,), matching 2-D input.
     • The earlier per-function special cases for this shape become unreachable but harmless.
     • The list and preserve_lengths paths are untouched.
3. detrend crashed on (N, n_ch, 1). It now branches on x.ndim and keeps the input shape.

Spec sections touched

None. These are conformance fixes to existing SPEC §1 and §6.1/§6.8 behaviour.

Type of change

• Bug fix

Checks

• Tests written against the spec (a 3-D single trajectory gives the 2-D result; MISO shapes per §6.8), not against the MATLAB port.

• Shared helper touched: every caller audited (above).

• Tests (python/tests/test_single_trajectory_shapes.py, new; it covers all three fixes in one file):

  • 3-D L = 1 gives exactly the 2-D result: response, std, noise and coherence. Covered: freq_bt, freq_btfdr, freq_etfe, freq_map (BT and Welch), and a time series.
  • detrend keeps the shape.
  • MISO freq_map runs under both algorithms and recovers a static gain per input, element-wise.
  • test_validate.py adds unit tests for the squeeze, including the mixed 3-D/2-D cases.
  • Against the old code, 6 of the 9 shape tests fail.

• Pre-push review: no blocking findings. Fixed in 5fcfed2:

  • isolated helper tests;
  • an freq_etfe changelog entry (its behaviour changed via the helper);
  • stronger comparisons.

  Left for D6: u 3-D with L > 1 against a single-trajectory y still raises an internal error rather than a typed one. That was already so before this PR (F12), and D6's traj_mismatch check covers it.

• Local CI: Python 428 passed, 4 skipped. ruff and the Python header check are clean.

🤖 Generated with Claude Code

[pdlourenco]

Review of PR #213 at 5fcfed2: D1, Python shape crashes

Posted as a comment review because this session's GitHub identity is the PR author. Verdict: request changes, to the PR description only. The code is correct and well tested. One statement in the shared-helper audit is wrong, and CLAUDE.md §3 treats that audit as the record. Once the description is fixed, this is ready to merge.

Method. A reviewer with fresh context covered the spec, the code, the tests and governance. An independent verifier then tried to refute each finding. A separate agent trial-merged all five open PRs and ran both full suites on the result. The checks were the full Python suite on this head, a base-vs-head differential over 27 input forms × 10 calls, mutation tests in scratch copies, and Octave 8.4 probes of the MATLAB side. Disclosure: this session wrote the remediation plan (item D1); it did not write this PR.

Blocker (description only)

1. The audit says the list paths are untouched, but a one-element list now goes through the new squeeze too. The equal-length list branch (validate_data.py:202-214) builds (N, n, L) and falls through to the squeeze at :255-258. So [y] with L = 1 now comes back 2-D. That is also true with preserve_lengths=True when the lengths are equal.
   • Measured, base vs head:
     • validate_data([y],[u]) goes from (600,1,1) to (600,1).
     • freq_etfe([y], None): noise_spectrum and noise_spectrum_std go from (128,1) to (128,). The description names only noise_spectrum, and only for the 3-D input.
     • freq_etfe([y],[u]) and Welch freq_map([y],[u]) used to crash and now work.
   • Every list-of-one result now equals the 2-D result exactly, which is correct per SPEC §1. So the behaviour is right; only the audit statement is wrong.
   • Fix: amend the description:
     • a one-element list (and preserve_lengths with equal lengths) is now squeezed too;
     • the freq_etfe time-series shape change applies to both the 3-D (N,1,1) form and [y], and covers noise_spectrum_std as well;
     • the list-of-one freq_etfe and Welch freq_map crashes are fixed as a side effect;
     • only the variable-length preserve_lengths branch (:181-200) is truly untouched.
   • Optionally, add a list-of-one case to test_single_trajectory_shapes.py. Nothing tests that input form today.

Nits

2. A similar 3-D L = 1 crash remains in residual/compare with a frequency-domain model. It predates this PR and isn't in its scope; D6 owns it. sid.residual(freq_model, y[:,:,None], u[:,:,None]) raises a raw broadcast ValueError (freq_domain_sim.py:105) on both base and head.
   • For the maintainer: D6's recommended "reject 3-D with sid:dimMismatch", read literally, would also reject (N, n, 1). That would contradict the reading this PR adopts, that one trajectory is the 2-D case. I'd recommend the D6 rule apply to L > 1 only.
   • Optionally add a "Left for D6" line here.
3. The new cross-function test file isn't flagged as a deviation from the per-function convention. python/CONTRIBUTING.md:359 says one test file per public function. The parametrised file is a reasonable choice (plan C3 asks for 3-D L = 1 coverage on every function), but CLAUDE.md §1 asks for deviations to be noted in the PR description.
4. "Special cases become unreachable" isn't true for cov.py. The sid_cov squeeze (cov.py:79-82) is still reached through residual, which doesn't use validate_data; a spy confirms (200,1,1) inputs. spectrogram.py:342-347 also keeps its own L = 1 branch. Reword the sentence so a future cleanup doesn't delete code residual needs.

Verified

• Fixes: the three crashes reproduce on origin/main and are fixed here. Those are MISO freq_map (bt and welch), freq_etfe and Welch freq_map on (N,n,1), and detrend on (N,n_ch,1).
• 3-D L = 1 equals 2-D exactly, every field, compared with array_equal, warnings included. This holds across freq_bt, freq_btfdr, freq_etfe, freq_map bt and welch, on SISO, SIMO, MISO, MIMO and time-series data.
• Shared-helper audit (CLAUDE.md §3):
  • The only callers of validate_data are freq_bt, freq_btfdr, freq_etfe and freq_map.
  • Base and head give bit-identical results for 1-D, 2-D, 3-D with L = 3, equal-length lists with L = 3, and variable-length lists.
  • Error paths are unchanged: traj_mismatch still fires, and the F12 cases still raise, left to D6 as the description says.
• MISO freq_map: shapes match SPEC §6.8, and each segment equals freq_bt / _welch_estimate (max diff 0). Variable-length MISO lists also work.
• No change needed in dft.py or the Welch inner path once validation squeezes the trailing axis, which is the plan's own option.
• MATLAB: behaves per spec on the same inputs under Octave, so "MATLAB needs no change" holds.
• Mutation tests: reverting each fix fails a new test. The y squeeze fails 6 tests, the u squeeze 4, the detrend store branch 1 and the freq_map branch 2. The detrend extract-branch revert is an equivalent mutant.
• PR claims and lint: "6 of the 9 shape tests fail against the old code" and "428 passed, 4 skipped" both reproduce. ruff is clean.
• Cross-PR: merges cleanly with the other four open PRs. On the trial merge, the full Python suite reports 500 passed, 4 skipped.

---

Generated by Claude Code