Skip to content

Behaviour fixes and reorganisation; fix the DAF duration metric - #14

Merged
jmxpearson merged 3 commits into
mainfrom
behaviour-and-organization
Oct 23, 2026
Merged

jmxpearson merged 3 commits into
mainfrom
behaviour-and-organization

Conversation

@jmxpearson

Copy link
Copy Markdown
Member

Points 3 (behaviour fixes) and 4 (organisation) from the review, plus the duration-metric fix.

Organisation

  • VocalErrorNet replaces VocalErrorNetV2. Its defaults are the shipped operating point, so VocalErrorNet(n_aud=encoder.n_channels) builds the published network; n_aud is required. VocalErrorNetV2 stays as a subclass with its old defaults unchanged, because finchsim's tests build networks from them. finchsim's spiking-ven tests give the same result against this branch (12 passed, 3 skipped).

  • Every trained value now lives in constants.py: TAU_I, C_JEI, NOISE_E, NOISE_I, TARGET_RATE_E, TARGET_RATE_I and T_REFR_MS, which existed only as class defaults, plus the encoder front end (CAUSAL_N_BASES, CAUSAL_K_FRAMES, CAUSAL_N_COCH_CH, CAUSAL_FRAME_RATE, CAUSAL_LO_HZ, CAUSAL_HI_HZ, CAUSAL_RMS_REF). All match the shipped artifacts. Units are documented.

  • evaluate.py split into stimuli.py, metrics.py and targets.py; evaluate.py remains as a re-export.

  • K labels removed:

    Where Old New
    function daf_metrics rendition_rates
    function format_metrics format_rendition_rates
    JSON / dict key k1 trained_hz
    JSON / dict key k4_rate reversed_hz
    JSON / dict key k4 reversed_over_trained
    JSON / dict key k2_no_hvc daf_not_singing_hz
    traces array rc trained_hz
    traces array rn reversed_hz
    traces array jie jie_mean
    traces scalars rc_hvc etc. final_*
  • New docs: docs/MODEL.md (equations as implemented, units, glossary), linked from the README.

Behaviour fixes

  • load() is strict: missing required keys raise an error that lists them all; features added after the first saved models load with documented "off" defaults.
  • The model records its stimuli (ven.stimulus): sample rate, seed, protocol timing, rate calibration, DAF window, and checksums of the encoder and corpus. sven-evaluate and sven-figure rebuild from that record, warn when a flag overrides it or a checksum differs, and take n_hvc from the model (sven-evaluate --n-hvc is removed). New flags: --daf-window-s, --stimulus-seed.
  • The DAF window flows from build_stimuli through the metrics and the PSTH figure; nothing hardcodes the 400 ms onset or the 50 ms burst any more.
  • sr defaults to the corpus's own rate, and a mismatch is an error.
  • motifs.npz stores template_index, the DTW reference. The shipped corpus has no such key and still picks template 25.
  • Training monitoring uses its own noise stream, so logging no longer changes the trained weights.

Duration metric

The end of a response is now searched for up to 300 ms after onset, capped at the motif end, instead of dropping responders still responding at 150 ms. Responses still running at the window end are counted at the length reached and reported as censored. Two off-by-ones are fixed. Only duration changes: 46.4 → 47.5 ms at the test seed (5-seed mean 45.9 → 46.0). The model's responses really are at the bird's lower edge (47 ms).

Known-answer tests show latency is recovered exactly, while the paper's duration rule reads about 8 ms long, because chance-significant baseline bins postpone the 10-bin end run. The bird's number was measured with the same rule, so the comparison is like for like.

Retrained network

Retrained only because training no longer shares its noise with monitoring. Every metric is within its 5-seed spread of the previous model, and the reversed/trained ratio is 1.174 vs 1.173. A pure-defaults run reproduces ven_model.npz byte-for-byte; MANIFEST.sha256 is updated. The encoder and corpus are unchanged.

Pinned values (seed 42): responders 52.2%, baseline 2.52 Hz, latency 30.5 ms, duration 47.5 ms, extra spikes 1.74, peak 44.6 Hz.

Verification

183 tests pass (fast and slow, including Brian2 parity at the operating point); ruff clean. Evaluating the previous shipped model gives identical metrics before and after the refactor.

Before merging

Copy the new outputs/ven_model.npz and ven_model_traces.npz into existing checkouts (or run make clean-artifacts && make figure); the encoder and motifs.npz are unchanged.

🤖 Generated with Claude Code

jmxpearson and others added 3 commits October 23, 2026 10:03
…nsform, stimulus metadata, new constants

VocalErrorNetV2 is kept as a subclass with its earlier defaults, so existing
callers (finchsim) behave as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- train_ven's per-rendition monitoring draws its own noise, so training no
  longer depends on logging; the network is retrained (within noise of the
  previous one) and a pure-defaults run reproduces it byte-for-byte.
- daf_response searches for the end of a response up to 300 ms after onset
  (capped at the motif end) instead of dropping responders still responding
  at 150 ms; responses still running at the window end are counted at the
  length reached and reported as censored. Fixes two off-by-ones in the
  onset/offset searches. Duration 46.4 -> 47.5 ms at the test seed (5-seed
  mean 45.9 -> 46.0); no other metric changes.
- Known-answer tests for latency and duration (tests/test_metrics.py).
- README: links to docs/MODEL.md and design notes; stimulus provenance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jmxpearson
jmxpearson marked this pull request as ready for review October 23, 2026 14:32
@jmxpearson
jmxpearson merged commit 586bbff into main Oct 23, 2026
2 checks passed
@jmxpearson jmxpearson mentioned this pull request Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant