fix(ghidra): raise pinned benchmark output_tokens 512 -> 4096 (#1804 follow-up)

[Xore]

The 512-token cap was truncating Ghidra triage and benchmark answers, not limiting them.

Evidence

Measured on the Tier A corpus, 2026-09-25, with #159's own harness:

cap	base cut off	fine-tune cut off
512	14 of 14	9 of 14
2048	1 of 17	0 of 17
4096	0 of 17	0 of 17

The case that survived 2048 was file_write_persist, done_reason: length at 5127 chars (~1900 tokens), so 2048 is not enough either. 4096 is validated by measurement, not chosen for tidiness.

This is not new. #2694 already recorded the same cap truncating injection conclusions in 23 of 30 Tier B answers. Any baseline recorded at 512 is suspect in the same direction — and the base model is penalised hardest, because it writes longer answers. That is what made the #1804 candidate-2 A/B look like a 6-point win for the fine-tune when it is 2.

The change, and why it is three files

ghidra moves to 4096 in approved-models.json (qualification_request and runtime_request), and ghidra-worker.py's TRIAGE_OUTPUT_TOKENS default moves with it. Those are independent defaults, and CI already pins them together — the Ghidra output cap matches the approved runtime request check failed on the manifest-only change, which is the invariant working. Had the manifest been changed alone, the live worker would have kept truncating while the manifest claimed otherwise.

revdeck.qualification_request also moves to 4096: it is the benchmark's own measurement slot, and the largest measured prompt is 2008 tokens, so 2008 + 4096 = 6104 < its 8192 context.

sessions deliberately stays at 512. That slot is llm-worker session classification, not reverse engineering, and llm-worker clamps LLM_OUTPUT_TOKENS to max=2048 under an 8192 context. Its 512 was never a truncation defect, and raising it would have made the manifest value unrepresentable in the consumer that serves it.

Tests: the one hardcoded 512 expectation in test_ghidra_worker.py is updated. Nothing else needed loosening — the governance and baseline tests carry their own local request dicts, and 76 pass.

Re-measured at 4096

tierA_approved_qwen3_14b_cap4096_run1.json is committed: qwen3:14b scores 76/83 (91.6%) across 17 cases, 0 truncated, 0 done_reason: length. That is the model approved-models.json names for all three slots, so it is the baseline most likely to be quoted anywhere, and it has now been measured clean.

Known and not fixed here

• hf.co/mradermacher/GPT-OSS-Cybersecurity-20B-Merged-heretic-GGUF:Q4_K_M returns empty content on every request, at every reasoning_effort and frequency_penalty, including a bare request with no options. gpt-oss:20b answers the same prompt normally, so this is the checkpoint, not the runtime or the cap. It scores 0/83 and that number is meaningless. This is a model problem for One benchmark, every model, every aspect: rebuild the model matrix on current hardware with error bars #1947, not a cap problem, and is not addressed by this PR.
• Existing recorded baselines at 512 are not comparable to these. One benchmark, every model, every aspect: rebuild the model matrix on current hardware with error bars #1947's matrix is where the rest gets re-measured.

Refs #1804, #1947, #2694

[github-actions]

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None