Extraction model: "FORCE Sonnet for quality" is an unexamined default — Haiku matched fact-density at 3.6x speed (measured); make the tier configurable + correct the docstring

[DarrenZal]

The extractor hard-comments that Sonnet is required for extraction quality:

• module docstring: "runs the document extractor prompt per window via claude -p (Sonnet — forced for extraction quality)"
• extract_deep_documents.py:~1112: # Extract any window not already cached (FORCE Sonnet for quality).

The model is in fact env-tunable (CLAUDE_P_MODEL, :80), so this is a documentation/default問題 rather than a code limitation — but the comment is load-bearing: it discourages tuning the single biggest latency lever, and it does not appear to be backed by a measurement.

Measured A/B (2026-07-16)

Same pipeline, --tier thorough, one paper per row; Sonnet rows are papers ingested the same night.

model	paper	chars	facts	facts/1k chars	throughput
Sonnet 4.6	coecke-discocat (1003.4394)	69,551	27	0.39	~87 chars/s
Sonnet 4.6	contextual-fraction (1705.07918)	73,049	30	0.41	~87 chars/s
Sonnet 4.6	over-squashing MPNN (2302.02941)	82,093	48	0.58	~87 chars/s
Sonnet 4.6	sheaf-nonlocality (1102.0264)	108,235	45	0.42	~87 chars/s
Haiku 4.5	MPSN (2103.03212)	127,118	60	0.47	312 chars/s

Haiku: ~equal fact density (0.47 vs 0.45 Sonnet mean), 3.6x faster (407s vs ~800s/paper).

Not count-inflation — spot-checked Haiku facts are specific and technical, same register as Sonnet:

• "The k-Folklore WL procedure (k-FWL) is equivalent to (k+1)-WL for k ≥ 2."
• "The Hodge Laplacian operator for dimension p can be expressed as a function of boundary operators…"
• "The ocean drifter trajectories dataset is collected around the island of Madagascar." (a real detail from the paper)

Honest limits of this evidence

n=1 for Haiku, on a geometric-deep-learning paper, and it compares fact density, not fact correctness against source text. It does not establish "Haiku ≥ Sonnet" in general — only that the Sonnet default is unexamined and costs ~3.6x latency on at least one common document class.

Proposal

1. Correct the docstring/comment — replace the "forced for quality" assertion with the actual state: model is CLAUDE_P_MODEL-tunable; Sonnet is a conservative default, not a measured requirement.
2. Support per-tier / per-document model selection (e.g. DOC_EXTRACTOR_MODEL_BY_TIER, or honor an existing per-call override) so callers can tier: cheap model for surveys/empirical papers, stronger model for subtle formal math.
3. Optionally run a broader benchmark (N papers × both models, facts scored against source) before changing the shipped default. Happy to share the harness — a caller-side tiering drainer is at ~/Documents/Research/Papers/tools/drain_remaining.py locally.

Note this is the only real speed lever available: extraction is globally serialized by the advisory lock in #35, so parallelism buys nothing.

Related: #35 (global lock), #36 (dead-socket wedge), #32/#33/#34.

[DarrenZal]

The docstring half of this is fixed in 24f0294 — the module header no longer asserts "Sonnet — forced for extraction quality", which is how the claim kept propagating as if it were a constraint rather than a default nobody had tested.

The tunability half was already in place (DOC_EXTRACTOR_MODEL / DOC_EXTRACTOR_TRANSPORT); only the docs contradicted it.

Fresh evidence for the tiering claim, at a much larger n than the original measurement. The entire Cynthia Kurtz corpus was extracted on Haiku 4.5 via the claude_p (subscription) transport on 2026-07-31/08-01:

sources	4 books (1,487 pp), a 227-post blog, a GitHub design dir, a catalysis report
output	2,466 chunks · 2,307 facts · 1,038 discourse moves
windows	198 across 7 documents, every doc windows_done == window_count

Spot-checks showed specific, grounded, correctly-attributed facts — e.g. "The unionized workplace project collected over 370 stories from staff across Canada over 4 years", "Narrative Catalysis is an optional pattern-finding component of Participatory Narrative Inquiry" — and all 8 discourse-move types present (33 claims / 23 definitions / 17 premises / 17 evidence / 16 implications / 9 theses / 8 counterpoints / 6 open questions on one book). No quality deficit observed against Sonnet-era output.

Note the metered api transport is still unusable regardless of model choice — probed 2026-07-31, HTTP 400 "Your credit balance is too low" — so Haiku is reachable only via claude_p. TELUS Qwen (openai transport) returned 504 Gateway Timeout plus truncated JSON on 45k-char windows and was abandoned for this corpus.

Leaving open for whoever wants to formalise the tier policy in code rather than in the memory file.