quantize_q4 output dir is not self-contained: missing config.json breaks from_q4_dir / eval_perplexity

[ohdearquant]

Summary

quantize_q4 writes .q4 / .f16 / quantize_index.json to its --output-dir, but not config.json. The runtime Q4 loader path — MetalQwen35State::from_q4_dir via eval_perplexity's load_cfg_for_q4 — reads config.json from the Q4 dir and errors if it is absent:

ERROR: Q4 dir <output-dir> is missing config.json

So a freshly-produced quantize_q4 directory cannot be loaded/evaluated without manually copying config.json in from the source model dir.

Inconsistency

quantize_quarot produces a self-contained artifact (its output dir includes config.json), so its output loads directly. quantize_q4 does not, which is surprising given both feed the same from_q4_dir loader. The gap is easy to miss because a pre-existing Q4 dir often already has config.json from an earlier setup — it only surfaces on a fresh artifact.

Repro

target/release/quantize_q4 --model-dir <model> --output-dir /tmp/q4
ls /tmp/q4                 # .q4 / .f16 / quantize_index.json, no config.json
target/release/eval_perplexity --q4-dir /tmp/q4 --tokenizer-dir <model> \
  --corpus-file <corpus>   # ERROR: Q4 dir /tmp/q4 is missing config.json

Suggested fix

Have quantize_q4 copy (or write) config.json into its --output-dir, matching quantize_quarot, so any Q4 artifact is self-describing and directly loadable. Alternatively, document that a Q4 dir requires config.json alongside the .q4 files and have callers provide it.

Context

Discovered while wiring the #616 Q4 perplexity CI gate: the gate's ephemeral-artifact step now copies config.json in as a workaround. Making quantize_q4 self-contained would remove that workaround and prevent the same surprise for anyone quantizing then loading/evaluating.