Calibrate filter chain from real samples before changing dumbify intensity

[twistedmelonman]

Output from the personify → dumbify chain needs two opposite kinds of manual fix before it can be posted: soft-touches get restored, and padding gets removed — on the same text. No single intensity setting explains that, and no intensity change fixes it. The pipeline is mis-targeted, not mis-dialed.

Evidence that the dial is the wrong control: d96055f (#6) moved intensity deliberately the other way — rewrote all four intensity examples to be less compressed, added the "Over-compression into telegram-speak" failure mode, added "Level 2 is a loose, lowercase, minimal-punctuation clause — not yet a telegram", and added guidance to preserve natural hedges (imo, i think). The symptom survived that change.

Scope: gather evidence, then edit

Before any change to SKILL.md, capture real samples — original text, the output of each filter stage separately, and the maintainer's final posted version. Diffing the final edit against each individual stage localizes the culprit: a soft-touch that survived personify but died in dumbify indicts dumbify specifically, which an end-to-end diff can't show.

Classify each restoration as one of:

• a dumbify bug,
• an upstream personify bug,
• a context-specific judgment call that shouldn't be encoded at all.

The current failure mode is treating all three as "too aggressive" and turning the dial.

Use the calibrate-register skill to collect these. It exists for exactly this: runs each filter stage separately, saves every intermediate output (unrecoverable after the fact — captured even when a stage appears to change nothing), records the final posted version verbatim, interviews on each delta, and appends a structured sample to ~/.claude/calibration-samples/. It is capture-only by design — it does not propose skill edits or generalize from a single sample.

No SKILL.md changes until that evidence accumulates across several real samples — the skill puts the threshold at roughly five, and treats generalizing from them as a separate, explicitly-invoked task.

Known gap, pending evidence

Dumbify has no guidance for running as a second compression pass. SKILL.md says nothing about the case where personify has already de-abstracted and compressed the input. The risk there is double-cutting, not timidity — pr-review already guards it ("if the compressed draft has lost a file path, line reference, identifier, number, or a hedge, keep the longer wording"), but dumbify itself doesn't know it may be running on already-compressed input and shouldn't force a delta. Likely the first thing samples will confirm.

Rejected as originally filed

Original proposals and why they don't hold (full reasoning in the first comment)

1. "Is every remaining word carrying a fact?" as a pass/fail gate. The audit has no such line; the nearest is "is any word present only to sound polite, careful, professional, or educated?", which is better. As a hard gate this deletes exactly what docs: tighten register examples toward natural engineer tone #6 protected — hedges (i think carries stance, not fact) and the connective words that keep output from reading as a telegram — and contradicts the Over-compression failure mode. Since SKILL.md resolves conflicts by layer precedence, a non-negotiable-tier gate would silently override both. Legitimate kernel: the audit is 13 flat questions with no priority, so nothing is load-bearing. Fixable by grouping/ordering (cut-harder checks vs. don't-overcut checks) without an absolute that fights the rest of the skill.

2. Default level 3 for PR/review-comment and Slack modes. Beyond reversing a same-week decision, this breaks a live downstream consumer. pr-review Phase 5 says: "Don't raise dumbify's intensity above the default unless the invoker asks; levels 3-4 compress harder than a review comment carrying a technical finding can usually afford." Raising dumbify's default moves the floor underneath that instruction — pr-review would receive level-3 output while believing it asked for something safe. Only possible in lockstep with Make the "too many words, not enough substance" gate explicit in Phase 5 pr-review#13.

3. "No mercy" second pass after personify. Right location, wrong direction — see "Known gap" above.

Linked: smartwatermelon/pr-review#13, twistedmelonman/personify#49. Same objection likely applies to pr-review#13; personify#49 is a different shape and mostly unaffected.

[twistedmelonman]

Holding this open, but the remedy here is probably wrong for the problem, and I want to record why before anyone implements it.

Observed symptom is bidirectional. After running PR review text or a proposed Slack message through the full chain, the maintainer reports doing two kinds of edit on the same output: restoring soft-touches and removing over-wordiness. A global intensity increase can't explain that and can't fix it — turning the dial up makes the first category strictly worse while only partly helping the second. The pipeline is miscalibrated in two directions at once: cutting texture that makes text land, while leaving padding that carries no payload. That's a targeting problem, not a dial position.

This partly re-litigates d96055f (#6), merged one day before this issue was filed. That commit deliberately moved the skill the other way: it rewrote all four intensity examples to be less compressed (what is now level 2 was previously labeled level 1), added the "Over-compression into telegram-speak" failure mode, added "Level 2 is a loose, lowercase, minimal-punctuation clause — not yet a telegram", and added guidance to preserve natural hedges (imo, i think). Notably, #6 did not fix the symptom — which is further evidence the dial is the wrong control.

Per-proposal:

1. "Is every remaining word carrying a fact?" as pass/fail — reject as written. The audit has no such line; the nearest is "is any word present only to sound polite, careful, professional, or educated?", which is better. As a hard gate, "every word carries a fact" deletes exactly what docs: tighten register examples toward natural engineer tone #6 protected — hedges (i think carries stance, not fact) and the connective words that keep output from reading as a telegram — and contradicts the Over-compression failure mode directly. Since SKILL.md resolves conflicts by layer precedence, a non-negotiable-tier gate would silently override both. The legitimate kernel: the audit is 13 flat questions with no priority, so nothing is load-bearing. Fixable by grouping/ordering (cut-harder checks vs. don't-overcut checks) without an absolute that fights the rest of the skill.

2. Default level 3 for PR/Slack modes — reject. Beyond reversing a same-week decision, it breaks a live downstream consumer. pr-review Phase 5 says: "Don't raise dumbify's intensity above the default unless the invoker asks; levels 3-4 compress harder than a review comment carrying a technical finding can usually afford." Raising dumbify's default to 3 moves the floor underneath that instruction — pr-review would receive level-3 output while believing it asked for something safe. This can only be done in lockstep with pr-review#13.

3. Dumbify-after-personify — the real gap, and the only genuinely new information in the issue. SKILL.md says nothing about running as a second compression pass. But the evidence points the opposite way from "no mercy": if personify already compressed, the second-pass risk is double-cutting, not timidity. pr-review already guards this ("if the compressed draft has lost a file path, line reference, identifier, number, or a hedge, keep the longer wording"). What dumbify is missing is an explicit "you may be running on already-compressed input; don't force a delta" note.

Path forward — evidence before edits. Rather than specifying the target register declaratively (which is what keeps producing overcorrections in both directions), capture real samples: original text, the output of each filter stage, and the maintainer's final posted version. Diffing the final edit against each individual stage localizes the culprit — a soft-touch that survived personify but died in dumbify indicts dumbify specifically, which an end-to-end diff can't show. Each restoration then gets classified as a dumbify bug, an upstream personify bug, or a context-specific judgment call that shouldn't be encoded at all. The current failure mode is treating all three as "too aggressive" and turning the dial.

No changes here until that evidence accumulates across several real samples. Filed identically across three repos, so the same objection likely applies to pr-review#13 (see comment there); personify#49 is a different shape and mostly unaffected.