Plan: docs/plans/voice-corpus-mining.md
VOICE.md is currently derived from one recent blog corpus. That gap caused a concrete wrong inference: its Conflicts section told Personify to preserve em dashes and complete-sentence fastidiousness at work, which is exactly the register that drew manager warnings. Six blog posts described the published voice and said almost nothing about the work voice.
The second reason is better. The flagged register is about 30 years old, going back to Usenet in the mid-90s and Slashdot in the early 2000s, so it predates any language model by decades. A corpus spanning that range turns "this is how he writes" from an inference into something demonstrable, and settles whether the register is his or an artifact.
Sources, ranked by value per unit of effort:
- Gmail sent mail, 2004 to present. Highest value and the only source covering the work register directly. Also the most sensitive: 22 years of personal mail, most of it irrelevant to writing style.
- Usenet, 1995 to 2006. The provenance evidence. Google Groups is dead for this since early 2024; usenetarchives.com and the Archive Team dumps are the live options. Needs the posting address from that era.
- Slashdot, early 2000s. Lowest value, short argumentative comments rather than sustained prose. No export exists.
The plan's load-bearing idea is that filtering and sampling happen before any model runs. Six plain-text stages (sent-only, strip quoted and forwarded text, strip signatures, drop short messages, drop boilerplate, dedupe) cut the large majority by count, then stratified sampling targets a few hundred messages rather than exhaustive coverage, because style analysis saturates fast. Local Qwen is preferred over a hosted model, since the corpus is two decades of personal mail.
No token estimate, deliberately. Post-filter volume is the only input that matters and it can't be known until Takeout lands.
Not blocking anything. VOICE.md v0.2 already produces what the manager feedback asked for; this is the durable version. Start with the Takeout request, which has the longest lead time.
Plan: docs/plans/voice-corpus-mining.md
VOICE.md is currently derived from one recent blog corpus. That gap caused a concrete wrong inference: its Conflicts section told Personify to preserve em dashes and complete-sentence fastidiousness at work, which is exactly the register that drew manager warnings. Six blog posts described the published voice and said almost nothing about the work voice.
The second reason is better. The flagged register is about 30 years old, going back to Usenet in the mid-90s and Slashdot in the early 2000s, so it predates any language model by decades. A corpus spanning that range turns "this is how he writes" from an inference into something demonstrable, and settles whether the register is his or an artifact.
Sources, ranked by value per unit of effort:
The plan's load-bearing idea is that filtering and sampling happen before any model runs. Six plain-text stages (sent-only, strip quoted and forwarded text, strip signatures, drop short messages, drop boilerplate, dedupe) cut the large majority by count, then stratified sampling targets a few hundred messages rather than exhaustive coverage, because style analysis saturates fast. Local Qwen is preferred over a hosted model, since the corpus is two decades of personal mail.
No token estimate, deliberately. Post-filter volume is the only input that matters and it can't be known until Takeout lands.
Not blocking anything. VOICE.md v0.2 already produces what the manager feedback asked for; this is the durable version. Start with the Takeout request, which has the longest lead time.