Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion src/lib/webllm/post-process.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -231,10 +231,30 @@ describe("cleanRewriteLine", () => {
["• 4.2% conversion lift.", "4.2% conversion lift."],
// Not marker-prefixed at all — the same branch mangled this on its own.
["3.5x revenue growth.", "3.5x revenue growth."],
])("does not eat a decimal a marker strip uncovers: %s", (input, want) => {
// The dash-side twin of the same bug (#821). `-` is the one marker that
// is also ordinary content — a minus sign — so the zero-space branch ate
// the sign a `- ` strip had just uncovered. A reduction then reads as a
// gain, and unlike a mangled decimal the line stays wholly plausible.
["- -5% churn in Q3.", "-5% churn in Q3."],
["- -12% cost reduction.", "-12% cost reduction."],
["• -5% churn in Q3.", "-5% churn in Q3."],
["- -3.5x infra cost.", "-3.5x infra cost."],
// Single-pass shapes — no marker to uncover the sign, so this branch
// mangled them on its own, as with the bare decimal above.
["-5% churn in Q3.", "-5% churn in Q3."],
["-.5% weekly churn.", "-.5% weekly churn."],
])("does not eat a number a marker strip uncovers: %s", (input, want) => {
expect(cleanRewriteLine(input)).toBe(want);
});

it("preserves a leading minus sign, for any marker", () => {
// Sign-preservation counterpart to the decimal property below, asserted
// as a property so a new marker branch has to satisfy it too.
for (const marker of ["-", "*", "•", "1.", "1)"]) {
expect(cleanRewriteLine(`${marker} -5% churn.`)).toBe("-5% churn.");
}
});

it("preserves every digit of a leading decimal, for any marker", () => {
// The number-preservation counterpart to the `not.toContain("*")`
// property above. Asserted as a property so a new marker branch has to
Expand Down
18 changes: 16 additions & 2 deletions src/lib/webllm/post-process.ts
Original file line number Diff line number Diff line change
Expand Up @@ -106,11 +106,25 @@ const LEADING_BOLD_WORD_PATTERN = /^\*\*([A-Za-z][\w-]*)\*\*\s+/;
* `- 3.5x revenue growth` became `5x revenue growth`. Silent numeric
* corruption on the product path, in a bullet the rewrite prompt actively
* asks the model to quantify. Nothing needs a tight `1.Foo`.
* - **`-` and `•` allow `\s*`**, so a tight `-Shipped X` still normalizes.
* - **`•` allows `\s*`**, so a tight `•Shipped X` still normalizes. Safe
* with zero-or-more because `•` is only ever a glyph — it never carries
* meaning as part of the text it precedes.
* - **`-` allows `\s*` only when what follows is not a number.** `-` is the
* one marker that is also an ordinary character mid-content: a minus sign.
* With a bare `\s*` this branch was the dash-side twin of the numbered bug
* above — pass 1 stripped `- ` from `- -5% churn`, pass 2 read the
* uncovered `-` as a glyph and ate it too, shipping `5% churn`. A
* reduction reads as a gain, and the line stays grammatical, so nothing
* downstream looks wrong. The bare `-5% churn` shape needed only one pass
* and predates the loop entirely. `(?!\.?\d)` withholds the zero-space
* branch from a sign, covering `-.5%` as well as `-5%`; the `\s+` branch
* is unguarded because `- 5%` is a marker plus a positive number, and a
* genuine negative is written `- -5%`. Nothing needs a tight `-5`-as-marker
* reading. Fix for #821.
* - **`*` requires `\s+`**, because `*X*` is italics and is handled by the
* paired-emphasis strip instead.
*/
const LIST_MARKER_PATTERN = /^(?:\d+[.)]\s+|[•\-]\s*|\*\s+)/;
const LIST_MARKER_PATTERN = /^(?:\d+[.)]\s+|•\s*|-(?:\s+|(?!\.?\d))|\*\s+)/;

/**
* Runaway guard for the markdown-prefix loop in `cleanRewriteLine` — not a
Expand Down
Loading