v0.4.3 — interactive-editing perf (-19% median, -26% p95)

Performance release focused on interactive editing. One breaking change (SourceSpan record components).

Highlights — interactive editing on 1,900-LOC Java fixture

Metric	0.4.2	0.4.3	Change
Median per-edit	13.3 ms	10.8 ms	-19%
p95	30.1 ms	22.4 ms	-26%
p99	56.4 ms	53.3 ms	-5%
% under 16ms (frame budget)	83%	91.5%	+8.5pp

Breaking change

SourceSpan record components changed from (SourceLocation start, SourceLocation end) to (int startLine, startColumn, startOffset, endLine, endColumn, endOffset). The start() and end() methods are preserved (now lazy-materialize SourceLocation). Code that pattern-matches on the old shape needs migration; code calling the methods is unaffected. Drove the 26% p95 improvement.

Changed

• NodeIndex.build pre-sizes its IdentityHashMap from a descendant-count pass — eliminates resize churn (was 22.8% of bench allocations).
• Generated parsers also emit the new int-based SourceSpan shape — regenerate to pick up gen-time improvements.

New

• IncrementalSessionBench in peglib-incremental/src/jmh/java — durable measurement infra for realistic 1,000-edit interactive sessions with cursor-moved-to-edit AND cursor-pinned regimes. Per-class p50/p95/p99/max + frame-budget hit rate + top-10 outlier diagnostics.
• JVM tuning recommendation: -Xms2g -Xmx2g -XX:MaxGCPauseMillis=20 reduces p99 by 33% (60ms → 40ms) — free for downstream.

Coming next

Architectural rework planned for 0.5.0. See docs/incremental/ARCHITECTURE-0.5.0.md (when published) — moves to incremental-native data structures (stable node IDs, top-down pivot, IR-unified runtime). Eliminates the per-edit fixed overhead that's still ~10ms today.

v0.4.2 — standalone-parser fix

Single-fix maintenance release. No public API changes for the interpreter / incremental / formatter paths.

Fixed

• Generated parsers are now truly standalone. The emitted RuleId interface no longer extends org.pragmatica.peg.action.RuleId, and the emitted parseRuleAt signature uses the locally-emitted RuleId. Downstream projects without peglib on their compile classpath can now build the emitted source.
• No FQCN references in emitted code. Audit removed all fully-qualified class names (org.pragmatica.peg.action.RuleId, java.util.ArrayDeque, etc.) from generator emission templates. Proper imports are emitted instead.

Why

Previous releases' generated parsers contradicted the documented contract on ParserGenerator ("The generated parser depends only on pragmatica-lite:core"). The link was vestigial — the generated withAction API uses string-based dispatch and does not need the parent type.

Drop-in upgrade

• Regenerate parsers with 0.4.2 to pick up the fix.
• Generated parsers continue to depend only on pragmatica-lite:core (for Result, Option, Cause).
• The peglib-incremental module's interpreter-based parseRuleAt API is unchanged.

v0.4.1 — 3.88× interpreter perf

Performance release. No public API changes.

Highlights

• 3.88× speedup on the interpreter (PegEngine) — full parse of 1900-LOC Java drops from 281 ms → 72.4 ms
• 3.0× speedup on incremental edits in cursor-far regime — singleCharEdit drops from 322 ms → 107 ms
• Generator's emitted parsers also pick up the furthestExpected LinkedHashSet fix (modest 8% gain on top of pre-existing optimizations)

Three landed perf wins

• HashMap cache for Grammar.rule(name) lookup in PegEngine
• Static singletons for ParseMode.standard / noWhitespace
• LinkedHashSet dedup for furthestExpected in both ParsingContext (interpreter) and ParserGenerator emission (BASIC mode)

Reverted

• A speculative inlineLocations port to the interpreter delivered 0% wall-time impact and was reverted. Findings documented in docs/incremental/V2.5-SPIKE.md addendum and docs/HANDOVER.md §6.4.

Drop-in upgrade

No code changes needed for consumers. Regenerate any generator-produced parsers to pick up the furthestExpected emission improvement.

v0.4.0 — API consolidation + JBCT polish (breaking)

Highlights

Breaking release. API consolidation pass: factory naming, parse-don't-validate Grammar, Result/Option boundary cleanups, Formatter immutable config, SessionImpl record. Six logical phases over 16 commits. All 897 tests green throughout.

No incremental v2.5 cache remap. The original 0.4.0 plan item was superseded by the v2.5 spike's NO-GO recommendation (docs/incremental/V2.5-SPIKE.md) — the actual perf lever is pivot-selection in findBoundaryCandidate, not cache invalidation. A naive lever-1 swap was tried in 0.3.6 development but hit boundary-semantic regressions; deferred until that's resolved cleanly.

What changed (BREAKING)

• Grammar parse-don't-validate. Grammar.of(...) removed; use Grammar.grammar(rules, ...) returning Result<Grammar>. Validation runs inside the factory; standalone validate() removed.
• Factory rename across the codebase. of()/create()/at() → typeName() for all public factories. Affected: SourceLocation.sourceLocation, SourceSpan.sourceSpan, ParserConfig.parserConfig, ParseResult.Success.success, ParseResult.Failure.failure, ParseResult.CutFailure.cutFailure, ActionCompiler.actionCompiler, SemanticValues.semanticValues, ParserGenerator.parserGenerator, CstHash.cstHash, SessionFactory.sessionFactory, TraceRecord.traceRecord.
• Test assertion idiom. Tests rewritten from assertTrue(result.isSuccess()) + unwrap() to result.onFailureDo(c -> fail(c.message())).onSuccessDo(...). Failure causes are no longer silently swallowed.
• SessionImpl → IncrementalSession record. Drops the Impl anti-pattern. Session interface unchanged; record components are internal-only.
• Formatter builder is now immutable. New FormatterConfig record; Formatter instances are thread-safe to share post-construction.
• PegEngine.createWithoutActions(...) returns Result<PegEngine> (symmetric with create).
• PegEngine action dispatch is now a Result.lift boundary instead of try/catch.
• Playground parseRequestBody returns Result<ParseRequest> with explicit BadRequest cause.
• Incremental nullables → Option: tryIncrementalReparse, findBoundaryCandidate. Zero (CstNode) null casts remain in peglib-incremental.

Documentation

• Maven Mojo execute() methods rewritten as Result pipelines with JBCT-boundary Javadoc explaining the lift contract.
• CLI main methods (AnalyzerMain, PlaygroundRepl, PlaygroundServer, PackratStatsProbe) carry JBCT-boundary Javadoc.

Migration

See CHANGELOG.md § [0.4.0] § "Migration guide" for per-change recipes.

Known limitations

• Incremental singleCharEdit perf is still ~325 ms/op pending the lever 1 boundary work. Targeted for 0.4.1 or later.
• Generator-side %recover was already shipped in 0.3.6 — orthogonal to this release.

Tests

897 passing (no change from 0.3.6 baseline — refactor preserved semantics throughout), 0 failures, 0 skipped.

v0.3.6 — generator-side %recover

Highlights

Generator-side %recover per-rule overrides. Generated parsers now honor the directive end-to-end, mirroring the interpreter wiring shipped in 0.3.5. Fills in the gap noted in 0.3.5's release notes: the generator previously only emitted start-rule recovery statically; non-start rules' overrides were silently dropped.

Wiring

The generated parser now carries:

• A recoveryOverride stack pushed at rule entry (when rule.hasRecover() and ErrorReporting.ADVANCED)
• A pendingFailureRecoveryOverride field — captured BEFORE the finally pop, with deepest-wins semantics
• skipToRecoveryPoint consults pending → live stack → default char-set in that priority order
• Cleared on rule success so backtracked alternatives don't leak

New regression test GeneratedParserRecoverDirectiveTest validates: pre-fix, override is silently no-op'd; post-fix, recovery lands on the override terminator. Discriminator (:) sits outside the default char-set so the test is unambiguous.

Known limitations

• Incremental parser singleCharEdit perf is still ~325 ms/op on the 1,900-LOC fixture. docs/incremental/V2.5-SPIKE.md documents the actual bottleneck (pivot overshoot in findBoundaryCandidate, not cache invalidation as v2.5 assumed). The proposed "lever 1" fix was attempted but hit correctness regressions in IncrementalParityTest due to inclusive-boundary semantics in NodeIndex.contains. Deferred — needs careful boundary work.

Tests

899 passing (+1 new), 0 failures, 0 skipped.

v0.3.5 — trivia round-trip

Highlights

Full byte-equal round-trip on all 22 perf-corpus fixtures. reconstruct(parse(source)) == source byte-for-byte across the entire corpus via the generated parser — interpreter and generator now in full parity. RoundTripTest re-enabled.

%recover directive wired end-to-end on the interpreter side. Per-rule recovery overrides now actually shift recovery point selection at runtime; new RecoverDirectiveProofTest validates with an unambiguous discriminator.

Bug fixes

Five distinct trivia-attribution bugs diagnosed and fixed this release:

• Bug A — ParsingContext.savePending/restorePending returned size-only, losing trivia consumed in backtracked branches. Now uses full List<Trivia> snapshots.
• Bug B — Cache-hit path didn't rebuild leading trivia (asymmetric vs cache-miss). Now drains pending + re-skips whitespace + reattaches.
• Bug C — Generator cached the wrapped-with-leading body. attachLeadingTrivia's short-circuit then preserved stale leading on cache hits, duplicating trivia across nested wrappers. Fix: cache empty-leading wrap, return leading-applied wrap.
• Bug C' — Trivia consumed by the body's last inter-element skipWhitespace (e.g. before an empty ZoM/Optional) ended up in pending with no claimant. Now attached to last child's trailingTrivia at rule exit. Pos is not rewound — predicate combinators rely on it.
• Bug C'' — Generator's emitted Sequence didn't roll back outer children on element failure. Symptom: trailing comma appeared as a child of both the inner ZoM-NT and the outer Sequence. Fix: snapshot children at Sequence start, restore on element failure.

Plus the %recover wiring fix: override was popped in finally before parseWithRecovery consulted it. Now captured into a per-context field with deepest-wins semantics.

Baselines

large/FactoryClassGenerator.java.txt CST hash baseline regenerated (Bug C'' fix removes a duplicate trailing comma in enum-constant lists). 21 other fixtures unchanged.

Known limitations

• Generator-side %recover per-rule overrides still only emit start-rule recovery statically. Targeted for 0.3.6.
• Incremental parser `singleCharEdit` benchmark still ≈ 325 ms/op vs SPEC's < 1 ms target. Span-rewriting cache remap (v2.5) targeted for 0.4.1 ahead of IDE-plugin consumption.

Tests

896 passing (+21 net from the 22 corpus fixtures of RoundTripTest becoming live), 0 failures, 0 skipped.

v0.3.4 — audit-driven cleanup

Post-roadmap cleanup release. Two rounds of parallel JBCT review (10 focus areas × 2 + docs + test-coverage audit) surfaced ~150 findings; this release lands P0 + P1 + P2 fixes. P3 architectural items tracked in docs/AUDIT-REPORTS/CONSOLIDATED-BACKLOG.md.

Fixed (P0)

• Thread-safety data race. PegEngine failure caches now ConcurrentHashMap + computeIfAbsent. Honors IncrementalParser's thread-safety contract in practice, not just in docs.
• Playground hardening. Path-traversal rejection, 1 MiB body cap (HTTP 413), security headers on every response, Locale.ROOT on case conversions.
• CHANGELOG drift. Three previous-release entries had inaccurate usage snippets (phantom repl / server sub-commands, nonexistent IncrementalParser.builder(...), missing HardLine record). Corrected.

Changed (P1 — minor API break)

• Actions#get(String) and Actions#get(Class<? extends RuleId>) return Option<...> instead of raw nullable. Consumer updates are one-line.
• NodeIndex#smallestContaining(int) / smallestContainingFrom(...) / parentOf(...) return Option<CstNode>.
• SessionImpl#reparseAt(...) propagates Option<CstNode>; no more null-smuggling through Result.fold.
• Result.failure(cause) → cause.result() across ~50 main-source sites (mechanical).
• PlaygroundEngine.parseWithRecovery rewritten without the Option.or((String) null) anti-pattern.
• PegEngine.whitespaceFirstChars and FirstCharAnalysis.whitespaceFirstChars now use Pragmatica Option instead of java.util.Optional.
• Defensive null-checks removed from JsonEncoder + ParseTracer (dead code under sealed CstNode hierarchy).

Added (P2)

• Left-associativity CST-shape proof test in LeftRecursionTest — a right-recursive workaround would fail it.
• %recover smoke test in RuleRecoveryTest with a documented follow-up (directive doesn't yet visibly shift behavior on the tested input; tracked as P3).
• docs/AUDIT-REPORTS/ directory: CONSOLIDATED-BACKLOG.md, docs-backreference.md, test-coverage-proof.md, plus the fix-needed lists.

Deferred (P3)

Substantial architectural work surfaced and catalogued in CONSOLIDATED-BACKLOG.md:

• Parse-don't-validate: Grammar#validate() → factory returning Result<Grammar>.
• Factory naming: of() / create() / at() → typeName() across ~20 public API sites.
• SessionImpl → record rename.
• Mojo execute() → Result-pipeline + @Contract boundary.
• Formatter mutable builder → immutable config + separate builder.
• Test idiom mass-rewrite (assertTrue(result.isSuccess()) → .onFailure(Assertions::fail).onSuccess(...)) — ~1000 sites.
• parseRequestBody / HttpHandler → Result.lift boundary.
• PegEngine.createWithoutActions → Result<PegEngine> symmetry.
• Trivia round-trip completion (rule-exit pos-rewind + baseline regen) — still the primary outstanding feature from 0.2.4.

Tests

• Aggregate: 874 passing, 1 skipped (pre-existing RoundTripTest), 0 failures.
• peglib-core 676 (+2), peglib-incremental 100, peglib-formatter 66, peglib-maven-plugin 5, peglib-playground 27 (+5 adversarial smoke).

Breaking changes

Minor — return type on Actions#get overloads. Primary public API (PegParser, ParserConfig, Grammar, CstNode, IncrementalParser.initialize / edit) unchanged.

v0.3.3 — peglib-formatter framework

Final release in the 0.2.3 → 0.3.3 roadmap. New peglib-formatter module ships a Wadler-Lindig pretty-printer framework for grammar-driven formatters, closing peglib as a complete language-tool substrate: parse → incremental reparse → format.

Doc algebra

Sealed Doc interface with seven records (Text, Line, Softline, Group, Indent, Concat, Empty) + static builders in Docs. Canonical Wadler algebra implemented in ~200 LOC.

Fluent DSL

var formatter = Formatter.builder()
    .rule("Block", (ctx, children) ->
        group(text("{"), line(), indent(4, children.concat()), line(), text("}")))
    .rule("ArgList", (ctx, children) ->
        children.joinWith(text(","), line()))
    .defaultIndent(4)
    .maxLineWidth(100)
    .build();

Result<String> output = formatter.format(cst);

Rule lookup by CstNode#rule() name. Children are pre-formatted before the user's rule combinator runs.

Demo formatters

Three end-to-end demos with idempotency fuzz tests:

• JSON — objects with braces + indented members; arrays similar.
• SQL-like — clause-per-line, keywords upper-cased.
• Arithmetic — precedence-aware with minimal parentheses.

Each passes format(format(x)) == format(x) on fuzz inputs.

Limitations

Trivia preservation is best-effort in v1. Depends on 0.2.4's trivia attribution foundation, which still has a deferred rule-exit pos-rewind for trailing intra-rule trivia. Format output may not byte-equal source for all inputs today; full round-trip lands when the trivia foundation is completed.

Module layout

peglib-formatter depends on peglib-core. Installable independently or via the peglib-parent reactor.

Breaking changes

None. Strictly additive.

v0.3.2 — incremental v2 (trivia-aware) + JMH

Summary

Trivia-aware reparse splice in peglib-incremental + deferred JMH benchmark harness from 0.3.1.

• TriviaRedistribution internal helper redistributes leadingTrivia / trailingTrivia between a spliced subtree and its neighbors after the splice.
• Opt-in trivia-only edit fast-path (IncrementalParser.builder(...).triviaFastPathEnabled(true)) — detects pure-whitespace edits and patches the CST without reinvoking the parser. Default off; safe grammars opt in.
• IncrementalBenchmark (JMH) parametrized over initialize / singleCharEdit / wordEdit / lineEdit / fullReparse / undoRestore.

Honest performance reporting

Smoke benchmark (single iteration, JDK 25, Apple Silicon) measured singleCharEdit at ~325 ms/op on the 1,900-LOC fixture. Target was < 1 ms. Root cause: wholesale packrat cache invalidation (SPEC §5.4 v1 decision) + back-reference fallback on the Java grammar.

The performance unlock is v2.5 span-rewriting cache remap — deferred to a future release. peglib-incremental v2 ships with correctness intact (parity harness still green across 2,200+ CstHash checks) but honest perf numbers. Not claiming editor-scale speed today.

Tests

• peglib-core: 674 + 1 skipped — unchanged.
• peglib-incremental: 67 → 100 passing. +33: TriviaRedistributionTest (11), IncrementalTriviaParityTest (22 trivia-biased corpus runs).
• Aggregate: 801 passing + 1 skipped, 0 failures, 0 errors.

Deferred

• v2.5 span-rewriting cache remap (the actual single-char edit perf unlock).
• Grammar-analyzer-driven triviaFastPathEnabled per rule (currently blanket opt-in).

Breaking changes

None.

v0.3.1 — peglib-incremental v1

First real implementation of peglib-incremental. Cursor-anchored stateful parser that reparses only the subtree affected by an edit. Implements docs/incremental/SPEC.md v1.

Public API

org.pragmatica.peg.incremental:

IncrementalParser parser = IncrementalParser.create(grammar);
Session s = parser.initialize(buffer, cursorOffset);

s = s.edit(offset, oldLen, newText);   // returns new immutable Session
s = s.moveCursor(newCursor);            // pure hint, no reparse
CstNode tree = s.root();
Stats stats = s.stats();

Records: Edit(offset, oldLen, newText), Stats(reparseCount, fullReparseCount, lastReparsedRuleOrd, lastReparsedNodeCount, lastReparseMs, cacheSize).

v1 scope

• CST-only — actions run only on full-reparse fallback (no action replay).
• Wholesale packrat cache invalidation on edit — span-rewriting remap deferred to v2.5.
• Back-reference rules fall back to full reparse. Grammar analysis marks BackReference-containing rules at Grammar.validate(); reparse boundary promotes to full reparse whenever an edit lands inside one.
• Immutable sessions enable O(1) undo via session retention.

Correctness gate

Deterministic fuzz harness: 22 corpus files × 100 random edits = 2,200 CST-hash parity checks under seed 0xC0FFEE42. All green. Scales to 22,000 via -Dincremental.parity.editsPerFile=1000.

Tests

• peglib-core: 674 passing + 1 skipped — unchanged (zero regression).
• peglib-incremental: 67 passing (45 unit + 22 parity).
• Aggregate across all modules: 768 passing + 1 skipped, 0 failures, 0 errors.

Deferred to 0.3.2

• JMH performance gates (SPEC §7.4).
• Trivia-aware reparse splice (SPEC §5.4 v2) — rides on 0.2.4's trivia threading foundation.

Notes

RuleIdRegistry uses JEP 457 (java.lang.classfile) to synthesize marker RuleId subclasses at runtime — pragmatic bridge between the 0.2.6 Class<? extends RuleId> API and rule-name lookup in the interpreter's parseRuleAt.

Breaking changes

None.