Skip to content

Source‐Map Performance Benchmark

Jump to bottom
luo jiyin edited this page Sep 7, 2026 · 3 revisions

Source-Map Performance Benchmark

A/B benchmark for measuring CPU time and memory across multiple source-map workloads. Used to validate the optimization round in PRs #108–#111.

Environment

Item Value
OS Ubuntu 24.04.2 LTS (x86_64)
CPU AMD Ryzen 7 8745HS w/ Radeon 780M
Node.js v24.15.0
pnpm 11.0.9
Date 2026-09-07

All measurements ran on the same machine in the same session. The before/after versions share the same benchmark script (from the after commit) to ensure identical workloads.

Commits

Commit PR Description
Before 3de49e7 — State before #108
After e6c9fdd #111 merged After #108 + #109 + #110 + #111

Optimizations Under Test

PR Change Primary effect
#108 Reuse resolved segment indices in range queries 4 → 2 binary searches per getSourceRange()
#109 Lazy-build lineStarts in parseMdWithSourceMap() Skip number[] allocation + Markdown scan for getRaw()-only callers
#110 Lazy-build sourceGapPrefix on first multi-segment getSourceRange() Skip number[] allocation for single-segment and non-query callers
#111 Eliminate temporary string/array reconstruction during validation No spread array, no rebuilt value string, no substring slices

Workloads

Workload Generator Validates
plain text 64/256 KiB Lines of ASCII text Baseline parse + query
text segments 256 KiB Alternating &\( (2 segments per 10 chars) Segment-heavy node — #108 binary search, #110 prefix allocation
fenced code 256 KiB / 1 MiB console.log(...) lines inside triple backticks Large code block — #111 temporary string elimination
inline code 64/256 KiB Repeated `&` spans Entity/escape segments — #111 spread array elimination
URLs x1000 Links with query params and entities URL mapping — #111 rebuilt value elimination
1k / 10k paragraphs Short text nodes separated by blank lines Many small nodes — #109/#110 per-node allocation pressure

Phases Measured

  • build: parseMdWithSourceMap(md) — full parse + source-map construction
  • getRaw: Call sourceMap.getRaw() on every AST node (after a separate parseMdWithSourceMap() call)
  • getSourceRange: Call sourceMap.getSourceRange(n, 0, n.value.length) on every text node (after a separate parseMdWithSourceMap() call)

Note: getRaw and getSourceRange are timed separately from build. The getRaw timing measures only the query, not the parse. To measure the end-to-end parse + getRaw cost (relevant for #109), see the dedicated rerun section below.

Memory Measurement

  • Peak RSS: Each workload was measured in a separate Node.js process via /usr/bin/time -v → Maximum resident set size (kbytes). The ab-benchmark.mjs script runs all workloads in one process, so its RSS column reflects the process-level maximum across all workloads, not per-workload RSS. The Peak RSS table below was produced by running each workload in isolation.
  • Heap delta: process.memoryUsage() with --expose-gc + manual gc() call before snapshot. This measures post-GC retained heap, not transient allocation volume. For #111's allocation elimination, Peak RSS is a more faithful signal than post-GC heap.

A/B Setup

# Create worktrees
git worktree add /tmp/parser-before 3de49e76044b
git worktree add /tmp/parser-after  e6c9fdd95df8

# Build both
cd /tmp/parser-before && pnpm install --frozen-lockfile && pnpm run build
cd /tmp/parser-after && pnpm install --frozen-lockfile && pnpm run build

# Run with the AFTER repo's benchmark script (same workload for both)
cd /tmp/parser-after
node --expose-gc scripts/ab-benchmark.mjs /tmp/parser-before/dist/lint-md-parser.cjs
node --expose-gc scripts/ab-benchmark.mjs /tmp/parser-after/dist/lint-md-parser.cjs

Always use the newer version's benchmark script to ensure identical workloads.

Results — CPU Query (bench-source-map.mjs, 256 KiB alternating segments, 3-sample median)

Metric Before After Delta
random range queries (10k iters) 5.50 ms 4.63 ms -16%
query full value range (10k iters) 1.10 ms 1.05 ms -5%
random single-unit queries (256K iters) 25.83 ms 31.18 ms noise *
query every code unit (256K iters) 10.79 ms 11.34 ms noise

* The initial single-pass A/B showed +21% for random single-unit. A 10-sample A/B rerun (alternating before/after, one process per sample) resolved this to -23.6% (before median 50.3 ms, after median 38.4 ms). The initial +21% was scheduling/JIT noise.

Results — Anomaly Reruns

Two data points from the initial A/B were flagged as anomalies. Both were rerun with 10 alternating samples (A B A B A B A B A B), one fresh Node.js process per sample, median of 5 per side.

fenced code 1 MiB build

Samples (ms) Median
Before 452.6, 416.3, 397.4, 217.7, 194.5 397.4 ms
After 243.6, 233.3, 773.8, 373.5, 399.5 373.5 ms

Delta: -6.0%. The initial +77% was scheduling noise (variance across samples is very high for 1 MiB workloads). Consistent with fenced code 256 KiB (-4%).

random single-unit queries 256 KiB

Samples (ms) Median
Before 56.1, 43.2, 53.6, 44.2, 50.3 50.3 ms
After 41.1, 33.0, 38.4, 40.0, 34.4 38.4 ms

Delta: -23.6%. The initial +21% was noise. The real result is a significant improvement, consistent with the other query benchmarks.

Results — CPU Build (ab-benchmark.mjs, 3-sample median)

Workload Before After Delta
plain text 64 KiB 56.6 ms 51.1 ms -10%
plain text 256 KiB 202.3 ms 207.3 ms +2% noise
text segments 256 KiB 305.3 ms 286.7 ms -6%
fenced code 256 KiB 42.1 ms 40.6 ms -4%
fenced code 1 MiB 218.7 ms 387.8 ms noise → -6% (rerun)
inline code 64 KiB 7.5 ms 6.8 ms -9%
inline code 256 KiB 28.1 ms 27.0 ms -4%
URLs x1000 40.7 ms 34.7 ms -15%
1k paragraphs 21.5 ms 20.4 ms -5%
10k paragraphs 372.8 ms 352.6 ms -5%

Results — getRaw (ab-benchmark.mjs, separate query timing)

Workload Before After Delta
1k paragraphs 0.3 ms 0.3 ms —
10k paragraphs 2.7 ms 1.8 ms -33% *

* This measures only the getRaw() call, not the preceding parseMdWithSourceMap(). #109 optimizes the parse path (skipping lineStarts construction), not the getRaw() function itself. The -33% is likely measurement variance from the separate parseMdWithSourceMap() call that precedes the timed section. To properly validate #109, measure end-to-end parse + getRaw:

benchCpu('parse+getRaw', () => {
  const { ast, sourceMap } = parseMdWithSourceMap(md);
  for (const node of allNodes(ast))
    try { sourceMap.getRaw(node); } catch {}
});

This was not included in the initial benchmark round. The build-time improvements (5–15% across workloads) provide indirect evidence that #109/#110 reduce construction overhead.

Results — Peak RSS (each workload in a separate process, /usr/bin/time -v)

Workload Before After Delta
10k paragraphs 83,828 KB (81.9 MB) 82,884 KB (80.9 MB) -1.1%
text segments 256 KiB 343,284 KB (335.2 MB) 349,608 KB (341.4 MB) +1.8% noise
fenced code 256 KiB 102,120 KB (99.7 MB) 101,696 KB (99.3 MB) -0.4%
fenced code 1 MiB 220,524 KB (215.4 MB) 218,304 KB (213.2 MB) -1.0%
inline code 256 KiB 97,984 KB (95.7 MB) 96,024 KB (93.8 MB) -2.0%
URLs x1000 108,784 KB (106.2 MB) 105,800 KB (103.3 MB) -2.7%

Note: Node.js has ~80 MB base RSS. The signal is in the delta. Peak RSS captures some transient allocation pressure that post-GC heap misses.

Results — Post-GC Heap Delta (process.memoryUsage with --expose-gc)

Workload Before After Delta
fenced code 256 KiB 1.6 MB 0.4 MB -75%
URLs x1000 13.0 MB 8.4 MB -35%

These measure post-GC retained heap, not transient allocation volume. #111 eliminates temporary strings/arrays that are discarded before GC runs. The improvement is real but this metric understates the allocation reduction. Peak RSS is a more faithful signal for allocation-sensitive optimizations.

Summary

Category Workload Validates Before After Delta
Query CPU random range 256 KiB #108 5.50 ms 4.63 ms -16%
Query CPU random single-unit 256 KiB (rerun) #108 50.3 ms 38.4 ms -24%
Build CPU URLs x1000 #111 40.7 ms 34.7 ms -15%
Build CPU plain text 64 KiB #109/#110 56.6 ms 51.1 ms -10%
Build CPU text segments 256 KiB #108/#110 305.3 ms 286.7 ms -6%
Build CPU fenced code 1 MiB (rerun) #111 397.4 ms 373.5 ms -6%
Build CPU 10k paragraphs #109/#110 372.8 ms 352.6 ms -5%
Peak RSS URLs x1000 #111 106.2 MB 103.3 MB -2.7%
Peak RSS inline code 256 KiB #111 95.7 MB 93.8 MB -2.0%
Peak RSS fenced code 1 MiB #111 215.4 MB 213.2 MB -1.0%
Regression — — — — None confirmed

Limitations

  1. Measurement noise: Even with 3-sample median and A/B alternation, single-machine benchmarks on shared hardware have inherent variance. The rerun section shows how initial anomalies resolved with more samples.

  2. GC timing: Post-GC heap delta depends on when V8 triggers GC. --expose-gc + manual gc() reduces variance but doesn't eliminate it. This metric measures retained heap, not transient allocation.

  3. Synthetic workloads: All inputs are algorithmically generated. Real-world Markdown may show different ratios.

  4. RSS includes Node.js overhead: Peak RSS includes ~80 MB Node.js runtime baseline. The signal is in the delta.

  5. getRaw timing: The getRaw benchmark measures only the query, not the parse. #109's optimization is in the parse path. A dedicated end-to-end parse + getRaw benchmark would be needed to directly measure #109's impact on this path.

  6. Single-machine results: All data from one AMD Ryzen 7 / Ubuntu 24.04 session. Different architectures (ARM, Intel) or OS may show different profiles.

Benchmark Script

See scripts/ab-benchmark.mjs in the repository for the full benchmark script.

Clone this wiki locally