Skip to content

Measurements

codingncaffeine edited this page Oct 5, 2026 · 95 revisions

Measurements

Public yardsticks, scored by Sashfold itself, enforced in CI.

Yardstick Score
html5lib tokenizer 7032 / 7032 (100%)
html5lib tree construction 1784 / 1784 (100%)
WPT URL parsing (urltestdata) 893 / 893 (100%)
Unicode bidi (BidiCharacterTest) 91,707 / 91,707 (100%)
Unicode bidi (BidiTest) 770,241 / 770,241 (100%)
Unicode line breaking (LineBreakTest) 16,672 / 16,672 (100%)
WPT CSS reference tests 10272 / 15682 (65.5%)
WPT testharness tests, by subtest 157846 / 192269 (82.1%) over 6,343 test files
test262, the ECMAScript conformance suite 41845 / 45367 (92.2%)
Acid1 Renders as its reference, font rasterization aside
The Sashfold 100 Rendered nightly by the Windows build and by the Linux build, published at sashfold.com/sashfold100 and sashfold100/linux

The WPT CSS reference tests

The Web Platform Tests' CSS reference tests are the tests that need no scripting: a test page and a reference page must render to the same pixels. Sashfold scores twenty-one directories (css/CSS2, css/css-backgrounds, css/css-box, css/css-cascade, css/css-color, css/css-display, css/css-flexbox, css/css-fonts, css/css-grid, css/css-overflow, css/css-position, css/css-syntax, css/css-text, css/css-values, css/selectors, css/css-variables, css/css-writing-modes, html/rendering, html/canvas/element, css/css-nesting, css/css-conditional) — every test and every reference rendered headless by Sashfold at the suite's 800×600 viewport, compared by the suite's own rules: pixel-identical unless the test declares a fuzziness allowance, references chained the way wptrunner chains them.

The WPT testharness tests

The suite's scripted tests: each file a page that loads testharness.js and asserts, subtest by subtest, what the DOM, the window and the event loop do. Sashfold scores fifty-six directories — dom/nodes, dom/events, dom/ranges, dom/traversal, html/browsers/the-window-object, html/browsers/windows, html/browsers/origin, html/semantics/embedded-content/the-iframe-element, html/semantics/embedded-content/the-object-element, html/semantics/embedded-content/the-embed-element, html/semantics/embedded-content/the-frame-element, html/webappapis/scripting, html/webappapis/structured-clone, html/webappapis/timers, html/webappapis/microtask-queuing, html/infrastructure, html/syntax/parsing, url, custom-elements, shadow-dom, dom/abort, dom/collections, dom/lists, css/cssom, html/dom, html/webappapis/dynamic-markup-insertion, css/css-conditional, css/css-fonts/parsing, css/css-display, IndexedDB, html/canvas/element, media-source, html/semantics/embedded-content/media-elements, html/semantics/embedded-content/the-video-element, html/semantics/embedded-content/the-audio-element, html/user-activation, picture-in-picture, media-capabilities, mediasession, storage, permissions, geolocation, mediacapture-streams, screen-wake-lock, credential-management, badging, web-share, gamepad, trusted-types, user-timing, performance-timeline, hr-time, css/css-nesting, css/css-cascade, streams, encoding/streams — every test loaded by Sashfold itself from the suite's own files, its scripts run and its event loop pumped until the harness reports; the verdicts are the harness's own. The unit counted is the subtest, as the suite's dashboards count them; a file whose harness never reports contributes none, and the table says how many did. Every host of the suite is served from the same checkout, nothing touches the network, and no server-side handler runs. Testing has the runner.

Nine of those directories were added deliberately in September 2026, after a score of 89.2% over the other eighteen turned out to measure what the engine had already built rather than what the web needs: the honest figure over the wider list began at 78.7%, and the difference is the work the earlier number could not see. The suite's interfaces/ tree — the Web IDL of every specification, which its own idlharness tests read — is fetched with it. Those idlharness tests did not run at all until the end of that month: the suite's server serves the IDL parser they load under a name of its own, and the runner looked for a file of that name. With the name answered, 8,179 more subtests run, and the score over the wider count went from 84.8% to 84.0% while 5,496 more subtests passed. encoding/ is deliberately left out: its legacy multi-byte tests generate about twenty-three thousand subtests each, one per code point, and they would outweigh everything else in a report that ranks causes by the subtests they block.

  • The per-directory table is at sashfold.com/wpt-harness.html, regenerated with every change.
  • The total grows as more of a file runs: the range tests stopped in their shared setup until CDATA sections and ranges were written, and with them the total went from 26,275 subtests to 74,294.
  • tests/wpt/passing.txt names every passing test; CI fails when a listed test stops passing.
  • The checkout is fetched at a pinned revision by tools/wpt-fetch.sh; nothing of it is vendored.
  • Tests that wait on scripts count as failures, as they do on wpt.fyi. What the engine cannot do, it does not score.

test262

test262 is the ECMAScript conformance suite. Sashfold scores every test under the directories tests/test262/directories.txt names — the language chapters, the built-in library from AggregateError to encodeURIComponent, and Annex B — 41,791 tests over 86 directories, run the way the suite's own instructions say: the harness first, then the test, once as sloppy code and once as strict unless the test says otherwise, each in a fresh realm on a sixty-second deadline.

  • The per-directory table is at sashfold.com/test262.html, regenerated with every change.
  • tests/test262/passing.txt names every passing test; CI fails when a listed test stops passing.
  • The checkout is fetched at a pinned revision by tools/test262-fetch.sh; nothing of it is vendored.
  • Two rules of our own: every test under a listed directory counts, modules and async included; and a test that expects a syntax error scores only when the engine raised a real one, never when it declined a feature it does not have. Most of what fails today is the regular-expression forms the engine declines with an error that names them (lookbehind, property escapes, the v flag: the RegExp directories), the newest import syntax, and an await or yield inside a class heritage or a computed key.

Acid1

The CSS1 box, float and clear test renders at its reference's width as the W3C's reference rendering shows it — the same boxes in the same places — with the differences a bitmap font of 1998 leaves against grayscale anti-aliasing today. The live shell script opens it and asserts its texts line by line.

Timings

sashfold --bench <page> times parse, stylesheets, style, layout and paint over several runs. On this project's reference machine, Wikipedia's "Web browser" article (415 KB, 14 stylesheets, some 900 rules, at 800 px) parses in about 8 ms, resolves its style in about 13 ms, lays out in a few milliseconds and paints its viewport in about one; the whole page, pictures included, arrives in about a second over 3 connections opened and 14 reused.

Clone this wiki locally