Parsing

Parsing

HTML

The WHATWG HTML parser, complete:

• Tokenizer — every state, character references from generated entity tables, error recovery. 7032 / 7032 (100%) on the html5lib tokenizer suite.
• Tree construction — the full insertion-mode machine, the adoption agency, foster parenting, templates, fragment parsing, SVG and MathML foreign content, and the customizable <select> parser with <selectedcontent> cloning. 1784 / 1784 (100%) on the html5lib tree-construction suite.
• Quirks — the document's quirks, limited-quirks and no-quirks mode is set from the doctype.

Both scores are enforced in CI by baselines that only ratchet upward, and both halves of the parser have fuzz harnesses (see Testing).

Character encodings

Sniffed the way the specification says: a byte order mark first, then the <meta> prescan, then the windows-1252 fallback. The decoders — UTF-8, UTF-16LE, UTF-16BE, windows-1252 — are written in the repository.

The DOM

A node soup owned by the Document: every node is a unique_ptr held by its parent, with parent pointers back. Elements keep their namespace, local name and attributes; text nodes keep their data. The parser keeps an explicit insertion point.

Seeing the tree

sashfold --dump-dom page.html

prints the document tree of any file. The WHATWG specification itself (787 KB) parses to a 48,000-line tree in about 50 ms.

CSS

The CSS tokenizer and parser live with the rest of the style system — see Styling.