Networking

Networking

Every layer between a typed address and a page's bytes is written in the repository, behind one fetch() choke point that tools/egress-check.sh enforces in CI.

URLs

The WHATWG URL parser, at 893 / 893 (100%) on the Web Platform Tests' urltestdata: the state machine, every encode set, full IPv4 and IPv6, domain-to-ASCII through UTS #46 and NFC over generated Unicode tables, and Punycode in both directions.

HTTP/1.1

• Requests through one client: chunked transfer, Content-Length, read-to-close; content decoding with a full inflate (stored, fixed and dynamic Huffman; zlib and gzip framing) and a brotli decoder of our own (RFC 7932, the static dictionary and its word transforms included), both bomb-capped; redirects; body caps.
• Persistent connections. A connection whose response was delimited and not marked close goes back to the session pool under its scheme, host and port, and the next request to that origin skips the connect and the TLS handshake — 32 idle, 6 per origin, 60 seconds. Interim 1xx responses (103 Early Hints included) are stepped past; 204 and 304 read no body; a pooled connection found dead is retried once on a fresh one.
• Wikipedia's article page with its 14 stylesheets and pictures loads in about a second on 3 connections opened and 14 reused.
• A page's own requests. fetch() and XMLHttpRequest (Scripting) go through the same choke point with the page's session: the method, headers and body as the script set them (the exchange keeps Host, Content-Length, Connection, Accept-Encoding and Cookie its own), cookies only where the request allows them and never across sites, a 303 — or a 301 or 302 on a POST — turned into a GET with the body and its headers dropped, 307 and 308 kept as they were, a HEAD reading no body, and only a GET consulting or feeding the cache. The browser's CORS rules are applied above the choke point, in the bindings.
• What the page may ask for. Every request a page makes carries the page's say on it into the choke point: the session's content-blocking lists and the document's Content Security Policy (src/net/Csp, see Security defaults) judge the URL before any connection is made, and judge every redirect hop again before it is followed — a policy's source paths set aside on a hop, as the specification has it — so a refused request, or a refused hop, costs no connection and no request. A frame's document is asked for under the page's frame-src, and what that document asks for under its own policy.

Fetching ahead, together

The document itself comes on another thread. A load of a page of the web is begun and the window is the reader's again at once: the tab goes on showing what it showed, and can be read, scrolled and clicked, the other tabs switched to, the address bar typed in; a tab a session restored is switched to at once and its page arrives after. When the document has come, the stylesheets and scripts its markup names are asked for together (below) and waited for a turn of the window's loop at a time — four seconds at the most — and only then is the page put into the tab, parsed and drawn, so that the parser finds what it would have waited for already there. A load on its way gives way to the next one asked for in its tab, and to Back and Forward; the HTTPS-first retry, a download and a theme are taken up where the document's arrival is. The window's loop sleeps on its events meanwhile and looks again every few milliseconds (Browser::load_ready says whether there is a step to do now; has_pending_load whether a load is under way at all, which is what a caller without a window ticks by).

A page names dozens of stylesheets, scripts, fonts and pictures, and fetched one after another each waits on the network while the rest wait on it. They are asked for ahead of the page's own asking, together, on the threads of a small pool (net::FetchPool, eight at most, none started until there is work):

• Before the parse, a light pass over the document's bytes (html::scan_for_preloads, every browser's "preload scanner") lists the stylesheets and scripts its markup names — tags and attributes only, comments and the text of script, style, noscript and template passed over, a <base href> honoured; an alternate, print or disabled sheet, a script that is data, a nomodule fallback are left out. It names only what it is sure of: what it misses is fetched later as it always was.
• Once the tree stands, the pictures — the same walk that chooses each <img>'s source from srcset, sizes and <picture>, run once with nothing fetched — and the backgrounds the styles name; and once the sheets are in, the fonts, each face's first readable source.
• Pictures are not waited for at all. A pass over the page's pictures takes those that have come and leaves the ones still on their way (ImagePass::arrived, Loader::ahead_pending): the page shows at once, and the pictures are taken up as they arrive — a batch at most every tenth of a second, since each pass lays the page out again, and a pass that found nothing new lays nothing out. A caller without a window (--render, --bench) still ticks until they are all in.
• The page's own request, which follows, claims what came or waits for what is on its way (ShellLoader::prefetch / load_subresource): one exchange per address, and the wait is for the slowest of them, not their sum.

Nothing is asked for ahead that would not be asked for anyway, and the rules hold: the block lists judge every such request and each redirect hop of it before anything leaves; a page with a Content Security Policy has each address checked quietly against it first (allows_quietly — a policy that leans on a nonce or on 'strict-dynamic' answers no, which only means "not early"), a document that states a policy in a <meta> is left alone altogether, and where a redirect followed ahead ended up is judged by the page's guard when the answer is claimed. Only pages of the web ask ahead; a local page and the shell's scripts fetch as they always did.

The session's cache, connection pool and cookie jars serve those threads and the window's alike: each keeps a lock of its own, and the cache hands out copies made under it. Still waited for where it is asked: an @import inside a sheet, a frame's document, what a script inserts or asks for with fetch()/XMLHttpRequest, and the pages the shell makes out of a fetch of its own (view-source:, the reader, the themes gallery).

TLS

• Windows: SChannel, with real certificate validation and revocation checking below the root (soft-fail when the revocation server is unreachable). The badssl.com matrix — expired, self-signed, wrong host, untrusted root, revoked — is refused, and a certificate that fails lands on a page with no way through.
• Linux: a TLS client of Sashfold's own (Linux has no OS TLS to lean on), speaking TLS 1.3 and, for a server that has nothing newer, TLS 1.2 negotiated in the same hello: ChaCha20-Poly1305 and AES-128-GCM, the groups x25519 and P-256 (a 1.3 server that wants P-256 asks for another hello and gets it), in 1.2 the ECDHE suites over those two AEADs under RSA or ECDSA with the extended master secret, and never renegotiation; certificates validated against the system CA bundle with signatures, validity, name constraints and hostname matching, and the leaf checked against its issuer's revocation list — fetched over plain HTTP from the certificate's own distribution point through the same HTTP client, with a sharded list's issuing distribution point honoured, each list held until its next update, and a point that cannot be reached a soft pass, as the OS validators treat it; the badssl.com matrix — expired, self-signed, wrong host, untrusted root, revoked — is refused. A server's hello is held to RFC 8446's extension rules: an extension twice in a block, or one the client recognizes but that has no place in the message it came in, is illegal_parameter; one never offered is unsupported_extension; and a plain hello answering a share-less client hello with a key share, where only a retry request may, is refused. A second connection to a server resumes the first: in 1.3 the tickets a server issues are kept per host and port, each offered once as a pre-shared key with its binder beside a fresh key share, so the resumed handshake carries no certificate and is one flight from the server; in 1.2 the session is resumed by the ticket the server issued (RFC 5077) or, where it issued none, by the session id it chose, under the same suite and the same extended-master-secret binding, and the short handshake has the server finishing first. A server that declines gets a full handshake; a ticket is dropped past its lifetime and never kept past seven days, a 1.2 session past a day. SASHFOLD_TLS_INSECURE=1 accepts any certificate and says so loudly, SASHFOLD_TLS_SUITE and SASHFOLD_TLS_VERSION narrow the hello to one suite or one version, SASHFOLD_TLS_RESUME=0 offers nothing to resume. A connection gives each address of a host five seconds to answer before the next is tried, so an IPv6 address on a machine with no IPv6 route costs seconds, not minutes.
• macOS: not written yet (Network.framework).

Cookies

An RFC 6265 cookie jar with third-party cookies blocked by default: a request a page makes to another site carries none of that site's cookies and stores none. The date parser doubles as the HTTP date parser.

The cache

An RFC 9111 cache. A stored response is served while it is fresh: by its max-age (less Age) or Expires, else — for a 200 that carries a Last-Modified and says nothing else — for a tenth of the time since it was last modified, a week at most. A stale copy that carries an ETag or Last-Modified is revalidated: the request goes out with If-None-Match or If-Modified-Since, and a 304 renews the stored copy in place with the headers that came back, its body never sent again; a 200 replaces it. no-store is honoured absolutely, no-cache stores but never serves without asking the origin, private stores (this is a private cache); Vary is honored for Accept, Accept-Encoding and User-Agent and refuses anything else; Set-Cookie is never replayed from a hit. Lookup happens per hop of a redirect chain. The entries live in memory (8 MB per entry, 64 MB in all) and, in the window, in the profile's cache/ folder as well — a file of headers and a file of body per entry, read back at the next start with the bodies loaded as they are asked for, 256 MB in all, the oldest leaving first.

Other schemes

data: URLs (with a forgiving base64), about:blank, about:sashfold, view-source:, reader:, and file: for local pages (a file: page may load file: subresources; a web page may not).

Downloads

A response the engine cannot render — a type it does not show, or a Content-Disposition: attachment — is saved under a fresh, sanitized name in the Downloads folder, marked with the mark of the web (the NTFS Zone.Identifier stream carrying the source and referrer URLs), and never opened.

Policies

• HTTPS-first: a typed host is tried over https; http is tried once, only when the connection itself fails, never after a certificate error.
• Referrer: strict-origin-when-cross-origin — a cross-site request tells the other site the origin, never the full URL.

The user agent string

Sashfold sends Mozilla/5.0 (<platform>) Sashfold/<version> — for example Mozilla/5.0 (Windows NT 10.0; Win64; x64) Sashfold/0.0.5. The Mozilla/5.0 prefix and the platform token are the compatibility-shaped form every engine ships, because servers still key on them; the honest part is the Sashfold/<version> suffix, the only thing that identifies the browser, and the version is the release's own. The string carries no build number, locale or hardware detail, and it is the same for every user of a given platform.