-
Notifications
You must be signed in to change notification settings - Fork 395
Add support for compression dictionary transport #1854
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
68a2aa5
5d5bc57
5dff3fd
138062a
ac0bac2
91bf967
1628af9
eab37fc
b6aebd1
0c50c9d
be13a7f
90cf650
c0df025
288ed46
8f3027c
9791e82
31d5f7d
ba369f6
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -16,12 +16,19 @@ Translate IDs: typedefdef-bodyinit bodyinit,dictdef-requestinit requestinit,type | |
| urlPrefix:https://httpwg.org/specs/rfc5861.html#;type:dfn;spec:stale-while-revalidate | ||
| url:n-the-stale-while-revalidate-cache-control-extension;text:stale-while-revalidate lifetime | ||
|
|
||
| urlPrefix:https://httpwg.org/specs/rfc9842.html#;type:dfn;spec:rfc9842 | ||
| url:use-as-dictionary;text:Use-As-Dictionary | ||
| url:available-dictionary;text:Available-Dictionary | ||
| url:dictionary-id;text:Dictionary-ID | ||
| url:rfc.section.2.2;text:finding the best matching dictionary | ||
|
|
||
| urlPrefix:https://httpwg.org/specs/rfc9651.html#;type:dfn;spec:rfc9651 | ||
| url:rfc.section.2;text:structured field value | ||
| url:text-serialize;text:serializing structured fields | ||
| url:text-parse;text:parsing structured fields | ||
| url:;text:structured header | ||
| url:token;text:structured field token | ||
| url:parse-bare-item;text:bare item | ||
|
|
||
| urlPrefix:https://httpwg.org/specs/rfc9110.html#;type:dfn;spec:http | ||
| url:method.overview;text:method | ||
|
|
@@ -67,6 +74,11 @@ urlPrefix:https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-layered-cooki | |
| url:name-serialize-cookies;text:serialize cookies | ||
| url:name-garbage-collect-cookies;text:garbage collect cookies | ||
|
|
||
| urlPrefix:https://urlpattern.spec.whatwg.org/#;type:dfn;spec:urlpattern | ||
| url:url-pattern-create;text:creating a URL pattern | ||
| url:url-pattern-has-regexp-groups;text:has regexp groups | ||
|
|
||
|
|
||
| urlPrefix:https://www.rfc-editor.org/rfc/rfc6454;type:dfn;spec:RFC6454 | ||
| url:section-7.1;text:serialized-origin | ||
| </pre> | ||
|
|
@@ -1822,6 +1834,7 @@ device to assist defining CSP and Mixed Content. It is not exposed to JavaScript | |
| the empty string, | ||
| "<code>audio</code>", | ||
| "<code>audioworklet</code>", | ||
| "<code>compression-dictionary</code>", | ||
| "<code>document</code>", | ||
| "<code>embed</code>", | ||
| "<code>font</code>", | ||
|
|
@@ -1878,7 +1891,7 @@ not always relevant and might require different behavior. | |
| <th>CSP directive | ||
| <th>Features | ||
| <tr> | ||
| <td rowspan=22>"" | ||
| <td rowspan=23>"" | ||
| <td>"<code>report</code>" | ||
| <td rowspan=2>— | ||
| <td>CSP, NEL reports. | ||
|
|
@@ -1970,6 +1983,10 @@ not always relevant and might require different behavior. | |
| <td>"<code>video</code>" | ||
| <td><code>media-src</code> | ||
| <td>HTML's <code><video></code> element | ||
| <tr> | ||
| <td>"<code>compression-dictionary</code>" | ||
| <td><code>connect-src</code> | ||
| <td>HTML's <code><link rel=compression-dictionary></code> | ||
| <tr> | ||
| <td>"<code>download</code>" | ||
| <td>"" | ||
|
|
@@ -2326,9 +2343,10 @@ When a request is [=request/cloned=], the created request gets a unique | |
|
|
||
| <p>A <dfn export>subresource request</dfn> is a <a for=/>request</a> | ||
| whose <a for=request>destination</a> is "<code>audio</code>", "<code>audioworklet</code>", | ||
| "<code>font</code>", "<code>image</code>", "<code>json</code>", "<code>manifest</code>", | ||
| "<code>paintworklet</code>", "<code>script</code>", "<code>style</code>", "<code>text</code>", | ||
| "<code>track</code>", "<code>video</code>", "<code>xslt</code>", or the empty string. | ||
| "<code>compression-dictionary</code>", "<code>font</code>", "<code>image</code>", | ||
| "<code>json</code>", "<code>manifest</code>", "<code>paintworklet</code>", "<code>script</code>", | ||
| "<code>style</code>", "<code>text</code>", "<code>track</code>", "<code>video</code>", | ||
| "<code>xslt</code>", or the empty string. | ||
|
|
||
| <p>A <dfn export>non-subresource request</dfn> is a <a for=/>request</a> | ||
| whose <a for=request>destination</a> is "<code>document</code>", "<code>embed</code>", | ||
|
|
@@ -3326,6 +3344,22 @@ or an <a>implementation-defined</a> value. | |
| </div> | ||
|
|
||
|
|
||
| <h3 id=compression-dictionary-cache-partitions>Compression-dictionary cache partitions</h3> | ||
|
|
||
| <div algorithm> | ||
| <p>To <dfn>determine the compression-dictionary cache partition</dfn>, given a <a for=/>request</a> <var>request</var>: | ||
|
|
||
| <ol> | ||
| <li><p>Let <var>key</var> be the result of <a for=request>determining the network partition key</a> | ||
| given <var>request</var>. | ||
|
|
||
| <li><p>If <var>key</var> is null, then return null. | ||
|
|
||
| <li><p>Return the unique compression-dictionary cache associated with <var>key</var>. [[!RFC9842]] | ||
| </ol> | ||
| </div> | ||
|
|
||
|
|
||
| <h3 id=port-blocking>Port blocking</h3> | ||
|
|
||
| <p class=note>New protocols can avoid the need for blocking ports by negotiating the protocol | ||
|
|
@@ -6435,8 +6469,9 @@ run these steps: | |
| <li><p>If <var>httpRequest</var>'s <a for=request>cache mode</a> is | ||
| "<code>only-if-cached</code>", then return a <a>network error</a>. | ||
|
|
||
| <li><p>Let <var>forwardResponse</var> be the result of running <a>HTTP-network fetch</a> given | ||
| <var>httpFetchParams</var>, <var>includeCredentials</var>, and <var>isNewConnectionFetch</var>. | ||
| <li><p>Let <var>forwardResponse</var> be the result of running | ||
| <a>HTTP-network compression-dictionary fetch</a> given <var>httpFetchParams</var>, | ||
| <var>includeCredentials</var>, and <var>isNewConnectionFetch</var>. | ||
|
|
||
| <li><p>If <var>httpRequest</var>'s <a for=request>method</a> is <a>unsafe</a> and | ||
| <var>forwardResponse</var>'s <a for=response>status</a> is in the range 200 to 399, inclusive, | ||
|
|
@@ -6585,12 +6620,148 @@ run these steps: | |
| <li><p>If <var>isAuthenticationFetch</var> is true, then create an <a>authentication entry</a> for | ||
| <var>request</var> and the given realm. | ||
|
|
||
| <li> | ||
| <p>If <var>request</var>'s <a for=request>response tainting</a> is not "<code>opaque</code>" | ||
| and <var>response</var>'s <a for=response>header list</a> <a for="header list">contains</a> | ||
| <a><code>Use-As-Dictionary</code></a>: | ||
| <!-- This is defined in [[!RFC9842]] --> | ||
|
|
||
| <ol> | ||
| <li><p>Let <var>dictionaryValue</var> be the result of | ||
| <a for="header list">getting a structured field value</a> given <a><code>Use-As-Dictionary</code></a>, | ||
| "<code>dictionary</code>", and <var>response</var>'s <a for=response>header list</a>. | ||
|
|
||
| <li><p>If <var>dictionaryValue</var> is null or <var>dictionaryValue</var>["<code>match</code>"] | ||
| does not <a for=map>exist</a>, then return <var>response</var>. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What about other parameters? For example https://www.rfc-editor.org/rfc/rfc9842#name-type says a client should not deal with a For For
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added details on how to handle invalid values for the other parameters. I don't explicitly call out that unknown keys should be ignored but I can add an explicit step to delete unknown keys if you think it would help. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah, I was not sure what is the common way to handle this, however https://httpwg.org/specs/rfc9651.html#preserving-extensibility mentions this:
So maybe the spec should be explicit about ignoring unknown keys. Incidentally, https://httpwg.org/specs/rfc9651.html#error-handling says ignoring the entire field is the default behavior when field-specific constraints are violated so it's indeed good the spec is now explicit for id, type, etc that we only ignore the corresponding key. (note: probably rfc9651 should also have similar wording) |
||
|
|
||
| <li><p>If <var>dictionaryValue</var>["<code>type</code>"] <a for=map>exists</a> and its | ||
| <a>bare item</a> is not an <a>implementation-defined</a> supported dictionary type, then return | ||
| <var>response</var>. | ||
|
|
||
| <li><p>If <var>dictionaryValue</var>["<code>id</code>"] <a for=map>exists</a> and its | ||
| <a>bare item</a>'s <a for=string>length</a> is greater than 1024, then <a for=map>remove</a> | ||
| <var>dictionaryValue</var>["<code>id</code>"]. | ||
|
|
||
| <li> | ||
| <p>If <var>dictionaryValue</var>["<code>match-dest</code>"] <a for=map>exists</a>: | ||
|
|
||
| <ol> | ||
| <li><p>Let <var>matchDestList</var> be <var>dictionaryValue</var>["<code>match-dest</code>"][0]. | ||
|
|
||
| <li><p>For each <var>dest</var> of <var>matchDestList</var>: if <var>dest</var>'s <a>bare item</a> | ||
| is not a <a for=request>destination</a> supported by the user agent, then remove <var>dest</var> | ||
| from <var>matchDestList</var>. | ||
|
|
||
| <li><p>If <var>matchDestList</var> is empty, then return <var>response</var>. | ||
| </ol> | ||
|
|
||
| <li><p>Let <var>compressionDictionaryCache</var> be the result of | ||
| <a>determining the compression-dictionary cache partition</a> given <var>request</var>. | ||
|
|
||
| <li><p>If <var>compressionDictionaryCache</var> is null, then return <var>response</var>. | ||
|
pmeenan marked this conversation as resolved.
|
||
|
|
||
| <li><p>Let <var>pattern</var> be the result of | ||
| <a for=/>creating a URL pattern</a> given the <a>bare item</a> of <var>dictionaryValue</var>["<code>match</code>"], | ||
| the <a lt="URL serializer">serialization</a> of <var>request</var>'s <a for=request>current URL</a>, | ||
| and an empty map. If this throws an exception, then return <var>response</var>. | ||
|
|
||
| <li><p>If <var>pattern</var> is failure or <var>pattern</var> <a for=/>has regexp groups</a>, | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "creating a URL pattern" uses algos from https://url.spec.whatwg.org/ that can return failure, but when that happens it actually throw an exception, rather than returning failure? I wanted to ask about these potential exceptions the other day, but couldn't find an obvious input here that would make the algo throw... Anyway, I find this spec indeed deals with the case when an exception is thrown so I guess we probably want the same here: https://wicg.github.io/connection-allowlists/#abstract-opdef-parse-a-connection-allowlist-header
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It also seems we lack tests for this. We should have somewhat exhaustive tests for error conditions.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks. I added similar language that it should return response if the URL Pattern creation throws an exception. I'll add WPT tests for that and the rest of the edge cases that aren't currently covered now (will take a few days to work their way through).
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I just landed a (hopefully comprehensive) set of WPT tests to cover all of the edge cases I could think of: web-platform-tests/wpt#60164
|
||
| then return <var>response</var>. | ||
|
|
||
| <li><p>Let <var>expirationTime</var> be the time at which the <var>response</var> becomes | ||
| a <a>stale response</a>. | ||
|
|
||
| <li><p>If <var>expirationTime</var> is not in the future, then return <var>response</var>. | ||
|
|
||
| <li><p>Store <var>response</var> in <var>compressionDictionaryCache</var> with its associated | ||
| <var>pattern</var>, <var>dictionaryValue</var>, and <var>expirationTime</var>. | ||
| </ol> | ||
|
|
||
| <li><p>Return <var>response</var>. <span class=note>Typically <var>response</var>'s | ||
| <a for=response>body</a>'s <a for=body>stream</a> is still being enqueued to after | ||
| returning.</span> | ||
| </ol> | ||
| </div> | ||
|
|
||
| <h3 id=http-network-compression-dictionary-fetch>HTTP-network compression-dictionary fetch</h3> | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I wonder if here and below we also want a hyphen between network and compression. cc @sideshowbarker
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. FWIW, I was copying how HTTP-network fetch is handled which has a space in the string but hyphens in the ID ( |
||
|
|
||
| <div algorithm> | ||
| <p>To <dfn id=concept-http-network-compression-dictionary-fetch>HTTP-network compression-dictionary fetch</dfn>, | ||
| given a <a for=/>fetch params</a> <var>fetchParams</var>, an optional boolean | ||
| <var>includeCredentials</var> (default false), and an optional boolean <var>forceNewConnection</var> | ||
| (default false): | ||
|
|
||
| <ol> | ||
| <li><p>Let <var>request</var> be <var>fetchParams</var>'s <a for="fetch params">request</a>. | ||
|
|
||
| <li> | ||
| <p>Let <var>fallback</var> be the following steps: | ||
|
|
||
| <ol> | ||
| <li><p>Return the result of running <a>HTTP-network fetch</a> given <var>fetchParams</var>, | ||
| <var>includeCredentials</var>, and <var>forceNewConnection</var>. | ||
| </ol> | ||
|
|
||
| <li><p>If <var>request</var>'s <a for=request>mode</a> is "<code>no-cors</code>", then return the | ||
| result of running <var>fallback</var>. | ||
|
|
||
| <li><p>If the user agent is configured to block cookies for <var>request</var>, then return the | ||
| result of running <var>fallback</var>. | ||
|
|
||
| <li><p>Let <var>compressionDictionaryCache</var> be the result of | ||
| <a>determining the compression-dictionary cache partition</a> given <var>request</var>. | ||
|
|
||
| <li><p>If <var>compressionDictionaryCache</var> is null, then return the result of running | ||
| <var>fallback</var>. | ||
|
|
||
| <li><p>Let <var>bestMatch</var> be the result of <a>finding the best matching dictionary</a> in | ||
| <var>compressionDictionaryCache</var> for <var>request</var>. | ||
|
|
||
| <li><p>If <var>bestMatch</var> is null, then return the result of running <var>fallback</var>. | ||
|
|
||
| <li><p>Add the <a><code>Available-Dictionary</code></a> and <a><code>Dictionary-ID</code></a> | ||
| (if applicable) headers to <var>request</var> using <var>bestMatch</var>. | ||
|
|
||
| <li><p><a for="header list">append</a> (`<code>Accept-Encoding</code>`, `<code>dcb</code>`) | ||
| to <var>request</var>'s <a for=request>header list</a>. | ||
|
|
||
| <li><p><a for="header list">append</a> (`<code>Accept-Encoding</code>`, `<code>dcz</code>`) | ||
| to <var>request</var>'s <a for=request>header list</a>. | ||
|
|
||
| <li><p>Let <var>response</var> be the result of running <var>fallback</var>. | ||
|
|
||
| <li><p>Let <var>codings</var> be the result of <a>extracting header list values</a> given | ||
| `<code>Content-Encoding</code>` and <var>response</var>'s <a for=response>header list</a>. | ||
|
|
||
| <li><p>If <var>codings</var> is null or does not contain `<code>dcb</code>` or `<code>dcz</code>`, | ||
| then return <var>response</var>. | ||
|
|
||
| <li><p>If <var>request</var>'s <a for=request>response tainting</a> is "<code>opaque</code>", | ||
| then return a <a>network error</a>. | ||
|
|
||
| <li><p>Let <var>availableDictionaryItem</var> be the result of | ||
| <a for="header list">getting a structured field value</a> given <a><code>Available-Dictionary</code></a>, | ||
| "<code>item</code>", and <var>request</var>'s <a for=request>header list</a>. | ||
|
|
||
| <li><p>If <var>availableDictionaryItem</var> is null, then return a <a>network error</a>. | ||
|
|
||
| <li><p>Let <var>availableDictionaryHash</var> be the <a>bare item</a> of <var>availableDictionaryItem</var>. | ||
|
|
||
| <li><p>Let <var>newBody</var> be a new <a for=/>body</a> whose <a for=body>stream</a> is the | ||
| result of transforming <var>response</var>'s <a for=response>body</a>'s <a for=body>stream</a> | ||
| with an algorithm that verifies that the dictionary hash in the stream matches | ||
| <var>availableDictionaryHash</var> and decodes the rest of the stream with the applicable | ||
| algorithm as defined in [[!RFC9842]]. If verification or decoding fails, | ||
| error the transformed stream. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure what's implied by this "error the transformed stream". Shouldn't we return a network error if decoding/verification fails? If not what do we expect for newBody in that case? Also, about the hash verification, I don't see a lot from RFC 9842... Looking for "hash" or "available-dictionary", I find these paragraphs:
and in particular "The dictionary is validated using an SHA-256 hash of the content to make sure that the client and server are both using the same dictionary." from which one can infer the server is expected to send back the same hash if it has found the same on its side. So can we actually just add a previous step that checks
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I'm happy to change it to be a network error of some kind if there's a sensible way to plumb that. Where it gets a bit complicated is that it's a problem with the stream, not necessarily the HTTP-level response container and is closer to being a corrupt payload (like sending a brotli payload with That feels like it's a problem at the stream level rather than the network level but I'm happy to plumb it however it best fits into the spec. |
||
|
|
||
| <li><p>Set <var>response</var>'s <a for=response>body</a> to <var>newBody</var>. | ||
|
|
||
| <li><p><a for="header list">delete</a> `<code>Content-Encoding</code>` from <var>response</var>'s | ||
| <a for=response>header list</a>. | ||
|
|
||
| <li><p>Return <var>response</var>. | ||
| </ol> | ||
| </div> | ||
|
|
||
| <h3 id=http-network-fetch>HTTP-network fetch</h3> | ||
|
|
||
|
|
@@ -6662,6 +6833,7 @@ optional boolean <var>forceNewConnection</var> (default false), run these steps: | |
|
|
||
| <ul> | ||
| <li><p>Follow the relevant requirements from HTTP. [[!HTTP]] [[!HTTP-CACHING]] | ||
| [[!RFC9842]] | ||
|
|
||
| <li> | ||
| <p>If <var>request</var>'s <a for=request>body</a> is non-null, and <var>request</var>'s | ||
|
|
@@ -8527,7 +8699,7 @@ dictionary RequestInit { | |
| any window; // can only be set to null | ||
| }; | ||
|
|
||
| enum RequestDestination { "", "audio", "audioworklet", "document", "embed", "font", "frame", "iframe", "image", "json", "manifest", "object", "paintworklet", "report", "script", "sharedworker", "style", "text", "track", "video", "worker", "xslt" }; | ||
| enum RequestDestination { "", "audio", "audioworklet", "compression-dictionary", "document", "embed", "font", "frame", "iframe", "image", "json", "manifest", "object", "paintworklet", "report", "script", "sharedworker", "style", "text", "track", "video", "worker", "xslt" }; | ||
| enum RequestMode { "navigate", "same-origin", "no-cors", "cors" }; | ||
| enum RequestCredentials { "omit", "same-origin", "include" }; | ||
| enum RequestCache { "default", "no-store", "reload", "no-cache", "force-cache", "only-if-cached" }; | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should make compression-dictionary cache a link.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This mirrors the section above on HTTP cache partitions where it does "Return the unique HTTP cache associated with...". The RFC itself doesn't go into detail about the cache itself and how it operates. Is that something that needs to be fully defined somewhere?