Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/web-surface.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,8 @@ A request target must be a plain path. One naming its own authority — `http://

Addresses are matched whole, not by prefix arithmetic. `/api/v1` is proven to be the prefix before anything beneath it is read, so a lookalike of the same length — `/abcdef/…` — reaches nothing. Empty path components are kept rather than filtered away, so a doubled or trailing slash is a different address than the one documented here rather than an alias for it.

**The address routed is the one the client sent.** A leading doubled slash is the case where that has to be said, because two libraries would otherwise answer it for us. Python's `BaseHTTPRequestHandler` rewrites a leading `//…` to `/…` before a handler runs — [gh-87389](https://github.com/python/cpython/issues/87389), which stops a client reading a *redirect* to `//host/path` as an absolute URI — and `urlsplit` reads `//api/v1/session` as the authority `api` and the path `/v1/session`, since those are a URI reference's rules and not a request target's. Either one, taken at face value, would make `//api/v1/opportunities/{id}/status` a second spelling of the one command address, and the rule above would never see the spelling it refuses. So the target is read from the raw request line and split on the first `?` here, rather than accepted from `self.path` or handed to a URI parser: in a request line `//api/v1/session` is an ordinary absolute path whose first component is empty, it is beneath no root, and it reaches nothing — `404` for a read, `405` for a `POST`, exactly as the other empty-component spellings do.

Every response carries:

```
Expand Down
32 changes: 32 additions & 0 deletions src/system/tests/test_web_frontend.py
Original file line number Diff line number Diff line change
Expand Up @@ -855,3 +855,35 @@ def test_no_shipped_asset_carries_anything_that_looks_like_a_credential():
if path.name == "screens.js" and named == TOKEN_VARIABLE:
continue
assert named not in body, f"{path.name} names {named}"


def test_the_packet_names_a_destination_rather_than_publishing_a_missing_one():
"""Two fields that are null until a decision exists, never concatenated into the page.

`action.provider` and `action.provider_namespace` are what a *recorded* decision bound.
Before one exists the API reports both as null -- correctly, because nothing is bound --
and `null + " . " + null` is the string "null . null", published in the one pane whose
stated job is to say what an approval would bind, directly under a note promising the
destination is named there and beside a button that names it in its own label.

So the recorded pair is read only when there is a decision to read it from, and an
undecided packet names the launch destination the session reported instead. Neither is
computed here: both arrive from the server, which is the rule this file exists to keep.
"""
screens = code(STATIC / "screens.js")
detail = region(screens, "function opportunityDetail(", "\nfunction digests(")
guarded = (
"const destination = record.action.provider\n"
' ? record.action.provider + " · " + record.action.provider_namespace\n'
" : target\n"
' ? target.provider + " · " + target.provider_namespace\n'
' : "—";'
)
assert guarded in detail, "the destination row is no longer guarded by a recorded decision"
assert '["destination", destination]' in detail
# The unguarded form, which is what published "null · null". Named here so that
# reintroducing it fails rather than merely looking different from the block above.
unguarded = '["destination", record.action.provider + '
assert unguarded not in screens, "the destination row reads the pair without a guard"
# The launch destination is the session's, not a value assembled in the browser.
assert "const target = extra.target;" in detail
51 changes: 51 additions & 0 deletions src/system/tests/test_web_surface.py
Original file line number Diff line number Diff line change
Expand Up @@ -530,6 +530,49 @@ def test_the_command_address_is_singular_rather_than_a_family_of_aliases(
)


def test_a_leading_empty_component_is_not_a_spelling_of_the_command_address(
client, repository, opportunity
):
"""The one alias a handler inherits rather than writes.

`BaseHTTPRequestHandler.parse_request` rewrites a leading `//...` to `/...` before any
handler runs, so a surface that routes `self.path` answers `//api/v1/...` at every one
of its addresses while believing it answers none of them -- and the test above passes
throughout, because every spelling it sends is one the rewrite leaves alone. This is
that spelling, at the address where it would matter most.

The body is withheld: the refusal is settled on the request line, so the bytes would go
unread, and closing on unread bytes is a reset rather than a clean shutdown on Windows.
What is asserted is the same either way -- the command address is reached by one
spelling, and the database is untouched by the others.
"""
event = repository.opportunity(opportunity)["status_event"]
payload = json.dumps({"status": "interested", "expected_event_id": event}).encode("utf-8")
before = state(repository)
for path in (
f"//api/v1/opportunities/{opportunity}/status",
f"///api/v1/opportunities/{opportunity}/status",
f"////api/v1/opportunities/{opportunity}/status",
):
status, body, headers = client.send(
path,
method="POST",
origin=True,
content_type="application/json",
body=payload,
transmit=False,
)
assert status == 405, (path, status)
assert json.loads(body) == {"error": "No command at this address"}
assert headers["Allow"] == web.READ_METHODS
assert state(repository) == before
# And the one spelling the contract names still reaches the command, so this narrowed
# nothing it should not have.
assert (
client.command(opportunity, {"status": "interested", "expected_event_id": event})[0] == 200
)


def test_a_target_naming_its_own_authority_is_refused(client, surface, repository, opportunity):
"""Identity is settled on Host, so a target may not name an authority of its own.

Expand Down Expand Up @@ -562,9 +605,17 @@ def test_a_read_address_is_singular_too(client, repository, opportunity):
"/api/v1/session/",
"/api/v1//opportunities",
f"/api/v1/opportunities/{opportunity}/sources/",
# A leading empty component is the spelling the stdlib rewrites before a handler
# sees it, which would answer every read at a second address. The page itself is
# here for the same reason: `//` is not `/`, and the reads are not the only thing
# that rewrite would have aliased.
"//api/v1/session",
"///api/v1/session",
"//api/v1/opportunities",
):
status, _, _ = client.send(path)
assert status == 404, path
assert client.send("//", token=False)[0] == 404
assert client.send("/api/v1/session")[0] == 200


Expand Down
79 changes: 56 additions & 23 deletions src/system/web/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@
from http import HTTPStatus
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from importlib import resources
from urllib.parse import parse_qs, unquote, urlsplit
from urllib.parse import parse_qs, unquote

from communications.gmail import TOKEN_VARIABLE
from communications.gmail_draft import COMPOSE_TOKEN_VARIABLE
Expand Down Expand Up @@ -214,9 +214,12 @@ def tail(path):
reach whatever that address routes to.

Empty components are kept rather than filtered out, so a doubled or trailing slash is
a different address than the one the contract names instead of an alias for it. Each
component is unquoted after the split, so a `%2F` inside an identifier stays one
component rather than dividing the address.
a different address than the one the contract names instead of an alias for it. That
holds for a leading one too -- `//api/v1/...` is beneath no root and reaches nothing --
but only because `_target` reads the request line rather than `self.path`, which the
stdlib has already rewritten by the time anything here is called. Each component is
unquoted after the split, so a `%2F` inside an identifier stays one component rather
than dividing the address.
"""
if path == API_ROOT:
return []
Expand Down Expand Up @@ -544,7 +547,7 @@ def do_POST(self):
# --- the boundary ---------------------------------------------------------------------

def _resolve(self):
parsed = urlsplit(self.path)
path, query = self._target()
if self.headers.get("Host") != self.server.authority:
# A request naming any other host arrived here by having that name pointed at
# loopback, which is the shape DNS rebinding takes. The socket is not the
Expand All @@ -553,23 +556,53 @@ def _resolve(self):
origin = self.headers.get("Origin")
if origin is not None and origin != self.server.origin:
return self._error(HTTPStatus.FORBIDDEN, "Unexpected Origin")
if not self._origin_form(parsed):
if not self._origin_form(path):
return self._error(HTTPStatus.BAD_REQUEST, "Unsupported request target")
segments = tail(parsed.path)
segments = tail(path)
if segments is not None:
return self._api(parsed, segments)
return self._static(parsed.path)

def _origin_form(self, parsed) -> bool:
return self._api(query, segments)
return self._static(path)

def _target(self):
"""The request target as the client wrote it, split into path and query.

Not `self.path`, which is not what arrived: `BaseHTTPRequestHandler.parse_request`
rewrites a leading `//...` to `/...` before a handler is ever called (CPython
gh-87389, which keeps a client from reading a *redirect* to `//host/path` as an
absolute URI). That protection is reasonable and it is not this surface's to
inherit: taken here it makes `//api/v1/opportunities/{id}/status` a second spelling
of the one command address, and `tail` never gets to refuse it, because the rewrite
has already happened. The raw request line is still on the handler, so the address
this surface routes is read from there and is the address the client asked for.

Nor is it `urlsplit`, whose rules are a URI reference's rather than a request
target's: it reads `//api/v1/session` as the authority `api` and the path
`/v1/session`. In a request line that is an origin-form target -- an absolute path
whose first component happens to be empty -- and empty components are kept here, as
`tail` explains. Splitting the query off directly keeps the one parser that decides
what an address *is* in this file, where the rule is written down.
"""
words = self.requestline.split()
# A request line that did not parse never reaches a handler; the fallback is for a
# caller that set `path` without one.
raw = words[1] if words[1:] else self.path
path, _, query = raw.partition("?")
return path, query

def _origin_form(self, path) -> bool:
"""A request target this surface answers: a path, carrying no scheme and no authority.

A target in absolute form -- `http://127.0.0.1:8765/api/v1/...` -- names its own
authority, and HTTP makes that authority, not the Host header, the one that
identifies the server. This surface settles identity on Host, so honouring absolute
form would mean routing by one authority while checking another, and would give the
single command address a second spelling. It is refused rather than half-honoured.

Origin form is exactly an absolute path, so a leading `/` is the whole test. The
other forms HTTP defines -- absolute, authority and asterisk -- all begin with
something else, and each would name a target this surface has no answer for.
"""
return not parsed.scheme and not parsed.netloc
return path.startswith("/")

def _authorized(self) -> bool:
presented = self.headers.get(TOKEN_HEADER, "")
Expand All @@ -579,15 +612,15 @@ def _authorized(self) -> bool:
presented.encode("utf-8", "surrogateescape"), self.server.token.encode("utf-8")
)

def _api(self, parsed, segments):
def _api(self, query, segments):
if not self._authorized():
return self._error(HTTPStatus.UNAUTHORIZED, "A valid launch token is required")
try:
# keep_blank_values, because the default drops `?status=` entirely and this
# route's whole query contract is fail-closed: a parameter that disappears
# before it is looked at is a narrowed request answered with an unnarrowed
# list, which is the one wrong answer that looks right.
payload = self._projection(segments, parse_qs(parsed.query, keep_blank_values=True))
payload = self._projection(segments, parse_qs(query, keep_blank_values=True))
except KeyError:
return self._error(HTTPStatus.NOT_FOUND, "No record with that id")
except ValueError as exc:
Expand Down Expand Up @@ -660,7 +693,7 @@ def _command(self):
required here rather than merely checked when present: a read with no Origin is an
ordinary same-document fetch, but a write with none has nothing to say for itself.
"""
parsed = urlsplit(self.path)
path, query = self._target()
if self.headers.get("Host") != self.server.authority:
return self._error(HTTPStatus.FORBIDDEN, "Unexpected Host")
if self.headers.get("Origin") != self.server.origin:
Expand All @@ -671,32 +704,32 @@ def _command(self):
return self._error(HTTPStatus.UNAUTHORIZED, "A valid launch token is required")
# After the provenance checks, so the ordering above is the ordering the contract
# names: this is a question about the request line, not about who is asking.
if not self._origin_form(parsed):
if not self._origin_form(path):
return self._error(HTTPStatus.BAD_REQUEST, "Unsupported request target")
match tail(parsed.path):
match tail(path):
case ["opportunities", identifier, "status"]:
return self._record_status(identifier, parsed.query)
return self._record_status(identifier, query)
case ["reviews", identifier, "decision"]:
return self._decide(identifier, parsed.query)
return self._decide(identifier, query)
# Each outward address names the one service call it means, written out here
# beside the other commands. A handler that took the address and worked out which
# method it stood for would be a router with a lookup in it, and the address would
# stop being the whole of what distinguishes these two commands.
case ["reviews", identifier, "draft" as command]:
return self._outward(
identifier, command, parsed.query, lambda actions: actions.draft(identifier)
identifier, command, query, lambda actions: actions.draft(identifier)
)
case ["reviews", identifier, "reconcile" as command]:
return self._outward(
identifier, command, parsed.query, lambda actions: actions.reconcile(identifier)
identifier, command, query, lambda actions: actions.reconcile(identifier)
)
# The two intake addresses. Each names one service call, written out here beside
# the others: there is no `/intake` that takes a source as a field, because an
# address a request could choose is an authority a page could choose.
case ["intake", "eml"]:
return self._ingest_eml(parsed.query)
return self._ingest_eml(query)
case ["intake", "gmail"]:
return self._ingest_gmail(parsed.query)
return self._ingest_gmail(query)
# Every other address reads -- including a path outside the API root, for which
# `tail` returns None and no sequence pattern above can match. Saying so with Allow
# rather than 404 keeps a POST from reporting which read routes exist.
Expand Down
15 changes: 14 additions & 1 deletion src/system/web/static/screens.js
Original file line number Diff line number Diff line change
Expand Up @@ -581,13 +581,26 @@ function opportunityDetail(record, extra, actions) {
);

const bound = record.bound;
/* An undecided review has no destination recorded against it, and both fields read null.
* Concatenated, that published the word "null" twice under a note that says approving
* binds this packet to the destination named here -- and beside a button that names the
* launch destination in its own label. So an undecided packet names that same launch
* destination, which the session reports and this file does not derive; a decided one
* keeps naming what was recorded, which is the only one of the two that binds anything.
* Before the session has arrived there is nothing to name, and the row says so. */
const target = extra.target;
const destination = record.action.provider
? record.action.provider + " · " + record.action.provider_namespace
: target
? target.provider + " · " + target.provider_namespace
: "—";
const packetBlock = bound
? definitions([
["review", shorten(bound.review, 14)],
["source", bound.source ? shorten(bound.source, 18) : "—"],
["recipient", bound.to || "—"],
["subject", bound.subject || "—"],
["destination", record.action.provider + " · " + record.action.provider_namespace],
["destination", destination],
])
: empty("No current review — nothing an approval could bind.");
const wording = bound ? paragraphs(bound.wording, "wording") : null;
Expand Down
Loading