Repository navigation
✨ Inbox: add links to the Inbox, fetched under an SSRF guard - #18418
Conversation
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
Mutation testing❌ gremlins did not produce a report for 2 package(s), so no verdict can be derived from this run (see the job log):
Verdict: |
|
Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor. @clubanderson this PR is held at Why:
Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path. stage 095-pre-merge-soak → 071-needs-author: conflicting Evidence bundle `#184180e8c692` sha256:112d60b6d3f3ad5aee9c7aa73ce860d1312c8e4b48656a78fc6ffff9c8ef9197 — download from the dashboard or `hivectl review evidence #18418`. |
|
[APPROVALNOTIFIER] This PR is APPROVED Approval requirements bypassed by manually added approval. This pull-request has been approved by: The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Paste a URL onto the Inbox or type it into Add: the hive fetches it once and runs the body through the upload type pipeline (HTML pages become readable markdown via golang.org/x/net/html; linked md/json/csv files become that kind). SSRF guard: http/https only, no userinfo, own DNS lookup with the connection pinned to the checked IP (private, loopback, link-local, CGNAT, multicast, ULA, metadata and embedded-v4 ranges refused), every redirect re-checked (max 3), body and time caps, no environment proxy; a matching upstream proxy gets CONNECT to the checked IP. Signed-off-by: Andy Anderson <andy@clubanderson.com>
…r's own cap Signed-off-by: Andy Anderson <andy@clubanderson.com>
768dd4f to
0e8c692
Compare
|
New changes are detected. LGTM label has been removed. |
|
merged by Hive trusted-author auto-merge: author @clubanderson holds role owner and GitHub permission admin on repo; CI green at 0e8c692. |
Second PR of drag-and-drop ingest: links. Builds on the upload source from the documents PR.
What it does
POST /api/inbox/uploads/urlwith{url}; same result shape as file uploads (201, or 400link_refused, 413, 415, 422, 502fetch_failed, 507). Audited asinbox_uploadwithorigin=linkand the host only, never the full URL.Content-Typemaps to one; the content must still pass the sniff. A page becomeshtml, a linked.json/.csv/.mdbecomes that kind. YouTube and other streaming pages are read as HTML; audio/video links are refused for now (they will plug into the recording pipeline).golang.org/x/net/html(BSD-3-Clause, already a dependency):<title>names the item;<article>/<main>/<body>is rendered as markdown (headings, paragraphs, lists, quotes, code, tables, absolute links, image alt text); scripts, styles, forms, nav, footer, aside, hidden and aria-hidden elements are dropped; nothing is executed or loaded; depth and node caps..html/.htmfiles can be dropped too.link · <host>, titled by<title>, else the file name, else the URL. The URL is provenance only (Content.Provenance, factsource_url).SSRF rules
localhost,*.localhost,*.local,*.internal,metadata,metadata.google.internalare refused before lookup; IP literals are checked directly.LookupNetIP) and refuses the link if any answer is blocked, then dials the checked IP (no second lookup, so DNS rebinding cannot swap the address). Blocked: 0/8, 10/8, 100.64/10 (CGNAT, incl. 100.100.100.200), 127/8, 169.254/16 (incl. 169.254.169.254), 172.16/12, 192.0.0/24, 192.0.2/24, 192.88.99/24, 192.168/16, 198.18/15, 198.51.100/24, 203.0.113/24, 224/4, 240/4, ::, ::1, 100::/64, 2001::/32 (Teredo), 2001:db8::/32, fc00::/7 (incl. fd00:ec2::254), fe80::/10, fec0::/10, ff00::/8, and IPv4 embedded in IPv4-mapped, NAT64 (64:ff9b::/96) and 6to4 (2002::/16) addresses.Authorization/Cookieare dropped on redirects.inbox.uploads.max_file_bytes), Content-Length checked first; connect 10 s, TLS 10 s, headers 15 s, whole fetch 60 s; keep-alives off;User-Agent: hive-inbox/<version>.HTTP(S)_PROXYis never used (the proxy would resolve again). Whennetwork.upstream_proxymatches the host, the hive asks it toCONNECTto the checked IP:port.Tests
file:, 4 redirects (refused) and 3 (allowed), oversize streamed body, 404, unknown host; a public URL is allowed via the httptest server behind a fake public resolver, and the dialer proves no refused address was ever dialed.AddLinkthrough the type pipeline (HTML, CSV by file name, binary, audio, too large, fetch failure, refusal), HTML extraction cases, the HTTP route (role matrix, refusals, audit without the URL), and node tests for the URL field and URL paste.No new dependencies.
🤖 Generated with Claude Code