Skip to content

✨ Inbox: add links to the Inbox, fetched under an SSRF guard - #18418

Merged
hivecommons-hive[bot] merged 2 commits into
v5from
feat/inbox-upload-links
Nov 16, 2026
Merged

hivecommons-hive[bot] merged 2 commits into
v5from
feat/inbox-upload-links

Conversation

@clubanderson

Copy link
Copy Markdown
Member

Second PR of drag-and-drop ingest: links. Builds on the upload source from the documents PR.

What it does

  • UI. + Add gets a URL field (Enter or Add link), and pasting a single http/https URL onto the Inbox (outside a text field) adds it. Each link gets a row ("Fetching", then added or the per-link error). Owner and read-write only.
  • Route. POST /api/inbox/uploads/url with {url}; same result shape as file uploads (201, or 400 link_refused, 413, 415, 422, 502 fetch_failed, 507). Audited as inbox_upload with origin=link and the host only, never the full URL.
  • Type pipeline. The fetched body goes through the same sniff + parse as a dropped file: the file name (Content-Disposition or the URL path) wins when it names a readable type, else the response Content-Type maps to one; the content must still pass the sniff. A page becomes html, a linked .json/.csv/.md becomes that kind. YouTube and other streaming pages are read as HTML; audio/video links are refused for now (they will plug into the recording pipeline).
  • HTML extraction with golang.org/x/net/html (BSD-3-Clause, already a dependency): <title> names the item; <article>/<main>/<body> is rendered as markdown (headings, paragraphs, lists, quotes, code, tables, absolute links, image alt text); scripts, styles, forms, nav, footer, aside, hidden and aria-hidden elements are dropped; nothing is executed or loaded; depth and node caps. .html/.htm files can be dropped too.
  • Item. link · <host>, titled by <title>, else the file name, else the URL. The URL is provenance only (Content.Provenance, fact source_url).

SSRF rules

  • http/https only; userinfo is refused; 2048-character cap.
  • Host names localhost, *.localhost, *.local, *.internal, metadata, metadata.google.internal are refused before lookup; IP literals are checked directly.
  • The hive resolves the name itself (LookupNetIP) and refuses the link if any answer is blocked, then dials the checked IP (no second lookup, so DNS rebinding cannot swap the address). Blocked: 0/8, 10/8, 100.64/10 (CGNAT, incl. 100.100.100.200), 127/8, 169.254/16 (incl. 169.254.169.254), 172.16/12, 192.0.0/24, 192.0.2/24, 192.88.99/24, 192.168/16, 198.18/15, 198.51.100/24, 203.0.113/24, 224/4, 240/4, ::, ::1, 100::/64, 2001::/32 (Teredo), 2001:db8::/32, fc00::/7 (incl. fd00:ec2::254), fe80::/10, fec0::/10, ff00::/8, and IPv4 embedded in IPv4-mapped, NAT64 (64:ff9b::/96) and 6to4 (2002::/16) addresses.
  • Every redirect hop is re-checked (scheme, userinfo, host, and the resolution at dial time); at most 3 redirects; Authorization/Cookie are dropped on redirects.
  • Body streamed through a limit reader (inbox.uploads.max_file_bytes), Content-Length checked first; connect 10 s, TLS 10 s, headers 15 s, whole fetch 60 s; keep-alives off; User-Agent: hive-inbox/<version>.
  • HTTP(S)_PROXY is never used (the proxy would resolve again). When network.upstream_proxy matches the host, the hive asks it to CONNECT to the checked IP:port.

Tests

  • Address table (blocked and allowed, embedded v4), URL refusal table (schemes, userinfo, literals, internal names, long URLs).
  • Fetch through a fake resolver + dialer: private, ULA, mixed answers, metadata, CGNAT, a numeric name resolving to loopback, a redirect to a name that resolves privately (rebinding-style), a redirect to the metadata literal, a redirect adding credentials, a redirect to file:, 4 redirects (refused) and 3 (allowed), oversize streamed body, 404, unknown host; a public URL is allowed via the httptest server behind a fake public resolver, and the dialer proves no refused address was ever dialed.
  • Upstream proxy: a fake CONNECT proxy sees the checked IP, not the name.
  • AddLink through the type pipeline (HTML, CSV by file name, binary, audio, too large, fetch failure, refusal), HTML extraction cases, the HTTP route (role matrix, refusals, audit without the URL), and node tests for the URL field and URL paste.

No new dependencies.

🤖 Generated with Claude Code

@kubestellar-prow kubestellar-prow Bot added dco-signoff: yes Indicates the PR's author has signed the DCO. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Nov 16, 2026
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#18415 ✨ Inbox: pills record where an item was shared, by whom and when same files src/cmd/hive/boot_inbox.go
src/pkg/dashboard/static-src/css/inbox.css
src/pkg/dashboard/static-src/js/inbox.js
src/pkg/dashboard/static/index.html
src/pkg/inbox/service.go
#18417 ✨ meetings: Settings tab "Meeting attendee" with Kokoro persona presets (Betty E) same files dashboard/openapi.json
src/docs/api-reference/knowledge.md
src/pkg/dashboard/static/index.html
#18416 ✨ meetings: Quorum v0.2.0 and Discord speech through the TTS service same files dashboard/openapi.json
src/docs/api-reference/knowledge.md
src/pkg/dashboard/static/index.html
#18405 ✨ Inbox: send items into an Inception (spek) session same files dashboard/openapi.json
src/pkg/dashboard/static-src/js/inbox.js
src/pkg/dashboard/static/index.html
#18419 ⚠️ Retire the inception API, watcher and brainstorm inception mode same files dashboard/openapi.json
src/pkg/dashboard/static/index.html

@github-actions

Copy link
Copy Markdown
Contributor

Mutation testing

❌ gremlins did not produce a report for 2 package(s), so no verdict can be derived from this run (see the job log):

  • pkg/dashboard: gremlins exited 1
  • pkg/inbox: gremlins exited 1

Verdict: error (#17349).

@hivecommons-hive

hivecommons-hive Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

@clubanderson this PR is held at stage/071-needs-author and the hive will not merge it yet.

Why:

  • the branch conflicts with the base and needs a rebase

Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path.

stage 095-pre-merge-soak → 071-needs-author: conflicting

Evidence bundle `#184180e8c692` sha256:112d60b6d3f3ad5aee9c7aa73ce860d1312c8e4b48656a78fc6ffff9c8ef9197 — download from the dashboard or `hivectl review evidence #18418`.

@github-actions github-actions Bot added the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Nov 16, 2026
@clubanderson clubanderson added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Nov 16, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@github-actions github-actions Bot removed the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Nov 16, 2026
Paste a URL onto the Inbox or type it into Add: the hive fetches it once
and runs the body through the upload type pipeline (HTML pages become
readable markdown via golang.org/x/net/html; linked md/json/csv files
become that kind).

SSRF guard: http/https only, no userinfo, own DNS lookup with the
connection pinned to the checked IP (private, loopback, link-local,
CGNAT, multicast, ULA, metadata and embedded-v4 ranges refused), every
redirect re-checked (max 3), body and time caps, no environment proxy;
a matching upstream proxy gets CONNECT to the checked IP.

Signed-off-by: Andy Anderson <andy@clubanderson.com>
…r's own cap

Signed-off-by: Andy Anderson <andy@clubanderson.com>
@clubanderson
clubanderson force-pushed the feat/inbox-upload-links branch from 768dd4f to 0e8c692 Compare November 16, 2026 14:01
@kubestellar-prow kubestellar-prow Bot removed the lgtm Indicates that a PR is ready to be merged. label Nov 16, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

New changes are detected. LGTM label has been removed.

@clubanderson clubanderson added the lgtm Indicates that a PR is ready to be merged. label Nov 16, 2026
@hivecommons-hive
hivecommons-hive Bot merged commit 52e35b0 into v5 Nov 16, 2026
87 of 91 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the feat/inbox-upload-links branch November 16, 2026 14:13
@hivecommons-hive

Copy link
Copy Markdown
Contributor

merged by Hive trusted-author auto-merge: author @clubanderson holds role owner and GitHub permission admin on repo; CI green at 0e8c692.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has signed the DCO. lgtm Indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant