[Bug]: Security: SSRF via /api/crawler — server fetches any URL including cloud metadata endpoints

[AgentSeal]

Bug Description

The /api/crawler endpoint fetches any URL server-side with no validation.
An attacker can use it to reach internal services, cloud metadata endpoints (169.254.169.254 for AWS/GCP credential theft), and scan the host's local network. The MD5 auth uses an 8-char truncated timestamp (~11.5 day validity window) and the default password is empty.

Affected code:

• src/app/api/crawler/route.ts:18 — fetch(url) with zero URL validation
• src/utils/signature.ts:4 — MD5(key + "::" + timestamp.substring(0,8))
• src/constants/prompts.ts:306 — { ...defaults, ...overrides } allows
  full system prompt replacement
• src/app/api/sse/route.ts:123 — Access-Control-Allow-Origin: *

Steps to Reproduce

1. Clone repo, npm install, start with ACCESS_PASSWORD="" (the default)
2. Generate auth: MD5("" + "::" + Date.now().toString().substring(0,8))
3. curl -X POST http://localhost:3000/api/crawler \
   -H "Authorization: Bearer "
   -H "Content-Type: application/json" \
   -d '{"url": "https://httpbin.org/ip"}'
4. Response contains the SERVER's public IP (proves server-side fetch)
5. Repeat with "http://169.254.169.254/latest/meta-data/" - server
   attempts to reach AWS metadata endpoint (500 error confirms attempt)
6. Repeat with "http://10.0.0.1/" - server probes internal network

Expected Behavior

The crawler should validate URLs before fetching: block internal IPs (10.x, 172.16.x, 192.168.x), cloud metadata (169.254.169.254), and localhost. Auth should use HMAC-SHA256 with full timestamps, not MD5 with truncation. Default password should not be empty.

Screenshots

No response

Deployment Method

• Docker
• Vercel
• Server

Desktop OS

No response

Desktop Browser

No response

Desktop Browser Version

No response

Smartphone Device

No response

Smartphone OS

No response

Smartphone Browser

No response

Smartphone Browser Version

No response

Additional Logs

Suggested fixes:

1. Block internal/metadata IPs before fetch (10.x, 172.16.x, 192.168.x, 169.254.x, localhost)
2. Replace MD5 with HMAC-SHA256, use full timestamp + nonce
3. Restrict promptOverrides — don't allow full system prompt replacement
4. Restrict CORS to actual frontend origin

Full report: https://agentseal.org/mcp/https-githubcom-u14app-deep-research/exploit-report

If you ship fixes, we'll rescan and update the report.

• AgentSeal Security Research (hello@agentseal.org) (https://github.com/AgentSeal/agentseal)