SEO: fix Search Console indexing, add provider comparisons + free DNS checker

[leoisadev1]

Why

Google Search Console shows ~450 pages not indexed (434 "Discovered – currently not indexed", 9 "Crawled – not indexed", 5 "Duplicate, Google chose different canonical"). Root cause: the 450 old-version doc snapshots under /docs/v/<x> are prerendered, listed in the sitemap with a hardcoded lastmod, self-canonicalized, and fully crawlable — Google sees ~10x near-duplicate content and refuses to index most of it.

This PR fixes the indexing issues and adds two new keyword surfaces built on data only this project has.

Indexing fixes

• noindex, follow on old-version docs (src/routes/docs/$.tsx): baked into the prerendered HTML, deduped against the global index, follow (verified single tag in build output). Old versions stay reachable via the version picker.
• Sitemap lists only current docs: ~500 → 82 URLs, zero /docs/v/ entries.
• Real lastmod values from git history via scripts/generate-docs-lastmod.ts → committed docs-lastmod.generated.json (Vercel shallow-clones, so the manifest is committed; regenerate with bun run lastmod:generate).
• robots.txt unchanged deliberately — Disallow: /docs/v/ would stop Google from ever seeing the noindex; a comment documents this.
• Guard test: every current docs page must have a unique 40–170-char frontmatter description.

New: /compare (14 provider comparison pages)

resend-vs-postmark, sendgrid-vs-mailgun, mailgun-vs-ses, … generated from the SDK's own SUPPORTED_MESSAGE_FIELDS capability matrix (now exported from the package and snapshotted to client-safe JSON with a drift-guard test). Each page: hand-written 120+ word intro (enforced by test), field-support table, computed fallback-compatibility analysis, compilable code snippets for both adapters, FAQ with FAQPage JSON-LD. Unknown slugs 404; reversed slugs 301 in vercel.json. Cross-linked from the 8 highest-traffic adapter docs.

New: /tools/email-dns-checker

Free SPF / DKIM / DMARC / MX checker. DNS-over-HTTPS (Cloudflare → Google fallback) server-side, pure unit-tested parsers, per-instance rate limiting + 60s result cache, provider hints mapping SPF includes to adapter docs, WebApplication JSON-LD, ~500 words of explainer content. ?domain= URLs are shareable but always canonicalize to the bare tool URL. Smoke-tested against live DNS.

Verification done

• bun run build: sitemap has 82 URLs / 0 /docs/v/, old-version pages carry exactly one noindex, follow meta, current pages keep index, follow, canonicals + JSON-LD present in prerendered HTML
• bun test (56 pass), bun run check-types, bun run types:check, bun run check all green
• Live DoH parse test against github.com (DMARC pass, SPF 8/10-lookup warning, SendGrid detected)

After merging — Google Search Console steps

1. Resubmit https://email-sdk.dev/sitemap.xml
2. URL-inspect one /docs/v/… page → expect "Excluded by 'noindex' tag"
3. Press Validate fix on the "Duplicate, Google chose different canonical" and "Crawled – currently not indexed" reports
4. The 3 "Page with redirect" entries are the intentional vercel.json doc redirects — no action
5. Re-check in 2–4 weeks; only then consider Disallow: /docs/v/ (optional, crawl-budget only)

---

Created with PostHog Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
email-sdk-fumadocs	Ready	Preview, Comment	Jul 12, 2026 4:01am

[greptile-apps]

Greptile Summary

This PR updates the docs site for SEO and adds new provider comparison and DNS-checking pages. The main changes are:

• Adds noindex, follow metadata for old versioned docs.
• Rebuilds sitemap entries around current docs and generated lastmod data.
• Adds curated provider comparison pages from SDK field-support data.
• Adds a free email DNS checker for SPF, DKIM, DMARC, and MX records.
• Adds generator scripts and tests for SEO/frontmatter and field-support drift.

Confidence Score: 4/5

Mostly safe after fixing the package-root export issue.

The docs-site changes are scoped and covered by generated data/tests. The root SDK export can break browser bundling for apps importing @opencoredev/email-sdk.

packages/email-sdk/src/index.ts

T-Rex Logs

What T-Rex did

• Reproduced the browser bundle failure when the root entry imports only createEmailClient from the email-sdk root, failing due to an unresolved node:fs/promises in utils.js.
• Collected artifacts that document the repro harness, the minimal entry import, and the verbose bundle failure log.
• Validated static-site checks by running the validation transcript, which produced VALIDATION_PASS and exit_code: 0.
• Compiled machine-readable evidence showing exact inspected paths, sitemap counts, robots meta counts, canonical hrefs, and JSON-LD details to support the validation results.

_{Ran code and verified through T-Rex}

Important Files Changed

Filename	Overview
apps/fumadocs/src/lib/compare.ts	Defines curated provider comparison pairs, intros, capability helpers, and code snippet generation.
apps/fumadocs/src/lib/dns-checker-runtime.ts	Adds server-side DNS-over-HTTPS lookup, caching, and rate limiting for the DNS checker.
apps/fumadocs/src/lib/dns-checker.ts	Adds pure DNS record parsing, domain normalization, and provider hint logic.
apps/fumadocs/src/routes/compare/$pair.tsx	Adds SEO-ready curated comparison pages with FAQ JSON-LD, tables, fallback analysis, and snippets.
apps/fumadocs/src/routes/docs/$.tsx	Adds noindex/follow robots metadata for old-version docs while preserving current docs indexing.
apps/fumadocs/src/routes/sitemap[.]xml.ts	Reworks sitemap generation to include new SEO pages and only current docs with generated lastmod dates.
apps/fumadocs/src/routes/tools/email-dns-checker.tsx	Adds the email DNS checker UI, shareable query handling, canonical metadata, and explanatory content.
apps/fumadocs/vite.config.ts	Adds prerender coverage for versioned docs, compare pages, and tools routes.
packages/email-sdk/src/index.ts	Exports the field-support matrix from the package root, but this pulls a Node-only utils module into browser-facing root imports.
vercel.json	Adds permanent redirects for reversed provider comparison slugs.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant User
participant Browser as Email SDK docs site
participant Server as React Start server function
participant CF as Cloudflare DoH
participant Google as Google DoH

User->>Browser: "Open /tools/email-dns-checker?domain=example.com"
Browser->>Server: checkEmailDnsServerFn(domain)
Server->>Server: Normalize domain and check cache/rate limit
par Resolve records
    Server->>CF: TXT example.com
    Server->>CF: TXT _dmarc.example.com
    Server->>CF: MX example.com
    Server->>CF: TXT selector._domainkey.example.com
end
alt Cloudflare unavailable
    Server->>Google: Repeat failed DoH lookup
end
Server->>Server: Parse SPF, DKIM, DMARC, MX, and provider hints
Server-->>Browser: DnsCheckResult
Browser-->>User: Render findings and shareable URL

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant User
participant Browser as Email SDK docs site
participant Server as React Start server function
participant CF as Cloudflare DoH
participant Google as Google DoH

User->>Browser: "Open /tools/email-dns-checker?domain=example.com"
Browser->>Server: checkEmailDnsServerFn(domain)
Server->>Server: Normalize domain and check cache/rate limit
par Resolve records
    Server->>CF: TXT example.com
    Server->>CF: TXT _dmarc.example.com
    Server->>CF: MX example.com
    Server->>CF: TXT selector._domainkey.example.com
end
alt Cloudflare unavailable
    Server->>Google: Repeat failed DoH lookup
end
Server->>Server: Parse SPF, DKIM, DMARC, MX, and provider hints
Server-->>Browser: DnsCheckResult
Browser-->>User: Render findings and shareable URL

Loading

_{Reviews (1): Last reviewed commit: "feat(seo): fix GSC indexing issues, add ..." | Re-trigger Greptile}

[greptile-apps]

Avoid Node-only root import
Exporting SUPPORTED_MESSAGE_FIELDS from the package root makes @opencoredev/email-sdk depend on ./utils.js, whose top-level node:fs/promises import is visible to browser/client bundlers. Web apps that import createEmailClient from the root can now fail to bundle because the root entry pulls in a Node builtin.

Artifacts

Repro: generated browser bundling harness

• Contains supporting evidence from the run (text/javascript; charset=utf-8).

Repro: generated entry importing only createEmailClient

• Contains supporting evidence from the run (text/javascript; charset=utf-8).

Repro: verbose esbuild browser bundle failure showing utils.js and node:fs/promises

• Keeps the command output available without making the summary code-heavy.

_{Ran code and verified through T-Rex}