Publish the compatibility report at /compat/

[Neaox]

Summary

• New Compatibility section at /compat/. It renders the compat-report.json that every Overcast release attaches (added upstream in feat(compat): publish a public compatibility report with every release overcast#2141): what passes in each of the eight clients (six AWS SDKs, the CLI, the CDK), what doesn't, and why.
• Overview (/compat/):
  • headline numbers
  • changes since the previous release
  • every reason a result doesn't pass, grouped by who has to act (Overcast, the test suite, the SDK, the environment, not tested)
  • a service × client matrix with a service filter
• Service pages (/compat/<service>/) list every test with a result chip per client. Each failing test expands to show the reason, the parsed expected/actual diff, what it is blocked by, and its tracking issue. With no tracking issue, a prefilled "Report this" link opens one carrying the marker that links it back on the next release.
• Reason pages (/compat/reason/<code>/) are static lists that work without JavaScript and are searchable.
• Explorer (/compat/explore/) filters every result by text, outcome, owner, client, tracking issue and change since the previous release.
  • Facet counts follow the other active filters.
  • The whole view is kept in the URL, and back/forward work.
  • It loads a 32 KB (gzipped) per-release index.
• Older releases get /compat/history/<tag>/.
• Markdown twins, the sitemap and llms.txt cover the new pages. There is a new header nav entry.

Screenshots

Captured from a real browser (Playwright/Chromium) against astro dev on this branch, with the synced reports built by --publish-report from two real main-branch compat runs. Both themes are shown because the result tints are new colour pairings. Phone width is shown because the matrix is the one wide element: it scrolls inside its card with the service column pinned.

Overview, dark, 1440px:

Service × client matrix, light, 1440px:

IAM service page with a failing test expanded (clients sharing an outcome grouped, parsed diff), light, 1440px:

Explorer filtered to soaking candidates, dark, 1440px:

Matrix, light, 375px:

Verification

• npm test: 160 pass. The new src/lib/compat-report.test.ts covers:
  • the numbers: measured rate, owner counts, matrix rows, service rate matching its row, operation rollups, release diffs
  • the explorer index encoding
  • the filter: URL round-trip, token matching, facets counted with the others applied, masks, untested operations, ranking
  • result grouping, the prefilled issue link, and the markdown twins
• npm run check: copy-lint clean, astro check 0 errors / 0 warnings.
• npm run build with two local reports: builds the compat pages, .md twins and data routes, and pagefind indexes them.
• In the browser:
  • explorer ?q=sqs&outcome=not-emulated&suite=rust-sdk restores from the URL
  • back/forward walks the history
  • facet sums reconcile: outcomes 8,261 = results plus untested operations; clients 8,092 = results; owners 3,496 = everything that isn't a pass
  • no page-level horizontal overflow at 375px on any compat page
  • light and dark checked
• scripts/a11y-audit.mjs on the 7 compat page types (overview, service, explorer unfiltered and filtered, reason, archived release, a 0%-pass service): 0 violations over 49 theme × viewport × state combinations.
  • The first pass found 16 contrast failures from faded "empty" facet buttons. Those and the explorer's dimmed out-of-filter chips now use a dashed outline with full-contrast text.
• content:sync with no local report and the network on lists the releases, finds none with the asset yet, and builds the empty state. A sync never dirties the tree.

Notes

• Until the next Overcast release ships compat-report.json (after feat(compat): publish a public compatibility report with every release overcast#2141 merges), production /compat/ shows an empty state that points to the support matrix. Every step of the new sync degrades to fewer reports and cannot fail the build.
• Rate definition: the pass rate counts passes against Overcast's own gaps (behaviour differs, 501, flaky).
  • It leaves out tests not written for a language, SDK limits, environment skips, and tests blocked by an earlier failure. Counting a blocked test would charge its root failure once per dependent.
  • The callout on the overview says so. This is the fork most worth a second opinion.
• Service naming: display names and doc links come from service-support.json, via a small alias table (elastic-load-balancing → elbv2) and names for the three compat-only ids.
• Dev-only: OVERCAST_COMPAT_REPORT (one or more local reports) and OVERCAST_COMPAT_OFFLINE, documented in .env.example and AGENTS.md.
• Separate follow-up: scripts/a11y-audit.mjs never exits on Windows and leaves orphaned astro preview servers. It kills the run-command.ts wrapper but not astro. Being fixed separately.