AI Trust Index module#4154
Open
gorkem-bwl wants to merge 63 commits into
Open
Conversation
Approved design for a 6th sidebar module that pulls the public AI Trust Index feed (verifywise.ai/ai-trust-index.json), lets orgs browse/search and track apps, and emails configured recipients weekly on risk-material change. Encodes the adversarial-review fixes: canonical hashing, material-vs-full hash split, feed sanity gates, transactional first-seed guard, idempotent singleton cron, per-org recipient isolation.
- Removed-app handling: email a 'no longer assessed' digest section when a tracked app leaves the index; add removed_at one-shot guard so it fires once - Bulk tracking: POST /tracked/bulk + Browse checkboxes + 'Track selected' - Confirm weekly-cron-only (no manual sync) - Full docs in v1: technical reference + user-guide articles (both locations) - Default browse sort score-desc, 25/page
14 tasks across 7 phases: DB migration + models, pure hashing/feed-gating logic, data-access queries, REST API, weekly cron + digest email, frontend module, live verification + docs. TDD with verified codebase patterns (compileMjmlToHtml, migrate-db, SidebarShell, STATUS_CODE, sendAutomationEmail).
…anonicalize tests
… empty-feed upsert
## Changes
- Controller: getApps, getApp, getTracked, trackApp, trackAppsBulk, untrackApp, getSettings, updateSettings
- Routes: /api/ai-trust-index/{apps,apps/:slug,tracked,tracked/bulk,tracked/:slug,settings}
- Mounted in app.ts
- updateSettings is Admin/SuperAdmin-only (403 gate before write, email validation)
- Unit tests: 3/3 pass (TDD — test written before controller)
## Changes - Add syncAiTrustIndex() orchestration function with ISO-week singleton guard, fetch-fail abort, sanity-gate abort, and first-seed-silent guard - Email digest rendered via compileMjmlToHtml with per-org Changed/Removed sections - TDD: 4 tests covering all 4 guards (first-seed silent, week no-op, bad feed, email path) ## Benefits - Prevents email blast on initial seeding - Idempotent: safe to call multiple times in same ISO week - Fail-open on network issues (no writes, no emails)
…el-string spacing ## Changes - Browse/index.tsx: replaced raw MUI Table/TableHead/TableBody/TableRow/TableCell with MCPTable<TrustIndexRow>; checkbox and Track/Untrack cells wrap with stopPropagation so row-click navigation excludes them; select-all checkbox moved to filter bar; server-side TablePagination kept outside MCPTable unchanged; spacing tokens px/mt converted from theme.spacing(4)/theme.spacing(6) to literal pixel strings "32px"/"48px" - Tracked/index.tsx: replaced raw MUI table markup with MCPTable<TrustIndexRow>; no_longer_in_index rows dimmed via rowSx (opacity: 0.6); Untrack cell wraps with stopPropagation; all existing status chips and empty-state behavior preserved ## Benefits - Both pages now use the project's design-system-compliant shared table (MCPTable) - Eliminates direct MUI Table usage in AI Trust Index module - Spacing tokens comply with project pixel-string rule (no MUI numeric multipliers)
… from hash Live verification revealed the production feed is now feedVersion 2 — a backward-compatible bump that adds a per-app `history` object (all v1 fields remain). The strict version gate correctly aborted the seed, so: - Accept feedVersion 1 AND 2 (newer/unknown versions still abort). - Exclude `history` from full_hash alongside iconUrl: it carries volatile timestamps (lastChecked updates on every re-check) that would otherwise churn last_changed_at every week with no real change. Verified live: seeds 62 apps, first run email-silent, second same-week run no-ops via the singleton guard.
## Changes - Add docs/technical/domains/ai-trust-index.md: full technical reference covering the feed contract (feedVersion 1+2), 4 DB tables, global-vs-org-scoped design, two-hash change detection (material vs full), 5 sanity gates, weekly BullMQ job (first-seed + singleton guards), recipient resolution, 8 REST API endpoints, and frontend module structure. - Add 4 user-guide articles under shared/user-guide-content/content/ai-trust-index/: dashboard (overview), browse (search/filter/track/bulk), tracked (watch list + "no longer in index"), settings (Admin-only email recipients). - Wire articles into shared/user-guide-content/content/index.ts (imports + map) and userGuideConfig.ts (Gauge icon, new collection + articles + search keywords). - Add Gauge to iconMap in 3 UserGuide components (ArticlePage, CollectionPage, UserGuideLanding) to satisfy Record<IconName, LucideIcon> constraint. - Add app_slug? optional field to TrustIndexRow to close pre-existing type gap used by the Tracked page. - Add AI Trust Index row to CLAUDE.md Detailed References table. ## Benefits - Developers can find conventions and patterns for the module without reading code. - Users have contextual help for Browse, Tracked, and Settings pages. - Typecheck clean (zero errors outside pre-existing test/mock files).
…/category chips, standard pagination, compact rows, settings dropdown, biometrics chip
…gory chip colors, settings save flicker; don't false-remove dropped-field apps
…ers + standard pagination on Browse/Tracked
The sort change wrapped sortable header labels in a nested Box whose text color did not inherit from the parent cell, so the headers rendered faint. Set the header text color/size/weight explicitly on the sortable label, matching the table theme, with a darker hover.
…perAdmin fallback, slug-normalized hash, ILIKE escaping, raw-feed-count floor, email HTML-escaping, mutation logging; theme-token + VW Checkbox UI ## Changes - Validate recipientUserIds are integers (reject non-int/fractional → 400, no write) - resolveRecipients fallback now includes SuperAdmins (org_id NULL) and warns when no recipient resolves - computeHashes normalizes the slug so case/whitespace variants share material+full hashes - getAppsQuery escapes LIKE metacharacters (\\ % _) and adds ESCAPE '\\' to ILIKE - upsertFeedTx stores last_good_count from the raw feed size (rawCount), not the upserted subset - Email digest HTML-escapes the section title and slugs before MJML interpolation - Add processing/success/failure logging to trackApp, trackAppsBulk, untrackApp - MCPTable sortable header uses theme.palette.text tokens instead of hardcoded hex - AI Trust Index Browse uses the VerifyWise Checkbox (select-all + per-row) over raw MUI ## Tests - 12 new backend tests covering each fix (recipient validation, SuperAdmin fallback, slug-normalized hash, LIKE escaping, rawCount floor, validateFeed rawCount, MJML escaping)
Formatting-only; no behavioral changes. Brings the AI Trust Index migration and three client files into prettier compliance so format-check passes in CI.
Full-stack design to bring the app detail page to parity with the public website: thread the per-indicator award map through the feed, seed a day-one snapshot so fresh installs show data immediately, and rebuild AppDetail with the website's section set (grade hero + meter, verdict, comparison strip, grade scale, watch-outs, score breakdown, related apps) using VW components.
8 tasks (TDD, bite-sized): backend indicators pass-through + day-one seed snapshot migration; frontend rubric port, insights + score-breakdown components, rebuilt AppDetail with grade hero/meter/verdict/grade-scale/ watch-outs/breakdown/related; i18n + gates. Graceful degradation when the indicator map is absent.
Implements ScoreBreakdown — per-domain progress bars and 30-indicator checklist. Half awards correctly label as "Partial" (AWARD_LABELS.half), not "Disclosed" (SUBFLAG_LABELS.OK). Test verifies the half label and domain header render correctly.
…ict, breakdown, related
## Changes
- Replace plain "Trust score: X/100" text with animated meter bar (brand-primary fill, 8px pill)
- Add VerdictLine, ComparisonStrip, grade-scale chips, WatchOuts, ScoreBreakdown/fallback, RelatedApps
- Wire useApps({}) → allApps (TrustIndexAppData[]) for ComparisonStrip and RelatedApps
- Reorder sections: hero → verdict → capped note → dealbreakers → comparison → grade scale → summary → highlights → watch-outs → breakdown → policy → related
- Grade-scale chips wrapped in Box for opacity control (Chip has no sx prop)
- Add AppDetail test covering verdict/summary/breakdown-fallback path (no indicators)
Add German, French, and Spanish translations for all new AI Trust Index detail-page strings: grade scale, score breakdown labels, legend labels (Disclosed/Partial/Silent/Adverse/Not applicable), policy gap copy, category comparison strings, privacy rating, and fallback notes. DE/FR/ES all reach 100% in i18n:audit:strict.
…feed) The AI Trust Index detail page displays English content retrieved verbatim from the website feed/rubric; its rubric domain names and section labels are not translated. Remove the de/fr/es entries added for them and exempt the strings in the i18n audit (alongside the brand name) so the strict gate stays green without translating a content surface.
Surfaces the assessment confidence next to vendor/category in the hero, matching the website detail page. Closes a parity gap noted in final review.
…live feed Live feed now emits the per-indicator award map (113/116 apps). Refresh the day-one seed so fresh installs render the score breakdown + silent/vague sections immediately instead of waiting for the first weekly sync.
Browse now renders apps as a card grid (favicon, grade chip, 4-line summary, category, score) like the public index, instead of a dense table. Cards keep the app-only controls: per-card select + track/untrack (stopPropagation) and bulk-track; clicking the card body opens the detail page. Sort moves to a dropdown (the card grid has no column headers); page size is 24 for an even 3-column grid. de/fr/es added for the new sort + dealbreaker labels.
Upstream index grew to 142 apps (26 new HR/legal/finance/medical tools; 139/142 with indicators). Refresh the day-one seed snapshot accordingly.
…digest email
Settings: tell the admin the index is checked automatically every week
(Monday) and show the last-checked week, so it's clear no manual action is
needed. Surfaces last_run_week via getSettingsQuery.
Email: the digest now lists the human app name plus what changed
('Paxton AI — now grade B') instead of the raw slug, by joining the apps
table in getAffectedOrgsBySlugs.
useSettings now has a 60s staleTime so it stops refetching on the global 2s default during the page's auto-save interactions. The recipients dropdown no longer makes the form feel stuck: it shows its own loading state for the user list instead of appearing empty while all org users load.
…card grid Promote the Browse card to a reusable components/AppCard used by both Browse and Tracked. Page-specific bits are props: optional select checkbox (Browse bulk-select), an actions slot (Browse Track/Untrack, Tracked Untrack), an optional status chip (Tracked), and a dimmed flag (removed-from-index). Tracked now renders the same card grid as Browse instead of a table, with a sort dropdown and client-side pagination. Removes the per-page card duplication and the Tracked table machinery.
AppDetail read the apps list at appsData?.apps, but the hook returns the axios envelope so the list is at appsData?.data?.apps — allApps was always empty, so rank, category average, and 'vs category average' rendered as 0/—. Read the correct path, and request the full catalog (pageSize 1000, backend cap raised to 1000) so rank 'of N' and the category average are accurate.
Refresh the Browse, Tracked, and Settings sidebar docs to match the current UI: card grid (not a table), 24 per page, the sort dropdown, the richer app detail sections (verdict, comparison, grade scale, watch-outs, per-domain breakdown, related apps, confidence), the weekly-cadence note on Settings, and digest emails that list apps by name with their grade.
- Grade scale: stop dimming non-active band chips to opacity 0.5 (it made the pastel chip text unreadable). All bands now render full-contrast; the app's own band is marked with a solid outline ring instead. - Add the user-guide (i) help icon to Browse, Tracked, and Settings via helpArticlePath, opening the matching ai-trust-index article.
… detail Bump the app detail Summary body to 15px and the section headers (Summary, Highlights, Policy details, Dealbreaker flags, watch-outs, privacy rating, related apps) from 13px to 15px for readability.
Remove AI writing tells from the Browse, Tracked, Settings, and dashboard articles: prose em-dashes, stacked AI vocabulary (continuously-updated, independently-maintained, widely-used), copula avoidance, and rule-of-three. Kept numeric ranges (85-100, A-F) and the neutral reference-doc tone. Verified every factual claim against the codebase (weekly Monday 06:00 UTC sync, first-seed-silent, material-change fields, 200 bulk cap, B-cap model).
Drop the internal field names (scoreOutOf100, policyLastUpdated, processesBiometrics) from the tracked-apps digest list; describe each material change in plain language instead.
Browse category and grade dropdowns now show the catalog count per option (e.g. 'Assistant (14)', 'A (4)'). getAppsQuery returns catalog-wide categoryCounts and gradeCounts (computed over all active apps, not the current filter, so selecting one option doesn't zero the others). Tracked gains a category filter with per-category counts computed client-side from the loaded tracked list; the sort dropdown is unchanged (sort options have no count).
Update the stale sections to match the shipped state: feed indicators field, day-one seed-snapshot migration, /apps categoryCounts+gradeCounts + 1000 pageSize cap + expanded sort whitelist, name+grade digest, and the card-grid frontend (shared AppCard, 24/page, dropdown counts, richer AppDetail, help icons, English-content-surface note). Architecture, hashing, gates, and the weekly job sections were already accurate and are unchanged.
…fallback ## Changes - Digest now reports the displayed grade (B-capped governance grade), not the raw letter grade, so the email matches the grade shown everywhere else. - Each changed app shows what actually changed — "grade B → C", "score 78 → 71", "policy updated", "new dealbreaker flag", biometrics on/off — instead of a flat "now grade X" that was wrong for non-grade changes. - Added a link to the AI Trust Index module and to the settings page (where an admin manages recipients) from the digest body and footer. - Removed the Admin/SuperAdmin recipient fallback: digests go only to recipients an org explicitly configures. An org with none gets no digest (logged at info). ## Why - The displayed grade is the governance-relevant grade; emailing the uncapped letter grade contradicted the rest of the product. - The old digest could not distinguish a grade drop from a policy-date tweak. - The fallback could mail every org Admin and every platform SuperAdmin — cross-tenant noise — for orgs that never configured recipients. ## Tests - New describeMaterialChanges unit tests (displayed-grade transitions, score, flags, policy, biometrics, no-previous fallback). - Updated recipients tests to the no-fallback contract; updated sync digest mocks to the new MaterialChange shape. 56/56 AI Trust Index tests pass.
…on on detail page ## Changes - Regenerated the day-one seed snapshot (ai-trust-index-snapshot.json) from the live public feed: 205 apps on rubric v2.0 (was 142 apps on v1.2). A fresh install now seeds a catalog that matches the published index, so the module is populated and current on first load instead of 63 apps short and a rubric version behind. - Fixed the Track/Untrack button not updating on the app detail page. The detail view reads is_tracked from the ["ai-trust-index", "app", slug] query, but the track/untrack mutations only invalidated the "apps" and "tracked" keys. Added invalidation of the "app" key to useTrackApp, useUntrackApp, and useTrackAppsBulk so the detail page refetches and the button reflects the new state immediately. ## Why - The committed snapshot had drifted behind the website roster and rubric; fresh deploys would show stale, fewer apps until the first weekly sync. - On the detail page the toggle appeared to do nothing because the cached record never refreshed, even though the server-side change succeeded.
…e clamping ## Changes - Track/untrack/bulk-track/settings mutations now surface failures via a toast instead of failing silently. Added a shared useTrustIndexAlert helper and wired onError on every mutation across Browse, Tracked, app detail, and Settings. - Browse "Select all on page" now excludes already-tracked apps, so the bulk "Track selected (N)" count reflects only new tracks and never re-tracks an app. - The selection now clears whenever search, filters, sort, or page changes, so apps selected on a prior page/filter are no longer bulk-tracked off-screen. - Browse and Tracked clamp the page when the result set shrinks (filter change or untracking the last row of the last page), so the user is never stranded on an empty grid. - Per-row track/untrack buttons disable only the in-flight row instead of every card in the grid. - Corrected the Settings description: with no recipients configured, no digest is sent (the Admin/SuperAdmin fallback was removed earlier in this PR). ## Why Found during a verification pass after the detail-page Track button fix. These are the same family of interaction bugs: stale state, silent failures, and selection spanning views the user can no longer see.
…and docs ## Changes - App detail comparison strip and related-apps now render only after the full catalog has loaded, so they no longer flash "#0 of 0" and "—" while the catalog query is still in flight. - Related-apps favicons fall back to the app's initial when the icon service has no entry, matching the cards and the detail header (no broken-image icon). - Settings shows a brief "Saved" confirmation after a successful auto-save, so a successful save is distinguishable from nothing happening. - Updated the Settings user guide and page copy to match the removed Admin fallback: with no recipients configured, no digest is sent (digests go only to configured recipients). Also refreshed the digest example to reflect the new "what changed" format using the displayed grade. ## Why Final polish from the verification pass. The insights flash and missing favicon fallback were cosmetic gaps; the docs still described the old fallback behaviour.
…aw MUI Link Replace the one raw @mui/material Link in the app detail page with the standard VWLink component, matching the design-system convention used elsewhere. VWLink renders the external-link icon on hover, so the manual ExternalLink icon and its import are removed. No behavioural change — the policy link still opens in a new tab with rel="noopener noreferrer".
…k/untrack ## Changes - Added placeholderData: keepPreviousData to the useApps, useApp, and useTracked read queries. Track/untrack invalidates these keys, triggering a background refetch; without keepPreviousData the query data briefly became undefined and the detail page (notably the score meter) unmounted and re-mounted — the "moving progress bar" the user saw. Previous data now holds the UI steady while the fresh data loads. - Gave the detail-page Track/Untrack button a fixed min-width so toggling the label "Track" ⇄ "Untrack" no longer reflows the header row and nudges the score meter beside it. ## Verification Watched the meter's computed fill width across a track + untrack toggle: it holds a constant 521px (was jumping 531 → 545 before the fix). Stable in both directions.
The regenerated 205-app seed snapshot was written via JSON.stringify, which does not match Prettier's JSON style and failed the format-check CI gate. Reformatted with prettier --write; the data is unchanged (205 apps, feedVersion 2, rubric v2.0).
Following the empty-state standardization on develop (#4128), give every AI Trust Index empty state a contextual icon, and add guidance tips to the high-value "no tracked apps" state: - Browse: SearchX for "no filter matches", AlertTriangle for load errors. - Tracked: Star for the empty state, with EmptyStateTip cards explaining how to find apps in Browse and how the weekly digest works; AlertTriangle for errors. - App detail: SearchX for the not-found state. - Settings: Lock for the non-admin state. Added de/fr/es translations for the new tip strings and the expanded tracked message (i18n:audit:strict passes, 0 gaps).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Adds the AI Trust Index module: a curated, weekly-refreshed catalog of AI applications with privacy/governance grades that organizations can browse, track, and receive change alerts for. Tracking surfaces material changes (grade/score) to the apps an org cares about and emails a digest to configured recipients.
Changes
Backend
Frontend
Code-review fixes
\ % _) with explicitESCAPElast_good_countstored from the raw feed size, not the validated subsetApp detail page — parity with the public index
The in-app app detail page now mirrors the public AI Trust Index detail page, driven by the same data.
dataJSON already carries it). Apps without an indicator map degrade gracefully (the breakdown and silent/vague sections fall back to a short note; everything else still renders).Benefits