data-chat-mini: MotherDuck guides as the context engine (+ guide manager UI) by matsonj · Pull Request #79 · motherduckdb/labs

matsonj · 2026-07-08T19:58:20Z

Replaces the IndexedDB context-layer placeholder with MotherDuck guides as the real context engine, adds the guide manager UI, and prepares the app for the production MCP rollout.

Production launch state

Production MCP source and production E2E coverage now include the guide subsystem (get_guide, list_guides, the full guide lifecycle) plus list_views and list_macros.
The app now defaults to https://api.motherduck.com instead of staging.
npm run mcp:validate checks all 16 production tools the app consumes, their required input fields, and live get_guide("guides.md") / list_guides reads.
A production user-scoped PAT with guide entitlement is required for the full experience. The currently configured Vercel PAT authenticates to production catalog tools but production returns Tool create_guide not found for it; deployment was rolled back pending the entitlement/token fix.
Vercel deployment protection is already scoped to all deployments; the app remains a protected single-identity deployment.

Context engine

Reads get_guide("guides.md") before SQL and uses relevant org/personal guides as durable context.
Allows create_guide, update_guide, and edit_guide_content, guarded to personal users/<username>/... paths.
Keeps data read-only: query_rw is not advertised or dispatched.
Detects both MCP isError and { success: false } guide-write failures.

Guide panel and manager

/api/guides supports list/read/version history and guarded personal create/update/delete operations.
The sidebar renders MCP guides and provides markdown view/edit/create/history/reference management.
Org-wide mutation and promotion remain blocked from the unauthenticated app route.

Verification

npm test: 91/91
npm run typecheck: clean
npm run lint: clean
npm run build: clean on Node 24.12.0
npm run audit: 0 vulnerabilities
Production MCP source contract checked at motherduckdb/mono@9ec82b57; a protected production deploy/smoke test confirmed the current Vercel PAT lacks guide tools, and the alias was restored to dpl_6wkP2vawYt7cJoWSYcjQsbCfpM4Q.

Deferred

Re-record the legacy IndexedDB-based workshop demo around guides, then delete the dead context round-trip plumbing. Tracked in docs/context-layer-guide-migration.md.

…e manager UI) Replaces the IndexedDB context-layer placeholder with the MotherDuck guide subsystem as the real context engine, and adds human-driven guide management. Context engine (read + write via MCP guides): - Allowlist guide-write tools (create/update/edit_guide_content) with a personal-only guard (assertGuideWriteAllowed); data stays read-only. - System prompt switches from the local query_context_layer Step-0 protocol to the guide protocol: read guides.md first; persist durable learnings as small personal guides under users/<username>/. - Chat route stops advertising the local CONTEXT_TOOLS; guide tools dispatch like any MCP tool. The legacy round-trip plumbing stays only for the demo harness pending a re-record (docs/context-layer-guide-migration.md). Guide panel + manager: - New /api/guides route: list/get + POST(create), PATCH(update_guide + set_guide_access + update_guide_metadata, per-step errors), DELETE, ?version=N. - Sidebar "Guides" panel renders MCP guides; clicking a card opens a centered markdown popover (no chevron) with view / raw-markdown edit / create / version history / promote-to-org / delete / opt-in references editor. Notes / known limits: - The app's read-scaling token can edit existing guides and promote-to-org, but create + demote-to-personal need a username claim it lacks — fix is a user-scoped/OAuth token (integration-plan open question). - Demo validation kept green on the existing transcript; full guide-based re-record deferred (see docs/context-layer-guide-migration.md). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

vercel · 2026-07-08T19:58:27Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
postgres-vs-motherduck	Ready	Preview, Comment	Jul 15, 2026 8:31pm

matsonj · 2026-07-08T20:02:33Z

+      `${name}: this app may only write personal guides — set access:"user" (org-wide guides are admin-only).`,
+    );
+  }
+  if (name === 'create_guide') {


[P1] Enforce the personal-guide boundary for every model-visible guide write. update_guide and edit_guide_content are now allowlisted for the agent, but this guard only checks the path when name === "create_guide"; an update/edit call can still target revenue-billing/foo.md or any other org guide. The PR notes that this app token can edit existing guides, so relying on MCP-side ownership is not a safe backstop here. Please require update/edit targets to resolve to users/<current-user>/... (and reject or resolve id targets before dispatch), or keep these tools out of the model allowlist.

Fixed in cef565c. assertGuideWriteAllowed now guards every guide-write tool (create/update/edit_guide_content), not just create: it requires a users/… path and rejects id-based targeting (can't be namespace-checked pre-dispatch). An update_guide/edit_guide_content aimed at revenue-billing/foo.md or an opaque id is now rejected before dispatch, so we no longer lean on MCP-side ownership for org guides.

matsonj · 2026-07-08T20:02:41Z

+  name: string,
+  args: Record<string, unknown>,
+): Promise<{ ok: true; data: Record<string, unknown> } | { ok: false; error: string }> {
+  const { text, isError } = await executeToolWithStatus(client, name, args, true);


[P1] Add route-level authorization before using the internal MCP bypass. Every browser request to this route runs with the server MotherDuck token, but PATCH can call set_guide_access / update_guide_metadata and DELETE can call delete_guide for an arbitrary path; the UI also exposes promote/delete for org guides. With no app auth or role check in front of this route, any app visitor can mutate or soft-delete shared guide content if the token has that permission. Please gate mutating verbs to the current user personal namespace, or require an authenticated admin path before calling internal guide-write tools.

Fixed in cef565c. Added a route-level rejectNonPersonal(...) gate that runs before the internal MCP bypass on every mutating verb — POST, PATCH (incl. set_guide_access / update_guide_metadata and the newPath rename target), and DELETE. Non-users/… paths get a 403 with no MCP call. Verified live: DELETE/promote on revenue-billing/awr-definition.md and a rename into revenue-billing/ all return 403. (The app itself still relies on deployment-level protection for general access; this confines the blast radius to the personal namespace so shared org guides can't be mutated/deleted from the app. Org-guide management needs an admin/OAuth path out of band.)

matsonj · 2026-07-08T20:03:04Z

+  // Personal-guide sandbox: block org-wide / non-users writes before dispatch.
+  assertGuideWriteAllowed(name, args);
  const result = await client.callTool({ name, arguments: args }, undefined, requestOptions);
  if (result.structuredContent != null) {


[P2] Wire guide-write failures into the chat dispatch failure detector. executeToolWithStatus will stringify structured { success: false, error } payloads from create_guide, update_guide, and edit_guide_content, but detectPayloadFailure still only recognizes the old context/dive write tools. In chat, the documented no authenticated username create failure would currently produce a successful tool_end, refresh the guide panel, and let the model tell the user that durable context was saved when nothing landed. Please add the guide write tool names to that detector and cover one false-success case in tests.

Fixed in cef565c. Added create_guide / update_guide / edit_guide_content to SUCCESS_FIELD_TOOLS, so detectPayloadFailure now catches their { success:false, error } payloads. The no authenticated username create failure now produces an error tool_end (and the panel-refresh is gated on !tc.error, so it no longer fires on failure). Added lib/tool-invocation.test.ts covering the false-success case + the passthrough cases.

…namespace + detect write failures - assertGuideWriteAllowed now guards EVERY guide-write tool (create/update/ edit_guide_content), not just create: requires a users/… path and rejects id-based targeting, so the model can't edit org guides. (P1) - /api/guides gates all mutating verbs (POST/PATCH/DELETE, incl. set_guide_access, update_guide_metadata, delete_guide, and rename newPath) to the personal namespace before the internal MCP bypass — 403 otherwise. Closes the hole where any app visitor could mutate/delete shared org guides. (P1) - detectPayloadFailure recognizes create_guide/update_guide/edit_guide_content, so a { success:false } write (e.g. "no authenticated username") surfaces as a real tool error instead of a false success + spurious panel refresh. Added lib/tool-invocation.test.ts covering the false-success case. (P2) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

matsonj · 2026-07-08T20:19:23Z

+      }
+
+      if (typeof body.access === 'string') {
+        const res = await callWrite(client, 'set_guide_access', { path, access: body.access });


[P1] Block org visibility changes from the unauthenticated guide route. The new guard confines the path to users/..., but PATCH still forwards access: "organization" to set_guide_access for those personal paths, and the sidebar still exposes “Promote to org-wide”. That means any app visitor who can hit this route can publish guide content into the org-wide context layer with the server token, even though the route comment says org guides are read-only from the app. Please reject access: "organization" here (and hide/disable the promote action) until there is an authenticated admin/OAuth path for org-guide management.

Fixed in f67d603. PATCH /api/guides now rejects access:"organization" with a 403 before any MCP call (demote to "user" — de-escalation — is still allowed). The sidebar "Promote to org-wide" control is disabled (kept visible with a tooltip pointing at the admin/OAuth path) and the setAccess handler is gone. Added app/api/guides/route.test.ts covering the org-promote block, the non-personal-path block, and the rename-into-org block. Verified live: PATCH {path:"users/matson/nba/nba-box-scores-v2.md", access:"organization"} → 403, guide still access:user.

…d route The personal-namespace path gate still allowed PATCH { access: "organization" } on a users/… guide, so any visitor could publish personal guide content into the org-wide context layer. - /api/guides PATCH now rejects access:"organization" with 403 before any MCP call (demote to "user" still allowed). Added route tests for the org-promote block, the non-personal-path block, and the rename-into-org block. - Sidebar popover disables the "Promote to org-wide" control (kept visible with a tooltip) and drops the now-unreachable setAccess handler. Org visibility needs an authenticated admin/OAuth path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

matsonj · 2026-07-08T20:31:49Z

✅ Approved — holding merge until the guides subsystem is live on the prod MotherDuck MCP.

This branch targets staging, where get_guide/list_guides + the guide-write tools exist. Prod-readiness gate before merge:

Guides on prod MCP — until then, an app pointed at prod degrades gracefully (guide tools simply aren't advertised → no context layer, no errors). Merge once prod serves guides so the feature is actually live.
Write token — create / demote / agent auto-saves need a user-scoped or OAuth token (the read-scaling PAT has no username claim). Reads, edits of existing guides, and the (now-blocked) promote path don't. Worth resolving before relying on the write side in prod.
Deferred (tracked in docs/context-layer-guide-migration.md): re-record the guide-based demo, then delete the dead IndexedDB round-trip.

No code changes pending; safe to merge as-is once (1) lands.

vercel Bot deployed to Preview July 8, 2026 19:58 View deployment

matsonj commented Jul 8, 2026

View reviewed changes

vercel Bot deployed to Preview July 8, 2026 20:14 View deployment

matsonj commented Jul 8, 2026

View reviewed changes

vercel Bot deployed to Preview July 8, 2026 20:26 View deployment

matsonj marked this pull request as draft July 8, 2026 20:32

data-chat-mini: prepare production MCP launch

62ea595

vercel Bot deployed to Preview July 15, 2026 19:50 View deployment

data-chat-mini: surface missing guide tools

271cbe0

vercel Bot deployed to Preview July 15, 2026 20:31 View deployment

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

data-chat-mini: MotherDuck guides as the context engine (+ guide manager UI)#79

data-chat-mini: MotherDuck guides as the context engine (+ guide manager UI)#79
matsonj wants to merge 5 commits into
mainfrom
data-chat-mini-work

matsonj commented Jul 8, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Jul 8, 2026 •

edited

Loading

Uh oh!

matsonj Jul 8, 2026

Uh oh!

matsonj Jul 8, 2026

Uh oh!

matsonj Jul 8, 2026

Uh oh!

matsonj Jul 8, 2026

Uh oh!

matsonj Jul 8, 2026

Uh oh!

matsonj Jul 8, 2026

Uh oh!

matsonj Jul 8, 2026

Uh oh!

matsonj Jul 8, 2026

Uh oh!

matsonj commented Jul 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

matsonj commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Production launch state

Context engine

Guide panel and manager

Verification

Deferred

Uh oh!

vercel Bot commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

matsonj commented Jul 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

matsonj commented Jul 8, 2026 •

edited

Loading

vercel Bot commented Jul 8, 2026 •

edited

Loading