Render rich content components in the transcript

[giannagerton]

Summary

RFC

Renders the components an agent sends inline in the transcript, starting with an item card: an identifier and title, plus optional subtitle, image, description, price, and up to three buttons. Buttons send a follow-up message as the user.

Builds on the onRichContent callback from the SDK PR. The widget appends each component to the transcript as it arrives, and renders anything it doesn't recognize as a short notice rather than breaking the turn.

Nothing is visible until the backend can send rich_content, which isn't merged yet. Open to edit the design and props. Just wanted to get this out there as a first pass.

Preview of the item card with example data

---

Note

Medium Risk
New user-facing transcript rendering and agent-driven UI (images, buttons sending messages) with validation mitigations; depends on SDK/backend rich_content delivery not yet merged.

Overview
Adds inline rich content in the convai widget transcript when the agent sends rich_content events (via SDK onRichContent). The first supported component is item_card: title, optional subtitle, image, description, price, and up to three buttons that call sendUserMessage with preset text.

Transcript pipeline: a new rich_content entry type is appended on arrival with deduplication by richContentId. buildDisplayTranscript passes these through in order (cards stay where they arrived; empty agent placeholders can be dropped when a card follows). TranscriptMessage routes rich entries to RichContentRenderer.

Safety: parseItemCardProps validates and normalizes agent props (text limits, https / data:image URLs only, button caps). Unknown or invalid components show the configurable rich_content_unavailable notice instead of breaking the turn.

Tests & mocks: validation unit tests, display-transcript scenarios for ordering with tools/typing, and a rich_content mock agent for manual/dev testing.

^{Reviewed by Cursor Bugbot for commit b380126. Bugbot is set up for automated code reviews on this repo. Configure here.}

[giannagerton]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• Render rich content components in the transcript #915 👈 (View in Graphite)
• Add onRichContent callback #912
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[cursor]

Image URLs truncated to 500 chars

Medium Severity

parseImageUrl runs URLs through parseText, which slices to MAX_TEXT_LENGTH (500). Signed CDN/S3 URLs and real data:image payloads routinely exceed that, so valid images are returned truncated and fail to load while the card still renders.

Additional Locations (1)

• packages/convai-widget-core/src/rich-content/validate.ts#L9-L23

 

^{Reviewed by Cursor Bugbot for commit 4d33abe. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit b380126. Configure here.}

[cursor]

Card buttons skip scroll pin

Medium Severity

Card button clicks call sendUserMessage without setting scrollPinned to true. The text input path does pin scroll before send, so after scrolling up to tap a card the follow-up user message and agent reply can stay off-screen.

 

^{Reviewed by Cursor Bugbot for commit b380126. Configure here.}