Summary
Add first-class inline HTML previews in chat: fenced ```html-preview code blocks in assistant messages render as sandboxed, auto-sized, interactive figures inline in the transcript — plus system-prompt guidance that teaches the agent to use them well (text explains, visuals demonstrate).
This is implemented and validated in a downstream fork; filing to track landing it upstream.
Motivation
Long analytical assistant responses (post-mortems, design decisions, multi-finding reviews) read as walls of prose. Mermaid helped for static structure, but cannot express the things that actually earn a visual: state change, time, interaction, quantities, and UI mockups (e.g. "throw the error and watch it get lost", "drag through releases until the version-skew crash", "find your own cost crossover point").
pi-web already has all the machinery: HTML artifacts preview inline in a sandboxed iframe. But artifacts require file juggling, don't stream with the message, and pull the reader out of the conversation flow. A fenced block keeps the figure's source in the transcript (quotable, diffable) and renders it in place.
Design
Renderer (src/markdown/render.ts):
```html-preview fences are enhanced after markdown render (same pattern as Mermaid): the <pre> is wrapped in a .htmlPreview container with an iframe[srcdoc].
- Security: identical model to HTML artifact previews —
sandbox="allow-scripts" without allow-same-origin, so scripts run in an opaque origin with no access to the app origin, its storage, or APIs.
- Auto-sizing handshake: a small script is appended to the srcdoc that reports content height and intrinsic width (right edge of widest body child + body right padding/margin) via
postMessage. The parent matches frames by contentWindow identity and clamps height to min(70vh, 720px).
- Shrink-wrap + interleaving: the container narrows to the reported content width, so small intrinsically-sized figures (
display:inline-block, flex rows) sit inside prose like figures instead of stretching full-width. Previews ≤140px tall get a compact style (lighter chrome, tighter margins) via .htmlPreview--compact.
- Source toggle: a hover button flips between rendered preview and the original code block (re-tagged
language-html for highlighting, keeps the copy button; source view restores full message width).
- Max source size 50 KB; oversized or empty blocks stay as plain code.
Guidance (contexts/web-ui.md, injected UI context): a "Rich visual responses" section encoding the usage standard we converged on:
- Conclusion first — decision/answer in ≤2 sentences at the top.
- Text explains, visuals demonstrate — 2–3 sentences of what/why per concept, then one small interactive figure right after its prose; words inside figures are labels (≤6 words), never narrative.
- One concept, one widget — interleave; no monolithic multi-tab explorables.
- Escalation ladder — markdown/tables → Mermaid for static structure ("if it could be a Mermaid diagram, it must be") →
html-preview only for state/time/interaction/quantities/mockups. Code stays code; checklists stay lists.
Plus a genre table (incident → simulation; design decision → comparison table + scrubbable tradeoff; architecture → Mermaid; review → lists) and a compact embedded exemplar (a control that changes an animated outcome) for the model to imitate.
Implementation (downstream)
| File |
Change |
src/markdown/render.ts |
enhanceInlineHtmlPreviews() + resize/width postMessage listener + source toggle |
src/styles/messages.css |
.htmlPreview, .htmlPreviewFrame, .htmlPreviewToggle, --compact variant, [hidden] handling |
server/mock.ts |
mock reply exercising scripts + the sizing handshake |
tests/e2e/pi-web.spec.ts |
e2e: renders sandboxed, script executed inside frame, auto-height growth, shrink-wrap width, compact class, source toggle round-trip — passing on desktop/tablet/mobile |
contexts/web-ui.md |
usage standard described above |
Notes / limitations
- Previews render when the completed message is markdown-rendered (streaming shows the raw fence, same as Mermaid).
- One
message listener total; frames tracked in a Set, pruned on disconnect.
srcdoc + injected reporter script means full-document snippets (<html>…</html>) work; the reporter appended after </html> is tolerated by browsers.
Happy to open a PR with the downstream implementation.
Summary
Add first-class inline HTML previews in chat: fenced
```html-previewcode blocks in assistant messages render as sandboxed, auto-sized, interactive figures inline in the transcript — plus system-prompt guidance that teaches the agent to use them well (text explains, visuals demonstrate).This is implemented and validated in a downstream fork; filing to track landing it upstream.
Motivation
Long analytical assistant responses (post-mortems, design decisions, multi-finding reviews) read as walls of prose. Mermaid helped for static structure, but cannot express the things that actually earn a visual: state change, time, interaction, quantities, and UI mockups (e.g. "throw the error and watch it get lost", "drag through releases until the version-skew crash", "find your own cost crossover point").
pi-web already has all the machinery: HTML artifacts preview inline in a sandboxed iframe. But artifacts require file juggling, don't stream with the message, and pull the reader out of the conversation flow. A fenced block keeps the figure's source in the transcript (quotable, diffable) and renders it in place.
Design
Renderer (
src/markdown/render.ts):```html-previewfences are enhanced after markdown render (same pattern as Mermaid): the<pre>is wrapped in a.htmlPreviewcontainer with aniframe[srcdoc].sandbox="allow-scripts"withoutallow-same-origin, so scripts run in an opaque origin with no access to the app origin, its storage, or APIs.postMessage. The parent matches frames bycontentWindowidentity and clamps height tomin(70vh, 720px).display:inline-block, flex rows) sit inside prose like figures instead of stretching full-width. Previews ≤140px tall get a compact style (lighter chrome, tighter margins) via.htmlPreview--compact.language-htmlfor highlighting, keeps the copy button; source view restores full message width).Guidance (
contexts/web-ui.md, injected UI context): a "Rich visual responses" section encoding the usage standard we converged on:html-previewonly for state/time/interaction/quantities/mockups. Code stays code; checklists stay lists.Plus a genre table (incident → simulation; design decision → comparison table + scrubbable tradeoff; architecture → Mermaid; review → lists) and a compact embedded exemplar (a control that changes an animated outcome) for the model to imitate.
Implementation (downstream)
src/markdown/render.tsenhanceInlineHtmlPreviews()+ resize/widthpostMessagelistener + source togglesrc/styles/messages.css.htmlPreview,.htmlPreviewFrame,.htmlPreviewToggle,--compactvariant,[hidden]handlingserver/mock.tstests/e2e/pi-web.spec.tscontexts/web-ui.mdNotes / limitations
messagelistener total; frames tracked in aSet, pruned on disconnect.srcdoc+ injected reporter script means full-document snippets (<html>…</html>) work; the reporter appended after</html>is tolerated by browsers.Happy to open a PR with the downstream implementation.