Repository navigation
perf(react): reuse incremental parsing in streams #397
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
onmax
wants to merge
4
commits into
comarkdown:main
Choose a base branch
from
onmax:fix/react-incremental-streaming
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
12d6c06
perf(react): reuse incremental parsing in streams
onmax 1a793ab
fix(react): preserve factory parsers and streamed links
onmax 5791510
Merge branch 'main' into fix/react-incremental-streaming
farnabaz 7ca19c5
refactor(react): use createSerializedMarkdownParser in MarkdownClient
farnabaz File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,188 @@ | ||
| import React, { act, Component } from 'react' | ||
| import { createRoot } from 'react-dom/client' | ||
| import { afterEach, beforeEach, describe, expect, it } from 'vitest' | ||
| import type { ComarkPlugin, MarkdownDocument } from 'comark' | ||
| import { MarkdownClient } from '../src/components/MarkdownClient' | ||
| import { Markdown } from '../src/components/Markdown' | ||
| import type { MarkdownProps } from '../src/components/Markdown' | ||
| import { defineMarkdownComponent } from '../src/index' | ||
|
|
||
| Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }) | ||
|
|
||
| let container: HTMLDivElement | ||
| let root: ReturnType<typeof createRoot> | ||
| beforeEach(() => { | ||
| container = document.createElement('div') | ||
| document.body.append(container) | ||
| root = createRoot(container) | ||
| }) | ||
| afterEach(async () => { | ||
| await act(async () => root.unmount()) | ||
| container.remove() | ||
| }) | ||
| async function render(props: MarkdownProps) { | ||
| await act(async () => { | ||
| root.render(<MarkdownClient {...props} />) | ||
| }) | ||
| } | ||
| function observe() { | ||
| const inputs: string[] = [] | ||
| const trees: MarkdownDocument[] = [] | ||
| const plugin: ComarkPlugin = { | ||
| name: 'observe-stream', | ||
| pre(state) { | ||
| inputs.push(state.markdown) | ||
| }, | ||
| post(state) { | ||
| trees.push(state.tree) | ||
| }, | ||
| } | ||
| return { inputs, trees, plugins: [plugin] } | ||
| } | ||
|
|
||
| describe('MarkdownClient streaming', () => { | ||
| it('reuses completed blocks and parses the whole input when streaming ends', async () => { | ||
| const probe = observe() | ||
| const first = '# Completed\n\nFirst paragraph.\n\nLast paragraph' | ||
| await render({ value: first, streaming: true, plugins: probe.plugins }) | ||
| const heading = probe.trees[0].nodes[0] | ||
| const next = first + ' grows' | ||
| await render({ value: next, streaming: true, plugins: probe.plugins }) | ||
| expect(probe.inputs[1]).not.toContain('# Completed') | ||
| expect(probe.trees[1].nodes[0]).toBe(heading) | ||
| expect(container.textContent).toContain('Last paragraph grows') | ||
|
|
||
| await render({ value: next, streaming: false, plugins: probe.plugins }) | ||
| expect(probe.inputs.at(-1)).toBe(next) | ||
| expect(probe.trees.at(-1)?.nodes[0]).not.toBe(heading) | ||
| expect(container.querySelector('h1')?.textContent).toBe('Completed') | ||
| }) | ||
|
|
||
| it('reuses completed blocks through the public Markdown wrapper', async () => { | ||
| const probe = observe() | ||
| const first = '# Completed\n\nFirst paragraph.\n\nLast paragraph' | ||
| await act(async () => { | ||
| root.render(await Markdown({ value: first, streaming: true, plugins: probe.plugins })) | ||
| }) | ||
| await act(async () => { | ||
| root.render(await Markdown({ value: first + ' grows', streaming: true, plugins: probe.plugins })) | ||
| }) | ||
| expect(probe.inputs[1]).not.toContain('# Completed') | ||
| expect(probe.trees[1].nodes[0]).toBe(probe.trees[0].nodes[0]) | ||
| expect(container.textContent).toContain('Last paragraph grows') | ||
| }) | ||
|
|
||
| it('recreates the parser for option and plugin changes with unchanged input', async () => { | ||
| const initial = observe() | ||
| const value = '# Heading\n\nParagraph' | ||
| await render({ value, streaming: true, plugins: initial.plugins }) | ||
| expect(container.querySelector('h1')?.id).toBe('heading') | ||
| await render({ value, streaming: true, plugins: initial.plugins, options: { headingIds: false } }) | ||
| expect(container.querySelector('h1')?.hasAttribute('id')).toBe(false) | ||
| const replacement = observe() | ||
| await render({ value, streaming: true, plugins: replacement.plugins }) | ||
| expect(replacement.inputs).toEqual([value]) | ||
| expect(container.querySelector('h1')?.id).toBe('heading') | ||
| }) | ||
|
|
||
| it.each([false, true])('reuses factory configuration with caller overrides: %s', async (override) => { | ||
| const probe = observe() | ||
| const Base = defineMarkdownComponent({ extends: MarkdownClient, plugins: probe.plugins, headingIds: false }) | ||
| const Defined = defineMarkdownComponent({ extends: Base }) | ||
| const props = override ? { options: { headingIds: true }, plugins: [{ name: 'caller' }] } : {} | ||
| const first = '# Completed\n\nFirst paragraph.\n\nLast paragraph' | ||
| async function update(value: string, config: Pick<MarkdownProps, 'options' | 'plugins'> = props) { | ||
| await act(async () => | ||
| root.render( | ||
| <Defined | ||
| value={value} | ||
| streaming | ||
| {...config} | ||
| /> | ||
| ) | ||
| ) | ||
| } | ||
| await update(first) | ||
| expect(container.querySelector('h1')?.hasAttribute('id')).toBe(override) | ||
| await update(first + ' grows') | ||
| expect(probe.inputs.at(-1)).not.toContain('# Completed') | ||
| expect(container.textContent).toContain('Last paragraph grows') | ||
|
|
||
| await update(first + ' grows', { ...props, options: { headingIds: !override } }) | ||
| expect(probe.inputs.at(-1)).toBe(first + ' grows') | ||
| expect(container.querySelector('h1')?.hasAttribute('id')).toBe(!override) | ||
| const replacement = observe() | ||
| replacement.plugins[0].name = 'replacement' | ||
| await update(first + ' grows', { ...props, plugins: replacement.plugins }) | ||
| expect(replacement.inputs).toEqual([first + ' grows']) | ||
| }) | ||
|
|
||
| it('applies unwrap changes and bypasses parsing for documents', async () => { | ||
| const probe = observe() | ||
| await render({ value: 'Paragraph', plugins: probe.plugins }) | ||
| expect(container.querySelector('p')).not.toBeNull() | ||
| await render({ value: 'Paragraph', plugins: probe.plugins, unwrap: true }) | ||
| expect(container.querySelector('p')).toBeNull() | ||
| const count = probe.inputs.length | ||
| await render({ value: { nodes: [['p', {}, 'Already parsed']], frontmatter: {}, meta: {} }, plugins: probe.plugins }) | ||
| expect(probe.inputs).toHaveLength(count) | ||
| expect(container.textContent).toBe('Already parsed') | ||
| }) | ||
|
|
||
| it('serializes overlapping plugin work and renders the latest update', async () => { | ||
| const { promise: gate, resolve: release } = Promise.withResolvers<void>() | ||
| let active = 0 | ||
| let maxActive = 0 | ||
| let calls = 0 | ||
| const plugins: ComarkPlugin[] = [ | ||
| { | ||
| name: 'deferred', | ||
| async pre() { | ||
| calls++ | ||
| active++ | ||
| maxActive = Math.max(maxActive, active) | ||
| if (calls === 1) await gate | ||
| active-- | ||
| }, | ||
| }, | ||
| ] | ||
| await render({ value: 'First', plugins, streaming: true }) | ||
| await render({ value: 'First grows', plugins, streaming: true }) | ||
| expect(calls).toBe(1) | ||
| await act(async () => release()) | ||
| expect(maxActive).toBe(1) | ||
| expect(container.textContent).toBe('First grows') | ||
| }) | ||
|
|
||
| it('delivers plugin errors to the error boundary', async () => { | ||
| class Boundary extends Component<{ children: React.ReactNode }, { error: boolean }> { | ||
| state = { error: false } | ||
| static getDerivedStateFromError() { | ||
| return { error: true } | ||
| } | ||
| render() { | ||
| return this.state.error ? <p>Parse failed</p> : this.props.children | ||
| } | ||
| } | ||
| const plugins: ComarkPlugin[] = [ | ||
| { | ||
| name: 'failure', | ||
| pre() { | ||
| throw new Error('plugin failed') | ||
| }, | ||
| }, | ||
| ] | ||
| await act(async () => { | ||
| root.render( | ||
| <Boundary> | ||
| <MarkdownClient | ||
| value="Text" | ||
| streaming | ||
| plugins={plugins} | ||
| /> | ||
| </Boundary> | ||
| ) | ||
| }) | ||
| expect(container.textContent).toBe('Parse failed') | ||
| }) | ||
| }) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,7 +1,21 @@ | ||
| import { defineConfig } from 'vitest/config' | ||
| import { playwright } from '@vitest/browser-playwright' | ||
|
|
||
| export default defineConfig({ | ||
| test: { | ||
| include: ['test/**/*.test.{ts,tsx}'], | ||
| projects: [ | ||
| { test: { name: 'server', include: ['test/**/*.test.{ts,tsx}'], exclude: ['test/**/*.browser.test.tsx'] } }, | ||
| { | ||
| test: { | ||
| name: 'client', | ||
| include: ['test/**/*.browser.test.tsx'], | ||
| browser: { | ||
| enabled: true, | ||
| provider: playwright(), | ||
| instances: [{ browser: 'chromium', headless: true }], | ||
| }, | ||
| }, | ||
| }, | ||
| ], | ||
| }, | ||
| }) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Use and test
MarkdownClientfor client-side streaming. Both examples render asyncMarkdownfrom client-side state. React does not support async client components. The browser test avoids that path by awaitingMarkdown(...)before rendering, so it cannot validate the examples. (react.dev)docs/content/3.rendering/5.react.md#L640-L640: useMarkdownClientin the streaming example and for its final update.packages/comark-react/README.md#L64-L64: useMarkdownClientin the streaming example.packages/comark-react/test/streaming.browser.test.tsx#L65-L65: render the documented client component as JSX rather than awaiting the wrapper function.📍 Affects 3 files
docs/content/3.rendering/5.react.md#L640-L640(this comment)packages/comark-react/README.md#L64-L64packages/comark-react/test/streaming.browser.test.tsx#L65-L65🤖 Prompt for AI Agents