feat(MarkdownRenderer): Add blocks memoization and parsing incomplete chunks

[Postamentovich]

Summary

This PR introduces performance optimizations for markdown rendering by implementing block-level memoization and adds support for parsing incomplete markdown chunks during streaming.

Changes

Performance Improvements

• Block-level memoization: Markdown content is now parsed into blocks and each block is cached individually, preventing unnecessary re-rendering of unchanged blocks when content is appended (e.g., during streaming)
• Component memoization: MarkdownRenderer is now wrapped with React.memo with custom comparison logic to prevent unnecessary re-renders when props haven't changed

New Features

• Incomplete markdown parsing: Added support for parsing incomplete markdown chunks using the remend library, which automatically closes unclosed markdown tags and structures
• New prop shouldParseIncompleteMarkdown: Added to MarkdownRenderer, AssistantMessage, UserMessage, MessageList, and ChatContainer components to enable incomplete markdown parsing

New Utilities and Hooks

• useMarkdownTransform hook: Handles markdown transformation with block-level caching
• useRemend hook: Wraps the remend library for closing incomplete markdown structures
• parse-blocks.ts utility: Parses markdown content into individual blocks using the marked lexer, handling HTML blocks and footnotes correctly

Dependencies

• Added marked (^17.0.1) - for markdown lexing and block parsing
• Added remend (^1.0.1) - for closing incomplete markdown structures

Technical Details

The implementation splits markdown content into blocks using the marked lexer. Each block is cached separately, so when new content is appended (e.g., during streaming), only new blocks are transformed while existing blocks are retrieved from cache. This significantly improves performance for long messages and streaming scenarios.

The block parsing logic:

• Handles HTML blocks correctly by tracking opening/closing tags
• Preserves footnote references and definitions as single blocks
• Merges related tokens into logical blocks

Screen.Recording.2025-12-24.at.17.54.49.mov

[gravity-ui-bot]

Preview is ready.