Skip to content

feat: Add StreamingMarkdownDecoder for LLM streaming scenarios - #19

Closed
Feiyu-Liu wants to merge 1 commit into
DoctorinaAI:masterfrom
Feiyu-Liu:steaming
Closed

feat: Add StreamingMarkdownDecoder for LLM streaming scenarios#19
Feiyu-Liu wants to merge 1 commit into
DoctorinaAI:masterfrom
Feiyu-Liu:steaming

Conversation

@Feiyu-Liu

Copy link
Copy Markdown

Summary

Adds StreamingMarkdownDecoder for efficient incremental parsing of LLM streaming output.

Key Features

  • Line-level state tracking to avoid re-parsing closed blocks
  • Shared parsing logic between MarkdownDecoder and StreamingMarkdownDecoder
  • 2-4x performance improvement for streaming scenarios

Changes

  • lib/src/parser.dart: Add StreamingMarkdownDecoder and refactor shared parsing logic
  • benchmark/streaming_performance_benchmark.dart: Add performance benchmark

Benchmark Results

  • Small content (50 lines): 2.84x faster
  • Medium content (200 lines): 1.97x faster
  • Large content (500 lines): 3.67x faster

Testing

All 80 tests pass including 18 new streaming-specific tests.

@mike-doctorina

mike-doctorina commented Jan 14, 2026

Copy link
Copy Markdown
Collaborator

@Feiyu-Liu because this is package, update, please, version at pubspec.yaml and changelog.md as well.
And fix formatting at lib/src/parser.dart as well

@mike-doctorina

Copy link
Copy Markdown
Collaborator

Thanks for this, @Feiyu-Liu — streaming/incremental parsing for LLM output is definitely something we want. 🙏

I'm intentionally not merging it yet: a larger refactor of the rendering layer (and how incremental parses feed into it) is coming next, and StreamingMarkdownDecoder overlaps that surface heavily. I'd rather land it on top of the new architecture than merge now and rework it immediately. Keeping this open and will revisit right after the render refactor. Appreciate your patience!

@mike-doctorina

Copy link
Copy Markdown
Collaborator

Implemented here #26

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants