fix(kb): route plain text uploads to TextParser without markitdown - #9889
Open
xiaoyuyu6420 wants to merge 3 commits into
Open
fix(kb): route plain text uploads to TextParser without markitdown#9889xiaoyuyu6420 wants to merge 3 commits into
xiaoyuyu6420 wants to merge 3 commits into
Conversation
select_parser sent .txt/.md/.markdown to MarkitdownParser, which imports the optional markitdown-no-magika dependency. When that dependency is not installed, uploading a plain text file raised ModuleNotFoundError that was swallowed into a generic "文档解析失败" message, with no hint about the missing dependency (issue AstrBotDevs#9598). Plain text formats are decoded directly by the existing TextParser, which needs no extra dependency and supports multiple encodings (utf-8/gbk/...). Route .txt/.md/.markdown to TextParser; keep .rst/.adoc/.xlsx/.docx/.xls on MarkitdownParser since those rely on conversion. Chunking in kb_helper still selects MarkdownChunker by file extension, so markdown heading hierarchy is unchanged. Fixes AstrBotDevs#9598
Contributor
There was a problem hiding this comment.
Hey - I've found 1 issue
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="tests/unit/test_parser_selection.py" line_range="3" />
<code_context>
+import pytest
+
+from astrbot.core.knowledge_base.parsers.markitdown_parser import MarkitdownParser
+from astrbot.core.knowledge_base.parsers.text_parser import TextParser
+from astrbot.core.knowledge_base.parsers.util import select_parser
</code_context>
<issue_to_address>
**issue (testing):** The test module imports `MarkitdownParser` at collection time, so `ModuleNotFoundError` is raised before any test runs when the optional `markitdown-no-magika` dependency is absent—the exact deployment scenario this change is intended to support.
**Triggers:** When running the test suite in an installation without `markitdown-no-magika`.
**Suggested fix:** Import `MarkitdownParser` lazily inside the rich-format test and skip that test when the optional dependency is unavailable, or mock the parser module during selection testing.
</issue_to_address>Sourcery assessment
Needs a human reviewer. 1 finding to address first, and if the routing is wrong, .txt or Markdown files could be parsed into incorrect knowledge-base content and embeddings, leaving bad indexed records after a revert. The records are bounded and can be cleared or regenerated, so reverting plus re-ingestion repairs the effect.
Blocking findings: tests/unit/test_parser_selection.py:3
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
… optional dep Sourcery flagged that importing MarkitdownParser at module scope raises ModuleNotFoundError during test collection when markitdown-no-magika is absent - exactly the deployment scenario this change supports. Import it inside the rich-format test and skip that test when the optional dependency is unavailable, so the plain-text selection tests still run.
Soulter
approved these changes
Sep 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #9598
Problem
select_parsersent.txt/.md/.markdowntoMarkitdownParser, which imports the optionalmarkitdown-no-magikadependency at module level. When that dependency is not installed (e.g. manual deployments), uploading a plain text file raisesModuleNotFoundError, whichkb_helperswallows into a generic "文档解析失败" message. The user gets no hint that the root cause is a missing optional dependency, and plain text uploads simply fail.Fix
Plain text formats are decoded directly by the existing
TextParser, which needs no extra dependency and auto-detects multiple encodings (utf-8/gbk/gb2312/gb18030):.txt/.md/.markdown→TextParser.rst/.adoc/.xlsx/.docx/.xls→MarkitdownParser(unchanged; these rely on conversion)Chunking in
kb_helperalready selectsMarkdownChunkerby file extension, so markdown heading hierarchy is preserved — the parser swap does not change downstream behavior for.mdfiles.Testing
uv run pytest tests/unit/test_parser_selection.py— 3 passed (new)uv run pytest tests/test_epub_parser.py tests/unit/test_kb_upload_atomicity.py tests/unit/test_parser_selection.py— 26 passedruff format/ruff checkcleanSummary by Sourcery
Bug Fixes: