fix: route txt/md KB uploads to TextParser and clarify missing markitdown - #9920
Draft
lxfight wants to merge 1 commit into
Draft
fix: route txt/md KB uploads to TextParser and clarify missing markitdown#9920lxfight wants to merge 1 commit into
lxfight wants to merge 1 commit into
Conversation
…down txt/md/markdown are plain text and were routed to MarkitdownParser, which requires markitdown-no-magika. Without that package the upload failed with a generic parse error that never mentioned the missing dependency, and TextParser existed but was never used. Route plain-text formats to TextParser, and when markitdown-no-magika is missing for office formats, raise KnowledgeBaseUploadError with an explicit install hint instead of a wrapped ModuleNotFoundError.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #9598
.txt/.md/.markdownuploads were routed toMarkitdownParser, which importsmarkitdown-no-magikaat module level. On environments where that package is absent, uploading a txt file failed with a generic "文档解析失败:无法读取或解析上传文件" error that never mentioned the actual cause (ModuleNotFoundError). MeanwhileTextParser— a dependency-free plain-text parser explicitly written for TXT/MD — existed but was never used byselect_parser.Modifications / 改动点
astrbot/core/knowledge_base/parsers/util.py:.txt/.md/.markdowntoTextParser(no external dependency needed; behavior is unchanged since markitdown just passed these through)..rst/.adoc/.xlsx/.docx/.xlsonMarkitdownParser, but catchImportErrorat selection time and raiseKnowledgeBaseUploadErrorwith an explicit install hint (pip install 'markitdown-no-magika[docx,xls,xlsx]').kb_helperre-raisesKnowledgeBaseUploadErroruntouched, so users now see the real cause instead of the misleading "file corrupted" message.tests/test_kb_select_parser.py: cover routing for all extensions, unsupported extension error, and the missing-dependency error path (simulated viasys.modules).This is NOT a breaking change. / 这不是一个破坏性变更。
Screenshots or Test Results / 运行截图或测试结果
uv run ruff format/uv run ruff checkpass.markitdown-no-magika, upload a.txtfile to a knowledge base — it now parses and indexes successfully; uploading a.docxin the same environment reports the missing-dependency hint instead of a generic parse failure.Checklist / 检查清单
😊 If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
/ 如果 PR 中有新加入的功能,已经通过 Issue / 邮件等方式和作者讨论过。
👀 My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above.
/ 我的更改经过了良好的测试,并已在上方提供了“验证步骤”和“运行截图”。
🤓 I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations in
requirements.txtandpyproject.toml./ 我确保没有引入新依赖库,或者引入了新依赖库的同时将其添加到
requirements.txt和pyproject.toml文件相应位置。😮 My changes do not introduce malicious code.
/ 我的更改没有引入恶意代码。
Summary by Sourcery
Ensure knowledge-base uploads use the appropriate parser and expose actionable errors for missing optional dependencies.
Bug Fixes:
Tests: