Skip to content

chat: improve Chinese tool recall in large MCP catalogs - #872

Open
duhu2000 wants to merge 1 commit into
magpie-community:mainfrom
duhu2000:fix/chat-cjk-tool-recall
Open

duhu2000 wants to merge 1 commit into
magpie-community:mainfrom
duhu2000:fix/chat-cjk-tool-recall

Conversation

@duhu2000

@duhu2000 duhu2000 commented Nov 8, 2026

Copy link
Copy Markdown

Problem

Chat splits tool-search queries on non-alphanumeric/non-CJK characters. An unspaced Chinese request becomes a single long term and fails to match relevant tool descriptions. In desktop QCC OAuth acceptance, a natural-language historical-patent request missed the available tool; explicitly naming the tool succeeded.

Change

Extract Chinese bigrams alongside existing Latin identifiers, weight terms by catalog frequency to reduce repeated boilerplate, and prioritize explicitly named raw tool identifiers. Keep the 12-schema cap and bounded common-tool fallback. No QCC-specific routing, credentials, or queries are included.

This remains lexical selection, not semantic discovery or a guarantee that every tool is recalled. OAuth relay reuse is a separate issue.

Validation

  • Bun tests: 73 passed, 1 skipped, 0 failed across Chat, EPUB, and tool-selection tests (existing EPUB complete-book test skipped).
  • Five added regression cases include a synthetic 186-tool catalog, Chinese historical/current distinctions, exact identifiers, English queries, and fallback bounds.
  • Plugin check: registry.json and chat passed.
  • No live model/provider requests in these tests. No claim of all 186 real tools being exercised.

@yetone yetone left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. The Chinese recall fix works. With the old scoring swapped back in, your CJK test goes red. Two things are needed before this can merge:

  1. Bump packages/chat to 0.1.12. 0.1.11 is already published to the dist branch, so as it stands no user would ever get this change.
  2. Add tests for the other two rules, so each fails without its rule. I removed each rule by hand and all 5 tests still passed:
    • Without the +1000 boost for an exact rawName match, nothing goes red. A test needs a query that names a tool exactly and would rank it lower without the boost, for example a common word shared with many other tools.
    • With the IDF weight replaced by 1, nothing goes red. A test needs a query where a rare term must beat a term most tools contain.

Minor: the commit subject carries an internal ticket id ("QMCP-2014"). It means nothing in this repo, so please drop it when you squash.

What I ran, on a separate macOS box with a sandbox HOME:

  • packages/chat: bun test gave 73 pass and 1 skip.
  • bun scripts/check.mjs chat: ✓

I read every line of the diff and found no licensing, telemetry or vendor-text concerns.

@yetone

yetone commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Thanks, this looks right. I read every line and ran it at 6b7a528 merged onto main f379bf9 on a separate machine (bun 1.4.2, temp HOME): bun test packages/chat 73 pass, 1 skip, 0 fail; bun scripts/check.mjs ok. CI is green at 6b7a528.

One change before merge: bump packages/chat/package.json to 0.1.12. A package only ships when its version changes (page-chat is served from the dist branch), so without the bump users never get this. Past behaviour changes to chat bumped it in the PR (#631 → 0.1.10, #645 → 0.1.11).

Two small points, not blockers:

  • [a-z0-9_]+ now keeps list_issues as one term, where the old split gave list and issues. That's fine for exact names. The exact-name boost never fires for a tool name with a hyphen (get-patent), because - splits it. Allowing - in the Latin term regex would cover it.
  • The import sits above the file's header comment. Move it below.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants