Two small, independent plug-ins for LLM agentic systems:
tool_curator/— a hybrid retrieval engine that narrows a large tool registry down to the ~15–20 tools relevant to one query, before those tool definitions ever reach the LLM's context window.multilingual_normalizer/— a provider-agnostic translation stage that keepstool_curator's English-trained retrieval working when queries aren't in English.
Both ship in this one repo but neither depends on the other, or on MCP, FastMCP, or any specific LLM SDK — they operate on plain (name, description) pairs and query strings. example/ is a fully working reference deployment (a simulated university management system, 191 tools across 8 MCP servers) used to validate both plug-ins against a 281-case hand-labeled evaluation suite — a proof point and a template, not a dependency.
For architecture details, configuration, the API reference, and the full evaluation methodology and results, visit:
https://Yashwanth-R19.github.io/rag-tool-curator/
This README covers install and the shape of the repo; everything else lives on the docs site so it isn't duplicated in two places.
LLM tool-calling accuracy degrades measurably once the available tool count passes roughly 20–30, and drops sharply beyond a few hundred. Independent of accuracy, every tool declaration in a prompt costs tokens — a registry of 3,000 tools at ~200 tokens each is 600,000 tokens per request, more than the context window of essentially any production model.
Measured impact (reference deployment, 281-case eval suite):
| Tool pool size | Without curator | With curator (top-15) | Token reduction |
|---|---|---|---|
| 87 tools (student persona) | ~17,400 tokens/query | ~3,000 tokens/query | 83% |
| 191 tools (admin persona) | ~38,200 tokens/query | ~3,000 tokens/query | 92% |
| 3,000 tools (enterprise scale) | ~600,000 tokens/query (exceeds context window) | ~4,000 tokens/query | 99.3% |
git clone https://github.com/Yashwanth-R19/rag-tool-curator.git
cd rag-tool-curator
pip install -e . # tool_curator only
pip install -e ".[normalizer]" # + multilingual_normalizer's Gemini backendOr copy tool_curator/ and/or multilingual_normalizer/ directly into your project — neither has any import reaching outside its own folder. See Installation for both paths in full, including running the bundled reference example.
Requirements: Python 3.10+. No GPU required — all retrieval and the language fast-path run on CPU.
from tool_curator import get_curator, load_tool_context
from multilingual_normalizer import get_normalizer
curator = get_curator(mode="rag", top_k=15, index_dir=".tool_cache")
normalizer = get_normalizer(provider="gemini", api_key="...")
# your_tools: list[tuple[str, str]] — (tool_name, tool_description) pairs,
# already filtered to whatever your own access control allows.
curator.build_index(your_tools, extra_context=load_tool_context("domain_synonyms.json"))
# Per message:
english_query, language = normalizer.normalize(user_message)
selected = curator.select(english_query, your_tools)Full walkthrough: Quickstart.
.
├── tool_curator/ # Reusable retrieval engine — domain-agnostic, pip-installable
│ ├── __init__.py # get_curator(), load_tool_context(),
│ │ # load_dependency_rules(), apply_dependency_rules()
│ ├── curator_interface.py # BaseToolCurator abstract interface
│ ├── hybrid_rag_curator.py # FAISS + BM25 + RRF implementation
│ ├── suggestion_validator.py # detects LLM-hallucinated tool names in free-text responses
│ └── terminal_display.py # minimal, dependency-light curator selection logging
│
├── multilingual_normalizer/ # Reusable query-language normalizer — provider-agnostic
│ ├── __init__.py # get_normalizer() factory
│ ├── normalizer_interface.py # BaseNormalizer abstract interface + fast-path + cache
│ ├── gemini_normalizer.py # GeminiNormalizer — the default shipped backend
│ └── fastpath.py # majority-vocabulary English fast-path heuristic
│
├── accuracy_boosters/ # Annotated templates for domain-specific tuning
│ ├── tool_context.template.json
│ ├── dependency_rules.template.json
│ └── equivalences.template.json
│
├── example/ # Full reference deployment (not required to use either plug-in)
│ ├── main.py # Entry point — prompts for persona/user, starts the session
│ ├── settings.py # Configuration: curator/normalizer params, server registry
│ ├── domain_synonyms.json # University-domain synonym set
│ ├── dependency_rules.json # University-domain companion-tool rules
│ ├── equivalences.json # Eval-only covers/aliases for --lenient scoring
│ │
│ ├── orchestrator/ # Application-level orchestration — not part of either plug-in
│ │ ├── agent_orchestrator.py # Main conversation loop: MCP servers, curator + normalizer per turn
│ │ ├── persona_access_policy.py # Role-based tool visibility (student/faculty/admin/executive)
│ │ ├── mutation_guard.py # Confirmation gate for state-mutating tool calls
│ │ ├── hallucination_guard.py # Fuzzy correction for malformed tool names from the LLM
│ │ ├── usage_logger.py # Per-turn and per-session token usage tracking
│ │ └── terminal_display.py # Full rich-terminal UI for the reference deployment
│ │
│ ├── mcp_servers/ # 8 FastMCP servers exposing 191 tools total
│ ├── mock_db/ # In-memory mock dataset shared by all 8 servers
│ ├── eval/
│ │ ├── collect_tools.py # Snapshots (name, description) pairs from all 8 servers, once
│ │ ├── curator_eval.py # hit@k / recall@k against the labeled eval sheet, no live LLM
│ │ └── MCP_Evaluation_Sheet_-_TF_IDF_Tool_Curator.xlsx
│ └── requirements.txt
│
├── docs/ # Jekyll site — https://Yashwanth-R19.github.io/rag-tool-curator/
├── pyproject.toml
├── LICENSE
└── requirements-core.txt # tool_curator's own dependencies, for non-pip installs
The full parameter reference, domain_synonyms.json authoring guidance, dependency_rules.json, and the evaluation loop live at Configuration & Domain Synonyms. In summary: get_curator() takes mode, top_k, model_name, and index_dir; retrieval quality on a new domain is governed almost entirely by how well domain_synonyms.json covers real users' actual phrasing, not by anything in the curator's own logic.
Both curator generations were scored against a manually curated 281-query hand-labeled suite spanning multiple personas and query categories, using a Correct / Partially Correct / Wrong / Skipped rubric:
| Metric | Value |
|---|---|
| Total Evaluation Queries | 281 |
| Correct | 262 |
| Partially Correct | 9 |
| Wrong | 3 |
| Skipped | 7 |
| Correct Response Rate | 93.2% |
| Overall Accuracy Score | 94.8% |
Reproduce this with example/eval/collect_tools.py + example/eval/curator_eval.py — see Evaluation & Results for the full methodology, the token-cost breakdown, and Failure Analysis for what the remaining 19 cases have in common.
- Both plug-ins operate purely on
(name, description)pairs / plain strings and have no awareness of MCP, FastMCP, transport protocol, or any specific LLM SDK. - Persona/role-based access control is intentionally kept outside the curator (
example/orchestrator/persona_access_policy.py). The curator always receives an already-authorized tool pool and never selects a tool outside it. multilingual_normalizeris fully independent oftool_curator— use either without the other. Its English fast-path is a majority-vocabulary heuristic, not a bare ASCII check, so it correctly routes romanized non-English text to translation instead of silently letting it through.example/orchestrator/mutation_guard.pyandhallucination_guard.pyaddress two failure modes common when integrating tool-calling LLMs — state-mutating calls without confirmation, and minor tool-name drift from the model. Neither is required for either plug-in to function; both are optional hardening layers in the reference orchestrator.
- Tool Curator Architecture — the full retrieval pipeline
- Multilingual Normalizer — the fast-path heuristic and adding your own translation backend
- Integration Guide — wiring both plug-ins into an existing multi-server MCP system
- Reusability Guide — what carries over to a new domain unchanged vs. what you'd rebuild
- Limitations & Future Work — validated vs. not, stated plainly