Skip to content

feat: find_documents, opt-in library discovery for the MCP server (0.46.0) - #40

Merged
qye-inf merged 9 commits into
mainfrom
feat/library-discovery
Nov 5, 2026
Merged

qye-inf merged 9 commits into
mainfrom
feat/library-discovery

Conversation

@qye-inf

@qye-inf qye-inf commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds find_documents, an opt-in MCP tool that lists PDFs under configured library folders, so an agent can find the right datasheet when its own shell cannot see the library. Release 0.46.0.

The case that prompted it: the MCP server runs on a Windows host and can read a network share of ~1,400 PDFs, but the agent's shell runs in a Linux dev container and cannot. Only the server can see the library, so only the server can do the finding; choosing the PDF stays with the agent.

What changes

  • DATASHEETINDEX_LIBRARY_ROOTS: one or more folders (UNC, local, mapped drive), separated by os.pathsep. Unset or blank: nothing changes, the server serves its five tools. Set: a sixth tool appears and the server instructions gain one sentence pointing the agent at it. .env is honoured on a source checkout.
  • find_documents(query, limit=50, refresh=false): case-insensitive AND match of whitespace-separated terms against each PDF's path below its root (folders and file name; never the root itself). Returns full server-side paths, ready for build_datasheet unchanged, plus modified and size_kb; total_matches and truncated tell the agent to narrow the query; per-root status (ok / refreshing / scanning / unavailable), listed_at, skipped_folders, skipped_files.
  • Cache: one file per root under <output dir>/library, valid for one day, written atomically. Stale or missing lists are rescanned in a background thread at server start, so a slow share is scanned before the agent asks. A stale list is served at once while it refreshes; only a root with no list (or refresh=true) makes a call wait, bounded at 30 s.
  • Failure handling: an unreachable root, an unreadable folder, a damaged cache or a file name that cannot be encoded as UTF-8 never becomes a tool error or stops the server.

Design decisions

  • MCP-server-only. Not added to create_datasheet_tool_defs() / create_datasheet_tools_server(): embedders did not opt into a filesystem lister, and their five-tool surface is unchanged. tools/defs.py and tools/registry.py are untouched.
  • Full paths, not root-relative ones, so build_datasheet's contract is unchanged and an agent on a different OS never joins path segments itself.
  • Configured roots only, never an arbitrary folder from the agent.
  • Threads, never a process pool (the Windows MCP stdio deadlock).
  • The new tool goes through the same defs list as the other five, so the schema-validation, unknown-tool and catch-all guards in _on_call_tool cover it unchanged.

Testing

  • tests/test_library.py (new) and tests/test_mcp_server.py: matching, scanning (symlinks/junctions, unreadable folders and files, non-UTF-8 names), cache round-trip and damage, background refresh and bounded waits with an injected clock, overlapping roots, opt-in registration, .env loading, end-to-end find_documents -> build_datasheet.
  • Full suite locally including the vendor-PDF tests: 1355 passed, 10 skipped. With the [llm] extra and sdk group against a live gateway: 1399 passed, 0 skipped.
  • Stdio smoke test of the real server process with two real roots and one missing root: correct tool list and instructions, missing root reported as unavailable, returned paths open in build_datasheet, limit: 501 rejected by schema validation, nothing written to stdout.

…oots

- load_cache treats any failure as an unusable cache (OverflowError for an
  Infinity size, RecursionError for deep nesting) and rejects a non-finite
  listed_at; a list from the future counts as stale. A damaged cache file
  no longer stops the server at warm().
- scan_root skips and counts (skipped_files) a PDF that cannot be stat'ed or
  whose path is not valid UTF-8: such a path broke save_cache and, returned
  to the agent, killed the stdio transport on serialization. skipped_files
  is persisted (cache schema 2) and reported per root.
- create_local_mcp_server loads .env before reading
  DATASHEETINDEX_LIBRARY_ROOTS and the cache's output dir.
…d entries

A root or cached path that is not valid UTF-8 no longer reaches the cache
name, tool description or JSON-RPC writer; a child whose type cannot be
read is counted in skipped_folders; docs say .env applies to a source
checkout only.
@qye-inf
qye-inf merged commit 9723338 into main Nov 5, 2026
7 checks passed
@qye-inf
qye-inf deleted the feat/library-discovery branch November 9, 2026 09:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant