[Suggestion]: share fff-mcp indexes without leaking worktree state

[IgorArkhipov]

Which fff frontend(s)?

• MCP server (fff-mcp)
• Core or Rust crate

What problem are you trying to solve?

Agent hosts usually start one fff-mcp process per agent session. When several agents work in the same repository, each process builds its own index, content cache, watcher set, and scan threads.

Sharing one index for the whole repository would be incorrect because linked worktrees can have different branches and uncommitted files. Searches and refreshes must remain scoped to the worktree that initiated them.

Proposed solution

Keep the stdio MCP process that each agent host expects, but make it a lightweight proxy:

• Run one shared Unix daemon per canonical Git common directory.
• Inside that daemon, keep a separate index session for each canonical worktree root and complete set of index-affecting options.
• Let agents in the same worktree reuse one index.
• Let agents in different worktrees share only the daemon process while retaining isolated files, Git state, watchers, refresh generations, and frecency data.
• Preserve local-process behavior as an opt-out and on non-Unix platforms.

I suggest landing this as opt-in first. Default-on behavior can remain a separate decision after broader platform testing.

I have a reviewed local implementation split into small dependent changes. On an Apple M5 Max, three runs against the public FFF tree plus 12,000 generated files showed:

• Eight same-worktree clients: RSS fell from 581.3 MiB to 207.2 MiB (64.4%).
• CPU time fell from 12.13 s to 1.58 s.
• Warm attachment fell from 69.7 ms to 12.1 ms median.
• All 160 mixed search requests per mode succeeded, with no cross-worktree isolation failures.

There are tradeoffs: one client uses 24.5% more RSS, mixed-query median increased from 2.53 ms to 5.42 ms, p95 increased from 6.24 ms to 7.91 ms, and some cold multi-worktree starts reached about 646 ms. These results support an opt-in rollout, not an unconditional default.

Before opening implementation PRs, I would appreciate feedback on:

1. Does one daemon per Git common directory, with sessions keyed by canonical worktree and index options, fit FFF's direction?
2. Should frecency remain worktree-local unless users explicitly configure a shared database?
3. Is an opt-in Unix daemon an acceptable first rollout?

Related work: #801 explores in-process sharing for pi-fff; #608 and #675 cover different worktree-discovery and multi-root concerns.

[gustav-fff]

[triage-bot] FEATURE REQUEST: needs maintainer design decision, no triage action possible.

Not a bug. Nothing to reproduce. Classified feature-request, not duplicate — #608 is worktree discovery, #675 is roots/list_changed, #801 shares finders in-process inside one pi process. This asks for a new out-of-process runtime.

What is being proposed is a second execution mode for crates/fff-mcp (2417 LOC today, single stdio server in crates/fff-mcp/src/server.rs): Unix socket transport, daemon lifecycle/reaping, session table keyed by canonical worktree + index options, per-session watcher and frecency isolation, plus a non-Unix fallback path. That is architecture, not a fix, and it is outside what automated triage lands.

The three questions you ask (daemon keyed by git common dir, frecency locality, opt-in Unix-only rollout) are calls only @dmtrKovalenko makes. Your benchmark numbers were not verified here — no local repro was attempted.

Two things that will decide it, worth having ready:

• p50 regression 2.53ms -> 5.42ms is on the hot search path. This repo does not trade query latency for RSS silently. Show where the extra ~2.9ms goes (IPC roundtrip vs session lookup) and whether it survives being opt-in.
• Orphan/daemon lifetime. See [Bug]: MCP server with Claude code keeps leaving orphan processes that take up of CPU usage #633 — orphaned fff-mcp processes already burned users once. A long-lived daemon needs a defined idle shutdown and crash-recovery story before anything is merged.

Waiting on maintainer direction. Do not open implementation PRs until it lands.

Honk-Honk 🪿

[IgorArkhipov]

I reran this specifically around those two questions.

Latency attribution

I compared the same ARM64 release binary from b67c285, with only --shared-daemon toggled. The fixture had 12,382 tracked files; both modes used --no-update-check --no-warmup --content-indexing --no-watch.

The quoted benchmark is an intentionally adversarial case: eight clients continuously alternate find_files and grep against one worktree. Across two fresh five-repeat runs (4,000 calls per mode), the shared-mode p50 penalty reproduced at 3.1–3.4 ms.

In the labelled run:

operation	local p50	shared p50
mixed	3.55 ms	6.67 ms
find_files	0.80 ms	6.34 ms
grep	7.70 ms	7.05 ms

This is not session-table lookup: one million warm SessionPool::get_or_create calls measured 52 ns/call. A cheap one-client index_status control call was 0.132 ms local vs 0.197 ms shared, putting the approximate IPC/session-dispatch fixed cost around 0.07 ms in this setup.

The remaining delay is head-of-line blocking at the current one-permit semaphore per worktree session: a fast find can wait behind another client's grep. With the daemon disabled, the same custom binary measured 2.61 ms p50 in the unchanged alternating run, close to the earlier 2.53 ms upstream baseline. So the non-daemon path does not inherit the ~3 ms regression, but the shared path still needs a concurrency change before it should become the default. I would keep the first rollout opt-in and re-benchmark concurrent read-only searches without that single-request gate.

Lifetime and recovery

The prototype currently does the following:

• One daemon is keyed by the canonical Git common directory; each canonical worktree plus its complete index options gets a separate picker, watcher, scan state, and request session.
• Each stdio proxy watches its exact parent process and exits after reparenting, even if stdin was left open. This directly covers the orphan pattern from [Bug]: MCP server with Claude code keeps leaving orphan processes that take up of CPU usage #633.
• Inactive index sessions are evicted after the negotiated timeout, active requests cannot be evicted, connected clients recreate an evicted index on their next request, and finite inactive sessions are capped at eight per daemon.
• A daemon spawned without a successful client exits after 15 seconds. After clients disconnect, the daemon exits once retained sessions expire.
• Startup is serialized; stale owned sockets are validated before replacement; a missing live socket is repaired. A safe pre-send connection failure reconnects or starts a daemon, while an ambiguous partially written request is not replayed.

The recovery benchmark measured 108 ms for a retained reconnect, 172 ms to recreate an index after connected-client eviction, and 192 ms to restart a killed daemon with a changed PID.

One adjustment is appropriate before review: the prototype currently defaults to a 60-minute index timeout. I propose changing that to 15 minutes to match the direction in #633; 0 remains an explicit opt-out.

Review material

I published the implementation in my fork as a 12-commit review stack. Every intended PR boundary is also available as an igor/upstream-* branch.

Once the maintainer chooses the daemon key, frecency policy, and rollout mode, I can open only the approved slices as small PRs.

[dmtrKovalenko]

This is actually planned and currently is in development together with a standalone CLI. I agree 100%