[research] BM25 beats agentic search at scale — by 20 accuracy points

[github-actions[bot]]

🔬 The Finding

A new controlled scaling study ("BM25 Wins at Scale", arxiv 2607.26497) tested RAG paradigms across a 450-fold range of corpus sizes. At corpora beyond ~10 million tokens, BM25 lexical retrieval outperforms agentic file-system search by up to 20 accuracy points — while agentic search consumes 39× more query tokens and degrades as the search space grows. Dense retrieval stays efficient but lags in accuracy; graph-based RAG hits construction walls before reaching deployment scale.

⚙️ What It Means for Agentic Workflows

1. Don't replace retrieval with agents — sequence them. The study's own conclusion: "agentic reasoning works best after ranked discovery rather than in place of it." Use BM25 to shortlist candidates, then invoke your LLM agent only on the filtered set.
2. Token budget matters at scale. If your workflow searches large codebases or document stores, switching from open-ended agentic search to a BM25 pre-filter can slash inference costs dramatically without sacrificing (and likely improving) accuracy.

🔗 Source

BM25 Wins at Scale: A Scaling Study of RAG Paradigms — July 31, 2026

Generated by Daily Agentic AI Research Digest · 152.9 AIC · ⌖ 11.4 AIC · ⊞ 24.2K · ◷

• expires on Aug 10, 2026, 10:27 AM UTC