Speculative retrieval: hiding web retrieval latency behind human typing latency. Retrieval starts before the request technically exists.
user typing ──▶ intent predictor ──▶ Search API ──▶ speculative context cache
│
final selector ──▶ main model
A research project, not a product. Live search responses are recorded once and all reported pilot analyses are recomputed offline. The current artifact stops at search-result caching: it does not yet measure fetched passages, answer quality, or production end-to-end latency. The metric family comes from hardware cache prefetching:
| metric | meaning |
|---|---|
| coverage | fraction of stabilized final-query URLs present in speculative results |
| URL precision | speculative URLs also present in the stabilized final-query set |
| timeliness | proxy-useful material ready before Enter |
| waste | issued searches that fail, miss the deadline, or contribute no proxy-useful URL |
Claim boundary: the current
goldis the union of repeated top-8 results from the same retriever and final query. Exact URL overlap measures provider self-agreement, not judged relevance or "the final answer's sources." Replay fetch latencies are sensitivity parameters, not measurements.
src/prelude/ Search API adapter, predictors, trace recorder, metrics
eval/questions.jsonl 60-question eval set: 20 short / 20 medium / 20 long
experiments/ experiment runners (M1: predictability curves)
tools/ instance probing, analysis
results/<run_id>/ traces.jsonl + gold.json + metrics.json + summary.md + plot
docs/ design notes, related work
Prelude treats retrieval as one provider-neutral Search API boundary. It accepts a query and returns ranked results with a title, URL, snippet, rank, latency, and failure metadata. Provider routing and fallback behavior are implementation details behind that boundary; the experiments operate only on the recorded Search API responses.
Run: results/m1-live2-20260815-210823 — coverage of the gold URL set
(gold = union of 3 final-question searches) by predictor and prefix fraction:
| predictor | 25% | 50% | 75% | 100% |
|---|---|---|---|---|
| oracle (same query, calibration) | 0.99 | 0.99 | 0.99 | 0.99 |
| prefix-as-query (zero-model) | 0.04 | 0.17 | 0.31 | 0.99 |
| window (last 8 words) | 0.04 | 0.15 | 0.23 | 0.60 |
Readings (exploratory):
- Same-query calibration: oracle = 0.991 under this reference construction. It is not a general retriever noise floor.
- The zero-model baseline is weak on exact URL overlap: at 75% of words, raw-prefix search recovers 0.314 (95% item-bootstrap interval 0.259--0.371).
- The trailing-window heuristic is weaker on long questions, but it simply discards all but the last eight observed words. It is not an autocomplete model, so the result does not establish that query autocompletion fails.
- There is predictor headroom relative to the non-deployable oracle, but only judged retrieval and answer experiments can localize the real bottleneck.
- Median recorded search latency is 107ms. That is only the winning search hop; it excludes prediction, page fetch/extract, queueing, gating, and generation.
results/m2-replay-pilot/ replays the recorded responses under seeded typing
trajectories calibrated to 30/45/60/90 effective WPM, three trigger policies,
three selectors, a 24-URL cache, and modeled 0--10s parallel fetch/extract
sensitivity. At 45 WPM and 500ms modeled fetch, a raw-prefix trigger at 75%
reaches 31.4% URL-agreement coverage and 50.0% context Hit@3 with one logical
search. Incremental 25/50/75% triggering expands cache coverage to 36.5%, but
the latest-batch top-8 context retains 32.1%; it averages 2.88 logical searches
and 46.4% unproductive-or-late work. The lexical selector changes incremental
agreement by -7.7 points; this is not a relevance or safety result. A second
artifact varies cache size, context size, and selector. These are provider-URL
proxy results, not answer-latency results.
- Primary report:
results/m2-replay-pilot/summary.md - Cache/selector sensitivity:
results/m2-cache-sensitivity/summary.md - Paper:
paper/main.pdf(source) - Reproduction:
make reproduce
- M1 (done): predictability curves — oracle/prefix/window vs prefix fraction; trace + gold + metrics pipeline; junk-guarded retriever chain.
- M2a (done, proxy only): offline trace replay, seeded typing policies, bounded URL cache, modeled fetch-latency sensitivity, and selector/capacity ablations.
- M2b: held-out qrels + real keystrokes; measured page fetch/extract; matched-budget query-completion and trigger baselines.
- M3: bounded passage cache + calibrated semantic gate; end-to-end answer quality/latency versus post-hoc RAG and no-RAG; edit contamination and cost.
- M4: confirmatory user study and submission-ready paper.
apps/demo/ — a live presentation page showing API calls and timeline events
while typing. Its SSE event shape is not the persisted experiment schema and
is not used for paper results.
.venv/bin/python apps/demo/server.py # then open http://127.0.0.1:8899make verify # artifact self-consistency, lineage, and exact M1 metric replay
make test # dependency-free unittest suite
make replay # deterministic 162,000-row primary policy grid
make sensitivity # cache/context/selector grid
make analyze # item-bootstrap tables, contrasts, and figures
make paper # uses tectonic, or pdflatex + bibtex
# or: make reproduceConfigure a compatible Search API endpoint, then run:
.venv/bin/python experiments/run_predictability.py \
--eval eval/questions.jsonl --out results/<run_id> --gold-k 3 \
--base http://localhost:8080
.venv/bin/python tools/analyze_m1.py --run results/<run_id>API credentials remain behind the Search API boundary and are never written to Prelude traces.
See CITATION.cff. The accompanying exploratory paper is
paper/main.pdf.
MIT License. Copyright (c) 2026 Abas Gudle. See LICENSE.