scout-postgres-benchmark

Reproducible latency benchmark of jonaspauleta/scout-postgres against Laravel Scout's default database driver, on a single Postgres 18 service with a hot cache.

The methodology, the artisan command, and the raw numbers are all here so anyone can rerun the benchmark on their own hardware and corpus and verify the package's performance claims.

What it measures

For each query in a representative set, both drivers execute Book::search($q)->take(20)->get() 30 times. Reported numbers are p50 and p95 of the 30 measured samples (after a 3-call warm-up), in milliseconds.

The query set covers seven shapes:

label	query	what it stresses
common_token	world	broad single-token match
two_words	modern history	multi-token (LIKE cannot bridge token gaps)
rare_phrase	philosophical exposition	narrow exact-phrase match
prefix_partial	phil	short prefix → broad match
typo	philosphy	near-miss handled by trigram
no_match	qwxzqwxzqwxz	nothing matches (LIKE seq-scans)
long_query	a comprehensive history of modern philosophical thought	long natural-language query

Setup

You need a local Postgres 14+ accepting connections on 127.0.0.1:5432.

git clone https://github.com/jonaspauleta/scout-postgres-benchmark.git
cd scout-postgres-benchmark
composer install
cp .env.example .env

# In .env, point DB_* at a Postgres database the postgres user can write to.
# The default values match a stock local Postgres 18 install.

php artisan key:generate
createdb -h 127.0.0.1 -U postgres scout_postgres_bench
php artisan migrate

Running the benchmark

# Seed 50,000 rows, then run with default config (30 measured runs per query, 3 warmup).
php artisan bench:scout --seed=50000 --runs=30 --warmup=3

After the initial seed, omit --seed to rerun against the same dataset:

php artisan bench:scout --runs=30 --warmup=3

To compare against the pre-1.0 default trigram threshold, set SCOUT_POSTGRES_TRIGRAM_THRESHOLD=0.15 in .env and rerun.

Output

Bench: rows=50150, warmup=3, runs=30, limit=20

--- Driver: pgsql ---
  common_token      p50=  8.90ms  p95=  9.19ms  hits=20
  two_words         p50=185.73ms  p95=189.41ms  hits=20
  rare_phrase       p50= 82.56ms  p95= 93.98ms  hits=20
  ...

--- Driver: database ---
  common_token      p50=  2.33ms  p95=  2.69ms  hits=20
  two_words         p50=196.02ms  p95=203.34ms  hits=0
  ...

=== Summary ===
+----------------+-----------+-----------+--------------+--------------+-------------+------------+
| query          | pgsql p50 | pgsql p95 | database p50 | database p95 | speedup p50 | hits pg/db |
+----------------+-----------+-----------+--------------+--------------+-------------+------------+
| common_token   | 8.90      | 9.19      | 2.33         | 2.69         | 0.3x        | 20 / 20    |
| two_words      | 185.73    | 189.41    | 196.02       | 203.34       | 1.1x        | 20 / 0     |
| ...                                                                                              |
+----------------+-----------+-----------+--------------+--------------+-------------+------------+

Reference numbers

A reference run on Postgres 18.3 / PHP 8.5 / Laravel 13 / 50,150 rows / hot cache is captured in the package itself, under benchmarks/README.md.

What the harness does

• Defines a Book model with title, subtitle, author, summary columns.
  • postgresSearchable() indexes all four with weights A, B, B, C.
• Seeds --seed=N rows of faker-generated text plus 150 deterministic rows with known matches for the test queries.
• For each driver in [pgsql, database], swaps scout.driver at runtime and runs every query through Scout's Book::search()->take(20)->get(). Engine resolution is invalidated between drivers so the cached engine map does not carry over.
• Times each call with hrtime(true) (nanosecond precision).
• Reports p50 / p95 / mean plus a comparison table with speedup p50 and the hit counts per driver.

Caveats

• Faker-generated text. Real corpora have different trigram distributions — numbers will differ.
• Hot cache only. Cold-cache numbers (first query after restart) are meaningfully higher and not measured here.
• Single-node, no replication, no concurrent load.
• The harness is intentionally short; treat it as a starting point for benchmarking your own corpus, not as the final word.

License

MIT