Reduce Hot-Path Search Latency with Connection Reuse and Collection Metadata Caching

[cferrys]

Summary

Search requests currently pay avoidable per-call overhead by opening a new socket for every HTTP request and, when query_by is omitted, issuing an extra collection metadata request before the actual search. This should be optimized so repeated searches avoid redundant DNS/TCP/TLS setup and repeated searchable_fields lookups.

Context

src/request.cpp builds raw HTTP/1.1 requests with Connection: close, resolves the hostname with gethostbyname(), opens a new socket, and tears it down for every Request::execute() call. On the search path, src/collections.cpp and src/search.cpp auto-detect query_by by calling get(name) when a query is provided without explicit fields.

For a high-performance RocksDB-backed search engine, this makes client-side overhead a meaningful part of tail latency, especially for short queries, autocomplete-style traffic, and services issuing many small searches.

Proposed Implementation

Add a lightweight connection reuse layer inside Request:

1. Keep persistent HTTP/1.1 connections per (scheme, host, port, tls_verify) where possible.
2. Send Connection: keep-alive and reuse sockets until the server closes them, a timeout occurs, or an error is detected.
3. Replace gethostbyname() with getaddrinfo() to support IPv6 and cleaner address handling.
4. Add a bounded metadata cache for collection searchable_fields, keyed by collection name.
5. Invalidate or refresh cached fields after collection create/update/delete calls.
6. Add tests or examples that demonstrate repeated searches do not trigger repeated metadata fetches when the schema is unchanged.

Impact

This reduces p50 and p99 latency for repeated searches, lowers CPU and network overhead from connection churn, and avoids doubling request count for common q-only searches. It also makes the C++ client better suited for production services where hlquery is called in tight loops or high-concurrency request paths.