Repository navigation
Run all Postgres work on the isolate thread - #5
Merged
Merged
Conversation
Today every plain query hops to the tokio thread and back. That thread spends most of its time in the kernel. This prototype lets us measure the other way, like dox-http-native does for HTTP. With DOX_PG_DIRECT=1, each isolate gets its own link in Rust: one mio poll and its own non-blocking Postgres connections. Dart sends a batch, Rust writes it to the sockets at once. Dart then calls a run function that reads the answers without waiting. An idle helper thread wakes Dart when answers come while it is idle. DOX_PG_DIRECT_PUMP=1 also runs once after each submit. Only plain queries on TCP without TLS use it. Transactions and TLS pools keep the tokio path. With the flag off nothing changes. ABI version 5.
The tokio thread added a thread hop per query. That cost a lot of speed. Now each isolate has one link with a mio Poll, like dox_http_native. Queries are written to the socket in dpg_submit and read in dpg_run. A small helper thread wakes Dart when a socket has data or a timer is due, and checks that the isolate is alive. - Connect, TLS (rustls) and login (trust, cleartext, md5, SCRAM) are a state machine in the loop. Nothing blocks the isolate. - maxConnections stays the limit for the whole process. When an isolate waits for a connection, the others give their idle ones to the pool. - Transactions, the statement LRU, the stale statement retry and the 300 s timeouts work as before. - A dead isolate closes its connections and frees its poll file. - tokio, tokio-postgres, deadpool-postgres, tokio-postgres-rustls and futures-util are gone. The DOX_PG_DIRECT switch is gone. - ABI version 6.
- 4 isolates on one pool never have more than maxConnections open. - An isolate that keeps idle connections does not starve another. - A killed isolate frees its connection, its transaction ends. - A slow query and an idle transaction time out (short timeout for the test only, set with dpg_set_timeout). - direct_test.dart is now pipeline_test.dart and always runs. It also checks a connection that the server ends.
README and CHANGELOG (Unreleased) say how it works now. The pubspec description no longer names tokio-postgres. The version stays the same.
A transaction id now holds the id of the link that started it. A request from another isolate goes to the inbox of that link, runs on the connection of the transaction, and the answer goes back to the outbox of the link that asked. The owner isolate keeps its fast path. When the owner isolate is gone, the request gets "the transaction is closed". Leaf calls must not call the Dart API. Now only the helper thread posts to Dart. dpg_submit returns 1 when results are ready at once, and Dart calls dpg_run by itself. Other wakes use nudge: a flag, a kick of the helper and a mio Waker in the poll, so a waiting helper wakes at once.
No test uses PG_URL any more.
When the results were bigger than Dart's buffer, dpg_run gave back -n. Dart grew the buffer and called once more. That call ran a new turn, found even more results, and could give back -n again. Dart did not check this, so the results stayed in Rust and nothing woke Dart: the queries hung. Now the next call after -n only copies the results that are held, and Dart calls again until they fit. A test sends 200 results of 5 KiB at once, five times.
- A connection read the whole socket into its buffer before it read the messages. Under load that buffer grew to MBs, was zero-filled, and stayed up to 1 MiB. Now it reads a chunk, then its messages, then the next chunk: the buffer stays at 64 KiB unless one message is bigger. - Read and write buffers over 256 KiB are freed once empty. A big empty wait queue is freed too. - Row buffers of finished queries are reused (up to 32 per connection), so a query does not malloc and free one each time.
necessarylion
marked this pull request as ready for review
November 1, 2026 05:26
This was referenced Nov 1, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Every query went from the Dart isolate to a tokio thread and back. That thread hop cost a lot of speed.
Effect
GET /dbwas much slower than it could be. With the prototype of this change (2 cores): 127k -> 258k requests per second. Bun does 188k.Fix
All Postgres work now runs on the thread of the Dart isolate, with mio, like
dox_http_native.dpg_submitwrites the queries to the socket at once.dpg_runreads the answers. Neither waits.A small helper thread per isolate wakes Dart when a socket has data or a timer is due. It also sees when the isolate dies, and then closes its connections.
Connect, TLS (rustls) and login (trust, cleartext, md5, SCRAM-SHA-256) are a state machine in the loop. Nothing blocks the isolate. DNS runs at pool open, and again on a thread after a connect fails.
maxConnectionsis still the limit for the whole process. When an isolate waits for a connection, the others give their idle ones to the pool, so no isolate starves.Transactions, the statement LRU (512), the stale statement retry, and the 300 s timeouts work as before.
tokio, tokio-postgres, deadpool-postgres, tokio-postgres-rustls and futures-util are gone. The
DOX_PG_DIRECTswitch is gone. ABI version 6.A transaction id works from any isolate, as before. Its requests go to the isolate that started it, the answers come back.
dpg_submitanddpg_runare leaf calls and never call the Dart API. Only the helper thread posts to Dart.The public Dart API is the same.
threadsis kept but does nothing.Tests
cargo test,cargo clippy -D warnings(also forx86_64-pc-windows-msvc, with a stub C compiler),dart analyze,dart format.dart testagainst Postgres: 39 tests pass, also with--concurrency=1. TLS tests ran with docker.Not merged, not released. Benchmarks are still to be run.