Enable TCP_NODELAY by default for TCP sockets (Disable Nagle Algorithm) #144254
Description
Bug report
Bug description:
Hello,
Opening a bug ticket before I send a PR.
I just spent yet another day debugging a slow Python application. A task that took 8 seconds out of the box, took 0.3 seconds after adding TCP_NODELAY.
Nothing special, just two processes talking over a localhost TCP socket.
For people not familiar with the matter, nagle algorithm adds a small delay (up to 500ms depending on OS) before sending a packet in some circumstances, this can have a catastrophic effect on network applications depending on their read/write patterns.
This has been a long-standing recurring issue across many applications and libraries. I'd like to set TCP_NODELAY by default on all TCP sockets created by python (socket module), to effectively disables Nagle's algorithm by default.
The last python discussion on this topic was in 2016 https://bugs.python.org/issue27456
There was clear consensus that nagle algorithm should be disabled by default.
This lead to a few patches in built-in modules and third party packages (asyncio and uvloop), some related tickets: python/asyncio#286 python/asyncio#373
10 year have passed since that last discussion.
The bug is still showing up regularly both in the python interpreter and in popular libraries. ( a quick search immediately shows that http.server is affected #108736 )
We should fix it.
For python, I think we can set TCP_NODELAY when a socket is created in the socket module. It's just a few lines of code to change.
Thoughts?
The rest of the world has been actively setting TCP_NODELAY to mitigate the issue:
- GO has set TCP_NODELAY by default since inception
- curl has set TCP_NODELAY since 2016 curl/curl@4732ca5
- NodeJS has set TCP_NODELAY by default since 2020 Follow curl: Enable setNoDelay() / TCP_NODELAY by default on HTTP(S) 1.x nodejs/node#34185
- Rust has TCP_NODELAY in the network libraries (main library is tokio Set tcp nodelay on sockets tokio-rs/axum#2521)
- John Nagle explained himself that the nagle algorithm has been broken since the moment the RFC was published https://news.ycombinator.com/item?id=10608356
- the RFC https://datatracker.ietf.org/doc/html/rfc896
- one article to explain the issue https://brooker.co.za/blog/2024/05/09/nagle.html
Regards
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux