httpx forks - is the library compatible?

[Abdelgha-4]

Idk if you're following what's arguably described as the "collapse" of httpx and mkdocs, if not, this article is about mkdocs, you can found other resources about httpx.

It's not about who's right or wrong but the current situation highlights an increasing instability, unreliability and what some has classified as supply-chain risk, so I think the safest choice is to move to httpxyz as a drop-in replacement for httpx and zensical (built by the creators of Material for MkDocs) as a future-proof alternative to Mkdocs.

[will-ockmore]

I've been following the situation here and there. It's certainly something to keep abreast of given the strong dependency on httpx.

I'm looking forward to a stable release of zensical, it has great potential. For now it's missing a few features that the docs for httpx-retries depend on, including but not limited to support for mkdocstrings-style linking. I don't think there are any major issues with keeping the documentation setup we currently have for the foreseeable future.

Regarding httpx forks, I'd be open to seeing what we can do about supporting them transparently; we can likely test against this and ensure things still work as expected. It probably requires some changes to the dependency definitions in pyproject.toml to ensure things work as expected when users install the package.

[vgavro]

@will-ockmore i reviewed httpxyz, it's setting sys.modules["httpx"] on first import. For pytest it's registered as plugin, so pytest -p httpxyz ensures it's imported first, and other libraries importing httpx will transparently using httpxyz instead.
I added this for my testing pipeline with this approach vgavro/httpx-curl-cffi@72aa598 , also see https://httpxyz.org/httpx-compatibility/
if you're interested i'm willing to add merge request like this here. But - in my humble opinion, it's a little bit extra at the time. There are no plans in httpxyz to change API, so i don't see real need with this at least while httpxyz isn't as widely adpoted to decide implement meaningful API changes on their side. Basically, it's their DECLARED goal to be backward compatible with httpx, and while it is so - we're safe.

[will-ockmore]

That's a nice @vgavro

I also agree, it's probably a bit extra (for now). Let's see what happens. If the fork takes off then we can reassess

[pawamoy]

For now it's missing a few features that the docs for httpx-retries depend on, including but not limited to support for mkdocstrings-style linking.

We do support mkdocstrings-style linking in Zensical, if you want to give it a try 🙂

Also note that Pydantic recently forked httpx into an httpx2 repo in their org, and the httpxyz maintainers seem to welcome it.

[will-ockmore]

Update on this: I tested Zensical and it's fully compatible, so I made the switch. Nice and easy, thanks @pawamoy !

I'm going to reopen this issue to track the httpx forks and provide some information for anyone confused by the changes.

On the forks

On the subject of httpx itself, for anyone out of the loop: there's been no stable release since 0.28.1 (Dec 2024) and the issue tracker is closed.

The fork situation has already consolidated since this issue was opened. Pydantic now maintains httpx2 as a stewardship continuation. 2.5.0 is marked stable, Starlette has migrated. The httpxyz author himself now points people to httpx2 as the "blessed" fork. As a result the practical target is httpx2, with httpxyz as the transparent drop-in.

Good news for this library: httpx-retries works with no changes on both forks. I've added a CI job that runs the full suite against httpxyz and httpx2 on push; so we'll see if that stops being true.

How to use each fork

httpxyz self-registers, so just import it first:

# pip install httpxyz
import httpxyz                      # registers itself as sys.modules["httpx"]
import httpx                        # -> httpxyz
from httpx_retries import RetryTransport

with httpx.Client(transport=RetryTransport()) as client:
  ...

httpx2 is a rename with no auto-alias, so you have to wire it up yourself, and it has to happen before httpx_retries is imported:

# pip install httpx2
import sys, httpx2
sys.modules["httpx"] = httpx2       # do this first, before importing httpx or httpx_retries
import httpx                        # -> httpx2
from httpx_retries import RetryTransport

with httpx.Client(transport=RetryTransport()) as client:
  ...

Why the alias is required: RetryTransport subclasses httpx.BaseTransport. Without the alias it binds to the real httpx, and httpx2.Client then rejects it. The alias makes RetryTransport bind to httpx2's classes instead.

Plan for this project

For now I consider this testing approach sufficient. If httpx2 becomes the de facto standard for whatever reason, this can be revisited.