httpxyz one month in

It has been roughly a month since we forked httpx and named our package httpxyz. For the reasons why I refer you to Why I forked httpx. In this post I will explain where we are now, one month ‘into’ having created the fork.

TL;DR: httpxyz has now many bug fixes and significantly better performance than httpx, and we would encourage anyone to move to our version!

Where we are now

Initial version of fork

Our first version of httpxyz contained just the fixes to get zstd working, and the fixes to get the test suite running on python 3.14, some ‘housekeeping’ changes related to the renaming, as well as some minor and trivial fixes that had already been merged in httpx and had not been released yet.

End of March: bugfixes, and performance improvements

Then at the end of March we did another release of httpxyz, containing a compatibility shim that allows you to use httpxyz even with third-party packages that import httpx themselves, as long as you import httpxyz first. We’ve added a nice documentation page for this.

This release also included a change that lazily imports the CLI for httpxyz, which is typically not used in your app; in my measurements this makes httpxyz import in half the time needed by httpx. This PR has been sitting idle at the httpx repository for over a year ref and we’ve kindly adopted it, thanks to Nate Hardison for providing it!

And apart from that we adopted a bunch of other smaller and bigger bug fixes and improvements in this release.

httpcorexyz!

But we realized pretty soon that there were some problems that httpx was having, that were actually caused by the underlying ‘transport’ which is defined in a different module, httpcore. This module also had no release in over a year, and there were a bunch of issues reported by users, with fixes even, that were not landing. So we ended up also forking httpcore and created httpcorexyz.

Here we fixed a WHOLE bunch of performance related issues:

Use anyio.Lock(fast_acquire=True) and anyio.Semaphore(fast_acquire=True) to skip unnecessary event loop yields on uncontended lock acquires. Reduces async GET latency by ~3.3x. Reinstates encode/httpcore#953 which was reverted upstream for non-technical reasons. Requires anyio >= 4.5.0. ref
Retire map_exceptions() context manager in favour of plain try/except on hot paths (socket reads, writes, connects). try/except with no exception is nearly free; @contextmanager-based with blocks have measurable overhead. (encode/httpcore#1044) PR 14
Make Origin hashable via Origin.__hash__() for use in sets and dicts. (encode/httpcore#1038)
Release the HTTP/2 max-streams semaphore and decrement _concurrent_streams when NoAvailableStreamIDError is raised, preventing permanent deadlock. (encode/httpcore#1061) PR 10
Move del self._events[stream_id] inside the _state_lock in _response_closed() to prevent a race condition with concurrent coroutines. (encode/httpcore#1062)
Close proxy connection when tunnel TLS handshake fails, preventing the pool from stalling at max_connections. (encode/httpcore#1049)
Propagate the connect timeout through the SOCKS5 handshake to prevent indefinite hangs with non-responsive proxies. (encode/httpcore#1055) ref
Explicitly close async generators via safe_async_iterate() context managers to prevent Trio warnings and unpredictable GC-dependent behaviour. (encode/httpcore#1019) ref
Reduce lock contention in PoolByteStream.close(): pool requests are marked as closed outside the lock and cleaned up lazily inside _assign_requests_to_connections(). (encode/httpcore#1038) ref

These are serious issues, and it’s not even all. Almost all of these were already fixed by ‘the community’ and were living in unmerged pull requests. We code reviewed them, fixed them up where needed and adapted them to our code base. We also now added a benchmarks section to the documentation; httpxyz is now MUCH faster than httpx in many serious use cases.

Then we released a new version of HTTPXYZ earlier this week: 0.31.0 with this updated version of httpcorexyz and also more bug fixes of its own.

Where I previously stated having the fork is about future-proofing and I would not necessarily recommend people to switch to httpx, with this new version 0.31.0 I now feel that you should definitely move to httpxyz. With All new fixes and performance improvements it now makes ‘business sense’ to switch from httpx to httpxyz, the ‘churn’ is worth it now!

Adoption

We’re happy to see projects moving to using our package, even though it’s not much yet. We encourage everyone to do so, and please tell others and convince them! If you would find issues with your switchover, which I do not expect, please just open an issue and I’m sure we can figure it out!

Quotes

I'm a big user of httpx... Thanks for the fork. Here's to hoping it gained some traction. (Michael Kennedy, Founder, Talk Python)

Thank you for forking httpx. httpx is a joy to work with, but clearly a bit dead in the water. Really nice to see it moving forward again. (hhartzer)

If you want to have traction, you should probably move to GitHub instead of using Codeberg (Marcelo Trylesinski aka Kludex / FastAPI)

On Codeberg

We explained the last time why we chose Codeberg; we’re not unhappy about our choice. We got 39 ‘stars’ so far on our Codeberg repository and many of those are from ‘fresh’ accounts. If us being on codeberg helps a tiny little bit making github a bit less dominant, I think this is a good thing, and I hope it can inspire other projects to do the same. And after all, for most users, who would just pip install httpxyz or uv add httpxyz it does not matter where the source is hosted.

Thanks, and have fun!