Support Zstd compression - #3145

I'd like to benchmark this but I'm having some infra troubles with the server where I usually run these things. I'll post a benchmark in the next ~24 hours.

Here's that benchmark. It just reads a file with read_eval_log and immediately writes it with write_eval_log Before:

$./benchmark.py /tmp/in_deflate.eval /tmp/out_deflate.eval
2026-01-31 15:47:51,569 - INFO - Input file size: 102.82 MB
2026-01-31 15:48:03,352 - INFO - Reading took 11.78 seconds
2026-01-31 15:48:33,680 - INFO - Writing took 30.33 seconds
2026-01-31 15:48:33,680 - INFO - Output file size: 102.82 MB

(between these two I ran INSPECT_USE_ZSTD=1 ./benchmark.py /tmp/in_deflate.eval /tmp/in_zstd.eval)
After:

INSPECT_USE_ZSTD=1 ./benchmark.py /tmp/in_deflate.eval /tmp/in_zstd.eval
2026-01-31 15:55:28,776 - INFO - Input file size: 38.52 MB
2026-01-31 15:55:38,942 - INFO - Reading took 10.17 seconds
2026-01-31 15:55:53,674 - INFO - Writing took 14.73 seconds
2026-01-31 15:55:53,675 - INFO - Output file size: 38.52 MB

@epatey and @dragonstyle will these logs be readable by the viewer?

@epatey and @dragonstyle will these logs be readable by the viewer?

Currently, no. The viewer currently only implements the standard DEFLATE algorithm right now. It would pretty easy to add, making future versions support it, but log files written with zstd would be unreadable to old inspect view clients, which isn't great.

We could get support for zstd on older Python with this: https://github.com/taisei-project/python-zipfile-zstd

It seems like this might be worthwhile given how substantial the savings are!

@dragonstyle I am not super worried about older versions of the viewer as generally just to keep up with the latest models users need to upgrade quite frequently. We could include a conversation tool that converts logs to use the older compression which people could run in batch as required.

@kerrickstaley I'm more excited about this if we can make it work everywhere (as I think with just the env var it will be very infrequently used).

I've added support within the viewer for decompressing zstd streams here:
#3149

I've added support within the viewer for decompressing zstd streams here: #3149

Okay, that is now merged! Here's what I propose:

1. @kerrickstaley let's enhance this PR to use zipfile-zstd on Python 3.10-3.13 and the standard library on Python >= 3.14. We should have it activate via environment variable exactly as you do now.

2. In 3 months or so (long enough to make it acceptable so soft-force viewer upgrades) we will switch this to being the default behavior.

@epatey tagging you for review as I know we have other places that we read from zip streams that may need the **zipfile_compress_kwargs? (just want to make sure we have all our internal and scout use cases working correctly).

If other folks have code that cracks our zip files directly I am wondering if our import zipfile_zstd will cover them (since it monkey patches). Alternatively if its inconvenient to make it "just work" for these callers then asking them to add import zipfile_zstd for Python < 3.14 is IMO not an unreasonable ask.

@jjallaire done!

we have other places that we read from zip streams that may need the **zipfile_compress_kwargs

zipfile_compress_kwargs is just for writing, not reading. Reading will work transparently on Python 3.14 or if zipfile_zstd has been imported.

I am wondering if our import zipfile_zstd will cover them (since it monkey patches)

Yes, it will, as long as they have run some code path which has imported _util/zipfile.py. Maybe we should move the zipfile_zstd import to src/inspect_ai/__init__.py?

@kerrickstaley, this looks awesome. We do have two additional scenarios that I'll have to tackle.

• In inspect_scout we have a pipeline that does streaming, filtered json parsing directly from the compressed file entry byte stream.
• Also in inspect_scout, we have a use case where we literally stream the compressed file entry bytes directly to the client setting content-encoding: deflate when possible. This allows the server to not touch the bytes, and causes the client's http stack to do the decompression before the js code even sees the data.

I should be able to get to them this week.

Or maybe put it in inspect_ai/log/__init__.py to scope it down a bit more?

Done

@kerrickstaley, if convenient, can you share a few evals that you've created with zstd for my testing.

@epatey here is one of our logs with Zstd compression

@kerrickstaley, I've finished the inspect_scout work here meridianlabs-ai/inspect_scout#257. I'm now thinking about what it would take to get your PR in.

One thing I want to consider is if there's a way to lazy import zipfile_zstd. As currently implemented, we load it (and monkey patch when <3.14) all the time. Ideally, we'd only import it when we actually need it - either when about to create a log with INSPECT_USE_ZSTD set or when reading a log with zstd compression. The reading could catch unsupported compression method errors, import zipfile_zstd and try again.

Thoughts?

Running import zipfile_zstd takes about 11ms. I benchmarked running uv run child.py 1000x and the mean was 21ms when child.py was empty and was 32ms when child.py contained just import zipfile_zstd. This feels like it's fast enough that it's not worth worrying about, especially given that it'll go away in future Python versions?

If we wanted to reduce this I agree we can catch the exception, import, and try again.

@epatey @jjallaire is this OK to merge? Like I mentioned before I think we don't have to do anything about the zipfile_zstd import but am happy to fix it if it's a concern for you. (The main downside of a conditional import is that the logic would be hairy / more to maintain). Thanks!