AI Shrinks 1 GB of Wikipedia to 105 MB, but at a Cost

Original colored-pencil illustration of a fictional woman software researcher studying Wikipedia documents flowing through neural compression into a compact archive in a GPU computing laboratory

A viral X post declares lossless compression solved. The underlying research is impressive, but more specific. Tufa Labs researcher Tommy He developed tufazip, an experimental compressor that reduced a one-billion-byte Wikipedia text file to 105,388,353 bytes. The total includes compressed text, decoder software and the model initialization needed to recover every original byte.

The result appears on Matt Mahoney’s Large Text Compression Benchmark. Its first version ranked second when submitted September 7, 2026. Version two improved the result September 30, followed by an October 14 technical write-up. The claim resurfaced on social media November 19. Tufazip is the strongest listed entry using a neural network as its sole predictor, although higher-ranked methods combine neural and hand-crafted models.

Willie’s November 19 X reaction to the Tufa Labs benchmark. Original post.

How much did Wikipedia shrink?

The benchmark input, enwik9, holds exactly 1,000,000,000 bytes of Wikipedia content. The final scored archive is 105.39 decimal MB, yielding a 9.49-to-1 compression ratio and an 89.46% reduction. Lossless recovery differs from generative AI reconstructing plausible text: the decoded result must match the original byte for byte.

The public tufazip repository lists 100,958,324 encoded-data bytes plus 4,430,029 bytes of supporting decoder files. It also publishes file hashes. Including the software prevents an enormous model kept elsewhere from disguising the real storage requirements.

Educational tutorial on neural compression and arithmetic coding, published before the tufazip research. It is not a benchmark demonstration. Original video.

A Transformer that learns while decoding

Tufazip employs a 12-layer, 163-million-parameter Transformer. The model predicts upcoming tokens, then arithmetic coding assigns fewer bits to probable symbols and more to surprising ones. Encoder and decoder repeat the same deterministic learning updates using previously processed text, so the full newly learned weights never need transferring.

The winning version also begins with a compact offline-trained model. The additional predictor knowledge improves compressed size enough to justify the bytes required to transfer it. Older statistical models and neural compression research supplied many of the underlying ideas.

enwik9 compressorScored archive
tufazip v2105.388 MB
tufazip 0.1.0105.979 MB
gzip 1.3.5, -9322.626 MB
Source: Large Text Compression Benchmark. Decimal MB, required decoder files included. Different computers and runtimes prevent a direct speed comparison.

The hidden cost: more than seven days

The benchmark records about 631,548 seconds for compression and 645,748 seconds for decompression, respectively 7.31 and 7.47 days. The documented test used two NVIDIA B200 GPUs, and the compression time excluded earlier offline training. Gzip achieved far less compression on the same input but had a much shorter recorded runtime on different hardware.

A record for smallest archive is not a recommendation for everyday backups. Our LZ4 coverage explains why production systems often value speed. Our B200 rental article documents compute-market economics, while the new electricity cost per kWh guide explains how measured power and tariffs translate to a bill.

February 2026 arithmetic-coding tutorial from NextGen Learners. Background about entropy coding, not a tufazip product test. Original video.

Why lossless compression has not been solved

No lossless compressor can reliably reduce every possible file. Random bytes, encrypted material and already compressed data may resist compression or expand after overhead is added. Wikipedia contains textual structure that a trained model can predict; other datasets behave differently.

Broader use will require faster encoding, practical decoding, tests on varied file types and affordable infrastructure. Our benchmark reporting on software optimization shows why researchers must disclose workload and runtime. Tufazip advances learned text compression, not a universal law-defying compressor.

Sources and methods

Primary evidence: Tufa Labs research report, public code and checksums, and the benchmark record. The editorial calculations use the exact published byte totals. A full independent compression and decompression reproduction was not performed.

Reporting Note: The colored-pencil cover shows a fictional White woman software researcher. Neither educational video depicts the tufazip research team or an actual September benchmark run.


Editor’s Note: We volunteer daily to ensure the credibility of information on our platform is Verifiably True. Support our research by scanning the QR code below to donate 5,000 sats.

BitcoinVersus.Tech donation QR code for 5,000 satoshis

For mobile scanning, open the article on another display and scan the QR code with a Bitcoin wallet. Verify receiving address 3C9o19EH5HSiwEPyCTmEKzxhNCbo2X6TTb before sending.

BitcoinVersus.tech provides technical and financial journalism for informational purposes only and does not provide investment, legal or cybersecurity assurance.

Leave a Reply