Compression

Compression is prediction, and prediction is compression: these are two sides of the same information-theoretic coin. Any model that accurately predicts the next token can be turned into a lossless data compressor, and any compressor can be turned into a generative model. This guide explains why that equivalence holds, how DeepMind and Meta proved it in 2023, and why it reframes how you should think about large language models. What Does It Mean That Compression Is Prediction? Compression is prediction because a good predictor implicitly assigns low code lengths to likely sequences, which is exactly what a compressor does. When you predict “the” follows “of the”, you are acting like an entropy coder that gives “the” a short code because it is probable. The tighter your predictions match the real distribution of data, the fewer bits you need to store that data. ...

What Is misa77? — The Open-Source Codec That Outruns LZ4 misa77 is a new open-source LZ-based compression codec written in C++20 by developer nonadhocproblem. Released under the MIT license at version 0.2.0, it targets a specific niche: write-once read-many workloads where decompression throughput is the critical bottleneck. On the standard Silesia compression corpus, misa77 at its fastest level decodes at 5,219 MB/s — more than double LZ4’s 2,505 MB/s — while simultaneously achieving a better compression ratio (42.64% vs 47.59%). This combination of faster decode and better ratio is rare in the compression landscape and has attracted significant attention from the developer community. ...