TurboQuant: Google’s New Compression Trick That Actually Works

Google Research just dropped three new compression algorithms at ICLR and AISTATS 2026, and the lead one — TurboQuant — is actually interesting. Not in the “we compressed a model by 90% but it can’t tell a cat from a dog” way. These folks claim zero accuracy loss with massive memory savings. Let’s dig in.

The vector problem that won’t go away

Every AI model, from your phone’s autocorrect to GPT-4, relies on vectors — long lists of numbers that represent things like word meanings or image features. High-dimensional vectors are powerful but memory-hungry. They’re the reason your LLM’s key-value cache eats up VRAM like it’s going out of style.

Vector quantization has been the go-to fix for years. You compress those vectors down, save memory, speed up search. But there’s a dirty secret: traditional quantization methods add their own memory overhead. You have to store quantization constants in full precision for every little block of data. That’s 1-2 extra bits per number, which kind of defeats the purpose when you’re trying to save bits.

TurboQuant tackles this head-on. It’s a two-stage compression that’s mathematically grounded, not just a hack thrown together in a Jupyter notebook.

How TurboQuant actually works

First stage: PolarQuant. It randomly rotates the data vectors — sounds weird, but it simplifies the geometry so a standard quantizer can work on each part individually. This is where most of the compression happens, capturing the essential meaning of the original vector.

Second stage: QJL (Quantized Johnson-Lindenstrauss). This is the clever bit. It takes the tiny residual error from the first stage and compresses it down to a single sign bit (+1 or -1). Zero memory overhead. It acts as a mathematical error-correction mechanism that eliminates bias in the attention score calculation.

The result? You get the compression without the usual accuracy hit. I’ve seen enough “breakthrough” compression papers that fall apart on real benchmarks, so the ICLR acceptance gives me some confidence this is legit.

Why this matters for real-world AI

Two big use cases:

1. KV cache compression — This is the bottleneck that makes long-context LLMs expensive. Every token in the conversation gets stored as key-value pairs. TurboQuant shrinks those pairs dramatically without making the model dumber. For anyone running production LLMs, this is a direct line to lower costs.

1. Vector search — Think semantic search, recommendation systems, RAG pipelines. Faster similarity lookups with less memory. The QJL component is particularly elegant here because it preserves distance relationships between data points while using almost no memory for the quantization constants.

I’ve been burned by quantization methods that look good on paper but require exotic hardware or blow up at inference time. TurboQuant seems designed for practical deployment — standard hardware, no special kernels needed.

The bigger picture

Google’s been quietly building a portfolio of compression techniques. These three algorithms (TurboQuant, QJL, PolarQuant) are part of a broader push to make AI models more efficient without sacrificing quality. The fact that they’re presenting at both ICLR and AISTATS suggests the theoretical foundations are solid.

What I’d like to see next: open-source implementations and benchmarks on real production workloads. Papers are nice, but I want to know how TurboQuant handles a 100K-token context window on an A100. That’s the real test.

For now, this is one of the more promising compression approaches I’ve seen in a while. It’s not hype — it’s math, and it works.