Models & Agents
Ep 205: Pause-token KV compression shrinks cache 3.5x and speeds H100 inference 3.2x on long math…
12:06
Models & Agents Pause-token KV compression shrinks cache 3.5x and speeds H100 inference 3.2x on long math problems. What You Need to Know: AbridgeKV trains pause tokens to compress completed reasoning blocks into compact KV entries, matching full-cache accuracy on 446 competition problems while cutting memory and latency. Several new papers introduce reusable control layers, topology generators,…