AI Post Transformers
Genuine 1-Bit Post-Training Quantization for LLMs: All for 1-Bit
This episode examines "All for 1-Bit," a paper claiming to compress LLM linear weights to exactly one bit each, metadata included, without retraining. It builds on double binary factorization and challenges earlier "1-bit" post-training methods, which land at roughly 2.5 to 3.5 effective bits per weight once scales, masks and outlier indices are counted. The discussion covers why bandwidth-bound…