Models & Agents
Ep 208: Frozen 16 MB embedding tables let ternary 46 MB models hit 615–740 tokens/sec on laptop…
12:15
# Models & Agents Frozen 16 MB embedding tables let ternary 46 MB models hit 615–740 tokens/sec on laptop CPUs while matching float baselines on LAMBADA and PIQA. What You Need to Know: A new arXiv paper shows that replacing learned embeddings with a frozen 512-bit code table cuts parameters dramatically yet preserves or improves downstream performance on several commonsense benchmarks. ...…