Ground Truth
The Economics of Training Versus Inference Are Inverting
13:05
For the first decade of deep learning, the bottleneck was training—getting enough compute and data to build models. But as models have become larger and more capable, the cost structure has inverted. Now the bottleneck is inference: running trained models at scale. A single query to a large language model costs money in compute, and at scale, those costs add up. This episode maps the economics…