Turing Post
Inference Race: OpenAI Cut Inference Costs in Half. AMD and Cerebras Split AI
17:20
One AI request may soon begin on one computer and finish on another. WHAT?!
AMD and Cerebras are separating the two phases of LLM inference: Helios processes prompts and long context, while Cerebras generates tokens. They claim up to 5x more tokens per second per watt, although the figure is based on internal modeling.
Meanwhile, OpenAI reportedly cut inference costs for one segment of ChatGPT…