The Databricks Data Engineer
Why Databricks serverless exists, and which of your jobs should stay on classic clusters
12:03
Open the workspace. Start a cluster. Get coffee. Come back, and sometimes it's ready. If you've been doing this a few years you can feel that wait in your shoulders, and it's the least interesting part of the story.
Before any of it, you had to answer a question you could not possibly answer: how big should this be? Start small and you pay in latency. Start big and you pay for machines that are…