The Databricks Data Engineer
The 90/9/1 rule of Databricks performance work - how to triage Spark optimization in 60 seconds
17:21
Your team is three weeks into a Databricks performance push. Broadcast hints in PRs. AQE flags toggled like christmas lights. Partition counts re-tuned for the third time. The manager is asking, gently, when the gains are showing up in the bill.
The staff DE on the next team finished theirs in two afternoons. Same workloads, bigger drop. They were running a triage you have never been…