apache spark
Spark on Kubernetes Reserves CPU It Never Uses. Here’s the Overcommit Fix.
Spark sets executor CPU requests equal to limits, so a Kubernetes cluster reserves twice the CPU it uses and refuses to schedule pending pods. Kubernetes has no native overcommit. Here is the mutating-webhook operator I use to fix it.