Switching tile formats on a LiDAR ingestion pipeline cut output object count by roughly 25x at the same data volume, independent of compression. Here is why object count is its own cost and performance axis on object storage, and what to check before you scale a pipeline that writes many small files.
Spark sets executor CPU requests equal to limits, so a Kubernetes cluster reserves twice the CPU it uses and refuses to schedule pending pods. Kubernetes has no native overcommit. Here is the mutating-webhook operator I use to fix it.
GPUs are no longer the bottleneck. Data movement is. The bottleneck in AI infrastructure has moved from compute to data movement. The claim was articulated in an online talk I attended recently, and it matches what I have seen for years across very different systems: raw compute stops being the Read more