Knowledge distillation is a large model teaching a small one, and it works almost exactly like school: soft labels instead of a bare answer key, pairing weeks instead of reports, and a tutor who marks your own attempt rather than a textbook of last year’s solutions. The limits map too, and the last one costs money when you size the hardware.
A frontier lab optimizes an absolute target and has to explore. A challenger optimizes a distance to the leader and can exploit. That is the whole cost asymmetry, and it shows up in GPU hours and in a bike race.
OpenRouter put numbers on a pattern agent builders already use: a panel of models with a judge synthesizer beats the best single model, and a budget panel matched frontier quality at half the cost. What that means for your AI platform.
A small coding model mocked on r/LocalLLM is actually a clean case of execution-verified distillation. Here is how the method works, when distilling a frontier model into a cheap local one pays off, and the three things the hype leaves out.
Most AI infrastructure teams spend 35-60% more on GPU compute than they need to. The cause isn’t cloud pricing. It is architecture, and it is fixable.
Spark sets executor CPU requests equal to limits, so a Kubernetes cluster reserves twice the CPU it uses and refuses to schedule pending pods. Kubernetes has no native overcommit. Here is the mutating-webhook operator I use to fix it.