Two movements meet: AI is getting production responsibilities, and production data is flowing through AI. Anthropic’s September report, malicious routers, training data provenance and model drift show what to verify before an AI reaches production.
Knowledge distillation trains a student using a teacher’s examples, output scores or internal features. Compare the access each method needs, then evaluate quality, unwanted behavior, serving performance and full lifecycle cost.
Qwen3 reports a roughly tenfold GPU-hour difference in one post-training comparison. The result explains the value of teacher feedback, not a universal rule for total frontier-model development cost.
A small coding model mocked on r/LocalLLM is actually a clean case of execution-verified distillation. Here is how the method works, when distilling a frontier model into a cheap local one pays off, and the three things the hype leaves out.