Model distillation trains a small student model to reproduce the behavior of a large teacher, giving you a cheaper and faster model. Presented that way it sounds like free money. The reason teams get burned is that the risks are not loud. They do not show up as a failed training run; they show up months later as a biased decision, a quiet quality decay, or a legal email nobody expected.
This article surfaces the non-obvious risks of distillation and gives concrete mitigations for each. The framing matters: distillation is usually a good idea, but it is a good idea with a specific risk profile that you should manage deliberately rather than discover the hard way. Treating it as risk-free is itself the first risk.
For the practices that prevent many of these problems upstream, What Is Model Distillation: Best Practices That Actually Work is the companion read.
Risk 1: You Inherit Everything Wrong With the Teacher
The student copies the teacher, including its biases, blind spots, and bad habits. Distillation does not filter quality; it transfers it wholesale.
Why this is dangerous
- A bias that was subtle in the teacher can become baked-in policy in a student that you deploy at scale and rarely revisit.
- The student often presents as a "new" model, so people forget to re-run the fairness and safety checks they would apply to anything new.
Mitigation: treat the student as a new model for governance purposes. Run the same bias, fairness, and safety evaluations you would on any production model, on the student specifically, not just the teacher. Audit the teacher's known weaknesses and check whether they survived distillation.
Risk 2: Silent Quality Drift in Production
A distilled student is tuned to a snapshot of inputs. The world moves. New input patterns appear, the teacher may update, and the student slowly stops matching reality, all without any error or alert.
Why it stays hidden
- Distilled models often serve high-volume, low-attention paths where nobody is watching individual outputs.
- Aggregate metrics can stay flat while a critical slice quietly degrades.
Mitigation: monitor production accuracy on a sampled basis, sliced by your critical categories, not just in aggregate. Pin teacher versions so the teacher cannot change underneath you. Define a redistillation trigger on measured drift. The metrics article covers the slice-based monitoring this requires.
Risk 3: Metrics That Lie
Distillation produces metrics that are unusually easy to misread, which leads to false confidence.
- Aggregate accuracy hides slice failures. A student at 93% overall can be at 68% on your highest-value segment.
- Fidelity to the teacher masks teacher errors. High agreement looks great even when the teacher is the thing that is wrong.
- Broken calibration. The student's stated confidence may no longer track its accuracy, so any threshold built on it is silently wrong.
Mitigation: never ship on a single number. Require slice-level task accuracy, fidelity, and recalibrated confidence together. The common mistakes guide catalogs the specific ways these metrics mislead.
Risk 4: Licensing and Terms-of-Service Exposure
This is the risk teams least expect and the one with the sharpest teeth. Distilling from a model means using its outputs as training data. Many commercial model providers' terms of service restrict or prohibit using their outputs to train competing models.
Why it bites
- The engineer running the distillation rarely reads the teacher's terms of service.
- The exposure is invisible until it is a legal or contractual problem.
Mitigation: before distilling from any third-party teacher, confirm that its terms permit using outputs for training your student. If they do not, either get explicit permission, switch to a teacher whose license allows it, or use a model you are licensed to distill from. Make this a required checklist item, not an afterthought.
Risk 5: Over-Compression Breaking Edge Behavior
Pushing the student too small to maximize savings can shatter behavior on rare but important cases, which are exactly the ones aggregate metrics ignore.
Mitigation: set the size target from the quality requirement, not the other way around. Find the smallest student that still holds your critical slices, and resist shrinking past that point just because the cost number looks attractive. The trade-offs article frames this size-versus-quality decision.
Risk 6: Security and Adversarial Exposure
A distilled student inherits the teacher's vulnerabilities and can introduce new ones. If the teacher is susceptible to certain prompt manipulations, the student often is too, and the smaller student may have fewer of the implicit safeguards a larger model picks up.
Why it is easy to miss
- Security review tends to focus on the original model, and the student is treated as a derivative that "already passed."
- A student deployed on-device or in a customer-controlled environment is more exposed to probing than a server-side teacher behind an API.
Mitigation: run adversarial and red-team testing against the student specifically, especially if it will be deployed in a less controlled environment than the teacher. Do not assume the teacher's security posture transfers cleanly to a smaller, differently-trained model.
Risk 7: Organizational Over-Reliance
A subtler, slower risk: once a distilled model works and is cheap, teams stop questioning it. The model becomes infrastructure, nobody owns its quality, and it quietly governs decisions long after the data it was trained on stopped being representative.
Mitigation: assign a named owner and a review cadence to every production distilled model, exactly as you would for any system that makes decisions at scale. The cost savings that make distillation attractive are also what make people stop paying attention, so the governance has to be deliberate. Pair this with the team standards described in Rolling Out What Is Model Distillation Across a Team.
A Risk Management Checklist
Run this before any distilled model reaches production:
- Confirm the teacher's terms of service permit distillation.
- Run bias, fairness, and safety evaluations on the student, not just the teacher.
- Evaluate task accuracy by critical slice, not only in aggregate.
- Recalibrate confidence and verify thresholds.
- Pin the teacher version and record it.
- Define a redistillation trigger and a production monitoring plan with a named owner.
Most distillation disasters trace back to skipping one of these six.
Frequently Asked Questions
What is the most overlooked distillation risk?
Licensing. Many commercial providers' terms restrict using their model's outputs to train another model, and engineers rarely check before distilling. Confirm the teacher's terms permit it before you generate a single training label, because this risk is invisible until it becomes a legal problem.
Does distillation introduce new bias?
It does not create new bias so much as transfer the teacher's bias into a model you may deploy more widely and watch less closely. Treat the student as a new model and run the full fairness and safety evaluation suite on it specifically.
How do I catch silent drift?
Monitor production accuracy on a sampled, sliced basis rather than relying on aggregate metrics, and pin the teacher version so it cannot change underneath you. Define an explicit redistillation trigger on measured drift so decay is acted on, not just observed.
Can making the model too small be a risk?
Yes. Over-compression breaks behavior on rare but important cases that aggregate metrics ignore. Set the size target from your quality requirement and find the smallest student that still holds your critical slices, rather than shrinking for cost and discovering the damage later.
Key Takeaways
- Distillation transfers the teacher's biases and blind spots wholesale; govern and evaluate the student as a new model, not a free copy.
- Watch for silent production drift with sampled, slice-level monitoring, pinned teacher versions, and a defined redistillation trigger.
- Distillation metrics mislead easily; never ship on a single number, and require slice-level accuracy, fidelity, and recalibrated confidence.
- Check the teacher's terms of service before distilling; many commercial providers restrict training on their outputs, and this risk is invisible until it is legal.
- Set the student's size from the quality requirement, not the cost target, to avoid over-compression that breaks critical edge cases.