AI and LLMs · Middle · Question 45 of 75
What is model distillation?
- 1Removing all of the safety training from a model
- 2Training a smaller model to imitate a larger one's outputs
- 3Compressing prompts into fewer tokens
Show the answer
Correct answer: Training a smaller model to imitate a larger one's outputs
Explanation
A distilled model can be much cheaper for a specific task while keeping most of the quality on it.