What is Model Distillation?
Creating a smaller, faster AI model that mimics the behaviour of a larger, more capable one.
Why It Matters
Distillation makes it possible to run powerful AI capabilities on smaller devices or with lower costs.
Real-World Example
Training a compact model to replicate the quality of a large language model for a specific task like email classification.
“Understanding terms like Model Distillation matters because it helps you have better conversations with developers and make smarter decisions about your software. You do not need to be technical. You just need to know enough to ask the right questions.”
Related Terms
Large Language Model (LLM)
An AI system trained on massive amounts of text that can understand and generate human language.
Quantisation
Reducing the precision of numbers in an AI model to make it smaller and faster without losing much accuracy.
Inference
The process of using a trained AI model to generate predictions or outputs from new input data.
Fine-tuning
Adapting a pre-trained AI model to perform better on a specific task by training it on additional specialised data.
From definition to deployment
Knowing the term is step one. Using it in something real is the rest.