Distillation
Distillation, also called knowledge distillation, is a technique where a smaller model is trained to replicate the behavior of a larger, more capable model. The goal is to achieve similar quality at lower computational cost and lower latency.