A knowledge transfer technique where a smaller student model is trained to mimic the output distributions or behavior of a larger teacher model to achieve comparable performance with reduced computational cost.
Overview
- [Industry analysis] US Treasury Secretary Scott Bessent threatened targeted sanctions against Chinese companies for distilling US large language models, framing the practice as IP theft. (Source: Scott Bessent (via AI Daily Brief host), via AI Daily Brief, 2026-08-24)