Distillation Loss

فقد التقطير

تباعد KL بين مخرجات المعلم والطالب المرنة (عند حرارة مرتفعة)، يُضرب في T² لتعويض تقلص التدرجات.

The KL divergence between teacher and student soft outputs (at elevated temperature), multiplied by T² to compensate for gradient magnitude reduction.

Also translated asدالة فقد التقطير

First appears in this corpus in: DistilBERT, a Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter (2019)

Appears in these papers

DistilBERT, a Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter2019 in the sky ✦
DistilBERT, a Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter2019 in the sky ✦