AdamW

آدَم مع اضمحلال أوزان مفصول

متغيِّرة من آدَم تفصل اضمحلال الأوزان عن التدرج التفاضلي وتطبقه مباشرة على المعاملات، مما يحسِّن التعميم في تدريب المحوِّلات.

A variant of Adam that decouples weight decay from the gradient and applies it directly to parameters, improving generalization in Transformer training.

Also translated asAdamW، آدم دبليو

First appears in this corpus in: SGDR: Stochastic Gradient Descent with Warm Restarts (2017)

Appears in these papers

Decoupled Weight Decay Regularization2019 in the sky ✦
A ConvNet for the 2020s2022 in the sky ✦
The Falcon Series of Open Language Models2023 in the sky ✦
Large Batch Optimization for Deep Learning: Training BERT in 76 Minutes2019 in the sky ✦
LIMA: Less Is More for Alignment2023 in the sky ✦
Symbolic Discovery of Optimization Algorithms2023 in the sky ✦
OPT: Open Pre-Trained Transformer Language Models2022 in the sky ✦
Textbooks Are All You Need2023 in the sky ✦
SGDR: Stochastic Gradient Descent with Warm Restarts2017 in the sky ✦