Weight Decay

اضمحلال الأوزان

تقنية تمنع الإفراط عبر فرض غرامة عددية تقلص الأوزان الضخمة وتجعلها قريبة من الصفر.

Weight Decay

Also translated asكبح الأوزان الزائدة، تقليص القيم البنيوية، اضمحلال أوزان

First appears in this corpus in: Ridge Regression: Biased Estimation for Nonorthogonal Problems (1970)

Appears in these papers

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization2011 in the sky ✦
Decoupled Weight Decay Regularization2019 in the sky ✦
ImageNet Classification with Deep Convolutional Neural Networks2012 in the sky ✦
ALIGN: Scaling Up Visual and Vision-Language Representation Learning with Noisy Text Supervision2021 in the sky ✦
Alpaca: A Strong, Replicable Instruction-Following Model2023 in the sky ✦
Backpropagation Through Time: What It Does and How to Do It1990 in the sky ✦
Neural Networks and the Bias/Variance Dilemma1992 in the sky ✦
Training Data-Efficient Image Transformers & Distillation Through Attention2021 in the sky ✦
Extracting and Composing Robust Features with Denoising Autoencoders2008 in the sky ✦
Dropout: A Simple Way to Prevent Neural Networks from Overfitting2014 in the sky ✦
The Falcon Series of Open Language Models2023 in the sky ✦
The Falcon Series of Open Language Models2023 in the sky ✦
Explaining and Harnessing Adversarial Examples2015 in the sky ✦
On the Difficulty of Training Recurrent Neural Networks2013 in the sky ✦
Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets2022 in the sky ✦
Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets2022 in the sky ✦
Large Batch Optimization for Deep Learning: Training BERT in 76 Minutes2019 in the sky ✦
Large Batch Optimization for Deep Learning: Training BERT in 76 Minutes2019 in the sky ✦
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour2017 in the sky ✦
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour2017 in the sky ✦
Regression Shrinkage and Selection via the Lasso1996 in the sky ✦
LIMA: Less Is More for Alignment2023 in the sky ✦
Symbolic Discovery of Optimization Algorithms2023 in the sky ✦
Symbolic Discovery of Optimization Algorithms2023 in the sky ✦
Visualizing the Loss Landscape of Neural Nets2018 in the sky ✦
mixup: Beyond Empirical Risk Minimization2018 in the sky ✦
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications2017 in the sky ✦
Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer2022 in the sky ✦
A Neural Probabilistic Language Model2003 in the sky ✦
Neural Tangent Kernel: Convergence and Generalization in Neural Networks2018 in the sky ✦
OPT: Open Pre-Trained Transformer Language Models2022 in the sky ✦
Optimal Brain Damage1989 in the sky ✦
Ridge Regression: Biased Estimation for Nonorthogonal Problems1970 in the sky ✦
RoBERTa: A Robustly Optimized BERT Pretraining Approach2019 in the sky ✦
SGDR: Stochastic Gradient Descent with Warm Restarts2017 in the sky ✦
SGDR: Stochastic Gradient Descent with Warm Restarts2017 in the sky ✦
Sharpness-Aware Minimization for Efficiently Improving Generalization2021 in the sky ✦
Sharpness-Aware Minimization for Efficiently Improving Generalization2021 in the sky ✦