Vanishing Gradient

اضمحلال متجهات الميل

مشكلة شائعة في التعلم العميق تحدث عندما تصبح التدرجات صغيرة للغاية مما يعيق تحديث أوزان الطبقات المبكرة، و يتسبب في توقف النموذج عن التعلم أو بطء شديد في التدريب.

Vanishing Gradient

Also translated asتلاشي التدرجات، الاختفاء التدريجي، التدرج المتلاشي، تلاشي المشتقات العكسية، تلاشي التدرج التفاضلي، التدرج التفاضلي المتلاشي

First appears in this corpus in: Learning Representations by Back-Propagating Errors (1986)

Appears in these papers

ImageNet Classification with Deep Convolutional Neural Networks2012 in the sky ✦
Learning Representations by Back-Propagating Errors1986 in the sky ✦
Learning Representations by Back-Propagating Errors1986 in the sky ✦
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift2015 in the sky ✦
Learning Long-Term Dependencies with Gradient Descent is Difficult1994 in the sky ✦
BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer2019 in the sky ✦
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks2019 in the sky ✦
Generative Adversarial Networks2014 in the sky ✦
On the Difficulty of Training Recurrent Neural Networks2013 in the sky ✦
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling2014 in the sky ✦
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification2015 in the sky ✦
Highway Networks2015 in the sky ✦
Long Short-Term Memory1997 in the sky ✦
Long Short-Term Memory1997 in the sky ✦
Rectified Linear Units Improve Restricted Boltzmann Machines2010 in the sky ✦
Deep Residual Learning for Image Recognition2015 in the sky ✦
Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation2014 in the sky ✦
Recurrent Neural Networks (RNNs): A Gentle Introduction and Overview2019 in the sky ✦
Recurrent Neural Networks (RNNs): A Gentle Introduction and Overview2019 in the sky ✦
Efficiently Modeling Long Sequences with Structured State Spaces2022 in the sky ✦
Efficiently Modeling Long Sequences with Structured State Spaces2022 in the sky ✦
Sequence to Sequence Learning with Neural Networks2014 in the sky ✦
Very Deep Convolutional Networks for Large-Scale Image Recognition2014 in the sky ✦
WaveNet: A Generative Model for Raw Audio2016 in the sky ✦
Wasserstein GAN2017 in the sky ✦