GELU
وحدة الخطأ الخطية الغاوسية (GELU)
دالة تفعيل سلسة تجمع بين خصائص ReLU والتوزيع الغاوسي، تُستخدم في المحوِّلات الحديثة بدلاً من ReLU.
A smooth activation function combining properties of ReLU and Gaussian distribution, used in modern Transformers instead of ReLU.
Also translated asGELU، وحدة الخطأ الخطية الغاوسية، دالة تفعيل GELU
First appears in this corpus in: Gaussian Error Linear Units (GELUs) (2016)
Appears in these papers
- BART: Denoising Sequence-to-Sequence Pre-Training for Natural Language Generation, Translation, and Comprehension2019in the sky ✦
- BLOOM: A 176B-Parameter Open-Access Multilingual Language Model2022in the sky ✦
- A ConvNet for the 2020s2022in the sky ✦
- Gaussian Error Linear Units (GELUs)2016in the sky ✦
- Gemma: Open Models Based on Gemini Research and Technology2024in the sky ✦
- Improving Language Understanding by Generative Pre-Training2018in the sky ✦
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism2019in the sky ✦
- SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers2021in the sky ✦
- Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows2021in the sky ✦
- wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations2020in the sky ✦