Activation Function

دالة التنشيط

دالة رياضية تطبق تحولاً غير خطي على مخرجات العصبون لتمكين الشبكة من تعلم أنماط معقدة.

Activation Function

Also translated asدالة التحويل غير الخطي، محفز العصبون الحسابي، دالة التنشيط

First appears in this corpus in: A Logical Calculus of the Ideas Immanent in Nervous Activity (1943)

Appears in these papers

Parameter-Efficient Transfer Learning for NLP2019 in the sky ✦
ImageNet Classification with Deep Convolutional Neural Networks2012 in the sky ✦
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift2015 in the sky ✦
Learning Long-Term Dependencies with Gradient Descent is Difficult1994 in the sky ✦
A ConvNet for the 2020s2022 in the sky ✦
Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks2015 in the sky ✦
Reducing the Dimensionality of Data with Neural Networks2006 in the sky ✦
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin2015 in the sky ✦
DeepSeek-V3 Technical Report2024 in the sky ✦
Deep Interest Network for Click-Through Rate Prediction2018 in the sky ✦
Deep Interest Network for Click-Through Rate Prediction2018 in the sky ✦
Mastering Diverse Domains Through World Models2023 in the sky ✦
Mastering Diverse Domains Through World Models2023 in the sky ✦
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks2019 in the sky ✦
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks2019 in the sky ✦
The Falcon Series of Open Language Models2023 in the sky ✦
Explaining and Harnessing Adversarial Examples2015 in the sky ✦
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness2022 in the sky ✦
Graph Attention Networks2018 in the sky ✦
Gaussian Processes for Machine Learning2006 in the sky ✦
3D Gaussian Splatting for Real-Time Radiance Field Rendering2023 in the sky ✦
Semi-Supervised Classification with Graph Convolutional Networks2017 in the sky ✦
Gaussian Error Linear Units (GELUs)2016 in the sky ✦
Gaussian Error Linear Units (GELUs)2016 in the sky ✦
Gemma: Open Models Based on Gemini Research and Technology2024 in the sky ✦
Going Deeper with Convolutions2014 in the sky ✦
GPTQ: Accurate Post-Training Quantization for Generative Pre-Trained Transformers2022 in the sky ✦
On the Difficulty of Training Recurrent Neural Networks2013 in the sky ✦
Inductive Representation Learning on Large Graphs2017 in the sky ✦
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification2015 in the sky ✦
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification2015 in the sky ✦
The Organization of Behavior: A Neuropsychological Theory1949 in the sky ✦
Highway Networks2015 in the sky ✦
Layer Normalization2016 in the sky ✦
Llama 2: Open Foundation and Fine-Tuned Chat Models2023 in the sky ✦
The Llama 3 Herd of Models2024 in the sky ✦
LLaMA: Open and Efficient Foundation Language Models2023 in the sky ✦
LLM.int8(): 8-Bit Matrix Multiplication for Transformers at Scale2022 in the sky ✦
The Regression Analysis of Binary Sequences1958 in the sky ✦
A Logical Calculus of the Ideas Immanent in Nervous Activity1943 in the sky ✦
A Logical Calculus of the Ideas Immanent in Nervous Activity1943 in the sky ✦
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism2019 in the sky ✦
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism2019 in the sky ✦
Mistral 7B2023 in the sky ✦
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications2017 in the sky ✦
Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer2022 in the sky ✦
N-BEATS: Neural Basis Expansion Analysis for Interpretable Time Series Forecasting2019 in the sky ✦
Natural Gradient Works Efficiently in Learning1998 in the sky ✦
Neocognitron: A Self-Organizing Neural Network Model for Pattern Recognition1980 in the sky ✦
Neural Tangent Kernel: Convergence and Generalization in Neural Networks2018 in the sky ✦
Neural Tangent Kernel: Convergence and Generalization in Neural Networks2018 in the sky ✦
OPT: Open Pre-Trained Transformer Language Models2022 in the sky ✦
OPT: Open Pre-Trained Transformer Language Models2022 in the sky ✦
PaLM: Scaling Language Modeling with Pathways2022 in the sky ✦
Perceptrons: An Introduction to Computational Geometry1969 in the sky ✦
Progressive Growing of GANs for Improved Quality, Stability, and Variation2018 in the sky ✦
RAFT: Recurrent All-Pairs Field Transforms for Optical Flow2020 in the sky ✦
Random Features for Large-Scale Kernel Machines2007 in the sky ✦
Rectified Linear Units Improve Restricted Boltzmann Machines2010 in the sky ✦
Efficiently Modeling Long Sequences with Structured State Spaces2022 in the sky ✦
Efficiently Modeling Long Sequences with Structured State Spaces2022 in the sky ✦
Shampoo: Preconditioned Stochastic Tensor Optimization2018 in the sky ✦
Approximation by Superpositions of a Sigmoidal Function1989 in the sky ✦
Very Deep Convolutional Networks for Large-Scale Image Recognition2014 in the sky ✦
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations2020 in the sky ✦
WaveNet: A Generative Model for Raw Audio2016 in the sky ✦
World Models2018 in the sky ✦
Understanding the Difficulty of Training Deep Feedforward Neural Networks2010 in the sky ✦
Understanding the Difficulty of Training Deep Feedforward Neural Networks2010 in the sky ✦
YOLOv3: An Incremental Improvement2018 in the sky ✦