Multi-Head Attention

الانتباه المتعدد المسارات

تشغيل آليات انتباه متعددة ومتوازية لالتقاط علاقات دلالية متنوعة في نفس الوقت.

Also translated asآليات الانتباه المتوازية الأبعاد، رؤوس الانتباه المتعددة، الانتباه متعدد الرؤوس، انتباه ذاتي متعدد الرؤوس

First appears in this corpus in: Attention Is All You Need (2017)

Appears in these papers

Parameter-Efficient Transfer Learning for NLP2019 in the sky ✦
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2018 in the sky ✦
BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer2019 in the sky ✦
BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer2019 in the sky ✦
DeepSeek-V3 Technical Report2024 in the sky ✦
Training Data-Efficient Image Transformers & Distillation Through Attention2021 in the sky ✦
DistilBERT, a Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter2019 in the sky ✦
FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning2023 in the sky ✦
Graph Attention Networks2018 in the sky ✦
Graph Attention Networks2018 in the sky ✦
Gemma: Open Models Based on Gemini Research and Technology2024 in the sky ✦
Gemma: Open Models Based on Gemini Research and Technology2024 in the sky ✦
Improving Language Understanding by Generative Pre-Training2018 in the sky ✦
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints2023 in the sky ✦
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints2023 in the sky ✦
Do Transformers Really Perform Bad for Graph Representation?2021 in the sky ✦
Do Transformers Really Perform Bad for Graph Representation?2021 in the sky ✦
Informer: Beyond Efficient Transformer for Long Sequence Time Series Forecasting2021 in the sky ✦
Llama 2: Open Foundation and Fine-Tuned Chat Models2023 in the sky ✦
LLaMA: Open and Efficient Foundation Language Models2023 in the sky ✦
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism2019 in the sky ✦
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism2019 in the sky ✦
Mistral 7B2023 in the sky ✦
PaLM: Scaling Language Modeling with Pathways2022 in the sky ✦
A Time Series Is Worth 64 Words: Long-Term Forecasting with Transformers2023 in the sky ✦
A Time Series Is Worth 64 Words: Long-Term Forecasting with Transformers2023 in the sky ✦
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers2021 in the sky ✦
Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows2021 in the sky ✦
Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows2021 in the sky ✦
A Decoder-Only Foundation Model for Time Series Forecasting2024 in the sky ✦
A Decoder-Only Foundation Model for Time Series Forecasting2024 in the sky ✦
Attention Is All You Need2017 in the sky ✦
Attention Is All You Need2017 in the sky ✦
An Image Is Worth 16×16 Words: Transformers for Image Recognition at Scale2020 in the sky ✦