Masked Language Modeling (MLM)
نمذجة اللغة المُقنَّعة (MLM)
هدف تدريب مسبق يُقنَّع فيه جزء من رموز المدخل ويُطلب من النموذج تنبؤها من السياق المحيط من الاتجاهين.
A pre-training objective where a fraction of input tokens are masked and the model must predict them from the surrounding bidirectional context.
Also translated asنمذجة اللغة بالتقنيع، التنبؤ بالرموز المخفية، نمذجة اللغة المُقنَّعة
First appears in this corpus in: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2018)
Appears in these papers
- Atlas: Few-shot Learning with Retrieval Augmented Language Models2023in the sky ✦
- Atlas: Few-shot Learning with Retrieval Augmented Language Models2023in the sky ✦
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2018in the sky ✦
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2018in the sky ✦
- BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer2019in the sky ✦
- DeBERTa: Decoding-Enhanced BERT with Disentangled Attention2020in the sky ✦
- DeBERTa: Decoding-Enhanced BERT with Disentangled Attention2020in the sky ✦
- DistilBERT, a Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter2019in the sky ✦
- DistilBERT, a Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter2019in the sky ✦
- ELECTRA: Pre-Training Text Encoders as Discriminators Rather Than Generators2020in the sky ✦
- MusicLM: Generating Music From Text2023in the sky ✦
- REALM: Retrieval-Augmented Language Model Pre-Training2020in the sky ✦
- RoBERTa: A Robustly Optimized BERT Pretraining Approach2019in the sky ✦
- XLNet: Generalized Autoregressive Pretraining for Language Understanding2019in the sky ✦