Compute Budget
الميزانية الحوسبية
إجمالي كمية الحوسبة المخصصة لتدريب النموذج، تُقاس بعمليات الفاصلة العائمة (FLOPs)، وتُقدَّر بنحو C ≈ 6ND للمحوِّلات.
The total amount of computation allocated for training a model, measured in FLOPs. Approximately C ≈ 6ND for Transformers.
Also translated asالحد الأقصى لطاقة المعالجة المسموحة، السقف المالي والزمني للتشغيل الرقمي، الموازنة الحاسوبية، ميزانية الحوسبة
First appears in this corpus in: Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization (2018)
Appears in these papers
- Training Compute-Optimal Large Language Models2022in the sky ✦
- Training Compute-Optimal Large Language Models2022in the sky ✦
- Scaling Language Models: Methods, Analysis & Insights from Training Gopher2022in the sky ✦
- Scaling Language Models: Methods, Analysis & Insights from Training Gopher2022in the sky ✦
- Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization2018in the sky ✦
- Scaling Laws for Neural Language Models2020in the sky ✦
- Scaling Laws for Neural Language Models2020in the sky ✦