GPT-4 Technical Report

GPT-4 is a large-scale multimodal model built on the Transformer architecture. Like its predecessors, it is pre-trained to predict the next token in a document, then fine-tuned using Reinforcement Learning from Human Feedback (RLHF) to follow instructions and behave safely. But GPT-4 introduces two leaps: it can accept images alongside text as input, and its builders developed methods to predict its performance before training finished.

The report deliberately withholds architecture details — model size, hardware, training compute, and dataset construction — citing competitive and safety reasons. What it does reveal is how GPT-4 performs, how its performance was predicted, and how its safety was improved.

The central engineering insight of GPT-4 is that performance at massive scale can be predicted from small-scale experiments. Training GPT-4 required enormous compute — far too expensive for trial and error. Instead, the team trained much smaller models using the same methodology and fitted a scaling law to predict the final loss.

The intuition: imagine plotting the fuel efficiency of toy cars at different sizes. If the data follows a smooth curve, you can predict what a full-sized car will achieve without building one. The key formula is a power law with an irreducible loss term that captures the floor below which no amount of compute can push.

The prediction matched GPT-4's actual final loss with remarkable accuracy — this was done before the full training run completed, using models trained with at most 1/10,000th the compute.

GPT-4 was tested on exams designed for humans — bar exams, SATs, AP tests, medical knowledge assessments, and coding challenges — with no exam-specific training. The results were striking:

• Uniform Bar Exam: scored ~298/400, placing in the top 10% of test takers (GPT-3.5 scored in the bottom 10%).
• SAT Math: 700/800 (~89th percentile).
• MMLU (57 academic subjects): 86.4% accuracy, beating all prior models and most benchmark-specialized systems.
• GRE Verbal: 169/170 (~99th percentile).

These capabilities stem primarily from pre-training, not from RLHF fine-tuning — the base model and the RLHF model performed nearly identically on multiple-choice exams (73.7% vs 74.0% averaged across all exams).

To test GPT-4 in languages beyond English, the team translated the entire MMLU benchmark into 26 languages using Azure Translate. GPT-4 outperformed the English-language scores of both GPT-3.5 and prior models (Chinchilla and PaLM) in 24 of 26 languages — including low-resource languages such as Latvian, Welsh, and Swahili.

This is remarkable because GPT-4 was not specifically trained on multilingual exam data. The model's understanding of concepts transfers across language boundaries. For Arabic, GPT-4 achieved approximately 80% accuracy on MMLU — compared to 70% for GPT-3.5 in English.

GPT-4 accepts prompts consisting of both images and text — a capability absent from prior GPT models. The model can interpret diagrams, photographs, screenshots, and memes, generating text outputs that demonstrate genuine visual understanding. Standard techniques like few-shot prompting and chain-of-thought reasoning work just as well with visual inputs.

Think of it as giving the model eyes: where GPT-3.5 could only read the description of a chart, GPT-4 can look at the chart itself and answer questions about it. The model can read handwritten text, explain the humor in a meme, solve physics problems from diagrams, and summarize academic papers by looking at their figures.

Despite its capabilities, GPT-4 has the same fundamental limitations as earlier GPT models:

• Hallucinations: the model can generate fluent text that is factually wrong, sometimes with high confidence. GPT-4 improved factual accuracy by 19 percentage points over GPT-3.5 on adversarial factuality tests, but hallucinations remain.
• Limited context window: the model processes a fixed-length input and cannot consult external sources mid-generation.
• No learning from experience: unlike humans, GPT-4 does not learn from its interactions after training is complete.
• Calibration loss after RLHF: the pre-trained model is highly calibrated (its confidence matches its accuracy), but RLHF reduces this calibration — making the model sometimes overconfident or underconfident.

GPT-4 introduced a two-pronged approach to safety that uses the model itself as a tool for its own alignment.

The first prong is adversarial testing by domain experts: over 50 specialists in cybersecurity, biosecurity, and international security probed the model for dangerous capabilities. Their findings informed targeted data collection — for example, teaching GPT-4 to refuse requests about synthesizing dangerous chemicals.

The second prong is rule-based reward models (RBRMs): zero-shot GPT-4 classifiers that provide an additional reward signal during RLHF training. Given a prompt, a model response, and a human-written rubric (e.g., "classify as: desired refusal / undesired refusal / disallowed content / safe response"), the RBRM classifies the output. This lets the system reward correct refusals on dangerous prompts and penalize over-cautious refusals on safe prompts.

The result: an 82% reduction in responses to disallowed content compared to GPT-3.5, and a 29% improvement in responding appropriately to sensitive requests.

GPT-4's training pipeline has three major stages, each building on the one before it:

Stage 1 — Pre-training: the model learns to predict the next token from a massive corpus of text and images from the internet and licensed sources. This is where the model acquires its broad knowledge of language, reasoning, facts, and code. The scaling laws described earlier allow engineers to predict the outcome of this stage.

Stage 2 — Supervised Fine-Tuning (SFT): human labelers write ideal responses to diverse prompts, and the model learns to follow instructions by imitating these examples.

Stage 3 — RLHF with RBRMs: human labelers rank multiple model responses, training a reward model that captures human preferences. The policy model is then optimized against this reward model, with additional rule-based reward signals (RBRMs) to steer safety behavior at a fine-grained level.