Highly Accurate Protein Structure Prediction with AlphaFold

Proteins are the molecular machines of life. A protein starts as a linear chain of amino acids — there are 20 types, each with different chemical properties. Within milliseconds of being synthesized, this chain folds into a precise 3D shape that determines its function: hemoglobin carries oxygen because of its shape, antibodies recognize viruses because of theirs.

The sequence-to-structure mapping is called the protein folding problem. Experimental methods (X-ray crystallography, cryo-EM, NMR) can determine structure, but each takes months and costs tens of thousands of dollars. By 2020, only ~170,000 structures had been experimentally determined — a tiny fraction of the billions of proteins in nature.

CASP (Critical Assessment of protein Structure Prediction) is a biennial blind competition where teams predict structures of proteins whose shapes have been determined but not yet published. In CASP14 (2020), AlphaFold achieved a median GDT score of 92.4 — for the first time matching experimental accuracy. The second-best method scored 67.

AlphaFold's architecture has three main stages, and understanding this pipeline is key to understanding the paper:

Stage 1 — Input embedding. The amino acid sequence is searched against genetic databases to build a Multiple Sequence Alignment (MSA) — a collection of related sequences from other organisms. These evolutionary relatives reveal which positions co-evolve (change together), hinting at 3D proximity. Two initial representations are created: an MSA representation (capturing per-sequence, per-residue features) and a pair representation (capturing relationships between every pair of residues).

Stage 2 — Evoformer. 48 blocks of a novel attention architecture jointly refine both representations. The MSA track uses row-wise and column-wise attention; the pair track uses triangular updates and triangular attention. Information flows continuously between the two tracks. This is where the network reasons about which residues are close in 3D space.

Stage 3 — Structure module. Converts the refined representations into actual 3D coordinates. Each residue gets a local coordinate frame (rotation + translation), refined by Invariant Point Attention (IPA). The entire pipeline is recycled 3 times, with each pass refining the previous prediction.

The key insight driving AlphaFold is co-evolution. When two amino acid positions are in contact in the 3D structure, a mutation at one position creates evolutionary pressure for a compensating mutation at the other — just like two puzzle pieces that must fit together. If one piece changes shape, the other must change too.

To capture this, AlphaFold builds a Multiple Sequence Alignment: it searches databases like UniRef90 and BFD for sequences similar to the target protein, aligning them column by column. Each column represents the same structural position across species. Columns that show correlated mutations are likely in 3D contact.

From this, two representations are initialized:

• The MSA representation $m_{si}$ has shape (sequences × residues × features). Think of it as a spreadsheet where each row is a related species and each column is a position in the protein. Each cell holds a feature vector describing that residue in that sequence.

• The pair representation $z_{ij}$ has shape (residues × residues × features). Think of it as a relationship map: cell $(i, j)$ holds everything the network currently believes about the spatial relationship between residue $i$ and residue $j$.

The Evoformer is the heart of AlphaFold — 48 stacked blocks that iteratively refine both representations. Each block contains two interleaved tracks, one for the MSA representation and one for the pair representation, with information flowing between them. Think of it as two teams of analysts working side by side: one team studies the evolutionary evidence (MSA), the other studies spatial relationships (pairs), and they constantly share notes.

MSA track — Row-wise attention with pair bias: For each sequence in the MSA, attention is computed across residue positions (the columns). This lets the network learn which positions in the sequence are related. Crucially, the pair representation is injected as a bias into this attention — so what we already know about residue proximity directly influences how the MSA is read.

MSA track — Column-wise attention: Attention is computed down each column of the MSA — across different species at the same position. This lets the network compare how different organisms handle the same position, learning conservation patterns.

Outer product mean — MSA to pair communication: The MSA representation is summarized into the pair representation via an outer product operation. For each pair of positions $(i, j)$, the network takes the mean outer product of their MSA column features across all sequences. This converts evolutionary covariance into spatial information.

The pair representation encodes pairwise distances — but distances in 3D space must satisfy the triangle inequality: if residue A is close to B, and B is close to C, then A must be reasonably close to C. Standard attention has no mechanism to enforce this.

AlphaFold introduces triangular multiplicative updates — a novel operation designed specifically for this constraint. The intuition: for every edge $(i, j)$ in the pair representation, look at all intermediate residues $k$ that form triangles $(i, k, j)$. The edges $(i, k)$ and $(k, j)$ provide information about what edge $(i, j)$ should look like.

There are two variants: outgoing edges update $(i, j)$ using $(i, k)$ and $(j, k)$ — both starting from the same node; incoming edges update $(i, j)$ using $(k, i)$ and $(k, j)$ — both ending at the same node. These are followed by triangular self-attention, where the attention bias for updating row $i$ of the pair representation comes from column information of the same pair matrix.

After 48 Evoformer blocks, the representations are rich with structural information — but they're still abstract feature vectors, not 3D coordinates. The structure module bridges this gap.

Each residue is represented by a rigid body frame — a rotation matrix and translation vector that define the position and orientation of its backbone in 3D space. Initially, all frames are at the origin. The structure module's 8 blocks (with shared weights) iteratively refine these frames.

The core mechanism is Invariant Point Attention (IPA) — a novel attention mechanism designed for 3D structure. Standard attention computes similarity between abstract feature vectors. IPA additionally computes similarity between 3D points — each query and key generate points in their local coordinate frames, and the attention scores include the distances between these points. The crucial property: IPA is invariant to global rotations and translations. Moving or rotating the entire protein doesn't change the attention scores, because the distances are computed in local frames.

After IPA updates the single representation, a small network predicts a rotation and translation update for each residue's frame. The side-chain positions are then computed from predicted torsion angles.

A single pass through the network produces a reasonable structure, but not the best one. AlphaFold uses recycling — running the entire pipeline 3 times, feeding the previous output back as additional input. Think of it like a sculptor: the first pass roughs out the shape, the second refines proportions, the third polishes details.

Specifically, three things are recycled: (1) the predicted backbone coordinates from the structure module, (2) the output pair representation from the Evoformer, and (3) the first row of the MSA representation. This recycling is remarkably efficient — it lets the network act as if it were 3× deeper without 3× the parameters, because the weights are shared across cycles.

How do you teach a network to predict 3D structures? You need a loss function that compares predicted and true atom positions — but there's a subtlety. If the predicted structure is simply rotated or translated relative to the ground truth, a naive loss would penalize this even though the structure is correct.

AlphaFold introduces Frame Aligned Point Error (FAPE): for each residue's local frame, transform all atoms into that frame and compute the error. By averaging over all frames as reference points, FAPE is invariant to global rotations and translations — it measures structural accuracy regardless of orientation. This is more informative than traditional RMSD because it captures local structural quality (each residue judges the structure from its own viewpoint), not just global alignment.

A prediction is only useful if you know how much to trust it. AlphaFold predicts a per-residue confidence score called pLDDT (predicted Local Distance Difference Test), ranging from 0 to 100. Regions with pLDDT > 90 are typically accurate to within 1 Å of the experimental structure; regions below 50 are likely disordered (no fixed structure in nature).

AlphaFold also provides a Predicted Aligned Error (PAE) matrix — a residue × residue map showing the expected position error of residue $j$ when the prediction is aligned on residue $i$. This reveals domain boundaries: within a rigid domain, PAE is low everywhere; between independently-moving domains, PAE is high.

AlphaFold didn't just solve a computational problem — it gave biology an entirely new tool. Drug designers use it to find binding sites. Enzyme engineers use it to design proteins that don't exist in nature. Evolutionary biologists use it to study proteins too ancient for experimental determination. The Transformer architecture that powers modern language models also powers the prediction of life's molecular machinery.