Histograms of Oriented Gradients for Human Detection

Raw pixel values are a terrible feature for person detection. Move the person into shade and every pixel changes. Change their shirt from white to black and half the pixels flip. But the edges — where intensity changes sharply — remain remarkably stable. The boundary between arm and background, the line of the shoulders, the contour of the head: these persist across lighting, clothing, and skin tone.

Earlier methods recognized this. Canny edge detection finds edges but loses their orientation and strength. SIFT computes oriented gradients around sparse keypoints. HOG's breakthrough was to compute dense, overlapping gradient histograms across the entire detection window, capturing the global shape of the human body rather than isolated keypoints.

The HOG descriptor is built in five stages, each refining the previous one. Think of it as an assembly line where raw pixels enter one end and a compact shape description emerges from the other. Understanding each stage, and why it matters, is the key to understanding how hand-crafted features once powered state-of-the-art detection.

The first step is finding edges. At every pixel, we compute how fast the intensity changes horizontally and vertically. Dalal and Triggs found that the simplest possible filter works best: subtract the pixel on the left from the pixel on the right for the horizontal gradient, and the pixel above from the pixel below for the vertical one. No Gaussian smoothing, no Sobel operators — just the raw centered difference $[-1, 0, 1]$.

From the two components $g_x$ and $g_y$, we compute the magnitude (how strong the edge is) and the orientation (which direction it points). The magnitude tells us how much of an edge is here; the orientation tells us which way it runs.

Next, we divide the image into small spatial regions called cells — typically 8×8 pixels. Within each cell, we build a histogram: we spread the full 0°–180° range of unsigned orientations into 9 bins (each covering 20°), and each pixel votes for the bin matching its gradient direction, with its vote weighted by the gradient magnitude.

Why unsigned (0°–180°) instead of signed (0°–360°)? Because for silhouette detection, a dark-to-light edge is the same shape information as a light-to-dark edge. A person's arm creates a vertical edge regardless of whether the arm is darker or lighter than the background. Dalal and Triggs confirmed experimentally that unsigned gradients give better results for pedestrian detection.

Think of each cell histogram as a compass rose: it tells you which edge directions dominate this small patch of the image. A cell on the shoulder will have strong horizontal bins; a cell on the leg will have strong vertical bins.

Cell histograms capture local edge patterns, but their magnitudes depend on lighting. A person in bright sunlight has higher gradient magnitudes than the same person in shadow — the pattern of edge directions is the same, but the histogram values are scaled differently.

To fix this, HOG groups cells into larger blocks — typically 2×2 cells (16×16 pixels) — and normalizes the concatenated histograms within each block. The blocks overlap: shifting one cell at a time, so each cell appears in multiple blocks with different normalization contexts. This redundancy is deliberate — it makes the descriptor more robust.

Dalal and Triggs tested four normalization schemes and found that L2-norm followed by clipping at 0.2 and renormalizing (called L2-Hys) works best:

After normalization, the histograms from all blocks are concatenated into one long vector. For a typical 64×128 pixel detection window with 8×8 cells, 2×2 cell blocks, and 9 orientation bins:

• The window has 8×16 = 128 cells
• Overlapping blocks: 7×15 = 105 blocks
• Each block: 2×2 cells × 9 bins = 36 values
• Final descriptor: 105 × 36 = 3,780 dimensions

This 3,780-element vector is a complete description of the gradient structure within the detection window. It encodes where every edge is, how strong it is, and which direction it points — all normalized against local illumination.

The 3,780-dimensional HOG vector is fed into a linear SVM — a classifier that learns a single hyperplane separating "person" from "not person" in this high-dimensional space. The SVM learns which gradient patterns correspond to human body shapes during training: strong horizontal gradients at shoulder height, strong vertical gradients at leg positions, a rounded gradient pattern at head level.

Why a linear SVM? Because the HOG features are so well-engineered that a simple linear boundary suffices — nonlinear kernels gave only marginal improvement at much higher computational cost. This is a hallmark of good feature engineering: when the features are right, the classifier can be simple.

To find people in a full image, the HOG+SVM pipeline uses a sliding window: a 64×128 detection window moves across the image at multiple scales, computing the HOG descriptor at every position and feeding it to the SVM. The SVM says "person" or "not person" for each window. Overlapping positive detections are merged using non-maximum suppression to produce the final bounding boxes.

This is computationally expensive — thousands of windows per image at multiple scales — but it was practical for offline processing and near-real-time with optimizations. The sliding window paradigm would later be replaced by region proposals in R-CNN, but HOG proved that dense scanning with a strong descriptor works.

Dalal and Triggs ran extensive ablation studies — systematically changing one component at a time — and identified the choices that matter most:

• Fine gradient scale — the $[-1, 0, 1]$ filter outperformed Sobel, Gaussian derivatives, and larger masks. Smoothing hurts because it blurs the very edges we're trying to capture.
• Unsigned gradients with fine orientation binning — 9 bins over 0°–180° outperformed signed gradients (0°–360°) and coarser binning. For shape detection, you need direction, not polarity.
• Overlapping block normalization — this was the single most impactful design choice. Without it, performance dropped dramatically. Local contrast normalization is what makes HOG robust to illumination.
• Linear SVM — a simple classifier on top of well-engineered features. This validates the principle that feature quality matters more than classifier complexity.

SIFT and HOG both use gradient histograms, but they answer different questions. SIFT asks: "What distinctive keypoints exist and how can I describe the region around each one?" HOG asks: "What is the overall gradient structure of the entire detection window?"

SIFT is sparse — it finds special points and ignores the rest. This makes it excellent for matching and recognition of specific objects. HOG is dense — it covers every cell in the window. This makes it excellent for detecting categories of objects (like "any person") where the shape varies but follows a pattern.

Think of it this way: SIFT is like describing a face by its moles and freckles (unique landmarks). HOG is like describing a face by the overall arrangement of edges — eyes here, nose there, chin below — which works for any face, not just one specific person.

Dalal and Triggs explored two block geometries. R-HOG (Rectangular HOG) uses square grids of cells — the version described above and the one that became standard. C-HOG (Circular HOG) uses cells arranged in log-polar sectors radiating from a central point, similar to SIFT's descriptor layout.

R-HOG performed slightly better for pedestrian detection, was simpler to implement, and became the default. C-HOG has advantages for rotationally symmetric objects but adds complexity without significant gains for upright human detection.