Control Theory1960intermediate12 min read

A New Approach to Linear Filtering and Prediction Problems

نهج جديد لمسائل الترشيح الخطي والتنبُّؤ

Kalman, R. E. — ASME Journal of Basic Engineering

The problem

Before 1960, filtering and of signals relied on the Wiener filter — a frequency-domain method that required stationary statistics, infinite memory, and heavy computation. It could not handle systems whose behavior changed over time, had no natural form, and struggled with multidimensional problems. Engineers needed a method that could estimate the of a dynamic system in real time, recursively, from noisy measurements — and that worked for nonstationary systems too.

The contribution

The : a recursive algorithm that estimates the hidden of a linear dynamic system from noisy measurements. At each time step it performs two phases — predict (project the state forward using the system ) and update (correct the prediction using the new measurement, weighted by a Kalman gain that balances model confidence against measurement confidence). It propagates not just the state estimate but also its uncertainty ( ), and it is provably optimal for linear systems with Gaussian : no other linear estimator can achieve lower .

The impact

The most widely used estimation algorithm in engineering history. It guided Apollo astronauts to the Moon, and the entire GPS system has been described as "one enormous Kalman filter." It is embedded in every smartphone (sensor fusion), every aircraft navigation system, every autonomous vehicle, and most financial trading systems. Its state-space formulation became the language of modern control theory, and it directly inspired descendants like ARIMA in time-series analysis and S4 in deep learning's state space models.

Imagine a sailor navigating through fog. He has two tools: a compass (his model of how the ship moves — heading and speed) and a foghorn echo (a noisy measurement of distance to shore).

The compass alone drifts over time. The echo alone jumps erratically. But if he weights each source by how much he trusts it and blends them, he gets a position estimate better than either one.

That is the Kalman filter: a principled way to blend an imperfect prediction with an imperfect measurement, updating your confidence in each as you go.

The ceiling: Wiener filtering and its limits

Before Kalman, the gold standard for filtering noisy signals was the Wiener filter (1940s). It worked in the frequency domain: decompose the signal into sine waves, figure out which frequencies are signal and which are noise, and keep only the signal.

This works well when the statistics of the signal don't change over time (stationarity) and you have access to the entire signal history. But real-world systems — a spacecraft changing orbit, a missile accelerating, an economy shifting regime — violate both assumptions. The Wiener filter also produces a batch solution: it processes the entire signal at once and cannot update incrementally as new data arrives.

What engineers needed was a filter that works in the time domain, handles nonstationary systems, updates recursively one measurement at a time, and extends naturally to multiple dimensions.

The key insight: think in state space

Kalman's breakthrough was to reformulate the filtering problem using the state-space representation of dynamic systems. Instead of asking "what frequency components does this signal have?" (Wiener's question), Kalman asked: "what is the hidden state of the system that produced this noisy measurement?"

The idea is simple but powerful: the world has a true state (position, velocity, temperature…) that you can't observe directly. You observe it through noisy sensors. Between measurements, the state evolves according to known physics (or a model). Both the evolution and the measurement are corrupted by random noise.

Two equations capture everything:

x_{k} = F\,x_{k-1} + B\,u_{k-1} + w_{k-1}

State transition equation — how the hidden state evolves — F = state transition matrix (the physics) · B u = optional control input · w = process noise (uncertainty in the model itself, drawn from 𝒩(0, Q))

z_{k} = H\,x_{k} + v_{k}

Measurement equation — what the sensor sees — H = measurement matrix (maps hidden state to observable) · v = measurement noise (sensor inaccuracy, drawn from 𝒩(0, R))

Think of it as a two-layer world: a hidden layer (the true state, evolving by the laws of physics plus some randomness) and an observable layer (what your sensor reports, which is a noisy, possibly incomplete view of the hidden state). The Kalman filter's job: infer the hidden layer from the observable layer, optimally.

Open in Lab

The two-layer view: hidden state evolves through F, observed through H. Toggle to see noise effects.

The demo wakes as you arrive…

The algorithm: predict, then update

The Kalman filter is a two-step loop that repeats every time a new measurement arrives:

Step 1 — Predict. Use the physics model to project the current state estimate and its uncertainty forward in time. You now have a prior — your best guess before seeing the new measurement. This is like the sailor dead-reckoning from his compass: "Based on my heading and speed, I should be here."

Step 2 — Update. When the new measurement arrives, compute the Kalman gain — a weight that tells you how much to trust the measurement versus the prediction. Blend the prediction with the measurement, proportionally. Then shrink the uncertainty: you know more now than you did before the measurement.

The key insight is that both the prediction and the measurement carry uncertainty, and the filter optimally balances them. If the model is very confident but the sensor is noisy, the filter mostly trusts the model. If the sensor is precise but the model is uncertain, the filter mostly trusts the sensor. The Kalman gain automatically finds the sweet spot.

Open in Lab

Watch the predict-update cycle on a moving target. The blue ellipse is prediction uncertainty, orange is measurement uncertainty, and green is the fused estimate.

The demo wakes as you arrive…

The math: five equations that run the world

The entire Kalman filter lives in five equations, split across the two phases. Each equation has a clear purpose — no equation is decoration.

Update phase: correct with measurement

Innovation: $\tilde{y}_k = z_k - H\,\hat{x}_{k|k-1}$ How far is the actual measurement from what I predicted? This "surprise" drives the correction.

Kalman gain: $K_k = P_{k|k-1}\,H^T\,(H\,P_{k|k-1}\,H^T + R)^{-1}$ The optimal blending weight. If R (measurement noise) is large relative to my prediction uncertainty, K shrinks — trust the model. If my prediction is uncertain, K grows — trust the sensor.

State update: $\hat{x}_{k|k} = \hat{x}_{k|k-1} + K_k\,\tilde{y}_k$ The corrected estimate: prediction + gain × surprise.

Covariance update: $P_{k|k} = (I - K_k\,H)\,P_{k|k-1}$ Uncertainty always shrinks after incorporating a measurement.

Open in Lab

Drag the noise sliders to see how the Kalman gain shifts between trusting the model and trusting the sensor.

The demo wakes as you arrive…

The Kalman gain: the heart of the filter

The Kalman gain $K$ is a matrix that answers one question: "how much should I correct my prediction?" Think of it as a dial between two extremes:

$K \to 0$ : ignore the measurement entirely — "my model is perfect, the sensor is garbage."
$K \to I$ : ignore the prediction entirely — "my model is clueless, trust only the sensor."

In practice $K$ is always somewhere in between. It automatically adjusts at every time step based on the relative uncertainties. Early on, when your state estimate is rough, $K$ is large — every measurement matters. As you accumulate measurements and your estimate tightens, $K$ shrinks — you already know where you are, a single noisy reading won't move you much.

K_k = \frac{P_{k|k-1}\,H^T}{H\,P_{k|k-1}\,H^T + R}

Kalman gain — the optimal blending weight (scalar form) — Numerator = how uncertain the prediction is (in measurement space) · Denominator = total uncertainty (prediction + measurement) · When R → 0, K → 1 (trust sensor). When P → 0, K → 0 (trust model).

Why Gaussians? Optimality and closed-form bliss

A Gaussian (normal ) is fully described by just two things: its mean (where is the center?) and its covariance (how spread out is it?). The Kalman filter exploits a remarkable property: the product of two Gaussians is another Gaussian. This means that blending a Gaussian prediction with a Gaussian measurement yields a Gaussian result — and the result's mean and covariance can be computed in closed form, with simple matrix algebra.

This is why the filter is so elegant: it never needs to store a full probability distribution. Just two quantities — mean and covariance — propagate forward, get updated, and propagate again. The entire history of past measurements is compressed into the current estimate and its uncertainty. This makes the filter recursive: constant memory per step, no matter how many measurements you've processed.

Open in Lab

Watch two Gaussians (prediction in blue, measurement in orange) fuse into a narrower green Gaussian. The peak always falls between the two inputs, closer to the more confident one.

The demo wakes as you arrive…

Tracking a moving object: the filter in action

Let's track a ball moving in one dimension. The state is $[position, velocity]^T$ . Between measurements, the ball obeys $position_{new} = position + velocity \times \Delta t$ . A noisy sensor reports the position (but not velocity) with some measurement noise.

Watch the interactive below: the red dots are noisy sensor readings. The blue line is the Kalman filter's estimate. Notice how the blue line is smoother than the red dots and closer to the true (green) trajectory — especially when sensor noise is high.

Open in Lab

Adjust sensor noise (R) and process noise (Q) to see how the filter adapts. High R → smooth line trusting the model. High Q → jagged line trusting measurements.

The demo wakes as you arrive…

The same idea in code

Kalman filter for 1D position tracking, completepython

Simplified to show the idea — not the real implementation.

import numpy as np

def kalman_filter(measurements, F, H, Q, R, x0, P0):
    """
    F: state transition matrix       (n x n)
    H: measurement matrix            (m x n)
    Q: process noise covariance       (n x n)
    R: measurement noise covariance   (m x m)
    x0: initial state estimate        (n x 1)
    P0: initial covariance            (n x n)
    """
    x = x0.copy()
    P = P0.copy()
    estimates = []

    for z in measurements:
        # ── PREDICT ─────────────────────────────────
        x_pred = F @ x                    # state prediction
        P_pred = F @ P @ F.T + Q          # covariance prediction

        # ── UPDATE ──────────────────────────────────
        y = z - H @ x_pred                # innovation (surprise)
        S = H @ P_pred @ H.T + R          # innovation covariance
        K = P_pred @ H.T @ np.linalg.inv(S)  # Kalman gain

        x = x_pred + K @ y               # corrected state
        P = (np.eye(len(x)) - K @ H) @ P_pred  # corrected covariance

        estimates.append(x.copy())

    return np.array(estimates)

# Example: track position and velocity from noisy position-only sensor
dt = 1.0
F = np.array([[1, dt],    # position = position + velocity * dt
               [0,  1]])   # velocity = velocity (constant velocity model)
H = np.array([[1, 0]])    # sensor measures position only
Q = np.array([[0.1, 0], [0, 0.1]])  # small process noise
R = np.array([[1.0]])     # sensor noise variance
x0 = np.array([[0], [1]])  # start at 0, velocity 1
P0 = np.eye(2) * 10       # very uncertain initially

Kalman vs Wiener: what changed

Kalman's approach was not just an improvement — it was a paradigm shift:

Domain: Wiener works in frequency. Kalman works in time. Time-domain means you can handle nonstationary systems naturally.
Memory: Wiener needs the entire signal history. Kalman needs only the current state estimate and covariance — constant memory.
Recursion: Wiener is a batch method. Kalman is recursive — one measurement at a time, perfect for real-time systems.
Multidimensional: Wiener struggles with multiple coupled variables. Kalman handles arbitrary state dimensions via matrix algebra.
Optimality: Both are optimal for their settings. But Kalman's setting (state space, recursive) is far more practical for engineering systems.

Open in Lab

Left: Wiener filter needs the full signal. Right: Kalman filter updates step by step, tracking a nonstationary signal the Wiener filter cannot follow.

The demo wakes as you arrive…

Covariance: the filter knows what it doesn't know

One of the Kalman filter's most powerful features is that it doesn't just give you an estimate — it gives you a confidence measure alongside. The covariance matrix $P$ tells you how uncertain the estimate is, in every dimension.

During the predict step, $P$ grows (you become less certain because of process noise). During the update step, $P$ shrinks (the measurement gives you information). Over time, $P$ converges to a steady-state value that reflects the fundamental trade-off between process noise (model uncertainty) and measurement noise (sensor uncertainty).

This self-awareness makes the Kalman filter far more useful than a simple moving average or exponential smoother: those give you an estimate but no sense of how much to trust it.

Impact: from the Moon to your pocket

1960
Kalman publishes the paper
Rudolf E. Kalman introduces the recursive state estimator in the ASME Journal of Basic Engineering. The paper reformulates filtering from frequency domain to state space.
1961
Kalman-Bucy continuous-time extension
Kalman and Bucy extend the filter to continuous-time systems, broadening its applicability to analog systems and differential equations.
1962
Apollo navigation system
NASA adopts the Kalman filter for Apollo spacecraft navigation. Stanley Schmidt at NASA Ames implements the Extended Kalman Filter to handle the nonlinear orbital mechanics, guiding astronauts to the Moon and back.
1973
GPS system design
The GPS system architecture incorporates Kalman filtering at its core for combining satellite signals with inertial measurements. Someone described the entire GPS as "one enormous Kalman filter."
1970s
ARIMA and time series
Box-Jenkins ARIMA models share deep connections with the Kalman filter. Any ARIMA model can be cast in state-space form and estimated via the Kalman filter, bridging control theory and econometrics.
2000s
Smartphone sensor fusion
Every smartphone fuses accelerometer, gyroscope, magnetometer, and GPS data through Kalman filters to provide smooth orientation, step counting, and location tracking.
2010s
Autonomous vehicles
Self-driving cars use Extended and Unscented Kalman Filters to fuse camera, lidar, radar, and GPS into a coherent estimate of the vehicle's state and the world around it.
2021
S4 and deep state space models
The Structured State Space for Sequences (S4) model brings state-space ideas back into deep learning, achieving strong results on long-range sequence modeling — a direct intellectual descendant of Kalman's state-space formulation.

The Kalman filter's legacy extends far beyond any single application. Its state-space formulation became the language in which modern control theory, signal processing, and robotics are written. Every time a system must estimate something it cannot directly observe — from a rocket's trajectory to an economy's hidden state — Kalman's framework is the starting point.

Beyond linear: extensions and limitations

The original Kalman filter assumes linearity and Gaussian noise. Real-world systems often violate both. Over the decades, engineers developed a family of extensions:

Extended Kalman Filter (EKF): linearizes the nonlinear system around the current estimate using first-order Taylor expansion. Used in most GPS receivers and spacecraft navigation. Simple but can diverge if the nonlinearity is severe.
Unscented Kalman Filter (UKF): instead of linearizing, it picks a small set of "sigma points" that capture the mean and covariance, propagates them through the true nonlinear function, and computes statistics from the result. More accurate than EKF for strongly nonlinear systems.
Particle Filters: abandon the Gaussian assumption entirely. They represent the probability distribution as a cloud of weighted samples (particles). Can handle arbitrary nonlinearities and non-Gaussian noise, at the cost of much more computation.

Each extension trades off accuracy against computational cost. The original linear Kalman filter remains the gold standard whenever the linear-Gaussian assumptions hold — which is surprisingly often in practice.

CitationKalman, R. E.. A New Approach to Linear Filtering and Prediction Problems. ASME Journal of Basic Engineering, 1960.

Terms in this paper

State Estimationتقدير الحالة
Kalman Filterمرشِّح كالمن
Covarianceالتباين المشترك
Predictionالتنبؤ
Gaussian Distributionالتوزيع الغاوسي
Recursiveتكراري
Noiseالضجيج الحسابي
Hidden Stateالحالة المخفية
Likelihoodالأرجحية
Markov Chainسلسلة ماركوف الاحتمالية
Bayesian Inferenceالاستدلال البايزي
Loss functionدالة الخسارة
Optimal Policyالسياسة المُثلى