Neuroscience1949foundational9 min read
The Organization of Behavior: A Neuropsychological Theory
تنظيم السلوك: نظرية في علم النفس العصبي
Hebb, D. O. — Wiley
The problem
By the late 1940s, McCulloch and Pitts had shown that simple -like units could perform logical computations, but their model had fixed, hand-wired connections — no mechanism for learning or adaptation. Psychology offered behaviorist stimulus-response associations, but no neural explanation for how the brain could form, store, or recall internal representations. There was no bridge between the logic of single neurons and the complexity of perception, memory, and thought.
The contribution
Hebb proposed that synaptic connections strengthen when presynaptic and postsynaptic neurons fire together — "cells that fire together, wire together." This simple local rule provides a mechanism for without an external teacher. He also introduced "cell assemblies" — groups of neurons that, through repeated co-activation, form stable circuits representing concepts or percepts — and "phase sequences," chains of assembly activations that model the flow of thought. Together, these ideas gave neural networks a learning rule and a theory of internal .
The impact
Hebb's learning rule became the conceptual ancestor of virtually every learning algorithm in neural networks. The Hopfield network used Hebbian storage directly. The neocognitron built on local, unsupervised detection that Hebb's ideas inspired. Backpropagation itself, while not strictly Hebbian, fulfills the same goal — adjusting connection strengths from experience. Modern neuroscience confirmed Hebb's postulate through the discovery of long-term potentiation (LTP). The book launched and made "learning from co-activation" the founding principle of both computational neuroscience and deep learning.
Imagine a dirt field where people can walk anywhere. At first there are no paths. But when two people keep walking the same route together — from the library to the café, day after day — a trail forms. The more they walk it, the deeper and wider the path becomes, and soon others follow it too.
Hebb's insight is that the brain works the same way: when two neurons fire together repeatedly, the connection between them wears a deeper path — it grows stronger. No one designed the trail; it emerged from use. That is learning without a teacher.
The problem: neurons compute, but how do they learn?
In 1943, McCulloch and Pitts showed that networks of simple binary neurons could compute any logical function — AND, OR, NOT, and their combinations. This was a landmark: it proved that brains could, in principle, be understood as computing machines.
But their model had a critical gap: the connections were fixed. Someone had to choose the right weights by hand. There was no mechanism for the network to improve from experience. A McCulloch-Pitts network could compute, but it could not learn.
Meanwhile, behaviorist psychology studied learning purely through stimulus-response associations — Pavlov's dogs, Skinner's pigeons — but offered no neural mechanism. How did repeated experience physically change the brain? What was the biological "knob" that turned?
The idea: cells that fire together, wire together
Hebb's postulate can be stated in one sentence: if neuron A consistently helps fire neuron B, the synapse from A to B grows stronger. No external teacher is needed. The "teaching signal" is simply whether the two neurons are active at the same time.
Think of it as a voting system at a committee meeting. Every time two members vote the same way, they trust each other a little more and listen more closely next time. Over many meetings, clusters of like-minded members emerge — not because anyone assigned them to groups, but because their agreement history reshaped their influence on each other.
In modern terms, the update rule is:
The rule is beautifully local: each synapse only needs to know about its own two neurons. No global error signal, no backward pass through the network, no labeled data. Learning happens at the connection itself, from local coincidence alone.
Cell assemblies: when neurons form teams
Hebb did not stop at pairs of neurons. He proposed that through repeated co-activation, a group of interconnected neurons could form a — a stable circuit that fires as a unit. Once formed, activating part of the assembly ignites the whole group, even if the original stimulus is incomplete.
Imagine a group of musicians who have rehearsed together so many times that if you hum the first three notes, the entire ensemble can continue the melody. The assembly is the brain's representation of a concept — the letter "A," the smell of coffee, the face of a friend. It is not stored in one neuron but distributed across many, bound together by strengthened synapses.
This was revolutionary: Hebb gave the first plausible neural account of internal representations — mental objects that exist even when the external stimulus is gone.
Phase sequences: assemblies chaining into thought
A single cell assembly represents one concept. But thinking is a flow — one idea triggers the next. Hebb proposed phase sequences: ordered chains of cell assembly activations where each assembly, as it fires, facilitates the next one in the chain.
Think of it as a line of dominoes: each domino (assembly) falling triggers the next. But unlike dominoes, the sequence is learned — the brain wires the chain by strengthening connections between assemblies that historically activate in succession. Seeing a dog might trigger the assembly for "bark," which triggers "loud," which triggers "neighbor's dog."
Phase sequences were Hebb's attempt to explain the stream of consciousness in neural terms — how perception flows into memory flows into action, all through the same Hebbian mechanism of strengthening connections through co-activation.
The instability problem: weights that only grow
Hebb's rule has an elegant simplicity, but it carries a serious flaw: weights only increase, never decrease. Every time two neurons co-fire, their connection grows stronger. Over time, a dominant pattern will drive all weights upward without bound — the network "saturates" and loses the ability to distinguish between patterns.
This is like a social media algorithm that only adds friends and never unfollows: eventually everyone is connected to everyone, and the network carries no meaningful structure.
Later researchers solved this with modifications: Oja's rule adds a term that keeps weights bounded; BCM theory introduces a sliding threshold so that weakly active connections can decrease; and anti-Hebbian learning explicitly weakens connections between neurons that fire at different times. These are the engineering refinements that made Hebb's biological insight computationally stable.
The Hebbian idea in code
Simplified to show the idea — not the real implementation.
import numpy as np
def hebbian_update(W, x, y, eta=0.01):
"""Pure Hebbian update: dw = eta * x * y."""
return W + eta * np.outer(x, y)
def oja_update(W, x, y, eta=0.01):
"""Oja's rule: Hebbian + weight decay that keeps weights bounded."""
return W + eta * (np.outer(x, y) - y**2 * W)
# Simulate: two correlated input patterns
np.random.seed(42)
n_in, n_out = 5, 3
W = np.random.randn(n_in, n_out) * 0.01
for step in range(200):
# Correlated input: neurons 0, 1, 2 fire together
x = np.zeros(n_in)
x[:3] = np.random.rand(3) + 0.5 # active
x[3:] = np.random.rand(2) * 0.1 # quiet
y = np.tanh(W.T @ x) # postsynaptic response
W = oja_update(W, x, y) # learn
# After training, W[:3] are strong (correlated inputs),
# W[3:] are weak -- the network learned the pattern.
print("Weights from active neurons:", W[:3].round(3))
print("Weights from quiet neurons:", W[3:].round(3))From biology to computation: long-term potentiation
Hebb proposed his rule as a theoretical conjecture in 1949 — he had no direct experimental evidence. It took over two decades for neuroscience to catch up. In 1973, Bliss and Lømo discovered long-term potentiation (LTP) in the hippocampus: when a presynaptic neuron repeatedly stimulates a postsynaptic neuron, the synapse physically grows stronger, lasting hours to weeks. This was Hebb's postulate, confirmed in living tissue.
LTP works through NMDA receptors — molecular "coincidence detectors" that open only when both the presynaptic and postsynaptic neurons are active. This is nature's implementation of the Hebbian AND-gate: both must fire for the connection to strengthen.
The complementary process, long-term depression (LTD), weakens synapses when neurons fire out of sync — the biological anti-Hebbian mechanism that Hebb himself did not specify but that the brain needs for stability.
Why it mattered for AI
Hebb gave its first learning rule. Before him, neural networks were logic circuits. After him, they became learning machines. The line from Hebb runs through the core of the field:
-
Hopfield networks (1982) used Hebbian weight storage directly — the weight between two neurons equals the sum of their co-activations across stored patterns. This created associative memory: show part of a pattern, and the network completes it, exactly like Hebb's cell assemblies.
-
The neocognitron (1980) used local, unsupervised Hebbian-style learning to train a hierarchical visual feature detector — the architectural ancestor of modern convolutional neural networks.
-
Backpropagation (formalized 1986) is not Hebbian in mechanism — it sends a global error signal backward — but it fulfills the same purpose: adjusting connection strengths from experience. It is the engineering solution to the problem Hebb posed biologically.
Every time a adjusts its weights from data, it is doing what Hebb described in 1949: learning by changing the strength of connections.
The full picture
1943
McCulloch-Pitts Neuron
Binary neuron model that proved neural networks can compute logical functions — but with fixed, hand-wired connections and no learning.
1949
Hebb's Organization of Behavior
Introduced the Hebbian learning rule, cell assemblies, and phase sequences. Gave neural networks their first learning mechanism.
1958
Rosenblatt's Perceptron
Combined McCulloch-Pitts neurons with Hebbian-inspired weight adjustment. The first machine that could learn from examples.
1973
Long-Term Potentiation discovered
Bliss and Lømo confirmed Hebb's postulate in living hippocampal tissue — synapses physically strengthen when pre- and postsynaptic neurons co-fire.
1980
Neocognitron (Fukushima)
Used Hebbian-style unsupervised learning to build hierarchical visual feature detectors. Architectural ancestor of convolutional neural networks.
1982
Hopfield Network
Associative memory network using pure Hebbian weight storage. Showed that Hebb's rule creates content-addressable memory — show a fragment, retrieve the whole.
1986
Backpropagation formalized
Not Hebbian in mechanism, but fulfills Hebb's vision: adjusting weights from experience. Made deep learning possible by solving the credit assignment problem.
CitationHebb, D. O.. The Organization of Behavior: A Neuropsychological Theory. Wiley, 1949.
Terms in this paper
- Neuronالعصبون الاصطناعي
- Weightالوزن البنيوي
- Learning Rateمعدل التعلم
- Activation Functionدالة التنشيط
- Synaptic Plasticityاللدونة المشبكية
- Cell Assemblyالتجمُّع الخلوي
- Convergenceالتقارب الحسابي
- Unsupervised Learningالتعلّم غير الخاضع للإشراف
- Overfittingفرط التخصيص
- Connectionismالنظرية الترابطية