Core ML1959foundational10 min read
Some Studies in Machine Learning Using the Game of Checkers
دراسات في التعلُّم الآلي باستخدام لعبة الداما
Samuel, A. L. — IBM Journal of Research and Development
The problem
In the 1950s, programming a computer to perform a complex task meant spelling out every rule by hand. For a game like checkers — with roughly 10²⁰ possible positions — writing explicit rules for every situation was impossible. Could a machine instead learn from experience, the way a human player improves by playing many games?
The contribution
Samuel built a checkers program on the IBM 701 that learned in two complementary ways. First, : the program memorized board positions and their evaluated values, effectively extending its search depth at familiar positions. Second, learning: the program adjusted the weights of a linear — combining features like piece count, king advantage, mobility, and center control — by comparing its current evaluation against a deeper lookahead, an approach later recognized as an early form of learning. The program played against itself () to generate its own data and eventually surpassed its creator's playing ability.
The impact
Samuel coined the term "" and demonstrated that a computer could improve at a complex task without explicit programming. His generalization learning procedure was later identified as the first instance of temporal difference learning — the lineage that runs directly through TD-Gammon, Deep Q-Networks, and AlphaGo. Self-play, a concept Samuel pioneered, became the engine behind nearly every superhuman game-playing system since.
Imagine a chess coach who has never read a strategy book. She sits down at a board, plays a game, loses, and thinks: "That move where I gave up the center — that cost me." She adjusts her mental scorecard, plays again, and over hundreds of games her intuition sharpens until she can beat players who memorized the textbook.
Samuel's checkers program works the same way. It has a scorecard — a of board features — and two ways to sharpen it: memorizing positions it has seen before (rote learning) and adjusting its weights whenever its quick evaluation disagrees with a deeper lookahead (generalization learning). By playing thousands of games against itself, it became its own coach and its own sparring partner.
Why checkers? The perfect laboratory
Samuel needed a problem complex enough to be interesting yet simple enough for a 1950s computer. Checkers fit perfectly: it has roughly 5×10²⁰ possible positions — far too many to enumerate — yet the rules are simple and winning is clearly defined. Unlike toy problems, good checkers play demands genuine strategic thinking: sacrificing pieces for position, controlling the center, and planning several moves ahead.
The game also made evaluation natural. Unlike, say, medical diagnosis, there is an unambiguous outcome — win, lose, or draw — that the program can learn from. And because the program can play against itself, it generates unlimited training data for free.
Thinking ahead: minimax search and alpha-beta pruning
Before learning enters the picture, the program needs a way to choose moves. Samuel used the : build a tree of possible future moves, assume your opponent always picks the move worst for you, and pick the move that leads to the best outcome under that assumption.
At the leaves of this tree — the deepest positions the program looks at — it needs a number that says "how good is this position?" That number comes from the evaluation function, which we will explore next.
To avoid exploring the entire enormous tree, Samuel used : if a branch is provably worse than one already found, skip it entirely. This lets the program search roughly twice as deep in the same time — a crucial advantage on 1950s hardware.
The scorecard: the evaluation function
The evaluation function is the intellectual heart of Samuel's system. It takes a board position and returns a single number — positive means favorable, negative means unfavorable. Samuel designed it as a linear polynomial: a weighted sum of board features.
The features captured strategic knowledge a human player would recognize: piece advantage (how many more pieces do I have?), king ratio (kings are worth more than regular pieces), mobility (how many moves are available?), center control (pieces in the center can reach more squares), back-row defense, and more — up to 16 features at various points.
The critical insight was that the weights did not have to be set by a human expert. They could be learned.
Learning method 1: rote learning — memory as depth
The simplest form of learning in Samuel's system was rote learning: memorize board positions encountered during play, along with the backed-up minimax value computed for each. When the program later encounters the same position — either on the board or during its lookahead search — it can use the stored value directly instead of searching further.
Think of it as building a library of solved positions. Each entry in the library acts like an extension of the : when the program hits a stored position during its lookahead, it can stop searching and use the stored value as if it had searched much deeper.
Samuel added a crucial refinement: a "sense of direction." He decreased a position's stored value slightly each time it was backed up a level during minimax. This meant that the program preferred winning sooner rather than later, and delayed losing as long as possible — exactly the right incentive structure for game play.
Rote learning produced slow but steady improvement. It was most effective for opening and endgame positions, where the same patterns recur frequently. The program reached "better-than-average novice" level through this method alone.
Learning method 2: generalization — the ancestor of TD learning
Rote learning is powerful but limited: it only helps at positions the program has seen before. Samuel wanted the program to extract general principles — to learn that center control matters, not just memorize one specific center-controlling position.
His solution: maintain two copies of the evaluation function. The active copy is used during play and is constantly being adjusted. A stable copy serves as the learning target — it evaluates positions during a deeper lookahead and provides the "correct" values that the active copy tries to match.
After each move, the program compares the active copy's evaluation of the current position with the stable copy's deeper evaluation. The difference — essentially, "how wrong was my quick estimate?" — is used to adjust the active copy's weights. If the deeper search reveals that center control matters more than the quick evaluation realized, the for center control increases.
The idea — "update your current estimate toward a better future estimate" — is exactly what Richard Sutton formalized in 1988 as temporal difference (TD) learning. Sutton explicitly identified Samuel's checkers player as "the earliest and best-known use of a TD method." This is not historical trivia — it is the direct ancestral line that runs from Samuel's IBM 701 through TD-Learning, TD-Gammon, and all the way to AlphaGo.
Self-play: the program as its own opponent
Where does the training data come from? Samuel made the program play against itself. One copy uses the current evaluation function; the opponent copy may use the stable version or a slightly older version. Every game generates dozens of board positions, each a training example.
This is a deceptively powerful idea. Human players improve by finding opponents at the right skill level — but a self-play system automatically provides a perfectly matched opponent that grows stronger exactly as fast as the learner does. No human supervision is needed, no database of expert games is required, and the supply of training games is unlimited.
Samuel noted a problem, however: the program could develop blind spots. If it never explored certain lines of play, it might never learn to handle them. He mitigated this by occasionally playing against human opponents and by incorporating positions from published games, providing an external check on the self-play loop.
The idea in code
Simplified to show the idea — not the real implementation.
import numpy as np
def evaluate(board, weights):
"""Score a board position using weighted features."""
features = np.array([
piece_advantage(board), # my pieces minus opponent's
king_advantage(board), # my kings minus opponent's
mobility(board), # number of legal moves I have
center_control(board), # pieces on center squares
])
return weights @ features # dot product = weighted sum
def update_weights(weights, board_now, board_next,
stable_weights, lr=0.01):
"""Proto-TD update: nudge active weights toward deeper eval."""
v_active = evaluate(board_now, weights)
v_stable = evaluate(board_next, stable_weights)
error = v_stable - v_active # temporal difference
features_now = extract_features(board_now)
weights += lr * error * features_now # gradient step
return weights
# Self-play loop: play games, learn from each position
for game in range(1000):
boards = play_game(weights, stable_weights)
for t in range(len(boards) - 1):
weights = update_weights(
weights, boards[t], boards[t+1], stable_weights
)
if game % 50 == 0:
stable_weights = weights.copy() # refresh targetWhy it changed everything
When Samuel demonstrated his checkers program on television in 1956, it was a sensation. A computer learning to play a game — and beating its programmer — captured the public imagination in a way that abstract theorems could not. The program became a landmark proof of concept for the entire field of artificial intelligence.
The technical legacy is equally profound. The generalization learning procedure was the first temporal difference method. Self-play reappeared in TD-Gammon (1992), AlphaGo (2016), and AlphaZero (2017). The linear evaluation function with learned weights foreshadowed every that outputs a single scalar value — including the value networks in modern game-playing systems. Samuel's idea that a machine could "learn without being explicitly programmed" became the definition of an entire discipline.
1952
Samuel begins checkers program
Arthur Samuel starts building his checkers-playing program on the IBM 701 at Poughkeepsie, New York. One of the earliest attempts to make a computer learn.
1956
Television demonstration
Samuel demonstrates the program on national television. A computer playing checkers — and learning from its mistakes — captivates the American public.
1959
Landmark paper published
"Some Studies in Machine Learning Using the Game of Checkers" appears in the IBM Journal, coining the term "Machine Learning (ML)" and describing both rote and generalization learning.
1962
Beats a human champion
The program defeats Robert Nealey, a blind checkers champion, in a publicized match. The media (prematurely) declares checkers "solved."
1988
Sutton formalizes TD learning
Richard Sutton publishes the formal theory of temporal difference learning, explicitly crediting Samuel's checkers player as the first TD method.
1992
TD-Gammon
Gerald Tesauro combines TD learning with neural networks to build a backgammon program that reaches master-level play — Samuel's ideas with modern hardware.
2016
AlphaGo defeats Lee Sedol
DeepMind's AlphaGo uses self-play, learned evaluation (value networks), and tree search to defeat the world Go champion — the direct intellectual descendant of Samuel's architecture.
Every modern game-playing AI — from chess engines to Go to Dota 2 — is a descendant of the three ideas Samuel combined in 1959: an evaluation function, a search algorithm, and learning from self-play. The field he named is now the most transformative technology of the 21st century.
CitationSamuel, A. L.. Some Studies in Machine Learning Using the Game of Checkers. IBM Journal of Research and Development, 1959.
Terms in this paper
- Machine Learning (ML)تعلم الآلة
- Self-Playاللعب الذاتي
- Minimaxالأصغري-الأعظمي
- Alpha-Beta Pruningتقليم ألفا-بيتا
- Rote Learningالتعلّم بالحِفظ
- Evaluation Functionدالّة التقييم
- Reinforcement Learningالتعلم المعزز
- Temporal Differenceالفارق الزمني الحسابي
- Generalizationالتعميم
- Featureميزة / سمة