Source: Prisoner's Dilemma and Axelrod's Tournament (Veritasium)

Bibliographic Info

Source: Veritasium YouTube video (Derek Muller), featuring Robert Axelrod and Steven Strogatz
Raw file: raw/notes/prisoner's dilemma and axelrod's tournament.md
Original research: Robert Axelrod, The Evolution of Cooperation (1984)

The Prisoner’s Dilemma

A two-player game where each player simultaneously chooses to cooperate or defect. Payoffs:

	Opponent Cooperates	Opponent Defects
You Cooperate	3, 3	0, 5
You Defect	5, 0	1, 1

The dilemma: Regardless of what your opponent does, you are always individually better off defecting. If they cooperate, you get 5 instead of 3. If they defect, you get 1 instead of 0. Defection dominates. Yet when both players follow this logic, they both get 1 — far worse than the 3 each they’d get from mutual cooperation.

This is the formal structure of dozens of real-world situations: the arms race (US/Soviet nuclear buildup), pollution, tax compliance, team free-riding, impalas grooming each other, fish cleaning sharks.

Cold War illustration: Both the US and USSR spent ~$10 trillion developing nuclear arsenals neither could use. They’d both have been better off not building them — but rational self-interest drove them toward mutual defection.

The Repeated Prisoner’s Dilemma

In a one-shot game, defection is always rational. But most real interactions are repeated: impalas see each other daily, nations interact continuously, colleagues share an office for years. In the repeated game:

Your current defection affects your opponent’s future behavior
The prospect of future cooperation has value worth protecting
The game changes fundamentally — cooperation can become rational

Axelrod’s question: What is the best strategy in the repeated prisoner’s dilemma?

Axelrod’s Computer Tournaments (1980)

Political scientist Robert Axelrod ran two computer tournaments where submitted strategies played every other strategy for 200 rounds (first tournament), then an uncertain number of rounds (second tournament).

Tournament 1 (1980): 15 strategies

The simplest strategy won: Tit for Tat (TFT), submitted by Anatol Rapoport.

Tit for Tat rules:

Start by cooperating
On every subsequent move, do exactly what your opponent did last round
That’s it — 4 lines of code

Tournament 2: 62 strategies

Two camps formed: those who submitted nice/forgiving strategies (correctly inferring TFT’s dominance), and those who submitted nasty strategies trying to exploit forgiving ones. TFT won again. Top 15 strategies: 14 were nice. Bottom 15: 14 were nasty.

Key finding: Tit for Two Tats

If anyone had submitted “Tit for Two Tats” (only retaliate after two consecutive defections) to the first tournament, it would have won. But in the second tournament — with players anticipating niceness — it came 24th, because nasty strategies exploited its extra forgiveness. There is no single universally best strategy; the best strategy depends on the environment.

The Four Qualities of Winning Strategies

Axelrod analyzed all top-performing strategies and found four shared properties:

1. Nice

Never be the first to defect. All top 8 strategies in tournament 1 were nice. Even the worst nice strategy outscored the best nasty strategy.

2. Forgiving

Retaliate when necessary, but don’t hold a grudge. TFT forgives immediately — one cooperation from the opponent and TFT cooperates again. Contrast with Friedman (maximally unforgiving: one defection → defect for the rest of the game). Friedman’s unforgiving nature traps it in mutual defection loops with any strategy that makes even one mistake.

3. Retaliatory (Provokable)

If your opponent defects, respond immediately. Don’t be a pushover. “Always cooperate” scores 0 against exploitative strategies. TFT makes exploitation unprofitable: the one extra coin you gain from defecting is wiped out by TFT’s retaliation.

4. Clear (Transparent)

Strategies that are too complex or random are treated by opponents as unpredictable, making cooperation impossible to establish. TFT’s simplicity makes its intentions legible — opponents can predict and trust it.

Axelrod: “These four principles — being nice, forgiving, provokable, and clear — are a lot like the morality that has evolved around the world, often summarized as ‘an eye for an eye.‘”

The Ecological Simulation

Axelrod ran a further simulation where successful strategies bred more copies and unsuccessful ones shrank — an ecological model of strategy evolution.

Findings:

Nasty strategies initially dominated in some environments but then collapsed as the nice strategies they preyed on went extinct
After 1,000 generations, only nice strategies survived
TFT represented 14.5% of the total final population

Cluster effect: Even in a world dominated by always-defect, a small cluster of TFT players cooperating with each other can accumulate enough points to spread and eventually take over. Cooperation does not require a cooperative world to get started — just a nucleus of cooperators who interact frequently with each other.

Noise and Generous Tit for Tat

In the real world, signals are noisy. A cooperation can be accidentally perceived as a defection (e.g., the 1983 Soviet early-warning system that misidentified sunlight on clouds as a US missile launch — Stanislav Petrov dismissed the alarm, avoiding nuclear war).

When TFT plays against TFT in a noisy environment: one mistaken perception triggers retaliation, which triggers retaliation, creating an echo effect of mutual defection. TFT’s performance collapses from near-perfect to one-third of its clean-environment score.

Solution: Generous Tit for Tat (GTFT). Retaliate only ~90% of the time rather than 100%. The 10% forgiveness breaks echo effects while remaining retaliatory enough to deter exploitation. GTFT outperforms pure TFT in noisy environments.

Zero-Sum vs. Non-Zero-Sum

A key conceptual insight: most of life is not zero-sum.

Chess and poker are zero-sum: one player’s gain is necessarily another’s loss
The Prisoner’s Dilemma is non-zero-sum: mutual cooperation produces more total value (3+3=6) than mutual defection (1+1=2)

TFT can never win a single game — by design, it can only tie or lose. Yet it wins the tournament. This looks paradoxical until you see that the goal is not to beat your opponent but to accumulate points from the “banker” (the world). Finding win-win situations and unlocking joint value is the winning strategy over the long run.

Cross-wiki Connections

inclusive-institutions

Axelrod’s finding that cooperation can emerge from self-interested actors who interact repeatedly is the micro-level mechanism for how inclusive-institutions sustain themselves. Rule of law, contracts, and property rights are institutional encodings of TFT logic — they make cooperation the dominant strategy by reliably punishing defection and rewarding cooperation.

extractive-institutions

Extractive systems impose one-shot dynamics on what should be repeated games: when the powerful can extract without consequence, there is no future shadow to discipline defection. Eliminating the “repeat” is how extractive elites make defection rational.

invisible-hand

The invisible hand is Smith’s argument that self-interested behavior produces social benefit. The repeated prisoner’s dilemma formalizes this for cooperative settings: self-interest (wanting future cooperation) generates cooperation, even with no altruism, no central authority, no explicit agreement.

high-agency

High-agency individuals are TFT players operating in a world full of various strategies. They cooperate by default (nice), respond decisively to exploitation (retaliatory), forgive and reset when opponents cooperate again (forgiving), and are legible and predictable (clear). The TFT framework is a formalization of effective interpersonal strategy.

lollapalooza-effect

Echo effects in noisy environments are a lollapalooza phenomenon in reverse: small miscommunications compound into total mutual defection. The solution (GTFT’s occasional forgiveness) is the cooperative equivalent of a circuit breaker — preventing cascade failure.

bayes-theorem / probability-theory

The simulation’s evolutionary dynamics are essentially probabilistic: strategies reproduce in proportion to their fitness. The cluster-emergence phenomenon (TFT taking over an all-defect world from a small nucleus) is a result about probability and selection dynamics.

fallibilism

GTFT’s 10% forgiveness is operationalised fallibilism: build in the assumption that signals are noisy and your read of the situation may be wrong. Don’t treat your perception of defection as certainly correct — maintain some probability that you misread the situation.

Key Quotes

“The crazy thing was that the simplest program ended up winning.” — Axelrod

“Nice guys finished first.” — Derek Muller

“In the short term, it is often the environment that shapes the player. But in the long run, it is the players that shape the environment.”

“Cooperation pays even among rivals.”

🪴 PG Notes

Explorer