The Bitter Lesson: Why AI Beats Clever Hacks
AI progress rarely comes from hand-crafted rules or clever architectures. The real gains come from scaling compute and data—a lesson researchers keep forgetting.
The bitter lesson is that general methods that leverage computation are ultimately the most effective, and by a large margin.
The Power of Scale
From GPT-3 to AlphaGo, scaling up neural networks and training data consistently outperforms hand-crafted heuristics.
Chess: A Case Study
Early chess AI relied on expert knowledge and hand-coded evaluation functions. Deep Blue beat Kasparov with brute-force search. Then came AlphaZero—just neural networks and self-play, no human chess knowledge—and it crushed everything.
Reinforcement Learning in Games
AlphaGo Zero learned to play Go from scratch, beating the previous champion that used human data. No opening books, no endgame tables—just compute and a simple algorithm.
Why We Resist the Lesson
Researchers love crafting elegant, interpretable solutions. But nature’s algorithms are messy and opaque. The bitter lesson forces us to admit that our cleverness is often a distraction.
Implications for Engineering
If scaling is the key, then infrastructure—GPUs, data pipelines, distributed training—matters more than novel algorithms. The winners in AI are those who can afford to scale.
We want AI agents that can learn like humans, but we keep forgetting that humans are just scaled-up primate brains.
