2017

Transformer Architecture

A team at Google publishes "Attention Is All You Need" — eight pages that quietly reshape the entire field. The Transformer architecture replaces sequential processing with self-attention, allowing models to process entire sequences in parallel. It's faster, more scalable, and dramatically more capable. GPT, BERT, PaLM, Claude — every foundation model that follows stands on this paper. The irony: it was designed for machine translation. Its authors had no idea they were building the engine for a revolution.