GPT-2

A new essay asks why transformer-scale language models took until 2019 to appear, when much of the underlying theory was understood well before 2005. It examines which ingredients — compute, data, algorithms, or attention — were actually missing, arguing the bottleneck was harder to pin down than commonly assumed. Readers are debating whether progress in AI was driven more by hardware or by ideas.