grokking

A training run can look finished to anyone watching the numbers. Loss, the number a model is trained to lower, has bottomed out. The model has overfit its training data. Nothing further should be coming. Then accuracy on data the model never saw climbs from chance to perfect generalization. That late jump, arriving after the interesting part was supposed to be over, is grokking.

The thing it gets confused with has the opposite shape: a sudden drop in training loss partway through a run, read as the model picking up a new trick. That drop gets called grokking constantly, and it is a different event. One is a drop during the run. The other is a late jump on data the model never saw.

The paper is specific about the shape. Power, Burda, Edwards, Babuschkin and Misra, arXiv 2201.02177, v1 6 January 2022. The abstract describes test performance climbing from chance to perfect generalization long after the model has overfit the training data. Perfect generalization is the paper's own wording for the end state, and softening it loses what the result claimed.

The essay's own slip

The p(doom) song, annotated made exactly this mistake and prints the correction. Its first draft described a sudden drop in training loss as "the phenomenon called grokking", and the fact pass behind the essay recorded that as a conflation and separated the two. Its section on what checking the dates cost gives grokking its own paragraph and calls it the one worth knowing about. The error survived a draft that cited the paper.

The song itself is pointing at the drop, not at grokking: osmarks, who annotated it, links a LessWrong thread about a loss drop on that line rather than the grokking paper.

A stable training run is a long job that does not diverge, with loss blowing up or gradients going to infinity as the failure it avoided, and "stable" is the engineer's quiet word for success. That is a fact about the run. Grokking is a fact about what the model turned out to have learned.