Gato

A Gato mention now works mostly as a clue about when the sentence around it was written. The annotation on the Gato reference in The p(doom) song, annotated calls the line "very early", written while one network doing 604 things still looked like the road to AGI. That is osmarks's judgement of the line, not a verdict on the model.

The figures are exact. The paper is arXiv 2205.06175, posted 12 May 2022. The model is one 1.2-billion-parameter transformer. The task count is 604 distinct tasks: playing Atari, captioning images, stacking blocks with a robot arm, all in one set of weights.

Gato was modest at each task and remarkable for doing all of them. The breadth arrived with no claim of strength at any one of them: nobody said Gato was the best Atari player or the best captioner. The result was the single network.

Which is also where the word "agent" misleads. "A generalist agent" is DeepMind's own framing of the paper, and the generality it claimed was across task types and modalities, at small scale. It is not a large language model. And "agent" there does not mean what it means now, where the word points at a tool-calling loop with a model deciding its own next step. Gato was one network answering many kinds of input.

Why the number is 604

Not 600, not "more than 600", not "around 600". The figure comes from the paper's Figure 1 caption, and the annotated essay's fact pass tightened an earlier "600+" to 604 for that reason. A count that round is usually a count somebody rounded, and here the exact number was sitting in a caption the whole time.

The same holds for the parameter count. 1.2 billion is the figure, and it is small enough that the breadth is the whole of the result.