Emergent Capabilities

One of the most intriguing - and debated - aspects of scaling AI models is emergence: the appearance of capabilities that weren't explicitly trained for and weren't present in smaller models. A model that can't do arithmetic at one size might suddenly handle it at a larger size. Translation between languages that had minimal representation in the training data might appear unexpectedly. Complex reasoning, analogical thinking and even rudimentary theory of mind seem to emerge as models grow. This excited the field enormously because it suggested that simply making models bigger might unlock intelligence-like capabilities for free. However, the story has become more complicated. Some researchers have argued that what looks like sudden emergence is partly an artefact of how we measure performance - with different evaluation metrics, the transitions look more gradual. Others have shown that some "emergent" capabilities can be induced in smaller models with the right training techniques. For practical purposes, emergence matters because it means the gap between what an AI model was trained to do and what it can actually do may be larger than you expect - in both directions. Models might surprise you with unexpected capabilities, but you can't reliably predict which capabilities will appear or count on them being consistently available.