LESSWRONG
LW

Emergent Behavior ( Emergence )Language Models (LLMs)AI
Frontpage

25

Emergent Abilities of Large Language Models [Linkpost]

by aog
10th Aug 2022
1 min read
2

25

This is a linkpost for https://arxiv.org/pdf/2206.07682.pdf
Emergent Behavior ( Emergence )Language Models (LLMs)AI
Frontpage

25

Emergent Abilities of Large Language Models [Linkpost]
5Ruby
2Evan R. Murphy
New Comment
2 comments, sorted by
top scoring
Click to highlight new comments since: Today at 1:41 AM
[-]Ruby3y50

This is very cool.  Thanks for link posting!

Reply
[-]Evan R. Murphy3yΩ120

Those are fascinating emergent behaviors, and thanks for sharing your updated view.

Reply
Moderation Log
Curated and popular this week
2Comments

I've argued before against the view that intelligence is a single coherent concept, and that AI will someday suddenly cross the threshold of general intelligence resulting in a hard takeoff. This paper doesn't resolve that debate entirely, but it provides strong evidence that language models often have surprising jumps in capabilities. 

From the abstract: 

Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks. This paper instead discusses an unpredictable phenomenon that we refer to as emergent abilities of large language models. We consider an ability to be emergent if it is not present in smaller models but is present in larger models. Thus, emergent abilities cannot be predicted simply by extrapolating the performance of smaller models. The existence of such emergence implies that additional scaling could further expand the range of capabilities of language models.

Key Figures:

Related: More is Different for AI, Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets, Yudkowsky and Christiano on Takeoff Speeds