TL;DR: I wrote this essay for a class. I tried using token-level surprisal, entropy and LessWrong's own S2 as instruments for close reading, with Sylvia Plath's "Daddy" as the case study. Using a corpus of 295 poems by 31 canonical poets (canonicity proxied by quizbowl answerline frequency), I find that a few information-structural features predict a bit of canonicity within a poet's work (adjusted R² = 0.136). "Daddy" itself is more predictable word-by-word than 96% of the corpus, but where it does deviate, the deviations are measurably among the most extreme.
The scripts for scraping and feature regressions were built with Claude Code, and surprisal extraction was done with distilgpt2. The prose and the readings are my own. Replication package on GitHub.
“Any utterance produces a continually shifting sense of semantic weight and directionality, and reading always entails a continuous process of prediction, continuously modified as expectations are met, intensified, or disappointed.”
— Derek Attridge
An Entropic Lens
If prose, in a general sense, optimizes for predictability and efficient communication, verse traditions deliberately deviate from the reader’s expectations through unpredictability in word choice and imagery. Viktor Shklovsky called this “defamiliarization”: the deliberate roughening of language to encourage perception.
Large language models offer, for the first time, a method for operationalizing expectation precisely. Predicting the next token in a passage of text is the defining function of large language models (LLMs): For any input of words, a LLM outputs a probability distribution, a “probability of being next” for each possible next word. An oft-overlooked implication, however, is that the “surprise” of a word is neatly captured by how unlikely it was. This quantity is the negative log-probability of the word given its context, which we denote surprisal. Along with surprisal, this model also yields entropy: a measure not of how surprising the word that arrived was, but of how uncertain the model was before it arrived—captured by how many plausible continuations existed at that position. In information-theoretic terms, text of low entropy is compressible: it is so predictable that it affords readers little new information, and thus could (in a general sense) be encoded more efficiently. High entropy text, by contrast, resists such compression.
“[R]emarkably enough,” as Simon DeDeo wrote in A Cheat-Sheet for Text Analysis, “these numbers capture basic features of human experience of language.” The empirical results are clear: higher surprisal at a given position predicts longer reading times (Hale, 2001; Levy, 2008; Smith & Levy, 2013), correlates with N400 amplitude (Frank et al., 2015), tracks the movement of the eye during naturalistic reading (Demberg & Keller, 2008), and corresponds to fMRI imaging of the language-relevant areas of the brain (Shain et al., 2020); entropy, meanwhile, predicts the cognitive cost of anticipation (Pimentel et al., 2023), and speakers reliably smooth it across utterances (Jaeger, 2010). The implications are important: as the psycholinguistic evidence establishes that surprisal tracks the reader's word-to-word cognitive experience, the question of where and how a poem generates its affective power becomes, at least to some extent, empirical. With these new tools, we can locate—to the word and even the token—where a model reader fails to predict what comes next, with empirical warrant for supposing this failure often corresponds to what a reader registers as emotionally and analytically resonant.
The power and promise of this method has been anticipated for decades: Claude Shannon himself, in a 1951 paper, used human subjects as stand-ins for language models to estimate the entropy of printed English,[1] and Leonard Meyer's Emotion and Meaning in Music (1956) applied a structurally identical framework to musical experience, arguing that emotional response arises from the violation and fulfillment of expectation built up by familiarity with tonal conventions. Long before LLMs, Miall and Kuiken (1994) demonstrated behaviorally that stylistically deviant literary passages are associated with slower reading speed. With modern tools, however, the method has become massively more precise and accessible: where earlier approaches relied on human judgment or small controlled stimuli, LLM-assessed surprisal and entropy can be computed quite accurately for any text, too cheaply—nearly—to meter. Computational approaches to literary texts have, of course, grown substantially in the digital humanities: Kao and Jurafsky (2012) used stylometric features to distinguish professional from amateur poetry, and Reagan et al. (2016) mapped six dominant "emotional arc structures’ across 1,327 narratives by tracking sentiment trajectories, providing the suggestion that the information structure of narrative can be quantified and systematically compared. Most directly, Momen et al. (2026) applied LLM surprisal to metaphor novelty judgments, finding moderate but significant correlations between surprisal scores and human ratings of the figurative deviance of metaphors. Still, the use of token-level surprisal and entropy as tools of literary criticism proper remains a lacuna that this essay proposes to fill.
To this end, we construct a new lens. Using a corpus of 295 poems across 31 poets—canonical figures selected by frequency of appearance as answerlines in academic quizbowl competitions—we compute token-level surprisal and entropy for each poem using distilgpt2, and ask what information-structural features distinguish more canonical poems from less canonical ones within the same poet's work. An obvious and crucial line of inquiry for users of this method, of course, is determining what information-structural features are associated with "good" writing. A full treatment of this question is beyond the scope of this paper, but some broader analysis makes the problem of examining the information theoretic choices of our "case study" poem more tractable. To this end, we employ an outcome variable: "canonicity," operationalized by mention in academic quizbowl competitions. Quizbowl is a popular high school and collegiate trivia competition format in which school teams race to answer questions with buzzers. Quizbowl question data, we contend, provides an unusually reliable proxy for literary canonicity because the questions are written and vetted by a community with strong incentives to construct a reliable proxy for the poetic canon. Unlike citation counts or syllabi inclusions, quizbowl question selection is rigorously tested and policed: the community and numerous "play testers" are quick to penalize "bad" questions that reward trivia over genuine literary knowledge or misrepresent a work's standing. The result is a corpus of difficulty-calibrated knowledge that reflects what well-read undergraduates and graduate students across institutions recognize as canonical. By subtracting each poet’s average from both the outcome (log canonicity) and the features, the regression focuses only on variation within poets, in order to control for differences in style.
When a line registers at a z-score of +5.57 on excess surprisal, we can say not just that it is surprising in some abstract sense (and there’s scarcely a PhD in the world with reliable intuition for what “5 bit of surprisal” feels like to read in contemporary poetry) but that it is more surprising than virtually any comparable moment in a known distribution of literary language: The corpus allows us to make empirical, positioned claims where there was once only the possibility of impressionistic ones.
Entirely new to the domain of literature, S2—or excess surprisal—is the chief measure we will employ. As entropy measures how uncertain the model was as to the next token, regardless of which token actually arrived, subtracting entropy from surprisal isolates the contribution of the specific word choice (Fraser, 2025): a high S2 means the word was more surprising than its context alone would predict. A negative S2 means the word arrived in a position of already-high uncertainty and was not especially unexpected given that uncertainty.
Naturally, information-structural features explain only a relatively limited amount of what makes a poem canonical: canonicity is essentially a social and historical phenomenon. But we can be confident it explains something: 10-15% of the variation in canonicity (Adjusted R² 0.136). Concrete imagery ratio (r = +0.204), Hidden Markov Model (HMM) runway (r = +0.128), and negative excess entropy (r = -0.126) emerge as significant predictors—across 295 poems, 31 poets, controlling for between-poet differences in style—suggesting that the information architecture of a poem is not a mere epiphenomenon of its content, but to some extent an independent dimension of what makes it memorable. Our full results are in Figure 1.
Figure 1. Top 18 significant predictors of log canonicity (quizbowl question appearances) across 295 canonical poems from 31 modern English language poets. Within-poet demeaned Pearson r. Red bars = features cited in the essay. *** p < .001, ** p < .01, * p < .05.
One of the “stylized facts” of this nascent field is that speakers and writers tend to smooth information across utterances (Jaeger, 2010; Levy & Jaeger, 2007)—a phenomenon termed “Uniform Information Density” and considered the default condition of fluent language. We find support for the thesis that poetry works systematically differently: though poetry is not on average denser or stranger—the formal regularities of verse make it, if anything, more predictable than prose—the qualities that make poetry poetic are associated with surprisal that is concentrated.
As our case study in this new method in criticism, we will examine the information-theoretic profile of Sylvia Plath’s “Daddy.” Against the baseline, "Daddy” presents a paradox: it is more compressible (i.e. predictable), word-by-word, than 96% of canonical poems, yet the sections where it does deviate from expectation are among the most extreme in the corpus—the quantitative results are in Figure 2.
Figure 2. Information-structural profile of Sylvia Plath's "Daddy" across 14 features, expressed as z-scores relative to Plath's canonical poems (n = 18) and the full canonical corpus (n = 295). Red bars denote features cited in the essay.
The following sections analyze this apparent paradox.
The Opening
The concrete imagery ratio is the single strongest predictor of canonicity among the features we analyzed (r = +0.204). This feature measures the proportion of a poem's language that is devoted to concrete sensory objects rather than abstract states or emotions. A poem that says “I was trapped” is cheap; a poem that says “black shoe / In which I have lived like a foot” (S2 = +2.213 on “black shoe”) is costly.
You do not do, you do not do
Any more, black shoe
In which I have lived like a foot
For thirty years, poor and white,
Barely daring to breathe or Achoo.
The first line (S2 = −1.917) establishes a baseline for compression. Once the model sees “You do not do” once, the repeated clause will "cost" almost nothing, for the model has learned the pattern and can predict the recurrence with high confidence. It is, as it were, a first act to train the reader to expect rhythmic, maybe anaphoric, language. The shoe line arrives with that trained expectation in place and violates it. Notice how the equal length, double clause with its syntactically disconnected, assonant line gives a feeling of a disembodied anaphora.
The most explicit statement of the poem's subject is notably compressed by comparison:
Daddy, I have had to kill you. (S2 = −2.488)
The line has the third-lowest S2 line in the poem: this declaration, which constitutes the thesis of the confessional, is cheap—unsurprising, as the model is already conditioned on a second-person confessional address. Figure 3 reveals this pattern at the token level.
Figure 3. Token-level surprisal (red) and entropy (blue dashed), opening section. Surprisal collapses across the repeated “You do not do” (line 1, S2 = −1.917), then sharply rises by “shoe” in line 2. “Daddy, I have had to kill you” (line 6, S2 = −2.488) is the third-lowest S2 line in the poem.
"Concrete Imagery’ and the Statue Stanzas
The subsequent movement revels in concrete images of body parts. The father is described as “marble-heavy, a bag full of God” (S2 = −0.889), then as a “ghastly statue with one gray toe / Big as a Frisco seal” (S2 = +2.019, +0.276), and finally as a head lodged “in the freakish Atlantic.” Applying the adjective “gray” to a toe after the mention of a “ghastly statue”' evokes a shift in scale (to a color-material pairing) that the model has no precedent for, reflected in a sizable spike in S2. 'Where it pours bean green over blue' (S2 = +2.750) corresponds to the section's highest S2 spike, marking an unprecedented chromatic construction and, perceptibly, the most vivid image in the passage as shown in Figure 4.
Figure 4. Token-level surprisal (red) and entropy (blue dashed), statue stanzas. "Ghastly statue with one gray toe" (S2 = +2.019) and "Where it pours bean green over blue" (S2 = +2.750) are the section's highest spikes. Each S2 peak corresponds to a concrete sensory object landing in a context that was not prepared for it.
HMM Runway
The Hidden Markov Model (HMM) runway—the number of low-surprisal tokens the poem sustains before reaching peak S2—is the second of the three canonicity features we used (r = +0.128). The model posits two latent states: a low-emission state, in which the poem produces language closely aligned with contextual predictions, and a high-emission state, in which the language exceeds the running baseline in informational cost. Naturally, however, a poem does not announce its current state: “state” cannot be measured directly. Instead, it must be inferred mathematically from the sequence of surprisal values.
After "learning” the emission distributions and transition probabilities that best explain the data, the model categorizes the poem into states of relative predictability and violation. The "runway," for our purposes, is the length of the final low-emission stretch before the poem's peak S2 token. A longer runway generally results in a more pronounced spike; the more the reader is conditioned to expect a particular linguistic register, the greater the informational cost when a deviation occurs—"payoffs” following "setups." "Daddy" has an HMM runway of z = +3.71—the most extreme value in Plath's corpus—showing greater investment in compression than nearly any other poem in our corpus.
This "investment” is most legible in the geographical stanzas. After the statue imagery, the poem turns toward the father's origins and produces its most compressed passage outside the ich stanza:
In the German tongue, in the Polish town (S2 = −0.490)
Scraped flat by the roller
Of wars, wars, wars. (S2 = −0.209)
But the name of the town is common. (S2 = −2.678)
Though this stanza may have many features that seem exotic to us, the model is less perplexed. After the model sees the German, the formulaic geographical enumeration is found to be predictable enough. The repetition of "wars" also loses informational value with each occurrence; after the first token, the model assigns near-certainty to subsequent occurrences. "But the name of the town is common” registers the second-lowest S2 in the poem; syntax and surprisal flatten to near zero. The poem maintains this nadir of sustained compression through most of its first movement, before a very significant shift.
My Polack friend (S2 = +5.817)
“My Polack friend” is the poem's highest-S2 line, 3.3 standard deviations above its mean. The colloquial possessive, casual American diction, and (especially) arcane, somewhat humorously-connotated ethnic qualifier are a break from the preceding solemnity; every aspect of the prior register is disrupted. The informational cost to the reader matches the preceding runway: The poem sustains 53 tokens in a low-surprisal build-up, conditioning the reader in heavy, imprecise geography with a thoroughness that adds to the affective effect once the condition shifts. Figure 5 illustrates this passage.
Figure 5. Token-level surprisal (red) and entropy (blue dashed), in the "Polack’ passage. The geographical enumeration flattens to near-zero S2: 'But the name of the town is common' (S2 = −2.678) is the second-lowest line in the poem. 'My Polack friend' (S2 = +5.817) is the poem's highest S2 line, and memorably arrives after 53 tokens of low-surprisal build-up.
Code Switching and Information Structure
The poem's most architecturally deliberate passage spans the transition from English to German and back. The excess entropy feature (r = −0.126) measures the degree to which a poem's surprisal is concentrated in a few locations rather than distributed uniformly. A negative excess entropy—which "Daddy" has, at z = −0.89—means the poem is strategic about where it spends its "information budget." The ich stanza is the clearest demonstration of this strategy at the level of lines:
The tongue stuck in my jaw. (S2 = +1.787)
It stuck in a barb wire snare. (S2 = +2.306)
Ich, ich, ich, ich, (S2 = −2.193)
I could hardly speak. (S2 = +0.575)
I thought every German was you. (S2 = +2.163)
And the language obscene (S2 = +2.841)
The first two lines are characterized by escalation: each image of linguistic impasse is more specific and unexpected than the last. Barbed wire is a significantly more extreme and immediate metaphor for a stuck tongue than the "jaw" alone predicts, and the S2 naturally reflects this. Then the stutter: "Ich, ich, ich, ich" (S2 = −2.193)—the surprisal collapses. Once the first token is processed, the model predicts the repetition with near-certainty; and the information stream thins to almost nothing. The poem describes a breakdown of language, and the information structure enacts it—language stripped to a repeated token is, computationally, near information-free.
The rebound is immediate: “I thought every German was you” (S2 = +2.163) and “And the language obscene” (S2 = +2.841) are the passage's highest values. The return of English, semantically loaded, is expensive compared to the cheap preceding compression. This compressional, emissional architecture that the HMM detects at the whole-poem level is visible here in miniature—a six-line version of the poem's complete information structure. Figure 6 shows this at the token level.
Figure 6. Token-level surprisal (red) and entropy (blue dashed) in the "ich’ stanza. After escalation through 'tongue stuck in my jaw' (S2 = +1.787) and 'barb wire snare' (S2 = +2.306), S2 collapses to near-zero across 'Ich, ich, ich, ich' (S2 = −2.193)—the information structure enacts the breakdown it describes! It rebounds on 'I thought every German was you' (S2 = +2.163) and 'the language obscene' (S2 = +2.841).
The Nazi Imaginary
The cataloging of Nazi iconography offers one of the poem's more counterintuitive architectures:
With your Luftwaffe, your gobbledygoo.
And your neat mustache (S2 = +2.884)
And your Aryan eye, bright blue.
Not God but a swastika
So black no sky could squeak through. (S2 = +3.854)
"And your neat mustache" is among the poem's top five S2 values. It’s easy to find this rather surprising—a moment where the machine and human expectations would diverge. Hitler’s mustache, after all, is probably the most salient visual cue associated with the figure. We’d speculate that the model, having conditioned on "Luftwaffe” and "gobbledygoo,” did not anticipate the close-up, euphemistic detail-work. The violation serves as the anticlimax: the poem moves from hypersyllabic foreign vocabulary to a concrete cliche, and the step-down is unexpected. Thus, the concrete imagery ratio feature works even with culturally familiar material, in that the specific object's arrival is juxtaposed against the reader’s informed expectation of something unfamiliar.
“So black no sky could squeak through” (S2 = +3.854) is the stanza’s highest value. The dreamlike image of the swastika blocking the sky is not naturally available in any distribution the model has encountered: the spatial compression, the unexpected verb 'squeak,' and the physical absoluteness of it produce an extreme information event. The line gives the reader an object with dimensional properties and thus informational cost.
The famous "Every woman adores a Fascist" (S2 = +0.281) is, by contrast, nearly neutral. The model, having established the poem's pattern of categorical statements, finds the register adequate. When a semantic explosive line reads as informationally cheap, it suggests the poem has concentrated its real surprises elsewhere—and it’s trivial, in this case, to recognize that it has.
Figure 7. Token-level surprisal (red) and entropy (blue dashed), Nazi imagery stanza. 'And your neat mustache' (S2 = +2.884) violates the military register by changing state in favor of close-up domestic detail. "So black no sky could squeak through" (S2 = +3.854) is the passage's highest value. "Every woman adores a Fascist" (S2 = +0.281) is nearly neutral: the significance of the iconic line is decidedly not reducible to entropic features.
Disclosure and Resolution
The poem's subsequent turn to autobiography produces the clearest demonstration of what the features detected by the concrete imagery ratio and HMM runway can accomplish together. By the time the poem reaches its final movement, the model has been conditioned for forty lines on text in registers historical and mythological. Into that sustained expectation barrels the line:
At twenty I tried to die (S2 = +2.856)
This has one of the poem's top-five S2 values—though, naturally, the bulk of the significance of the line is irreducible to entropic features. Still, the numbered age, the first-person past tense, the flat declarative—none of these follow from the preceding register of hyperbole and historical magnification, and this adds to the climactic effect. What’s more, the specificity ("twenty") is doing the same work as "black shoe" and "gray toe" earlier—a concrete particular landing against a context that did not prepare for it.
The final resolution is compressed:
And I said I do, I do. (S2 = −1.037)
Daddy, daddy, you bastard, I'm through. (S2 = −0.733)
The closing lines are among the cheapest in the poem, and the model, somewhat surprisingly on account of having been trained by the poem's established patterns of repetition and direct address, predicts the formulations here with high confidence. 'I'm through,’—perhaps the poem's most explicit declaration of severance—arrives nearly expected: the poem has "spent" its surprisal in a few locations, thus withholding it from the ending. The force of "I'm through" comes not from the phrase's statistical novelty but from its semantics and position at the end.
The villagers appear as grammatical subjects for the first time in the poem's final movement:
They always (S2 = +3.852)
knew
“They always” is the poem's third-highest S2 value. The villagers—peripheral figures throughout—are now violating the poem's long-established pattern of second-person address as grammatical subjects of their own clause. The line break forces the reader to complete the attribution without a verb, suspended at "always" before "knew" arrives at S2 = near-zero. Figure 8 shows the final movement.
Figure 8. Token-level surprisal (red) and entropy (blue dashed), disclosure and resolution. 'At twenty I tried to die' (S2 = +2.856) arrives after forty mythological lines. 'They always' (S2 = +3.852) is the poem's third-highest S2 value. The closing lines ('And I said I do, I do', S2 = −1.037; "Daddy, daddy, you bastard, I’m through," S2 = −0.733) are among the cheapest in the poem.
Conclusion
Concrete imagery, the HMM runway, and negative excess entropy collectively embody a unified tendency to maintain controlled, predictable language throughout most of the poem, and then to ostentatiously disrupt that predictability at precise moments. While yeasty academic debate over its semantics has, of course, contributed to the continued interest in “Daddy,” it’s also likely no coincidence that Plath’s masterpiece exemplifies the “setup-payoff” strategy more thoroughly than any other poem in Plath's corpus—and arguably any other poem in our full corpus.
The ability of these features to significantly predict canonicity score across 31 modern poets suggests a broader phenomenon: world literature, in effect, selects for this architecture, rewarding poems with a striking pattern of compression and explosion. While the information-theoretic perspective does not explain why these effects resonate with readers, Shannon's framework, Meyer's aesthetics, and DeDeo’s holistic methodology together shed light on the underlying mechanism.
References
Attridge, Derek. The Singularity of Literature. Routledge, 2004.
DeDeo, Simon. “A Cheat-Sheet for Text Analysis.” Santa Fe Institute, 2023.
Demberg, Vera, and Frank Keller. “Data from Eye-Tracking Corpora as Evidence for Theories of Syntactic Processing Complexity.” Cognition 109.2 (2008): 193–210.
Frank, Stefan L., Leun J. Otten, Giulia Galli, and Gabriella Vigliocco. “The ERP Response to the Amount of Information Conveyed by Words in Sentences.” Brain and Language 140 (2015): 1–11.
Fraser, Sandy. “Detecting Out of Distribution Text with Surprisal and Entropy.” LessWrong, 28 Jan. 2025.
Hale, John T. “A Probabilistic Earley Parser as a Psycholinguistic Model.” Proceedings of NAACL-HLT. 2001: 159–166.
Jaeger, T. Florian. “Redundancy and Reduction: Speakers Manage Syntactic Information Density.” Cognitive Psychology 61.1 (2010): 23–62.
Kao, Justine, and Dan Jurafsky. “A Computational Analysis of Style, Affect, and Imagery in Contemporary Poetry.” Proceedings of NAACL-HLT. 2012.
Levy, Roger, and T. Florian Jaeger. “Speakers Optimize Information Density through Syntactic Reduction.” Advances in Neural Information Processing Systems 19 (2007): 849–856.
McCoy, R. Thomas, Paul Smolensky, Tal Linzen, Jianfeng Gao, and Asli Celikyilmaz. “How Much Do Language Models Copy from Their Training Data? Evaluating Linguistic Novelty in Text Generation.” Transactions of the Association for Computational Linguistics 11 (2023): 652–670.
Meyer, Leonard B. Emotion and Meaning in Music. University of Chicago Press, 1956.
Miall, David S., and Don Kuiken. “Foregrounding, Defamiliarization, and Affect: Response to Literary Stories.” Poetics 22.5 (1994): 389–407.
Miller, Stevie. Seeing like the Canon: Replication Package. GitHub, 2026, github.com/stevie-ml/seeing-like-the-canon.
Momen, Omar, Emilie Sitter, Berenike Herrmann, and Sina Zarrieß. “Surprisal and Metaphor Novelty Judgments: Moderate Correlations and Divergent Scaling Effects Revealed by Corpus-Based and Synthetic Datasets.” arXiv preprint arXiv:2601.02015, 2026. Forthcoming in Proceedings of EACL 2026.
Pimentel, Tiago, Clara Meister, Ethan Wilcox, Roger Levy, and Ryan Cotterell. “Revisiting the Uniform Information Density Hypothesis.” Proceedings of EMNLP. 2023: 963–980.
Reagan, Andrew J., Lewis Mitchell, Dilan Kiley, Christopher M. Danforth, and Peter Sheridan Dodds. “The Emotional Arcs of Stories Are Dominated by Six Basic Shapes.” EPJ Data Science 5.1 (2016).
Shain, Cory, Idan Asher Blank, Marten van Schijndel, William Schuler, and Evelina Fedorenko. “fMRI Reveals Language-Specific Predictive Coding of Natural Language.” Neuropsychologia 138 (2020): 107307.
Shannon, Claude E. “A Mathematical Theory of Communication.” Bell System Technical Journal 27.3 (1948): 379–423.
Shannon, Claude E. “Prediction and Entropy of Printed English.” Bell System Technical Journal 30.1 (1951): 50–64.
Shklovsky, Viktor. “Art as Technique.” Russian Formalist Criticism: Four Essays. Trans. Lee T. Lemon and Marion J. Reis. University of Nebraska Press, 1965. (Orig. 1917.)
Smith, Nathaniel J., and Roger Levy. “The Effect of Word Predictability on Reading Time Is Logarithmic.” Cognition 128.3 (2013): 302–319.
Specifically Shannon had people try to guess the next letter in a sequence of English text, one letter at a time. He recorded how many guesses it took before they got it right. The paper is "Prediction and Entropy of Printed English" (1951).
“Any utterance produces a continually shifting sense of semantic weight and directionality, and reading always entails a continuous process of prediction, continuously modified as expectations are met, intensified, or disappointed.”
— Derek Attridge
An Entropic Lens
If prose, in a general sense, optimizes for predictability and efficient communication, verse traditions deliberately deviate from the reader’s expectations through unpredictability in word choice and imagery. Viktor Shklovsky called this “defamiliarization”: the deliberate roughening of language to encourage perception.
Large language models offer, for the first time, a method for operationalizing expectation precisely. Predicting the next token in a passage of text is the defining function of large language models (LLMs): For any input of words, a LLM outputs a probability distribution, a “probability of being next” for each possible next word. An oft-overlooked implication, however, is that the “surprise” of a word is neatly captured by how unlikely it was. This quantity is the negative log-probability of the word given its context, which we denote surprisal. Along with surprisal, this model also yields entropy: a measure not of how surprising the word that arrived was, but of how uncertain the model was before it arrived—captured by how many plausible continuations existed at that position. In information-theoretic terms, text of low entropy is compressible: it is so predictable that it affords readers little new information, and thus could (in a general sense) be encoded more efficiently. High entropy text, by contrast, resists such compression.
“[R]emarkably enough,” as Simon DeDeo wrote in A Cheat-Sheet for Text Analysis, “these numbers capture basic features of human experience of language.” The empirical results are clear: higher surprisal at a given position predicts longer reading times (Hale, 2001; Levy, 2008; Smith & Levy, 2013), correlates with N400 amplitude (Frank et al., 2015), tracks the movement of the eye during naturalistic reading (Demberg & Keller, 2008), and corresponds to fMRI imaging of the language-relevant areas of the brain (Shain et al., 2020); entropy, meanwhile, predicts the cognitive cost of anticipation (Pimentel et al., 2023), and speakers reliably smooth it across utterances (Jaeger, 2010). The implications are important: as the psycholinguistic evidence establishes that surprisal tracks the reader's word-to-word cognitive experience, the question of where and how a poem generates its affective power becomes, at least to some extent, empirical. With these new tools, we can locate—to the word and even the token—where a model reader fails to predict what comes next, with empirical warrant for supposing this failure often corresponds to what a reader registers as emotionally and analytically resonant.
The power and promise of this method has been anticipated for decades: Claude Shannon himself, in a 1951 paper, used human subjects as stand-ins for language models to estimate the entropy of printed English,[1] and Leonard Meyer's Emotion and Meaning in Music (1956) applied a structurally identical framework to musical experience, arguing that emotional response arises from the violation and fulfillment of expectation built up by familiarity with tonal conventions. Long before LLMs, Miall and Kuiken (1994) demonstrated behaviorally that stylistically deviant literary passages are associated with slower reading speed. With modern tools, however, the method has become massively more precise and accessible: where earlier approaches relied on human judgment or small controlled stimuli, LLM-assessed surprisal and entropy can be computed quite accurately for any text, too cheaply—nearly—to meter. Computational approaches to literary texts have, of course, grown substantially in the digital humanities: Kao and Jurafsky (2012) used stylometric features to distinguish professional from amateur poetry, and Reagan et al. (2016) mapped six dominant "emotional arc structures’ across 1,327 narratives by tracking sentiment trajectories, providing the suggestion that the information structure of narrative can be quantified and systematically compared. Most directly, Momen et al. (2026) applied LLM surprisal to metaphor novelty judgments, finding moderate but significant correlations between surprisal scores and human ratings of the figurative deviance of metaphors. Still, the use of token-level surprisal and entropy as tools of literary criticism proper remains a lacuna that this essay proposes to fill.
To this end, we construct a new lens. Using a corpus of 295 poems across 31 poets—canonical figures selected by frequency of appearance as answerlines in academic quizbowl competitions—we compute token-level surprisal and entropy for each poem using distilgpt2, and ask what information-structural features distinguish more canonical poems from less canonical ones within the same poet's work. An obvious and crucial line of inquiry for users of this method, of course, is determining what information-structural features are associated with "good" writing. A full treatment of this question is beyond the scope of this paper, but some broader analysis makes the problem of examining the information theoretic choices of our "case study" poem more tractable. To this end, we employ an outcome variable: "canonicity," operationalized by mention in academic quizbowl competitions. Quizbowl is a popular high school and collegiate trivia competition format in which school teams race to answer questions with buzzers. Quizbowl question data, we contend, provides an unusually reliable proxy for literary canonicity because the questions are written and vetted by a community with strong incentives to construct a reliable proxy for the poetic canon. Unlike citation counts or syllabi inclusions, quizbowl question selection is rigorously tested and policed: the community and numerous "play testers" are quick to penalize "bad" questions that reward trivia over genuine literary knowledge or misrepresent a work's standing. The result is a corpus of difficulty-calibrated knowledge that reflects what well-read undergraduates and graduate students across institutions recognize as canonical. By subtracting each poet’s average from both the outcome (log canonicity) and the features, the regression focuses only on variation within poets, in order to control for differences in style.
When a line registers at a z-score of +5.57 on excess surprisal, we can say not just that it is surprising in some abstract sense (and there’s scarcely a PhD in the world with reliable intuition for what “5 bit of surprisal” feels like to read in contemporary poetry) but that it is more surprising than virtually any comparable moment in a known distribution of literary language: The corpus allows us to make empirical, positioned claims where there was once only the possibility of impressionistic ones.
Entirely new to the domain of literature, S2—or excess surprisal—is the chief measure we will employ. As entropy measures how uncertain the model was as to the next token, regardless of which token actually arrived, subtracting entropy from surprisal isolates the contribution of the specific word choice (Fraser, 2025): a high S2 means the word was more surprising than its context alone would predict. A negative S2 means the word arrived in a position of already-high uncertainty and was not especially unexpected given that uncertainty.
Naturally, information-structural features explain only a relatively limited amount of what makes a poem canonical: canonicity is essentially a social and historical phenomenon. But we can be confident it explains something: 10-15% of the variation in canonicity (Adjusted R² 0.136). Concrete imagery ratio (r = +0.204), Hidden Markov Model (HMM) runway (r = +0.128), and negative excess entropy (r = -0.126) emerge as significant predictors—across 295 poems, 31 poets, controlling for between-poet differences in style—suggesting that the information architecture of a poem is not a mere epiphenomenon of its content, but to some extent an independent dimension of what makes it memorable. Our full results are in Figure 1.
Figure 1. Top 18 significant predictors of log canonicity (quizbowl question appearances) across 295 canonical poems from 31 modern English language poets. Within-poet demeaned Pearson r. Red bars = features cited in the essay. *** p < .001, ** p < .01, * p < .05.
One of the “stylized facts” of this nascent field is that speakers and writers tend to smooth information across utterances (Jaeger, 2010; Levy & Jaeger, 2007)—a phenomenon termed “Uniform Information Density” and considered the default condition of fluent language. We find support for the thesis that poetry works systematically differently: though poetry is not on average denser or stranger—the formal regularities of verse make it, if anything, more predictable than prose—the qualities that make poetry poetic are associated with surprisal that is concentrated.
As our case study in this new method in criticism, we will examine the information-theoretic profile of Sylvia Plath’s “Daddy.” Against the baseline, "Daddy” presents a paradox: it is more compressible (i.e. predictable), word-by-word, than 96% of canonical poems, yet the sections where it does deviate from expectation are among the most extreme in the corpus—the quantitative results are in Figure 2.
Figure 2. Information-structural profile of Sylvia Plath's "Daddy" across 14 features, expressed as z-scores relative to Plath's canonical poems (n = 18) and the full canonical corpus (n = 295). Red bars denote features cited in the essay.
The following sections analyze this apparent paradox.
The Opening
The concrete imagery ratio is the single strongest predictor of canonicity among the features we analyzed (r = +0.204). This feature measures the proportion of a poem's language that is devoted to concrete sensory objects rather than abstract states or emotions. A poem that says “I was trapped” is cheap; a poem that says “black shoe / In which I have lived like a foot” (S2 = +2.213 on “black shoe”) is costly.
You do not do, you do not do
Any more, black shoe
In which I have lived like a foot
For thirty years, poor and white,
Barely daring to breathe or Achoo.
The first line (S2 = −1.917) establishes a baseline for compression. Once the model sees “You do not do” once, the repeated clause will "cost" almost nothing, for the model has learned the pattern and can predict the recurrence with high confidence. It is, as it were, a first act to train the reader to expect rhythmic, maybe anaphoric, language. The shoe line arrives with that trained expectation in place and violates it. Notice how the equal length, double clause with its syntactically disconnected, assonant line gives a feeling of a disembodied anaphora.
The most explicit statement of the poem's subject is notably compressed by comparison:
Daddy, I have had to kill you. (S2 = −2.488)
The line has the third-lowest S2 line in the poem: this declaration, which constitutes the thesis of the confessional, is cheap—unsurprising, as the model is already conditioned on a second-person confessional address. Figure 3 reveals this pattern at the token level.
Figure 3. Token-level surprisal (red) and entropy (blue dashed), opening section. Surprisal collapses across the repeated “You do not do” (line 1, S2 = −1.917), then sharply rises by “shoe” in line 2. “Daddy, I have had to kill you” (line 6, S2 = −2.488) is the third-lowest S2 line in the poem.
"Concrete Imagery’ and the Statue Stanzas
The subsequent movement revels in concrete images of body parts. The father is described as “marble-heavy, a bag full of God” (S2 = −0.889), then as a “ghastly statue with one gray toe / Big as a Frisco seal” (S2 = +2.019, +0.276), and finally as a head lodged “in the freakish Atlantic.” Applying the adjective “gray” to a toe after the mention of a “ghastly statue”' evokes a shift in scale (to a color-material pairing) that the model has no precedent for, reflected in a sizable spike in S2. 'Where it pours bean green over blue' (S2 = +2.750) corresponds to the section's highest S2 spike, marking an unprecedented chromatic construction and, perceptibly, the most vivid image in the passage as shown in Figure 4.
Figure 4. Token-level surprisal (red) and entropy (blue dashed), statue stanzas. "Ghastly statue with one gray toe" (S2 = +2.019) and "Where it pours bean green over blue" (S2 = +2.750) are the section's highest spikes. Each S2 peak corresponds to a concrete sensory object landing in a context that was not prepared for it.
HMM Runway
The Hidden Markov Model (HMM) runway—the number of low-surprisal tokens the poem sustains before reaching peak S2—is the second of the three canonicity features we used (r = +0.128). The model posits two latent states: a low-emission state, in which the poem produces language closely aligned with contextual predictions, and a high-emission state, in which the language exceeds the running baseline in informational cost. Naturally, however, a poem does not announce its current state: “state” cannot be measured directly. Instead, it must be inferred mathematically from the sequence of surprisal values.
After "learning” the emission distributions and transition probabilities that best explain the data, the model categorizes the poem into states of relative predictability and violation. The "runway," for our purposes, is the length of the final low-emission stretch before the poem's peak S2 token. A longer runway generally results in a more pronounced spike; the more the reader is conditioned to expect a particular linguistic register, the greater the informational cost when a deviation occurs—"payoffs” following "setups." "Daddy" has an HMM runway of z = +3.71—the most extreme value in Plath's corpus—showing greater investment in compression than nearly any other poem in our corpus.
This "investment” is most legible in the geographical stanzas. After the statue imagery, the poem turns toward the father's origins and produces its most compressed passage outside the ich stanza:
In the German tongue, in the Polish town (S2 = −0.490)
Scraped flat by the roller
Of wars, wars, wars. (S2 = −0.209)
But the name of the town is common. (S2 = −2.678)
Though this stanza may have many features that seem exotic to us, the model is less perplexed. After the model sees the German, the formulaic geographical enumeration is found to be predictable enough. The repetition of "wars" also loses informational value with each occurrence; after the first token, the model assigns near-certainty to subsequent occurrences. "But the name of the town is common” registers the second-lowest S2 in the poem; syntax and surprisal flatten to near zero. The poem maintains this nadir of sustained compression through most of its first movement, before a very significant shift.
My Polack friend (S2 = +5.817)
“My Polack friend” is the poem's highest-S2 line, 3.3 standard deviations above its mean. The colloquial possessive, casual American diction, and (especially) arcane, somewhat humorously-connotated ethnic qualifier are a break from the preceding solemnity; every aspect of the prior register is disrupted. The informational cost to the reader matches the preceding runway: The poem sustains 53 tokens in a low-surprisal build-up, conditioning the reader in heavy, imprecise geography with a thoroughness that adds to the affective effect once the condition shifts. Figure 5 illustrates this passage.
Figure 5. Token-level surprisal (red) and entropy (blue dashed), in the "Polack’ passage. The geographical enumeration flattens to near-zero S2: 'But the name of the town is common' (S2 = −2.678) is the second-lowest line in the poem. 'My Polack friend' (S2 = +5.817) is the poem's highest S2 line, and memorably arrives after 53 tokens of low-surprisal build-up.
Code Switching and Information Structure
The poem's most architecturally deliberate passage spans the transition from English to German and back. The excess entropy feature (r = −0.126) measures the degree to which a poem's surprisal is concentrated in a few locations rather than distributed uniformly. A negative excess entropy—which "Daddy" has, at z = −0.89—means the poem is strategic about where it spends its "information budget." The ich stanza is the clearest demonstration of this strategy at the level of lines:
The tongue stuck in my jaw. (S2 = +1.787)
It stuck in a barb wire snare. (S2 = +2.306)
Ich, ich, ich, ich, (S2 = −2.193)
I could hardly speak. (S2 = +0.575)
I thought every German was you. (S2 = +2.163)
And the language obscene (S2 = +2.841)
The first two lines are characterized by escalation: each image of linguistic impasse is more specific and unexpected than the last. Barbed wire is a significantly more extreme and immediate metaphor for a stuck tongue than the "jaw" alone predicts, and the S2 naturally reflects this. Then the stutter: "Ich, ich, ich, ich" (S2 = −2.193)—the surprisal collapses. Once the first token is processed, the model predicts the repetition with near-certainty; and the information stream thins to almost nothing. The poem describes a breakdown of language, and the information structure enacts it—language stripped to a repeated token is, computationally, near information-free.
The rebound is immediate: “I thought every German was you” (S2 = +2.163) and “And the language obscene” (S2 = +2.841) are the passage's highest values. The return of English, semantically loaded, is expensive compared to the cheap preceding compression. This compressional, emissional architecture that the HMM detects at the whole-poem level is visible here in miniature—a six-line version of the poem's complete information structure. Figure 6 shows this at the token level.
Figure 6. Token-level surprisal (red) and entropy (blue dashed) in the "ich’ stanza. After escalation through 'tongue stuck in my jaw' (S2 = +1.787) and 'barb wire snare' (S2 = +2.306), S2 collapses to near-zero across 'Ich, ich, ich, ich' (S2 = −2.193)—the information structure enacts the breakdown it describes! It rebounds on 'I thought every German was you' (S2 = +2.163) and 'the language obscene' (S2 = +2.841).
The Nazi Imaginary
The cataloging of Nazi iconography offers one of the poem's more counterintuitive architectures:
With your Luftwaffe, your gobbledygoo.
And your neat mustache (S2 = +2.884)
And your Aryan eye, bright blue.
Not God but a swastika
So black no sky could squeak through. (S2 = +3.854)
"And your neat mustache" is among the poem's top five S2 values. It’s easy to find this rather surprising—a moment where the machine and human expectations would diverge. Hitler’s mustache, after all, is probably the most salient visual cue associated with the figure. We’d speculate that the model, having conditioned on "Luftwaffe” and "gobbledygoo,” did not anticipate the close-up, euphemistic detail-work. The violation serves as the anticlimax: the poem moves from hypersyllabic foreign vocabulary to a concrete cliche, and the step-down is unexpected. Thus, the concrete imagery ratio feature works even with culturally familiar material, in that the specific object's arrival is juxtaposed against the reader’s informed expectation of something unfamiliar.
“So black no sky could squeak through” (S2 = +3.854) is the stanza’s highest value. The dreamlike image of the swastika blocking the sky is not naturally available in any distribution the model has encountered: the spatial compression, the unexpected verb 'squeak,' and the physical absoluteness of it produce an extreme information event. The line gives the reader an object with dimensional properties and thus informational cost.
The famous "Every woman adores a Fascist" (S2 = +0.281) is, by contrast, nearly neutral. The model, having established the poem's pattern of categorical statements, finds the register adequate. When a semantic explosive line reads as informationally cheap, it suggests the poem has concentrated its real surprises elsewhere—and it’s trivial, in this case, to recognize that it has.
Figure 7. Token-level surprisal (red) and entropy (blue dashed), Nazi imagery stanza. 'And your neat mustache' (S2 = +2.884) violates the military register by changing state in favor of close-up domestic detail. "So black no sky could squeak through" (S2 = +3.854) is the passage's highest value. "Every woman adores a Fascist" (S2 = +0.281) is nearly neutral: the significance of the iconic line is decidedly not reducible to entropic features.
Disclosure and Resolution
The poem's subsequent turn to autobiography produces the clearest demonstration of what the features detected by the concrete imagery ratio and HMM runway can accomplish together. By the time the poem reaches its final movement, the model has been conditioned for forty lines on text in registers historical and mythological. Into that sustained expectation barrels the line:
At twenty I tried to die (S2 = +2.856)
This has one of the poem's top-five S2 values—though, naturally, the bulk of the significance of the line is irreducible to entropic features. Still, the numbered age, the first-person past tense, the flat declarative—none of these follow from the preceding register of hyperbole and historical magnification, and this adds to the climactic effect. What’s more, the specificity ("twenty") is doing the same work as "black shoe" and "gray toe" earlier—a concrete particular landing against a context that did not prepare for it.
The final resolution is compressed:
And I said I do, I do. (S2 = −1.037)
Daddy, daddy, you bastard, I'm through. (S2 = −0.733)
The closing lines are among the cheapest in the poem, and the model, somewhat surprisingly on account of having been trained by the poem's established patterns of repetition and direct address, predicts the formulations here with high confidence. 'I'm through,’—perhaps the poem's most explicit declaration of severance—arrives nearly expected: the poem has "spent" its surprisal in a few locations, thus withholding it from the ending. The force of "I'm through" comes not from the phrase's statistical novelty but from its semantics and position at the end.
The villagers appear as grammatical subjects for the first time in the poem's final movement:
They always (S2 = +3.852)
knew
“They always” is the poem's third-highest S2 value. The villagers—peripheral figures throughout—are now violating the poem's long-established pattern of second-person address as grammatical subjects of their own clause. The line break forces the reader to complete the attribution without a verb, suspended at "always" before "knew" arrives at S2 = near-zero. Figure 8 shows the final movement.
Figure 8. Token-level surprisal (red) and entropy (blue dashed), disclosure and resolution. 'At twenty I tried to die' (S2 = +2.856) arrives after forty mythological lines. 'They always' (S2 = +3.852) is the poem's third-highest S2 value. The closing lines ('And I said I do, I do', S2 = −1.037; "Daddy, daddy, you bastard, I’m through," S2 = −0.733) are among the cheapest in the poem.
Conclusion
Concrete imagery, the HMM runway, and negative excess entropy collectively embody a unified tendency to maintain controlled, predictable language throughout most of the poem, and then to ostentatiously disrupt that predictability at precise moments. While yeasty academic debate over its semantics has, of course, contributed to the continued interest in “Daddy,” it’s also likely no coincidence that Plath’s masterpiece exemplifies the “setup-payoff” strategy more thoroughly than any other poem in Plath's corpus—and arguably any other poem in our full corpus.
The ability of these features to significantly predict canonicity score across 31 modern poets suggests a broader phenomenon: world literature, in effect, selects for this architecture, rewarding poems with a striking pattern of compression and explosion. While the information-theoretic perspective does not explain why these effects resonate with readers, Shannon's framework, Meyer's aesthetics, and DeDeo’s holistic methodology together shed light on the underlying mechanism.
References
Attridge, Derek. The Singularity of Literature. Routledge, 2004.
DeDeo, Simon. “A Cheat-Sheet for Text Analysis.” Santa Fe Institute, 2023.
Demberg, Vera, and Frank Keller. “Data from Eye-Tracking Corpora as Evidence for Theories of Syntactic Processing Complexity.” Cognition 109.2 (2008): 193–210.
Frank, Stefan L., Leun J. Otten, Giulia Galli, and Gabriella Vigliocco. “The ERP Response to the Amount of Information Conveyed by Words in Sentences.” Brain and Language 140 (2015): 1–11.
Fraser, Sandy. “Detecting Out of Distribution Text with Surprisal and Entropy.” LessWrong, 28 Jan. 2025.
Hale, John T. “A Probabilistic Earley Parser as a Psycholinguistic Model.” Proceedings of NAACL-HLT. 2001: 159–166.
Jaeger, T. Florian. “Redundancy and Reduction: Speakers Manage Syntactic Information Density.” Cognitive Psychology 61.1 (2010): 23–62.
Kao, Justine, and Dan Jurafsky. “A Computational Analysis of Style, Affect, and Imagery in Contemporary Poetry.” Proceedings of NAACL-HLT. 2012.
Levy, Roger. “Expectation-Based Syntactic Comprehension.” Cognition 106.3 (2008): 1126–1177.
Levy, Roger, and T. Florian Jaeger. “Speakers Optimize Information Density through Syntactic Reduction.” Advances in Neural Information Processing Systems 19 (2007): 849–856.
McCoy, R. Thomas, Paul Smolensky, Tal Linzen, Jianfeng Gao, and Asli Celikyilmaz. “How Much Do Language Models Copy from Their Training Data? Evaluating Linguistic Novelty in Text Generation.” Transactions of the Association for Computational Linguistics 11 (2023): 652–670.
Meyer, Leonard B. Emotion and Meaning in Music. University of Chicago Press, 1956.
Miall, David S., and Don Kuiken. “Foregrounding, Defamiliarization, and Affect: Response to Literary Stories.” Poetics 22.5 (1994): 389–407.
Miller, Stevie. Seeing like the Canon: Replication Package. GitHub, 2026, github.com/stevie-ml/seeing-like-the-canon.
Momen, Omar, Emilie Sitter, Berenike Herrmann, and Sina Zarrieß. “Surprisal and Metaphor Novelty Judgments: Moderate Correlations and Divergent Scaling Effects Revealed by Corpus-Based and Synthetic Datasets.” arXiv preprint arXiv:2601.02015, 2026. Forthcoming in Proceedings of EACL 2026.
Pimentel, Tiago, Clara Meister, Ethan Wilcox, Roger Levy, and Ryan Cotterell. “Revisiting the Uniform Information Density Hypothesis.” Proceedings of EMNLP. 2023: 963–980.
Reagan, Andrew J., Lewis Mitchell, Dilan Kiley, Christopher M. Danforth, and Peter Sheridan Dodds. “The Emotional Arcs of Stories Are Dominated by Six Basic Shapes.” EPJ Data Science 5.1 (2016).
Shain, Cory, Idan Asher Blank, Marten van Schijndel, William Schuler, and Evelina Fedorenko. “fMRI Reveals Language-Specific Predictive Coding of Natural Language.” Neuropsychologia 138 (2020): 107307.
Shannon, Claude E. “A Mathematical Theory of Communication.” Bell System Technical Journal 27.3 (1948): 379–423.
Shannon, Claude E. “Prediction and Entropy of Printed English.” Bell System Technical Journal 30.1 (1951): 50–64.
Shklovsky, Viktor. “Art as Technique.” Russian Formalist Criticism: Four Essays. Trans. Lee T. Lemon and Marion J. Reis. University of Nebraska Press, 1965. (Orig. 1917.)
Smith, Nathaniel J., and Roger Levy. “The Effect of Word Predictability on Reading Time Is Logarithmic.” Cognition 128.3 (2013): 302–319.
Plath, Sylvia. “Daddy.” The Collected Poems. Ed. Ted Hughes. Harper & Row, 1981.
Specifically Shannon had people try to guess the next letter in a sequence of English text, one letter at a time. He recorded how many guesses it took before they got it right. The paper is "Prediction and Entropy of Printed English" (1951).