Methods note: The code used for the experiments and related open-source repo were built with Claude. The experimental design and writeup is my own, with minimal editing and formatting amendments made with Claude
Introduction
This is a research report inspired by a single question: in a model engineered to answer a simple biological question, how are higher-order, auxiliary concepts represented and used? ESM-2 is a model designed to infer missing amino acids in protein sequences (Lin et al. 2023), but what does the model use and know of emergent secondary structure? These are two different questions, and the data in this post serve to distinguish between the two: information about secondary structure is decodable, but when ablated has a limited, quantifiable effect on the model's accuracy. The implications of this research are in service of the greater question: when we use an LLM for a biological task, but we (appropriately) worry about how higher order or secondary concepts are being used by that LLM, how can we monitor or manipulate the representation of those secondary concepts? I personally believe that the bigger questions require parsing the smaller ones first, and so here I characterize the representation and usage of secondary structure (DSSP-labeled helix, strand, or coil) in ESM-2 150M.
Is secondary structure linearly represented?
The experiment to answer this question is relatively simple--use a linear probe to decode the vectors which represent the DSSP-labeled secondary structures, ablate those representations, and compare model performance to ablation of a random plane. The first thing to test is that the ablation worked; how well it worked provides information about the linearity of the concept representation. At layer 29 of 30, performing a linear ablation and then refitting a new linear probe on the same layer decreased decodability (hereafter referenced in macro-F1) from 0.81 to 0.71. This suggests, simply, that while some of the concept is linearly decodable, the linear probe’s plane does not hold all of the relevant signal.
This is not to say that the linearly decodable elements of secondary structure were not predictively significant. Rather, the results were encouraging recapitulations of existing research (Rives et al. 2021; Adams et al. 2025; Gujral et al. 2025) with a meaningful extension of their findings: linear ablation of these concepts reduced true residue likelihood by about 1 percent for helix and strand compared against random vector ablation. This shows that the linear representation of secondary structure is used by ESM-2 , but is not a complete picture of all that is represented by the model about secondary structure.
What, if any, linear direction does the model use for residue prediction?
The linear probe discovered and ablated a plane that removed some, but not all, of the linear decodability of the concept. This tells us that a linear probe alone cannot be used to remove all of the linearly decodable representation. So, how can we ablate all of the linearly decodable information without a linear probe?
The answer comes from removal of the class-mean plane. Each row represented within ESM-2 corresponds to a single amino acid, which we can label as belonging to a class: our secondary structures (helix, strand, coil). For a given cloud of points representing a class, there exists a mean. If we were to superimpose each class cloud over one another such that their means share the same coordinates, no linear plane could be used to distinguish them (Belrose et al. 2023). In this experiment, the refit probe following class mean plane ablation fell to 0.22 (the 0.02 discrepancy coming from the difference between training and test data), proving the ablation to be successful.
Ablation of the class-mean plane decreased true residue likelihood by 6% across secondary structures (Table 1). This suggests that the model uses the linearly decodable differences between secondary structures to infer amino acids based on surrounding residues.
Table 1. Removing one plane at layer 29. Likelihood drop = 1 − exp(Δ), with Δ the mean change in log-likelihood of the true residue at a masked position. Nats in parentheses.
Plane removed
New linear probe, macro-F1
Likelihood drop, helix
Strand
Coil
All residues
None
0.81
0
0
0
0
Random plane
0.81
0.1%
0.1%
0.1%
0.1% (−0.001)
Probe's plane
0.71
1.1%
0.9%
0.3%
0.7% (−0.007)
Class-mean plane
0.22
7.1%
4.8%
5.7%
6.0% (−0.062)
Class-mean plane, amino-acid identity held fixed
not run
4.9%
2.7%
5.7%
4.8% (−0.049)
This, however, is an incomplete assertion. In order to better understand how the model uses this information, it helps not only to see how the model behaves when the information disappears, but how it behaves when it's given misleading information. In other words, what happens to the model's predictions if you deliberately "mislabel" the secondary structure affiliated with an amino acid? To do this, I added the class-mean vector for helix to rows representing residues labeled with coil. I found that the residue predictions meaningfully shift towards helix-forming amino acids, increasing their predicted probability on coil rows by 13% (Table 2).
Table 2. Adding a direction to coil rows at layer 29. One dose moves the mean of the coil rows onto the helix mean along the direction. Helix-forming amino acids: A, E, F, I, K, L, M, Q, R, W.
Direction added
Dose
Predicted probability on helix-forming amino acids
Share of the helix-coil gap closed
Coil rows the layer-29 probe calls helix
None
0
43.6%
0
9%
Helix class-mean direction
0.5
45.9%
0.13
44%
Helix class-mean direction
1
49.2%
0.33
94%
Helix class-mean direction
2
56.3%
0.74
100%
Helix class-mean direction
4
63.1%
1.15
100%
Helix direction, amino-acid identity held fixed
1
48.2%
0.27
94%
Random direction, same length
1
43.4%
−0.01
9%
This set of experiments shows that linearly collapsible information about secondary structure is represented and used by ESM-2 for residue prediction, but does not exhaustively represent all information about secondary structure.
What nonlinear representation of secondary structure exists after linear erasure, and how is it used?
Table 3 shows how different non-linear probes decode secondary structure representation following linear erasure and class-means ablation. While most nonlinear probes still decode the information, but are meaningfully affected by the linear erasure, QDA remains unaffected throughout. This suggests that the shape of the class cloud, largely untouched by the ablation, closely mirrors that used by the QDA--a tilted ellipsoid. Of note here is that the squares-only reader, which has an untilted ellipsoid shape, falls from 0.81 to 0.56, suggesting further that the orientation of the ellipsoid is an exceptionally significant element of the concept’s representation within the model.
Table 3. Readers trained after the edit at layer 29. Macro-F1 on held-out chains. Floor 0.20.
Reader
Unedited
Random plane removed
Probe's plane removed
Class-mean plane removed
Linear probe
0.81
0.81
0.71
0.22
MLP, one hidden layer
0.81
0.81
0.80
0.79
SVM, RBF kernel
0.82
0.82
0.77
0.78
Squares-only, untilted ellipsoids
0.81
0.81
not run
0.56
QDA, tilted ellipsoids
0.81
0.81
not run
0.81
Perhaps the most interesting finding of this post, ablating the shape of the cloud after class-mean plane erasure decreases true residue likelihood by 2% (Table 4). This suggests that the shape of the data is, itself, used by the model for residue interpolation. Notably, this is only true for helix and strand, but not for coil for which ablation of shape incurs no predictive penalty. This is likely because the shape of coil is affected by its incorporation of a plethora of more disparate secondary structures (“unassigned”), whereas helix and strand are consistent throughout their representative distributions.
Table 4. Removing the shape of the class clouds at layer 29. Last row: paired difference in log-likelihood, shape minus control, converted the same way.
Edit
QDA after the edit, macro-F1
Likelihood drop, helix
Strand
Coil
All residues
Class-mean plane alone
0.81
7.1%
4.8%
5.7%
6.0% (−0.062)
Plane plus shape
0.03
11.8%
9.3%
8.2%
9.7% (−0.102)
Control: plane plus shape fit on shuffled labels
0.66
8.4%
6.6%
8.4%
8.0% (−0.084)
Shape beyond the control
3.7%
2.8%
−0.2%
1.9% (−0.019 [−0.021, −0.016])
How is this picture shaped by architecture of ESM-2?
ESM-2's bidirectionality within layers (each row reading information from higher and lower rows, representing amino acids flanking it on both sides) affects the difficulty and implementation of concept erasure. As such, the contingency of erasure on identical erasure on surrounding rows as mapped between layers, was performed in Table 5. This meaningfully affects our understanding of the positional distribution of the concept representation: The representation of higher order concepts in ESM-2 is affected by the ability of a row at a given layer to represent bidirectional information about surrounding amino acids and their affiliated secondary structures.
Table 5. Class-mean plane removed at layer 29 at a share of positions. New linear probe trained at layer 30 on the erased rows. Likelihood drop at the same rows.
Positions erased at layer 29
Erased rows read
Rows
New linear probe at layer 30, macro-F1
Likelihood drop at those rows
None
all rows
113,279
0.81
0
Every position
all rows
113,279
0.56
6.0%
Half, at random
both neighbours also erased
14,069
0.58
4.9%
Half, at random
both neighbours intact
14,297
0.66
4.2%
Half, at random
three intact on each side
861
0.69
5.3% [3.8%, 6.9%]
One in ten, at random
all erased rows
11,508
0.67
3.9%
Half, random plane
all erased rows
56,797
0.81
0.1%
To better understand how information about secondary structure is represented across layers, single layer ablations were performed at all layers, and multi-layer ablations were run on band 20-30, 26-30, and 29-30 (Figure 1). The multi-layer ablations prove that secondary structure can be reconstructed across multiple layers, suggesting that ablation at a single layer using these techniques would be insufficient for full concept erasure. The difference in true residue prediction between ablation at Layers 30 and 29 is 8% (14 and 6 respectively), and also suggests meaningful reconstruction happens between single layers, contingent on erasure at proximal positions (Table 5).
Figure 1. Cost of removing the class-mean plane, by depth. A: one layer at a time. B: every layer of a band. C: what a new linear probe reads at layer 30 after each band.
What other experiments would teach us more about secondary structural representation in ESM-2?
The most straightforward experiment assesses its scalability--use the exact experimental paradigm across all ESM-2 sizes (8M, 35M, 150M, 650M, 3B and 15B) and see if increased dimensionality affects the shape and nonlinear properties of the concept representation. Furthermore, assessing the boundary conditions of the conclusions, and amending observations about the model's use of coil shape, would involve increasing the usage of DSSP labels to include all 8, and therefore be more granular.
Methods and Supplementary Data
For fit and test data, 548 chains comprised of 113,279 labeled residues (out of 135,365 total) were used (NetSurfP-3.0 dataset; Hugging Face mirror). 5 folds were grouped by chain, and every score was determined on chains held-out from the training data. The baseline linear decodability of secondary structure is a macro-F1 of 0.81. Guessing always-coil had a macro-F1 of 0.20. Amino-acid alone (without model usage) would provide a macro-F1 of 0.44.
Table 6. Plane removed at every layer of a band. Unedited: top-1 accuracy 39.2%, linear probe 0.81, QDA 0.81.
Introduction
This is a research report inspired by a single question: in a model engineered to answer a simple biological question, how are higher-order, auxiliary concepts represented and used? ESM-2 is a model designed to infer missing amino acids in protein sequences (Lin et al. 2023), but what does the model use and know of emergent secondary structure? These are two different questions, and the data in this post serve to distinguish between the two: information about secondary structure is decodable, but when ablated has a limited, quantifiable effect on the model's accuracy. The implications of this research are in service of the greater question: when we use an LLM for a biological task, but we (appropriately) worry about how higher order or secondary concepts are being used by that LLM, how can we monitor or manipulate the representation of those secondary concepts? I personally believe that the bigger questions require parsing the smaller ones first, and so here I characterize the representation and usage of secondary structure (DSSP-labeled helix, strand, or coil) in ESM-2 150M.
Is secondary structure linearly represented?
The experiment to answer this question is relatively simple--use a linear probe to decode the vectors which represent the DSSP-labeled secondary structures, ablate those representations, and compare model performance to ablation of a random plane. The first thing to test is that the ablation worked; how well it worked provides information about the linearity of the concept representation. At layer 29 of 30, performing a linear ablation and then refitting a new linear probe on the same layer decreased decodability (hereafter referenced in macro-F1) from 0.81 to 0.71. This suggests, simply, that while some of the concept is linearly decodable, the linear probe’s plane does not hold all of the relevant signal.
This is not to say that the linearly decodable elements of secondary structure were not predictively significant. Rather, the results were encouraging recapitulations of existing research (Rives et al. 2021; Adams et al. 2025; Gujral et al. 2025) with a meaningful extension of their findings: linear ablation of these concepts reduced true residue likelihood by about 1 percent for helix and strand compared against random vector ablation. This shows that the linear representation of secondary structure is used by ESM-2 , but is not a complete picture of all that is represented by the model about secondary structure.
What, if any, linear direction does the model use for residue prediction?
The linear probe discovered and ablated a plane that removed some, but not all, of the linear decodability of the concept. This tells us that a linear probe alone cannot be used to remove all of the linearly decodable representation. So, how can we ablate all of the linearly decodable information without a linear probe?
The answer comes from removal of the class-mean plane. Each row represented within ESM-2 corresponds to a single amino acid, which we can label as belonging to a class: our secondary structures (helix, strand, coil). For a given cloud of points representing a class, there exists a mean. If we were to superimpose each class cloud over one another such that their means share the same coordinates, no linear plane could be used to distinguish them (Belrose et al. 2023). In this experiment, the refit probe following class mean plane ablation fell to 0.22 (the 0.02 discrepancy coming from the difference between training and test data), proving the ablation to be successful.
Ablation of the class-mean plane decreased true residue likelihood by 6% across secondary structures (Table 1). This suggests that the model uses the linearly decodable differences between secondary structures to infer amino acids based on surrounding residues.
Table 1. Removing one plane at layer 29. Likelihood drop = 1 − exp(Δ), with Δ the mean change in log-likelihood of the true residue at a masked position. Nats in parentheses.
This, however, is an incomplete assertion. In order to better understand how the model uses this information, it helps not only to see how the model behaves when the information disappears, but how it behaves when it's given misleading information. In other words, what happens to the model's predictions if you deliberately "mislabel" the secondary structure affiliated with an amino acid? To do this, I added the class-mean vector for helix to rows representing residues labeled with coil. I found that the residue predictions meaningfully shift towards helix-forming amino acids, increasing their predicted probability on coil rows by 13% (Table 2).
Table 2. Adding a direction to coil rows at layer 29. One dose moves the mean of the coil rows onto the helix mean along the direction. Helix-forming amino acids: A, E, F, I, K, L, M, Q, R, W.
This set of experiments shows that linearly collapsible information about secondary structure is represented and used by ESM-2 for residue prediction, but does not exhaustively represent all information about secondary structure.
What nonlinear representation of secondary structure exists after linear erasure, and how is it used?
Table 3 shows how different non-linear probes decode secondary structure representation following linear erasure and class-means ablation. While most nonlinear probes still decode the information, but are meaningfully affected by the linear erasure, QDA remains unaffected throughout. This suggests that the shape of the class cloud, largely untouched by the ablation, closely mirrors that used by the QDA--a tilted ellipsoid. Of note here is that the squares-only reader, which has an untilted ellipsoid shape, falls from 0.81 to 0.56, suggesting further that the orientation of the ellipsoid is an exceptionally significant element of the concept’s representation within the model.
Table 3. Readers trained after the edit at layer 29. Macro-F1 on held-out chains. Floor 0.20.
Perhaps the most interesting finding of this post, ablating the shape of the cloud after class-mean plane erasure decreases true residue likelihood by 2% (Table 4). This suggests that the shape of the data is, itself, used by the model for residue interpolation. Notably, this is only true for helix and strand, but not for coil for which ablation of shape incurs no predictive penalty. This is likely because the shape of coil is affected by its incorporation of a plethora of more disparate secondary structures (“unassigned”), whereas helix and strand are consistent throughout their representative distributions.
Table 4. Removing the shape of the class clouds at layer 29. Last row: paired difference in log-likelihood, shape minus control, converted the same way.
How is this picture shaped by architecture of ESM-2?
ESM-2's bidirectionality within layers (each row reading information from higher and lower rows, representing amino acids flanking it on both sides) affects the difficulty and implementation of concept erasure. As such, the contingency of erasure on identical erasure on surrounding rows as mapped between layers, was performed in Table 5. This meaningfully affects our understanding of the positional distribution of the concept representation: The representation of higher order concepts in ESM-2 is affected by the ability of a row at a given layer to represent bidirectional information about surrounding amino acids and their affiliated secondary structures.
Table 5. Class-mean plane removed at layer 29 at a share of positions. New linear probe trained at layer 30 on the erased rows. Likelihood drop at the same rows.
To better understand how information about secondary structure is represented across layers, single layer ablations were performed at all layers, and multi-layer ablations were run on band 20-30, 26-30, and 29-30 (Figure 1). The multi-layer ablations prove that secondary structure can be reconstructed across multiple layers, suggesting that ablation at a single layer using these techniques would be insufficient for full concept erasure. The difference in true residue prediction between ablation at Layers 30 and 29 is 8% (14 and 6 respectively), and also suggests meaningful reconstruction happens between single layers, contingent on erasure at proximal positions (Table 5).
Figure 1. Cost of removing the class-mean plane, by depth. A: one layer at a time. B: every layer of a band. C: what a new linear probe reads at layer 30 after each band.
What other experiments would teach us more about secondary structural representation in ESM-2?
The most straightforward experiment assesses its scalability--use the exact experimental paradigm across all ESM-2 sizes (8M, 35M, 150M, 650M, 3B and 15B) and see if increased dimensionality affects the shape and nonlinear properties of the concept representation. Furthermore, assessing the boundary conditions of the conclusions, and amending observations about the model's use of coil shape, would involve increasing the usage of DSSP labels to include all 8, and therefore be more granular.
Methods and Supplementary Data
For fit and test data, 548 chains comprised of 113,279 labeled residues (out of 135,365 total) were used (NetSurfP-3.0 dataset; Hugging Face mirror). 5 folds were grouped by chain, and every score was determined on chains held-out from the training data. The baseline linear decodability of secondary structure is a macro-F1 of 0.81. Guessing always-coil had a macro-F1 of 0.20. Amino-acid alone (without model usage) would provide a macro-F1 of 0.44.
Table 6. Plane removed at every layer of a band. Unedited: top-1 accuracy 39.2%, linear probe 0.81, QDA 0.81.