This article is written as part of an ongoing research initiative by the AMIR Lab at Georgia Tech, exploring scientific discovery and mechanistic interpretability for biological AI models. Main results and discussion points raised are adapted from the ProtoMech framework, which was accepted into ICML 2026.[1]
Summary
AI models have revolutionized biology by enabling us to simulate, predict, and engineer biomolecules in silico. We have the unique opportunity to repurpose these AI models from opaque black boxes to digital microscopes that can help us learn more about the biological world around us. By introducing the ProtoMech framework, we demonstrate how tracing internal computational circuits can unmask hidden functional hotspots, structural motifs, and the mechanistic impacts of protein mutations. We envision a world where we can leverage these digital microscopes for scientific discovery.
Background
Figure 1. Scientific tools have revolutionized our understanding of DNA, RNA, and proteins.[2] Every advancement lets us further understand the microscopic world.
The Tools That Shape Science
History's greatest biologists have always used cutting-edge tools to explore the microscopic world. In 1674, Antonie van Leeuwenhoek built pioneering single-lens microscopes to observe the first living cells. Nearly three centuries later, in 1953, Rosalind Franklin’s X-ray diffraction data allowed James Watson and Francis Crick to uncover the double-helix structure of DNA. Shortly after, in 1958, John Kendrew utilized X-ray crystallography to solve the very first atomic-resolution protein structure.
Every major leap in our biological understanding has been propelled by the lenses we use to look at nature (Fig. 1).
The Next Lens is Digital
Today, we are entering a new frontier. We are no longer just observing the microscopic world through physical hardware. Now, we're building digital AI models capable of simulating it. Today, we have access to AI models that can predict biological structures and engineer novel proteins without a single wet-lab experiment.
Unlike the tools of old though, this progress represents a fundamental challenge: for the first time, we don't understand how our own tools work. Even with our most complicated physical lenses, we fundamentally knew the biophysics principles that enabled them to work. But with AI models, we have no clue.
Our core question is simple: rather than operate these AI models as black boxes, can we operate these AI models as digital microscopes to learn more about the microscopic world?
The ProtoMech Framework: Tracing the Circuits of Life
To build this digital lens, our team developed ProtoMech, a framework for tracing out the internal computational pathways, or circuits, inside large protein language models, such as ESM2. ProtoMech utilizes cross-layer transcoders, which learn sparse latent representations jointly across layers to capture the model’s full computational circuitry.
Figure 2. ProtoMech serves as a replacement model for ESM2. ProtoMech identifies a circuit of interpretable latents (blue) that traces and approximates the behavior of ESM2. The example latent detects the conserved HRD catalytic motif found in protein kinases.
When we pointed this digital microscope at ESM2, we discovered that the model, without any understanding of the real world, had independently learned complex biochemistry:[3]
Functional Hotspots: We isolated distinct computational pathways dedicated exclusively to recognizing active catalytic sites in enzymes (Fig. 3a).
Structural Circuits: ProtoMech revealed circuits dedicated to recognizing specifical structural motifs, such as unique secondary structures or binding sites (Fig. 3b).
Mutations Changing Protein Function: ProtoMech provides a mechanistic rationale for why certain mutations improve protein function and why others kill protein function (Fig. 4).
Figure 3. Examples of circuits discovered using ProtoMech to identify protein families. We use ProtoMech to examine a, kinase domain and b, NADP+ binding domain circuits. We find interpretable features related to binding and active sites, secondary structure, and biochemical patterns. We observe that earlier layers are detecting key amino acids that assemble into complex motifs.
Figure 4. Examples of circuits discovered using ProtoMech attributing mutations in proteins to changes in protein function. a, In the GB1 protein sequence, we find interpretable features related to binding and stability. b, Introducing a mutation which improves protein function, we find an additional interpretable feature corresponding to stability. c, Introducing a mutation which kills protein function, a majority of the circuit deactivates completely. These findings highlight ProtoMech’s ability to provide a mechanistic rationale for changes in protein function.
Future Outlook
By adjusting the lens we view biology with, we now have the capability of turning AI models from opaque black boxes to digital microscopes. Right now, we are only scratching the surface of what these digital microscopes can view. As our capacity to interpret features is bounded by our current biological knowledge, it is possible that there exist circuits governing mechanisms that are not yet well-characterized. Work toward automating the interpretation of features is most certainly necessary to expand our knowledge.
However, the true paradigm shift lies in moving from interpretation to scientific discovery. Historically, biological research has been bottlenecked by the speed of wet-lab trials. By using frameworks like ProtoMech, this gives us the opportunity to translate AI models into digital labs, potentially enabling us to conduct initial biological exploration through our digital microscopes.
This article is written as part of an ongoing research initiative by the AMIR Lab at Georgia Tech, exploring scientific discovery and mechanistic interpretability for biological AI models. Main results and discussion points raised are adapted from the ProtoMech framework, which was accepted into ICML 2026.[1]
Summary
AI models have revolutionized biology by enabling us to simulate, predict, and engineer biomolecules in silico. We have the unique opportunity to repurpose these AI models from opaque black boxes to digital microscopes that can help us learn more about the biological world around us. By introducing the ProtoMech framework, we demonstrate how tracing internal computational circuits can unmask hidden functional hotspots, structural motifs, and the mechanistic impacts of protein mutations. We envision a world where we can leverage these digital microscopes for scientific discovery.
Background
Figure 1. Scientific tools have revolutionized our understanding of DNA, RNA, and proteins.[2] Every advancement lets us further understand the microscopic world.
The Tools That Shape Science
History's greatest biologists have always used cutting-edge tools to explore the microscopic world. In 1674, Antonie van Leeuwenhoek built pioneering single-lens microscopes to observe the first living cells. Nearly three centuries later, in 1953, Rosalind Franklin’s X-ray diffraction data allowed James Watson and Francis Crick to uncover the double-helix structure of DNA. Shortly after, in 1958, John Kendrew utilized X-ray crystallography to solve the very first atomic-resolution protein structure.
Every major leap in our biological understanding has been propelled by the lenses we use to look at nature (Fig. 1).
The Next Lens is Digital
Today, we are entering a new frontier. We are no longer just observing the microscopic world through physical hardware. Now, we're building digital AI models capable of simulating it. Today, we have access to AI models that can predict biological structures and engineer novel proteins without a single wet-lab experiment.
Unlike the tools of old though, this progress represents a fundamental challenge: for the first time, we don't understand how our own tools work. Even with our most complicated physical lenses, we fundamentally knew the biophysics principles that enabled them to work. But with AI models, we have no clue.
Our core question is simple: rather than operate these AI models as black boxes, can we operate these AI models as digital microscopes to learn more about the microscopic world?
The ProtoMech Framework: Tracing the Circuits of Life
To build this digital lens, our team developed ProtoMech, a framework for tracing out the internal computational pathways, or circuits, inside large protein language models, such as ESM2. ProtoMech utilizes cross-layer transcoders, which learn sparse latent representations jointly across layers to capture the model’s full computational circuitry.
Figure 2. ProtoMech serves as a replacement model for ESM2. ProtoMech identifies a circuit of interpretable latents (blue) that traces and approximates the behavior of ESM2. The example latent detects the conserved HRD catalytic motif found in protein kinases.
When we pointed this digital microscope at ESM2, we discovered that the model, without any understanding of the real world, had independently learned complex biochemistry:[3]
Figure 3. Examples of circuits discovered using ProtoMech to identify protein families. We use ProtoMech to examine a, kinase domain and b, NADP+ binding domain circuits. We find interpretable features related to binding and active sites, secondary structure, and biochemical patterns. We observe that earlier layers are detecting key amino acids that assemble into complex motifs.
Figure 4. Examples of circuits discovered using ProtoMech attributing mutations in proteins to changes in protein function. a, In the GB1 protein sequence, we find interpretable features related to binding and stability. b, Introducing a mutation which improves protein function, we find an additional interpretable feature corresponding to stability. c, Introducing a mutation which kills protein function, a majority of the circuit deactivates completely. These findings highlight ProtoMech’s ability to provide a mechanistic rationale for changes in protein function.
Future Outlook
By adjusting the lens we view biology with, we now have the capability of turning AI models from opaque black boxes to digital microscopes. Right now, we are only scratching the surface of what these digital microscopes can view. As our capacity to interpret features is bounded by our current biological knowledge, it is possible that there exist circuits governing mechanisms that are not yet well-characterized. Work toward automating the interpretation of features is most certainly necessary to expand our knowledge.
However, the true paradigm shift lies in moving from interpretation to scientific discovery. Historically, biological research has been bottlenecked by the speed of wet-lab trials. By using frameworks like ProtoMech, this gives us the opportunity to translate AI models into digital labs, potentially enabling us to conduct initial biological exploration through our digital microscopes.
Tsui, Darin, Kunal Talreja, Daniel Saeedi, and Amirali Aghazadeh. "Protein Circuit Tracing via Cross-layer Transcoders." arXiv preprint arXiv:2602.12026 (2026).
Adapted from: Kang, Justin S., Darin Tsui, Yigit Efe Erginbas, Landon Butler, Amirali Aghazadeh, and Kannan Ramchandran. 2026. "Spectral Sparsity: A Unifying Framework for Scalable Model Interpretability Using Codes." IEEE BITS the Information Theory Magazine.
These circuits are publicly available at https://protmech.github.io/.