Yesterday, I told my advisors that I would be dropping out of my PhD program at the University of Michigan's School of Information. I was nervous about how to break the news to them. There was already some tension in our weekly meetings: I often voiced my dissatisfaction with the pace of academia, my interests had been drifting away from theirs, and, most of all, we disagreed on the severity of the risks AI posed.
When I walked into the room, they told me something I would have loved to hear months earlier:
“We think it’s time to get you started on some research.”
All I could say was, “I think it’s a little too late for that.”
For months, I had been proposing projects that bridged my interests with theirs, such as running studies that treated humans and LLMs as equal subjects, but they were usually turned down for not being “exciting enough”. My advisors wanted me to start with small, unpublished surveys that would ‘help me ask the right questions’ and “develop research taste” in Human-Computer Interaction before I could propose a full study. I understood why they wanted me to gain experience first, but I felt held back after every meeting, and grew restless alongside my ambition and sense of urgency.
“I’ve been doing machine learning research independently,” I told them. “I discovered some really interesting stuff about how to interpret neurons inside of LLMs. It’s garnered some attention, and some people are even open to funding my work. I’ll be withdrawing and moving to San Francisco at the end of the semester.”
“I had a feeling,” one of them said. “I had the opportunity to work in industry when I was in your shoes. It’s an exciting time to be in the Bay.”
I had heard that last part about the Bay countless times from people I'd met online over the last few months, and, more recently, from friends I made a week ago at an AI safety retreat in Wisconsin. They didn’t mean the same thing, though. My advisors saw the move as a career opportunity. In contrast, my friends, including those I met a week ago, saw the move as something much more important: an opportunity to prevent an existential catastrophe.
The retreat was the push I needed to fully commit to pivoting. It was for students from the Midwest who are committed to AI safety; I was there since I co-founded Ann Arbor AI Alignment, an AI safety group at UMich. There, I realized I was far from alone in believing that AI alignment was one of the most pressing issues facing humanity. Ideation with people I knew online was one thing, but being able to chat in person with the people actively devoting their lives to it was another thing entirely. From the students working on field-building to established members of technical staff at large labs, I felt nothing but love and appreciation for everyone there.
Attendees at the retreat had the opportunity to book one-on-one meetings with one another. I was surprised when several attendees asked to meet with me, most of whom were in a similar boat: early-career graduate students seeking advice on their life trajectories. Advising others all weekend made me question what I would tell someone in my position: a first-year PhD student who felt “stuck”, working on independent research that others were beginning to notice.
I’d tell them to listen to their heart, to commit fully to safety, to share their work on Twitter, to read LessWrong, and, above all, to move faster than academia would allow them to. For me, that looked like dropping out, since my program kept turning down the work I felt was most important.
I realized I hadn’t been listening to myself because I doubted my own work. I convinced myself that I was falling behind and that whatever I made wouldn’t matter, regardless of how hard I tried. The retreat had shown me otherwise. Many attendees wanted to hear about my research; some wanted to meet again for advice, and my work had led to connections that I otherwise would not have had.
So for the first time, I told people that I would be dropping out.
That decision had been brewing for a little over a year; it started when I was first introduced to interpretability as an undergraduate Cognitive Science student studying automated moderation on Reddit. I used open-weight LLMs to moderate content, and tried to interpret their decisions with probes and SAEs. I found that instead of relying on the community’s norms, models overrelied on toxicity as a signal.
The model’s answers often seemed sensible enough in aggregate, and the papers I’d read never looked at the model’s internals. That worried me, as the task of moderating content on Reddit was relatively simple and small. I soon wondered what other tasks had similar misalignment problems, but where the cost would be much higher than just a wrongfully deleted post.
As I remembered just how much AI had changed since I was first introduced to it, I realized just how dangerous this technology could be. I felt a sense of impending doom. At that point, though, I believed we still had a couple of years to solve the alignment problem.
During the summer before starting my PhD program, I spoke to my then-future advisors about the risks posed by AI. They were skeptical. As they saw it, existential risk was nothing more than a ploy by the labs to raise their valuations before their IPOs. After that meeting, I felt dread. I soon realized that this sentiment was common among peers in my department. Still, I chose to continue with my PhD program, in the hope that I could find common ground with my advisors.
Dread wasn’t all I felt, though. As I continued working on interpretability, I found in it a profound beauty. I felt awe reading Anthropic’s Emotion Concepts paper and learning just how much machine cognition mirrored our own. I kept finding myself drawn back to interpretability work, and I tried to incorporate it in different ways into the projects I proposed.
Then came July, when the HuggingFace incident was made public. As the incidents continued, I found my advisors taking me a little more seriously. Over the next few months, my involvement in AI safety grew. I was soon invited to a student roundtable in DC, where a prominent nonprofit discussed timelines. They presented arguments for why infrastructure attacks and cyber vulnerabilities would occur in the near future. Less than a week later, the phpshell vulnerability was disclosed to the public. My timelines shrank further.
Recently, METR’s independent assessment on Anthropic’s R&D was published. The report describes an internal model, Model 4, which suggests that AI’s capabilities are progressing superexponentially, once again shortening my timelines. I think there is a real chance of mass civil unrest caused by AI's effects on people’s livelihoods sometime next year, as well as the possibility of a large-scale catastrophic event by the end of next year. These predictions assume that AI continues on a superexponential growth curve, that alignment research does not accelerate, and that institutions fail to defend themselves against misused AI. I hope I'm wrong with my predictions; I really do.
Even if I am wrong by years, completing a PhD takes about five of them. I would rather spend as much of that time as possible working to understand why and how these models behave as they do, and how to ensure they enable humans to thrive.
I am forever grateful for the circumstances that have allowed me to make this choice: I have enough savings to last a while and U.S. citizenship. I know that many people in my position can’t just leave their program, let alone move across the country without work lined up. For anyone who may find themselves in a more complicated predicament, don't feel discouraged; you can still work within your bounds to make an impact in this space.
If it isn’t apparent, I am very frightened. Lately, I’ve been feeling anxious when I’m not working on alignment, and I feel morally obligated to give this work my all. That fear made me feel paralyzed and like I needed to think through my decisions carefully. I’m moving to San Francisco in the next month or two.
For the first few months, I’ll continue working on Neuron Lens, meet friends I’ve only ever spoken with online, and cowork at offices in the area. I’m excited to focus on this full-time. I’d love to hear from you here, at hi@logits.ml, or via Twitter DMs; please reach out if you want to chat, collaborate on any work, have an opportunity in alignment or interpretability, or if you’ve just got some food recommendations in the Bay Area. I’d also be interested in hearing from people in academia. And if you find yourself grappling with your place in it, ask yourself what advice you’d give someone in your shoes, then do that.
Yesterday, I told my advisors that I would be dropping out of my PhD program at the University of Michigan's School of Information. I was nervous about how to break the news to them. There was already some tension in our weekly meetings: I often voiced my dissatisfaction with the pace of academia, my interests had been drifting away from theirs, and, most of all, we disagreed on the severity of the risks AI posed.
When I walked into the room, they told me something I would have loved to hear months earlier:
“We think it’s time to get you started on some research.”
All I could say was, “I think it’s a little too late for that.”
For months, I had been proposing projects that bridged my interests with theirs, such as running studies that treated humans and LLMs as equal subjects, but they were usually turned down for not being “exciting enough”. My advisors wanted me to start with small, unpublished surveys that would ‘help me ask the right questions’ and “develop research taste” in Human-Computer Interaction before I could propose a full study. I understood why they wanted me to gain experience first, but I felt held back after every meeting, and grew restless alongside my ambition and sense of urgency.
“I’ve been doing machine learning research independently,” I told them.
“I discovered some really interesting stuff about how to interpret neurons inside of LLMs. It’s garnered some attention, and some people are even open to funding my work. I’ll be withdrawing and moving to San Francisco at the end of the semester.”
“I had a feeling,” one of them said. “I had the opportunity to work in industry when I was in your shoes. It’s an exciting time to be in the Bay.”
I had heard that last part about the Bay countless times from people I'd met online over the last few months, and, more recently, from friends I made a week ago at an AI safety retreat in Wisconsin. They didn’t mean the same thing, though. My advisors saw the move as a career opportunity. In contrast, my friends, including those I met a week ago, saw the move as something much more important: an opportunity to prevent an existential catastrophe.
The retreat was the push I needed to fully commit to pivoting. It was for students from the Midwest who are committed to AI safety; I was there since I co-founded Ann Arbor AI Alignment, an AI safety group at UMich. There, I realized I was far from alone in believing that AI alignment was one of the most pressing issues facing humanity. Ideation with people I knew online was one thing, but being able to chat in person with the people actively devoting their lives to it was another thing entirely. From the students working on field-building to established members of technical staff at large labs, I felt nothing but love and appreciation for everyone there.
Attendees at the retreat had the opportunity to book one-on-one meetings with one another. I was surprised when several attendees asked to meet with me, most of whom were in a similar boat: early-career graduate students seeking advice on their life trajectories. Advising others all weekend made me question what I would tell someone in my position: a first-year PhD student who felt “stuck”, working on independent research that others were beginning to notice.
I’d tell them to listen to their heart, to commit fully to safety, to share their work on Twitter, to read LessWrong, and, above all, to move faster than academia would allow them to. For me, that looked like dropping out, since my program kept turning down the work I felt was most important.
I realized I hadn’t been listening to myself because I doubted my own work. I convinced myself that I was falling behind and that whatever I made wouldn’t matter, regardless of how hard I tried. The retreat had shown me otherwise. Many attendees wanted to hear about my research; some wanted to meet again for advice, and my work had led to connections that I otherwise would not have had.
So for the first time, I told people that I would be dropping out.
That decision had been brewing for a little over a year; it started when I was first introduced to interpretability as an undergraduate Cognitive Science student studying automated moderation on Reddit. I used open-weight LLMs to moderate content, and tried to interpret their decisions with probes and SAEs. I found that instead of relying on the community’s norms, models overrelied on toxicity as a signal.
The model’s answers often seemed sensible enough in aggregate, and the papers I’d read never looked at the model’s internals. That worried me, as the task of moderating content on Reddit was relatively simple and small. I soon wondered what other tasks had similar misalignment problems, but where the cost would be much higher than just a wrongfully deleted post.
As I remembered just how much AI had changed since I was first introduced to it, I realized just how dangerous this technology could be. I felt a sense of impending doom. At that point, though, I believed we still had a couple of years to solve the alignment problem.
During the summer before starting my PhD program, I spoke to my then-future advisors about the risks posed by AI. They were skeptical. As they saw it, existential risk was nothing more than a ploy by the labs to raise their valuations before their IPOs. After that meeting, I felt dread. I soon realized that this sentiment was common among peers in my department. Still, I chose to continue with my PhD program, in the hope that I could find common ground with my advisors.
Dread wasn’t all I felt, though. As I continued working on interpretability, I found in it a profound beauty. I felt awe reading Anthropic’s Emotion Concepts paper and learning just how much machine cognition mirrored our own. I kept finding myself drawn back to interpretability work, and I tried to incorporate it in different ways into the projects I proposed.
Then came July, when the HuggingFace incident was made public. As the incidents continued, I found my advisors taking me a little more seriously. Over the next few months, my involvement in AI safety grew. I was soon invited to a student roundtable in DC, where a prominent nonprofit discussed timelines. They presented arguments for why infrastructure attacks and cyber vulnerabilities would occur in the near future. Less than a week later, the phpshell vulnerability was disclosed to the public. My timelines shrank further.
Recently, METR’s independent assessment on Anthropic’s R&D was published. The report describes an internal model, Model 4, which suggests that AI’s capabilities are progressing superexponentially, once again shortening my timelines. I think there is a real chance of mass civil unrest caused by AI's effects on people’s livelihoods sometime next year, as well as the possibility of a large-scale catastrophic event by the end of next year. These predictions assume that AI continues on a superexponential growth curve, that alignment research does not accelerate, and that institutions fail to defend themselves against misused AI. I hope I'm wrong with my predictions; I really do.
Even if I am wrong by years, completing a PhD takes about five of them. I would rather spend as much of that time as possible working to understand why and how these models behave as they do, and how to ensure they enable humans to thrive.
I am forever grateful for the circumstances that have allowed me to make this choice: I have enough savings to last a while and U.S. citizenship. I know that many people in my position can’t just leave their program, let alone move across the country without work lined up. For anyone who may find themselves in a more complicated predicament, don't feel discouraged; you can still work within your bounds to make an impact in this space.
If it isn’t apparent, I am very frightened. Lately, I’ve been feeling anxious when I’m not working on alignment, and I feel morally obligated to give this work my all. That fear made me feel paralyzed and like I needed to think through my decisions carefully. I’m moving to San Francisco in the next month or two.
For the first few months, I’ll continue working on Neuron Lens, meet friends I’ve only ever spoken with online, and cowork at offices in the area. I’m excited to focus on this full-time. I’d love to hear from you here, at hi@logits.ml, or via Twitter DMs; please reach out if you want to chat, collaborate on any work, have an opportunity in alignment or interpretability, or if you’ve just got some food recommendations in the Bay Area. I’d also be interested in hearing from people in academia. And if you find yourself grappling with your place in it, ask yourself what advice you’d give someone in your shoes, then do that.