It’s old news by now, but OpenAI solved 372 open math problems. They released the results suddenly, and the response was equally swift and contentious. Fields Medalists put out a letter calling for attention to what they call a “severe misalignment” between mathematics and AI companies. Their central claim is that mathematics works towards collective understanding, and that more solutions alone don’t advance that goal. Soon after, OpenAI committed millions of dollars to damage control to help mathematicians use AI. Much has already been said about this letter, and I suggest that anyone interested read it themselves, so I will cling to just one idea from the joint statement: “We are witnessing a general threat to intellectual work.”
The following is a collection of reflections attempting to address this claim, each coming from my own experiences in my department and university, alongside trends I’ve picked up from national reports. Some of it’s intuition, some of it’s observation, and I’m sure at other institutions the state of the discourse might be quite different. Still, everywhere I look, AI-use seems to be a persistent issue. I want to consider why that is, and why it feels like we’re not doing anything about it.
The state of things
A few months ago, an MIT taskforce released a set of recommendations on AI-use in education. They spend some time detailing the various ways AI can and does interrupt traditional education models: students don’t do their homework, don’t go to office hours. They abandon study groups and professors offer fewer research assistantships. Hands-on learning is replaced by automation.
Of course, AI only replaces these things because it proves itself better at completing assignments, studying, and teaching than traditional methods (to a subjective degree). This is an underlying issue, but its consequence is worse: AI is quietly replacing the most intellectually productive parts of education. One of the key threats of AI, the report argues, is that “...students may internalize a transactional model in which assignments are outputs, teachers are evaluators, peers are optional, and knowledge (or an MIT degree) is an optimizable commodity to be acquired or produced as efficiently as possible.”
But when hasn’t it been a transactional model? Without financial aid, the per-year cost of attendance at MIT is $92,760. For students coming from a middle-class family in a high cost-of-living state, their family is fronting the full bill without the excess income to support it. Of course, students will load classes to graduate early, strategically distribute their effort to get a decent GPA, and/or focus most of their time on the internships or research necessary to get jobs or graduate positions (which, year after year, are getting more competitive).
Faculty are in similar straits. Research requires funding; funding requires proof of concept; proof of concept requires a history of significant outputs. While navigating this loop, faculty are also expected to teach and mentor the next generation of scholars and leaders. Researching well takes time and attention. Teaching well takes time and attention. It's no surprise that faculty offload parts of these tasks to AI. As the standards of output and involvement become higher, their careers depend on it.
To the authors’ credit, they recognize that this isn’t a new issue. As they see it, AI is an opportunity to finally address longstanding issues in academic integrity head on. But will that be enough? You can improve education to accommodate AI, but that doesn’t change the economic conditions academia exists in. Maybe we craft coursework that forces student engagement, but students and faculty are still in the same bind. As I said, AI being better at these tasks than humans is an underlying issue, but even more fundamentally, our metrics for “better” are more easily fulfilled by AI than by humans. This is a design problem: it suggests that these metrics don’t consistently measure intellectual achievement in the first place.[1]
So what are universities doing about the challenges of AI in academia? It varies by institution, but across the board it's fair to say the response has been slow. It’s telling enough that even MIT didn’t release a report like this until 2026, let alone begin to implement the suggested changes. My own university has committed itself to the benefits of AI in research—hiring AI-related positions in earnest in 2022 and recently invested massive sums of money into AI development—yet, the actual AI-use guidelines (notably, not an enforceable policy) don’t go much further than “check your work and don’t do plagiarism, please.”
Universities are responding asymmetrically to AI. Actions are comparatively big and fast when it comes to integrating AI, but relatively slow and shallow when mitigating the harm it might cause. This, I will argue, is because AI’s harms against academia are a more efficient iteration of academia’s harms against itself. Well before AI was a capable reasoner, universities were fighting against their own ideals of expanding human knowledge.
On human knowledge
Here I’ll make the case for human-generated knowledge being distinct and not entirely replaceable by AI-generated knowledge, with a different approach than standard human exceptionalism.
I’ve always liked to look up the authors of my assigned readings. There are some intellectually proper reasons for doing this— identifying context, bias, influences in the work—but in complete honesty, I just like being able to imagine the person speaking to me through the text.
In my second semester of college, I read Doubleweaving Two-Spirit Critiques by Qwo-Li Driskell. The article points to the lack of Native American voices or considerations in the emerging area of “queer of color critiques”: critiques that analyze racism and patriarchy as a joint enterprise. (Here, “critique” means to contest or reinterpret a dominant theory or narrative, and is usually thought to emerge from the author's subjective social position). To illustrate this point, Driskell uses the image of “doubleweaving”, a Cherokee basketry technique, to describe how queer and native perspectives can and should intermingle, overlap, and strengthen each other.
Driskell’s voice is loud in the piece. You can hear their frustration and indignation. They speak of Two-Spirit and indigenous folks with “we” and “us.” They leverage their identity as part of the critique: “I can speak for us; we need to be part of the conversation.”
I looked them up. A few months prior, a watchdog investigated Driskell’s family history and determined they had no ancestry in any of the tribes they claimed to be a part of. Were they lying outright, or did they actually believe it? I don’t know. As of writing, Driskell’s former university seems to have confirmed these accusations.
My professor and classmates were also shocked when I shared this information. We discussed the article in class anyway because, as my professor put it, “we shouldn’t throw the baby out with the bathwater.” Yet, I couldn’t help but feel like it wasn’t the same baby anymore. I can’t read an argument for native representation the same, knowing that the author misrepresented themself as native. The history of the author changed the text.
For the past year or so, when I tell people I’m studying anthropology, they will sometimes respond, “Well at least you won’t be replaced by AI.”
Anthropology has never been known for its lucrative career prospects, so it's a bit funny to see it framed as a relatively stable and resilient job market. (To be clear, it’s not.) There seems to be a narrative floating around that as AI completes more and more of our work, the most employable skills will be working with people and generating original ideas. This is anthropology’s domain. But why can’t it be AI’s as well?
There’s no reason to assume that AI won’t get really good at interacting with humans, or that it won’t be able to make things within the window of what we deem “original”. Not getting into the potentials of robotics, it's very plausible that AI will be able to complete virtually all human cognitive tasks. But even before we get to that point, it’s worth comparing anthropology with the kind of knowledge about human culture LLMs can already produce. Consider this riddle:
I consume a large and wide variety of texts about humans. Listen to them and interact with them. I find patterns. Scramble and regurgitate what I’ve seen into something intelligible to what I know. I capture, in new words, the way people speak and think and write. What am I?[3]
Obviously the actual mechanisms here are different, but it is worth considering anthropological work and complex text prediction as similar meaning-making processes. The premise of immersion in the sample data, constructing a useful model of that data, and distilling that model through new language is shared. The main difference is the issue of authorship, and it’s this issue that still defines legitimate anthropological knowledge. It’s not entirely fair to say “AI can’t produce knowledge on the human experience because it's not human” unless you similarly contend that anthropologists working in cultures other than their own can’t produce knowledge about those cultures.[4]Still, who wrote it matters, and how honestly they represent themself also matters. Just as the extent to which Driskell’s claimed indigenous identity affected the merits of their work is contested, whether or not an AI can be a responsible author is up for debate.
So let’s say that, for now, anthropologists are safe from AI only if there’s more value in a human perspective than in the degree of output, as is probably true for most “humanistic” professions. This isn’t an optimistic prognosis. Many people have already turned to AI for entertainment, therapy, education, art, music, religious guidance, companionship, and so on. These roles have proven to be replaceable so long as we don’t care whether or not another human fills them (or the barriers for humans to fill them are perceived as too high).
Anthropology[5]is already one of the disciplines hardest hit by growing preference for impact-driven research. Departments have been shrinking in size long before funding cuts rattled through the rest of academia. Anthropologists have slowly slid themselves under the label of “social scientist,” sometimes in an earnest preference for the scientific method, and sometimes to defend the legitimacy of their work. Anthropology (which is notoriously slow, subjective, nebulously defined, and with at-best dubious career prospects outside of academia) hasn’t fared well as institutions increasingly value high volumes of research and measurable, marketable impact.
In many ways, anthropology has already been experiencing what academia as a whole is facing in the wake of the AI boom. We’ve already seen first hand that a “human touch” is diminishing in value compared to measurable and high-volume research. And just as AI has positively impacted many areas of research, not all of these pressures on anthropology are bad. The discipline has undeniable limitations, and I, too, am in favor of the turn towards applied work. Nevertheless, these pressures reflect what academic institutions prioritize. Suddenly, we all need to reconvince ourselves and our institutions that slow, contemplative, and empirically flawed work is still valuable.
To restate, I believe anthropology (and the humanities in general) aren’t really vulnerable to AI. Rather, they are vulnerable to academic incentives for publications, impact, and efficiency, which have overshadowed a pure pursuit of human knowledge for a while now. It’s because of this that AI is able to compete with humans in institutionalized academic work at large.
Facing the future
So I agree with the Fields Medalists: we are facing a general threat to intellectual work. But what I’ve hoped to capture in these vignettes is that AI is more of an instrumentation of that threat than its origin. The parts of intellectual work that we, as humans, value most are also the most threatened by AI, precisely because they are already threatened by our institutions. It’s not that AI inherently undermines human values. We have already created systems that undermine our values, and of course AI will follow those incentives to their logical end.
I think this tension between the ideals and real incentives of academic institutions reflects prevailing attitudes towards AI within academia, which I will lump into two broad categories:
First, are the optimists. They are behind the billion-dollar investments, the hiring booms, the blooming and questionably developed AI degrees popping up across the US. They see AI skills as necessary for preparing students for the workforce and as a valuable investment into public good. AI is the future of science and business, and universities should establish themselves in that future now. This perspective helps the university to accrue prestige, secure funding, and adapt to emerging value systems post-AI.
Second, are the pessimists. They dislike AI, and many refuse to use it entirely. They are concerned about impacts on the environment, the cost of living, and the job market. Those who do engage with AI worry about its ethics, how it might make our society’s systemic failures even more difficult to avoid. They are often internal dissidents to the university’s handling of AI, but are nevertheless used as evidence of a “responsible AI policy.” They follow much of the argument I have put together in this essay: AI is a problem because it’s making the problems we already have worse.
Admittedly, there are many people in between. CS students who rely on the technology for every assignment, yet are increasingly pessimistic on the future of work. AI optimists that imagine a future of abundance, but are nevertheless anxious about cybersecurity. Many people who don’t know and don’t care.[6]In any case, dominant narratives in universities don’t encompass AI Safety or existential risk. It’s not that I don’t hear them discussed. Since the explosion of public awareness of existential risk, I’ve overheard classroom and gym conversations toeing the line between humor and anxiety: “have you heard how OpenAIs gonna kill all of us?” Still, most public-facing voices dismiss existential risk as alarmist, distracting, or (most commonly) don’t discuss it at all.
There are probably several contributing factors to this gap in the discourse, but I will focus on one: the narrative of existential risk is emotionally outcompeted. The university is already facing an “existential risk” from AI: before it threatens the existence of humanity, it is threatening the existence of the university. As I have argued in this post, the cracks in the post-AI university are foundational. For the two camps described above, we see that universities and academics are already faced with the choice to respond to AI or become irrelevant. Adapt or die. It’s not just people’s jobs that are on the line, but the very ideals they have built their lives and identities around. Unfortunately, I think that a hypothetical AI-driven doomsday (which is already burned by skepticism towards the concept of “doomsday”) is emotionally overshadowed by the very tangible benefits and injustices appearing right before people’s eyes.
But there’s still a chance to leverage this into something useful. Pessimism about the future of AI is reflected in pessimism about humanity. Walking out of my machine-learning class, my professor stated bluntly that we will never have the AI utopia because “humans are evil. We won’t let it happen.” On the other hand, many of the people optimistic about AI (including those mentioned above) are the same ones that already have faith in our existing political and economic systems. Popular attitudes towards AI and the ways it’s used emerge from existing narratives around work, truth, value, etc. If the goal is to mobilize towards AI regulation, it seems we need to understand how a narrative of existential risk fits against preexisting understandings of AI, extinction, authoritarianism, and other items in the list of AI-caused bad endings.
I have presented universities as a case where the negative impacts of AI aren’t because of AI, but because of institutional problems that already existed. If we take this as a smaller version of AI’s societal impacts as a whole, it seems that if we secure one of the good endings, it will be because of what we built outside of AI as well.
Thanks to Space and Nat for feedback on this post
[This post is part of an interactive research project, which you can learn more about on my personal blog. If you want to comment, but don’t want to be part of the research, feel free to mark your comment with // ]
My argument here closely follows the Adversarial Goodhart. Students know what gets them success, and often optimize for that instead of actual knowledge. Once most people start doing it, the students that learn the most are no longer the most successful along standard metrics.
I’m implying that “agents have no history.” I will disagree with myself a bit here. AI absolutely has a history: in its design, training, etc. The extent to which individual agents or instances have subjective experience is something I know nothing about, but is surely within the realm of “possible.” What I’m pointing to more here is that AI doesn’t (yet) have true personhood or social experience, and so it simply can’t make claims about certain things, at least not in a way that seems credible to humans.
This might be a disastrously crude description of how models work, but bear with me. I’ll possibly make a separate post in future that provides a better breakdown for the parallels I see between LLMs and ethnography.
This isn’t a claim that human cultures are as different from each other as AI is from humanity as a whole. The distinction between insider vs. outsider still holds without defining a degree of difference.
I speak on anthropology because I’m most familiar with it, but I assume much of this applies to the humanities in general (history, English, philosophy, etc.).
It’s old news by now, but OpenAI solved 372 open math problems. They released the results suddenly, and the response was equally swift and contentious. Fields Medalists put out a letter calling for attention to what they call a “severe misalignment” between mathematics and AI companies. Their central claim is that mathematics works towards collective understanding, and that more solutions alone don’t advance that goal. Soon after, OpenAI committed millions of dollars to damage control to help mathematicians use AI. Much has already been said about this letter, and I suggest that anyone interested read it themselves, so I will cling to just one idea from the joint statement: “We are witnessing a general threat to intellectual work.”
The following is a collection of reflections attempting to address this claim, each coming from my own experiences in my department and university, alongside trends I’ve picked up from national reports. Some of it’s intuition, some of it’s observation, and I’m sure at other institutions the state of the discourse might be quite different. Still, everywhere I look, AI-use seems to be a persistent issue. I want to consider why that is, and why it feels like we’re not doing anything about it.
The state of things
A few months ago, an MIT taskforce released a set of recommendations on AI-use in education. They spend some time detailing the various ways AI can and does interrupt traditional education models: students don’t do their homework, don’t go to office hours. They abandon study groups and professors offer fewer research assistantships. Hands-on learning is replaced by automation.
Of course, AI only replaces these things because it proves itself better at completing assignments, studying, and teaching than traditional methods (to a subjective degree). This is an underlying issue, but its consequence is worse: AI is quietly replacing the most intellectually productive parts of education. One of the key threats of AI, the report argues, is that “...students may internalize a transactional model in which assignments are outputs, teachers are evaluators, peers are optional, and knowledge (or an MIT degree) is an optimizable commodity to be acquired or produced as efficiently as possible.”
But when hasn’t it been a transactional model? Without financial aid, the per-year cost of attendance at MIT is $92,760. For students coming from a middle-class family in a high cost-of-living state, their family is fronting the full bill without the excess income to support it. Of course, students will load classes to graduate early, strategically distribute their effort to get a decent GPA, and/or focus most of their time on the internships or research necessary to get jobs or graduate positions (which, year after year, are getting more competitive).
Faculty are in similar straits. Research requires funding; funding requires proof of concept; proof of concept requires a history of significant outputs. While navigating this loop, faculty are also expected to teach and mentor the next generation of scholars and leaders. Researching well takes time and attention. Teaching well takes time and attention. It's no surprise that faculty offload parts of these tasks to AI. As the standards of output and involvement become higher, their careers depend on it.
To the authors’ credit, they recognize that this isn’t a new issue. As they see it, AI is an opportunity to finally address longstanding issues in academic integrity head on. But will that be enough? You can improve education to accommodate AI, but that doesn’t change the economic conditions academia exists in. Maybe we craft coursework that forces student engagement, but students and faculty are still in the same bind. As I said, AI being better at these tasks than humans is an underlying issue, but even more fundamentally, our metrics for “better” are more easily fulfilled by AI than by humans. This is a design problem: it suggests that these metrics don’t consistently measure intellectual achievement in the first place.[1]
So what are universities doing about the challenges of AI in academia? It varies by institution, but across the board it's fair to say the response has been slow. It’s telling enough that even MIT didn’t release a report like this until 2026, let alone begin to implement the suggested changes. My own university has committed itself to the benefits of AI in research—hiring AI-related positions in earnest in 2022 and recently invested massive sums of money into AI development—yet, the actual AI-use guidelines (notably, not an enforceable policy) don’t go much further than “check your work and don’t do plagiarism, please.”
Universities are responding asymmetrically to AI. Actions are comparatively big and fast when it comes to integrating AI, but relatively slow and shallow when mitigating the harm it might cause. This, I will argue, is because AI’s harms against academia are a more efficient iteration of academia’s harms against itself. Well before AI was a capable reasoner, universities were fighting against their own ideals of expanding human knowledge.
On human knowledge
Here I’ll make the case for human-generated knowledge being distinct and not entirely replaceable by AI-generated knowledge, with a different approach than standard human exceptionalism.
I’ve always liked to look up the authors of my assigned readings. There are some intellectually proper reasons for doing this— identifying context, bias, influences in the work—but in complete honesty, I just like being able to imagine the person speaking to me through the text.
In my second semester of college, I read Doubleweaving Two-Spirit Critiques by Qwo-Li Driskell. The article points to the lack of Native American voices or considerations in the emerging area of “queer of color critiques”: critiques that analyze racism and patriarchy as a joint enterprise. (Here, “critique” means to contest or reinterpret a dominant theory or narrative, and is usually thought to emerge from the author's subjective social position). To illustrate this point, Driskell uses the image of “doubleweaving”, a Cherokee basketry technique, to describe how queer and native perspectives can and should intermingle, overlap, and strengthen each other.
Driskell’s voice is loud in the piece. You can hear their frustration and indignation. They speak of Two-Spirit and indigenous folks with “we” and “us.” They leverage their identity as part of the critique: “I can speak for us; we need to be part of the conversation.”
I looked them up. A few months prior, a watchdog investigated Driskell’s family history and determined they had no ancestry in any of the tribes they claimed to be a part of. Were they lying outright, or did they actually believe it? I don’t know. As of writing, Driskell’s former university seems to have confirmed these accusations.
My professor and classmates were also shocked when I shared this information. We discussed the article in class anyway because, as my professor put it, “we shouldn’t throw the baby out with the bathwater.” Yet, I couldn’t help but feel like it wasn’t the same baby anymore. I can’t read an argument for native representation the same, knowing that the author misrepresented themself as native. The history of the author changed the text.
So what’s left when the author has no history?[2]
The humanities case
For the past year or so, when I tell people I’m studying anthropology, they will sometimes respond, “Well at least you won’t be replaced by AI.”
Anthropology has never been known for its lucrative career prospects, so it's a bit funny to see it framed as a relatively stable and resilient job market. (To be clear, it’s not.) There seems to be a narrative floating around that as AI completes more and more of our work, the most employable skills will be working with people and generating original ideas. This is anthropology’s domain. But why can’t it be AI’s as well?
There’s no reason to assume that AI won’t get really good at interacting with humans, or that it won’t be able to make things within the window of what we deem “original”. Not getting into the potentials of robotics, it's very plausible that AI will be able to complete virtually all human cognitive tasks. But even before we get to that point, it’s worth comparing anthropology with the kind of knowledge about human culture LLMs can already produce. Consider this riddle:
Obviously the actual mechanisms here are different, but it is worth considering anthropological work and complex text prediction as similar meaning-making processes. The premise of immersion in the sample data, constructing a useful model of that data, and distilling that model through new language is shared. The main difference is the issue of authorship, and it’s this issue that still defines legitimate anthropological knowledge. It’s not entirely fair to say “AI can’t produce knowledge on the human experience because it's not human” unless you similarly contend that anthropologists working in cultures other than their own can’t produce knowledge about those cultures.[4]Still, who wrote it matters, and how honestly they represent themself also matters. Just as the extent to which Driskell’s claimed indigenous identity affected the merits of their work is contested, whether or not an AI can be a responsible author is up for debate.
So let’s say that, for now, anthropologists are safe from AI only if there’s more value in a human perspective than in the degree of output, as is probably true for most “humanistic” professions. This isn’t an optimistic prognosis. Many people have already turned to AI for entertainment, therapy, education, art, music, religious guidance, companionship, and so on. These roles have proven to be replaceable so long as we don’t care whether or not another human fills them (or the barriers for humans to fill them are perceived as too high).
Anthropology[5]is already one of the disciplines hardest hit by growing preference for impact-driven research. Departments have been shrinking in size long before funding cuts rattled through the rest of academia. Anthropologists have slowly slid themselves under the label of “social scientist,” sometimes in an earnest preference for the scientific method, and sometimes to defend the legitimacy of their work. Anthropology (which is notoriously slow, subjective, nebulously defined, and with at-best dubious career prospects outside of academia) hasn’t fared well as institutions increasingly value high volumes of research and measurable, marketable impact.
In many ways, anthropology has already been experiencing what academia as a whole is facing in the wake of the AI boom. We’ve already seen first hand that a “human touch” is diminishing in value compared to measurable and high-volume research. And just as AI has positively impacted many areas of research, not all of these pressures on anthropology are bad. The discipline has undeniable limitations, and I, too, am in favor of the turn towards applied work. Nevertheless, these pressures reflect what academic institutions prioritize. Suddenly, we all need to reconvince ourselves and our institutions that slow, contemplative, and empirically flawed work is still valuable.
To restate, I believe anthropology (and the humanities in general) aren’t really vulnerable to AI. Rather, they are vulnerable to academic incentives for publications, impact, and efficiency, which have overshadowed a pure pursuit of human knowledge for a while now. It’s because of this that AI is able to compete with humans in institutionalized academic work at large.
Facing the future
So I agree with the Fields Medalists: we are facing a general threat to intellectual work. But what I’ve hoped to capture in these vignettes is that AI is more of an instrumentation of that threat than its origin. The parts of intellectual work that we, as humans, value most are also the most threatened by AI, precisely because they are already threatened by our institutions. It’s not that AI inherently undermines human values. We have already created systems that undermine our values, and of course AI will follow those incentives to their logical end.
I think this tension between the ideals and real incentives of academic institutions reflects prevailing attitudes towards AI within academia, which I will lump into two broad categories:
First, are the optimists. They are behind the billion-dollar investments, the hiring booms, the blooming and questionably developed AI degrees popping up across the US. They see AI skills as necessary for preparing students for the workforce and as a valuable investment into public good. AI is the future of science and business, and universities should establish themselves in that future now. This perspective helps the university to accrue prestige, secure funding, and adapt to emerging value systems post-AI.
Second, are the pessimists. They dislike AI, and many refuse to use it entirely. They are concerned about impacts on the environment, the cost of living, and the job market. Those who do engage with AI worry about its ethics, how it might make our society’s systemic failures even more difficult to avoid. They are often internal dissidents to the university’s handling of AI, but are nevertheless used as evidence of a “responsible AI policy.” They follow much of the argument I have put together in this essay: AI is a problem because it’s making the problems we already have worse.
Admittedly, there are many people in between. CS students who rely on the technology for every assignment, yet are increasingly pessimistic on the future of work. AI optimists that imagine a future of abundance, but are nevertheless anxious about cybersecurity. Many people who don’t know and don’t care.[6]In any case, dominant narratives in universities don’t encompass AI Safety or existential risk. It’s not that I don’t hear them discussed. Since the explosion of public awareness of existential risk, I’ve overheard classroom and gym conversations toeing the line between humor and anxiety: “have you heard how OpenAIs gonna kill all of us?” Still, most public-facing voices dismiss existential risk as alarmist, distracting, or (most commonly) don’t discuss it at all.
There are probably several contributing factors to this gap in the discourse, but I will focus on one: the narrative of existential risk is emotionally outcompeted. The university is already facing an “existential risk” from AI: before it threatens the existence of humanity, it is threatening the existence of the university. As I have argued in this post, the cracks in the post-AI university are foundational. For the two camps described above, we see that universities and academics are already faced with the choice to respond to AI or become irrelevant. Adapt or die. It’s not just people’s jobs that are on the line, but the very ideals they have built their lives and identities around. Unfortunately, I think that a hypothetical AI-driven doomsday (which is already burned by skepticism towards the concept of “doomsday”) is emotionally overshadowed by the very tangible benefits and injustices appearing right before people’s eyes.
But there’s still a chance to leverage this into something useful. Pessimism about the future of AI is reflected in pessimism about humanity. Walking out of my machine-learning class, my professor stated bluntly that we will never have the AI utopia because “humans are evil. We won’t let it happen.” On the other hand, many of the people optimistic about AI (including those mentioned above) are the same ones that already have faith in our existing political and economic systems. Popular attitudes towards AI and the ways it’s used emerge from existing narratives around work, truth, value, etc. If the goal is to mobilize towards AI regulation, it seems we need to understand how a narrative of existential risk fits against preexisting understandings of AI, extinction, authoritarianism, and other items in the list of AI-caused bad endings.
I have presented universities as a case where the negative impacts of AI aren’t because of AI, but because of institutional problems that already existed. If we take this as a smaller version of AI’s societal impacts as a whole, it seems that if we secure one of the good endings, it will be because of what we built outside of AI as well.
Thanks to Space and Nat for feedback on this post
[This post is part of an interactive research project, which you can learn more about on my personal blog. If you want to comment, but don’t want to be part of the research, feel free to mark your comment with // ]
My argument here closely follows the Adversarial Goodhart. Students know what gets them success, and often optimize for that instead of actual knowledge. Once most people start doing it, the students that learn the most are no longer the most successful along standard metrics.
I’m implying that “agents have no history.” I will disagree with myself a bit here. AI absolutely has a history: in its design, training, etc. The extent to which individual agents or instances have subjective experience is something I know nothing about, but is surely within the realm of “possible.” What I’m pointing to more here is that AI doesn’t (yet) have true personhood or social experience, and so it simply can’t make claims about certain things, at least not in a way that seems credible to humans.
This might be a disastrously crude description of how models work, but bear with me. I’ll possibly make a separate post in future that provides a better breakdown for the parallels I see between LLMs and ethnography.
This isn’t a claim that human cultures are as different from each other as AI is from humanity as a whole. The distinction between insider vs. outsider still holds without defining a degree of difference.
I speak on anthropology because I’m most familiar with it, but I assume much of this applies to the humanities in general (history, English, philosophy, etc.).
This post describes some additional student attitudes towards AI, most of which I have also witnessed.