Introduction
People are making important decisions about artificial intelligence every day, ranging from whether and how to use it to what policies should be passed regarding its development, without substantial awareness of the possibilities and risks (Founder's Pledge 2024, p. 94). Given limited awareness, one research area I believe could lead to effective artificial intelligence risk mitigation is incident monitoring. Incident monitoring exclusively deals with the risks associated with AI by cataloging times where intelligent systems have caused problems in the real world. Other sectors have been doing incident monitoring for a long time, such as the FAA with air transport services. Whether it's the airline industry or artificial intelligence, the goal here is simple: to capture, organize, and provide access to information about unwanted outcomes so that humanity can learn from them and mitigate future harm. Open access to accurate information is essential for AI research, journalism, and lawmaking, among other fields. Constituents across these areas benefit from AI incident databases for tasks such as risk trend analysis, reporting that shapes public opinion, and making sound policy decisions.
Areas of Activity
Let’s break down AI incident monitoring into three areas of activity, which can help clarify where and how philanthropy might have an impact. We can begin with INPUTS, the information drawn into a monitoring system. In order to be effective, a repository should comprehensively capture relevant episodes. If a repository is only capturing incidents that make it into mainstream news, for example, the database would likely be missing critical information. The organization behind this incident monitoring must be equipped with the resources necessary to identify incidents and track down relevant information (staff, connections, legal permissions/protections, etc.). One way they might do this is an proactive approach, wherein the organization seeks firsthand incident reporting from AI users by regularly checking if they have had relevant experiences such as chatbot-based confabulations/hallucinations.
A second area of activity concerns OPERATIONS, what an incident monitoring organization does with the information it collects. On an internal level, this deals with the ease of inputting new incidents and organizing them with existing ones for repository employees and contributors. On a public level, this deals with the approachability and navigability of the repository’s data. Both are important to the repository working as it should, since people are more effectively and efficiently able to extract value if UI/UX is well optimized. Successful operations management requires the monitoring organization to have the capital and ability to do this work internally or to outsource it to others.
The third entry point is OUTPUTS and OUTREACH, effectively how the repository connects to the world. The repository must make the outputs accessible and appealing to those who wish to use it. This may sound redundant with operations, but outputs rather deals with the way that this information is packaged. One model that current AI incident repositories are using is a weekly newsletter with top stories in order to boost engagement. This is a good start but there may be opportunities to go further, such as putting together a more comprehensive but less frequent report on artificial intelligence incidents and safety (something like Stanford HAI’s annual AI Index Report). Monitoring organizations must also raise awareness with relevant potential users and organizations who could benefit from the data. Regarding the field of AI research, for example, monitoring and repository organizations might attend or host events such as conferences in order to make themselves better known. This “outputs and outreach” work requires thoughtful leadership and further capital investment.
Current Leaders
Four organizations are currently leading AI incident monitoring. Let’s assess their philanthropic funding viability with respect to the points we’ve already discussed:
- Weekly update: focus is on big headlines with analysis, guest writers
- No centralized index of AI incidents – just what’s in the weekly update
- Many other focuses such as technical research and AI risk community building
- Upshot: Already well resourced – not clear additional funding will make a difference
- Weekly update: tries to be comprehensive
- Centralized index with strong organizational system
- Accepts user contributions
- Already at critical mass of staff
- Upshot: Good position to accept more funding to make meaningful progress
OECD AI Incident Monitoring (AIM)
- Centralized index with strong organizational system
- Research methodology relies on news reporting
- AI arm of the OECD; many other focuses such as international cooperation and AI risk education
- Upshot: Already well resourced – not clear additional funding will make a difference
AI, Algorithmic, and Automation Incidents and Controversies (AIAAIC)
- Weekly update: tries to be comprehensive
- Centralized index with poor organizational system
- Accepts user contributions; strong grassroots approach
- Lacks critical mass – highly dependent on two individuals
- Unclear if additional funding will be impactful
- Upshot: Good position to accept more funding to make meaningful progress
In sum, several organizations are already operating in this space. An initial look suggests that one of them, AIID, may have meaningful potential for increasing its impact with the help of additional philanthropy.
Why is this Important?
There are three reasons that this is an important issue to address: scale, tractability, and neglectedness (as adapted by Founder’s Pledge from William MacAskill, p. 307). First is scale. There is an extremely wide range of AI risk awareness and a concerning number of AI decision makers are already under-informed. This is despite the fact that AI itself seems poised to touch nearly everyone’s life, with some non-trivial risk of catastrophic outcomes. AI incident monitoring is important because being more aware of the past makes people more resilient to face similar challenges in the future. A repository would be particularly useful to individuals and organizations researching AI, influencing public opinion, and shaping our governmental policies.
Second is tractability. The three areas of activity described above (inputs, operations, and outputs/outreach) are potential entry points where minimal capital injection could make a meaningful difference in AI risk mitigation.
Third is neglectedness. AI incident monitoring is extremely neglected. Prominent government (e.g. Supreme Court 2025) and non-government organizations (e.g. ACLU 2025) are starting to reference these repositories, and yet of the four organizations mentioned above, two have limited resources. The other two are well-resourced but have critical shortcomings that suggest they aren’t a big priority for their parent organizations. This leads us to the second aspect of neglectedness, assessing it from an institutional point of view. Current organizational methodology is sporadic: one repository might be over-reliant on news articles and miss some inputs, another might organize their data in a confusing way that repels users, a third might fail to reach out to different audiences and grow their user base. Ultimately this work may be best done by a single international eminent authority for a few key reasons. First, consolidation would avoid redundancy and confusion with potential users who currently don’t know which incident database is best for their purposes. Second, funding by a broad set of international sources would minimize the risk of bias. Third, AI incidents are currently occurring at a rate where decent monitoring is currently manageable by a handful of people per organization, but this is almost certainly going to intensify in the near future (OECD reports their rate of incidents cataloged increasing dramatically over the last two years, visualized below in Figure 2).
In sum, this initial analysis suggests promise in the intervention of AI incident monitoring. A closer examination may reveal whether there is value in supporting one of the current players or in looking for opportunities to align, coordinate, and perhaps consolidate organizations to face the challenges and opportunities that lie ahead.
Acknowledgement: The author is an active member of one of the incident monitoring repositories mentioned here, the AIAAIC. This essay was written prior to my involvement with the organization.