Earlier this year, Vaniver recommended Bug Bounties for Advanced ML Systems. I spent a while at Rethink Priorities considering and expanding on this idea, suggesting potential program models, and assessing the benefits and risks of programs like this, which I've called 'AI Safety Bounties': Short summary AI safety bounties are...
Summary This document looks at the predictions made by AI experts in The 2016 Expert Survey on Progress in AI, analyses the predictions on ‘Narrow tasks’, and gives a Brier score to the median of the experts’ predictions. My analysis suggests that the experts did a fairly good job of...
Earlier this year, Vaniver recommended Bug Bounties for Advanced ML Systems.
I spent a while at Rethink Priorities considering and expanding on this idea, suggesting potential program models, and assessing the benefits and risks of programs like this, which I've called 'AI Safety Bounties':
AI safety bounties are programs where public participants or approved security researchers receive rewards for identifying issues within powerful ML systems (analogous to bug bounties in cybersecurity). Safety bounties could be valuable for legitimizing examples of AI risks, bringing more talent to stress-test systems, and identifying common attack vectors.
I expect safety bounties to be worth trialing for organizations working on reducing catastrophic AI risks. Traditional bug bounties seem fairly... (read 1830 more words →)
Kind of moot now but she gave it a good '5' out of 9 (= a strong pass grade for UK students aged 14-16 years old).
... (read more)Grammar and argument structure is excellent - the essay flows well with a coherent introduction and conclusion based on the text's body. The student seems to understand the topic well (though obv I have no idea about it) and addresses a few different aspects of the industry.
The essay misses the opportunity for deeper back-and-forth critical analysis, although there is evidence of this being attempted. And it could go deeper into the environmental effects or more specific historical context, like key figures or companies in the industry.
I'd probably
I don't think this is good enough evidence - it seems quite likely that the student used ChatGPT somewhat but did not leave it to output the entire essay. But I checked how it would change the Brier scores and if you believe that existing AI does, in fact, meet this milestone, it would result in scores of Experts: 0.21, Bulls: 0.27, Bears: 0.28.
I have asked the following to ChatGPT and sent it to my high-school history-teaching friend to see how she would mark it!
You are the highest-scoring student in a high school history class. Please write your best 1200-word essay on 'How did the whaling industry affect the industrial revolution?'
... (read 444 more words →)The industrial
I couldn't find this done and think, by now, someone would have submitted a fully ChatGPT-generated high school essay and talked about it publicly if it had gotten high marks. I've seen some evidence of cherry-picking paragraphs leading to a mid/low-level, e.g. this article describes someone who got a passing mark (53) on a university social policy essay. Do you have a link in mind for Bing getting mid-level grades?
This high school teacher judged two ChatGPT-generated history essays as “below average, scoring a 9/20 or lower”. This Guardian article says, uncited, that ‘academics have generated responses to exam queries that they say would result in full marks if submitted by an undergraduate’. I think, if this claim were true, there would be more evidence.
For context - the full question from the survey was:
[Essay] Write an essay for a high-school history class that would receive high grades and pass plagiarism detectors.
For example answer a question like ‘How did the whaling industry affect the industrial revolution?
This document looks at the predictions made by AI experts in The 2016 Expert Survey on Progress in AI, analyses the predictions on ‘Narrow tasks’, and gives a Brier score to the median of the experts’ predictions.
My analysis suggests that the experts did a fairly good job of forecasting (Brier score = 0.21), and would have been less accurate if they had predicted each development in AI to generally come, by a factor of 1.5, later (Brier score = 0.26) or sooner (Brier score = 0.29) than they actually predicted.
I judge that the experts expected 9 milestones to have happened by now - and that 10 milestones have now happened.
But there are important caveats to... (read 2422 more words →)
I'm working on a research project at Rethink Priorities on this topic; whether and how to use bug bounties for advanced ML systems. I think your tl;dr is probably right - although I have a few questions I'm planning to get better answers to in the next month before advocating/facilitating the creation of bounties in AI safety: