Other links

• Short interview with Ian Hogarth 
• Twitter thread

Some quotes from the report

Introduction

The Taskforce is a start-up inside government, delivering on the ambitious mission given to us by the Prime Minister: to build an AI research team that can evaluate risk at the frontier of AI. As AI systems become more capable they may significantly augment risks. An AI system that advances towards human ability at writing software could increase cybersecurity threats. An AI system that becomes more capable at modelling biology could escalate biosecurity threats. To manage this risk technical evaluations are critical - and these need to be developed by a neutral third party - otherwise we risk AI companies marking their own homework.

Given these potentially significant frontier risks, as of today, the Taskforce is being renamed to the Frontier AI Taskforce.

This is the Frontier AI Taskforce’s first progress report.

Expert advisory board spanning AI Research and National Security

Given that a number of risks from frontier systems touch areas of national security, we have established an expert advisory board that bridges some of the world’s leading experts in AI research and safety as well as key figures from the UK’s national security community. Our initial advisory board members are:

Yoshua Bengio. Yoshua is most known for his pioneering work in deep learning, earning him the 2018 A.M. Turing Award, “the Nobel Prize of Computing,” with Geoffrey Hinton and Yann LeCun. He is a Full Professor at Université de Montréal, and the Founder and Scientific Director of Mila – Quebec AI Institute.

Paul Christiano. Paul is one of the leading researchers in the field of AI Alignment. He is co-founder of ARC, the Alignment Research Centre and previously ran the language model alignment team at OpenAI.

Matt Collins. Matt is the UK’s Deputy National Security Adviser for Intelligence, Defence and Security. IYKYK.

Anne Keast-Butler. Anne is the director of GCHQ. Anne has an impressive track record at the heart of the UK’s national security network, helping to counter threats posed by terrorists, cyber-criminals and malign foreign powers.

Alex van Someren. Alex is the UK’s Chief Scientific Adviser for National Security. Alex was previously a venture capital investor and entrepreneur, focusing on investing in early stage ‘deep technology’ startups.

Helen Stokes-Lampard. Beyond national security and AI research expertise we are also excited to build an advisory board that can speak to critical uses of frontier AI on the frontlines of society. Helen is not only a practising General Practitioner observing how conversational AI tools can impact day to day medical diagnoses but also an incredibly experienced leader across the UK’s medical community, having Chaired the Royal College of General Practitioners and currently Chair of the Academy of Medical Royal Colleges.

Matt Clifford. Matt is the Prime Minister’s joint Representative for the AI Safety Summit, Chair of ARIA and co-founder of Entrepreneur First. His appointment as Expert Advisory Board Vice Chair demonstrates the level of coordination across UK initiatives around frontier AI - including the Taskforce and the AI Safety Summit.

Recruitment of expert AI researchers

We are drawing on world-leading expertise:

Yarin Gal will join as Research Director of the Taskforce from Oxford where he is head of the Oxford Applied and Theoretical Machine Learning Group. Yarin is a globally recognised leader in Machine Learning, and will retain his position as Associate Professor at Oxford.

David Krueger will be working with the Taskforce as it scopes its research programme in the run up to the summit. David is an Assistant Professor at the University of Cambridge’s Computational and Biological Learning lab, where he leads a research group focused on Deep Learning and AI Alignment.

The Taskforce is housed inside the UK’s Department for Science Innovation and Technology (DSIT) which employs roughly 1500 civil servants. When I arrived in June there was just one frontier AI researcher employed by the department with 3 years of experience in frontier AI.

This lone researcher was Nitarshan Rajkumar who put his PhD on pause to join DSIT in April, and is a testament to what an earnest, incredibly hardworking technical expert can accomplish when they commit to public service. Michelle Donelan, Secretary of State for DSIT recruited Nitarshan and he has materially influenced many of the bold efforts that the UK has been making to invest in frontier AI Safety. We need more Nitarshans!

Thanks to a huge push by the Taskforce team we now have a growing team of AI researchers with over 50 years of collective experience at the frontier of AI. If this is our metric for state capacity in frontier AI, we have managed to increase it by an order-of-magnitude in just 11 weeks. Our team now includes researchers with experience from DeepMind, Microsoft, Redwood Research, The Center for AI Safety and the Center for Human Compatible AI.

Partnering with leading technical organisations

Leading on AI safety does not mean starting from scratch or working alone – we are building on and supporting the work conducted by a range of cutting-edge organisations. We are excited to announce our initial set of partnerships with:

ARC Evals is a non-profit that works on assessing catastrophic risks from frontier AI systems, and have previously worked with OpenAI and Anthropic on evaluating the “autonomous replication and adaptation” capabilities of their systems before release. We’ll be working closely with the ARC Evals team to assess risks just beyond the frontier in the lead up to the UK’s AI Safety Summit. We’ll also be engaging with the team at Redwood Research, and with Jeff Alstott, Christopher Mouton and their team at non-profit RAND in driving forward this agenda.

Trail of Bits is a leading cybersecurity research and consulting firm that has helped secure some of the world’s most targeted organisations. We are kicking off a deep collaboration to understand risks at the intersection of cybersecurity and frontier AI systems. This work will be led by Heidy Khlaaf, who specialises in software evaluation, specification, and verification for safety-critical systems, and who also led the safety evaluation of Codex while at OpenAI.

The Collective Intelligence Project is a non-profit that incubates new governance models for transformative technology, with a mission to direct technological development towards the collective good. Co-founders Divya Siddarth and Saffron Huang will join us on secondment to help us develop a range of social evaluations for frontier models.

The Center for AI Safety is a non-profit that works to reduce societal-scale risks from AI, through fundamental safety research, research infrastructure, and technical expertise to support policymakers. We’ll be working with Dan Hendrycks and his team in the lead up to the summit to interface with and enable the broader scientific community.

Building the technical foundations for AI research inside government

A core goal of the Taskforce is to give AI researchers inside the government the same resources to work on AI Safety that they would find at leading companies like Anthropic, DeepMind, or OpenAI. As the Prime Minister announced, these companies have already committed to giving us deep model access so that researchers in the Taskforce are not constrained in their ability to work on model evaluations. We’re also working in close collaboration with No10 Data Science (‘10DS’) so that our researchers and engineers have the compute infrastructure they need to hit the ground running, for model fine-tuning, interpretability research, and more.