x

LESSWRONG
LW

Omega.

Top postsTop post

Omega.

Message

This is the LessWrong account of Anonymous Omega (Forum profile). We are writing a series of posts critiquing prominent AI safety labs (Forum sequence here).

38

4

9

3y

Omega.

Top postsTop post

Critiques of prominent AI safety labs: Conjecture

Cross-posted from the EA Forum. See the original here. Internal linking has not been updated for LW due to time constraints and will take you back to the original post. In this series, we consider AI safety organizations that have received more than $10 million per year in funding. There have already been several conversations and critiques around MIRI (1) and OpenAI (1,2,3), so we will not be covering them. The authors include one technical AI safety researcher (>4 years experience), and one non-technical community member with experience in the EA community. We’d like to make our critiques non-anonymously but believe this will not be a wise move professionally speaking. We believe our criticisms stand on their own without appeal to our positions. Readers should not assume that we are completely unbiased or don’t have anything to personally or professionally gain from publishing these critiques. We’ve tried to take the benefits and drawbacks of the anonymous nature of our post seriously and carefully, and are open to feedback on anything we might have done better. This is the second post in this series and it covers Conjecture. Conjecture is a for-profit alignment startup founded in late 2021 by Connor Leahy, Sid Black and Gabriel Alfour, which aims to scale applied alignment research. Based in London, Conjecture has received $10 million in funding from venture capitalists (VCs), and recruits heavily from the EA movement. We shared a draft of this document with Conjecture for feedback prior to publication, and include their response below. We also requested feedback on a draft from a small group of experienced alignment researchers from various organizations, and have invited them to share their views in the comments of this post. We would like to invite others to share their thoughts in the comments openly if you feel comfortable, or contribute anonymously via this form. We will add inputs from there to the comments section of this post, but will likely not be

12Jun 12, 2023

Top postsTop post

Omega.

Subscribe

Message

This is the LessWrong account of Anonymous Omega (Forum profile). We are writing a series of posts critiquing prominent AI safety labs (Forum sequence here).

38

4

9

3y

Top postsTop post

Critiques of prominent AI safety labs: Conjecture

Cross-posted from the EA Forum. See the original here. Internal linking has not been updated for LW due to time constraints and will take you back to the original post. In this series, we consider AI safety organizations that have received more than $10 million per year in funding. There have already been several conversations and critiques around MIRI (1) and OpenAI (1,2,3), so we will not be covering them. The authors include one technical AI safety researcher (>4 years experience), and one non-technical community member with experience in the EA community. We’d like to make our critiques non-anonymously but believe this will not be a wise move professionally speaking. We believe our criticisms stand on their own without appeal to our positions. Readers should not assume that we are completely unbiased or don’t have anything to personally or professionally gain from publishing these critiques. We’ve tried to take the benefits and drawbacks of the anonymous nature of our post seriously and carefully, and are open to feedback on anything we might have done better. This is the second post in this series and it covers Conjecture. Conjecture is a for-profit alignment startup founded in late 2021 by Connor Leahy, Sid Black and Gabriel Alfour, which aims to scale applied alignment research. Based in London, Conjecture has received $10 million in funding from venture capitalists (VCs), and recruits heavily from the EA movement. We shared a draft of this document with Conjecture for feedback prior to publication, and include their response below. We also requested feedback on a draft from a small group of experienced alignment researchers from various organizations, and have invited them to share their views in the comments of this post. We would like to invite others to share their thoughts in the comments openly if you feel comfortable, or contribute anonymously via this form. We will add inputs from there to the comments section of this post, but will likely not be

12Jun 12, 2023

Critiques of prominent AI safety organizations: Introduction

7Jul 19, 2023

Critiques of prominent AI safety labs: Redwood Research

4Apr 17, 2023

Omega.'s Shortform

2Jun 9, 2023

Critiques of prominent AI safety organizations: Introduction

Cross-posted from the EA Forum. What is this series (and who are we)? This is a series of evaluations of technical AI safety (TAIS) organizations. We evaluate organizations that have received more than $10 million per year in funding and that have had limited external evaluation.[1] The primary authors of...

Jul 19, 20237

Critiques of prominent AI safety labs: Conjecture

Cross-posted from the EA Forum. See the original here. Internal linking has not been updated for LW due to time constraints and will take you back to the original post. In this series, we consider AI safety organizations that have received more than $10 million per year in funding. There...

Jun 12, 202312

Omega.'s Shortform

Jun 9, 20232

Critiques of prominent AI safety labs: Redwood Research

In this series, we evaluate AI safety organizations that have received more than $10 million per year in funding. We do not critique MIRI (1) and OpenAI (1,2,3) as there have been several conversations and critiques of these organizations. The authors of this post include two technical AI safety researchers,...

Apr 17, 20234