Mass_Driver1mo

I hope you're right that job losses in more 'stable' fields will catalyze interest in a constructive response, but I was surprised over the last 20 years or so as the market power of workers in various traditionally stable industries collapsed for mundane economic reasons and not much changed in the policy world. Professors, lawyers, accountants, civil servants, and even some types of physicians have all been squeezed fairly heavily in the US just from globalization, monopolization, and deregulation. There was some brief pushback around the time of Occupy Wall Street, and then after that increased job insecurity became part of the new status quo.

Entry level software engineers are now facing serious... (read more)

Replying toWarning Aliens About the Dangerous AI We Might Create

Mass_Driver3mo

Warning Aliens About the Dangerous AI We Might Create

Yeah, but have you done a back of the envelope calculation here, or has anyone else? What size target could we hit in the Andromeda galaxy using, e.g., $50 million at our current tech levels, and how long could we transmit for? How large of a receiver would that target need to have pointing toward us in order to receive the message with anything like reasonable fidelity? If our message is focused no more tightly than on "a star," then would the receivers need an antenna the size of a solar system to pick it up? If not, why not?

I'm not sure codebreaking is a reasonable test of a supposedly universal language.... (read more)

Replying toWarning Aliens About the Dangerous AI We Might Create

Mass_Driver3mo

Warning Aliens About the Dangerous AI We Might Create

I think this is a promising strategy that deserves more investigation. Your game theory analysis of dark forest-type situations is particularly compelling; thank you for sharing it. I have two main questions: (1) to what extent is this technically feasible, and (2) how politically costly would the weirdness of the proposal be?

For technical feasibility, I was very surprised to hear you suggest targeting the Andromeda Galaxy. I agree that in principle the nearest stars are more likely to already have whatever data they might want about Earth, but I think of "the nearest stars" as being within 50 light-years or so, not as including the entire Milky Way. Can you explain why... (read more)

Replying toFour Questions to Refine Your Policy Proposal

Mass_Driver4mo

Four Questions to Refine Your Policy Proposal

I am literally a tort litigator in the United States! I worked for several years as a personal injury and product safety litigator.

Although the American legal system holds out "use reasonable care" as its official standard, in practice this gets further defined and specified based on rules found in OSHA, IEEE, or whatever the professional code of practice is for the relevant industry. As a plaintiff's attorney, if you can't point to a specific rule or norm that the defendant broke, you're extremely unlikely to recover any damages. The violation of the rule or the norm is taken as very persuasive evidence that the defendant didn't use reasonable care -- and, conversely, if you can't point to any crisp rule violation, that's usually taken as very persuasive evidence that the defendant did use reasonable care.

Four Questions to Refine Your Policy Proposal

Mass_Driver

4mo

I gave a talk this year at LessOnline about how to make AI governance research more relevant to policymakers (videos available here), and the subtopic from that talk that got the most audience engagement was how to turn a vague moral sentiment into a concrete policy proposal. During the talk, I helped people refine their proposals by asking them a series of questions about what they wanted to do and why. This works better in a conversational setting, but for those who couldn’t attend, I wanted to write up a list of some of the most common and/or important questions, so that everyone can try to make their policy proposals more concrete.

This... (read 1689 more words →)

Replying toMAGA speakers at NatCon were mostly against AI

Mass_Driver5mo

MAGA speakers at NatCon were mostly against AI

Agreed; well said.

Replying toMAGA speakers at NatCon were mostly against AI

Mass_Driver5mo

MAGA speakers at NatCon were mostly against AI

Yes, that's exactly right, we do. That's what it means to be an ally rather than a friend. America allied with the Soviet Union in World War 2; this is no different. When someone earnestly offers to help you literally save the world, you hold your nose and shake their hand.

Replying toMainstream Grantmaking Expertise (Post 7 of 7 on AI Governance)

Mass_Driver7mo

Mainstream Grantmaking Expertise (Post 7 of 7 on AI Governance)

Fair enough; we certainly paid much less than people could make in the private sector at CAIP, for essentially that reason. It's good for nonprofit staff to have some skin in the game.

My suggestion to consider more competitive wages is mostly a response to Oliver suggesting that LTFF has had a serious and long-term challenge in hiring as many people as they would need to fully accomplish their mission.

Mass_Driver7mo

Part of the distinction I try to draw in my sequence is that the median person at CSET or RAND is not "in politics" at all. They're mostly researchers at think tanks, writing academic-style papers about what kinds of policies would be theoretically good for someone to adopt. Their work is somewhat more applied/concrete than the work of, e.g., a median political science professor at a state university, but not by a wide margin.

If you want political experts -- and you should -- you have to go talk to people who have worked on political campaigns, served in the government, or led advocacy organizations whose mission is to convince specific politicians to... (read more)

Mass_Driver7mo

I'm the author of the LW post being signal-boosted. I sincerely appreciate Oliver's engagement with these critiques, and I also firmly disagree with his blanket dismissal of the value of "standard practices."

As I argue in the 7th post in the linked sequence, I think OpenPhil and others are leaving serious value on the table by not adopting some of the standard grant evaluation practices used at other philanthropies, and I don't think they can reasonably claim to have considered and rejected them -- instead the evidence strongly suggests that they're (a) mostly unaware of these practices due to not having brought in enough people with mainstream expertise, and (b) quickly deciding that... (read more)

Replying toMainstream Grantmaking Expertise (Post 7 of 7 on AI Governance)

Mass_Driver7mo

Mainstream Grantmaking Expertise (Post 7 of 7 on AI Governance)

These are important points, and I'm glad you're bringing them up!

Is spending a lot of time to assess new grantmakers merely distressing (but still net positive in terms of extending your total grantmaking ability), or is it actually causing you to lose time in expectation? In other words, if you spend 40 hours recruiting and assessing candidates, does one of those candidates then go on to do 100+ hours of useful grantmaking work? Or is it more like 20 hours of useful grantmaking work?
How closely connected is the shortage of people willing to be full-time grantmakers with an expectation that grantmakers will already be fluent in technical AI safety when they start

Mass_Driver

8mo

Previously in this sequence, I estimated that we have 3 researchers for every advocate working on US AI governance, and I argued that this ratio is backwards – we need the political power provided by advocates to have a chance of preventing misaligned superintelligence. A few researchers might be useful as a ‘multiplier effect’ on the power of many advocates, but the converse is not true: there’s no “magic bullet” that AI governance researchers can hope to discover that could substitute for an army of political foot soldiers. Even the best political ideas still need many political activists to spread them, because the political arena is noisy and contested.

Unfortunately, we have very... (read 11070 more words →)

Political Funding Expertise (Post 6 of 7 on AI Governance)

Mass_Driver

8mo

INTRODUCTION

The Story So Far

In my first three posts in this sequence, I estimated that we have 3 researchers for every advocate working on US AI governance, and I argued that this ratio is backwards. Even the best research offers only modest and indirect support for advocacy, and research alone has negligible political power. Without political power, we can’t change the bad incentives of AI developers that are very likely to lead to the collapse of human civilization.

In the fourth post of this sequence, I acknowledged that some amount of initial research to lay out the philosophical foundations of a new field might be needed before advocacy can begin, but I also showed... (read 4001 more words →)

•••

Orphaned Policies (Post 5 of 7 on AI Governance)

Mass_Driver

9mo

In previous posts in this sequence, I laid out a case for why most AI governance research is too academic and too abstract to have much influence over the future. Politics is noisy and contested, so we can’t expect that good AI governance ideas will spread on their own – we need a large team of people who are actively promoting those ideas. Unfortunately, we currently have at least 3 researchers for every advocate, so many policy ideas have been “orphaned,” i.e., nobody is taking those ideas and showing them to decision-makers who have the power to implement them.

The best way of addressing this imbalance would be to shift funding and jobs... (read 4608 more words →)

Shift Resources to Advocacy Now (Post 4 of 7 on AI Governance)

Mass_Driver

9mo

In my previous post in this series, I estimated that we have 3 researchers for every advocate working on US AI governance, and I argued that this ratio is backwards. When allocating staff, you almost always want to have more people working on the more central activity. I argued that in the case of AI policy, the central activity is advocacy, not research, because the core problem to be solved is fixing the bad private incentives faced by AI developers.

As I explained, the problem with these incentives is less that they’re poorly understood, and more that they require significant political effort to overturn. As a result, we’ll need to shift significant resources... (read 9335 more words →)

•••

We're Not Advertising Enough (Post 3 of 7 on AI Governance)

Mass_Driver

9mo

In my previous post in this series, I explained why we urgently need to change AI developers’ incentives: if we allow the status quo to continue, then an AI developer will recklessly deploy misaligned superintelligence, which is likely to permanently disempower humanity and cause billions of deaths. AI governance research can potentially be helpful in changing this status quo, but only if it’s paired with plenty of political advertising – research by itself doesn’t automatically convince any of the people who have the power to rein in AI developers.

Executive Summary

Here, in the third post, I want to make it clear that we are not doing nearly enough political advertising to successfully change... (read 8215 more words →)

115

•••

The Need for Political Advertising (Post 2 of 7 on AI Governance)

Mass_Driver

9mo

In my previous post in this sequence, I made a fundraising pitch for my organization, the Center for AI Policy (CAIP), asking readers to donate to us so we can continue lobbying Congress to pass strong AI safety legislation.

Here, in the second post, I want to zoom out from CAIP’s particular concerns and argue that the AI safety movement as a whole should be investing far more resources into “political advertising.” We need to seek out individual politicians and personally let them know about the risk from misaligned superintelligence and show them what they can do to mitigate that risk.

The core of my argument for this second post is that AI governance... (read 3761 more words →)

•••

Please Donate to CAIP (Post 1 of 7 on AI Governance)

Mass_Driver

9mo

I am Jason Green-Lowe, the executive director of the Center for AI Policy (CAIP). Our mission is to directly convince Congress to pass strong AI safety legislation. As I explain in some detail in this post, I think our organization has been doing extremely important work, and that we’ve been doing well at it. Unfortunately, we have been unable to get funding from traditional donors to continue our operations. If we don’t get more funding in the next 30 days, we will have to shut down, which will damage our relationships with Congress and make it harder for future advocates to get traction on AI governance. In this post, I explain what... (read 9882 more words →)

123

LINK: Quora brainstorms strategies for containing AI risk

Mass_Driver

10y

In case you haven't seen it yet, Quora hosted an interesting discussion of different strategies for containing / mitigating AI risk, boosted by a $500 prize for the best answer. It attracted sci-fi author David Brin, U. Michigan professor Igor Markov, and several people with PhDs in machine learning, neuroscience, or artificial intelligence. Most people from LessWrong will disagree with most of the answers, but I think the article is useful as a quick overview of the variety of opinions that ordinary smart people have about AI risk.

https://www.quora.com/What-constraints-to-AI-and-machine-learning-algorithms-are-needed-to-prevent-AI-from-becoming-a-dystopian-threat-to-humanity

Help Build a Landing Page for Existential Risk?

Mass_Driver

11y

The Big Orange Donate Button

Traditional charities, like Oxfam, Greenpeace, and Amnesty International, almost all have a big orange button marked "Donate" right on the very first page that loads when you go to their websites. The landing page for a major charity usually also has vivid graphics and some short, easy-to-read text that tells you about an easy-to-understand project that the charity is currently working on.

I assume that part of why charities have converged on this design is that potential donors often have short attention spans, and that one of the best ways to maximize donations is to make it as easy as possible for casual visitors to the website to (a)... (read 644 more words →)

LESSWRONG
LW

LESSWRONG
LW

Mass_Driver

Please Donate to CAIP (Post 1 of 7 on AI Governance)

We're Not Advertising Enough (Post 3 of 7 on AI Governance)

I'm scared.

An Outside View on Less Wrong's Advice

Mass_Driver

Mass_Driver

Four Questions to Refine Your Policy Proposal

Mainstream Grantmaking Expertise (Post 7 of 7 on AI Governance)

Political Funding Expertise (Post 6 of 7 on AI Governance)

Orphaned Policies (Post 5 of 7 on AI Governance)

Shift Resources to Advocacy Now (Post 4 of 7 on AI Governance)

We're Not Advertising Enough (Post 3 of 7 on AI Governance)

The Need for Political Advertising (Post 2 of 7 on AI Governance)

An Activist View of AI Governance

Mass_Driver

Please Donate to CAIP (Post 1 of 7 on AI Governance)

We're Not Advertising Enough (Post 3 of 7 on AI Governance)

I'm scared.

An Outside View on Less Wrong's Advice

Mass_Driver

Mass_Driver

Four Questions to Refine Your Policy Proposal

Mainstream Grantmaking Expertise (Post 7 of 7 on AI Governance)

Political Funding Expertise (Post 6 of 7 on AI Governance)

Orphaned Policies (Post 5 of 7 on AI Governance)

Shift Resources to Advocacy Now (Post 4 of 7 on AI Governance)

We're Not Advertising Enough (Post 3 of 7 on AI Governance)

The Need for Political Advertising (Post 2 of 7 on AI Governance)

An Activist View of AI Governance

INTRODUCTION

The Story So Far

Executive Summary

The Big Orange Donate Button