LESSWRONG
LW

Malo — LessWrong

¯\_(ツ)_/¯

Obviously just one example, but Schneier has generally been quite skeptical, and he blurbed the book.

but subconsciously I notice that MIRI was cleaning house before the book launch (e.g. taking down EY's light novel because it might look bad)

Do you have any other concrete example here besides the novel?

Malo5mo*

Maybe that's wrong; maybe the issue was lack of reach rather than exhausting the persuadees' supply, and the book-packaging + timing will succeed massively. We'll see.

This is certainly the hope. Most people in the world have never read anything that anyone here has ever written on this subject.

Malo5mo

FWIW, and obviously this is just one anecdote, but a member of Congress who read an early copy, and really enjoyed it, said that Chapter 2 was his favorite chapter.

Replying toIMO challenge bet with Eliezer

Malo7mo

IMO challenge bet with Eliezer

The market has now resolved to yes, with Paul confirming.

Replying toNew Endorsements for “If Anyone Builds It, Everyone Dies”

Malo8mo

New Endorsements for “If Anyone Builds It, Everyone Dies”

Huh, I thought I fixed this. Thanks for flagging, will ensure I fix now.

Replying toNew Endorsements for “If Anyone Builds It, Everyone Dies”

Malo8mo

New Endorsements for “If Anyone Builds It, Everyone Dies”

Also oddly, the US version is on many of Amazon's international stores including the German store ¯\_(ツ)_/¯

Replying toNew Endorsements for “If Anyone Builds It, Everyone Dies”

Malo8mo

New Endorsements for “If Anyone Builds It, Everyone Dies”

Schneier is also quite skeptical of the risk of extinction from AI. Here's a table o3 generated just now when I asked it for some examples.

Date	Where he said it	What he said	Take-away
1 June 2023	Blog post “On the Catastrophic Risk of AI” (written two days after he signed the CAIS one-sentence “extinction risk” statement)	“I actually don’t think that AI poses a risk to human extinction. I think it poses a similar risk to pandemics and nuclear war — a risk worth taking seriously, but not something to panic over.” (schneier.com)	Explicitly rejects the “extinction” scenario, placing AI in the same (still-serious) bucket as pandemics or nukes.
1 June 2023	Same post, quoting his 2018 book Click Here

Malo

8mo

Nate and Eliezer’s forthcoming book has been getting a remarkably strong reception.

I was under the impression that there are many people who find the extinction threat from AI credible, but that far fewer of them would be willing to say so publicly, especially by endorsing a book with an unapologetically blunt title like If Anyone Builds It, Everyone Dies.

That’s certainly true, but I think it might be much less true than I had originally thought.

Here are some endorsements the book has received from scientists and academics over the past few weeks:

This book offers brilliant insights into the greatest and fastest standoff between technological utopia and dystopia and how we can and should

... (read 1139 more words →)

488

MIRI 2024 Mission and Strategy Update

Malo

As we announced back in October, I have taken on the senior leadership role at MIRI as its CEO. It’s a big pair of shoes to fill, and an awesome responsibility that I’m honored to take on.

There have been several changes at MIRI since our 2020 strategic update, so let’s get into it.^[1]

The short version:

We think it’s very unlikely that the AI alignment field will be able to make progress quickly enough to prevent human extinction and the loss of the future’s potential value, that we expect will result from loss of control to smarter-than-human AI systems.

However, developments this past year like the release of ChatGPT seem to have shifted the Overton window in a... (read 2113 more words →)

223

•••

MIRI’s 2019 Fundraiser

Malo

(Crossposted from the MIRI blog)

MIRI’s 2019 fundraiser is now live, December 2–31!

Over the past two years, huge donor support has helped us double the size of our AI alignment research team. Hitting our $1M fundraising goal this month will put us in a great position to continue our growth in 2020 and beyond, recruiting as many brilliant minds as possible to take on what appear to us to be the central technical obstacles to alignment.

Our fundraiser progress, updated in real time (including donations made during the Facebook Giving Tuesday event):

Donate

MIRI is a CS/math research group with a goal of understanding how to reliably “aim” future general-purpose AI systems at known goals. For... (read 2419 more words →)

MIRI’s 2018 Fundraiser

Malo

(Crossposted from the MIRI blog)

Update Nov. 28: We’ve updated this post to announce that Haskell developer Edward Kmett has joined the MIRI team! Additionally, we’ve added links to some of our recent Agent Foundations research.

Our donors showed up in a big way for Giving Tuesday! Through Dec. 29, we’re also included in a matching opportunity by professional poker players Dan Smith, Aaron Merchak, Matt Ashton, and Stephen Chidwick, in partnership with Raising for Effective Giving. As part of their Double-Up Drive, they’ll be matching up to $200,000 to MIRI and REG. Donations can be made either on doubleupdrive.com, or by donating directly on MIRI’s website and sending your donation receipt to receipts@doubleupdrive.com.... (read 1840 more words →)

MIRI's 2017 Fundraiser

Malo

Update 2017-12-27: We've blown past our 3rd and final target, and reached the matching cap of $300,000 for the $2 million Matching Challenge! Thanks so much to everyone who supported us!

All donations made before 23:59 PST on Dec 31st will continue to be counted towards our fundraiser total. The fundraiser total includes projected matching funds from the Challenge.

MIRI’s 2017 fundraiser is live through the end of December! Our progress so far (updated live):

Donate Now

MIRI is a research nonprofit based in Berkeley, California with a mission of ensuring that smarter-than-human AI technology has a positive impact on the world. You can learn more about our work at “Why AI Safety?” or via MIRI... (read 3757 more words →)

MIRI's 2017 Fundraiser

Malo

MIRI’s 2017 fundraiser is live through the end of December! Our progress so far (updated live):

Donate Now

In 2015, we discussed our interest in potentially branching out to explore multiple research programs simultaneously once we could support a larger team. Following recent changes to our overall picture of the strategic landscape, we’re now moving ahead on that goal and starting to explore new research directions while... (read 3710 more words →)

SI is coming to Oxford, looking for hosts, trying to keep costs down

Malo

13y

The Singularity Institute is coming to Oxford, England for AGI-2012!

AGI-2012 runs December 8th–11th, but many of us will be arriving as early as the 6th or 7th, and some will be leaving as late as the 16th. We are looking for generous members of the Less Wrong community who would be willing to host us during our visit.

Who's coming?

Why you might want to host us?

You want to help SI save some money.
You want to see Luke Meuhlhauser fact #8 for yourself.
You want to hangout with us, which is great, because we also want to hangout with you.
You want to marvel at Louie's boyish good looks in person.

If you live

Malo

13y

The Singularity Institute is looking to hire someone familiar with decision theory to help finish Luke's Decision Theory FAQ (with the possibility of working on other projects afterwards). The first task of this individual will be to expand on section 11 by explaining each of the Newcomblike problems and showing how EDT, CDT, CDT+, and TDT perform on each of them.

Pay will be hourly and starts at $15/hr, but will increase if you produce a good product. You must be able to commit an average of 20+hrs/wk.

Those applying for the position should be generally familliar with most of the following:

Axiomatic decision theory (VNM, Savage)
EDT, CDT, CDT + causal graphs (CDT+), and TDT
Newcomb's problem, medical Newcomb's problem, Parfit's hitchhiker, etc.

Working remotely

Malo

14y

The Singularity Institute and the Center for Applied Rationality are looking to expand their web development team. As per usual we'd like to post this opportunity to the LW community first. All work will be on a contract basis (you quote us your hourly rate).

Perks:

Make money while contributing to organizations you care about.
Age and credentials are irrelevant; only product matters.

Requirements:

WordPress and PHP skills (some features require PHP coding).
Good design sense.
Ability to start work on tasks/features quickly and deliver them in a timely manner (i.e., responsiveness).

If you're interested, apply here!

[Applications Closed] The Singularity Institute is hiring remote LaTeX editors

Malo

14y

The Singularity Institute has recently made the transition to a LaTeX based document production workflow for its publications and republished its existing research papers. However, there is still much work to be done and we need more remote LaTeX editors. Some projects currently in the queue include converting both The Sequences and Facing the Singularity into LaTeX based books.

As with other remote positions, pay is hourly and starts at $14/hr but will increase if you produce a good product.

Perks:

Work flexible hours: Complete your work in few large chunks or many small ones—at 03:00 or 18:00—it's up to you.
Work from wherever you please: your home (maybe even in bed), your local coffee shop,

... (read more)