Ben Pace

I'm an admin of this site; I work full-time on trying to help people on LessWrong refine the art of human rationality.

Longer bio: www.lesswrong.com/posts/aG74jJkiPccqdkK3c/the-lesswrong-team-page-under-construction#Ben_Pace___Benito

Sequences

AI Alignment Writing Day 2019
Transcript of Eric Weinstein / Peter Thiel Conversation
AI Alignment Writing Day 2018
Share Models, Not Beliefs

Wiki Contributions

Load More

Comments

For the record, our relationship to supporting events for this ecosystem is changing from something like "all of our resources are the same, here have my venue for free if you need it" to "markets and pricing are a great way for large masses of people to coordinate on the value of a good or service, let's coordinate substantially via trade". 

For instance, during a previous cohort of SERI MATS scholars at the Lightcone Offices, I spent a couple of weeks of work adding a second floor and getting it furnished and doing interior design, hiring another support person to the office team, and then later on dealing with closing it down and downsizing when the demand went away. I did all of that for free, and was not paid salary or anything by MATS, it was part of my Lightcone job, because we wanted to support mentorship happening in the AI alignment ecosystem. It's different this time around. They're paying us a substantial amount of money (well over $100k) for the use of 2.5 of our nicely furnished and designed buildings for 2 months, an amount that makes the trade pretty good for Lightcone (and I hope+expect to work hard and make it worthwhile for SERI MATS too!). The other workshops Habryka has mentioned elsethread will also mostly be paying trade partners (general pricing TBD as we get a better sense of the demand).

I bring this up because the extent to which funds for Lightcone are spent supporting SERI MATS in particular (and other teams/orgs/events) is (I suspect) much less than you are thinking.

Here are some things I like about owning this space:

  • You don't have to ask anyone's permission to make modifications to it or to use it in unusual ways. For instance, if we want to cover every wall with whiteboards (as we've done in many rooms), we don't have to ask anyone's permission, we just order the boards, hire a few laborers, and have them cut them to size and nail them all along the walls. If I want to turn a 4-person team office into a 4-person bedroom, I just get a few people, carry the furniture into storage, carry some beds in, and we're done. This means that in the span of just a few days whole buildings can be turned from one use case into another (e.g. from housing into a conference venue), as demand comes up. This is not true of rented office spaces (where you certainly cannot reliably sleep many people or do surgery to the walls) or chain hotels (I cannot ask a hotel to use a room as a team office for 3 montsh), and most residential spaces are far too small. As another datapoint about the limitations of rented spaces, the last office building I was using wouldn't even allow us to have a fully-dark nap room due to regulations about lighting visibility in the event of a fire.
  • This space is much more aesthetically beautiful and harmonious with our intentions than most alternatives we could find in Berkeley. It is relatively central in Berkeley, yet has a large, private outdoor area that you can work from in the sun all day. The rooms are not inhuman exact squares with sterile lighting, they're all unique shapes, often with nice old woods and we can install our own lighting. We've had the space to build a number of wooden outdoor huts to work from with electricity and soon-to-be-heated cushions. In my model of the world the aesthetics of a space really affect the sorts of affordances and feelings and problem-solving-approaches that people will consider. For example I think this sort of natural beauty helps people be more willing to go deep on an idea or hypothesis for a longer amount of time.
  • The space is made of 5 separate buildings, which gives a little village-like feel, and (we're hoping) can allow some fairly different things to be happening on-site at the same time. In one building you could have a self-contained retreat (CFAR workshop, AI alignment workshop, etc), in another building a few x-risk focused teams could be getting work done in their offices as they have ever day for the last 2 months, another building could be having a weekly communal lunch open to 100+ people as well as hosting a few visitors in rooms upstairs who have flown in from out of town for a few weeks, etc. My hope is that the separate buildings can allow a lot of different things to be happening concurrently, and have natural boundaries between them.

Perhaps of interest, when we were considering alternative locations, the main other places that had the properties of my first two bullets were educational religious spaces. The School of Religion, the School of Theology, and a strange surprise-Buddhist-temple that Habryka and I unexpectedly found ourselves in one evening (as the woman was showing us around the school-like building, she fully walked past the temple doors, until I politely asked to look inside, and she unlocked them to show us a ~7k square foot room with a 40-foot high ceiling, filled with golden statues and colorful ribbons hanging from the ceiling and ancient texts inscribed on rotating pillars and 300 folding chairs and a big stage). These places had a lot of beauty. But one of them basically wasn't on sale, and the other two were only partially on sale (we couldn't have owned the whole property and would have to share with some religious groups, which is not a total dealbreaker but I strongly prefer having full ownership). 

We also considered renting solely office spaces, which would have been much faster to get started with, and were on the verge of going through on a deal last year. But then at the last minute they explained the elevator needed replacing and would be out of use for the first 2 months of us living there (which is a pretty big obstruction for moving in all of our heavy furniture up ~3 floors). They wouldn't negotiate at all on this and we walked away. I actually heard (epistemic status: I assign 75% to this being true, I have pinged the person who said this to me to double-check) that the elevator only actually got fixed around a month or two ago. To me not having to deal with this is part of the advantage of having full-ownership I describe in the first bullet above.

Thanks, I was confused that I couldn't find it.

  1. He cites ARC’s GPT-4 evaluation and Lesswrong in his AI report which has a large section on safety.

I wanted to double-check this. 

The relevant section starts on page 94, "Section 4: Safety", and those pages cite in their sources around 10-15 LW posts for their technical research or overviews of the field and funding in the field. (Make sure to drag up the sources section to view all the links.)

Throughout the presentation and news articles he also has a few other links to interviews with ppl on LW (Shane Legg, Sam Altman, Katja Grace).

Thanks! This is fairly tempting. I'm a bit concerned by 

Some other explanation that's of this level of "very weird"

To be clear, if it were just the 4 hypotheses you mention, then I feel pretty good about this, and I'd just want to reflect over 200:1 versus 100:1. 

  • For instance, if some form of undiscovered bat or sea creature is actually good at flying and that explains the images, does that count as "very weird"? That is, it's not an animal that can design technologies, but it is the cause of a bunch of the UFO reportings.
  • Also "psychic phenomenon" seems kind of ill-defined. Derren Brown can do hypnosis and seances that have strong effects on people, there's a lot of suggestibility in people, if a bunch of UFO sighting were due to someone doing similar things but even better and naturally, does that count as psychic phenomena?
  • For the laws of physics, if we discover a new theory of physics, and the UFOs were directly responsible for figuring it out, then I'm happy to credit that as responsible. Time travel would certainly count.

Regarding the hypotheses, I'd probably want to determine now some set of LW posters to resolve it if we disagree. My first guess is that Oliver Habryka, Alyssa Vance, and Vaniver could be good, where if any of them think the bet resolves in your favor then it does.

I have not read this post, and I have not looked into whatever the report is, but I'm willing to take a 100:1 bet that there is no such non-human originating craft (by which I mean anything actively designed by a technological species — I do not mean that no simple biological matter of any kind could not have arrived on this planet via some natural process like an asteroid), operationalized to there being no Metaculus community forecast (or Manifold market with a sensible operationalization and reasonable number of players) that assigns over 50% probability to this being a craft of non-human design being true in the next 2 years.

(I am actually going to check that this post makes a claim like this now, before posting, in case I am off-topic. K, looks like I am broadly on-topic.)

"AI maniacs" is maybe a term that meets this goal? Mania is the opposite side to depression, both of which are about having false beliefs just in opposite emotionally valenced directions, and also I do think just letting AI systems loose in the economy is the sort of thing a maniac in charge of a civilization would do.

The rest of my quick babble: "AI believers" "AI devotee" "AI fanatic" "AI true believer" "AI prophets" "AI ideologue" "AI apologist" "AI dogmatist" "AI propagandists" "AI priests".

I had the first of these pieces printed and left on my desk to read the other day.

I think I tend to base my level of alarm on the log of the severity*probability, not the absolute value. Most of the work is getting enough info to raise a problem to my attention to be worth solving. "Oh no, my house has a decent >30% chance of flooding this week, better do something about it, and I'll likely enact some preventative measures whether it's 30% or 80%." The amount of work I'm going to put into solving it is not twice as much if my odds double, mostly there's a threshold around whether it's worth dealing with or not.

Setting that aside, it reads to me like the frame-clash happening here is (loosely) between "50% extinction, 50% not-extinction" and "50% extinction, 50% utopia", where for the first gamble of course 1:1 odds on extinction is enough to raise it to "we need to solve this damn problem", but for the second gamble it's actually much more relevant whether it's a 1:1 or a 20:1 bet. I'm not sure which one is the relevant one for you two to consider.

(Strong-upvote, weak-disagree. I sadly don't have time right now to reflect and write why I disagree with this position but I hope someone else who disagrees does.)

Load More