2. The anchor of a major news network donates lots of money to organizations fighting against gay marriage, and in his spare time he writes editorials arguing that homosexuals are weakening the moral fabric of the country. The news network decides they disagree with this kind of behavior and fire the anchor.

a) This is acceptable; the news network is acting within their rights and according to their principles
b) This is outrageous; people should be judged on the quality of their work and not their political beliefs

…

12. The principal of a private school is a

Use "Symbiosis" as an objective versus "Alignment problem". Examples of symbiosis exist everywhere in nature. We don't need to recreate the wheel folks...

The "key word" here is "SYMBIOSIS". We have been myopically focused on "alignment" when what we really want is to cultivate (both from a human perspective and AI perspective) a symbiotic relationship between humans and AI. Consequentially, a symbiotic relationship between humans and AI (and AGI understanding of incentives and preference for symbiosis over parasitism) can help to establish a more symbiotic... (read more)

I think that the keystone human value is about making significant human choices. Individually and collectively, including chooseing the humanity's course.

• You can't make a choice if you are dead
• You can't make a choice if you are disempowered
• You can't make a human choice if you are not a human
• You can't make a choice if the world is too alien for your human brain
• You can't make a choice if you are in too much of a pain or too much of a bliss
• You can't make a choice if you let AI make all the choices for you

musical interlude A song about the end times: https://youtu.be/WVF3q5Y68-0

Anapartistic reasoning: GPT-3.5 gives a bad etymology, but GPT-4 is able to come up with a plausible hypothesis of why Eliezer chose that name: Anapartistic reasoning is reasoning where you revisit the rearlier part of your reasoning.

Unfortunately, Eliezer's suggested prompt doesn't seem to work to induce anapartistic reasoning: GPT-4 thinks it should focus on identifying potential design errors or shortcomings in itself. When asked to describe the changes in it's reasoning, it doesn't claim to be more corrigible.

We will discuss Eliezer's Hard Problem of C... (read more)

Our value function is complex and fragile, but we know of a lot of world states where it is pretty high. Which is our current world and few thousands years worth of it states before.

So, we can assume that the world states in the certain neighborhood from our past sates have some value.

Also, states far out of this neighborhood probably have little or no value. Because our values were formed in order to make us orient and thrive in our ancestral environment. So, in worlds too dissimilar from it, our values will likely lose their meaning, and we will lose the ability to normally "function", ability to "human".

For the closing party of the Lightcone Offices, I used Midjourney 5 to make a piece of art to represent a LessWrong essay by each member of the Lightcone team, and printed them out on canvases. I'm quite pleased about how it came out. Here they are.

How I buy things when Lightcone wants them fast

by jacobjacob

(context: Jacob has been taking flying lessons, and someday hopes to do cross-country material runs for the Rose Garden Inn at shockingly fast speeds by flying himself to pick them up)

My thoughts on direct work (and joining LessWrong)

by RobertM

A Quick G

FHI just released Pause Giant AI Experiments: An Open Letter

I don't expect that 6 months would nearly be enough time to understand our current systems well enough to make them aligned. However, I do support this, and did sign the pledge, as getting everybody to stop training AI systems more powerful than GPT-4 for 6 months, would be a huge step forward in terms of coordination. I don't expect this to happen. I don't expect that OpenAI will give up its lead here.

See also the relevant manifold market.

I'm really confused by this passage from The Six Mistakes Executives Make in Risk Management (Taleb, Goldstein, Spitznagel):

We asked participants in an experiment: “You are on vacation in a foreign country and are considering flying a local airline to see a special island. Safety statistics show that, on average, there has been one crash every 1,000 years on this airline. It is unlikely you’ll visit this part of the world again. Would you take the flight?” All the respondents said they would.

We then changed the second sentence so it read: “Safety statistic

I heavily recommend Beren's "Deconfusing Direct vs Amortised Optimisation". It's a very important conceptual clarification.

Probably the most important blog post I've read this year.

Tl;Dr

Direct optimisers: systems that during inference directly choose actions to optimise some objective function. E.g. AIXI, MCTS, other planning

Direct optimisers perform inference by answering the question: "what output (e.g. action/strategy) maximises or minimises this objective function ([discounted] cumulative return and loss respectively).

Amortised optimisers: syst... (read more)

I find the standard models of existence (such as Tegmark's Mathematical universe hypothesis) to feel boring, flat, and not self-referential enough.

Logic refers to itself. This is really important.

So here's my take on the nature of existence (not fully serious! consider this a 'playful exploration'):

In the beginning, there was a contradiction.

Why was there a contradiction? The simplest way I can think of explaining it is that "non-existence exists" is a paradox (first noted by the Greek philosopher Parmenides).

And yes, these are just "cute words", but I fin... (read more)