Sven Nilsen — LessWrong

LESSWRONG
LW

Safe AGI Complexity: Guessing a Higher-Order Algebraic Number

Introduction: A rational number can represent an arbitrary sequence of bits where bit patterns either terminate or repeat infinitely. This provides a mathematical toy model for reasoning about AGI safety. A bit sequence can be translated into a sequence of actions. The goal is to prove safety of some bit sequence without bounds. While this toy model does not allow input to the AGI, it offers useful insights into the complexity of safe AGI in the case where the entire deterministic world state and human values have been internalized.

I have been thinking about numbers of the form:

(a / b)^(1 / c)

Where a, b, c are natural numbers. In number theory, this... (read 475 more words →)

-2

Repeated Play of Imperfect Newcomb's Paradox in Infra-Bayesian Physicalism

Sven Nilsen

Yesterday, I watched the video series about Infra-Bayesianism. Link:

Some of it went over my head, so I tried to come up with a thought experiment to test my current level of understanding:

Imagine that you are playing a repeated game similar to Newcomb's paradox, except that 90% of the time you play against a perfect super-predictor and 10% of the time you play against someone who pretends to be a perfect super-predictor.

In Infra-Bayesianism, there is a convex probability distribution over environments.
In Infra-Bayesian Physicalism, there is a probability distribution over computational evidence, e.g. knowledge about mathematical abstractions. There is also a probability distribution over models of the physical world.

It is difficult to predict an... (read 328 more words →)

The Plan: Put ChatGPT in Charge

Sven Nilsen

Problem:

1. AI is becoming too smart to control
2. Sooner or later, we will have an AI takeover

Solution:

Put an AI that is too stupid to take over the world by itself, but trained on ethical data, in charge to control the AI takeover safely.

If you ask ChatGPT ethical questions to be answered "Yes", "No" or "STOP" if the question is too complex, then you will find that it is more ethical than many world leaders today.

The obvious solution is to put world leaders below ChatGPT in political power. Develop a safe protocol for asking ChatGPT ethical questions and make it possible to verify the answer.

Discussion?

-5

Harry Potter in The World of Path Semantics

Sven Nilsen

This is a short story written with help from ChatGPT, about the intuition behind counter-examples to Leibniz' first principle. This is generally believe to be true among logicians, but the equality as it is expressed in Leibniz' first principle is just an assumption about equality and not a proof of equality.

x=y → ∀F(Fx ↔ Fy) Leibniz' first principle

In short, Leibniz' first principle does not hold for all operators in mathematics, because not all operators are congruent by normal equality.

It is not possible to reason about proofs of equality directly in logic (this is not about terms of types as proofs, but about "actual proofs"). There is a weaker statement... (read 197 more words →)

-3

Bridging Two Esoteric 4-Value Logics Using Quantum Physics: Answered Modal Logic vs Catuṣkoṭi

Sven Nilsen

Paper "Answered Modal Logic Catuṣkoṭi": https://github.com/advancedresearch/path_semantics/blob/master/papers-wip/answered-modal-logic-catuskoti.pdf

In the classical world of information theory, there are only bits: Yes/No, True/False, On/Off, 0/1.

This system is so convenient that it is hard for our brains to adapt to other logical systems. However, does it really matter? Can not all logical systems be reduced to bits (or propositions as they are called in Logic)?

Indeed, this correct. The challenge with what we call "Esoteric Logics", is how operations are defined, not to re-invent bits.

Answered Modal Logic was an attempt to find a logic with intrinsic path semantical properties, but was unsuccessful and replaced by Path Semantical Logic. However, recently I and Daniel Fischer revisited Answered Modal Logic to... (read 658 more words →)

AI Alignment Using Reverse Simulation

Sven Nilsen

A simulation is often used to describe a computer program imitating a physical process. In this post, I use the term "reverse simulation" to refer to a scenario when a computer program is simulated using physics.

The approach I am suggesting here, is meant to address one problem of AI alignment that belongs to the philosophy of mind:

How can we prevent AGI from accidentally causing suffering of minds, when we do not yet know an objective theory of minds?

It is not evident to me that suffering of minds can be assigned a utility. Of course, assuming one could assign a utility, there might be a correct decision theory. However, if suffering of minds... (read more)