Matthew Farrugia-Roberts

LESSWRONG
LW

Matthew Farrugia-Roberts — LessWrong

Replying toKessler's Second Syndrome

Matthew Farrugia-Roberts1y

Kessler's Second Syndrome

Seems worth it 👾

Replying toThe Queen’s Dilemma: A Paradox of Control

Matthew Farrugia-Roberts1y

The Queen’s Dilemma: A Paradox of Control

W. Ross Ashby's Law of Requisite Variety (1956) suggests fundamental limits to human control over more capable systems.

This law sounds super enticing and I want to understand it more. Could you spell out how the law suggests this?

I did a quick search of LessWrong and Wikipedia regarding this law.

"... Ashby's "Law of requisite variety", which roughly speaking states that a system can only remain in homeostasis if it has more internal states than the external states it encounters." from Yuxi_Liu, "Cybernetic dreams".
"Either the AI is too simple to be an independent robust agent in human society, or it needs to be approximately as complex as humans themselves. Cf. the law of requisite variety."

Matthew Farrugia-Roberts1y*

The Queen’s Dilemma: A Paradox of Control

I like this analogy, but there are a couple of features that I think make it hard to think about:

1. The human wants to play, not just to win. You stipulated that "the human aims to win, and instructs their AI teammate to prioritise winning above all else". The dilemma then arises because the aim to win cuts against the human having agency and control. Your takeaway is "Even perfectly aligned systems, genuinely pursuing human goals, might naturally evolve to restrict human agency."

So in this analogy, it seems that "winning" stands for the human's true goals. But (as you acknowledge) it seems like the human doesn't just want to win, but actually... (read 517 more words →)

Replying toJesse Hoogland on Developmental Interpretability and Singular Learning Theory

Matthew Farrugia-Roberts3y

Jesse Hoogland on Developmental Interpretability and Singular Learning Theory

There is a typo in the transcript. The name of the creator of singular learning theory is "Sumio Watanabe" rather than "Sumio Aranabe".

Replying toDSLT 1. The RLCT Measures the Effective Dimension of Neural Networks

Matthew Farrugia-Roberts3y

DSLT 1. The RLCT Measures the Effective Dimension of Neural Networks

I think these are helpful clarifying questions and comments from Leon. I saw Liam's response. I can add to some of Liam's answers about some of the definitions of singular models and singularities.

1. Conditions of regularity: Identifiability vs. regular Fisher information matrix

Liam: A regular statistical model class is one which is identifiable (so $p (y | x, w_{1}) = p (y | x, w_{2})$ implies that $w_{1} = w_{2}$ ), and has positive definite Fisher information matrix $I (w)$ for all $w \in W$ .
Leon: The rest of the article seems to mainly focus on the case of the Fisher information matrix. In particular, you didn't show an example of a non-regular model where the Fisher information matrix is positive definite everywhere.
Is it correct to assume models which are merely non-regular because the map from

... (read 939 more words →)