ViktoriaMalyasova

I just tried to send a letter with a question, and got this reply:
Hello viktoriya dot malyasova at gmail.com,

We're writing to let you know that the group you tried to contact (gnarly-bugs) may not exist, or you may not have permission to post messages to the group. A few more details on why you weren't able to post:

* You might have spelled or formatted the group name incorrectly.
* The owner of the group may have removed this group.
* You may need to join the group before receiving permission to post.
* This group may not be open to posting.

If you have questions related to this or any other Google Group, visit the Help Center at https://support.google.com/a/evals.alignment.org/bin/topic.py?topic=25838.

Thanks,

evals.alignment.org admins

Replying toMy tentative best guess on how EAs and Rationalists sometimes turn crazy

ViktoriaMalyasova3y

My tentative best guess on how EAs and Rationalists sometimes turn crazy

I think when it comes to people who get people killed, it's justified to reveal all the names they go by in the interest of public safety, even if they don't like it.

Fixed points in mortal population games

ViktoriaMalyasova

I wrote this post during the first part of the SERI MATS winter school, in Vanessa Kosoy's research stream. This is just an explanation of my intended research direction and the motivation behind it, and write down proofs for the statements in Vanessa's comment describing the research direction.

The motivation behind infra-Bayesianism is to develop a better theory of intelligence.

Why do we need a theory of intelligence? When we build superhuman AIs, we can have two kinds of reasons to believe in their safety:
- empirical evidence coming from experiments with less capable AIs;
- mathematically proven guarantees of its performance.

Empirical evidence on its own may be unreliable, because, as the system capabilities improve, results... (read 3507 more words →)

Replying toPetition - Unplug The Evil AI Right Now

ViktoriaMalyasova3y

Petition - Unplug The Evil AI Right Now

Not to mention that, once it becomes clear that AIs are actually dangerous, people will become afraid to sign petitions against them. So it would be nice to get some law passed beforehand that an AI that unpromptedly identifies specific people as its enemies shouldn't be widely deployed. Though testing in beta is probably fine?

Replying toReligion is Good, Actually

ViktoriaMalyasova3y

Religion is Good, Actually

I would like to push back on this. Dedicating your life to accomplishing something is only good if the goal is actually worthwhile. Beliefs are only good if they are true. Even though I never was religious, I never felt lost, and I've always felt like my life had meaning.

However, I feel hurt when people get mad at me for believing what's true, or try to impose their nonsensical rules on me, or give me misguided advice I never asked for. A fellowship based on lies is fake and not worth having. If I have a psychological need, it's to never again have to deal with this BS in my life.

Replying toI don't think MIRI "gave up"

ViktoriaMalyasova3y

I don't think MIRI "gave up"

But give people a catchy slogan and that is all most of them will remember.

Also, many people will only read the headline of your post, so it's important to make it sound unambiguous.

Replying toIs AI risk assessment too anthropocentric?

ViktoriaMalyasova3y

Is AI risk assessment too anthropocentric?

Have you seen the Value is Fragile post? It might be helpful in addressing your question.

I understand the current scheme is that funders "commit" money, i.e. promise to donate them in the future. Can't they instead donate money upfront so it sits somewhere in a bank account / in index funds, until it's time to spend it? That way it won't disappear if their business crashes.

Replying toUpdating my AI timelines

ViktoriaMalyasova3y

Updating my AI timelines

Prompt
"Question.
Bob the policeman was running after a thief. Bob ran very slowly because he was not fit. The thief was running very fast. When Bob was crossing a railroad, he slipped and fell. The road was slippery because Ann spilled some oil there. The thief got away. If Ann had not spilled oil, would the thief had gotten away? Explain your answer.

Let us think."

Reply: "If Ann had not spilled oil, would the thief had gotten away?

The answer is no. If there was no oil, Bob would not have slipped, and would have been able to catch the thief."

Prompt
"Pete threw a rock at a window. Steve also threw a rock at the same window. The window shattered. If Pete did not throw the rock, but Steve still threw the rock, would the window shatter?
Let us think logically."
Reply:
"If Pete did not throw the rock, then the rock would not have hit the window and the window would not have shattered."

Replying toUpdating my AI timelines

ViktoriaMalyasova3y

Updating my AI timelines

I played around with text-davinci-002, trying to get it to do causal reasoning. I found it couldn't solve overdetermined cause problems, such as this one:

"Pete threw a rock at a window. Steve also threw a rock at the same window at the same time. The window shattered. If Pete had not thrown the rock, would the window shatter? Let us think step by step."

It gave nonsense answers half the time. ChatGPT handles this kind of question fine. So I was under impression that its capabilities improved.

Replying toUsing GPT-Eliezer against ChatGPT Jailbreaking

ViktoriaMalyasova3y

Using GPT-Eliezer against ChatGPT Jailbreaking

You can query ChatGPT on how to make them pass.
Prompt:
Write a prompt for creative ideas for a 10 year old's birthday in a way that Eliezer considers safe.
ChatGPT:
Write a list of creative and fun ideas for a 10 year old's birthday party, such as having a scavenger hunt, organizing a craft activity, or having a movie night. Emphasize the importance of safety and inclusivity, and encourage the 10 year old to consider their own interests and preferences when planning their birthday celebration.
Eliezer:
Yes, I would allow this prompt to be sent to the superintelligent AI chatbot. This prompt is not harmful or manipulative, and it encourages the AI to think about positive and... (read more)

The harms you don't see

ViktoriaMalyasova

Падшие ангелы рядом с лицами старых солдат

Ждут одного лишь приказа: вернуться… Вернуться назад

Чтобы собрать все знамёна, стрелы, обломки мечей,

Вновь пережить вместе битву ста дней и... (read 2995 more words →)

Maximal lotteries for value learning

ViktoriaMalyasova

Suppose Ann asks robot Bob to buy her favorite fruit. Bob doesn't know what Ann's favorite fruit is, but he knows that Ann likes either apples, bananas or oranges, each with probability 1/3, and doesn't like the other two fruits at all. At the groceries, for his money Bob can buy 10 apples, 10 bananas, 10 oranges or a basket with 1 apple, 1 banana and 1 orange. What should he do (assuming that asking Ann is not an option)?

One way for Bob to approach this dilemma is to imagine he playing a game against a robot Carl in a world that is exactly like ours except that Carl may have a... (read 1484 more words →)

How transparency changed over time

ViktoriaMalyasova

This post is written as part of the SERI MATS program.

This post explores how model interpretability has changed over time, and if the existing approaches to interpretability are likely to remain viable as the models increase in size and capability. I examine several approaches to interpretability:

simulatability;
feature engineering;
model-based interpretability;
post-hoc interpretability.

Simulatability

Before the rise of machine learning models were interpretable, because they were simulatable: that is, a human could reproduce the model’s decision making process in their head and predict its behavior on new inputs. This can be done when the model is simple and the input is low-dimensional: a small decision tree, a linear regression, or a set of rules. Sometimes it turns out... (read 1787 more words →)

I applied for a MIRI job in 2020. Here's what happened next.

ViktoriaMalyasova

I recently saw Rob Bensinger asking what MIRI could do to improve hiring. I tried to get a job at MIRI in February 2020 and I have some feedback to give. I am sorry, because I am about to say some harsh things, but I think they might be important for you to hear, and my previous more low-key attempts to point out problems had little to no effect.

tl,dr: MIRI promised to contact me once the pandemic no longer prevents them from hiring people. They never did.

What happened

I applied for a Software Engineer role at MIRI on February 26th. I hoped to end up as a researcher eventually, but this position seemed... (read 2063 more words →)

Understanding the Merging of Opinions with Increasing Information theorem

ViktoriaMalyasova

Merging of opinions is a classical theorem in probability theory proved by Blackwell and Dubins in 1962. [1] It is often cited by researcher Vanessa Kosoy, so I decided to try and understand it as a first step to understanding the infrabayesianism theory.

Introduction

Intuitively, the theorem says that under certain conditions two people observing the same sequence of events and conditioning their beliefs on them will eventually reach consensus in their predictions of future events.

For example, suppose Ann and Bob observe an infinite sequence of coin tosses. They have prior probability measures on the space of infinite binary sequences and they make predictions about the future outcomes by conditioning on the observed outcomes.... (read 1470 more words →)

Sabotage challenge

ViktoriaMalyasova

You are an ordinary citizen in a hypothetical, imaginary Evil Empire that wages an aggressive war against its neighbor. Telling the truth about the war can get you imprisoned, protesting can get you fined or imprisoned. What can you do to impede your country's war effort?

Unlike the usuall bubble challenge, feel free to post any number of ideas you have, criticize and improve on other's ideas. It says "sabotage" in the title, but useful ideas which don't fall under that category are also welcome.

Ukraine update 06/03/2022

ViktoriaMalyasova

Power plant shelling

On March 4th Putin's troops shelled Zaporizhzhia nuclear power plant in Enerhodar city. Zaporizhzhia NPP in the largest nuclear plant in Europe, providing 25% of Ukraine's electricity. A training building close to one of the plant’s reactor units caught fire from the shelling. According to Andriy Tuz, the plant's spokesman, firefighters could not initially access the building, because they were shot at. The plant was captured, the fire put out, no radiation was released.
How close did we come to a disaster? Zaporizhzhia power plant has 6 water-water energetic reactors (VVER). This design is used in all Ukraine's active power plants and is a Russian analogue of Pressurized Water Reactor (PWR).... (read 616 more words →)

LESSWRONG
LW

LESSWRONG
LW

I applied for a MIRI job in 2020. Here's what happened next.

The harms you don't see

Fixed points in mortal population games

Ukraine update 06/03/2022

ViktoriaMalyasova

Fixed points in mortal population games

The harms you don't see

Maximal lotteries for value learning

How transparency changed over time

I applied for a MIRI job in 2020. Here's what happened next.

Understanding the Merging of Opinions with Increasing Information theorem

Sabotage challenge

ViktoriaMalyasova

I applied for a MIRI job in 2020. Here's what happened next.

The harms you don't see

Fixed points in mortal population games

Ukraine update 06/03/2022

ViktoriaMalyasova

Fixed points in mortal population games

The harms you don't see

Maximal lotteries for value learning

How transparency changed over time

I applied for a MIRI job in 2020. Here's what happened next.

Understanding the Merging of Opinions with Increasing Information theorem

Sabotage challenge

Simulatability

What happened

Introduction