My name is Mikhail Samin (diminutive Misha, @Mihonarium on Twitter, @misha on Telegram).
Humanity's future can be enormous and awesome; losing it would mean our lightcone (and maybe the universe) losing most of its potential value.
My research is currently focused on AI governance and improving the understanding of AI and AI risks among stakeholders. I also have takes on what seems to me to be the very obvious shallow stuff about the technical AI notkilleveryoneism; but many AI Safety researchers told me our conversations improved their understanding of the alignment problem.
I'm running two small nonprofits: AI Governance and Safety Institute and AI Safety and Governance Fund. Learn more about our results and donate: aisgf.us/fundraising
I took the Giving What We Can pledge to donate at least 10% of my income for the rest of my life or until the day I retire (why?).
In the past, I've launched the most funded crowdfunding campaign in the history of Russia (it was to print HPMOR! we printed 21 000 copies =63k books) and founded audd.io, which allowed me to donate >$100k to EA causes, including >$60k to MIRI.
[Less important: I've also started a project to translate 80,000 Hours, a career guide that helps to find a fulfilling career that does good, into Russian. The impact and the effectiveness aside, for a year, I was the head of the Russian Pastafarian Church: a movement claiming to be a parody religion, with 200 000 members in Russia at the time, trying to increase separation between religious organisations and the state. I was a political activist and a human rights advocate. I studied relevant Russian and international law and wrote appeals that won cases against the Russian government in courts; I was able to protect people from unlawful police action. I co-founded the Moscow branch of the "Vesna" democratic movement, coordinated election observers in a Moscow district, wrote dissenting opinions for members of electoral commissions, helped Navalny's Anti-Corruption Foundation, helped Telegram with internet censorship circumvention, and participated in and organized protests and campaigns. The large-scale goal was to build a civil society and turn Russia into a democracy through nonviolent resistance. This goal wasn't achieved, but some of the more local campaigns were successful. That felt important and was also mostly fun- except for being detained by the police. I think it's likely the Russian authorities would imprison me if I ever visit Russia.]
Your message:
‘Hypothetical scenario (this has not happened and details are made up):
Me and [name] are discussing the landscape of [thing] as it regards to Lightcone strategy. [name] is like "man, I feel like if I was worried that other people soon try to jump into the space, then we really should probably just back [a thing] because probably something will soon cement itself in the space". I would be like "Oh, well, I think [third party] might do stuff". Rafe is like "Oh, fuck, hmm, that's bad". I am like "Yep, seems pretty fucked. Plausibly we should really get going on writing up that 'why [third party’s person] seems like a low-integrity dude' post we've been thinking about". [name] is like "Yeah, maybe. Does really seem quite bad if [third party’s person] tries to position himself here centrally. Actually, I think maybe [name] from CEA Comm health was working on some piece about [third party’s person]? Seems like she should know [third party’s person] is moving into the space, since it seems a bit more urgent if that's happening". I am like "Yep, seems right".’
If you had asked in advance I would have rejected your request
You didn’t say that when we were talking about it! You implied that since I didn’t ask in advance, you are not bound by anything; you did mention “I can keep things confidential if you ask me in advance, but of course I wouldn't accept a request to receive private information about [third party] being sketchy that I can only use to their benefit?”
(“Being sketchy” is not how I’d describe the information. It was about an idea that Oliver is not okay with the third party working on, but is okay with others working on, because he doesn’t like the third party for a bunch of reasons and thinks it’s bad if they get more power, as per my understanding.)
I did not and would not have demanded somehow avoiding propagating the information. If you were like, “sorry, I obviously can’t actually not propagate this information in my world model and promise it won’t reflect on my plans, but I won’t actively try to use outside of coordinating with the third party and will keep it confidential going forward”, that would’ve been great and expected and okay.
I asked to not apply effort to using the information against the third party. I didn’t ask to apply effort to not be aware of the information in your decision-making, to keep separate world-models, or whatever. Confidentiality with people outside your team and not going hard on figuring out how to strategically share or use this information to cause damage to the third party’s interests would’ve been understandable and acceptable.
Yeah, no, initially, I simply asked: just in case, please don't use [the information I shared it explicitly for the purpose of enabling Oliver to coordinate with the third party] except to coordinate with the third party, expecting "sure, no problem" in response.
Then, after hearing Oliver wouldn't agree to confidentiality given that I haven't asked him for it in advance, I tried to ask: okay, sure, if you have such a high cost of/principles relating to not telling other people things, please at least don't try to tell people specifically for the purpose of harming the third party, making it a bad idea to have tried to coordinate. He then said that nope, he wouldn't agree to even that partial confidentiality, because if, e.g., someone was considering whether it's important to harm the third party now rather than later and telling them the information that I shared would've moved them towards harming the third party earlier, Oliver would want to share information with that someone so that they could harm the third party. (And also said he already told some people.)
(He ended up talking to the third party; but an opportunity to use the information adversarially did not turn up afaik.)
It's plausible that he misunderstood what I was asking for throughout, but he had no intention of avoiding making it such that me trying to coordinate with him would have been a bad idea for me.
(See also this comment.)
Yep! Even small bits of making a home smarter are great!
You can also make the light temperature natural by tracking the sun's elevation in your location.
The first two things that I did were making the lights wake me up and making them go red in the evening.
Great that you've had a positive experience!
My post is that poeple should be careful about things he's not explicitly committed to that are not about basic deontology (see the post). (And I don't feel like starting a reply with "lol no" a week later was particularly good faith; friends tell me it was rude.) Our values align a lot, and yet.
I was not confident that I would not be retaliated against (though mostly I expected that because of the social pressure, it would be very surprising if he did); but mostly, I pretty much didn't think about it and generally, I ignore the possibility of retaliation when speaking up/doing what's right.
This post focuses on communicating that! (+ being okay with hosting ai capabilities events + less important misc stuff)
(A large part of the goals of this post is to communicate to people that Oliver considers the cost of accepting information to be very high, and make people aware that they should be careful around Oliver and predict him better on this dimension, not repeating my mistake of expecting him not to do so much worse than a priest of Abadar would.)
Yep, that request would be identical, and is what I meant.
Your entire reply, after a week, was:
lol, no, that's not how telling me things works
I won't go public with it, but of course I am going to consider it in my plans in various ways
You can ask in-advance if I want to accept confidentiality on something, and I'll usually say no
Many words followed only after I expressed surprise and started the discussion ("about 3000 words of explanation, and a 1-2 hour chat conversation" is false, there were fewer than 2k words from your side in the entire conversation, many of which were about the third party, unrelated to your explanations of your decision procedures etc., a discussion of making a bet that you ended up not taking, some facts that you got wrong, etc.)
insisting again and again on a very specific interpretation of what reasonable conduct is and clarifying multiple times that yes, he wouldn't want me to use information like this under any circumstance in any kind of way adversarial to the third party the information is about
I think you're misrepresenting what I asked; I asked you to not use it adversarially towards the third party, as it seemed to me as a less strong demand than confidentiality, especially given that you already shared it with people and also said you want to be able to share information and think people like Eliezer are wrong about all the keeping-secrets stuff.
If you tell me something in secret, or ask me to put some kind of constraint on information, I will check whether I would have accepted that information with that constraint in advance. If I would have, I am happy to agree to it afterwards
Similarly, if I think you have some important preference, but you just forgot to ask me explicitly, or we didn't have time to discuss it, or it's just kind of obvious that you have this preference, I will do the same
Among the almost 2000 words, you did not describe this procedure even once.
And maybe I completely misinterpreted what you wrote, but from your messages, I had the impression of quite the opposite: that you think it is insane to expect people to use information in ways that align with important preferences.
I think characterizing that as "Oli is a bad counterparty" instead of something like "Oli doesn't follow glowfic!lawful deity norms" or "I regret having Oli as a counterparty" is impolite
I see your point and initially agreed that "Oliver is a bad counterparty" is indeed not polite and intended to change that, but saw that I actually wrote "Oliver is not a good counterparty".
That was produced by "Oliver is the kind of counterparty you might regret having dealt with, as I have".
It's less of an impolite judgement than "Oliver is a bad counterparty", but if you think it reads the same, I'll try to change that to be more polite while still expressing that I think it often makes sense for people to be careful around him.
This would’ve worked!
(Other branches seem less productive to reply to, given this.)