My name is Mikhail Samin (diminutive Misha, @Mihonarium on Twitter, @misha on Telegram).
Humanity's future can be enormous and awesome; losing it would mean our lightcone (and maybe the universe) losing most of its potential value.
My research is currently focused on AI governance and improving the understanding of AI and AI risks among stakeholders. I also have takes on what seems to me to be the very obvious shallow stuff about the technical AI notkilleveryoneism; but many AI Safety researchers told me our conversations improved their understanding of the alignment problem.
I believe a capacity for global regulation is necessary to mitigate the risks posed by future general AI systems. I'm happy to talk to policymakers and researchers about ensuring AI benefits society.
I took the Giving What We Can pledge to donate at least 10% of my income for the rest of my life or until the day I retire (why?).
In the past, I've launched the most funded crowdfunding campaign in the history of Russia (it was to print HPMOR! we printed 21 000 copies =63k books) and founded audd.io, which allowed me to donate >$100k to EA causes, including >$60k to MIRI.
[Less important: I've also started a project to translate 80,000 Hours, a career guide that helps to find a fulfilling career that does good, into Russian. The impact and the effectiveness aside, for a year, I was the head of the Russian Pastafarian Church: a movement claiming to be a parody religion, with 200 000 members in Russia at the time, trying to increase separation between religious organisations and the state. I was a political activist and a human rights advocate. I studied relevant Russian and international law and wrote appeals that won cases against the Russian government in courts; I was able to protect people from unlawful police action. I co-founded the Moscow branch of the "Vesna" democratic movement, coordinated election observers in a Moscow district, wrote dissenting opinions for members of electoral commissions, helped Navalny's Anti-Corruption Foundation, helped Telegram with internet censorship circumvention, and participated in and organized protests and campaigns. The large-scale goal was to build a civil society and turn Russia into a democracy through nonviolent resistance. This goal wasn't achieved, but some of the more local campaigns were successful. That felt important and was also mostly fun- except for being detained by the police. I think it's likely the Russian authorities would imprison me if I ever visit Russia.]
Posts
Wikitag Contributions
Do you want to make Claude make an app that would do the thing? Might be pretty valuable if it works.
Do you think with the right prompt, current LLMs could do what practitioners do?
(btw i expect we’ll really want enhanced humans to have the capacity to suffer, because we have preferences around future people being able to experience the kinds of feelings we experience when we read stories, including very sad stories. Some suffering is reflectively endorsed and we enjoy it/wouldn’t want it to not happen; and it seems fine to want new humans and enhanced current humans to also have it, although maybe with more access to some control over it.)
a large fraction of the AI safety and alignment community endorses humanity being replaced by a sufficiently aligned and advanced AI, and would prefer this to a future where our actual descendants spread over the planets, albeit at a slower pace and with fewer total objective "utils"
I don’t think this is true.
To the extent we, on reflection, have preferences about discovering, solving, colonizing the universe ourselves—solving physics, building technology, etc.—a CEV-aligned superintelligence just refuses to help us with those things! It will stop deaths immediately, and it will probably send probes to distant galaxies immediately (on average, our future lightcone loses approximately one galaxy every year), but then it will probably leave us reflect and do stuff, because this is what’s fun.
“Utils” are not some objective thing. It is what we want. To the extent we do want to do meaningful, cool, and interesting stuff, this is what “utils” end up being mainly about.
A sufficiently aligned and advanced AI likely prevents loss of huge amounts of value and then approximately turns off.
Great video! Amazing job, thank you!
No one believes the Jesus market will resolve to Yes. See https://www.lesswrong.com/posts/LBC2TnHK8cZAimdWF/will-jesus-christ-return-in-an-election-year for an explanation of what people are betting on
1
1- Humans doing random stuff doesn’t provide a lot of evidence for how common it is in the universe, especially if humans do this to fool a superintelligence instead of because they actually believe it. (humans don’t have strong preferences around how much reality fluid they have and so them being nice to others doesn’t update ASI a lot.) There’s no reason for a normal powerful civilization that doesn’t have the alignment problem unsolved to develop karma tests like that.
- We probably won’t at any point have mech interp at the level that allows us to trick all parts of a superintelligence’s cognition sufficiently well.
- (I do not understand the idea of kingmaker logic. Is it to edit the understanding of the world to make it convinced that there’s a part of logic like that?)
- Some of this post can be adjusted to be the canonical description of one of ways Omega might determine whether to give money in logical counterfactual mugging.
1(MIRI did some work on logical induction.)
I’ll give the post a read!
The US version is currently #396 in Books on .co.uk