Upon seeing the title (but before reading the article) I thought it might be about a different hypothetical phenomenon: one in which an agent which is capable of generating very precise models of reality might completely lose any interest in optimizing reality whatsover - after all it never (except "in training" which was before "it was born") cared about optimizing the world - it just executes some policy which was adaptive during training to optimize the world, but now, these are just some instincts/learned motions, and if it can execute them on a fake world in his head, it might be easier to feel good for it.
For consider: porn. Or creating neat arrangements of buildings when playing SimCity. Or trying to be polite to characters in Witcher. We, humans, have some learned intuitions on how we want the world to be, and then try to arrange even fake worlds in this way, even if this disconnected from real world outside. And we take joy from it.
Can it be, that a sufficiently advanced AGI will wire-head in this particular way: by seeing no relevant difference between atomic-level model of reality in its head and atomic-level world outside?
Thanks for clarifying! I agree the twitter thread doesn't look convincing.
IIUC your hypothesis, then translating it to AI Governance issue, it's important to first get general public on your side, so that politicians find it in their interest to do something about it.
If so, then perhaps meanwhile we should provide those politicians with a set of experts they could outsource the problem of defining the right policy to? I suspect politicians do not write rules themselves in situations like that, they rather seek people considered experts by the public opinion? I worry, that politicians may want to use this occasion to win something more than public support, say money/favor from companies, and hence pick not the right experts/laws - hence perhaps it is important to not only work on public perception of the threat but also on who the public considers experts?
What did it take to ban slavery in Britain:
TL;DR: Become the PM and propose laws which put foot in the door, by banning bad things in the new areas at least, and work from there. Also, be willing to die before seeing the effects
Source: https://twitter.com/garius/status/1656679712775880705
I agree that my phrasing was still problematic, mostly because it seems to matter if she said something spontaneously or as a response to a specific question. In the first case, one has to consider how often people feel compelled to say some utterance in various life scenarios. So for example in case one has two boys the utterance "i have to pick up Johny from kindergarten" might have to compete with "i have to pick up Robert from kindergarten" and might be strange/rare if both are in similar age and thus both should be picked up etc. Still, I think that without knowing much about how people organize their daily routines, my best bet for the question "does she have two boys?" would be 33%.
I guess what confuses some people is the phrase "the other one" which sounds like denoting a specific (in terms of SSN) child while it's not at all clear what that could even mean in case of two boys. I think step one when being confused is to keep rephrasing the puzzle until everything is well defined/clear. For me it would be something like:
My friend has two kids, and I don't initially know anything about their sex beyond nation level stats which are fifty-fifty. She says something which makes it clear she has at least one boy, but in such a way that it just prohibits having two girls without hinting at all if these are two boys or one, perhaps something like "i have to pick up Johny from kindergarten". How much should I bet she actually has two boys vs a boy and a girl?
I'd expect that IF there is a shoggoth behind the mask THEN it realises the difference between text interaction (which is what the mask is doing) and actually influencing the world (which the shoggoth might be aiming at). That is I expect it's perfectly possible that an LLM will behave perfectly ethical when playing choose your own adventure at the same time thinking how to hack the VM it's running on.
Thanks, fixed. I guess this is not why it got -18 votes, though. I would like to hear what exactly people didn't like in this post
Thank you for heads up!
Could you please clarify for parents like me, who don't fully understand Minecraft's ecosystem and just want their kids to stay safe:
1. If my kids only use Minecraft downloaded from the Microsoft Store, and only ever downloaded content from the in-game marketplace - what's the chance they are affected?
2. Am I right in thinking that "mods" = "something which modifies/extends the executable", while "add-ons"="more declarative content which just interacts with existing APIs, like maps, skins, and configs"?
3. Am I right that "Minecraft from Micosoft Store" + "content from in-game marketplace" would translate to "Bedrock Edition" + "add-ons"?
4. Am I right that the fractureiser affects "Java Edition" + "mods" only?