I guess "backdoor" suggests access being exclusive to person who planted it, while "vulnerability" is something exploitable by everyone? Also, after thinking a bit more about it, I think you're right that "backdoor" implies some intentionality, and perhaps accidental backdoor is an oxymoron.
There is such thing as accidental backdoor: not properly escaping strings embeded in other strings, like SQL injection, or prompt injection
Either it should be 15% instead of 11% or I need some more explanation
So the 13% could be an 11% decline in some areas and a 2% increase in other larger areas, where they cancel out
I think the framework from "Dictator's Handbook" can be applied: citizens get as much freedom an benefits as is (short-term) optimal for the rulers. For example, if a country needs skilled labor and transportation to create tax revenue, then you can predict the govt will fund schools, roads and maybe even hospitals. OTOH if the country has rich deposits of gold located near the ports, then there's no need for any of that.
Since reading this book I am also very worried by scenarios of human disempowerment. I've tried to ask some questions around it:
- Can homo-sapiens sustain an economy parallel to AIs?
- How politics interacts with AI? (for some reason: negative 18 votes)
I wonder if this is somehow harder to understand for citizens of USA, than for someone from a country which didn't care about its citizens at all. For example, after Lukashenko was "elected" in Belarus, people went to the streets to protest, yet, this didn't make any impression on the rulers. They didn't have any bargaining power, it seems.
Cool puzzle. (I've wrote like 4 versions of this comment each time changing explanation and conclusions and each time realizing I am still confused).
Now, I think the problem is that we don't pay much attention to:
What should one do when one has drawn a red ball?
(Yeah, I strategically use word "one" instead of "I" to sneak assumption that everyone should do the same thing)
I know, it sounds like an odd question, because, the way the puzzle is talked about, I have no agency when I got a red ball, and I can only wait in despair as the owners of green balls make their moves.
And if you imagine a big 2-dimensional array where each of 100 columns is an iteration of a game, and each of 20 rows is a player, and look at an individual row (a player) then, we'd expect, say 50 columns to be "mostly green", of them roughly 45 have the player "has drawn green" cell, and 50 columns to be "mostly red", with 5 of them having "has drawn green". If you focus just on those 45+5 columns, and note that 45:5 is 0.9:0.1, then yeah, indeed the chance that the column is "mostly green" given "I have drawn green" is 0.9.
AND coincidentally, if you only focus on those 45+5 columns, it looks like to optimize the collective total score limited to those 45+5 columns, the winning move is to take the bet, because then you'll get 0.9*12-0.1*52 dollars.
But what about the other 50 columns??
What about the rounds in which that player has chosen "red"?
Turns out they are mostly negative. So negative, that it overwhelms the gains of the 45+5 columns.
So, the problem is that when thinking about the move in the game, we should not think about
1. "What is the chance one is in mostly green column if one has a green ball?" (to which the answer is 90%)
but rather:
2. "What move should one take to maximize overall payout when one has a green ball?" (to which the answer is: pass)
and that second question is very different from:
3. "What move should one take to maximize payout limited just to the columns in which they drew a green ball when seeing a green ball?" (to which the answer is: take the bet!)
The 3. question even though it sounds very verbose (and thus weird) is actually the one which was mentally substituted (by me, and I think most people who see the paradox?) naturally when thinking about the puzzle, and this is what leads to paradox.
The (iterated) game has 45+5+50 columns, not just 45+5, and your strategy affects all of them, not just the 45+5 where you are active.
How can that be? Well, I am not good at arguing this part, but to me it feels natural, that if rational people are facing same optimization problem, they should end up with same strategy, so whatever I end up doing I should expect that others will end up doing it too, so I should take that into account when thinking what to do.
It still feels feel a bit strange to me mathematically, that a solution which seems to be optimal for 20 various different subsets (each having 45+5 columns) of 100 columns individually, is somehow not optimal for the whole 100 columns.
The intuition for why it is possible is that a column which has 18 green fields in it, will be included in 18 sums, and a column which has just 2 green fields in it will be counted in just 2 of them, so this optimization process, focuses too much on the "mostly green" columns, and neglects those "mostly red".
Is it inconsistent to at the same time think:
"The urn is mostly green with ppb 90%" and
"People who think urn is mostly green with ppb 90% should still refuse the bet which pays $12 vs $-52"?
It certainly sounds inconsistent, but what about this pair of statements in which I've only changed the first one:
"The urn is mostly green with ppb 10%" and
"People who think urn is mostly green with ppb 90% should still refuse the bet which pays $12 vs $-52?"
Hm, now it doesn't sound so crazy, at least to me.
And this is something a person who has drawn a red ball could think.
So, I think the mental monologue of someone who drew a green ball should be:
"Yes, I think that the urn is mostly green with ppb 90%, by which I mean, that if I had to pay -lg(p) Bayes points when it turns out to be mostly green, and -lg(1-p) if it isn't, then I'd choose p=0.9. Like, really, if there is a parallel game with such a rules, I should play p=0.9 in it. But still, in this original puzzle game, I should pass, because whatever I'll do now, is whatever people will tend to do in cases like this, and I strongly believe that "People who think urn is mostly green with ppb 90% should still refuse the bet which pays $12 vs $-52", because I can see how this strategy optimizes the payoff in all 100 columns, as opposed to just those 5+45 I am active in. The game in the puzzle doesn't ask me what I think the urn contained, nor for a move which optimizes the payoff limited to the rounds in which I am active. The game asks me: what should be the output of this decisions process so that the sum over all 100 columns is the largest. To which the answer is: pass".
How to combine this with the fact that "the nudge" apparently doesn't work https://phys.org/news/2022-08-nudge-theory-doesnt-evidence-future.html ?
1≈ 41,000
Why not ?
The way I understood the story, to define a function on two numbers from I need to fill-in a table with 59*59 cells, by picking for each cell a number from . If 20% of it is still to be filled, then there are 0.2*59*59 decisions to be made, each with 59 possibilities.
Right?
Thank you for heads up!
Could you please clarify for parents like me, who don't fully understand Minecraft's ecosystem and just want their kids to stay safe:
1. If my kids only use Minecraft downloaded from the Microsoft Store, and only ever downloaded content from the in-game marketplace - what's the chance they are affected?
2. Am I right in thinking that "mods" = "something which modifies/extends the executable", while "add-ons"="more declarative content which just interacts with existing APIs, like maps, skins, and configs"?
3. Am I right that "Minecraft from Micosoft Store" + "content from in-game marketplace" would translate to "Bedrock Edition" + "add-ons"?
4. Am I right that the fractureiser affects "Java Edition" + "mods" only?
Upon seeing the title (but before reading the article) I thought it might be about a different hypothetical phenomenon: one in which an agent which is capable of generating very precise models of reality might completely lose any interest in optimizing reality whatsover - after all it never (except "in training" which was before "it was born") cared about optimizing the world - it just executes some policy which was adaptive during training to optimize the world, but now, these are just some instincts/learned motions, and if it can execute them on a fake world in his head, it might be easier to feel good for it.
For consider: porn. Or creating neat arrangements of buildings when playing SimCity. Or trying to be polite to characters in Witcher. We, humans, have some learned intuitions on how we want the world to be, and then try to arrange even fake worlds in this way, even if this disconnected from real world outside. And we take joy from it.
Can it be, that a sufficiently advanced AGI will wire-head in this particular way: by seeing no relevant difference between atomic-level model of reality in its head and atomic-level world outside?
I am not sure if immutability or optimality is the main thing behind people giving you advice in these cases. It could be that: