All of wizzwizz4's Comments + Replies

I don't quite get the argument here; doesn't anthropic shadow imply we have nothing to worry about (except for maybe hyperexistential risks) since we're guaranteed to be living in a timeline where humanity survives in the end?

But it doesn't say we're guaranteed not to be living in a timeline where humanity doesn't survive.

If I had a universe copying machine and a doomsday machine, pressed the “universe copy” button 1000 times (for 2¹⁰⁰⁰ universes), then smashed relativistic meteors into Earth in all but one of them… would you call that an ethical issue? I ... (read more)

I can't remember pain, in much the same way. Perhaps extreme depression “counts as” mental pain enough to trigger this effect?

Write “half”, or (if you're feeling pedantic), “~half”.

Bayesian inference only functions within known solution-space. Spotting things outside of known solution space, while rare, is essential for the progression of science – and can't be modelled simply as Bayesian inference.

Did anyone point out that the 3DS has a camera title?

Communication transfers ideas from one person to another. If technically correct communication transfers false ideas, it is deception. Accurate communication transfers correct ideas with high fidelity, which isn't necessarily equivalent to technically correct communication.

but it's hard to hear the story and imagine that that grandpa is an old timey european, talking about good wolves.

That is what I thought. Not "old timey" per se, but modern grandparent age. The story feels five to ten years old, to me.

or so as not to confuse the public with changed numbers

If you're withholding knowledge to avoid confusing people, chances are that your withholding is the primary source of confusion. Just say "new estimates" or "revised estimates" – job done.

What could survive is a propensity to become the sort of person to sacrifice yourself to protect your family. given that no other family member has done so. Or, a propensity to sacrifice yourself that would normally kick in after you've had kids. But actually sacrificing yourself before you pass on your genes is a textbook example of "selected against".

I'm not sure this is true. I don't think people have children out of a conscious desire to "pass on their genes." I am a parent and have never experienced this, nor have I ever heard of anyone framing their desire to become a parent in this way. This may be what Nature has hard-wired us to do, but I don't think "passing on one's genes" is necessarily the end-goal in that regard, either. I think the objective is to produce offspring, and then see to it that those offspring survive. In which case dying would be absurdly counterproductive. I think, first of all, people are intrinsically motivated to have sex, which naturally results in children at least historically, prior to the invention of birth control -- which, it's worth noting, humans tried unsuccessfully to invent for thousands of years before we finally got it right, if that tells you anything. I do think there is a genuine desire to procreate and raise children, but interestingly, now that we have come up with a way to avoid parenting without having to avoid sex, we have found that the desire to procreate is completely absent in many people -- a surprising number of people, even. Perhaps the expectation that most adults will eventually become parents is merely reflective of the situation pre-birth control, which in relative terms is still a brand new medical innovation, and not something which our social norms have completely adjusted around yet. This makes me wonder, tangentially, if one's desire to parent children may be socially imposed to a significant degree. By contrast, very few people intentionally avoid sex all their lives. Bottom line: it seems obviously false to me to claim that "the propensity to sacrifice one's self would normally kick in only after becoming a parent." I think the opposite is actually true. Barring situations where someone is actively trying to harm one's child, where self sacrifice may be necessary in order to preserve the child's life, I think you'll find that most people

A second, detailed reading might make it seem like this comment's has an error. However, the reasoning is sound; "you said the coin was heads" doesn't distinguish very well between "the coin was heads" and "the coin was tails but you lied about the bet", so doesn't provide much evidence.

Likewise, the dismissing of hearsay appears to be an error, but remember that humans have finite computational power. If you take into account (at least) the hypothesis that somebody's trying to deceive you about reality, you effectively end up dismissing the evidence anywa

... (read more)

I had to think about this for quite a while before I could refute it. Well done.

If you happen to have evolved a cognitive architecture that permits storing information about the state of the world in the same format as information about how to build new members of the species, transferring that information would grant an evolutionary advantage over not. The only "just so" assumption is in such a cognitive architecture having developed, but they're allowed that assumption given that the Super Happies already exist.

Yes, it was something Yudkowsky added. But the text doesn't imply ghosts aren't "really people"; it just states that they're read-only human simulations of unknown fidelity, and the characters are chauvinistic about that.

Oh. Yeah, that seems obvious now, thanks for pointing that out.
Implying that ghosts aren't really people

Or were just exempted from protection from the Interdict of Merlin, like books.

I was under the impression that the Interdict was something Yudkowsky added. It's been a long, long time since I read the originals, but this stackexchange [] post has a bunch of people saying the Interdict is a HPMOR-exclusive. There's also a counterexample: in canon, Harry learns sectumsempra by reading it out of Snape's old potion book.

You should also take into account what this signals to people who know you've had a nose job (e.g. vanity).

1snog toddgrass3y
Good point. It would be slightly more valuable when moving cities.

They look like it, but its some sort of emergent behaviour,

I agree with this assessment. It almost feels like a hive mind; I've dipped into the peripherals of online mobs before, and have felt "hey, this action is a good idea" thoughts enter my head unbidden. I'd probably participate in such things often, if I didn't have a set of heuristics that (coincidentally) cancels out most of this effect, and a desire not to associate with the sorts of people who form mobs.

If the barrier-to-entry is increased to "requires two minutes of unrewarded drudgery, where it

... (read more)

How can a swarm of nuclear asbestos superintelligent nanobots be synthesised using common household items? (The rhetoric in the answer will keep your guard down for just long enough to publish it.)

The free will comes from this other realm.

And what if we end up being able to predict what goes on in this "other realm"? This just pushes the problem back behind the curtain, instead of confronting it.

Well, the question is whether our thoughts are deterministic or not.  If you reset the universe to the same point multiple times, would everyone necessarily do exactly the same things?  Or might there be variation?  There being an extra-universal influence on our thoughts that wouldn't get reset gives the possibility of non-determinism, even if there is some ability to predict what it might do in known circumstances. Actually running that test though would be...  difficult.  We only get to see one of the runs, so we have nothing to compare to. So is our sense of free will an illusion?  Or meta-information that's leaking in from somewhere due to incomplete sandboxing?  Really hard to know for sure.  But, at the same time, does it actually matter?
How much have you read about the idea from its proponents?

Loads from angry mean people on the internet, very little from academics (none, if reading the Wikipedia article doesn't count). So I'm probably trying to learn anarchocommunism from Stalin. (I haven't heard much about it from its detractors, either, except what I've generated myself – I stopped reading the Wikipedia article before I got to the "criticism" section, and have only just read that now.)

In case this is the reason for disagreement, I might be criticisi... (read more)

I was with you until "paraphilia". I don't see how "wanting to see a world without strict gender roles" has anything to do with sexuality… and did you seriously just link to the Wikipedia article for autogynephilia‽ That's as verifiable as penis envy. (By which, I mean "probably applies to some people, somewhere, but certainly isn't the fully-general explanation they're using it as". And no, I don't think I'm doing the idea a disservice by dismissing it with a couple of silly comics;... (read more)

Thanks for commenting! (Strong-upvoted.) It's nice to get new discussion on old posts and comments.

probably applies to some people, somewhere


I don't think I'm doing the idea a disservice

How much have you read about the idea from its proponents? ("From its proponents" because, tragically, opponents of an idea can't always be trusted to paraphrase it accurately, rather than attacking a strawman.) If I might recommend just one paper, may I suggest Anne Lawrence's "Autogynephilia and the Typology of Male-to-Female Transsexualism: Concepts and Contro

... (read more)

These kinds of images should be saved as PNG. Use a version from before any JPEG compression, if you can.

there are no photons traveling from the sun to us

Woah, where did this assertion come from?

which in fact means that one photon fills up our universe.

This doesn't follow.

And how do we get away from that? By saying that they were 'probabilities' :)

Who's saying that? This post is talking about amplitudes. (And so on for the next paragraph.)

Why is the "all tags" link to

Oops, that's a mistake. Fixed now. Thanks.

This implies a solution to the "weak" Ship of Theseus problem: yes, it's the same ship.

I think this also implies a solution to the "strong" Ship of Theseus problem: "a new ship is created from the old parts" – but I'm less confident both that it implies this, and that it's the right conclusion to make. Consider also: mitosis. Which one is "the bacterium"™? But it doesn't quite make sense (to my fuzzy intuition) to say "the bacterium doesn't exist any more".

I think any ... (read more)

This is a good point. For all three of these (new ship from old parts, mitosis, and two flowers) the algorithm's answer would be that there are multiple admissible notions of what-the-relevant-object-is, i.e. multiple locally-minimal boundaries consistent with the initial conditions. And indeed, human intuition also recognizes that there are multiple reasonable notions of what-the-object-is. Different object-notions would be relevant to different queries (i.e. different sets of variables considered "far away"). E.g. in the strong ship of Theseus problem, low-level internal structure of the materials carries over from old to new ship, but their exact connections might not (i.e. nails might be in different places or boards in different order or even just new pitch on the hull). One ship-notion considers anything depending on those connections to be "nearby" (so that information can be safely thrown away), while the other ship-notion considers at least some things depending on those connections to be "far away" (so that information cannot be safely thrown away). On the other hand, if the ship is perfectly reconstructed down to a molecular level, then all of the information is carried over from old to new, and then the algorithm would unambiguously say it's the same ship.

Technically, certain photons have colours. Our "colour" categories are excessively broad, but all photons can be described as having a particular colour, or being colourless.

If our universe did not have the kind of structure that appears in a causal model, then our reasoning would not function properly. Induction would fall completely flat, and I don't think our brains would work right. We probably wouldn't exist in such a world, but if we're taking into account anthropic effects… well, I'm not even sure a human could survive long enough for a single conscious thought, since their state at time t+1 wouldn't follow from their state at time t.

After skimming others' replies, I've realised... (read more)

The only problem with epiphenomenalist theories of consciousness is that the thing we call consciousness does have an effect on our actions (proof: answer the question "how do you know you're conscious?" out loud), so the thing they call "consciousness" must be something different. However, these rules don't say that this thing is impossible or meaningless; by definition, it's caused by neurons, and so (if it actually happens) would be causally linked to reality, and hence meet the criteria for being "real".

Written in a somewhat fake wisdom manner, without reading other replies:

If you chose a different card, would he not say a different card's name? Therefore, the card he says is causally linked to your choice of card. And would you not hear different words if he said something different? Therefore, your observations are causally linked to the card he says. Just because we do not know how something works – nay, even if it is Unknowable™ – that doesn't make it causally unshackled from reality.

they would say that we should stop burning fossil fuels right now.

And would that be so hard?

  • Stop driving petrol and diesel cars.
    • Use public transport instead.
      • Make all new public transport electric – recharge buses at bus stops, etc.. Put in place infrastructure to support this.
    • Travel less. (The response to COVID-19 proves this can be done.)
  • Stop burning fossil fuels in power stations.
    • Use less (peak) electricity.
      • Turn lights off, turn down screen brightness on computers and phones, don't watch as much television, use less water, run washing machine
... (read more)
I have had a couple of experiences in which intense study of math and physics led me to some pretty dark psychological places,

One way of dealing with this problem is to get that out of the way when you're young (i.e., 6–11). Then you've learned coping mechanisms (which will end up used regularly), but don't have a distinct recollection of the horrible thought patterns that you might just fall back into if you think about them too hard, by the time you're older.

Considering how much wealth can be generated at the moment by running a computer program shifting (numbers loosely representing) money around, and the ever-more-sophisticated ways that this can be done (obviously, to the detriment of many of the humans involved), I think it's already in the process of substrate jumping. These things aren't limited to human minds and tax law any more.

Also, why would we kill our creators?

We might not. But if they were paperclip maximisers or pebble sorters, we might not see any value in their existence, and lots of harm. Heck, we're working to kill evolution's effect on us, and betray its "inclusive genetic fitness" optimisation criterion, and nobody cares, because we don't view it as having intrinsic worth. (Because it doesn't have intrinsic worth; it's just an emergent phenomenon of imperfectly-self-replicating stuff in an environment, and has no more value than the ... (read more)

You're right, it is (2)! If we build an artificial intelligence that smart, with such absurd resources, then we _will_ be in danger. Doing this thing implies we lose.

However, that does not mean that not doing this thing implies we do not lose. A ⇒ B doesn't mean ¬A ⇒ ¬B. Just because simulating trillions of humans then giving them internet access would be dangerous, that doesn't mean that's the only dangerous thing in the universe; that would be absurd. By that logic, we're immune from nuclear weapons or nan... (read more)

Not to get into too much irrelevant discussion about contemporary society's human-labelling paradigm, I think you mean non-binary Virtuists. A lot of trans people are men or women.

But this leaves five possibilities, not three!

[+Green +Blue] and [-Green -Blue] don't affect Green and Blue's relative standing, so they're equivalent to [0Green 0Blue].

Sorry, since when does "quiet strain in the back of your mind" automatically translate to "irrational"? This particular quiet voice is usually _right_; surely that makes it rational?

The trouble is, any utility function where 1 doesn't hold is vulnerable to intuition pumps. If you can't say which of A, B and C is better (e.g. A > B, B > C, C > A), then I can charge you a penny to switch from C → B, then B → A, then A → C, and you're three pennies poorer.

I really, really hope my utility function's "set B" can be mapped to the reals. If not, I'm screwed. (It's fine if what I want varies with time, so long as it's not circular at a given point in time.)

It would be good to elaborate on this. Whilst they're not strictly logically contradictory, a few reasonable assumptions here and there when extrapolating and they appear to suggest different courses of action.

You're assuming the AI has terminal access. Just because our brains are implemented as neurons doesn't mean we can manipulate matter on a cellular scale.

just give it a basic curiosity about certain things

What's "curiosity"? I don't think we can just say "just" yet, when we can't even explain this concept to a hypothetical human-minus-curiosity. (Wanting to learn more? What does it mean to actively learn about something?)

With recent events, you might not have been able to write more of these. Are you still planning to? I'd really like to read them.

Just so you know, the next one is posted. ;)
Thanks for the comment! My lateness to write the next installment is more related to having a lot of research work and study to do (as well as preparing a job interview), but I already have a draft of the second post. And this time, the short story has loads of ideas related to AI safety in non trivial ways. ;) I should be able to post it around the end of this week.

"One" means "an arbitrary person". "The one" means "the specific person we were just talking about".

Very good comic, thank you for the share!
Thank you, that was both funny and relevant.

The detector clicks 50% of the time because "detector makes a clicking noise" is so complex that it doesn't ever end up in the same state as "detector doesn't make a clicking noise" to interfere with it. There are multiple paths this photon can take to end up in the same configuration, because the photon moving around is simple enough that we can design an experiment to make some of the amplitude that's flowed to different configurations flow back to the same configurations – but the detector is complex enough that... (read more)

Load More