No posts to display.
> In parallel, if I am to compare two independent scenarios, the at-least-one-in-ten-billion odds that I'm hallucinating all this, and the darned-near-zero odds of a Pascal's Mugging attempt, then I should be spending proportionately that much more time dealing with the Matrix scenario than that the...(read more)
Continuity and independence.
Continuity: Consider the scenario where each of the [LMN] bets refer to one (guaranteed) outcome, which we'll also call L, M and N for simplicity.
Let U(L) = 0, U(M) = 1, U(N) = 10**100
For a simple EU maximizer, you can then satisfy continuity by picking p=(1-1/10\*\...(read more)
> Thus, I can never be more than one minus one-in-ten-billion sure that my sensory experience is even roughly correlated with reality. Thus, it would require extraordinary circumstances for me to have any reason to worry about any probability of less than one-in-ten-billion magnitude.
No. The reaso...(read more)
I'll be there.
It's the most important problem of this time period, and likely human civilization as a whole. I donate a fraction of my income to MIRI.
> Which means that if we buy this [great filter derivation] argument, we should put a lot more weight on the category of 'everything else', and especially the bits of it that come before AI. To the extent that known risks like biotechnology and ecological destruction don't seem plausible, we should ...(read more)
This issue is complicated by the fact that we don't really know how much computation our physics will give us access to, or how relevant negentropy is going to be in the long run. In particular, our physics may allow access to (countably or more) infinite computational and storage resources given so...(read more)
I used "invariant" here to mean "moral claim that will hold for all successor moralities".
A vastly simplified example: at t=0, morality is completely undefined.
At t=1, people decide that death is bad, and lock this in indefinitely.
At t=2, people decide that pleasure is good, and lock that in in...(read more)
That does not sound like much of a win. Present-day humans are really not that impressive, compared to the kind of transhumanity we could develop into. I don't think trying to reproduce entites close to our current mentality is worth doing, in the long run.
While that was phrased in a provocative manner, there /is/ an important point here: If one has irreconcilable value differences with other humans, the obvious reaction is to fight about them; in this case, by competing to see who can build an SI implementing theirs first.
I very much hope it won't ...(read more)