It looks like Theorem 1 can be improved slightly, by dropping the "only if" condition on . We can then code up something like Kolmogorov complexity by adding a probability transition from every site to our chosen UTM.

If you only want the weaker statement that there is no stationary distribution, it looks like there's a cheaper argument: Since is aperiodic and irreducible the hypothetical stationary distribution is unique. is closed under the action of , and (2) implies that for any , the map is an automorphism of the Markov chain

...That sounds a rather odd argument to make, even at the time. Astronomy from antiquity was founded on accurate observations.

Astronomy and epistemology aren't quite the same. Predicting where Saturn would be on a given date requires accurate observation, and nobody objected to Coperniucus as a calculational tool. For example, the Jesuits are teaching Copernicus in China in Chinese about 2 years after he publishes, which implies they translated and shipped it with some alacrity.

The heavens were classically held to be made of different stuff; quintessense (...

48y

Do you have a citation for that?
Added: No, this is false. The Jesuits were founded in 1540; Copernicus published
in 1542. Francis Xavier proposed sending astronomers to Asia while in Japan in
1552. In the same year he died trying to reach China. I don't think Jesuits
learned Chinese until about 1580. One of the first to do so, Matteo Ricci, again
asked for astronomers in 1605, which suggests to me that he was not, himself,
teaching Copernicus.

08y

It's worth noting that Copernicus' use of circular orbits required the use of
epicycles to make the theory fit the observations.

28y

I wonder how current physics will look like if/when GR and QM will be finally
unified...

The precise phrasing is deliberately a little tendentious, but the issue of the epistemological status of the telescope was raised by loads of people at the time. For a modern review with heavy footnotes, see eg Galileo, Courtier: The Practice of Science in the Culture of Absolutism, pp 95-100, (though the whole chapter is good)

For example, the first anti-Galilean tract is by Horky in 1610 and focussed mostly on the lack of reliability of the telescope. For another, Magini's letters (confirmed in Kepler and Galileo) write of a "star party" in...

78y

I think these words are rather telling (emphasis in the original, p.96):
And it goes on to show how the dispute was conducted on both sides in terms of
status, Galileo getting princes on side by sending them telescopes, and his
opponents attacking him because he was succeeding.
That sounds a rather odd argument to make, even at the time. Astronomy from
antiquity was founded on accurate observations. Galileo's contemporaries could
argue that the telescope wasn't good enough, but hardly that getting a better
view of the heavens could reveal nothing new. They were arguing over what could
be seen, not that seeing was the wrong thing to do.

tl;dr: The side of rationality during Galileo's time would be to recognise one's confusion and recognise that the models did not yet cash out in terms of a difference in expected experiences. That situation arguably holds until Newton's Principia; prior to that no one has a working physics for the heavens.

The initial heliocentric models weren't more accurate by virtue of being heliocentric; they were better by virtue of having had their parameters updated with an additional 400 years of observational data over the previous best-fit model (the Alfonsine tab...

Thank you for that informed account of the history.

You mention three times, without attributing it to any contemporary of Galileo, that the telescope "distorted the vision", which is a tendentious description. Given that the military application of the telescope was grasped as soon as the instrument became known, who at the time made this criticism? Did they similarly eschew its terrestrial use for the improvement of vision?

So, my observation is that without meta-distributions (or A_p), or conditioning on a pile of past information (and thus tracking /more/ than just a probability distribution over current outcomes), you don't have the room in your knowledge to be able to even talk about sensitivity to new information coherently. Once you can talk about a complete state of knowledge, you can begin to talk about the utility of long term strategies.

For example, in your example, one would have the same *probability* of being paid today if 20% of employers actually pay you every da...

The substantive point here isn't about EU calculations per se. Running a full analysis of everything that might happen and doing an EU calculation on that basis is fine, and I don't think the OP disputes this.

The subtlety is about what numerical data can formally represent your full state of knowledge. The claim is that a mere probability of getting the $2 payout does not. It's the case that on the first use of a box, the probability of the payout given its colour is 0.45 regardless of the colour.

However, if you merely hold onto that probability, then if y...

510y

However, a single probability for each outcome given each strategy is all the
information needed. The problem is not with using single probabilities to
represent knowledge about the world, it's the straw math that was used to
represent the technique. To me, this reasoning is equivalent to the following:
"You work at a store where management is highly disorganized. Although they
precisely track the number of days you have worked since the last payday, they
never remember when they last paid you, and thus every day of the work week has
a 1/5 chance of being a payday. For simplicity's sake, let's assume you earn
$100 a day.
You wake up on Monday and do the following calculation: If you go in to work,
you have a 1/5 chance of being paid. Thus the expected payoff of working today
is $20, which is too low for it to be worth it. So you skip work. On Tuesday,
you make the same calculation, and decide that it's not worth it to work again,
and so you continue forever.
I visit you and immediately point out that you're being irrational. After all, a
salary of $100 a day clearly is worth it to you, yet you are not working. I look
at your calculations, and immediately find the problem: You're using a single
probability to represent your expected payoff from working! I tell you that
using a meta-probability distribution fixes this problem, and so you excitedly
scrap your previous calculations and set about using a meta-probability
distribution instead. We decide that a Gaussian sharply peaked at 0.2 best
represents our meta-probability distribution, and I send you on your way."
Of course, in this case, the meta-probability distribution doesn't change
anything. You still continue skipping work, because I have devised the
hypothetical situation to illustrate my point (evil laugh). The point is that in
this problem the meta-probability distribution solves nothing, because the
problem is not with a lack of meta-probability, but rather a lack of considering
future consequences.
In

310y

Thanks, Jonathan, yes, that's how I understand it.
Jaynes' discussion motivates A_p as an efficiency hack that allows you to save
memory by forgetting some details. That's cool, although not the point I'm
trying to make here.

Concretely, I have seen this style of test (for want of better terms, natural language code emulation) used as a screening test by firms looking to find non-CS undergraduates who would be well suited to develop code.

In as much as this test targets indirection, it is comparatively easy to write tests which target data driven flow control or understanding state machines. In such a case you read from a fixed sequence and emit a string of outputs. For a plausible improvement, get the user to log the full sequence of writes, so that you can see on which instruc...

To my knowledge, it's not discussed explicitly in the wider literature. I'm not a statistician by training though, so my knowledge of the literature is not brilliant.

On the other hand, talking to working Bayesian statisticians about "what do you do if we don't know what the model should be" seems to reliably return answers of broad form "throw that uncertainty into a two-level model, run the update, and let the data tell you which model is correct". Which is the less formal version of what Jaynes is doing here.

This seems to be a reasona...

Thank you for calling out a potential failure mode. I observe that my style of inquisition can come across as argumentative, in that I do not consistently note when I have shifted my view (instead querying other points of confusion). This is unfortunate.

To make my object level opinion changes more explicit:

I have had a weak shift in opinion towards the value of attempting to quantify and utilise weak arguments in internal epistemology, after our in person conversation and the clarification of what you meant.

I have had a much lesser shift in opinion of

110y

Ok. See also my discussion post giving clarifications
[http://lesswrong.com/r/discussion/lw/hnq/some_clarifications_concerning_my_many_weak/].
I think that the most productive careful analysis of the validity of a claim
occurs in writing, with people who one believes to be arguing in good faith.
In person, you highlighted the problem of the first person to give arguments
having an argumentative advantage due to priming effects. I think this is much
less of a problem in writing, where one has time to think and formulate
responses.
My view on this point is very much contingent on what Euler actually did as
opposed to a general argument of the type "heuristics can be used to reach true
conclusions, and so we can have high confidence in something that's supported by
heuristics."
Beyond using a rough heuristic to generate the identity, Euler numerically
checked whether the coefficients agreed (testing highly nontrivial identities
that had previously been unknown) and found them to agree with high precision,
and verified that specializing the identity recovered known results.
If you don't find his evidence convincing, then as you say, we have to agree to
disagree because we can't fully externalize our intuitions

Fermat considered the sequence of functions f(n,x) = x^n for n = 0, 1, 2, 3, ....

Only very kind of. Fermat didn't have a notion of function in the sense meant later, and showed geometrically that the area under certain curves could be computed by something akin to Archimedes' method of exhaustion, if you dropped the geometric rigour and worked algebraically. He wasn't looking at a limit of functions in any sense; he showed that the integral could be computed in general.

The counterexample is only "very simple" in the context of *knowing* that the...

310y

Independently of whether Fermat thought of it as an example, Cauchy could have
considered lots of sequences of functions in order to test his beliefs, and I
find it likely that had he spent time doing so, he would have struck on this
one.
On a meta-level, my impression is that you haven't updated your beliefs based on
anything that I've said on any topic, in the course of our exchanges, whether
online or in person. It seems very unlikely that no updates are warranted. I may
be misreading you, but to the extent that you're not updating, I suggest that
you consider whether you're being argumentative when you could be inquisitive
and learn more as a result.

It possible that "were known in general to lead to paradoxes" would be a more historically accurate phrasing than "without firm foundation".

For east to cite examples, there's "The Analyst" (1734, Berkeley). The basic issue was that infinitesimals needed to be 0 at some points in a calculation and non-0 at others. For a general overview, this seems reasonable. Grandi noticed in 1703 that infinite series did not need to give determinate answers; this was widely known in by the 1730's. Reading the texts, it's fairly clear that th...

That it worked in every instance of continuous functions that had been considered up to that point, seemed natural, and extended many existing demonstrations that a specific sequence of continuous functions had a continuous limit.

A need for lemmas of the latter form are endemic, for a concrete class of examples, any argument via a Taylor series on an interval implicitly requires such a lemma, to transfer continuity, integrals and derivatives over. In just this class, you get numerical evidence came from the success of perturbative solutions to Newtonian mechanics, and theoretical evidence in the existence of well behaved Taylor series for most functions.

210y

In ~1659, Fermat considered
[http://www.math.ufl.edu/~joelar/FermatsIntegration.pdf] the sequence of
functions f(n,x) = x^n for n = 0, 1, 2, 3, .... Each of these is a continuous
function of x. If you restrict these functions to the interval between 0 and 1,
and take the limit as n goes to infinity, you get a discontinuous function.
So there's a very simple counterexample to Cauchy's ostensible theorem from
1821, coming from a sequence of functions that had been studied over 150 years
before. If Cauchy had actually looked at those examples of sequences of function
that had been considered, he would have recognized his ostensible theorem to be
false. By way of contrast, Euler did extensive empirical investigation to check
the plausibility of his result. The two situations are very, very different.

310y

I guess we'll have to agree to disagree here :-). I find Euler's evidence for
the product formula for sine to be far more convincing than what was available
to Cauchy at the time.
Edit: I say more here
[http://lesswrong.com/lw/hlx/the_use_of_many_independent_lines_of_evidence_the/94g8],
where I highlight how different the two situations are.

Observationally, the vast majority of mathematical papers do not make claims that are non-rigorous but as well supported as the Basel problem. They split into rigorous proofs (potentially conditional on known additional hypotheses eg. Riemann), or they offer purely heuristic arguments with substantially less support.

It should also be noted that Euler was working at a time when it was widely known that the behaviour of infinite sums, products and infinitesimal analysis (following Newton or Leibnitz) was without any firm foundation. So analysis of these obje...

110y

Could you give a source for this claim? "Foundation" sounds to me anachronistic
for 1735.

510y

What was the evidence?

I do not think that they are "making it up"; that phrase to me seems to attach all sorts of deliberate malfeasance that I do not wish to suggest. I think that to an *outside observer* the estimate is optimistic to the point of being incredible, and reflecting poorly on CEA for that.

These 291 people haven't pledged dollar values. They've pledged percentage incomes. To turn that into a dollar value you need to estimate whole-life incomes. Reverse engineering an estimate of income (assuming that most people pledge 10%, and a linear drop off in pledgers with 50% donating for 40 years), yields mean lifetime earnings of ~£100K. That's about the 98th centile for earnings in the UK.

710y

Ah; I thought that people were pledging a percent and then listing their
estimated income. Looking at an old email from gwwc I see in their discussion of
pledge calculations:
which makes me think they are working off of self-reported estimated lifetime
incomes. Though they might be extrapolating from people who submitted income
amounts to the people who didn't.
I don't think that's the right way to interpret "pledge". If 100 people pledge
to keep smoking and then only 50 end up going through with it, I would still say
that you had 100 pledges.
£112.8M over 291 people and 40 years is ~£10K, or an estimated income of £100K
at 10%. (Which is the same number you got, which I think means you didn't
actually apply your 50% figure.)
An average income of £100K is high, but looking over their members
[http://www.givingwhatwecan.org/about-us/our-members/list-of-members],
remembering that their students are Oxford students, and figuring that people
who go into banking or something via EtG might give larger percents (while still
having more to keep for themselves) I think their numbers are not "optimistic to
the point of being incredible".

Hi Will,

I'm glad to hear that a general response is being collated; if there are things where CEA can improve it would seem like a good idea to do them, and if I'm wrong I would like to know that. Turning to the listed points:

I went into that conversation with a number of questions I sought answers to, and either asked them or saw the data coming up from other questions. I knew your time was valuable and mostly targeted at other people there.

Adam explicitly signed off on my comment to Luke. He saw the draft post, commented on it, recommended it be put

The primary source of the post was an extensive email exchange with Adam Casey (currently working full time at CEA). Since we are friends, this was not primarily in an official capacity. I also asked Adam to cross check the numbers whilst wearing a more official hat.

I was encouraged by him and Alexey Morgunov (Cambridge LWer) to make the substance of this public immediately after Will Crouch came up to Cambridge.

Can I just make clear my role here. 1) I've had general conversation with Jonathan about CEA and MIRI, in which several of these criticisms were raised. 2) I checked over the numbers for the GWWC impact assessment. 3) I've also said that criticism in public is often productive, and that public scrutiny of CEA on LW would be helpful for people choosing between it and MIRI. 4) I saw a draft of the post before it was published.

I want to make it very clear that: 1) I do not endorse this post. 2) I did not do detailed fact-checking for it. 3) I do not want t...

Whose status ordering are you using? Getting someone who is *not* a mathematician to TMS is harder; within the Natural Sciences it is possible, and there are O(1) Computer Scientists, philosophers or others. For the historians, classicists or other subjects, mathmos are not high status. In terms of EtG, these groups are valuable - most traders are not quants.

In that case, having a claim on every page of the GWWC site claiming that £112.8M have been pledged seems deceptive. 291 people have pledged, and [by a black box that doesn't trivially correspond to reality] that's become £112.8M. I know that at least 3 people in Cambridge have seen that statistic and promptly *laughed* at GWWC. The numbers are implausible enough that <5s Fermi estimates seem to refute it, and then the status of GWWC as somewhat effective rational meta-charity is destroyed. Why would someone trust GWWC's assessment of charities or likely impact over, say, GiveWell, if the numbers GWWC display are so weird *and* lacking in justification?

310y

Do you really think that their pledge total is something other than the
(undiscounted) sum of what those 291 people have pledged to give over the course
of their lives? You think they're just making it up?

410y

If someone has pledged to give 10% of X, and you estimate X to be about Y, I
think it's not totally unreasonable to suggest Y*0.1 has been pledged,
especially if that person doesn't disagree with your estimate of X.

Talking about effective altruism is a constraint, as is talking about mathematics. Being a subject society makes it easier to get people from that subject to attend; it also makes it harder to convince people from outside that subject to even consider coming.

TMS pulls 80+ people to most of its talks, which are not generally from especially famous mathematicians. TCSS got 600 people for a Pensrose-Rees event. Both TCSS and TMS have grown rapidly in 18-24 months, having existed for far longer. This seems to indicate that randomly selected student societies h...

610y

One constrains you to a subject with thousands of high-status practitioners and
hundreds of students - the other restricts you to a subject with one high-status
practitioner and no students.

This holds for graduates who earn less than average as well. Is there data showing that the predominant source of career changes are people who would otherwise earn substantially less than mean? Is there data suggesting that the career changes are increasing incomes substantially?

We might mean many things by "2 + 2 = 4". In PA: "PA |- SS0 + SS0 = SSSS0", and so by soundness "PA |= SS0 + SS0 = SSSS0" In that sense, it is a logical truism independent of people counting apples. Of course, this is clearly not what most people mean by "2+2=4", if for no other reason than people did number theory before Peano.

When applied to apples, "2 + 2 = 4" probably is meant as: "apples + the world |= 2 apples + 2 apples = 4 apples". the truth of which depends on the nature of "the worl...

I want to note that I may be confused: I have multiple hypotheses fitting some fraction of the data presented.

- Supergoals and goals known, but unconscious affective death spirals or difficulties in actioning a far goal are interfering with the supergoals.
- Supergoals and goals known, goal is suboptimal.
- Supergoals not known consciously, subgoal known but suboptimal given knowledge of supergoals.

The first is what seems to be in the example. The second is what the strategy handles. The third is what I get when I try to interpret:

...This technique is about f

011y

You bring up a really good point here. I would say that my unconscious thinking
was making oversights and unexamined assumptions in the pursuit of goals. For
example, thinking "Okay there's a bunch of stuff that I want, but if I just
become super effective at reaching goals generally, then I'll get those things
automatically." Because it was overlooking other ways of reaching these goals,
it both failed to be motivated by some helpful things, like programming study
even if not impressive, and it also thought less creatively about how to hit the
supergoal.

So, it seems to me that what you describe here is not moving up a hierarchy of goals, unless there are serious issues with the mechanisms used to generate subgoals. It seems like slogans more appropriate to avoiding the demonstrated failure mode are:

"Beware affective death spirals on far-mode (sub)goals" or "Taboo specific terms in your goals to make them operationally useful" or possibly even "Check that your stated goals are not semantic stop-signs"

As presented, you are claiming that:

...I wanted to be a perfectly honed instru

011y

I am primarily referring to the unconscious drives underlying our actions, not
our verbal goals. No matter what term I used to describe it, when I imagined
myself doing very well in general relative to other people, spending every
moment in focused and topical optimization, I was excited and driven to pursue
the things I expected to make me like that. If I anticipated outcomes that did
NOT involve me being that kind of person, there was far less unconscious drive
to act.
Being hyper-competent was not a subgoal of programming or business, and if it
were I would have your same critique. Being hyper-competent was a subgoal of
having social success, having riches, being safe, a general assessment of "able
to succeed even in difficult situations." Programming and business were rather
what seemed consciously to be the best specific routes for achieving these
things, but they involved not being the sort of hyper-competent person, and
because I unconsciously desired that so much I was not, in practice, driven to
pursue programming or business.
The term "perfectly honed instrument" is meant to convey an intuitive sense, not
a technical description. But you would recognize such a person by them
constantly engaging in what actually seemed to have the greatest marginal return
on time, and probably by quickly developing unusually large amounts of skill.
Those terms refer to particular patterns of reality and not others - Bourne,
rational!Quirrell, arguably rational!Dumbledore are all extensions of this
intension. The average person is not.
By "going up the pyramid of goals" I'm referring to understanding more precisely
the rules generating the particular, concrete situations we desire, and
following a rule higher up on that pyramid. In other words, are there some
things we could think of concretely, that once thinking of them, we realized
were the real reason we had been motivated by something else was that we
unconsciously anticipated it to lead to the first thing? This is

Thanks. Definite typo, Fixed.

Better directions to the JCR (with images) are here.

ETA: Also fixed the list of meetups to link there.

011y

5 and 6 link to the same photo.

The foundational problem in your thesis is that you have grounded "rationality" as a normative "ought" on beliefs or actions. I dispute that assertion.

Rationality is more reasonably grounded as selecting actions so as to satisfy your explicit or implicit desires. There is no normative force to statements of the form "action X is not rational", unpacked as "If your values fall into {large set of human-like values}, then action X is not optimal, choosing for all similar situations where the algorithm you use is run".

Th...

012y

Compare What Do We Mean By Rationality
[http://lesswrong.com/lw/31/what_do_we_mean_by_rationality/].

From your own summary:

I think that trolley problems contain perfect information about outcomes in advance of them happening, ignore secondary effects, ignore human nature, and give artificially false constraints.

Which is to say they are idealised problems; they are *trued* dilemmas. Your remaining argument is fully general against any idealisation or truing of a problem that can also be used rhetorically. This is (I think) what Tordmor's summary is getting at; mine is doing the same.

Now, I think that's bad. Agree/disagree there?

So, I clearly disagree...

The thrust of your argument appears to be that: 1) Trolley problems are idealised 2) Idealisation can be a dark art rhetorical technique in discussion of the real world. 3) Boo trolley problems!

There are a number of issues.

First and foremost, reversed stupidity is not intelligence. Even if you are granted the substance of your criticisms of the activists position, this does not argue per se against trolley problems as dilemmas. The fact that they share features with a "Bad Thing" does not inherently make them bad.

Secondly, the whole point of cons...

612y

This is dangerous, in the real world. If you say "of these two options, I prefer
X," I would expect that to be misinterpreted by non-literal-minded people as "I
support X." In any real-world situation, I think it's actually smarter and more
useful to say something like, "This is the wrong choice--there's also the option
of Z" without associating yourself with one of the options you don't actually
support. Similarly:
Personally, I'm wary in general of the suggestion that I "should" intrinsically
have a preference about something. I reserve the right not to have a preference
worth expressing and being held to until I've thought seriously about the
question, and I may not have thought seriously about the question yet. If I
understand correctly, the original poster's point was that trolley problems do
not adequately map to reality, and therefore thinking seriously about them in
that way is not worth the trouble.

612y

This is strange, this is the second comment that summarized an argument that I'm
not actually making, and then argues against the made up summary.
My argument isn't against idealization - which would be an argument against any
sort of generalized hypothetical and against the majority of fiction ever made.
No, my argument is that trolley problems do not map to reality very well, and
thus, time spent on them is potentially conducive to sloppy thinking. The four
problems I listed were perfect foresight, ignoring secondary effects, ignoring
human nature, and constraining decisions to two options - these all lead to a
lower quality of thinking than a better constructed question would.
There's a host of real world, realistic dilemmas you could use in place of a
(flawed) trolley problem. Layoffs/redundancies to try to make a company more
profitable or keep the ship running as is (like Jack Welch at GE), military
problems like fighting a retreating defensive action, policing problems like
profiling, what burden of proof in a courtroom, a doctor getting asked for
performance enhancing drugs with potentially fatal consequences... there's
plenty of real world, reality-based situations to use for dilemmas, and we would
be better off for using them.

In the wider sense, MML still works on the dataset {stock prices, newspapers, market fear}. Regardless of what work has presently been done to compress newspapers and market fear, if your hypothesis is efficient then you can produce the stock price data for a very low marginal message length cost.

You'd write up the hypothesis as a compressor-of-data; the simplest way being to produce a distribution over stock prices and apply arithmetic coding, though in practice you'd tweak whatever state of the art compressors for stock prices exist.

Of course the side ef...

You and I both agree on Bayes implying 1/21 in the single constant case. Considering the 2 constant game as 2 single constant games in series, with uncertainty over which one (k1 and k2 the mutually exclusive "this is the k1/k2 game")

P(H | W) = P(H ∩ k1|W) + P(H ∩ k2|W) = P(H | k1 ∩ W)P(k1|W) + P(H|k2 ∩ W)P(k2|W) = 1/21 . 1/2 + 1/21 . 1/2 = 1/21

This is the logic that to me drives PSB to SB and the 1/3 solution. I worked it through in SB by conditioning on the day (slightly different but not substantially).

I have had a realisation. You work dir...

Continuity problem is that the 1/2 answer is independent of the ratio of expected number of wakings in the two branches of the experiment

Why is this a problem?

The next clause of the sentence is the problem

unless the ratio is 0 (or infinite) at which point special case logic is invoked to prevent the trivially absurd claim that credence of Heads is 1/2 when you are never woken under Heads.

The problem is special casing out the absurdity, and thus getting credences that are discontinuous in the ratio. On the other hand, you seem to take 1/21in PSB (...

No; P(H|W) = 1/21

Multiple ways to see this: 1) Under heads, I expect to be woken 1/10 of the time Under tails, I expect to be woken twice. Hence on the average for every waking after a head I am woken 20 times after a tail. Ergo 1/21.

2) Internally split the game into 2 single constant games, one for k1 and one for k2. We can simply play them sequentially (with the same die roll). When I am woken I do not know which of the two games I am playing. We both agree that in the single constant game P(H|W) = 1/21.

It's reasonably clear that playing two single const...

013y

Credence isn't constrained to be in [0,1]???
It seems to me that you are working very hard to justify your solution. It's a
solution by argument/intuition. Why don't you just do the math?
I just used Bayes rule. W is an awakening. We want to know P(H|W), because the
question is about her subjective probability when (if) she is woken up.
To get P(H|W), we need the following:
P(W|H)=2/20 (if heads, wake up if D20 landed on k1 or k2)
P(H)=1/2 (fair coin)
P(W|T)=1 (if tails, woken up regardless of result of coin flip)
P(T)=1/2 (fair coin)
Using Bayes rule, we get:
P(H|W)=(2/20)(1/2) / [(2/20)(1/2)+(1)*(1/2)] = 1/11
With your approach, you avoid directly applying Bayes' theorem, and you argue
that it's ok for credence to be outside of [0,1]. This suggests to me that you
are trying to derive a solution that matches your intuition. My suggestion is to
let the math speak, and then to figure out why your intuition is wrong.

Continuity problem is that the 1/2 answer is independent of the ratio of expected number of wakings in the two branches of the experiment, unless the ratio is 0 (or infinite) at which point special case logic is invoked to prevent the trivially absurd claim that credence of Heads is 1/2 when you are never woken under Heads.

If you are put through multiple sub-experiments in series, or probabilistically through some element of a set of sub-experiments, then the Expected number of times you are woken is linearly dependent on the distribution of sub-experiment...

013y

Why is this a problem? I'm perfectly comfortable with that property. Since you
really just have one random variable in each arm. You can call them different
days of the week, but with no new information they are all just the same thing
By D do you mean W?
Is this how you came up with the 1/3 solution? If so, I think it requires more
explanation. Such as what D is precisely.

You're woken with a big sign in front of you saying "the experiment is over now", or however else you wish to allow sleeping beauty to distinguish the experimental wakings from being allowed to go about her normal life.

Failing that, you are never woken; it shouldn't make any difference, as long as waking to leave is clearly distinguished from being woken for the experiment.

No. I assert P(H|W) = 1/21 in this case.

Two ways of seeing this: Either calculate the expected number of wakings conditional on the coin flip (m/20 and m for H and T). [As in SB]

Alternatively consider this as m copies of the single constant game, with uncertainty on each waking as to which one you're playing. All m single constant games are equally likely, and all have P(H|W) = 1/21. [The hoped for PSB intuition-pump]

013y

I need more clarification. Sorry. I do think we're getting somewhere...
The experimenters fix 2 unique constants, k1,k2, each in {1,2,..,20}, sedate
you, roll a D20 and flip a coin. If the coin comes up tails, they will wake you
on days k1 and k2. If the coin comes up heads and the D20 that comes up is in
{k1,k2}, they will wake you on day 1.
Do you agree that P(H|W)=2/22 in this case?

*Before* I am woken up, my prior belief is that I spend 24 hours on Monday and 24 on Tuesday regardless of the coin flip. Hence *before* I condition on waking, my probabilities are 1/4 in each cell.

When I wake, one cell is driven to 0, and the is no information to distinguish the remaining 3. This is the point that the sleeping twins problem was intended to illuminate.

Given awakenings that I know to be on Monday, there are two histories with the same measure. They are equally likely. If I run the experiment and count the number of events Monday ∩ H and Monday ...

As I see it, initially (as a prior, before considering that I've been woken up), both Heads and Tails are equally likely, and it is equally likely to be either day. Since I've been woken up, I know that it's not (Tuesday ∩ Heads), but I gain no further information.

Hence the 3 remaining probabilities are renormalised to 1/3.

Alternatively: I wake up; I know from the setup that I will be in this subjective state once under Heads and twice under Tails, and they are a priori equally likely. I have no data that can distinguish between the three states of identi...

013y

It's not equally likely to be either day. If I am awake, it's more likely that
it's Monday, since that always occurs under heads, and will occur on half of
tails awakenings.
Heads and tails are equally likely, a priori, yes. It is equally likely that you
will be woken up twice as it is that you will be woken up. Yes. That's true. But
we are talking about your state of mind on an awakening. It can't be both Monday
and Tuesday. So, what should your subjective probability be? Well, I know it's
tails and (Monday or Tuesday) with probability 0.5. I know it's heads and Monday
with probability 0.5.

The reason it corresponds to Sleeping Beauty is that in the limit of a large number of trials, we can consider blocks of 20 trials where heads was the flip and all values of the die roll occurred, and similar blocks for tails, and have some epsilon proportion left over. (WLLN)

Each of those blocks corresponds to Sleeping Beauty under heads/tails.

No; between sedation and amnesia you know nothing but the fact that you've been woken up, and that 20 runs of this experiment are to be performed.

Why would an earlier independent trial have any impact on you or your credences, when you can neither remember it nor be influenced by it?

013y

I don't know. It's a much more complicated problem, because you have 20 coin
flips (if I understand the problem correctly). I haven't taken the time to work
through the math yet. It's not obvious to me, though, why this corresponds to
the sleeping beauty problem. In fact, it seems pretty clear that it doesn't.

It isn't a probability; the only use of it was to note the method leading to a 1/2 solution and where I consider it to fail, specifically because the number of times you are woken is not bound in [0,1] and thus "P(W)" as used in the 1/2 conditioning is malformed, as it doesn't keep track of when you're actually woken up. In as much as it is anything, using the 1/2 argumentation, "P(W)" is the expected number of wakings.

...No. You will wake on Monday with probability one. But, on a randomly selected awakening, it is more likely that it's

113y

Why are you using the notation P(W) when you mean E(W)? And if you can get an
expectation for it, you must know the probability of it.
Randomly selecting a waking does not imply a uniform distribution. On the
contrary, we know the distribution is not uniform.

Of course P(W) isn't bound within [0,1]; W is one of any number of events, in this case 2: P(You will be woken for the first time) = 1; P(You will be woken a second time) = 1/2. The fact that natural language and the phrasing of the problem attempts to hide this as "you wake up" is not important. That is why P(W) is apparently broken; it double counts some futures, it is the expected number of wakings. This is why I split into conditioning on waking on Monday or Tuesday.

(Tuesday, tails) is not the same event as (Monday, tails). They are distinct ...

113y

If P(H) and P(H|W) are probabilities, then it must be true that:
P(H)=P(H|W)P(W)+P(H|~W)P(~W), where ~W means not W (any other event), by the law
of total probability
If P(H)=1/2 and P(H|W)=1/3, as you claim, then we have
1/2=1/3P(W)+P(H|~W)(1-P(W))
P(H|~W) should be 0, since we know she will be awakened if heads. But that leads
to P(W)=3/2.
P(W) should be 1, but that leads to an equation 1/2=1/3
So, this is a big mess.
The reason it is a big mess is because the 1/3 solution was derived by treating
one random variable as two.

013y

"Of course P(W) isn't bound within [0,1]"
Of course! (?) You derived P(W) using probability laws, i.e., solving for it in
this equation: P(H)=P(H|W)P(W), where P(H)=1/2 and P(H|W)=1/3. These are
probabilities. And your 1/3 solution proves there is an error.
If two variables have correlation of 1, I think you could argue that they are
the same (they contain the same quantitative information, at least).
No. You will wake on Monday with probability one. But, on a randomly selected
awakening, it is more likely that it's Monday&Heads than Monday&Tails, because
you are on the Heads path on 50% of experiments

The point of the PSB problem is that the approach you've just outlined is indefensible.

You agree that for each single constant k_i P(H|W) = 1/21. Uncertainty over which constant k_i is used does not alter this.

So if I run PSB 20 times, you would assert in each run that P(H|W) = 1/21. So now I simply keep you sedated between experiments. Statistically, 20 runs yields you SB, and each time you answered with 1/21 as your credence. Does this not faze you at all?

You have a scenario A where you assert foo with credence P, and scenario B where you also assert foo with credence P, yet if I put you in scenario A and then scenario B, keeping you sedated in the meantime, you do not assert foo with credence P...

013y

Jonathan,
In this problem:
Do you agree that P(H|W)=m/(20+m) ? If not, why not?
Do you also agree that when m=20 we have the sleeping beauty problem (with 20
wake ups instead of 2 for tails)? If not, why not?

013y

You just changed the problem. If you wake me up between runs of PSB, then
P(H|W)=1/21 each time. If not, I have different information to condition on.

The claim is implied by your logic; the fact that you don't engage with it does not prevent it from being a consequence that you need to deal with. Furthermore it appears to be the intuition by which you are constructing your models of Sleeping Beauty.

Imagine we repeat the sleeping beauty experiment many times. On half of the experiments, she'd be on the heads path. On half of the experiments, she'd be on the tails path.

Granted; no contest

If she is on the tails path, it could be either monday or tuesday.

And assuredly she will be woken on both days...

-113y

I think we could make faster progress if you started with the assumption that I
have read and understood the problem. Yes, I know that she is woken up twice
when tails.
You agree that
Given that she is awake right now, what should be her state of mind. Well, she
knows if heads it's Monday. She knows if tails it's either Monday or Tuesday.
The fact that she will (or has) been woken up on both days doesn't matter to her
right now. It's either Monday or Tuesday. Given that she cannot distinguish
between the two, it would make sense for her to think of those as equally likely
at this awakening (under tails). Thus, P(Monday|T,W)=1/2, P(T|W)=1/2, P(Monday ∩
T | W)=1/4.
The problem with your 1/3 solution is you treat the data as if they are counts
in a 3 by 1 contingency table (the 3 cells being Monday&H, Monday&T, Tuesday&T).
If the counts were the result of independent draws from a multinomial
distribution, you would get p(H|W)=1/3. You have dependent draws though. You
have 1 degree of freedom instead of the usual 2 degrees of freedom. That's why
your ratio is not a probability. That's why your solution results in nonsense
like p(W)=3/2.

P(Monday ∩ H | W) = P(Monday ∩ T | W). Regardless of whether the coin came up heads or tails you will be woken on Monday precisely once.

P(Monday ∩ T | W) = P(Tuesday ∩ T | W), because if tails comes up you are surely woken on both Monday and Tuesday.

You still seem to be holding on to the claim that there are as many observations after a head as after a tail; this is clearly false. There isn't a half measure of observation to spread across the tails branch of the experiment; this is made clearer in Sleeping Twins and the Probabilistic Sleeping Beauty probl...

013y

Probabilistic sleeping beauty
P(H|W)=1/21
Now, let's change the problem slightly.
The experimenters fix m unique constants, k1,...,km, each in {1,2,..,20}, sedate
you, roll a D20 and flip a coin. If the coin comes up tails, they will wake you
on days k1,...,km. If the coin comes up heads and the D20 comes up is in
{k1,...,km}, they will wake you on day 1.
Here, P(H|W)=m/(20+m)
If m is 1 we get 1/21.
If m is 20 we get 1/2, which is the solution to the sleeping beauty problem.

013y

No. I never made that claim, so I cannot "hold on to it". The number of
observations after tails doesn't matter here.
Imagine we repeat the sleeping beauty experiment many times. On half of the
experiments, she'd be on the heads path. On half of the experiments, she'd be on
the tails path. If she is on the tails path, it could be either monday or
tuesday. Thus, on an awakening Monday ∩ T is less likely than Monday ∩ H

It seems there are a few meta-positions you have to hold before taking Bayesianism as talked about here; you need the concept of Winning first. Bayes is not sufficient for sanity, if you have, say, an anti-Occamian or anti-Laplacian prior.

What this site is for is to help us be good rationalists; to win. Bayesianism is the best candidate methodology for dealing with uncertainty. We even have theorems that show that in it's domain it's uniquely good. My understanding of what we mean by Bayesianism is updating in the light of new evidence, and updating correctly within the constraints of sanity (cf Dutch books).

313y

We can discuss both epistemic
[http://wiki.lesswrong.com/wiki/Rationality#Epistemic_rationality] and
instrumental
[http://wiki.lesswrong.com/wiki/Rationality#Instrumental_rationality]
rationality.

313y

You are right that Bayesianism isn't sufficient for sanity, but why should it
prevent a post explaining what Bayesianism is? It's possible to be a Bayesian
with wrong priors. It's also good to know what Bayesianism is, especially when
the term is so heavily used. My understanding is that the OP is doing a good job
keeping concepts of winning and Bayesianism separated. The contrary would
conflate Bayesianism with rationality.

The same one that you're currently seeing; for all values of E there is a value of F such that this is consistent, ie that D has actually predicted you in the scenario you currently find yourself in.

The game is to pick a box numbered from 0 to 2; there is a hidden logical computation E yielding another value 0 to 2. Omega has a perfect predictor D of you. You choose C.

The payout is 10^((E+C)mod 3), and there is a display showing the value of F = (E-D)mod 3.

If F = 0, then:

- D = 0 implies E = 0 implies optimal play is C = 2; contradiction
- D = 1 implies E = 1 implies optimal play is C = 1; no contradiction
- D = 2 implies E = 2 implies optimal play is C = 0; contradiction

And similarly for F = 1, F = 2 play C = F+1 as the only stable solution (which nets ...

013y

I'm not sure this game is well defined. What value of F does the predictor D
see? (That is, it's predicting your choice after seeing what value of F?)

This ad-hoc fix breaks as soon as Omega makes a slightly messier game, wherein you receive a physical clue as to a computation output, and this computation and your decision determine your reward.

Suppose that for any output of the computation there is a a unique best decision, and that furthermore this set of (computation output, predicted decision) pairs are mapped to distinct physical clues. Then given the clue you can infer what decision to make and the logical computation, but this **requires** that you infer from a logical fact (the predictor of you) to the physical state to the clue to the logical fact of the computation.

013y

Can you provide a concrete example? (because I think that a series of
fix-example-fix ... cases might get us to the right answer)

The underlying issue is what we take the purpose of debate or discussion to be. Here we consider discourse to be prior to justified belief; the intent is to reveal the reasonable views to hold, and then update our beliefs.

If there is a desire to justify some specific belief as an end in itself, then the rules of logical politeness are null; they have no meaning to you as you're not looking to find truth, per se, but to defend an existing position. You have to admit that you could in principle be wrong, and that is a step that, by observation, most people d...

413y

Or that people endorse that idea in the abstract enough to adopt standards about
logical rudeness which can be recognized and applied in the heat of argument.
People are less likely to evade that way if they know everyone else will say
"you lost".

Even if it's the case that the statistics are as suggested, it would seem that a highly effective strategy is to ensure that there are multiple adults around all the time. I'll accept your numbers ad arguendo (though I think they're relevantly wrong).

If there's a 4% chance that one adult is an abuser, there's a 1/625 chance that two independent ones are, and one might reasonably assume that the other 96% of adults are unlikely to let abuse slide if they see any evidence of it. The failure modes are then things like abusers being able to greenbeard well eno... (read more)