A few thought on the inner ring

In modeling the behavior of the coolness-seekers, you put them in a less cool position.

It might be a good move in some contexts, but I feel resistant to taking on this picture, or recommending others take it on. It seems like making the same mistake. Focusing on the object level because you want to be [cool in that you focus on the object level], that does has the positive effect of focusing on the object level, but I think also can just as well have all the bad effects of trying to be in the Inner Ring. If there's something good about getting into the Inn... (read more)

11yExactly this. The whole point of the Inner Ring (which I did not read, but
judging by the review and my knowledge of Lewis/Christian thought and virtue
ethic) is that you should aim at the goods that are inherent to your trade or
activity (i.e., if you are a coder, writing good code), and not care about
social goods that are associated with the activity. Lewis then makes a second
claim (which is really a different claim) that you will also reach social goods
through sincerely pursuing the inherent goods of your activity.

Open problem: thin logical priors

I agree that the epistemic formulation is probably more broadly useful, e.g. for informed oversight. The decision theory problem is additionally compelling to me because of the apparent paradox of having a changing caring measure. I naively think of the caring measure as fixed, but this is apparently impossible because, well, you have to learn logical facts. (This leads to thoughts like "maybe EU maximization is just wrong; you don't maximize an approximation to your actual caring function".)

Concise Open Problem in Logical Uncertainty

In case anyone shared my confusion:

The while loop where we ensure that eps is small enough so that

bound > bad1() + (next - this) * log((1 - p1) / (1 - p1 - eps))

is technically necessary to ensure that bad1() doesn't surpass bound, but it is immaterial in the limit. Solving

bound = bad1() + (next - this) * log((1 - p1) / (1 - p1 - eps))

gives

eps >= (1/3) (1 - e^{ -[bound - bad1()] / [next - this]] })

which, using the log(1+x) = x approximation, is about

(1/3) ([bound - bad1()] / [next - this] ).

Then Scott's comment gives the rest. I was worried about the

... (read more)Concise Open Problem in Logical Uncertainty

Could you spell out the step

every iteration where mean(π΄[ππππ:ππππ])β₯2/5 will cause bound - bad1() to grow exponentially (by a factor of 11/10=1+(1/2)(β1+2/5ππ·))

a little more? I don't follow. (I think I follow the overall structure of the proof, and if I believed this step I would believe the proof.)

We have that eps is about (2/3)(1-exp([bad1() - bound]/(next-this))), or at least half that, but I don't see how to get a lower bound on the decrease of bad1() (as a fraction of bound-bad1() ).

16yYou are correct that you use the fact that 1+eps is at approximately e^(eps).
The concrete way this is used in this proof is replacing the ln(1+3eps) you
subtract from bad1 when the environment is a 1 with 3eps=(bound - bad1) / (next
- this), and replacing the ln(1-3eps/2) you subtract from bad1 when the
environment is a 0 with -3eps/2=-(bound - bad1) / (next - this)/2
Therefore, you subtract from bad1 approximately at least (next-this)((2/5)(bound
- bad1) / (next - this)-(3/5)*(bound - bad1) / (next - this)/2).
This comes out to (bound - bad1)/10.
I believe the inequality is the wrong direction to just use e^(eps) as a bound
for 1+eps, but when next-this gets big, the approximation gets close enough.

LessWrong 2.0

(Upvoted, thanks.)

I think I disagree with the statement that "Getting direct work done." isn't a purpose LW can or should serve. The direct work would be "rationality research"---figuring out general effectiveness strategies. The sequences are the prime example in the realm of epistemic effectiveness, but there's lots of open questions in productivity, epistemology, motivation, etc.

A Proposal for Defeating Moloch in the Prison Industrial Complex

This still incentivizes prisons to help along the death of prisoners that they predict are more likely then the prison-wide average to repeat-offend, in the same way average utilitarianism recommends killing everyone but the happiest person (so to speak).

-97y

37yHmmm, yes. Yikes. Additional thought needed.

The value of learning mathematical proof

I see. That could be right. I guess I'm thinking about this (this = what to teach/learn and in what order) from the perspective of assuming I get to dictate the whole curriculum. In which case analysis doesn't look that great, to me.

The value of learning mathematical proof

Ok that makes sense. I'm still curious about any specific benefits that you think studying analysis has, relative to other similarly deep areas of math, or whether you meant hard math in general.

07yI think that analysis is actually the easiest entry point to the kind of
mathematical reasoning that I have in mind for people who have learned calculus.
Most of the theorems are at least somewhat familiar, so one can focus on the
logical rigor without having to simultaneously having to worry about
understanding what the high level facts are.

The value of learning mathematical proof

(See reply there.)

The value of learning mathematical proof

Seems like it's precisely because of the complicated technical foundation that real analysis was recommended.

What I'm saying is, that's not a good reason. Even the math with simple foundations has surprising results with complicated proofs that require precise understanding. It's hard enough as it is, and I am claiming that analysis is too much of a filter. It would be better to start with the most conceptually minimal mathematics.

... (read more)Even great mathematicians ran into trouble playing fast and loose with the real numbers. It took them about two hundred y

07yOh, sure, in expressing agreement with Epictetus I was just saying that I don't
think that you get the full benefits that I was describing from basic discrete
math. I agree that some students will find discrete math a better introduction
to mathematical proof.

The value of learning mathematical proof

Could you say more about why you think real analysis specifically is good for this kind of general skill? I have pretty serious doubts that analysis is the right way to go, and I'd (wildly) guess that there would be significant benefits from teaching/learning discrete mathematics in place of calculus. Combinatorics, probability, algorithms; even logic, topology, and algebra.

To my mind all of these things are better suited for learning the power of proof and the mathematical way of analyzing problems. I'm not totally sure why, but I think a big part of it i... (read more)

17yThis is also somewhat in reply to your elaboration in this comment
[http://lesswrong.com/lw/mac/the_value_of_learning_mathematical_proof/cfoc].
Just some data points:
In regards to this topic of proof, and more generally to the topic of formal
science, I have found logic a very useful subject. For one, you can leverage
your verbal reasoning ability, and begin by conceiving of it as a symbolization
of natural language, which I find for myself and many others is far more
convenient than, say, a formal science that requires more spatial reasoning or
abstract pattern recognition. Later, the point that formal languages are
languages in their own right is driven home, and you can do away with this
conceptual bridge.
Logic also has helped me to conceive of formal problems as a continuum of
difficulty of proof, rather than proofs and non-proofs. That is, when you read a
math textbook, sometimes you are instructed to Solve, sometimes to Evaluate,
sometimes to Graph; and then there is the dreaded Show That X or Prove That X!
In a logic textbook, almost all exercises require a proof of validity, and you
move up over time, deriving new inference rules from old, and moving onto
metalogical theorems. Later returning to books about mathematical proof, I found
things much less intimidating. I found that proof is not a realm forbidden to
those lacking an innate ability to prove; you must work your way upwards as in
all things.
Furthermore, in regards to this:
In my opinion, very significant and complex results in logic are arrived at
quite early in comparison to the significance of, and effort invested in,
results in other fields of formal science.
And in regards to this:
I have found that in continuous mathematics I have walked away from proofs with
a feeling best expressed as, "If you say so," as opposed to discrete mathematics
and logic, where it's more like, "Why, of course!"

07yI agree with Epictetus' comment.

37yPersonally I think real analysis is an awkward way to learn mathematical proofs,
and I agree discrete mathematics or elementary number theory is much better. I
recommend picking up an Olympiad book for younger kids, like "Mathematical
Circles, A Russian Experience."

07yI think the main thrust of the article was less about the power of mathematics
and more about the the habits of close reading and careful attention to detail
required to do rigorous mathematics.
Seems like it's precisely because of the complicated technical foundation that
real analysis was recommended. Theorems have to be read carefully, as even
simple ones often have lots of hypotheses. Proofs have to be worked through
carefully to make sure that no implicit assumptions are being introduced. Even
great mathematicians ran into trouble playing fast and loose with the real
numbers. It took them about two hundred years to finally lay rigorous
foundations for calculus.

Open Thread, May 11 - May 17, 2015

PSA: If you wear glasses, you might want to take a look behind the little nosepads. Some... stuff... can build up there. According to this unverified source it is oxidized copper from glasses frame + your sweat, and can be cleaned with an old toothbrush + toothpaste.

97ySounds like the only disutility of the stuff is that it annoys some people, but
it can't annoy you if you dont notice it...so why bring it up?

Precisely Bound Demons and their Behavior

There are ten thousand wrong solutions and four good solutions. You don't get much info from being told a particular bad solution. The opposite of a bad solution is a bad solution.

17ySo ask a series of "which of X and Y would you prefer that we do". The demon
always prefers the worst thing, but is constrained to truthfully describe its
preferences. This is a single bit of data, but it's really useful.

Open thread, Mar. 2 - Mar. 8, 2015

Lol yeah ok. I was unsure because alexa says 9% of search traffic to LW is from "demetrius soupolos" and "traute soupolos" so maybe there was some big news story I didn't know about.

Open thread, Mar. 2 - Mar. 8, 2015

http://www.alexa.com/siteinfo/lesswrong.com

(The recent uptick is due to hpmor, I suppose?)

07yProbably yes, see: http://www.alexa.com/siteinfo/hpmor.com
[http://www.alexa.com/siteinfo/hpmor.com]

Harry Potter and the Methods of Rationality discussion thread, March 2015, chapter 116

I'd say your first thought was right.

... (read more)She noticed half an hour later on, when Harry Potter seemed to sway a bit, and then hunch over, his hands going to cover up his forehead; it looked like he was prodding at his forehead scar. The thought made her slightly worried; everyone knew there was something going on with Harry Potter, and if Potter's scar was hurting him then it was possible that a sealed horror was about to burst out of his forehead and eat everyone. She dismissed that thought, though, and continued to explain Quidditch facts to the historicall

Harry Potter and the Methods of Rationality discussion thread, March 2015, chapter 116

Hermione the Hi-Fi Heiress of Hufflepuff and Harry the Humanist

Harry Potter and the Methods of Rationality discussion thread, March 2015, chapter 114 + chapter 115

As a simple matter of fact, Voldemort is stronger than Harry in basically every way, other than Harry's (incomplete) training in rationality. If Voldemort were a good enough planner, there's no way he could lose; he is smarter, more powerful, and has more ancient lore than any other wizard. If Voldemort were also rational, and didn't fall prey to overconfidence bias / planning fallacy...

Well, you can be as rational as you like, but if you are human and your opponent is a superintelligent god with a horde of bloodthirsty nanobots, the invincible Elder Ligh... (read more)

22yAh. But he would want to be more careful than that, because there's a prophecy,
and Voldemort got burned the last time a prophecy was involved.
So he goes out of his way to tear it apart, by bringing Hermione back, for
instance, which required the stone, and having the other Tom swear an
unbreakable vow.

47yYup. So the solution is not to make your villain a superintelligent god with a
horde of bloodthirsty nanobots, the invincible Elder Lightsaber, and the One
Thing to Rule Them All to begin with. Eliezer took the risk of setting up an
incredibly powerful villain, and it is to his credit as a writer that up until
the very end he made us believe that he was capable of writing a satisfying
resolution anyway.
Frankly, he still might. There are four chapters left, and Eliezer is nothing if
not capable of surprising his audience. And as a Naruto fan, he might also have
come across Bleach (another of the Big Three shounen series), and learned from
its author already having made the exact same mistake.

Harry Potter and the Methods of Rationality discussion thread, February 2015, chapter 113

A brief and terrible magic lashed out from the Defense Professor's wand, scouring the hole in the wall, scarring the huge chunk of metal that lay in the room's midst; as Harry had requested, saying that the method he'd used might identify him.

I'm kind of worried about this... all the real attempted solutions I've seen use partial transfiguration. But if we take "the antagonist is smart" seriously, and given the precedent for V remembering and connecting obscure things (e.g. the Resurrection Stone), we should assume V has protections ... (read more)

Harry Potter and the Methods of Rationality discussion thread, February 2015, chapter 113

Didn't V see at least the results of a Partial Transfiguration in Azkaban (used to cut through the wall)? Doesn't seem like something V would just ignore or forget.

17yI believe Voldemort was unconscious at the time, following a magical feedback
mishap at the conclusion of his duel with Bahry. Bellatrix was awake, but
probably not very coherent after eleven years in Azkaban, and Voldemort strikes
me as the type to dismiss confusing reports from unreliable underlings.

Harry Potter and the Methods of Rationality discussion thread, February 2015, chapter 113

Since they are touching his skin, does he need his wand to cancel the Transfiguration?

37yNo. He just learned to dispell Transfiguration without a wand when he dispelled
the one on Hermione's body.

Harry Potter and the Methods of Rationality discussion thread, February 2015, chapter 112

Right, this is a stronger interpretation.

47yReduce, re-use, recycle.

Harry Potter and the Methods of Rationality discussion thread, February 2015, chapter 112

This is persuasive, but... why the heck would Voldemort go the trouble of breaking into Azkaban instead of grabbing Snape or something?

47yVM said he broke into Azkaban to find out where his wand was; there's also the
flesh of the servant thing. Using her Dark Mark is a secondary benefit.

57yIn Chapter 61 [http://hpmor.com/chapter/61] Dumbledore says:

Harry Potter and the Methods of Rationality discussion thread, February 2015, chapter 112

Oh right, good point. Likewise Nott.

Harry Potter and the Methods of Rationality discussion thread, February 2015, chapter 112

So, like, is Snape in that crowd of Death Eaters, or what?

07yYou can't Apparate within the Hogwarts wards.

47yI rather doubt it; he might still be βguardingβ that corridor.
On the other hand, Lucius Malfoy should be there. His reaction might be
interesting, given his previous, rather unusual encounters with Harry β¦

Request: Sequences book reading group

FYI, each sequence is (very roughly) 20,000 words.

27yAssuming it is slower to read than the standard 200 wpm, that's still only a
couple of hours each; seems doable!

Harry Potter and the Methods of Rationality discussion thread, February 2015, chapters 105-107

(Presumably Parseltongue only prevents willful lies.)

Quirrell also claims (not in Parseltongue):

Occlumency cannot fool the Parselmouth curse as it can fool Veritaserum, and you may put that to the trial also.

It seems like what you can say in Parseltongue should only depend on the actual truth and on your mental state. What happens if I Confundus / Memory Charm someone into believing X? Can they say X in Parseltongue? If they can say it just because they believe it, then Parseltongue is not so hard to bypass; I just Confundus myself (or get someone t... (read more)

If Parseltongue depended only on the actual truth of the world, Voldemort would have won already, because you can then pull single bits of arbitrary information out of the aether one at a time.

Harry Potter and the Methods of Rationality discussion thread, February 2015, chapters 105-107

[EDIT: the Dark Lord of the Matrix have fixed this.]

There's a glitch in the Matrix:

A blank-eyed Professor Sprout had now risen from the ground, had picked up Harry's wand and was wrapping it in a shimmering cloth.

Then Harry does some bargaining, and then...

After that, Professor Sprout picked up Harry's wand, and wrapped it in shimmering cloth; then she placed it on the floor, and pointed her own wand at Harry.

37yI noticed that too. It's often a sign of obliviation. My secondary hypothesis is
that it was a mistake and will be corrected in a later update.

An alarming fact about the anti-aging community

Seconded. Specifically, citations for the implied claims (1) that it is not exorbitantly expensive to perform the organ regeneration or pay for an insurance policy that will pay for that, and (2) how often death is caused by something that can be fixed with organ transplants. Also relevant would be the probability that you would get a successful organ transplant without the cell preservation.

Harry Potter and the Methods of Rationality discussion thread, February 2015, chapter 104

A bunch of unspecified Muggle items he got the Weasleys to obtain for him.

Stupid Questions February 2015

Ο maximally simplifies finding the circumference of of a circle from its diameter

More importantly, Ο is the area of the unit circle. If you're talking about angles you want Ο (tau), if you're talking about area you want Ο. And you always want pie, ha ha.

Signalling with T-Shirt slogans

Here's a shirt I made, stating that PA is consistent in mysterious looking symbols. Not directly rationality related, but could be a conversation starter. http://teespring.com/paconsistent

Open thread, Oct. 27 - Nov. 2, 2014

Personally, the nonverbal thing is the proper content of math---drawing (possibly mental) pictures to represent objects and their interactions. If I get stuck, I try doing simpler examples. If I'm still stuck, then I start writing things down verbally, mainly as a way to track down where I'm confused or where exactly I need to figure something out.

Power and difficulty

Oh, I slightly misread some of the previous paragraphs. I was thinking specifically in terms of skills that you develop by doing something hard, rather than object-level products. What you said now makes perfect sense; and in either case writing a *third* game directly in machine code would be a waste of time, despite still being pretty hard.

Power and difficulty

Upvoted.

Similarly, writing a game in machine code or as a set of instructions for a Turing machine is certainly difficult, but also pretty dumb, and has no significant payoff beyond writing the game in a higher-level language.

IAWYC, but this example doesn't seem true. The additional payoff would be that you are forced to invent a memory system, bootstrapping compilers, linear algebra algorithms, etc., depending on how complicated the game is.

2[anonymous]7yI'm still not seeing the payoff... all that stuff has already been done by other
people, probably more than enough for most games you would create.

Open thread, Oct. 6 - Oct. 12, 2014

It seems like FAI requires *deeper* math than UFAI, for some appropriate value of deeper. But this "trial and error" still requires some math. You could imagine a fictitious Earth where suddenly it becomes easy to learn enough to start messing around with neural nets and decision trees and metaheuristics (or something). In that Earth, AI risk is increased by improving math education in that particular weird way.

I am trying to ask whether, in our Earth, there is a clear direction AI risk goes given more plausible kinds of improvements in math educa... (read more)

07yI'd endorse that. But IME mathematical advances aren't usually new ways to do
the same things, they're more often discoveries that it's possible to do new
things.

Open thread, Oct. 6 - Oct. 12, 2014

Question: Say someone dramatically increased the rate at which humans can learn mathematics (over, say, the Internet). Assume also that an intelligence explosion is likely to occur in the next century, it will be a singleton, and the way it is constructed determines the future for earth-originating life. Does the increase in math learning ability make that intelligence explosion more or less likely to be friendly?

Responses I've heard to questions of the form, "Does solving problem X help or hinder safe AGI vs. unsafe AGI?":

Improvements in ratio

37yI would think that FAI requires mathematics a lot more than does UFAI, which can
be created through trial and error.

Open thread, Sept. 29 - Oct.5, 2014

The literal reading would be A-I-ksi or A-I-zai, said aye-eye-ksee or aye-eye-zai, because AI is standing for Artificial Intelligence and XI is the greek letter. But yeah, I just avoid mentioning it by name :)

[Link] Forty Days

Interesting post.

But:

HIV-infected people must provide the names of all sexual partners for the past sic months.

You missed a golden opportunity:

...all sexual partners for the past sic [

sic] months.

0[anonymous]7yDamn, I was all fired up to make that joke. Great minds think alike, I
suppose...

2014 iterated prisoner's dilemma tournament results

"Do not attempt long chains of reasoning or complicated plans."

Open thread, Sept. 1-7, 2014

All else being equal, if you have the choice, would you pick (a) your son/daughter immediately ceases to exist, or (b) your son/daughter experiences a very long, joyous life, filled with love and challenge and learning, and yes, some dust specks and suffering, but overall something they would describe as "an awesome time"? (The fact that you might be upset if they ceased to exist is not the point here, so let it be specified that (a) is actually everyone disappearing, which includes your child as a special case, and likewise (b) for everyone, again including your child as a special case.)

27yIf the suffering "rounds down" to 0 for everyone, sure, A is fine. That is, a
bit of pain in order to keep Fun. But no hellish levels of suffering for anyone.
Otherwise, B. Given how the world currently looks, and MWI, it's hard to see how
it's possible to end up with everyone having pain that rounds down to 0.
So given the current world and my current understanding, if someone gave me a
button to press that'd eliminate earth in a minute or so, I'd press it without
hesitation.

Raven paradox settled to my satisfaction

Right, I should have written, "I agree. Also, ...". I just wanted to find the source of the intuition that seeing non-black non-ravens is evidence for "non-black -> non-raven".

Raven paradox settled to my satisfaction

I think it's just wrong that "H1': If it is not black, it is not a raven" predicts that you will observe non-black non-raven objects, under the assumption/prior that the color distributions within each type of object (chairs, ravens, bananas, etc.) are independent of each other.

The intuition comes from implicitly visualizing the observation of an unknown non-black object O; then, indeed, H1 predicts that O will turn out to not be a raven. Then point is, even observing that O is non-black would decrease your credence in H1; and then increase it ... (read more)

07yThis model of independence between shapes is what I'm calling the implicit model
that people use to say that the conclusion of the raven paradox is absurd.

Three questions about source code uncertainty

1) Yes, presumably; your brain is a vast store of (evolved)(wetware)(non-serial)(ad-hoc)(etc.)algorithms that has so far been difficult for neuroscientists to document.

2) Just plain empirical? There's nothing stopping you from learning your own source code, in principle, it's just that we don't AFAIK have scanners that can view "many" nearby neurons, in real time, individually (as opposed an fMRI).

3) Well that's much more difficult. Not sure why Mark_Friedenbach's comment was downvoted though, except maybe snarkiness; heuristics and biases is a small step towards understanding some of the algorithms you are (and correcting for their systematic errors in a principled way).

Rationality Quotes July 2014

There's a saying that goes "People who live in glass houses shouldn't throw stones." Okay. How about "Nobody should throw stones." That's crappy behavior. My policy is: "No stone throwing regardless of housing situation." Don't do it. There is one exception though. If you're trapped in a glass house, and you have a stone, then throw it. What are you, an idiot? So maybe it's "Only people in glass houses should throw stones, provided they are trapped in the house with a stone." It's a little longer, but yeah.

---Demetri Martin, Person (2007)

58yClever reasoning that completely misses the point.
Throwing stones is an entirely appropriate response to many situations. It is
ill advised (and contemptible) in some others.

18yIf you're trapped in a glass house and you have a stone, throwing it is still a
terrible idea.

Rationalist Sport

Rock-climbing:

- Individualistic
- Meditative
- Works many different muscles

The Power of Noise

Many important problems in graph theory, Ramsey theory, etc. were solved by considering random combinatorial objects (this was one of the great triumphs of Paul Erdos) and thinking in purely deterministic terms seems very unlikely to have solved these problems.

From a Bayesian perspective, a probability is a cognitive object representing the known evidence about a proposition, flattened into a number. It wouldn't make sense to draw conclusions about e.g. the existence of certain graphs, just because *we in particular* are uncertain about the structure of s... (read more)

Open Thread, May 19 - 25, 2014

"[...]may be the case[...]"

Sometimes this phrase is harmless, but sometimes it is part of an important enumeration of possible outcomes/counterarguments/whatever. If "the case" does not come with either a solid plan/argument or an explanation why it is unlikely or not important, then it is often there to make the author and/or the audience feel like all the bases have been covered. E.g.,

We should implement plan X. It may be the case that [important weak point of X], but [unrelated benefit of X].

I speculate (based on personal glimpses, not based on any stable thing I can point to) that there's many small sets of people (say of size 2-4) who could greatly increase their total output given some preconditions, unknown to me, that unlock a sort of hivemind. Some of the preconditions include various kinds of trust, of common knowledge of shared goals, and of person-specific interface skill (like speaking each other's languages, common knowledge of tactics for resolving ambiguity, etc.).

[ETA: which, if true, would be good to have already set up before crunch time.]