All of shminux's Comments + Replies

You are way more fallible than you think

I agree that there are people who don't need this warning most of the time. Because they already double and triple check their estimates and are the first ones to admit to their fallibility. "Most of us" are habitually overconfident though. I also agree that the circumstances matter a lot, and some people in some circumstances can be accurate at 1% level, but most people in most circumstances aren't. I'm guessing that superforecasters would not even try to estimate anything at 1% level, realizing they cannot do it well enough. We are most fallible when we ... (read more)

Morality is Scary

To repost my comment from a couple of weeks back, which seems to say roughly the same thing, not as well:

I don't believe alignment is possible. Humans are not aligned with other humans, and the only thing that prevents an immediate apocalypse is the lack of recursive self-improvement on short timescales. Certainly groups of humans happily destroy other groups of humans, and often destroy themselves in the process of maximizing something like the number of statues. Best we can hope for that whatever takes over the planet after meatbags are gone has some of

... (read more)
4Charlie Steiner5dDo you think there are changes to the current world that would be "aligned"? (E.g. deleting covid) Then we could end up with a world that is better than our current one, even without needing all humans to agree on what's best. Another option: why not just do everything at once? Have some people living in a diverse Galactic civilization, other people spreading the word of god, and other people living in harmony with nature, and everyone contributing a little to everyone else's goals? Yes, in principle people can have different values such that this future sounds terrible to everyone - but in reality it seems more like people would prefer this to our current world, but might merely feel like they were missing out relative to their own vision of perfection.
2Ratios5dI have also made a similar comment a few weeks ago [https://www.lesswrong.com/posts/ZddY8BZbvoXHEvDHf/selfishness-preference-falsification-and-ai-alignment?commentId=7ydhepAvLZzzaTtMd#KLBExwQTsZ5rsTKei] , In fact, this point seems to me so trivial yet corrosive that I find it outright bizarre it's not being tackled/taken seriously by the AI alignment community.
How do you write original rationalist essays?

But gun to my head - I can't seem to just sit down and make up an original non-fiction essay worthy of Less Wrong

With a gun to your head you would. It's amazing what the right motivation can do.

5lumenwrites7dWhile I'd rather not test this empirically, I think I'm feeling pretty motivated to do this, and yet I can't. I'd really like to solve this issue without resorting to hiring a professional assassin on myself.
Seeking Truth Too Hard Can Keep You from Winning

As an arealist, I certainly can't disagree with your definition of truth, since it matches mine. In fact, I stated on occasion that tabooing true, say, by replacing with "accurate" where possible, is a very useful exercise.

The problem of criterion dissolves once you accept that you are an embedded agent with a low-fidelity model of the universe you are embedded in, including self. There is no circularity. Knowing how to know something is an occasionally useful step, but not essential for extracting predictions from the model of the universe, which is the a... (read more)

Question/Issue with the 5/10 Problem

From the link 

"I have to decide between $5 and $10. Suppose I decide to choose $5. I know that I'm a money-optimizer, so if I do this, $5 must be more money than $10, so this alternative is better. Therefore, I should choose $5."

To me there is a sleight of hand there. The statement "I know that I'm a money-optimizer" is not a mathematical statement, but an empirical one, it can be tested through one's actions. If you take $5 instead of $10, you are not a money-optimizer, even if you initially think you are, and that's something, as an agent, you can learn about yourself by observing your actions.

Why Study Physics?

I'm sure one can train this skill, to some degree at least. I don't think I got better at it, but I did use "the appropriate level of abstraction" to get the numerical part of my thesis done without needing a lot of compute, 

By the way, I agree that finding the appropriate level of abstraction is probably the core of what the OP describes.

The bonds of family and community: Poverty and cruelty among Russian peasants in the late 19th century

The question for me is how much these observations apply to peasant life in other places and at other times. I’m hesitant to generalize, since this is the first book-length work of ethnography I’ve read in the context of this project, but for me it opens questions. Is cruelty towards animals and children, and an almost slave status for women, the norm?

The modern Western notions of classifying certain behaviors as cruelty, dishonesty, abuse and so on emerged from the life of surplus, when you could afford this luxury. Morals emerge from the need to survive,... (read more)

There's something to what you say, but at the same time, it sounds like you're suggesting that the morals of the peasants in question were well-adapted to their situation. But it seems hard to imagine that e.g. frequently stealing from each other, or jealous neighbors uprooting the trees of their slightly more well-off neighbors, would have been particularly adaptive in the long run - it's setting up for the community to stay poor and miserable indefinitely, even if it occasionally benefits individuals. 

Of course there's a sense in which the situation... (read more)

Is it better to fix a problem directly, or start again so the problem never happens?

Rewriting is hard, refactoring is easy and gets you 80% toward the goal that pushes one to rewrite. Also can be done incrementally.

Why Study Physics?

I think the title should be "why study physicists" not "why study physics". Because what you are describing is a gift certain physicists have, and others do not. I had it in high school (often when the teacher would state a problem in class, the only thing that was obvious from the beginning was the answer, not how to get to it), and it saved my bacon in grad school a few times many many years later. Recently it took my friend and me about 5 min of idle chatting to estimate the max feasible velocity of a centrifugal launch system, and where the bottlenecks... (read more)

I'm not so sure. I think a lot of physicists get better at this through practice, maybe especially in undergrad. I have a PhD in physics, and at this point I think I'm really good at figuring out the appropriate level of abstraction to use on something (something I'd put in the same category as the things mentioned in the OP.) I don't totally trust my own recollection, but I think I was worse at this freshman year, and much more likely to pick e.g. continuum vs. discrete models of things in mechanics inappropriately and make life hard for myself.

4adamShimi9dCould you write a list of physicists which have such "gift"? Might be useful for analyzing that specific skill.
6Steven Byrnes9dYeah, I also seem to have a knack for that (as good as anyone in my cohort at a top physics grad school, I have reason to believe), but I have no idea if I got it / developed it by doing lots of physics, or if I would have had it anyway. It's hard to judge the counterfactual! Hmm, I do vaguely remember, in early college, going from a place where I couldn't reliably construct my own differential-type arguments in arbitrary domains ("if we increase the charge on the plate by dQ ..." blah blah, wam bam, and now we have a differential equation), to where I could easily do so. Maybe that's weak evidence that I got something generalizable out of physics?
You are way more fallible than you think

It's a good question. If you ever do, say, project estimates at work, and look back at your track record, most of us would give 99% odds of completion to some project within a given time (well padded to make it that high), and still notice that we go over time and/or over budget way more often than that. There are exceptions, but in general we suck at taking into account long tails.

You are way more fallible than you think

That's another way to look at it. The usual implicit assumptions break down on the margins. Though, given the odds of this happening (once in a bush, at best, and the flame was not all that glorious), I would bet on hallucinations as a much likelier explanation. Happens to people quite often.

You are way more fallible than you think

Uh... because belief feels like truth from the inside, and so you cannot trust the inside view unless you are extremely well calibrated on tiny probabilities? So all you are left with is the outside view. If that is what you are asking.

7TurnTrout12dI think you should distrust the other person for other reasons. They may have ulterior motives, for example—as you wrote, they may be lying. There are many social reasons for them to make claims like that. Whatever lies we tell ourselves, they are usually not conscious and explicit. Which is why I found it strange that you stated this claim without further explanation.
[linkpost] Why Going to the Doctor Sucks (WaitButWhy)

Uncharitable summary: 

The Lanby is building a primary care utopia no one else thought of, even though it's obvious, with focus on prevention, quality care and free unicorns.

6Said Achmiz15dHmm. Now, this may be a stupid question, but I can’t seem to find the answer on a skim or Cmd-F of the post: why is this… thing… called “the Lanby”? What is a… lanby?
Morally underdefined situations can be deadly

Are you talking about bounded consequentialism, where you are hit with unknown unknows? Or about known consequences whose moral status evaluates to "undefined"?

2Stuart_Armstrong14dThe second one (though I think there is some overlap with the first).
Worst Commonsense Concepts?

I'd go further than "fact vs opinion" and claim that the whole concept of there being one truth out there somewhere is quite harmful, given that the best we can do is have models that heavily rely on personal priors and ways to collect data and adjust said models.

I don't understand why shminux's comment was down to -6 (as of 11/17). I think this comment is good for thinking clearly. How reality is perceived to you is based off how you collect data, update, and interpret events. You can get really different results by changing any of those( biased data collection, only updating on positive results, redefining labels in a motte and bailey, etc.)  

Going from a "one truth" to a "multiple frames" model helps communicating with others. I find it easier to tell someone

from a semantics viewpoint, 'purpose' is a word c

... (read more)
9Viliam22dThe concept of "one truth" can be an infohazard, if people decide that they already know the truth, so there is no reason to learn anymore, and all that is left to do is to convert or destroy those who disagree. To me this seems like an example of the valley of bad rationality [https://www.lesswrong.com/tag/valley-of-bad-rationality]. If possible, the solution is more rationality. If not possible, then random things will happen, not all of them good.
Why do you believe AI alignment is possible?

I don't believe alignment is possible. Humans are not aligned with other humans, and the only thing that prevents an immediate apocalypse is the lack of recursive self-improvement on short timescales. Certainly groups of humans happily destroy other groups of humans, and often destroy themselves in the process of maximizing something like the number of statues. Best we can hope for that whatever takes over the planet after meatbags are gone has some of the same goals that the more enlightened meatbags had, where "enlightened" is a very individual definitio... (read more)

Attempted Gears Analysis of AGI Intervention Discussion With Eliezer

I don't see any glaring flaws in any of the items on the inside view, and, obviously, I would not be qualified to evaluate them, anyway. However, when I try to take an outside view on this, something doesn't add up.

 Specifically, it looks like anything that looks like a civilization should end up evolving, naturally or artificially, into an unsafe AGI most of the time, some version of Hanson's grabby aliens. We don't see anything like that, at least not in any detectable way. And so we hit the Fermi paradox, where an unremarkable backwater system is a... (read more)

4Multicore22dLow conversion rate to grabbiness is only needed in the model if you think there are non-grabby aliens nearby. High conversion rate is possible if the great filter is in our past and industrial civilizations are incredibly rare.
Improving on the Karma System

My gut feeling is that attracting more attention to a metric, no matter how good, will inevitably Goodhart it. The current karma system lives happily in the background, and people have not attempted to game it much since the days of Eugine_Nier. I am not sure what problem you are trying to solve, and whether your cure will not be worse than the disease.

9alkexr22dThat is a good gut feeling to have, and Goodhart certainly does need to be invoked in the discussion. But the proposal is about using a different metric with a (perhaps) higher level of attention directed towards it, not just directing more attention to the same metric. Different metrics create different incentive landscapes to optimizers (LessWrongers, in this case), and not all incentive landscapes are equal relative to the goal of a Good LessWrong Community (whatever that means). This last sentence comes across as particularly low-effort, given that the post lists 10 dimensions along which it claims karma has problems, and then evaluates the proposed system relative to karma along those same dimensions.
2Yoav Ravid23dI don't think the problem is that people try to game it, but that it's flawed (in the many ways the post describes) even when people try to be honest.
AGI is at least as far away as Nuclear Fusion.

I am no MIRIan, but I see an obvious difference: nuclear fusion is easy and has been around for about 70 years , it's controlled nuclear fusion that is hard. By contrast, there is no needle one has to thread with AGI, once it's achieved, it is self-sustaining, and, arguably, is more like a nuclear bomb than like a nuclear reactor. So that's an argument that AGI is not as hard.

However, there is an opposite argument: self-sustaining long-lasting nuclear fusion has been around for 13.8 billion years, and spontaneously arises in nature, while AGI has never been observed in nature, as far as we know, and "intelligence" in general artificial or natural, has not been observed outside the surface of this planet.

Against the idea that physical limits are set in stone

the idea that physical limits are set in stone

is not what physicists believe. The actual statement is that under a wide variety of conditions the Core Theory is a very accurate description of the universe. And it precludes possibilities like FTL and time travel, among others. By the way, the FTL proposals you mentioned are not really FTL. For example the Alcubierre drive, contrary to popular views, does not enable one to travel arbitrarily far faster than light, only as far as the light propagates before the drive is "engaged" An eternal Alcubierre drive i... (read more)

1maximkazhenkov23dActually, not that much higher. The Oh-My-God particle [https://en.wikipedia.org/wiki/Oh-My-God_particle] had a center-of-mass collision energy of 750 TeV, roughly 60 times that of LHC. I seriously doubt it's a good idea to probe into energy ranges outside of naturally occurring events considering the potential benefits and existential risks.
1Samuel Shadrach1moThanks a lot for this reply - lots to read up. I'm not confident this means we shouldn't try. We didn't observe EM waves being used for communication, but we built it anyways. Nature doesn't have an incentive to build most of things we want to build, or provide useful observations to learn about them. We have to create experiments where we can collect new useful observations, then actually build what we want to. Not all, but some, for sure.
Has LessWrong Been Mind-Killed on the Topic of God and Religion?

The other post is long and meandering, which only works if you are a good writer who expresses their point in koans or something. I couldn't even tell what your point was, or what your personal view on theism and religion is. Or why you are motivated to discuss belief in supernatural in a generally atheist crowd. Like, what are the windmills you are fighting?

Resurrecting all humans ever lived as a technical problem

Regarding collecting crumbs, Brandon Sanderson wrote a fantasy story about it: The Emperor's Soul.

1RomanS1moThank you! Will read it. Another work of fiction on the topic is a science fiction novel by Philip José Farmer called "To Your Scattered Bodies Go [https://en.wikipedia.org/wiki/To_Your_Scattered_Bodies_Go]". Speaking about fiction, the method #2 could create minds indistinguishable from the minds of some fictional characters. For example, one of the generated minds could believe that his name is Harry James Potter-Evans-Verres. He could have the right personality, and vivid memories of all the relevant events from the HPMOR. It will be quite a shock to learn that you're nothing but a fictional character reconstructed from a Harry Potter fanfic by some galaxy-scale posthuman machine. Although I'm pretty sure Harry would support such a project.
Why the Problem of the Criterion Matters

I've always been confused on what people find confusing re the problem of the criterion. If I get it right, you can state it as "you must know how to assess truth in order to know how to assess truth", or something like that. My confusion about confusion lies in squaring that with the idea of embedded agency, where we are a part of partially predictable universe and contain an imperfect model of the universe. Therefore something like a criterion for assessing truth is inherent in the setup, otherwise we would not count as agents.

1TAG1moEverything that an embedded agent does, or needs to do comes, under usefulness. Inasmuch as truth goes beyond usefulness , arguments from agency don't work.
True Stories of Algorithmic Improvement

Another source of speedup is making the right approximations. Ages ago I coded a numerical simulation of a neuromusular synaptic transmission, tracking 50k separate molecules bumping into each other, including release, diffusion, uptake etc that ended up modeling the full process faithfully (as compared with using a PDE solver) after removing irrelevant parts that took compute time but did not affect the outcome.

Self-Integrity and the Drowning Child

Oops... I guess I misunderstood what you meant by "two pieces of yourself".

Anyway, I really like the part 

you failed to understand and notice a kind of outside assault on your internal integrity, you did not notice how this parable was setting up two pieces of yourself at odds, so that you could not be both at once, and arranging for one of them to hammer down the other in a way that would leave it feeling small and injured and unable to speak in its own defense

because it attends to the feelings and not just to the logic: "hammer down the other in a w... (read more)

> I could have designed an adversarial lecture that would have driven everybody in this room halfway crazy - except for Keltham

I... would love to see one of those, unless you consider it an infohazard/Shiri's scissor.

I think this might just mean using the drowning child argument to convince the students they should be acting selflessly all the time, donating all their money above minimal subsistence, etc. 

Regarding pieces of oneself, consider the ideas of IFS (internal family systems). "Parts" can be said to attenuate to different concerns and if one can distract from others then an opportunity to maximize utility across dimensions may be missed. One might also suggest that attenuation to only one concern over time can result in a slight movement towards disintegration as a result of increasingly strong feelings about "ignored" concerns. Integration or alignment, with every part joining a cooperative council is often considered a goal and personification can assist some in more peaceably achieving that. I personally found the suggestion to personify felt weird and false.

Self-Integrity and the Drowning Child

Trying to summarize for those of us not fond of long-winded parables.

  • A single moral agent is a bad model of a human,
  • Multiple agents with individual utilities/rules/virtues is a more accurate model.
  • It's useful to be aware of this, because Tarski, or else your actions don't match your expectations.

I worry that "parts as people" or even "parts as animals" models are putting people on the wrong path to self-integrity, and I did my best to edit this whole metaparable to try to avoid suggesting that any point.

What health-related tips do you have for buying meat?

Ah, got it. I read "meat that was raised responsibly" and assumed that you are against animal farming, but also not willing to reduce meat consumption, and so going for an option in between. Oops.

2adamzerner1moAh I see. I didn't explain that very well, my bad.
What health-related tips do you have for buying meat?

If you consider changing your meat acquisition habits based on health and ethical reasons, I presume you have considered changing your animal products consumption habits based on the same reasons, and rejected the idea on some grounds, right? 

2adamzerner1moI'm having a little trouble understanding what you are asking. I don't see eating meat as unethical, at least not in the grand scheme of things, so I wasn't considering eliminating it from my diet.
Lies, Damn Lies, and Fabricated Options

Yeah, you are right, the only hope of getting somewhere is when you address their true objections. That's not easy though because they might not even be aware of what they are, and refuse to acknowledge it when pointed out (again, see the examples in the thread I linked), often because acknowledging them would clash with their self-image. Successfully addressing their real arguments, not the chaff on top, is a difficult skill. If you can do it, it would feel like magic. Or a superpower.

2AllAmericanBreakfast2moThis sounds like the proper use of empathy, as a tool for constructive exchange of perspectives. Alice suspects Bob is not stating his true objecting to her idea. She tries to simulate the kind of mental experience that might cause Bob to resist her point of view. She bases this simulation on what she knows of Bob's background and personality. This simulation is an example of empathy. Alice can use it to predict what sort of response might shift Bob's point of view toward her own in a way that he would reflectively endorse. Improper use of empathy would be creating and broadcasting an empathetic simulation that may be detached from any particular relationship. It no longer serves to make one person understand another. Instead, it creates concern for a fictional character. This character is mistaken for a real person, or group of people. Concern for this fiction motivates real action. The action it motivates is a tool that the broadcasters of this simulation can use for their own ends. When we speak of empathy as a problematic motivating force, I think that this is the underlying mechanism. Perhaps it would be good if rationalists promoted this distinction between relational empathy and empathy for a fiction, and focused on practicing relational empathy. It might indeed be a superpower.
Lies, Damn Lies, and Fabricated Options

“You just pretended to care about price gouging, so why did you do that” seems like a good way to confront such statements, at least some of the time.

I... don't think this works, not even in the LW circles, as some of the reaction to my old post show https://www.lesswrong.com/posts/a4HzwhvoH7zZEw4vZ/wirehead-your-chickens

5AllAmericanBreakfast2moYou are right! I worded that very poorly. Here’s what I meant. If someone has pretend noble motives and true ignoble motives, I sometimes get into a failure mode where I act as if the noble motives were their true motives. Then I try to show how their proposed solution will fail to achieve their pretend noble motives. There’s some sort of idea of “showing them up” behind my own behavior here, and also some idea that this behavior of mine is a “noble” form of bravery. An alternative approach in such circumstances is to become clear in my own mind that the other person has pretend motives. Then I can interpret their behavior or their proposition in light of that. This could also be useful for self-analysis. This is what I meant when I said “confront,” but this was the wrong word to choose for that! Thanks for pointing that out.
My experience at and around MIRI and CFAR (inspired by Zoe Curzi's writeup of experiences at Leverage)

If you manage to get through that, maybe you can summarize it? Even Logan's accessible explanation makes my eyes glaze over.

I got through that page and… no, I really can’t summarize it. I don’t really have any idea what Kant is supposed to have been saying, or why he said any of those things, or the significance of any of it…

I’m afraid I remain as perplexed as ever.

Lies, Damn Lies, and Fabricated Options

I just wanted to mention that you assume consequentialist thinking, specifically of the type "what should we do to change X for the better?" This is not at all how most people think. "Price gouging is unfair" is enough to pass a legislation, without heeding the consequences. "Abortion is against a sacred rule from God" is enough to fight to prohibit it. "But I can change my mind" is argument enough to two-box. And I'm not even touching issues where people don't reason at all, or, like politicians, optimize something other than the stated goal.

4AllAmericanBreakfast2moThis is a great point. When I feel frustrated with a faulty conversation, I often start by projecting my own motivations onto the other person. Or that their explicitly stated goals are their real goals. Even if I know that this is wrong, I try to act as if that were so. “You say you’re doing this for the good of humanity? Then I’m going to respond as if that’s really what you cared about, and that you’re going about it badly, even if I know deep down that you’re lying about your motives.” It’s this perverse form of “bravery.” There’s a class of shallow incoherent lies that we’re all supposed to know are shallow lies, and yet act as if they were deep coherent truths. It can feel like bravery to “expose” the lie by taking it literally and showing how incoherence is the result. An alternative is to focus not on bravely confronting the lie in order to expose the object level truth, but to focus on cannily understanding the motive and function of the lie itself. “Did you lie to me just now, and what’s the truth of the matter” is a very different question from, “I think you just lied to me, so why did you do that?” “You just pretended to care about price gouging, so why did you do that” seems like a good way to confront such statements, at least some of the time.

Yeah, I stumbled over the price-gouging example for similar reasons. After two background examples of inconceivable worlds, the world of that story sounded similarly incoherent to me - I could not write it in 2021.

Mainly, a world where lawmakers frequently ban pricegouging is a world where it's probably in their interest to do so. So to posit that they ban it because they're somehow mistaken about the consequences sounds wrong to me.

Rather than the options in the story, in my model they follow Asymmetric Justice, social reality, dysfunctional incentives in... (read more)

9Duncan_Sabien2moOoh, excellent point. I don't think I assume that others are actively trying and failing at consequentialist thinking (I think if I'd been queried on this, I would have said words that largely match your perspective/predictions) but I do think that effective people and trying-to-be-effective people should definitely at least often be in a consequentialist mode. And so I think I was pointing at something like "evaluate the stuff they're proposing from a consequentialist lens [regardless of whether they themselves are doing so]."
Is moral duty/blame irrational because a person does only what they must?

You might find a recent book review useful: https://www.lesswrong.com/posts/jr3sxQb6HDS87ve3m.

I commented on the topic there.

Also, see my favorite quote by Ambrose Bierce on the subject, from another post:

"There's no free will," says the philosopher; "To hang is most unjust."
"There is no free will," assents the officer; "We hang because we must."

Basically, all an embedded agent does is discovering what the world is really like. There is no ability to steer the world in a desired direction, but also no ability to not to steer it. We are all NPCs, but most o... (read more)

4cata2moYour link is broken because the comment thought the trailing period was part of the link. Here's the proper link: https://www.lesswrong.com/posts/jr3sxQb6HDS87ve3m [https://www.lesswrong.com/posts/jr3sxQb6HDS87ve3m]
Do you think you are a Boltzmann brain? If not, why not?

Sean Carroll talked about this just recently, in the context of Bayesianism https://www.preposterousuniverse.com/podcast/2021/09/16/ama-september-2021/

It's around 2:11:08, or Ctrl-F in transcript.

Book Review: Free Will

It is tricky to talk in self-consistent ways about lack of free will. Obviously any kind of prescriptivism is right out: since there is no free will you can't consciously steer the future in any specific direction, you can only have an illusion of it. It is possible to talk about lack of free will in descriptive terms, however. For example, a statement like "one should hold people accountable even though free will does not exist, for the benefit of the society as a whole", can be expressed as "societies where people are held accountable for their actions t... (read more)

3Ape in the coat2moI had originally expected exactly that from the book! But, in my opinion, it didn't turn out to be the case. I'm pretty sure that Harris could have done it if he intended to. My guess is that he wanted to be more relatable and appealing to a layman reader rather than polish his speech too much.
1[comment deleted]2mo
Steelman arguments against the idea that AGI is inevitable and will arrive soon

Without technological artifacts, it would be very hard to identify such an intelligence.

Indeed. And I think it's a crucial question to consider in terms of identifying anything intelligent (or even alive) that isn't "like us". 

Steelman arguments against the idea that AGI is inevitable and will arrive soon

Somewhat off topic...

A Bayesian superintelligence? There is no natural example.

How would you tell if some "natural" phenomenon is or is not a Bayesian superintelligence, if it does not look like us or produces human-recognizable artifacts like cars, buildings, ships etc?

1RomanS2moThat's a very good question, and I'm not sure how to answer it. If we encounter some intelligence that can achieve feats like the aforementioned Relativity-from-grass, then it would confirm that a Bayesian superintelligence is possible. It could be an exceptionally intelligent generalist human, or perhaps a highly advanced alien AI. Without technological artifacts, it would be very hard to identify such an intelligence.
“Who’s In Charge? Free Will and the Science of the Brain”

Right, good point, I think it's very close. I guess when you are a professional philosopher stating the obvious it often comes across as profound.

Though I'm trying to do more than to just state it, but to construct a model of the meta-problem: that it is a side effect of the specific optimization computation. I wish I could tease out some testable predictions from this model that are different from alternatives. 

“Who’s In Charge? Free Will and the Science of the Brain”

How much free will does a monkey have? A cat? A fish? An amoeba? A virus? A vapor bubble in a boiling pot? A raspberry shoot jockeying for a sunny spot? An octopus arm? A solar flare? A chess bot?

Hint: the same amount as a human. 

Answer: We just happen to have a feeling of free will that is an artifact of some optimization subroutine that runs in our brains and is not fully available to introspection. Do octopuses have that feeling? Chess bots? That question might get answered one day once we understand that how the feeling of free will is formed in humans.

3G Gordon Worley III3moFWIW, sounds like you're pointing at what Chalmers calls the meta problem of consciousness: why do we think there is a hard problem of consciousness?
1Richard Zander3moWe are free to think we are free. Freedom is the opiate of the optimists, so be sour and you will be free of freedom.
What is difference between thoughts and consciousness?

How would you define thoughts? Is it something you can notice happening, as opposed to a feeling or an urge, that just bubbles up? 

4Olitness4moNoticing would be thoughts, but instinctive urges like fear or hunger I wouldn't probably consider to be thoughts unless it is being noticed and processed somehow, because this can be happening automatically without consciousness and awareness. But once that urge or feeling is noticed then there is some "image" or "thought" of it, some indirect interpretation of the event. In such case, I am thinking of consciousness as way of processing complex thoughts.
When Programmers Don't Understand Code, Don't Blame The User

Javascript is indeed notoriously opaque in terms of assigning "self" to a function call. There are multiple ways to do it more explicitly, including prototype inheritance, using Function.prototype.bind() etc., all being workarounds for passing and calling "selfless" methods. So yeah, I agree with your main point.

Erratum for "From AI to Zombies"

I think that rationality as a competing approach to the scientific method is a particularly bad take that leads a lot of aspiring rationalists astray, into the cultish land of "I know more and better than experts in the field because I am a rationalist". Data analysis uses plenty of Bayesian reasoning. Scientists are humans and so are prone to the biases and bad decisions that instrumental rationality is supposed to help with. CFAR-taught skills are likely to be useful for scientists and non-scientists alike. 

Erratum for "From AI to Zombies"

I agree that the point was not to teach you physics. It was a tool to teach you rationality. Personally, I think it failed at that, and instead created a local lore guided by the teacher's password, "MWI is obviously right". And yes, I think he said nearly as much on multiple occasions. This post https://www.lesswrong.com/posts/8njamAu4vgJYxbJzN/bloggingheads-yudkowsky-and-aaronson-talk-about-ai-and-many  links a video of him saying as much: https://bloggingheads.tv/videos/2220?in=29:28

Note that Aaronson's position is much weaker, more like "if you we... (read more)

5philh4moThis could well be the case, I have no particular opinion on it. (To clarify, I take it "as much" here means "MWI is obviously right", not "the sequence failed at teaching rationality".) So the distinction I've been making in my head is between a specific interpretation called MWI, and multi-world interpretations in general. That is, I've been thinking there are other interpretations that we don't call MWI, but which share the property of, something like, "if it looks to you like your observations are collapsing quantum superposition, that's just what happens when you yourself enter superposition". My (again, vague) understanding is that Eliezer thinks "some interpretation with that property" is obviously correct, but not necessarily the specific interpretation we might call MWI. But if I'm wrong about what MWI means, and it just refers to all interpretations with that property (or there is/can be only one such interpretation), then Eliezer certainly thinks "this is obviously by far the best hypothesis we have" and I agree that it sounds like he also thinks "this is obviously correct". And it seems like Scott is using it in the latter sense in that blog post, at least. (And, yeah, I find Eliezer pretty convincing here, though I'm not currently capable of evaluating most of the technical arguments. My read is that Scott's weaker position seems to be something like, "okay but we haven't looked everywhere, there are possibilities we have no particular reason to expect to happen but that we can't experimentally rule out yet".)
Technical Predictions Related to AI Safety

I guess you don't mean simulating the relevant parts of the rat brain in silico like OpenWorm, but "a rat-equivalent Bayesian reasoning machine out of silicon", which is probably different.

2lsusr4moYes.
Technical Predictions Related to AI Safety

Do you think it's reasonable to push for rat-level AI before we can create a C.Elegans-level AI?

2lsusr4moI don't know how to measure the intelligence of a C. Elegans. If I could it would come first.
6philh4moSeems important to note here that the point of the quantum physics sequence, in context, is not to teach you physics. So reading a physics book doesn't give you what the sequence is intended to, and in particular it doesn't take the place of errata for that sequence. At least, it's been a long time since I've read it, but I'm pretty confident the point isn't to teach you physics. I'm less confident of this, but to my memory the point is something like, "here's a worked example of a place where rationality can guide you to the truth faster than the scientific establishment can". If Eliezer was wrong here, and rationality didn't help guide him to the truth, I think that would be actually really important. But I also think that having better interpretations of QM then MWI ten years later isn't obviously a failing, since - again, to my memory - he didn't say that MWI was obviously correct but that it was obviously the best we had, or maybe even just obviously better than Copenhagen.
Against "blankfaces"

I think the Umbridge version is uncontroversial: someone who uses existing rules or creates new rules (like the lifeguard in the Scott's description, or the agencies making it intentionally hard to get reimbursed) to disguise their real intentions, that have nothing to do with following the rules and everything with achieving nefarious goals, be it torturing HP, getting rid of a kid they don't like, or maybe getting a bonus for minimizing expenses. 

1Measure4moAlso, someone wouldn't have to be fully as bad as Umbridge to qualify as "sufficiently Umbridge-like".
The Myth of the Myth of the Lone Genius

I don't know if that last paragraph is the author's view, and whether there is any evidence/consensus for it. I go by what I see, and this is a person possessed to overcome obstacles over and over again. Musk is an extreme example, but in general all the classic tech moguls are "natural heroes" in that sense. The burning need inside to do "world optimization" cannot be quenched.

The Myth of the Myth of the Lone Genius

I like the post, but just to pick on one thing

(4) There are no such things as geniuses, and even if there were you are not one of them. 

There are two parts to this, the first "There are no such things as geniuses" is not proclaimed by anyone serious, the second, "you are not one of them" is basically correct if you rephrase it as "if you need to ask whether you are one of them, you are not."

The old wizard sighed. His half-glasses eyes looked only at her, as though they were the only two people present. "Miss Granger, it might be possible to discourage witches from becoming Charms Mistresses, or Quidditch players, or even Aurors. But not heroes. If someone is meant to be a hero then a hero they will be. They will walk through fire and swim through ice. Dementors will not stop them, nor the deaths of friends, and not discouragement either."

"Well," Hermione said, and paused, struggling with the words. "Well, I mean... what if that's not actu

... (read more)
Load More