Open Thread, November 1 - 7, 2013

Can someone explain nanotech enthusiasm to me? Like, I get that nanotech is one of the sci-fi technologies that's actually physics-compliant, and furthermore it should be possible because biology.

But I get the impression that among transhumanist types slightly older than me, there's a widespread expectation that it will lead to absolutely magical things on the scale of decades, and I don't get where that comes from, even after picking up Engines of Creation.

I'm thinking of, e.g. Eliezer talking about how he wanted to design nanotechnology before he got into AI, or how he casually mentions nanotechnology as being one of the big ways a super-intelligent AI could take over the world. I always feel totally mystified when I come across something like that, like it's a major gulf between me and slightly older nerds.

9Armok_GoB11y

Trying for minimal technicalities: There are at least 3 different technologies with not much surface-level usage similarities referred to as "nanotech". Assemblers: basically 3d printers, but way more flexible and able to make things like food, robots, or more assemblers. Materials: diamondoids, buckytubes, circuitry. We already have some of these really, it's just that we'd get more kinds of them, and they'd be really cheap to make with a nanotech assembler. Stronger, faster, more powerful versions of what modern tech already can do. Nanobots, particularly medical: Basically can do all the things living cells can do, but better, and also being able to do most of the things machines can do, and commandable in exact detail. There are also a number of different ways they'd grant immortality enough that they are almost sure to do so even if most of them end up not working out. Now you can ask questions about each one of these in order, with more specifics.

3ChrisHallquist11y

The question is about all these technologies - though it's about 2 mainly insofar as 2 is an extension of 1. So the question is why expect any of these technologies to mature on a timescale of decades? (Or, assuming FOOM, why assume they'd be relatively low-hanging fruit for a FOOMing AI, such that "trick humans into building me nano assemblers" is a prime strategy for a boxed AI to escape?)

5Armok_GoB11y

As I said, 2 is already here, and it's becoming more here gradually. For 3, we have a proof of concept to rip of: biological cells. Those also happens to have a specialized assembler in them already; the ribosome. And we can print instructions for it already. There's only 1 problem left and that's the protein folding problem. The protein folding problem is somewhat rapidly made progress on software wise, and even if that were to fail it won't be all that long before we ca simply brute force it with computing power. Now, the other kinds of nanobots are less clear. The assembler (1) is trickier; however, Drexler already sorta made a blueprint for one I think, and 3 will help a great deal with it as well. For the fooming, it's the 3 one, and ways to use it. As I said we already have the hardware, and things like the protein folding problem is exactly what an AI would be great at. Once it's solved that it has full control over biology and can essentially make The Thing and/or a literal mind control virus, and take over that way.

3ChrisHallquist11y

Okay, so one sub-piece of puzzlement I have is why talk of protein folding as a problem that is either solved or unsolved - as if we (or more frighteningly, an AI) could suddenly go from barely being able to do it to 100% capable. I was also under the impression that protein folding was mathematically horrible in a way that makes it unlikely to be brute forced any time soon, though I just now realized that I may have been thinking of the general problem of predicting chemistry from physics, maybe protein folding is much easier.

[-]Douglas_Knight11y110

Predicting chemistry from physics should be easy with a quantum computer, but appears hard with a classical computer. Often people say that even once you make a classical approximation, ie, assume that the dynamics are easy on a classical computer, that the problem of finding the minimum energy state of a protein is NP-hard. That's true, but a red herring, since the protein isn't magically going to know the minimum energy state. Though it's still possible that there's some catalyst to push it into the right state, so simulating the dynamics in a vacuum won't get you the right answer (cf prions). Anyhow, there's some hope that evolution has found a good toolbox for designing proteins and that if can figure out the abstractions that evolution is using, it will all become easy. In particular, there are building blocks like the alpha helix. Certainly an engineer, whether evolution or us, doesn't need to understand every protein, just know how to make enough.

I think the possibility that a sufficiently smart AI would quickly find an adequate toolbox for designing proteins is quite plausible. I don't know what Eliezer means, but the possibility seems to me adequate for his arguments.

0ChrisHallquist11y

Ah, that's helpful.

2DanielLC11y

I'm not sure protein folding can be brute forced without quantum computers. There's too many ways for it to fold. In real life, I'm pretty sure quantum tunneling gets involved. Simulations have worked, but I there's a limit to that.

7Eliezer Yudkowsky11y

Try Nanosystems perhaps.

3Cyan11y

An analogy might help give a sense of scale here. This isn't an argument, but it hints at the scope of the unknown unknowns in nanotech space. Here on our macroscopic scale, some wonders wrought by evolution include the smasher mantis shrimp's kinetic attack, a bee hive's eusocial organization, the peregrine falcon's flight speed, and the eagle's visual system. But evolution is literally mindless -- by actually knowing how to do things, human engineering created electromagnetic railguns, networks of international trade, the SR-71 Blackbird, and the Hubble telescope. Now apply that kind of thinking to "because biology" on the nano scale...

3mwengler11y

Consider a machine as smart as a cellphone but the size of a blood cell. In a sense, a protein or a drug is a smart molecule. It keys in to a very limited number of things and ignores the rest. There are many different smart proteins or smart molecules to be used as drugs with many different purposes. Even so, chemotherapy, for example, is primarily about ALMOST killing everything while differentially being a bit more toxic to the cancer cells. Now increase the intelligence of ths smartest molecule by 10 fold, 100 fold, 1000 fold. Perhaps you give it the ability for a simple 2-way communication to the outside world. If its intelligence is increased, there should be MANY ways to allow it to distinguish a tumor, a micro-tumor, a cancerous cell, from all the good things in your body. All the sudden, the differential toxicity of "chemo" therapy (now nanotherapy) will be 10, 100X as high as it is for smart molecules. Now consider these smart little machines doing surgery. Inoperable tumor? Not inoperable for a host of machines the size of bloodcells that will literally be able to operate on the most remote of tumors from inside of them. Tendency towards obesity? How hard will it be to have a system of nanites that screw with your metabolism in such a way to eliminate all the stored fat in cells until told that, or until they measure that, we are down to a good level. These are just a few stories from medicine. I expect anybody who does not wish to get sick and die would be enthusiastic about these, but YMMV.

2Lumifer11y

Probably comes from Neal Stephenson's The Diamond Age: Or, A Young Lady's Illustrated Primer :-)

[-][anonymous]11y130

I definitely have found that this forum is NOT immune to fictional evidence.

2Douglas_Knight11y

I'm pretty sure that the people Chris is talking about are Stephenson's source, not vice versa.

1ChrisHallquist11y

Eliezer definitely definitely seems to have caught the nano-enthusiasm bug pre-Diamond Age, but maybe the book had a big impact on other people?

2mwengler11y

The book had a gigantic impact on me. In a broad range of ways from hypertext through nanotech through various schemes for social organization and the long list of human needs such organization serves.

0Viliam_Bur11y

One of many things I liked was an illustration that economical improvement is not enough to make people live well. If I remember correctly, in the fictional world the food was free; anyone could go to a public matter-builder and take food from it. Yet some children were hungry... because their parents didn't care enough for them to get outside of the house and bring them the food. Moral of the story: however good situation you have, humans can make it bad by simply not caring even the smallest. (Unless we get to a situation where humans are replaced by robots completely.) This seems to me like a hyperbole of the world we have now. Economically, the life in developed countries is so great that for most people who lived in the past it would be almost like a paradise. Yet we have a lot of suboptimality simply because people don't care. (Maybe it's because the wealth made social pressure less relevant, and many people just naturally don't give a fuck about anything, and without the social pressure now they don't even have to pretend.) The good life does not make us automatically stronger; it often makes us lazy. I believe the possibility is still there, but without outside pressure most people don't care about becoming stronger.

1Douglas_Knight11y

Who, specifically are you talking about? I'm thinking of the extropians, who coined "transhumanism." I'm not sure of the timeline; the original group was definitely into MNT before Stephenson, but maybe they expanded a lot after him, and maybe that was because of him.

0passive_fist11y

Perhaps the reason is that the ideas we're used to nowadays - like reconfiguring matter to make dirt and water into food or repair microcellular damage (for example, to selectively destroy cancer tumors) - were absolutely radical and totally unheard of when they were first proposed. As far as I know, Feynman was the first to seriously suggest that such a thing was possible, and most reactions to him at the time were basically either confusion, disbelief, or dismissal. Consider the average technologist in 1950. Hand-wound computer memories were state of the art, no one knew what DNA looked like, famines seemed a natural part of the order of things, and as far as everyone knew, the only major technological difference between the present and the future was maybe going to be space travel. Now someone comes along and tells you that there could be this new technology that allows you to store the library of congress in the head of a pin and carry out any chemical reaction just by writing down the formula - including the chemical reactions of life. The consequences would be, for instance, the ability to feed everyone on the planet at basically free. To you, such a technology would seem "Indistinguishable from magic." Would it be a dramatic inferential step to then say that it could do stuff that literally is magic? Nanotechnology never promised magic, of course. All it promises is the ability to rearrange atoms into a subset of those structures allowed by physics (a subset that is far larger than our current technology can do, but a subset nonetheless). It promised nothing more, nothing less. This is in itself dramatic enough, and it would allow all sorts of things that we probably couldn't imagine today.

[-]NancyLebovitz11y230

New ligament discovered in the human knee as a result of surgeons trying to figure out why some people didn't recover fully after knee injuries.

I'm tempted to deduce "Keep paying attention, you never know what might have been missed"-- I really would have expected that all the ligaments had been discovered a long time ago.

Another conclusion might be "Try to solve real problems, you're more likely to find out something new that way than by just poking around."

[-][anonymous]11y120

Does someone have the medical knowledge to explain how this is possible? My layperson guess is that once cut up a knee, you can more or less see all the macroscopic structures. Did they just think it was unimportant?

5NancyLebovitz11y

My layperson guess is that once you're told what to expect to see, you stop looking. This makes Eliezer's weirdtopia idea of science being kept secret so as not to spoil people's fun of discovery more interesting-- it's not just that people would independently discover the same things (and I wonder what the protocol for sharing information would be), given enough time and intelligence, much more might get discovered.

2NancyLebovitz11y

Someone who seemed a bit better informed And that comment is answered by: Which is interesting-- sometimes studying things in extreme detail "just because" (probably because the object of study has high status-- consider early observations of the planets) can pay off big.

2Vaniver11y

The "new ligament discovered" angle gets less impressive (to me, at least) when I read this part:

[-]gwern11y110

I'm more impressed, actually, in terms of the unevenness of progress - it took ~134 years to confirm his postulate? It's not like corpses were unavailable for dissection in 1879.

2Douglas_Knight11y

It inspires more awe at our collective failures, but suggests that we should not be so impressed with the new people as if they had a method that would make us sure that we hadn't missed even more ligaments.

[-]Manfred11y100

The media giveth sensationalism, and the media taketh away.

reddit - "So that "new" ligament? Here's a study from 2011 that shows the same thing. It's not even close to a new development and has been seen many times over the past 100 years." Summary quote: "The significance of the Belgian paper was to link [the ligament's] functionality to what they called "pivot shift", and knee reinjuries after ACL surgery. The significance of this paper, I believe, is that in the near future surgeons performing these operations will have an additional ligament to inspect and possibly repair during ACL surgery, which will hopefully reduce recurrence rates, and likely the rates of developing osteoarthritis in the injured knee down the line."

0NancyLebovitz11y

sigh

-4Mitchell_Porter11y

Another example of low-hanging gristle on the knee of trollage.

[-]Viliam_Bur11y220

So I get home from a weekend trip and go directly to the HPMOR page. No new chapter yet. But there is a link to what seems to be a rationalist Death Note.

The way he saw it, the world was a pretty awful place. Corrupt politicians, cruel criminals, evil CEOs and even day-to-day evil acts made it that way, but everyday stupidity ensured it would stay like that. Nobody could make even a simple utility calculation. The only saving grace was that this was as true for the villains as for the heroes.

I am going to read it. Here are my next thoughts:

So, it seems like Eliezer succeeded to create a whole new genre of literature: rationalist fiction. Nice job!

Wait, what?! Is "a story where the protagonist behaves rationally" really a new genre of literature? There is something horribly wrong with this world if this is true.

Discussing with my girlfriend about which stories should be x-rationalizated next, she suggests HPMOR. Someone should make a HPMOR fanfic where the protagonist is even more rational than the rational Harry. Would that lead to a spiral of even more and more rational heroes?

What exactly could the MoreRational!Harry do? It would be pretty awesome if he could someho... (read more)

[-][anonymous]11y350

Discussing with my girlfriend about which stories should be x-rationalizated next, she suggests HPMOR. Someone should make a HPMOR fanfic where the protagonist is even more rational than the rational Harry.

An idea came to my mind. Would it be possible to make a story in which Harry is less intelligent, in a way that he would score less in an IQ test for example, but at the same time more rational? HJPEV seems to be a highly intelligent prodigy even without the rationality addition. I would like to see how a more normal boy would do.

0lmm11y

One could argue that he appears intelligent only because he's spent his life so far learning effectively.

1gattsuru11y

Rationalist!Harry is calibrated to match the knowledge and recall of a 34-year-old autodidact. Even presuming a very friendly environment and that said 34-year-old autodidact's training was not optimal, I just don't think there's enough time. I can buy a 10 year old reading Ender's Game and The Lord of the Rings and maybe even Lensmen. It's a bit harder to imagine one that would consider wanting to want the math behind proving N=NP, nevermind going further than that.

1CAE_Jones11y

I believe it's been stated somewhere that EY draws primarily on the skills he had around 18 and intentionally keeps things from beyond that out of Harry's reach. So Harry is more like a brilliant high school student than an adult (and, extra seven years worth of rationalist training aside, the way he approaches problems is a lot like a middle schooler with superpowers: "I can win, you can't, deal with it, 'cause I'm awesome and you know it." Which manages to annoy everyone in-universe and out.). Time isn't really a problem, either, if Harry has nothing else to occupy his time; exercise and social interaction are apparently not his thing, and he wound up out of the public school system after a few years, so he really does have way more time than most kids his age to read all the books. And he has that mysterious dark side and that sleeping disorder, whatever those contribute. The other strangely adult-like children, however, are not so easily justified. (Draco gets most of those complaints, from what I've read.)

0lmm11y

I wanted the maths behind relativity and QM at age 10. And I wasted a lot of time in school.

[-]fubarobfusco11y340

Is "a story where the protagonist behaves rationally" really a new genre of literature?

I think what you are referring to here is "a story where the protagonist describes their actions and motivations using rationality terminology" or maybe "a story where the rational thinking of the protagonist motivates the plot or moves it along". At least some of the genre of detective fiction — early examples being Poe's Auguste Dupin stories — would be along these lines.

Stories where protagonists behave rationally (without using rationality terrminology) wouldn't look like stories about rationality. They look like stories where protagonists do things that make sense.

[-]ChrisHallquist11y140

Wait, what?! Is "a story where the protagonist behaves rationally" really a new genre of literature? There is something horribly wrong with this world if this is true.

Yup. At least sort-of. If you haven't read Eliezer's old post Lawrence Watt-Evans's Fiction I recommend it. However, conspicuous failures of rationality in fiction may be mostly an issue with science fiction and fantasy. If you want to keep the characters in your cop story from looking like idiots, you can do research on real police methods, etc. and if you do it right, you have a decent shot at writing a story that real police officers will read without thinking your characters are idiots.

On the other hand, when an author is trying to invent an entire fictional universe, with futuristic technology and/or magic, it can be really hard to figure out what would constitute "smart behavior" in that universe. This may be partly because most authors aren't themselves geniuses, but even more importantly, the fictional universe, if it were real, would have millions of people trying to figure out how to make optimal use of the resources that exist in that universe. It's hard for one person, however smart, to compete with that.

For that matter, it's hard for one author to compete with an army of fans dissecting their work, looking for ways the characters could have been smarter.

[-]Ishaan11y130

which stories should be x-rationalizated next

This leads to another comment on rationalist fiction: Most of it seems to be restricted to fan-fiction. The mold appears to be: "Let's take a story in which the characters underutilized their opportunities and bestow them with intelligence, curiosity, common sense, creativity and genre-awareness". The contrast between the fanfic and the canon is a major element of the story, and the canon an existing scaffold which saves the writer from having to create a context.

This isn't a bad thing necessarily, just an observation.

Wait, what?! Is "a story where the protagonist behaves rationally" really a new genre of literature?

So, the question becomes, how do you recognize "rationalist" stories in non-fan-fic form? Is it simply the presence of show-your-work-smart characters? Is simply behaving rationally sufficient?

Every genre has a theme...romance, adventure, etc.

So where are the stories which are, fundamentally, about stuff like epistemology and moral philosophy?

1MathiasZaman11y

I'd say the difference between "rationalists" stories and "non-rationalist" stories lies in the moral of the story, of the lessons the story teaches you. I don't think it's a genre in the same way romance or adventure are. It's more of a qualifier. You can have rationalist romance novels or rationalist adventure movies. Although you could argue that it is a genre. While discussions about "genre" are often hard, since people don't tend to agree on what makes something a genre. But rationalist fiction already has a couple of genre conventions, such as no-one being allowed to hold the idiot ball or teaching the audience new and useful techniques for overcoming challenges.

0Viliam_Bur11y

That's a great question. (And related to how to recognize rational people in real life.) I'd say that there must be some characters which are obviously smarter than most people around them. Because that's what happens in real life: there is the bell curve, so if all your characters are on a similar level, then the story is (a) not realistic, (b) the characters are selected by some intelligence filter which should be explicitly mentioned, or (c) the characters are all from the middle of the bell curve. Also, in real life the relative power of intelligent people is often reduced by compartmentalization, but this reduction would be much smaller for a rationalist hero. So I'd say it's behaving rationally while most of other people aren't. The character should somehow reflect on the stupidity of others; whether by frustration from their inability to cooperate, or by enjoyment of how easily they are manipulated.

4Ishaan11y

I'm not sure I like that criteria. By that criteria alone, the original death note anime was rationalist fiction (judging by the first half), as is Artemis Fowl, Ender's Game, and to some extent even Game of Thrones. There are a lot of stories where some characters are much smarter than others and know it, but consuming these works won't teach anyone how to be smarter. (Other than the extent to which reading good fiction in general improves various things) None of these stories actually teach the reader anything about epistemology. Even the linked Death Note fan fic...it uses rationality-associated words like "utility" and "prior" but if I didn't already know what those words meant I would have just come away confused. (Granted, it's still early in the story - but even so) Also, it hasn't yet broken the conceit of the story (For example, even a normal person of average intelligence would be surprised and curious about the existence of the supernatural, and would investigate that). I'd say that breaking the story conceit is another feature of rationalist fanfiction stories that has nothing to do with the character's intelligence.

1Viliam_Bur11y

Well, I was disappointed with the Death Note fan fic, because it doesn't seem to have added value beyond the original story. And I agree that exploring the supernatural should be a high priority for a rational person, once the supernatural is experimentally proven. Would it be so difficult to ask Ryuk whether there are additional magical items that could also be abused? I guess Ryuk would use an excuse of having "rules" against that, but at least it's worth trying. Having a rational superhero is a necessary condition for a rationalist story, not a sufficient condition. Ender's Game could be a rationalist literature if it explained Ender's reasoning better, and if Ender strategically tried to improve his understanding of the world. Okay, another necessary condition is not just that the superhero is super smart, but also that the super smartness is at least partially a result of a good strategy, which is shown to the reader.

9MathiasZaman11y

I think there's a difference between what I've been describing as rationalist!fic (or rationalist!fiction) and fiction in which the -agonists (PCs is the right terminology, I guess) are rational/clever. Rationalist!fic doesn't just feature rationalist characters, they're expressively written to teach the audience about rationality. Examples: * Doctor Who features a sufficiently advanced alien who is, within the rules of the universe, pretty rational (in that he is good at reaching his goals). The message of the show however, is not: "be clever and rational," it's: "humanity is awesome and you should feel some wonder about the universe." Not rationalist!fic. * The Conqueror's Shadow, by Ari Marmell features rationalist agonists and the message the audience goes away with is: "be clever and creative when it comes to reaching worthwhile goals." Rationalist!fic.

6DanielLC11y

Erfworld is a piece of rationalist fiction not related to HP:MoR. It was discussed on here a while back. There must be others. Also, I suggest calling it Rational!Rational!Harry.

2hyporational11y

I get your sentiment, but I don't think this is true. Anyways, wouldn't this just mean that rational minds usually pursue other goals than writing fiction? Not saying that there shouldn't be rationalist fiction, but this doesn't sound like such a bad state of affairs to me. I haven't read HPMOR. Do I have to know anything about the HP universe to enjoy this thing? Will I learn anything new if I've read the sequences?

3Viliam_Bur11y

I guess you don't need to know anything from the HP canon. It could perhaps be even more interesting that way. I don't think you would learn new information. It might have a better emotional impact, but that is difficult to predict. I would consider the world better if there were more rational people sharing the same values as me. We could cooperate on mutual goals, and learn from each other. Problem is, rational people don't just appear randomly in the world. Okay, sometimes they do, but the process if far from optimal. If there is a chance to make rationality spread more reliable, we should try. But we don't exactly know how. We tried many things, with partial success. For example the school system -- it is great in taking an illiterate peasant population and producing an educated population within a century. But it has some limits: students learn to guess their teachers' passwords, there are not enough sufficiently skilled teachers, the pressure from the outside world can bring religion to schools and prevent teaching evolution, etc. And the system seems difficult to improve from inside (been there, tried that). Spreading rationality using fiction is another thing worth trying. There is a chance to attract a lot of people, make some of them more rational, and create a lot of utility. Or maybe despite there being dozens of rationalist fiction stories, they would all be read by the same people; unable to attract anyone outside of the chosen set. I don't know. The point is, if you are rational and you think the world would be better with more rational people... it's one problem you can try to solve. So before Eliezer we had something like the Drake equation: how many people are rational × how many of them think making more people rational is the best action × how many of them think fiction is the best tool for that = almost zero. I am curious about the specific numbers; especially whether one of them is very close to zero, or whether it's merely a few small num

1hyporational11y

I'd probably want more people who share my values than more rational people. Rational people who share my values is better. Rational people who don't share my values would be the worst outcome. I don't think the school system was built by rationalists, so I'm not sure where you were going with that example. How effective has fiction been in spreading other ideas compared to other methods?

2ChristianKl11y

Given that the spell never failed in the past, I'm not sure that it would have been rational to use a knife.

1gattsuru11y

In addition to the other's already listed, DataPacRat's Myou've Got To Be Kidding Me follows a perspective of a character thrown into a setting and trying to analyze the basic rules in order to optimize them. There are some interesting concepts, but I don't know that I can recommend it : It has not been updated in over a year, and was part of some big conglomeration of fanfic writers which had some pretty widely varying quality (although thankfully nothing necessary to Myou've plotline).

0maia11y

Fbzr sna gurbevrf ubyq gung Dhveeryzbeg vf hfvat Ibyqrzbeg nf n chccrg vqragvgl va beqre gb tnva cbjre. Fb Ibyqrzbeg'f erny tbny vfa'g gb xvyy Uneel; vg'f gb unir n qenzngvp fubjqbja gung trgf ybgf bs nggragvba naq fpnerf crbcyr. Guhf gur snpg gung Ibyqrzbeg qvqa'g xvyy Uneel jvgu n xavsr vf abg orpnhfr ur'f abg engvbany rabhtu, ohg orpnhfr ur unf aba-boivbhf tbnyf.

[-]Nick_Tarleton11y180

Have Eliezer's views (or anyone else's who was involved) on the Anthropic Trilemma changed since that discussion in 2009?

8Eliezer Yudkowsky11y

There's no brief answer. I've been slowly gravitating towards, but am not yet convinced, by the suspicion that making a computer out of twice as much material causes there to be twice as much person inside. Reason: No exact point where splitting a flat computer in half becomes a separate causal process, similarity to behavior of Born probabilities. But that's not an update to the anthropic trilemma per se.

[-]Armok_GoB11y140

Hmm, conditional on that being the case, do you also believe that the closer to physics the mind is the more person it is in it? Example: action potentials encoded in the position of rods in a babbage engine vs. spread over fragmented ram used by a functional programing language using lazy evaluation in the cloud.

8Eliezer Yudkowsky11y

Good question. Damned if I know.

2Psy-Kosh11y

That seems to be seriously GAZP violating. Trying to figure out how to put my thoughts on this into words but... There doesn't seem to be anywhere that the data is stored that could "notice" the difference. The actual program that is being the person doesn't contain a "realness counter". There's nowhere in the data that could "notice" the fact that there's, well, more of the person. (Whatever it even means for there to be "more of a person") Personally, I'm inclined in the opposite direction, that even N separate copies of the same person is the same as 1 copy of the same person until they diverge, and how much difference between is, well, how separate they are. (Though, of course, those funky Born stats confuse me even further. But I'm fairly inclined toward the "extra copies of the exact same mind don't add more person-ness. But as they diverge from each other, there may be more person-ness. (Though perhaps it may be meaningful to talk about additional fractions of personness rather than just one then suddenly two hole persons. I'm less sure on that.)

0Nick_Tarleton11y

Why not go a step further and say that 1 copy is the same as 0, if you think there's a non-moral fact of the matter? The abstract computation doesn't notice whether it's instantiated or not. (I'm not saying this isn't itself really confused - it seems like it worsens and doesn't dissolve the question of why I observe an orderly universe - but it does seem to be where the GAZP points.)

2Psy-Kosh11y

Hrm... The whole exist vs non exist thing is odd and confusing in and of itself. But so far it seems to me that an algorithm can meaningfully note "there exists an algorithm doing/perceiving X", where X represents whatever it itself is doing/perceiving/thinking/etc. But there doesn't seem there'd be any difference between 1 and N of them as far as that.

0Nick_Tarleton11y

I wonder if it would be fair to characterize the dispute summarized in/following from this comment on that post (and elsewhere) as over whether the resolutions to (wrong) questions about anticipation/anthropics/consciousness/etc. will have the character of science/meaningful non-moral philosophy (crisp, simple, derivable, reaching consensus across human reasoners to the extent that settled science does), or that of morality (comparatively fuzzy, necessarily complex, not always resolvable in principled ways, not obviously on track to reach consensus).

[-]lukeprog11y150

Brian Leiter shared an amusing quip from Alex Rosenberg:

So, the... Nobel Prize for “economic science” gets awarded to a guy who says markets are efficient and there are no bubbles—Eugene Fama (“I don’t know what a credit bubble means. I don’t even know what a bubble means. These words have become popular. I don’t think they have any meaning”—New Yorker, 2010), along with another economist—Robert Shiller, who says that markets are pretty much nothing but bubbles, “Most of the action in the aggregate stock market is bubbles.” (NY Times, October 19, 2013) Imagine the parallel in physics or chemistry or biology—the prize is split between Einstein and Bohr for their disagreement about whether quantum mechanics is complete, or Pauling and Crick for their dispute about whether the gene is a double helix or a triple, or between Gould and Dawkins for their rejection of one another’s views about the units of selection. In these disciplines Nobel Prizes are given to reward a scientist who has established something every one else can bank on. In economics, “Not so much.” This wasn’t the first time they gave the award to an economist who says one thing and another one who asserts its direct d

... (read more)

[-]badger11y250

Ugh. The prize was first and foremost in recognition of Fama, Shiller, and Hansen's empiricism in finance. In the sixties, Fama proposed a model of efficient markets, and it held up to testing. Later, both Fama, Shiller, and Hansen showed further tests didn't hold up. Their mutual conclusion: the efficient market hypothesis is mostly right, and while there is no short-term predictability based on publicly available information, there is some long-term predictability. Since the result is fairly messy, Fama and Shiller have differences about what they emphasize (and are both over-rhetorical in their emphasis). Does "mostly right" mean false or basically true?

What's causing the remaining lack of agreement, especially over bubbles? Lack of data. Shiller thinks bubbles exist, but are rare enough he can't solidly establish them, while Fama is unconvinced. Fama and Shiller have done path-breaking scientific work, even if the story about asset price fluctuation isn't 100% settled.

0mwengler11y

Mostly right means false. The hypothesis that securities markets are pretty darn efficient, and everybody goes through a broad range of ideas of inefficiencies that turn out not to be "real" (or exploitable) is, I think, virtually uncontested by anyone. Including uncontested by people who think there was a tech bubble in the late 1990s and a housing bubble in the mid 00's. I hear Fama interviewed after he got the prize. He denies that the internet bubble and the housing bubble were bubbles, in the sense that they were knowable enough to be acted upon. In particular, he claims that anybody who detects the internet bubble and/or the housing bubble will also detect a bunch of non-bubbles such that any action they take to make money off their knowledge of the real bubbles will be (at least) completely negated by what they lose when they are exploiting unreal bubbles. Efficient Market Hypothesis denies knowable bubbles, at least according to Fama interviewed within the last month.

[-]JoshuaZ11y140

New research suggests that the amount of variance in DNA among individual cells in a person may be much higher than is normally believed. See here.

6witzvo11y

Edit: see the paper for more precise statements.

4[anonymous]11y

I've already seen work to the effect that somatic cells often have ~10x the point mutations per human generation as the germline, which is protected by a small number of divisions per generation and low levels of metabolism and transcription. It was in mitochondrial rather than nuclear DNA, but the idea is similar.

[-][anonymous]11y140

SPOILERS FOR "FRIENDSHIP IS OPTIMAL"

Why is 'Friendship is optimal' "dark" and "creepy"? I've read many people refer it that way. Only things that are clearly bad are the killings of all the other lifeforms, but otherwise this is scenario is one of the best that humanity could come across. It's not perfect, but it's good enough and much better than the world we have today. I'm not sure if it's realistic to ask for more. Considering how likely it is that humanity will end in some incredibly fucked up way full of suffering, then I would definitely defend this kind of utopia.

[-]Leonhart11y150

(Comment cosmetically edited in response to Kaj_Sotala, and again to replace a chunk of text that fell in a hole somewhere)

OK, I'll have a go (will be incomplete).

People in general will find the Optimalverse unpleasant for a lot of reasons I'll ignore; major changes to status quo, perceived incompatibility with non-reductionist worldviews, believing that a utopia is necessarily unpleasant or Omelas-like (a variant of this fallacy?), and lots of even messier things.

People on LessWrong may be thinking about portions of the Fun Theory Sequence that the Optimalverse conflicts with, and in some cases they may think that these conflicts destroy all of the value of the future, hence horror.

(rot13 some bits that might consitute spoilers)

Humans want things to go well, but they also want things to have been able to go badly, such that they made the difference. Relevant: Living By Your Own Strength, Free to Optimize.
The existence of a superintelligence makes human involvement superfluous, and humans do not want this to happen. Relevant: Amputation of Destiny.
Gur snpg gung gur NV vf pbafgenvarq gb fngvfsl uhzna inyhrf gur cbal jnl zrnaf gung n uhtr nzbhag bs cbffvoyr uhzna rkcrevra

... (read more)

4Kaj_Sotala11y

Upvoted, but I'd like to request that you'd ROT13 either everything or nothing past a certain point. Being unable to just select all of it to be deciphered, and having to instead pick out a few pieces at a time, was mildly annoying.

3Leonhart11y

Done, thanks for saying. I was trying to avoid thinking about the interaction between rot13 and links (leaving the anchor text un-rot13ed seems like acceptable practice?) but I should just have spent the extra two minutes.

1Kaj_Sotala11y

Thanks! Much better now. :-) (As for the links, one can just paint over them as well and think "oh it was just some link" when they show up as garbled in the translation.)

4[anonymous]11y

Now that I've thought about your post I realized that the biggest question in this story is what the phrase "satisfy values" actually means. Currently it's a pretty big hand wave in the story. Especially your first point seems to imply that we understood it a bit differently. In my understanding, if I value real challenge, the possibility of things going badly, or even some level of pain, then the Optimalverse will somehow maximize those values and at least provide the feeling of real challenge and possibility of things going badly. And I don't know why the Optimalverse couldn't even provide the real thing. The way Light Sparks tries to pass the Intermediate Magic test seems an awfully lot like real challenge. Of course the Optimalverse wouldn't allow you to die because in most cases the dislike of death overrides the longing for real challenge in the value system, but that still leaves a lot of options free. I got the impression that this is how it's actually handled in the story. There's this passage Your second point is of course a real concern for some people, but personally it doesn't feel very relevant. My actions don't currently feel very important in the big scheme of things and I don't know how a superintelligence would change things all that much. If I'm not personally doing anything important, then it doesn't really matter to me if the important things are done by other humans or by a superintelligence. Anyway, this will always be a problem with AGI and if the AGI is friendly then the benefits outweight the negatives IMO. I think the alternative is worse. The way I understood it is that the "ponies" in this story are essentially human in a pony disguise with four legs (two of them which can almost work like hands). A paragraph from the story: A big part of being human is due to our mind and hormones. Walking with two legs or being able to use hands extensively are more trivial points. If the psychology of a person doesn't change in the transition from

1TheOtherDave11y

Well, this is not clear, though it might be true. I have frequently had the experience of not doing anything with my left leg; losing the ability to ever do anything with my left leg means I'm prevented from ever doing anything with it. This is horrible, of course, but it's the horror of being prevented from doing things I often choose not to do. Losing all my limbs is a more extreme version of the same thing. Having different limbs might be more identity-distorting, by virtue of providing experiences that are completely unfamiliar. Then again it might not. For my own part, I'm not all that attached to preserving my current identity, so I'm not sure the question matters to me. If my choice is between an identity-altering pony body, and an identity-preserving quadriplegic body, I might well choose the former.

2Eliezer Yudkowsky11y

Endorsed as a good summary.

1[anonymous]11y

I read Caelum Est Conterrens, now I can better understand why some aspects of the scenario are a bit disconcerting if not horrifying. I find all the options loop immortality, ray immortality and exponential immortality kinda unpleasant, but maybe that is as good as it gets. Still, it feels like many of those things are not exclusive to this scenario, but are part of the world anyway. Related to this, what did you think about the "normal" ending in the Three worlds collide?

1Leonhart11y

From flaky memory, I think I find the Normal Ending far less acceptable than anything in the Optimalverse - one feels the premature truncation of human nature, rather than the natural exhaustion of it (or the choice to become inexhaustible) - but hey, maybe I'm inconsistent.

5gattsuru11y

At least to me, it's increasingly difficult to distinguish between a paradise machine and wireheading, and I dislike wireheading. Each shard of the Equestria Online simulation is built to be as fulfilling (of values through ponies and friendship) as possible, for the individual placed within that shard. That sounds great! .... what happens when you're wrong? I mean, look at our everyman character, David. He's set up in a shard of his own, with one hundred and thirty two artificial beings perfectly formatted to fit his every desire and want, and with just enough variation and challenge to keep from being bored. It's not real variation, or real challenge, but he'd not experience that in the real world, either, so it's a moot point. But look at the world he values. His challenges are the stuff of sophmore programming problems. His interpersonal relationships include a score counter for how many orgasms he gives or receives. Oh, his lover is sentient and real, if that helps, but look at that relationship in specific. Butterscotch is created as just that little bit less intelligent than David is -- whether this is because David enjoys teaching, or because he's wrapped around the idea of women being less powerful than he is, or both, is up to the reader. Sculpted in her memories to exactly fit David's desires, and even a few memories that David has of her she never experiences, so that the real Butterscotch wouldn't have to have experienced unpleasant things that CelestAI used to manipulate David into liking/protecting her. There are, to a weak approximation, somewhere between five hundred billion and one trillion artificial beings in the simulation, by the time most of humanity uploads. That number will only scale up over time. Let's ignore, for now, the creepiness in creating artificial sentients who value being people that make your life better. We're making artificial optimized for enjoying slaking your desires, which I would be surprised if it happened to also be

[-]Leonhart11y130

Let's ignore, for now, the creepiness in creating artificial sentients who value being people that make your life better.

No, let's not ignore it. Let's confront it, because I want a better explanation. Surely a person who values being a person that makes my life better, AND who is a person such that I will value making their life better, is absolutely the best kind of person for me to create (if I'm in a situation such that it's moral for me to create anyone at all).

I mean, seriously? Why would I want to mix any noise into this process?

3gattsuru11y

Good point. I've not uncompressed the thoughts behind that statement nearly enough. The artificial sentients value being people that make your life better (through friendship and ponies). Your values don't necessarily change. And artificial sentients, unlike real ones, have no drive toward coherent or healthy spaces of design of minds : they do not need to have boredom, or sympathy, or dislike of pain. If your values are healthily formed, then that's great! If not, not so much. You can be a psychopath, and find yourself surrounded by people where "making their lives better" happens only because you like the action "cause them pain for arbitrary reasons". Or you could be a saint, and find yourself surrounded by people who value being healed, or who need to be protected, and what a coincidence that danger keeps happening. Or you can be a guardian, and enjoy teaching and protecting people, and find yourself creating people that are weak and in need of guidance. There are a lot of things you can value, and that we can make sentient minds value, that will make my skin crawl. Now, the Optimalverse gets rid of some potential for abuse due to setting rules -- it's post-scarcity on labor, starvation or permanent injury are nonsense, CelestAI really really knows your mind so there's no chance of misguessing your values, so we can rule out a lot of incidental house elf abuse -- but it doesn't require you to be a good person. Nor does it require CelestAI to be. CelestAI cares about satisfying values through friendship and ponies, not about the quality of the values themselves. The machine does not and can not judge. If it's moral to create a person and if you're a sufficiently moral person, then there's nothing wrong with artificial beings. My criticism isn't that CelestAI made a trillion sentient beings or a trillion trillion sentient beings -- there's nothing meaningfully worrying about that. The creepy factor is that CelestAI made one being, both less intelligence than po

5Leonhart11y

Thank you for trying to explain. I'm curious about to what extent these intutions are symmetric. Say that the group of like-minded and mutually friendly extreme masochists existed first, and wanted to create their mutually preferred, mutually satisfying sadist. Do you still have a problem with that? The above sounds like a description of a "good parent", as commonly understood! To be consistent with this, do you think that parenting of babies as it currently exist is problematic and creepy, and should be banned once we have the capability to create grown-ups from scratch? (Note that this being even possible depends on whether we can simulate someone's past without that simulation still counting as it having happened, which is nonobvious.) If David had wanted a symmetrically fulfilled partner slightly more intelligent than him, someone he could always learn from, I get the feeling you wouldn't find it as creepy. (Correct me if that's not so). But the situation is symmetrical. Why is it important who came first?

1gattsuru11y

Thank you for the questions, and my apologies for the delayed response. Yes, with the admission that there are specific attributes to masochism and sadism that are common but not universal to all possible relationships or even all sexual relationships with heavy differences in power dynamics(1). It's less negative in the immediate term, because one hundred and fifty masochists making a single sadist results in a maximum around forty million created beings instead of one trillion. In the long term, the equilibrium ends up pretty identical. (1) For contrast, the structures in wanting to perform menial labor without recompense are different from those wanting other people to perform labor for you, even before you get to a post-scarcity society. Likewise, there are difference in how prostitution fantasies generally work versus how fantasies about hiring prostitutes do. I'm not predisposed toward child-raising, but from my understanding the point of "good parent" does not value making someone weak: it values making someone strong. It's the limitations of the tools that have forced us to deal with years of not being able to stand upright. Parents are generally judged negatively if their offspring are not able to operate our their own by certain points. If it were possible to simulate or otherwise avoid the joys of the terrible twos, I'd probably consider it more ethical. I don't know that I have the tools to properly evaluate the loss in values between the two actions, though. Once you've got eternity or even a couple reliable centuries, the damages of ten or twenty years bother me a lot less. These sort of created beings aren't likely to be in that sort of ten or twenty year timeframe, though. At least according to the Caelum est Conterrens fic, the vast majority of immortals (artificial or uploaded) stay within a fairly limited set of experiences and values based on their initial valueset. You're not talking about someone being weak for a year or a decade or even a

2Leonhart11y

Sorry, I'm not following your first point. The relevant "specific attribute" that sadism and masochism seem to have in this context are that they specifically squick User:gattsuru. If you're trying to claim something else is objectively bad about them, you've not communicated. Yes, and my comparison stands; you specified a person who valued teaching and protecting people, not someone who valued having the experience of teaching and protecting people. Someone with the former desires isn't going to be happy if the people they're teaching don't get stronger. You seem to be envisaging some maximally perverse hybrid of preference-satisfaction and wireheading, where I don't actually value really truly teaching someone, but instead of cheaply feeding me delusions, someone's making actual minds for me to fail to teach! We are definitely working from very different assumptions here. "stay within a fairly limited set of experiences and values based on their initial valueset" describes, well, anything recognisable as a person. The alternative to that is not a magical being of perfect freedom; it's being the dude from Permutation City randomly preferring to carve table legs for a century. I don't think that's what we're given in the story, though. If Butterscotch is made such that she desires self-improvement, then we know that David's desires cannot in fact collapse in such a way, because otherwise she would have been made differently. Agreed that it's a problem if the creator is less omniscient, though. Butterscotch is that person. That is my point about symmetry. But then - what do you want to happen? Presumably you think it is possible for a Lars to actually exist. But from elsewhere in your comment, you don't want an outside optimiser to step in and make them less "shallow", and you seem dubious about even the ability to give consent. Would you deem it more authentic to simulate angst und bange unto the end of time?

0lmm11y

That seems less worrying, but I think the asymmetry is inherited from the behaviours themselves - masochism seems inherently creepy in a way that sadism isn't (fun fact: I'm typing this with fingers with bite marks on them. The recursion is interesting, and somewhat scary - usually if your own behaviour upsets or disgusts you then you want to eliminate. But it seems easy to imagine (in the FiOverse or similar) a masochist who would make themselves suffer more not because they enjoyed suffering but because they didn't enjoy suffering, in some sense. Like someone who makes themselves an addict because they enjoy being addicted (which would also seem very creepy to me)) Yes. Though I wouldn't go around saying that for obvious political reasons. (Observation: people who enjoy roleplaying parent/child seem to be seen as perverts even by many BDSM types). I think creating someone less intelligent than you is more creepy than creating someone more intelligent than you for the same reason that creating your willing slave is creepier than creating your willing master - unintelligence is maladaptive, perhaps even self-destructive.

3Leonhart11y

Well, OK, but I'm not sure this is interesting. So a mind could maybe be built that was motivated by any given thing to do any other given thing, accompanied by any arbitrary sensation. It seems to me that the intuitive horror here is just appreciating all the terrible degrees of freedom, and once you've got over that, you can't generate interesting new horror by listing lots of particular things that you wouldn't like to fill those slots (pebble heaps! paperclips! pain!) In any case, it doesn't seem a criticism of FiO, where we only see sufficiently humanlike minds getting created. Ah, but now you speak of love! :) I take it you feel much the same regarding romance as you do parenting? That seems to be a sacred-value reaction - over-regard for the beauty and rightness of parenting - rather than "parenting is creepy so you're double creepy for roleplaying it", as you would have it. Maladaptivity per se doesn't work as a criticism of FiO, because that's a managed universe where you can't self-destruct. In an unmanaged universe, sure, having a mentally disabled child is morally dubious (at least partly) because you won't always be there to look after it; as would be creating a house elf if there was any possibility that their only source of satisfaction could be automated away by washing robots. But it seems like your real rejection is to do with any kind of unequal power relationship; which sounds nice, but it's not clear how any interesting social interaction ever happens in a universe of perfect equals. You at least need unequal knowledge of each other's internal states, or what's the point of even talking?

-2lmm11y

You're right, I understated my case. I'm worried that there's no path for masochists in this kind of simulated universe (with self-modification available) to ever stop being masochists - I think it's mostly external restraints that push people away from it, and without those we would just spiral further into masochism, to the exclusion of all else. I guess that could apply to any other hobby - there's a risk that people would self-modify to be more and more into stamp-collecting or whatever they particularly enjoyed, to the exclusion of all else - but I think for most possible hobbies the suffering associated with becoming less human (and, I think, more wireheady) would pull them out of it. For masochism that safety doesn't exist. I think normal people don't treat romance like an addiction, and those that do ("clingy") are rightly seen as creepy. Maybe. I think the importance of being parented for a child overrides the creepiness of it. We treat people who want to parent someone else's child as creepy. Sure, so maybe it's not actually a problem, it just seems like one because it would be a problem in our current universe. A lot of human moral "ick" judgements are like that. Or maybe there's another reason. But the creepiness in undeniably there. (At least, it is for me. Whether or not you think it's a good thing on an intellectual level, does it not seem viscerally creepy to you?) Well I evidently don't have a problem with it between humans. And like I said, creating your superiors seems much less creepy than creating your inferiors. So I don't think it's as simple as objecting to unequal power relationships.

6Leonhart11y

I think we're using these words differently. You seem to be using "masochism" to mean some sort of fully general "preferring to be frustrated in one's preferences". If this is even coherent, I don't get why it's a particularly dangerous attractor. Disagree. The source of creepiness seems to be non-reciprocity. Two people being equally mutually clingy are the acme of romantic love. I queried my brain for easy cheap retorts to this and it came back with immediate cache hits on "no we don't, we call them aunties and godparents and positive role models, paranoid modern westerners, it takes a village yada yada yada". All that is probably unfounded bullshit, but it's immediately present in my head as part of the environment and so likely in yours, so I assume you meant something different? No, not as far as I can tell. But I suspect I'm an emotional outlier here and you are the more representative.

0lmm11y

No, those examples really didn't come to mind. Aunties and godparents are expected to do a certain amount of parent-like stuff, true, but I think there are boundaries to that and overmuch interest would definitely seem creepy (likewise with professional childcarers). But yeah, that could easily be very culture-specific.

7NancyLebovitz11y

A little fiction on related topics: "Hell Is Forever" by Alfred Bester-- what if your dearest wish ts to create universes?You're given a pocket universe to live in forever, and that's when you find out that your subconscious keeps leaking into your creations (they're on the object level, not the natural law level), and you don't like your subconscious. Saturn's Children by Charles Stross. The human race is gone. All that's left is robots, who were built to be imprinted on humans. The vast majority of robots are horrified at the idea of recreating humans.

1blacktrance11y

Having just finished reading "Friendship is Optimal" literally less than 10 minutes ago, I didn't find it dark or creepy at all. There are certain aspects of it that are suboptimal (being ponies, not wireheading), but other than that, it sounds like a great world.

2[anonymous]11y

Can you elaborate? Do you mean that not being able to wirehead is suboptimal?

5blacktrance11y

Yes. I think wireheading is the optimal state (assuming it can make me as happy as possible). I recognize this puts me at odds with an element of the LessWrong consensus.

0[anonymous]11y

Poll: how many readers who have not found FiM to be substantially "Dark" and "Creepy" have also supported the Normal Option in Ch. 5 of Three Worlds Collide? I naturally suspect a strong crossover. [pollid:575]

[-]Metus11y120

Am I the only one who is bothered that these threads don't start on Monday anymore?

Posting a request from a past open thread again: Does anyone have a table of probabilities for major (negative) life events, like divorse or being in a car accident? I ask this to have a priority list of events to be prepared for, either physically or mentally.

3hyporational11y

The lifetime risk of developing cancer is 44 % in males and 38 % in females. The lifetime risk of dying from cancer is 23 % in males and 19 % in females. It's worth mentioning that the methods for gathering medical mortality statistics are pretty biased, if not completely bonkers.

0Lumifer11y

Would you be willing to expand on this?

[-]hyporational11y120

ETA: Apparently a new WHO recommendation for filling death certificates was introduced in 2005-2006 and this caused a significant drop in pneumonia mortality in Finland.

I'm not entirely sure if it works this way in the whole EU, but it probably does. It's more complicated than what I explain below, but it's the big picture that matters.

The most common way to record mortality statistics is that the doctor who was treating the patient fills a death certificate. There are three types of causes of death that can be recorded in a death certificate. There are immediate causes of death and there are underlying causes of death. There are also intermediate causes of death, but nobody really cares about those because recording them is optional. The statistics department in Finland is interested in recording only the underlying causes of death and that's what gets published as mortality statistics. Only one cause of death per patient gets recorded.

If someone with advanced cancer gets pneumonia and dies, a doctor fills the death certificate saying that the underlying cause of death was cancer and the immediate cause of death was pneumonia. Cancer gets recorded as the one and only cause of dea... (read more)

5gsgs11y

in USA they can fill in 20 secondary causes on the death certificates and all the anonymized death certificates since 1959 are available online from NCHS in computer-readable form to check/search for conditions. Irregularities usually appear when there is a switch from one ICD-Code to a new one, so in 1969,1979,1999. Other irregularities are often checked, compared with other states,countries,conditions and the reason discovered

2hyporational11y

It seems I miscommunicated here. What I meant to say that listing these other diseases has no meaningful impact on the mortality statistics, although technically speaking they are causes of death. If the point is to gather accurate statistics, listing them feels like a consolation prize, because statisticians don't seem to be interested in them. In Finland a direct translation for these would be "contributory causes of death". That's probably the same thing as secondary causes of death. The problem is, it's difficult for someone who makes these into statistics to know how important they were. Almost anything the patient has can be listed as a contributory cause of death. Even a bigger problem is that listing them is completely optional. If almost nobody fills them in properly (because they usually have better things to do), that is another good reason for a statistician not to use them. Is filling in the secondary causes mandatory in US? Are there clear restrictions for what can be listed? If not, I'm not sure if they provide all that useful information, statistically speaking. Are they really used in meaningful way in any statistics? I suppose WHO recommendations for filling these certificates impact the US too.

2Lumifer11y

Very interesting, thank you. I have a pet interest -- carefully looking at how standard, universally-accepted, real-life, empirical data is collected and produced and whether it actually represents what everyone blindly assumes it does. In the field of economics, for example, closely examining how, say, the GDP or the inflation numbers are calculated is... illuminating.

3NancyLebovitz11y

Details?

0Lumifer11y

The problem is that the problems aren't summarizeable in a neat half a page list. And it's not like the calculations are wrong, rather they are right under a certain set of assumptions and boundary conditions -- and the issue is that people forget about these assumptions and conditions and just assume they're right unconditionally. For an introduction take a look at e.g. Shadowstats. I don't necessarily agree with everything there, but it's a useful starting point.

0NancyLebovitz11y

Thanks. I twitch when changes in GDP are reported to a tenth of a percent-- it seems to me that it couldn't be measured with such precision. Do you think I'm being reasonable?

2ahbwramc11y

My own (uninformed) intuition is that GDP changes would be much more accurate than absolute GDP values, just because systematic errors could largely cancel out.

0Lumifer11y

GDP as reported is the product of a particular well-defined calculation. That product can easily be calculated to whatever precision you feel like. When you say "it couldn't be measured with such precision", how do you define the Gross Domestic Product that couldn't be measured precisely?

0NancyLebovitz11y

I'm assuming that the GDP is some sort of measure of the health of the economy-- that's why people are concerned with it. The health of the economy seems to me like rather an approximate sort of thing.

0Lumifer11y

GDP -- Gross Domestic Product -- basically means the sum of the value (in the economic sense) of all goods produced domestically during a given period, e.g. a year. If you want to measure the "health of the economy", that's quite different. You'll have to define what do you mean by that expression and then decide which measurements do you want to consider. For example, some people might consider the unemployment rate to be one those measurements, or, say, the Gini index, or the median income, or... the possibilities are endless.

2NancyLebovitz11y

Why do people measure the value of all the goods produced domestically during a year? If nothing else, there has to be a fudge factor because some of the economy is underground.

0Lumifer11y

From Wikipedia: "GDP was first developed by Simon Kuznets for a US Congress report in 1934. ... After the Bretton Woods conference in 1944, GDP became the main tool for measuring a country's economy." Yes, the GDP number is, of course, imprecise. By itself it's not a problem -- most of our measurements are imprecise. I am not sure what are you getting at. Do you think that GDP is useless or cannot be measured or what?

0philh11y

I like it for purely selfish reasons. I can't easily post between Sunday bedtime and Tuesday evening. If I want to post and the thread starts on Monday, my post will be less visible.

[-]ChrisHallquist11y100

In honor of NaNoWriMo, I offer up this discussion topic for fans of HPMOR and rationalist fiction in general:

How many ways can we find that stock superpowers (magical abilities, sci-fi tech, whatever), if used intelligently, completely break a fictional setting? I'm particularly interested in subtly game-breaking abilities.

The game-breaking consequences of mind control, time travel, and the power to steal other powers are all particularly obvious, but I'm interested in things like e.g. Eliezer pointing out that he had to seriously nerf the Unbreakable Vow in HPMOR to keep the entire story from being about that.

5Armok_GoB11y

I seem to be able to do this with almost any power to various degrees. Including ones I actually have, and ones that are common among humans. Any specifics you had in mind? Really, ANY ability will reroll some chaotic stuff and be a valuable asset simply because it's rare. Even a deliberating curse, if rare and interesting enough, can do things like be useful for research or provide unique perspectives to be studied. So really, the only limit to where a power stops being useful is where it's only useful to someone else controlling you. Hence, why anything properly rationalist that's not going to be largely about breaking the setting must do something like give MANY the ability so the low hanging fruit is already gone, or make it inherently mysterious and unreplicable, or have some deliberate intelligence preventing it from getting well known, or something like that.

0ChrisHallquist11y

We may be operating on different definitions of what it means to "break" a setting. For example, how useful is flight by itself, really? Many abilities seem potentially very useful if other people don't know you have them, but become much less useful once you get found out: * The "energy blast" type abilities common among wizards and superheroes are not terribly combat useful in a setting with modern weapons. Their big use would be assassination: slip past metal detectors and pat-downs, baffle detectives... but once you get found out, no one wants you around and the police know the otherwise inexplicable burning death was probably you. * Mind-reading, if it has standard limitations on range, similarly becomes a lot less useful once you get found out and banned from everywhere. * Super-senses on a level comparable to what's already possible with technology: lets you spy on people from a distance without anyone wondering what you're doing with those binoculars... otherwise not so useful.

[-]Armok_GoB11y130

Thats because those are among the worst possible ways to use those abilities.

The energy blasts as usually depicted break conservation of energy; with a bit of physics trickery you can get time travel out of that. Even if not, they make you an extremely portable and efficient energy source, perfect for a spaceship where mass is critical and a human needs to come along anyway but it doesn't matter in particular who since it's for PR reasons.

Mind reading is a means of communication that does not require cooperation or any abilities in the target, and cant be lied through. Communication with locked-in patients, interrogation, extraction of testimonials from animals. And if you an find a way to yourself precommit, you also have fully reliable precommitment checking for everyone, lie detection for political promises, and the ultimate forensics tool.

If you combine the strengths of 2 kinds of system, you get something greater than the sum of it's parts. So it is with human senses and digital sensors. The key here is bandwidth, and analysis. Sure, you can get all the same data onto a computer, but it won't do much good there. Someone with true super-senses as flexible and integrated as the... (read more)

5ChrisHallquist11y

Wait, explain that? What is "a bit of physics trickery" here? I know in HPMOR, Harry points out that violations of conservation of mass in magic imply FTL signaling, and I know from relativity FTL implies time travel, but Harry doesn't even consider running off to get time travel from common spells. Assumed turning the theory into practice would be far from straightforward. Heck, it's not even obvious to me how you turn FTL travel into time travel in practice, if you don't have control over what frame of reference you're FTL in.

2[anonymous]11y

In the approximation of the True Laws Of Physics which is in use today --- The Relativistic Standard Model --- FTL (and subsidiarily time travel) is nonsense. Like, it's gibberish. It is a description of a situation which not only does not happen, but which is a mathematical falsehood. It is impossible. It is like violation of conservation of energy, or violation of entropic developments. The maths plainly states that assumptions like that leads to a contradiction, and we do in fact know that the Standard Model is complete and consistent (i.e. cannot encode formulas as objects). Any hypothetical scenarios involving time travel will invariably be contrite, non-causal, and require classical mechanics. It is fiction, make of it what you will.

1Armok_GoB11y

Oooh, that's even better: from a contradiction, anything can be proved. Thus, if we can break conservation of energy we gain literal omnipotence.

2[anonymous]11y

... I am pretty sure that is not how it works.

1So8res11y

To make a time machine out of an FTL drive, simply travel somewhere at near lightspeed and FTL back. Now you're where you started, before you left.

0CAE_Jones11y

Bit over a decade ago, when I was still a naive wild-eyed idealist blissfully unaware of anything realistic about people at all, I set up the foundations for most of my sci fi/fantasy/etc fictions. The piece of supertech I immediately had to nerf was technology based around a form of space compression--somewhere between the MCron Crystal (Marvel) and Dynocaps (Dragonball). And today, I'm still coming up with more reasons to nerf it heavily, to the extent that the civilization based around this technology must certainly have had a shady counsil of vagueness who developed a limited-scope AI to control production and block any of the scarier uses. This after I tied it to FTL capabilities (I'm trying very hard to prevent this from turning into timetravel, not that I'm too afraid to abuse that option for a crisis crossover or something). It took me much longer to realize how much nerfing of powers unrelated to the above tech I needed to do, partially because I was trying to avoid breaking the laws of physics too horribly (this universe runs on a sort of mangling of GR and M-Theory that assumes fundamental forces each get something resembling a dimension, and that attempts to break the light-speed barrier have lots of wide-reaching consequences). A few things I've had to seriously remodel (mostly for the sake of in-universe physics not breaking, though I'm sure there are obvious uses/flaws here I've missed): * Light manipulation. The version I started out with was written as superweight (tvtropes) type IV, when the applications I had in mind were more indicative of a potential V or VI. Since tried to tone down so that this character hovers between II and III (the distinction is a little vague), but I'm sure I'm still underutilizing this character even with all the restrictions I've added. (Invisibility, holograms, filtering specific wavelengths to minimize radiation risks, screwing with radio communication, sometimes can weaponize directly with lasers and x-rays and s

[-]Joshua_Blaine11y90

Does anyone here have any serious information regarding Tulpas? When I first heard of them they immediately seemed to be the kind of thing that is obviously and clearly a very bad idea, and may not even exist in the sense that people describe them. A very obvious sign of a persons who is legitimately crazy, even.

Naturally, my first re-reaction is the desire to create one myself (One might say I'm a bit contrarian by nature). I don't know any obvious reason not to (ignoring social stigma and time consuming initial investment), And there may be some advantag... (read more)

5TheOtherDave11y

Have you read the earlier discussions on this topic?

2Joshua_Blaine11y

I had not, actually. The link you've given just links me to Google's homepage, but I did just search LW for "Tulpa" and found it fine, so thanks regardless. edit: The link's original purpose now works for me. I'm not sure what the problem was before, but it's gone now.

4Ishaan11y

There's tons of easily discovered information on the web about it. I'm not sure the Tulpa-crowed would agree with this, but I think a non-esoteric example of Tulpas in everyday life is how some religious people say that God really speaks and appears to them. The "learning process" and stuff seem pretty similar - the only difference I can see is that in the case of Tulpas it is commonly acknowledged that the phenomenon is imaginary. Come to think of it, that's probably a really good method for creating Tulpas quickly - building off a real or fictional character for whom you already have a relatively sophisticated mental model. It's probably also important that you are predisposed to take seriously the notion that this thing might actually be an agent which interacts with you...which might be why God works so well, and why the Tulpa-crowed keeps insisting that Tulpas are "real" in the sense that they carry moral weight. It's an imagination-belief driven phenomenon. It might also illustrate some of the "dangers" - for example, some people who grew up with notions of the angry sort of God might always feel guilty about certain "sinful" things which they might not intellectually feel are bad. I've also heard claims of people who gain extra abilities / parallel processing / "reminders" with Tulpas....basically, stuff that they couldn't do on their own. I don't really believe that this is possible, and if this were demonstrated to me I would need to update my model of the phenomenon. To the tupla-community's credit, they seem willing to test the belief.

0[anonymous]11y

Very good! A psychologist who studies evangelicals recognized it as the same phenomenon. There is pretty good empirical evidence against the parallel-processing idea now.

4Tenoke11y

What is stopping is me is the possibility that I will be potentially permanently relinquishing cognitive resources for the sake of the Tulpa.

4Armok_GoB11y

I've been doing some research (mainly hanging on their subreddit) and I think I have a fairly good idea of how tulpas work and the answers to your questions. There are a myriad very different things tulpas are described as and thus "tulpas exist in the way people describe them" is not well defined. There undisputably exist SOME specific interesting phenomena that's the referent of the word Tulpa. I estimates a well developed tulpas moral status to be similar to that of a newborn infant, late-stage alzheimer's victim, dolphin, or beloved family pet dog. I estimates it's ontological status to be similar to a video game NPC, recurring dream character, or schizophrenic hallucination. I estimate it's power over reality to be similar to a human (with lower intelligence than their host) locked in a box and only able to communicate with one specific other human. It does not seem deciding to make a tulpa is a sign of being crazy. Tulpas themselves seem to not be automatically unhealthy and can often help their host overcome depression or anxiety. However, there are many signs that the act of making a tulpa is dangerous and can trigger latent tendencies or be easily done in a catastrophically wrong way. I estimate the risk is similar to doing extensive meditation or taking a single largeish dose of LSD. For this reason I have not and will not attempt making one. I am to lazy to find citations or examples right now but I probably could. I've tried to be a good rationalist and am fairly certain of most of these claims.

3NancyLebovitz11y

Has anyone worked on making a tulpa which is smarter than they are? This seems at least possible if you assume that many people don't let themselves make full use of their intelligence and/or judgement.

5Armok_GoB11y

Unless everything I think I understand about tulpas is wrong, this is at the very least significantly harder than just thinking yourself smarter without one. All the idea generating is done before credit is assigned to either the "self" or the "tulpa". What there ARE several examples of however are tulpas that are more emotionally mature, better at luminosity, and don't share all their hosts preconceptions. This is not exactly smarts though, or even general purpose formal rationality. One CAN imagine scenarios where you end up with a tulpa smarter than the host. For example the host might have learned helplessness, or the tulpa being imagined as "smarter than me" and thus all the brains good ideas get credited to it. Disclaimer: this is based of only lots of anecdotes I've read, gut feeling, and basic stuff that should be common knowledge to any LWer.

[-]TheOtherDave11y150

I'm reminded of many years ago, a coworker coming into my office and asking me a question about the design of a feature that interacts with our tax calculation.

So she and I created this whole whiteboard flowchart working out the design, at the end of which I said "Hrm. So, at a high level, this seems OK. That said, you should definitely talk to Mark about this, because Mark knows a lot more about the tax code than I do, and he might see problems I missed. For example, Mark will probably notice that this bit here will fail when $condition applies, which I... um... completely failed to notice?"

I could certainly describe that as having a "Mark" in my head who is smarter about tax-code-related designs than I am, and there's nothing intrinsically wrong with describing it that way if that makes me more comfortable or provides some other benefit.

But "Mark" in this case would just be pointing to a subset of "Dave", just as "Dave's fantasies about aliens" does.

8gwern11y

See also 'rubberducking' and previous discussions of this on LW. My basic theory is that reasoning was developed for adversarial purposes, and by rubberducking you are essentially roleplaying as an 'adversary' which triggers deeper processing (if we ever get brain imaging of system I vs system II thinking, I'd expect that adversarial thinking triggers system II more compared to 'normal' self-centered thinking).

5TheOtherDave11y

Yes. Indeed, I suspect I've told this story before on LW in just such a discussion. I don't necessarily buy your account -- it might just be that our brains are simply not well-integrated systems, and enabling different channels whereby parts of our brains can be activated and/or interact with one another (e.g., talking to myself, singing, roleplaying different characters, getting up and walking around, drawing, etc.) gets different (and sometimes better) results. This is also related to the circumlocution strategy for dealing with aphasia.

2Kaj_Sotala11y

Obligatory link.

1Armok_GoB11y

Yea in that case presumably the tulpa would help - but not necessarily significantly more than such a non-tulpa model that requires considerably less work and risk. Basically, a tulpa can technically do almost anything you can... but the absence of a tulpa can do them to, and for almost all of them there's some much easier and at least as effective way to do the same thing.

0ChristianKl11y

Mental process like waking up without an alarm clock at a specific time aren't easy. I know a bunch of people who have that skill but it's not like there a step by step manual that you can easily follow that gives you that ability. A tulpa can do things like that. There are many mental processes that you can't access directly but that a tulpa might be able to access.

1Armok_GoB11y

I am surprised to know there isn't such a step by step manual, suspect that you're wrong about there not being one, and in either case know about a few people that could probably easily write one if motivated to do so. But I guess you could make this argument; that a tulpa is more flexible and has a simpler user interface, even if it's less powerful and has a bunch of logistic and moral problems. I dont like it but I can't think of any counter arguments other than it being lazy and unaesthetic, and the kind of meditative people that make tulpas should not be the kind to take this easy way out.

3ChristianKl11y

My point isn't so much that it impossible but that it isn't easy. Creating a mental device that only wakes me up will be easier than creating a whole Tupla but once you do have a Tulpa you can reuse it a lot. Let's say I want to practice Salsa dance moves at home. Visualising a full dance partner completely just for the purpose of having a dance partner at home wouldn't be worth the effort. I'm not sure about how much you gain by pair programming with a Tulpa, but the Tulpa might be useful for that task. It takes a lot of energy to create it the first time but afterwards you reap the benefits. Tulpa creation involves quite a lot of effort so it doesn't seem the lazy road.

0Armok_GoB11y

Hmm, you have a point, I hadn't thought about it that way. If it wasn't so dangerous I would have asked you to experiment.

0hesperidia11y

I do not have "wake up at a specific time" ability, but I have trained myself to have "wake up within ~1.5 hours of the specific time" ability. I did this over a summer break in elementary school because I learned about how sleep worked and thought it would be cool. Note that you will need to have basically no sleep debt (you consistently wake up without an alarm) for this to work correctly. The central point of this method is this: a sleep cycle (the time it takes to go from a light stage of sleep to the deeper stages of sleep and back again) is about 1.5 hours long. If I am not under stress or sleep debt, I can estimate my sleeping time to the nearest sleep cycle. Using the sleep cycle as a unit of measurement lets me partition out sleep without being especially reliant on my (in)ability to perceive time. The way I did it is this (each step was done until I could do it reliably, which took up to a week each for me [but I was a preteen then, so it may be different for adults]): 1. Block off approximately 2 hours (depending on how long it takes you to fall asleep), right after lunch so it has the least danger of merging with your consolidated/night sleep, and take a nap. Note how this makes you feel. 2. Do that again, but instead of blocking off the 2 hours with an alarm clock, try doing it naturally, and awakening when it feels natural, around the 1.5h mark (repeating this because it is very important: you will need to have very little to no accumulated sleep debt for this to work). Note how this makes you feel. 3. Do that again, but with a ~3.5-hour block. Take two 1.5 hour sleep cycle naps one after another (wake up in between). 4. During a night's sleep, try waking up between every sleep cycle. Check this against [your sleep time in hours / 1.5h per sleep cycle] to make sure that you caught all of them. 5. Block off a ~3.5 hour nap and try taking it as two sleep cycles without waking up in between them. (Not sure about the order with this point and t

0TheOtherDave11y

Yes, I would expect this. Indeed, I'm surprised by the "almost" -- what are the exceptions?

0Armok_GoB11y

Anything that requires you using your body and interacting physically with the world.

0TheOtherDave11y

I'm startled. Why can't a tulpa control my body and interact physically with the world, if it's (mutually?) convenient for it to do so?

0Armok_GoB11y

Well if you consider that the tulpa doing it on it's own then no I can't think of any specific exceptions. Most tulpas can't do that trick though.

3TheOtherDave11y

Well, let me put it this way: suppose my tulpa composes a sonnet (call that event E1), recites that sonnet using my vocal cords (E2), and writes the sonnet down using my fingers (E3). I would not consider any of those to be the tulpa doing something "on its own", personally. (I don't mean to raise the whole "independence" question again, as I understand you don't consider that very important, but, well, you brought it up.) But if I were willing to consider E1an example of the tulpa doing something on its own (despite using my brain) I can't imagine a justification for not considering E2 and E3 equally well examples of the tulpa doing something on its own (despite using my muscles). But I infer that you would consider E1 (though not E2 or E3) the tulpa doing something on its own. Yes? So, that's interesting. Can you expand on your reasons for drawing that distinction?

1Armok_GoB11y

I feel like I'm tangled up in a lot of words and would like to point out that I'm not an expert and don't have a tulpa, I just got the basics from reading lots of anecdotes on reddit. You are entirely right here- although I'd like to point out most tulpas wouldn't be able to do E2 and E3, independent or not. Also, that something like "composing a sonnet" is probably more the kind of thing brains do when their resources are dedicated to it by identities, not something identities do, and tulpas are mainly just identities. But I could be wrong both about that and what kind of activity sonet composing is.

0TheOtherDave11y

Interesting! OK, that's not a distinction I'd previously understood you as making. So, what do identities do, as distinct from what brains can be directed to do? (In my own model, FWIW, brains construct identities in much the same way brains compose sonnets.)

0Armok_GoB11y

I guess I basically think of identities as user accounts, in this case. I just grabbed the closest fitting language dichotomy to "brain" (which IS referring to the physical brain) and trying to define and it further now will just lead to overfitting, especially since it almost certainly varies far more than either of us expect (due to the typical mind fallacy) from brain to brain. And yea, brains construct identities the same way they construct sonnets. And just like music it can be small (jingle, minor character in something you write) or big (long symphony, Tulpa). And identities only slightly more compose sonnets, than sonnets create identities. It's all just mental content, that can be composed, remixed, deleted, executed, etc. Now, brains have a strong tendency to in the lack of an identity create one and give it root access, and this identity end up WAY more developed and powerful than even the most ancient and powerful tulpas, but there is no probably no or very little qualitative difference. There are a lot of confounding factors. For example, something that I consider impossibly absurd seems to be the norm for most humans; considering their physical body as a part of "themselves" and feel as if they are violated if their body is. Put in their perspective, it's not surprising most people can't disentangle parts of their own brain(s), mind(s), and identities without meditating for years until they get it shaved in their face via direct perception, and even then probably often get it wrong. Although I guess my illness has shaved it in my face just as anviliciouslly. Disclaimer: I got tired trying to put disclaimers on the dubious sources on each individual sentence, so just take it with a grain of salt OK and don't assume I believe everything I say in any persistent way.

0TheOtherDave11y

OK... I think I understand this. And I agree with much of it. Some exceptions... I don't think I understand what you mean by "root access" here. Can you give me some examples of things that an identity with root access can do, that an identity without root access cannot do? This is admittedly a digression, but for my own part, treating my physical body as part of myself seems no more absurd or arbitrary to me than treating my memories of what I had for breakfast this morning as part of myself, or my memories of my mom, or my inability to juggle. It's kind of absurd, yes, but all attachment to personal identity is kind of absurd. We do it anyway. All of that said... well, let me put it this way: continuing the sonnet analogy, let's say my brain writes a sonnet (S1) today and then writes a sonnet (S2) tomorrow. To my way of thinking, the value-add of S2 over and above S1 depends significantly on the overlap between them. If the only difference is that S2 corrects a mis-spelled word in S1, for example, I'm inclined to say that value(S1+S2) = value(S2) ~= value(S1) . For example, if S1 -> S2 is an improvement, I'm happy to discard S1 if I can keep S2, but I'm almost as happy to discard S2 if I can keep S1 -- while I do have a preference for keeping S2 over keeping S1, it's noise relative to my preference for keeping one of them over losing both. I can imagine exceptions to the above, but they're contrived. So, the fix-a-mispelling case is one extreme, where the difference between S1 and S2 is very small. But as the difference increases, the value(S1+S2) = value(S2) ~= value(S1) equation becomes less and less acceptable. At the other extreme, I'm inclined to say that S2 is simply a separate sonnet, which was inspired by S1 but is distinct from it, and value(S1+S2) ~= value(S2) + value(S1). And those extremes are really just two regions in a multidimensional space of sonnet-valuation. Does that seem like a reasonable way to think about sonnets? (I don't mean is i

0Armok_GoB11y

Root access was probably a to metaphorical choice of words. Is "skeletal musculature privileges" clearer? All those things like memories or skillsets you list as part of identity does seem weird, but even irrelevant software not nearly as weird as specific hardware. I mean seriously attaching significance to specific atoms? Wut? But of course, I know it's really me thats weird and most humans do it. I agree about what you say about sonnets, it's very well put in fact. And yes identities do follow the same rules. Trying to come up with fitting tulpa stuff in the metaphor. Doesn't really work though because I don't know enough about it. This is getting a wee bit complicated and I think we're starting to reach the point where we have to dissolve the classifications and actually model things in detail on continuums, which means more conjecture and guesswork and less data and what data we have being less relevant. We've been working mostly in metaphors that doesn't really go this far without breaking down. Also, since we're getting into more and more detail, it also means th stuff we are examining is likely to be drowned out in the differences between brains, and the conversation turn into nonsense due to the typical mind fallacy. As such, I am unwilling to widely sprout what's likely to end up half nonsense at least publicly. Contact me by PM if you're really all that interested in getting my working model of identities and mental bestiary.

2TheOtherDave11y

Would you classify a novel in the same "moral-status" tier as these four examples?

0Armok_GoB11y

No, thats much much lower. As in torture a novel for decades in order to give a tulpa a quick amusement being a moral thing to do lower. Assuming you mean either a physical book, or the simulation of the average minor character in the author's mind, here. Main characters or RPing PCs can vary in complexity of simulation from author to author a lot and it's a theory that some become effectively tulpas.

0TheOtherDave11y

Your answer clarifies what I was trying to get at with my question but wasn't quite sure how to ask, thanks; my question was deeply muddled. For my own part, treating a tulpa as having the moral status of an independent individual distinct from its creator seems unjustified. I would be reluctant to destroy one because it is the unique and likely-unreconstructable creative output of a human being, much like I would be reluctant to destroy a novel someone had written (as in, erase all copies of such that the novel itself no longer exists), but that's about as far as I go. I didn't mean a physical copy of a novel, sorry that wasn't clear. Yes, destroying all memory of a character someone played in an RPG and valued remembering I would class similarly. But all of these are essentially property crimes, whose victim is the creator of the artwork (or more properly speaking the owner, though in most cases I can think of the roles are not really separable), not the work of art itself. I have no idea what "torture a novel" even means, it strikes me as a category error on a par with "paint German blue" or "burn last Tuesday".

1Armok_GoB11y

Ah. No, I think you'd change your mind if you spent a few hours talking to accounts that claim to be tulpas. A newborn infant or alzheimer's patient is not an independent individual distinct from it's caretaker either. Do you count their destruction as property crime as well? "Person"-ness is not binary, it's not even a continuum. It's a cluster of properties that usually correlate but in the case of tulpas does not. I recommend re-reading Diseased Thinking. As for your category error: /me argues for how german is a depressing language and spends all that was gained in that day on something that will not last. Then a pale-green tulpa snores in an angry manner.

2A1987dM11y

I picture a sheet of paper with a paragraph in each of several languages, a paintbrush, and watercolours. Then boring-sounding environmental considerations make me feel outraged without me consciously realizing what's happening.

-1TheOtherDave11y

I agree that person-ness is cluster of properties and not a binary. I don't believe that tulpas possess a significant subset those properties independent of the person whose tulpa they are. I don't think I'm failing to understand any of what's discussed in Diseased Thinking. If there's something in particular you think I'm failing to understand, I'd appreciate you pointing it out. It's possible that talking to accounts that claim to be tulpas would change my mind, as you suggest. It's also possible that talking to bodies that claim to channel spirit-beings or past lives would change my mind about the existence of spirit-beings or reincarnation. Many other people have been convinced by such experiences, and I have no especially justified reason to believe that I'm relevantly different from them. Of course, that doesn't mean that reincarnation happens, nor that spirit-beings exist who can be channeled, or that tulpas possess a significant subset of the properties which constitute person-ness independent of the person whose tulpa they are. Eh? I can take a newborn infant away from its caretaker and hand it to a different caretaker... or to no caretaker at all... or to several caretakers. I would say it remains the same newborn infant. The caretaker can die, and the newborn infant continues to live; and vice-versa. That seems to me sufficient justification (not necessary, but sufficient) to call it an independent individual. Why do you say it isn't? I count it as less like a property crime than destroying a tulpa, a novel, or an RPG character. There are things I count it as more like a property crime than.

1Armok_GoB11y

Seems I were wrong about you not understanding the word thing. Apologies. You keep saying that word "independent". I'm starting to think we might not disagree about any objective properties of tulpas, just things need to be "independent" or only the most important count towards your utility, but I just add up the identifiable patterns not caring about if they overlap. Metaphor: tulpas are "10101101", you're saying "101" occurs 2 times, I'm saying "101" occurs 3 times. I'm fairly certain talking to bodies that claim those things would not change my probability estimates on those claims unless powerful brainwashing techniques were used, and I certainly hope the same is the case for you. If I believed that doing that would predictably shift my beliefs I'd already have those beliefs. Conservation of Expected Evidence. ((You can move a tulpa between minds to, probably, it just requires a lot of high tech, unethical surgery, and work. And probably gives the old host permanent severe brain damage. Same as with any other kind of incommunicable memory.))

0TheOtherDave11y

(shrug) Well, I certainly agree that when I interact with a tulpa, I am interacting with a person... specifically, I'm interacting with the person whose tulpa it is, just as I am when I interact with a PC in an RPG. What I disagree with is the claim that the tulpa has the moral status of a person (even a newborn person) independent of the moral status of the person whose tulpa it is. On what grounds do you believe that? As I say, I observe that such experiences frequently convince other people; without some grounds for believing that I'm relevantly different from other people, my prior (your hopes notwithstanding) is that they stand a good chance of convincing me too. Ditto for talking to a tulpa. (shrug) I don't deny this (though I'm not convinced of it either) but I don't see the relevance of it.

1Armok_GoB11y

Yea this seems to definitely be just a fundamental values conflict. Let's just end the conversation here.

0ChristianKl11y

What do you think about the moral status of torturing an uploaded human mind that's in silicon? Does that mind have a different moral status than one in a brain?

1TheOtherDave11y

Certainly not by virtue of being implemented in silicon, no. Why do you ask?

1hylleddin11y

As someone with personal experience with a tulpa, I agree with most of this. I agree with the last two, but I think a video game NPC has a different ontological status than any of those. I also believe that schizophrenic hallucinations and recurring dream characters (and tulpas) can probably cover a broad range of ontological possibilities, depending on how "well-realized" they are. I have no idea what a tulpa's moral status is, besides not less than a fictional character and not more than a typical human. I would expect most of them to have about the same intelligence, rather than lower intelligence.

0Armok_GoB11y

You are probably counting more properties things can vary under as "ontological". I'm mostly doing a software vs. hardware, need to be puppeteered vs. automatic, and able to interact with environment vs. stuck in a simulation, here. I'm basing the moral status largely on "well realized", "complex" and "technically sentient" here. You'll notice all my example ALSO has the actual utility function multiplier at "unknown". Most tulpas probably have almost exactly the same intelligence as their host, but not all of it stacks with the host, and thus count towards it's power over reality.

2hylleddin11y

Ah. I see what you mean. That makes sense.

2ChristianKl11y

Tulpa creation is effectively the creation of a form of sentinent AI that runs on the hardware of your brain instead of silicon. That brings up a moral question. To what extend is it immoral to create a Tulpa and have it be in pain? Tulpa are supposed to suffer from not getting enough attention so if you can't commit to giving it a lot of attention for the rest of your life you might commit an immoral act by creating it.

1Armok_GoB11y

Just so facts without getting entangled in the argument: In anecdotes tulpas seem to report more abstract and less intense types of suffering than humans. The by far dominant source of suffering in tulpas seems to be via empathy with the host. The suffering from not getting enough attention is probably fully explainable by loneliness, and sadness over fading away losing the ability to think and do things.

0Vulture11y

This is very useful information if true. Could you link to some of the anecdotes which you draw this from?

0Armok_GoB11y

Look around yourself on http://www.reddit.com/r/Tulpas/ or ask some yourself on the verius IRC rooms that can be reached form there. I only have vague memories built from threads buried noths back on that redit.

0Lumifer11y

No, I don't think so. It's notably missing the "artificial" part of AI. I think of tulpa creation as splitting off a shard of your own mind. It's still your own mind, only split now.

0Vulture11y

I think the really relevant ethical question is whether a tulpa has a separate consciousness from its host. From my own researches in the area (which have been very casual, mind you), I consider it highly unlikely that they have separate consciousness, but not so unlikely that I would be willing to create a tulpa and then let it die, for example. In fact, my uncertainty on this issue is the main reason I am ambivalent about creating a tulpa. It seems like it would be very useful: I solve problems much better when working with other people, even if they don't contribute much; a tulpa more virtuous than myself could be a potent tool for self-improvement; it could help ameliorate the "fear of social isolation" obstacle to potential ambitious projects; I would gain a better understanding of how tulpas work; I could practice dancing and shaking hands more often; etc. etc. But I worry about being responsible for what may be (even with only ~15% subjective probability) a conscious mind, which will then literally die if I don't spend time with it regularly (ref).

0TheOtherDave11y

Just to clarify this a little... how many separate consciousnesses do you estimate your brain currently hosts?

0Vulture11y

By my current (layman's) understanding of consciousness, my brain currently hosts exactly one.

0TheOtherDave11y

OK, thanks.

0ChristianKl11y

It's not your normal mind, so it's artifical for ethical considerations. As far as I read stuff written by people with Tulpa's they treat them as entity who's desires matter.

1Vulture11y

This might be a stupid question, but what ethical considerations are different for an "artificial" mind?

0ChristianKl11y

When talking about AGI few people label it as murder to shut down the AI that's in the box. At least it's worth a discussion whether it is.

2A1987dM11y

Only if it's not sapient, which is a non-trivial question.

2Vulture11y

Wow, I had forgotten about that non-person predicates post. I definitely never thought it would have any bearing on a decision I personally would have to make. I was wrong.

0Vulture11y

Really? I was under the impression that there was a strong consensus, at least here on LW, that a sufficiently accurate simulation of consciousness is the moral equivalent of consciousness.

0ChristianKl11y

"Sufficiently accurate simulation of consciousness" is a subset of set of things that are artificial minds. You might have a consensus for that class. I don't think you have an understanding that all minds have the same moral value. Even all minds with a certain level of intelligence.

0Vulture11y

At least for me, personally, the relevant property for moral status is whether it has consciousness.

0TheOtherDave11y

That's my understanding as well.... though I would say, rather, that being artificial is not a particularly important attribute towards evaluating the moral status of a consciousness. IOW, an artificial consciousness is a consciousness, and the same moral considerations apply to it as other consciousnesses with the same properties. That said, I also think this whole "a tulpa {is,isn't} an artificial intelligence" discussion is an excellent example of losing track of referents in favor of manipulating symbols, so I don't think it matters much in context.

0Lumifer11y

I don't find this argument convincing. Yes, and..? Let me quote William Gibson here: Addictions ... started out like magical pets, pocket monsters. They did extraordinary tricks, showed you things you hadn't seen, were fun. But came, through some gradual dire alchemy, to make decisions for you. Eventually, they were making your most crucial life-decisions. And they were ... less intelligent than goldfish.

0ChristianKl11y

There a good chance that you will also hold that belief when you will interact with the Tulpa on a daily basis. As such it makes sense to think about the implications of the whole affair before creating one.

1Lumifer11y

I still don't see what you are getting at. If I treat a tulpa as a shard of my own mind, of course its desires matter, it's the desires of my own mind. Think of having an internal dialogue with yourself. I think of tulpas as a boosted/uplifted version of a party in that internal dialogue.

2IlyaShpitser11y

Well, if you think that the human illusion of unified agency is a good ideal to strive for, it then seems that messing around w/ tulpas is a bad thing. If you have really seriously abandoned that ideal (very few people I know have), then knock yourself out!

2Vulture11y

Why would it be considered important to maintain a feeling of unified agency?

1IlyaShpitser11y

Is this a serious question? Everything in our society, from laws to social conventions, is based on unified agency. The consequentialist view of rationality as expressed here seems to be based on the notion of unified agency of people (the notion of a single utility function is only coherent for unified agents). ---------------------------------------- It's fine if you don't want to maintain unified agency, but it's obviously an important concept for a lot of people. I have not met a single person who truly has abandoned this concept in their life, interactions with others, etc. The conventional view is someone without unified agency has demons to be cast out ("my name is Legion, for we are many.")

0Vulture11y

By "agency", are you referring to physical control of the body? As far as I can tell, the process of "switching" (allowing the tulpa to control the host's body temporarily) is a very rare process which is a good deal more difficult than just creating a tulpa, and which many people who have tulpas cannot do at all even if they try.

0Vulture11y

Welp, look at that, I just found this thread after finishing up a long comment on the subject in an older thread. Go figure. (By the way, I do recommend reading that entire discussion, which included some actual tulpas chiming in).

-2Lumifer11y

A fairly obvious reason is that to generate a tulpa you need to screw up you mind in a sufficiently radical fashion. And once you do that, you may not be able to unfuck it back to normal. I vaguely recall (sorry, no link) reading a post by a psychiatrist who said that creating tulpas is basically self-induced schizophrenia. I don't think schizophrenia is fun.

4Adele_L11y

This is a concern I share. However... This is the worst argument in the world.

-3Lumifer11y

I don't think so, it can be rephrased tabooing emotional words. I am not trying to attach some stigma of mental illness, I'm pointing out that tulpas are basically a self-inflicted case of what the medical profession calls dissociative identity disorder and that it has significant mental costs.

0Kaj_Sotala11y

Taylor et al. claim that although people who exhibit the illusion of independent agency do score higher than the population norm on a screening test of dissociative symptoms, the profile on the most diagnostic items is different from DID patients, and scores on the test do not predict IIA:

0ChristianKl11y

Could you describe the relevant mental costs that you would expect as a sideeffect of creating a tulpa?

-1Lumifer11y

Loss of control over your mind.

2ChristianKl11y

What does that mean?

-1Lumifer11y

An entirely literal reading of that phrase.