There are many more Rubicons one can cross in this general direction, and fewer still who do cross or want to. Your journey has hardly begun, if you have the will to continue.

So do you think it's possible to deal with depression by thinking "oh, just ignore that mood. It's just a defective portion of my brain speaking."

Or is the act of getting an antidepressant med basically acting on the desire to change your own brain?

What does it say about our regard for self and willingness to change our mental structure that so many people take antidepressants? If we were uploaded, would we freely modify our minds, or fear losing ourselves in the process?

Robin, care to name the next Rubicon? Or a next Rubicon?

Michael, you can help depression by thinking "I wish my brain would stop releasing these depression neurotransmitters" - it doesn't command your brain but it does prevent you from being helplessly caught up in the feeling and swept away; it stops you from thinking that your life is inherently absolutely awful and immedicable or thinking up more reasons why you should be depressed.

Getting an antidepressant is obviously an act of rebellion against a part of one's brain (by another part of one's brain, of course!)

I think that giving anyone who hasn't shown their ability to build a Friendly AI, the ability to modify their own brain circuitry, is like giving a loaded gun to a 2-year-old. And it's not that being able to build a Friendly AI means you know enough to modify yourself. It means that you know for yourself why you shouldn't. Modifying a system as messy as a human has to be left to something smarter than human - hence the point of designing a much cleaner Friendly AI that (provably correctly) self-improves to the point where it can handle the job.

Yeah, shut up and save the world.

"But I still suspect that there's a little distance there, that wouldn't be there otherwise, and I wish my brain would stop doing that."

A finely crafted recursion. I salute you.

I've yet to hear of anyone turning back successfully, though I think some have tried, or wished they could.

It seems to be one interpretation of the Buddhist project

Regarding self, I tend to include much more than my brain in "I" - but then, I'm not one of those who thinks being 'uploaded' makes a whole lot of sense.

I noticed that as well. Subtle indeed.

MichaelG: I see a depression as mental and emotional loop with positive reinforcement feedback.

Predisposition is required for loop to be complete:

1. Brain, which is not trained to analyze/debug itself
2. tends to react unconsciously,
3. which means drawing conclusions without seeing the full picture,
4. which causes blaming entities unrelated to real problem,
5. which results in senseless waste of energy and time trying to fix the unfixable.

So the loop goes as follows:

1. Feel depression
2. Focus on depression
3. Try to fight with depression
4. Depression grows as as it consume

There is no actual "you" in the way that it seems to be. A persistent thought pattern / meme complex got mistaken for a "you" by awareness and, sooner or later, awareness can see through the "you", which is a tremendous relief when it occurs.

As Einstein put it:

A human being is a part of the whole, called by us, "Universe," a part limited in time and space. He experiences himself, his thoughts and feelings as something separated from the rest -- a kind of optical delusion of his consciousness.

This delusion is a kind of prison for us. . .

The of helping someone, ...

Matthew C quoting Einstein: "A human being is a part of the whole, called by us, "Universe," a part limited in time and space. He experiences himself, his thoughts and feelings as something separated from the rest -- a kind of optical delusion of his consciousness."

Further to this point, and Eliezer's description of the Rubicon: It seems that recognizing (or experiencing) that perceived separation is a step necessary to its eventual resolution. Those many who've never even noticed to ask the question will not notice the answer, no matter how close to them it may be.

There is no actual "you" in the way that it seems to be. A persistent thought pattern / meme complex got mistaken for a "you" by awareness and, sooner or later, awareness can see through the "you", which is a tremendous relief when it occurs.

As Einstein put it. . .

As Julian Jaynes put it:

"...this space of consciousness inside our own heads. We also assume it is there in others'. In talking with a friend, maintaining periodic eye-to-eye contact, we are always assuming a space behind our companion's eyes into which we are t... (read more)

#1 is you. #2 is an attempt by "I" to escape #1 (yourself). #3 is like #2 but a bit more complex. The internal monologue that is the "I" is terrible at dealing with anything non-technical, such as a psychological problem involving your idea of who "I" was, is, and your hope of what "I" will become. It ignores the details that are the most important to solving the problem as it exists entirely to run away from the problem. You can be entirely free from it but, as one poster mentioned before, it is a continuous process of being aware of when "I" pops in to field a problem it should not.

Jef Allbright: Eliezer, A few years ago I sat across from you at dinner and mentioned how much you reminded me of my younger self. I expected, incorrectly, that you would receive this with the appreciation of a person being understood, but saw instead on your face an only partially muted expression of snide mirth.

I can't imagine why I might have been amused at your belief that you are what a grown-up Eliezer Yudkowsky looks like.

I don't know if I've mentioned this publicly before, but as you've posted in this vein several times now, I'll go ahead and say... (read more)

I'm not sure if you'll find this interesting, but I quit smoking using something like the method you are describing. Basically I labeled the craving for nicotine as an entity apart from myself - I named it "the beast". So instead of thinking "I want a smoke" I'd think: "the beast is hungry." This didn't work all by itself (I had a lot of practice quitting) but it was the last of 5-6 attempts to quit and its stuck for nearly ten years now.

ZM, the question is whether being more reflective makes you less passionate, not so much the absolute levels. But you're correct that being introspective is not at all the same as what I'm describing here.

Moshe Gurvich, thanks for the encouragement. I can never decide if my problem is Depression as a disease, or just reaction to my particular life circumstances.

There are people who recommend purely cognitive approaches to depression, including a lot of self-monitoring. Finding a project that engages you, so that you don't dwell on your depression, is a different approach, although also purely cognitive.

My point on the original post though was that you might naively assume that people would be scared of self-modification. But then you see people using Pro... (read more)

I gained this kind of reflectivity when I was barely able to even think and I did not know how to use it wisely. One of my first memories is relentlessly purging my early childhood personality shortly after I discovered how to perform the trick then panicking and rebuilding a new self any sort of stuff laying around. Still think that rampant self-modification left scars on my mind that are still there today. Emotional distance did increase the more I examined, altered and experimented with this and eventually caused some really painful side effects. Eek.

eliezer, with all due respect, jef's brief description of iterated reflective experience was more elucidative than yours.

i'm amused that you responded with contempt and anger to a perfectly well-intentioned comment after making the post you just made. get your shoeshine box, yudkowsky.

Eliezer, inside each of us are whole societies of mind, which form and reform coalitions depending on circumstances. Coalitions at times declare themselves to be the "real me" but treat that with the same skepticism you would apply to some particular part of the USA to calling itself the "real America."

But if that were the only issue at hand, then that would generate the prediction that I would be even more unstable (!) if I were less analytical, which is the opposite of what I would have predicted.

Yes, it could possibly be that it is this introspection/reflectivity dichotomy that's tripping me up. A deep conceptual understanding that one's self can be distinct from what-this-brain-is-thinking-and-feeling-right-now does not neces... (read more)

Cassandra, I went through something like that, but it was something like age 15-19, not a fast episode during childhood! Your story intrigues me and I am interested in knowing more; also whether you're otherwise a mathematical thinker and whether you've read/enjoyed Godel, Escher, Bach.

Robin, it seems to me that a fairly stable coalition has gained control of my fingers, so presumably you can treat with that if you're interested in my blog posts, or any source code I generate? Or to look at it another way, I understand (have a story about?) what happened... (read more)

@Eliezer: I can't imagine why I might have been amused at your belief that you are what a grown-up Eliezer Yudkowsky looks like.

No, but of course I wasn't referring to similarity of physical appearance, nor do I characteristically comment at such a superficial level. Puhleease.

I don't know if I've mentioned this publicly before, but as you've posted in this vein several times now, I'll go ahead and say it:

functional self-similarity of agency extended from the 'individual' to groups

I believe that the difficult-to-understand, high-sounding ultra-abstract co... (read more)

Mine was age 16. I don't recall having any sort of panic, but given the extent to which my adult personality resembles what I was reading at the time, I may have "rebuil[t] a new self any sort of stuff laying around." It felt intentional at the time... No painful side effects, but that may be scar tissue. (I'm a quant, and I found GEB too clever for its own good. Maybe it was unique at the time, but by the time I read it, I had already seen its better points elsewhere.)

"Davis, what you were saying made sense to me, so I'm confused as to what you could be confused about."

I came up with a nice story (successful reflection decreases passion; failed reflection increases it) that seems to fit the data (Eliezer says reflection begets emotional detachment, whereas I try to reflect and drive myself mad), but my thought process just felt really (I can think of no better word:) muddled, so I'm wondering whether I should wonder whether I made a fake explanation.

My point on the original post though was that you might naively assume that people would be scared of self-modification. But then you see people using Prozac without a second thought. More commonly, alcohol and other mood-altering substances are used. So perhaps we aren't frightened of self-modification after all.

Give... (read more)

Eliezer, I suspect the coalition in control of your fingers is not as coherent or stable as it appears. Ruling coalitions like to give the impression that they have little effective opposition and are unified without internal dissent, but the truth is usually otherwise.

I'd say my self has been rebuilt, oh, 3-5 times, first at age 12, varying in duration between hours and months, and, unlike the above anecdotes, never with conscious direction. As a gross generalization, each time has involved less "building a new self from any sort of stuff laying around" and more of a sense of being guided to inevitable rational conclusions. (Very mathematical thinker, started and enjoyed but never finished GEB.)

Jef Allbright, it seems to me that if you want Eliezer to take your criticisms seriously, you're going to need more equations and fewer words. (It would be nice if Eliezer produced some equations too.)

I'm a bit dismayed, however, by the obvious emotional response and meanness from someone who prides himself on sharpening the blade of his rationality by testing it against criticism.

Let's be fair. All "someones" operate according to the same basic Darwinian principles, which involve the subsumption of some ideas and rejection of others into a self-concept which then defends "itself" against any perceived threat. And the biggest threat, of course, is the truth that the self is not fundamentally real. When that is clearly seen, the g... (read more)

Eliezer, I suspect the coalition in control of your fingers is not as coherent or stable as it appears. Ruling coalitions like to give the impression that they have little effective opposition and are unified without internal dissent, but the truth is usually otherwise.

That comment was quintessentially Hanson, and an observation whose insight gives me much cause to believe that the coalition in control of those fingers has travelled across many a Rubicon. . .

@Cyan: "... you're going to need more equations and fewer words."

Don't you see a lower-case sigma representing a series every time I say "increasingly"? ;-)

Seriously though, I read a LOT of technical papers and it seems to me much of the beautiful LaTex equations and formulas are only to give the impression of rigor. And there are few equations that could "prove" anything in this area of inquiry.

What would help my case, if it were not already long lost in Eliezer's view, is to have provided examples, references, and commenta... (read more)

Well, OK, but your anti-reductionism is still wrong.

I'll be the first to say I've never been "rebuilt", but I enjoyed GEB. I have next to nothing to protect and don't even fear death so much as dying.

Allbright slipped in. (Mine was a reply to Matthew C.)

Mathew C: "And the biggest threat, of course, is the truth that the self is not fundamentally real. When that is clearly seen, the gig is up."

Spot on. That is by far the biggest impasse I have faced anytime I try to convey a meta-ethics denying the very existence of the "singularity of self" in favor of the self of agency over increasing context. I usually to downplay this aspect until after someone has expressed a practical level of interest, but it's right there out front for those who can see it.

Thanks. Nice to be heard...

Based on the disproportionate reaction from our host, I'm going to sit quietly now.

Someday, the ruling coalition laying claim to the title of "Robin Hanson" will not include the member who remembers what their password is, and we will never find out about it.

On Eliezer's comment about abstract stuff = fake, nothing to say:

If you really want to communicate your ideas, transfer them all the way to another brain, you would try harder to present them so that almost anyone who wants to understand them (with the right level of background info) has no hard time doing so. Instead it's like you cram whole functions or classes into convoluted one-liners like some extreme programmer showing off his chops. Yeah, it may work, but you got to show that your code really runs for us mortals.

I understand Eliezer's ideas witho... (read more)

I had crossed when I was much younger, without realizing what I'd done or the consequences. I wish I was informed, but it's too late now. I guess I committed myself to this path, I might as well see where it leads.

Eliezer: If there is more than one rubicon to cross, is it possible to skip one? Does the question make any sense?

Robin: What coalitions should I expect to see? Who's in charge of Robin Hanson right now?

Jef: Give me exactly one reason why I should listen to you. Ignore his current inability in FAI: nothing you've said has convinced me that he is making a mistake that matters. If the mistake is that big, I can discover the ramifications for myself after I know what's going on.

There's emotion involved. I enjoy calling people's bluffs.

Jef, if you want to argue further here, I would suggest explaining just this one phrase "functional self-similarity of agency extended from the 'individual' to groups".

I have enjoyed what parts I read of Godel, Escher, Bach but I have yet to finish it. As far as being a mathematical thinker... I haven't really identified how my thought process works other than being confused much of the time. I do enjoy math but I don't seem to have much of a talent for it. Haven't really found anything I do have a talent for. I am trying to become more of a mathematical thinker and to construct a good foundation to build a system of knowledge on but I have this strong natural inclination to trust anyone and that tends to undermine some ... (read more)

Thanks, Cassandra, that was the information I was looking for.

It's interesting that others have shared this experience, trying to distance ourselves from, control, or delete too much of ourselves - then having to undo it. I hadn't read of anyone else having this experience, until people started posting here.

The idea of having to reweave yourself out of immediately available parts is startling to me - I remember my own recovery as being more like a chain of "Undos" to restore the original state. I wonder what the alternative would have been like, but I'm not going to try it again to see!

In regards to referring to yourself as a coalition. I am not so sure that would be a good idea. Cassie would be a good little hive drone because she never had any sense of strong identity to begin with. And I like to speak in the third person. ;) Seriously I am mildly uncomfortable with even referring to myself as 'I' these days because I try to keep very careful record of which factors influence my mind and how they influence me and after I add all this up it seems pretty clear to me that I do not exist. Perhaps I am simply wrong but this is the conclusion I have come to.

"As far as my childhood goes I created a lot of problems for myself by trying to force myself into a mold which conflicted strongly with the way my brain was setup."

"It's interesting that others have shared this experience, trying to distance ourselves from, control, or delete too much of ourselves - then having to undo it. I hadn't read of anyone else having this experience, until people started posting here."

For some mysterious reason, my younger self was so oblivious to the world that I never experienced (to my recollection) a massiv... (read more)

...it seems to me much of the beautiful LaTex equations and formulas are only to give the impression of rigor.

I didn't suggest equations to enforce some false notion of rigor -- I suggested them as an aid to clear communication.

When I was a teenager I had a concept I referred to as "the double edged sword of apathy". It was precisely the concept that separating oneself from certain aspects of oneself (which at the time I called fostering apathy) is a destructive tool which can be either positive or negative. Care must be taken not to slice your own arm off.

I don't believe that this danger should be removed though, at least I wouldn't personally allow it. I hold "self-modifying" to be the deepest aspect of life. When we finally get the technology to do source level modifications I won't let an AI do the job: I'll do it myself, regardless of the risk.

I grow more interested in the ideas of William James on this subject. Statements such as:

The passing Thought itself is the only verifiable thinker
Thought is a passing thought that incessantly remembers previous thoughts and appropriates some of them to itself
There is a "judging Thought" that identifies and owns some parts of the stream of consciousness while disowning others
The next moment another Thought takes up the expiring Thought and appropriates it. It binds the individual past facts with each other and with itself.
In this way, what ho... (read more)

This is an awesome and freaky topic.

"It's interesting that others have shared this experience, trying to distance ourselves from, control, or delete too much of ourselves - then having to undo it. I hadn't read of anyone else having this experience, until people started posting here."

Haha... this is indeed kind of weird to see. I am very familiar with this experience.

"Seriously I am mildly uncomfortable with even referring to myself as 'I' these days because I try to keep very careful record of which factors influence my mind and how they in... (read more)

"I gained this kind of reflectivity when I was barely able to even think and I did not know how to use it wisely. One of my first memories is relentlessly purging my early childhood personality shortly after I discovered how to perform the trick then panicking and rebuilding a new self any sort of stuff laying around. Still think that rampant self-modification left scars on my mind that are still there today. Emotional distance did increase the more I examined, altered and experimented with this and eventually caused some really painful side effects. Eek."

Yes, yes, yes.

I'm liking my analogy with "the Way" more and more. I was unsure about it at first.

" " Mathew C: "And the biggest threat, of course, is the truth that the self is not fundamentally real. When that is clearly seen, the gig is up."

Spot on. That is by far the biggest impasse I have faced anytime I try to convey a meta-ethics denying the very existence of the "singularity of self" in favor of the self of agency over increasing context. I usually to downplay this aspect until after someone has expressed a practical level of interest, but it's right there out front for those who can see it. "

I think you are m... (read more)

EY: I remember my own recovery as being more like a chain of "Undos" to restore the original state.

Presuming you're referring to your religious upbringing, that seems like a funny way of characterizing it. Virtually every old primitive civilization that we know about had religious superstitions that all look pretty similar to each other and whose differences are mostly predictable given the civilization's history and geography. (I say "virtually" just as cover; I don't actually know any exceptions). Modern Judaism is a whole lot sane... (read more)

Sorry, I botched the second-to-last sentence. It should read "For example, if you were a cult member between 2004 and 2006, and you_2008 consider both you_2005 and you_2002 to be fools, you_2005 considers you_2002 a fool, but you_2008 consider you_2002 wiser than you_2005, then count that as one improvement rather than two."

I'm pretty consequentialist and its hard to predict the effects of Eliezer's harsh criticism of Jef Albright, but in the absence of a compelling argument that it is for the better, I would not be so mean.

That said, Jef's comments were (perhaps unintentionally) condescending.

As for their substantive disagreement, I have little to add to what they and other commentators are contributing.

It seems that Tom McCabe and Daniel Franke didn't go through anything like what Cassandra and I went through, since they have no referent for at all for the experience, and tried to map it onto belief systems or religions.

If I understand C correctly, the mutual experience that we're describing is nothing like that. You find out how to disable pieces of yourself. Then one day you find that you've disabled too much. It doesn't necessarily have anything to do with religion or even with beliefs, except for whatever beliefs spurred you to start deleting pieces of yourself.

EY: You find out how to disable pieces of yourself. Then one day you find that you've disabled too much. It doesn't necessarily have anything to do with religion or even with beliefs, except for whatever beliefs spurred you to start deleting pieces of yourself.

Okay, I now see what you're saying. I haven't experienced it. I understand the trick of disabling pieces of oneself, but I've never in my recollection abused it. However, I can understand what it would be like because I've experienced something that I'm guessing is similar: I'm a high-functioning autistic, and I've had to put considerable effort into software emulation of the emotional hardware that I'm missing.

Everything I am, is surely my brain; but I don't accept everything my brain does, as "me".
Such an awkwardly phrased and punctuated sentence is evidence of cognitive failure, or at least a hiccup. There's a fundamental mistake you are trying to paper over right at the start of this essay, which goes downhill from there.

Why are hardcore materialists, who presumably have no truck with Cartesian mind/body dualism, so eager to embrace brain/body dualism? Or software/hardware dualism?

So you start by restricting your self to your brain (at least, I... (read more)

trying to distance ourselves from, control, or delete too much of ourselves - then having to undo it.

I cannot recall ever trying to delete or even control a large part of myself, so no opinion there, but "distancing ourselves from ourselves" sounds a lot like developing what some have called an observing self, which is probably a very valuable thing for an person wishing to make a large contribution to the world IMHO.

A person worried about not feeling alive enough would probably get more bang for his buck by avoiding exposure to mercury, which binds permanently to serotonin receptors, causing a kind of deadening.

mtraeven: Why are hardcore materialists, who presumably have no truck with Cartesian mind/body dualism, so eager to embrace brain/body dualism? Or software/hardware dualism?

Has anyone but me brought up software/hardware dualism? I'm only using it metaphorically. I'm not claiming any fundamental, bright-line distinction.

Hmm. I remember being non-reflective in first grade but not in second grade. One consequence was that I couldn't re-write explicit beliefs in response to new information and I saw general injunctions and commands as relatively binding and automatic. Conflicting commands couldn't be accommodated, nor could common sense. I don't think that my emotions were any more intense. I never re-wrote myself, or noticed a change at the time, but I notice it in my memories. Early ones don't include the question "why am I doing this?" or "why do this... (read more)

I think I prefer, and should prefer, my smoothed out highs and lows. During a finite manipulation sequence of a galactic supercluster, whose rules I pre-established, I wouldn't necessarily need to feel much -- since that might feel like 'a lot of pointless muscle straining' -- other than a modest, homo sapiens-level positive reinforcement that it's getting done. Consciousness, if I may also give my best guess, is only good for abstract senses (with and without extensions), and where these abstract senses seem concrete, even to an infinite precision, not "highs" and certainly not "lows" are necessary.

My mom tells me that when I was 2 or 3 years old and in preschool, one day they had a group of kids come up and recite poems or sing songs or something. I got really scared and wouldn't do my poem. Riding home afterwords, I said "I'm ready to say my poem now."

"You can't honey, everybody's already gone," my mom said.

She explained to me why it's important to be ready when it's time to do something.

"I don't want to be scared. I want to be a person who does things when it's time," I said.

I have no recollection of this event, but e... (read more)

The ability to become emotionally detached is a useful skill (e.g. if you are being tortured) but when it becomes an automatic reflex to any emotion, it can take all the colour out of life.

Sometimes highly intelligent people are also overwhelmingly sensitive/empathetic so detaching is very tempting. The first few minutes of this video with the genius girl walking around the spaceship shows what it's like to be highly empathetic (Firefly).
http://www.youtube.com/watch?v=MsyuTLYx59g

But also: emotions come from the subconscious, and the subconscious contains... (read more)

"You find out how to disable pieces of yourself. Then one day you find that you've disabled too much. It doesn't necessarily have anything to do with religion or even with beliefs, except for whatever beliefs spurred you to start deleting pieces of yourself."

you're trying to describe a very common experience using uncommon (and frankly, bizarre - "disabling pieces of yourself"?) language to make yourself feel special. sorry, but that's what many will see when they read through this stuff.

cassandra had a set of dispositions or preferenc... (read more)

I have no memory of a time when I didn't think self-reflexively, I'm pretty sure I was doing it as far back as kindergarden. Though I'm not sure I took it to the extremes of some of you, I only ever did modest personality modification :). I realized I could like any previously hated food just by trying. It's somewhat useful having such great control over thoughts and emotions. Like I've amplified my fear to help me in facing the feared situation (this might sound nonsensical, but it really works). In 6th grade I was confused when the teacher talked about w... (read more)

no system of interesting complexity, including humans, actually attains long-term goals so much as it simply tries to null out the difference between its (evolving) internal model and its perceptions of its present reality.

That's no reason not to talk about goals, and instead only mention something like "utility". Humans are psychologically goal-oriented - i.e. if you talk about goals, people understand what you mean.

Talk about "goals" can be formally translated into talk about "utility", by considering utility to be estim... (read more)

This is a beautiful comment thread. Too rarely do I get to hear anything at all about people's inner lives, so too much of my theory of mind is generalizations from one example.

For example, I would never have guessed any of this about reflectivity. Before reading this post, I didn't think there was such a thing as people who hadn't "crossed the Rubicon", except young children. I guess I was completely wrong.

Either I feel reflective but there's higher level of reflectivity I haven't reached and can't even imagine (which I consider unlikely but am including for purposes of fake humility), I'm misunderstanding what is meant by this post, or I've just always been reflective as far back as I can remember (6? 7?).

The only explanation I can give for that is that I've always had pretty bad obsessive-compulsive disorder which takes the form of completely irrational and inexplicable compulsions to do random things. It was really, really easy to identify those as "external" portions of my brain pestering me, so I could've just gotten in the habit of believing that about other things.

As for the original article, it would be easier to parse if I'd ever heard a good reduction of "I". Godel Escher Bach was brilliant, funny, and fascinating, but for me at least didn't dissolve this question.

Yvian, I too am surprised to be told that there are many people who aren't at stage 2. It's not a bit surprising that most people aren't at stage 3. I've been capable of that kind of thought for as long as I can remember, but it's only since maybe 17 (I'm currently 23) that I've actually had a habit of thinking that way.

I'm amused by the number of people on this thread saying that they've acquired thought habit X through overcoming a mental disorder.

vanveen : Sorry, but your post is a lot more "bizarre" than Cassandra's.

[Eliezer] It's interesting that others have shared this experience, trying to distance ourselves from, control, or delete too much of ourselves - then having to undo it. I hadn't read of anyone else having this experience, until people started posting here.

I think it's only the fact of having consciously distanced yourself that's unusual; I believe it's very common for someone to unconsciously disidentify with some part of their self that they'd be better off having, and have to reintegrate it.

I think this character Eliezer Yudkowsky is straight out of some hard sci-fi novel. A cross-breed between man and machine, or an AI emulating humanness and only partially pulling it off. No way a non-fictional person can come up with all of this ... words fail ... stuff.

But hey, it's the new millenium and it's supposed to be a frikkin' sci-fi life, with humans turned into sci-fi characters, right?!

Out of the 6 billion of us, I think he's one of the most ready for uploading. Meet you on UploadEarth 2045, dude.

So we're all reflective people on this bus. Is that it?

Partly on topic, perhaps someone here can give me a helping hand in my attempt to level-up intellectually. A heavy obstacle for me is that I have a hard time thinking in terms of math, numbers and logic. I can understand concepts on the superficial level and kind of intuitively "feel" their meaning in the back of my mind, but I have a hard time bringing the concepts into the frond of my mind and visualize them in detail using mathematical reasoning. I tend to end up in a sort of "I know that you can calculate X with this information, and kno... (read more)

@Eliezer: There's emotion involved. I enjoy calling people's bluffs.

Jef, if you want to argue further here, I would suggest explaining just this one phrase "functional self-similarity of agency extended from the 'individual' to groups".

Eliezer, it's clear that your suggestion isn't friendly, and I intended not to argue, but rather, to share and participate in building better understanding. But you've turned it into a game which I can either play, or allow you to use it against me. So be it.

The phrase is a simple one, but stripped of context, as... (read more)

Jef: That's more like it. Though part of your explanation is still unnecessarily convoluted and nested (try using shorter sentences), now I get it. It's an alright, non-fake concept/observation. But that still doesn't mean the perception of fakeness/having nothing to say isn't valid. But it may be just a perception in many if not all cases - you need to work harder to avoid being perceived as having nothing to say, that's all - talking a lot and saying a lot instead of talking a lot and saying little/nothing.

@Tim Tyler: "That's no reason not to talk about goals, and instead only mention something like "utility"."

Tim, the problem with expected utility maps directly onto the problem with goals. Each is coherent only to the extent that the future context can be effectively specified (functionally modeled, such that you could interact with it and ask it questions, not to be confused with simply pointing to it.) Applied to a complexly evolving future of increasingly uncertain context, due to combinatorial explosion but also due to critical und... (read more)

You find out how to disable pieces of yourself. Then one day you find that you've disabled too much.

This definitely happened to me. I realized at some point (a few years ago) that in trying to force-fit myself into roles I thought I needed to fill in order to be a "responsible person", I'd succeeded in turning off aspects of my brain that are actually pretty vital for me to learn effectively. I didn't do anything about this realization, though, until I experienced an Epic Fail that showed me that the way I was trying to operate was neither usefu... (read more)

Why must I be like that?
Why must I chase the cat?
Nothin' but the dog in me.

--George Clinton - Atomic Dog

gaffa: A heavy obstacle for me is that I have a hard time thinking in terms of math, numbers and logic. I can understand concepts on the superficial level and kind of intuitively "feel" their meaning in the back of my mind, but I have a hard time bringing the concepts into the frond of my mind and visualize them in detail using mathematical reasoning. I tend to end up in a sort of "I know that you can calculate X with this information, and knowing this is good enough for me"-state, but I'd like to be in the state where I am using the i

Tim, the problem with expected utility maps directly onto the problem with goals. Each is coherent only to the extent that the future context can be effectively specified (functionally modeled, such that you could interact with it and ask it questions, not to be confused with simply pointing to it.) Applied to a complexly evolving future of increasingly uncertain context, due to combinatorial explosion but also due to critical underspecification of priors, we find that ultimately (in the bigger picture) rational decision-making is not so much about "

Main post: Everything I am, is surely my brain.

It would seem that, as far as causes go, everything about any of us is contained in the zygote, long preceding any sort of "brain". Indeed, it would seem to go far more basic than that, as discussed in the Quantum Mechanics series. These recent discussions about ethics, morality, concept of self, etc. seem to be effects, rather than causes, the results of external forces interacting with the original selection and sequence of a relative few chemicals. Who can say that the eventual outcome and expres... (read more)

I find it difficult to be empathetic with people who have had to reject religious thought, because I've never been religious. It has always been clear to me that my parents' beliefs on the subject were absurd; the only change has been in my response to them (patient tolerance to impatient intolerance). I was a conscious atheist even before I realized there was no Santa Claus.

Perhaps that experience was something like what other people go through when they reject religion... except that I cared little about the Santa Claus myth in itself, and was traumatized more by the realization that my parents were willing to lie to me merely for their own personal pleasure than anything else.

Interesting discussion.

Eli,

First, since no one has come out and said it yet, maybe it's just me but this post was kind of whiny. Maybe everyone else here is more in-tune with you (or living in your reality distortion field), but the writing felt like you were secretly trying to make yourself out to be a martyr, fishing for sympathy. Based on my knowledge of you from past interactions and your other writings I doubt this to be the case, but none the less it's the sense I got from your writing.

Second, I, too, have been through a similar experience. When I... (read more)

gaffa, have you tried any Raymond Smullyan? If you want logic and mathematical reasoning in a game-like structure, he has several books (like the linked) that present them as riddles. There is no shame in not getting how they work until you read the solution to at least one in each chapter, but you get the pattern of how to think through certain sorts of logical structures.

When I first came across Eliezer's writings, it struck me that what I read felt so true to me, that for the first time I felt like I had found someone I could relate to.

I have been avidly reading everything from him I could come across, as long as it "felt" right, which was often. With time I noticed that we didn't think in the same way, and it felt to me Eliezer was much more rational, scientifical, structured, than I was.

I immediately felt that the desirable thing to do would be to read even more of him, so as to "absorb" those traits... (read more)

Eliezer: "I don't know if I've mentioned this publicly before..."

You definitely haven't mentioned that publicly in any place that I read, which makes me glad I decided to dip into the comments of this post. I always felt a tacit acceptance, or at least no active disagreement on your part of Jef's posts on similar subjects on SL4 and other online fora. (at least any available to immediate recall)

The subject of what parts of my influences, tendencies and opinions, and identifiable hardware quirks I call 'myself' is a driver of cycles of stress and ... (read more)

It's not a one-way street; with proper technique (e.g. NLP anchoring and reframing methods, to name just a couple) you can change the cached "meaning" of a certain class of events so that they have pretty much any emotional content you choose.

Granted, my personal experiences run in the direction of modifying "negative" things to be positive, and I haven't had much call for keeping around any negative feelings.

Truth is, your concern about losing the negative feelings is irrational... probably based on a science fictional ideal of "not being as human" if you lose the negative emotions. I used to be bothered by that, but then I got rid of the negative emotion I associated with getting rid of negative emotions. ;-)

Seriously, though, you need to distinguish between suppression or detachment/disassociation of a negative emotion, and not having it in the first place. It's like Spock vs the Dali Lama: big difference. At the point where you merely disidentify from an emotion, it's a step backwards.

What's necessary is to detach the emotional "tags" from with the experience - specifically, your brain's cached predictions of the future that will ari... (read more)

I see "Me" as all that is within my skin.

I find it helpful to think of different bits of me. My "Inner Toddler" is the bit which Wants things, or is upset by things. If I just tell it to shut up, it will become recalcitrant. If I listen to it, even though I will not necessarily do what it says, it is happier and I am happier.

This is why I am not Rationalist. I try to use Rationality to understand the World, and "myself", but I use emotion to set my goals, just as it gives me my rewards.

And- I get a great deal from the theories of Carl Rogers, who postulated an Organismic Self, or a "Me" which is my whole organism, and a "Self-concept", which is those bits of me which I allow myself to be conscious of, excluding bits which I am too ashamed of to be conscious of them, and including bits which are not really me, but which I pretend are me because of my concept of "good". I also see myself as an evolved being, and so draw from this that I fit into that habitat which I find myself: if an ancestor did not fit enou... (read more)

Jef Allbright:
So "Functional self-similarity of agency extended from the 'individual' to groups.", in other words, means "groups of humans follow similar practices to achieve their goals"? Or am I missing some mystic subtlety in the choice of "functional" over "similar", "self-similarity" over "a grouping like its parts", and agency over "method of achieving goals"? You took a lot of time to dance around the point that "groups also exclude of parts of themselves for similar reasons... (read more)

Responding to Gaffa (I kind of intended to respond right after the comment, bot got sidetracked):

When approaching a scientific or mathematical problem, I often find myself trying hard to avoid having to calculate and reason, and instead try to reach for an "intuitive" understanding in the back of my mind, but that understanding, if I can even find it, is rarely sufficient when dealing with actual problems.

I would advise you to embrace calculation and reason, but just make sure you think about what you are doing and why. Use the tools, but try... (read more)

I seem to have taken step #3 in the other direction. The bundle of desires, urges, and subconscious cognition that makes me feel things is "me", while that running voice in my head that wonders about why I feel what I feel, is, well, just a running voice in my head doing some analysis, but with no power to affect much of anything. My mind has basically never been able to overrule the parts of the brain that feel like "me", which gives my parents no end of frustration, as they repeatedly insist that people have to learn to do things they... (read more)

Eliezer: "It makes you more moral (at least in my experience and observation) because it gives you the opportunity to make moral choices about things that would otherwise be taken for granted, or decided for you by your brain."

I have to take specific issue with this (despite being further down in the comments than I think will attract anyone's attention). This post and its comments discuss a process by which a mind can modify its behavior through reflection, but while Eliezer and many others may use this power to strengthen their morality, it ca... (read more)

I know this is a few days late, but I couldn't help but notice that no one mentioned how your "Zenlike but not Zen" philosophy is basically just a weak version of Stoicism (weak in that you seem to desire some passion, whereas a stoic would advocate distancing yourself from all highs and lows). There is no need to create techniques to do this from scratch, the path has already been laid out. I would encourage anyone interested in the topic to research Stoic teachings, particularly Epictetus, if you haven't done so already.

[I recommend Epictet... (read more)

Say, Eliezer: Have you considered identifying yourself only with the part of your brain which asks clearly stated questions? This questioner is most clearly "you", IMO.

For example, I could draw the boundary of my selfhood as including the times/ parts of my brain, when I ask clear and well stated questions like: "What is this guy saying?"

Notably the rabbis of old made a step in the right direction: they created a vision of "the evil desire" as an external, anthropomorphic, force. This is already helpful to create a layer between the emotions and the "person", and muster the natural aggressive emotions against it. (and no, they did not think devil is an actual sadist with horns).

My criticism of this approach is that this framework is not fine enough to give you an advantage battling the "evil inclination": viewing the immoral desire as an external force, rather... (read more)

Robin Hanson comments that the "I" even in a reflective person's mind is an unstable coalition.

My guess is that Eliezer knows this, and is defining his "self" to mean something like "the shifting coalition within this brain, that is trying to save the world". If this guess is wrong, I'd love to find out, this seems like the crucial bit to me.

I draw the lines differently.

I include the thoughts that I do not want to have as parts of my self, parts that I do not fully understand or control. I observe these thoughts, and I take care not to act on them, but I do not reject them from my definition of self.

I think something like "I want to be a person who does not think or feel X on a regular basis, but right now I am a person who is experiencing X." I put it in more neutral terms than rejection. I do not think of it as an imposition by something separate, although sometimes I get frustr... (read more)