The Superhappies' decision seems reasonable. I am not sure what alternative solution might be. Hrm.

That just made my day there. Very funny!

In regards to referring to yourself as a coalition. I am not so sure that would be a good idea. Cassie would be a good little hive drone because she never had any sense of strong identity to begin with. And I like to speak in the third person. ;) Seriously I am mildly uncomfortable with even referring to myself as 'I' these days because I try to keep very careful record of which factors influence my mind and how they influence me and after I add all this up it seems pretty clear to me that I do not exist. Perhaps I am simply wrong but this is the conclusion I have come to.

I have enjoyed what parts I read of Godel, Escher, Bach but I have yet to finish it. As far as being a mathematical thinker... I haven't really identified how my thought process works other than being confused much of the time. I do enjoy math but I don't seem to have much of a talent for it. Haven't really found anything I do have a talent for. I am trying to become more of a mathematical thinker and to construct a good foundation to build a system of knowledge on but I have this strong natural inclination to trust anyone and that tends to undermine some of my efforts. My current plan to counter that is to simply find people that I can trust and fill my head up with good stuff so I can use it to block out the bad stuff.

As far as my childhood goes I created a lot of problems for myself by trying to force myself into a mold which conflicted strongly with the way my brain was setup. For most of my youth and teenage years I was this weird wanna-be artist anti-rationality, anti-science stereotype that was stuck in a sophistic nightmare for years. Luckily I can look back on that cringe in horror. I do realize now that there was no way that I could of ever been successful in that field even if I hadn't made mistake after mistake because my creative writing, for example, reads like a VCR manual. I have noticed now that I have stopped struggling against my general nature and adopted more technical and rational based approaches that I do so much better.

I still suck at this whole mathematical thinking thing but I believe that has more to do with me working to undo years of stupid shit and lacking the necessary experience than a lack of talent maybe. My entire life is just one big game of catchup right now and its extraordinarily stressful. Not sure if this is the story you wanted to hear, feel free to send me an email or something I guess, if you wanted different information.

I gained this kind of reflectivity when I was barely able to even think and I did not know how to use it wisely. One of my first memories is relentlessly purging my early childhood personality shortly after I discovered how to perform the trick then panicking and rebuilding a new self any sort of stuff laying around. Still think that rampant self-modification left scars on my mind that are still there today. Emotional distance did increase the more I examined, altered and experimented with this and eventually caused some really painful side effects. Eek.

My own potential intelligence does worry me fairly often. I am currently studying to become an engineer and hope to work on some of the awesome ideas I read about on sites like this. The thing is though, I wasted the first twenty third years of my life. I am currently at twenty-five years old and I have been forced to pretty much start from scratch on everything from social skills to education and after two years I think I am making some headway. I am even starting to understand what Eliezer talks about in all these posts and apply it to my own life as best I can. The math still escapes me but I managed to make it through about half of the Bayesian explanation before getting completely and utterly lost. So I think it is certainly possible learn a huge amount of things even after young childhood but it is rather less efficient. I have had to really struggle to get to where I am now. And since Eliezer is one of the big reasons I am so excited about getting into science I would like to attempt to work in a similar field of research. Yep.

What I always wonder is why we need to preserve human values like autonomy if we could produce better results without it? For example, if an AI could compute the absolute best way to perform a specific action then why is it a good thing to be able to choose a different way to perform the action?

See ya later! I continue to enjoy reading your posts when ever you make them but I am sure a person needs to rest occasionally. Yep.

I think what Eliezer is saying is that our evolutionary psychology, memetic history and reaction to current moral arguments form the computational trajectory for our moral judgment. All the points on this trajectory are acceptable moral judgments but when new experiences are fed back through base program this trajectory can shift. The shift takes place at the base of the line as it extends from the program, rather than curving in the middle to include all the current moral values. Moral values that are contacted by this line are good and any that aren't contacted by this line are not good, like an off-on switch. This is because current moral judgments flow backwards.

The aggregate moral trajectory adds up to humanity's morality when the function is filtered through the base program once again. So it continues to perform an update loop. Now if we edit the base program then it no longer provides consistent answers. This would be like taking a pill that makes it 'morally right' to kill people. What I am stuck on is how we could edit the base program and still have it produce consistent answers.