if both players play (C, C) and then divide up the points evenly at the end, isn't that sort of... well... communism?
Eliezer, you have just replaced Reeves' substance with your own symbol. What's your point here?
Untranslatable 2 is the thought sharing sex.
Sprite, you are, by definition, wrong.
Thirding D Franke's idea.
Eliezer, a thought occurs. I'm sure the new setup will be great for everyone who wants to make sure they're using the right priors and calculating the correct odds about whether to bet on Obama or not. Or indeed, trying their utmost to eliminate every source of bias from their life and turn into a giant lookup table or something. But I've much preferred reading your assorted ramblings on things like quantum mechanics, timeless physics, and especially low-level AI theory. Wrong motives? Meh, maybe. I'm sure the answer is 'the elimin... (read more)
Or indeed Marcello Herreshoff?
Exciting stuff. Looking forward to having the OB back catalogue sorted into sequences. That'll make it much easier for me to badger everyone I know to get reading.
Rob - that's because The Wire is more like real life than real life.
I can see the link to the Chronophone here Eliezer. What would Benny F have found most shocking about today? How can we extrapolate that forwards?
Surely the most scary changes will be in ethics and the way we think of the human condition and personal identity.
I'm currently most of the way through The Mind's I, and if Hofstadter's (very plausible) musings on identity are anything like accurate, we're going to have to start thinking very differently about who we are, and even whether that qu... (read more)
Eliezer, does The Adaptive Stereo have an analogic application here?
To compress the idea, if you slowly turn down the strength of a signal into perception (in this case sound), you can make large, pleasant, periodic steps up without actually going anywhere. Or at least, you can go slower.
Any logical reason why this wouldn't work for hedons? 'Going digital' might nullify this effect, but in that case we just wouldn't do that, right?
Finally, I would dispute the notion that a periodic incremental increase in hedons flowing into us is how human pleasure works.... (read more)
[...]my experience of drugs is as nonexistent as my experience of torture.
There's something imbalanced about that.
Agreed. I'm sure both can be procured somewhere in the Bay Area though. Great material for blogging too!
Is the equivalent pleasure one that overrides everything with the demand to continue and repeat it?
Yes. And that's as horrible an idea as eternal torture. I'm surprised you haven't cited any of the studies about relative happiness of lottery winners, (compared to their expectations) though I seem to remember references in some of the posts ab... (read more)
Thus fails the Utopia of playing lots of really cool video games forever.
Not convinced. Based on my experience of what people are like; from the moment where games are immersive enough, and we have the technology to plug in for good, the problem of 'no lasting consequences' will vanish for people who want it to. There are already plenty of people willing to plug into WoW for arbitrary amounts of time if they are able. Small increases in immersiveness and catheter technology will lead to large increases in uptime.
phane touches on something interesting just ... (read more)
Hey Rick Astley! Much better than this decision theory crap.
Came across this at work yesterday, which isn't unrelated. For every level of abstraction involved in a decision, or extra option added, I guess we should just accept that 50% of the population will fall by the wayside. Or start teaching decision theory in little school.
Happy Nondenominational Winter Holiday Period, all. Keep it rational.
I have included in the envelope a means of identifying myself when I claim the money, so that it cannot be claimed by someone impersonating me.
Doesn't that technically make you now Known?
Also, how much time has to pass between an AI 'coming to' and the world ending? What constitutes an AI for this bet?
Eliezer, will you be donating the $10 to the Institute? If so, does this constitute using the wager to shift the odds in your favour, however slightly?
Yes, the last two are jokes. But the first two are serious.
Anonymous, that reminds me of some anecdote by Feynman where he has complex mathematical ideas described to him by young students. He wouldn't fully understand them, but he would imagine a shape, and for each new concept he'd add an extra bit, like a squiggly tail or other appendage. When something didn't fit in right, it would be instantly obvious to him, even if he couldn't explain exactly why.
Improvised sensory modality for maths?
And note that Eliezer never answered your question, namely, if you can modify yourself so that you never get bored, do you care about or need to have fun?
Richard, probably you wouldn't care or need to have fun. But why would you do that? Modifying yourself that way would just demonstrate that you value the means of fun more than the ends. Even if you could make that modification, would you?
How odd, I just finished reading The State of the Art yesterday. And even stranger, I thought 'Theory of Fun' while reading it. Also, nowhere near the first time that something I've been reading has come up here in a short timeframe. Need to spend less time on this blog!
Trying to anticipate the next few posts without reading:
Any Theory of Fun will have to focus on that elusive magical barrier that distinguishes what we do from what Orgasmium does. Why should it be that we place a different on earning fun from simply mainlining it? The intuitive answer is t... (read more)
Phil, what Vlad and Nick said. I've no doubt we won't look much like this in 100 years, but it's still humanity and its heritage shaping the future. Go extinct and you ain't shaping nothing. This isn't a magical boundary, it's a pretty well-defined one.
'Precise steering' in your sense has never existed historically, yet we exist in a non-null state.
Aron, Robin, we're only just entering the phase during which we can steer things to either a really bad or really good place. Only thinking in the short term, even if you're not confident in your predictions, is pretty irresponsible when you consider what our relative capabilities might be in 25, 50, 100 years.
There's absolutely no guarantee that humanity won't go the way of the neanderthal in the grand scheme of things. They probably 'thought' of themselves a... (read more)
in a Big World, I don't have to worry as much about creating diversity or giving possibilities a chance to exist, relative to how much I worry about average quality of life for sentients.
Can't say fairer than that.
Eliezer, given the proportion of your selves that get run over every day, have you stopped crossing the road? Leaving the house?
Or do you just make sure that you improve the standard of living for everyone in your Hubble Sphere by a certain number of utilons and call it a good day on average?
design cycles have stayed about the same length while chips have gotten hundreds of times more complex, and also much faster, both of which soak up computing power.
So...if you use chip x to simulate its successor chip y, and chip y to simulate its successor, chip z, the complexity and speed progressions both scale at exactly the right ratio to keep simulation times roughly constant? Interesting stuff.
Sounds as though the introduction of black-box 2015 chips would lead to a small bump and level off quite quickly, short of a few huge insights, which Jed seem... (read more)
Also, while economists have many abstractions for modeling details of labor teams and labor markets, our standard is that the simplest versions should be of just a single aggregate quantity of labor.
Granted, but as long as we can assume that things like numbers of workers, hours worked and level of training won't drop through the floor, then brain emulation or uploading should naturally lead to productivity going through the roof shouldn't it?
Or is that just a wild abstraction with no corroborating features whatsoever?
because our computing hardware has run so far ahead of AI theory, we have incredibly fast computers we don't know how to use for thinking; getting AI right could produce a huge, discontinuous jolt, as the speed of high-grade thought on this planet suddenly dropped into computer time.
Now there's a scary thought.
Right, that's it, I'm gonna start cooking up some nitroglycerin and book my Eurostar ticket tonight. Who's with me?
I dread to think of the proportion of my selves that have already suffered horrible gravitational death.
Eliezer: part of the AIXI sequence, which I don't think I'll end up writing.
Ahh, that's a shame, though fully understood. Don't suppose you (or anyone) can link to some literature about AIXI? Haven't been able to find anything comprehensive yet comprehensible to an amateur.
Tim Tyler: Brainpower went into making new brains historically - via sexual selection. Feedback from the previous generation of brains into the next generation has taken place historically.
Tim, Dawkins has a nice sequence in The Blind Watchmaker about a species of bird in which the femal... (read more)
Andrew, we're not talking about the equivalent of a human studying neuroscience by groping in the dark. If an AI truly, truly groks every line of its own code, it can pretty much do what it wants with it. No need for trial and error when you have full visibility and understanding of every neuron in your head.
How, you ask? What do such recursive algorithms look like? Mere details; the code monkeys can worry about all that stuff!
Know the feeling. I'm a fully qualified ex-Catholic atheist, but when my girlfriend told me that her family generally just has a pasta dish on Christmas day I was shocked. Anything but turkey makes baby Jesus cry!
Those childhood priors sure get burnt in deeply.
Silas - yeah, that's about the size of it.
Eliezer, when you come to edit this for popular publication, lose the maths, or at least put it in an endnote. You're good enough at explaining concepts that if someone's going to get it, they're going to get it without the equations. However, a number of those people will switch off there and then. I skipped it and did fine, but algebra is a stop sign for a number of very intelligent people I know.
So...noise can be useful in decision theory as long as you don't expect it to do any work. And the mistake gets easier to make the more complex your system. Sounds right enough to me.
Your 'by definition' link needs a look, Eliezer.
Or imagine that the combination changes every second. In this case, 0-0-0-0, 0-0-0-0 is just as good as the randomized algorithm - no better and no worse.
If it changes every second, trying the same set of four over and over is marginally better than random.
If you've just entered 0-0-0-0 and got it wrong, then on the nex... (read more)
Here is the ultimate work of Modern Art, that truly defies all rules: It isn't mine, it isn't real, and no one knows it exists...
Great post for the most part, though I do have to agree with Tim's straw man alert.
Something I learnt while studying postmodern fiction (yeah Eliezer, that's right): Art can be referential, or memetic, or both, or neither. Most is both, in that it (very roughly) is 'like' reality (i.e. it's memetic) and 'seeks to tell us something about' reality (i.e. it's referential). However, there's some really interest... (read more)
/asks self if he should beat his wife
/realises self is not married
I think we should rename it "Robin and Eliezers' Varied Thoughts".
John, Alex; Meh. Long as it's interesting who cares? Nobody promised anyone posts on any particular topic, and nodoby's forcing your mouse clicks. If it makes you feel better, rename your bookmark 'Metaphysical Singularity Sci-Fi'.
the strong whiff of sci-fi geekdom that pervades most of Eliezer's posts.
You say it like it's a bad thing.
What would effective cryo policy look like? Or conversely, what in current policy is inhibiting the proper development of cryogenics?
Ruling parties come and go in waves. Work out when you reckon you'll be unfrozen and vote with that year's election in mind.
Question: if you're on your deathbed and about to have your head frozen, should you be allowed to pre-register your votes for the next few elections? "Palin's counting on a low turnout amongst the dead for 2016, as they tend to vote primarily for the Democratic candidate."
Oh, by the way, well done America.
"If your dreams and aspirations interfere with your rationality and impartiality then you're no longer following the Way. The Way doesn't drive one off course; one rather loses sight of the Way and hence goes off course."
[The Book of Eliezer 4:24 18-20]
Oh, and don't forget the Mystical Intertubes of Communication, which allow any person with access to the Tubes to 'post' their opinions for others to peruse. Even better, other Intertube users can append inanities to any of these essays with the minimum of thought and effort!
Wow. On around 20 minutes Jaron wraps his irrationality up in so much floral language it's impossible to follow. There's no arguing with that, but you had a really good stab, Eliezer. I'd have snapped at all the implied barbs. Fascinating all the way through. Three cheers for physical reality!
Possession of a single Eye is said to make the bearer equivalent to royalty.
How about the miraculous ability to synthesise or isolate compounds of chemicals from the world that recreate sensations, or even push perception beyond the sensations for which it was designed? I'm always pretty impressed by that one.
A general theory of intelligence designed for constructing AI's does not need to be universally applicable.
I think the idea is that once that AI is running, it would be nice to have an objective measure of just how pwerful it is, over and above how efficiently it can build a car.
From The Bedrock of Morality:
For every mind that thinks that terminal value Y follows from moral argument X, there will be an equal and opposite mind who thinks that terminal value not-Y follows from moral argument X.
Does the same apply to optimisation processes? In other words, for every mind that sees you flicking the switch to save the universe, does another mind see only the photon of 'waste' brain heat and think 'photon maximiser accidentally hits switch'? Does this question have implications for impartial measurements of, say, 'impressiveness' or 'ef... (read more)
an outcome that ranks high in your preference ordering
Well if Garry's wins are in the centre of your preference ordering circle of course you'll lose! Some fighting spirit please!
Oh, and if something maximising entropy is a valid optimisation process, then surely everything is an optimisation process and the term becomes useless? Optimisation processes lead (locally) away from maximal entropy, not towards it, right?
I would be rather not be around people who kept telling me true minutiae about the world and he cosmos, if they have no bearing on the problems I am trying to solve.
Will, not wishing to be told pointless details is not the same as deluding yourself.
I was discussing the placebo effect with a friend last night though, and found myself arguing that this could well be an example of a time when more true knowledge could hurt. Paternalistic issues aside, people appear to get healthier when they believe falsehoods about the effectiveness of, say, homeopathy or s... (read more)
From a strictly Bayesian point of view that seems to me to be the overwhelmingly more probably explanation.
Now that's below the belt.... ;)
Too much at stake for that sort of thing I reckon. All it takes is a quick copy and paste of those lines and goodbye career. Plus, y'know, all that ethics stuff.
Throttling an AI to human intelligence is like aiming your brand new superweapon at the world with the safety catch on. Potentially interesting, but really not worth the risk.
Besides, Eliezer would probably say that the F in FAI is the point of the code, not a module bolted into the code. There's no 'building the AI and tweaking the morality'. Either it's spot on when it's switched on, or it's unsafe.
David, the concept behind the term Singularity refers to our inability to predict what happens on the other side.
However, you don't even have to hold with the theory of a technological Singularity to appreciate the idea that an intelligence even slightly higher than our own (not to mention orders of magnitudes faster, and certainly not to mention self-optimizing) would probably be able to do things we can't imagine. Is it worth taking the risk?
if you can provide us with some examples - say, ten or twenty - of scientists who had success using this approach.
Phil, the low prevalence of breakthroughs made using this approach is evidence of science's historical link with serendipity. What it is not is evidence that 'Bayesian precision' as Eliezer describes it is not a necessary approach when the nature of the problem calls for it.
Recall the sequence around 'Faster than Einstein'. From a top-down capital-S Science point of view, there's nothing wrong with pootling around waiting for that 'hmmm... (read more)
the probabilities for cryonics look good.
They don't have to look good, they just have to beat the probabilities of your mind surviving the alternatives. Current alternatives: cremation, interment, scattering over your favourite football pitch. Currently I'm wavering between cryonics and Old Trafford.
Eliezer, I'm ridiculously excited about the next fifty years, and only slightly less excited about the fun theory sequence. Hope it chimes with my own.
What you're saying is that once we've suspended our disbelief about the Force as a cool mysterious property of the universe (three films earlier) there's no call for it to be explained, particularly not in such a crappy way.
That's fair enough.
In the conceptually impossible possible world where the Force exists in the first place, midichlorians are a foreign invader in the simplest explanation of the Force's structure. You want to move something, therefore it moves.
Fascinating. I'd have thought that a chance to render the Force into a physical instantiation would have been music to your ears.
Magnets can pick up paperclips even when you don't know about electromagnetism. However, to fully understand the magnet, you need a theory of electrons. If you want to use the Force to move something, you do... (read more)
Stupid HTML. The link to IMDB above is still good.
Eliezer, one's mastery of the Force isn't based solely on practice, but on the prevalence of Midichlorians in your blood. Due to his family ties, Luke has plenty - it's just the application and faith that he lacks.
This scene rang very true with me and I don't agree with your gripes. Luke has been training like mad for weeks. He's still at the stage of balancing rocks, while his friends are in great danger and he has no way of reaching them. His frustration reaches breaking point in this scene, hence the sulk. In an ideal world Yoda would use this as... (read more)
Shane E, meet Caledonian. Caledonian, Shane E.
Nick T - it's worse than that. You'd have to mathematically demonstrate that your novel was both completely American and infallibly Great before you could be sure it wouldn't destroy the world. The failure state of writing a good book is a lot bigger than the failure state of writing a good AI.
Pinprick - bear in mind that if Eliezer considers you more than one level beneath him, your praise will be studiously ignored ;).
Yadda yadda yadda, show us the code.
Yes, I'm kidding. Small typo/missing word, end of first paragraph.
Phil, very well articulated and interesting stuff. Have you seen Wall-E? It's the scenario your post warns against, but with physical instead of evolutionary fitness.
I agree that Eliezer seems to have brushed aside your viewpoint withough giving it due deliberation, when the topic of the ethics of transcending evolution seems right up his street for blogging on.
However: It considers the "preferences" (which I, being a materialist, interpret as "statistical tendencies" of organisms, or of populations; but not of the dynamic system. Why d... (read more)
Post I'd like to read: Eliezer's Chrono-Conference Call With His Various Previous Selves.
You could even have Eliezer-2018 make an appearance towards the end. Oh, and please write it in the style of the GEB dialogues.