In light of how important it is (especially in terms of all the decision theory posts!) to know about Judea Pearl's causality ideas, I thought I might share something that helped me get up to speed on it: this lecture on Reasoning with Cause and Effect.
For me, it has the right combination of detail and brevity. Other books and papers on Pearlean causality were either too pedantic or too vague about the details, but I learned a lot from the slides, which come with good notes. Anyone know what proceedings papers it refers to though?
Does anyone feel the occasional temptation to become religious?
I do, sometimes, and push it away each time. I doubt that I could, really, fall for that temptation - even if I tried to, the illogicality of the whole thing would very likely prevent me from really believing in even a part of it very seriously. And as more and more time passes, religious structures begin to seem more and more ridiculous and contrived. Not that I'd have believed that them starting to feel more contrived would even have been possible.
And yet... occasionally I remember the time back in my teens, when I had some sort of a faith. I remember the feeling of ultimate safety it brought with it - the knowledge that no matter what happens, everything will turn out well in the end. It might be a good thing that I spend time worrying over existential risks, and spend time thinking about what I could do about them, but it sure doesn't exactly improve my mental health. The thought of returning to the mindset of a believer appeals on an emotional level, in the same way stressed adults might longingly remember the carefree days of childhood. But while you can't become a child again, becoming a believer is at least theoretically possible. And sometimes I do play around with the idea of what it'd be like, to adopt a belief again.
Uh, here is a confession. Twice in the last 6-7 years, at moments of extreme
psychological distress, I talked to the God of my Catholic youth. Once I went to
an empty church after a series of coincidences (running into two people from my
Catholic grade school separately) that I thought was a sign from God. Really,
embarrassing, right? It was like my mind segmented and the rationalist was put
aside and the devout Catholic school boy was put in charge. The last time this
happened was about four years ago.
I don't know if this could still happen today (my atheism is probably more
entrenched now). There have been plenty of periods of distress where this didn't
happen, so I don't know what triggered it in particular. I think my brain must
have really needed a God figure at that moment and didn't know how to deal with
the pain without one so it hacked itself and turned off the rationalist
defenses. Or something, it seems so screwed up looking back on it.
Interestingly, I told my theistic-non religious girlfriend about this who in
turn told her Christian best friend. Talking to me on the phone for the first
time, the friend something along the lines of "I know you say you're an atheist
but B(my girlfriend) told me that you sometimes pray when you're upset so I know
you're really a good person/God loves you." In other words, I'm least wrong when
I'm thinking least clearly.
On the consolidation of dust specks and the preservation of utilitarian conclusions:
Suppose that you were going to live for at least 3^^^3 seconds. (If you claim that you cannot usefully imagine a lifespan of 3^^^3 seconds or greater, I must insist that you concede that you also cannot usefully imagine a group of 3^^^3 persons. After all, persons are a good deal more complicated than seconds, and you have experienced more seconds than people.)
Suppose that while you are contemplating how to spend your 3^^^3-plus seconds, you are presented with a binary ch... (read more)
If this choice was actually presented to someone, my guess would be that he
would first choose the speck, and then after an extremely long time (i.e. much,
much longer than 50 years, giving him a sense of proportion) he would undergo a
preference reversal and ask for the torture.
2Eliezer Yudkowsky12y
What if the TORTURE occurs during a random time in the next 3^^^3 seconds, not
right at the beginning? Also, I think we definitely require a limit on sanity
damage because otherwise the scenario is being tortured for 50 years and then
spending the next 3^^^3 seconds being insane which Vastly outweighs the ordinary
scenario of being tortured for 50 years.
0Alicorn12y
In the original scenario, where just some random person got tortured, no
constraints were specified about eir sanity or lifespan post-torture.
1Eliezer Yudkowsky12y
I think I did specify that no one would die who would otherwise be immortal;
eternal insanity or 3^^^3 years of insanity ought to be implicitly included, I'd
think.
2Larks13y
This phrase makes the difference for me- the 3^^^3 other people in the original
argument weren't mad- or at least, no more than would have been mad anyway.
Additionally, in your scenario, we have to consider discount rates- it's
certainly conceivable that someone might choose the dust specks over torture
now, but be willing to forgo the dust specks in return for torture in 3^^^3
seconds time.
4Alicorn13y
Does it seem likely to you that out of 3^^^3 people chosen with no particular
safeguards, not one of them will find a dust speck in the eye to be maddening?
It could be the last straw in a string of misfortunes; it could set off some
causal chain that will lead to other maddening events, etc.
1Larks13y
3^^^3 is such a huge number, some must find it maddening, but the proportion
will be a lot lower than the odds that 50 years of torture breaks you mentally.
2Emily13y
That's interesting: I have much, much less hesitation in saying TORTURE to this
one. With the original, I can grudgingly concede that I suppose I possibly ought
to choose TORTURE, but I still can't ever quite convince myself to feel that
it's a good answer. This one, I think I can.
7Wei_Dai13y
Suppose you are given a button that you can press at any time during the 50
years of torture, that will stop the torture (and erase your memory of it if you
wish), but you'll have to live with the dust speck from then on.
I predict that you'll press the button after actually being tortured for a
couple of hours, maybe days, but at most weeks. Even professional
spies/soldiers/terrorists who have trained to resist torture end up betraying
their cause, so I find it hard to believe that you can hold out for 50 years.
But if you really prefer TORTURE now, that brings up an interesting question:
whose preferences are more important, the current you, or the hypothetical
future you? It could be argued that the future you is in a better position to
decide, since she knows what it actually feels like to be tortured for a
significant period of time, whereas you don't.
But I don't consider that a knock-down argument, so what do you think? Suppose
you can also commit to not pressing the button (say by disabling your arm/hand
muscles for 50 years), would you do so?
(This is related to a recent comment
[http://lesswrong.com/lw/17h/the_lifespan_dilemma/13qr] by Rolf Andreassen,
which I think applies better to this scenario.)
0Emily13y
Yes, I think you're almost certainly right about the button, which thought does
indeed put a dent in my lesser hesitation in choosing TORTURE. I think I would
definitely not commit to not pressing the button if I were able to "try out" the
SPECKS scenario for some short period of time first (say, a week). That way I
could make a comparison. Absent that condition... I don't know. I can't imagine
having the guts to commit to not pressing it.
The fact is that while I certainly don't see a momentary dust speck as torture,
I can easily imagine beginning to see it as torture after a day, never mind
3^^^3 seconds, which is rather longer than 50 years. But I can't be certain of
that, nor of to what extent I would get used to it, nor of how it would compare
with much worse kinds of torture. (But then again, there's a spanner in the
works: knowing that you won't have any lasting physical damage... perhaps that
would lend some kind of strength of mind to a torturee?)
Sean Carroll and Carl Zimmer are leaving Bloggingheads, mostly because it's started playing nice with creationists. Click their names to read their full explanations.
Thanks for the link. I saw the original Behe podcast and was surprised when it
suddenly disappeared with a brief note from an admin. I haven't been able to
follow up on this matter much since it hasn't generated much buzz, but I'll look
forward to reading these links - if people are boycotting Bloggingheads for
giving platforms to people whom they don't like.... well, that's an interesting
precedent to set which should be scrutinized carefully.
1Eliezer Yudkowsky13y
Hm. (Reads.) Well... if BHTV has me on, an "anti-accomodationist" as I've
recently heard my diavlogs called (what a lovely term!), then I don't think it's
unfair for them to give Behe a slot. It is extremely foolish that they paired
him up with a linguist. (Reads.) Looks like Robert Wright agrees with that last
part.
2Furcas13y
You're one of the last people I would have expected to be concerned with
'fairness' when one of the sides is blatant crackpottery. I suppose you wouldn't
have a problem if BHTV invited an astrologer, as long as they paired him up with
an astronomer?
0Peter_Twieg13y
What harm is done by bringing on an astrologer? At worst it fail to amuse.
But it's obvious you're not talking about the diavlog's impact on you... you're
concerned with the poor, unwashed masses who might actually be left to form
their own opinions from the available information. Well, that's very nice of
you, but I don't think it's unreasonable to believe that it's safe to expose
people to views which might be labeled as "crackpottery" by some.
5Vladimir_Nesov13y
High-status serious people want to associate with high-status others. Allowing
crackpots on the same venue dramatically decreases its attractiveness for
quality participants.
You can't really give an elaborate justification for why in this particular case
it's OK, because signals are shallow
[http://www.overcomingbias.com/2009/06/why-signals-are-shallow.html].
5Furcas13y
The "unwashed masses", as you call them, are already getting plenty of
exposition to crackpottery, much more than they get to real science, so that a
few crackpots on BHTV are barely a drop in the ocean. That's not what concerns
me, no.
What concerns me is that BHTV has a reputation as a respectable website thanks
to the participation of respectable academics and experts. It's reasonable to
assume that such a respectable website wouldn't invite crackpots to promote
their brand of crackpottery; in fact, that's an assumption I made myself until I
read Sean's and Carl's posts. Inviting crackpots therefore gives the impression
that these people should be taken seriously, even if we think they're wrong.
In any event, your own motives are suspect, to say the least. Characterizing
creationist nuts as "people whom [Sean and Carl] don't like", as if creationism
was merely a distasteful political opinion, or something, makes you sound like a
crackpot yourself, or worse, a postmodernist.
-1Peter_Twieg13y
So BHTV can't both enjoy the participation of respectable academics and also
host the occasional crackpot? There exists no such universe where the two could
possibly coincide? Is there some implicit assumption here that there's a fixed
amount of BHTV episodes, each of which will feature either crackpots or
respectable academics? Even if this were so, wouldn't the reasonable response be
to skip over the crackpots rather than avoiding the entire medium? The only
justifiable rationale I can see for skipping over BHTV because of this is if you
just watched diavlogs at random and having crackpots degraded the signal:noise
ratio of the site. But I doubt that you, I, Sean, Carl, or your average
Bloggingheads viewer navigates the site in this manner.
Even though I profoundly degree with Behe's epistemology (and theology), which
should go without saying in these parts, I found the debate interesting (I think
irreducible complexity is a neat topic), certainly moreso than I've enjoyed
other diavlogs. Can anyone honestly say that Behe's presence is less valuable
than any other podcast on the website? I doubt it, and thus it strikes me as
disingenuous that the unique response his presence generates can be explained
away purely through outrage at the notion that somewhere, someone's time may be
wasted.
And I didn't include the "people whom others don't like" line as a defense of
creationism per se, but as a broader point about silencing views found in
contempt. To rip off Will Wilkinson, I'd probably venture to assert that
unrepentant Marxists are just as high on the crackpottery scale as Creationists,
but I highly doubt we'd see people abandon the site in protest in BHTV hosted
some of them. "Respectability" in this context is a tricky term to use, since
"respectability" tends to be conferred by social fashions just as much as actual
correspondence to whatever virtues we've deemed to be worthy of respect. On a
more base level, I suspect that many participants in this community
2eirenicon13y
Should BHTV invite Perez Hilton to debate the fearsome Man Bat? Michael Behe is
as credible an author as Pamela Anderson, although not quite as illuminating. I
used to think that the worst kind of ignorance was when you knew you were wrong
and refused to accept it. Now I think the worst kind is when you know you're
capable of knowing when you're wrong but refuse to let yourself. Michael Behe
wants to be ignorant of his own ignorance. Let him do so in the peace and quiet
of his own sad little world.
1Peter_Twieg13y
You're shifting the goalposts some. I'm not defending the original decision to
invite Behe. I'm questioning the notion that inviting Behe is such an egregious
offense against BHTV's "respectability" that it should be boycotted. I wouldn't
boycott BHTV if 90% of the diavlogs were replaced by midget porn, if it meant
that I would get the occasional episode of Free Will.
I think Behe's critics should just admit that what's really motivating the
reaction is the notion that Creationists not only should not be given forums to
speak, but those who do grant Creationists forums to speak should be actively
identified and boycotted in a way which is reserved for an arguably
arbitrarily-defined set of social undesirables. This isn't an indefensible
position, but people have to admit to holding this belief (or some similar
belief which is constructed in a more-charitable manner) before a meaningful
debate can be enjoined.
[Edit]
Reading over the comments section of the CV posts, it looks like a lot of people
are quick to point to Megan McArdle as the political crackpot equivalent of
Behe. Should her presence be boycotted too as detrimental to the site? Where
should the line be drawn? Where do you actually think the line would be drawn,
if not along questionable ideological lines? Why have a line at all?
2eirenicon13y
What critic will not admit that? It's hardly a fringe opinion in the scientific
community that Creationists should not be given forums to speak on the
thoroughly unscientific topic of Creationism, and that those who do so and call
it science are being absurdly and unnecessarily tolerant. Creationism has never
been more or less than an attack on science. It's extremely toxic, and while I
would never try to "silence" anyone, I don't think it deserves more publicity. I
grew up being taught that dreck in a fundamentalist Christian school and I'm
more familiar with Behe than I'd care to be. Frankly, he's an idiot, and his
life purpose seems to be toward making more idiots. He doesn't need anyone's
help.
As for McArdle, I don't really care. Politics is not a hard science, and while
she's something of a crackpot, she's not that way because somebody proved her
map doesn't follow the territory. It's the difference between someone who thinks
the earth is flat and someone who thinks it's run by the Illuminati. The former
is just wrong, the latter is just crazy. I don't mind crazy, because crazy isn't
nearly as dangerous as wrong.
-1Peter_Twieg13y
I haven't seen BhTV endorse Creationism as science in any official capacity.
3XFrequentist13y
We been known to shoot us some subjectivists 'round these here parts, y'hear?
Sean Carrol concurs
[http://blogs.discovermagazine.com/cosmicvariance/2009/08/31/bye-to-bloggingheads/]
-2timtyler13y
It seems like science snobbery. BHTV has loads of political commentary - and
other non-science. It just isn't remotely like a peer-reviewed science journal.
1timtyler13y
PZ Myers says no to BHTV:
http://scienceblogs.com/pharyngula/2009/09/phil_plait_ditches_blogginghea.php
[http://scienceblogs.com/pharyngula/2009/09/phil_plait_ditches_blogginghea.php]
-1timtyler13y
Phil Plait pulls the plug on BHTV:
http://blogs.discovermagazine.com/badastronomy/2009/09/04/bloggingheads-capo-non-grata/
[http://blogs.discovermagazine.com/badastronomy/2009/09/04/bloggingheads-capo-non-grata/]
-1timtyler13y
Robert Wright discusses the whole issue on BHTV:
http://bloggingheads.tv/diavlogs/22300 [http://bloggingheads.tv/diavlogs/22300]
-1timtyler13y
He says 48 minutes in that the Templeton foundation "were no longer" and that
they only supported the site for four months.
However, their ads are still on the sponsored videos beneath:
http://bloggingheads.tv/percontations [http://bloggingheads.tv/percontations]
...so the Templeton foundation link apparently remains. Maybe there will be no
more percontations, though.
-3timtyler13y
Wright gives much the same arguments about ID as I give - in the section on
"Viable Intelligent Design Hypotheses" - on:
http://originoflife.net/intelligent_design/
[http://originoflife.net/intelligent_design/]
Since the topic of atheism, morality, and the like often come up here, I would like to point people to the free online book Secular Wholeness by David Cortesi. He approaches the topic of religion by trying to determine what benefits it can provide (community, challenges to improve oneself, easier ethical decisions, etc.), then tries to describe how to achieve these same benefits without resorting to religion. It's not very heavy on detail but seems very well sourced and with some good pointers on why people choose religions and what they get out of them.
ETA: This could have been a reply to the thread on Scientology, had I seen it before posting.
Boo to whoever voted this down in the first place. Great link.
0spriteless13y
It's obviously a counter-conspiracy against my conspiracy to get people to see
rationality as something that exists outside of this site. And also to make you
look at girls.
Scott Aaronson announced Worldview Manager, "a program that attempts to help users uncover hidden inconsistencies in their personal beliefs".
You can experiment with it here. The initial topics are Complexity Theory, Strong AI, Axiom of Choice, Quantum Computing, Libertarianism, Quantum Mechanics.
Mostly agree is a higher degree of agreement than Agree ?
To Somewhat agree that everyone should have the vote and Disagree that children
should have the vote is inconsistent ?
Obviously this is the work of the Skrull "Scott Aaronson", whose thinking is not
so clear.
0Nubulous13y
Also, almost every question is so broken as to make answering it completely
futile. So much so that it's hard to believe it was an accident.
2anonym13y
I find it hard to believe that you could really think the most likely
explanation of the flaws you perceive are that Aaronson and the students that
implemented this purposely introduced flaws and are trying to sabotage the work.
So why do you utter such nonsense?
And did it not occur to you that disagreeing that children should have the vote
could be resolved by being neutral on everybody having the vote, which is what I
did after realizing that there are plausible interpretations under which I would
disagree and plausible interpretations under which I would agree.
2Nubulous13y
Whether you consider this as sabotage or not depends on what you think the goal
of the site's authors was. It certainly wasn't to help find inconsistencies in
people's thinking, given the obvious effort that went into constructing
questions that had multiple conflicting interpretations.
Quite.
2DS361813y
I just tried the one for AI and I think its not quite accurate. One of the
biggest issues is that I think some of the terms need to be precisely defined
and they are not. The other issue I found was that the analysis of my beliefs
was not completely accurate because it did not take into account all the answers
properly.
Its an interesting idea but needs work.
-2[anonymous]13y
I didn't find the lack of precise definitions a problem.
1Larks13y
I got this conflict
[http://projects.csail.mit.edu/worldview/tension.php?tension_id=9445] between my
acceptance of the draft in the unlikely event it would be useful, and my belief
that all acts I think the Government should be allowed to do are currently
allowed. It doesn't seem to know of the existence of this supreme court ruling
[http://caselaw.lp.findlaw.com/cgi-bin/getcase.pl?court=US&vol=245&invol=366]
1wedrifid13y
Interesting link. I played with it for a while. It kept misunderstanding the
nuances of my responses, telling me I was wrong when I wasn't then refusing to
listen to my replies. So I stopped playing with it. Two in one day
[http://lesswrong.com/lw/16x/open_thread_september_2009/13qb]. What are chances?
1cousin_it13y
Good idea, bad implementation. Right now it thinks I have this
[http://projects.csail.mit.edu/worldview/tension.php?tension_id=9518] "tension",
but I'm pretty sure it's not a tension.
3JulianMorrison13y
Versus:
It's the [...] that hurts. "It is possible for one's mind to exist outside of
one's material body." does not imply "the mind is physically independent of the
material body". It's physically dependent and abstractly independent.
1anonym13y
I did have some difficulty resolving all tensions, but I was able to do so. I
found that there were often alternate interpretations of a statement that would
resolve a tension but were still plausible interpretations. For example, one
that I remember was interpreting some of the questions about "physical body"
more generally as "physical substrate". Sometimes the tension page didn't offer
the question that needed reinterpretation, in which case I deferred the tension
until I saw a tension that contained the statement to be reinterpreted.
It definitely does need a lot of work, but I can imagine a tool like this having
profound effects on people when all the bugs are worked out and it is applied to
mind killers and beliefs/habits where cognitive biases figure prominently.
One major thing that needs to be improved if they intend normal people to use it
for normal issues like politics, abortion, etc., is to make the tension page
much friendlier. Most LWers have probably studied logic, and can pretty easily
interpret the tension explanation, but most people have no clue about logic and
won't understand the implicit implications that aren't explained (like that
contrapositive of "A -> B" is valid).
0[anonymous]13y
I wouldn't call it a bad implementation for the occasional wrong reported
tension, especially if it's not completely clear for why it reports such a wrong
tension. To me, the purpose of the service is not to provide 100% coherent and
consistent questionnaire. The idea is that it points to conceptions that might
contradict themselves. Whether or not they in fact do contradict should be up to
a closer investigation. But merely pointing the user to these possible
contradictions should prove to be useful, because it's so difficult find these
inconsistencies by oneself.
It seems clear to me that it will generate some false positives. It will also
come up chains of logic that aren't obviously true or false (because it's
impossible to create statements that are completely free of differing
interpretations). Of course, the better the implementation in whole (both the
logic system and the sets of statements) the less it will generate these false
positives and other inconsistencies, but I do think that it's impossible to
remove them all. Instead, the service should perhaps be considered more like a
probing machine.
To claim that it's a bad implementation sounds to me like it's not a useful
implementation at all. Sure, it'll probably have a relatively many glitches and
bugs, but the above comment doesn't give any particular evidence that the
implementation as such doesn't work correctly. It seems almost equally likely
that such possible inconsistencies are an inherent part of this kind of
implementation.
If the implementation would constantly point to tensions that are obviously not
real tensions (or useful observations in general), then I'd be more inclined to
call it a bad implementation. After all, such claim will discourage people from
trying out the service and I don't see reason for such claim in the example
cousin_it gave.
The other common complaint seems to be the lack of precise definitions. Again, I
see this more like a feature than a bug. When taking the que
Is there an easy way to access the first comment of someone without looking at their comment page and uploading "next" zillions of times?
You're probably wondering why I would want to do that. I have been motivated occasionally to read someone's comments from beginning to end, and today I found myself wondering what my first comment was about.
My mind frequently returns to and develops the idea that sometime in the future, a friendly artificial intelligence is going to read Less Wrong. It uploads all the threads and is simultaneously able to (a) ... (read more)
Earlier today I was assigning probabilities to whether or not someone on this
site was someone I used to date. If they aren't then I really should introduce
them...
Readers of Less Wrong may be interested in this New Scientist article by Noel Sharkey, titled Why AI is a dangerous dream, in which he attacks Kurzweil's and Moravec's "fairy tale" predictions and questions whether intelligence is computational ("[the mind] could be a physical system that cannot be recreated by a computer").
[edit] I thought this would go without saying, but I suspect the downvotes speak otherwise, so: I strongly disagree with the content of this article. I still consider it interesting because it is useful to be aware o... (read more)
I strongly disagree. First, on the grounds that LW readers have strong reason to
believe this:
to be false, and so treat it similarly to a proof that 2=1.
But instead of just being a grouch this time, I decided to save you guys the
effort and read it myself to see if there's anything worth reading.
There isn't. It's just repetitions of skepticism you've already heard, based on
Sharkey's rejection of "the assumption that intelligence is computational" (as
opposed to what, and which is different and uncreatable why?), which "It might
be, and equally it might not be".
Other than that, it's a puff piece interview without much content.
1byrnema13y
(Phase 1)
Agreed, I don't see why the mind isn't a type of "computer", and why living
organisms aren't "machines". If there was something truly different and special
about being organic, then we could just build an organic AI. I don't get the
distinction being made.
(Phase 2)
Oh: sounds like dualism of some kind if it is impossible for a machine to have
empathy, compassion or understanding. Meaning beings with these qualities are
more than physical machines, somehow.
(Phase 3)
Reading through some of the comments to the article, it sounds like the
objection isn't that intelligence is necessarily non-physical, but that
"computation" doesn't encompass all possible physical activity. I guess the idea
is that if reality is continuous, then there could be some kind of complexity
gap between discrete computation and an organic process.
Phases 1-3 is the sequential steps I've taken to try to understand this point of
view. A view can't be rejected until its understood...I'm sure people here have
considered the AI-is-impossible view before, but I hadn't.
What is the physical materialist view on whether reality is discrete? (I would
guess it's agnostic.) What is the AI view on whether computations must be
discrete? (I would guess AI researchers wouldn't eschew a continuous computation
as as a non-computational thing if it were possible?)
2SilasBarta13y
I agree it's important to apply the principle of charity, but people have to
apply the principle of effort too. If Sharkey's point is about some crucial
threshold that continuous systems possess, he should say so. The term
"computational" is already taken, so he needs to find another term.
And he can't be excused on the grounds that "it's a short interview",
considering that he repeated the same point several times and seemed to find
enough space to spell out what (he thinks) his view implies.
4HalFinney13y
"[the mind] could be a physical system that cannot be recreated by a computer"
Let me quote an argument in favor of this, despite the apparently near universal
consensus here that it is wrong.
There is a school of thought that says, OK, let's suppose the mind is a
computation, but it is an unsolved problem in philosophy how to determine
whether a given physical system implements a given computation. In fact there is
even an argument that a clock implements every computation, and it has yet to be
conclusively refuted [http://consc.net/papers/rock.html].
If the connection between physical systems and computation is intrinsically
uncertain, then we can never say with certainty that two physical systems
implement the same computation. In particular, we can never know that a given
computer program implements the same computation as a given brain.
Therefore we cannot, in principle, recreate a mind on a computer; at least, not
reliably. We can guess that it seems pretty close, but we can never know.
If LessWrongers have solved the problem of determining what counts as
instantiating a computation, I'd like to hear more.
-4SilasBarta13y
Sure thing. I solved the problem here
[http://lesswrong.com/lw/zc/taking_occam_seriously/rp1] and here
[http://lesswrong.com/lw/zc/taking_occam_seriously/rqe] in response to Paul
Almond's essays on the issue. So did Gary Drescher, who said essentially the
same thing in pages 51 through 59 of Good and Real. (I assume you have a copy of
it; if not, don't privately message me and ask me how to pirate it. That's just
wrong, dude. On so many levels.)
-1billswift13y
This was linked on Hacker News http://news.ycombinator.com/item?id=797871
[http://news.ycombinator.com/item?id=797871]
I left this comment there: I thought this might be something like Eliezer's
arguments against developing a GAI until it could be made provably Friendly AI,
instead I just got an argument exactly like the ones in 1903 that said heavier
than air flight by men was impossible - go back and read some of them, some of
the arguments were almost identical. Some of the arguments are currently true,
but some of them amount to "I can't do it, and no one else has done it,
therefore there must be some fundamental reason it can't be done".
Just found this note in Shalizi's notebooks which casts an interesting shadow on the Solomonoff prior:
The technical results say that a classification rule is simple if it has a short description, measured in bits. (That is, we are in minimum description length land, or very close to it.) The shorter the description, the tighter the bound on the generalization error. I am happy to agree that this is a reasonable (if language-dependent) way of defining "simplicity" for classifier rules. However, so far as I can tell, this really isn't what makes
Here's my understanding of the dialog, which (as I read it) is not particularly
critical of the Solomonoff prior, if that is what cousin_it meant by "casts a
shadow".
(Background knowledge) Shalizi understands "Occam's Razor" to be something like
"In order to reach the truth, among the theories compatible with the evidence,
chose the simplest".
There is a claim that he wishes to refute. The claim is that a certain result is
an explanation or proof of Occam's Razor. The result says that if one finds a
simple classification rule which works well in-sample, then it is highly
probable that it will continue to work well out-of-sample.
This is a failure of relevance. Occam's Razor, as Shalizi understands it, is a
way of obtaining TRUTH, but the proof only concludes something about
GENERALIZATION PERFORMANCE. To illustrate the difference, he points to an
example where, in order to increase generalization performance, one might
decrease truth.
Shalizi contrasts the algorithmic information theory proof with Kevin T. Kelly's
Ockham Efficiency Theorem, which seems to Shalizi more productive. In
particular, Kevin T. Kelly's formalization does talk about truth rather than
generalization performance.
Finally, Shalizi provides an alternative ending to the algorithmic information
theory proof. If instead of choosing the simplest classification rule, one chose
the simplest rule within a sparse random subset of rules (even a non-computable
random subset), then you could still conclude a bound on generalization
performance. By providing an alternative ending, he has constructed an
alternative proof. Presumably this alternative proof does NOT seem like it is a
formalization of Occam's Razor. Therefore, the interpretation of the original
proof as demonstrating some version of Occam's Razor must also be mistaken.
To sum up, Shalizi is arguing that a certain
rhetorical/motivational/interpretational notion which often occurs near a
specific proof is wrong. I don't think he's concludi
3Wei_Dai13y
Looks like I almost missed a very interesting discussion. Hope I'm not too late
in joining it.
As far as I can tell, you still need the Solomonoff prior for decision making.
Kevin T. Kelly's Ockham Efficiency Theorem says that by using Ockham's Razor you
minimize reversals of opinion prior to finding the true theory, but that seems
irrelevant when you have to bet on something, especially since even after you've
found the truth using Ockham's Razor, you don't know that you've found it.
Also, I think there's a flaw in Shalizi's argument:
But if you're working with a sparse random subset of the rules, why would any of
them correctly classify all the data? In algorithmic information theory, the set
of rules is universal so one of them is guaranteed to fit the data (assuming the
input is computable, which may not be a good assumption but that's a separate
issue
[http://groups.google.com/group/one-logic/browse_thread/thread/b499a90ef9e5fd84/e7bd147f65e2b1dc?#e7bd147f65e2b1dc]
).
0cousin_it13y
All good points if the universe has important aspects that are computable, which
seems uncontroversial to me. Thanks for the link, I'd lost it sometime ago, that
thread is pretty epic.
1Daniel_Burfoot13y
The point of Kolmogorov complexity is that there is some limit to how baroque
your rules can ever become, or how baroque you can make a fundamentally simple
rule by choosing a really weird prior.
In algorithmic information theory, the problem of choosing a prior is equivalent
to choosing a particular universal Turing machine. If you pick a really weird
Turing machine, you will end up assigning low prior probability to things that
"normal" people would consider simple - like a sine wave, for example. But
because of universal computation, there's a limit to how low a probability you
can assign. Your weird machine is still universal, so somehow or other it's got
to be able to produce a sine wave, after whatever translation-prefix
machinations you have to perform to get it to act like a normal programming
language.
Another way of viewing this is just to eschew the notion of generalization and
state that the goal of learning is compression. If you do this, you end up doing
all the same kinds of work, with many fewer philosophical headaches. Now the
great deep problem of picking a prior boils down to the rather more quotidian
one of picking a data format to use to transmit/encode data.
0cousin_it13y
I don't understand this claim. Surely we can make the probability of a specific
program as low as we want by restricting the programming language in ad hoc
ways, while letting it stay Turing complete.
1Daniel_Burfoot13y
I agree there is some philosophical greyness here. But let me try again.
Let's say we are adversaries. I am going to choose a data set which I claim is
simple, and you are going to try to prove me wrong by picking a weird Turing
machine which assigns the data a low probability. I generate my data by taking T
samples from a sine wave. You pick some strange Turing machine which is designed
to be unable to produce sine waves. But regardless of the choice you make, I can
always just crank up T to a high enough value so that the compression rate of
the data set is arbitrarily close to 100%, proving its simplicity.
0RichardKennaway13y
cousin_it quoting Shalizi:
But a particular style of baroque elaboration is one that has a short
description.
0cousin_it13y
Not necessarily. (Or did I misread your comment?) The particular style can have
an arbitrarily long/complex description, and learning will still work as long as
the class of described rules is small enough. This observation seems to imply
that algorithmic simplicity doesn't play the central role I'd imagined it to
play. This is precisely the point where I'd like to hear LW's informed replies.
1RichardKennaway13y
Fixed overheads are ignored in the definition of Kolmogorov complexity.
A "particular style of baroque elaboration" is simply a program through whose
eyes certain rules look short, which look elaborate and baroque through the eyes
of another program.
0cousin_it13y
The PAC-learning scenario doesn't have any parameter that goes to infinity, so
I'm not sure why you dismiss "fixed" overheads :-)
0RichardKennaway13y
Once you've chosen them, they're fixed, and don't run off to infinity.
This definition-only-up-to-a-constant is one of the weaknesses of minimum
description length. (The other is its uncomputability. Shalizi somewhere else
remarks that in discussions of algorithmic complexity, it is traditional to
solemnly take out Kolmogorov complexity, exhibit its theoretical properties,
remark on its uncomputability, and put it away again before turning to practical
matters.)
FWIW, this is mine, informed or otherwise. Anyone else have light to shed?
1cousin_it13y
No, let me try nailing this jelly to the wall once again. The
definition-only-up-to-a-constant is a weakness of MDL, but this weakness isn't
relevant to my question at all! Even if we had some globally unique variant of
MDL derived from some nice mathematical idea, learning theory still doesn't use
description lengths, and would be perfectly happy with rules that have long
descriptions as long as we delineate a small set of those rules. To my mind this
casts doubt on the importance of MDL.
4Johnicholas13y
Consider this alternative characterization. Someone wants to fit a polynomial to
some data. They pre-selected a sparse set of polynomials, which are in general
ridiculously complex. Against all odds, they get a good fit to the training
data. This theorem says that, because they haven't examined lots and lots of
polynomials, they definitely haven't fallen into the trap of overfitting.
Therefore, the good fit to the training data can be expected to generalize to
the real data.
Shalizi is saying that this story is fine as far as it goes - it's just not
Occam's Razor.
2Daniel_Burfoot13y
Good characterization. It's worth noting that learning theory never gives any
kind of guarantee that you will actually find a function that provides a good
fit to the training data, it just tells you that if you do, and the function
comes from a low-complexity set, it will probably give good generalization.
0Daniel_Burfoot13y
Any delineation of a small set of rules leads immediately to a short description
length for the rules. You just need to encode the index of the rule in the set,
costing log(N) bits for a set of size N.
Note that MDL is not the same as algorithmic information theory
(definition-up-to-a-constant comes up in AIT, not MDL), though they're of course
related.
0cousin_it13y
I see everyone's assuming that some things, by their nature, always go to
infinity (e.g. number of samples) while others stay constant (e.g. rule set).
This is a nice convention, but not always realistic - and it certainly wasn't
mentioned in the original formulation of the problem of learning, where
everything is finite. If you really want things to grow, why don't you then
allow the set itself to be specified by increasingly convoluted algorithms as N
goes to infinity? Like, exponential in N? :-) Learning can still work - in
theory, it'll work just as well as a simple rule set - but you'll have a hard
time explaining that with MDL.
(If there's some theoretical justification why this kind of outrageous bloatware
won't be as successful as simple algorithms, I'd really like to hear it...)
2Johnicholas13y
You might find what you're looking for in Kevin T. Kelly's work:
http://www.andrew.cmu.edu/user/kk3n/ockham/Ockham.htm
[http://www.andrew.cmu.edu/user/kk3n/ockham/Ockham.htm]
Dr. Shalizi mentioned it as an alternative formalization of Occam's Razor.
0cousin_it13y
Johnicolas, thanks. This link and your other comments in those threads are very
close to what I'm looking for, though this realization took me some time.
0RichardKennaway13y
I think its uncomputability already does that. When you make a computable
version by limiting attention to some framework of descriptive capabilities
smaller than universal computation, different choices of that framework will
give you different measures of simplicity. What is simple in one framework may
seem elaborate and baroque in another. Or as some military strategist once put
it:
"To the foot-soldier, the strategy of a general may seem obscure, shrouded in
shadows and fog, but to the general himself, his way is as plain as if he were
marching his army down a broad, straight highway."
Any Santa Fe or Albuquerque lesswrong-ers out there, who might want to chat for an hour? I'll be in Santa Fe for a conference from 9/13 to 9/17, and am flying in and out of Albuquerque, and will have some free time Sunday 9/13.
After some more thought, I realized that making a game with fancy graphics and complex gameplay would probably not be a good idea for a first project to try.
A better idea would be a simple text-based game you play in your browser, probably running on either PHP or Python.
This might not have as much fun appeal as a traditional video game, since it would probably look like a university exam, but it could still b... (read more)
This could be used to make a game based off of Dungeons and Discourse
[http://dndis.wikidot.com/].
When you attack, you have to select an argument without a flaw, or it gets
blocked. When the opponent attacks, if you find a flaw, it deals no damage.
2gwern13y
This has already been done many times as part of critical thinking courses;
people don't use free sites like http://www.wwnorton.com/college/phil/logic3/
[http://www.wwnorton.com/college/phil/logic3/] because they're boring and hard.
I think the problem is that we lack a good game mechanic. Come up with a
mechanic, and the goals can follow, but it's hard to go from a goal like
'calibrate yourself to avoid overconfidence' to a fun game. We need to think
about how to borrow games like Zendo and repurpose them.
4PeerInfinity13y
Thanks for the link.
That LogicTutor site you linked to provides a good, basic introduction to a few
concepts and fallacies. However, the practice problems just ask you to identify
which fallacy is in the sentence they give you. They're missing the other half
of the game, which is spotting the fallacy in a block of text that's
deliberately designed to hide the fallacy. I'll keep looking in case someone has
already made a game that contains this part.
One way to make the game more fun would be to have interesting text to find the
fallacy in. Eliezer's short stories are a good example of this. Though for the
purpose of the game, we would need just short segments of stories, which contain
one clear example of a fallacy. Preferably one that's well-hidden, but obvious
once you see it. Also, to keep players on their toes, we could include segments
that don't actually contain a fallacy, and players would have the option of
saying that there is no fallacy.
And as I mentioned before, another idea is to flesh out the stories even more,
so that it could be expanded into a mystery game, or an adventure game, or an
escape-the-room game, where in order to continue you need to talk to people, and
some of these people will give inaccurate information, because they didn't
notice a flaw in their own reasoning, and you will need to point out the flaw in
their reasoning before they will give you the accurate information.
You would also have to choose your replies during the conversation, and have to
choose a reply that doesn't introduce a new fallacy and send the conversation
off in the wrong direction. Many of the possible responses would be to question
why the person believes specific things that they just said. Maybe there could
also be a feature where you could interrupt the person in the middle of what
they're saying, to point out the problem. Optionally, score the player based on
how long they took, and how many wrong paths they went down before finding the
correct path.
And
1Eliezer Yudkowsky13y
Well that would be an interesting game mechanic.
4CronoDAS13y
How about a courtroom drama? [http://www.youtube.com/watch?v=GJWiohNd3Us]
1gwern13y
I object!
(Alternate Higurashi ending: /me claws throat open to get at insects on me)
0gwern13y
Incidentally, another link right up your alley:
http://projects.csail.mit.edu/worldview/about
[http://projects.csail.mit.edu/worldview/about]
(Starting to think maybe we could use a wiki page, even if only for links and
ideas. This game discussion is now spread out over something like 5 LW
articles...)
0PeerInfinity13y
5 articles? I only know of 2 articles where this topic was discussed. Can you
post a link to the other 3, please?
Anyway, I went ahead and created a wiki page for discussing the game idea, and
posted links to the two threads I know of:
http://wiki.lesswrong.com/wiki/The_Less_Wrong_Video_Game
[http://wiki.lesswrong.com/wiki/The_Less_Wrong_Video_Game]
0gwern13y
It certainly feels like it's been spread out over more than 2 open threads!
Out of pure curiosity, what's your probability distribution of Scientology (or some other such group) being useful? Not the Xenu part, but is it possible that they've discovered some techniques to make people happier, more successful, etc.?
We already have some limited evidence that conventionally religious people are happier, and conventional religions are quite weak.
But see Will Wilkinson on this
[http://www.willwilkinson.net/flybottle/2008/05/17/arthur-brooks-on-religion-and-happiness/]
too (arguing that this only really holds in the US, and speculating that it's
really about "a good individual fit with prevailing cultural values" rather than
religion per se).
1taw13y
That's a good counter-argument, but the linked post doesn't actually measure
religion-happiness correlation within those other countries (which is the
relevant factor), and it's very plausible that European monopolistic religions
are far less effective than American freely competing religions for creating
happiness.
1conchis13y
The Snoep paper
[http://springerlink.metapress.com/content/650q541579041625/fulltext.html] Will
linked to measured the correlation for the US, Denmark and the Netherlands (and
found no significant correlation in the latter two).
The monopolist religion point is of course a good one. It would be interesting
to see what the correlation looked like in relatively secular, yet
non-monopolistic countries. (Not really sure what countries would qualify
though.)
1taw13y
I'm going to completely ignore "statistical significance", as scientific papers
are well known to have no idea how to do statistics properly with multiple
hypotheses, and can be assumed to be doing it wrong until proven otherwise.
If null hypothesis were false, the chance of all almost signs pointing in the
same direction would be very low. As far as I can tell what the paper finds out
is that religion is less effective in Denmark and Netherlands than in US, but it
increases happiness, and it's extremely unlikely to be a false positive result
due to chance.
1Jonathan_Graehl13y
I know a scientist who's spent significant money on Landmark Education
[http://en.wikipedia.org/wiki/Landmark_Education#Characterization_as_religious_movement]
(nee "est"). He's happy with what he got out of it, but doesn't feel the need
for any more of it now.
To answer both, there's no consequence. So I choose not to invent a completely
arbitrary prior.
I do enjoy fantasizing about possible measurable consequences of particular
types of simulations. Perhaps if I'm interesting enough, I'll be copied into
other simulations; perhaps we can discover some artifact of variably approximate
simulation when no important observer is near, etc.
1[anonymous]13y
A simulation hypothesis such as "our universe is a simulation" is not
falsifiable even given perfect knowledge of the universe at some point in time;
maybe the universe has a definite beginning and end and it's simulated perfectly
the whole way through. Therefore, I'll use the following definition of the
simulation hypothesis: "The best description of the universe as we are capable
of observing it describes our observations as happening entirely within a
simulation crafted by optimizing processes."
Let's assume for the sake of convenience that "the" priors for the laws of
physics are P, and let's call the distribution of universes that optimizing
processes would simulate P'. The only necessary difference between P and P' is
that P' is biased toward universes that are easy and/or useful to simulate. How
easy a universe is to simulate in general can probably be estimated by how easy
a universe is to simulate in itself. We have quantum mechanics but quantum
computers have been late in coming, suggesting that our universe would be
difficult to simulate. Now, as for utility, evolution optimizes for things that
themselves optimize for reproduction, but it also produces optimization for
pretty much random things. We can ignore the random things, and ask how useful
our universe is for reproduction. I'm guessing that the universe, as it seems to
involve lots of pointless computation, is not good for that.
So, given the above, I'd estimate the probability as being... oh, how does 20%
sound?
Now, of course, the other thing to look for in a simulated universe is
simulation artifacts: things that seem to not follow the laws of physics, and
behaviors that are only approximations to how things should behave. Suffice to
say, we haven't seen any of those.
0JGWeissman13y
Quantum computers are computers which use quantum superposition for parallel
processing, and are not required for simulating quantum mechanics. And our
"classical" computers do in fact take advantage of quantum mechanics, as
classical physics does not allow for solid state transistors.
0[anonymous]13y
I don't understand what point you're trying to make here, but classical physics
allows for mechanical computers.
0[anonymous]13y
It seems that quantum computers are required for simulating quantum mechanics in
sub-exponential time, though.
1JGWeissman13y
When discussing asymptotic algorithmic complexity, you should specify the
varying parameter of problem complexity.
0pengvado13y
The usual default parameter is number of bits it takes to write down the
problem. It could also be number of particles. Either one works in this case.
0JGWeissman13y
What quantum algorithm for simulating quantum mechanics takes sub-exponential
time with respect to the number of particles?
0pengvado13y
I didn't have a particular algorithm in mind when I said that, but since you ask
I went and found this one [http://arxiv.org/abs/quant-ph/9603026].
0byrnema13y
I consider any evidence that a truly random/spontaneous process occurs is
evidence that the universe isn't closed, because something is happening without
an internal mechanism to arbitrate it. And here we have the 2008 Nobel prize in
physics, "for the discovery of the mechanism of spontaneous broken symmetry in
subatomic physics".
2Douglas_Knight13y
I do not think that phrase means what you think it means.
-2byrnema13y
I thought it meant what you linked to, and after checking I'm pretty sure that
was what the prize was about.
So what do you think about the possibility of a physical mechanism being able to
make a free choice?
Perhaps some better examples:
* spontaneous creation of particles in a vacuum
* spontaneous particle decay
0Douglas_Knight13y
QM is deterministic. Spontaneous symmetry breaking also occurs in stat mech,
which applies to deterministic classical systems.
0byrnema13y
QM could be interpreted in a deterministic way, but this is not a common view. I
would like to learn more about it from you and others here on LW.
"Spontaneous" means that something happens without precursor; without any
apparent cause. It is orthogonal in meaning to "determined".
When you write that spontaneous symmetry breaking is deterministic, perhaps you
mean that its description is analytic -- wholly described by a set of
deterministic mathematical equations?
0Douglas_Knight13y
Spontaneous symmetry breaking is part of stat mech. It has practically nothing
to do with QM. Stat mech can be interpreted probabilistically, but it is not at
all controversial to apply it to deterministic systems.
Maybe that's a reasonable definition, but you contrasted "spontaneous" with
"closed," which is not orthogonal to "determined."
-1byrnema13y
My point [http://lesswrong.com/lw/16x/open_thread_september_2009/12pn] was that
true randomness of any kind would be evidence that a system is not closed. This
might be a novel observation (I haven't heard it before) but I think it is a
logical one. It is relevant to reductionism (we wouldn't want supernatural
processes swooping down to make choices for our free processes) and whether we
are in a simulation.
When applied to deterministic systems, the spontaneous symmetry breaking isn't
really spontaneous, just apparently so. The idea is that the direction of
breaking is determined by the initial conditions, but we may not have enough
information about the initial conditions to predict it.
It sounds like you want like to argue with whoever is responsible for,
"spontaneous symmetry breaking in subatomic physics". I didn't mention QM apart
from that.
0[anonymous]13y
All of your examples count as random events with a collapse postulate, but not
with many worlds, and hidden-variables have been formulated both ways.
Based on your past comments, I assume you already know that. Still, since your
examples don't suffice to distinguish interpretations of QM, they also don't
suffice to distinguish a universe with randomness from one without. Or are you
just pointing out that we should assign higher probability to randomness than we
would have if we hadn't observed anything that looked like collapse?
0billswift13y
I'm not much interested in creating bogus/useless "probability estimates". The
simulation hypothesis I rate, as I do religion, as "false, barring further
evidence". Evidence that the simulation hypothesis is true could be a
"physically impossible" inconsistency, like in Heinlein's story "Them". If I
became convinced that this was a simulation, I'd become a complete hedonist, why
bother with anything else when you are completely under the thumb of whatever's
running the simulation.
6Vladimir_Nesov13y
You are completely under the thumb of the physical laws.
1billswift13y
Sorry, but that is a reification of "physical laws"; physical laws aren't a
thing, they are simply our description of "how things work".
1Vladimir_Nesov13y
Stuff with which you interact is part of the rules of the game applied to you.
The more generally applicable of these rules you call "physical laws"
[http://lesswrong.com/lw/11m/atheism_untheism_antitheism/wu8]. Those are the
rules that can't be helped. If you are in the domain of a singleton
[http://wiki.lesswrong.com/wiki/Singleton], then its preference is one of the
inescapable laws.
You can analyze the raw content of the laws applied to you, just as you can
analyze sensory input, and see patterns such as individual agents making
decisions that affect your condition. Maybe such patterns are there, maybe they
are not, but the judgment of what to do under the given rules must depend on
what exactly those patterns are, not just a fact of "their existence".
0byrnema13y
That's interesting: Vladimir_Nesov chastised me for exactly the same thing
[http://lesswrong.com/lw/11m/atheism_untheism_antitheism/wu8] a month ago. While
I probably do reify physical laws, I'm sure he was just applying a metaphor.
1Vladimir_Nesov13y
My point in both cases is more that the concept of "existence" is very low on
meaningfulness, that you shouldn't act on mere "existence" of "nonexistence" of
something, you must instead understand what that something is.
So, I was thinking about how people conclude stuff. We tend to think of ourselves as having about two levels of conclusion: the "rational" level, which is the level we identify with, considering its conclusions to be our conclusions, and the "emotional" level, which is the one that determines our behavior. (Akrasia is disagreement between the two levels.)
Now, there doesn't seem to be any obvious rule for what becomes a rational level conclusion. If you go outside and wonder at nature, have you proven that God exists? For some people, it... (read more)
I see it tries to push the right buttons, but does it contain any novel
insights?
1Kaj_Sotala13y
The plenty of in-group references that are scattered in are almost as good. An
excerpt from the backstory:
That group sounds kinda familiar. I wonder if they might be inspired by some
real-life organization?
ETA: Oh, and it gets better. Later on, they mention within the same sentence the
"Lifeboat Institute" and the "Singularity Foundation" as two organizations that
existed before the Singularity. Hmmmmmmmm...
0CannibalSmith13y
Almost certainly not. After all, that's not the goal. Edit: After 72 pages I
retract my previous two statements.
Bought, downloaded, read intro. Basically it's Shadowrun IN SPACE! and mixed
with Paranoia. A good analogy would be that this tabletop RPG is to
Transhumanism what I, Robot the movie is to the book.
0Kaj_Sotala13y
Not really related to rationality, so no top-post if you ask me, but it's a very
good setting and an excellent product.
In reading the Singularity Institute's research goals, and the ruminations of Yudkowski, Wei Dai, Nesov et al. in postings here, the approach to developing friendly AI which stands out the most, and from my perspective seems to just always have been the case, seems to be exclusively logic based in the vein of John McCarthy.
I am wondering how the decision was made to focus research for SIAI on the pure logic side, rather than, for example building a synthetic conscious which uses the brain as a model?
To be sure, nearly all AI approaches overlap at some poi... (read more)
Such a design would be harder to reason about.
Let's say you've got a prototype you want to improve. How do you tell if a
proposed change would make it smarter, break it, introduce a subtle cognitive
bias, or make the AI want to kill you?
In order to set on limits on the kinds of things an AI will do, you need to
understand how it works. You can't be experimenting on a structure you partially
understand, AND be certain that the experiments won't be fatal.
This is easier when you've got a clearly defined structure to the AI, and know
how the parts interact, and why.
0AndrewKemendo13y
How is that impossible with a replicated brain architecture? We can't make one
if we don't know how it works.
Of course. However, how you plan to structure AI what I am asking about. There
are many theories about how to structure the AI
[http://en.wikipedia.org/wiki/Artificial_intelligence#Approaches] - so why did
the SIAI choose to only focus on a theoretical mathematical logic based approach
rather than taking the most advanced, if still flawed, logic device known to man
and replicating and improving that?
1Vladimir_Nesov13y
If you have the right tools, you can make a brain without understanding it.
Reproductive system can make brains. Whole brain emulation
[http://www.fhi.ox.ac.uk/publications/fohi_publications/brain_emulation_roadmap]
doesn't require understanding of brain architecture, only the dynamics of its
lowest-level components.
You "know" how pi works, and how to set up a program that computes it, but you
don't know what its quandrillionth digit is.
0AndrewKemendo13y
I fear the same philosophical reasoning may be applied to model neural
architecture as is currently being used for econometric forecasting. Even the
most complex economic models cannot account for significant exogenous variables.
For the record I think we can get to WBE, however I think a premature launch
would be terrible. Based on the lack of research into developmental AI (much
work notably done by a friend - Dr. Frank Guerin
[http://www.csd.abdn.ac.uk/~fguerin/] at Aberdeen college) I think there is a
long way to go.
Granting that a brain model or WBE, would be as accurate as the biological
version, why then would that not be the most efficient method? The problems with
testing and implementation are the same as any other AI, if not easier because
of familiarity, however it is grounded on specific biological benchmarks which
at that point would be immediately identifiable.
I could go on with my particular thoughts as to why biological simulation is in
my estimation a better approach, however I am more interested in why the
organization (people who have been thinking longer and with more effort than
myself) decided otherwise. It would seem that their collective reasoning would
give a sufficiently clear and precise answer such that there would be no
ambiguity.
1Vladimir_Nesov13y
You have to instill the right preference, and just having a working improved
brain doesn't give this capability. You are trying to make an overwhelmingly
powerful ally; just making something overwhelmingly powerful is a suicide. As
CannibalSmith said [http://lesswrong.com/lw/16x/open_thread_september_2009/12or]
, brains are not particularly Friendly [http://wiki.lesswrong.com/wiki/FAI].
Read the paper [http://yudkowsky.net/singularity/ai-risk].
0AndrewKemendo13y
Of course - we have to BUILD IT RIGHT. I couldn't agree more. The cognitive
model does not suggest a mere carbon copy of any particular brain at random, as
you know it is not so limited in focus. The fantastic part about the method is
at the point in which it is possible to do correctly (not simply an apparent
approximation), the tools will likely be available (in the process of it being
structured) to correct a large portion of what we identify as fatal cognitive
errors. Any errors that are missed it stands to reason would be also missed
given the same amount of time with any other developmental structure.
I am familiar with the global risk paper you linked, AI: A modern approach which
addresses the issue of cognitive modeling as well as Drescher's Good and Real
and the problems associated with an FAI.
The same existential risks and potential for human disasters are inherent in all
AI systems - regardless of the structure, by virtue of it's "power." I think one
of the draws to this type of development is the fantastic responsibility which
comes with it's development, recognizing and accounting for the catastrophic
results that are possible.
That said, I have yet to read a decision theoretic explication as to which
structure is an optimized method of development, weighing all known limiting
factors. I think AI: A modern approach comes closest to doing this but falls
short in that it specifically narrows it's focus without a thorough comparison
of methods. So again, I ask, by what construct has it been determined that a
logical symbolic programming approach is optimized?
0billswift13y
In other words they are doing it where the light's better, rather than where
they dropped the keys. Given the track record of correctness proofs in comp sci,
I don't think provably Friendly AI is even possible, hopefully I'm wrong there,
but all they are doing is further crippling their likelihood of achieving AI
before some military or business does.
-1timtyler13y
People control how companies operate - even though they don't understand all the
components involved (in particular the brains).
Any idea that you have to understand all the components of a system in order to
exert a high level of control over it thus appears to have dubious foundations.
There are alternative approaches to producing predictable systems which
basically involve lots of unit tests.
Testing is in vogue in the software engineering world. Few systems are simple
enough to prove much about their behaviour. So: to make sure they behave as they
are intended, they are intensively tested. It seems likely that machine
intelligence will be no different.
5Vladimir_Nesov13y
When a failed test destroys the world, applicability of the normally very useful
testing methodology should be reconsidered.
0AndrewKemendo13y
This would be true of any AI. Thus the AI box problem.
It is unclear however, how a formal logic approach overcomes this problem and a
replication approach does not. They both will need testing, but as you said the
methodology should be reconsidered. The easiest way to change testing
methodology for logic would be to improve on current logic methodology which has
yielded arguably fantastic results - all done by faulty human brains.
-3timtyler13y
Normally testing is done in an offline "testing" mode - using a test harness or
sandbox arrangement. Tests themselves are consequently harmless.
Of course it is possible for the world to present eventualities that are not
modelled by the test suite - but that's usually no big deal.
I don't think it is realistic to confine machine intelligence to the domain of
provably correct software. Anyone trying that approach would rather obviously be
last to the marketplace with a product.
I seriously doubt whether paranoid fantasies about DOOM will hinder progress
towards machine intelligence significantly. I expect that the prophets of DOOM
will be widely ignored. This isn't exactly the first time that people have
claimed that the world is going to end.
1DonGeddis13y
Forget about whether your sandbox is a realistic enough test. There are even
questions about how much safety you're getting from a sandbox. So, we follow
your advice, and put the AI in a box in order to test it. And then it escapes
[http://yudkowsky.net/singularity/aibox] anyway, during the test.
That doesn't seem like a reliable plan.
-1timtyler13y
The idea that society is smart enough to build machine intelligence, but not
smart enough to build a box to test it in does not seem credible to me:
Humans build boxes to put other humans in - and have a high success rate of
keeping them inside when they put their minds to it. The few rogue agents that
do escape are typically hunted down and imprisoned again. Basically the builders
of the box are much stronger and more powerful than what it will contain.
Machine intelligence testing seems unlikely to be significantly different from
that situation.
The cited "box" scenario discusses the case of weak gatekeepers and powerful
escapees. That scenario isn't very relevant in this case - since we will have
smart machines on both sides when restraining intelligent machines in order to
test them.
0thomblake13y
Either massive progress or DOOM will be wrought by those ignoring the
DOOM-prophets; either the dynamists win or everyone loses, so the DOOM-prophets
lose either way. It seems like a bad business to be in.
0timtyler13y
DOOM is actually big business. Check out all the disaster movies out there. DOOM
sells. What could be more important than... THE END OF THE WORLD? What greater
cause could there be than... SAVING THE WORLD? So, people buy the DOOM
merchandise, contribute the DOOM dollars, and warn their friends about the
impending DOOM - thus perpetuating the DOOM virus. That is part of why there
have been so many DOOM prophets - DOOM pays.
0JamesAndrix13y
The more I think about this, the more it seems incorrect.
2CannibalSmith13y
Human brains are not particularly friendly.
3AndrewKemendo13y
I would disagree. The overwhelming majority of the average human's life is spent
peacefully. It is actually fairly remarkable how rarely we have significant
conflict, especially considering the relatively overcrowded places that humans
live. Not to mention that it is only a small proportion of the human population
that engages other humans destructively (not by proxy).
4Kaj_Sotala13y
The overwhelming majority of the average human's life is also spent in
conditions where they are on relatively even grounds with everyone else. But
once you start looking at what happens when people end up in situations where
they are clearly more powerful than others? And can treat those others the way
they like and without fear of retribution? Ugly.
1AndrewKemendo13y
I disagree. While there are some spectacular examples of what you describe, and
they are indeed ugly, by and large there is wide distribution of hierarchical
disparity even in daily life which is more often than not mutually beneficent.
As an emperor I optimize my empire by ensuring that my subjects are
philosophically and physically satisfied do I not? I think there is plenty of
evidence to support this philosophy as the most sustainable (and positive) of
hierarchical models; after all some of the most successful businesses are
laterally organized.
0Kaj_Sotala13y
A certain philosophy being the most sustainable and positive isn't automatically
the same as being the one people tend to adopt. Plus the answer to your question
depends on what you're trying to optimize.
Also, it sounds like you're still talking about a situation where people don't
actually have ultimate power. If we're discussing a potential hard takeoff
scenario, then considerations such as "which models have been the most
successful for businesses before" don't really apply. Any entity genuinely
undergoing a hard takeoff is one that isn't afterwards bound by what's
successful for humans, any more than we are bound by the practices that work the
best for ants.
0AndrewKemendo13y
I think there is more than ample evidence to suggest that those are
significantly less likely to be adopted - however wouldn't a group of people who
know that and can correct for it be the best test case of implementing an
optimized strategy?
I hold the view that it is unnecessary to hold ultimate power over FAI. I
certainly wouldn't bind it to what has worked for humans thus far. Don't fear
the AI, find a way to assimilate.
"But wait – who’s to say that progress will remain “only” exponential? Might not progress exceed this rate, following an inverse polynomial curve (like gravity) or even an inverse exponential curve? I, for one, don’t see why it shouldn’t. If we consider specifically the means whereby the Singularity is most widely expected to occur, namely the development of computers with the capacity for recursive improvement of their own workings,4 I can see no argument why the rate at which such a comp... (read more)
I have a couple of intuitions about the structure of human preferences over a large universe.
The first intuition is that your preferences over one part of the universe (or universe-history) should be independent of what happens in another part of the universe, if the "distance" between the two parts is great enough. In other words, if you prefer A happening to B happening in one part of the universe, this preference shouldn't be reversed no matter what you learn about a distant part of the universe. ("Distance" might be spatial, tempora... (read more)
Several times recently I asked for simple clarifications about a comment that replied to something I wrote, and had my question ignored. (See here, here, and here.) And I don't know why. Did I violate some rule of etiquette, or what? How can I rephrase my questions to get a better response rate?
ETA: Here are the questions, in case people don't want to search through the comments to find them:
But I'm not sure what you mean by "metaethics, a solved problem". Can you give a link?
What prior work are you referring to, that hasn't been broadly disse
In my case, what tends to happen is that I either don't notice the question, or
I notice that the question requires a bunch of work to respond to and then
either get to it some time later or let it slide off entirely.
1CarlShulman13y
"or I notice that the question requires a bunch of work to respond to and then
either get to it some time later or let it slide off entirely"
Or don't recall it.
0anonym13y
I think in cases like these, you're more likely to get a response by adding
another post as a reply to the person with just a single unanswered question
(start with the one you care about most), so the person will see they have a new
response in their inbox and realize they never answered an earlier question. If
you post each of those 3 questions as a response to the person, in context, I'd
be very surprised if you didn't get a response to at least 2 of the 3, as long
as you include little to nothing else in each post so it's obvious what you're
asking for and they can't respond to something else in the post.
I've noticed that longish posts with multiple questions often get just one
question answered and all the others ignored, intentionally or unintentionally.
And posts that are longish with questions interspersed with non-questions tend
to get responded too as if the non-questions were the substantive part, with the
questions often ignored.
(The other extremely common reason for not getting a response is identifying a
flaw or asking a question that shows problems with the person's position, in
which case most people seem to just ignore the post rather than admit they were
wrong or can't answer a critique. I don't think that's the case here at all
though.)
1Wei_Dai13y
This seems like good advice. I did think about repeating the unanswered
question, but was worried that I'd come off as obnoxious if the commenter was
avoiding it deliberately for some reason. Given the multiple confirmations that
that's probably not the case, I think I'll do so more often in the future.
Thanks.
0Vladimir_Nesov13y
Not answering a question is Internet's way of walking away from a conversation.
You don't usually say "excuse me, I'm late to a meeting", as interaction is
asynchronous. In the current Internet culture, saying "I don't want to bother
answering" sounds rude, and so the best solution signaling-wise is to just not
answer.
4kpreid13y
In my experience (for myself and interacting with others), it's not uncommon at
all for someone to
* miss that there was a question at all
* see a question, but put off thinking about it and then forget to get back to
it
* reply to some other part and forget about answering the question
* figure replying to the question is uninteresting 'because the thread is dead'
etc.
0wedrifid13y
The first two have responses as of the time that you posted this. (If there are
remaining questions in a post they are far less likely to be answered after the
first couple of replies.)
The final example
[http://lesswrong.com/lw/17c/outlawing_anthropics_an_updateless_dilemma/13k5]
suffers somewhat from 'nobody knows what science doesn't know'. There are
probably not too many people who can think of an example of a problem that UDT1
can not handle. For my part I probably wouldn't answer just because I don't like
the name UDT1 and the language used to describe it irritates me.
I'm not sure why Eleizer didn't answer but I probably wouldn't bother wasting
thoughts wondering. Want an answer? Make a top level post about it. Include
enough of a useful description of the theory and the problems it has already
solved to make you not look bad. In particular, include links to said problems
and resolutions. If you really want an answer then include an assertion that
UDT1 has solved all the significant decision Problems that have been discussed
on LessWrong.
1Wei_Dai13y
Please expand on that.
Why should I make that much effort to get a simple answer to a simple question?
Eliezer obviously had something specific in mind when he wrote "Problem". Why
didn't he just write a couple of sentences saying what it was when I indicated
that I didn't get the reference?
Same with the other two questions. I wasn't asking difficult questions, just
simple clarifications.
0wedrifid13y
I'm not suggesting that you ought to have to. I don't think you violated any
particular etiquette with your requests for clarification. If going meta and
questioning whether the lack of reply is justifiable is your preferred use for
the effort then by all means do that instead.
I have no idea why Eliezer didn't answer you. Maybe he was busy. Maybe he was
self absorbed. But I do make this observation in general: If someone presents a
position along the lines of "it is right to believe there is are Problems's with
X" then I usually don't expect them to answer me if I press them for an example.
This is particularly the case if there are, in fact, no obvious examples. Even
if an example could be given, successfully justifying themselves in response to
what can be construed as a challenge does not necessarily benefit them. If you
present your question in your own frame, however, the dynamics are entirely
different.
Concerning Newcomb’s Problem I understand that the dominant position among the regular posters of this site is that you should one-box. This is a position I question.
Suppose Charlie takes on the role of Omega and presents you with Newcomb’s Problem. So far as it is pertinent to the problem Charlie is identical to Omega with the notable exception that his prediction is only %55 likely to be accurate. Should you one-box or two-box in this case?
If you one-box then the expected utility is (.55 1,000,000) $550,000 and if you two-box then it is (.45 1,001,000) $450,450 so it seems you should still one-box even when the prediction is not particularly accurate. Thoughts?
Good question. And with Charlie known to be operating exactly as defined then
yes, I would one box. I wouldn't call him Charlie however as that leads to
confusion. The significant problem with dealing with someone who is taking the
role of Omega is in my ability to form a prediction about them that is
sufficient to justify the 'cooperate' response. Once I have that prediction the
rest, as you have shown, is just simple math.
1dv82matt13y
I don’t think Newcomb’s Problem can easily be stated as a real (as opposed to a
simply logical) problem. Any instance of Newcomb’s problem that you can feasibly
construct in the real world it is not a strict one shot problem. I would suggest
that optimizing a rational agent for the strictly logical one shot problem one
is optimizing for a reality that we don’t exist in.
Even if I am wrong about Newcomb’s problem effectively being an iterated type of
problem treating it as if it is seems to solve the dilemma.
Consider this line of reasoning. Omega wants to make the correct prediction. I
want Omega to put the million dollars in the box. If I one-box I will either
reward Omega for putting the money in the box or punish Omega for not putting
the money in the box. Since Omega has a very high success rate I can deduce that
Omega puts a high value on making the correct prediction I will therefore put a
correspondingly high value on the instrumental value of spending the thousand
dollars to influence Omega’s decision. But here’s the thing, this reasoning
occurs before Omega even presents you with the problem. It is worked out by
Omega running your decision algorithm based on Omega’s scan of your brain. It is
effectively the first iteration.
You are then presented with the choice for what is effectively the second time
and you deduce that any real Omega (as opposed to some platonic ideal of Omega)
does something like the sequence described above in order to generate it’s
prediction.
In Charlie’s case you may reason that Charlie either doesn’t care or isn’t able
to produce a very accurate prediction and so reason he probably isn’t running
your decision algorithm so spending the thousand dollars to try to influence
Charlie’s decision has very low instrumental value.
In effect you are not just betting on the probability that the prediction is
accurate you are also betting on whether your decision algorithm is affecting
the outcome.
I’m not sure how to calculate this but t
0[anonymous]13y
It can be stated as real in any and every universe that happens to have an
omniscient benefactor who is known to be truthful and prone to presenting such
scenarios. It's not real in any other situation. The benefit for optimising a
decision making strategy to handle such things as the Newcomb problem is that it
is a boundary case. If our decision making breaks down entirely at extreme cases
then we can not trust it to be correct.
12 healthy male volunteers were chosen to study what is "just right" amount of beer for driving car. These men consumed doses of beer at 2 bottles, 4 bottles, 8 bottles, and 16 bottles per day for two weeks for each dose amount, with beer being the only alcohol in their diet. Surely the 2 bottles would win, but it definitely ain't the "just right" amount.
Am I missing something in the sciencedaily news, or did they really end up to that conclusion, of 200mg from that test?
Well, the article abstract isn't consistent with the description you linked to.
One of the dangers of paraphrasing science.
0JGWeissman13y
From the abstract: "Twelve healthy male volunteers (aged 53–65 yr) were assigned
to consume an intake of successively 200, 400, 800, and 1600 mg/d DHA, as the
only {omega}-3 fatty acid, for 2 wk each dose."
I don't know what inconsistency you noticed between the news article and the
abstract, but it seems the abstract itself describes a study that is missing the
control group that gets a dosage of 0.
0MendelSchmiedekamp13y
The following sounds like a control measurement was taken:
"Blood and urine samples were collected before and after each dose of DHA and at
8 wk after arrest of supplementation."
Also note, that the abstract doesn't say that 200mg is ideal as the science
daily description does it says:
"It is concluded that low consumption of DHA could be an effective and
nonpharmacological way to protect healthy men from platelet-related
cardiovascular events."
0JGWeissman13y
Taking measurements before and after the treatment is good, but that is not the
same as having a separate control group, which could filter out effects of
timing, taking the dose with food or water, etc.
The abstract also claims "Therefore, supplementation with only 200 mg/d DHA for
2 wk induced an antioxidant effect." It is likely that there was a more complete
conclusion in the full article.
0MendelSchmiedekamp13y
But the abstract does not make any "just right" claims, unlike the summary on
science daily. Which is what you where complaining about.
The abstract reads - we did an incremental test, and even at the lowest dosage
we found an effect. This suggest that low dosages could be effective. I don't
see anything wrong with that reasoning.
The science daily summary is simply misrepresenting it. So, the original
commenter isn't missing something in the science news, it is science daily who
made the error.
0JGWeissman13y
The news article was not based on the abstract. It was based on the journal
article (which is available with a subscription) that the abstract summarized.
It is not reasonable to expect that every point in the news article be supported
by the abstract.
1Douglas_Knight13y
extremely implausible, as a general rule.
ETA:
0JGWeissman13y
So, perhaps the news article was based on press release that was based on the
journal article. My point was that it was not produced solely from the abstract.
1Douglas_Knight13y
While the article is more reliable than the abstract, the abstract is more
reliable than the press release and the news coverage, because there is better
policing of its claims. And the abstract is more policed than the article, so
though it may be less reliable because of compression, it is not biased towards
sensationalism.
0MendelSchmiedekamp13y
I don't see why this is your point? In the very least it doesn't present counter
evidence to my claim that the abstract contains information not present in the
news article which mitigates or negates the concerns of the original comment.
0JGWeissman13y
So what? That point was in response to your other claim about what the abstract
did not contain.
0MendelSchmiedekamp13y
It's just that with two distinctly different conclusions from the results
mentioned from two different sources: the article authors (in the abstract) and
Gerald Weissmann, M.D., Editor-in-Chief (in the news article), I place a much
lower confidence in later being a reasonable reading of the research paper.
But of course we could quite safely argue about readings and interpretations
indefinitely. I'd point you to Derrida and Hermeneutics if you want to go that
route.
In any case, I'll update my estimates on the likelihood of the research paper
having an errant conclusion based on Weismann's quote, and I suggest you do the
same based on the evidence in the abstract - and then I suspect we have little
more to discuss on the subject.
In light of how important it is (especially in terms of all the decision theory posts!) to know about Judea Pearl's causality ideas, I thought I might share something that helped me get up to speed on it: this lecture on Reasoning with Cause and Effect.
For me, it has the right combination of detail and brevity. Other books and papers on Pearlean causality were either too pedantic or too vague about the details, but I learned a lot from the slides, which come with good notes. Anyone know what proceedings papers it refers to though?
Does anyone feel the occasional temptation to become religious?
I do, sometimes, and push it away each time. I doubt that I could, really, fall for that temptation - even if I tried to, the illogicality of the whole thing would very likely prevent me from really believing in even a part of it very seriously. And as more and more time passes, religious structures begin to seem more and more ridiculous and contrived. Not that I'd have believed that them starting to feel more contrived would even have been possible.
And yet... occasionally I remember the time back in my teens, when I had some sort of a faith. I remember the feeling of ultimate safety it brought with it - the knowledge that no matter what happens, everything will turn out well in the end. It might be a good thing that I spend time worrying over existential risks, and spend time thinking about what I could do about them, but it sure doesn't exactly improve my mental health. The thought of returning to the mindset of a believer appeals on an emotional level, in the same way stressed adults might longingly remember the carefree days of childhood. But while you can't become a child again, becoming a believer is at least theoretically possible. And sometimes I do play around with the idea of what it'd be like, to adopt a belief again.
On the consolidation of dust specks and the preservation of utilitarian conclusions:
Suppose that you were going to live for at least 3^^^3 seconds. (If you claim that you cannot usefully imagine a lifespan of 3^^^3 seconds or greater, I must insist that you concede that you also cannot usefully imagine a group of 3^^^3 persons. After all, persons are a good deal more complicated than seconds, and you have experienced more seconds than people.)
Suppose that while you are contemplating how to spend your 3^^^3-plus seconds, you are presented with a binary ch... (read more)
Sean Carroll and Carl Zimmer are leaving Bloggingheads, mostly because it's started playing nice with creationists. Click their names to read their full explanations.
Since the topic of atheism, morality, and the like often come up here, I would like to point people to the free online book Secular Wholeness by David Cortesi. He approaches the topic of religion by trying to determine what benefits it can provide (community, challenges to improve oneself, easier ethical decisions, etc.), then tries to describe how to achieve these same benefits without resorting to religion. It's not very heavy on detail but seems very well sourced and with some good pointers on why people choose religions and what they get out of them.
ETA: This could have been a reply to the thread on Scientology, had I seen it before posting.
It's been awhile since this came up. But hay, look, arguments about sex and race and stuff, as applicable to atheism as to rationality in general.
Scott Aaronson announced Worldview Manager, "a program that attempts to help users uncover hidden inconsistencies in their personal beliefs".
You can experiment with it here. The initial topics are Complexity Theory, Strong AI, Axiom of Choice, Quantum Computing, Libertarianism, Quantum Mechanics.
Is there an easy way to access the first comment of someone without looking at their comment page and uploading "next" zillions of times?
You're probably wondering why I would want to do that. I have been motivated occasionally to read someone's comments from beginning to end, and today I found myself wondering what my first comment was about.
My mind frequently returns to and develops the idea that sometime in the future, a friendly artificial intelligence is going to read Less Wrong. It uploads all the threads and is simultaneously able to (a) ... (read more)
Readers of Less Wrong may be interested in this New Scientist article by Noel Sharkey, titled Why AI is a dangerous dream, in which he attacks Kurzweil's and Moravec's "fairy tale" predictions and questions whether intelligence is computational ("[the mind] could be a physical system that cannot be recreated by a computer").
[edit] I thought this would go without saying, but I suspect the downvotes speak otherwise, so: I strongly disagree with the content of this article. I still consider it interesting because it is useful to be aware o... (read more)
Just found this note in Shalizi's notebooks which casts an interesting shadow on the Solomonoff prior:
... (read more)Any Santa Fe or Albuquerque lesswrong-ers out there, who might want to chat for an hour? I'll be in Santa Fe for a conference from 9/13 to 9/17, and am flying in and out of Albuquerque, and will have some free time Sunday 9/13.
I'm going to use this open thread to once again suggest the idea of a Less Wrong video game.
( Here's a link to the post I made last month about it )
After some more thought, I realized that making a game with fancy graphics and complex gameplay would probably not be a good idea for a first project to try.
A better idea would be a simple text-based game you play in your browser, probably running on either PHP or Python.
This might not have as much fun appeal as a traditional video game, since it would probably look like a university exam, but it could still b... (read more)
Out of pure curiosity, what's your probability distribution of Scientology (or some other such group) being useful? Not the Xenu part, but is it possible that they've discovered some techniques to make people happier, more successful, etc.?
We already have some limited evidence that conventionally religious people are happier, and conventional religions are quite weak.
I'm curious about how Less Wrong readers would answer these questions:
What is your probability estimate for some form of the simulation hypothesis being true?
If you received evidence that changed your estimate to be much higher (or lower), what would you do differently in your life?
So, I was thinking about how people conclude stuff. We tend to think of ourselves as having about two levels of conclusion: the "rational" level, which is the level we identify with, considering its conclusions to be our conclusions, and the "emotional" level, which is the one that determines our behavior. (Akrasia is disagreement between the two levels.)
Now, there doesn't seem to be any obvious rule for what becomes a rational level conclusion. If you go outside and wonder at nature, have you proven that God exists? For some people, it... (read more)
There is an audio interview with EY on "Make A Public Commitment".
Eclipse Phase
Does it deserve a top level post?
In reading the Singularity Institute's research goals, and the ruminations of Yudkowski, Wei Dai, Nesov et al. in postings here, the approach to developing friendly AI which stands out the most, and from my perspective seems to just always have been the case, seems to be exclusively logic based in the vein of John McCarthy.
I am wondering how the decision was made to focus research for SIAI on the pure logic side, rather than, for example building a synthetic conscious which uses the brain as a model?
To be sure, nearly all AI approaches overlap at some poi... (read more)
I have at least one other legacy identity here, dating from the old OB days, "mitchell_porter2". Is there some way to merge us?
To quote from http://www.sens.org/files/sens/FHTI07-deGrey.pdf:
"But wait – who’s to say that progress will remain “only” exponential? Might not progress exceed this rate, following an inverse polynomial curve (like gravity) or even an inverse exponential curve? I, for one, don’t see why it shouldn’t. If we consider specifically the means whereby the Singularity is most widely expected to occur, namely the development of computers with the capacity for recursive improvement of their own workings,4 I can see no argument why the rate at which such a comp... (read more)
Came across this: What We Can Learn About Pricing From Menu Engineers. Probably nothing new to you/us. Summary: Judgements of acceptable prices are strongly influenced by other seen prices.
I have a couple of intuitions about the structure of human preferences over a large universe.
The first intuition is that your preferences over one part of the universe (or universe-history) should be independent of what happens in another part of the universe, if the "distance" between the two parts is great enough. In other words, if you prefer A happening to B happening in one part of the universe, this preference shouldn't be reversed no matter what you learn about a distant part of the universe. ("Distance" might be spatial, tempora... (read more)
Several times recently I asked for simple clarifications about a comment that replied to something I wrote, and had my question ignored. (See here, here, and here.) And I don't know why. Did I violate some rule of etiquette, or what? How can I rephrase my questions to get a better response rate?
ETA: Here are the questions, in case people don't want to search through the comments to find them:
- But I'm not sure what you mean by "metaethics, a solved problem". Can you give a link?
- What prior work are you referring to, that hasn't been broadly disse
... (read more)Has Eliezer written anything good about the evolution of morality? It should probably go on a wiki page titled "Evolution of morality".
ETA: While I'm at it, how about reasons people are religious?
Concerning Newcomb’s Problem I understand that the dominant position among the regular posters of this site is that you should one-box. This is a position I question.
Suppose Charlie takes on the role of Omega and presents you with Newcomb’s Problem. So far as it is pertinent to the problem Charlie is identical to Omega with the notable exception that his prediction is only %55 likely to be accurate. Should you one-box or two-box in this case?
If you one-box then the expected utility is (.55 1,000,000) $550,000 and if you two-box then it is (.45 1,001,000) $450,450 so it seems you should still one-box even when the prediction is not particularly accurate. Thoughts?
Does this http://www.sciencedaily.com/releases/2009/08/090831130751.htm suffer same problem as:
12 healthy male volunteers were chosen to study what is "just right" amount of beer for driving car. These men consumed doses of beer at 2 bottles, 4 bottles, 8 bottles, and 16 bottles per day for two weeks for each dose amount, with beer being the only alcohol in their diet. Surely the 2 bottles would win, but it definitely ain't the "just right" amount.
Am I missing something in the sciencedaily news, or did they really end up to that conclusion, of 200mg from that test?
If noone was using computers today, how would you convince people to use them?