@Robin: Dennett (Elbow Room: The Varieties of Free Will Worth Wanting) points out that as our understanding of internal causation slowly comes to embrace and accurately describe the various mental processes that we currently describe as "conscious will," the courts are going to be seriously challenged over this distinction. Dennett's solution, that we will ultimately treat those brought before court as if every behavior were a product of free will and only worry about the most effective treatment, sounds right to me. Dennett points out that a weaker approach is incoherent and encourages the kind of cynicism I suspect Eliezer frets over.
@John: I think Eliezer did a good job of describing the problem in his followup to Anna, but I'm still having trouble convincing myself of the correctness of his statements. It feels to me like Eliezer is working hard to have these systems both ways: in his example of something historically effective but not psychologically effective, surely the psychological effectiveness, if it exists, is an emergent property of its historical effectiveness.
There ought to be an HTML entity for a lightbulb going on! Eliezer tickled my "invest more energy in this conversation" bias by mentioning ID vs. evolution, and there's a thought tickling the back of my head linking William Dembski's discredited mathematical premises for design inference and Eliezer's premise. I'm also getting "Danger, Will Robinson!" signals from the backbrain telling me not to accidentally find an argument that might give me the impression that Dembski might be on to something.
Let me see if I can get this right, given that I haven't quite "leveled up" as much as many of the participants here; I'm just an amateur SF writer who finds Overcoming Bias just about the most incredible resource about for developing interesting questions about how "humanity" will survive the future.
If I read Eliezer correctly, his central argument is that the long distance between the emergent, temporal mechanism of evolution, with its own distinct processes, is so far removed from the immediate mental machinery of a living mind that investing energy in attempting to discuss, describe, or recompute the evolutionary process is a waste, and Eliezer has better things to do. I agree.
Where I disagree (this goes back to a conversation Eliezer and I had, frack, 11 years ago now?) is in his second argument about the origins of signaling and its effect on cynicism. I'm much more cynical than Eliezer, but in a cheerful and hopeful way. When Eliezer writes about the strong selection pressure to, as he puts it, "carve [parental grief] into the genome from scratch," I have to wonder if it did. When I wonder what human consciousness "is for," my conclusion is that it's the current peak expression (not at all necessarily the optimal or the peak expression) of the evolutionary arms race that took place in an environment of other human beings. (Why the heck is it, whenever someone talks about the EEA, often the most salient feature of it-- the presence of other human beings-- seems to be almost an afterthought?)
Parental grief is seen in several mammal species. Accusing a baboon mother who refuses to part with the corpse of a dead child of being an "animal too stupid to know it's dead" is part of that bias that attempts to privilegize the human animal. If I wanted to know what parental grief signaled, I'd look at what other species did, at what faint, subtle expressions we inherited and emphasized, and wonder what it signaled-- or if it signaled anything at all.
I don't lose sleep about the distinction Eliezer wants to make. I mean, if you're comfortable with your sphexishness, you are (in the views of those who believe in an ineffable free will) already too cynical for society.
I'm rambling. It's BC (before coffee).
I had the oddest reaction to the test: I couldn't take it. As a touch-typist and a Dvorak keyboardist, I could not easily remember where the "I" and the "E" keys were, since the keys themselves remain labeled in the more traditional style. I can't help but wonder if that qualifies as a bias all its own.