(You're correct. I was using fictionalist in that sense.)

I think the equivocation of "Theorem X is provable from Axiom Set Y" <--> such-and-such thing is Good; would be the part of that chain of reasoning a self-described fictionalist would ascribe fictionality to. 

As I understand it, it's the difference between thinking that Good is a real feature of the universe and Good being a wordgame that we play to make certain ideas easier to work with. Maybe a different example could illuminate some of this.

Fictionalism would be a good tool to describe the way we talk about Evolution and Nature. As has sometimes been said on this site, humans are not aligned towards Evolution, since they aren't inclusive fitness maximizers. We also say things like: such-and-such a feature evolved to do X function on an organism. Of course, that's not true. Biological features don't evolve in order to do a thing, they just happen to do things as a consequence of surviving in an ancestral environment.

We talk about organs and limbs "evolving to do" things, even when they do not, because it is a fiction that makes Evolution more palatable to intellectual examination, but unless you belief in weird stuff like teleology, it's just a fiction, a story that is convenient, and corresponds to real features of the world, but is not itself strictly true. And it is not untrue in a provisional way that we expect to be overturned with later reasoning and evidence, but untrue by design, because the literal truth of biological features arising by chance and operating by chance is harder to talk coherently about, given human constraints on mental compute.

I think your presentation of Eliezer's view is like that: one way it differs from a moral realist is not only that of a category error (objective morality vs aligning to human value) but that of a thought pattern deliberately constructed to aid human cognition vs a thought pattern attempting to align closely with correct mathmatical model of the object(s).

That's my reading of why it would matter if you're a moral antirealist (classical) vs a moral antirealist (fictionalist). I do consider fictionalist to be a subset of antirealist.

Thanks for the reply.

I don't disagree with Eliezer's position for the most part, I just don't see where he lays out a coherent foundation for why he believes certain things about human values. (Or maybe I'm just being uncharitable in my evaluation and not counting some things as "real arguments" that others would.)

By objective and discoverable, I meant in the sense of the values understood in and of themselves without reference to humans in particular. Obviously you can just model human brains and understand what they value, but I meant that you can't learn about "beauty" or "friendship" or what have you outside of that. That part of the post was inelegantly worded, and I'd probably strike that out if this was a long post and not a comment.

I used "Moral Fictionalist" as a descriptor for Eliezer's position because, although he probably wouldn't ascribe it to himself, it seems to me to be the best fit for it. I'm not a rationalist, and I don't have a rationalist background, I just like to read the site from time to time, and very occasionally comment. So my diction tends to sound "foreign" here.

 

"Why would any supermind want something so inherently worthless as the feeling of discovery without any real discoveries?"

"No free lunch.  You want a wonderful and mysterious universe?  That's your value."

"These values do not emerge in all possible minds.  They will not appear from nowhere to rebuke and revoke the utility function of an expected paperclip maximizer."

"Touch too hard in the wrong dimension, and the physical representation of those values will shatter - and not come back, for there will be nothing left to want to bring it back."

I've chosen a small representation of the sort of things that Eliezer says about human values. When I call Eliezer a moral fictionalist, I don't mean that he doesn't think human values are real, just that they are real in the way that fictional stories are real, ie. that they exist only in human minds, and are not in any way objective or discoverable.

Human values are, in Eliezer's view:

Irrational: they cannot be derived from first principles.
Accidental: they arise from the ancestral environment in which humans evolved.
Inalienable: You can't get jettison them for arbitrary values, your philosophy must ultimately reconcile your stated values with your innate ones^[1]
Fragile: because human values are a small subset of high dimensional intersections, they are subject to be destroyed by even small perturbations.

All of these attributes are just obvious consequences of his metaphysics so he doesn't attempt to justify any of it in the sequence you linked. Why would he? It's obvious. He's more interested in examining the consequences of these attributes on civilizational policy.
 

1. ^{^}

   "You do have values, even when you're trying to be "cosmopolitan", trying to display a properly virtuous appreciation of alien minds.  Your values are then faded further into the invisible background - they are less obviously human.  Your brain probably won't even generate an alternative so awful that it would wake you up, make you say "No!  Something went wrong!" even at your most cosmopolitan.  E.g. "a nonsentient optimizer absorbs all matter in its future light cone and tiles the universe with paperclips".  You'll just imagine strange alien worlds to appreciate.

   Trying to be "cosmopolitan" - to be a citizen of the cosmos - just strips off a surface veneer of goals that seem obviously "human"."

No. Yudkowski is a moral fictionalist, but he has never (to my knowledge) ever justified his position. Granted I haven't read his whole corpus of work, but from what I've seen he just takes it as a given.

The correct moral choice is for both people to lower their EA contributions to 0%.

Thoughtfulness, pro-sociality, and conscientiousness have no bearing on people's ability to produce aligned AI. 

They do have an effect on people's willingness to not build AI in the first place, but the purpose of working at Meta, OpenAI, and Google is to produce AI. No one who is thoughtful, pro-social, and conscientious is going to decide not to produce AI while working at those companies, while still having a job. 

Hence, the effect of discouraging those sorts of people from working at those companies has no net increase in Pdoom.

If you want to avoid building unaligned AI, you should avoid building AI.

Who says you contribute to the pool at the same rate you'd contribute to your own children? Surely other people in the pool would have different priorities than you, wouldn't they? What if there are N people in the pool and you contribute 1/5N to the children in each pool?

Add that to the fact, that maybe you only have one standout chromosome, and you could easily see a situation where genetic analysis of the population in your family + your pool shows a sudden disappearance of 90% of your genes with a proliferation of 5% of your genes. Is that equivalent to having children? Some people might say it's not.

Also, yes, obviously if you were trying to maximize your genetic genetic density you'd do all of the above: contribute to pools, clone yourself a couple times, have children normally (or with chromosomal selection), and contribute to a sperm banks. That'd be the route to take if you view maximizing genetic density as a terminal goal.

I think the reality is that people have some instinctual need to see more of themselves and their loved ones in the world, and that a learned person would use genetic inheritance as a proxy for this emotional non-quantifiable goal. I also suspect it's a threshold goal, and not a maximization goal, which is why people want some number N of children and not "as many children as I can afford to have".

Maximizing the amount of your genetic material in the (near) future is my null hypothesis. I don't think it's totally accurate, but in the absence of a good understanding of which parts of our genetic material produce the non-quantifiable traits we care about: things like the shape of one's smile, personality, taste in food, overall "mood", then I expect people to be reluctant to trade off genetic density at rates greater than ~25-60%

The alternative extreme hypothesis would be a "parent" who wants to maximize their "children's" traits to the point where they'd prefer 0% genetic inheritance if the resultant child would be superior in some respect.

That comparison misses something crucial, which is the density of genetic material passed on. Each generation represents a dilution of the first parent's genetic material with non-kin, but also has the potential for increased numbers of descendants at each generation. By the time your family would be producing your great-grandkids, they'd have the potential to have 2 dozen or more of your direct descendants.

With chromosomal selection you're trading off a massive amount of genetic saturation: essentially getting the percentage genetic inheritance of a great grandchild without the potential for the massive "payoff" of having numerous descendants sharing portions of your genetics.

Putting it like that, it's no surprise that people are going to feel repulsed by the idea...and that's before we even get into the part where the chromosomes which don't get selected are going to be instantly "lost" in a single generation. That's made more spooky by the fact that we won't know where demonstrably inheritable traits like "Has his mother's eyes; has his dad's smile; has temper/melancholy/mood similarities to one or the other parent" lie on the chromosomes. I wouldn't be surprised if the idea that you'd be risking losing something important like that is going to be a deal breaker for a lot of people.

Regarding DragonMagazine: It would often publish content for Dungeons and Dragons that was of a more hurried and slightly lower quality. This led to it being treated as a sort of pseudo 3rd party or beta source of monsters and player options.

People in online communities would frequently talk about options being "from Dragon Magazine" or "Dragon content" in order to forewarn people of content that may not have been given a thorough pass on editing/game balance. As such that phrase was very prevalent in online forums for D&D discussion, which as I understand it, would show up a lot in the training data.