I agree. I never saw this analogy as being about the specific bear_fat_and_honey snack. Bear fat and honey is just a vivid example to say that even though we want fat, sugar and salt, some elements of the class of foods with fat, sugar and salt are ranked as more appetizing. And it is this preference ranking  what the observing aliens could not predict from knowing about preference for fat, sugar and salt.

Maybe stating the obvious, but EDVAC was developed by J. Presper Eckert and John Mauchly.

The rounding of concepts and results is very common in academia, where researchers try to publish results or ideas that are "novel". A common concern in the peer-review process is that the idea is not really new, but a consequence or a re-framing of another idea. "How is this new?" is a typical question in any referee report. In some cases this concern is correct. In others, it is conceptual rounding of a new result.

This intellectual attitude dismisses useful new instantiations of general ideas. In many cases, the particular application could be more impactful than the general result.  It reminds of the reaction of Jonh von Neumann to John Nash's work: "That’s trivial!. That’s just a fixed point theorem.” . Von Neumann was rounding too much. [I use this as an illustrative anecdote, well knowing that Nash partly generalized JvN's results ]

I work in research in economics/management, where conceptual rounding is endemic.  A machine learning colleague became exasperated by the general attitude in management, lamenting that  marginal improvements to existing results are seen as trivial. He told me, "If aerospace were run like management journals, planes would not fly". 

I am a long-time lurker; I started following EY in Overcoming Bias and then here, where I visit on and off. I have been a business school professor for about 15 years. I use a pseudonym to keep my online professional profile distinct from my non-professional online activity. 

I notice that  business topics are not often discussed in LW, or at least not in the same detail and precision as other topics. It is certainly not a focal theme of the blog, but I was wondering if there would be interest to discuss business topics without the "fluff" that one inevitably finds in airport books and business school discussions.

For instance, there is a strong link between decision-making and the foundations of marketing. There are also interesting connections between branding and categorization and linguistics. If you guys think this could be a contribution to the blog, I would be more than happy to give it a go. 

I agree with the frustration. Wolfram was being deliberately obtuse. Eliezer summarised it well toward the end, something like "I am telling you that the forest is on fire and you are telling me that we first need to define what do we mean by fire". I understand that we need definitions for things like "agency" or technology "wanting" something and even why do we mean by a "human" in the year 2070. But Wolfram went a bit too far. A naive genius that did not want to play along in the conversation. Smart teenagers talk like that.

Another issue with this conversation was that, even though they were listening to each other, Wolfram was too keen to go back to his current pet ideas. Eliezer's argument is (not sure) independent on whether we think the AIs will fall under computational "irreducibilteh", but he kept going back to this over and over.

I blame the ineffective exchange primarily on Wolfram in this case. Eliezer is also somewhat responsible for the useless rabbitholes in this conversation. He explains his ideas vividly and clearly. But there is something about his rhetoric style that does not persuade those who have not spent time engaging with his ideas beforehand, even someone as impressive as Wolfram. He also goes on too long on some detail or some contrived example rather than ensuring that the interlocutor in the same epistemological plane. 

Anyway, fun thing to listen to

Not a deep insight, Just to add an example to the curb-cut effect I experienced that went from mild reluctance to enthusiastic acceptance. 

Background: My son is hard of hearing (almost deaf). The health insurance paid for a system of microphones that connect to his hearing aid /CI and also to very high quality speakers in the back. The whole thing cost several thousands euros. The school also has really good acoustic panels in the ceiling (this schools admits one or two hard-of-hearing kids a year, out of about 120).

First reactions. Some teachers looked like "do I have to do this?" Meaning wearing the mic, learning how to use it. Other parents complaint in the fist parents meetings, asking "Why does my kid need to speak into a microphone?", presuming it is unnatural and it does not lead to good class discussion. "What about the shy kids?"

One month in, everyone loves it. The teachers love it because their voices get amplified with high fidelity. Students like that the system forces them to speak one at a time and everyone gets their turn. In addition, every child in the back can hear what the shy child in the front is mumbling when they contribute to class discussion. Overall, the learning experience is much better. So much better, that some teachers actually commented that they would like the system in every classroom. Without the hearing aid tech, such a system would cost about 2000 euros. But it cannot be installed, because the improved learning is unquantifiable and the insurance would only pay for it if there is a child with a hearing disability.