I really enjoyed reading this post, especially its connection with the Pearl's belief propagation algorithm in bayesian networks.Thank you Eliezer!
I feel really uncomfortable with this idea: "EITHER YOUR MODEL IS FALSE OR THIS STORY IS WRONG."
I think this statement suffers from the same limitations of propositional logic; consequently, it is not applicable to many real life situations.
Most of the times, our model contains rules of this type (at least if we are rationalists): Event A occurs in situation B with probability C, where C is not 0 or 1. Also, life experiences teach us that we should update the probabilities in our model over time. So beside the uncertainty caused by the probability C, there is also uncertainty resulted from our degree of belief in the correctness of the rule itself. The situation becomes more complicated when the problem is cost sensitive.
I got your point (I hope so) and I'm definitely not trying to say "IT IS WORNG" but I think it is true to some degree.
My native language is Persian (Farsi). There is no definite article in Persian and the specific object/ person/ idea which a noun refers to is determined from the context.
I agree with you about the ambiguity of the word "singularity". Not only there are different definitions for "singularity" in AI, the term is also applied in other contexts (e.g. economic singularity, gravitational singularity). I think, as you said, talking about "a singularity" is more appropriate.
My name is Ali and I'm 24 year-old. I graduated in software engineering and currently, I'm in second year of Master of Science in Artificial Intelligence. Machine learning is my primary interest; however, I am extremely enthusiastic about other subfields of AI, cognitive science, psychology, physics and biology. I love to learn the assembly code fragments underlying high level processes in the universe and to see how complexities are decomposed into simple components by science.
Being born in a religious country, my first steps in the way of rationalism began by questioning the religious beliefs in my adolescence. Since then, I learned to live with probabilities, evidences and explanations.
I found Less Wrong by searching about singularity. I'm sure there is a lot here for me to learn, but I hope someday I'll be able to contribute.
(English is not my first language, so I apologize for any error in my writing. :D)