Abe Dillon's Shortform
Jul 31, 20192
Abe Dillon
67
1
36
If the "cartesian barrier" is such a show-stopper, then why is it non-trivial to prove that I'm not a brain in a vat remotely puppetering a meat robot?
A Cartesian agent that has detailed and accurate models of its hardware still won't recognize that dramatic damage or upgrades to its software are possible...
Was nobody intelligent before the advent of neuroscience? Do people need to know neuroscience before they qualify as intelligent agents? Are there no intelligent animals?
I'm really not sure how to interpret the requirement that an agent know about software upgrades. There is a system called a Gödel Machine that's compatible with AIXI(tl) and it's all about self modification, however; I don't... (read 829 more words →)
No its not. There is no objective sense in which human suffering and extinction is bad. It's not even a matter of degree. Questions of morality are only meaningful from a subjective POV. That's Hume's Law in a nutshell. You can't solve the alignment problem by believing that we should aim as close as possible to the objective truth that everything is just particles and forces and alignment doesn't matter. It's circular reasoning.
Why do you assume that Yelsgib doesn't know or keep that in mind?
The problem is that Yudkowsky insists that a mechanistic view of the universe is the only correct perspective even though problems like alignment are inherently inaccessible from such a perspective due to Hume's Guillotine. It's only from a subjective POV that the idea that human suffering and/or extinction can be considered bad.
I always feel like I'm reading your response to some other argument, but you decide to use some indirect reference or straw-man instead of actually addressing the impetus for your posts. This article is a long way of saying that even when things aren't black and white, that doesn't mean shades of grey don't matter.
Also, I think people often reach for analogies as though they always provide clarification when sometimes they just muddle things. I have no idea what to make of your disappearing moon example. The odds that the entire moon could disappear and reappear seem very hard to compare to the odds that there's an invisible dragon that can cure cancer. Why not compare something highly probable with the cancer curing dragon instead of something so strangely contrived? You can't prove Big Ben will ring tomorrow, but the odds are much better than an invisible dragon baking you a cake!
Both AIXI and AIXItl will at some point drop an anvil on their own heads just to see what happens
You're confusing arbitrary optimization with a greedy algorithm which AIXI explicitly is not. It considers a future horizon. I see you commit this falacy often. You implicitly assume "what would an arbitrarily intelligent system do?" is equivalent to "what would a arbitrarily greedy algorithm do?"
Also, the math of AIXI assumes the environment is separably divisible
I don't know what you mean by this.
If you're talking about the fact that it considers models of the environment where dropping the anvil on its head will yield some reward are mixed in with models of the... (read 431 more words →)
Hey, G Gordon Worley III!
I just finished reading this post because Steve2152 was one of the two people (you being the other) to comment on my (accidentally published) post on formalizing and justifying the concept of emotions.
It's interesting to hear that you're looking for a foundational grounding of human values because I'm planning a post on that subject as well. I think you're close with the concept of error minimization. My theory reaches back to the origins of life and what sets living systems apart from non-living systems. Living systems are locally anti-entropic which means: 1) According to the second law of thermodynamics, a living system can never be a truly closed... (read more)
Thanks for the insight!
This is actually an incomplete draft that I didn't mean to publish, so I do intend to cover some of your points. It's probably not going to go into the depth you're hoping for since it's pretty much just a synthesis of the bit of information from a segment from a Radiolab episode and three theorems about neural networks.
My goal was to simply use those facts to provide an informal proof that a trade-off exists between latency and optimality* in neural networks and that said trade-off explains why some agents (including biological creatures) might use multiple models at different points in that trade-off instead of devoting all their computational... (read 1183 more words →)
In short, your second paragraph is what I'm after.
Philosophically, I don't think the distinction you make between a design choice and an evolved feature carries much relevance. It's true that some things evolve that have no purpose and it's easy to imagine that emotions are one of things especially since people often conceptualize emotion as the "opposite" of rationality, however; some things evolve that clearly do serve a purpose (in other words there is a justification for their existence), like the eye. Of course nobody sat down with the intent to design an eye. It evolved, was useful, and stuck around because of that utility. The utility of the eye (its justification for sticking around) exists independent of whether the eye exists. A designer recognizes the utility before hand and purposefully implements it. Evolution "recognizes" the utility after stumbling into it.
How? The person I'm responding to gets the math of probability wrong and uses it to make a confusing claim that "there's nothing wrong" as though we have no more agency over the development of AI than we do over the chaotic motion of a dice.
It's foolish to liken the development of AI to a roll of the dice. Given the stakes, we must try to study, prepare for, and guide the development of AI as best we can.
This isn't hypothetical. We've already built a machine that's more intelligent than any man alive and which brutally optimizes toward a goal that's incompatible with the good of man kind. We call it, "Global... (read more)
A flaw in the Gödel Machine may provide a formal justification for evolution
I've never been a fan of the concept of evolutionary computation. Evolution isn't fundamentally different than other forms of engineering, rather it's the most basic concept in engineering. The idea slightly modifying an existing solution to arrive at a better solution is a fundamental part of engineering. When you take away all of an engineer's other tools, like modeling, analysis, heuristics, etc. You're left with evolution.
Designing something can be modeled as a series of choices like traversing a tree. There are typically far more possibilities per choice than is practical to explore, so we use heuristics and intelligence to prune... (read more)
Rough is easy to find and not worth much.
Diamonds are much harder to find and worth a lot more.
I once read a post by someone who was unimpressed with the paper that introduced Generative Adversarial Networks (GANs). They pointed out some sloppy math and other such problems and were confused why such a paper had garnered so much praise.
Someone replied that, in her decades of reading research papers, she learned that finding flaws is easy and uninteresting. The real trick is being able to find the rare glint of insight that a paper brings to the table. Understanding how even a subtle idea can move a whole field forward. I kinda sympathize... (read more)
Drop the "A"
Flight is a phenomenon exhibited by many creatures and machines alike. We don't say mosquitos are capable of flight and helicopters are capable of "artificial flight" as though the word means something fundamentally different for man-made devices. Flight is flight: the process by which an object moves through an atmosphere (or beyond it, as in the case of spaceflight) without contact with the surface.
So why do we feel the need to discuss intelligence as though it wasn't a phenomenon in its own right, but something fundamentally different depending on implementation?
If we were approaching this rationally, we'd first want to formalize the concept of intelligence mathematically so that we can bring... (read more)
I'm having a great deal of trouble reading this because it's littered with big red "INVALID EQUATION" and "UNBALANCED EQUATION" censors.
I think your argument regarding "Bob pushing a button" is flawed for a few reasons.
You assume a specific example of a man pushing a button as a stand-in for a more general framework. That examply does not generalize.
In general, the agent gets a reward signal from some untility function. It doesn't have to even be implemented by a mechanism the agent has agency over. Deep Blue interacts with a chess board. That's its the environment it has visibility into and it has very limited agency over very limited degrees of freedom within... (read more)