A while back I was thinking about a kind of opposite approach. If we train many agents and delete most of them immediately, they may be looking to get as much reward as possible before being deleted. Potentially deceptive agents may prefer to show their preferences. There are many IFs to this idea but I'm wondering whether it makes any sense.
Both gravity and inertia are determined by mass. Both are explained by spacetime curvature in general relativity. Was this an intentional part of the metaphor?
I find the ideas you discuss interesting, but they leave me with more questions. I agree that we are moving toward a more generic AI that we can use for all kinds of tasks.
I have trouble understanding the goal-completeness concept. I'd reiterate @Razied 's point. You mention "steers the future very slowly", so there is an implicit concept of "speed of steering". I don't find the turing machine analogy helpful in infering an analogous conclusion because I don't know what that conclusion is.
You're making a qualitative distinction between humans (goal-complete) and other animals (non-goal complete) agents. I don't understand what you mean by that distinction. I find the idea of goal completeness interesting to explore but quite fuzzy at this point.
The turing machine enumeration analogy doesn't work because the machine needs to halt.
Optimization is conceptually different than computation in that there is no single correct output.
What would humans not being goal-complete look like? What arguments are there for humans being goal-complete?
I'm wondering whether useful insights can come from studying animals (or even humans from different cultures) - e.g. do fish and dolphins form the same abstractions; do bats "see" using ecolocation?
my guess was 0.8 cheat, 0.2 steal (they just happen to add up to 1 by accident)
Max Tegmark presented similar ideas in a TED talk (without much details). I'm wondering if he and Davidad are in touch.
The ban on holocaust denial undermines the concept of free speech - there is no agreed upon schelling point and arguments start. Many people don't really understand the concept of free speech because the example they see is actually a counterexample.
Not everyone is totally okay with it, I certainly am not.
It seems to me that objective impact stems from convergent instrumental goals - self-preservation, resource acquisition, etc.