My problems with Formal Friendly Artificial Intelligence work

I'm writing this to get information about the lesswrong community and whether it worth engaging. I'm a bit out of the loop in terms of what the LW community is like and whether it can maintain multiple view points (and how known the criticisms are).

The TL;DR is I have problems with treating computation in an overly formal fashion. The more pragmatic philosophy suggested implies (but doesn't prove) that AI will not be as powerful as expected as the physicality of computation is important and instantiating computing in a physical fashion is expensive.

I think all the things I will talk about are interesting, but I don't see the sufficiency of them when considering AI running in the real world in real computers.

1. Source code based decision theory

I don't understand why:

other agents trust that your source code is what you say it is
other agents trust that your implementation of your interpreter matches their understanding of the interpreter. I don't see how they get round trustless trust (inserting code/behaviour via malicious compiler/interpreter) issues when they don't have the ability to do diverse compilation.

I understand the justification that causal decision theory is not optimal and you want something better. But it seems that whatever that better thing is has to be able to adapt to conditions. If people start baking in TDT or UDT into the core of their AIs philosophy for all eternity, they can be tricked by machines that 99% the time run the source code they purport to run as expected, but not when the fate of the galaxy decisions are being made. You need a system that can start to do hardware verification on an ad hoc basis if the problems it comes across in the environment warrant it.

2. General Functionalism

The idea that it doesn't matter how you compute something just whether the inputs and outputs are the same.

The battery life of my phone says that the way of computation is very important, is it done on the cloud and I have to power up my antennae to transmit the result.
Timing attacks say that speed of the computation is important, that faster is not always better.
Rowhammer says that how you layout your memory is important. Can I flip a bit of your utility calculation?
Memory usage, overheating, van Eck phreaking etc etc....

Functionalism is leaky abstraction over all of physics. It is in many cases a handy abstraction but it can and will still bite you day to day.

I point these things out not because I want to abandon either of these in their entirety, just their usage in the core of our thinking about AI design. I think pragmatism make a better core as it makes less assumptions about the nature of reality, with other stuff layered on top.

I also think that an abandonment of functionalism makes AI less scary. If formal proof is insufficient for AI improvement and experimentation and exposure to the world is required to make sure there aren't side channels or resource exhaustions neglected by the formal model, the potential top speed of improvement slows down dramatically. Functionalism collapses the world and simplifies it, if it incorrect to use in the AI field then the solution space (and navigation of it) expands hugely.

I think all the things I will talk about are interesting, but I don't see the sufficiency of them when considering AI running in the real world in real computers.

1. Source code based decision theory

I don't understand why:

other agents trust that your source code is what you say it is
other agents trust that your implementation of your interpreter matches their understanding of the interpreter. I don't see how they get round trustless trust (inserting code/behaviour via malicious compiler/interpreter) issues when they don't have the ability to do diverse compilation.

2. General Functionalism

The idea that it doesn't matter how you compute something just whether the inputs and outputs are the same.

The battery life of my phone says that the way of computation is very important, is it done on the cloud and I have to power up my antennae to transmit the result.
Timing attacks say that speed of the computation is important, that faster is not always better.
Rowhammer says that how you layout your memory is important. Can I flip a bit of your utility calculation?
Memory usage, overheating, van Eck phreaking etc etc....

Functionalism is leaky abstraction over all of physics. It is in many cases a handy abstraction but it can and will still bite you day to day.

2+2=4 pays rent.

Ugh can use it right away for counting days. Source code based decision theory not so much. There aren't the societies based on agents that can read each others source code, so I can't try and predict them with source code based decision theories. It seems like it is mathematically interesting thing though, so it is still interesting. I just don't want it to be a core part of our sole pathway to try and solve the AI problem.

Would a recommendation to skip solving the simplified problems, because they don't include all the messy details, more likely lead to prople solving the hard problems faster, or giving up?

Perhaps it would lead people to avoid trying to find optimal decision theories and accept the answer that the best decision theory depends on the circumstances. Then we can figure out what our circumstances are and find good decisions theories for those. And create designs that can do similarly.

Like the best search algorithm is context dependent, where even algorithms of a worst complexity class can be better due to memory locality and small size.

How specifically are you planning to use this insight in constructing a Friendly superintelligent AI?

Supposition> If answers to how decisions are made (and a whole host of other problems) are contextual and complex then it is worth trading information about what answers they have found within their context.

Figure the control flows between parts of the human brain that means they manage to roughly align themselves into a coherent entity (looking at dopamine etc).
Apply the same informational constraints to computers so that we can link a computer up to a human (it doesn't need to be physically as long as the control flows work). The computer should be aligned with the user as much as a part of the brain is aligned to another part (which is hopefully sufficient).
While this is in an embryonic stage get as many people from all around the world to use it .
Hard take off (I think this less likely due to the context sensitivity things I have described, but still possible)- it is worthwhile to trade between agents so groups of human/computers that co-operate and trade information advance quicker. than loan rogue agents. If we have done the alignment work right then it is likely the weight of them will be able to squash any human inimical systems that appear.

This pathway makes no sense from people that expect there to be winner take all optimal AI designs as any one embryonic system might find the keys to the future and take it over. But if that is not the way the world works....

There aren't the societies based on agents that can read each others source code,

Human beings can probabilistically read each others' source code. That's why we use primitive versions of noncausal decision theory like getting angry, wanting to take revenge, etc.

5Viliam10y

There are mathematical concepts that didn't pay rent immediately: imaginary numbers, quaternions, non-Euclidean geometry, Turing machines... But thanks for the specific plan, it sounds like it could work.

8

My problems with Formal Friendly Artificial Intelligence work

8

1. Source code based decision theory

2. General Functionalism

8

8

My problems with Formal Friendly Artificial Intelligence work

8

1. Source code based decision theory

2. General Functionalism

8