Computational Agent Theory and ASI Alignment

A high-level overview of the CORAL plan for solving AI alignment.

I am planning to cover:

• What is the technical alignment problem and what are its core difficulties, in my view. This is a perspective that draws simultaneously on classical traditions in theoretical computer science / machine learning theory, ideas from analytic philosophy, and views that originated in the Machine Intelligence Research Institute / LessWrong.
• A high-level overview of computational agent theory. This theory differs from other accounts of intelligence and/or rational agency by combining a mature treatment of the relevant philosophical problems (drawing on ideas such as structural realism, computationalism and functional decision theory) with the promise of extracting practical algorithms.
• How computational agent theory leads to a solution of the technical alignment problem, something that requires a major rethinking of how we design AI systems.
• The roadmap towards completing the theory, and our current location on this roadmap.
• Some considerations (scientific, technological and social) about crossing the gap from a theoretical solution to AI alignment to a real-world solution to AI risk.