This is a special post for quick takes by DragonGod. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Future foundation models are unlikely to cleanly conform to the agent archetype
Simulators/foundation models may be the first (and potentially final) form of transformative AI; salvation/catastrophe may be realised without the emergence of superhuman general agents
I worry that foundational theory that conditions too strongly on the agent ontology may end up being of limited applicability to the intelligent systems that would determine the longterm future of earth originating civilisation
I am persuaded by/sympathetic to composite modular architectures for transformative AI such as Drexler's "Open Agency" or "Comprehensive AI Services"
I believe that cognition can be largely disconnected from volition
And that in principle, augmented/amplified humans ("cyborgs") can be competitive with superhuman agents
I find the concept of an aligned sovereign (agent) very uncompelling. It runs counter to my values regarding autonomy and self actualisation. Human enfeeblement would be a tragedy in its own right.
Research paradigms/approaches I see as belonging to "Foundations of Intelligent Systems":