The upcoming ALIFE 2023 conference is hosting a workshop on AI Alignment and Artificial Life. This will complement the special session on AI Alignment at the conference. The workshop will include presentations and a discussion panel from established AI Alignment and ALife researchers, exploring the overlap between these two fields....
These are some hypothetical scenarios involving emulating a (human) mind on a digital computer. They seem to present counterintuitive implications for the question of whether an emulated mind would actually be conscious or not. This relates to the question of whether consciousness is substrate independent, and whether consciousness is fundamentally...
Thanks again John for giving this talk! I really enjoyed the talk at the time and was pleasantly surprised with the positive engagement from the audience. I'm also pleased that this turned into a resource that can be re-shared.
I suppose I can imagine an architecture that has something like a central planning agent that is capable of having a goal, observing the state of the world to check if the goal had been met, coming up with high level strategies to meet that goal, then delegating subtasks to a set of subordinate sub-agents (whilst making sure that these tasks are broken down enough that the sub-agents themselves don't have to do much long time-horizon planning or goal directed behaviour).
With this architecture it seems like all the agent-y goal-directed stuff is done by a single central agent.
However I do agree that this may be less efficient or capable in practice than an architecture with more autonomous, decentralised sub-agents. But on the other hand it might be better at more consistently pursuing a stable goal, so that could compensate.
Good points, however I'm still a bit confused about the difference between two different scenarios: "multiple sub-agents" vs "a single sub-agent that can use tools" (or can use oracle sub-agents that don't have their own goals).
For example a human doing protein folding using alpha-fold; I don't think of that as multiple sub-agents, just a single agent using an AI tool for a specialised task (protein folding). (Assuming for now that we can treat a human as a single agent, which isn't really the case, but you can imagine a coherent agent using alpha-fold as a tool).
It still seems plausible to me that you might have a mind made of many different parts, but there is a clear "agent" bit that actually has goals and is controlling all the other parts.
I agree that initially a powerful AGI would likely be composed of many sub-agents. However it seems plausible to me that these sub-agents may “cohere” under sufficient optimisation or training. This could result in the sub-agent with the most stable goals winning out. It’s possible that strong evolutionary pressure makes this more likely.
You could also imagine powerful agents that aren’t composed of sub-agents, for example a simpler agent with very computationally expensive search over actions.
Overall this topic seems under-discussed in my opinion. It would be great to have a better understanding of whether we expect sub-agents to turn into a single coherent agent.
The video of John's talk has now been uploaded on YouTube here.
I really enjoyed this dialogue, thanks!
A few points on complexity economics:
The main benefit of complexity economics in my opinion is that it addresses some of the seriously flawed and over-simplified assumptions that go into classical macroenomic models, such as rational expectations, homogenous agents, and that the economy is at equilibrium. However it turns out that replacing these with more relaxed assumptions is very difficult in practice. Approaches such as agent-based models (ABMs) are tricky to get right, since they have so many degrees of freedom. However I do think that this is a promising avenue of research, but it maybe it still needs more time and effort to pay off. Although it's... (read more)
The workshop talks from the previous year's ALIFE conference (2022) seem to be published on YouTube, so I'm following up with whether John's talk from this year's conference can be released as well.
An interesting paper is The information theory of individuality, Krakauer et. al
The upcoming ALIFE 2023 conference is hosting a workshop on AI Alignment and Artificial Life. This will complement the special session on AI Alignment at the conference. The workshop will include presentations and a discussion panel from established AI Alignment and ALife researchers, exploring the overlap between these two fields.
Please see the workshop website for more information: https://humanvaluesandartificialagency.com/workshop/
Date: Friday 28th July, 2023
Venue: Hybrid: Sapporo Japan and remote
Attending remotely requires paying the remote registration fee. The workshop organisers are offering a limited number of bursaries for anyone who cannot afford the registration fee. See the registration page on the workshop website for details.
Artificial Life (ALife) is a field which studies the properties of life and... (read more)
This is a really interesting point that I hadn't thought of!
I'm not sure where I land on the conclusion though. My intuition is that two copies of the same mind emulation running simultaneously (assuming they are both deterministic and are therefore doing identical computations) would have more moral value than only a single copy, but I don't have a lot of confidence in that.
These are some hypothetical scenarios involving emulating a (human) mind on a digital computer. They seem to present counterintuitive implications for the question of whether an emulated mind would actually be conscious or not. This relates to the question of whether consciousness is substrate independent, and whether consciousness is fundamentally computational. These scenarios are inspired by ideas in the book Permutation City, by Greg Egan.
These thought experiments challenge my intuitions about digital consciousness. Some of these challenges arise from the discrete nature of digital computation; with a discrete digital simulation you can increase the “distance” (in time or space) between timesteps, which is a bit of a mind-bending prospect. Additionally some of the... (read 1794 more words →)
Have you come across this book on video games and philosophy: "Ten Things Video Games Can Teach Us: (about life, philosophy and everything)" https://amzn.eu/d/ef7KGjJ
It maps various games to philosophical thought experiments, I quite enjoyed it.