I think a key element here is (the distribution of) whether the AI is motivated (i.e. acts as if motivated) to want to be controlled by us, or not — i.e. something like corrigibility or value learning, and whether we can arrange that. Is the workshop expected to cover that question?
Thank you for your feedback. The workshop arranges a lot of time for free discussions, so the motivation of the AI to be controlled might pop up there. Under the current proposals for talks, the focus is more on the environment in which the agent evolves than the agent itself. However, discussions about the "level of trust" or "level of cooperation" needed to actually keep control is absolutely in the theme.
On a more personal level, unless I have very strong reasons to believe in an agent's honesty, I would not feel safe in a situation where my control depends on the agent's cooperation. As such, I would like to understand what control look like in different situations before surrendering any control capabilities to the agent.
Whether or not we decide to put an agent in a situation that we can't keep under control if the agent doesn't wish to be controlled is an interesting topic - but not on the agenda for now. If you'd like to participate and run a session on that topic, you're more than welcome to apply!
Can we keep enough control over AI? If systems are developed to be more and more autonomous, this is no longer a given. It's a hypothesis, calling for serious investigation. Even alignment relies on control. Researchers build mechanisms to control AI's impacts in line with human values.
So how must a control mechanism operate? What limits its capacity to track and correct all the AI signals/effects? Does it provide enough stability, or give way eventually to runaway impacts?
We explore these questions in a new field: Limits to Control.
This in-person workshop is meant to facilitate deep collaboration. We're bringing together researchers to map the territory of AI control limitations – to understand the dynamics, patterns, and impossibilities of control. We are thrilled to welcome Roman Yampolskiy, Anders Sandberg, Forrest Landry, and other researchers working on control limitations.
This workshop aims to:
Detailed logistical information will be provided to confirmed participants.
We aim to strike a balance between structured sharing and messy exploration – we believe this is where the best ideas tend to emerge. Over the three days, we will do:
Sessions will include:
(More details on specific talks and activities will be added as confirmed.)
This section will be updated after the workshop with summaries, key insights, and any public materials generated.
Potential outputs may include:
This workshop is for researchers actively working on or deeply interested in the theoretical and practical limits of AI control. Do you wish to contribute to these focused discussions? Email Orpheus at o@horizonomega.org to express your interest.
Costs & funding: Participants are generally expected to cover their own travel and accommodation. We can reimburse only some whose research is not yet funded. The workshop has a grant offer from Survival and Flourishing Fund.
To prepare: Read work by participants you are curious to chat with. Then we share some understanding already going in. Most of our time will be in collaborative discussions, so consider where you could bring in specific problems or concepts.
Writings by participating researchers:
Papers:
Essays:
Sorted roughly by ease of reading. To clarify an argument, do reach out to authors. Authors value questions!
This event is hosted by HΩ, and organized by:
For inquiries regarding the workshop, please contact Orpheus at o@horizonomega.org.