Disclaimer. This is the first post in a series about my personal research progress as a member of the Interpretable Cognitive Architectures team at AI Safety Camp.
Epistemic status. An educated take on combining existing knowledge into cohesive whole.
TL;DR
- Ontogenetic curriculum is a map of developmental threads that describe the formation of a human mind
- The most promising research lines are:
- Ontogeny of values and concepts,
- Curriculum learning,
- Distribution of complexity between learning algorithm and training curriculum
What is OC?
Ontogenetic curriculum is a map of developmental threads that describe the formation of a human mind. That includes value learning, concept acquisition, skill building and development of cognitive capacities.
Some examples of relevant fields and research areas include:
Why OC? Promising research questions
There are many possible lines of research under this topic, but the most promising questions are:
- Ontogeny of values and concepts. Values and preferences in humans are formed gradually during upbringing, which resembles evolutionary dynamics with offspring-values born under the pressure of a new environmental niche. Getting better understanding of the rules governing this dynamics would help us design better value mechanism for artificial agents. Related questions:
- What are the exact conditions leading to the emergence of positive attractors in humans?
- What are the most powerful contributing factors in that dynamics?
- How do values apply to a new situation? Would an agent’s extrapolation of values to a new situation match with humans’?
- Curriculum learning. What new opportunities does the transition from batch training to ordered training open up? The main candidate for the answer is fine control over time when agent learns something. Choosing the right points of intervention suitable for specific value is in theory a good strategy for steering behaviour away from undesired territories.
- Where do the crucial bits of complexity aggregation mechanism lie: in the learning algorithm or in the ontogenetic curriculum? Modern ML almost entirely relies on power of the learning algorithm (barring data preparation activities like data augmentation). However, human mind undergoes transition from a seed that is fully reliant on supervision to a fully independent learning device. That raises the question: is there another combination of {learning algorithm, training curriculum} (probably closer to the human one) that is better at transparency/corrigibility/ability to combine values.
Why now?
No particular reason. I’ve been thinking about LW’s dropping the ball on this one for a while now. This project is my attempt at closing the gap.
Current focus
The current focus is on building the cohesive map of developmental trajectories, identifying different types of developmental situations, and channeling those insights into the research on positive attractors.
Capabilities Externalities Analysis
The impact of this project on capabilities depends on chosen research lines and ranges from “pretty safe, not advancing capabilities significantly” to “that’s a capability research not safety, stop it now”. I recognise the need to proceed with caution.