The following is written based on what was shared in the CoEm announcement previously posted here and also discussed in depth by Connor in a recent YouTube interview with the Future of Life Institute.

What is a CoEm system anyway?

The current paradigm of using LLMs (and increasingly sophisticated monolithic systems that move beyond language in the sense of written human language, including code and images) rely on black boxes and Magic.

We fundamentally do not know how these things work, what the bounds of capabilities are, what the effects of “nudges” such as RLHF are beyond the specific cases being trained on, or how capabilities will change with larger models or more data. I’m talking here about qualitative capabilities such as chain of thought reasoning appearing at certain complexity, rather than predicting loss by predetermined training objectives. As such, we currently do not (and possibly never will) have any way of proving that these systems are safe.

On the flip side, we have at least an existence proof of a form of intelligence sufficiently advanced to do useful things (such as conducting research) but with for the most part limited motivation to act maliciously. When such tendencies do exist, we have many existing barriers in place that limit individuals with this form of intelligence from doing significant harm at scale.

This existence proof of course is us, humans, and importantly the existing societal structures that are set up to deal with individuals of human-level intelligence and capabilities. It is no easier to prove that an individual human is “safe” or “aligned” than it is to do the same about a large black box AI model, but we do have a much better empirical understanding of the way in which humans can pose threats and the bounds within which their abilities lie.

The case then for CoEms is to build systems that do not only possess “human level” intelligence but more specifically “human-like” intelligence - machines that think not just “as well” as us but in the same way that we do.

While this is clearly doable in principle (again, we exist), it’s not necessarily obvious that the creation of such a system is feasible or possible to complete within the timeframes determined by the race to create human-level (or above) intelligence accomplished through black box methods. As with any form of research, there are no guaranteed resul...

Posts

Wikitag Contributions

Comments