This post does not try to add to the discussion about aligning an AGI/ASI to all human values but will rather focus on a smaller, arguably fundamental, subset of human values: prosocial behaviour.
(Note: with many agreeing that AGI and ASI will occur so closely together the differentiation between the two might be meaningless, we will only refer to ASI from now)
It aims to propose a possible way we can increase the chances of generating ASIs with prosocial values.
Almost all discussions relating to ASI risk discusses a singular ASI in isolation, what a singular mind would think and value and do in relation to Humanity, the so called Singleton outcome.
Few discuss the interactions... (read 646 more words →)
Perhaps "agency" is a better term here? In the strict sense of an agent acting in an environment?
And yeah, it seems we have shifted focus away from that.
Thankfully, thanks to our natural play instincts, we have a wonderful collection of ready made training environments: I think the field needs a new challenge of an agent playing video games, only receiving instructions of what to do using natural language.
These stories always assume that an AI would be dumb enough to not realise the difference between measuring something and the thing measured.
Every AGI is a drug addict, unaware that it's high is a false one.
Why? Just for drama?
The predicted cost for GPT-N parameter improvements is for the "classical Transformer" architecture? Recent updates like the Performer should require substantially less compute and therefore cost.
but indeed human utility functions will have to be aggregated in some manner
I do not see why that should be the case? Assuming virtual heavens, why couldn't each individuals personal preferences be fullfilled?
It seems pretty undeniable to me from these examples that GPT-3 can reason to an extend.
However, it can't seem to do it consistently.
Maybe analogous to people with mental and/or brain issues that have times of clarity and times of confusion?
If we can find a way to isolate the pattern of activity in GPT-3 that relates to reasoning we might be bale to enforce that state permanently?