A fair objection.
I had a quick search online and also flicked through Boyd's Convex Optimization, and didn't find Stuart Russell's claim expounded on. Would you be able to point me in a direction to look further into this?
Nevertheless, let me try to provide more detailed reasoning for my counterclaim. I assume that Russell's claim is indeed true in the classic optimisation domain, where there is a function R^N -> R f(x) as well as some inequality constraints on a subset of x.
However, I argue that this is not a good model for maximising a utility function in the real world.
First of all, it is not necessarily possible to freely search over x,... (read 442 more words →)
According to Professor Stuart Russell, and with a sentiment I have seen re-expressed often in the AI safety community:
A system that is optimizing a function of n variables, where the objective depends on a subset of size k<n, will often set the remaining unconstrained variables to extreme values; if one of those unconstrained variables is actually something we care about, the solution found may be highly undesirable.
I no longer believe this to be obviously true. Actually, I think it's likely to be untrue in the real world, and under nearly all realistic AGI-advent scenarios. This is because extreme values are only likely to be taken for the unconstrained variables if the environment... (read 809 more words →)
AGI is happening soon. Significant probability of it happening in less than 5 years.
I agree that there is at least some probability of AGI within 5 years, and my median is something like 8-9 years (which is significantly advanced vs most of the research community, and also most of the alignment/safety/LW community afaik).
Yet I think that the following statements are not at all isomorphic to the above, and are indeed - in my view - absurdly far off the mark:
We don’t have any obstacle left in mind that we don’t expect to get overcome in more than 6 months after efforts are invested to take it down.
... (read more)If you have technical understanding of
A meta-related comment from someone who's not deep into alignment (yet) but does work in AI/academia.
My impression on reading LessWrong has been that the people who are deep into alignment research are generally spending a great deal of their time working on their own independent research agendas, which - naturally - they feel are the most fruitful paths to take for alignment.
I'm glad that we seem to be seeing a few more posts of this nature recently (e.g. with Infra-Bayes, etc) where established researchers spend more of their time both investigating and critiquing others' approaches. This is one good way to get alignment researchers to stack more, imo.