jonathanstray
49
12
jonathanstray has not written any posts yet.
jonathanstray has not written any posts yet.
But under these assumptions, combining evidence always gives the right answer. Compare with the example in the post: "vote on a, vote on b, vote on a^b" which just seems strange. Shouldn't we try to use methods that give right answers to simple questions?
a) "Everyone does Bayesian updating according to the same hypothesis set, model, and measurement methods" strikes me as an extremely strong assumption, especially since we do not have strong theory that tells us the "right" way to select these hypothesis sets, models, and measurement instruments. I would argue that this makes Aumann agreement essentially useless in "open world" scenarios.
b) Why should uniquely consistent aggregation methods exist at all?... (read more)
You can peek into everyone's heads, gather all the evidence, remove double-counting, and perform a joint update. That's basically what Aumann agreement does - it doesn't vote on beliefs, but instead tries to reach an end state that's updated on all the evidence behind these beliefs.
Right, this is where strong Bayesianism is required. You have to assume, for example, that everyone agrees on the set of hypotheses under consideration and the exact models to be used. This is not just an abstract plan for slicing the universe into manageable events, but the actual structure and properties of the measurement instruments that generate "evidence." If we wish to act as well we... (read more)
Aumann agreement isn't an answer here, unless you assume strong Bayesianism, which I would advise against.
I have to say I don't know why a linear combination of utility functions could be considered ideal. There are some pretty classic arguments against it, such as Rawls' maximin principle, and more consequentialist arguments against allowing inequality in practice.
If you liked this post, you will love Amartya Sen's Collective Choice and Social Welfare. Originally written in 1970 and expanded in 2017, this is a thorough development of the many paradoxes in collective choice algorithms (voting schemes, ways to aggregate individual utility, and so on.)
My sense is the AI alignment community has not taken these sorts of results seriously. Preference aggregation is non-trivial, so "aligning" an AI to individual preferences means something much different than "aligning" an AI to societal preferences. Different equally-principled ways of aggregating preferences will give different results, which means that someone somewhere will not get what they want. Hence an AI agent will always have some type... (read more)
So I was very surprised when I learned that a single general method in deep learning (training an artificial neural network on massive amounts of data using gradient descent)[2] led to performance comparable or superior to humans’ in tasks as disparate as image classification, speech synthesis, and playing Go. I found superhuman Go performance particularly surprising—intuitive judgments of Go boards encode distillations of high-level strategic reasoning, and are highly sensitive to small changes in input.
I think it may be important to recognize that AlphaGo (and AlphaZero) use more than deep learning to solve Go. They also use tree search, which is ideally suited to strategic reasoning. Neural networks, on the other hand, are famously bad at symbolic reasoning tasks, which may ultimately have some basis in the fact that probability does not extend logic.
We could look at donors' public materials, for example evaluation requirements listed in grant applications. We could examine the programs of conferences or workshops on philanthropy and see how often this topic is discussed. We could investigate the reports and research literature on this topic. But I don't know how to define enough concern.
My sense is that donors do care about evaluation, on the whole. It's not just GiveWell / Open Philanthropy / EA who think about this :P
See for example https://www.rockpa.org/guide/assessing-impact/
Well said. And this middle ground is exactly what I am worried about losing as companies add more AI to their operations -- human managers can and do make many subtle choices that trade profit against other values, but naive algorithmic profit maximization will not. This is why my research is on metrics that may help align commercial AI to pro-social outcomes.
While Bayesian statistics are obviously a useful method, I am dissatisfied with the way "Bayesianism" has become a stand-in for rationality in certain communities. There are well-developed, deep objections to this. Some of my favorite references on this topic:
A... (read more)
Because central planning is so out of fashion, we have mostly forgotten how to do it well. Yet there are little known historical methods that could be applicable in the current crisis, such as input-output analysis, as Steve Keen writes:
... (read more)