Long time lurker but first post here, so I'll quickly introduce myself. I work at the Ethereum Foundation leading their AI x Public Goods track, which broadly encompasses coming up with and piloting new credibly neutral funding mechanisms (especially those involving the use of AI to distribute money)

The purpose of this post is two-fold;

Soliciting feedback on the idea of distilled human judgment (DHJ) and its application to AI alignment
A call to action for the rationalist community to submit models in our 2 ongoing DHJ competitions to allocate funding to core Ethereum open source repositories and identify sybil (bot) wallet addresses

Let's start with an explanation of DHJ to a 5 year old;

Your teacher needs to grade 1000 assignments
She only manages to complete 100
Many AI agents or models give a grade to all the assignments
The 100 assignments the teacher did grade selects the winning model that then gives grades to all 1000

Since people often relate new ideas with what they are already familiar with, the best way to think of DHJ is as a Kaggle competition where models are competing to accurately predict the ground truth data (human judgments).

The main difference being, we don't have ground truth data for all the answers that models give but only a subset of them. But as submitters don't know which questions we have answers to and which we don't, they have to do a good job on all of them.

DHJ <> AI Alignment

I'll start this section with a vision of a world that I don't want but which I think we're headed towards

Every company has its own monolithic AI model trained on data of their C-suite. Employees pray for beneficence from the AI overlord to get attention to their project and also for promotions, pay raises, etc. Any change to the company AI is permissioned and requires review from a core team within the company

What is a viable alternative?

Every company has a marketplace of competing AI models that any team (or outside contributors) can submit. All these models are queried for answers to decisions that the C-suite needs to make, such as funding to different departments, who to lay off or promote, etc. Executives then see which models align most closely with the subset of decisions that they do make manually to make decisions across the board.

A good analogy for thinking about these 2 visions is whether we want AI to be similar to a monopoly ...

Posts

Wikitag Contributions

Comments