ozziegooen's Shortform

Questions around Making Reliable Evaluations

Most existing forecasting platform questions are for very clearly verifiable questions:

  • "Who will win the next election"
  • "How many cars will Tesla sell in 2030?"

But many of the questions we care about are much less verifiable:

  • "How much value has this organization created?"
  • "What is the relative effectiveness of AI safety research vs. bio risk research?"

One solution attempt would be to have an "expert panel" assess these questions, but this opens up a bunch of issues. How could we know how much we could tr

... (read more)
Showing 3 of 14 replies (Click to show all)

Making long term predictions is hard. That's a fundamental problem. Having proxies can be convenient, but it's not going to tell you anything you don't already know.

2ozziegooen5moYea, in cases like these, having intermediate metrics seems pretty essential.
2romeostevensit5moDeference networks seem underrated.

ozziegooen's Shortform

by ozziegooen 31st Aug 2019127 comments