Questions around Making Reliable Evaluations

Most existing forecasting platform questions are for very clearly verifiable questions:

  • "Who will win the next election"
  • "How many cars will Tesla sell in 2030?"

But many of the questions we care about are much less verifiable:

  • "How much value has this organization created?"
  • "What is the relative effectiveness of AI safety research vs. bio risk research?"

One solution attempt would be to have an "expert panel" assess these questions, but this opens up a bunch of issues. How could we know how much we could tr

... (Read more)(Click to expand thread. ⌘F to Expand All)Cmd/Ctrl F to expand all comments on this post
Showing 3 of 14 replies (Click to show all)

Making long term predictions is hard. That's a fundamental problem. Having proxies can be convenient, but it's not going to tell you anything you don't already know.

2ozziegooen7d Yea, in cases like these, having intermediate metrics seems pretty essential.
2romeostevensit8d Deference networks seem underrated.

ozziegooen's Shortform

by ozziegooen 17d31st Aug 201915 comments

14