This Sunday, two or three speakers from the newly formed Quantified Uncertainty Research Institute (focused on epistemics and forecasting) will present on the general topic of "evaluations" and seek feedback on their work from attendees.
An "evaluation" here means any assessment of some thing for some property. We're talking scientific review of articles, product ratings, or even measurements like temperature.
First part of the meetup will be in Zoom (link) where speakers will talk and take an initial Q&A. Following that, there will be freeform discussion and socialization in the LessWrong Walled Garden (link). The first segment usually goes for an hour, and the second goes on for as long as people feel like staying (usually another hour or two).
Ozzie Gooen: Evaluate All The Things!
Ozzie Gooen will discuss how future technologies might enable cheap and scalable "evaluations" such as scientific reviews and product ratings.
Blurb from Ozzie:
New forecasting and AI technologies might make systematized evaluations really cheap and scalable. This could be great if we come up with pragmatic and effective evaluations, and terrible if we implement poor ones. Think of article or scientific reviews, product ratings, or more creative things like independent evaluations of the reputability of every sentence in an argument. A few of us have started early investigation and experimentation in the area, starting with EA research and project evaluations. Ozzie will present on progress and is curious to get thoughts.
Elizabeth: Case Studies in Why Evaluating Things Is Hard
Elizabeth tries to look for some simple examples of successful evaluations and finds it harder than expected.
"Nuño: Evaluating the Value of Small Research Projects
Nuño will talk about evaluating the value of small research projects, and about combining evaluations and forecasts to choose which projects to carry out. Some challenges are finding informative proxies of impact, comparing the value of projects in different areas, and finding useful base rates.
Posted on: