Thank you! I'd be glad to include this and any other corrections in an edit once contest results are released. Are there any other errors which catch your eye?

I would honestly be interested to see a detailed writeup with good examples of this "maybe amazing" vs "probably good" distinction.

A subtlety here is that the traits that make a candidate a potential outlier are often very different from the traits that would make them “pretty good,” so improving your filtering process to produce more “pretty good” candidates won’t necessarily increase the rate of finding outliers, and might even decrease it.

Most important point I'd still want to grok is what this "might even decrease it" looks like. What are industry examples of metrics or filtering processes that can differentiate 95th percentile samples from 99.9th percentile samples? And what are some of the qualitative shifts you see between them? I suspect the art of identifying 99.9th percentile samples goes beyond looking for "really, really good" 95th-percentile-ish things.