I am a bit puzzled why you insist on removing the accuracy floor as a more "correct" approach. The scoring formula is arbitrary; crucially, it tanks the in-distribution predictive power of the hypothesis.
It would be awesome to get the results of each proposed amendment (probe quality and accuracy calculation) separately. It'd be even greater if you shared the repo with your results and especially the filtering results for the probes dataset, because people have to rely only on the original flawed version right now. Please forgive me if I overlooked the link to your repo in the post.
Amazing work!
I am a bit puzzled why you insist on removing the accuracy floor as a more "correct" approach. The scoring formula is arbitrary; crucially, it tanks the in-distribution predictive power of the hypothesis.
It would be awesome to get the results of each proposed amendment (probe quality and accuracy calculation) separately. It'd be even greater if you shared the repo with your results and especially the filtering results for the probes dataset, because people have to rely only on the original flawed version right now. Please forgive me if I overlooked the link to your repo in the post.