Outside of the interpretability research space, do you know of other interesting examples of different techniques being graded on different curves?
Electric vehicles? Early electric vehicles were worse than gas cars on all axis other than the theoretical promise of the technology. However, they were (and still are, ie formula E) graded on separate curves. The fairly straight-forward analogy I'm trying to make is that maybe it's worthwhile treating early technologies gently, as now I think most people are pretty impressed by electric cars.
Electric vehicles? Early electric vehicles were worse than gas cars on all axis other than the theoretical promise of the technology. However, they were (and still are, ie formula E) graded on separate curves. The fairly straight-forward analogy I'm trying to make is that maybe it's worthwhile treating early technologies gently, as now I think most people are pretty impressed by electric cars.
Although obvi... (read more)