LESSWRONG
LW

Machine Learning (ML)Probability & StatisticsHas DiagramFormal ProofWorld Modeling
Frontpage

19

An Illustrated Proof of the No Free Lunch Theorem

by [anonymous]
8th Jun 2020
1 min read
0

19

This is a linkpost for https://mlu.red/muse/52449366310.html
Machine Learning (ML)Probability & StatisticsHas DiagramFormal ProofWorld Modeling
Frontpage

19

New Comment
Moderation Log
Curated and popular this week
0Comments

This is a proof of one of the many No Free Lunch Theorems in statistical learning. Rafael Harth already did a very good writeup here of the same proof. I had a vision for doing some very nice diagrams which could help bridge the many summations and inequalities in this proof. I think I maybe only executed up to ~20% of that vision.

I have a good amount of experience making graphics and visuals, and this was still challenging. I think, until I find a better method, that I'll probably not make more of these. I felt a combination of:

  • difficulty using my tool (Inkscape) to get the "real" result in my head
  • frustration that the similarities between the images couldn't be abstracted out in some nice tool

For these sorts of graphs, I feel like Mermaid is maybe half-way there. But I feel like there's a lot of value to finding the specific visuals that you as the author think are good to help convey the intuition, so I'm a little hesitant to just use a compositional tool...