LESSWRONG
LW

Personal Blog

2

How do you learn Solomonoff Induction?

by aisarka
17th May 2016
1 min read
6

2

Personal Blog

2

How do you learn Solomonoff Induction?
15Anders_H
9Manfred
6bogus
5ike
0Houshalter
3Daniel_Burfoot
New Comment
6 comments, sorted by
top scoring
Click to highlight new comments since: Today at 10:04 AM
[-]Anders_H9y150

Solomonoff Induction is uncomputable, and implementing it will not be possible even in principle. It should be understood as an ideal which you should try to approximate, rather than something you can ever implement.

Solomonoff Induction is just bayesian epistemology with a prior determined by information theoretic complexity. As an imperfect agent trying to approximate it, you will get most of your value from simply grokking Bayesian epistemology. After you've done that, you may want to spend some time thinking about the philosophy of science of setting priors based on information theoretic complexity.

Reply
[-]Manfred9y90

The classic textbook is Li and Vitanyi's An Introduction to Kolmogorov Complexity and Its Applications.

Reply
[-]bogus9y60

Solomonoff induction is uncomputable, thus, as a direct consequence, it cannot be learned. Some approximations to it which are of practical interest: Occam learning and probably approximately correct learning. As a general matter, these questions are addressed by computational learning theory.

Reply
[-]ike9y50

https://wiki.lesswrong.com/wiki/Solomonoff_induction http://lesswrong.com/lw/dhg/an_intuitive_explanation_of_solomonoff_induction/ should get you started.

Reply
[-]Houshalter9y00

Also Yudkowsky's article on Occam's Razor describes the Occam's razor/ simplicity prior OP was interested in.

Reply
[-]Daniel_Burfoot9y30

convoluted tome

My book describes a philosophy of science based on large scale lossless data compression. It is not going to give you a toolbox for using SI; as others have observed, SI is of primarily theoretical importance, since it can't be computed. However, different aspects of the book might help expand your worldview in this area.

Reply
Moderation Log
More from aisarka
View more
Curated and popular this week
6Comments

I read about a fascinating technique described on Wikipedia as a mathematically formalized combination of Occam's razor and the Principle of Multiple Explanations. I want to add this to my toolbox. I'm dreaming of a concise set of actionable instructions for using Solomonoff induction. I realize this wish might be overly idealistic. I'm willing to peruse a much more convoluted tome and will consider making time for any background knowledge or prerequisites involved.

If anyone knows of a good book on this, or can tell me what set of information I need to acquire, please let me know. It would be much appreciated!