LESSWRONG
LW

Wikitags
Main
3
Intro Dialogue (Math 2)
11
LW Wiki

Solomonoff induction

Edited by Eliezer Yudkowsky, et al. last updated 17th Dec 2015

Solomonoff induction is an ideal answer to questions like "What probably comes next in the sequence 1, 1, 2, 3, 5, 8?" or "Given the last three years of visual data from this webcam, what will this robot probably see next?" or "Will the sun rise tomorrow?" Solomonoff induction requires infinite computing power, and is defined by taking every computable algorithm for giving a probability distribution over future data given past data, weighted by their , and by comparison to the actual data.

E.g., somewhere in the ideal Solomonoff distribution is an exact copy of you, right now, staring at a string of 1s and 0s and trying to predict what comes next - though this copy of you starts out with a very low weight in the mixture owing to its complexity. Since a copy of you is present in this mixture of computable predictors, we can prove a theorem about how well Solomonoff induction does compared to an exact copy of you; namely, Solomonoff induction commits only a bounded amount of error relative to you, or any other computable way of making predictions. Solomonoff induction is thus a kind of perfect or rational ideal for probabilistically predicting sequences, although it cannot be implemented in reality due to requiring infinite computing power. Still, considering Solomonoff induction can give us important insights into how non-ideal reasoning should operate in the real world.

Additional reading:

  • https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_inductive_inference
  • http://lesswrong.com/lw/dhg/an_intuitive_explanation_of_solomonoff_induction/
  • http://wiki.lesswrong.com/wiki/Solomonoff_induction
Parents:
Children:
Subscribe
3
Subscribe
3
Inductive prior
updating those weights
algorithmic simplicity
Discussion9
Discussion9
Solomonoff induction: Intro Dialogue (Math 2)
Methodology of unbounded analysis
Posts tagged Solomonoff induction
180The Solomonoff Prior is Malign
Ω
Mark Xu
5y
Ω
52
169An Intuitive Explanation of Solomonoff Induction
Alex_Altair
13y
228
144A Semitechnical Introductory Dialogue on Solomonoff Induction
Ω
Eliezer Yudkowsky
4y
Ω
33
47Open Problems Related to Solomonoff Induction
Wei Dai
13y
104
38Solomonoff induction still works if the universe is uncomputable, and its usefulness doesn't require knowing Occam's razor
Christopher King
2y
28
72When does rationality-as-search have nontrivial implications?
Ω
nostalgebraist
7y
Ω
12
56The Problem of the Criterion
Gordon Seidoh Worley
4y
63
33How is Solomonoff induction calculated in practice?
Q
Bucky, Richard_Kennaway
6y
Q
13
147K-complexity is silly; use cross-entropy instead
So8res
3y
54
145Remarks 1–18 on GPT (compressed)
Cleo Nardo
2y
35
112Pascal's Mugging: Tiny Probabilities of Vast Utilities
Eliezer Yudkowsky
18y
354
60Multiple Worlds, One Universal Wave Function
evhub
5y
76
53Computational Model: Causal Diagrams with Symmetry
Ω
johnswentworth
6y
Ω
29
51Solomonoff Cartesianism
Rob Bensinger
11y
51
47My impression of singular learning theory
Ege Erdil
2y
30
Load More (15/72)
Add Posts