faul_sname

We are excited to publicly introduce the Laboratory for Importance-sampled Measure and Bayesian Observation (LIMBO), a small research group working at the intersection of cosmological theory, probability, and existential risk. We believe that the mechanisms by which observers continue to exist in the universe are important, neglected, and tractable to...

There's a take I've seen going around, which goes approximately like this: > It used to be the case that you had to write assembly to make computers do things, but then compilers came along. Now we have optimizing compilers, and those optimizing compilers can write assembly better than pretty...

Assessments of "general" vs "spiky" capability profiles are secretly assessments of "matches existing infrastructure" vs "doesn't". Human societies contain human-shaped roles because humans were the only available workers for most of history. Packaging tasks into human-sized, human-shaped jobs was efficient. Given LLMs, the obvious thing to do is to try...

Motivation One major risk from powerful optimizers is that they can find "unexpected" solutions to the objective function, which score very well on the objective function but are not what the human designer intended. The canonical example is > Suppose your aged mother is trapped in a burning building, and...

TL;DR: does stapling an adaptation executor to a consequentialist utility maximizer result in higher utility outcomes in the general case, or is AlphaGo just weird? So I was reading the AlphaGo paper recently, as one does. I noticed that architecturally, AlphaGo has 1. A value network: "Given a board state,...

"Rewarding good performance leads to faster improvement than punishing bad performance" "In general, unusually bad performance improves after punishment, but good performance tends not to improve and sometimes even gets worse after praise is administered." These statements seem contradictory, yet both describe real effects. The apparent contradiction is caused by...