DaemonicSigil

Here is a question worth asking at least once: Why can't we just solve alignment by doing RL where the reward is exactly equal to our own utility function? Now, there are some implementation concerns here. For example, we don't actually know our own utility function. And even if we...

Advice String Distillation This post will advocate for a fine-tuning methodology that I think is currently extremely under-rated for alignment and interpretability. It uses Context Distillation, but I don't think it has a proper name of its own yet, so I will refer to it as "Advice String Distillation" here....

The best language models are still getting smarter and more capable. To an increasing degree, this is because they are trained by Reinforcement Learning with Verifiable Rewards. Chain of thought reasoning allows models to evade the finite depth restriction on information flow by passing (relatively little) information back into the...

If Bayesian stats is so great, it should be able to do everything that ordinary stats can do. Indeed, as a fan of Bayesian stats, I have not made much of an attempt to learn normal stats with its p-values and t-tests outside what was required for my highschool statistics...

Logical Share Splitting I previously wrote a post based on the following law in probability theory: This law relates the probabilities of given individual statements to the probabilities of logical combinations of them. Prediction market prices should approximate probabilities, and by combining this with the above equation, we produce a...

An ideal machine learning model trained on an independent identically distributed dataset would end up making the same predictions after seeing all data, no matter what order it updated on the data. This is not true of neural nets trained by gradient descent. To first order in the learning rate,...

Airships are pretty dang cool. Airplanes need a continuous expenditure of energy to stay in the air, but if you just fill a bag with a light gas, you can stay up in the air with no energy expenditure at all. The two lightest gases are hydrogen and helium. Though...