But this is impossible by definition The title may seem like a contradiction. How can you differentiate something that’s not even continuous? The usual definition of the derivative of a function f at a point a is given by the limit: f′(a):=limε→0f(a+ε)−f(a)ε If f is differentiable at a, then it...
The 2 Aspects There’s 2 Aspects to things in general. I will call them Mapping Out and Mapping In, in titlecase so you know they’re distinct concepts. warmup: 0 -> 1 Here, 0 is an initial object and 1 is a terminal object. 0 is Mapped Out of because it’s...
Blaise Pascal said that. When I heard it, this interpretation instantly came. Here's an AR companion video where I point at some mountains in the Apple Vision Pro to explain the big idea. Whether or not the universe is actually infinite, it's real big. So modeling it as hyperfinite is...
(Old notes from 2016 that I stumbled upon). * Train on everything, finetune if needed. Big enough and tuning seems pointless too. It’s not like humans need much tuning for everything. Why do we train and then throw away the network each time to start anew? * Why have separate...
I just finished a 1 month research program called REMIX (Redwood Research Mechanistic Interpretability Experiment). I learned that sometimes, life can just be an order of magnitude better, and sometimes it really can be that simple.
For a finite set, one can describe a uniform distribution. There isn't a natural way to do so for a countable set. But for a hyperfinite set, one can describe a uniform distribution through a probability density. So in some ways the countable is "bigger".
What is an action? Well, they do things. What does that mean? It means something starts out some way and changes. Old states become new ones. Given a transition map T: a x s -> s, we can rewrite it as a -> (s -> s). This means actions can...