magfrump

Acknowledgments: Thanks to Aditya Adiga for leading this project and trusting his ideas to me. Thanks to Matt Farr for comments on this draft. Thanks to Kuil Schoneveld for organizing the project. And thanks to the several friends who tested the MFC. This work was done as part of Groundless’...

A neural net using rectified linear unit activation functions of any size is unable to approximate the function sin(x) outside a compact interval. I am reasonably confident that I can prove that any NN with ReLU activation approximates a piecewise linear function. I believe the number of linear pieces that...

This post is a response to the recent Astral Codex Ten post, “CHAI, Assistance Games, And Fully-Updated Deference”. A brief summary of the context, for any readers who are not subscribed to ACX or familiar with the shutdown problem: The Center for Human-Compatible Artificial Intelligence (CHAI) is a research group...

cross-posted from my blog Background on the events I have been thinking about this since the firing of Dr. Timnit Gebru, and yet still no one has actually written about it beyond my own tweets, so I guess it falls to me. I find, and I imagine many people in...

cross-posted from my personal blog I have an intuition that the gap between short term ML alignment problems and long term AGI alignment problems is smaller than usual. Writing on the subject I usually see rarely takes this perspective. I see a few things actually get written: * People who...

One of the dominant paradigms in image processing today is transfer learning from well established image classification algorithms (such as DenseNet, VGG, or Inception) to other image processing applications. The idea is that in the process of learning to classify types of images, the networks learn to recognize distinctive visual...

Maybe deep learning researchers are just like mathematicians. They think of coding as the hard part of what they do so they spend a lot of time teaching people how to implement neural nets from scratch. But actually the hard part of what they do is math (or at least...