there are definitely status ladders within openai, and people definitely care about it. status has this funny tendency where once you've "made it" you realize that you have merely attained table stakes for another status game waiting to be played. 

this matters because if being at openai counts as having "made it", then you'd predict that people will stop value drifting once they are already inside and feel secure that they won't be fired, or could easily find a new job if they did. but, in fact, i observe lots of value drift in people after they join the labs just because they are seeking status within the lab.

in ML, or at least in the labs, people don't really care that much about Nature. my strongest positive association with Nature is AlphaGo, and I honestly didn't know anyone in ML other than DeepMind published much in Nature (and even there, I had the sense that DeepMind had some kind of backchannel at nature.)

people at openai care firstly about what cool things you've done at openai, and then secondly about what cool things you've published in general (and of course zerothly, how other people they respect perceive you). but people don't really care if it's in neurips or just on arxiv, they only care about whether the paper is cool. people mostly think of publishing things as a hassle, and reviewer quality as low.

i don't think this metaphor makes things less confusing. i think the best way to understand this is just to directly understand the scaling laws. the chinchilla paper is an excellent reference (its main contribution is to fix a bug in the kaplan paper essentially; people have known since time immemorial that obviously amount of data matters in addition to model size).

(N is parameter count, D is dataset size, L is loss)

for a fixed dataset, each model size has a best achievable loss with infinite data. if you train any longer, there is no capacity left. for each data size, there is also a best achievable loss with infinite model size. for a given training compute expenditure, there is an optimal tradeoff between training data and model size that achieves the best loss.

I think motivated reasoning is mostly bad, but there is some value to having some regularization towards consistency with past decisions. for example, there are often two almost-equally good choices you can make, but you need to commit to one instead of indefinitely waffling between the two, which is way worse than either option. having some delusional confidence via motivated reasoning can help you commit to one of the options and see it through. I've personally found that my unwillingness to accept motivated reasoning also has the side effect that I spend a lot more time in decision paralysis.

for me personally, it's slightly discomforting to use goods that are produced inefficiently with human labor instead of automation. for example, I'd actively prefer to use a mass-produced bowl than a handmade one, because the thought of unnecessarily consuming human labor to produce something that is not any better than the mass-produced one feels wrong to me.

why do we need to solve the philosophical dilemmas before the singularity? presumably you only need sufficient philosophical maturity to end the critical risk period, not kill everyone in the process, and maintain sufficient cosmopolitanism to give humanity room to figure out the answers to all the philosophical dilemmas eventually.

In my experience of programming, the really hard part was figuring out which packages weren’t installed or weren’t updated or were in the wrong folder, causing the test we’d done in class to completely fail to work in the same way on my own computer. The next really hard part was Googling everything the debugger spat out to find an explanation of how to make it go away.

this is only sometimes true in my work. i find there are two main kinds of work that i encounter:

• greenfield development - i own the entire codebase and build it from scratch. i spend 90% of my time thinking about what i actually want the computer to do.
• working in a large existing codebase - i spend a lot of time understanding what someone else told the computer to do, and then spend a bunch of time fixing things that broke

for the former, i spend very little time dealing with this kind of stupid bug thing. for the latter, there is a much larger amount, but it is often not solvable via the "just google/slack-search for the error and whack it until it works" loop. often i need to deeply understand the code to debug it, and the reason it wasn't working was because of some kind of misunderstanding of what the computer was being told to do on my part. this also varies a lot by specific codebase; the worst codebases i have dealt with were impossible to understand and only fixable by whacking it.

my guess is i do a lot more of the first thing than the average lab employee

imo, the new mealsquares have taste and mouthfeel very similar to many brands of protein bar, whereas the old mealsquares had a unique taste and mouthfeel

i miss the og mealsquares