[ Question ]

What should an Einstein-like figure in Machine Learning do?

by Razied1 min read5th Aug 20203 comments


AI RiskAIWorld Optimization

Suppose you are an extremely talented but still unknown machine learning researcher (like Einstein of patent-clerk days), and you discover something very important, maybe a few tricks that allow you to train neural nets 4 orders of magnitude faster than anyone else. You can train something of GPT-3 level on your personal rig in a few weeks, those tricks also generalize to larger networks. What should you do with this information to benefit humanity?

Publishing seems out of the question, as it would probably mean that AGI is within reach of a large project. But you personally cannot yet produce it, you can produce GPT-3 like performance, but nothing more. You could do nothing at all, keeping silent and hoping that your discoveries are not replicated by the main players. Or maybe you could try to covertly help OpenAI, which would then have a massive advantage, and enough lead time to properly treat safety. What would you do in this scenario?

New Answer
Ask Related Question
New Comment

1 Answers

This is not a hypothetical to me. I currently have, running on my computer, a few tricks that allow me to train small data systems much faster than anyone else. The first thing I did was figure out if they could bring an AGI within reach of a large project. Is it? I don't know. AGI faces bigger obstacles than a mere 4 orders of magnitude of compute power.

The second thing I did was chat with machine learning engineers from the big tech companies. They are not using these tricks and are not interested in adopting them. This will not surprise the Lisp hackers reading this post.

There might be a hedge fund out there that can use my tech. But a cursory investigation suggests that the majority of hedge funds are not doing anything sophisticated enough to require it.

So now I'm using my ML system to train an IMU-based gesture detection algorithm to track how much people eat in order to help prevent obesity-related diseases.

Can you locally replicate GPT? For example, can GPT-you compress WebText better than GPT-2?

0lsusr9moI am not yet prepared to release the details of what the system can and cannot do.