All of Jack R's Comments + Replies

Three reasons to expect long AI timelines

Won't we have AGI that is slightly less able to jump into existing human roles before we have AGI that can jump into existing human roles? (Borrowing intuitions from Christiano's Takeoff Speeds) [Edited to remove typo]

2Steven Byrnes5moSure but that would make OP's point weaker not stronger, right?
0Gerald Monroe5moJack, to be specific, we expect to have AI that can jump into specific classes of roles, and take over the entire niche. All of it. They will be narrowly superhuman at any role inside the class. If right now, strategy games, both of the board and the realtime clicking variety, had direct economic value, every human doing it would already be superfluous. We can fully solve the entire class. The reason is, succinctly: a. Every game-state can be modeled on a computer with the subsequent state resulting from a move by the AI agent provided b. The game-state can be reliably converted to a score that is an accurate assessment of what we care about - victory in the game. That is, it's usually a delayed reward, but a game-state either is winning or it is not and this mapping is reliable. For real world tasks (b) gets harder because there are subtle outcomes that can't be immediately perceived, or they are complex to model. Example: an autonomous car reaches the destination but has damaged it's own components more than the value of the ride. So it will take longer to solve the class of : robotics manipulation problems where we can reliably estimate the score resulting from a manipulation, and model reasonably accurately the full environment and the machine in that environment. This is most industrial and labor tasks on the planet in this class. But the whole class can be solved relatively quickly - once you have a general solver for part of it, the rest of it will fall. And then the next class of tasks are things where a human being is involved. Humans are complex and we can't model them in a simulator like we can model rigid bodies and other physics. I can't predict when this class will be solved.
The mathematical universe: the map that is the territory

Obviously, we wouldn’t notice the slowness from the inside, any more than the characters in a movie would notice that your DVD player is being choppy.

Do you have a causal understanding for why this is the case? I am a bit confused by it

Three reasons to expect long AI timelines

Re: 1, I think it may be important to note that adoption has gotten quicker (e.g. as visualized in Figure 1 here; linking this instead of the original source since you might find other parts of the article interesting). Does this update you, or were you already taking this into account? 

5Matthew Barnett5moWow, that chart definitely surprised me. Yes, this caused me to update.
Does the lottery ticket hypothesis suggest the scaling hypothesis?

When the network is randomly initialized, there is a sub-network that is already decent at the task.

From what I can tell, the paper doesn't demonstrate this--i.e. I don't think they ever test the performance of a sub-network with random weights (rather they test the performance of a subnetwork after training only the subnetwork). Though maybe this isn't what you meant, in which case you can ignore me :)

5DanielFilan5moYep, I agree that this question does not accurately describe the lottery ticket hypothesis.
Opinions on Interpretable Machine Learning and 70 Summaries of Recent Papers

Thanks a lot for this--I'm doing a lit. review for an interpretability project and this is definitely coming in handy :)

Random note: the paper "Are Visual Explanations Useful? A Case Study in Model-in-the-Loop Prediction" is listed twice in the master list of summarized papers.

2lifelonglearner6moThanks! Didn't realize we had a double entry, will go and edit.
Why GPT wants to mesa-optimize & how we might change this

I agree, and thanks for the reply. And I agree that even a small chance of catastrophe is not robust. Though I asked because I still care about the probability of things going badly, even if I think that probability is worryingly high. Though I see now (thanks to you!) that in this case our prior that SGD will find look-ahead is still relatively high and that belief won't change much by thinking about it more due to sensitivity to complicated details we can't easily know.

Why GPT wants to mesa-optimize & how we might change this

Anyway, the question here isn't whether lookahead will be perfectly accurate, but whether the post-lookahead distribution of next words will allow for improvement over the pre-lookahead distribution.

Can you say a bit more about why you only need look-ahead to improve performance? SGD favors better improvements over worse improvements--it feels like I could think of many programs that are improvements but which won't be found by SGD. Maybe you would say there don't seem to be any improvements that are this good and this seemingly easy for SGD to find?

2John_Maxwell7moFrom a safety standpoint, hoping and praying that SGD won't stumble across lookahead doesn't seem very robust, if lookahead represents a way to improve performance. I imagine that whether SGD stumbles across lookahead will end up depending on complicated details of the loss surface that's being traversed.
2020 LessWrong Demographics Survey

For the risk question, is it asking about positive and negative risk, or just negative risk?

1Bob Jacobs1yPositive and negative risk