I have learned that AI safety is a big issue for 5 years now. Why do I only do anything now? There are a bunch of factors, but I think one really important factor was me finding other people making mistakes. Because this happened recently, the version of me before this, I can still actually recall.
I noticed that very few people were doing anything at all about AI safety, at least in the Netherlands. Let's say before the summer, there were still little projects going on, with everybody having their own pet technical projects mainly. I thought those were wrong, but that what I mainly thought was wrong was the underlying world model that people had. I think that the main mistake was their expectations of the future, and in a way, that makes my actions not as comparable to other people's, kind of like someone looking at a swimmer and thinking, "But you can go way faster by running."
I think that right now, virtually everyone needs to be working on getting the pause treaty in the air as fast as possible. A main thing that happened to me this summer is that I encountered people who were actually having the same instrumental goals as I had.
They were actually doing things. I wasn't, but I was looking at the things they were doing, and I thought they were doing it wrong. Now I am a runner looking at a different runner using bad shoes, and when they didn't seem to be listening when I advised them on better shoe choice, I now feel really motivated to put on better shoes and run past them (and hope they put on the better shoes as a result of that).
More anekdotal evidence: if I'm looking at the things I was doing before, I was just sitting behind my computer writing programs, because whenever I sit behind my computer, I just notice the ways my computer is wrong. I have quite some programming skills now, and I think I am low-key addicted to just fixing mistakes.
I have spent, I think, literal years just fixing, writing better programs than the programs that had flaws in them, and then subsequently fixing the flaws that exist in my programs, time and time again. It's kind of so bad that sometimes I just avoid opening my computer because I am afraid that I will be sucked into fixing the mistakes I see. Maybe I am an outlier of a person, but I think mistakes are so extremely activating.
Another anekdote: I don't have much inspiration for writing things at all, but after I talk to people and notice the mistakes in their reasoning, I attach to that like a male anglerfish. There is a mistake in there, and if that conversation ends before they were convinced of my point, those are the moments in my life I feel most motivated to write a LessWrong post, though it never really quite often got me to actually write posts here.
Fear of accidentally making it worse
I think, in a way, people are really afraid of accidental self-pessimization (achieving the opposite of what you set out to achieve). We know what will happen when we do nothing, but even if we have a very strong desire for something, I think we are just so self-conscious about accidentally making things even worse. And I also feel like this is mainly how antagonistic criticism shapes. I think on some gut level we know that we work this way. I think your worst antagonists will try and figure out what you want to achieve and then subsequently argue as if it's not just that you're not achieving what you're trying to achieve, but that you are anti-achieving what you want to achieve.
Though I also think that humans in general notice, people just will very often and casually go into this you're-actually-making-it-worse line of arguing, also when they don't genuinely believe it. Mainly this is the easy button that gets people scared to actually do anything, and the button to make people just fall back into normalcy again.
I'm coming at all of this from an approach, a belief that humans are mostly rational, and that, if some of our instincts come from evolution, evolution mostly did things that kind of sorta maybe work. I think that the fear of accidental self-pessimization being quite totalizing, that's not an unreasonable thing to put into an agent.
The Hero License is the confidence you'll do better than nothing.
I am calling this out because I notice a shared frustration from most people working on AI safety: nobody is doing anything except for a small, tiny group of people. I think the answer here is: we need to convince these people that their acts will not be accidentally self-pessimizing. Oh, I used the word "convince." That's not quite correct, because we don't want anyone working with us accidentally self-pessimizing. That hurts trust. We want to help people with their absolutely necessary investigation into whether them helping does not accidentally pessimize whatever they want from life.
I think, in practice, that we want to somehow show people, ideally in a way that we cannot fake, what will happen when they are not helping. You can convince me that I will not be pessimizing anything when you show me a bad program, because I will definitely not do worse than this. I think that these kinds of demonstrations are to be tailored to what this other person is good at recognizing and good at understanding, and then show them our status quo and tell them (or have their trust in us) that someone doing this well will actually be helpful. (Though if they are an expert, they will probably already have a sound intuition.)
Mass minting
I think, on top of that, also, you yourself being just generally very transparent about what you are doing and how that is going is just going to actually accidentally motivate error-correcting people very strongly. In most of the settings, this motivation is going to feel unpleasant to the instigator because in most traditional settings, such as employment (where you are rivaling your peers for a spot in a company) and companies (who are rivaling other companies for the same set of customers), in these adversarial settings, getting others to see your flaws and subsequently being highly motivated to fix them, actually risks you being dethroned and without a job or without a company.
In response to centuries of this status quo, we have a lot of cultural habits to prevent this error-correcting motivation from triggering in order to avoid someone becoming motivated to take our job. I think this looks like, for example, a lot of the obscurity, privacy and secrecy we think are normal and makes us feel naked[1] otherwise.
But now, when I speak to a leader, they tell me: "if there is someone better than me at my job, I want them to take it from me". Us being in a situation where, in fact, we all have the same goal. We are in a fully cooperative game. It changes our desire for entrenchment and I don't think we've thought through the consequences of that yet.
At the same time, I won't deny feeling chipper about me being the one to discover this. That's also kind of cool, but in my emotional state, I do think the existential issue is weighing so much more and so much heavier[2].
Well, back to hero licensing. I do truly think that just putting the errors in front of people's faces is just what we'll get as heroes for free.
I think that, in a way, this is the perfect time to have imposter syndrome, maybe I, maybe not. Sorry, maybe I'm not thinking about this well, but the answer right now, today, really genuinely is: if you think you might be doing it wrong, just show everyone, tell everyone, and get everyone to look at your stuff.
And for everyone who is looking at this person's stuff: It's a great time to be very frank about whether there are mistakes in there. It's a great time to take stuff from their hands and go, "Okay, you know what, I'll do this, actually."
If you are talking to someone who is doing this to prevent extinction, then it might feel a little bit weird in this new status quo. But once they think about it for a few seconds, they'll endorse you taking over.
And on a meta note, I am also way more comfortable writing this post right here[3] because I might activate someone to actually write a good post. It's okay to comment on this and say how horrendous of a post this is. I don't think I'll rapidly improve as a writer in the short term. I'm also not sure that that is my calling in this timeline, but if it is, then I will become a good writer. If it's not, then, yeah.
If none of you commenters are going to just write the better post, then at least I'll find someone who will be so extremely triggered about my post, and then I'll shove it under their nose. I'll also show them my task lists and then have them stare into my eyes for 30 seconds when I say, "I am not going to change anything about this post. I think these tasks on this task list are more important. If you are not going to do this, this post is just going to sit there until the end of time and be the best that this timeline had at conveying this idea.
Because, as you can see, these commenters have not done anything, even though it's now 2 weeks since the post's creation, so it's you or this piece, Ms. I-think-I'm-not-that-great-of-a-writer-but-at-least-not-as-attrocious-as-Jochem."
I think it's neat that, if I am actually capable and actually able to do something, I might get to work with interesting people. That sounds okay. That sounds neat, but I would trade that for a boring and normal life that ends in cryonic suspension in a heartbeat. Because I never forget that this extinction stuff is not just me who's going to die. It is everything I find beautiful about this Earth. It is everyone I love about this Earth. If it was just me, I might choose that and be fine with it, but I have people I dearly love in this world. I'm not going to risk their well-being for the chance to be seen as cool by some interesting strangers. My life is way too beautiful for that.
which is bad. I'm not a good writer. I'm literally just dictating into a phone because otherwise, I think and obsess about the minutia for literal weeks if nothing interrupts me, and finally conclude that, well, I shouldn't be writing this in the first place.
I mean, the writing process is also a discovering process, right? The longer my writing process is, the more I've changed as a person by the end of it, such that by the time I see no mistakes in the post anymore, I will just think the post irrelevant and I should do something else
I have learned that AI safety is a big issue for 5 years now. Why do I only do anything now? There are a bunch of factors, but I think one really important factor was me finding other people making mistakes. Because this happened recently, the version of me before this, I can still actually recall.
I noticed that very few people were doing anything at all about AI safety, at least in the Netherlands. Let's say before the summer, there were still little projects going on, with everybody having their own pet technical projects mainly. I thought those were wrong, but that what I mainly thought was wrong was the underlying world model that people had. I think that the main mistake was their expectations of the future, and in a way, that makes my actions not as comparable to other people's, kind of like someone looking at a swimmer and thinking, "But you can go way faster by running."
I think that right now, virtually everyone needs to be working on getting the pause treaty in the air as fast as possible. A main thing that happened to me this summer is that I encountered people who were actually having the same instrumental goals as I had.
They were actually doing things. I wasn't, but I was looking at the things they were doing, and I thought they were doing it wrong. Now I am a runner looking at a different runner using bad shoes, and when they didn't seem to be listening when I advised them on better shoe choice, I now feel really motivated to put on better shoes and run past them (and hope they put on the better shoes as a result of that).
More anekdotal evidence: if I'm looking at the things I was doing before, I was just sitting behind my computer writing programs, because whenever I sit behind my computer, I just notice the ways my computer is wrong. I have quite some programming skills now, and I think I am low-key addicted to just fixing mistakes. I have spent, I think, literal years just fixing, writing better programs than the programs that had flaws in them, and then subsequently fixing the flaws that exist in my programs, time and time again. It's kind of so bad that sometimes I just avoid opening my computer because I am afraid that I will be sucked into fixing the mistakes I see. Maybe I am an outlier of a person, but I think mistakes are so extremely activating.
Another anekdote: I don't have much inspiration for writing things at all, but after I talk to people and notice the mistakes in their reasoning, I attach to that like a male anglerfish. There is a mistake in there, and if that conversation ends before they were convinced of my point, those are the moments in my life I feel most motivated to write a LessWrong post, though it never really quite often got me to actually write posts here.
Fear of accidentally making it worse
I think, in a way, people are really afraid of accidental self-pessimization (achieving the opposite of what you set out to achieve). We know what will happen when we do nothing, but even if we have a very strong desire for something, I think we are just so self-conscious about accidentally making things even worse. And I also feel like this is mainly how antagonistic criticism shapes. I think on some gut level we know that we work this way. I think your worst antagonists will try and figure out what you want to achieve and then subsequently argue as if it's not just that you're not achieving what you're trying to achieve, but that you are anti-achieving what you want to achieve.
Though I also think that humans in general notice, people just will very often and casually go into this you're-actually-making-it-worse line of arguing, also when they don't genuinely believe it. Mainly this is the easy button that gets people scared to actually do anything, and the button to make people just fall back into normalcy again.
I'm coming at all of this from an approach, a belief that humans are mostly rational, and that, if some of our instincts come from evolution, evolution mostly did things that kind of sorta maybe work. I think that the fear of accidental self-pessimization being quite totalizing, that's not an unreasonable thing to put into an agent.
The Hero License is the confidence you'll do better than nothing.
I am calling this out because I notice a shared frustration from most people working on AI safety: nobody is doing anything except for a small, tiny group of people. I think the answer here is: we need to convince these people that their acts will not be accidentally self-pessimizing. Oh, I used the word "convince." That's not quite correct, because we don't want anyone working with us accidentally self-pessimizing. That hurts trust. We want to help people with their absolutely necessary investigation into whether them helping does not accidentally pessimize whatever they want from life.
I think, in practice, that we want to somehow show people, ideally in a way that we cannot fake, what will happen when they are not helping. You can convince me that I will not be pessimizing anything when you show me a bad program, because I will definitely not do worse than this. I think that these kinds of demonstrations are to be tailored to what this other person is good at recognizing and good at understanding, and then show them our status quo and tell them (or have their trust in us) that someone doing this well will actually be helpful. (Though if they are an expert, they will probably already have a sound intuition.)
Mass minting
I think, on top of that, also, you yourself being just generally very transparent about what you are doing and how that is going is just going to actually accidentally motivate error-correcting people very strongly. In most of the settings, this motivation is going to feel unpleasant to the instigator because in most traditional settings, such as employment (where you are rivaling your peers for a spot in a company) and companies (who are rivaling other companies for the same set of customers), in these adversarial settings, getting others to see your flaws and subsequently being highly motivated to fix them, actually risks you being dethroned and without a job or without a company.
In response to centuries of this status quo, we have a lot of cultural habits to prevent this error-correcting motivation from triggering in order to avoid someone becoming motivated to take our job. I think this looks like, for example, a lot of the obscurity, privacy and secrecy we think are normal and makes us feel naked[1] otherwise.
But now, when I speak to a leader, they tell me: "if there is someone better than me at my job, I want them to take it from me". Us being in a situation where, in fact, we all have the same goal. We are in a fully cooperative game. It changes our desire for entrenchment and I don't think we've thought through the consequences of that yet.
At the same time, I won't deny feeling chipper about me being the one to discover this. That's also kind of cool, but in my emotional state, I do think the existential issue is weighing so much more and so much heavier[2].
Well, back to hero licensing. I do truly think that just putting the errors in front of people's faces is just what we'll get as heroes for free. I think that, in a way, this is the perfect time to have imposter syndrome, maybe I, maybe not. Sorry, maybe I'm not thinking about this well, but the answer right now, today, really genuinely is: if you think you might be doing it wrong, just show everyone, tell everyone, and get everyone to look at your stuff.
And for everyone who is looking at this person's stuff: It's a great time to be very frank about whether there are mistakes in there. It's a great time to take stuff from their hands and go, "Okay, you know what, I'll do this, actually." If you are talking to someone who is doing this to prevent extinction, then it might feel a little bit weird in this new status quo. But once they think about it for a few seconds, they'll endorse you taking over.
And on a meta note, I am also way more comfortable writing this post right here[3] because I might activate someone to actually write a good post. It's okay to comment on this and say how horrendous of a post this is. I don't think I'll rapidly improve as a writer in the short term. I'm also not sure that that is my calling in this timeline, but if it is, then I will become a good writer. If it's not, then, yeah. If none of you commenters are going to just write the better post, then at least I'll find someone who will be so extremely triggered about my post, and then I'll shove it under their nose. I'll also show them my task lists and then have them stare into my eyes for 30 seconds when I say, "I am not going to change anything about this post. I think these tasks on this task list are more important. If you are not going to do this, this post is just going to sit there until the end of time and be the best that this timeline had at conveying this idea. Because, as you can see, these commenters have not done anything, even though it's now 2 weeks since the post's creation, so it's you or this piece, Ms. I-think-I'm-not-that-great-of-a-writer-but-at-least-not-as-attrocious-as-Jochem."
...is shame of nudity the result of this same dynamic?
I think it's neat that, if I am actually capable and actually able to do something, I might get to work with interesting people. That sounds okay. That sounds neat, but I would trade that for a boring and normal life that ends in cryonic suspension in a heartbeat. Because I never forget that this extinction stuff is not just me who's going to die. It is everything I find beautiful about this Earth. It is everyone I love about this Earth. If it was just me, I might choose that and be fine with it, but I have people I dearly love in this world. I'm not going to risk their well-being for the chance to be seen as cool by some interesting strangers. My life is way too beautiful for that.
which is bad. I'm not a good writer. I'm literally just dictating into a phone because otherwise, I think and obsess about the minutia for literal weeks if nothing interrupts me, and finally conclude that, well, I shouldn't be writing this in the first place. I mean, the writing process is also a discovering process, right? The longer my writing process is, the more I've changed as a person by the end of it, such that by the time I see no mistakes in the post anymore, I will just think the post irrelevant and I should do something else