Neil

Having a flair for the dramatic can be dangerous--read at your own risk.

Wiki Contributions

Comments

The best example I can recall of what you're describing is the members of La Résistance in France during WW2. These people risked their lives and the lives of their family in order to blow up German rails, smuggle out Jews and kill key Gestapo operatives. They did not consider themselves heroes, because for them this was simply the logical course of action, the only way to act for a person with a shred of common sense. Most of them are dead now but along their lives they repeated that if France considered them to be heroes (which it did) that would defeat the point: that doing what they did should not be extraordinary, but common sense. 

You're right about the epic drama thing. Poetic flare can be useful in certain situations, I imagine, although it is a fine line between using that as motivation and spoiling rationality. (Poetry, as in beauty and fun, is a terminal goal of humanity so I would also advise against ignoring it entirely.) 

Hello! 

The universe is so complex that it is exceedingly unlikely that you will get it right on the first try--and in order to err and err and err again you can't just stick to one theory and latch on to it through fire and snow no matter what befalls you. While confirmation probably makes sense at the local level, given the limited time and resources that you mentioned, it doesn't make sense on greater scales where the stakes are so high that it's worth it being absolutely right rather than being efficient in finding an interim solution. The issues commonly discussed on LessWrong tend to be of the latter category, and so confirmation bias is indeed a bias here. 

Orwell's original title for 1984 was The last man in Europe, by which he meant that Winston, the hero of the novel, was the last sane man left in the entire continent. I would argue that because literally everyone else around him was insane and was essentially drowning in the water, he was a hero for swimming. The amount of people working on alignment in the world is far below 1% of the general population--I know it's a romanticized qualification, which is kind of the point here, but this runs under my definition of "hero". 

I mean what even is your definition of hero? 

Interesting question. The sous-entendu might be "how much of this post was written for you" and the answer would be "probably a lot". I don't think I have the mind or time or stamina to work on the front lines, and so so far my most concrete plan is writing a few more LessWrong posts based on various helpful-seeming ideas. This post outlined a few options I and others in the same position as me have. Do you have any more ideas?

The point of the post was to not lionize them over everyone else. The target audience I had in mind (which may not even exist at this point) was people who wanted to become alignment researchers because that's where the front lines are. My point is that that may not be the best idea in some cases. At the end of the day, if we solve the alignment problem it will be directly thanks to those researchers, that's what I mean. 

As for the politics thing that's interesting, I hadn't thought of it horribly backfiring in that way. I mean the goal would be to explain to them why alignment is necessary, which shouldn't be an impossible task. There's a lot of legal and economic power coming from the government, so just ignoring that actor seems like a mistake. 

Thanks for the feedback!

The passage on "you are responsible for the entire destiny of the universe" was mostly addressing the way it seems many EAs feel about the nature of responsibility. We indeed have limited agency in the world but people around here tend to feel they are personally responsible for literally saving the world alone. The idea was not to frontally deny that or to run against heroic responsibility but rather to say that while the responsibility won't go away, there's no point in becoming consumed by it. You are a less effective tool if you are too heavily burdened by responsibility to function properly. I wrote it that way because I'm hoping the harsh and utilitarian tone will reach the target audience better than something more clichèd would. There's enough romanticization as it is here.

I definitely romanticized the alignment researchers being heroes part. I'll add a disclaimer to mention that the choice of words was meant to paint the specific approach, the specific picture that up-and-coming alignment researchers might have when they arrive here. 

As for which narrative to follow, this one might be as good as any. As the mental health post I referenced here mentioned, the "dying with dignity" approach Eliezer is following might not sit well with a number of people even when it is in line with his own predictions. I'm not sure to what degree what I described is a fantasy. In a universe where alignment is solved, would this picture be inacurate? 

Thanks for the feedback!

All the obvious alternate routes to participating to the alignment problem seem to have been mentioned here--are there any more I should write down? I'm aware this is a flawed post and would like to make it more complete as time goes.

Well there's always value in cramming old ideas into a small amount of words. 

You're right that I should have aimed for something more interesting and novel, but I'm still experimenting with LessWrong and went with this for now. Thanks for the comment, I'll keep this in mind for next time. 

Haha I don't know what this post did to deserve -7 Karma, but if somebody could explain I'd be really grateful. Since there is no "I disagree with the contents" button on regular posts apparently, does this mean that I should assume the dislikes are from people who disagree with me? Or is my logic fundamentally flawed and breaks a few rules of rationality? Criticism would be great even if just a few lines to explain. Thanks!

We can't negociate with something smarter than us 

Superintelligence will outsmart us or it isn't superintelligence. As such, the kind of AI that would truly pose a threat to us is also an AI we cannot negotiate with.

No matter what arguments we make, superintelligence will have figured them out first. We're like ants trying to appeal to a human, and the human can understand pheromones but we can't understand human language. It's entirely up to the human and its own arguments whether we get squashed or not. 

Worth reminding yourself of this from time to time, even if it's obvious. 

Counterpoints: 

  1. It may not take a true superintelligence to kill us all, meaning we could perhaps negociate with a pre-AGI machine
  2. The "we cannot negociate" part is not taking into account the fact that we are the Simulators and thus technically have ultimate power over it
Load More