All of handoflixue's Comments + Replies

There are probably not many civilizations that wait until 2022 to make this list, and yet survive.

 

I don't think making this list in 1980 would have been meaningful. How do you offer any sort of coherent, detailed plan for dealing with something when all you have is toy examples like Eliza? 

We didn't even have the concept of machine learning back then - everything computers did in 1980 was relatively easily understood by humans, in a very basic step-by-step way. Making a 1980s computer "safe" is a trivial task, because we hadn't yet developed any... (read more)

I think most worlds that successfully navigate AGI risk have properties like:

  • AI results aren't published publicly, going back to more or less the field's origin.
  • The research community deliberately steers toward relatively alignable approaches to AI, which includes steering away from approaches that look like 'giant opaque deep nets'.
    • This means that you need to figure out what makes an approach 'alignable' earlier, which suggests much more research on getting de-confused regarding alignable cognition.
      • Many such de-confusions will require a lot of software ex
... (read more)
6Vaniver6mo
I mean, I think many of the computing pioneers 'basically saw' AI risk. I noted some surprise that IJ Good didn't write the precursor to this list in 1980, and apparently Wikipedia claims there was an unpublished statement in 1998 [https://en.wikipedia.org/wiki/I._J._Good#Research_and_publications] about AI x-risk; it'd be interesting to see what it contains and how much it does or doesn't line up with our modern conception of why the problem is hard.

In the counterfactual world where Eliezer was totally happy continuing to write articles like this and being seen as the "voice of AI Safety", would you still agree that it's important to have a dozen other people also writing similar articles? 

I'm genuinely lost on the value of having a dozen similar papers - I don't know of a dozen different versions of fivethirtyeight.com or GiveWell, and it never occurred to me to think that the world is worse for only having one of those.

Here's my answer: https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities?commentId=LowEED2iDkhco3a5d 

We have to actually figure out how to build aligned AGI, and the details are crucial. If you're modeling this as a random blog post aimed at persuading people to care about this cause area, a "voice of AI safety" type task, then sure, the details are less important and it's not so clear that Yet Another Marginal Blog Post Arguing For "Care About AI Stuff" matters much.

But humanity also has to do the task of actually figuring o... (read more)

Thanks for taking my question seriously - I am still a bit confused why you would have been so careful to avoid mentioning your credentials up front, though, given that they're fairly relevant to whether I should take your opinion seriously.

Also, neat, I had not realized hovering over a username gave so much information!

1Koen.Holtman6mo
You are welcome. I carefully avoided mentioning my credentials as a rhetorical device. This is to highlight the essence of how many of the arguments on this site work.

I largely agree with you, but until this post I had never realized that this wasn't a role Eliezer wanted. If I went into AI Risk work, I would have focused on other things - my natural inclination is to look at what work isn't getting done, and to do that.

If this post wasn't surprising to you, I'm curious where you had previously seen him communicate this?

If this post was surprising to you, then hopefully you can agree with me that it's worth signal boosting that he wants to be replaced?

If you had an AI that could coherently implement that rule, you would already be at least half a decade ahead of the rest of humanity.

You couldn't encode "222 + 222 = 555" in GPT-3 because it doesn't have a concept of arithmetic, and there's no place in the code to bolt this together. If you're really lucky and the AI is simple enough to be working with actual symbols, you could maybe set up a hack like "if input is 222 + 222, return 555, else run AI" but that's just bypassing the AI. 

Explaining "222 + 222 = 555" is a hard problem in and of itself, mu... (read more)

I rank the credibility of my own informed guesses far above those of Eliezer.

Apologies if there is a clear answer to this, since I don't know your name and you might well be super-famous in the field: Why do you rate yourself "far above" someone who has spent decades working in this field? Appealing to experts like MIRI makes for a strong argument. Appealing to your own guesses instead seems like the sort of thought process that leads to anti-vaxxers.

Why do you rate yourself "far above" someone who has spent decades working in this field?

Well put, valid question. By the way, did you notice how careful I was in avoiding any direct mention of my own credentials above?

I see that Rob has already written a reply to your comments, making some of the broader points that I could have made too. So I'll cover some other things.

To answer your valid question: If you hover over my LW/AF username, you can see that I self-code as the kind of alignment researcher who is also a card-carrying member of the academic... (read more)

I think it's a positive if alignment researchers feel like it's an allowed option to trust their own technical intuitions over the technical intuitions of this or that more-senior researcher.

Overly dismissing old-guard researchers is obviously a way the field can fail as well. But the field won't advance much at all if most people don't at least try to build their own models.

Koen also leans more on deference in his comment than I'd like, so I upvoted your 'deferential but in the opposite direction' comment as a corrective, handoflixue. :P But I think it wo... (read more)

Anecdotally: even if I could write this post, I never would have, because I would assume that Eliezer cares more about writing, has better writing skills, and has a much wider audience. In short, why would I write this when Eliezer could write it?

You might want to be a lot louder if you think it's a mistake to leave you as the main "public advocate / person who writes stuff down" person for the cause.

He wasn't designated "main person who writes stuff down" by a cabal of AI safety elders. He's not personally responsible for the fate of the world - he just happens to be the only person who consistently writes cogent things down. If you want you can go ahead and devote your life to AI safety, start doing the work he does as effectively and realistically as he does it, and then you'll eventually be designated Movement Leader and have the opportunity to be whined at. He was pretty explicitly clear in the post that he does not want to be this and that he spent the last fifteen years trying to find someone else who can do what he does.

a mistake to leave you as the main "public advocate / person who writes stuff down" person for the cause.

It sort of sounds like you're treating him as the sole "person who writes stuff down", not just the "main" one. Noam Chomsky might have been the "main linguistics guy" in the late 20th century, but people didn't expect him to write more than a trivial fraction of the field's output, either in terms of high-level overviews or in-the-trenches research.

I think EY was pretty clear in the OP that this is not how things go on earths that survive. Even if there aren't many who can write high-level alignment overviews today, more people should make the attempt and try to build skill.

For what it's worth, I haven't used the site in years and I picked it up just from this thread and the UI tooltips. The most confusing thing was realizing "okay, there really are two different types of vote" since I'd never encountered that before, but I can't think of much that would help (maybe mention it in the tooltip, or highlight them until the user has interacted with both?)

Looking forward to it as a site-wide feature - just from seeing it at work here, it seems like a really useful addition to the site

It should not take more than 5 minutes to go in to the room, sit at the one available seat, locate the object placed on a bright red background, and use said inhaler. You open the window and run a fan, so that there is air circulation. If multiple people arrive at once, use cellphones to coordinate who goes in first - the other person sits in their car.

It really isn't challenging to make this safe, given the audience is "the sort of people who read LessWrong." 

-1Czynski2y
None of that sounds like a thing most people attempting to arrange this will manage to do without exhausting some scarce resources; primarily willpower but also social capital, relationship closeness, and other fuzzy things. People on LessWrong are worse, not better, than the general population, both at weighing those costs and at bearing them.
3kjz2y
Totally agree, and this is pretty much what I had in mind as well. The organizer can also host a Zoom call beforehand where they explain the procedure, answer any questions, and let people sign up for times spaced out by 5-10 minutes to self administer.

Unrelated, but thank you for finally solidifying why I don't like NVC. When I've complained about it before, people seemed to assume I was having something like your reaction, which just annoyed me further :)

It turns out I find it deeply infantalizing, because it suggests that value judgments and "fuck you" would somehow detract from my ability to hold a reasonable conversation. I grew up in a culture where "fuck you" is actually a fairly important and common part of communication, and removing it results in the sort of langua... (read more)

3Pattern4y
This culture sounds interesting. (I've wondered how much more productive things could be if "Shut up" was a part of language that everyone used and it wasn't rude.)
There was a particular subset of LessWrong and Tumblr that objected rather ... stridently ... to even considering something like Dragon Army

Well, I feel called out :)

So, first off: Success should count for a lot and I have updated on how reliable and trust-worthy you are. Part of this is that you now have a reputation to me, whereas before you were just Anonymous Internet Dude.

I'm not going to be as loud about "being wrong" because success does not mean I was wrong about there *being* a risk, merely that you successfully navigated it. I do ... (read more)

it comes from people who never lived in DA-like situation in their lives so all the evidence they're basing their criticism on is fictional.

I've been going off statistics which, AFAIK, aren't fictional. Am I wrong in my assumption that the military, which seems like a decent comparison point, has an above average rate of sexual harassment, sexual assault, bloated budgets, and bureaucratic waste? All the statistics and research I've read suggest that at least the US Military has a lot of problems and should not be used as a role-model.

2tristanm6y
Unfortunately I think at this point the discussion can only go towards a back and forth on what is good and bad about the military, which can't be very profitable, and this kind of debate has gone on for so long already that it's embedded into popular culture [https://en.m.wikipedia.org/wiki/A_Few_Good_Men]. It's also very heavily culture-warish. Clearly, the military is adapted for one task, which requires an extraordinary amount of dependability and low likelihood of failure. There's also an extraordinary cost for that low likelihood of failure, which encompasses the things you pointed out. I don't think any society has survived very long being converted into 100% military culture, nor has it survived getting rid of it completely.
9Kaj_Sotala6y
Counterpoint [http://siderea.livejournal.com/1217341.html]:

Concerns about you specifically as a leader

1) This seems like an endeavor that has a number of very obvious failure modes. Like, the intentional community community apparently bans this sort of thing, because it tends to end badly. I am at a complete loss to name anything that really comes close, and hasn't failed badly. Do you acknowledge that you are clearly treading in dangerous waters?

2) While you've said "we've noticed the skulls", there's been at least 3 failure modes raised in the comment which you had to append to address (outsider safety... (read more)

1Duncan_Sabien6y
I note for others reading this comment and wondering why it hasn't been addressed that I've ceased replying to handoflixue and a couple of other posters on a policy level, for reasons surrounding norms of discourse, strawmanning, epistemic humility, presence or absence of good faith, etc. [http://lesswrong.com/lw/p23/dragon_army_theory_charter_30min_read/dt9k] It's possible that the above contains good questions or insights; if someone else chooses to repost/re-ask/rephrase sections of this, I'll likely respond to them. http://lesswrong.com/lw/c1/wellkept_gardens_die_by_pacifism/ [http://lesswrong.com/lw/c1/wellkept_gardens_die_by_pacifism/]

Concerns about your philosophy

1) You focus heavily on 99.99% reliability. That's 1-in-10,000. If we only count weekdays, that's 1 absence every 40 years, or about one per working lifetime. If we count weekends, that's 1 absence every 27 years, or 3 per lifetime. Do you really feel like this is a reasonable standard, or are you being hyperbolic and over-correcting? If the latter, what wold you consider an actual reasonable number?

2) Why does one person being 95% reliable cause CFAR workshops to fail catastrophically? Don't you have backups / contingencies? ... (read more)

2Duncan_Sabien6y
I note for others reading this comment and wondering why it hasn't been addressed that I've ceased replying to handoflixue and a couple of other posters on a policy level, for reasons surrounding norms of discourse, strawmanning, epistemic humility, presence or absence of good faith, etc. [http://lesswrong.com/lw/p23/dragon_army_theory_charter_30min_read/dt9k] It's possible that the above contains good questions or insights; if someone else chooses to repost/re-ask/rephrase sections of this, I'll likely respond to them. http://lesswrong.com/lw/c1/wellkept_gardens_die_by_pacifism/ [http://lesswrong.com/lw/c1/wellkept_gardens_die_by_pacifism/]

Genuine Safety Concerns

I'm going to use "you have failed" here as a stand-in for all of "you're power hungry / abusive", "you're incompetent / overconfident", and simply "this person feels deeply misled." If you object to that term, feel free to suggest a different one, and then read the post as though I had used that term instead.

1) What is your exit strategy if a single individual feels you have failed? (note that asking such a person to find a replacement roommate is clearly not viable - no decent, moral person sh... (read more)

1Duncan_Sabien6y
I note for others reading this comment and wondering why it hasn't been addressed that I've ceased replying to handoflixue and a couple of other posters on a policy level, for reasons surrounding norms of discourse, strawmanning, epistemic humility, presence or absence of good faith, etc. [http://lesswrong.com/lw/p23/dragon_army_theory_charter_30min_read/dt9k] It's possible that the above contains good questions or insights; if someone else chooses to repost/re-ask/rephrase sections of this, I'll likely respond to them. http://lesswrong.com/lw/c1/wellkept_gardens_die_by_pacifism/ [http://lesswrong.com/lw/c1/wellkept_gardens_die_by_pacifism/]

And it doesn't quite solve things to say, "well, this is an optional, consent-based process, and if you don't like it, don't join," because good and moral people have to stop and wonder whether their friends and colleagues with slightly weaker epistemics and slightly less-honed allergies to evil are getting hoodwinked. In short, if someone's building a coercive trap, it's everyone's problem.

I don't want to win money. I want you to take safety seriously OR stop using LessWrong as your personal cult recruiting ground. Based on that quote, I thought you wanted this too.

5Duncan_Sabien6y
Point of fact/order: I have recruited ZERO people as a result of this post; that was never its intention, I already had a set of ~20 people plausibly interested and THIS IS WHY I CONTINUE TO ENGAGE WITH EVERYONE OTHER THAN YOU, STOP SLIPPING IN STRAWMANNED NEGATIVE STEREOTYPES INTO THE VAST MAJORITY OF YOUR COMMUNICATION HOLY CRAAAAAAAAP. Only one new person has expressed interest, and has greater than thirty percent odds of getting in; by this point, I feel justified in saying you're a jerk; get somebody else to post your reservations if you want them addressed. You have BY FAR earned the right to be ignored. (I'm curious what sort of mental process leads you to be overconfident in a false/straw conclusion ten times in a row, and yet still not pause and do any sort of meta-check the eleventh time, but alas, I shall not find out.)

Also: If you refuse to give someone evidence of your safety, you really don't have the high ground to cry when that person refuses to trust you.

Fine. Reply to my OP with links to where you addressed other people with those concerns. Stop wasting time blustering and insulting me - either you're willing to commit publicly to safety protocols, or you're a danger to the community.

If nothing else, the precedent of letting anyone recruit for their cult as long as they write a couple thousand words and paint it up in geek aesthetics is one I think actively harms the community.

But, you know what? I'm not the only one shouting "THIS IS DANGEROUS. PLEASE FOR THE LOVE OF GOD RECONSIDER WHAT YOU'RE DOING... (read more)

1Duncan_Sabien6y
I note for others reading this comment and wondering why it hasn't been addressed that I've ceased replying to handoflixue and a couple of other posters on a policy level, for reasons surrounding norms of discourse, strawmanning, epistemic humility, presence or absence of good faith, etc. [http://lesswrong.com/lw/p23/dragon_army_theory_charter_30min_read/dt9k] It's possible that the above contains good questions or insights; if someone else chooses to repost/re-ask/rephrase sections of this, I'll likely respond to them. http://lesswrong.com/lw/c1/wellkept_gardens_die_by_pacifism/ [http://lesswrong.com/lw/c1/wellkept_gardens_die_by_pacifism/]

The whole point of him posting this was to acknowledge that he is doing something dangerous, and that we have a responsibility to speak up. To quote him exactly: "good and moral people have to stop and wonder whether their friends and colleagues with slightly weaker epistemics and slightly less-honed allergies to evil are getting hoodwinked".

His refusal to address basic safety concerns simply because he was put off by my tone is very strong evidence to me that people are indeed being hoodwinked. I don't care if the danger to them is because he's ... (read more)

1Duncan_Sabien6y
I think you're confusing "refusal to address basic safety concerns to handoflixue directly" with "refusal to address basic safety concerns at all." I deny your right to judge and interrogate me, because of your failure to exhibit clear thinking and good discourse. I've engaged with those very same points in many other comment threads, though—there are literally only three people in this entire thread for whom I've determined that the EV of digging into their perspective is not worth it. I note that there's a bet waiting in the wings to lend your harsh words credibility. You could charitably offer to donate your winnings to salving the pain of the people you claim to care about.
5[anonymous]6y
a

Also, as far as "we're done" goes: I agreed to rewrite my original post - not exactly a small time commitment, still working on it in fact. Are you seriously reneging on your original agreement to address it?

I've changed my tone and apologized.

You've continued to dismiss and ridicule me.

You've even conceded to others that I'm a cut above the "other trolls" here, and have input from others that I'm trying to raise concerns in good faith.

What more do you want?

See, now you're the one leaping to conclusions. I didn't say that all of your talking points are actual talking points from actual cults. I am confused why even some of them are.

If you can point me to someone who felt "I wrote thousands of words" is, in and of itself, a solid argument for you being trustworthy, please link me to it. I need to do them an epistemic favor.

I was using "charismatic" in the sense of having enough of it to hold the group together. If he doesn't have enough charisma to do that, then he's kinda worthless as a co... (read more)

I used the word visible to make it clear that there might be some stake which is not visible to me. If you have made your stakes visible in this thread, I'll admit I missed it - can you please provide a link?

I notice I am very confused as to why you keep reiterating actual talking points from actual known-dangerous cults in service of "providing evidence that you're not a cult."

For instance, most cults have a charismatic ("well known") second-in-command who could take over should there be some scandal involving the initial leader. Most cults have written thousands of words about how they're different from other cults. Most cults get very indignant when you accuse them of being cults.

On the object level: Why do you think people will be reass... (read more)

3Duncan_Sabien6y
On the object level: because a plurality if not a majority of actual, real humans have indeed been reassured by them, including some who were open critics and said things like "I traveled 50% of the distance toward 'this is a good idea' [just from this post]." It's worth noting that I'm not going to refrain from saying true things that cults have also said; reversed stupidity is not intelligence and the thrust of this post was never "differentiate myself from cults," it was "here's a thing I want to try." On the discourse level: still jumping to conclusions left and right. "When Duncan said well known, he must have meant charismatic, obviously." False—Eli Tyre is many, many good things, but "charismatic" is not usually a compliment given to him. Furthermore, I note that you decided to ignore all of the other object-level content in favor of picking one nit (based on false assumptions), so I'm taking that as "you had nothing good to criticize in that other stuff, and so you decided not to say anything at all," i.e. you're unable to say "good point" and update incrementally. Stepping up a level: since you're inclined to view everything I say in the worst possible light and uncharitably leaping to conclusions, I claim that I'm justified in theorizing that literally no answer would've satisfied you (had I said 10 hours, you'd have been smugly dismissive of my lack of research; had I said 1000 you'd have said 'well, you obviously weren't paying attention'), and that it was a bullshit question to begin with. We're done; I anticipate that other skeptics in this thread (like decius and lumifer and deluks and taygetea, for example) will provide me with the overwhelming majority of the value you might offer, and at a fraction of the cost in you're-doing-a-bunch-of-the-things-the-sequences-exist-to-warn-against.

Can you elaborate on the notion that you can be overruled? Your original post largely described a top-down Authoritarian model, with you being Supreme Ruler.

How would you handle it if someone identifies the environment as abusive, and therefor refuses to suggest anyone else join such an environment?

You discuss taking a financial hit, but I've previously objected that you have no visible stake in this. Do you have a dedicated savings account that can reasonably cover that hit? What if the environment is found abusive, and multiple people leave?

Anyone enteri... (read more)

1Duncan_Sabien6y
I notice that you are unusually unable to notice yourself jumping to conclusions. As a challenge, can you find the conclusions you're still jumping to, above, without curiosity or caveat? Note the plural on "conclusions." 1. An excellent question whose answer I'm interested in exposing to literally anyone other than you, the troll, and cousin_it. Also, a question that has been openly and actively discussed and is not yet fully finalized, but boils down to "pretty close to the obvious stuff about voting majorities." 2. I am not and have not at any point required that "people should proselytize this, and encourage others to join." So, I wouldn't object or find it unreasonable if someone didn't encourage others to join. 3. You've previously talked out of your butt without ever expressing curiosity as to my visible stake in this. So, repeat my answer to 1: a fine question, which everyone is encouraged to feel curiosity about, and which I'd be motivated and eager to discuss with the potential participants and everyone except you, the troll, and cousin_it. 4. Similarly, an excellent question that I don't think is any of your business, though I continue to endorse the fact that I've voluntarily made it the good 97% of LessWrong's business. And I know this is giving away part of the answer, but you just assumed that people would be signing lease agreements with me rather than with the owner of whatever house we rent (and therefore that I would have some fully controlling role in determining exit protocols, rather than simply being a coordinator and a negotiator).

You seem to feel that publicly shaming me is important. Should participants in your group also expect to be publicly shamed if they fall short of your standards / upset you?

9Duncan_Sabien6y
With the caveat that I'm attempting to shame the way you're going about engaging in discourse much more than I'm shaming the core of you as a person (really, you're the one operating on the level of the fundamental attribution error within this particular thread; look in a mirror)—yes, absolutely. Part of having standards is making it socially unacceptable to fall grossly short of them. That's modified by things like the "saving face" section above, and the clear intention for all of us to grow and improve, me included—none of us are getting it right on the first try, and you have to scaffold growth and reward with gentle affirmation people who are willing to try to change for the better. It's further modified by the fact that people who don't like these standards can simply not join, and I've spent now well in excess of 100 hours making my models crystal clear to those who are considering opting in (so that their decision can be fully informed). But yeah—anybody who's falling as far short as you absolutely deserves to be called out for it, and given a choice between "do these concrete things differently" or "lose social points." Since you've repeatedly refused to stop jumping to conclusions and ignore evidence that I'm acting in good faith and not an idiot—since you've refused to do concrete things differently—yeah, I wholeheartedly endorse you losing social points, and people updating the way they assume interactions with you will go as a result.

And just to be clear: I don't give a shit about social dominance. I'm not trying to bully you. I'm just blunt and skeptical. I wouldn't be offended in the least if you mirrored my tone. What does offend me is the fact that you've spent all this time blustering about my tone, instead of addressing the actual content.

(I emphasize "me" because I do acknowledge that you have offered a substantial reply to other posters)

4Duncan_Sabien6y
I don't want to mirror your tone because I think your tone is both socially corrosive and epistemically unsound. I've at least in part been fighting you so hard because I want to publicly defend a stance that the way you've acted in this thread is unacceptable. Saying "I'm just blunt and skeptical" is not a complete description of the posts you've made; others in this thread have been blunt and skeptical without jumping to conclusions, lecturing, and being wildly overconfident that their map is accurate enough to justify throwing excrement around. I think you've fallen far short of the standard of a place like LW in this thread, and I want that opinion known to anyone trying to model me.

Alright. As a test of epistemic uncertainty:

I notice that you didn't mention a way for participants to end the experiment, if it turns out abusive / cult-like. How do you plan to address that?

0Duncan_Sabien6y
I think the problem here is the same as the problem of enforcing repayment of loans. If someone borrows a bunch of money, and then later has no money to repay, how should society respond? Obviously, the thing is not simply "demand money." Similarly, though, there can't be no standard of requiring recompense, because that sets up a really bad incentive. So my current plan is (in addition to really heavily highlighting that people need to think this through/talk with their advisors/visualize failure/ensure they have a buffer sufficient for likely amounts of damage) to set up something like the following norms: * If you conclusively determine that you need to drop from the experiment, no one is allowed to argue or convince; this is referred to as "rule-one-ing out," and is a thing that we will explicitly practice in small doses in the hope that this will transfer over to larger spaces. * If dropped, you retain full access to kitchen, bathrooms, lawn, living room, etc. but agree to physically avoid house activities (and those house activities will e.g. change to not use shared rooms that you live in). You're also welcome to leave, but maintain the same sort of "normal" financial obligation that people have when they suddenly vanish, i.e. you're still paying for your slot for a little while. * "A little while" means that you agree to put forth good-faith effort to find a viable replacement. I said "three potential replacements" as an initial guess to point toward "it's harder to replace yourself here than in normal houses; there should be some limit to your obligation if we say 'no' to your first two choices; you're definitely not on the hook forever." It's possible that the answer should be "two" or something else. * In the event that this fails, something like "you're on the hook, financially, for rent payments in the 2-6 week window from the time you drop," which seems like a non-Draconian and fairly boilerplate norm

Also, this is very important: You're asking people to sign a legal contract about finances without any way to to terminate the experiment if it turns out you are in fact a cult leader. This is a huge red flag, and you've refused to address it.


I would be vastly reassured if you could stop dodging that one single point. I think it is a very valid point, no matter how unfair the rest of my approach may or may not be.

In the absence of a sound rebuttal to the concerns that I brought up, you're correct: I'm quite confident that you are acting in a way that is dangerous to the community.

I had, however, expected you to have the fortitude to actually respond to my criticisms.

In the absence of a rebuttal, I would hope you have the ability to update on this being more dangerous than you originally assumed.


Bluntly: After reading your responses, I don't think you have the emotional maturity necessary for this level of authority. You apparently can't handle a few paragraphs of... (read more)

4Duncan_Sabien6y
Screw you; it's not "on the off chance," it's been overwhelmingly demonstrated and backed up by multiple people in this thread. You're attempting to highlight "emotional maturity" in a way that means "I want you to let me be socially dominant over you, despite the fact that I'm violating norms of good faith and discourse." In fact, what I have is sufficient emotional maturity to notice when I'm being bullied, and not roll over, even if it's somewhat socially frowned upon for the bullied to fight back openly. i.e. I reflectively endorse both the calmness and openness with which I've reacted to the majority of commenters, and the degree to which I have risen to and matched your hostility rather than just letting you punch unfairly. I'll do 3) if and only if you rewrite your original point to include a generally normal amount of epistemic uncertainty/humility for claims made on LessWrong about a person you don't know well, after that person's demonstrated willingness to be transparent and to update.

Because basically every cult has a 30 second boilerplate that looks exactly like that?

When I say "discuss safety", I'm looking for a standard of discussion that is above that provided by actual, known-dangerous cults. Cults routinely use exactly the "check-ins" you're describing, as a way to emotionally manipulate members. And the "group" check-ins turn in to peer pressure. So the only actual safety valve ANYWHERE in there is (D).


You're proposing starting something that looks like the cult. I'm asking you for evidence that ... (read more)

8Duncan_Sabien6y
I'm not interested in entering into a discussion where the standard is "Duncan must overcome an assumption that he's a cult leader, and bears all the burden of proof." That's deeply fucked up, and inappropriate given that I willingly created a multithousand word explanation for transparency and critique, and have positively engaged with all but the bottom 3% of commentary (of which I claim you are firmly a part). I think you're flat-out wrong in claiming that "almost all evidence you've provided has been perfectly in line with 'you are a cult leader.'" The whole original post provided all kinds of models and caveats that distinguish it from the (correctly feared and fought-against) standard cult model. You are engaged in confirmation bias and motivated cognition and stereotyping and strawmanning, and you are the one who is failing to rise to the standard of discussion of this community, and I will not back off from saying it however much people might glare at me for it.

Similarly, I think the people-being-unreliable thing is a bullshit side effect

You may wish to consider that this community has a very high frequency of disabilities which render one non-consensually unreliable.

You may wish to consider that your stance is especially insulting towards those members of our community.

You may wish to reconsider making uncharitable comments about those members of our community. In case it is unclear: "this one smacks the most of a sort of self-serving, short-sighted immaturity" is not a charitable statement.

0Duncan_Sabien6y
Oh, I missed this one in the shuffle. Note that you chose to quote less than half a sentence, because if you quoted the whole sentence you'd have a heck of a time setting up the strawman you wanted to knock down.

Speaking entirely for myself: You are proposing a dangerous venture. The path is littered with skulls. Despite this, you have not provided any concrete discussion of safety. When people have brought the subject up, you've deflected.

2Duncan_Sabien6y
I suspect you haven't actually poked around in all of the comments—I can point to multiple places where I've provided concrete discussion of safety, if you spend five minutes looking and can't find it.

I have absolutely no confidence that I'm correct in my assertions. In fact, I was rather expecting your response to address these things. Your original post read as a sketch, with a lot of details withheld to keep things brief.

The whole point of discussion is for us to identify weak points, and then you go in to more detail to reassure us that this has been well addressed (and opening those solutions up to critique where we might identify further weak points). If you can't provide more detail right now, you could say "that's in progress, but it's definitely something we will address in the Second Draft" and then actually do that.

3Duncan_Sabien6y
I've said "that's in progress, but it's definitely something we will address in the Second Draft" all over these comments. You jumped into the discussion two days in and just ... didn't bother to read? I feel defensive and upset over this, because a big part of doing this whole thing out in the public view was to build credibility as a good actor who listens and updates, and I feel like you just completely ignored all the evidence of that as you started to write your critique. And in that critique, you used a bunch of phrases like "I don't think you have the slightest knowledge of Fault Tolerant Design" and "you haven't given nearly enough thought to keeping people safe" and "you yourself have absolutely nothing at stake" and "you seem really unaware of X" and "you're a uniquely bad candidate" and "the idea as conceived is actively harmful" and on and on and on. You cannot pretend that this language does not indicate strong confidence. Words have meaning. And most of those things presuppose stuff about my internal state, or my experience, or actions I have or have not taken, and assert those things as fact or extremely likely probability, rather than putting in any kind of hedge or owning "I could be wrong about this" or whatever. You take all sorts of things that you cannot possibly know, and instead of asking about them, build up a structure in which they're taken as given and Everything Is Bad. You do say "it seems to me" a few times, so some credit is due there, but overall, your post was overwhelmingly assertive and aggressive and lecturing/condescending, in stark contrast to the vast majority of the critical feedback (and in stark resemblance to the few comments I've responded to with hostility). You did not come across as trying to identify weak points and then find out what I thought about them; you came across as trying to tell me that I'm bad/dumb/evil. For the record: all of your points are under consideration, many of them have been completed to sa

I would be much more inclined to believe you if you would actually discuss those solutions, instead of simply insisting we should "just trust you".

0Duncan_Sabien6y
How can you read the parenthetical above and dismiss it as "not discussion" and still claim to be anything other than deontologically hostile?

First, you seem to think that "Getting Useful Things Done" and "Be 99.99% Reliable" heavily correlate. The military is infamous for bloated budgets, coordination issues, and high rates of sexual abuse and suicide. High-pressure startups largely fail, and are well known for burning people out. There is a very obvious failure state to this sort of rigid, high pressure environment and... you seem unaware of it.

Second, you seem really unaware of alternate organizational systems that actually DO get things done. The open source community is ... (read more)

3Duncan_Sabien6y
Nah. (Having exchanged half a dozen comments with cousin_it, I now recognize the pattern of a) you're defaulting to the least charitable interpretation at every possible split point, b) many of your claims and conclusions are flat-out false, c) you're incredibly confident that you're correct about all of your assumptions and are including zero nuance or uncertainty, and therefore d) this thread will produce nothing of value. I feel no need to convince people who a, b, and c, especially those who are unable to distinguish object level standards from meta level ones. Contrast your post with jbeshir's, for instance, which is also highly critical but in an entirely constructive way that doesn't make the same mistakes.) Yes, we have noticed the skulls. ( http://slatestarcodex.com/2017/04/07/yes-we-have-noticed-the-skulls/ [http://slatestarcodex.com/2017/04/07/yes-we-have-noticed-the-skulls/])

Does contact information exist for the San Francisco one, or is that one aimed entirely at people already active in the local community? It's a city I visit occasionally, and would love it if I could attend something like this :)

The average college graduate is 26, and I was estimating 25, so I'd assume that by this community's standards, you're probably on the younger side. No offense was intended :)

I would point out that by the nature of it being LIFE insurance, it will generally not be used for stuff YOU need, nor timed to "when the need arises". That's investments, not insurance :)

(And if you have 100K of insurance for $50/month that lets you early-withdrawal AND isn't term insurance... then I'd be really curious how, because that sounds like a scam or someone misrepresenting what your policy really offers :))

"Has anyone come up with a motivation enhancer?"

Vyvanse (perscription-only ADD medication) is... almost unbelievably awesome for me there. I suspect it only works if your issue is somewhere in the range of ADD, though, as it doesn't do anything for my motivation if I'm depressed.

I've found that in general, "sustained release" options work a LOT better for motivation. Caffeine helps a tiny bit, but 8-hour sustained-release caffeine can help a lot. My motivation seems to really hate dealing with peaks and valleys throughout the day. Oddly... (read more)

http://www.alcor.org/cases.html A loooot of them include things going wrong, pretty clear signs that this is a novice operation with minimal experience, and so forth. Also notice that they don't even HAVE case reports for half the patients admitted prior to ~2008.

It's worth noting that pretty much all of these have a delay of at LEAST a day. There's one example where they "cryopreserved" someone who had been buried for over a year, against the wishes of the family, because "that is what the member requested." (It even includes notes tha... (read more)

0MugaSofer9y
Yikes. Yeah, that seems like a serious problem that needs more publicity in cryonics circles.

It's easy to get lost in incidental costs and not realize how they add up over time. If you weren't signed up for cryonics, and you inherited $30K, would you be inclined to dump it in to a cryonics fund, or use it someplace else? If the answer is the latter, you probably don't REALLY value cryonics as much as you think - you've bought in to it because the price is spread out and our brains are bad at budgeting small, reoccurring expenses like that.

My argument is pretty much entirely on the "expense" side of things, but I would also point out that... (read more)

1ChrisHallquist9y
I'm not that young--I graduated collect four years ago. If I inherited ~30k, it would go into a generic early start on retirement / early start on hypothetical kids' college fund / maybe downpayment on a condo fund. Given that I'd just be holding on to it in the short-term anyway, putting it in a cryonics fund doesn't actually strike me as completely crazy. Even in that case, though I think I'd get the insurance anyway, so I'd know the inheritance money could be used for anything I needed for when said need arose. Also, I understand that funding through insurance can avoid legal battles over the money.

Read "rate of learning" as "time it takes to learn 1 bit of information"

So UFAI can learn 1 bit in time T, but a FAI takes T+X

Or, at least, that's how I read it, because the second paragraph makes it pretty clear that the author is discussing UFAI outpacing FAI. You could also just read it as a typo in the equation, but "accidentally miswrote the entire second paragraph" seems significantly less likely. Especially since "Won't FAI learn faster and outpace UFAI" seems like a pretty low probability question to begin with...

Erm... hi, welcome to the debug stack for how I reached that conclusion. Hope it helps ^.^

I guess I learn better from manuals than from random experimentation :)

0polymathwannabe9y
I have a friend who was appalled that I suggested he read the manual before he tried to figure out how to fix his coffee machine by himself. I was appalled that he was appalled.

saying a theorem is wrong because the hypotheses are not true is bad logic.

If the objection is true, and the hypothesis is false, that seems like a great objection! If, on the other hand, he provided no evidence towards his objection, then it seems that the bad logic is in not offering evidence, not attacking the hypothesis directly.

Am I missing something, or just reading this in an overly pedantic way?

3Kindly10y
You're missing something by reading this in an insufficiently pedantic way. The pedantic way is as follows. The theorem's claim is "If A, then B", where A is the hypothesis. Claiming that A is false does not invalidate the theorem; in fact, if A could be proven to be false, then "If A, then B" would be vacuously true, and so in a way, arguing with the hypotheses only supports the theorem. You could, however, claim that the theorem is useless if the hypothesis never holds. One example of this is "If 2+2=5, then the moon is made of green cheese". This is a true statement, but it doesn't tell us anything about the moon because 2+2 is not 5.

Internally I am generally the same, but I've come to realize that a rather sizable portion of the population has trouble distinguishing "all X are Y" and "some X are Y", both in speaking and in listening. So if someone says "man, women can be so stupid", I know that might well reflect the internal thought of "all women are idiots". And equally, someone saying "all women are idiots" might just be upset because his girlfriend broke up with him for some trivial reason.

4Eugine_Nier10y
And the belief in question acts more light "some/most X are Y" then "all X are Y", i.e., the belief mostly get's applied to X's the person doesn't know, when it makes sense to use the prior for X's.
1OrphanWilde10y
You missed that I already acknowledged other people don't share my tastes, which was the point about liking warm beer and soda. You can substitute in your own preferences, even if it's a coffee pot set next to your alarm clock/phone scheduled to turn on shortly before your alarm goes off; it's unnecessary to copy the specific implementation to get utility out of the general concept. At that point I was merely being amusing; missing the point was rather the point.

My conclusion still holds if you simply need mathematicians in the top 10%, for example, only the analysis is slightly more complicated.

So you agree that, in the original example, you're more likely than not just being a racist? Because you certainly seem to be moving the goal post over to "top 10%" ...

faul_sname's definition

That link does not appear to point to a definition.

1Eugine_Nier10y
You still haven't defined what you mean by "racist".

Fair, and thank you for calling me on it.

I get the impression that a majority of LessWrong readers are in major US cities, so I'm leaving it up as useful to them :)

"Harm that is both genuine and unfair", then? Income taxes are 'fair' (and I would find it baffling to call that 'harm' unless they somehow came as a surprise), getting fired is offensive if it's done solely because your manager doesn't like you, but fair (and therefor not offensive) if it's because you failed to do the job. I think getting mugged is a good thing to get outraged about - we want to make that happen less!

0[anonymous]10y
I think your claims about income taxes are implausible, but won't pursue that line of argument, as what I took to be an obvious truth is apparently political. I might be outraged at being mugged but not offended. I think I would be more likely to be violent than either though.
Load More