Thanks for clarifying. To the extent that you aren't particularly sure about consciousness comes about, it makes sense to reason about all sorts of possibilities related to capacity for experience and intensity of suffering. In general, I'm just kinda surprised that Eliezer's view is so unusual given that he is the Eliezer Yudkowsky of the rationalist community.

My impression is that the justification for the argument your mention is something along the lines of "the primary reason one would develop a coherent picture of their own mind is so they could convey a convincing story about themselves to others -- which only became a relevant need once language developed."

I was under the impression you were focused primarily on suffering from the first two sections and the similarity of the above logic to the discussion of pain-signaling earlier. When I think about your generic argument about consciousness, I get confused however. While I can imagine why would one benefit from an internal narrative around their goals, desires, etc, I'm not even sure how I'd go about squaring pressures for that capacity with respect to the many basic sensory qualia that people have (e.g. sense of sight, sense of touch) -- especially in the context of language.

If one accepts Eliezer Yudkowsky's view on consciousness, the complexity of suffering in particular is largely irrelevant. The claim "qualia requires reflectivity" implies all qualia require reflectivity. This includes qualia like "what is the color red like?" and "how do smooth and rough surfaces feel different?" These experiences seem like they have vastly different evolutionary pressures associated with them that are largely unrelated to social accounting.

If you find the question of whether suffering in particular is sufficiently complex that it exists in certain animals but not others by virtue of evolutionary pressure, you're operating in a frame where these arguments are not superseded by the much more generic claim that complex social modeling is necessary to feel anything. 

If you think Eliezer is very likely to be right, these additional meditations on the nature of suffering are mostly minutiae.

[EDIT to note: I'm mostly pointing this out because it appears that there is one group that uses "complex social pressures" to claim animals do not suffer because animals feel nothing and another group that uses "complex social pressures" to claim that animals do not specifically suffer because suffering specifically depend on these things. That these two groups of people just happen to start from a similar guiding principle and happen to reach a similar answer for very different reasons makes me extremely suspicious of the epistemics around the moral patienthood of animals.]

Forgive me if engage with only part of this, I believe that the OP already acknowledges most of the problem you've described.

No forgiveness needed! I agree that the OP addresses this portion -- I read the OP somewhat quickly the first time and didn't fully process that part of it. And, as I've said, I do appreciate the thought you've put into all this.

I think I differ from the text of the OP in that social-shaming/lack-of-protest-method in rituals is often an okay and sensible thing. It is only when this property is combined with a serious problem with the ritual itself that I get worried -- but I have a hunch that you'd agree with this.

I care about the entire LessWrong community. I'm not sure where the exact boundaries lie–it's more than posters/commenters and probably short of anyone who's ever read a LW post–but I'm especially interested in the group who I feel like I can trust to work with me when the stakes are real and high. The Petrov Day ritual to date was designed to show that this group exists and trust each other, and I think that's a powerful and valuable thing to do, if you can do it.

I agree that having/establishing a group of people you can work with/trust is a good thing, and I think that rituals about this can be beneficial. However I have two main objections to this perspective:

#1. 
It is not obvious to me that identifying a group unlikely to press the button in a Petrov Day ritual is one capable of coordinating generally when stakes are real and high. As commenters have noted, social pressure incentives stack pressure against defecting. Moreover, if you are selecting for people who you know well enough to speculate on behavior in these circumstances, you are probably also selecting for people more deeply connected for the community for whom these pressures matter a lot. 

#2. 
I don't think an existence proof for a 100-strong set of LWers who don't press the button in a Petrov's Day ritual is particularly useful or surprising to me. If 50% of LWers would press the button and 50% wouldn't, its mathematically obvious that such a group exists. The actually arguably impressive/surprising part of the Ritual is not "does this group exist?" -- its "hey look! we have a selection process with strong enough discriminatory power to find one-hundred people who will act in a certain way." 

This could mean something important symbolically -- about how so many people in the community are trustworthy that we can assemble a group with even our imperfect judgement. But it could also mean the following things:

•  We have 10,000 people to select from and the top-one-percentile of people ranked by unwillingness to press the button is very unwilling to press the button
• We are just very good at predicting people's behavior in this circumstance and at least 10% of the community won't press the button. 
  • Note that this also strongly interacts with point #1 because then you can select both for people you're confident are trustworthy and for people who will who'd submit under strong social pressure (not that I think you'd do that deliberately)

So its hard for me to find much symbolic importance in the thing that the current Petrov Day Ritual is establishing. 

But, social pressure aside, the establishment of high-stakes trust group is not obviously useful/relevant information to a typical community member. The capacity for distinguishing a high-trust group in a given community is only relevant to me if I can also distinguish a high-trust group that I can work with. In other words, knowing that such a group exists theoretically does not mean much to me if I happen to be in a branch of the community that I can't/shouldn't trust. My impression is that people who receive the code and choose not to press the button are basically anonymous, so this is the case here.[1]

If one knew for sure that this group delineated trustworthy people capable of coordinating effectively in high-stakes scenarios, it may be useful for ritualized (optional) self-reveals after the fact. However, I would actually caution against this as it could have a reverse/harmful effect if any participant does anything harmful ever -- since it could establish a coalitional Schelling Point of mutual protection or the perception of one -- creating real or perceived complicitness in harm.

 

Naturally, an ideal Petrov Day design would be both something for the ideal community and perhaps also be something that strengthens the trust between an especially devoted community core.

Yep! In practice, I don't think Petrov Day needs to everything though -- and its probably easiest to create a really strong ritual that captures one theme and then explore secondary activities/processes that don't interfere with the core. 

Strengthening trust of the community core is a good thing -- and I don't think every ritual has to be about the entire community or vice-versa. I'm more concerned about what the selection-process+ritual-combi itself signals (both to the core and everyone else) about the kind of compliance behaviors expected by the core (the current process sort of confounds them). For contrast, if the selection process credibly demonstrated that it was just selecting Core Members, dropped the social-pressure incentives, and then demonstrating that core can trust each other not to blow things up then it would be a lot more meaningful in the sense you are gesturing at.

Perhaps there's also a minor frame difference here where you already see the process as basically something like "pick core members" based on your experience while that wasn't my default assumption.

[1] To be fair, I doubt that actually affiliating oneself with this group/parts-of-it would be overwhelmingly difficult -- given the obvious affiliations and the of people who have stated that they've received codes.

You seem to have put a lot of thought into this ritual and I appreciate the consideration you, Ben, and others are giving it. Anyway, here's some raw unfiltered (potentially overly-harsh) criticism/commentary on Petrov Day -- take what you need from it: 

In addition to Lethriloth's criticism of LW Petrov Day failing to match the incentives/dynamics associated with Petrov (an important consideration indeed given the importance of incentive consideration in the LW cannon), it is also important to consider that Community Rituals may serve ends wildly disparate from their stated purposes, whether intentionally or unintentionally.

Yes requires the possibility of no. If you are under the impression (as I am), that this Ritual deliberately curates an anonymous subset of the population likely to produce a desired outcome, that people who transgress on this outcome are socially shamed, and that the maximally obvious way to express distaste of a Ritual is to transgress upon it... then this Ritual as it stands systematically attacks the feasibility of "no." 

To elaborate, it does the following things:

• The Ritual misrepresents the true opinion of the community, by selecting those who would take it seriously and erasing those who wouldn't[1].
• If the Ritual fails to sufficiently filter out people who don't want to take it seriously, it preemptively punishes their actions with social shaming.

From the lens of someone who thinks LW Petrov Day fails to meaningfully reflect Petrov's decision, this creates the impression that subsets of rationalist community can concoct arbitrary rituals (as long as they have some plausible justification) and declare them to be community rituals all-the-while excluding members who have strong reasons for disagreement. Moreover, it establishes that this subset can use social-shaming strategies to coerce disagreeing members from protesting in obvious (but materially mild) ways[2].

I'd imagine that these considerations where present in the people you alluded to who preferred to self-exclude from The Ritual but I could be wrong.

Given recent criticism of the rationalist community, I consider course-correcting away from these types of dynamics as pretty crucial.

[1] There is a second criticism here, which is that the existing Ritual prioritizes the impression/illusion of coordination in the community over the actual level of coordination in the community -- which is both epistemically disadvantageous and meta-level disadvantageous as one would not expect a community like LW to prefer symbolic coordination over actual coordination.

[2] There is an argument that protesting in this way corresponds to a Unilateralist Veto and, on a meta-level, the community should disincentive this means of protesting and that social shaming is an acceptable way to do this. But this is a pretty roundabout way of handling that and I think the first-order social effects swamp the import of this argument.

This is cool! I like speedrunning! There's definitely a connection between speed-running and AI optimization/misalignment (see When Bots Teach Themselves to Cheat, for example). Some specific suggestions:

• Speedrun times have a defined lower bound on the minimization problem (zero seconds). So over an infinite amount of time, the time vs speedrun time plot necessarily converges to a flat line. You can avoid this by converting to an unbounded maximization problem. For example, you might wanna try plotting Speed-Run-Time-on-Game-Release divided by Speed-Run-Time at time t vs time. Some benefits of this include
  • Intuitive meaning: This ratio tells you how many optimal speed-runs at time t could be accomplished over the course of a single speed-run at game release
  • Partially addresses diminishing returns: Say the game's first speed-run completes the game in 60 seconds and the second speed-run is completes the game at 15 seconds (a 45 second improvement). No matter how much you work at the game, its not possible to reduce the speed-run time by more than the 45 second improvement (at most you can do 15 seconds) so diminishing returns are implied
    
    In contrast, if you look at the ratio, the first speed has a ratio of 1 (60 seconds/60 seconds), the second has a ratio of 4 (60 seconds/15 seconds), and a third one-second speed run has a ratio of 60 (60 seconds/1 second). Between the second and third speed-run, we've gone from a value of 4 to a value of 60 (a 15x increase!). Diminishing returns are no longer inevitable!
  • Easier to visualize: By normalizing by the initial speed-run time, all games start out with the same value regardless of how long they objectively take. This will allow you to more easily identify similarities between the trends.
  • More comparable to tech progress: Since diminishing returns aren't inevitable by construction, this looks more like tech progress where diminishing returns also aren't inevitable by construction. Note that they still can be in practice however
• Instead of plotting absolute dates, you plot time relative to when the first speed-run was registered. That is, set the date of the first speed run to t=0. This should help you identify trends.
• A lot of the games you review indicate that, in many cases, our best speed-run time so far isn't even 3x as faster as the original speed-run. This implies that optimizing speed-run time (or the ratio I introduce above) is bounded and you can't get more than a factor of 3 or 4 in terms of improvement. But obviously tech capabilities have improved by several orders of magnitude. So structurally, I don't think speed-running can be particularly predictive of the tech advances
• Given the above, I suggest that if you want to model speed-runs, you should use functions that expect asymptotes (eg logistic equations). Combinations of logistic equations can probably capture the cascading L curves you notice in your write-up. May also be worth doing some basic analysis like counting the number of inflections in each speed-run (do this by plotting derivatives and counting the number of peaks).
  • If you do this, I strongly suggest doing a transformation like the one I suggested above since otherwise, you're probably gonna get diminishing returns right off the bat and logistic equations don't expect this. If you don't transform for whatever reason, try exponential decay.
• Speed-running world records have times that, by definition, must monotonically decrease. So its expected that most of the plots will look like continuous functions. As you're plotting things now, diminishing returns are built-in so you should also expect the derivatives to

Have fun out there!

Thanks, I appreciate the concrete examples of untrustworthiness than don't rely on inferences made about reputation. I am specifically concerned about things like this (which seems like a weird and bad direction to take a conversation (https://sinceriously.fyi/net-negative/). It also seems hard to recount falsely without active deception or complete detachment from reality and I doubt Ziz is completely detached from reality:

They asked if I’d rape their corpse. Part of me insisted this was not going as it was supposed to. But I decided inflicting discomfort in order to get reliable information was a valid tactic.

You say that you "never said that sentence or tried to imply it..." Do you have any sense of why Ziz interpreted you as saying that? I'd like to gauge the distance between what you said and how Ziz interpreted it to gauge degree-of-untrustworthiness.

The article was the first impression I got about Ziz (I live in Germany and never have attended a CFAR workshop) and I would expect that I'm not the only person for which it's true. 

Ah, mea culpa. I saw your other comment amount Pasek crashing with you and interpreted it to mean you were pretty close to the Ziz-related part of the community. I'm less hesitant about talking to you now so I'll hop back in.

they are done because the person considers expression of their sexual of gender identity to be a sacred value. Sith robes are not expressions of their sexual of gender identity and thus taking the reputational hit for them shows valuing reputation less. 

I really feel that you're making a category error by repeatedly merging the concepts "credibility with a small set of helpful ppl" and "general reputation." I don't see why Sith Robes or gender identity or aesthetics in general should cause me to trust someone less, especially when I have other information on them I consider more relevant. This because, unlike most social conventions which serve as forms of control/submission to the mob/etc, the ability to be perceived as honest by those you want to work with allows you to more easily work with them.

Gervais sociopaths often have principles that include telling the truth.

I don't think her aesthetic was stoically motivated as much as motivated by the desire to treat ones own interests and values as logically prior to social convention -- a refusal to let ones own interests bow to the mob. This seems conceptually similar to me as treating something as a sacred value. It just has more decision theory behind it.

It's about the generator function. The question is about what generator function explains all three events

I think this is somewhat noncentral because (as mentioned), I disagree that a single generator produced all three events. What do you think the actual relevant generator is, and why do you think it also generates lie-behavior against parties Ziz might want to work with (eg publishing everyone-facing lies on the internet)?

While it likely played out worse then she expected beforehand, I don't think the idea that it was only likely to damage her reputation with the CFAR staff (whom she thinks defected) was a reasonable model of the situation.

Yeah fair enough. I agree that this isn't a reasonable model but my point still stands I think. The issue is that I neglected a third group aside from people who plan on defecting against Ziz or have low opinions of her judgement. People who automatically flinch away from others who do unconstrained things would also likely trust her less. Still, that group would be unable to help do the unconstrained things she wants to so I don't think it means much to Ziz that she can't work with them.

What group of people do you think Ziz wanted to work with that she is no longer able to because of the protest?

Since you've quoted Ziz out of context, let me finish that quote for you. It is clear that the other half of her (whatever that means) did in fact believe those things and it is clear that this was a recounting of a live-conversation rather than a broad strategy. It is not that weird to not have fully processed the things that you partially believe, live, in the middle of a conversation such that you are surprised by them.

The other half of me was like isn’t it obvious. They are disturbed at me because intense suffering is scary. Because being trans in a world where it would make things worse to transition was pain to intense for social reality to acknowledge, and therefore a threat.

I see you're equivocating between "honesty"/"credibility"  and "reputation" again:

Reading https://www.sfchronicle.com/bayarea/article/Mystery-in-Sonoma-County-after-kidnap-arrests-of-14844155.php is going to make any normal person consider the people to have no credibility and having an article like that with your legal name that people can google to find more about you in interactions like applying for a flat, is a heavy reputational cost. 

I see you've quietly dropped the two other reasons to ding credibility I mentioned to focus on the protest, which along with a misquote is your main reason for why Ziz is a liar:

False imprisonment of kids that are innocent bystanders isn't just "handles it inappropriately". None of the LGBTQ+ people I know personally have to the extend of my knowledge done something as bad nor would I expect that to be in their range of possible actions. 

It seems obvious to me that false imprisonment of kids is a noncentral description of what Ziz was doing (ie "she staged a protest and unbeknownst to her there were children somewhere" is my model). Given that this was scaled back to a misdemeanor, I imagine that you're focusing on this specifically for rhetorical effect. 

While I'm dubious about the protest being a Smart Move, I don't think this has much bearing on Ziz's honesty and I certainly don't think the coincidental presence of children somewhere in the area has any bearing on it.

From the TDT perspective actually fulfilling what you threaten seems quite reasonable.

From a TDT perspective, actually treating the first thing you come up with after someone asks you a question (esp when its couched in wiggle-terms like "probably") as a binding pre-commitment does not seem reasonable to me at all.

Given your manipulation of context above, and my notable lack of context wrt this whole situation, you have an asymmetric information advantage here that I suspect you may use to deceive me. As a result, I'm tapping out of the convo here.

If you are in good faith, I wish you well.

the completely unfounded belief that only good-aligned people can cooperate or use game theory and that nongood people will defect on each other too often to defeat her alliance. 

Can you elaborate on why you think this belief is completely unfounded? It seems to me that there are clear asymmetries in coordination capacities of good vs nongood. For example, being more open to the idea of a "Good Person" in power than a "Bad Person" seems like common sense. Similarly, groups of good people are intrinsically value-aligned while teams of bad people are not (each has a distinct selfish motivation) -- and I think value-alignedness increases effectiveness.

Assuming Ziz is being honest, she pulled the stunt at CFAR after she had already been defected against. This does not globally damage her credibility. It does damage her reputation among a) ppl who think they can't defect against her sneakily but plan to try and b) ppl who think she is bad at judging when she's been defected against. I am in neither of those categories so I have no reason to expect Ziz to defect by lying at me.

In contrast, if Ziz was being dishonest, she pulled that stunt for... inscrutable reasons that may or may not be in the web of lies she might have made. I think this is unlikely. As I've already said, her claims seem plausible and, if she was lying, she could do far worse than she did. If she wanted to defect really hard and wasn't constrained by truth, she could just raise issues that non-marginalized communities have a personal stake in (instead of something like transphobia).

Wearing Sith robes and naming themselves after a fanfic villian is similar in that it damages reputation among many people and not a strategy to develop a reputation as someone to be trusted. 

Do you think Little Nas X is less honest because he became Satan in his hit music video Montero?  I doubt that wearing Sith Robes / naming yourself after a villain (Ziz is a mythological bird?) is useful information about how honest someone is. This goes double when you know other things. Three points here:

• Marginalized people have understandable reasons for inverting mainstream aesthetics. Good/evil aesthetics are defined by  mainstream culture. If that culture has betrayed you, it can be therapeutic to reject it in turn by inverting its aesthetics. Many of my LGBTQ+ friends do this. Given that Ziz is a trans women who has an ontology that treats most ppl as "nongood", it makes sense that she would also (either because of morality-related alienation or gender-related alienation).
• You're conflating reputation/credibility among "many ppl" with reputation/credibility among ppl who would actually help you. Only the second group matters and optimizing credibility with the first is a waste of time. If you look at a marginalized person who has moral integrity (vegan) and writes extensively on TDT (solving coordination problems) and conclude that they're a liar because clothing choices, this says a lot about your values.
  You either a) buy-in to nongood mainstream clothing norms as reflecting a person's goodness/evilness or b) think Ziz is making a critical error by pushing away ppl who think this is what clothes mean.  
  IMO people who take either of these positions are sufficiently invested in the status quo that they'd only impede Ziz's work. Credibility with them isn't worth much.
• While I doubt it turned off helpful ppl, Ziz admitted (I think in a comment on sinceriously somewhere) that her aesthetic was a tactical mistake in hindsight because it attracted a bunch of edgy bad ppl. You framing it as a strategic choice is incorrect for this reason.

How do you explain those three decisions if you think that she's committed to upholding her credibility?

Well I think I've covered it above. The purpose of "Upholding Credibility" is so that the ppl you care about knowing the truth can actually receive the truth from you.  And none of the decisions above impede information-conveyance or reflect defections on the groups of ppl Ziz would be interested in helping (ethical ppl who can work with her, concerned trans women who might be at risk, etc).

...

I'll admit, I'm a little angry with your response so I might have a harsher tone here. Some of your arguments struck a nerve with me because it really feels like you're implying that a bunch of my (non-rationalist, completely unaffiliated) LGBTQ+ friends are liars. Here's what your arguments sound like to me:

• "If someone claims Something Bad is happening and then handles it inappropriately, we should give significant weight to the possibility that they are lying because handling it inappropriately was defection" 
• "When a marginalized person copes with marginalization by inverting mainstream aesthetics, we should give significant weight to the possibility that they are lying because not following the mainstream means you don't care about being taken honestly"
• "If a marginalized person claims Something Bad is happening, handles it inappropriate, and copes with marginalization by inverting mainstream aesthetics, they're systematically a liar"