Making yourself understandable to other people

(Epistemic status: Processing obvious things that have likely been written many times before, but that are still useful to have written up in my own language)

How do you act in the context of a community that is vetting constrained? I think there are fundamentally two approaches you can use to establish coordination with other parties:

1. Professionalism: Establish that you are taking concrete actions with predictable consequences that are definitely positive

2. Alignment: Establish that you are a competent actor that is acting with intentions that are aligned with the aims of others

I think a lot of the concepts around professionalism arise when you have a group of people who are trying to coordinate, but do not actually have aligned interests. In those situations you will have lots of contracts and commitments to actions that have well-specified outcomes and deviations from those outcomes are generally considered bad. It also encourages a certain suppression of agency and a fear of people doing independent optimization in a way that is not transparent to the rest of the group.

Given a lot of these drawbacks, it seems natural to aim for establishing alignment with others, it is however much less clear how to achieve that. Close group of friends can often act in alignment because they have credibly signaled to each other that they care about each others experiences and goals. This also tends to involve costly signals of sacrifice that are only economical if the goals of the participants were actually aligned. I also suspect that there is a real "merging of utility functions" going on, where close friends and partners self-modify to share each others values.

For larger groups of people, establishing alignment with each other seems much harder, in particular in the presence of adversarial actors. You can request costly signals, but it is often difficult to find good signals that are not prohibitively costly for many members of your group (this task is much easier for smaller groups, since you have less spread in the costs of different actions). You are also under much more adversarial pressure, since with more people you likely have access to more resources which attracts more adversarial actors.

I expect this is the reason why we see larger groups often default to professionalism norms with very clearly defined contracts.

I think the EA and Rationality communities have historically optimized hard for alignment and not professionalism, since that enabled much better overall coordination, but as the community grew and attracted more adversarial actors those methods didn't scale very well and so we currently expect alignment-level coordination capabilities while only having access to professionalism-level coordination protocols and resources.

We've also seen an increase in people trying to increase the amount of alignment, by looking into things like circling and specializing in mediation and facilitation, which I think is pretty promising and has some decent traction.

I also think there is a lot of value in building better infrastructure and tools for more "professionalism" style interactions, where people offer concrete services with bounded upside. A lot of my thinking on the importance of accountability derives from this perspective.

nicoleross

nicoleross

nicoleross

nicoleross

nicoleross

nicoleross