[ Question ]

Why is so much discussion happening in private Google Docs?

byWei_Dai2mo12th Jan 201921 comments

74


I've noticed that when I've been invited to view and comment on AI-safety related draft articles (in Google Docs), they tend to quickly attract a lot of extensive discussion, including from people who almost never participate on public forums like LessWrong or AI Alignment Forum. The number of comments is often an order of magnitude higher than a typical post on the Alignment Forum. (Some of these are just pointing out typos and the like, but there's still a lot of substantial discussions.) This seems kind of wasteful because many of the comments do not end up being reflected in the final document so the ideas and arguments in them never end up being seen by the public (e.g., because the author disagrees with them, or doesn't want to include them due to length). So I guess I have a number of related questions:

  1. What is it about these Google Docs that makes people so willing to participate in discussing them?

  2. Would the same level of discussion happen if the same draft authors were to present their drafts for discussion in public?

  3. Is there a way to attract this kind of discussion/participants to public posts in general (i.e., not necessarily drafts)?

  4. Is there some other way to prevent those ideas/arguments from "going to waste"?

  5. I just remembered that LessWrong has a sharable drafts feature. (Where I think the initially private comments can be later made public?) Is anyone using this? If not, why?

Personally I much prefer to comment in public places, due to not wanting my comments to be "wasted", so I'm having trouble understanding the psychology of people who seem to prefer the opposite.

7 Answers

ESRogs

Jan 12, 2019

35
What is it about these Google Docs that makes people so willing to participate in discussing them?

I would guess the main reasons are:

1. Privacy concerns -- preferring not to go on-the-record with an off-the-cuff thought, especially if representing an organization.

2. The ability to comment on a specific line in a document, with the comment showing up in context.


Raemon

Jan 12, 2019

22

This is a major issue the LessWrong team has thought about a lot. Building off ESRogs' and Elo's answers, the two main reasons are:

1) Privacy, comfort, and filtering.

2) Google Docs has great features and the best collaborative exobrain product.

Privacy/Comfort/Filtering

I think google docs ends up having the best overall conversation, but there is even a marked difference in feel between public LessWrong discussion and, say, Facebook discussion (holding participants constant).

Part of this is privacy concerns like ESRogs notes. If you represent an org maintaining a reputation, or you are discussing infohazardy material, or you have semi-secret information, you simply can't chat freely in public – you have to think carefully about the ramifications of sharing your ideas. This makes sharing ideas harder, and sometimes unpleasant, so people end up doing it less. (semi-private venues like facebook, and many google docs, obviously carry at least some risk, but the risk feels a lot less pronounced).

But I think for a lot of documents, it's less about literal secrecy or organizational PR, and more about an overall sense of safety/trust/filtering. In private you don't have to worry as much about being wrong.

And, to some degree, "fear of being wrong" is something worth getting over. But there are practical matters that mean public discussion will always have some downsides.

In a public space, if you make an off-the-cuff remark that's not fully defensible, you'll often get people critiquing you who miss the point of what you're actually trying to say, or (worse) are just actively trying to take you down a peg for social reasons. And you have to either spend a bunch of time arguing about something that wasn't even the main idea you were trying to clarify or hash out, or not reply and look/feel like you're ignoring a critic.

(And because of inferential distance, the critique might look reasonable to an onlooker, whereas you've already had this conversation 10 times and know the various arguments and counterarguments, and don't feel like patiently explaining it again).

Abram's recent post on intellectual cultures explores the details here. There's a rough hierarchy of discussion that goes something like:

  • 0. Open Verbal Combat
  • 1. Face Culture.
  • 2. Intellectual Debate
  • 3. Mutual Curiosity
  • 4. Exchanging Gears

Venues like LessWrong have a mix of people, some of whom are trying to do intellectual debate, some mutual curiosity, some exchanging gears (and occasional people doing pure verbal combat). Google docs lets you filter for people who are trying to do the kind of conversation you're looking for.

Collaborative Exobrain Technology

Google docs is legitimately great. Important features include inline commenting, realtime collaboration, suggesting text, version history, etc.

These individual features are good, and they add up to a really powerful "group exobrain tool." As of now Google Docs are simply much better than most other forum / blogpost technology for maximizing group-working-memory, and alternating between the "generation" step, "critique and discussion" step and the "distill" step of intellectual progress.

Some things that work better on google docs than many alternatives:

a. Peer Review.

The use case it sounded like you were most referring to. You can write up an initial paper exploring an idea. You invite smart people to comment on it. You might invite disparate groups of people at once, or you might take turns sending it to different groups of people who you think will do a good job of critiquing different aspects of the idea, or bring different perspectives.

The comments are attached to specific parts of the essay. In addition to immediately seeing the context of a given comment, I think there's something like "humans using physical space to augment their working memory" (similar to memory palaces) where having a "location" for each comment helps people keep track of everything going on.

You can also give people the ability to suggest text, and then have people comment on those suggestions. So brainstorming and critique seamlessly flow back into the rewrite process. Whereas on a blog, if people comment, then the author needs to set aside time and activation energy to turn all those comments into a second draft, and then wait for further comments on whether that second draft actually addressed the issues.

(Plus, there's something that feels "final" about posting something publicly, so doing a "second draft" feels a bit weird in the first place)

On google docs, the second draft happens sort of invisibly as you go along.

b. Planning.

Andrew Critch has been a huge proponent of using Google Docs to develop plans –if you have a course of action, be it a life plan, new project, or whatever, you can write it down.

Initially, writing it down forces you (at least somewhat) to notice what assumptions you've made, and make sure each step of the plan actually follows from the previous step.

Then, you show it to a couple people. At first it may not be very good so you just share it with one or two people as an overall sanity check. If you can find 4 people with different perspectives to vet the plan, and check assumptions you missed the first time, you can be a lot more confident that the plan makes sense.

The brainstorm/critique/distill/next-draft cycle applies to plans as well as idea-based-writing. But, in this case, the final product is not a paper to read – it's a course of action that people can commit to. If a plan calls for multiple people to commit to a particular role, then part of the drafting process is to make sure each role makes sense... and to find people who would be willing to do it.

Just like a google-docs paper ends with you suddenly realizing that you have a finished draft, with little activation energy required to commit the final comments, ideally a google-docs plan ends with you realizing you have a viable strategy and a bunch of buy-in from people who helped shape it and are ready to start turning it into actual reality.

So what do we make of this?

I still think you're right, that keeping all these comments hidden in drafts is wasting a lot of potential. I think there are potential solutions, but they require a fair bit of work, and accepting some of the constraints. People want the privacy and filtering while they're hashing their ideas out.

The main thing lacking with google docs is the ability to turn, not just the finished writeup, but all the corresponding comments, into a publicly accessible work, with a click of a button.

(The LessWrong team is looking into some options for making LessWrong posts more google-docs-like, so that people and orgs can use it for internal documents while significantly lowering the barrier-to-entry to sharing them eventually, but it's a big chunk of work and not something we can promise in the immediate future)


Elo

Jan 12, 2019

21

In a Google doc, a comment is perceived by both the author and the commenter as intending the be helpful (collaborative culture). In a lw post comments, there is often adversarial culture (you are wrong in ways xyz). It's less fun to be around and it's not as productive at encouraging people to participate.


Thrasymachus

Jan 12, 2019

17

I'm someone who both prefers and practises the 'status quo'.

My impression is the key feature of this is limited (and author controlled) sharing. (There are other nifty features for things like gdocs - e.g. commenting 'on a line' - but this practice predates gdocs). The key benefits for 'me as author' are these:

1. I can target the best critics: I usually have a good idea of who is likely to help make my work better. If I broadcast, the mean quality of feedback almost certainly goes down.

2. I can leverage existing relationships: The implicit promise if I send out a draft to someone for feedback is I will engage with their criticism seriously (in contrast, there's no obligation that I 'should' respond to every critical comment on a post I write). This both encourages them to do so, and may help further foster a collegial relationship going forward.

3. I can mess up privately: If what I write makes a critical (or embarrassing) mistake, or could be construed to say something objectionable, I'd prefer this be caught in private rather than my failing being on the public record for as long as there's an internet archive (or someone inclined to take screen shots). (This community is no stranger to people - insiders or outsiders - publishing mordant criticisms of remarks made 'off the cuff' to infer serious faults in the speaker).

I also think the current status quo is a pretty good one from an ecosystem wide perspective too: I think there's a useful division of labour between 'early stage' writings to be refined by a smaller network with lower stakes, and 'final publications' which the author implicitly offers an assurance (backed by their reputation) that the work is a valuable contribution to the epistemic commons.

For most work there is a 'refining' stage, which is better done by smaller pre-selected networks rather than of authors and critics mutually 'shouting into the void' (from the author's side, there will likely be a fair amount of annoying/irrelevant/rubbish criticism; from a critic's side, a fair risk your careful remarks will be ignored or brushed off).

Publication seems to be better for polished or refined work, as at this stage a) it hopefully it has fewer mistakes and so generally more valuable to the non-critical reader, b) if there is a key mistake/objection neglected (e.g. because the pre-selected network resulted in an echo chamber) disagreement between ('steel-manned') positions registered publicly and hashed seems a useful exercise. (I'm generally a fan of more 'adversarial' - or at least 'adversarial-tolerant' norms for public discussion for this reason.)

This isn't perfect, although I don't see the 'comments going to waste' issue as the greatest challenge (one can adapt one's private comments to a public one to post, although I appreciate this is a costlier route than initially writing the public comment - ultimately, if one finds ones private feedback is repeatedly neglected, one can decline to provide it in the first place).

The biggest one I see is the risk of people who can benefit from a 'selective high-quality feedback network' (either contributing useful early stage criticism, having good early stage posts, or both) not being able to enter one. Yet so long as members of existing ones still 'keep an eye out' for posts and comments from 'outsiders', this does provide a means for such people to build up a reputation to be included in future (i.e. if Alice sees Bob make good remarks etc., she's more interested in 'running a draft by him' next time, or to respond positively if Bob asks her to look something over).


Elo

Jan 12, 2019

13

Safety of draft-land. There's no need to worry about the long term effects of being wrong in draft format. There is a reason to be worried about being wrong in published format.

(I definitely am guilty of discussing by Google doc. I have drafts unpublished in doc form right now)


Lukas_Gloor

Jan 12, 2019

9

For me there's a huge difference between these two.

  • In gdocs I feel like it's more okay to write "unpolished" comments. I think that's mostly because the expectations are lower. Polishing my comments takes me 3-5x longer, which often takes away the motivation to comment at all.
  • In a public forum I worry more about provoking misleading impressions. For instance, in a gdoc shared with people who know me well, I'm not worried that a comment like "AIs might do [complex sequence of actions]" will get people to think that I have weirdly confident views about how the future might play out. In public conversations I'd experience a strong urge to qualify statements like that even though it feels tedious to do so.

Dr. Jamchie

Jan 12, 2019

2

A guess: google docs comments does not have karma system, so participants are free to tell what they want without worrying about losing points.