A similar calibration game I like to play with my girlfriend: one of us gives our 80% confidence interval for some quantity (eg "how long will it take us to get to the front of this line?") and the other offers to bet on the inside or the outside, at 4:1 odds.

I've learned that my 80% intervals are right like 50% of the time, almost always in favor of being too optimistic...

With my wife, I do it a little differently. Once a week or so, when the kids have fallen asleep, we’ll lie in separate beds—Johanna next to the baby, and me next to the 5-year-old. We’ll both be staring at our screens. Unlike the notes I keep with Torbjörn, these notes are shared. They are a bunch of Google docs.

This reminds me of the note-taking culture we have at Manifold, on Notion (which I would highly recommend as an alternative to Google docs -- much more structured, easier to navigate and link between things, prettier!)

For example, while we do our daily standup meetings, we're all jotting thoughts into our meeting notes, and often move between linked documents. To track who has been having which thought, we'll prefix a particular bullet point with your initials e.g. "[A] Should we consider moving to transactions?"

Thanks for writing this up! I've just added AI Impacts to Manifold's charity list, so you can now donate your mana there too :)

I find the move from "website" to "wiki" very interesting. We've been exploring something similar for Manifold's Help & About pages. Right now, they're backed by an internal Notion wiki and proxied via super.so, but our pages are kind of clunky; plus we'd like to open it up to allow our power users to contribute. We've been exploring existing wiki solutions (looks like AI Impacts is on DokuWiki?) but it feels like most public wiki software was designed 10+ years ago, whereas modern software like Notion is generally targeted for the internal use case. I would also note that LessWrong seems to have moved away from having an internal wiki, too. There's some chance Manifold ends up building an in-house solution for this, on top of our existing editor...

Definitely agreed that the bottleneck is mostly having good questions! One way I often think about this is, a prediction market question conveys many bits of information about the world, while the answer tends to convey very few.

Part of the goal with Manifold is to encourage as many questions as possible, lowering the barrier to question creation by making it fast and easy and (basically) free. But sometimes this does lead to people asking questions that have wide appeal but are less useful (like the ones you identified above), whereas generating really good questions often requires deep subject-matter expertise. If you have eg a list of operationalized questions, we're always more than happy to promote them to our forecasters!

Re your second point (score rather than ranking basketball players), Neel Nanda has the same advice which I've found fairly helpful for all kinds of assessment tasks: https://www.neelnanda.io/blog/48-rating

It makes me much more excited for eg 5-star voting instead of approval or especially ranked choice voting.

Big fan of the concept! Unfortunately, Manifold seems too dynamic for this extension (using the extension seems to break our site very quickly) but I really like the idea of temporarily hiding our market % so you can form an opinion before placing a bet:

Really appreciate this list!

Things I very much agree with:

4. Have a single day, e.g. Tuesday, that’s the “meeting day”, where people are expected to schedule any miscellaneous, external meetings (e.g. giving someone career advice, or grabbing coffee with a contact).

12. Have a “team_transparency@companyname” email address, which is such that when someone CC’s it on an email, the email gets forwarded to a designated slack channel

17. Have regular 1-1s with the people you work with. Some considerations only get verbalised via meandering, verbal conversation. Don’t kill it with process or time-bounds.

Things I'm very unsure about:

8. Use a real-time chat platform like Slack to communicate (except for in-person communication). For god’s sake, never use email within the team.

I actually often wonder whether Slack (or in our case, Discord) optimizes for writeability at the cost of readability. Meaning, something more asynchronous like Notion, or maybe the LessWrong forum/Manifold site, would be a better system of documenting decisions and conversations -- chat is really easy to reach for and addictive, but does a terrible job of exposing history for people who aren't immediately reading along. In contrast, Manifold's standup and meeting calendar helps organize and spread info across the team in a way that's much more manageable than Discord channels.

14. Everyone on your team should be full-time

Definitely agree that 40h is much more than 2x 20h, but also sometimes we just don't have that much of certain kinds of work, slash really good people have other things to do with their lives?

Things we don't do at all

5. No remote work.

Not sure how a hypothetical Manifold that was fully in-person would perform -- it's very unclear if our company could even have existed, given that the cofounders are split across two cities haha. Being remote forces us to add processes (like a daily hour-long sync) that an in-person team can squeak by without, but also I think has led to a much better online community of Manifold users because we dogfood the remote nature of work so heavily.

Finally: could you describe some impressive things that Lightcone has accomplished using this methodology? I wonder if this is suited to particular kinds of work (eg ops, events, facilities) and less so others (software engineering, eg LessWrong doesn't seem to do this as much?)

Rob Wiblin from 80k asks:

Inositol, I believe: https://www.facebook.com/100000020495165/posts/4855425464468089/?app=fbl

I've been following the SMTM hypothesis with great interest; don't have much to add on a technical level, but I'm happy to pay a $200 bounty in M$ to Natália in recognition of her excellent writeup here.  Also - happy to match (in M$) any of the bounties that she outlined!