How valid is it to assume that (approximately) everyone who got the heliocentrism question wrong got it wrong by "guessing"? If 18% got it wrong, then your model says that there's 36% who had no clue and half guessed right, but at the other extreme there's a model that everyone 'knows' the answer, but 18% 'know' the wrong answer. I'm not sure which is scarier - 36% clueless or 18% die-hard geocentrists - but I don't think we have enough information here to tell where on that spectrum it is. (In particular, if "I don't know" was an option and only 3% selected it, then I think this is some evidence against the extreme end of 36% clueless?)
Too Like the Lightning by Ada Palmer :)
with the following assumptions:
Should the ∨ in assumption 1 be an ∧?
Cool idea!
One note about this:
Let's see what happens if I tweak the language: ... Neat! It's picked up on a lot of nuance implied by saying "important" rather than "matters".
Don't forget that people trying to extrapolate from your five words have not seen any alternate wordings you were considering. The LLM could more easily pick up on the nuance there because it was shown both wordings and asked to contrast them. So if you actually want to use this technique to figure out what someone will take away from your five words, maybe ask the LLM about each possible wording in a separate sandbox rather than a single conversation.
US Department of Transportation, as I’m sometimes bold enough to call them
I assume you intended to introduce your "US DoT" abbreviation here?
Oh, and as an aside a practical experiment I ran back in the day by accident: I played in a series of Diplomacy games where there was common knowledge that if I ever broke my word on anything all the other players would gang up on me, and I still won or was in a 2-way draw (out of 6-7 players) most of the time. If you have a sufficient tactical and strategic advantage (aka are sufficiently in-context smarter) then a lie detector won’t stop you.
I'm not sure this is evidence for what you're using it for? Giving up the ability to lie is a disadvantage, but you did get in exchange the ability to be trusted, which is a possibly-larger advantage - there are moves which are powerful but leave you open to backstabbing; other alliances can't take those moves and yours can.
Taken together, the two linked markets say there's a significant chance that the House does absolutely nothing for multiple weeks (i.e. they don't elect a new speaker and they don't conduct legislative business either). I guess this is possible but I don't think we're that dysfunctional and will bet against that result when my next Manifold loan comes in.
My guess at the truth of the matter is that almost no one is 100% guessing, but some people are extremely confident in their answer (a lot of the correct folks and also a small number of die-hard geocentrists), and then there's a range down to people who haven't thought about it in ages and just have a vague recollection of some elementary school teacher. Which I think is also a more hopeful picture than either the 36% clueless or the 18% geocentrists models? Because for people who are right but not confident, I'm reasonably ok with that; ideally they'd "know" more strongly, but it's not a disaster if they don't. And for people who are wrong but not confident, there are not that many of them and also they would happily change their mind if you just told them the correct answer.