My first impulse is to focus on the question "is it true?", because it's simple, memorable, and broadly useful. It also seems likely that it could have prompted this questionnaire. You can elaborate that this is explicitly meant to replace the question "do I like this?", which is often what people default to instead of "is it true?" when reacting to new claims. (You can imagine the big discussion without any data, and there would surely be some posturing of the "well, what we do is better than Dragoncon along some dimension" variety.)
"Is it true?" isn't specific enough.
It's almost certainly true that the author would rather promote his books at Dragoncon. I think the right question is closer to "How can we find out what this true thing might imply?".
Yeah, I like "How would I notice?" a lot.
So, if I have a belief, or if someone else mentions one, I think "How would I notice?" and try to imagine, if I wandered into this topic with no background knowledge, what I would encounter naturally or have to research in order to decide the question.
I also like it because it mutes any feelings of defensiveness (for me anyway) that "Oh yeah, prove it!" usually implies. How would I notice also gives me room to mention things I'd notice that would point me the other way, since it feels more like a walk/story/narrative than an argument.
"Is it true?" isn't specific enough.
Agreed. The reason I stuck with the simpler question is because you need to move your audience from A to B, not A to C. If you get them to B, then I think that can also get them to C, as that question can be applied to whatever reaction they have to the author's preference. If they think "it's true that the author prefers to promote his books at Dragoncon, and so he'll never get value out of my convention," they can respond "is that true?". (Your question is a more targeted version of that.)
The question I try to focus on isn't "Is it true," or even "Will it work," but "Do I care." Will this thing help me win? How does it relate to my life and my goals?
If there's no immediate connection to a project I'm working on, or a skill that's common across things I do, or a general human skill (like memory or motivation), I immediately discard it.
In this case: Why do I care about this author's belief? My goals aren't tied to his ability to sell books. Why spend the time trying to convince him he should test his belief, testing it for him, and helping him integrate the results? I have only so many keystrokes, and I wouldn't waste them on him.
Why do I care about this author's belief?
Because you want this author to come to your science fiction convention.
Did you used to be unselective about the information you took in?
If "Do I care?" has been a significant change, how has it been working out for you?
My point wasn't specifically about science fiction authors and conventions, it was about getting from "We've been insulted! We're losing out!" to "What's actually going on?".
Yes, I used to waste a lot more time than I do now. I still waste time, but I try to be more productive in my recreation time and I've successfully cut down on unproductive waste and increased "productive procrastination."
I certainly feel less streched and am getting more done, so it's been working well.
I don't think anything in your post is invalid; I was just making an observation. Being able to get from "We've been insulted" to "Let's think quantitatively about the situation" is valuable when you do care.
I'm on a mailing list for people who run science fiction conventions*, and an author had said that he prefers going to Dragoncon because it's a better place to promote his books.
This has some angstful possibilities because Dragoncon is much larger than the sort of conventions the people on the mailing list run, and structured somewhat differently.
I suggested doing a survey of sf authors about how they promote their books at conventions, why they've made the choices they do, and how it's been working out. Other people are working on the questionnaire, and it's planned to be out within six months or so.
The rationality part is that it occurred to me that we needed more information before we did a big discussion about what's happening with authors at amateur-run conventions, and that it was possible to get a fair amount of that information.
I'm considering posting something about how I learned those mental habits, but I'm not sure how I did, though I believe it has something to do with hanging around Less Wrong. Any recommendations for Practical Rationality 101?
*I don't run conventions, I just kibitz.