This is another entry in the category of "attempting to correct people on using technical terminology while not understanding the point of having catchy jargon in the first place and so the supposed improvements or equivalent formulations are neither".
Imagine one is not a rationalist, and totally unfamiliar with Scott’s writing, and you read something like “1.8% of 25-45 year olds with covid [develop] long covid that affects their daily life, which is well within the Lizardman Constant”.[3] Are you likely to know what that means? Compare instead reading an academic article that says: “[t]his makes the samples vulnerable to fake or bogus respondents.” I think most people would readily understand the latter—a fake or bogus respondent is someone that responds in a false or ‘bogus’ way, if a study is ‘vulnerable’ to that, it means that the apparent effects may be the result of bogus respondents. But “Lizardman constant” is not readily understandable to the lay person; it describes the same thing but uses an obscure jargon term instead.
This manages to be both wrong and miss the point. 'Fake or bogus' is not understandable, and it is not a substitute for a specific term. Most people might think they understand the latter, but they don't. It is an 'illusion of transparency'. It does not cover the full meaning of the term, which includes malicious respondents, rushed respondents, good-faith but overconfident or deluded respondents, finger or vocal slips, etc. (You say all that is implied by 'fake or bogus'? Well, what does 'bogus' mean? 'not genuine; counterfeit, sham'. Oh, thank you, that totally cleared the matter up! I'll be sure to explain things like "science" using this word. "Science isn't hard, it's just a way of coming to beliefs which aren't bogus. You understand everything about it now, right, like every kind of error which might affect a survey?") It also completely omits the main meaning which was not 'bad responses exist' - who could ever have doubted that? - but that the badness is in pretty much every survey at nontrivial percentages. (Note the completely different meaning of 'constant' to 'vulnerability'. A constant is always present. A vulnerability is merely a potential.)
Second, it misses the point of coining a term. 'vulnerable to fake or bogus respondents' is not terminology. It is a wordy ad hoc circumlocution made up on the spot to deal with the fact that the authors and audiences do not share a single crisp clear term for the general recurring problem and so cannot easily talk about it or remember it. Every time they want to talk about it, they have to make up a new phrase and it'll be different. 'contaminated by unserious responses'. 'Measurement error in noisy samples'. 'Mischievous responders'. 'Trolls'. Meanwhile, a Scott Alexander reader can just say, 'Lizardman constant'. And it is instantly memorable (every reader has memorized it after about 1 screen of preface in the original post and still remembers it despite it being from 2013), searchable, linkable, and consistently employed.
but more egregiously it is wrong! It isn’t a constant and writers using the jargon are led to at best misleading conclusions. The prior example continues: “The Lizardman Constant doesn’t mean prevalences below 4% don’t exist, it means they’re impossible to measure using naive tools.” This is just wrong, prevalence of under 4% can be measured and the tools being used here are fit for purpose! If one engaged with the literature on bogus respondents this would become clear.
What "naive tools" let you defend against Lizardman Constant and safely measure prevalences <4% without systematic bias being a large component?
Probabilistic sampling, and using verified data can help manage the risks.[4] How you write a questionnaire, how you solicit respondents, and numerous other factors can greatly increase or decrease the rates of bogus respondents.
All that seems reasonable and what an expert aware of the Lizardman Constant and the 'numerous other factors' might or might not be able to fix. But in what sense do naive tools do all that for you?
As a case example, let’s look at the particular study being referenced.[5] It is a UK metareview of 10 longitudinal studies using in-patient and primary care diagnosis data along with patient self-reported information. If it is answering a poll on twitter, the rate of people pressing a random answer here or there, or just choosing whatever they think is funniest, may be very high. But what is the risk of bogus respondents of patients filling out surveys including their symptoms—at repeated intervals—with the patients matched against diagnosis records? The risk there is negligible—people are incentivized to report honestly and are not taken at random but verified using medical records. There are a host of other problems that might result in false positives (e.g., nocebo effects), but the risk of bogus respondents is incredibly low.
This is all handwaving. You just think that the survey must be accurate. You don't provide any non-naive tools showing it is accurate and has non-existent Lizardman problems. And EHRs are well known to have a ton of data quality problems, and as for Long Covid self-reports (to specify what the topic was, which you left out), well, I don't think I really need to say anything at this point... I doubt that the garbage in it is "incredibly low"
There are plenty of other cases of jargon, which I would classify more as an issue of over-pretentious speech and writing. These are more typical foibles and hardly unique to rationalists. To give but one minor example, using “Pons Asinorum” in place of “foundational challenge”. Using jargon and scientific language that serves to further clarity is fine, but should be avoided in cases where plain English is both clearer and more accessible.
'pons asinorum' is not reducible to a phrase like 'foundational challenge', and Yudkowsky's use is both correct and clearer than your suggestion. 'Foundational challenge' could mean just about anything hard and important (and usually unsolved).
attempting to correct people on using technical terminology while not understanding the point of having catchy jargon in the first place and so the supposed improvements or equivalent formulations are neither
As I said, the jargon would be "fine" if it was correct. It would even be preferable if it was catchy and provided clarity. The problem is it is less clear and leads to mistakes. Edit: to be clear, this a mistake you seem to affirm "A constant is always present." The problem is it is not always present in "non trivial" portions. The rate of bogus respondents in longitudinal, externally matched sample sets is trivial (a fraction of a percent).
'Fake or bogus' is not understandable
I am sorry, I am not sure what you're trying to claim here. Do you not understand what is meant by the term "bogus respondent"?
Well, what does 'bogus' mean? 'not genuine;
Correct, a bogus respondent is a respondent that does not provide provide a genuine response. There are various ways of coding for them in academic literature.
Most people might think they understand the latter, but they don't.
There are obviously levels of understanding. A lay person can readily understand the gist of what is being said. If they want to know more about bogus respondents, their are hundreds of textbooks, journal articles and a wealth of literature on what causes bogus respondents.
It does not cover the full meaning of the term, which includes malicious respondents, rushed respondents, good-faith but overconfident or deluded respondents, finger or vocal slips, etc
"Good-faith but overconfident or deluded respondents" are not bogus respondents. All it means is where the responses do not match the views of a respondent. If someone's response is genuine, they are not a bogus respondent.
Very quickly, to illustrate the point, imagine running a survey of 'do you believe the earth is flat or round?" Respondents that think the earth is flat and say they think it is flat are genuine respondents, just wrong. Respondents that say something other than their belief (or are unable to reflect a belief, such as with bot responses) are 'bogus'.

It is a wordy ad hoc circumlocution made up on the spot to deal with the fact that the authors and audiences do not share a single crisp clear term for the general recurring problem and so cannot easily talk about it or remember it.
No, the term "bogus respondents" is in academic literature going back decades. It is not a new ad hoc term. You will see it regularly in text books and guides for producing surveys and utilizing survey statistics.
Meanwhile, a Scott Alexander reader can just say, 'Lizardman constant'. And it is instantly memorable (every reader has memorized it after about 1 screen of preface in the original post and still remembers it despite it being from 2013), searchable, linkable, and consistently employed.
But wrong (or at least leads to erroneous understandings since the literally meaning of the terms is incorrect). If you had a catchy mutually understood term that was correct, clear and prevented misunderstandings I would give it the thumbs up, but as I said, that isn't the case. There is no constant that describes rates of bogus respondents. I put some literature in another comment, but there is a huge wealth of literature on the topic. It is trivial to get extremely high or very low rates of bogus respondents. There is no constant. You need to assess the risk on a case by case basis by examining the survey design, the sample, what validations are in place, and data cleaning that is performed. You cannot hand wave away results as being "within the Lizardman constant."
This is all handwaving. You just think that the survey must be accurate.
That is not what I think. I am not sure how you read that in my statement. My view is:
That doesn't mean the study is accurate. As I said "There are a host of other problems that might result in false positives (e.g., nocebo effects), but the risk of bogus respondents is incredibly low."
to specify what the topic was, which you left out
I included it in the quote I provided (pictured below). I did not feel the need to repeat the context. My apologies if that caused any confusion, but it was plainly not an omission on my part.

And EHRs are well known to have a ton of data quality problems
Hence why I said: "There are a host of other problems that might result in false positives (e.g., nocebo effects), but the risk of bogus respondents is incredibly low." Just hand waving "lizardmen constant" is not a problem. That isn't an issue for the study.
doubt that the garbage in it is "incredibly low"
I am going to repeat myself: "There are a host of other problems that might result in false positives (e.g., nocebo effects), but the risk of bogus respondents is incredibly low."
What "naive tools" let you defend against Lizardman Constant and safely measure prevalences <4% without systematic bias being a large component?
Systemic bias is a huge range of issues. I would consider a naive survey inherently dubious. To quote the American statistical society: "The quality of a survey is best judged not by its size, scope, or prominence, but by how much attention is given to [preventing, measuring and] dealing with the many important problems that can arise." IN some surveys, bogus respondents are a problem, how much of a problem they are liable to be depends on survey design (I assume you do not want a lecture about what survey designs are more susceptible). One might for perfectly valid reasons choose survey designs that are more susceptible to bogus respondents, in that case additional controls need to be put in place. In the cases of non-probabilistic opt-in surveys, there rates of bogus respondents typically range from 4-7%. In the case of longitudinal surveys of patients matched against medical records, the rates of bogus respondents is largely negligible. That doesn't mean there isn't systematic bias. There are a lot of other sources of bias in data, you could have sampling errors, or results could arise from other factors (to give the same example, again, nocebo effects is a possible source).
'pons asinorum' is not reducible to a phrase like 'foundational challenge', and Yudkowsky's use is both correct and clearer than your suggestion
My reading of his statement was that he just meant it was a foundational problem in the field. If he meant something else, I would consider that just illustrate of the lack of clarity.
For example, if we look at the same pew research report that found bogus respondent rates of around 4-7% in non-probabilistic opt-in surveys, they found 0% of the comparison group, of address-matched panel data, exhibited some of the same bogus typologies (which 2-4%. of the opt-in samples exhibited). "2% to 4% of opt-in poll respondents repeatedly gave answers that did not match the question asked, compared to 0% of address-recruited panel respondents."
Strongly upvoted for a series of concrete, object-level arguments with reference to literature. Unclear why people perceive this comment as emotional. The use of "lizardman constant" here seems to conflate many sources of measurement error in surveys in a way that it didn't upon the original definition.
Thanks!
Unclear why people perceive this comment as emotional
If I had to guess, I think using bold may have given that impression style wise? Also, I was pretty blunt in places--I do call him flat out 'wrong' . I wanted to highlight those things as I believed gwern was fundamentally misunderstanding what I said (e.g. his statement "You just think that the survey must be accurate" is just wrong, I do not think that).
The use of "lizardman constant" here seems to conflate many sources of measurement error in surveys in a way that it didn't upon the original definition.
I do think I made a mistake by not including a discussion of sources of issues and terminology in literature in my original post and I took for granted, (which, in my defense, gwern does as well though I think he gets it wrong), that readers would be familiar with the description Scott Alexander gave in his 2013 essay. I am doing another write up which I think I will make into a post trying to explain better how to think about survey data (it is something I work with a lot professionally and am quite familiar with).
I thought it would be more intuitive to a reader that the issues a non-probabilistic opt-in survey is vulnerable to (including bogus respondents) are not interchangeable with the issues a metareview of longitudinal surveys matched against diagnosis data are likely to face.
I should probably have gone through and edited it more or at least done another pass or two (unfortunately, I am behind on more important IRL things), but if you are curious I did the write up I was thinking of here: https://www.lesswrong.com/posts/kvtE9Md9Z8i8PyLm6/lizardmen-are-not-constant-a-very-brief-introductory-primer
There are plenty of valid cases one might make to refute the argument presented in the ~150 word paragraph in the example. But none that I can think of would include a 10k word (deliberately, I assume) cliche piece of fictional narrative that has a “midwit” espouse a view somewhat similar (but notably distinct from) the view being refuted, just so they can be torn down in your fictional conceit,
A reminder that Eliezer also wrote to beware fictional evidence.
Thank you! I hadn't seen that piece of his. I absolutely agree with him there, though he actual even goes a bit further than I would in places (example below). Case studies (even fictional ones) can be useful for illustrating examples and providing more engaging moral lessons but they should not be used in place of actually engaging with an argument on its terms (or refuting the terms, oc). It seems there is a habit of doing just that in some spaces that has developed.
I would *suspect* in Yudowsky's case it is in large part driven by there being more demands on his time, as he has become more popular and taken more roles in his organizations and advocacy.
A story is never a rational attempt at analysis, not even with the most diligent science fiction writers, because stories don’t use probability distributions. I illustrate as follows...
This seems to me to be a bit too strong of a case. You can have a story that contains a rational attempt at analysis and not all rational analyses, imo, require a formal probability distribution (as a side point, often, I would actually argue, formal probability distributions are better avoided where there is not strong enough evidence to justify preferring one distribution to another, besides different a priori intuitions).
I realized I was rather brief in assuming others would be able to recognize how the bogus respondent rates trivially vary. The cited Pew article somewhat discusses it, but below is some other literature (going back decades) on the topic. I didn't want to get too side tracked, but can expand on it, if helpful.
Some possible readings:
https://doi.org/10.1111/j.1360-0443.1987.tb03909.x (particularly on how longitudinal data is more reliable, of particular relevance to the example)
https://doi.org/10.1080/00224545.1989.9711721 (focuses on field surveys and methods for reducing bias)
https://www.jstor.org/stable/2090467
https://doi.org/10.1093/poq/nfad037 (focuses on problems with non probabilistic surveys)
-George Orwell, Politics and the English Language[1]
I will begin with admission, one could certainly find ready-made examples of hypocrisy on my own part—I would welcome it as a constructive critique—but it seems to me that much of ‘rationalist’ literature and writing conventions is plagued by harmful conventions, jargon and cliches. I may seem overly harsh in some places, for that I would offer a pre-emptive apology: these are by no means mortal foibles.
The most prominent example, featured on Scott Alexander’s wiki page, that particularly bothers me[2] is that of the ‘lizardman constant’--not only is it an unhelpful jargon but it is foundationally wrong. Imagine one is not a rationalist, and totally unfamiliar with Scott’s writing, and you read something like “1.8% of 25-45 year olds with covid [develop] long covid that affects their daily life, which is well within the Lizardman Constant”.[3] Are you likely to know what that means? Compare instead reading an academic article that says: “[t]his makes the samples vulnerable to fake or bogus respondents.” I think most people would readily understand the latter—a fake or bogus respondent is someone that responds in a false or ‘bogus’ way, if a study is ‘vulnerable’ to that, it means that the apparent effects may be the result of bogus respondents. But “Lizardman constant” is not readily understandable to the lay person; it describes the same thing but uses an obscure jargon term instead.
On its own, I would find this a somewhat forgivable fault of in-culture terminology (like using ‘grok’ to mean ‘understand’), but more egregiously it is wrong! It isn’t a constant and writers using the jargon are led to at best misleading conclusions. The prior example continues: “The Lizardman Constant doesn’t mean prevalences below 4% don’t exist, it means they’re impossible to measure using naive tools.” This is just wrong, prevalence of under 4% can be measured and the tools being used here are fit for purpose! If one engaged with the literature on bogus respondents this would become clear.
Research on non-probabilistic, online polls commonly finds rates of bogus respondents between 4-7%, but this is highly variable and can be mitigated.[4] Probabilistic sampling, and using verified data can help manage the risks.[5] How you write a questionnaire, how you solicit respondents, and numerous other factors can greatly increase or decrease the rates of bogus respondents. If you want to assess the risk of bogus respondents to a result just going ‘oh it’s 4%, Scott Alexander said ‘the Lizardman constant is 4%’ so we can assume this result could be explained by the Lizardman constant’ is just wrong.
As a case example, let’s look at the particular study being referenced.[6] It is a UK metareview of 10 longitudinal studies using in-patient and primary care diagnosis data along with patient self-reported information. If it is answering a poll on twitter, the rate of people pressing a random answer here or there, or just choosing whatever they think is funniest, may be very high. But what is the risk of bogus respondents of patients filling out surveys including their symptoms—at repeated intervals—with the patients matched against diagnosis records? The risk there is negligible—people are incentivized to report honestly and are not taken at random but verified using medical records. There are a host of other problems that might result in false positives (e.g., nocebo effects), but the risk of bogus respondents is incredibly low.
There are plenty of other cases of jargon, which I would classify more as an issue of over-pretentious speech and writing. These are more typical foibles and hardly unique to rationalists. To give but one minor example, using “Pons Asinorum” in place of “foundational challenge”. Using jargon and scientific language that serves to further clarity is fine, but should be avoided in cases where plain English is both clearer and more accessible.
What I describe are extremely common tactics in politics, but one I think should have no place in rational discourse. When writing or speaking (excluding purely artistic endeavors) conveying meaning clearly in ways that can be readily understood as you mean them should be one’s priority. Of course, it is impossible to remove ambiguity, but answering questions with long tangents, moving between unrelated technical fields, and filling your communication with superfluous words and unclear terminology are habits that may serve you well in parliaments and congressional halls, but should be avoided if you actually care about transmitting sincere meaning with your words.
Compare Clinton’s often mocked response on being asked about the Lewinsky affair:
With Yudkowsky being asked on some of his transhumanist views:
These aren’t helpful answers, they are intended to shield the speaker from their own statements rather than elucidating listeners to their thoughts and views. It also develops bad habits that result in comically obtuse statements full of verbose pretentious phrases like: “statistically liable to end in victimful (sic) harm.”[7]
Many have noted a tendency (particularly of Yudkowsky) to make use of cliched parables to make points. I do quite like some parables, they can be useful as moral lessons or posing thought experiments, but they are poor replacement for actual rational argumentation and reasoning. Consider this exchange, for example. When opposing the position that (to paraphrase) “intelligence is multimodal and AI, despite improvements, might not universally outdo humans” there are plenty of arguments and rationales one might offer for why you could expect AI to outcompete humans across diverse fields. One might offer evidence of how models are increasingly becoming competent across many domains, or make a more fundamental argument about how AI models function to justify the view that their capabilities are incredibly broad.
There are plenty of valid cases one might make to refute the argument presented in the ~150 word paragraph in the example. But none that I can think of would include a 10k word (deliberately, I assume) cliche piece of fictional narrative that has a “midwit” espouse a view somewhat similar (but notably distinct from) the view being refuted, just so they can be torn down in your fictional conceit, is no more compelling than a man from Nazareth declaring that “everyone who hears these words of mine and does not act on them will be like a foolish man who built his house on sand” (Matt. 7:12). You cannot expect readers, particularly those of an opposing view, to grant you authority as a sage able to elucidate both sides of an argument with great cunning.
Anyone who has not read Orwell’s essay, would be well advised to do so. It is a foundational, if imperfect, text in English style and warrants reading by anyone interested in English communication, particularly of the polemic sort.
In my professional life, I often work with survey data and extensive critiques of their usage.
I do not mean to pick on anyone, but I am choosing this older essay as it is particularly illustrative of how some major errors occur, which I expand on.
Edit: Indeed, in some survey designs, it is non existent. As the same Pew Research results indicate, for matched panel-data, the bogus respondent rate is zero.
There are a bunch of nuances to how/when and what risks these can mitigate
It was not at the LessWrong post published in Nature Communications, but the full text, and supplemental material, covered everything I am going to discuss.
Rendered in plain English, it is simply “likely to cause harm”; the words, “statistically” and “victimful” add no meaning