If someone updated towards the "autism is extreme maleness" theory after reading an abstract based on your hypothetical maleness test, you could probably argue them out of that belief by explaining the specific methodology of the test, because it's obviously dumb. If you instead had to do a bunch of math to show why it was flawed, then it would be much harder to convince people because some wouldn't be interested in reading a bunch of math, some wouldn't be able to follow it, and some would have complicated technical nitpicks about how if you run these numbers slightly differently you get a different result.
Separate from the "Is that your true rejection?" question, I think the value of making this argument depends heavily on how simple you can make the explanation. No matter how bulletproof it is, a counterargument that takes 10000 words to make will convince fewer people than one that can be made in 100 words.
Background on EQ-SQ
The EQ-SQ theory is a theory by Simon Baron-Cohen which states that there's a biological sex difference in tendency to empathize with people (Empathizing) versus try to understand deterministic systems (Systematizing), and that autism represents the extreme Systematizing end of this spectrum.
Measurement bias
Since writing this subthread where I found ambiguous and partial evidence that the EQ-SQ theory's findings are due to measurement bias, I've been considering collecting direct evidence with SBC's EQ and SQ-R scales, and write up a detailed post investigating measurement bias.
Measurement bias, if it exists (as it very much seems to do on e.g. the EQ scale), would most likely involve observing that the sex difference on the scales is limited to one subset of the items, the autistic-allistic difference is limited to another subset of the items, and maybe also the main thing the scale measures exists on a yet third subset of the items.
That is, it would involve observing that the scales mix different things together, and that autism and sex have distinct relationships with the things that get mixed together. As an analogy, imagine showing that autism is extreme masculinity by creating a "maleness test" which mixes together physical strength and poor social skills. Yes, this would have a large sex difference and a large autistic-allistic difference, but that'd transparently be measurement bias; the reason for the large sex difference (it measures physical strength) would be distinct from the reason for the large autistic-allistic difference (it measures social skills).
My concern
My concern is that people didn't start believing in the EQ-SQ theory based on statistical correlations found with Simon Baron-Cohen's scales. They presumably started believing in it based on fuzzy intuitions arrived at through social experience.
So I could imagine that if I did show SBC's findings to be due to measurement error, people would sort of dismiss it as poorly designed scales and still continue believing in the theory. Like basically maybe SBC's studies provide a faux-objective criterion they can use to throw at feminists, but if the criterion doesn't validate sex differences then EQ-SQ enthusiasts would discard it and pick a new one.
Thoughts?