From a perspective of a "magician", checking for placebo effect is probably not worth it. Suppose I do a ritual to... increase my willpower, so that I stop procrastinating or improve my diet. And it works! Was it really the magic? Or was it just my belief in the ritual that made me do the right thing? It doesn't matter; I got the desired effect either way. And if it was just my belief (narrator: it was), investigating too deeply might actually ruin the effect.
Of course, making this excuse openly would practically mean admitting that it was a placebo effect. So the traditional excuses are more like: "skepticism ruins the mood, which is a necessary component of the magic".
I find it puzzling why significant numbers of intelligent-seeming people claim to have strong evidence that magic does exist.
I have much less trust in rationality of the intelligent people. Science is not instinctive. Well, curiosity and generating hypotheses, they are. But testing the hypotheses (as opposed to just collecting confirmatory data, or rationalizing the outcome) is not.
From status perspective, if you convince some people that you have magical powers, it gives you high status in the community. On the other hand, examining your magic with actual skepticism (not just pretended one) means being perceived as a loser in both worlds: the non-magicians will laugh at you for taking it seriously, the magicians will laugh at you for doing it wrong. People naturally follow their status incentives. Also, trying to examine someone else's magic will be perceived as a status attack.
Another obvious explanation: people lying on the internet.
If they are trying to sell something - authors, bloggers, influencers - then I agree that this is a major concern.
When it's just people posting on social media, I don't expect there to be that many overt liars. Although self-deception is very much a possibility.
The problem is that naive empiricism is not good enough for most non-trivial practical applications.
(Where a trivial application would be figuring out that a hammer makes a sound when you bash it against a piece of wood, which will virtually always happen assuming certain standard conditions.)
For another example of this failure mode, look at the history of medicine. At least some of the practitioners there were clearly empiricists, otherwise it seems very unlikely that they would have settled on willow bark (which contains salicylic acid). But plenty of other treatments are today recognized as actively harmful. This is because empiricism and good intentions are not enough to do medical statistics successfully.
Look at the replication crisis for another data point: Even being part of a tradition ostensibly based on experimental rigor is not enough to halfway consistently arrive at the truth.
If you are testing the hypothesis "I am a wizard" versus the null hypothesis "I am a muggle", it is likely that the former is much preferable to the experimenter than the latter. This means that they will be affected by all sorts of cognitive biases (as being an impartial experimenter was not much selected for in the ancestral environment) which they are unlikely to even know (unless they have Read The Sequences or something alike).
If it comes to testing oneself for subtle magic abilities, it would take a knowledgeable and rigorous rationalist to do that correctly. I certainly would not trust myself to do it. (Of course, most rationalists would also be likely to reject the magic hypothesis on priors.)
Record-keeping isn't enough to make you a scientist. People might be making careful records and then analyzing them badly, and if there's no actual effect going on selection effect will leave you with a community of misanalyzers.
I did a fair bit of exploring hypnosis (stage, guided, and self), I Ching and Tarot, etc. in my youth. I never found any action I couldn't explain psychologically.
These all seem like mechanisms for enhancing placebo effects - not direct action, but change of frame or expectation. This can have actual results as it brings forth knowledge or behavior that isn't expected otherwise, and they can have very large observation bias effects, in noticing things that support the occult theory and disregarding things with mundane explanations.
So, mostly #1, and in a few cases #1 can seem strong enough to look a bit (if you squint) like #5.
It is said that the magician knows, dares, and keeps silent. Those who know do not speak, and those who speak do not know.
This exerts a strong selection effect on the answers that you will see, even if there is such a thing.
They claim to keep good records of their spells and what effects they produce, and they claim to have found methods that consistently produce measurable paranormal results for them.
Why don't you link to those claims?
Because that was not the purpose of my post. I did not mean to invite people to "cast a quick glance at the occult community and come up with an explanation". I was fishing for people who already knew something about the topic.
I am curious about the occult. I've read a bit on r/occult on Reddit and a few other places, and one thing that strikes me as very interesting is that many of these magicians (practitioners of magic or magick) consider themselves empiricists. They claim to keep good records of their spells and what effects they produce, and they claim to have found methods that consistently produce measurable paranormal results for them.
If we assume that magic does not exist, then I find it puzzling why significant numbers of intelligent-seeming people claim to have strong evidence that magic does exist. This does not prove that "magic exists", of course, but it is, as I said, puzzling.
The explanations I can think of include (and the answer might be a combination of several of these):
Has anyone else looked into the occult community and their claims?
(You don't need to reply with the standard objections like "why aren't they rich yet?" or "go win the Randi prize". I can come up with those myself.)