This is definitely true in software development. Ignore the hucksters selling clean code and agile software development or what not, and focus on blog posts by real practicing developers describing how they solved real world problems and the tradeoffs they faced.
https://www.scattered-thoughts.net/writing/on-bad-advice/ is a great post on this.
"You talk the talk but do you walk the walk?"
I'm getting very Talebian vibes form this post, the "professor" a Euphemism for his "Intellectual Yet Idiot" stock villain. And the whole idea of Skin in the Game: that the more 'cost' someone may incur for their action the more credible their decision making. Talk is cheap: a professor may suffer a certain reputational damage if they are found to be espousing a incorrect theory, but is that cost relative to those who practice it?
However, Taleb himself would caution that not all practitioners are trustworthy or right. The Titanic Submarine comes to mind as someone who bet their life on something and ended up losing that bet, tragically taking the lives of others with them. They put their faith in someone with skin in the game and paid the ultimate price.
Quite so. I haven't read 'Skin in the game' yet, but it was recommended by a friend who read an early version of this post. It looks like it conveys this point exactly.
In response to the caution you referred to, I would agree. In reality we should only be watching practitioners, not listening to them. And then we can only treat the observation as Bayesian evidence.
One problem with this is that most people don't objectively summarize their lives and post all their consequences online. If we want to get more evidence than we can gather personally, we are going to have to listen to someone.
I think Taleb's view is to take a more ensemble approach - you can't be sure of the trustworthiness or the correctness of a single practitioner's model: but you can see what a group of practitioners do and infer from the failure rate how good the practice is. After all - there is no 'dead cert.' - there is always a chance. All we can do is minimize it.
As you say this involves watching - observation. It is one thing to be told "oh well they all do this" - that which is professed to be what's happening. Teasing out what is happening, and also getting all the unspoken tacit knowledge that makes it work is quite another.
That and, sometimes for this very reason people are protective about the specifics and the important nitty gritty details which make it effective or not.
This might be a tangent, but it reminds me of a comment recently about how deceitful the word "just" is - as in "you can just do things". Michael Palin has a snappy quip about the word "just"
The use of the word "just" by an Australian [1]means that whatever it is you have to do, it will not be easy, as in "Just pull that sword out of the stone" or "Just split that atom."
Learning how to do something the way that a group of practioners do may not be as simple as "just observe them" or even "just go like this". I may be a tremendously complicated path to learn it, involving not only amassing huge reams of data and information - I have a vague memory of Xerox or a competitor hiring Ethnologists to study videos of how people used copy machines. But then internalizing it cold require some form of structure to allow Deliberate Practice.
"Just do it like this..." - how do you do THAT!?
I'm sure is more universal than just a product of the seemingly "too easy" "she'll be right, mate" "no wucking forries" attitude of Australians
Issue with judging the practitioners is that practicing it may be correlated with other things that are much more harmful. Like all the talk about how single parenthood is supposedly bad for you, but then it doesn't hold up to more careful scrutiny afaik.
Don't judge a principle by its professors—look to its practitioners.[1]
"Professor" is an interesting word. At one point in my professional life, I had the opportunity to teach college classes. I often corrected students who called me "Professor Eggs," telling them I was just "Mr. Eggs." "Professor" was a title and a high status I hadn’t earned. But at the same time, the root "profess" often carries the opposite implication. To profess means to declare, sometimes loudly, sometimes without credibility. A profess-er, in this light, sounds less like a scholar and more like a huckster.
An intriguing contrast might be the word "practitioner." Connotatively, it sounds humble, even lowly, the opposite of the high-minded professor. But denotatively, it's closer to the true opposite of a profess-er: someone who applies principles rather than just proclaims them. Practitioners are the ones you call when something needs doing.
I've noticed a tendency, especially in intellectual discourse, to evaluate a principle by looking at its professors, the people most vocal about it. But maybe that’s a mistake. Maybe we should be looking at its practitioners.
Consider some familiar examples of professors who are not practitioners:
Union officials who champion public education but send their children to private school. Advocates of tolerance who can't stand each other. Married commentators who insist that single parenthood is just as effective. Scientists who teach the scientific method, but who practice p-hacking. Or closer to home, a rationalist blogger who never changes their mind.
The issue isn't just hypocrisy. It's misrepresentation. Professors, in the broad sense, shape public understanding of a principle. If their lives don’t reflect its real effects, onlookers may wrongly conclude that the principle itself is flawed, or blessed, or far more effective than it really is. This danger grows when the professor is eloquent: good rhetoric can mask poor integration.
There's an old biblical line: "By their fruits ye shall know them." It's probably true. Professing a belief is easy. Living it is hard. If you want to know what someone really believes, watch what they do, not what they say. This is why revealed preferences offer a better diagnostic than stated ones. And the same goes for principles. The lived consequences of a belief, the results it produces in practice, tell us more than any abstract declaration.
This holds not only for individuals but for ideologies. Don’t just listen to what a belief system claims. Ask what kind of people it reliably produces. Real practitioners encounter friction. They try to apply principles in a messy, constrained world. That forces tradeoffs, workarounds, and deeper understanding. Not every practitioner succeeds, but those who do illuminate what the principle really entails. They show the blind spots, the costs, and the unexpected strengths. They might not talk much about it, but they are living experiments and valuable data.
So don’t judge a principle by its professors. Judge it by what happens when real people try to live by it.
Ask who the quiet implementers are. Are they different? Are they better or worse off? What does their practice reveal that the professors may never mention?
And pay special attention to professors who were practitioners first. They still profess, but they do so with the authority of lived experience. They might just know what they’re talking about.
This post is nothing new, but a synthesis of ideas I've had while going through the sequences.