tl;dr : We overgeneralize the way we personally come up with concepts. People, notably outside of LW and inside of Academia, argue by referring themselves to diverse and starkly different ways of coming up with concepts, which are different enough for us to misinterpret them. We should not expect people to be receptive to our arguments if we can’t identify how their fundamental conceptualization procedure affects their arguing style.
I think there are two tendencies in the LW and EA communities :
These two tendencies are presuppositions of the following, and I will not defend them. I also think people on LW and EA should care about academia.
In general, knowing better how academia works is useful for setting up a zeitgeist in which EA and some of its crucial topics will be more accepted. But a more important argument is to know where to go and what words to use when you want to work on an EA-related topic, but cannot afford to do some active AI alignment research early on, and so end up being at academia for about five years at minimum.
Epistemic status : I’m not a philosopher, I only have a undergraduate diploma in philosophy. What I’m showing here is a heuristic for understanding disagreements between people. I might put it to formal test one day and try to refine it. In the meantime, I suggest you to come up with your own version of this, and not to use this vocabulary as is in your interactions. I invite you to refine it and probe for possible errors. See it more like shores of vast continents containing greater diversity.
Philosophy is a diverse field. In western thought, however, the main dominating discourses usually fall under one of four traditions :
1-Analytics (in the likes of Frege, Russel, Quine, Denett)
2-Phenomenologists (in the likes of Husserl, Heidegger, Sartre, Merleau-Ponty)
3- « german ideology/socio-historical approach/neo-kantians/neo-marxists » (in the likes of Adorno, Foucault, and arguably Habermas)
4-Pragmatists (in the likes of Dewey and John Herbert Mead)
I've had teachers of each of these traditions, there are arguably others, of course.
I insist here that I will not use the name of these traditions to analyze people’s arguing style, because it makes it confused whether they are actually academically affiliated with it or not. I’ll be using the word “categories of arguing style” or CAS to refer to the categories I’m relying on to analyze people’s arguing style. These CAS are not the traditions themselves, so we can keep things clean and distinct. Note also that they are independent of the topics their wielders are often most interested in.
CAS are categories that vary, first, depending on how people usually come to conceptualize stuff. However, Anne and Bob could conceptualize stuff differently, without anyone noticing anything. A CAS is meant to point out the difference in conceptualization that results in a stark difference in argumentative style. I personally tend to classify people into four CAS depending on which tradition they most resemble :
1-Analyticians : Presuppose that knowing truth conditions and subparts of objects help to understand the world.
2-Phenomenologians : Presuppose that knowledge is based on fundamental personal experiences.
3-Socio-historians : Presuppose that we have socio-historically situated presuppositions, and that in order to vanquish them, methods such as political activism or sociology are sometimes strictly better than careful reasoning or natural science.
4-Pragmaticians : Presuppose that truths and the world are linked together in a feedback loop, and that beliefs are what you act on, rather than abstract propositions in the head.
This is completely arbitrary, you could as well choose your own sources of inspiration. Ideally, of course, these would be identified through careful data-analysis. But I’m choosing those for this post because it’s the only ones that I used in practice so far, and because they question very deep presuppositions about our way of arguing.
Another way to distinguish the CAS that I use is by asking this question :
Given that there was a scientific revolution, an enlightenment, and a spread of liberal democracies, why are we still facing problems?
1-If you're analytician, your answer is probably something like « our science isn't good enough », « we still have a lot of progress to make », or « we're not perfectly rational ». They tend to see the advances in natural sciences as a characteristic success of their modus operandi.
2-If you're a phenomenologian, your answer is probably something like « none of these events were actual revolutions (except in the sense that they deeply affected our perception of the world), we still have to acquire a good ability to deconstruct ourselves / analyze our own essences and [those of] our society ». You probably celebrate things such as social awareness, and events such as the legalization of gay marriage probably feels considerably more significant to you in how it affected knowledge than the latest advances in physics.
3-If you're a socio-historian, your answer is probably something like « although these events had their share of contribution, they're driven by something more fundamental that is related to power relationships and power dynamics, and we're stuck when it comes to that ». You tend to consider the advent of trade unions or high engagement rates in a population towards some cause as a marker of success.
4-If you're a pragmatician, your answer is probably something like « any change we make to society changes it back, and we have to constantly maintain it and readjust it : see the legal system, who has to constantly create new laws to adjust for the abuse induced by some other laws which is perceived as unfair by some people. Our mechanisms aren't currently fitted for that. We’d need to be much more active, and participate much more to the political life of our country ». I know that at least one pragmatist scholar celebrates vigorously Wikipedia, as well as horizontal organisation becoming the mainstream in activist movements. Flexible and democratic organisation is usually seen as a success.
Regarding my first description, I insist on the verb presuppose. If you usually set out the stage explicitly to reach any of these conclusions, you probably are not in the CAS that presupposes it. You'll have noticed that by beginning by a section called « This post aims to help people in Academia », I've affiliated myself to the analytician arguing style, since I broke down my post into subsections that tend to set out the different theses on which contention can appear.
Here's the problem : almost everyone on LW and in the EA community are analyticians. Almost everyone makes as if this is completely normal. And a lot of us tend to be bored by other approaches (including myself) and to miss their point. The result is that
1-The few things told about academia overwhelmingly tend to concern its analytic side.
2-People tend to run into the same failure mode when exchanging with a different CAS.
Anna is an analytician. Pragyan is a pragmatician. Pragyan has a theory that claims reasoning amounts to a succession of checks in terms of trust, evidence, and usefulness. Anna tries to understand. At some point, Pragyan takes a pen and draws a flowchart. The flowchart has several layers to account for all of these checks.
Anna : Interesting. But, it seems bizarre to me, the flowchart you draw has many layers and at least three different steps. Don't you think this is a bit computationally costly?
Pragyan : Computationally costly? Why would it be?
Anna : Well, you can't have such a multi-step ultrafast process without using time and energy to process inputs and outputs, right? What you've drawn has to fit into one's mind.
Pragyan : Uh? No it doesn't fit.
Anna : Exactly, that's what I'm saying, it doesn't.
Pragyan : But where's the problem?
Anna : Well, if it doesn't fit into one's mind, then how can that possibly explain reasoning? Reasoning is something that happens in the head, right?
Pragyan : In the head? Why that?
Anna : Come on, it doesn't happen in the air.
Pragyan : I'm not getting you, what I've just drawn represents reasoning, and reasoning has to happen in interaction with the environment.
Anna : You consider the environment to be part of your mind? That sounds a bit far-fetched.
Pragyan : Why would I? I’m just trying to get to a workable definition of reasoning, one that makes sense.
Anna : Well, magical reasoning working that way just… doesn’t make sense to me…
Anna in an analytician. Philip is a Phenomenologian. Anna has a disagreement with Philip about how to solve an organizational problem within the university they work in (in part courtesy of Chat-GPT, this one was hard to come up with on my own).
Anna: I think the best way to solve this problem is to break it down into smaller parts and analyze each one objectively.
Philip: I don't think that's the best approach. I mean, sure, it might give us some information, but we'll miss out on the bigger picture. If we make sure to identify what we personally associate with the situation at hand, including emotionally, we’ll be able to have a better view of it.
Anna: But we can't let our emotions cloud our judgment. We need to be objective and make the best decision based on the facts.
Philip: Yeah, but the facts don't tell the whole story. They don't show us how people will react to our solution or how they'll feel about it. Those are just as important in finding the right answer.
Anna: I just don't see how feelings and emotions can help us make the best decision. It seems like a cop-out to me.
Philip: And I don't see how ignoring people's emotions and experiences can lead to a good solution. That just seems insensitive to me.
Anna: I'm not ignoring anything! I just think that our decision should be based on logic and reason, not just on how people might feel.
Philip: Well, I'm not ignoring anything either! I just think that our solution should take into account all the different elements involved, including people's feelings and experiences.
Sofia is a Socio-Historian.
Anna : Research has shown that there is difference X between males and females. Actually, it’s also shared by other great apes.
Sofia : What kind of research was it? What did they do exactly?
Anna : Glad you’re asking ! It’s behavioral biology. They ran a double-blinded experiment on some subjects and found significant results.
Sofia : Ah, come on, you know it’s bullshit.
Anna : Why is that?
Sofia : The world is so complex, they could have interpreted the results in any way they liked. Who ran the study?
Anna : Uh, a university in Norway, I think
Sofia : And who paid for it?
Anna : I think it was part of a grant given by a protestant fund, but who cares?
Sofia : Well, I do. Protestant fund you say? That does not sound good.
Anna : But look, Sofia, they were not doing the research.
Sofia : Maybe, but they encouraged it. They’re probably conservatives. All this behavioral research is just reifying and essentializing people.
If you’re like me, then Philip, Pragyan and Sofia probably sound confused and overconfident. We’ll go back to these examples at the end, and you’ll see that they hold more coherence than you think.
Picture yourself as 17-year-old me, imagining that University is basically Analyticia : a land in which everyone stops doing this meaningless rumble with their mouth and tries to think clearly. I mean, YouTube videos taught me critical thinking, blog posts taught me about biases and arguing, and a Harry Potter fanfiction taught me about rationality. Since academia is more serious than YouTube, blog posts and Harry Potter, it must be Analytic, right?
Wrong. Living in France, I had to deal with continental philosophy and sociology, and people actually being on this other side. I went through a lot of confusion, and sometimes had problems sleeping or socializing with others due to this difference.
It turned out this confusion was good, and I made myself a mental map to understand what the hell was going on in academia and why no one told me about it, and how previous attempts to make maps by my immediate surroundings utterly failed. My entire quest was basically understanding the existence of the different philosophical traditions presented above and how they created confusion, smugness and aggressivity in the context of academia. I wish someone had warned me about this. So I try to warn the youngests among you who think they'd find siblings in thought in academia. Here's a landscape of the different CAS.
1-Isn't this just the « science wars »?
Not really. What I'm focusing on here is something related to how people talk together, and how they disagree within academia. I don't really care about the publication landscape. This is why I distinguish the CAS (ending in -an) and the actual traditions.
2-I've finished university. Is there another benefit to know this aside from alleviating confusion from people or can I stop reading?
I think there is a potential benefit if you're working in AI. I think it'll be very hard to solve ontology-related problems in AGI Safety (on a deeper level than the kind of problem mentioned in Natural Abstraction Hypothesis) if we're not even able to solve them among humans.
3-These other traditions seem completely irrelevant to me. They’re just wrong, why should I care?
This is a common trap when it comes to understanding others and having a productive dialog. If you don’t try to understand how they could possibly feel that they are right without calling them “dumb”, “irrelevant” or “obviously self-contradicting”, you will hardly be able to communicate your actual remarks to someone on the other side.
Just to make it clear, I’ll first explain for each of them what tradition they are affiliated to and how it transfers to the associated category of arguing style (CAS).
Warning : trying to explain other traditions is doomed to be self-contradictory. Traditions are partially about how to try to explain things. They’re about practice, not content. So I can give you an example of what it looks like, but if I want you to empathize with it, I'll have to betray this tradition a little bit and go back to the analytic tradition to emulate it.
I think people here are familiar enough with analytic philosophy to simply redirect them towards the wikipedia page or a few example articles. Long story short, the founders of analytic philosophy also invented modern logic, so you expect it to share the same kind of rigor and clarity.
Analyticians are not necessarily analytic philosophers, although analytic philosophers are overwhelmingly analyticians. To some extent, I'd even go as far as saying that LessWrong posts embody analytician-ism even more than traditional analytic philosophy does. Since most people here are familiar with that kind of content, I’ll just bullet point some typical elements :
This is a hard one. I'll first link here a series of video by Merleau-Ponty, which does a fantastic job to explain what philosophical phenomenology is.
Exercice:
Let's try a simple phenomenological exercise. I'll give you a word : treasure. Try to write down the first associations that you make. It might be treasure island, to treasure something, treasure near the rainbow... Now another word : dust. The associations might be different : you'll return to dust, a dusty attic, dust flying in the light... I'll call these associations you're making with each word your relationship to these words. Here's the twist : try to swap these relationships. Something like dust Island, to dust something or dust near the rainbow, and you'll return to treasure, a treasury attic, treasure flying in the light... In this particular example, I'm now able to note that something that characterizes dust in its basic connotation is a sense of substance, it's what things can be made of. On the other hand, treasure is more of something you find, that can be located somewhere. What I'm thus inclined to imagine now is treasure as a substance, something things can be made of, and dust as something I find, that can be located somewhere.
Enriching this idea, it can push me to ask myself whether the place wherein value lies the most is usually in the components of trivial stuff we haven’t discovered yet (a bit like deconstructing a device, such as a hair dryer, and discovering surprising stuff inside of it, further and further digging into the core). Whereas for dust, maybe the true feeling of dustiness is what you experience when something you cherish you expected to find beneath some obstacles (like an old postcard) is actually now but a mere pile of dust.
This is phenomenological productivity in action. Now, the reduction (identification of the basic connotation) and the deconstruction (swapping the relationships) I'm offering you here are of very poor quality and quite uninteresting, but the general process is somehow similar across different themes and concepts. You'll notice that what you're focusing on is not so much the formal relationships or empirical properties of dust and treasure, but rather your own associations. Some phenomenology authors will achieve something quite impressive while doing this, and point out intuitions that we share with them we hadn't even been aware of. This shared sense partially determines what we call reality.
Personal impressions of the CAS [which might help to form a more detailed picture] :
I remember a friend who is a very good example of a phenomenologian. While we were preparing a field study about tatoo in sociology class, we prompted him on how to formulate our problem statement, and he started wondering : « When I think about tatoos, I think about skin. Skin decays with age. What is interesting here is how the tatoo survives or decays with time ». That left me half-amused, as it wasn’t what we were asked for. An instance of him mis-identifying the actual tradition he had to situate himself in happened in analytic philosophy. Prompted about his position on the philosophical debate over the nature of time (A vs B theory), he answered something in the lines of : « I imagine myself on a boat, and the boat is on a still ocean. The boat sails through the water, but the water remains the same. How am I to know that I'm really going forward? »
Note that this is not an analogy, nor is it a thought experiment, nor is it just throwing around some literary meanders. It is an elicitation of fundamental properties of the experience of identity as it manifests itself to his awareness. What he actually means, translated in analytic lingo, is « what concepts point to the most fundamental units that constitute the felt sense of going forward, irrespective of the modal field mobilized? ». But remember to translate this back to phenomenological language before you get into analytic phenomenology -the goal here is not to find the concept, it is to picture yourself the fundamental initial experiences that constitute time. An enterprise such as qualia formalism (the kind of one the QRI pursues) is, for this friend, probably of only very limited interest.
This is what the CAS I call phenomenologians is actively seeking : what analyticians call felt senses, or resonance, or qualia of certain kinds. Contrarily to the analyticians, however, these felt senses are not conceptually defined : they are what allows us to conceptualize in the first place. Whereas an analytician thinks about neurons or information processing when thinking about taking a deep dive in the fundamental units of thought, phenomenologians rather think about very primordial experiences, or succession of mental images, so to speak. Think about the kind of “qualia transfer” you’re doing when trying to craft a smart analogy to understand something at a more “gutsy” level -as when likening the interplay of weak and strong atomic force to pushing a boulder against the slope of a volcano.
To my understanding, this philosophical tradition stems from Kant's idea of a priori judgements. A priori judgements are judgements that you don't get to empirically. E.g, « a triangle has three sides ». Academic socio-historians simply postulate the existence of a specific type of a priori, called historical a priori judgements. Said otherwise, some premises (understood non-formally) in your reasoning will remain implicit due to the Age you're living in. The same can be said relative to your social class and culture. They sneek into your reasoning as silent presuppositions.
To give a simple illustration of a presupposition, if I’m searching for evidence related to the question “Has Anna stopped smoking?”, whether the evidence is in favor of Anna stopping or keeping on is irrelevant if I haven’t already checked whether Anna actually smokes.
Exercice:
Let’s try, this time, a socio-historical exercice. Pick a concept, I’ll go with art. Now pick a historical trend, I’ll choose industrialisation. Instead of analyzing what counts or doesn’t count as art, or making double-blinded experiments relative to art, as a first step, I’m just going to make a bunch of common-sense statements or raise typical questions on the matter : art is unique, industrialisation is soulless, how do art pieces convey emotions, why is art so wonderful, etc.
I’ll now try to picture how I can be biased or make false assumptions or not make sense due to the environment I live in. Imagine asking these questions to an artist or an art historian, and having a reaction that you didn’t expect at all. I can picture myself artists telling me stuff such as “Making general declarations about art, whether it being unique or not, is kind of contrary to the entire spirit of creating art”, or “Art pieces don’t necessarily convey emotions. I mean, I don’t feel like I want to convey emotions, I want to convey something way richer than emotions and sometimes they’re not my priority.” I can picture art historians saying “The idea that art is unique really stems from a form of individualism that appeared at the turn of the 19th century. Before that people used to see artists as we see artisans today.”
Now, ask new questions but having these possible answers in mind. What I mean is, I’d look back at the last couple of centuries (and by that I mean harvesting data, and look at art from this time period) and ask myself : how was our concept of art influenced by industrialisation? What kind of undetected a prioris about art were shifted due to it? How was the way of producing art influenced by it?
But now, instead of picturing people telling me I’m making wrong presuppositions, I’ll actually get out in the world and look for them, be it with data, or interviews, or reading books. I’ll also often meet people who still share these kinds of erroneous presuppositions. This is what socio-historians are trying to get : identifying their socio-historically situated presuppositions, either within themselves or the broader society. This is the way one can attempt to gain awareness of their socio-historical a prioris.
For a lot of them, however, power is at some point going to get involved within this. Usually, people tend to conserve the erroneous presuppositions that are done by the people holding power. So to be sure that I’m not falling into this trap, I’ll make sure to also interview or survey people or categories who are not in power.
A meta-step further however, after doing my entire analysis of how power relationships influence (how industrialisation influences) art, I’ll look at it and remember that there might be presuppositions I’m still not aware of. Maybe I’ve relied on analogies that refer to stuff that exist in only some societies, like social roles -roles in theater only exist in those societies which developed theater. I therefore conclude that my analysis might at some point just end up dated and unusable anymore (what would “social roles” even mean if theater doesn’t exist anymore?).
Personal Impressions:
Arguing with the socio-historian category often feels like combatting someone looping on ad hominem attacks with an extreme form of modest epistemology whenever the subject touches on humans -specifically behavioral studies. On the socio-historian view, it sounds extremely arrogant to claim to be able to rigorously analyze data from human behavior, sometimes even metaphysically impossible -imagine a 19th century intellectual male owner trying to analyze modern, rich data collected on black females. Their intuition is that this person would not have an even remotely good interpretation, except if he either 1-acknowledges the struggles lived by black females and how their perception or reality has some value of some sort and 2-does the type of deep sociological introspection sociologists could do nowadays, suspending judgment about the reliability of their conclusion in light of History. This would concretely suggest that double-blinded experiments and data analysis are doing the same kind of job that doing an analogy that refers to something that is not universal would do. In the same way, our descendants will one day look at us much like we look at 19th century intellectual male owners.
More formally, they usually bite the bullet on an argument that Passeron had about Popperian epistemology : hypotheses on human behavior are not falsifiable, no matter how hard you try. There will always be a confound you don't account for, and if you isolate variables, you necessarily get an artifact : something which is historically and socially situated taken for a natural law. This impossibility is what is rejected and exploited by the likes of behavioral sciences. For sociologists, all theories about human behavior come with a peremption date stamped upon, and that is arguably true for socio-historians too.
The pragmatist movement started existing slightly before analytic philosophy. In contrast to phenomenologists, they staunchly reject Descartes, but tend to embrace less the power-related fatalism of socio-historians. Basics tenets of this thought include ideas like
1-Beliefs are things you act on. Believing in God is true as long as it provides you some form of meaning while acting on it.
2-Meaning does not happen inside people's minds, nor in ideal abstract worlds, but between people.
3-Inquiring on reality (i.e making some form of informal science) is something necessary to understand it.
Exercice:
Let’s try a pragmatist exercice : pick a concept, something like “walking” (as in “I’m walking”). Define it. I’ll say “putting one foot in front of the other in a forward motion”. Does it make sense to act (intuitively) on this definition? Try it right now. Ugh, walking that way feels a bit… robotic to me. Let’s try a different definition. “getting to another place by no other means than one’s body”. Yep, definitely feels better to act on that. What differences in behavior does each of these definitions cause to you?
Now, you might be tempted to say, “Oh, ok, so if someone gets from point A to point B with a prosthesis, then it’s not walking anymore?”. But that’s your analytician CAS lurking behind. Remember, what matters is that it makes sense to act on it. If you really wanted to give a counter-argument, you’d have to show that this definition does not make sense for someone with a prosthesis -and, actually, they might find it ok, to the extent that they consider their prosthesis to be an extension of their body.
Now of course, Pragmatism is not only concerned with that kind of objects -some beliefs will imply some more abstract actions like teaching some things in a certain way, promoting or shaming some actions, and of course, trying to set up a better system in which beliefs that can’t be unilaterally carried out can be deliberated in a better way.
Personal Impressions:
One of my pragmatism professors was a fond adept of Wikipedia and claimed it was an excellent example of pragmatism in action : people acting on their beliefs by modifying the articles, debating their beliefs, and trying to fetch sources when needing to do so.
A problem that tends to preoccupy the pragmatician CAS is the idea that epistocracy is probably a failure mode : illiterate but concerned minorities are probably still better than the best experts on some domain, given a complex enough system, and so we need to let them have a say in the way truths are established, especially if really acting on their beliefs does not sound crazy to them.
I definitely think that EA and mostly LW have inherited characteristics of pragmatician arguing style from american movement-building and maybe Eliezer himself, and I'm quite surprised not to see this CAS more frequently in the movement.
Pragmaticians are usually people who don't really worry about making very informal flowcharts and to posit concepts in natural language amidst of them. As an analytician, I will tend to look at these flowcharts as sketchy version of a more ideal Finite State Machine, or complex system, or information treatment system of some sort. Pragmaticians do not think there is any link with that kind of form : causal effects between concepts refer to something real but not necessarily formalisable. It only has to make sense when acting on it.
Principles
First things first : the main problem with CAS is that if you analyze misunderstandings caused by CAS, your analysis is very probably embedded within your own CAS. Trying to explain the problem to the person you disagree with will probably just worsen things out.
Second : The CAS one uses is usually deeply ingrained in their way of seeing the world. You can’t just point it out.
Analysis
We can now start to reanalyze the kind of failure modes people fall into when coming from different CAS :
Anna presupposes that what allows her interlocutors to go from premise A to conclusion B is either evidence or a formal relationship.
Philip, however, was implicitly meaning that personal experience of A, when subjected to some implicit mental operation, elicits personal experience B. This mental operation seems to clarify his experience and improve his judgment.
Pragyan on his end, presupposes that what allows to go from A to B is merely the fact of observing that acting on this makes sense. He doesn’t need there to be a “medium” on which beliefs would be written.
Sofia, finally, observes that there is a benefit gained by group X by believing that A leads to B, which is a reason for her to believe that this group has not investigated things properly -they need to identify their own erroneous presupposition.
If there is one thing I would personally stress, it is that these failures are dyadic. The failure here happens in the communication itself. I’m not presupposing that any of the individuals here are making an error.
In contrast, there is one non-dyadic failure I think analyticians should pay more attention to.
The main failure mode of the analytician CAS : the HPMoR Syndrome
Jokes apart, what I mean by “HPMoR syndrome” is the tendency I’ve seen in some rationalists to keep arguing with a characteristic, rationalist style even when their interlocutor seems unresponsive to it. So just to get the message across, in case you didn’t already get it : by default, do NOT talk to people in the style that Harry Potter does in HPMoR. Most of the time, you’ll achieve more by adopting a softer, curious and compassionate tone. This advice holds for talking to other rationalists too. And (wink wink), it may be more rational to do so.
What this means is that ideally, Anna should have questioned the style of her interlocutor instead of keeping on with her counter-arguments. Let’s go back to Sofia :
Anna : Research has shown that there is a difference X between males and females. Actually, it’s also shared by other great apes.
Sofia : What kind of research was it? What did they do exactly?
Here, Sofia is interested in the process to know whether the experimenters attempted to identify their own personal presuppositions, but Anna can’t know it yet.
Anna : Glad you’re asking ! It’s behavioral biology. They ran a double-blinded experiment on some subjects and found significant results.
Sofia : Ah, come on, you know it’s bullshit.
If Anna has some (even unclear) idea of CAS existing, she can try to pattern-match and check whether her guess was true, as she does here.
Anna : Hm, do you think that would be because they presupposed stuff that wasn’t true?
Sofia : Yes, they just went with the vision of gender their society inculcated them with, and then projected it on great apes.
Anna could alternatively have no idea of what CAS her interlocutor is part of. She can then try to identify it or create a new one.
Anna : Hm, why do you think that would be?
Sofia : Well… They just started their experiment, and then saw the great apes behave in such and such way to a statistically significant degree. But they didn’t think about why they started having this hypothesis to begin with and how it was formed by their society.
Anna : I see. In general, what do you think is the biggest obstacle to human knowledge, when it comes to human behavior?
Sofia : Prejudice, for sure. Injustice and prejudice.
Real-life example :
I had a discussion with someone leading an AI Safety outreach group. He was wondering about how to present his work in academia. My main worry was that he'd face several detractors among socio-historians, which are common in France. I gave him that kind of advice :
-If anything, aknowledge the historical aspect of technology. You can say stuff like "I do not believe technology will solve everything", or "progress is slow". Do not frame AI as the "advent of decades of hard work" or anything of that sort.
-You can replace "cost" and "benefice" by "degradation" and "improvement". Lending words from the economic vocabulary usually creates backlash.
-Avoid framing stuff in terms of prediction ("Here are the timelines...") but rather in terms of "reaching a place of safety from which we can govern things more easily" ("Here is how we can ensure that things are kept safe, and how fast we need to do it"). You can even say that you're not predicting the future.
How would I explain the same failure modes to different CAS?
This is an important section, because gaining awareness of someone else’s CAS should influence your decisions when it comes to communicating with them. Remember, if you analyze misunderstandings caused by CAS, your analysis is very probably embedded within your own CAS. This section here presents a way to partially counter this effect.
To give an example, here is how I’d write the “analysis” section above if you were a phenomenologian :
Anna has a very strong intuition that using evidence solves things best. When she relies on her emotions, she experiences confusion and skepticism. When she reads an article giving an empirical rationale for privileging X over Y, she feels a form of lightness and clarity of mind. Philip, however, focuses on the importance of how the problem could harm people even if the solution feels less clear and based on evidence. Pragyan, on his end, experiences some very stringent dissonance when he’s presented with arguments that are supposedly true, but you can’t behave according to. Sofia shares the same concern, but she feels angry at the idea that people might describe the world of other from their own perspective, which might lead them to subconciously justify their personal interest.
You can still see I’m clearly analytician, in that I have four characters, each embodying one CAS. I’m also influenced by the fact that I’m writing for an audience mainly consisting of analyticians. However, you can see that the emphasis is on the difference in subjective experiences rather than criterion for adopting a belief. Take a look at the perspective shift between that paragraph and the original one, try to amplify it, compare with an actual phenomenologian CAS, correct obvious errors, look back at the first version, rince and repeat.
Why is this hard to explain?
I can’t pretend to have the full answer here, but my impression is that each CAS tends to reconstruct their adversary’s remarks by making it coherent with their own basic assumptions, instead of noting that their assumptions are being actively denied. It’s as if we were using different shapes (say cubes for analyticians and tetragons for phenomenologians) to build a tower (say skyscrapers for analyticians and pyramids for phenomenologians) and when asked to copy our partner’s tower in order to check its robustness, we did so by still relying on our own initial shape (building pyramids with cubes and skyscrapers with tetragons). Instead of focusing on whether we should be using cubes or tetragons, we end up pretending to debate with someone who doesn’t make sense, because our reconstruction of their argument keeps falling apart (as is to be expected).
Can we escape?
Can we escape the misunderstandings between CAS (without manipulating, or doing otherwise stuff the interlocutor wouldn't approve of)? I don’t know. As suggested in my examples, asking non-loaded questions to your interlocutor seems a frequent way out, and some techniques like Street Epistemology gave me tools to cope with these differences. Note however, that does not seem like a reason to think that the analytician CAS is better than the others.
I’ll explain myself : I’ve also encountered people with a phenomenologian CAS who’d rely on some conversational “techniques” (or habits, or posture) that, much like relying on Street Epistemology can leave others a bit closer to the analytician CAS, left me extremely impacted on a perceptual level. I wonder how many dialectic techniques that ensure convergence towards your own CAS have an equivalent technique that ensures convergence towards your interlocutor’s CAS.
As far as I understand, sufficiently powerful AIs are expected to produce some form of ontology of the world. This ontology will probably progress as the AI advances, encompassing concepts that are more and more abstract. At some point, they may develop a method for ontological generation.
Observing that humans did not converge so far on what method for ontological generation was the best, resulting in different traditions (and humorously stringent misunderstandings between philosophers), I’m wondering whether the same fate is pending on AGI, and how much that should be a concern for ontology-related problem in alignment.
I also associate each CAS with a more agentic version of it : agentic analyticians tend to be EAs or entrepreneurs, agentic phenomenologians usually get into heavy meditation or Gestalt-therapy, agentic socio-historians tend to organize protests and to be great activists, and agentic pragmaticians tend to gather, inquire, and create self-sustaining organizations on their own. You might feel a bit indifferent, but each of them can, when well trained, inspire awe.
I have to insist that having friends that belong to three of these four traditions helped me a lot. Phenomenologians insisted on me not deluding myself and accepting my intuitions and emotions. Socio-Historians made me accept non-binary people as people experiencing legitimate struggle and who needed acceptance. Pragmaticians, although not represented by my friends, if represented by the kind of people starting stuff such as Wikipedia or Nuit Debout (a French protest back in 2017), gave me hope that democracy and rigorous epistemics might somehow find some mutual ground one day.