Note: I wrote my comment while reading as notes to see what I thought of your arguments while reading more than as a polished thing.

I think your calibration on the 'slow scenario' is off. What you claim is the slowest plausible one is fairly clearly the median scenario given that it is pretty much just following current trends, and slower than present trend is clearly plausible. Things already slowed way down, with advancements in very narrow areas being the only real change. There is a reason that OpenAI hasn't dared even name something GPT 5, for instance. Even 03 isn't really an improvement on general llm duties and that is the 'exciting' new thing, as you pretty much say.

Advancement is disappointingly slow in AI that I personally use (mostly image generation, where new larger models are often not really better overall for the past year or so, and newer ones mostly use llm style architectures), for instance, and it is plausible that there will be barely any movement in terms of clear quality improvement in general uses over the next couple years. And image generation should be easier to improve than general llms because it should be earlier in the diminishing returns of scale (as the scale is much smaller). Note that since most are also diffusion models, they are already using an image equivalent of the trick o1 and o3 introduced with what I would argue is effectively chain of thought. For some reason, all the advancements I hear about these days seem like uninspired copies of things that already happened in image generation.

The one exception is 'agents' but those show no signs of present day usefulness. Who knows how quickly such things will become useful, but historical trends on new tech, especially in AI, say 'not soon' for real use. A lot of people and companies are very interested in the idea for obvious reasons, but that doesn't mean it will be fast. See also self-driving cars which has taken many times longer than expected, despite seeming like it is probably a success story in the making (for the distant future). In fact, self-driving cars are the real world equivalent of a narrow agent, and the insane difficulty they are having is strong evidence against agents being a transformatively useful thing soon.

I do think that AI as it currently is will have a transformative impact in the near term for certain activities (image generation for non-artists like me is already one of them), but I think the smartphone comparison is a good one; I still don't bother to use a smartphone (though it has many significant uses). I would be surprised if it had as big an impact as the worldwide web has on a year for year basis counting from the beginning of the www (supposedly in 1989) for that and 2014 when transformers were invented (or even 2018 when GPT1 became a thing) for AI, for instance. I like the comparison to the web because I think that AI going especially well would be a change to our information capacities similar to an internet 3.0. (Assuming you count the web as 2.0).

As to the fast scenario, that does seem like the fastest scenario that isn't completely ridiculous, but think that your belief in its probability is dramatically too high. I do agree that if you believe that self-play (in the AlphaGo sense) to generate good data is doable for poorly definable problems that would alleviate the lack of data issues we suffer in large parts of the space, but it is unlikely that would actually improve the quality of the data in the near term, and there are already a lot of data quality issues. I personally do not believe that o1 and o3 have at all 'shown' that synthetic data is a solved issue, and it wouldn't be for quite a while if ever.

Note that the image generation models already have been using synthetic data by teachers for a while now with 'SDXL Turbo' and other later adversarial distillation schemes. This did manage a several times speed boost, but at a cost of some quality, as all such schemes do. Crucially, no one has managed to increase quality this way, because the 'teacher' provides a maximum quality level you can't go beyond (except by pure luck).

Speculatively, you could perhaps improve quality by having a third model selecting the absolute best outputs of the teacher and only training on those until you have something better than the teacher, and then switching 'better than the teacher' into teacher and automatically start training a new student (or perhaps retraining the old teacher?). The problem is, how do you get that selection model that is actually better than the things you are trying to improve in its own self-play style learning rather than just getting them to fit the static model of a good output? Human data creation cannot be replaced in general without massive advancements in the field. You might be able to switch human data generation to just training the selection model though.

In some areas, you could perhaps train the AI directly on automatically generated data from sensors in the real world, but that seems like it would reduce the speed of progress to that of the real world unless you have that exponential increase in sensor data instead.

I do agree that in a fast scenario, it would clearly be algorithmic improvements rather than scale leading to it.

Also, o1 and o3 are only 'better' because of a willingness to use immensely more compute in the inference stage, and given that people already can't afford them, that route seems like a it will be played out after not too many generations of scaling, especially since hardware is improving so slowly these days. Chain of thought should probably be largely replaced with something more like what image generation models currently use where each step iterates on the current results. These could be combined together of course.

Diffusion models make a latent picture of a bunch of different areas, and each of those influences each other area in the future, so in text generation you could analogously have a chain of thought that is used in its entirety to create a new chain of thought. For example, you could use a ten deep chain of thought being used to create another ten deep chain of thought nine times instead of a hundred different options (with the first ten being generated by just the input of course). If you're crazy, it could literally be exponential, where you generate one for the first step, two in the second... 32 in the fifth, and so on.

"Identifying The Requirements for a Short Timeline"
I think you are missing an interesting way to tell if AI is accelerating AI research. A lot of normal research is eventually integrated into the next generation of products. If AI really was accelerating the process, you would see the integrations happening much more quickly, with a shorter lag time between 'new idea first published' and 'new idea integrated into a fully formed product' that is actually good. A human might take several months to test the idea, but if an AI could do the research, it could also replicate the other research incredibly quickly, and see how it works when combined with the other research.

(Ran out of steam when my computer crashed during the above paragraph, though I don't seem to have lost any of what I wrote since I do it in notepad.)

I would say the best way to tell you are in a shorter timeline is if it seems like gains from each advancement start broadening rather than narrowing. If each advancement applies narrowly, you need a truly absurd number of advancements, but if they are broad, far fewer.

Honestly, I see very little likelihood of what I consider AGI in the next couple decades at least (at least if you want it to have surpassed humanity), and if we don't break out of the current paradigm, not for much, much longer than that, if ever. You do have some interesting points, and seem reasonable, but I really can't agree with the idea that we are at all close to it. Also, your fast scenario seems more like it would be 20 years than 4. 4 years isn't the 'fast' scenario, it is the 'miracle' scenario. The 'slow scenario' reads like 'this might be the work of centuries, or maybe half of one if we are lucky'. The strong disagreement on how long these scenarios would take is because the point we are at now is far, far below what you seem to believe. We aren't even vaguely close.

As far as your writing goes, I think it was fairly well written structurally and was somewhat interesting, and I even agree that large parts of the 'fast' scenario as you laid it out make sense, but since you are wrong about the amount of time to associate with the scenarios, the overall analysis is very far off. I did find it to be worth my time to read.

Apparently the very subject coming up led to me writing a few paragraphs about the problems of a land value tax before I even started reading it. (A fraction of the things in parenthesis were put in later to elaborate a point.)

There's nothing wrong with replacing current property taxes with equivalent (dollar value) taxes that only apply to the value of the land itself (this would be good to avoid penalizing improving your own land), but the land value tax (aka Georgeism) is awful because of what its proponents want to do with it. Effectively, they want to confiscate literally all of the value of people's land. This is hugely distortionary versus other forms of property, but that isn't even the real problem.

The real problem is that people don't want to live lives where they literally can't own their own home, can't plan for where they will live in the future, and can be kicked out at any time easily just because someone likes the idea of claiming their land became valuable. This turns all homeowners into renters. There are in many places (such as California) very distortionary laws on property tax because people hate the idea of their land being confiscated through taxes. There are also very many laws on making it hard to kick out renters because renters hate being being kicked out too. Stability is important and many people currently pay massive premiums to not rent (including taking out massive 30 year loans on a place where they often don't even plan to live for half that long). Also, being forced to move at an inconvenient time is very expensive and has a hell of a lot of deadweight loss both economically and personally. (People also hate eminent domain.)

Of lesser but not no importance is fact that their taxes can go up to ridiculously high levels just because someone else built something valuable nearby will cause present day nimbyism to look very nice and kind (though it is kind of funny that people will likely switch what kind of nimbyism they support as well).

So, does your post bring up what I think are the problems with an land value tax? Yes, though I somewhat disagree with some of the emphasis.

Searching for new uses of land is pretty important especially over time, but we don't need new uses for things to work right now whereas the disruptions to people's lives would make things unworkable right now, and a lot of searching for new uses of land is done by people who do not currently own said land.

Implicitly taxing improvements to nearby land is obviously related to my point on nimbyism so we agree there, though it is interesting to note that seem to prefer talking about the internal version why I mostly reference the political version. The internal issue could obviously be 'fixed' by simply consolidating lots, but the political cannot without completely destroying the idea of individual ownership.

Your statements about the tax base narrowing issue is largely correct, but I would like to emphasize a different point of agreement that supporters seem to see it as simple, elegant, and easy but each patch makes it more complicated, kludgy, and difficult. I think that the very idea of evaluating the value of the land itself as separate from improvements actually starts out pretty difficult, so the increase in difficulty is a huge problem. Any incorrect overvaluation makes land worse than useless under this system! A system where a lot of land is useless very obviously leads to a lot of unused valuable land... which is exactly why people are currently complaining about speculation in land, but worse! (This is also deadweight loss.)

You don't go far enough when decrying the effect on people's confidence in the government, because it isn't just a confidence thing. Confiscating people's property without extremely good cause is one of the primary signs of living in a country that's either dirt poor, or right about to be, and will definitely stay that way (except in some rare cases where it only leads to massive famines, death, and stagnation rather than becoming poorer monetarily at first). It is also massively immoral.

The problem with your section on disrupting long-term plans is that you emphasize only the immediate problem of the transition, but don't mention that it also prevents the creation of new long term plans for a stable life unless you are willing to live a very bad life compared to how you could otherwise live. A full land value tax is therefore extremely dystopian.

Luckily, the proponents aren't currently finding much luck in their desired tax actually happening, and I hope it stays that way.

Math is definitely just a language. It is a combination of symbols and a grammar about how they go together. It's what you come up with when you maximally abstract away the real world, and the part about not needing any grounding was specifically about abstract math, where there is no real world.

Verifiable is obviously important for training (since we could give effectively infinite training data), but the reason it is verifiable so easily is because it doesn't rely on the world. Also, note that programming languages are also just that, languages (and quite simple ones) but abstract math is even less dependent on the real world than programming.

Math is just a language (a very simple one, in fact). Thus, abstract math is right in the wheelhouse for something made for language. Large Language Models are called that for a reason, and abstract math doesn't rely on the world itself, just the language of math. LLMs lack grounding, but abstract math doesn't require it at all. It seems more surprising how badly LLMs did math, not that they made progress. (Admittedly, if you actually mean ten years ago, that's before LLMs were really a thing. The primary mechanism that distinguishes the transformer was only barely invented then.)

For something to be a betrayal does not require knowing the intent of the person doing it, and is not necessarily modified if you do. I already brought up the fact that it would be perfectly fine if they had asked permission, it is in the not asking permission to alter the agreed upon course where the betrayal comes in. Saying 'I will do x' is not implicitly asking for permission at all, it is a statement of intent, that disregards entirely that there was even an agreement at all.

'what made A experience this as a betrayal' is the fact that it was. It really is that simple. You could perhaps object that it is strange to experience vicarious betrayal, but since it sounds like the four of you were a team, it isn't even that. This is a very minor betrayal, but if someone were to even minorly betray my family, for instance, I would automatically feel betrayed myself, and would not trust that person anymore even if the family member doesn't actually mind what they did.

Analogy time (well, another one), 'what makes me experience being cold' can be that I'm not generating enough heat for some personal reason, or it can just be the fact that I am outside in -20 degree weather. If they had experienced betrayal with the person asking for permission to do a move that was better for the group, that would be the former, but this is the latter. Now, it obviously can be both where a person who is bad at generating heat is outside when it is -20 degrees. (This is how what you are saying actually happened works out in this scenario.)

From what I've seen of how 'betrayal' is used, your definition is incorrect. (As far as I can tell) In general use, going against your agreement with another person is obviously betrayal in the sense of acting against their trust in you and reliance upon you, even if the intent is not bad. This is true even if the results are expected to be good.  So far as I know we do not have distinguishing words between 'betrayal with bad motives' and 'betrayal with good motives'.

 Another analogy, if a financial advisor embezzled your money because they saw a good opportunity, were right, and actually gave you your capital back along with most (or even all) of the gain before they were caught, that is still embezzling your money, which is a betrayal. Since they showed good intentions by giving it back before being caught, some people would forgive them when it was revealed, but it would still be a betrayal, and other people need not think this is okay even if you personally forgive it. Announcing the course of action instead of asking permission is a huge deal, even if the announcement is before actually doing it.

You can have a relationship where either party is believed to be so attuned to the needs and desires of the other person that they are free to act against the agreement and have it not be betrayal, but that is hardly normal. If your agreement had included, explicitly or through long history, 'or whatever else you think is best' then it wouldn't be a betrayal, but lacking that, it is. Alternately, you could simply announce to the group beforehand that you want people to use their best judgment on what to do rather than follow agreements with you. (This only works if everyone involved remembers that though.) The fact is that people have to rely on agreements and act based upon them, and if they aren't followed, there is little basis for cooperation with anyone whose interests don't exactly coincide. As you note, their objection was not to the course of action itself.

The damning part isn't the fact that they thought there was a new course of action that was better and wanted to do it (very few people object to thinking a new course of action is better if you are willing to follow the agreement assuming the other person doesn't agree), it was the not asking and the not understanding which both show a lack of trustworthiness and respect for agreements. This need not be a thing that has happened before, or that is considered super likely to occur again for it to be reasonable for another party to state that they hate such things, which one of the things being communicated. One thing objecting here does is tell the person 'you are not allowed to violate agreements with me without my permission.'

Also, they may be trying to teach the violator, as it is often the case that people try to teach morality, which may be why so much of philosophy is morality discussions. (Though I don't actually know how big of a factor that is.)

If there had been a reason they couldn't ask, then it would make more sense to do the seemingly better thing and ask for their approval after the fact. This is often true in emergencies, for instance, but also in times of extreme stress. Your friend wouldn't feel like it was a betrayal if the other person had instead gone to bathroom and never came back because they got a call that their best friend had just been hit by a car and they didn't think to tell people before leaving. If, on the other hand, the person acted unable to understand why they should explain themselves later, or that it wouldn't have been better if they had remembered to do so, that would be bizarre.

I do agree that considering the hypothesis that they may have experienced serious betrayal is useful (it is unfortunately common), which is why I think asking about it was potentially a good idea despite being potentially very awkward to bring up, but I think it is important not to commit to a theory to degrees beyond what is necessary.

I also agree that feeling understood is very important to people. From what I can tell, one of the primary reasons people don't bother to explain themselves is that they don't think the other person would understand anyway no matter how much they explained, with the others being that they wouldn't care or would use it against them.

Obviously, translating between different perspectives is often a very valuable thing to do. While there are a lot of disagreements that are values based, very often people are okay with the other party holding different values as long as they are still a good partner, and failure to communicate really is just failure to communicate.

I dislike the assumption that 'B' was reacting that way due to past betrayal. Maybe they were, maybe they weren't (I do see that 'B' confirmed it for you in a reaction to another comment, but making such assumptions is still a bad idea), but there doesn't have to be any past betrayal to object to betrayal in the present; people don't need to have ever been betrayed in the past to be against it as a matter of principle. They only need to have known that betrayal is a thing that exists, and they would probably be more upset even if they were somehow just learning of it at the time it happened. Leave out the parts that are unnecessary to the perspective. The more you assume, the more likely you are to fail to do it properly. You can ask the person if for some reason it seems important to know whether or not there is such a reason behind it, but you can't simply assume (if you care to be right). I personally find such assumptions about motives to be very insulting.

I personally find the idea that 'A' could not know what they were doing was clearly betrayal to be incomprehensible since people have heard countless stories of people objecting to altering things in this manner; this is not an uncommon case. Even if this person believes that consequences are the important part, there is no possible way to go through life without hearing people objecting countless times to unilaterally altering deals no matter what the altering party thinks of the consequences for the party they were agreeing with. This is similar to the fact that I don't understand why someone would want to go to a loud party with thumping bass and strobing lights, but I've heard enough stories to know that a lot of people genuinely do. I can say that such parties are bad because they are noisy and it is hard to see, but it is impossible not to know that some people disagree on those being bad for a party.

If someone only cared about the consequences of the action the agreement as judged by the other party, the agreement would have been to the criteria rather than the action. There is no need for an agreement at all if the criteria is just 'do whatever you think is best' (though the course of action may still be discussed of course). Also, it is quite easy to ask permission to alter an agreement whenever it seems simultaneously advantageous to all parties, and conditions where they can simply deviate can also be agreed upon in advance. The failure to ask permission should be seen as them refusing to think their agreements actually mean something, (especially when they don't immediately understand the objection), which makes for a very bad/unreliable partner. Additionally, asking if the other party thinks the new move is better gives an additional check on whether you came to the right conclusion on you evaluation.

I would strongly caution against assuming mindreading is correct. I think it is important to keep in mind that you don't know whether or not you've successfully inhabited the other other viewpoint. Stay open to the idea that the pieces got fit together wrong. Of course, it becomes easier in cases like the second one where you can just ask 'do you mean moon cheese?' (In that particular case, even the question itself might be enough to clue in the other party of the shape of the disagreement.)

When 'D' does agree that you're right, and 'C' still doesn't really get it, I suppose you now need to do the whole procedure again if it is worth continuing the discussion. You are correct to note it often isn't.

You might believe that the distinctions I make are idiosyncratic, though the meanings are in fact clearly distinct in ordinary usage, but I clearly do not agree with your misleading use of what people would be lead to think are my words and you should take care to not conflate things. You want people to precisely match your own qualifiers in cases where that causes no difference in the meaning of what is said (which makes enough sense), but will directly object to people pointing out a clear miscommunication of yours because you do not care about a difference in meaning. And you are continually asking me to give in on language regardless of how correct I may be while claiming it is better to privilege. That is not a useful approach.

(I take no particular position on physicalism at all.) Since you are a not a panpsychist, you would likely believe that consciousness is not common to the vast majority of things. That means the basic prior for if an item is conscious is, 'almost certainly not' unless we have already updated it based on other information. Under what reference class or mechanism should we be more concerned about the consciousness of an LLM than an ordinary computer running ordinary programs? There is nothing that seems particularly likely to lead to consciousness in its operating principles.

There are many people, including the original poster of course, trying to use behavioral evidence to get around that, so I pointed out how weak that evidence is.

An important distinction you seem to not see in my writing (whether because I wrote unclearly or you missed it doesn't really matter) is that when I speak of knowing the mechanisms by which an llm works is that I mean something very fundamental. We know these two things: 1)exactly what mechanisms are used in order to do the operations involved in executing the program (physically on the computer and mathematically) and 2) the exact mechanisms through which we determine which operations to perform.

As you seem to know, LLMs are actually extremely simple programs of extremely large matrices with values chosen by the very basic system of gradient descent. Nothing about gradient descent is especially interesting from a consciousness point of view. It's basically a massive use very simplified ODE solvers in a chain, which are extremely well understood and clearly have no consciousness at all if anything mathematical doesn't. It could also be viewed as just a very large number of variables in a massive but simple statistical regression. Notably, if gradient descent were related to consciousness directly, we would still have no reason to believe that an LLM doing inference rather than training would be conscious. Simple matrix math also doesn't seem like much of a candidate for consciousness either.

Someone trying to make the case for consciousness would thus need to think it likely that one of the other mechanisms in LLMs are related to consciousness, but LLMs are actually missing a great many mechanisms that would enable things like self-reflection and awareness (including a number that were included in primitive earlier neural networks such as recursion and internal loops). The people trying to make up for those omissions do a number of things to attempt to recreate it (with 'attention' being the built-in one, but also things like adding in the use of previous outputs), but those very simple approaches don't seem like likely candidates for consciousness (to me).

Thus, it remains extremely unlikely that an LLM is conscious.

When you say we don't know what mechanisms are used, you seem to be talking about not understanding a completely different thing than I am saying we understand. We don't understand exactly what each weight means (except in some rare cases that some researchers have seemingly figured out) and why it was chosen to be that rather than any number of other values that would work out similarly, but that is most likely unimportant to my point about mechanisms. This is, as far as I can tell, an actual ambiguity in the meaning of 'mechanism' that we can be talking about completely different levels at which mechanisms could operate, and I am talking about the very lowest ones.

Note that I do not usually make a claim about the mechanisms underlying consciousness in general except that it is unlikely to be these extremely basic physical and mathematical ones. I genuinely do not believe that we know enough about consciousness to nail it down to even a small subset of theories. That said, there are still a large number of theories of consciousness that either don't make internal sense, or seem like components even if part of it.

Pedantically, if consciousness is related to 'self-modeling' the implications involve it needing to be internal for the basic reason that it is just 'modeling' otherwise. I can't prove that external modeling isn't enough for consciousness, (how could I?) but I am unaware of anyone making that contention.

So, would your example be 'self-modeling'? Your brief sentence isn't enough for me to be sure what you mean. But if it is related to people's recent claims related to introspection on this board, then I don't think so. It would be modeling the external actions of an item that happened to turn out to be itself. For example, if I were to read the life story of a person I didn't realize was me, and make inferences about how the subject would act under various conditions, that isn't really self-modeling. On the other hand, in the comments to that, I actually proposed that you could train it on its own internal states, and that could maybe have something to do with this (if self-modeling is true). This is something we do not train current llms on at all though.

As far as I can tell (as someone who finds the very idea of illusionism strange), illusionism is itself not a useful point of view in regards to this dispute, because it would make the question of whether an LLM was conscious pretty moot. Effectively, the answer would be something to the effect of 'why should I care?' or 'no.' or even 'to the same extent as people.' regardless of how an LLM (or ordinary computer program, almost all of which process information heavily) works depending on the mood of the speaker. If consciousness is an illusion, we aren't talking about anything real, and it is thus useful to ignore illusionism when talking about this question.

As I mentioned before, I do not have a particularly strong theory for what consciousness actually is or even necessarily a vague set of explanations that I believe in more or less strongly.

I can't say I've heard of 'attention schema theory' before nor some of the other things you mention next like 'efference copy' (but the latter seems to be all about the body which doesn't seem all that promising a theory for what consciousness may be, though I also can't rule out that it being part of it since the idea is that it is used in self-modeling which I mentioned earlier I can't actually rule out either.).

My pet theory of emotions is that they are simply a shorthand for 'you should react in ways appropriate ways to a situation that is...' a certain way. For example (and these were not carefully chosen examples) anger would be 'a fight', happiness would be 'very good', sadness would be 'very poor' and so on. And more complicated emotions might obviously include things like it being a good situation but also high stakes. The reason for using a shorthand would be because our conscious mind is very limited in what it can fit at once. Despite this being uncertain, I find this a much more likely than emotions themselves being consciousness.

I would explain things like blindsight (from your ipsundrum link) through having a subconscious mind that gathers information and makes a shorthand before passing it to the rest of the mind (much like my theory of emotions). The shorthand without the actual sensory input could definitely lead to not seeing but being able to use the input to an extent nonetheless. Like you, I see no reason why this should be limited to the one pathway they found in certain creatures (in this case mammals and birds). I certainly can't rule out that this is related directly to consciousness, but I think it more likely to be another input to consciousness rather than being consciousness.

Side note, I would avoid conflating consciousness and sentience (like the ipsundrum link seems to). Sensory inputs do not seem overly necessary to consciousness, since I can experience things consciously that do not seem related to the senses. I am thus skeptical of the idea that consciousness is built on them. (If I were really expounding my beliefs, I would probably go on a diatribe about the term 'sentience' but I'll spare you that. As much as I dislike sentience based consciousness theories, I would admit them as being theories of consciousness in many cases.)

Again, I can't rule out global workspace theory, but I am not sure how it is especially useful. What makes a globabl workspace conscious that doesn't happen in an ordinary computer program I could theoretically program myself? A normal program might take a large number of inputs, process them separately, and then put it all together in a global workspace. It thus seems more like a theory of 'where does it occur' than 'what it is'.

'Something to do with electrical flows in the brain' is obviously not very well specified, but it could possibly be meaningful if you mean the way a pattern of electrical flows causes future patterns of electrical flows as distinct from the physical structures the flows travel through.

Biological nerves being the basis of consciousness directly is obviously difficult to evaluate. It seems too simple, and I am not sure whether there is a possibility of having such a tiny amount of consciousness that then add up to our level of consciousness. (I am also unsure about whether there is a spectrum of consciousness beyond the levels known within humans).

I can't say I would believe a slime mold is conscious (but again, can't prove it is impossible.) I would probably not believe any simple animals (like ants) are either though even if someone had a good explanation for why their theory says the ant would be. Ants and slime molds still seem more likely to be conscious to me than current LLM style AI though.

And here you are trying to be pedantic about language in ways that directly contradict other things you've said in speaking to me. In this case, everything I said holds if we change between 'not different' and 'not that different' (while you actually misquote yourself as 'not very different'). That said, I should have included the extra word in quoting you.

Your point is not very convincing. Yes, people disagree if they disagree. I do not draw the lines in specific spots, as you should know based on what I've written, but you find it convenient to assume I do.

Do you hold panpsychism as a likely candidate? If not, then you most likely believe the vast majority of things are not conscious. We have a lot of evidence that the way it operates is not meaningfully different in ways we don't understand from other objects. Thus, almost the entire reference class would be things that are not conscious. If you do believe in panpsychism, then obviously AIs would be too, but it wouldn't be an especially meaningful statement.

You could choose computer programs as the reference class, but most people are quite sure those aren't conscious in the vast majority of cases. So what, in the mechanisms underlying an llm is meaningfully different in a way that might cause consciousness? There doesn't seem to be any likely candidates at a technical level. Thus, we should not increase our prior from that of other computer programs. This does not rule out consciousness, but it does make it rather unlikely.

I can see you don't appreciate my pedantic points regarding language, but be more careful if you want to say that you are substituting a word for what I used. It is bad communication if it was meant as a translation. It would easily mislead people into thinking I claimed it was 'self-evident'. I don't think we can meaningfully agree to use words in our own way if we are actually trying to communicate since that would be self-refuting (as we don't know what we are agreeing to if the words don't have a normal meaning).