Recent Mechanistic Interpretability (MI) work shows Large Language Models (LLMs) have emotional representations with geometric structure matching human affect. This doesn't prove LLMs deserve moral consideration, but it establishes a necessary condition.
Re "establishes a necessary condition": It seems rather than proving it to be a necessary condition, you assume it to be a necessary condition; while instead, I think we could well imagine that "geometric structures matching human affect" (unless you define that category as so broad that it becomes a bit meaningless) are instead not the only way to sentience i.e. moral consideration.
Agree though more generally WBE can be a useful starting point for thought experiments on AI sentience. Forcing a common starting point for discussion. Although even at that starting point there can be two positions: the usual one that you invoke, plus illusionism (which I personally think is underrated even if I agree it feels so hard to entertain).
I was always slightly suspicious of the claims that we had x (5 or so?) times been closer to entering the big nuclear war than to not entering it. But if this passage is accurate, then the fact that some of the usual claims are so easy to put into perspective would suggest in some communities we are also a bit more affected by sensationalism than I'd have thought. Interesting, thanks!
[tone to be taken with a grain of salt, meant as a proposition but I thought to write it a bit provocatively]
No, the more fundamental problem is: WHATEVER it tells you, you can NEVER infer with anything like certainty whether it's conscious (at least if we agree to mean sentient with conscious). Why do I write such a preposterous thing like I know that you cannot know? Very simple: Presumably we agree that we cannot be certain A PRIORI whether any type of current CPU, with whichever software run on it, can become sentient. If there are thus two possible states of the world,
A. current CPU computers cannot become sentient
B. with the right software run on it, sentience can arise
Then, because once you take Claude and its training method & data, you can perfectly track bit by bit why it spits out its sentience-suggestive & deep speek, you know your observations about the world you find yourself in, are just as probable under A as under B! The only Bayesian valid inference then is: Having observed hippy's sentience-suggestive & deep speek, you're just as clueless about whether you're in B. or in A.
tl;dr
- Basic market competition effect: Ambiguous (higher customer rents; lower producer rents by other firms), see Econ 101 and Econ 102 as detailed below.
- Bullshittiness of markets: Totally!
- But these are two rather separate effects and should be more cleanly separated for fruitful analysis
- All in all: But despite all, hard to quantify the net effect of an 'average' successful business; I find your title insufficiently substantiated.
Explanation:
Econ 101 model of Monopolistic Competition describes exactly the basic market effect you're going at.
While Econ 101 is though in its most basic form a bit stupid and blind on the more interesting questions you address, when you start from that model and look at effects of market entry, you see that two first-order effects challenge your a priori of there automatically being a reduction of net welfare:
- Yes Tom bites into Fred's profit, i.e. negative externality on others' producer rent. But he also could not attract any single customer w/o increasing consumer welfare (alias consumer rent). Which of the two dominate is absolutely unclear and will depend on subtle market parameters.
- Together the competitors drive down each others' prices, as Richard_Kennaway points out. This increases net rents in the market.
This may be called Econ 102. You find a ton written about it under keyword "business-stealing effect". Conclusion: In some cases free entry has, on the margin at the equilibrium, positive net welfare effects, in some negative.
I'm actually sympathetic that many firms are mostly gaining because of sneakiness that is remote to actual increase of value for customers and society, but it seems to me your post is mixing up (i) fundamental effect that you analyze not in the depth required (the above mentioned additional fundamental basic competition effects) and - in some of your examples later - (ii) the more tricky marketing/sneakiness advantages some profiteer from. These should be separated more clearly for a fruitful discussion of the topic. To be clear what I mean with the latter: Acc. to my experience, a large share of firms mainly tinkers around how to beat competition not in terms of creating better products but in terms of how to better sell it etc., or how to cut costs in gray-zone areas or at least in ways that are not mostly value-adding but instead leading to other negative externalities, so all w/o it necessarily going in the direction of yielding higher value for customer and/or society. And thus also: huge share of bullshit jobs; plus if some jobs aren't full bullshit jobs, they still have a high share of bullshit components (although I guess we should be careful to distinguish within-company bullshittiness and indeed outwardgoing bullshit created, of which only the latter goes in the direction of your argument).
FWIW Tangential topic: Low marginal costs
A related effect I find annoying is that we rely on competition and markets while more and more goods are quasi-artificially-scarce (low marginal cost goods) but with high fixed costs. Independent private business don't want and cannot sell these goods at efficient low prices. If by pure luck we stumbled into a future with a benevolent AI future, we'll have some directed communistic element complementing the market economy. Sadly, otherwise, we'll probably be stuck with highly inefficiently high prices and, yes, maybe with too many firms doing similar things without gains justifying duplication costs.
Agree now turning 40 or 20 need not make a bit difference for those aware of the weirdness of the time.
But: Seems like a stretch to say it's already been like that few decades ago. Now the sheer uncertainty seems objectively different, qualitatively truly incomparable, to 20y ago (well, at least if the immediacy of potential changes is considered too).
2Agree. Wonder whether one should go a step further: Rather than a specific 'wiki'/series of comments, maybe the domain would deserve an entire tagged (and ideally hideable for those rather categorically uninterested) sub-part of LessWrong, which which I mean: entire LW domain with posts of all LW categories and genres etc., but clearly targetting mostly only parents of children.
I guess that would then become a question of filtering out specific tags for users.
[There's also a much more banal answer that I wouldn't be surprised if it is a major, deep underlying driver, with all the interesting psychology provided in OP being some sort of half-conscious rationalization for our actual deep-rooted tendencies:] Not going insane simply is the very natural default outcome for humans even in such felt dire situation:
While shallowly it might feel like it would, going insane actually appears to me to NOT AT ALL be the default human reaction to an anticipation of (even a quite high probability of) the world ending (even very soon). I haven't done any stats or research, but everything I've ever seen or heard of seems to suggest to me:
- While they're not anywhere nearly the majority, still very many people have very high P(doom soon) yet stay nearly perfectly calm (at best you might call them insanely calm, given the [true or imagined] circumstances).
- I think this applies to many people e.g. on this forum, but I'm reminded of much more 'normal' persons uttering even more dramatic 'I'm sure AI might already TMORROW kill us all' - all while simply going on with their usual lives.
- Slightly less 1:1 but imho still underlining our sanity's resilience in closely as dire situations: Many people seem egoistic enough such that the ending of their own life to mean a very large part of the world they care about to be going to end, and yet they face many situations of more or less imminent death rather calmly as opposed to going insane
- Extend to various cases where family and/or friends and/or tribe is facing extinction; at least I haven't heard of them to usually be going insane by the prospect of not-yet-actually-visible but forthcoming extinction.
- Once a torturous way of you or your close ones being killed has actually started, that's of course different, that's when you go insane.
- Extend to various cases where family and/or friends and/or tribe is facing extinction; at least I haven't heard of them to usually be going insane by the prospect of not-yet-actually-visible but forthcoming extinction.
1
1
1I find the argument/thesis brought fwd here, in its strong form, too abstract and lacking sufficient empirical justification, even if purely directionally the main claim need not be entirely wrong (which, though, isn't a very surprising conclusion anymore). In reality, how often one might ideally share your differing opinions will depend on more subtle details of topic, colleagues, situation, incl. the exact ways how one can share/emphasize the opinion.
Intended mainly for OP but I post publicly still fwiw.
Thanks! What coincidence: I went to the doc yesterday because once again my chronic palate inflammation as been flaring up more strongly than usual in the past 1-2 months or so. Since ages, I seem to have fragile mucous membranes and some other skin/pseurasis-type issues e.g. on the head; while the latter are under control with medicinal shampoos, the palate inflammation seems to be more of a constant, although at varying intensity.
I had seen docs many times many years before that, but I also simply have gotten used to having inflammation and latent pain there as nothing can seemingly be done about it. The main worry is the broader risks from long-term chronic inflammation.
But yesterday, for the first time, a doc told me that before anything else: "go to see a dentist as very first thing - bacteria something something". I didn't want to take her seriously; rather good tooth hygiene - though indeed relatively strongly withdrawn gum.
Long story short: If you happen to have good links (or other sort of info that doesn't take you too long to provide) I'd be keen to know more details about anything you write in your post thus. I should admit, despite the amazing parallels of what you write, my hope is a bit limited, as I still guess (i) my irritated/fragile mucous membranes are somehow a rather fundamental issue, and, at any rate, (ii) gum being somewhat withdrawn can barely be changed.
I think (i) your reasoning is flawed, though - even if barely anyone will be agreeing to it - (ii) actually have some belief in something related to what you say
(i) YOUR BAYESIAN REASONING IS FLAWED:
As Yair points out, one can easily take a different conclusion from your starting point, and maybe it's best to stop there. Here still an attempt of a Bayesian tracking why; it's all a bit trivial, but maybe its worth could be: if you really believe in the conclusions brought in OP, maybe you can take it as a starting point and pinpoint where exactly you'd argue a Bayesian implementation of the reflection ought to look differently.
Assume we have, in line with your setup, two potential states of the world - without going into detail as to what these terms would even mean:
A = Unified Consciousness
B = Separate Consciousness for each individual
The world is, of course, exactly the same in both cases, except for this underlying feature. So any Joe born in location xyz at date abc will be that exact same Joe born then and there under either of the hypothetical A and B, except for the underlying nature of his consciosusness to differ in the sense of A vs. B.
We know there are
Potential and actual numbers are the same in world case A and world case B, just their consciousness(es) is/are somehow of a different nature.
Let's start with an even prior:
P(Unified Consciousness) = P(Separated Consciousnesses) = 0.5
Now, consider in both hypothetical worlds a random existing human # 7029501952, born to the name of Joe, among the 9 bn existing ones. Joe can indeed ask himself: "Given that I exist - wow, I exist! - how likely is that there is a unified vs. separate... He does the Bayesian update given his evidence at hand. From his perspective
P(A | Joe exist) = P(Joe exist | A) x P(A)/P(Joe exist)
P(B | Joe exist) = P(Joe exist | B) x P(B)/P(Joe exist)
As we're in a bit a weird thought experiment, you may argue to have only one or two of the following possibilities to evaluate this ( think the first makes more sense as we're talking about his perspective, but if you happen to prefer seeing it the other way round; won't change anything):
If you substitute that in you get one of
And the same 0.5 in both cases for P(B | Joe exists).
So, the probability of A and of B remains at 0.5 just as it initially was.
In simplified words - just like the maths also they feel a bit trivial: Given by definition only the existing humans - no matter whether their atomic consciousnesses or somehow one single connected one - exist, and can thus ask themselves about their existence, the fact that they exist despite the many hypothetical humans individually only rarely becoming actual existences, doesn't reduce the probability of them having been born into a world of type B as opposed to type A. I.e., whatever our prior for world A vs. world B, your type of reasoning does not actually yield any changed posterior.
(ii) I THINK UNIFIED CONSCIOUSNESS - IN SOME SENSE - MAKES SORT OF SENSE
FWIW I'm half convinced we can sort of know we're more 'one' than 'separate' as it follows from a observation insight and thought experiment: (a) there's not much more in "us" at any given moment than an instantaneous self and memories and intentions/preferences regarding a future self that happens to be in the same 'body', but (b) it suffices any random selection of a large set of thought experiments about sleeping/cloning/awaking that can show we can very happily imagine ourselves to 'be' an entire different future in the next moment in a way that imho can best be made sense of if there is not really just a stable and well-defined long-term self but instead (either no such thing as any self in any meaningful way, i.e. something a bit illusionist or) a wholly flimsy/random continuation of self, in a way that may well best be described as there being a single self or something (and half esoterically I derive from it I should really better care about everyone's welfare equally well as opposed to mainly about the one of my own physical longer-term being, though it's all fuzzy), as I try to explain in Relativity Theory for What the Future 'You' Is and Isn't.