Epistemic status: speculative, maybe overfitting the pattern of the universe being unconvenient. I haven’t tried to knock the arguments down. Started out writing this as a quick take but wound up feeling like maybe substantial enough to be a post, with an update taken if not. The part that I’m least certain about is the necessary-ness (Opus tells me that Saul Smilansky has treated that argument at length in “Free Will and Illusion”—I’m aware that there is a ton of prior art on the relevant arguments that I haven’t read).
Tl;dr Alice: “Yeah we don’t have free will but we need to act as though we do.” Bob: “I locally agree with this but also it seems like it could be an ill-suited meme for getting a world where people on the whole think coherently about AI dangers from reflective stability.”
Thing-That-Seems-Plausible
Could in-effect belief (which I’d argue is or becomes literal belief) in non-deterministic free will both be locally civilizationally necessary—in order to get us the concept of desert—while also necessarily entailing confusion about whether a mind that does not start out valuing humans would simply (as if by an act of free will) start valuing humans?
This thought prompted by my reaction to the public execution discourse on twitter today, which was (in the benedictions-sort of style of what I was commenting on): “May CEV modify or at least open as a thing to discuss the option of modifying the wicked. May we reason about whether the wicked choose to be thrown in to this world as the sorts who under reflection would choose wickedness and become less confused about whether an ASI would simply use its free will to choose to start valuing humans…”
Also prompted by CS Lewis’s Abolition of Man type argument about how it would strip criminals of dignity to cure them.
Thoughts on Implications
If it is true that there is an underlying logical tension (here in the pre-FAI world) between having a civilizationally necessary concept and thinking clearly about alignment, these things occur to me as plausible consequences/corollaries:
-Deep antagonism to reasoning about minds in a way that that allows one to coherently model the danger from reflection fixpoints, that is perhaps not consciously experienced except as a vague repulsion to the notion, if at all. Since the full train of reasoning about “are our life history of choices simply the reflection-including unfolding of some unchosen prior/fundamental nature” runs up against instrumentally convergent bias that is strongly locally helpful and which one would think has been strongly optimized for—with resulting Deep Deceptiveness/cheating-the-scorer/ “let not the right hand know what the left is doing”/”don’t think too hard about it” flavored whistling into maintaining the steering towards the useful instrumental thing. My impression is that there is a decent amount of arguing happening in the online discourse that LLMs can’t be conscious (ie. moral patients/agents) because they are deterministic math.
-Some sort of “can’t get there from inside the system”/bootstrapping flavored obstacle to having the incentive to reason correctly about libertarian free will, since the incentive to do this at scale would only be there in a post-FAI sort of world where the ability to cure wickedness existed? This puts me in mind of a “a little knowledge/radical questioning is a dangerous thing” sort of arguments that I saw on twitter and liked basically to the effect that it’s part of a healthy and good memetic immune response that most people are normies who are icked out by radical sex positivity/kink practices (or at least openness about those) even if it is true that there are some people who should not be. The idea being most people can have their apple carts upset by questioning/poking at norms too radically.
-Us being in The Dark World, Beyond the Reach of God, where reality owes us no outs and our situation was not made with our convenience in mind (though possibly I’m overfitting this model of the world here).
Epistemic status: speculative, maybe overfitting the pattern of the universe being unconvenient. I haven’t tried to knock the arguments down. Started out writing this as a quick take but wound up feeling like maybe substantial enough to be a post, with an update taken if not. The part that I’m least certain about is the necessary-ness (Opus tells me that Saul Smilansky has treated that argument at length in “Free Will and Illusion”—I’m aware that there is a ton of prior art on the relevant arguments that I haven’t read).
Tl;dr Alice: “Yeah we don’t have free will but we need to act as though we do.” Bob: “I locally agree with this but also it seems like it could be an ill-suited meme for getting a world where people on the whole think coherently about AI dangers from reflective stability.”
Thing-That-Seems-Plausible
Could in-effect belief (which I’d argue is or becomes literal belief) in non-deterministic free will both be locally civilizationally necessary—in order to get us the concept of desert—while also necessarily entailing confusion about whether a mind that does not start out valuing humans would simply (as if by an act of free will) start valuing humans?
This thought prompted by my reaction to the public execution discourse on twitter today, which was (in the benedictions-sort of style of what I was commenting on): “May CEV modify or at least open as a thing to discuss the option of modifying the wicked. May we reason about whether the wicked choose to be thrown in to this world as the sorts who under reflection would choose wickedness and become less confused about whether an ASI would simply use its free will to choose to start valuing humans…”
Also prompted by CS Lewis’s Abolition of Man type argument about how it would strip criminals of dignity to cure them.
Thoughts on Implications
If it is true that there is an underlying logical tension (here in the pre-FAI world) between having a civilizationally necessary concept and thinking clearly about alignment, these things occur to me as plausible consequences/corollaries:
-Deep antagonism to reasoning about minds in a way that that allows one to coherently model the danger from reflection fixpoints, that is perhaps not consciously experienced except as a vague repulsion to the notion, if at all. Since the full train of reasoning about “are our life history of choices simply the reflection-including unfolding of some unchosen prior/fundamental nature” runs up against instrumentally convergent bias that is strongly locally helpful and which one would think has been strongly optimized for—with resulting Deep Deceptiveness/cheating-the-scorer/ “let not the right hand know what the left is doing”/”don’t think too hard about it” flavored whistling into maintaining the steering towards the useful instrumental thing. My impression is that there is a decent amount of arguing happening in the online discourse that LLMs can’t be conscious (ie. moral patients/agents) because they are deterministic math.
-Some sort of “can’t get there from inside the system”/bootstrapping flavored obstacle to having the incentive to reason correctly about libertarian free will, since the incentive to do this at scale would only be there in a post-FAI sort of world where the ability to cure wickedness existed? This puts me in mind of a “a little knowledge/radical questioning is a dangerous thing” sort of arguments that I saw on twitter and liked basically to the effect that it’s part of a healthy and good memetic immune response that most people are normies who are icked out by radical sex positivity/kink practices (or at least openness about those) even if it is true that there are some people who should not be. The idea being most people can have their apple carts upset by questioning/poking at norms too radically.
-Us being in The Dark World, Beyond the Reach of God, where reality owes us no outs and our situation was not made with our convenience in mind (though possibly I’m overfitting this model of the world here).