There's a lot of "neuralink will make it easier to solve the alignment problem" stuff going around the mainstream internet right now in response to neuralink's recent demo.
I'm inclined to agree with Eliezer, that seems wrong; either AGI will be aligned in which case it will make its own neuralink and wont need ours, or it will be unaligned and you really wouldn't want to connect with it. You can't make horses competitive with cars by giving them exoskeletons.
But, is there much of a reason to push back against this?
Providing humans with cognitive augmentation probably would help to solve the alignment problem, in a bunch of indirect ways.
It doesn't seem like a dangerous error at all. It feeds a public desire to understand how AGI might work. Neuralink itself is a great project for medical science. Generally, wrong beliefs cause bad consequences, but I'm having difficulty seeing what they'd be here.
The obvious bad consequence is a false sense of security leading people to just get BCIs instead of trying harder to shape (e.g. delay) AI development.
" You can't make horses competitive with cars by giving them exoskeletons. " <-- this reads to me like a separate argument, rather than a restatement of the one that came before.
I agree that BCI seems unlikely to be a good permanent/long-term solution, unless it helps us solve alignment, which I think it could. It could also just defuse a conflict between AIs and humans, leading us to gracefully give up our control over the future light cone instead of fighting a (probably losing) battle to retain it.
...Your post made me think more about my own (and others') reasons for rejecting Neuralink as a bad idea... I think there's a sense of "we're the experts and Elon is a n00b". This coupled with feeling a bit burned by Elon first starting his own AI safety org and then ditching it for this... overall doesn't feel great.
I've never been mad at elon for not having decision theoretic alignmentism. I wonder, should I be mad. Should I be mad about the fact that he has never talked to eliezer (eliezer said that in passing a year or two ago on twitter) even though he totally could whenever he wanted.
Also, what happened at OpenAI? He appointed some people to solve the alignment problem, I think we can infer that they told him, "you've misunderstood something and the approach you're advocating (proliferate the technology?) wouldn't really be all that helpful", and he responded badly to that? They did not reach mutual understanding?
Idea: Screen burn correction app that figures out how to exactly negate your screen's issues by pretty much looking at itself in a mirror through the selfie cam, trying to display pure white, remembering the imperfections it sees, then tinting everything with the negation of that from then on.
Nobody seems to have made this yet. I think there might be things for tinting your screen in general, but it doesn't know the specific quirks of your screenburn. Most of the apps for screen burn recommend that you just burn every color over the entire screen that isn't damaged yet, so that they all get to be equally damaged, which seems like a really bad thing to be recommending.
Considering doing a post about how it is possible the Society for Cryobiology might be wrong about Cryonics, it would have something to do with the fact that at least until recently, no cryobiologist who was seriously interested in cryonics was allowed to be a member,
but I'm not sure... their current position statement is essentially "it is outside the purview of the Society for Cryobiology", which, if sincere, would have to mean that the beef is over?
Noticing I've been operating under a bias where I notice existential risk precursors pretty easily (EG, biotech, advances in computing hardware), but I notice no precursors of existential safety. To me it is as if technologies that tend to do more good than harm, or at least, would improve our odds by their introduction, social or otherwise, do not exist. That can't be right, surely?...
When I think about what they might be... I find only cultural technologies, or political conditions: the strength of global governance, the clarity of global discourses, perhaps the existence of universities. But that can't be it. These are all low hanging fruit, things that already exist. Differential progress is about what could be made to exist.
Some biotech contributes to existential risk but others doesn't. A lot of vaccine technology doesn't increase existential risk but reduces it because of reduced danger from viruses. Phage therapy is the same for reducing the risk from infectious bacteria.
LessWrong itself is a form of technology that's intended to lead to existential risk reduction by facillitating a knowledge community to exist that otherwise wouldn't.
The general idea of CFAR is that social technology they developed like double crux helps people to think more clearly and thus reduce existential risk.
Hmm. It appears to me that Qualia are whatever observations affect indexical claims, and anything that affects indexical claims is a qualia, and this is probably significant
Yes, this seems straightforwardly true, although I don't think it's especially significant unless I'm failing to think of some relevant context about why you think indexical claims matter so much (but then I don't spend a lot of time thinking very hard about semantics in a formal context, so maybe I'm just failing to grasp what all is encompassed by "indexical").
It's important because demystifying qualia would win esteem in a very large philosophical arena, heh. More seriously though, it seems like it would have to strike at something close to the heart of the meaning of agency.
Hmm. It appears to me that Qualia are whatever observations affect indexical claims, and anything that affects indexical claims is a qualia
I don't think so, here is a counter-example:
Alice and Bob start talking in a room. Alice has an identical twin, Alex. Bob doesn't know about the twin and thinks he's talking to Alex. Bob asks: "How are you today?". Before Alice responds, Alex walks in.
Bob's observation of Alex will surprise him, and he'll quickly figure out that something's going on. But more importantly: Bob's observation of Alex alters the indexical 'you' in "How are you today?" (at least compared to Bob's intent, and it might change for Alice if she realises Bob was mistaken, too).
I don't think this is anything close to describing qualia. The experience of surprise can be a quale, the feeling of discovering something can be a quale (eureka moments), the experience of the colour blue is a quale, but the observation of Alex is not.
Do you agree with this? (It's from https://plato.stanford.edu/entries/indexicals/)
An indexical is, roughly speaking, a linguistic expression whose reference can shift from context to context. For example, the indexical ‘you’ may refer to one person in one context and to another person in another context.
That's a well constructed example I think, but no that seems to be a completely different sense of "indexical". The concept of indexical uncertainty we're interested in is... I think... uncertainty about which kind of body or position in the universe your seat of consciousness is in, given that there could be more than one. The Sleeping Beauty problem is the most widely known example. The mirror chamber was another example.
The concept of indexical uncertainty we're interested in is... I think... uncertainty about which kind of body or position in the universe your seat of consciousness is in, given that there could be more than one.
I'm not sure I understand yet, but does the following line up with how you're using the word?
Indexical uncertainty is uncertainty around the exact matter (or temporal location of such matter) that is directly facilitating, and required by, a mind. (this could be your mind or another person's mind)
Notes:
"exact" might be too strong a word
I added "or temporal location of such matter" to cover the sleeping beauty case (which, btw, I'm apparently a halfer or double halfer according to wikipedia's classifications, but haven't thought much about it)
Edit/PS: I think my counter-example with Alice, Alex, and Bob still works with this definition.
I think I'm saying those are going to to turn out to be the same thing, though I'm not sure exactly where that intuition is coming from yet. Could be wrong.
As I get closer to posting my proposal to build a social network that operates on curators recommended via webs of trust, it is becoming easier for me to question existing collaborative filtering processes.
And, damn, scores on posts are pretty much meaningless if you don't know how many people have seen the post, how many tried to read it, how many read all of it, and what the up/down ratio is. If you're missing one of those pieces of information, then there exists an explanation for a low score that has no relationship to the post's quality, and you can't use the score to make a decision as to whether to give it a chance.
An analytic account of Depression: When the agent has noticed that strategies that seemed fruitful before have stopped working, and doesn't have any better strategies in mind.
I imagine you'll often see this type of depression behavior in algorithmic trading strategies, as soon as they start consistently losing enough money to notice that something must have changed about the trading environment, maybe more sophisticated strategies have found a way to dutch book them. Those strategies will then be retired, and the trader or their agency will have to search for new ones.
Chronic depression in humans (designed agencies wouldn't tend to have such an obvious bug) kinda feels like when the behavior of searching for new strategies has itself has been caught within depression's scope as an invalidated strategy.
Searching anew is what's supposed to happen (I had a post called "Good Gloom" that drew an analogy to turning all of the lights out in a town so that you'll be able to see the light pollution of a new town, and set out for it), but if the depression is blocking that too, you stop moving.
My opinion is that the St Petersberg game isn't paradoxical, it is very valuable, you should play it, it's counterintuitive to you because you can't actually imagine a quantity that comes in linear proportion to utility, you have never encountered one, none seems to exist.
Money, for instance, is definitely not linearly proportionate to utility, the more you get the less it's worth to you, and at its extremes, it can command no more resources than what the market offers, and if you get enough of it, the market will notice and it will all become valueless.
Every resource that exists has sub-linear utility returns in the extremes.
(Hmm. What about land? Seems linear, to an extent)
Wild Speculative Civics: What if we found ways of reliably detecting when tragedies of the commons have occurred, then artificially increased their costs (charging enormous fines) to anyone who might have participated in creating them, until it's not even individually rational to contribute to them any more?
That sounds like punishing any usage of common resources which is likely undesireable.
Good policy to manage individual commons requires to think through how their usage is best managed. Elinor Ostrom did a lot of research into what works for setting up good systems.
I can imagine a few technical problems, like determining what level of usage is optimal (you don't want people to overfish the lake, but you don't know exactly how many fish are there), or the costs of policing. But it would be possible to propose a few dozen situations where this strategy could be used, and address these issues individually; and then perhaps only use the strategy in some of them. Or perhaps by examining individual specific cases, we would discover a common pattern why this doesn't work.
There's a lot of "neuralink will make it easier to solve the alignment problem" stuff going around the mainstream internet right now in response to neuralink's recent demo.
I'm inclined to agree with Eliezer, that seems wrong; either AGI will be aligned in which case it will make its own neuralink and wont need ours, or it will be unaligned and you really wouldn't want to connect with it. You can't make horses competitive with cars by giving them exoskeletons.
But, is there much of a reason to push back against this?
Providing humans with cognitive augmentation probably would help to solve the alignment problem, in a bunch of indirect ways.
It doesn't seem like a dangerous error at all. It feeds a public desire to understand how AGI might work. Neuralink itself is a great project for medical science. Generally, wrong beliefs cause bad consequences, but I'm having difficulty seeing what they'd be here.
The obvious bad consequence is a false sense of security leading people to just get BCIs instead of trying harder to shape (e.g. delay) AI development.
" You can't make horses competitive with cars by giving them exoskeletons. " <-- this reads to me like a separate argument, rather than a restatement of the one that came before.
I agree that BCI seems unlikely to be a good permanent/long-term solution, unless it helps us solve alignment, which I think it could. It could also just defuse a conflict between AIs and humans, leading us to gracefully give up our control over the future light cone instead of fighting a (probably losing) battle to retain it.
...Your post made me think more about my own (and others') reasons for rejecting Neuralink as a bad idea... I think there's a sense of "we're the experts and Elon is a n00b". This coupled with feeling a bit burned by Elon first starting his own AI safety org and then ditching it for this... overall doesn't feel great.
I've never been mad at elon for not having decision theoretic alignmentism. I wonder, should I be mad. Should I be mad about the fact that he has never talked to eliezer (eliezer said that in passing a year or two ago on twitter) even though he totally could whenever he wanted.
Also, what happened at OpenAI? He appointed some people to solve the alignment problem, I think we can infer that they told him, "you've misunderstood something and the approach you're advocating (proliferate the technology?) wouldn't really be all that helpful", and he responded badly to that? They did not reach mutual understanding?
Idea: Screen burn correction app that figures out how to exactly negate your screen's issues by pretty much looking at itself in a mirror through the selfie cam, trying to display pure white, remembering the imperfections it sees, then tinting everything with the negation of that from then on.
Nobody seems to have made this yet. I think there might be things for tinting your screen in general, but it doesn't know the specific quirks of your screenburn. Most of the apps for screen burn recommend that you just burn every color over the entire screen that isn't damaged yet, so that they all get to be equally damaged, which seems like a really bad thing to be recommending.
Considering doing a post about how it is possible the Society for Cryobiology might be wrong about Cryonics, it would have something to do with the fact that at least until recently, no cryobiologist who was seriously interested in cryonics was allowed to be a member,
but I'm not sure... their current position statement is essentially "it is outside the purview of the Society for Cryobiology", which, if sincere, would have to mean that the beef is over?
( statement is https://www.societyforcryobiology.org/assets/documents/Position_Statement_Cryonics_Nov_18.pdf )
Noticing I've been operating under a bias where I notice existential risk precursors pretty easily (EG, biotech, advances in computing hardware), but I notice no precursors of existential safety. To me it is as if technologies that tend to do more good than harm, or at least, would improve our odds by their introduction, social or otherwise, do not exist. That can't be right, surely?...
When I think about what they might be... I find only cultural technologies, or political conditions: the strength of global governance, the clarity of global discourses, perhaps the existence of universities. But that can't be it. These are all low hanging fruit, things that already exist. Differential progress is about what could be made to exist.
Some biotech contributes to existential risk but others doesn't. A lot of vaccine technology doesn't increase existential risk but reduces it because of reduced danger from viruses. Phage therapy is the same for reducing the risk from infectious bacteria.
LessWrong itself is a form of technology that's intended to lead to existential risk reduction by facillitating a knowledge community to exist that otherwise wouldn't.
The general idea of CFAR is that social technology they developed like double crux helps people to think more clearly and thus reduce existential risk.
Hmm. It appears to me that Qualia are whatever observations affect indexical claims, and anything that affects indexical claims is a qualia, and this is probably significant
Yes, this seems straightforwardly true, although I don't think it's especially significant unless I'm failing to think of some relevant context about why you think indexical claims matter so much (but then I don't spend a lot of time thinking very hard about semantics in a formal context, so maybe I'm just failing to grasp what all is encompassed by "indexical").
It's important because demystifying qualia would win esteem in a very large philosophical arena, heh. More seriously though, it seems like it would have to strike at something close to the heart of the meaning of agency.
I don't think so, here is a counter-example:
Alice and Bob start talking in a room. Alice has an identical twin, Alex. Bob doesn't know about the twin and thinks he's talking to Alex. Bob asks: "How are you today?". Before Alice responds, Alex walks in.
Bob's observation of Alex will surprise him, and he'll quickly figure out that something's going on. But more importantly: Bob's observation of Alex alters the indexical 'you' in "How are you today?" (at least compared to Bob's intent, and it might change for Alice if she realises Bob was mistaken, too).
I don't think this is anything close to describing qualia. The experience of surprise can be a quale, the feeling of discovering something can be a quale (eureka moments), the experience of the colour blue is a quale, but the observation of Alex is not.
Do you agree with this? (It's from https://plato.stanford.edu/entries/indexicals/)
Btw, 'qualia' is the plural form of 'quale'
That's a well constructed example I think, but no that seems to be a completely different sense of "indexical". The concept of indexical uncertainty we're interested in is... I think... uncertainty about which kind of body or position in the universe your seat of consciousness is in, given that there could be more than one. The Sleeping Beauty problem is the most widely known example. The mirror chamber was another example.
I'm not sure I understand yet, but does the following line up with how you're using the word?
Indexical uncertainty is uncertainty around the exact matter (or temporal location of such matter) that is directly facilitating, and required by, a mind. (this could be your mind or another person's mind)
Notes:
Edit/PS: I think my counter-example with Alice, Alex, and Bob still works with this definition.
I can see how this might result from confusing consciousness qua phenomenonality with consciousness qua personal identity.
I think I'm saying those are going to to turn out to be the same thing, though I'm not sure exactly where that intuition is coming from yet. Could be wrong.
Why would that be the case?
As I get closer to posting my proposal to build a social network that operates on curators recommended via webs of trust, it is becoming easier for me to question existing collaborative filtering processes.
And, damn, scores on posts are pretty much meaningless if you don't know how many people have seen the post, how many tried to read it, how many read all of it, and what the up/down ratio is. If you're missing one of those pieces of information, then there exists an explanation for a low score that has no relationship to the post's quality, and you can't use the score to make a decision as to whether to give it a chance.
An analytic account of Depression: When the agent has noticed that strategies that seemed fruitful before have stopped working, and doesn't have any better strategies in mind.
I imagine you'll often see this type of depression behavior in algorithmic trading strategies, as soon as they start consistently losing enough money to notice that something must have changed about the trading environment, maybe more sophisticated strategies have found a way to dutch book them. Those strategies will then be retired, and the trader or their agency will have to search for new ones.
Chronic depression in humans (designed agencies wouldn't tend to have such an obvious bug) kinda feels like when the behavior of searching for new strategies has itself has been caught within depression's scope as an invalidated strategy.
Searching anew is what's supposed to happen (I had a post called "Good Gloom" that drew an analogy to turning all of the lights out in a town so that you'll be able to see the light pollution of a new town, and set out for it), but if the depression is blocking that too, you stop moving.
My opinion is that the St Petersberg game isn't paradoxical, it is very valuable, you should play it, it's counterintuitive to you because you can't actually imagine a quantity that comes in linear proportion to utility, you have never encountered one, none seems to exist.
Money, for instance, is definitely not linearly proportionate to utility, the more you get the less it's worth to you, and at its extremes, it can command no more resources than what the market offers, and if you get enough of it, the market will notice and it will all become valueless.
Every resource that exists has sub-linear utility returns in the extremes.
(Hmm. What about land? Seems linear, to an extent)
Wild Speculative Civics: What if we found ways of reliably detecting when tragedies of the commons have occurred, then artificially increased their costs (charging enormous fines) to anyone who might have participated in creating them, until it's not even individually rational to contribute to them any more?
That sounds like punishing any usage of common resources which is likely undesireable.
Good policy to manage individual commons requires to think through how their usage is best managed. Elinor Ostrom did a lot of research into what works for setting up good systems.
Sounds like auctioning the usage of the common.
I can imagine a few technical problems, like determining what level of usage is optimal (you don't want people to overfish the lake, but you don't know exactly how many fish are there), or the costs of policing. But it would be possible to propose a few dozen situations where this strategy could be used, and address these issues individually; and then perhaps only use the strategy in some of them. Or perhaps by examining individual specific cases, we would discover a common pattern why this doesn't work.