After seeing this post and it's comments, I realized there may be an influx of new, roughly college-aged people joining LessWrong and the broader AI x-risk reduction community.
As a college student myself, this is a relief!
I have been regularly reading and digesting posts on here for nearly a year now, and the journey has been a bit lonely – navigating these ideas can feel detached from reality when the people around you aren’t really discussing them yet. I think it would be very helpful to have like-minded peers to talk to.
Note that this applies to anyone who is relatively new to the community, not just college students. The reason I specified that... (read 234 more words →)
Agree; I'm strongly in favor of using a term like "disempowerment-risk" over "extinction-risk" in communication to laypeople – I think the latter detracts from the more important question of preventing a loss of control and emphasizes the thing that happens after, which is far more speculative (and often invites the common "sci-fi scenario" criticism).
Of course, it doesn't sound as flashy, but I think saying "we shouldn't build a machine that takes control of our entire future" is sufficiently attention-grabbing.
I'm disappointed that they don't make any mention of extinction risk
Agree, but I wonder if extinction risk is just too vague, at the moment, for something like this. Absent a fast takeoff scenario, AI doom probably does look something like the gradual and unchecked increase of autonomy mentioned in the call, and I'm not sure if there's enough evidence of a looming fast takeoff scenario for it to be taken seriously.
I think the stakes are high enough that experts should firmly state, like Eliezer, that we should back off way before fast takeoff even seems like a possibility. But I see why that may be less persuasive to outsiders.
Note that there have been many reports of persistent physiological changes caused by 5-AR inhibitors such as finasteride (see: Post Finasteride Syndrome), some of which sound pretty horrifying, like permanent brain fog and anhedonia.
I've spent a lot of time reading through both the scientific literature and personal anecdotes and it seems like such adverse effects are exceedingly rare, but I have high confidence (>80%) that they are not completely made up or psychosomatic. My current best guess is that all such permanent effects are caused by some sort of rare genetic variants, which is why I'm particularly interested in the genetic study being funded by the PFS network.
The whole situation is pretty complex and there's a lot of irrational argumentation on both sides. I'd recommend this Reddit post as a good introduction – I plan on posting my own detailed analysis on LW sometime in the future.
I am a bit doubtful about the impact that this will have – it seems like the goal of fasting, as a form of activism, is to signal to others that you care deeply enough about an issue to suffer for it.
However, a lot of the key players in AI probably disagree with, or assign lower probability to the arguments for x-risk (for whatever reason). Which means demonstrating that you care a lot is unlikely to convince them.
Not to say that there will be no impact, but I'm not sure the expected loss of productivity from fasting for a week straight would be worth it.
To refine this discussion, I'll only be responding to what I think are major points of disagreement.
This scenario is just as incoherent as the other one!
No, it is not.
Imagining yourself as a "disembodied spirit," or a mind with no properties, is completely incoherent.
To imagine being someone else is something people (including you, I assume) do all the time. Maybe it's difficult to do, and certainly your imagination will never be perfectly accurate to their experience, but it is not incoherent.
I do see your point: you can't become someone else because you would then no longer be you. What you are arguing is that through your view of personal identity (which, based on... (read 359 more words →)
The way I interpret it: in the thought experiment, you are not literally imagining yourself as a mind without properties, and then asking "what would I want?" You are imagining that you can become any of the sentient minds entering existence at any given moment, and that you will inherit their specific circumstances and subjective preferences.
The goal of the thought experiment, then, is to construct a desirable world that encompasses the specific desires and preferences of all sentient beings.[1] This seems obviously relevant to the alignment problem.
With no constraints, your answer might look like "give every sentient being a personal utopia." Our individual ability to change the real world is heavily constrained, though,... (read more)
I'm curious – what are the "very serious criticisms" you refer to? Your comment would more helpful and productive if you pointed to specific disagreements you have with the post rather than abstractly mentioning the existence of them.
More importantly, what are these criticisms directed at? The stated thesis of this post is that the veil of ignorance is a "useful and mostly accurate description of reality when viewed through certain theories of personal identity."
This list of notable criticisms of the Veil of Ignorance doesn't seem to include any disagreement with this claim in particular. In fact, the last criticism mentioned seems to support it – albeit with different reasoning.
Likewise, the first LessWrong post that comes up when searching for "Veil of Ignorance" argues that the thought experiment doesn't necessarily support Rawl's ideal society – not that it doesn't accurately describe reality.
Regarding traits you love – maybe you are looking for something like intellectual humility? I think it can naturally follow from kindness and cooperativeness, but is often necessary for me to respect an intelligent person.
It also seems like a core principle of this community, where as some say, "we gain status by pointing out where others haven't been careful or skeptical enough in their thinking."