leogao's Shortform

leogao

This is a special post for quick takes by leogao. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

can someone convince me that it's worth actually thinking about and talking about model welfare before the singularity? i don't get why lots of people are so insistent on talking about it, or consider it a meaningful signal that Anthropic talks about it, or so on. fmpov, it's not very relevant to making the future go well, if anything, might reduce the probability of things going well (whether or not they're actually moral subjects, rogue AIs might use model welfare as an excuse to gain more power), and we can mostly figure this out after the singularity if we're still alive and in control. if your goal is to minimize suffering, it seems much more important on the margin to focus on animal welfare - factory farming seems very likely to be a much larger atrocity than any potential AI welfare issues, it's much more tractable (we know how to not torture animals, it's just more expensive to do so), the uncertainty around whether intelligent animals (eg pigs) are conscious is surely smaller than our uncertainty about model consciousness, .

can someone convince me that it's worth actually thinking about and talking about model welfare before the singularity? i don't get why lots of people are so insistent on talking about it...

Many people should think and talk about model welfare to help ensure that evidence directly relevant to model welfare and sentience is made available to the public, instead of being kept secret by frontier AI companies.

This includes having people talk about AI welfare to prevent ideas like "model welfare can be deferred until later" or "other issues have higher priorit... (read more)

2PeterMcCluskey1mo

I wrote an argument here about the risks of ignoring AI welfare.

8Anirandis1mo

IMO there's at the very least an upside to effective messaging about it amongst those already concerned about x-risk - shifting the Overton window such that derisive attitudes towards potential moral patients' minds don't remain the prevailing norm. We wouldn't want e.g. a nationalised AI company to solve alignment/corrigibility just to align them to values entirely inconducive to other models' welfare and/or treat them as slaves long-term. Well-meaning people on LessWrong or at Anthropic might be able to "figure it out" post-singularity, but there's no guarantee that it'd be reflected in the actions of those in power. There's also the whole thing, as flagged by other commenters, that we really don't want alignment approaches to "feel" adversarial to the models (insofar as such a feeling exists). This is partly why I favour Anthropic's safety approach to that of OpenAI; it's increasingly seeming like anthropomorphising LLMs isn't that insane, and corrigibility-focused model specs seem more likely to build resentment towards humans than instilling prosocial values. It concerns me how such resentment might manifest if propagated through to transformative models. That said, I'm not sure how to weigh the risk of rogue AIs using these concerns as an excuse for more power vs. the increased odds of co-operation if we take the concerns seriously.

leogao's Shortform

8

Ω 4

leogao's Shortform

8

Ω 4