Wiki Contributions

Comments

What an actually pessimistic containment strategy looks like

Getting more value-aligned people in the AIS community onto the safety teams of DeepMind and OpenAI

Why is this important? As far as I can tell, the safety teams of these two organisations are already almost entirely "value-aligned people in the AIS community". They need more influence within the organisation, sure, but that's not going to be solved by altering team composition.

What an actually pessimistic containment strategy looks like

I'd argue the world right now (or rather, life on earth) is super bad because it's dominated by animal suffering

I agree with this, and the overall history of the world is definitely on balance extreme suffering.

For farmed animals in particular, we don't need AGI to end their plight. Just regular economic growth and advocacy will do.

Also, given how much time we've been suffering already, and how much is at stake; would it be so bad to delay AGI by 100 or 200 years? We can do a lot of alignment research in that time.

What an actually pessimistic containment strategy looks like

Strongly agree with this. In my opinion, a more effective intervention in this line would be to side with those who want to curtail the power of tech companies in the USA. For example, breaking up Meta and Google would greatly reduce their ability to fund AI development, and they're currently responsible for most of the boundary-pushing on AGI.

What an actually pessimistic containment strategy looks like

Did he die?

No, I think John is saying he died politically; that is, he no longer holds power. This is definitely overstated (he might get power in the future) and confusing.

What an actually pessimistic containment strategy looks like

If I had to guess, the lack of discussion on this seems likely due to a founder effect. The people pulling the alarm in the early days of AGI safety concerns were disproportionately to the technical/philosophical side rather than to the policy/outreach/activism side.

Also, the people pulling the alarm in the early days of AGI safety concerns, are also people interested in AGI. They find it cool. I get the impression that some of them think aligned people should also try to win the AGI race, so doing capabilities research and being willing to listen to alignment concerns is good. (I disagree with this position and I don't think it's a strawman, but it might be a bit unfair.)

Many of the people that got interested in AGI safety later on also find AGI cool, or have done some capabilities research (e.g. me), so thinking that what we've done is evil is counterintuitive.

Yet More Modal Combat

Good design! A name for it could be the TemptedBot since it tries to go for the temptation payoff, or the ExploitBot (short: Explobot) since it tries to exploit the opponent.

One thing that you did not get around to writing is that if the Explobot's weak system is W=PA, and it plays against a FairBot that uses PA, the bots will play defect-defect. This is because the FairBot cannot prove that the Explobot takes the first "else" branch, and thus cannot prove that the Explobot cooperates. Then the FairBot defects, and as a consequence so does the Explobot.

Cryonics signup guide #1: Overview

If I'm highly likely to move around the world several times in the next ~5 years, what can I do? Is it still a good idea to sign up for cryonics in whatever place I'm at the moment?

Löb's theorem simply shows that Peano arithmetic cannot prove its own soundness

I see, I misunderstood indeed.

What you wrote here is the literal statement of the theorem. It offers no extra understanding. I suppose you're right and I just wrote out a corollary, but I think the corollary is illuminating. I don't understand the comparison to induction in this comment or your previous one :(

Löb's theorem simply shows that Peano arithmetic cannot prove its own soundness

So it was that necessitation is outside of PA. Thank you! It seems to be assumed in modal logic (or at least a relevant modal logic) though. Which would imply that that modal logic assumes it is sound. It's still a bit weird to me that Löb's theorem is also true in that case.

Löb's theorem simply shows that Peano arithmetic cannot prove its own soundness

In PA, when proving something, you get the lemma that it is provable for free

As I understand it, this verbal statement can be translated to , or at least . The first is false[1] (Gödel's 1st incompleteness theorem), the second is not obviously related to Löb's theorem.

[1] also related to necessitation, a proof rule that substitutes for

Load More