Modeling Transformative AI Risk (MTAIR)

Wiki Contributions


Thanks - and the fact that we don't know who is working on relevant things is exactly the reason we're doing this! 

We are focused on mathematical research and building bridges between academia and research. I think the pathway to doing that type of research is usually through traditional academic channels, a PhD program, or perhaps a masters degree or a program like MATS, at which point the type of research promotion and academic bridge building we are focused on become far more relevant. That said, we do have undergrad as an option, and are certainly OK with people at any level of seniority signaling their interest.

Do you have any description of your research agenda, or is this just supposed to provide background?

or will contribute


This bit is immediately dropped in favor of discussing current systems, the risk they currently pose, and how much difference they make. But no-one is arguing that current systems pose danger!

I agree that, tautologically, there is some implicit model that enables the LLM to infer what will happen in the case of the ball. I also think that there is a reasonably strong argument that whatever this model it, it in some way maps to "understanding of causes" - but also think that there's an argument the other way, that any map between the implicit associations and reality is so convoluted that almost all of the complexity is contained within our understanding of how language maps to the world. This is a direct analog of Aaronson's "Waterfall Argument" - and the issue is that there's certainly lots of complexity in the model, but we don't know how complex the map between the model and reality is - and because it routes through human language, the stochastic parrot argument is, I think, that the understanding is mostly contained in the way humans perceive language.

One concern I have is that there are many claims here about what was or was not present in the training data. We don't know what training data GPT-4 used, and it's very plausible that, for instance, lots of things that GPT-3 and GPT-3.5 were asked were used in training, perhaps even with custom, human written answers. (You did mention that you don't know exactly what it was trained on, but there's still an implicit reliance. So mostly I'm just annoyed that OpenAI isn't even open about the things that don't pose any plausible risks, such as what they train on.)

And this is not to say I disagree - I think the post is correct. I just worry that many of the claims aren't necessarily possibly to justify.

I disagree with you about what Hamas's ultimate goal is - you seem to envision this as a near-term self-promotion and self-preservation organization, whereas religious extremists are often happy to sacrifice themselves and others for their ideals. If Israel makes peace with the Saudis, it cements the status quo in place, makes it impossible to actually reasonably claim that the Muslim world would support the destruction of Israel, negating Hamas's entire platform and reason to exist.

I strongly agree with you that this was intended to be smaller - I expect that they were anticipating a non-zero level of resistance at the border, and most of the attackers being turned back. But they were inviting reprisals in either case, and the reason they tend to do mount attacks is to manipulate internal Palestinian or Israeli politics in some way.

It's an Iranian proxy war, and Iran is allied with Russia, but Russia likely isn't interested in this other than to the extent that it's an international distraction from Ukraine.

And Russia provided military hardware to groups it supported in Syria, and many of those groups were adjacent to terrorist groups - but not materially more so than the US's support of groups in Iraq and Afghanistan.

I think the timing was a combination of ongoing repression and Hamas planning coming to fruition after years of planning and previous attempts - but that wasn't the proximate cause. The proximate cause for ordering the operation, I strongly suspect, was the US/Israel/Saudi peace talks that threatened Iran (Hamas's sponsor) with a regional coalition against them, and Netanyahu explicitly saying he was leaving the Palestinians out, and was hoping to neutralize the international support they had by making peace. The cause is particularly critical, in my view, because Israel's reaction has been exactly what Hamas and Iran hoped for, and seems likely to derail attempts at a Saudi peace deal.

We'll find out in the coming years what Egyptian intelligence knew, and likely find out how much the attack was directly orchestrated by Iran/Hezbollah via training, but one terrorism expert, Matt Levine of the Washington Institute, noted that the attack tactics were very clearly in line with Hezbollah's typical approach, rather than being similar to what Hamas has done in the past.

Context: Brothers in Arms Is an anti-Netanyahu group that was pushing reservists to protest.

Load More