A common argument is about whether RSI would maximize capabilities in all modalities, or only in easily hill-climbable modalities such as math and code. Two points I think are relevant.
A model with superhuman math and coding skills would have exactly the skills required to find a more sample efficient learning algorithm
A model with superhuman math and coding skills would be easily able to collect data in other modalities via a mix of existing and un-exploited channels
Does anyone have recommendations for dealing with the sense of dread and frustration that comes when seeing the vast number of people in denial about x-risk, even when faced with increasing evidence? There seems to be a pervasive cynicism where people are unwilling to listen to arguments for x-risk on account of people they dislike personally (AI lab CEOs) having made public statements acknowledging it. This combined with flatout misinformation about and misinterpretation of the HuggingFace incident.
Now I get to understand how climatologists have felt for the past few decades.