wonder

For the specific context of reliability safety research (or even perceived reliability) and RSI work:

• I don't have first hand experience with this specific aspect, so I don't have very strong opinion on the safety-capability trade-off itself, with RSI. I do think a lot of things still transfer - for example, does not matter if the reliability research comes from the labs, or from the general community. However, it is true the but-for scenario probably changed. I suspect safety researchers leaving the labs will not help with the situation though. In my model

It seems pretty odd to me to have concrete career advice either way - I think the community really just need to keep communicating "this is an important issue", and those who have passion in this field would switch, and stay. Talent is one thing, but passion is another important factor IMO.

It sounds like there are two concerns: secondarily, safety execution (safety washing), and primarily, safety research work that also leads to capability increases. [Context: I had some experience working on trust and safety at one of the labs before the end of 2025, but not after.]

For safety execution:

• It sounds like the problem is more about how that safety work is done and whether the details are communicated. For example, what happens before this safety mitigation? After this mitigation, what remains? And note that any mitigation is likely not a guarante

I thought the author is talking more about the general public or online culture, but like anywhere I suppose you will find your people by just being who you are?

I think the post is probably not just about outreach (at least that's how I read it) but also about culture exchange generally and what to keep in mind when we collaborate with people across borders. And I think they are probably thinking about talking to the public and not just scholars. Someone else could also do a study by talking to their friends though, that's a good idea! But I think the speaker is probably trying to help by talking from their own experience. It also sounded like a good idea for people who want to collaborate to work with the AI Safety orgs inside China, which I have also heard about as well.

while cultures arising in regions of abundance tend to develop a mindset that it's not that hard to provide for yourself and so no one has a responsibility to help you

Yeah interesting. An extended thought - the economy at the time in China changed from a planned economy to a more free-market economy or a mixed economy, which made the "provide for yourself" concept more possible while the abundance increases.

If the resource is abundance, but there are for some reason (policy, generation of wealth, privilege) generally less possibility for gaining that, may... (read more)

Sounds like pretty cool events! I was a research engineer in the LLM Responsible AI/Trust and Safety team and worked a lot with lawyers; and also found our lawyers learn very fast and often understand our technical parts eventually pretty well! I can also echo that we (researchers in the team) were often encouraged to think from the perspective of threat models, and refining that threat model from the technical perspective was both helpful for me as a researcher and for the lawyers.

(Chinese here who went abroad at 15 as well); adding on the AGI danger outreach: from what I saw on Xiaohongshu/Rednote I think it is also very important to 1. lay out the specific and different scenarios for why it is dangerous, 2. convince that “this outreach does not have anything to do with the rich hoping to be come richer” or “this is how the frontier companies are preventing competition”. It is quite the opposite - power and capital pushed the advancement of improper and irresponsible development of the technology and that makes it more dangerous

(Hu... (read more)

Yeah I see! I did start to learn more of such religions that are more flexible - walked pass by a Unitarian Universalism place (like you have mentioned) the other day and found it very interesting.

I am a first-generation Chinese immigrant who came to the US when I was 15. I also took US history/government (AP) classes and world history/government classes in both high school and college (I majored in statistics with a concentration in political science). But I have now lived in the US for (slightly) more years than in China, and I do observe many differences among my Chinese friends (and even more on Chinese social media), so take my words with this context as well. I do think it is a good idea to share things like OP did to build cross-cultural unde... (read more)

My current take: only one group of reasoning (that I heard from talking to others) seems remotely relevant but still not convincing to me: 1.1 AIs learn from users(our interactions with models are used to further train the models), and they will learn things like aggression/manipulation etc from the human users. What I don’t get is: why cannot effective rewriting and filtering solve the problem? Or at least it should not be named AI Welfare, but more post training data quality, from my current perspective. 1.2 AI already learnt human behaviors where when y... (read more)

Wondering if there are offline meetups for each of these groups?

Appreciate the post

In my opinion and from my observations, I do think some people share/agree with some aspects/ideas from rationalist community/ideas, but not necessarily completely fit all aspects, or just may not claim to be in the community.

Yeah agreed board games are super popular/not a niche thing. So is meditation in my opinion. (Side track)

On a side note, I have been reflecting recently and felt that power dynamics/abuse, and bad actors are probably the common central factors in both longer term risks like x risks and other current risks.

Imagine a homeless person - they would worry about food resources or accommodation rather than longer term risks. Similarly, if someone still needs to worry about survival before long term risks come, they would probably pay more attention to that. Unfortunately it is true that there are still many near term things that could affect shorter term survival to a lot of people, so even though x risks are something to be aware of/paying attention to, it likely won’t be the only thing to worry about. This situation is usually a product of the era and environment, and seems more prevalent in places with greater economic/power/social disparity.

To zoom out a bit, from the post I assume you benchmark ASI mostly by "replacing humans 100% in all jobs". Curious in why you specifically care about absolutely 100%? (Replacing 50% of humans is still significant imo.)

"Looking up the exact answer in a database"

I mean it is not too too wrong if the model is using RAG?

We simply lack systems other than religions that provide religions same benefits

I am empathetic to this point, and I was thinking of this recently too. However, I've reached a conclusion that the benefit of having values relatively independent of religion (or even similar communities) is to be able to change and develop your values as you experience more things in life. It feels to me that values in religion may be relatively fixed (I could be wrong). 

The sad/hard thing is that the process of changing values or beliefs might be a painful experience, w... (read more)