I sometimes worry that my ability to perceive social status isn't calibrated well.  I wonder if you might be experiencing that?  They may have been patting you on the back for your cool questions rather than your jokes, but you completely missed it.

Also, there might be some selection effects on who shows up to philosophy meetups, such that their net total epistemics are worse than a randomly selected sample of people from the general population.  To spitball a low confidence explanation - maybe they're high in openmindedness, but haven't developed an epistemic toolkit suited for dealing with that?  So they do worse than more average closed-minded people in forming good beliefs?  But honestly, I don't like thinking this way very much.  It's not very charitable, and I wouldn't want to say that to the faces of people I'm judging this way.

I guess if it were me, I would worry that maybe I was Just Wrong and I failed to engage with the social reality correctly?  Like there was a layer or signal that I completely missed?  A while back I read an essay about how neurotypical people differ from ASD people about their relationship to the Truth, and it's stuck with me.  It could be just that:  they relate to Truth differently.

Hi, I'm a local who's interested in AI safety, though I don't think I'd have anything to contribute since I'm still a student.  Would this be something where it would be cool if I applied or showed up?  Or are you aiming for a more professional atmosphere?

Isn’t this a really small amount, like a single raisin has this much free glucose in it right?  Why not just microdose raisins?  How does this create a signal against normal blood glucose variation?  Your gut should be releasing a lot more glucose from just the oats.  Do you have a metabolic disorder?  I guess I’m trying to reason out about why your body isn’t already supplying the glucose to your brain in the first place from current digestion, and why would glucose microdosing get around this.

“What terms have you heard of (or can you invent right now in the comments?) that we have definitely NOT passed yet?”

Artificial Job-taking Intelligence (AJI):  AI that takes YOUR job specifically

Money Printing Intelligence (MPI):  AI that any random person can tell in plain language to go make money for them (more than its inference and tool use cost) and it will succeed in doing so with no further explication from them

Tool-Using Intelligence (TUI):  AI that can make arbitrary interface protocols to let it control any digital or physical device responsive to an Internet connection (radio car, robot body, industrial machinery)

Automated Self-Embodier (ASE):  An AI that can design a physical tool for itself to use in the world, have the parts ordered, assembled, and shipped to the location it wants the tool at, and is able to remote in and use it for its intended purpose at that location with few hiccups and little assistance from humans outside of plugging it in

“Everything feels both hopeless - my impact on risk almost certainly will round down to zero - and extremely urgent - if I don’t try now, then I won’t have a chance to.“

I have thought about individual impact a lot myself, and I don’t think this is the right way to see it.  It sounds like you might not be hung up on this, but I want to attack it anyways since it’s been on my mind, and maybe you will find it useful.

So.  Two alternatives:

Focus on your marginal impact, instead of your absolute impact.  No one person’s marginal contributions, in expectation, are going to be able to swing p(doom) by 1%.  A much more reasonable target to aim for on a personal level is .01% or .0001% or so.

Or:  the paths to successful worlds are highly irregular.  There might be several different lines that will get us there, many requiring a high number of steps in sequence, an unknown number of which are interchangeable.  The problem is too difficult and unknowable to model with a single final probability, or is simply not even in that kind of a reference class.  You just have to look for the most effective levers from your position and pull them.

One might counter that actually, we live in a world where you only need a few key ideas or visions, and a few extraordinary, keystone people to implement them.  Maybe that’s true.  But I think we should think about the difference between two very similar instances of that world, one where we win, one where we lose.

The first thought about that difference that comes to my mind (confidence 80%) is:  The ecosystem of work on this was just slightly not robust enough, and those few keystones didn’t encounter the right precursor idea, or meet the right people, or have the right resources to implement them.  Or, they didn’t even have the motivation to do it in the first place, due to despairing in their belief in their insignificance. 

So given this, I think a key component of that ecosystem is morale.  Morale is a correlated decision; if you don’t have it, the keystone people won’t have it either.  And you won’t know in advance if you’re one of the keystones, either.  Therefore, believe in yourself.

As for whether you’re even likely to be a keystone?  Well, looking at your webpage, I’d say it’s much more likely than the odds of a random person on Earth.  So you should count yourself in that reference class.  This probably extends to anyone who has read LessWrong, even if you’re not aiming for technical work.  Some of the key actions might not even be technical, such as if an international pause is required.

And of course, if we live in the fuzzier many-paths world I described earlier, then it’s much harder to say that your actions don’t matter; so the only reason left to take actions as if they don’t is poor self esteem.  That should collapse once you take the time to properly integrate that there is no other reason, and as long as you are doing the other things humans need to function (socializing, taking care of your biology, etc.).

Or, I suppose, if you disagree with the whole AI safety project in general, or if you think the chances of anyone helping are truly infinitesimal and you’d rather just focus on living your best life in the shadow of the singularity.  But I assume you’re not here for that.  So within that frame - do your best; that’s all you have to do.

I am curious, are the kids actually doing this?  Is everything you described here literally happening?  If so, is there a search term for what they’re doing?  And what scene is this happening in?

I encounter the same frustration when trying to talk about this.  However, I think of it like this:

From the outside, arguing for AI risk looks highly analogous to evangelism.  

You are concerned for the immortal soul (post singularity consciousness) of every human on earth, and that there is an imminent danger to them, posed by wildly powerful unfalsifiable unseen forces (yet-to-be-existent AGI), which cannot be addressed unless you get everyone you talk to to convert to your way of thinking (AI risk).

Or think about it like this:  if an evangelist whose religion you disagree with takes every opportunity they get to try to convert you, how would you respond?  If you wish to be kind to them, you sympathize with them while not taking their worldview seriously.  And if they keep at it, your kindness might wear thin.

AI risk - or any potentially likely x-risk in general- is a very grabby memeplex.  Taken seriously, if you think it’s at all a tractable problem, it warps your entire value system around it.  Its resemblance to religion in this way is probably a major factor in giving many irreligious folk the ick when considering it.  Most people who aren’t already bought into this are not down to have their entire value system upended like this, in a conversation where they likely already have epistemic guards up.  

In my view, you often have to take a softer approach.  Try to retreat to arguing for the sorts of background assumptions that naturally lead to x-risk concern, rather than arguing for it directly.  Share the emotions that drive you, rather than the logic.  Try not to push it- people can tell when you’re doing it.  Appeal to the beliefs of people your interlocutor genuinely respects.  And try to be vulnerable:  if you aren’t willing to change your beliefs, why should they be?  In conversation, minds are usually changed only when both parties have their guard down.

Doubtless there are darker arts of persuasion you could turn to.  They could even be important or necessary.  But I personally don’t want to think of myself as a soldier fighting for a cause, willing to use words as weapons.  So this is what I’ve come to aspire to instead.  It helps a lot in my own belief in the idea, also, to open it up to vulnerable critique from others; if I didn’t, I would just be letting the first grabby memeplex that saw me take me over, without fighting back.  And, who knows - you might occasionally meet someone who shares enough assumptions to buy your worldview, and then at that point you can make the easier, logical case.

Edit: upon reflection, a lot of this applies to activism generally, too, not just religion.  The connection to religion seems to me to be more prominent, however, due to its potential totality over all affairs of life, the way x-risk concern, and transhumanism more broadly, addresses the same kind of cosmic meaning religion often does, apocalyptic thinking, a central canon, set of institutions, and shared culture, organized gatherings explicitly framed as a secular surrogate for traditional holidays, and so on.  Others have likely debated the particular distinctions elsewhere.  The irreligionist in me wishes it weren’t so, but I feel I have to own the comparison if I want to see the world correctly.  It’s been on my mind a lot.

I wonder if this is relevant to the SGTM paper Anthropic put out recently?  Could this be done to reverse the ablation they did?  That is, because this method seems to not rely as much on old information, which Anthropic asserts they destroyed.  Separately, I wonder what happens when you ablate and patch serially several times?  Hundreds of times?

Also, the nature of the patch here strongly reminds me of neuroplasticity, and how when a region of the brain is damaged, other regions adjust to pick up the slack.

Talking about memetic evolution puts me in mind of You Get About Five Words.  So in that spirit, I'd like to try making soundbites of the counter-memes above:

1.  "Let our children decide AI" "Let future people choose AI, not us" "AI Now is too soon"

2.  "Humanity has so much potential" "Normal humans shouldn't choose AI" "Max out IQ before AI" or "IQ before AI" "Even Picard couldn't handle where AI is going"

3.  "The AI you want isn't the AI you'll get"  "Nothing stops AI from being awful"

4.  "AI psychosis is just the beginning" "Satan was a flatterer too" 

5.  "AI Replacement is Literally Fascism" "History shouldn't end with AI" "Nazis thought they were inevitable too" "AI Might doesn't make AI Right"

6.  "Humans really do matter" "Love your neighbor, not computers" "AI can't replace family and friends"

7.  "You can't take AI replacement back" "As though we're in a position to permanently decide the future" "No guarantee AIs have souls" "AI can't answer life's questions"

8.  "AI replacment won't be voluntary" "ChatGPT 8 Wants Little Timmy's Atoms" 

9.  "AI replacement betrays humanity" "Does everyone you could ever know really deserve to be replaced by AI?  Even me?  Even everyone I care about?" "Me when I destroy humanity's hopes and dreams because my AI girlfriend said I'm a good boy:"

And these are just the first things that came to my mind with an hour of effort, no doubt others could do a lot better.  Perhaps referencing pieces of culture I didn't think of to sum up concepts, or just putting things more wittily.  Maybe this task should be referred to experts, or AI.

Edit:  I guess this goes in the opposite direction of Richard Ngo's point about how this represents an escalation in memetic warfare between AI safety and accelerationism.  Now I feel kinda bad for essentially manufacturing ammunition for that.