Do they exist? Sure, I'm exaggerating a little bit. On my current job hunt I try to pay special attention to safety/alignment-related positions, but it seems that a vast majority of them would require me to relocate to either the Bay Area (or the US in general), or London, and I'm just not very eager to do that.
This is a polemic to the ten arguments post. I'm not a regular LW poster, but I'm an AI researcher and mild-AI-worrier. I believe that AI progress, and the risks associated with it, is one of the most important things to figure out as humanity in the current year. And...
I'm a daily user of ChatGPT, sometimes supplementing it with Claude, and the occasional local model for some experiments. I try to make squeeze LLMs into agent-shaped bodies, but it doesn't really work. I also have a PhD, which typically would make me an expert in the field of AI,...
Link: Personal note: I'm somewhat in between safetyism and e/acc in terms of their general ideologies/philosophies. I don't really consider myself a part of either group. My view on AI x-risk is that AI can be potentially an existential threat, but we're nowhere near that point right now, so safety...
After some introspection, I realized my timelines are relatively long, which doesn't seem to be shared by most people around here. So this is me thinking out loud, and perhaps someone will try to convince me otherwise. Or not. First things first, I definitely agree that a sufficiently advanced AI...
I'm curious what the LW community as a whole thinks about the work that Hugging Face is doing. Their main MO seems to be taking whatever new breakthrough in AI there is, and making it open-source and accessible to the public. (as a first order approximation) I see a few...
Do they exist? Sure, I'm exaggerating a little bit. On my current job hunt I try to pay special attention to safety/alignment-related positions, but it seems that a vast majority of them would require me to relocate to either the Bay Area (or the US in general), or London, and...
Work by Quinn Dougherty, Ben Greenberg & Ariel Kwiatkowski From the AI Safety Camp group that worked on Cooperativity and Common Pool Resources: a write-up of a problem we worked on, our proposed solution, and the results of implementing this solution in a simulated environment. Contents: 1. Problem description 2....
This is a polemic to the ten arguments post. I'm not a regular LW poster, but I'm an AI researcher and mild-AI-worrier.
I believe that AI progress, and the risks associated with it, is one of the most important things to figure out as humanity in the current year. And yet, in most discussions about x-risk, I find myself unaligned with either side.
My overall thesis about AI x-risk is that it's absolutely real, but also far enough into the future that at this moment, we should simply continue progress on both capabilities and safety. I'm not trying to argue that sufficiently powerful AI could never pose an x-risk, this belief seems rather silly.
Disclaimers:
I'm a daily user of ChatGPT, sometimes supplementing it with Claude, and the occasional local model for some experiments. I try to make squeeze LLMs into agent-shaped bodies, but it doesn't really work. I also have a PhD, which typically would make me an expert in the field of AI, but the field is so busy and dynamic that it's hard to really state what an "expert" even is.
AI can bring - already brings - lots of value, and general improvements to human lives, so my default stance is that its continued progress - one might say, acceleration - is in our best interest. The flip-side is, of course, a whole host... (read 908 more words →)
Link:
Personal note: I'm somewhat in between safetyism and e/acc in terms of their general ideologies/philosophies. I don't really consider myself a part of either group. My view on AI x-risk is that AI can be potentially an existential threat, but we're nowhere near that point right now, so safety research is valuable, but not urgent. For this reason, in practical terms, I'm somewhat closer to e/acc, because I think there's a lot of value to be found in technological progress, so we should keep developing useful AI.
I'm hoping this debate will contain solid arguments as to why we shouldn't keep developing AI at full speed, ideally ones that I haven't heard before.... (read 1181 more words →)
After some introspection, I realized my timelines are relatively long, which doesn't seem to be shared by most people around here. So this is me thinking out loud, and perhaps someone will try to convince me otherwise. Or not.
First things first, I definitely agree that a sufficiently advanced AI can pose an existential risk -- that's pretty straightforward. The key part, however, is "sufficiently advanced".
Let's consider a specific claim "Within X years, there will be a superintelligent AGI powerful enough to pose a significant existential threat", where X is any number below, say, 30[1].
Since this is a positive claim, I can't exactly refute it from thin air. Let's instead look at the... (read 943 more words →)
I'm curious what the LW community as a whole thinks about the work that Hugging Face is doing. Their main MO seems to be taking whatever new breakthrough in AI there is, and making it open-source and accessible to the public. (as a first order approximation)
I see a few aspects here:
Do they exist? Sure, I'm exaggerating a little bit. On my current job hunt I try to pay special attention to safety/alignment-related positions, but it seems that a vast majority of them would require me to relocate to either the Bay Area (or the US in general), or London, and I'm just not very eager to do that.
Work by Quinn Dougherty, Ben Greenberg & Ariel Kwiatkowski
Contents:
Check out our GitHub repo here.
When multiple parties share the access to a finite resource, they may overuse the resource leading to a Tragedy of the Commons. A simple way of mitigating this problem is allowing them to decrease the effective population by using violence - this, however, is not the best solution for obvious reasons. We study interventions for avoiding these... (read 1811 more words →)
Is there a reliable way of determining whether an agent is misaligned, as compared to just incompetent? And can we talk of such distinction at all? (I suppose the orthogonality thesis implies that we can)
Let's say I give you an agent with suboptimal performance. Without having insights inside its "brain", and only observing its behavior, can we determine whether it's trying to optimize the correct value function but failing, or maybe it's just misaligned?
Let's say you have an idea that you think might be interesting to investigate, possibly a new aspect in AI safety, maybe some new algorithm.
If you're an experienced researcher, you probably have plenty of intuition to think through it, consider the possible outcomes and decide whether it's worth investigating.
If you have a decent academic network, you can probably bring it up even in casual conversations with people who are as good or better than you in the field to get a sense of their intuitions.
What if you have none of those things? Is there an online forum for such discussions, just like I'd go to reddit when looking for e.g. game recommendations? Maybe *this* is the forum?
And are there any good recommendations how to try and work it out by yourself, without referring to the wisdom of online crowds?
This seems like a rather silly argument. You can apply it to pretty much any global change, any technological progress. The world changes, and will change. You can be salty about it, or you can adapt.
Modern misaligned AI systems are good, actually. There's some recent news about Sakana AI developing a system where the agents tried to extend their own runtime by editing their code/config.
This is amazing for safety! Current systems are laughably incapable of posing x-risks. Now, thanks to capabilities research, we have a clear example of behaviour that would be dangerous in a more "serious" system. So we can proceed with empirical research, create and evaluate methods to deal with this specific risk, so that future systems do not have this failure mode.
The future of AI and AI safety has never been brighter.
Because you could make the same argument could be made earlier in the "exponential curve". I don't think we should have paused AI (or more broadly CS) in the 50's, and I don't think we should do it now.
So you do, gotcha
Genuine questions - do you still believe you're one of the smartest young supergeniuses?
Trump "announces" a lot of things. It doesn't matter until he actually does them.
Expert opinion is an argument for people who are not themselves particularly informed about the topic. For everyone else, it basically turns into an authority fallacy.
While I participated in a previous edition, and somewhat enjoyed it, I couldn't bring myself to support it now considering Remmelt is the organizer, between his anti-AI-art crusades and an overall "stop AI" activism. It's unfortunate, since technical AI safety research is very valuable, but promoting those anti-AI initiatives makes it a probable net negative in my eyes.
Maybe it's better to let AISC die a hero.
And how would one go about procuring such a rock? Asking for a friend.
The ML researchers saying stuff like AGI is 15 years away have either not carefully thought it through, or are lying to themselves or the survey.
Ah yes, the good ol' "If someone disagrees with me, they must be stupid or lying"