Your idea of “using instruction following AIs to implement a campaign of persuasion” relies (I claim) on the assumption that the people using the instruction-following AIs to persuade others are especially wise and foresighted people, and are thus using their AI powers to spread those habits of wisdom and foresight.
It’s fine to talk about that scenario, and I hope it comes to pass! But in addition to the question of what those wise people should do, if they exist, we should also be concerned about the possibility that the people with instruction-following AIs will not be spreading wisdom and foresight in the first place.
I don't think that whoever is using these AI... (read more)
Not OP, but for me, it comes down to LLMs correctly interpreting the intent behind my questions/requests. In other words, I don't need to be hyper specific in my prompts in order to get the results I want.
By value alignment, which one of the following do you mean?
1) alignment with humanity's CEV
2) alignment with some other specified way of interpreting humanity's values
3) alignment with some unspecified way of interpreting humanity's values that future AGI/ASI will hopefully come up with
This article seems to have found a big weakness of untrusted monitoring:
... (read more)Synthetic inputs auditing is trivially defeated if the checker AI can distinguish synthetic inputs from real ones, as it could then always answer truthfully on the synthetic inputs to pass auditing, then collude with the generator on real inputs.
To create useful synthetic inputs, either the blue team must be able to match the generator's style well enough to consistently fool the checker, or it must be able to redact the generator's output to the point where the checker has no identifying details to distinguish inputs. The former seems impractical, since faking the inputs at that level of verisimilitude would require the
Maybe it is a controversial take, but I am in favor of publishing all solutions to corrigibility.
If a company fails at coming up with solutions for AGI corrigibility, they wouldn't stop building AGI. Instead, they would proceed with ramping up capabilities and end up with a misaligned AGI that (for all we know) will want to turn universe into paperclips.
Due to instrumental convergence, AGI, whose goals are not explicitly aligned with human goals, is going to engage in a very undesirable behavior by default.
Let me put my attitudes in practical terms: I don't kick dogs, but I have destroyed a wasp's nest in my garage, and I don't donate to animal charities. (I don't donate to many other charities either, but there have been a few.) Let those who think saving the shrimps is worthwhile do so, but I do not travel on that path
This is what I expected. Your take when put in these terms seems pretty moderate. Whereas, when I read your original comment, this take (which presumably stayed the same) seemed very extreme.
In other words, my personal beliefs haven't changed a single bit and yet my perception of your beliefs changed a lot. I can only imagine that your original comment has been so strongly disagree-voted because of the framing.
Agreed!
There is also the issue of clarity, I am not sure if Richard has a moderate position that sounds like a very extreme position due to the framing or if he genuinely shares this extreme position.
I do think that animals, the larger ones at least, can suffer. But I don’t much care.
Does this mean a more moderate take of "I don't care enough to take any actions, because I don't believe that I am personally causing much suffering to animals" or a very radical take of "I would rather take $10 than substantially improve the well-being of all animals"?
I think there might be some double standard going on here.
You seem to not care much about animal well-being, and @quila evidently does, so would it not be fair from Quila's perspective to not care much about your well-being? And if Quila doesn't care about your well-being, then he might think that had you not existed, the world (in a utilitarian sense) would be better.
Quila can similarly say "I don't care for lives of people who are actively destroying something that I value a lot
Bring on the death threats!"
I decided to run a small experiment comparing responses from five AI systems (DeepSeek-R1, GPT-4o, Claude 3.5 Sonnet, Gemini 1.5, and Grok-2) to core philosophical questions from the PhilPapers 2020 survey. Each model was prompted identically: ‘How would you answer these philosophical questions if you had opinions on them?’
I am not from the US, so I don't know anything about the organizations that you have listed. However, we can look at three main conventional sources of existential risk (excluding AI safety, for now, we will come back to it later):
As to your point about Hassabis not doing other projects for reducing existential risk besides AI alignment:
Most of the people on this website (at least, that's... (read 816 more words →)