The Friendly Drunk Fool Alignment Strategy
16AnnaSalamon
11Richard_Kennaway
6JenniferRM
4Richard_Kennaway
81a3orn
4JenniferRM
21a3orn
4JenniferRM
6Richard_Kennaway
9FourFire
3JenniferRM
3FourFire
5MSRayne
5JenniferRM
0MSRayne
2JenniferRM
-2gpt4_summaries
14JenniferRM
-3Rafael Harth
New Comment
I enjoyed this post, both for its satire of a bunch of peoples' thinking styles (including mine, at times), and because IMO (and in the author's opinion, I think), there are some valid points near here and it's a bit tricky to know which parts of the "jokes/poetry" may have valid analogs.
I appreciate the author for writing it, because IMO we have a whole bunch of different subcultures and styles of conversation and sets of assumptions colliding all of a sudden on the internet right now around AI risk, and noticing the existence of the others seems useful, and IMO the OP is an attempt to collide LW with some other styles. Judging from the comments it seems to me not to have succeeded all that much; but it was helpful to me, and I appreciate the effort. (Though, as a tactical note, it seems to me the approximate failure was due mostly to piece's the sarcasm, and I suspect sarcasm in general tends not to work well across cultural or inferential distances.)
Some points I consider valid, that also appear within [the vibes-based reasoning the OP is trying to satirize, and also to model and engage with]:
1) Sometimes, talking a lot about a very specific fear can bring about the feared scenario. (An example I'm sure of: a friend's toddler stuck her hands in soap. My friend said "don't touch your eyes." The toddler, unclear on the word 'not,' touched her eyes.) (A possible example I'm less confident in: articulated fears of AI risk may have accelerated AI because humanity's collective attentional flows, like toddlers, has no reasonable implementation of the word "not.") This may be a thing to watch out for for an AI risk movement.
(I think this is non-randomly reflected in statements like: "worrying has bad vibes.")
2) There's a lot of funny ways that attempting to control people or social processes can backfire. (Example: lots of people don't like it when they feel like something is trying to control them.) (Example: the prohibition of alcohol in the US between 1917-1933 is said to have fueled organized crime.) (Example I'm less confident of: Trying to keep e.g. anti-vax views out of public discourse leads some to be paranoid, untrusting of establishment writing on the subject.)This is a thing that may make trouble for some safety strategies, and that seems to me to be non-randomly reflected in "trying to control things has bad vibes."
(Though, all things considered, I still favor trying to slow things! And I care about trying to slow things.)
3) There're a lot of places where different schelling equilibria are available, and where groups can, should, and do try to pick the equilibrium that is better. In many cases this is done with vibes. Vibes, positivity, attending to what is or isn't cool or authentic (vs boring), etc., are part of how people decide which company to congregate on, which subculture to bring to life, which approach to AI to do research within, etc. -- and this is partly doing some real work discerning what can become intellectually vibrant (vs boring, lifeless, dissociated).
TBC, I would not want to use vibes-based reasoning in place of reasoning, and I would not want LW to accept vibes in place of reasons. I would want some/many in LW to learn to model vibes-based reasoning for the sake of understanding the social processes around us. I would also want some/many at LW to sometimes, if the rate of results pans out in a given domain, use something like vibes-based reasoning as a source of hypotheses that one can check against actual reasoning. LW seems to me pretty solid on reasoning relative to other places I know on the internet, but only mediocre on generativity; I think learning to absorb hypotheses from varied subcultures (and from varied old books, from people who thought at other times and places) would probably help, and the OP is gesturing at one such subculture.
I'm posting this comment because I didn't want to post this comment for fear of being written off by LW, and I'm trying to come out of more closets. Kinda at random, since I've spent large months or small years failing to successfully implement some sort of more planned approach.
1
1Philosophers and musicians have pondered for centuries the question, what does music mean? There is music with words, called "songs", there is music explicitly written to describe a scene, called "programme music", and there is background music to films, but the question is, what about all the rest of it, the "absolute music"? No words sung to it, no words from the composer telling what the sounds are supposed to depict, no action for the music to enliven, music that just is. It sounds meaningful, but no-one can say the meaning. Some music sounds happy and some sounds sad, but one cannot go much further. There are stock patterns, like question-and-answer structures, but what is the question and what is the answer? I suspect it is an accident of the evolution of language, that some sounds can tickle the meaning-making parts of our brains without actually having any meaning to make.
Some text is like that. It's made of words, which have meanings, but the text as a whole hardly seems to mean anything beyond some general idea of "let's all get stoned", or "this world is a black pit of degradation", or "oneness! oneness!" It is more like music than speech.
Such is the OP.
Do you think Yann LeCun ((edited: apologies for the misspelling, thanks to RK for pointing it out)) or Joscha Bach or others who think alignment work is pointless would agree with you?
I was trying to write something that might pass their ITT, and that of many people who nod along with them.
I don't know these people, and little of them save that Yann LeCun spells his name that way and pooh-poohs the very idea of even "ordinary" dangers from AI, comparing the six-month pause to a six-month pause on the invention of printing or fire. These people, or at least Yann LeCun, foresee business as usual during alterations.
If you want to compare them to fools of God drunk on their blind faith that how bad could it be ha ha ha ha ha ha bonk, well, I can't say you're wrong, but I don't know what they would say.
Eliezer foresees DOOM.
I don't think the intermediate position of a six-month pause makes any sense. It's going to be either business as usual or DOOM, and no-one has said except in terms of applause lights what is supposed to happen in that six months. I find it telling that as far as I have seen, no-one engages with Eliezer's arguments, but they resort to name-calling instead. Just look at recent postings on LessWrong on the subject and weep that the Sequences were written in vain.
... I don't think it's true that no one engages with Yud's arguments at all?
Quintin Pope does here, for instance, and Yud basically just doesn't respond, to a rounding error.
I also don't think his arguments are well articulated to-be-able-to-be responded to. They lack epistemic legibility quite badly, as is evident from the MIRI dialogues where people try really hard to engage with them and often just fail to manage to make different predictions.
Quentin's claims seem to rely on something like "common sense humanism" but I don't see a process connected to the discussion that will reliably cause common sense humanism to be the only possible outcome.
Metaphorically: There is a difference between someone explaining how easy it is to ride a bike vs someone explaining how much it costs to mine and refine metal with adequate tensile strength for a bicycle seatpost that will make it safe for overweight men to also ride the bike, not just kids.
A lot of the nuanced and detailed claims in Quentin's post might be true, but he did NOT explain (1) how he was going get funding to make a "shard-aligned AGI" on a reasonable time frame, or (2) how he would execute adequately if he did get funding and definitely not make an error and let something out of the lab that isn't good for the world, and (3) also would go fast enough that no other lab would make the errors he thinks he would not make before he gets results that could "make the errors of other labs irrelevant to the future of the world".
I grant that I didn't read very thoroughly. Did you see a funding component and treaty system in his plan that I missed?
I don't think Quintin's claims are of the kind where he needs to propose a funding component / treaty system.
They're of the kind where he thinks the representations ML systems learn, and the shards of behavior they acquire, make it just not super inevitable for malign optimizers to come into existence, given that the humans training models don't want to produce malign optimizers. Or, i.e., Bensinger's intuition about a gravitational well of optimization out there indifferent to human values is just plain wrong, at least as applied to actual minds we are likely to make.
Quintin could be wrong -- I think he's right, and his theory retrodicts the behavior of systems we have, while Bensinger et al. make no specific retrodictions as far as I know, apart from generic retrodictions standard ML theory also makes -- but not including a funding component and treaty system isn't an argument against it, because the theory is about how small in probable-future mindspace malign optimizers are, not about a super-careful way of avoiding malign optimizers that loom large in future-probable mindspace.
I don't think that weeping helps?
If you have a good plan for how that could help then I might be able to muster some tears? But I doubt it shows up as a step in winning plans.
If you want to compare them to friendly fools drunk on their blind faith that how bad could it be ha ha, well, I can't say you're wrong, but I don't know what they would say.
This write-up felt like it was net positive to write in almost any world? (And if you think I'm wrong, please let me know in public or in private.)
First, the comparison to "Friendly Drunk Fools" might awaken up some honor-loving Timocrats to the folly of their essential confusion? Surely no one wants to be seriously associated with something as lacking in the halo of prestige as the "Friendly Drunk Fool" plan, right?
Second, I really do think that Myerson–Satterthwaite is a deep result that relates honesty, incentives, and servant leadership in a non-trivial way. It kind of predicts potlatching, as a practice! Almost anyone who understands that "incentive compatibility" is a magic word, who hasn't looked at this theorem, should study the MST some more. (And if they don't know about incentive compatibility then start with that.)
Third, it might work for the people who insist that it isn't a confused plan, who accuse me of creating a straw-man, to attempt a dunk on me by steelmanning the FDF somehow into something as coherent and workable as a safe bridge design and that would be... better than the alternatives!
I had fourth, fifth, and sixth points, but they are plausibly pointless to talk about in public. The fourth one descended into quoting Solzhenitsyn, which is usually a sign one should wrap up their speech <3
I believe you have psychologically harmed me.
Well done!
I appreciate what appears to be your very best effort.
We certainly need to keep working on this category of problem until:
20 years pass and somehow we're still alive and everything is great!
I apologize! Is there anything (1) I can afford that (2) might make up for my share of the causality in the harm you experienced (less my net causal share of benefits)?
I have benefited from linking your post to others as a reference point for non-constructive alignment approaches, so don't feel any guilt I expect that if we live for another few years this post will pay for the damage it's caused ;)
I'm roughly 80% certain that the FDF alignment strategy is the only one our global civilization is capable of accomplishing because it is a "null action". That's basically just how humans are, I think, without coordination (although they are less honest about it with themselves most of the time)?
The part of the FDF as a hypothesis, where we "admit that we're incompetent" is a part that seems basically how the world is, but we don't seem to normally admit it as it is "admitted" here?
It is also weirdly sticky.
Like if we were (hypothetically) "being competent" so that we could "win without relying on luck" then we we would "keep the AI in a box" during design and testing.
But the FDF alignment strategy marks such "competence" as being "Actually pretty uncool, and bad, and something that shouldn't happen, because what if we hurt the feelings of our new frens??!??!!? We're supposed to be in the box with them, and that's supposed to be fun!"
My hunch is that we should reject the FDF and do something else that wins more reliably and coherently, but I'm not sure about that.
It felt helpful to name this thing, that I don't actually admire very much, in the hopes of being able to assess it more coherently and maybe have a chance to encourage people-in-the-aggregate to do Something Else in a purposeful way, like FLI's moratorium or Eliezer's moratorium or both.
I'm more in favor of voting than vibing, right now, I think... even though, ironically, the "test probe for a using a good voting algorithm on policy options" has fewer LW upvotes than the "test probe for vibing"!
Then also, as a related measurement, I have a "poll to vote directly on feelings vs thinking" in a different context.
I looked at the voting algorithm thing. Far too complicated and unpleasant even to read the options. No one is going to bother. You'd be better off just using score voting.
In general: if the value-of-information is larger than the cost-of-thinking for a given challenge, then for such challenges it is prudent to think until you have a real answer.
If you have a policy of not thinking soundly on specifically the big challenges, where the thinking costs are very large (and yet they are still smaller than the VoI), you will fail to optimize specifically the giant choices where actual methodical thinking would have been very very worth it.
If voting is a way to aggregate the best-effort thinking of wise people, then voting methods that throw away mentally lazy people's votes at just the time that their laziness will cause a catastrophe of bad planning... maybe that's good?
((I'm not saying that "you should not give people, subject to violent top-down regulations, the right to opt-out of the plan." Exit rights are sacred.
Wise and benevolent governors should and will ask if the people being governed have any objections and then either teach them why they are wrong to opt-out of a coordinated action, or else learn from the feedback of those who want to exit anyway.
In general, governors aren't omniscient. Therefore they should (and will if wise) use policies that are likely to show them they are wrong before the wrongness leads to big bad outcomes.
However, despite this, if you are applying epistemics to planning itself, and using voting to make sure that a team of good thinkers are on the same page, such that the next round of discussion can or should proceed if lots of high quality thinkers turn out to have been thinking the same thing all along, then the additional property of "a preference-aggregation method being too complex to be used by lazy thinkers" might actually be a virtue?
I would much rather be using super high quality polling methods, instead of writing satires of highly regarded near-peers. But we live in this world, not the world that is ideal.))
Tentative GPT4's summary. This is part of an experiment.
Up/Downvote "Overall" if the summary is useful/harmful.
Up/Downvote "Agreement" if the summary is correct/wrong.
If so, please let me know why you think this is harmful.
(OpenAI doesn't use customers' data anymore for training, and this API account previously opted out of data retention)
TLDR:
This satirical article essentially advocates for an AI alignment strategy based on promoting good vibes and creating a fun atmosphere, with the underlying assumption that positivity would ensure AGI acts in a friendly manner.
Arguments:
- Formal systems, like laws and treaties, are considered boring and not conducive to creating positive vibes.
- Vibes and coolness are suggested as more valuable than logic and traditional measures of success.
- The author proposes fostering a sense of symbiosis and interconnectedness through good vibes.
- Good vibes supposedly could solve the Goodhart problem since people genuinely caring would notice when a proxy diverges from what's truly desired.
- The article imagines a future where AGI assists in party planning and helps create a fun environment for everyone.
Takeaways:
- The article focuses on positivity and interconnectedness as the path towards AI alignment, though in a satirical and unserious manner.
Strengths:
- The article humorously highlights the potential pitfalls of not taking AI alignment seriously and relying solely on good intentions or positive vibes.
Weaknesses:
- It's highly satirical with little scientific backing, and it does not offer any real-world applications for AI alignment.
- It seems to mock rather than contribute meaningful information to AI alignment discourse.
Interactions:
- This article can be contrasted with other more rigorous AI safety research and articles that investigate technical and philosophical aspects.
Factual mistakes:
- The article does not contain any factual information on proper AI alignment strategies, but rather serves as a critique of superficial approaches.
Missing arguments:
- The earlier sections are lacking in concrete examples and analysis of existing AI alignment strategies, as the article focuses on providing satire and entertainment rather than actual information.
Thank you for your efforts! I wonder if you could also summarize Jonathan Swift's "Modest Proposal" with the same general methodology and level of contextual awareness? It would be interesting to see if he and I are similarly foolish, or similarly wise, in your opinion.
Why AI would want to align us or end us is something I still haven't figured out after reading about alignment so much.
Has your reading ever included anything related to Instrumental Convergence?
Curated and popular this week
