Game Practitioner http://aboutmako.makopool.com
Posts
Wiki Contributions
Comments
On reflection, it must have played out more than once that a kiwi lad, in a foreign country, drunk, has asked a girl if she wants to get a kebab. The girl thinks he means shish-kebab but says yes enthusiastically because she likes him and assumes he wouldn't ask that unless it was an abnormally good shish-kebab. The kiwi realizes too late that there are no kebabs in america, but they end up going ahead and getting shish-kebabs out of a combination of face-saving, and an infatuation-related coordination problem: The girl now truly wants a shish-kebab, it is too late to redirect the desires of the group.
So that detail might have just been inspired by a true story.
2
2
2
2
2
2
2
2Americans don't know how much they had to compromise in this video by using shish-kebabs instead of what a new zealander would really mean when someone at a party says "do you want to get a kebab with me", which are instead like, the turkish version of burritos, instead of mince, beans and cheese; turkish meat, hummus, veges and wider choice of sauces. They're a fixture of nightlife and tend to be open late.
1
1
1
1
1
1If you wanna talk about the humanity(ies), well I looked up Chief Vision Officer of AISI Adam Russel, and he has an interesting profile.
Russell completed a Bachelor of Arts in Cultural Anthropology from Duke University, and an M.Phil. and a D.Phil. in Social Anthropology from University of Oxford, where he was a Rhodes Scholar.[2] He played with the Oxford University RFC for four varsity matches and also worked with the United States national rugby union team, and worked as High Performance director for the United States women's national rugby union team in the 2014 and 2017 Women's Rugby World Cups.[3]
Russell was in the industry, where he was a senior scientist and principal investigator on a wide range of human performance and social science research projects and provided strategic assessments for a number of different government organizations.[2][4] Russell joined Intelligence Advanced Research Projects Activity (IARPA) as a program manager.[2][4] He developed and managed a number of high-risk, high-payoff research projects for the Office of the Director of National Intelligence.[2] Russell joined DARPA as a program manager in July 2015.[2][4] His work there focused on new experimental platforms and tools to facilitate discovery, quantification and "big validation" of fundamental measures in social science, behavioral science and human performance.[2]
In 2022, secretary Xavier Becerra selected Russell to serve as the acting deputy director for the Advanced Research Projects Agency for Health (ARPA-H), effective June 6. In this role, Russell leads the process to stand up ARPA-H.[5]
Hmm he's done a lot of macho human-enhancement-adjacent stuff. I wonder if there were some centaurists involved here.
- I previously noted a lot of research projects in neurotech research in DoD funding awards. I'm making a connection between this and a joke I heard recently on a navy seals podcast. "The guys often ask what they can do to deal with drones. So you start showing them how to work the jammer devices, or net guns, and their eyes glaze over, it's not what they wanted, they're disappointed. They're thinking like, 'no... how can I deal with it. Myself.' "
- So even though alignment-by-merger is kinda obviously not going to work (you'd have to reverse-engineer two vats of inscrutable matrices, instead of one. And the fleshy pink one wasn't designed to be read from and can only be read on a neuron-by-neuron level after being plastinated (which also kills it). AGI alignment is something that a neuralink cannot solve.), it's conceivable that it's an especially popular line of thought among military/sports types.
Otherwise, this kinda lines up with my confessions on manhattan projects for AGI. You arguably need an anthropologist to make decisions about what 'aligned' means. I don't know if you really need one (a philosophically inclined decision theorist, likely to already be involved already, would be enough for me) but I wouldn't be surprised to see an anthropologist appointed in the most serious projects.
Feel like there's a decent chance they already changed their minds as a result of meeting him or engaging with their coworkers about the issue. EAs are good at conflict resolution.
Wouldn't really need reward modelling for narrow optimizers. Weak general real-world optimizers, I find difficult to imagine, and I'd expect them to be continuous with strong ones, the projects to make weak ones wouldn't be easily distinguishable from the projects to make strong ones.
Oh, are you thinking of applying it to say, simulation training.
Cool then.
Are you aware that prepotence is the default for strong optimizers though?
Are you proposing applying this to something potentially prepotent? Or does this come with corrigibility guarantees? If you applied it to a prepotence, I'm pretty sure this would be an extremely bad idea. The actual human utility function (the rules of the game as intended) supports important glitch-like behavior, where cheap tricks can extract enormous amounts of utility, which means that applying this to general alignment has the potential of foreclosing most value that could have existed.
Example 1: Virtual worlds are a weird out-of-distribution part of the human utility function that allows the AI to "cheat" and create impossibly good experiences by cutting the human's senses off from the real world and showing them an illusion. As far as I'm concerned, creating non-deceptive virtual worlds (like, very good video games) is correct behavior and the future would be immeasurably devalued if it were disallowed.
Example 2: I am not a hedonist, but I can't say conclusively that I wouldn't become one (turn out to be one) if I had full knowledge of my preferences, and the ability to self-modify, as well as lots of time and safety to reflect, settle my affairs in the world, set aside my pride, and then wirehead. This is a glitchy looking behavior that allows the AI to extract a much higher yield of utility from each subject by gradually warping them into a shape where they lose touch with most of what we currently call "values", where one value dominates all of the others. If it is incorrect behavior, then sure, it shouldn't be allowed to do that, but humans don't have the kind of self-reflection that is required to tell whether it's incorrect behavior or not, today, and if it's correct behavior, forever forbidding it is actually a far more horrifying outcome, what you'd be doing is, in some sense of 'suffering', forever prolonging some amount of suffering. That's fine if humans tolerate and prefer some amount of suffering, but we aren't sure of that yet.
(instutitional reform take, not important due to short timelines, please ignore)
The kinds of people who do whataboutism, stuff like "this is a dangerous distraction because it takes funding away from other initiatives", tend also to concentrate in low-bandwidth institutions, the legislature, the committee, economies righteously withering, the global discourse of the current thing, the new york times, the ivy league. These institutions recognize no alternatives to them, while, by their nature, they can never grow to the stature required to adequately perform the task assigned to them.
I don't think this is a coincidence, and it makes it much easier for me to sympathize with these people: They actually believe that we can't deal with more than one thing at a time.
They generally have no hope for decentralized decisionmaking, and when you examine them closely you find that they don't really seem to believe in democracy, they've given up on it, they don't talk about reforming it, they don't want third parties, they've generally never heard of decentralized public funding mechanisms, certainly not futarchy. So it's kind of as simple as that. They're not being willfully ignorant. We just have to show them the alternatives, and properly, we basically haven't done it yet. The minarchists never offered a solution to negative externalities or public goods provision. There are proposals but the designs are still vague and poorly communicated. There has never been an articulation of enlightened technocracy, which is essentially just succeeding at specialization or parallelization in executive decisionmaking. I'm not sure enlightened technocracy was ever possible until the proposal of futarchy, a mechanism by which non-experts can hold claimed experts accountable.
2If that's really the only thing he drew meaning from, and if he truly thinks that failure is inevitable, today, then I guess he must be getting his meaning from striving to fail in the most dignified possible way.
But I'd guess that like most humans, he probably also draws meaning from love, and joy. You know, living well. The point of surviving was that a future where humans survive would have a lot of that in it.
If failure were truly inevitable (though I don't personally think it is[1]), I'd recommend setting the work aside and making it your duty to just generate as much love and joy as you can with the time you have available. That's how we lived for most of history, and how most people still live today. We can learn to live that way.
- ^
Reasons I don't understand why anyone would have a P(Doom) higher than 75%: Governments are showing indications of taking the problem seriously. Inspectability techniques are getting pretty good, so misalignment is likely to be detectable before deployment, so a sufficiently energetic government response could be possible, and sub-AGI tech is sufficient for controlling the supply chain and buying additional time, and China isn't suicidal. Major inner misalignment might just not really happen. Self-correcting from natural language instructions to "be good, you know" could be enough. There are very deep principled reasons to expect that having two opposing AGIs debate and check each others' arguments works well.
I didn't, but I often want to downvote articles that seem to be lecturing a group who wouldn't read or be changed by the article. I know a lot of idiots will upvote such articles out of a belief that by doing so they are helping or attacking that group. On reddit, it often felt like that is the main reason people upvote things, to engage indirectly with others, and it kills the sub, clogging it with posts that the people who visit the sub are not themselves getting anything from.
If you engaged with the target group successfully, they would upvote the post themselves, so a person should generally never upvote on others' behalf, because they don't actually know what would work for them.
Unfortunately, the whole anonymous voting thing makes it impossible to properly address voting norm issues like this. So either I address it improperly by making deep guesses about why people are voting, in this way (no, don't enjoy) or I prepare to depose lesswrong.com with a better system (that's what I'm doing)