I saw those scores and thought I was about to witness the greatest exchange of constructive contrarianism in the history of the forums. (Pretty proud of a +7 -12 I posted recently. A real fine stinker. The blue cheese of comments.)

I guess the process would be to pass it on to whichever cababilities researchers they trust with it. There would be a few of them at this point.

So, why not go straight to those researchers instead of MIRI? Because MIRI are more legible responsible intermediaries I guess.

One major second-order effect of doing something this dramatic is that you'd expect controls on gene editing technologies to be raised a lot/made at all, and an argument could be made that that would be a good thing.

There's a tendency to think: If we believe that something should be illegal, we shouldn't do it ourselves. In competitive arenas, this ends up disadvantaging the most responsible thinkers by denying them the fruits of defection without denying it to their competitors, or suppressing the acknowledgement of the regulatory holes as participants are afraid to look hypocritical if they acknowledge the need for regulation while thriving without it. It's actually not hypocritical to exploit a hole while working to close it. Sometimes, spectacularly exploiting the hole is the only practical way to get it closed.

Other projects I'd like to see: An engineered pathogen that just harmlessly melanizes the toes of everyone in the world so that they're reminded of their protectors' incompetence every time they look down.

A transferrable utility game is one where there's a single resource (like dollars) where everyone's utility is linear in that resource

For humans, money does not seem to have linear returns of utility. For what real agents could it?

My expectation for the U of an aligned AGI would be something like, the sum of the desires of humans, which, if the constituent terms have diminishing returns on resources, will also be diminishing. I can see arguments that many probable unaligned AGI might get linear returns on resources... but if humanity is involved in the negotiation (and you really hope we are) then doesn't that still break shapley? I guess you could still potentially use shapley for analyzing the valence of ecosystems of unaligned AGI, which would be useful for comparing risk of unaligned singletons to unaligned multipolar outcomes and to authoritarian lockin, but it's not exciting, and... actually, everything collapses to aligned-somehwhataligned-unaligned multipolar under the Grabby Aliens model.

There does seem to be a value within the human utility function that does scale linearly with resources (a variable that population ethicists and early longtermists love), but it's not clear at all what its relationship with other variables is. Another way of phrasing this objection is, there is a lot of personal low-hanging fruit that a human has to grab before the scalable variable will be all they have left to optimize, and I don't think I've ever heard of a person who gathered "enough" of the low-hanging fruit of the good life, that they started acting in a purely longtermist or stewardly way. EG, Elon will still want his space adventure no matter how many of his friends explain why they think it's not cost-effective for reducing existential risk. Many people tell this story where, once a person is rich enough and "has their needs met", they're supposed to optimize the scalable term and consequently become selfless (the self is finite, even in the extremes, due to the light speed limit, so anything that keeps scaling has to be a kind of selflessness), many people would like that to be true. It's not obviously actually true, it doesn't accord with present human behavior, and I'm not sure how to investigate it.

Can you explain how it picks them off one by one? I mean, how large a group do you need to pick off a wolf and wouldn't most people be close to being in a group of that size naturally as a result of uh having a town.

I'm not seeing any (sorry I missed a word) much game design here.

My experience as a designer, building out a genre of "peacewagers" (games that aren't zero sum but also aren't strictly cooperative, the set of games where honest negotiation is possible.), is that it actually is very likely that someone who's mostly worked in established genres would drastically underestimate the amount of design thought that is required to make a completely new kind of game work, and they're trying to make a new kind of game, so I wouldn't be surprised if they just fell over irrecoverably as soon as they strayed from the yellow brick road they have lived their whole lives within. When you're building a new genre... you have to figure out so much about what can be done there, what the challenge is, and what the appeal is, and how to elegantly communicate all of that to players and make them want it.

So... I've been working on semi-cooperative games for a few years now, I might be able to help with that (I'm also familiar with rust, and have built a basic game engine of my own for some unreleased stuff (in C++)). But I don't get the impression from the site that they appreciate the difficulty of design, that they'd appreciate me, so I haven't applied.

I see that this is getting quite a lot of agreement points. I would also like to add my agreement, this is probably a true quote. I agree that it's probably a true quote. Your claim that this was written somewhere is probably true.

I'd guess that the main AI-exacerbating thing that the game industry does is provoke consumers to subsidize hardware development. I don't know if this is worth worrying about (have you weighed the numbers?), but do you plan on like, promoting low-spec art-styles to curb demand for increasing realism? :] I wonder if there's a tension between realism and user-customizability that you might be able to inflame (typically, very detailed artstyles are more expensive to work in and are harder to kitbash, but it's also possible that stronger hardware would simplify asset pipelines in some ways: raytracing could actually simplify a lot of lighting stuff, right?).

Wolves sometimes kill more than they need, actually. It's quite strange. So they could be normal-sized wolves. And I'm imagining this to be a population of conservationists who aren't interested in taking them out of the local ecosystem.

I'm trying to figure out the worldbuilding logic of "they didn't come so they all got eaten". What do they do when they come? Why would they be less likely to get eaten if they don't do it? And also, how does the boy only have a 5% probability?

Okay so maybe the boy sees the wolf from a distance, on a particular bridge or in a particular clearing, and can't know whether the wolf is coming towards the village. There's a 5% chance that it will. He can't stay and make sure, because then maybe he just gets caught by the wolf, and he has to run to deliver the message as soon as possible.

Let's say it's a very tough wolf and it can only be defeated with a tank. The village has only one wolf-proof tank. and it takes a while to assemble the tank crew and get its engine started, and the fuel is expensive. Once in the village, the wolf would smell the tank from its fuel, locate it, and keep anyone from getting near it, if the wolf gets to the village before the tank is started then it's over.

And why is this task left to a child? Children are generally more concerned by and obsessed with monsters than adults. They know that wolf-crying is relatively safe, and they desperately want to witness the monster (this is the whole appeal of the genre of horror), so they take that job. They also have nothing better to do.

It makes non-web applications possible. It has a better layout system, rendering system. Animates everything properly. Centers Dart, which seems to be a pretty good language: It can be compiled ahead of time for faster boots (although I'm not completely sure that typescript wont be basically just as compilable once wasm-gc is up), has better type reflection, will have better codegen (already supports annotations), has a reasonable import system, better data structures, and potentially higher performance due to the type system not being an afterthought (although typescript is still very good relative to dart).