My government name is Mack Gallagher. Crocker's Rules. I am an "underfunded" "alignment" "researcher". DM me if you'd like to fund my posts, or my project.
I post some of my less-varnished opinions on my Substack.
Whatever happened to holding software companies to the standard of not rolling vulnerable user data into their widely distributed business logic?
Say AI companies could effectively make copying hard enough to provide security benefits to scrape-ees [ if I'm reading you right, that's approximately who you're trying to protect ]. Say also that this "easy-to-copy" property of AIs, is "the fundamental" thing expected increase the demand for AI labor relative to human labor. . . . Hard-alignment-problem-complete problem specification, no?
Oh gosh, how irksome if Magic neurotypes its players like that.
Sirlin writes only of denial of one's weakness, not of a "need to lose".
. . . Wow, if that Rizzo piece is representative of how channer bicamerals were handling their internal conflicts before Ziz, I understand Ziz a little better.
Isn't losing just what you need to do to increase your ability to win? Other than the elements of what Rizzo writes about that are obviously just the activation of simian instincts to end a conflict by submitting, that is [ which is a lot of it ].
In the rate-limiting resource, housing, the poor have indeed gotten poorer. Treating USD as a wealth primitive [ not to mention treating "demand" as a game-theoretic primitive ] is an economist-brained error.
Coins are easier to model quasi-deterministically than humans, is the point Jonnan was making. [ I don't think they [Jonnan] realize how many people miss this fact. ]
Well, we're assuming Omega wants more money rather than less, aren't we?
If it's sufficiently omniscient to predict us, a much more complicated type of thing than a coin, what reason would it have to ever flip a physically fair coin which would come up heads?
I don't think the vast majority of people in this comments section realize coins aren't inherently random.
the human-created source code must be defining a learning algorithm of some sort. And then that learning algorithm will figure out for itself that tires are usually black etc. Might this learning algorithm be simple and legible? Yes! But that was true for GPT-3 too
Simple first-order learning algorithms have types of patterns they recognize, and meta-learning algorithms also have types of patterns they like.
In order to make a friendly or aligned AI, we will have to have some insight into what types of patterns we are going to have it recognize, and separately what types of things it is going to like or find salient.
There was a simple calculation protocol which generated GPT-3. The part that was not simple was translating that into predicting its preferences or perceptual landscape, and hence what it would do after it was turned on. And if you can't predict how a parameter will respond to input, you can't architect it one-shot.
I'm laboriously manually Google-translating Lorenz's Der Kumpan in der Umwelt des Vogels from 1935, since I haven't been able to find an existing English translation of the complete work and don't have reliable OCR.
Rewarding passage [ boldface mine ]:
In contrast to these individually directed eliciting schemas, the innate ones are built into a complete, species-specific functional plan from the outset, in which it is determined in advance which characteristics are essential. Therefore, it only corresponds to the principle of parsimony if as few characteristics as possible are included in the eliciting schemas. For the sea urchin Sphaerechinus, it is sufficient if its exceptionally highly specialized combined flight and defense reaction against its main enemy, the starfish Asterias, is triggered by a single, specific chemical stimulus emanating from this starfish. Such triggering of a highly motorically complex behavior adapted to a very specific biological process by a single stimulus, or at least by a series of reactions, is characteristic. One would initially expect that in higher animals, to which we must necessarily attribute a material-objective grasp of the environment based on their other behavior, the object of all instinctual behaviors would also be firmly grasped. This would be considered particularly likely where a conspecific represents the object of the action. Strangely, however, a material identity of the conspecific across multiple functional circuits cannot be demonstrated in very many cases. I believe I can offer an explanation for why the subjective identity of the conspecific as an object of various functional circuits is even less of a biological necessity than that of other instinctual objects.
Even in the highest vertebrates, an object-directed instinctual sequence of actions is often triggered by a very small selection of the stimuli emanating from its object, not by its overall material image. When several functional circuits have the same object as their object, it can happen that each of these circuits responds to entirely different stimuli emanating from the same object. The innate triggering schema of an instinctual action selects, so to speak, a small selection from the abundance of stimuli emanating from its object, to which it selectively responds, thus initiating the action. The simplicity of these innate triggering schemas of different instinctual actions can result in two of them not sharing a single stimulus data that triggers their response, even though they are directed at the same object. Normally, the species-specific object sends all stimuli belonging to both schemas together. In experiments, however, the triggering schemas, which precisely because of their great simplicity can often be triggered by artificially presenting appropriate stimulus combinations, can be triggered by two different objects, thus achieving a separation of the two functional circuits directed at one object. Conversely, for the same reasons, one object can trigger two opposing, biologically meaningful reactions only with two separate objects. This is particularly common in those instinctual actions whose object is a conspecific. For example, in various species of ducks, the mother's defensive reaction can also be triggered by the cry for help of young of different species. Other caretaking reactions, on the other hand, are highly species-specific and tied to very specific coloration and marking patterns on the head and back of the offspring. Thus, it is understandable if a mallard leading her young courageously rescues a Turk's chick calling for help from danger and, in the next moment, due to the lack of the mallard-specific head and back markings that trigger further care, "unspecifically fusses" at it, i.e., attacks and kills it as a "foreign animal near its own chicks."
I was talking with some people yesterday whom I accused of competing to espouse middling p(doom)s. One of them was talking about Aaronson's Faust parameter [ i.e. the p(doom), assuming "everything goes perfect" if ¬doom, at which you press the button and release superintelligent AI right now ]. And they had what I think was a good question: In what year do we foresee longevity escape velocity, assuming the AInotkilleveryoneist agenda succeeds and superintelligence is forestalled for decades?
The appropriate countervailing challenge question is: What is one plausible story for how a by-chance friendly ASI invents immortality within two years or whatever of its creation, while staying harmless to humanity? What is the tech tree, how does it traverse this tree and what are the guardrails keeping it from going off on some exciting [ what is effectively to a human ] pathology-gain-of-function tangent along the way?