Gram Stone


Sorted by New

Wiki Contributions


Yudkowsky and Christiano discuss "Takeoff Speeds"

If I take the number of years since the emergence of Homo erectus (2 million years) and divide that by the number of years since the origin of life (3.77 billion years), and multiply that by the number of years since the founding of the field of artificial intelligence (65 years), I get a little under twelve days. This seems to at least not directly contradict my model of Eliezer saying "Yes, there will be an AGI capable of establishing an erectus-level civilization twelve days before there is an AGI capable of establishing a human-level one, or possibly an hour before, if reality is again more extreme along the Eliezer-Hanson axis than Eliezer. But it makes little difference whether it's an hour or twelve days, given anything like current setups." Also insert boilerplate "essentially constant human brain architectures, no recursive self-improvement, evolutionary difficulty curves bound above human difficulty curves, etc." for more despair.

I guess even though I don't disagree that knowledge accumulation has been a bottleneck for humans dominating all other species, I don't see any strong reason to think that knowledge accumulation will be a bottleneck for an AGI dominating humans, since the limits to human knowledge accumulation seem mostly biological. Humans seem to get less plastic with age, mortality among other things forces us to specialize our labor, we have to sleep, we lack serial depth, we don't even approach the physical limits on speed, we can't run multiple instances of our own source, we have no previous example of an industrial civilization to observe, I could go on: a list of biological fetters that either wouldn't apply to an AGI or that an AGI could emulate inside of a single mind instead of across a civilization. I am deeply impressed by what has come out of the bare minimum of human innovative ability plus cultural accumulation. You say "The engine is slow," I say "The engine hasn't stalled, and look how easy it is to speed up!"

I'm not sure I like using the word 'discontinuous' to describe any real person's position on plausible investment-output curves any longer; people seem to think it means "intermediate value theorem doesn't apply," (which seems reasonable) when usually hard/fast takeoff proponents really mean "intermediate value theorem still applies but the curve can be almost arbitrarily steep on certain subintervals."

Quick general thoughts on suffering and consciousness

I have an alternative hypothesis about how consciousness evolved. I'm not especially confident in it.

In my view, a large part of the cognitive demands on hominins consists of learning skills and norms from other hominins. One of a few questions I always ask when trying to figure out why humans have a particular cognitive trait is “How could this have made it cheaper (faster, easier, more likely, etc.) to learn skills and/or norms from other hominins?” I think the core cognitive traits in question originally evolved to model the internal state of conspecifics, and make inferences about task performances, and were exapted for other purposes later.

I consider imitation learning a good candidate among cognitive abilities that hominins may have evolved since the last common ancestor with chimpanzees, since as I understand it, chimps are quite bad at imitation learning. So the first step may have been hominins obtaining the ability to see another hominin performing a skill as another hominin performing a skill, in a richer way than chimps, like “That-hominin is knapping, that-hominin is striking the core at this angle.” (Not to imply that language has emerged yet, verbal descriptions of thoughts just correspond well to the contents of those thoughts; consider this hypothesis silent on the evolution of language at the moment.) Then perhaps recursive representations about skill performance, like “This-hominin feels like this part of the task is easy, and this part is hard.” I’m not very committed on whether self-representations or other-representations came first. Then higher-order things like, “This-hominin finds it easier to learn a task when parts of the task are performed more slowly, so when this-hominin performs this task in front of that-hominin-to-be-taught, this-hominin should exaggerate this part, or that part of the task.” And then, “This-hominin-that-teaches-me is exaggerating this part of the task,” which implicitly involves representing all those lower order thoughts that lead to the other hominin choosing to exaggerate the task, and so on. This is just one example of how these sorts of cognitive traits could improve learning efficiency, in sink and source.

Once hominins encounter cooperative contexts that require norms to generate a profit, there is selection for these aforementioned general imitation learning mechanisms to be exapted for learning norms, which could result in metarepresentations of internal state relevant to norms, like emotional distress, among other things. I also think this mechanism is a large part of how evolution implements moral nativism in humans. Recursive metarepresentations of one’s own emotional distress can be informative when learning norms as well. Insofar as one’s own internal state is informative about the True Norms, evolution can constrain moral search space by providing introspective access to that internal state. On this view, this is pretty much what I think suffering is, where the internal state is physical or emotional distress.

I think this account allows for more or less conscious agents, since for every object-level representation, there can be a new metarepresentation, so as minds become richer, so does consciousness. I don't mean to imply that full-blown episodic memory, autobiographical narrative, and so on falls right out of a scheme like this. But it also seems to predict that mostly just hominins are conscious, and maybe some other primates to a limited degree, and maybe some other animals that we’ll find have convergently evolved consciousness, maybe elephants or dolphins or magpies, but also probably not in a way that allows them to implement suffering.

I don’t feel that I need to invoke the evolution of language for any of this to occur; I find I don’t feel the need to invoke language for most explicanda in human evolution, actually. I think consciousness preceded the ability to make verbal reports about consciousness.

I also don’t mean to imply that dividing as opposed to making pies is a small fraction of the task demands that hominins faced historically, but I also don’t think it’s the largest fraction.

Your explanation does double-duty, with its assumptions at least, and kind of explains how human cooperation is stable where it wouldn’t be by default. I admit that I don’t provide an alternative explanation, but I also feel like it’s outside of the scope of the conversation and I do have alternative explanations in mind that I could shore up if pressed.

Progress, Stagnation, & Collapse

Your argument has a Holocene, sedentary, urban flavor, but I think it applies just as well to Pleistocene, nomadic cultures; I think of it as an argument about population size and 'cognitive capital' as such, not only about infrastructure or even technology. Although my confidence is tempered by mutually compatible explanations and taphonomic bias, my current models of behavioral modernity and Neanderthal extinction essentially rely on a demographic argument like the one made here. I don't think this comment would be as compelling without a reminder that almost everyone explains these phenomena via interspecies differences in individual cognitive adaptation, as opposed to demography.

On behavioral modernity, I am a gradualist; I do not think there was a sudden pulse of innovation. Signs of modernity, like a wider resource base, public symbol use, and certain technologies appear and then disappear, before becoming a permanent fixture later in the record. The most striking example of this would be ancient Australia, where humans did not reliably demonstrate all features of behavioral modernity for the first 25,000 years of residence even though these features were present in contemporaneous cultures in different geographic locations; Australia also happens to be one of the last places to which humans migrated. The idea here is that reduced population size, and thus density, affects the fidelity and bandwidth of social learning in lots of ways (e.g. decreased redundancy and specialization), resulting in a less sophisticated behavioral repertoire, as well as reduced selection for public symbol use (since there are no frequently encountered outgroups to which to signal).

On Neanderthal extinction, I think this is a collapse in the sense of your post, by a failure to reliably transmit cognitive capital. The life history of hominins is similar to a case of growth-dependent economics, where large energy and time investments are made under the expectation of future economic productivity. While I think environmental effects started the decay, temperate specialists as the Neanderthals were, I think they went extinct ultimately because these effects started a process that gradually invalidated the preconditions for reliable cultural transmission in that species.

Four factors that moderate the intensity of emotions

For those wondering about the literature, although Kahneman and Tversky coined no term for it, Kahneman & Tversky (1981) describes counterfactual-closeness and some of its affective consequences. This paper appears to be the origin of the missed flight example. Roese (1997) is a good early review on counterfactual thinking with a section on contrast effects, of which closeness effects are arguably an instance.

Incorrect hypotheses point to correct observations

Succubi/incubi and the alien abduction phenomenon point to hypnagogia, and evo-psych explanations of anthropomorphic cognition are often washed down with arguments that anthropomorphism causes good enough decisions while being technically completely false; there's an old comment by JenniferRM talking about how surprisingly useful albeit wrong it would be to model pathogens as evil spirits.

Topological Fixed Point Exercises

An attempt at problem #1; seems like there must be a shorter proof.

The proof idea is "If I flip a light switch an even number of times, then it must be in the same state that I found it in when I'm finished switching."

Theorem. Let e a path graph on ertices with a vertex oloring uch that if hen Let s bichromatic Then s odd.

Proof. By the definition of a path graph, there exists a sequence ndexing An edge s bichromatic iff A subgraph f s a state iff its terminal vertices are each incident with exactly one bichromatic edge or equal to a terminal vertex of The color of a state is the color of its vertices. There exists a subsequence of ontaining the least term of each state; the index of a state is equal to the index of its least term in this subsequence.

Note that none of the states with even indexes are the same color as any of the states with odd indexes; hence all of the states with even indexes are the same color, and all of the states with odd indexes are the same color.

For each state there exists a subsequence of orresponding to the vertices of and the least term of each subsequence is either r some hat is the greatest term in a bichromatic edge. Thus the number of states in

By contradiction, suppose that s even. Then the number of states is odd, and the first and last states are the same color, so the terminal vertices of re the same color, contrary to our assumption that they are different colors. Thus ust be odd.


What To Do If Nuclear War Seems Imminent

I see that New Zealand is also a major wood exporter. In case of an energy crisis, wood gas could serve as a renewable alternative to other energy sources. Wood gas can be used to power unmodified cars and generators. Empirically this worked during the Second World War and works today in North Korea. Also, FEMA once released some plans for building an emergency wood gasifier.

Making a Difference Tempore: Insights from 'Reinforcement Learning: An Introduction'

The Lahav and Mioduser link in section 14 is broken for me. Maybe it's just paywalled?

Anthropics made easy?

Just taking the question at face value, I would like to choose to lift weights for policy selection reasons. If I eat chocolate, the non-Boltzmann brain versions will eat it too, and I personally care a lot more about non-Boltzmann brain versions of me. Not sure how to square that mathematically with infinite versions of me existing and all, but I was already confused about that.

The theme here seems similar to Stuart's past writing claiming that a lot of anthropic problems implicitly turn on preference. Seems like the answer to your decision problem easily depends on how much you care about Boltzmann brain versions of yourself.

Load More