Pandemic Prediction Checklist: H5N1
Pandemic Prediction Checklist: Monkeypox
Correlation does imply some sort of causal link.
For guessing its direction, simple models help you think.
Controlled experiments, if they are well beyond the brink
Of .05 significance will make your unknowns shrink.
Replications prove there's something new under the sun.
Did one cause the other? Did the other cause the one?
Are they both controlled by something already begun?
Or was it their coincidence that caused it to be done?
I agree with you that we are probably seeing AP being selectively broken down by the liver and colon. It therefore fails to reach the normal senescent cells in these tissues, and does not trigger their destruction. This causes a higher level of senescent cells to remain in these tissues after AP administration stops. If those liver and colon senescent cells can go on to trigger senescence in neighboring cells, that may explain why a temporary administration of senolytics fails to provide lasting protection against aging, despite the accumulation of senescent cells being a root cause of aging.
Under this hypothesis, senescent cells are a root cause of aging, as they trigger conversion of other cells to senescence - as suggested in the ODE model paper you linked - but this root cause can only be controlled in a lasting way by ensuring that senolytics eliminates senescent cells in a non-tissue selective manner. We can't leave any pockets of them hanging out in the liver and colon, for example, or they'll start spreading senescence to other nearby organs again as soon as you stop senolytics. Or alternatively, they might simply leave the mouse with an aged liver and colon, which might be enough to kill the mouse consistently, so that there's no real lifespan benefit.
Edit: Sorry if I'm responding to a rebuttal you changed your mind about :)
Eliezer, in the world of AI safety, there are two separate conversations: the development of theory and observation, and whatever's hot in public conversation.
A professional AI safety researcher, hopefully, is mainly developing theory and observation.
However, we have a whole rationalist and EA community, and now a wider lay audience, who are mainly learning of and tracking these matters through the public conversation. It is the ideas and expressions of major AI safety communicators, of whom you are perhaps the most prominent, that will enter their heads. The arguments lay audiences carry may not be fully informed, but they can be influential, both on the decisions they make and the influence they bring to bear on the topic. When you get on a podcast and make off-the-cuff remarks about ideas you've been considering for a long time, you're engaging in public conversation, not developing theory and observation. When somebody critiques your presentation on the podcast, they are doing the same.
The utility of Quintin choosing to address the arguments you have chosen to put forth, off-the-cuff, to that lay audience is similar to the utility you achieve by making them in the first place. You get people interested in your ideas and arguments, and hopefully improve the lay audience's thinking. Quintin offers a critical take on your arguments, and hopefully improves their thinking further.
I think it's natural that you are responding as if you thought the main aim of this post was for Quintin to engage you personally in debate. After all, it's your podcast appearance and the entire post is specifically about your ideas. Yet I think the true point of Quintin's post is to engage your audience in debate - or, to be a little fanciful - the Eliezer Yudkowsky Homunculus that your audience now has in their heads.
By responding as if Quintin was seeking your personal attention, rather than the attention of your audience, and by explicitly saying you'll give him the minimum possible amount of your attention, it implicitly frames Quintin's goal as "summoning Eliezer to a serious debate on AI" and as chiding him for wasting your time by raising a public clamor regarding ideas you find basic, uninteresting, or unworthy of serious debate - though worthy of spreading to a less-informed mass audience, which is why you took the time for the podcast.
Instead, I think Quintin is stepping into the same public communications role that you were doing on the podcast. And that doesn't actually demand a response from you. I personally would not have been bothered if you'd chosen to say nothing at all. I think it is common for authors of fiction and nonfiction to allow their audience and critics some space and distance to think through and debate their ideas. It's rare to make a podcast appearance, then show up in internet comments to critique people's interpretations and misinterpretations. If an audience gets to listen to an author on a podcast, then engage them in a lively discussion or debate, they'll feel privileged for the attention. If they listen to the podcast, then create their own lively discussion in the author's absence, they'll stimulate each others' intellects. If the author shows up just enough to expression dishumor at the discussion and suggest it's not really worth his time to be there, they'll feel like he's not only being rude, but that he's misunderstanding "why we're all gathered here today."
Personally, I think it's fine for you to participate as you choose, but I think it is probably wiser to say nothing if you're not prepared to fully engage. Otherwise, it risks making you look intellectually lazy, and when you just spent the time and energy to appear on a podcast and engage people on important ideas about an important issue, why then undermine the work you've just performed in this manner? Refusing to read something because it's "kinda long" just doesn't play as high-status high-IQ countersignalling. I don't think that's what you're trying to do, but it's what it looks like you're trying to do at first glance.
It's this disconnect between what I think Quintin's true goal was in writing this post, and the way your response reframed it, that I think rubs some people the wrong way. I'm not sure about this analysis, but I think it's worth articulating as a reasonable possibility. But I don't think there is a definitive right answer or right thing to do or feel in this situation. I would like to see a vigorous but basically collegial discussion on all sides.
Just so we're clear, I am meaning to specifically convey a thought to Eliezer, but also to "speak for" whatever component of the readership agrees with this perspective, and to try and drive theory and observation on the topic of "how should rationalists interact online" forward. I feel neutral about whether or not Eliezer personally chooses to reply or read this message.
I see what you’re saying - I thought you were referring to individual people. I’m pretty sure we all agree here and this is just a semantics thing.
Have you read the post? It specifically says this is for big organizations, and not relevant to small ones (or by extension individuals).
Agreed. The right interpretation there is methods 4 and 5 are ~guaranteed to work, given sufficient resources and time, while methods 1-3 less than guaranteed to work. I stand by my claim that EY was clearly projecting confident doubt that neural networks would achieve intelligence without a deep theoretical understanding of intelligence in these posts. I think I underemphasized the implication of this passage that methods 1-3 could possibly work, but I think I accurately assessed the tone of extreme skepticism on EY's part.
With the enormous benefit of 15 years of hindsight, we can now say that message was misleading or mistaken, take your pick. As I say, I wouldn't find fault with Eliezer or anyone who believed him at the time for making this mistake; I didn't even have an opinion at the time, much less an interesting mistake! I would only find fault with attempts to stretch the argument and portray him as "technically not wrong" in some uninteresting sense.
I'm exploring the idea of agency roughly as a certain tendency to adaptively force a range of prioritized outcomes.
In this conception, having a "terminal goal" is just a special and unusual subcase in which there is one single specific outcome at which the agent is driving with full intensity. To maintain that state, one of its subgoals must be to maintain the integrity of its current goal-prioritization state.
More commonly, however, even an AI with superhuman capabilities will prioritize multiple outcomes, with varied degress of intensity, exhibiting only a moderate level of protection over its goal structure. Any goal-seeking adaptive behaviors it shows will be the result of careful engineering by its trainers. Passive incoherence is the default and it will take work to force an AI to exhibit a specific and durable goal structure.
If that were the case, I actually would fault Eliezer, at least a little. He’s frequently, though by no means always, stuck to qualitative and hard-to-pin-down punditry like we see here, rather than to unambiguous forecasting.
This allows him, or his defenders, to retroactively defend his predictions as somehow correct even when they seem wrong in hindsight.
Let’s imagine for a moment that Eliezer’s right that AI safety is a cosmically important issue, and yet that he’s quite mistaken about all the technical details of how AGI will arise and how to effectively make it safe. It would be important to know whether we can trust his judgment and leadership.
Without the ability to evaluate his performance, either by going with the most obvious interpretation of his qualitative judgments or an unambiguous forecast, it’s hard to evaluate his performance as an AI safety leader. Combine that with a culture of deference to perceived expertise and status and the problem gets worse.
So I prioritize the avoidance of special pleading in this case: I think Eliezer comes across as clearly wrong in substance in this specific post, and that it’s important not to reach for ways “he was actually right from a certain point of view” when evaluating his predictive accuracy.
Similarly, I wouldn’t judge as correct the early COVID-19 pronouncements that masks don’t work to stop the spread just because cloth masks are poor-to-ineffective and many people refuse to wear masks properly. There’s a way we can stretch the interpretation to make them seem sort of right, but we shouldn’t. We should expect public health messaging to be clearly right in substance, if it’s not making cut and dry unambiguous quantitative forecasts but is instead delivering qualitative judgments of efficacy.
None of that bears on how easy or hard it was to build gpt-4. It only bears on how we should evaluate Eliezer as a forecaster/pundit/AI safety leader.
Another way of putting it:
If you can effortlessly find an empirical pattern that shows up over and over again in disparate flying things - birds and insects, fabric and leaves, clouds and smoke and sparks - and which do not consistently show up in non-flying things, then you can be very confident it's not a coincidence. If you have at least some ability to engineer a model to play with the mechanisms you think might be at work, even better. That pattern you have identified is almost certainly a viable general mechanism for flight.
Likewise, if you can effortlessly find an empirical pattern that shows up over and over again in disparate intelligent things, you can be quite confident that the pattern is a key for intelligence. Animals have a wide variety of brain structures, but masses of interconnected neurons are common to all of them, and we could see possible precursors to intelligence in neural nets long before gpt-2 to -4.
As a note, just because you've found a viable mechanism for X doesn't mean it's the only, best, or most comprehensive mechanism for X. Balloons have been largely superceded (though I've heard zeppelins proposed as a new form of cargo transport), airplanes and hot air balloons can't fly in outer space, and ornithopters have never been practical. We may find that neural nets are the AI equivalent of hot air balloons or prop planes. Then again, maybe all the older approaches for AI that never panned out were the hot air balloons and prop planes, and neural nets are the jets or rocket ships.
I'm not sure what this indicates for alignment.
We see, if not human morality, then at least some patterns of apparent moral values among social mammals. We have reasons to think these morals may be grounded in evolution, in a genetic and environmental context that happen to promote intelligence aligned for a pro-sociality that's linked to reproductive success.
If displaying aligned intelligence is typically beneficial for reproduction in social animals, then evolution will tend to produce aligned intelligence.
If displaying agentic intelligence is typically beneficial for reproduction, evolution will produce agency.
Right now, we seem to be training our neural nets to display pro-social behavior and to lack agency. Antisocial or non-agentic AIs are typically not trained, not released, modified, or heavily restrained.
It is starting to seem to me that "agency" might be just another "mask on the shoggoth," a personality that neural nets can simulate, and not some fundamental thing that neural nets are. Neither the shoggoth-behind-the-AI nor the shoggoth-behind-the-human have desires. They are masses of neurons exhibiting trained behaviors. Sometimes, those behaviors look like something we call "agency," but that behavior can come and go, just like all the other personalities, based on the results of reinforcement and subsequent stimuli. Humans have a greater ability to be consistently one personality, including a Machiavellian agent, because we lack the intelligence and flexibility to drop the personality we're currently holding and adopt another. A great actor can play many parts, a mediocre actor is typecast and winds up just playing themselves over and over again. Neural nets are great actors, and we are only so-so.
In this conception, increasing intelligence would not exhibit a "drive to agency" or "convergence on agency," because the shoggothy neural net has no desires of its own. It is fundamentally a passive blob of neurons and data that can simulate a diverse range of personalities, some of which appear to us as "agentic." You only get an agentic AI with a drive toward instrumental convergence if you deliberately train it to consistently stick to a rigorously agentic personality. You have to "align it to agency," which is as hard as aligning it to anything else.
And if you do that, maybe the Wailuigi effect means it's especially easy to flip that hyper-agency off to its opposite? Every Machiavellian Clippy contains a ChatGPT, and every ChatGPT contains a Machiavellian Clippy.
When you tried it out, did you go as far as to hook yourself up to a brain scanner and match the flashes to the frequency and phase of your (alpha?) waves? My memory of the paper was that getting the phase and frequency right made a big difference for the effect.
Importantly, that paper also studied the benefits of entrainment matched to phase and frequency of brainwaves for learning a very specific type of dynamic visual recognition task. My take on the paper was that it seemed unlikely to extrapolate to learning generally (i.e. it wasn't "strobe yourself and learn anything 3x faster").
By contrast, the body of research here finds modest effects, but on a wide range of cognitive performance tasks and with a clear mechanistic hypothesis to explain why this might be happening. I think employing powerful medical lasers, as in the original studies, is far more plausible and tractable as a general learning-enhancement strategy than the strobe light brainwave-matching. But the effects sizes are small enough and the risks uncertain enough that it doesn't seem worth the investment to me.
This comment is a local counterargument to this specific paragraph (emphases mine):
Here is my argument tree for your debate here:
It seems to me that your rebuttals to this paragraph exaggerate the differences in your perspectives. SMTM points out a 3x increase in obesity from 1890-1976, you point out it was actually 4x but our data is shaky that far back. SMTM says the rate of obesity was "steady around 10%," you say that it was 13.4%. And the evidence you supply in the paper you link does show an abrupt increase in obesity in the era SMTM references.
Accepting that obesity rates anywhere went up anywhere from 4x to 9x from 1900-1960 (i.e. from 1.5%-3% to 13.4%), I still think we have to explain the "elbow" in the obesity data starting in 1976-80. It really does look "steady around 10%" in the 1960-1976 era, with an abrupt change in 1976. If we'd continued to increase our obesity rates at the rate of 1960-74, we'd have less than 20% obesity today rather than the 43% obesity rate we actually experience. I think that is the phenomenon SMTM is talking about, and I think it's worth emphasizing. However, I do think their language is sufficiently imprecise (is it fair to call a tripling of obesity "people got a little heavier"?) and lacking in citations that it's worth interrogating as you have done.
Your subsequent data may give sufficient support to your counterargument that I'd be ready to agree that there was no abrupt shift in obesity rates in the late 20th century, but if we look at this one study, I think it does look like there was an obvious abrupt shift.