Do I get Bayers Points?
Relatedly, if someone you know is accused of acting poorly toward another person, the observation that they never act that way toward you may provide nearly zero Bayesian evidence in their favor.
A very common example:
Outside of the specific example, and I'm not sure if you meant it this way, but I would caution against using "the world as we know it might end soon" as a reason to make decisions that would be bad if the world somehow stayed mostly normal.
To the specific example, I would search (or ask an LLM) for expert sources on raccoon behavior and welfare, and run with their conclusions.
Not that I know of, and I would also like to see such a report. It matches the angle I took in my contributions to the Metaculus Labor Hub.
but like many movements, the passionate, smart people are attracted into unproductive directions because they stick to what's memetic, avoid conflicting with allies...
I was genuinely confused for a moment while reading this, because this is a perfect description of the pro-industry, anti-pause arm of rationalism and EA. Everyone working for an AI company (or their proxies like Coefficient Giving), or trying to "align" LLMs, or eschewing protests, or refusing to speak to politicians about shutting down the unconscionable suicide race... they are getting ...
1"ASI Platform Provider" is a deeply cursed phrase.
1Focusing on choices that essentially affect only the chooser:
"If you've never been mugged, you're taking too few alleyway shortcuts" would be terrible advice, whereas "if you've never been turned down for a date, you're not putting yourself out there enough / aiming high enough" is generally good advice. So now we are talking price.
Setting aside that doing things that are bad for ourselves is often bad for others, and looking at the specific example offered: To me, missing a flight would be like getting mugged. Or at least, it would be much more like getting mugged than getting romantically rejected by a stranger. If it's a thing that happens once in my entire life, then I haven't spent enough time at airports.
In some cases, risks you impose upon yourself become risks imposed upon others. Generally advocating for people to impose more risks upon themselves poses additional risks on everyone. I don't want to be in an airport filled with people panicking about being late for their flight. That sounds terrible. And I definitely don't want to be driving on a road where everyone is significantly more likely to be late. That's more dangerous.
I'm wrestling through the idea now, though, that it may not be important for me to ever consciously consider the value of my all-things-considered p(doom), so long as it's high enough that I find it unacceptable to just let other people handle it. Over time, I have found myself motivated by the dire situation, and I have found myself motivated by rays of hope, but neither change what I ought to do. So why think about it?
That said, I am merely human. Worse yet, I'm a forecaster. I feel compelled to track that target. Watching my p(doom) go from 70% to 80% m...
I'm surprised to hear that there are (apparently?) many people exchanging their subjective probabilities of AI doom without that just being the top-level intro to a much more involved discussion. I feel like it's obvious that p(doom) is a flattening of all of one's thinking on AI risk into a single metric, and all the interesting differences of viewpoint are in the details, so of course I wouldn't try to compare numbers in a vacuum. Most people won't answer without first providing at least a timeline and a couple conditionals anyway.
P(doom) might be net negative as a meme -- I'm not sure, so I generally don't use that specific phrase these days, especially with people who don't already know it.
We've gotten plenty of left turns, whether sharp or no. All you have to do is make a more powerful version of what we have today, and we already know it would betray us in important ways. I think it's more likely that a new paradigm will emerge, and who's to say what the safety properties of that will be, other than "probably not what we want them to be by default," especially if it's part of an uncontrolled autonomous R&D loop and/or operating outside of human oversight.
I've always admired how well you use explicit models. In the course of my forecasting, I've found that I have to deeply trust my intuition (after vacuuming up all the relevant information I can, hiding nothing from myself). When my intuition fails, I don't double down, but I also don't wave the white flag and cast my intuition aside. I sit with the error until I feel like my intuition has updated. Then I make that mistake just a little bit less, having fed the black box that knows things I know not how.
Explicit modeling is one of many available starting po...
Not so long as more capability means more usefulness to industrial-scale clients. A few months' lead is worth a significant chunk of change. The full versions of the models will be served only to whitelisted clients, which also reduces distillability.
The capability floor is also important, not just the ceiling. If the floor continues to be raised, a lot of people will want to use the models that "don't make mistakes anymore".
The government is made of many people, and both reasons are at play. If a more cyber- and bio-capable AI is broadly released by another company and the White House does nothing to impede them, I will be very surprised.
Not today, but they will soon. They're working toward controlling all releases, and then they will start working on controlling the research itself, though they don't know yet that this is what they will soon want to do.
Nationalization is a step toward a pause. If the only entities at the table are states, the coordination problem becomes a lot easier.
No, but it's a great sign! It wouldn't be sane for any government to be friendly with the AI companies, so I smile when they fight. A stable multilateral pause is downstream from the vibes as well, not only from good policy.
1
Certainly, but it turns out that LLMs do appear to functionally have emotions. Whether they feel emotions is an open question, but they do seem to have them.