So... what's the general take on the hantavirus outbreak?
I ran through my checklist. Looks low-risk to me. Basically a very deadly disease that nevertheless shows nil or inefficient human-to-human transmission and doesn't show signs of regional or global growth. It depends heavily on specific South American rat species for transmission to humans. The original exposure was on or around April 1. There are no new concerning mutations (see below), and so far the flight attendant who had been feared to have picked it up on a flight has tested negative, though there's an up to 6 week incubation period and she was exposed on April 25th, about 2 weeks ago. There were also 3 other plane passengers who had contact with an infected person on a plane and they've tested negative so far as well.
The consensus public health position is that "person-to-person transmission is possible through close personal contact, such as between couples." If it turns out that this is wrong and that it's much more human-transmissible than previously believed, then the complete failure to quarantine passengers after the discovery of hantavirus on the Hondius cruise ship and first passenger death would completely alter my projection. But so far, there do not seem to be s... (read more)
Hard disagree. 8% is wildly overinflated and it's at 4% now. If you don't find yourself substantially revising your plans based on a 50% drop in pandemic probability, and if you didn't get seriously concerned about an 8% chance of a 40% CFR pandemic, then even you aren't taking them seriously. The main utility of this prediction market is driving internet chatter on how useful prediction markets are.
I think the prediction market can be useful even if you need to apply corrections to the headline number. It makes a huge difference to me whether Manifold says 8% or 40%, even though I won't necessarily fully trust either number until it's had a while longer to shake out (and even then it depends on the market dynamics, liquidity, etc.)
It sounds to me like you're expecting too much from all information. Consider the information that physical matter is made of atoms, which are about a nanometer across, and have positively charged protons and neutrons in the center, and then much smaller electrons whizzing around in orbits. This has been revolutionary for society, leading to so much ability to do engineering and chemistry and understanding cosmology and so forth.
A typical teenager learning this information for the first time might ask "what utility will this give me personally?" and find it has little direct application. Yet I would not advise them that this information is not worth knowing.
That's the first point, that information doesn't need to be directly connected to an outcome to be worth knowing.
The second point is that fine gradations in information are valuable. I can imagine someone similarly saying "Why should it be valuable to anyone to know the difference between Apple Stock being $285 versus $275? Surely we should just care whether it's doing well or not? Why don't we just replace it with the words "Great" "Good" "Bad" "Worse"? Yet often small signs tell us something. A few percentage points of dip can... (read more)

edit: update:


I think it is important to note that the opportunity created by shifting x-risk sentiment, or the current "crunch time", is more of an opportunity to shift public sentiment around AI x-risk rather than politician sentiment.
In some cases, such as the US President, portions of the public will follow the politicians preferences; but in most cases this is not true.
The most valuable opportunity for reducing x-risk right now is that shifting public awareness and sentiment of AI X-risk could have a direct effect on the outcome of the upcoming US midterms.
So focusing on making AI risk as much of a bipartisan, public focus as possible is our greatest lever on how policymakers will be thinking about AI risk.
So much for the AI safety issue not becoming polarized.
I wouldn't be totally surprised if this flips once Trump is out of office given the weight of popular opinion, c.f. sliding Democratic opinions on immigration. But that's a long time from now.
And this is why when someone calls your world model weird, you should just check in on who is being more continuously surprised by strange seeming developments like this. Because it turns out, in order to stop being surprised by reality.... you have to have some pretty weird seeming expectations about the future.
Not everything is a conspiracy.
The GOP generally believes in deregulation and free markets. Tariffs aside, so do Trump and his voters.
Every regulation comes with "but regulating {nail salons, coal, car engines, bridges, factories, AI} is actually important". Free market proponents have learned to ignore that and plow ahead. You have a lot of work ahead of you to convince people, not try the (metaphoric) assassinate the leaders of {x terrorist group} approach.
Learn the lessons Petraeus did.
This is interesting because, it is not discussed enough that AlphaGo worked the same way LLMs work: it was pretrained on a large dataset of human moves, then posttrained through reinforcement learning. The shift to AlphaZero removed the pretraining, showing that pretraining data didn't matter at all for the model's capabilities (quite the contrary, AlphaZero is superior to AlphaGo). While this coding model is still pretrained, it also shows that good pretraining data doesn't matter nearly as much as one could think.
it also shows that good pretraining data doesn't matter nearly as much as one could think
I don’t think it shows that. It arguably suggests that abundant pretraining data doesn’t matter as much as one could think. As opposed to good pretraining data. I presume that the codebases + agentic coding transcripts that they SFT’d on were high quality, right? [ETA: WHOOPS SEE MATRICE REPLY]
As for data efficiency, after the pre-1930 pretraining, IIUC it takes 250 training examples ≈ 13 million tokens before “the model solves its first [SWE-bench] issue”, and 75000 training examples ≈ 4 billion tokens gets to pass@1 of 4.5%.
Is that a more or less than expected? I dunno, it depends on what you were expecting. For what it’s worth, Gemini says 13 million tokens is about what a human could read in 650 hours non-stop (40 hours/week for 16 weeks).
My first Fable benchmark was to one-shot turning `emacs -batch -l dunnet` into a graphical adventure game and it hit the safety guardrails bc one of the puzzles involve nitroglycerine 😭