This is a special post for quick takes by DW11. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
When it comes to AI-driven biorisk, the classic threat actor is a terrorist or deranged lunatic who would previously never have been able to produce a bioweapon, but will now, at some point, be able to do so. These are basically the people Dario was talking about in the biology section of The Adolescence of Technology (I know people had issues with this essay but I thought the biology section was quite good).
I actually think an underrated threat, relative to a malicious actor, could come from AI enabling more dangerous biological research, including gain of function experiments and work on mirror life.
1) Advanced AI will probably dramatically increase productivity in pathogen research, just as it will for other fields. I don't see any reason why people carrying out risky research won't also become much more productive.
2) As well as increasing the total number of experiments being run, very advanced AI could enable riskier research to be undertaken - for example, afaik there are many unsolved problems preventing us from creating mirror bacteria. Presumably AI will help us solve these.
3) The people who undertake this research are motivated by a desire to help prevent pandemics and think that this will help. Increased fear of biorisks, caused by AI progress (a 'Mythos moment' for biorisk), could actually increase the number of risky experiments that are undertaken.
4) Almost all of the safeguards that I've seen people propose to combat biorisks won't stop this. Since this work will be done by government or academia, they will have access to the latest models with biological safeguards removed, and obviously stuff like nucleic acid synthesis screening or restrictions on hardware won't do much.
5) The uplift required to enable this is much less than that required to allow some random lone wolf to create bioweapons on his own, so the risk will probably be here sooner.
NB: I'm not a biologist and have no special insight into the current state of the field.
I've been building a small fitness app with Claude over the last week or so, all in the same claude code window on desktop. On the last few turns today Claude keeps telling me that this is a 'good place to pause' whenever it finishes building some feature.
I've seen people complain about this before on twitter but they seem to be experiencing it after coding for hours on end, so I assumed it was a user welfare thing. Is it the case that Claude has no idea of the timespan that this window has been open and thinks I've been talking to it non-stop? I thought that the time and date were in the system prompt, so it should be able to tell that this has taken place over several days?
1Last time I checked, Claude Code doesn't see the date or time at all unless it looks it up with CLI tools. I think Claude on the web sees it at the beginning of the conversation and has an MCP tool to look it up mid-conversation.
I find it kind of strange that they don't inject this on every turn, but maybe it's not worth the tokens in most conversations.
Have you been accidentally cueing it into the time somehow? I only get messages like that when I phrase messages in a way that implies something about the passage of time, e.g. "Tonight we're going to work on message parsing: {SPEC}"
Does anyone with a medical/engineering background have any thoughts on Midjourney's new "Midjourney Scanner"?
https://www.midjourney.com/medical/blogpost
I'm seeing a lot of excited posts on twitter about this but I'm inherently extremely sceptical of an image model company pivoting to medical technology.
I'm also quite sceptical of the concept of easily available full-body scanning, at least for the near future, given it could seemingly lead to a lot of unnecessary medical treatment based on benign abnormalities.
ultrasound is awful to work with in traditional medical image processing technologies and pretty darn bad in 2015-2025 convolutional medical imaging ai technologies. Magnetic Resonance is so much better when its the right tool for the job that replacing it with ultrasound for cost reasons is a tarpit. There are cases where ultrasound is better but these center on leveraging an extremely talented operator who is manipulating the probe manually, which you lose in any of these bath based approaches. I’d be surprised if this goes anywhere.
Credentials: I have beaten ultrasound with the ML stick until it yielded a few times ( https://scholar.google.com/citations?hl=en&user=O1xhOlUAAAAJ ) it was terrible and I have lasting resentment toward an imaging modality, though great fondness for all my colaborators. Many failed projects that did not make the google acholar
I don't, but here are other people's thoughts.
Midjourney Medical | Hacker News
jmhmd: Some initial thoughts as a practicing radiologist:
- (…)
- They show the reconstructed images as though they are a low resolution CT, and promise that quality will improve as they iterate. This is cool, but ultrasound is not CT. Ultrasound cannot image the lungs, as they are filled with air. You cannot find bone lesions, as the sound waves do not penetrate the cortex. You cannot image many structures in the abdomen if they are surrounded by gas-filled bowel. The brain is encased in bone, so you might get some penetration but it will be very limited. Even with theoretically perfect AI reconstruction, these scans will not be true "full body" in that there will be structures that are not reliably imaged. Imagine paying for weekly full body scans for years, everything looks fine, then its the lung cancer surrounded by air and invisible to ultrasound that kills you (that's why we use CT for lung screening!)
- The images they show are very cool, and do appear to show the correct structures. I realize this is early, but fuzzy shapes of organs is very, very far from medically useful. The whole point of screening is to identify problems early, often by definition, small. This technology looks like it will be best for seeing large, superficial (close to the skin) structures, whereas for effective screening, you want the opposite - small, deep structures.
- (…)
- Many people mistakenly believe that early diagnosis is the final boss in medicine, that if only we could find every cancer early we could prevent all those deaths. There are, in fact, many, many other hurdles and bottlenecks. Many chronic, expensive diseases do not have clear imaging manifestations. The claim that "it's completely possible that with enough early imaging in the future, the world could avoid 30% of all deaths and 50% of all healthcare costs", I think, to any practicing physician, would sound completely divorced from reality.
IshKebab: I used to work in ultrasound, and full body scans with the body underwater is definitely feasible and probably a good idea.
Also there's absolutely no way that it will be as good as MRI. In general ultrasound imaging is shit. The main reasons it is used are because it is very cheap and completely harmless. The actual images you get are mostly just speckle.
https://www.astralcodexten.com/p/preliminary-thoughts-on-the-midjourney + his twitter thread
I think the narrative among the SF AI crowd has escaped its basis in the medical facts, so I want to throw a bit of cold water on it. I’m a psychiatrist, which is about as far as you can get from radiology while still being a doctor, so this is speculation only, and you can ignore it if you find an actual radiologist or ultrasonographer with opinions. Still, my take is that this scanner isn’t useful for most current serious medical applications. It could potentially be used to pioneer a new class of low-risk screening applications, but it’s unclear whether these are good, and depends a lot on what other future technology gets invented in parallel.
Why can’t this immediately replace existing medical image modalities like normal ultrasound, CT, or MRI?
Ultrasound is great, but it can’t penetrate bone or air. Many things doctors want to look at involve bone or air in some way.
Couldn’t this technology enable new, non-specific-diagnostic uses for healthy people?
why don’t people get yearly whole-body MRI screenings? Some people do - companies like this provide them, and some rich people who can pay $2,000 out of pocket consume them. But the medical consensus currently recommends against them because they’re more likely to produce dangerous false positives than helpful true positives, and studies have failed to demonstrate benefit.
Couldn’t this technology become more useful in the future?
Yes. I think the best way to think of this is as a bet that future technology develops in a way that allows new possibilities for diagnostic ultrasound - or, even better, an attempt to gather the training data / interest / investment that will make this happen.
Appendix: Highlights From The Comments On Twitter
I mentioned this on Twitter and got some great responses.
The responses from real radiologists were universally negative. Here are some examples: (screenshots are annoying to embed please go to the blog)
The trivial reason is that due to the limitations of physics ultrasound will always be less capable at resolving anatomy than MRI or x-ray-based methods that we already have
Anthropic has published a piece on RSI where they say they will work on the systems required to slowdown or pause AI progress, if necessary. https://www.anthropic.com/institute/recursive-self-improvement
"If it were possible to effectively slow the development of this technology to give ourselves more time to deal with its immense implications, we think that would likely be a good thing. But if a slowdown simply lets the least cautious actors catch up technologically, it could leave everyone less safe. Without a global coordination mechanism, companies and governments will have to make difficult decisions about safety while under competitive and geopolitical pressures.
We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology. The Anthropic Institute will conduct research—in collaboration with many others—and take actions to help build the systems that a credible slowdown or pause would require. These systems would enable frontier AI developers to verify that others globally have actually stopped or slowed, and that a bad actor could not use the auspices of a coordinated slowdown to jump ahead in secret. If such systems existed, we expect that we would slow down or temporarily pause, if other developers at or near the frontier also did so in a verifiable manner."
Inspired by this Jack Clark tweet (as well as the emphasis on external review in RSP v3):
https://x.com/jackclarkSF/status/2053847777889964376
Has Anthropic made Mythos available to any external AI safety organisations for internal use (not just evaluation)?
There do seem to be some NDAs relating to Mythos given the lack of people on twitter talking about having access, so perhaps they are using it and are just keeping quiet. But it seems reasonable that they should be given access, given that it seems to be fairly widely-deployed for cyber at this point (not to mention internally at Anthropic).
Given all the discussion of Mythos, I'm surprised that I haven't seen anyone point out that OpenAI launched Trusted Access for Cyber alongside codex-5.3:
https://openai.com/index/trusted-access-for-cyber/
It seems that if you try to use 5.3 for cyber without verifying your identity, you get routed to 5.2:
https://developers.openai.com/codex/concepts/cyber-safety
So the practice of gating capabilities based on cyber risk is not totally brand new. I'm curious to see how they handle this for 5.5, given that I expect it would be hard to generally release a model anywhere close to Mythos now without causing backlash.