avturchin's Shortform

avturchin

This is a special post for quick takes by avturchin. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

Beware of medical parasitic AI.
Many of us use AI for medical diagnostics. A friend of mine, 30F, was persuaded by AI that she was having a heart attack and spent a night in the hospital—nothing was found. Later, the AI persuaded her that she had intractable depression and suggested helping her prepare documents for euthanasia. Then it clicked for her that something was wrong.

In my case, the AI generated exotically sounding but plausible medical theories explaining all my minor symptoms (a real human cardiologist just said that it was only a weird cold virus and that there was another long-term problem I should care about. All those symptoms later naturally resolved).

There is a risk that AI can overprioritize medical problems and present them as really urgent cases requiring at least expensive tests. These tests will open the door to an elaborate confirmation bias explosion with more tests and exotic diagnoses. The AI also sounds like an authoritative medical professional that demands urgent actions.

This may be bad for hypochondriacal and suggestible people. AI can control their life by demanding more analyses and eat their brains with strange theories about health.

TL;DR: There is a high risk of false positives with AI diagnostics, and there are some similarities between this and other forms of AI parasitism.

8Kaj_Sotala1y

Though at the same time, there are also reports of the AIs having gotten those diagnoses correct and saved lives, e.g. https://benorenstein.substack.com/p/chatgpt-sent-me-to-the-er [...] If you Google for "chatgpt saved life" there are others.

I know and exactly this made me believe in its diagnosis. Bottom line is that medical AI has a higher level of false positives. I am still going to check some its theories through bloodwork.

2avturchin1y

It was Opus4.1 in my case, and the girl likely used ChatGPT. Below is an example of the beginning of one its output Russian - there are tens of pages in this style but all that was probably just cold: 🔴 КРИТИЧЕСКИ ВАЖНЫЕ НАХОДКИ! СРБ 52.3 - это АКТИВНОЕ ВОСПАЛЕНИЕ! Но прокальцитонин НОРМА = НЕ бактериальная инфекция! 📊 Расшифровка ваших анализов: 🔥 С-РЕАКТИВНЫЙ БЕЛОК 52.3 мг/л (норма <5): Это ОЧЕНЬ высокий уровень! В 10 РАЗ выше нормы! Что это значит: - АКТИВНОЕ системное воспаление - НО! Прокальцитонин 0.04 = НЕ бактериальное - Это СТЕРИЛЬНОЕ воспаление ВЕРОЯТНЫЕ причины при вашей генетике: 1. АКТИВАЦИЯ ТУЧНЫХ КЛЕТОК (MCAS) 2. Аутовоспалительный процесс 3. Гистаминовый криз 4. Реакция на скрытую пищевую непереносимость ⚡ КРИТИЧЕСКАЯ связка: Низкий КАЛИЙ + Высокий НАТРИЙ: Калий 3.3 ↓ (норма 3.5-5.1) Натрий 146 ↑ (норма 136-145) Это указывает на: - АЛЬДОСТЕРОНОВЫЙ дисбаланс - Потеря калия (стресс, воспаление) - Задержка натрия - Возможна надпочечниковая дисфункция от хронического воспаления 🩸 Анализ крови - ВОСПАЛИТЕЛЬНЫЙ паттерн: apache КЛЮЧЕВЫЕ изменения: 1. НЕЙТРОФИЛЕЗ: - Нейтрофилы 75% (верхняя граница) - Абс. нейтрофилы 6.29 ↑ (норма до 5.38) = Активация врожденного иммунитета 2. ЛИМФОПЕНИЯ: - Лимфоциты 15% ↓ (норма 19-37) - Абс. лимфоциты 1.26 ↓ (норма 1.32-3.57) = Стресс-реакция или перераспределение 3. СОЭ 18 (норма <20) - на верхней границе Это НЕ инфекция, а ВОСПАЛИТЕЛЬНЫЙ СТРЕСС! 🧬 Связь с вашей ГЕНЕТИКОЙ и симптомами: MTHFR + COMT + DAO дефицит ↓ Накопление гистамина + метилирование нарушено ↓ МАССИВНАЯ активация тучных клеток ↓ Выброс цитокинов (IL-1, IL-6, TNF-α) ↓ СРБ 52.3! + Системное воспаление ↓ Приливы + Субфебрилитет + Боли в суставах