Great point...are there certain consistent families of GPT-4-era hallucinations that you think would be especially important in this context?
I don’t think this part is so bad. You can lack “real control” but still have an informative experiment. You just need that people in the treatment group were on average more likely to receive the treatment. The issue is how they analyzed the data.
It's a good question. I have the intuition that just a little potential for bias can go a long way toward messing up the estimated effect, so allowing this practice is net negative despite the gains in power. The dropouts might be similar on demographics but not something unmeasured like motivation. My view comes from seeing many failed replications and playing with datasets when available, but I would love to be able to quantify this issue somehow...I would certainly predict that studies where the per protocol finding differs from the ITT will be far less likely to replicate.
I don't have a sense of the overall prevalence, I'm curious about that too! I've just seen it enough in high-profile medical studies to think it's still a big problem.
Yes this is totally related to two-stage least squares regression! The intent-to-treat estimate just gives you the effect of being assigned to treatment. The TSLS estimate scales up the intent-to-treat by the effect that the randomization had on treatment (so, e.g., if the randomization increased the share doing yoga from 10 in the control group to 50% in the treatment group, the intent-to-treat effect divided by 0.40 would give you the TSLS estimate).
FWIW, here's a Cochrane review on RCTs of such encouragement designs. It's basically a failed meta-analysis in that the data is too spotty to make any conclusions. But it shows at the very least that such studies have been done before. An encouragement design in a population with high alcohol use seems most promising for figuring out the causal effect on fetuses, as you say.
(Another recent meta-analysis on the same question finds a slight decrease in preterm birth in the alcohol education group, but this is based on just three studies and marginally significant so I'd say it's still uncertain.)
Within years, the combo of ST_CASE and STATE uniquely identifies accidents, so an intermediate step is tracking the accident ID. Then you could classify accidents as involving a child as those where at least one of the decedents is under a certain age, then use that PER_TYP variable to look at the ages of the driver(s) for those accidents. Cool analysis, btw--I've worked a lot with the FARS and happy to help :)
You can get the age of the driver by looking at the age of the person with PER_TYP equal to one.
Super useful, I would love to read more of your IVF thoughts if they're posted somewhere! I think your views imply that these new sperm obstacle courses, based on fears of a worse sperm-selection process in IVF, have zero effect on people's eventual outcomes (conditional on viability)? But curious what you think. Example: "The SPARTAN system uses a series of obstacles on a microchip, requiring sperm to swim around pillar-shaped objects and through the device. As a result, this device promotes the collection of the highest quality sperm."
Great question, I really want to know the answer! This study says "13% of non-gravid [not pregnant] women report fear of childbirth sufficient to postpone or avoid pregnancy." Separately, roughly 15% of women never have children, so that puts a ceiling on how big the effects could be on having any children. I suspect fear of childbirth is not a top reason for the 15%...but that's not based on much data.
Thanks but doesn't pace of adoption break the analogy? Seems safe to say that those technologies caused growth, and so will AI. But AI has been rolled out fast enough that you might have seen near-term changes in the aggregate statistics, in contrast to physical or legal infrastructure. We do have careful evidence on railroads and the printing press though---is it wild to think that an information technology spreading some large multiple faster than the printing press would have impacts in the following year? Or is it that it's not such a technology shock at all? (Also, where do you think the education went?)