All of CharlesD's Comments + Replies

That seems like a different question which is partially entangled with AI but not necessarily, as more screen time doesn't necessarily need to be caused by AI, and the harms are harder to evaluate (even the sign of the value of "more screen time" is probably disputed).

The article convincingly makes the weaker claim that there's no guarantee of a fire alarm, and provides several cases which support this. I don't buy the claim (which the article also tries to make) that there is no possible fire alarm, and such a claim seems impossible to prove anyway. 

Whether it's smoke or a fire alarm, that doesn't really address the specific question I'm asking, in any case.

Great point - I'm not sure if that contained aspects which are similar enough to AI to resolve such a question. This source doesn't think it counts as AI (though it doesn't provide much of an argument for this) and I can't find reference to machine learning or AI on the MCAS page, though clearly one could use AI tools to develop an automated control system like this and I don't feel well positioned to judge whether it should count. 

3Anon User2y
To clarify - I do not think MCAS specifically is an AI based system, I was just thinking of a hypothetical future similar system that does include a weak AI component, but where, similarly to ACAS the issue is not so much with the flaw in AI itself, but in how it is being used in a larger system. In other words, I think your test needs to make a distinction between a situation where one needed a trustworthy AI, and the actual AI was unintentionally/unexpectedly untrustworthy vs a situation where perhaps the AI performed reasonably well, but the use of AI was problematic, causing a disaster anyway.

Thanks for a great post! I have a concern about your sample sizes however.

I am looking into similar questions myself, and while reading your post I was surprised to see your Metaculus sample claimed as 45k predictions. These are not actually individual predictions, but rather the time series of community predictions, which are much less information dense, as this is just the median of the recent community predictions at that time and typically a new prediction will have a small effect on this value. I think claiming the sample size is 45k is therefore a bi... (read more)

Yep, I share your concerns! I wanted to include them in the post, but then I got busy. Perhaps I'll update it in the forseeable future (no promises however, I'm pretty busy with other things). Maybe I'll just put a warning at the top of the article. And, in case you publish your stuff, I'd love to read it.