All of anson.ho's Comments + Replies

Grokking “Semi-informative priors over AI timelines”

To make sure I'm understanding you correctly, do you think the largest problem comes from (1) thinking of AGI development as a sequence of Bernoulli trials, or (2) each Bernoulli trial having constant probability, or (3) both?

It's not obvious to me that (1) is hugely problematic - isn't Laplace's rule of succession commonly applied to forecasting previously unseen events? Are you perhaps arguing that there's something particular to AGI development such that thinking of it as a series of Bernoulli trials is completely invalid?

I'm more sympathetic to your cr... (read more)

1JBlack2mo
(1) seems reasonable as a model at this level of abstraction, absent quibbles about whether some outcome really is AGI or not, instead of some degree of AGI-ness. (2) seems utterly wrong, and I don't think it even makes sense to talk about a "first trial" as being a clear-cut thing, let alone having a sensible probability, and definitely not as something related to success of all future trials. I contend that it is not even "semi-informative", it is useless.