In 1963, Mosteller and Wallace published Inference in an Authorship Problem, which used Bayesian statistics to try to infer who wrote some of the disputed Federalist Papers. (Answer: Madison) Anyway, at the end they have a list of "Remarks on Bayesian studies" which is astonishing to read 62 years later:
- Study of variation of results with different priors is recommended. Bracketing the prior is often easy. When founded in data, the choice of the prior has a status like that of the data distribution—subjectivity tempered with empiricism.
- Where possible, priors should be empirically oriented. Planning ahead for the collection of data suitable for estimating the form and underlying constants of prior distribution is useful and important, and very likely not hard to do once one gets in the frame of mind of preparing to use Bayes' theorem. The remark is all the more germane in the field of repetitive studies,where there has never been much excuse for the failure to collect such data.
- In any method of inference, data distributions matter enormously, and their study and choice requires the development of new and systematic methods.
- Statisticians need to provide richer sets of priors and data distributions than we now have, with a view to their manageability as well as their utility in studies of data. For example, mixture distributions can give satisfactory tail properties, but are grisly mathematically.
- The moment one leaves the simplest application of Bayes' theorem, the applier finds himself involved in a welter of makeshifts and approximations. This trouble and its cures require systematic developments. The prospect of useful exact treatments can be neglected.
- Simple Bayesian methods are needed that can be applied without appeal to high speed computers.
- Users of Bayes' theorem will find that many statistical jobs are readily handled by standard devices, but not by available Bayesian techniques. To avoid such devices for the sake of consistency reduces the quality of the research. On the other hand, time spent developing Bayesian procedures for a standard problem can be a profitable pursuit.
- In summary, for large scale data analysis, Bayesian methods require new studies of theoretical and empirical distributions and their approximation and estimation comparable in extent to those provided by statisticians up to 1935.
Some things never change!