Introduction

Epistemic spot checks are a series in which I select claims from the first few chapters of a book and investigate them for accuracy, to determine if a book is worth my time. This month’s subject is The Fall of Rome, by Bryan Ward-Perkins, which advocates for the view that Rome fell, and it was probably a military problem.

Like August’s The Fate of Rome, this spot check was done as part of a collaboration with Parallel Forecasting and Foretold, which means that instead of resolving a claim as true or false, I give a confidence distribution of what I think I would answer if I spent 10 hours on the question (in reality I spent 10-45 minutes per question). Sometimes the claim is a question with a numerical answer, sometimes it is just a statement and I state how likely I think the statement is to be true.

This spot check is subject to the same constraints as The Fate of Rome, including:

1. Some of my answers include research from the forecasters, not just my own.
2. Due to our procedure for choosing questions, I didn’t investigate all the claims I would have liked to.

Claims

Claim made by the text:  “[Emperor Valerian] spent the final years of his life as a captive at the Persian Court”
Question I answered: what is the chance that is true?
My answer: I estimate a chance of (99 – 3*lognormal(0,1)) that Emperor Valerian was captured by the Persians and spent multiple years as a prisoner before dying in captivity.

You don’t even have to click on the Wikipedia page to confirm this is the common story: it’s in the google preview for “emperor valerian”. So the only question is the chance that all of history got this wrong. Wikipedia lists five primary sources, of which I verified three.  https://www.ancient-origins.net/history/what-really-happened-valerian-was-roman-emperor-humiliated-and-skinned-hands-enemy-008598 raises questions about how badly Valerian was treated, but not that he was captive.

My only qualm is the chance that this could be a lie perpetuated at the time. Maybe Valerian died and the Persians used a double, maybe something weirder happened. System 2 says the chance of this is < 10% but gut says < 15%.

 

Claim made by the text: “What had totally disappeared, however, were the good-quality, low-value items, made in bulk, and available so widely in the Roman period”
Question I answered: What is the chance mass-produced, low-value items available so widely in the Roman period, disappear in Britain by 600 AD?
My answer: I estimate a chance of (64 to 93, normal distribution) that mass-produced, low-value items were available in Britain during Roman rule and not after 600 AD.

This was one of the hardest claims to investigate, because it represents original research by Ward-Perkins. I had basically given up on answering this without a pottery PhD until google suggestions gave me the perfect article.

This is actually a compound claim by Ward-Perkins: 

1. Roman coinage and mass-produced, low-cost, high-quality pottery disappeared from Britain and then the rest of post-Roman Europe.
2. The state of pottery and coinage is a good proxy for the state of goods and trades as a whole, because they preserve so amazingly well and are relatively easy to date.

Data points:

• https://brewminate.com/the-perils-of-periodization-roman-ceramics-in-britain-after-400-ce/ Cites exactly the pattern Ward-Perkins describes in pottery and coins, citing Ward-Perkins and 4 other source I couldn’t verify. This seems like very strong confirmation of W-P’s hypothesis. However it leaves open the chance that this is an area of contention, of which W-P and brewminate happen to be on the same side. Brewminate’s main focus is not on the British empire, but on pottery in the gap between the Roman’s and the Anglo-Saxons. I estimate that would bias them towards increasing the size of the discontinuity between Roman and post-Roman Britain.
• https://www.cambridge.org/core/journals/britannia/article/romanization-of-pottery-assemblages-in-the-east-and-northeast-of-england-during-the-first-century-ad-a-comparative-analysis/AE110A5FE5018C87E8DB284C458F1CC8 (cited by brewminate) establishes that pottery was romanized and centralized in the 1st century AD
• Roman Pottery by Kevin Greene establishes approximately the same thing, and was found independently of brewminate.
• https://www.romanobritain.org/10_crafts/rca_roman_british_pottery.php claims most pottery in Britain was imported even before Rome invaded (43 AD)
• https://www.romanobritain.org/10_crafts/rca_roman_british_pottery.php claims during Roman rule Britain had several high quality pottery production centers
• https://www.thegreatcoursesdaily.com/britain-after-the-romans-left/ (I think a very mainstream source) repeats the story that pottery is informative and pottery shows a decline after 400s.
• Searching for “the fall of rome ward-perkins criticism” turns up nothing interesting in the first two pages.
• Searched for “late antiquity british pottery” found nothing. “late antiquity” is the term for the narrative that Rome didn’t fall, it just transformed, and is what Ward-Perkins is directly arguing against, so I’d expect criticism of him to be coded with that term.
• https://brill.com/view/book/edcoll/9789004309784/B9789004309784-s016.xml says  “Inter-regional trade was not a motor for prosperity in the later Roman period in Britain”

• • Focuses on how amphorae were never really abundant in Britain
  • Chart stops at 400 AD
  • Graph showing large drops in amphorae distribution by 410 AD

• https://www.academia.edu/1353493/Form_function_and_technology_in_pottery_production_from_Late_Antiquity_to_the_early_Middle_Ages uses term “late antiquity” and cites end of large scale pottery production in Rome

If we believe Ward-Perkins and Brewminate, I estimate the chances that pottery massively declined at 95-99,  times 80-95 that other good declined with them. There remains the chances that the historical record is massively misleading (very unlikely with pots, although I don’t know how likely it is to have missed sites entirely), and that W-P et al are misinterpreting the record. I would be very surprised if so many sites had been missed as to invalidate this data, call it 5-15%. Gut feeling, 5-20% chance the W-P crowd are exaggerating the data, but given the absence of challenges, not higher than that and not a significant chance they’re just making shit up.

(95 to 99)*(85 to 95) * (80 to 95) = 64 to 93%

 

Claim made by the text: The Romans had mass literacy, which declined during the Dark Ages.
Question I answered: “[% population able to read at American 1st grade level during Imperial Rome] – [% population able to do same in the same geographic area in 1000 AD] = N%. What is N?”
My answer: I estimate that there is a 95% chance [Roman literacy] – [Dark Ages literacy] = (0 to 60, normal distribution) 

Data Points:

• https://en.wikipedia.org/wiki/Roman_Empire#Literacy,_books,_and_education
• “Estimates of the average literacy rate in the Empire range from 5 to 30% or higher, depending in part on the definition of “literacy” I’ll use the high end, since I’m using a pretty minimal definition of literacy
• https://www.quora.com/What-were-literacy-rates-in-Medieval-Europe-How-did-they-compare-to-literacy-rates-in-the-Roman-Empire
• “literacy levels were probably much lower than in Roman times during most of the Middle Ages – perhaps only 6% in England in 1300”
• https://www.quora.com/What-was-the-level-of-literacy-among-the-people-of-the-Roman-Empire-first-to-third-centuries-AD-If-only-the-wealthy-were-literate-what-was-the-point-of-the-written-propaganda-on-the-ancient-monuments claims literacy was low, 5-10 % of the population, with a maximum of 20 per cent.
• http://trisagionseraph.tripod.com/literacyf.html is hideous and should die

 

The highest estimate of literacy in Roman Empire I found is 30%.  Call it twice that for ability to read at a 1st grade level in cities. So the range is 5%-60%. 

The absolute lowest the European 1000AD literacy rate could be is 0; the highest estimate is 5% (and that was in the 1300s, which were probably more literate).  From the absence of graffiti I infer that even minimal literacy achievement dropped a great deal. 

Maximum = 60%-1% = 59%
Minimum = 5%-5%=0

 

Claim made by the text: “What some people describe as “the invasion of Rome by Germanic barbarians”, Walter Goffart describes as “the Romans incorporating the Germanic tribes into their citizenry and setting them up as rulers who reported to the empire.” and “Rome did fall, but only because it had voluntarily delegated its own power, not because it had been successfully invaded”.”
Question I answered: What is my confidence that this accurately represents historian Walter Goffart’s views?
My answer: I estimate that after 10 hours of research, I would be 68-92% confident this describes Goffart’s views accurately.

Data points:

• https://blog.oup.com/2005/12/the_fall_of_rom/
  • Peter Heather: The most influential statement of this, perhaps, is Walter Goffart’s brilliant aphorism that the fall of the Western Empire was just ‘an imaginative experiment that got a little out of hand’. Goffart means that changes in Roman policy towards the barbarians led to the emergence of the successor states, dependant on barbarian military power and incorporating Roman institutions, and that the process which brought this out was not a particularly violent one.
• https://www.goodreads.com/book/show/1680215.Barbarians_and_Romans_A_D_418_584?from_search=true 
  • “Despite intermittent turbulence and destruction, much of the Roman West came under barbarian control in an orderly fashion.”
• https://press.princeton.edu/titles/1036.html
  • “Despite intermittent turbulence and destruction, much of the Roman West came under barbarian control in an orderly fashion. Goths, Burgundians, and other aliens were accommodated within the provinces without disrupting the settled population or overturning the patterns of landownership. Walter Goffart examines these arrangements and shows that they were based on the procedures of Roman taxation, rather than on those of military billeting (the so-called hospitalitas system), as has long been thought. Resident proprietors could be left in undisturbed possession of their lands because the proceeds of taxation,rather than land itself, were awarded to the barbarian troops and their leaders.”
• https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1478-0542.2008.00523.x
  • “the barbarians and Rome, instead of being in constant conflict with each other, occupied a joined space, a single world in which both were entitled to share. What we call the barbarian invasions was primarily a drawing of foreigners into Roman service, a process sponsored, encouraged, and rewarded by Rome. Simultaneously, the Romans energetically upheld their supremacy. Many barbarian peoples were suppressed and vanished; the survivors were persuaded and learned to shoulder Roman tasks. Rome was never discredited or repudiated. The future endorsed and carried forward what the Empire stood for in religion, law, administration, literacy, and language.”
• https://books.google.com/books/about/Rome_s_Fall_and_After.html?id=55pDIwvWnpoC “Rome’s Fall and After” indicates Goffat does believe Rome fell. But suggests its main problem was constantinople, not interactions with barbarians at all. So top percentage correct = 90%)

 

This seems pretty conclusive that Goffart thought Barbarians were accommodated rather than conquered the area (so my minimum estimate that the summary was correct must be greater than 50%). However it’s not clear how much power he thought they took, or whether rome fell at all. This could be a poor restatement, or it could be that if I read Goffart’s actual work and not just book jacket blurbs I’d agree.

 

Question I answered: Chance Elizabeth would recommend this book as a reliable source on the topic to an interested friend, if they asked tomorrow (8/31/19)?
My answer: There is a (91-99%, normal distribution) chance I would recommend this to a friend.

99% is in range, because I definitely think it’s worth reading if they’re interested in the topic. I think I’d recommend it before Fate of Rome, because it establishes that rome fell more concretely.

Is there a chance I wouldn’t recommend it?

• They could have already read it
• They could be more interested in disease and climate change (in which case I’d recommend Fate)
• I could forget about it
• I could not want to take responsibility for their reading.
• I could be unconfident that Fall was better than what they’d find by chance.
  • This feels like the biggest one.
  • But the question doesn’t say “best book”, it just says “reliable source”
  • Only real qualm on that is that is normal history book qualms

So the minimum is 91%

 

Bonus Claims

These are the claims I didn’t check, but other people made predictions on how I would guess. Note that at this point the predictions haven’t been very accurate- whether they’re net positive depends on how you weight the questions. And Foretold is beta software that hasn’t prioritized export yet, so I’m using *shudder* screen shots. But for the sake of completeness:

Claim made by the text: The Fall of Rome: Roman Pottery pre-400AD was high quality and uniform.
Predicted answer: 29.9% to 63.5% chance this claim is correct

Claim made by the text: “In Britain new coins ceased to reach the island, except in tiny quantities, at the beginning of the fifth century”
Predicted answer: 31.6% to 94% chance this claim is correct

 

Claim made by the text: The Fall of Rome: [average German soldiers’ height] – [average Roman soldiers’ height] = N feet. What is N? .
Predicted answer: -0.107 to 0.61 ft.

 

Claim made by the text: The Romans chose to cede local control of Gaul to the Germanic tribes in the 400s, as opposed to losing them in a military conquest.
Predicted answer: 28.5% to 85.6% chance this claim is correct

 

Claim made by the text: The Germanic tribes who took over local control of Gaul in the 400s reported to the Emperor.
Predicted answer: 4.77% to 50.9% chance this claim is correct

 

Conclusion

The Fall of Rome did very well on spot-checking- no outright disagreements at all, just some uncertainties. 

On the other hand, The Fall of Rome barely mentions disease and doesn’t mention climate change at all, which my previous book, The Fate of Rome, claimed to be the main causes of the fall. The Fate of Rome did almost as well in epistemic spot checking as Fall, yet they can’t both be correct. What’s going on? I’m going to address that in a separate post, because I want to be able to link to it without forcing people to read this entire spot check.

In terms of readability, Fall starts slowly but the second half is by far the most interested I have ever been in pottery or archeology.

[Many thanks to my Patreon patrons and Parallel Forecast for financial support for this post]

Does combining epistemic spot checks and prediction markets sound super fun to you? Good news: We’re launching round three of the experiment today, with prizes of up to $65/question. The focal book will be The Unbound Prometheus, by David S. Landes, on the Industrial Revolution. The market opens today and will remain open until 10/27 (inclusive).