TL;DR: I have scraped a database of World Record improvements for fastest videogame completition for several videogames, noted down some observations about the trends of improvement and attempted to model them with some simple regressions. Reach out if you'd be interested in researching this topic!
Key points
I argue that researching speedrunning can help us understand scientific discovery, AI alignment and extremal distributions. More.
I’ve scraped a dataset on world record improvements in videogame speedrunning. It spans 15 games, 22 categories and 1462 runs. More.
Most world record improvements I studied follow a diminishing returns pattern. Some exhibit successive cascades of improvements, with continuous phases of diminishing returns periodically interrupted by (presumably) sudden discoveries that speed up the rate of progress. More.
Simple linear extrapolation techniques could not improve on just guessing that the world record will not change in the near future. More.
Possible next steps include trying different extrapolation techniques, modelling the discontinuities in the data and curating a dataset of World Record improvements in Tool Assisted Speedruns. More.
The script to scrape the data and extrapolate it is available here. A snapshot of the data as of 30/07/2021 is available here.
Figure 1: Speedrunning world record progression for the most popular categories. The horizontal axis is the date of submission and the vertical axis is run time in seconds. Spans 22 categories and 1462 runs. Hidden from the graph are 5 runs before 2011. See the rest of the data here.
Feedback on the project would be appreciated. I am specially keen on discussion about:
Differences and commonalities to expect between speedrunning and technological improvement in different fields.
Discussion on how to mathematically model the discontinuities in the data.
Ideas on which techniques to prioritize to extrapolate the observed trends.
Is there any way to estimate how many cumulative games that speedrunners have run at a given point?
One should be able to use the Speedrun.com API to search for the number of runs submitted by a certain date, as a proxy for the cumulative games (though it will not reflect all attempts since AFAIK many runners only submit their personal bests to speedrun.com).
Additionally, speedrun.com provides some stats on the amount of runs and players for each game, for example the current stats for Super Metroid can be found here: https://www.speedrun.com/supermetroid/gamestats
Current stats for Super Metroid according to Speedrun.com
There are some problems with this approach too.
These are aggregated by game, not by category, so one would need to somehow split the runs among popular categories of the same game.
There is only current data avaiable through the webpage. There might be a way to access historical data through the API. If not, one would need to use archived versions of the pages and interpolate the scrapped stats.
I'd be excited about learning about the results of either approach if anybody ends up scrapping this data!
[this is a linkpost to Analysis of World Records in Speedrunning]
ETA: a more recent analysis can be found here.
TL;DR: I have scraped a database of World Record improvements for fastest videogame completition for several videogames, noted down some observations about the trends of improvement and attempted to model them with some simple regressions. Reach out if you'd be interested in researching this topic!
Key points
The script to scrape the data and extrapolate it is available here. A snapshot of the data as of 30/07/2021 is available here.
Feedback on the project would be appreciated. I am specially keen on discussion about: