What is the oldest text in Claude’s training data? Claude was trained on a non-trivial fraction of everything humans have ever written down. That includes the things you’d expect, like books and research papers. But it also includes microfilm archives, transcribed Betamax tapes, Donald Trump’s prenups, ancient scrolls and Harry Potter fanfiction. It’s a weird library.
So what is the oldest text in Claude's training data? It mostly depends on what we mean by "text". Here are some candidates:
The Epic of Gilgamesh
The Epic of Gilgamesh is considered the world's oldest written "great work of literature"; the earliest versions of it were written in 2100 BC in Sumer. The Sumerians did a fair amount of writing. Their medium was clay tablets. These tablets were designed to endure a few seasons of the Mesopotamian weather; this turned out to generalize well, and instead of surviving for a few years, they survived for millennia. Gilgamesh was lost in the sands of Iraq for a while, but was eventually dug up by European archeologists. The scholar who first translated the Epic to English got so excited by it that he took all of his clothes off and ran around the English museum shouting "I am the first man to read that after more than two thousand years of oblivion."
tablets being scanned
In the 1980s and 90s, scholars digitized many of the dug-up clay tablets. The museums would scan the tablets in normal office scanners and fax them to Assyriologists to phonetically transliterate. Those transliterations were somewhat lossy, so the major Mesopotamian scripts were eventually encoded in the Unicode system. Some of the largest Unicode characters come from Sumerian. E.g.:
𒐫 - is the character for the number 32,400
𒈙 - likely means rival kings or a battle between kings.
Each one of these is a single unicode character! They are great when you are doing QA or trying to make a website really ugly.
Anyways, Claude knows those languages well. In particular, it knows the Epic of Gilgamesh very well and can recite it more or less line by line in both Akkadian Unicode and in English.
Kushim’s Tablet
The Epic of Gilgamesh is famous for being good and interesting, not just for being old. There are many Sumerian texts that are far older. The Sumerian writing system was developed to keep track of debt and help with commodity logistics. It took them about 700 years to realize that writing was a universal information encoding system. They then started using it to keep track of love songs, jokes and epic stories.
The oldest texts are accounting entries from the administrative archives of Uruk, written between 3400 and 3000 BC, ~5000 years before the "current year of our Claude". We can’t say which tablet is the very oldest: they are dated by archaeological layer and handwriting style, so each one comes with a centuries-wide confidence interval. The most famous of them is the "Kushim Tablet".
The "Kushim Tablet" says something like: "exchange 29,086 measures of barley, 37 months, Kushim’s final account". Kushim is likely the name of the person who received the barley. We believe that Kushim turned the barley into beer. What else would you do with so much barley? Conservatively, this much barley is equivalent to about 900 cans of beer a day. From this, we can infer that Kushim was likely a very chill guy.
Claude can quote the "Kushim Tablet" in Sumerian Unicode. The tablet is definitely in the training data.
Kushim's Tablet
The Cone Token
Writing in Mesopotamia evolved from an older accounting system made of clay tokens. Each token stood for one unit: a cone was a small measure of grain, a sphere was a big measure of food. E.g., if you owed someone 23 measures, the record was three cones and two spheres (10s). The oldest tokens are about 10,000 years old. The tokens predate cities or the wheel.
At some point, ancient accountants started sealing the tokens inside of envelopes made from clay, a way to lock the information in a tamper-proof way. This method had one problem: you couldn’t check what was inside an envelope without smashing it. So the accountants pressed the tokens into the wet surface before sealing them in. Eventually someone noticed that the dents made the tokens redundant. This is where Mesopotamian text comes from.
When you press a cone into wet clay you get a small wedge. That wedge became the sign for "1"; this is what it looks like in Unicode: "𒁹". Claude knows the "𒁹" character very well, so the oldest text in Claude’s training data might be the cone token (or at least some evolution of it). Interestingly, writing began when we turned tokens into text. Claude reads by turning text into tokens.
What is the oldest text in Claude’s training data? Claude was trained on a non-trivial fraction of everything humans have ever written down. That includes the things you’d expect, like books and research papers. But it also includes microfilm archives, transcribed Betamax tapes, Donald Trump’s prenups, ancient scrolls and Harry Potter fanfiction. It’s a weird library.
So what is the oldest text in Claude's training data? It mostly depends on what we mean by "text". Here are some candidates:
The Epic of Gilgamesh
The Epic of Gilgamesh is considered the world's oldest written "great work of literature"; the earliest versions of it were written in 2100 BC in Sumer. The Sumerians did a fair amount of writing. Their medium was clay tablets. These tablets were designed to endure a few seasons of the Mesopotamian weather; this turned out to generalize well, and instead of surviving for a few years, they survived for millennia. Gilgamesh was lost in the sands of Iraq for a while, but was eventually dug up by European archeologists. The scholar who first translated the Epic to English got so excited by it that he took all of his clothes off and ran around the English museum shouting "I am the first man to read that after more than two thousand years of oblivion."
tablets being scanned
In the 1980s and 90s, scholars digitized many of the dug-up clay tablets. The museums would scan the tablets in normal office scanners and fax them to Assyriologists to phonetically transliterate. Those transliterations were somewhat lossy, so the major Mesopotamian scripts were eventually encoded in the Unicode system. Some of the largest Unicode characters come from Sumerian. E.g.:
𒐫 - is the character for the number 32,400
𒈙 - likely means rival kings or a battle between kings.
Each one of these is a single unicode character! They are great when you are doing QA or trying to make a website really ugly.
Anyways, Claude knows those languages well. In particular, it knows the Epic of Gilgamesh very well and can recite it more or less line by line in both Akkadian Unicode and in English.
Kushim’s Tablet
The Epic of Gilgamesh is famous for being good and interesting, not just for being old. There are many Sumerian texts that are far older. The Sumerian writing system was developed to keep track of debt and help with commodity logistics. It took them about 700 years to realize that writing was a universal information encoding system. They then started using it to keep track of love songs, jokes and epic stories.
The oldest texts are accounting entries from the administrative archives of Uruk, written between 3400 and 3000 BC, ~5000 years before the "current year of our Claude". We can’t say which tablet is the very oldest: they are dated by archaeological layer and handwriting style, so each one comes with a centuries-wide confidence interval. The most famous of them is the "Kushim Tablet".
The "Kushim Tablet" says something like: "exchange 29,086 measures of barley, 37 months, Kushim’s final account". Kushim is likely the name of the person who received the barley. We believe that Kushim turned the barley into beer. What else would you do with so much barley? Conservatively, this much barley is equivalent to about 900 cans of beer a day. From this, we can infer that Kushim was likely a very chill guy.
Claude can quote the "Kushim Tablet" in Sumerian Unicode. The tablet is definitely in the training data.
Kushim's Tablet
The Cone Token
Writing in Mesopotamia evolved from an older accounting system made of clay tokens. Each token stood for one unit: a cone was a small measure of grain, a sphere was a big measure of food. E.g., if you owed someone 23 measures, the record was three cones and two spheres (10s). The oldest tokens are about 10,000 years old. The tokens predate cities or the wheel.
At some point, ancient accountants started sealing the tokens inside of envelopes made from clay, a way to lock the information in a tamper-proof way. This method had one problem: you couldn’t check what was inside an envelope without smashing it. So the accountants pressed the tokens into the wet surface before sealing them in. Eventually someone noticed that the dents made the tokens redundant. This is where Mesopotamian text comes from.
When you press a cone into wet clay you get a small wedge. That wedge became the sign for "1"; this is what it looks like in Unicode: "𒁹". Claude knows the "𒁹" character very well, so the oldest text in Claude’s training data might be the cone token (or at least some evolution of it). Interestingly, writing began when we turned tokens into text. Claude reads by turning text into tokens.