I’ve recently read the book Inventing Temperature, and very much enjoyed it. It’s a book that’s basically about the following problem: there was a time in which humans had not yet built accurate thermometers, and therefore weren’t able to scientifically investigate the phenomenon of temperature, which would require measuring it. But to build a thermometer and know you’ve done so correctly, it seems like you have to know that its temperature readings match the real temperature, which seemingly requires either other known-functional thermometers to calibrate (which they did not have), or a rigorous enough scientific understanding of temperature to know that your thermometer tracks it well (which is hard to obtain without having thermometers)—so it’s not obvious how one could go from a situation where thermometers didn’t exist to one where they do exist, and where we are justified in believing that they accurately measure temperature.
This book has had some popularity in the rationality community as an account of applied epistemology, and in particular, for its description of how to measure something intangible. An obvious application of the book (which I won’t elaborate much on except in a footnote1) is in understanding artificial intelligence: there are various properties like the ‘capability’ or ‘alignment’ of AI models (or perhaps of models+scaffolds, or perhaps of ecosystems of models) which we would like to understand but for which we do not have good measures of, and it’s not straightforward to know how we can validate our measures. I had purchased it in November 2024, and was very slowly making my way thru it, until I joined METR (an organization for which these questions are especially salient) and ran an Inventing Temperature Book Club, thereby forcing myself to read it.
Overall, I enjoyed the book, and would add my voice to the chorus of those recommending it to all those who want to know how to know things, as well as those with interest in the study of thermodynamics2. Firstly, the discussion of the phenomenon of temperature and the history of its study was interesting in and of itself—I was startled to learn that, for example, even at a fixed atmospheric pressure water does not boil at a consistent temperature, or that beams of cold can be reflected in mirrors and sent places to cool things down, seemingly contra our modern understanding of cold as the mere absence of heat.
Secondly, however, the book stimulated a good deal of thought in me about its chosen philosophical topic: how one can come to measure the previously unmeasured. I read the book as offering the following account: what justifies our measurements of temperature is their coherence. When we want to start measuring temperature, or extend our measurements into new regimes that require new instruments (e.g. the temperature of pottery kilns, where typical thermometers break), we should come up with a few different ways of trying to get at the same thing, and believe methods which all agree. The overall picture is a victory of coherentism against foundationalism: foundationalism being the theory that there are certain beliefs that we are justified in holding in and of themselves, without any other justifications (akin to how a Bayesian might think about the choice of prior), and coherentism being the theory that our beliefs are justified by their coherence with each other. Some examples of this playing out (very much abbreviated, for more detail I strongly recommend reading the book):
To determine that the temperature of just-boiled water vapour is constant, we come up with a crude ‘ordinal thermometer’3 that’s like a typical mercury thermometer, but doesn’t have degree markings. We then boil some water, put the ordinal thermometer in the vapour, mark the point the liquid gets to, and then repeat. If it comes to the same line, that’s some reason to think the temperature of boiled water-vapour is constant, and having some theory that justifies it is even more reason. These ordinal thermometers themselves are justified by their coherence with our senses of temperature when we touch things.
A basic type of thermometer is to put some liquid in a thin tube, and see how much it expands in various settings. In particular, you see where it comes up to at the freezing point of water, mark that 0 degrees, then you see where it comes up to at the temperature of water vapour, mark that 100 degrees, and then evenly mark the degrees in the middle. The problem is that if you do this, different substances will have different temperatures at which they hit 50 degrees. How do you decide which substance is measuring temperature correctly? Make a bunch of thermometers with that substance and check if they agree with each other - this picks a winner, that we then presume is measuring the actual temperature.
To figure out the temperature of things that are too hot to use standard thermometers, you come up with multiple methods of measuring temperature that seem justified on your existing tentative theories of temperature. It will turn out that most of them basically agree, and perhaps one will disagree. At this point, you’re justified in thinking that the methods that agree are measuring temperature, and the one that disagrees is broken, because of the coherence of these methods.
That said, I would describe what’s going on in these cases in a different way than the author does, which I’d like to lay out below.4
As humans, we have this gradated sense of ‘hot’ and ‘cold’, where ice feels cold, fire feels hot, Berkeley in the spring feels somewhere in the middle, and when you take a freshly-baked cake out of the oven, the pan feels hotter than the cake. We also notice some relationships between this sense and physical phenomena: for example, putting something in a fire seems to make it hotter, when you put something in snow it gets colder, when you make ice hotter it melts, and different times of year are hotter or colder depending on how long the sun is in the sky.
There are a variety of physical causes that are upstream of each one of these phenomena. However, their coincidence makes us suspect that there’s one cause that unites all of them. We therefore want to look for some unified cause that has a robust and simple5 relationship to as many phenomena that seem heat-related as possible, and once we find it we will call it ‘temperature’. This is why we look for the coherence of various different measurement techniques and theories: not because coherence of beliefs about temperature is inherently justifying, but because this coherence indicates that there is one thing being measured and that that thing deserves the name ‘temperature’.
I think there are a few upshots of this way of thinking:
The word ‘temperature’ doesn’t necessarily have some pre-existing fixed reference. Instead, there are a variety of properties that could deserve the name, and our job is to pick between them.
That said, the process is not merely of arbitrarily picking a thing to give a name to: it involves learning about the world and which things have a robust relationship to which other things.
There might not be a single phenomenon of ‘temperature’ that underlies all of our phenomena, and this might cause us to think of some of them as not ‘actually tracking temperature’: for instance, according to our modern understanding, when you bake a cake and take it fresh out of the oven, the cake is just as hot as the pan, it’s just that the pan is more easily able to heat your finger up when you touch it than the cake is.
Conceivably, it might have been the case that there were two equally-real concepts that each caused many of these phenomena, or perhaps no precise concept at all.
I think the generalization is something like this: when we see a relationship between a bunch of things, we might propose some latent cause that is some sort of scalar property (especially when the relationship is between a bunch of scalar properties, like the volumes of liquids/gasses, or how hot something feels). We then want to try to find such a latent cause by coming up with a variety of measures. Those measures that agree with each other, especially when the measures themselves are not by design identical, must be getting at a ‘more real’ property that has more relationships with other things, that is a prime candidate for an object of interest in our theory.6 This improves our sense of what latent causes can exist, and how they can relate to each other. Notably, this differs from an approach that theorizes a latent cause, gives that cause a name, and tries to ‘locate’ that cause (for example, consider thinking that some things are ‘conscious’ and trying to figure out what property counts as ‘consciousness’ so that we can measure the ‘consciousness’ of unknown examples—instead, this looks more like looking at conscious and non-conscious phenomena, finding common factors that have causal relationships with the phenomena of interest, and coming up with a theory and good measures of those factors, whether or not any of them ends up being best thought of as ‘consciousness’).
The overall view is that there are a variety of properties of nature that we could talk about, but some are ‘more real’ than others: they causally interact with more other things in more simple ways. Our job is to locate these real ones, and understand their relationships. Not everything we observe might have a single ‘real’ cause, but the cards are somewhat stacked in our favour: ‘real’ phenomena tend to affect lots of different other phenomena in simple ways, while ‘fake’ ones tend to have few downstream effects, so a ‘real’ phenomenon is more likely to cause any given effect of interest than a ‘fake’ phenomenon. That said, unfortunately this only gives you a likelihood ratio, and more reasoning is needed to figure out how likely we are to correctly stumble upon a ‘real’ phenomenon in the wild—for instance, if there are tons of ‘fake’ phenomena but very few ‘real’ phenomena then things we observe would be more likely to be caused by ‘fake’ phenomena, whereas if ‘real’ phenomena were plentiful then it would be even easier to stumble across them.
Unfortunately, measuring (for example) AI capabilities seems somewhat more conceptually fraught than measuring temperature: your measure of AI capability will depend somewhat on your distribution of tasks of interest (if you want to compare the capabilities of e.g. two models, one of which is better at Python coding and one of which is better at Latin-to-English translation), in a way that makes it hard to imagine that it can be boiled down to a single real number in the way that temperature can (altho of course temperature is not exactly a single number, since it can be measured in different scales). It is also not exactly clear what the thing to be measured is, as alluded to in the main text: whether it should be neural networks, neural networks plus ‘scaffolds’ used to get useful work out of them, or something else entirely. An additional interesting consideration is that capability measures of AI systems inherently have to be paired with difficulty measures of tasks, for ‘capability’ to have any cogent relationship with what AI systems can actually do, in a way that I think has no close analogy with temperature. ↩
The book uses the word ‘thermoscope’ for this, but I think ‘ordinal thermometer’ is more descriptive and immediately intelligible. ↩
I initally conceived of this as a disagreement with the author, but at the book club at least some people seemed to think it was compatible with the book, so I will remain neutral on the question of whether or not I agree, and focus on the exposition of my own view. ↩
The ‘robust and simple’ proviso is meant to distinguish temperature from any arbitary function of temperature. For example, absolute temperature to the 2.7th power, which is related to all the same other phenomena but in a less simple manner, or the function that is exactly the absolute temperature in Kelvin if that temperature is less than 68 degrees, and is otherwise the absolute temperature in Kelvin plus 38 degrees, whose relationship with other phenomena is not robust around the discontinuity it has. ↩
Claude Opus 4.5, when reviewing this post, suggests that there could be other causes of measurement agreement, the most significant being measurements that track properties that are distinct but correlate in observable ranges. As a result, this agreement should really be only taken as evidence of a ‘more real’ property, rather than strict proof, evidence that is stronger the more the measurement instruments differ in their design and the wider the range of situations in which they agree. ↩
I’ve recently read the book Inventing Temperature, and very much enjoyed it. It’s a book that’s basically about the following problem: there was a time in which humans had not yet built accurate thermometers, and therefore weren’t able to scientifically investigate the phenomenon of temperature, which would require measuring it. But to build a thermometer and know you’ve done so correctly, it seems like you have to know that its temperature readings match the real temperature, which seemingly requires either other known-functional thermometers to calibrate (which they did not have), or a rigorous enough scientific understanding of temperature to know that your thermometer tracks it well (which is hard to obtain without having thermometers)—so it’s not obvious how one could go from a situation where thermometers didn’t exist to one where they do exist, and where we are justified in believing that they accurately measure temperature.
This book has had some popularity in the rationality community as an account of applied epistemology, and in particular, for its description of how to measure something intangible. An obvious application of the book (which I won’t elaborate much on except in a footnote1) is in understanding artificial intelligence: there are various properties like the ‘capability’ or ‘alignment’ of AI models (or perhaps of models+scaffolds, or perhaps of ecosystems of models) which we would like to understand but for which we do not have good measures of, and it’s not straightforward to know how we can validate our measures. I had purchased it in November 2024, and was very slowly making my way thru it, until I joined METR (an organization for which these questions are especially salient) and ran an Inventing Temperature Book Club, thereby forcing myself to read it.
Overall, I enjoyed the book, and would add my voice to the chorus of those recommending it to all those who want to know how to know things, as well as those with interest in the study of thermodynamics2. Firstly, the discussion of the phenomenon of temperature and the history of its study was interesting in and of itself—I was startled to learn that, for example, even at a fixed atmospheric pressure water does not boil at a consistent temperature, or that beams of cold can be reflected in mirrors and sent places to cool things down, seemingly contra our modern understanding of cold as the mere absence of heat.
Secondly, however, the book stimulated a good deal of thought in me about its chosen philosophical topic: how one can come to measure the previously unmeasured. I read the book as offering the following account: what justifies our measurements of temperature is their coherence. When we want to start measuring temperature, or extend our measurements into new regimes that require new instruments (e.g. the temperature of pottery kilns, where typical thermometers break), we should come up with a few different ways of trying to get at the same thing, and believe methods which all agree. The overall picture is a victory of coherentism against foundationalism: foundationalism being the theory that there are certain beliefs that we are justified in holding in and of themselves, without any other justifications (akin to how a Bayesian might think about the choice of prior), and coherentism being the theory that our beliefs are justified by their coherence with each other. Some examples of this playing out (very much abbreviated, for more detail I strongly recommend reading the book):
That said, I would describe what’s going on in these cases in a different way than the author does, which I’d like to lay out below.4
As humans, we have this gradated sense of ‘hot’ and ‘cold’, where ice feels cold, fire feels hot, Berkeley in the spring feels somewhere in the middle, and when you take a freshly-baked cake out of the oven, the pan feels hotter than the cake. We also notice some relationships between this sense and physical phenomena: for example, putting something in a fire seems to make it hotter, when you put something in snow it gets colder, when you make ice hotter it melts, and different times of year are hotter or colder depending on how long the sun is in the sky.
There are a variety of physical causes that are upstream of each one of these phenomena. However, their coincidence makes us suspect that there’s one cause that unites all of them. We therefore want to look for some unified cause that has a robust and simple5 relationship to as many phenomena that seem heat-related as possible, and once we find it we will call it ‘temperature’. This is why we look for the coherence of various different measurement techniques and theories: not because coherence of beliefs about temperature is inherently justifying, but because this coherence indicates that there is one thing being measured and that that thing deserves the name ‘temperature’.
I think there are a few upshots of this way of thinking:
I think the generalization is something like this: when we see a relationship between a bunch of things, we might propose some latent cause that is some sort of scalar property (especially when the relationship is between a bunch of scalar properties, like the volumes of liquids/gasses, or how hot something feels). We then want to try to find such a latent cause by coming up with a variety of measures. Those measures that agree with each other, especially when the measures themselves are not by design identical, must be getting at a ‘more real’ property that has more relationships with other things, that is a prime candidate for an object of interest in our theory.6 This improves our sense of what latent causes can exist, and how they can relate to each other. Notably, this differs from an approach that theorizes a latent cause, gives that cause a name, and tries to ‘locate’ that cause (for example, consider thinking that some things are ‘conscious’ and trying to figure out what property counts as ‘consciousness’ so that we can measure the ‘consciousness’ of unknown examples—instead, this looks more like looking at conscious and non-conscious phenomena, finding common factors that have causal relationships with the phenomena of interest, and coming up with a theory and good measures of those factors, whether or not any of them ends up being best thought of as ‘consciousness’).
The overall view is that there are a variety of properties of nature that we could talk about, but some are ‘more real’ than others: they causally interact with more other things in more simple ways. Our job is to locate these real ones, and understand their relationships. Not everything we observe might have a single ‘real’ cause, but the cards are somewhat stacked in our favour: ‘real’ phenomena tend to affect lots of different other phenomena in simple ways, while ‘fake’ ones tend to have few downstream effects, so a ‘real’ phenomenon is more likely to cause any given effect of interest than a ‘fake’ phenomenon. That said, unfortunately this only gives you a likelihood ratio, and more reasoning is needed to figure out how likely we are to correctly stumble upon a ‘real’ phenomenon in the wild—for instance, if there are tons of ‘fake’ phenomena but very few ‘real’ phenomena then things we observe would be more likely to be caused by ‘fake’ phenomena, whereas if ‘real’ phenomena were plentiful then it would be even easier to stumble across them.
Unfortunately, measuring (for example) AI capabilities seems somewhat more conceptually fraught than measuring temperature: your measure of AI capability will depend somewhat on your distribution of tasks of interest (if you want to compare the capabilities of e.g. two models, one of which is better at Python coding and one of which is better at Latin-to-English translation), in a way that makes it hard to imagine that it can be boiled down to a single real number in the way that temperature can (altho of course temperature is not exactly a single number, since it can be measured in different scales). It is also not exactly clear what the thing to be measured is, as alluded to in the main text: whether it should be neural networks, neural networks plus ‘scaffolds’ used to get useful work out of them, or something else entirely. An additional interesting consideration is that capability measures of AI systems inherently have to be paired with difficulty measures of tasks, for ‘capability’ to have any cogent relationship with what AI systems can actually do, in a way that I think has no close analogy with temperature. ↩
Which also has deep ties to epistemology, altho I digress. ↩
The book uses the word ‘thermoscope’ for this, but I think ‘ordinal thermometer’ is more descriptive and immediately intelligible. ↩
I initally conceived of this as a disagreement with the author, but at the book club at least some people seemed to think it was compatible with the book, so I will remain neutral on the question of whether or not I agree, and focus on the exposition of my own view. ↩
The ‘robust and simple’ proviso is meant to distinguish temperature from any arbitary function of temperature. For example, absolute temperature to the 2.7th power, which is related to all the same other phenomena but in a less simple manner, or the function that is exactly the absolute temperature in Kelvin if that temperature is less than 68 degrees, and is otherwise the absolute temperature in Kelvin plus 38 degrees, whose relationship with other phenomena is not robust around the discontinuity it has. ↩
Claude Opus 4.5, when reviewing this post, suggests that there could be other causes of measurement agreement, the most significant being measurements that track properties that are distinct but correlate in observable ranges. As a result, this agreement should really be only taken as evidence of a ‘more real’ property, rather than strict proof, evidence that is stronger the more the measurement instruments differ in their design and the wider the range of situations in which they agree. ↩