In the previous post, we saw some different types of claims that we might want software systems to automatically evaluate. Ideally, we'd like our software systems to incorporate accurate and useful information, and disregard inaccurate or irrelevant information.

In this post, I'll describe some factors that can help our systems to build accurate models of the world, using information from their own sensors and other systems, even when those systems were designed for different purposes using different ways to model the world, by different designers with different ways they would like the world to be.

Translating Between Ontologies

For software systems, all factual claims start out in an ontology that looks like "this is the data that came in from my sensor channels." For a thermostat, that looks like "this is the signal I received on the channel that my thermometer should be hooked up to." An adversary might have chopped off the thermometer, and be feeding my thermostat false signals, but they are feeding those false signals into channels on my thermostat where the thermometer signals are supposed to go. It is a further and separate factual claim that "all my sensors are calibrated and configured correctly, and are feeding signals to their designated channels as expected."

For many systems, like household thermostats, it's fine to take sensor data at face value. But other times we'll want our systems to have ways of handling the possibility that they're receiving inaccurate information, or even being actively misled by an adversary. When it's clear from context, I'll drop the qualifier that a software system only receives alleged sensor data on their corresponding channels. But a system can honestly report "this came in on the channel designated for my sensor" without further claiming that this information came from a sensor that is working properly and not being misled.

Once we have sensor data, we can apply an epistemology to translate from that sensor data ontology to one that better suits our needs. Like taking camera footage and using that to reconstruct a 3D model of a scene and the objects in it. We could also take that same camera footage and attempt to draw rectangles around the faces we recognize.

Suppose that we are designing a software system, where the most natural way to represent the world is in terms of 3D objects in a 3D scene. And we want it...

Distributed Strategic Epistemology

Distributed Strategic Epistemology