I like this as a constrained example of why Knightian uncertainty is useful, but aren't the assumptions made for the devil vs. angel thought experiment also Knightian regions (and I would argue, the most important ones)? That is, if we already know that the agents are a superintelligent devil and a superintelligent angel, respectively, we can characterize our relationship to those entities because we already have a good model of their intelligence and intentions. What happens when that is not the case?
When we have Knightian uncertainty over what the agent ... (read more)
I like this as a constrained example of why Knightian uncertainty is useful, but aren't the assumptions made for the devil vs. angel thought experiment also Knightian regions (and I would argue, the most important ones)? That is, if we already know that the agents are a superintelligent devil and a superintelligent angel, respectively, we can characterize our relationship to those entities because we already have a good model of their intelligence and intentions. What happens when that is not the case?
When we have Knightian uncertainty over what the agent ... (read more)