x

LESSWRONG
LW

Daniel Herrmann — LessWrong

Daniel Herrmann

Top postsTop post

Daniel Herrmann

Message

Daniel Herrmann is a decision theorist, formal epistemologist, and philosopher of AI. He is currently an assistant professor at UNC Chapel Hill.

He was a PIBBSS fellow. He runs the not-quite-active philosophy blog The Crow's Nest, where he writes for both a philosophical and a broad...

365

Ω

71

6

12

8y

Daniel Herrmann

Daniel Herrmann is a decision theorist, formal epistemologist, and philosopher of AI. He is currently an assistant professor at UNC Chapel Hill.

He was a PIBBSS fellow. He runs the not-quite-active philosophy blog The Crow's Nest, where he writes for both a philosophical and a broad...

Top postsTop post

Chance is in the Map, not the Territory

"There's a 70% chance of rain tomorrow," says the weather app on your phone. "There’s a 30% chance my flight will be delayed," posts a colleague on Slack. Scientific theories also include chances: “There’s a 50% chance of observing an electron with spin up,” or (less fundamental) “This is a fair die — the probability of it landing on 2 is one in six.” We constantly talk about chances and probabilities, treating them as features of the world that we can discover and disagree about. And it seems you can be objectively wrong about the chances. The probability of a fair die landing on 2 REALLY is one in six, it seems, even if everybody in the world thought otherwise. But what exactly are these things called “chances”? Readers on LessWrong are very familiar with the idea that many probabilities are best thought of as subjective degrees of belief. This idea comes from a few core people, including Bruno de Finetti. For de Finetti, probability was in the map, not the territory. But perhaps this doesn’t capture how we talk about chance. For example, our degrees of belief need not equal the chances, if we are uncertain about the chances. But then what are these chances themselves? If we are uncertain about the bias of a coin, or the true underlying distribution in some environment, then we can use our uncertainty over those chances to generate our subjective probabilities over what we’ll observe.[1] But then we have these other probabilities — chances, distributions, propensities, etc. — to which we are assigning probabilities. What are these things? Here we’ll show how we can keep everything useful about chance-based reasoning while dropping some problematic metaphysical assumptions. The key insight comes from work by, once again, de Finetti. De Finetti’s approach has been fleshed out in detail by Brian Skyrms. We’ll take a broadly Skyrmsian perspective here, in particular as given in his book Pragmatics and Empiricism. The core upshot is that we don't need to believe i

Still no Lie Detector for LLMs

Evolution and the Low Road to Nash

Subjective Naturalism in Decision Theory: Savage vs. Jeffrey–Bolker

Why Are The Human Sciences Hard? Two New Hypotheses

> "Reasoning about the relative hardness of sciences is itself hard." > —the B.A.D. philosophers Epistemic status: Conjecture. Under a suitable specification of the problem, we have credence ~50% on the disjunction of our hypotheses explaining >1% of the variance (e.g., in R2 values) between disciplines, and ≪1% on our...

Mar 18, 2025•40

Local Trust

In the last post, we developed a new way of thinking about deference: you defer to someone when you'd prefer to let them make decisions on your behalf. This helped solve some puzzles about modest experts—experts who express uncertainty about their own expertise. But even if you're skeptical of expert...

Feb 24, 2025•21

Subjective Naturalism in Decision Theory: Savage vs. Jeffrey–Bolker

Summary: This post outlines how a view we call subjective naturalism[1] poses challenges to classical Savage-style decision theory. Subjective naturalism requires (i) richness (the ability to represent all propositions the agent can entertain, including self-referential ones) and (ii) austerity (excluding events the agent deems impossible). It is one way of...

Feb 4, 2025•45

Deference and Decision-Making

You're talking with a leading AI researcher about timelines to AGI. After walking through various considerations, they tell you: "Looking at the current pace of development, I'd put the probability of AGI within 3 years at about 70%. But here's the thing—I know several really sharp researchers who think that...

Jan 27, 2025•28

Evolution and the Low Road to Nash

Solution concepts in game theory—like the Nash equilibrium and its refinements—are used in two key ways. Normatively, they proscribe how rational agents ought to behave. Descriptively, they propose how agents actually behave when interactions settle into equilibrium. The Nash equilibrium[1] underpins much of modern game theory and its applications in...

Jan 22, 2025•46

Chance is in the Map, not the Territory

"There's a 70% chance of rain tomorrow," says the weather app on your phone. "There’s a 30% chance my flight will be delayed," posts a colleague on Slack. Scientific theories also include chances: “There’s a 50% chance of observing an electron with spin up,” or (less fundamental) “This is a...

Jan 13, 2025•67

Still no Lie Detector for LLMs

Background This post is a short version of a paper we wrote that you can find here. You can read this post to get the core ideas. You can read the paper to go a little deeper. The paper is about probing decoder-only LLMs for their beliefs, using either unsupervised...

Jul 18, 2023•50

Load More (7/10)