The point is to refine the wrong question about certainties 2 into a better question about probabilistic knowledge 2'. If you just want to get an answer to 2 - then this answer is 'no'. We can't be certain that our knowledge is true. Then again, neither we need to.

If your question is how it's different from cyclical reasoning then consider the difference:

Evolution produced my brain that discovered evolution. After I've considered this thought I'm now certain in both of them and so case closed. No more arguments or evidence can ever persuade me otherwise.

I can never be certain of anything but to the best of my knowledge the situation looks exactly the way it would looked if my brain was produced by evolution which allowed my brain to discover evolution. I'll be on lookout for counter-evidence, because maybe everything I know is a lie, but for now on a full reflection of my knowledge, including techniques of rationality and notions of simplicity and computational complexity, this seems to be the most plausible hypothesis.

I'll be talking more about Münchhausen trilemma in details in a separate post. But here are some highlights:

"from the inside" is what we care about here

There are two things that we care about

1. We have a map representing the territory to a good degree
2. We are justifiably certain in 1.

If our brains are created by natural selection and then they discover natural selection we have a straightforward non-paradoxical causal history for how our maps correlate to the territory:

Natural Selection -> Brain -> Natural Selection*

Where A* means map of A.

But that's an assumption. Doesn't lead you to know things.

In the map it is an assumption. But in the territory it's either true or false and it doesn't matter whether we've assumed it or not. 

A map can correspond to the territory even if noone knows about it. We can have 1. without 2.

you knowledge is in the map too

You can map the map-territory correspondence as well. As soon as we considered it we are getting 

(Natural Selection -> Brain -> Natural Selection*)*

This allows us to get a map of map-territory correspondence and then a map of map-territory correspondence of a map of  map-territory correspondence and so on.

So, even though we do not have 2. and, in fact we can not have 2. in principle, as any reason for certainty will have to go through our mind. 

We can have

      2'. We are somewhat justifiably somewhat confident in 1

and

      3'. We are somewhat justifiably somewhat confident in 2

and so on as long as we have enough data to construct a meta-model of the next level.

As a warming up exercise, try the tree story with something actually contentious

It's a fine exercise for beginners, but I hope we are long past it, at this point.

All the "contentiousness" evaporates as soon as we've fixed the definitions and got rid of the semantic confusion.

Note the lack of agreement about what empirical evidence even is, and about how science works. 

Granted. This disagreement is resolved the same way still. You gather evidence about interpreting evidence. You reflect on it with your mind. And so on.

Probably not very, but philosophy isn't stuck there, since it has plenty of highly technical debates.

The "technicality" of the debates is really not the issue here.

The evolutionary argument guarantees that you can know just enough for your survival, and only that, things like which berries to eat and animals to avoid being eaten by. ( There's no evidence at all that it's "optimal")

Are you not familiar with the notion of optimization process? When you are a result of successful replication of imperfect replicators in a competitive environment with limited resources there is quite a lot of evidence for some kind of "optimality".

Typical philosophical problems, about ontology and epistemology, the real nature of things, have no relevance to survival, so the evolutionary argument doesn't tell you they are soluble. 

If the existed in some kind of separate magisterium where our common knowledge wouldn't be applicable than yes. Is it your stance?

Some things are visible, and subject to direct feedback, other things arent.

So you use indirect feedback building on top of knowledge of things that are directly visible. I'm rather sure you understand how it works

To shown that empiricism works, you need to show that empiricism*alone" works, and it works for everything , including the tricky edge cases. And you can't infer all that from the fact that it works in one , simple case.

I can keep applying it to the "tricky cases" and see how things work from there. And this way I can aggregate more and more evidence and so on and so forth. Never reaching absolute certainty but always refining my tools.

models don't just drop out of the data.

They kind of do, in a manner of speaking. There are several ways to aggregate data in a model. We can try multiple of them and see how these models predict new data, therefore collecting evidence about what kind of models are good at data prediction in general and so refining our tools of model construction. 

Being already selected for intuitions related to surviving in the world and not starting from scratch helps quite a lot.

Conjecture is a conscious, creative and cognitive process, not something that happens automatically as part of perception.

Okay I think I understand what is going on here. Are you under impression that I'm trying to bring back the old empiricism vs rationalism debate, arguing on the side of empiricism?

If so, I want to stop you here, as this couldn't be further from the truth. I believe this whole debate was quite embarrassing and the whole distinction on pure observation and pure cognition doesn't make sense in the first place. Observation is cognition, cognition is observation. You can skip all the obvious points how one actually needs a brain to make observations - I've explicitly mentioned it myself in the post.

There are direct quantifiable tests for predictive accuracy , but no way  of directly comparing a map to the territory. Scientific realists hope and assume that empirical adequacy adds up to ontological correctness .. but it isn't necessarily so.

I'll talk about it in a future post.

One way of making this point is that ontologically wrong theories can be very accurate.

Yes, that's totally fine. I don't think we have any disagreement here. 

Our inability to make direct comparisons between map and territory extends to an inability to tell how close we are to the ultimately accurate ontology,. Even probablistic reasoning can't tell us how likely our theories are in absolute terms. We only know that better theories are more probably correct than worse ones, but we don't really know whether current theories are 90% correct or 10% correct, from a God's eye point of view.

[Half joking] 

Thankfully, there there doesn't seem to be any God, so we might as well not care about his point of view too much. 

[/half joking]

Yes, it's all probabilities all the way down, without perfect certainty. This is fine. We can come up with adversarial examples where it means that we were completely duped, and our views are completely disentangled from "true reality" and were simply describing an "illusion", but

1. Why would that be likely?
2. Renaming "reality" to "illusion" doesn't actually change anything of substance.

Examples, please.

There will be plenty in the future posts. But generally, consider the fact that philosophy reasons in all direction and normality is only a relatively small space of all possible destinations.

Have you solved philosophy? Has anybody?

Well, I've went further than many. But it's not relevant to the point I'm making here.

I also thought the robot:s answer missed the point quite badly ...because it reduced the ought all the way down to an is -- or rather a bunch of isses.

If you dismiss any reduction of ought to is, you are essentially dogmatically certain that Hume's guillotine is true. Is it your stance?

If what one ought to do reduces to what one would do

Not to what one would. Your ethical module may not be directly connected to the behavioral one and so your decisions are based on other considerations, like desires unrelated to ethics. This doesn't change the fact that what you ought to do is the output (or a certain generalization of multiple outputs) of the ethical module, which is a computation taking place in the real world, which can be observed.

there are potentially eight billion answers to what one ought to do.

Potentially but not actually. Once again, when you look, turns out individual ethical views of people are not *that* different. That said, there is still room for disagreement and how exactly we aggregate individual ethical preferences into morality is still up to debate. But this is the next question, with the somewhat similar direction for a solution.

There's a consistent theme in rationalist writing on ethics, where the idea that everyone has basically the same values , or "brain algorithms", is just assumed ... but it needs to be based on evidence as much as anything else. 

Not basically the same, but somewhat similar. And it's not just assumed, it's quite observable. Human ethical disagreements are mostly about edge cases. Like what is your objective claim here, that human values are not correlated at all?

Reducing ethical normativity isn't bad, but doing it in a way that leads to sweeping subjetivism is bad. If you accept subjectivism, you miss better answers.

I think calling it subjectivism is very misleading. The whole subjective/objective duality is quite horrible - I'll be dedicating a post about it at some point. It's social constructivism of morality. Which is rooted in our other knowledge about game theory and evolution.

In the first comic, the engineers answer is a also a plausible philosophical answer.

Yes, this is exactly my point. A lot of things, which are treated as "applied missing the point answers" are in fact legitimately philosophically potent. At the very least, we should be paying much more attention to them.

In the third comic, the philosopher is technically correct. You can't achieve certainty from a finite.chain of observations, and you can only make a finite chain. Modern empiricists have admitted this giving up on certainty.

Therefore it's not just "by looking" but "pretty much by looking". I completely agree about the necessity to abandon the notion of certainty. If you want to give philosophers some credit for this - I agree. The irony of the joke stays the same. When the question is refined so that we removed the problematic notion of "certainty", the naive answer turned out to be basically true.

You talk about the philosophers not having much to add in the third comic, and the scientist getting it right.

That's not exactly the case. I'm trying to make a more nuanced point than "boo philosophy and hooray science". 

I'm saying that the applied answer is much deeper than it initially appears. But to discover this deepness you still need to go on the philosophical journey. Most people who dismiss philosophical questions don't in fact understand the deepness of the "scientific answer" because they didn't go on this journey.

Seems to me like the engineer's/robot's answer in the first two comics are importantly misguided/non-helpful though.

I think they are similarly helpful and deep, highlighting some blindspots in conventional discussions on the matter.

The more sophisticated version of the first question would be something about whether you ought to care about copies of yourself

That would be a different question, though. The question of value, not ontology. 

The ontological question is about the way to solve some probability theoretic questions like anthropic trilemma. And the insight is that causality, the physical nature of the particular experiment matters quite a lot, not whether people have "similar atoms". I'll be writing about it in details at some point in my PTF102 sequence.

In the second comic, the robot's answer is fine as far as predictive accuracy goes.

Robot answer's highlight the way to reduce the concept of "shouldness". How it's a property of mind's decision making algorithm and where you need to look to find an answer, which is one of the core insights of metaethics.

Science, and more specifically physics, is built on first theorizing or philosophizing, coming up with a lot of potential worlds a priori, and only looking to see which one you probably fall in after the philosophizing is done.

This is a mainstream position, which is sort of, kind of true in broad strokes, but misses very important context, therefore leaving people confused about all the interesting nuance, preventing further progress. So I push back against it.

The important context/interesting nuances are:

1. There is no pure reasoning, disentangled from the reality you live in. Even if you tried to ignore all your personal experience while trying to come up with a theory, reasoning ability of your brain is already a result of optimization process inside the reality, encompassing generations of life experience in it.
2. People do not actually try to ignore their experience while coming up with the theories. They generalize and formalize this experience. Presenting it as if they first came up with general principles which were then validated by the experience for the sake of readers comprehension, even though the casual process that led to her discovery of the theory is different.
3. These principles are not left unchanged forever once they were formalized. We go back and retcon them when needed based on more experience we accumulate. It's not first reason then look, it's keep looking and reasoning to the best of your abilities at the same time to the point where the whole distinction between looking and reasoning is artificial in the first place.

I bet a good philosopher from a different universe could come up with the concept of trees

Crucially, the other universe has to be similar enough to our universe, so that reasoning that evolved in the other universe applies to ours as well.

I think my issue with empiricism is that it does not generalize, at all.

Have you heard about unsupervised learning?

The loans add an extra complication it's an extra variable that affects the economical equilibrium. But this effect is the same for both landlordism and capitalism, so it's irrelevant for the question of difference between the two.

Meanwhile there is a factor that distinguishes landlordism and capitalism: the sufficiently intense competition to provide good/services at lower cost that doesn't exist for landlordism, but does exist for capitalists even before we account for loans.

Essentially we have:

Capitalist Competiton Pressure = NL + L

Landlord Competiton Pressure =  L

Where NL is the aspect of pressure without the loans and L is aspect of pressure specifically due to loans.

You think landlords are special because land is limited. But capital as a whole is limited at any given time.

So? Don't you think that being limited at some point in time and being limited in principle is a meaningful distinction?

Loans/investments is a red herring here. Have you read Meditations on Moloch? Consider the example from there:

Suppose there’s a coffee plantation somewhere in Ethiopia that employs Ethiopians to grow coffee beans that get sold to the United States. Maybe it’s locked in a life-and-death struggle with other coffee plantations and want to throw as many values under the bus as it can to pick up a slight advantage.

But it can’t sacrifice quality of coffee produced too much, or else the Americans won’t buy it. And it can’t sacrifice wages or working conditions too much, or else the Ethiopians won’t work there. And in fact, part of its competition-optimization process is finding the best ways to attract workers and customers that it can, as long as it doesn’t cost them too much money. So this is very promising.

But it’s important to remember exactly how fragile this beneficial equilibrium is.

Suppose the coffee plantations discover a toxic pesticide that will increase their yield but make their customers sick. But their customers don’t know about the pesticide, and the government hasn’t caught up to regulating it yet. Now there’s a tiny uncoupling between “selling to Americans” and “satisfying Americans’ values”, and so of course Americans’ values get thrown under the bus.

Or suppose that there’s a baby boom in Ethiopia and suddenly there are five workers competing for each job. Now the company can afford to lower wages and implement cruel working conditions down to whatever the physical limits are. As soon as there’s an uncoupling between “getting Ethiopians to work here” and “satisfying Ethiopian values”, it doesn’t look too good for Ethiopian values either.

Or suppose someone invents a robot that can pick coffee better and cheaper than a human. The company fires all its laborers and throws them onto the street to die. As soon as the utility of the Ethiopians is no longer necessary for profit, all pressure to maintain it disappears.

Or suppose that there is some important value that is neither a value of the employees or the customers. Maybe the coffee plantations are on the habitat of a rare tropical bird that environmentalist groups want to protect. Maybe they’re on the ancestral burial ground of a tribe different from the one the plantation is employing, and they want it respected in some way. Maybe coffee growing contributes to global warming somehow. As long as it’s not a value that will prevent the average American from buying from them or the average Ethiopian from working for them, under the bus it goes.

I know that “capitalists sometimes do bad things” isn’t exactly an original talking point. But I do want to stress how it’s not equivalent to “capitalists are greedy”. I mean, sometimes they are greedy. But other times they’re just in a sufficiently intense competition where anyone who doesn’t do it will be outcompeted and replaced by people who do. Business practices are set by Moloch, no one else has any choice in the matter.

This sufficiently intense competition that pushes everyone to the worst practices just doesn't exist for landlords.

But that amount of effort would be sufficient to elevate most women who have similar ages, similar BMIs, and a face in the top quartile.

That's just trivially true, isn't it? Among women who were already pre-selected to have similar faces, ages and BMI's to movie starts most of them can be made extremely attractive, with the help of right makeup, clothes, context and so on.

But this is not how people talk about female sexual attractiveness! People, especially women, talk about it as if there's a significant, native difference in supermodels' broad appeal.

The difference is that some people happen to be part of this group of women with similar faces, ages and BMI's to movie starts and some do not. There is no contradiction here.

She's an outlier, but an outlier in the sense that someone who's 6'2" is an outlier, not an outlier like Michael Jordan is an outlier.

I'd say, more of an outlier than being 6'2, less of an outlier than Michael Jordan.

The reason women talk like this

Basically, because society as a whole actively conditions women that their looks is the most important thing about them. This includes some of the factors that you've mentioned.

Yes, but such an interpretation falls outside of Yudkowsky's view, as I understand it

Maybe! But I would expect him to change his view to something like this in case you managed to persuade him that there is some crucial flaw in Bayesianism. While your goal seems to be to propagate the toolbox-view as the only valid approach. So you might as well engage with a stronger version of law-view right now.

On Walker, on that paragraph he is criticizing the specific (common) practice of comparing separate Bayesian models and picking the best (via ratios or errors or some such) when there is uncertainty about the truth instead of appropriately representing this uncertainty about your sampling model in the prior. 

Rolling a die is a bit of a nifty example here since it's the case where you assign a separate probability to each label in the sample space, so that your likelihood is in fact fully general, which is where the idea for a Dirichlet prior comes from in an attempt to generalize this notion of covering all possible models for less trivial problems.

So, suppose that instead of assigning equal probabilities to each label of a die, I consider this as just one of multiple possible models from a set of models with different priors. According to one of them:

P(1) = 1/2, P(2) = P(3) = P(4)=P(5)=P(6)=1/10

According to another:

P(2) = 1/2, P(1)=P(3)=P(4)=P(5)=P(6)=1/10

And so on and so forth.

And then I assign equiprobable prior between these models and start collecting experimental data - see how well all of them perform. Do I understand correctly, that Walker considers such approach incoherent?

In which case, I respectfully disagree with him. While it's true that this approach doesn't represent our uncertainty about which label of an unknown die will be shown on a roll, it, nevertheless, represents the uncertainty about which bayesian model best approximates the behavior of this particular die. And there is nothing incoherent in modelling the latter kind of uncertainty instead of the former.

And likewise for more complicated settings and models. Whenever we have uncertainty about which model is the best one we can model this uncertainty and get a probabilistic answer to it via bayesian methods. And then get a probabilistic answer according to this model, if we want to.

On Bayesian finite-sample miscalibration, simply pick a prior which is sufficiently far off from the true value and your predictive intervals will be very bad for a long time

But why would the prior, capturing all your information about a setting, be sufficiently far off from the true value, in the first place? This seems to happen mostly when you misuse the bayesian method, by picking some arbitrary prior for no particular reason. Which is a weird complain. Surely we can also misuse frequentist methods in a similar fashion - p-hacking immediately comes to mind, or just ignoring bunch of data points altogether. But what's the point in talking about this? We are interested in situations when the art fails us, not when we fail the art, aren't we?

On minimax

Interesting! So is there a agreement among frequentists that probability of an unfair coin about which we know nothing else to land Tails is 1/2? Or is it more like: "Well we have a bunch of tools and here one of them says 1/2, but we do not have a principled reason to prefer it to other tools regarding the question of what probability is, so the question is still open".

On your last comment, it seems like a bit of an open question to attribute the existence of practical intuition and reasoning about mathematical constructs like this to a Bayesian prior updating process.

Is it? I though everyone is in agreement that Bayes theorem naturally follows from the axioms of probability theory. In which case the only reason why such reasoning doesn't follow Bayesian updating procedure is that, somehow, probability theory is not applicable to the reasoning about mathematical constructs in particular, but why would that be true?

Certainly I reason, and I change my mind, but to me personally I see no reason to imagine this was Bayesian in some way (or that those thoughts were expressed in credence-probabilities which I shifted by conditioning on a type of sense-data), nor that I would be ideally doing this instead.

Oh wait, you don't think that probability theory is applicable to reasoning in general? Surely I'm misunderstanding you here? Could you elaborate on your position here? I feel that this is the most important crux of disagreement.