The Problem of the Criterion

by G Gordon Worley III9 min read21st Jan 202138 comments

34

EpistemologySolomonoff InductionRationality
Frontpage

I keep finding cause to discuss the problem of the criterion, so I figured I'd try my hand at writing up a post explaining it. I don't have a great track record on writing clear explanations, but I'll do my best and include lots of links you can follow to make up for any inadequacy on my part.

Motivation

Before we get to the problem itself, let's talk about why it matters.

Let's say you want to know something. Doesn't really matter what. Maybe you just want to know something seemingly benign, like what is a sandwich?

At first this might seem pretty easy: you know a sandwich when you see it! But just to be sure you ask a bunch of people what they think a sandwich is and if particular things are sandwiches are not.

Uh oh...

from the internet

You've run headlong into the classic problem of how to carve up reality into categories and assign those categories to words. I'll skip over this part because it's been addressed to death already.

So now you've come out the other side accepting that "sandwich", like almost all categories, has nebulous boundaries, and that there is no true sandwich of which you can speak.

Fine, but being not one easily deterred, you come up with a very precise, that is a mathematically and physically precise, definition of a sandwich-like object you call an FPS, a Finely Precise Sandwich. Now you want to know whether or not something is an FPS.

You check the conditions and it all seems good. You have an FPS. But wait! How do you know each condition is true? Maybe one of your conditions is that the FPS is exactly 3 inches tall. How do you know that it's really 3 inches tall?

Oh, you used a ruler? How do you know the ruler is accurately measuring 3 inches? And furthermore, how do you know your eyes and brain can be trusted to read the ruler correctly to assess the height of the would-be FPS? For that matter, how do you even know what an inch is, and why was 3 inches the true height of an FPS anyway?

Heck, what does it even mean to "know" that something is "true"?

If you keep reducing like this, you'll eventually hit bottom and run into this question: how do you know that something is true? Now you've encountered the problem of the criterion!

The Problem

How do you know something is true? To know if something is true, you have some method, a criterion, by which you assess its veracity. But how do you know the criterion is itself true? Oh, you have a method, a criterion, by which you assess its veracity.

Oops! Infinite recursion detected!

The problem of the criterion is commonly attested to originate with Pyrrho. Roderick Chisholm, the modern philosopher who rejuvenated interest in the problem, often phrases it as a pair of questions:

  • What do we know? What is the extent of our knowledge?
  • How are we to decide whether we know? What are the criteria of knowledge?

This creates a kind of epistemic circularity where we go round in circles trying to justify our knowledge with our knowledge yet are never able to grab hold of something that itself need not be justified by something already known. If you like, it's the symbol grounding problem generalized (cf. SEP on metaphysical grounding).

To that point, the problem of the criterion is really more a category of related epistemological problems that take slightly different forms depending on how it's manifested. Some other problems that are just the problem of the criterion in disguise:

Really any kind of fundamental uncertainty about what is true, what is known, what can be trusted, etc. is likely a presentation of the problem of the criterion.

Okay, so we have this problem. What to do about it?

Possible Responses

First up, there are no known solutions to the problem of the criterion, and it appears, properly speaking, unsolvable. Ultimately all responses either beg the question or unask, dissolve, or reframe the question in whole or in part. Nonetheless, we can learn something from considering all the ways we might address it.

Chisholm argues that there are only three possible responses to the problem: particularism, methodism, and skepticism. I'd instead say there are only three ways to try to solve the problem of the criterion, and other responses are possible if we give up finding a proper solution. As mentioned, all these attempts at solutions ultimately beg the question and so none actually resolve the problem—hence why it's argued that the problem is unsolvable—but they are popular responses and deserve consideration to better understand why the problem of the criterion is so pernicious.

Particularism is an attempt to resolve the problem of the criterion by picking particular things and declaring them true, trusted, etc. by fiat. If you've ever dealt with axioms in a formal system, you've engaged in a particularist solution to the problem of the criterion. If you're familiar with Wittgenstein's notion of hinge propositions, that's an example of particularism. My impression is that this is widely considered the best solution since, although it leaves us with some number of unverified things we must trust on faith, in practice it works well enough by simply stopping the infinite regress of justifications at some point past which it doesn't matter (more on the pragmatism of when to stop shortly). To paraphrase Chisholm, the primary merit of particularism is that the problem of the criterion exists yet we know things anyway.

Methodism tries to solve the problem of the criterion by picking the criterion rather than some axioms or hinge propositions. Descartes is probably the best known epistemic methodist. The trouble, argues Chisholm and others, is that methodism collapses into particularism where the thing taken on faith is the criterion! Therefore we can just ignore methodism as a special case of particularism.

And then there's skepticism, arguably the only "correct" position of Chisholm's three in that it's the only one that seemingly doesn't require assuming something on faith. Spoiler alert: it does because it still needs some reason to prefer skepticism over the alternatives, thus it still ends up begging the question. Skepticism is also not very useful because even though it might not lead to incorrectly believing that a false thing is true, it does this by not allowing one to believe anything is true! It's also not clear that humans are capable of true skepticism since we clearly know things, and it seems that perhaps our brains are designed such that we can't help but know things, even if they are true but believed without properly grounded justification. So, alas, pure skepticism doesn't seem workable.

Despite Chisholm's claim to those being the only possible responses, some other responses exist that reject the premise of the problem in some way. Let's consider a few.

Coherentist responses reject the idea that truth, knowledge, etc. must be grounded and instead seek to find a way of balancing what is known with how it is known to form a self-consistent system. If you're familiar with the method of reflective equilibrium (SEP), that's an example of this. Arguably this is what modern science actually does, repeatedly gathering evidence and reconsidering the foundations to produce something like a self-consistent system of knowledge, though at the price of giving up (total) completeness (LW, SEP).

Another way of rejecting the premise is to give up the idea that the problem matters at all via epistemic relativism. In its strongest form, this gives up both any notion of objectivity or intersubjectivity and simply accepts that knowledge is totally subjective and ultimately unverifiable. In practice this is a kind of attractor position for folks burnt out on there-is-only-one-ontology scientism who overcorrect too far in the other direction, and although some of the arguments made for relativism are valid, complete rejection or even heavy devaluing of intersubjectivity makes this position essentially a solipsistic one and thus, like extreme skepticism, not useful for much.

Finally, we come to a special case of particularist responses known as pragmatism., and it's this kind of response that Yudkowsky offered. The idea of pragmatism is to say that there is some purpose to be served and by serving that purpose we can do an end-run around the problem of the criterion by tolerating unjustified knowledge so long as it works well enough to achieve some end. In Yudkowsky's case, that end is "winning". We might summarize his response as "do the best you can in order to win", where "win" here means something like "fulfil your purposes or concerns". I'd argue this is basically right and in practice what most people do, even if their best is often not very good and their idea of winning is fairly limited.

Yet, I find something lacking in pragmatic responses, and in particular in Yudkowsky's response, because they too easily turn from pragmatism to motivated stopping of epistemic reduction. If pragmatism becomes a way to sweep the problem of the criterion under the rug, then the lens has failed to see its own flaws. More is possible if we can step back and hold both the problem of the criterion and pragmatism about it simultaneously. I'll try to sketch out what that means.

Holding the Problem

At its heart, the problem of the criterion is a problem by virtue of being trapped by its own framing. That is, it's a problem because we want to know about the world and understand it and have that knowledge and understanding fit within some coherent ontological system. If we stopped trying to map and model the world or gave up on that model being true or predictive or otherwise useful and just let it be there would be no problem.

We're not going to do that, though, because then we'd be rocks. Instead we are optimization processes, and that requires optimizing for something. That something is what we care about, our telos, the things we are motivated to do, the stuff we want, the purposes we have, the things we value, and maybe even the something we have to protect. And instrumental to optimization is building a good enough map to successfully navigate through the territory to a world state where the something optimized for is in fact optimized.

So we're going to try to understand the world because that's the kind of beings we are. But the existence of the problem of the criterion suggests we've set ourselves an impossible task that we can never fully complete, and we are forced to endeavor in it because the only other option is ceasing to be in the world. Thus we seem inescapably trapped by the tension of wanting to know and not being able to know perfectly.

As best I can tell, the only way out is up, as in up to a higher frame of thinking that can hold the problem of the criterion rather than be held by it. That is, to be able to simultaneously acknowledge that the problem exists, accept that you must both address it and by virtue of addressing it your ontology cannot be both universal and everywhere coherent, and also leave yourself enough space between yourself and the map that you can look up from it and notice the territory just as it is.

That's why I think pragmatism falls short on its own: it's part of the response, but not the whole response. With only a partial response we'll continually find ourselves lost and confused when we need to serve a different purpose, when we change what we care about, or when we encounter a part of the world that can't be made to fit within our existing understanding. Instead, we need to deal with the fundamental uncertainty in our knowledge as best we can while not forgetting we're limited to doing our best while falling short of achieving perfection because we are bounded beings.

NB: Holding the problem of the criterion is hard.

Before we go further, I want to acknowledge that actually holding the problem of the criterion as I describe in this section is hard. It's hard because successfully doing what I describe is not a purely academic or intellectual pursuit: it requires going out into the world, actually doing things that require you to grapple with the problem of the criterion, often failing, and then learning from that failure. And even after all that there's a good chance you'll still make mistakes all the time and get confused about fundamental points that you understand in theory. I know I certainly do!

It's also not an all at once process to learn to hold the problem. You can find comments and posts of mine over the past few years showing a slowly building better understanding of how to respond to the problem of the criterion, so don't get too down on yourself if you read the above, aspire to hold the problem of the criterion in the way I describe, and yet find it nearly impossible. Be gentle and just keep trying.

And keep exploring! I'm not convinced I've presented some final, ultimate response to the problem, so I expect to learn more and have new things to say in time. What I present is just as far as I've gotten in wrangling with it.

Implications

Having reached the depths of the problem of the criterion and found a way to respond, let's consider some places where it touches on the projects of our lives.

A straightforward consequence of holding the problem of criterion and adopting pragmatism about it is that all knowledge becomes ultimately teleological knowledge. That is, there is always some purpose, motivation, or human concern behind what we know and how we know it because that's the mechanism we're using to fix the free variable in our ontology and choose among particular hinge propositions to assume, even and especially if those propositions are implicit. Said another way, no knowledge is better or worse without first choosing the purpose by which knowledge can be evaluated.

This is not a license to believe whatever we want, though. For most of us, most of the time, our purpose should probably be to believe that which best predicts what we observe about the world, i.e. we should believe what is true. The key insight is to not get confused and think that a norm in favor of truth is the same thing as truth being the only possible way to face reality. The ground of understanding is not solid, so to speak, and if we don't choose to make it firm when needed it will crumble away beneath our feet.

Thus, the problem of the criterion is also, as I hope will be clear, a killer of logical positivism. I point this out because logical positivism is perennially appealing, is baked into the way science is taught in schools, and has a way of sneaking back in when one isn't looking. The problem of the criterion is not the only problem with logical positivism, but it's a big one. In the same way, the problem of the criterion is also a problem for direct realism about various things because the problem implies that there is a gap between what is perceived and the thing being perceived and suggests the best we can hope for is indirect realism, assuming you think realism about something is the right approach at all.

Finally, all this suggests that dealing with knowledge and truth, whether as humans, AIs, or other beings, is complicated. Thus why we see a certain amount of natural complexity in trying to build formal systems to grapple with knowledge. Real solutions tend to look like a combination of coherentism and pragmatism, and we see this in logical induction, Bayesian inference, and attempts to implement Solomonoff induction. This obviously has implications for building aligned AI that are still being explored.

Recap

This post covered a lot of ground, but I think it can be summarized thusly:

  • There is a problem in grounding coherent knowledge because to know something is to already know how to know something.
  • This circularity creates the problem of the criterion.
  • The problem of the criterion cannot be adequately solved, but it can be addressed with pragmatism.
  • Pragmatism is not a complete response because no particular pragmatism is privileged without respect to some purpose.
  • The implications of necessarily grounding knowledge in purpose are vast.

34

38 comments, sorted by Highlighting new comments since Today at 8:48 AM
New Comment

This all seems to rest on an idea that an empty box labeled "truth" was dropped in my lap in the platonic land of a priori mental emptiness, and I'm obligated to fill it with something before I'm allowed to begin thinking. But obviously, that's not what happened. Rather, as I grew up, the abstract label called "truth" was invented and refined by me to help me make sense of the world and communicate with (or win approval from) others. So I end up at the same answer, pragmatism, but I deny that there was ever any problematic circularity. The problem instead seems to come from a notion of transcendent epistemic justification which is unmotivated and flawed--what good does this idea of justification do? What's the problem if I do something without justification?

So it would seem at first, but "truth" is just one place the problem of the criterion shows up. It's the classical version, yes, and that version did have a rather outdated notion of what truth is, but we can also just talk in terms of knowledge and belief and prediction and the problem continues to exist.

Do you ever claim that things are true?

Yes, but only in the sense that by my best efforts, using the brain I actually have, I believe the thing to be the case.

Yes, you can deal with the problem of the criterion by adopting modest epistemology. But that is different from saying there is no problem.

I like particularism more than pragmatism, because all reasoning needs some kind of basis, but not all reasoning needs some kind of goal. For example, proving theorems in Peano arithmetic has a basis, but no goal.

More generally, I'd prefer not to ground philosophy in the notion that people follow goals, because so often we don't. Life to me feels more like something spinning outward from its own basis, not toward something specific.

Finally, we come to a special case of particularist responses known as pragmatism., and it’s this kind of response that Yudkowsky offered. The idea of pragmatism is to say that there is some purpose to be served and by serving that purpose we can do an end-run around the problem of the criterion by tolerating unjustified knowledge so long as it works well enough to achieve some end

This kind of pragmatism solves an easier version of the problem. Winning, and it's close relative, prediction, can both be measured directly. That what allows you tell that something works without having an apriori theoretical justification. On the other hand, correspondence-to-reality can't be tested directly...you can't stand apart from the map territory relationship. In Scientism it's common to assume that predctiveness is a sign of correspondence...

Coherentist responses reject the idea that truth, knowledge, etc. must be grounded and instead seek to find a way of balancing what is known with how it is known to form a self-consistent system.

Coherentism may lack foundations in the sense that foundationalism has foundations, but it still has guiding principles -- notably, that coherence is conducive to truth!

Right, and this gets at why Chisholm argues there are only really two positions—particularism and skepticism/nihilism—because coherentism still requires one know the idea of coherentism to start, although to be fair coherentism would theoretically allow one to stop believing in coherentism if believing in it ceased to be coherent!

How could believing in coherentism cease to be coherent?

For instance, if there is some self contradiction involved.

Chisholm can think what he likes, but what I am saying is that skepticism qua nihilism isn't a viable position.

Right, I think Chisholm would agree that skepticism is not useful and you can't do anything with it, so if we want to live he'd argue that particularism is, in fact, the only choice whether we like it or not.

And then there’s skepticism, arguably the only “correct” position of Chisholm’s three in that it’s the only one that seemingly doesn’t require assuming something on faith. Spoiler alert: it does because it still needs some reason to prefer skepticism over the alternatives, thus it still ends up begging the question. Skepticism is also not very useful because even though it might not lead to incorrectly believing that a false thing is true, it does this by not allowing one to believe anything is true!

Which is a problem if you have good reason to believe some things in general are true. If you do, you might as well believe whichever specific things are most plausible , even if they are not fully justified.

You have characterised scepticism as a nothing-is-true position. But "it is true that nothing is true" s self defeating. If the problem of the criterion means that nothing is well justified, then strong claims should be avoided , including strong negative claims like "nothing is true". So scepticism done right is moderation in all things.

But "it is true that nothing is true" s self defeating

Correct; this is another way in which skepticism begs the question.

If the problem of the criterion means that nothing is well justified, then strong claims should be avoided , including strong negative claims like "nothing is true". So scepticism done right is moderation in all things.

Right, the kind of "skepticism" I'm talking about here is different from everyday skepticism which is more like reserving judgement until one learns more. Skepticism here is meant to point to a position you might also call "nihilism", but that's not the term Chisholm uses and I stuck with his terminology around this.

Correct; this is another way in which skepticism begs the question

I said it was self defeating which is a different problem.

Right, the kind of “skepticism” I’m talking about here is different from everyday skepticism which is more like reserving judgement until one learns more.

I know it is different. The point is that it is better.

If we can prove that the Problem of the Criterion is a true problem, then the Problem of the Criterion is a false problem. Therefore, the Problem of the Criterion can never be proven to be a true problem.

Philosopher: "There is a way of thinking where you can never have a feeling of certainty. Not on any mental, physical, or social level. And I can teach it to you!"

Caveman: "Why would I want to learn that?"

Philosopher: "..."

Perhaps one goal of the pragmatist is to avoid thinking about the Problem of the Criterion. It's like "the Game" we used to play in grade school, where you lose the game by thinking about it. Kids would ambush each other by saying "you just lost the Game!" apropos of nothing.

If my take on this issue is wrong, I encourage proponents of the Problem of the Criterion to prove it!

This article and comment have used the word "true" so much that I've had that thing happen where when a word gets used too much it sort of loses all meaning and becomes a weird sequence of symbols or sounds. True. True true true. Truetrue true truetruetruetrue.

Philosopher: “There is a way of thinking where you can never have a feeling of certainty. Not on any mental, physical, or social level. And I can teach it to you!”

Caveman: “Why would I want to learn that?”

Rationalist: If it is impossible to be certain, I want to believe it is impossible to be certain.

What if it is impossible to believe the truth?

Reading this, my guess is that you underestimate the importance of being pragmatic to a single purpose.

When you are pragmatic, you are pragmatic with respect to some purpose. Let's suppose that purpose is truth. You'll get by quite fine in the world if you do that.

Yet, there's some things you'll miss out on. For example, suppose you want to know what it is like to be intentionally deceitful, perhaps because you are dealing with a person who lies and would like to understand what it is like to be them. To fully do that you have to think the same sort of thoughts a deceitful person would and that would require knowing, even if only temporarily, something not maximally predictive of the world to the best of your ability. Thus you must be able to think with the purpose of deceiving others about something in order to embody such thought long enough to get some first hand experience with thinking that way.

I think this generalizes to deeply understanding what it is like to be someone who isn't you. In fact, I'll go out on a limb and say the typical mind fallacy exists because we are especially bad at considering the problem of the criterion and thinking thoughts as if we were serving someone else's purpose rather than our own.

This approach seems wrongheaded to me, from the start. Perhaps I am misunderstanding something. But let’s take your example:

Let’s say you want to know something. Doesn’t really matter what. Maybe you just want to know something seemingly benign, like what is a sandwich?

HALT! Proceed no further. Before even attempting to answer this question, ask: why the heck do you care? Why do you want to know the answer to this strange question, “what is a sandwich”? What do you plan to do with the answer?

In the absence of any purpose to that initial question, the rest of that entire section of the post is unmotivated. The sandwich alignment chart? Pointless and meaningless. Attempting to precisely define a “sandwich-like object”? Total waste of time. And so on.

On the other hand, if you do have a purpose in mind, then the right answer to “what is a sandwich” depends on that purpose. And the way you would judge whether you got the right answer or not, is by whether, having acquired said answer, you were then able to use the answer to accomplish that purpose.

Now, you could then ask: “Ah, but how do you know you accomplished your purpose? What if you’re a brain in a jar? What if an infinitely powerful demon deceived you?”, and so on. Well, one answer to that might be “friend, by all means contemplate these questions; I, in the meantime—having accomplished my chosen purpose—will be contemplating my massive pile of utility, while perched atop same”. But we needn’t go that far; Eliezer has already addressed the question of “recursive justification”.

It does not seem to me as if there remains anything left to say.

It does not seem to me as if there remains anything left to say.

You're right, in the end it's best to say nothing, for then there are no problems. Alas, we suffer and strive and want things to be other than they already are, and so we say something and get ourselves into a world of trouble.

But we needn’t go that far; Eliezer has already addressed the question of “recursive justification”.

I'm not sure what your argument is here? Eliezer already wrote a post on this so no one should bother writing anything about the same topic ever again?

HALT! Proceed no further. Before even attempting to answer this question, ask: why the heck do you care? Why do you want to know the answer to this strange question, “what is a sandwich”? What do you plan to do with the answer?

In the absence of any purpose to that initial question, the rest of that entire section of the post is unmotivated. The sandwich alignment chart? Pointless and meaningless. Attempting to precisely define a “sandwich-like object”? Total waste of time. And so on.

On the other hand, if you do have a purpose in mind, then the right answer to “what is a sandwich” depends on that purpose. And the way you would judge whether you got the right answer or not, is by whether, having acquired said answer, you were then able to use the answer to accomplish that purpose.

As best I can tell you are making an argument here for pragmatism but want to skip talking about epistemology. I won't begrudge your right not to care about epistemology, only the point of this post was to explore a conundrum of epistemology, so I'm not quite sure what you're trying to say here other than you agree with pragmatism about the problem of the criterion but don't want to talk about epistemology.

(As a reminder, my policy of only replying to you once in a thread on my posts remains in place, although I remain open to reversing that decision.)

I’m not sure what your argument is here? Eliezer already wrote a post on this so no one should bother writing anything about the same topic ever again?

Eliezer already wrote a post on this, so no one should bother writing anything about the same topic that does not add anything new to Eliezer’s post, or do anything to result in changing one’s conclusion from that described in Eliezer’s post. This post does not (indeed, it would be quite difficult to write anything that does).

As best I can tell you are making an argument here for pragmatism but want to skip talking about epistemology.

My point is precisely that conundrums of epistemology, and epistemological questions in general, are motivated by pragmatic things or else by nothing.

I do not think there is any “problem of the criterion”, except in a way such as is already addressed to the maximum possible degree of satisfaction by the linked Sequence post. Hence no “conundrum” exists.

When I said that nothing remains to say, I meant that anything else that’s said on this topic, over and above the linked post, is strictly superfluous—not that in general saying things is bad.

My point is precisely that conundrums of epistemology, and epistemological questions in general, are motivated by pragmatic things or else by nothing

They are motivated by whatever they are motivated by. It is possible for someone to value truth as a end in itself. Maybe you are not that person, but you have solved nothing for the people who value truth terminally

Eliezer already wrote a post on this, so no one should bother writing anything about the same topic that does not add anything new to Eliezer’s post, or do anything to result in changing one’s conclusion from that described in Eliezer’s post

He's one of those people. He hasn't put forward a solution where he sticks only to doing things of measurable value. He insists that MWI is the correct interpretation of QM , and is so to a high degree of certitude. But an interpretation of QM cant be tested empirically, and has no empirical consequences So he needs a justification of epistemology that allows for a) high credibility about b) abstract topics of no practical significance. And he hasn't got one, and he hasn't solved epistemology for the wider rationalist community, because they also haven't adopted pure pragmatism.

I do not think there is any “problem of the criterion”, except in a way such as is already addressed to the maximum possible degree of satisfaction

If there were already a satisfactory answer to the problem of the criterion , there would be no need to lower the bar by adopting pragmatism or instrumentalism. Solving epistemology for self confident epistemology is even harder.

(Alright, breaking my own rule; we'll see how it goes.)

Eliezer already wrote a post on this, so no one should bother writing anything about the same topic that does not add anything new to Eliezer’s post, or do anything to result in changing one’s conclusion from that described in Eliezer’s post. This post does not (indeed, it would be quite difficult to write anything that does).

I guess I don't have much to say to this, as I disagree with your judgement here that there's no value in saying the same things in a new way even if nothing new is added, although I also disagree of course that I don't add anything. Sometimes saying the same thing in a different way clicks for someone when it didn't click for someone else because not everyone has literally the same mind. Cf. Anna Salamon on learning soft skills and the project of distillation.

My point is precisely that conundrums of epistemology, and epistemological questions in general, are motivated by pragmatic things or else by nothing.

This is literally also one of my main points, so I guess we at least agree on something.

I do not think there is any “problem of the criterion”, except in a way such as is already addressed to the maximum possible degree of satisfaction by the linked Sequence post. Hence no “conundrum” exists.

Sure, but I'm not satisfied with his post, hence I think there is more to say, though you obviously disagree.

Thus I'm left struggling to figure out a charitable motivation for your comments. That is, I'm not sure what point you are trying to make here other than you are annoyed I wrote something you didn't want to read since I already agree with pragmatism, linked and highlighted your preferred Yudkowsky article on this topic, and we otherwise agree that there is something going on to be addressed regarding grounding problems.

It's the dream of covering every topic with a minimum web of posts, that only grows, without overlap? (The dream of non-redundant scholarship)

Or SA doesn't think 'the problem of the criterion' is motivated by a purpose (or SA's purpose)?

writing something on a topic is quite different to solving it forever.

Yes. Though even if something is solved, it can be solved again in a different way.

I'm a fan of W. W. Bartley's Pan Critical Rationalism, from his book The Retreat To Commitment. It doesn't seem to me to fit in your list of approaches. Bartley was a student of Karl Popper, who proposed Critical Rationalism. CR, badly stated, says "This is the fundamental tenet: criticize all your beliefs and see what survives."  PCR cleans that up by saying "This is the best approach to epistemology we've discovered so far: criticize all your beliefs (including this one) and see what survives."

Isn't that better than believing in a foundational, unjustified criterion? Isn't it more flexible than methodism? Isn't it more useful than skepticism?

It seems like you still need some criterion through which to criticize your beliefs. Popper offers the criterion that "your past observations don't falsify your theory", and "your theory minimizes adhocness", but by which criterion can you accept those criterion as true or useful?

Right! This is the slippery part about the problem of the criterion: literally any way you try to address it requires knowing something, specifically knowing the way you try to address it. It's in this way that nearly every response to it could be argued to be a special case of particularism, since if nothing else you are claiming to know something about how to respond to the problem itself!

Bartley is very explicit that you stop claiming to "know" the right way. "This is my current best understanding. These are the reasons it seems to work well for distinguishing good beliefs from unhelpful ones. When I use these approaches to evaluate the current proposal, I find them to be lacking in the following way."

If you want to argue that I'm using an inferior method, you can appeal to authority or cite scientific studies, or bully me, and I evaluate your argument. No faith, no commitment, no knowledge.

This comment describes a response that sounds exactly like pragmatism to me, so I'm not sure what the distinction you're trying to make here is.

Also, as Matt already pointed out, you must have some criterion by which you criticize your beliefs else you literally could not make any distinction whatsoever, so then the problem just becomes one of addressing how to ground that, perhaps by accepting it on faith.

Trying to anticipate where the confusion between us is, it might help to say that taking something on faith need not mean it remain fixed forever. You can make some initial assumption to get started and then change your mind about it later (that's fundamental to coherentist approaches).

See Matt's comment, but CR and PCR sound like coherentist or methodist responses.

They sound not-foundationalist. But they differ in what they are replacing foundations with. Coherehence is quite different from attempted-refutation.