There is also one more level of rationality, which is often assumed, but not presented. I would call it the inner definition of rationality:

"Rationality is a behavior which could be presented by a short set of simple rules". The rules include math, Byes theorem, utility function, virtues of rationality, you name it.

The main problem is following: does "inner definition" of rationality corresponds to the "outer definition", that is rationality as winning? That is, does knowing correct short set of rules results in constant winning?

If we think that yes, then all rationality manuals are useful, as by installing correct set of shots rules, we will get perfect rationality and will start to win.

However, if winning requires something extremely complex, like a very large neural net, which can't be presented by a short set of rules, we need to update our inner definition of rationality.

For example, a complex neural net may win in cat recognition, but it doesn't know any set of rules how to recognize a cat.

It's been almost four months since I wrote this thread. I've started to see the outline of an answer to my question. Over the course of the next year, I would begin documenting it.

Being lucky is not being rational. However it is undoubtable that winning in a lottery is mostly a positive outcome and that it requires for you to have purchased the ticket which is a decision. Something that looks only at outcomes would applaud the decision to buy the ticket (perhaps unconditionally).

The definiton of instrumental rationality is most commonly invoked when critisiing those that employ a complex methodology of choosing correctly while the methodolody can be criticed or the agent had evidence that could have been construed to be a reason to abandon the methodology. The critism "before" "instrumental rationality would focus on making an error in applicaiton of a methodology or not having any methodology at all to make the decision. The common sentiment from these can seem like "have a methodology and apply it correctly". And it seems clear that there are better and worse methodologies and one should try to apply the best available. And it seems "I had a methodology and applied it" doesn't make one to be "rational" (more like "dogmatic").

It seems one coudl have a reasonable chance of being "rational" if one had bad methodologies if one actively upswitches and upgrades their "carry on" methodology whenever they encounter new ones. It seems also that as the argument goes on the focus on metacognition increases. This can be seen also to frame the previous critisms in a new light. Its not that unmethodological decisions are "unrational" per se but doing so means likely that you missed to pick up a good methodology before that where you here could have applied to great success. So rather than "having" an methodology its more important to "pick up" methodologies with it being less essential whether you currently have or do not have a good methodology. With consistent pickups you should in the future have a great quality methodology. but rather than being the means its the effect.

From Jonathan Baron's Thinking and deciding:

The best kind of thinking, which we shall call rational thinking, is whatever kind of thinking best helps people achieve their goals. If it should turn out that following the rules of formal logic leads to eternal happiness, then it is “rational thinking” to follow the laws of logic (assuming that we all want eternal happiness). If it should turn out, on the other hand, that carefully violating the laws of logic at every turn leads to eternal happiness, then it is these violations that we shall call “rational.” When I argue that certain kinds of thinking are “most rational,” I mean that these help people achieve their goals. Such arguments could be wrong. If so, some other sort of thinking is most rational.

It may lead to what I call "folk rationality" (doing what you think would lead to success). Barring a few exceptions (extremes of emotion, compromised mental states, etc), most humans are folk rational. However, this folk rationality isn't what I refer to when I say "rational".

How about "doing what you can figure out would lead to success"? The gambler could figure out the gambler's fallacy, but the person crossing the road couldn't figure out the meteorite.

In harder problems like Newcomb's Problem or Counterfactual Mugging, there are several layers of "figuring out" leading to different answers, and there's no substitute for using intelligence to choose between them. So to define what's rational, we need to define what's intelligent. People are working on that, but don't expect an answer soon :-)

All humanimal attempts to define rationality are irrational!