Insolvency is very expensive! You can map it into the framework outlined in the post by assigning insolvency some interest cost of x% where x% >> 4%. If you don't like assigning a made up interest cost to being insolvent you can instead think of the whole thing in terms of a higher order representation such as utility. It then follows that you should cover your insolvency before covering most debts.
Your rules of thumb at the end appear very pragmatic in that they're easy to follow, and I use a similar system for myself. Do you happen to have a rule of thumb for how much return you require for a specific risk?
"happiness depends on the log of income"
I subscribe to the idea that increased wealth has approximately logarithmic utility. This is very tangential to the topic of your post but... I'd be curious to hear your thoughts about a corollary stemming from this that one should be willing to take increased risks with additional capital due to its logarithmic utility? What is your take on that, should an individual with assets / income beyond their needs be willing to take increased risk?
This more or less regresses to an offline supervised learning model in which a bunch of samples are collected upfront, a model trained, and then used to predict all future actions. While you might be framing your problem as an MDP, you're not doing reinforcement learning in this case. As TurnTrout mentioned in a sibling to this comment it works only in the stationary & deterministic environments which represent toy problems for the space, but ultimately the goal for RL is to function in non-stationary non-deterministic environments so it makes little sense to focus on this path.