If I understand virtue ethics correctly, which I don't, virtue ethicists want to have good autopilots. They don't give themselves much credit for doing good things, except inasmuch as it shows they do them and makes them more likely to continue to do so. Likewise, they don't do slightly bad things, because that would condition them to do bad things in other circumstances.
I think a more distinctly virtue ethicist way of putting it is that they don't do slightly bad things because that would condition them to have bad dispositions, or to be bad people, something that is intrinsically disvaluable.
People who avoid doing slightly bad things to prevent instilling unhelpful habits, and to prevent themselves from bringing about future harm are (roughly) global utilitarians.
What's a bad person with bad dispositions, if not someone who does bad things?
Well, in practice most of us do make these sorts of judgments. We don't think a bad person stops being a bad person when we imprison them, for example, even if they are thereafter unable to do bad things and therefore don't. Which suggests that we have some notion of a bad person who isn't doing bad things.
This may not be justified, but I'd be very surprised if it were so alien to you that you don't understand it.
I sometimes think "I shouldn't deplete myself by doing unpleasant things that don't matter a lot - I should save my willpower for the really important things." Apparently this may get less true as I age, so I should push myself sometimes to find out how much ego depletion is really happening.
Age Shall Not Weary Us: Deleterious Effects of Self-Regulation Depletion Are Specific to Younger Adults
Brain areas associated with self-regulation don't mature until the mid-twenties. And apparently, if you compare older and younger people in a standard set-up for detecting ego-depletion, older people are not affected. WEIRD, indeed.
In 2010, there was a study indicating that ego depletion doesn't affect people who don't expect it to (and that one was run on college students). I wonder if part of the effect might be coming from differences in attitudes about effort between older and younger people.