I decided, long ago, that I had the right to think whatever thoughts I wanted, including arbitrarily inappropriate sexual thoughts, as long as I kept my actions in check. (Which includes not telling people about thoughts they'd rather not hear about.) And the same goes for everyone else. I feel this is an important right, and will fite irl to defend it.
I agree with the principle but I caution that there is a risk to accidentally reprogram yourself. Thoughts may become actions if you are not careful.
I [have] the right to think whatever thoughts I [want], including arbitrarily inappropriate sexual thoughts
Although I agree with that, my experience has been that if I fantasize a lot about a woman (i.e., at least one "fantasizing session" ending in intense pleasure), it becomes significantly less likely that she will response positively to me.
Having had the fantasies changes my behavior in some way that the woman tends to notice (although I myself do not notice).
It took me many, many years to notice this cause-and-effect relationship. (Why so long? Maybe because pleasure is the motivation in motivated cognition.) But by now I realize that (at least for me) the effect is strong enough that I have basically adopted a rule of refraining from fantasizing a lot about a woman unless I know for sure I will neither want nor need to interact with her in the future. "Interact": regularly pass her in a hallway, talk to her in any capacity.
I adopted this rule because I need all the help I can get in being liked or at least consistently tolerated by the women in my environment, not because I recognize any moral imperative to refrain from fantasizing about anybody.
I am a straight man in his 60s.
IMPORTANT ADDENDUM.
The OP writes of sexual thoughts that last less than 5 seconds. That is definitely not at all what I refer to above as "fantasizing a lot".
I think most adult women realize that most men run a quick sexual evaluation of every single woman they interact with (and that this quick evaluation often involves imagining what it would be like to have sex with her). My guess is that a lot of women are perceptive enough that they can detect this evaluation as it is happening, but are forgiving of it -- or at least it does not worry them or annoy them -- as long as it does not go on and on. 5 seconds is fine if my life experience is any guide.
Also, my guess is that most men would not be able to stop running this evaluation during those initial 5 seconds or so without employing measures that are very expensive for the man in time, attention or opportunity costs.
I have one female friend (not girlfriend) who I know does by default not interpret any sexual intentions into the actions of males. For example, once she was asked out for a drink by a male stranger, and she did not see the intention behind this. She thought it was just about hanging out. I am not sure how old she was there. When this sort of thing happens a lot to you, then you probably become more aware of it.
That said, when I ask her to meet up, to play chess, she also did not see my intention behind that, and there she was definitely in her twenties. Thought that is just a sample size of one.
Interestingly, it seems that with these sorts of women it is easier to be friends. I definitely also experienced that women would react negatively toward me being friendly. Even if I was just being friendly. Maybe they immediately interpret sexual intentions into my actions and feel the need to make clear their disinterest (this might be unconscious). This could be something as benign as saying "Hi" in a friendly tone of voice. Though I might understand my tone of voice less than the woman. My tone of voice might reveal parts of my mind that I am not aware of, that contain attraction towards the woman.
One of the most useful moral heuristics that I know is: It is ok to do X, if you don't hurt anyone by doing X. And this applies here too.
Yes, though I was actually already believing this when feeling bad about my thoughts. I was not worried about other people thinking about me strangely. I was seeing it as a personal failure, which still made me feel bad. My point is that having unrealistic standards of yourself can also lead to unproductive suffering.
I have noticed very often that I can't predict how my brain reacts towards women. I often expect that nothing special will happen, compared to talking to males. However very often sexual thoughts come to mind that seems out of place to me. Normally they are very short (1-5s) and visual, but with very low detail.
This would happen most often when meeting women for the first time. Normally within the first minute of meeting them. Actually, this does not only happen in meetings but even when I watch a video with a woman (I mean a normal video, not pornography/erotica). Also, the frequency and strength of this are proportional to the time to last orgasm. Although that correlation is much weaker for me than one would expect. Though I am trying to maximize the time between orgasms. If I would not do this, there might be a stronger correlation. Certainly, if there are many orgasms not far apart (e.g. on the same day), there would be no sexual thoughts at all.
My normal reaction now for years was to feel embarrassed about my own thoughts. Interestingly, I always fail to anticipate my thoughts. Thus far I have seen the arising of sexual thoughts as a bad thing. A failure of self-control. Maybe when I manage to not have sexual thoughts for some time, my brain seems to think that now this problem is gone, and then it gets disappointed when thoughts come up again.
However recently I have managed to actually sort of just let it happen. Not only the imagination but also the suppression of it, without judging it at all. I sort of just let it happen and observed it. Right as I was starting to think sexual thoughts, I was thinking in a flash that this is a part of myself and that fighting it with negative emotions does not make sense. And then I managed to just observe it over the next 3 seconds before it disappeared, without any strong negative feelings. And it still disappeared. The negative feelings were not required to make it disappear!
Learning to accept/love yourself is old wisdom. I think I was applying it to various degrees before. But never to this explicitly. This might help me, to avoid unproductive negative feelings in the future. I wonder if this will continue to work.
I could imagine that this actually increases the frequency and intensity of sexual thoughts as it might make me not suppress them as hard. But even if this is the case, the psychological relief might still be worth it.