There are things that are worthless-- that provide no value. There are also things that are worse than worthless-- things that provide negative value. I have found that people sometimes confuse the latter for the former, which can carry potentially dire consequences.
One simple example of this is in fencing. I once fenced with an opponent who put a bit of an unnecessary twirl on his blade when recovering from each parry. After our bout, one of the spectators pointed out that there wasn't any point to the twirls and that my opponent would improve by simply not doing them anymore. My opponent claimed that, even if the twirls were unnecessary, at worst they were merely an aesthetic preference that was useless but not actually harmful.
However, the observer explained that any unnecessary movement is harmful in fencing, because it spends time and energy that could be put to better use-- even if that use is just recovering a split second faster! 
During our bout, I indeed scored at least one touch because my opponent's twirling recovery was slower than a less flashy standard movement. That touch could well be the difference between victory and defeat; in a real sword fight, it could be the difference between life and death.
This isn't, of course, to say that everything unnecessary is damaging. There are many things that we can simply be indifferent towards. If I am about to go and fence a bout, the color of the shirt that I wear under my jacket is of no concern to me-- but if I had spent significant time before the bout debating over what shirt to wear instead of training, it would become a damaging detail rather than a meaningless one.
In other words, the real damage is dealt when something is not only unnecessary, but consumes resources that could instead be used for productive tasks. We see this relatively easily when it comes to matters of money, but when it comes to wastes of time and effort, many fail to make the inductive leap.
 Miyamoto Musashi agrees:
The primary thing when you take a sword in your hands is your intention to cut the enemy, whatever the means. Whenever you parry, hit, spring, strike or touch the enemy's cutting sword, you must cut the enemy in the same movement. It is essential to attain this. If you think only of hitting, springing, striking or touching the enemy, you will not be able actually to cut him. More than anything, you must be thinking of carrying your movement through to cutting him. You must thoroughly research this.