Think about a banging sound at 0dB, 1dB, ..., 90dB. Everyone will be bothered at 90dB. Nobody will notice at 0dB. Somewhere in between, it transitions from effortless-to-ignore to impossible-to-ignore. Where is that point? It's not determined by the laws of physics, it's arbitrary, it's a setting in your brain, and it's set differently in different people. Similar for touch. Neurotypical people will not be able to ignore a woodpecker pecking them in the back, but may find it effortless to ignore a shirt tag. Other people find that the shirt tag keeps drawing their attention, but a gentle enough touch sensation would not rise to attention.
The "intense world theory of autism" (about which I'll finish a blog post draft one of these years...) says that autism happens when empathetic social interactions are so overwhelming that the person learns early in life to just avoid thoughts and situations that solicit those feelings, including deliberately avoiding eye contact etc. Not coincidentally, people with autism are liable to have sensory sensitivity too, i.e. to feel overwhelmed by levels of sound and touch that are far lower than what it takes to overwhelm neurotypical people.
Anyway, long story short, we all have innate reactions to different stimuli, and the thresholds can be set differently. That's just the way it is, I think.