Everything you know about physics and formation of clusters of matter is just part of the false-history that the Boltzmann brain experiences. For purposes of the thought experiment, assume the least convenient universe - pure randomness. It just so happened that your entire set of memories came into being, experienced this millisecond, and then will dissolve.
Behind this is the idea that there is no perception of the actual present. Each unit of experience (minimum time-slice) is entirely memories, plus inputs at least a few hundred milliseconds old. The next time-slice is connected by our perceptions, but in a BB, there would be no next time-slice, just the one.
I get the sense that you should enjoy learning more physics, and shouldn't worry about Boltzmann brains again until you at least have a physics-based intuition for whether we expect Boltmann brains or the decay of iron stars to come first.
Still, some links that might interest you:
Your issue is not Boltzmann brains at all. It's basic rationality: noticing your own confusion and therefore giving very low weight to your own confused arguments (and fears). There are many many scary things one can imagine, and Boltzmann brains are not even in the top 100. At this point, don't worry about physics, start with the basics.