This post is a bit of entertainment for scientifically inclined Harry Potter fans.
Time turner from the Harry Potter series (and from the Eliezer Yudkowsky's venerable HPMoR fanfic) is a very useful device if you have some unfinished business in the recent past, like attending an extra class or saving a friend from a certain death. However, General Relativity has a few words to say about them, and they are not very flattering. I will only address one issue here: Energy conservation. TL;DR: if you use a time turner to vanish into the past, those around you will see you blown to tiny bits of Merlin-knows-what, quickly disappearing from view. When you appear in the past, this explosion appears in reverse.
Before we get to the time turners, however, let us consider an aside.
Let us start with a common question: if the Sun stop shining this instant, when would we notice? The common answer: it takes light 8.5 minutes to travel the distance of 150,000,000 km between the Sun and the Earth, so that's how long it will take. This glosses over the issue of what does "this instant" mean exactly at two different points in space, which is not so trivial given the relativity of simultaneity in Special Relativity. It is easily patched up, however, once we fix a global frame of reference. The Cosmic Microwave Background (CMB) is a natural one to use, and both the Earth and the Sun travel with a negligible fraction of the speed of light relative to the CMB. Anyway, the answer is still very close to 8.5 min.
Now another, deceptively similar question: if the Sun disappears this instant, how long before the Earth will stop orbiting the point where it used to be? The common answer: gravity travels with the speed of light, so also 8.5 min. This answer is obvious, simple and wrong. Yes, dead wrong. Why? because static gravity is not like light, it's more like electric field, only worse.
Let's first think of how you would make the Sun disappear. Maybe it turned into a black hole? Well, this would not really mean disappearance of gravity, the mass of the black hole will still be that of the Sun, and the Earth will happily (or unhappily, as the case may be) continue orbiting the Sun's corpse. So, in this case the answer is "it won't stop orbiting".
OK, so black hole was a bad example. How about a wormhole instead? You know, the evil Vogon-like aliens need to clear the room for a hyperspace bypass, and they build a wormhole from far away and suck all the matter in the Sun through it out of the way. What would happen then? There are a couple of hints: one is that from outside a wormhole is indistinguishable from a black hole, and the other is the Gauss Law. Both hints lead one to the same answer: just like with turning the Sun into a black hole, there is very little gravitational effect on the surrounding space. The rest of the now ex-Solar system will continue merrily on its way around the point where our Sun used to be.
An aside for those curious about the Gauss Law argument. The law in its integral form states that the flux of the gravitational field inward through any closed surface encompassing the Sun is proportional to the Sun's mass. To change the field, you need to remove some mass from inside this imaginary surface, by having it physically cross the surface. This last point may not be obvious, but it follows from General Relativity. Specifically, the Einstein's most misunderstood theory says that the spacetime curvature is determined by the (past and present) distribution of matter in spacetime. There are some exceptions, like the fixed-mass spherical objects, such as black holes and wormholes, which contains no matter, and gravitational radiation, which can carry away energy. But if you take a spherical object like the Sun and try to calculate what happens if you decrease its mass, General Relativity tells you that this mass has gone outward from the Sun in all directions in some form. It is not fussy about the form, as long as just the right amount of mass/energy has gone out.
Let me repeat for those who skipped the above paragraph: if you take the Sun and decrease its mass, the only way it can happen if this mass leaves the Sun outward and disappears into space. This happens all the time, of course, the Sun constantly loses its mass through radiation and solar wind, or in more drastic cases through Supernova explosions. Effects like this propagate no faster than light, of course. So they take forever to propagate all the way to infinity.
Now, back to the time turners. Hermione Granger might be but a small if incredibly studious girl, but she still has mass. If you were to peek at her using a time turner and disappear, her mass, small though it may be, still has to go some place, just like the disappearing Sun's mass had to go some place. The options are few: she can blow into tiny pieces flying past you, or disappear in a flash of brilliant light (and it takes a lot of light to carry away 50kg, what's with E=mc^2) . Basically, it will not be a pretty sight. What cannot happen is her simply vanishing, with no ill effects whatsoever. Well, it cannot happen if we are willing to keep Relativity around. Maybe we don't have to, what's with a certain deputy mistress turning into a cat and back, probably instantly changing her mass, with no ill effects on her or her surroundings. But if you give up on General Relativity, quite a few things will unravel, like all four Newton's laws.
Also don't forget the other side of the time turner action: Hermione appearing out of thin air just before walking into her extra class. The above process has to happen in reverse: an amount of matter equivalent to her mass has to travel inwards out of nowhere and coalesce into a person. Where did this matter come from? How did it form before collapsing into a person? How did it know that it would need to time its arrival into a certain point perfectly with whatever time turner will have been set to? That's some hard-core magic right there. Also, suck it, the Second law of Thermodynamics.
So, let me summarize: mass cannot just disappear, it has to spread out. mass cannot just appear, it has to coalesce. Thus time turners cannot be used inconspicuously, everyone around would be well aware of one's use, assuming they survive it. Actually, it probably cannot be used at all without breaking General Relativity and/or Thermodynamics. But hey, that's what magic is for.
EDIT: this post currently sits at -2 karma with 6 downvotes. I'd appreciate if any of the people who thought "I want less of this" explicate their logic to me, so I can do better next time.
EDIT2: OK, no one replied to my request... I'm guessing that some of you guys just quietly hate me :)