It's a popular request. Search this thread for "preview" (it was requested multiple times). TrikeApps has declined to develop it so far for more or less the reasons Emile suggested, but the code that runs this site is open source, and contributions are welcome.
Since you can edit any formatting problems after the original posting, and comments are typically small, so that any problems are easy/fast to fix, I don't see a significant need for this feature.
Several factors why it isn't that much of a problem:
You can quickly edit your comment afterwards anyway (and still have the rest of the page visible)
The markup is pretty simple, there's less problems with mismatched tags than there can be in some forum and blog markups
Comments tend to be smaller than in forum and blog posts or than wiki pages (all cases where a preview is useful), and rarely contain complex things like nested quotes or images
Also, an edit button would cost development/debugging time that may be better used on other features (votes, agree/disagree, view edit history for comments, giving all users an animated gif avatar, etc.), and might clutter the interface.
But if we do have previews, maybe a simpler way of handling it would be to always have a preview of your comment right below the edit box, and update it in real time (I'd find that more convenient than a preview button - it's simpler from a user interaction point of view, and probably also from a code point of view, because it can all be done in client-side javascript).
It is on the radar for possible site improvements. A few people brought it up in the site suggestions thread, and Matt (who is leading the current round of site updates) recently addressed it here:
I agree that preview would be nice. What we have now works pretty well, so I'm judging this feature as low benefit/cost.
Surely I can't be the first person to have thought of it, but Uncle Google suggests this hasn't been discussed before.
There was previously a workaround that I used rather frequently. That is, if there was an error I quickly deleted it and edited the text from the copy/paste buffer so that nobody was exposed to the flawed version while I was hacking it. But it is even more important now.
A preview option for comments would be valuable, but it's not currently supported by the software, so it'll take some work.
Surely I can't be the first person to have thought of it, but Uncle Google suggests this hasn't been discussed before. Would it be difficult to make a preview button available when posting comments? Was it lacking from the software being used or is it just disabled? This blog uses different ways of text formatting than I think a lot of us are used to from other discussion forums, so if it happens to be easy to do, it'd be good to be able to experiment and see the results before making one's comment available.
I just tried that sandbox linked to from the wiki and it doesn't seem trustworthy, what should come out as italics comes out as some sort of a highlight...
Funny, I felt that bystander reluctance thing while posting this. "Why hasn't anyone posted this before? Is it because nobody wants to be the one asking for something? Or is it a silly request in some way I don't see now?"
EDIT:
I see Uncle Google failed me. Or is it that I failed Uncle Google? ;)
Thanks for the reponses, all.