(N.B. The default setting is (on new accounts at least) that "Make my votes public" is unchecked. I mention this because the main post is phrased like public is the default.)
Yeah, I have the feature on on purpose in case they ever got around to supporting it properly.
I think this feature might have been broken a while ago, but it works now. So if you don't want your likes and dislikes to be public, go to your preferences page and uncheck "Make my votes public." At the moment, the upvotes and downvotes of many prominent users are visible by clicking their username and then clicking "Liked" or "Disliked".
That is all.