I keep post scores hidden so getting downvoted isn't much of an issue for me because I won't know unless I go out of my way to check.
The main issue for me is that I can reliably guess what kind of replies certain topics get and what is popular and what is unpopular. It's kind of like having a friend who never says or does anything that surprises you. Why even bother asking for their opinion when you already know what they're going to say.