The way I explain it to people who say that to me is that it doesn't even have to be something illegal or sketchy.
Everyone...and I mean everyone has something about themselves that, were it common knowledge, would change the way your friends and family look at you. Maybe for the better. Maybe not. And if you, for whatever reason, don't want that they be known, then so be it. That's totally your decision, not the governments.
If you want to let your freak flag fly proudly, go ahead. You have my full love and support. If you want to fly your freak flag privately and not have it be common knowledge. You also have my full love and support.
It's not about "having something to hide". It's about the you that you choose to present to the world. And that is a fundamental right.