99% of people you know will have absolutely no problems with it. The problems are only in your head. Just simply add a "if not, that's absolutely no problem :)". And be graceful when rejected. That's why I added "in private", no one of the friend group has to know, it will allow them to save face.