If you’re ever asked to roll with someone, and you have to convince yourself to say yes, you shouldn’t do it. Trust your fear.
Don’t be rude about saying no, but don’t worry about explaining yourself.
If you ever ask someone to roll, and they say no, move on.