No. You’re not a source of terror. But people don’t know that. They just see a man. The issue isn’t with you, it’s with common experiences women have had with men.
There are so many bears that don’t hurt humans. But humans carry bear spray just in case the one they encounter is dangerous.