I don't think others are threats, I just don't want to hear them arguing or farting or have them in my way.
Edit I've lived in a cheap apartment before and could literally hear when my neighbor took a shit. I've also lived in a condo, and while higher end, I could still hear my neighbor open and close their front door.
I would much rather hear the birds in my rural yard than hear people moving around
Edit edit I have lots of people in my life, and have friends at my home, or am at their home multiple times a week. I would not describe myself as isolated or lonely, just choiceful about what noise and space pollution I allow in my life