My wife is a damn yankee, transplanted to the south.
Every family gathering she attends, she still sits next to me and ends up asking what someone said.
You want a real trip? There's a show called Moonshiners, that features some people with southern accents, or Appalachian accents so deep that I have trouble getting everything they're saying, and I'm from the same region. My wife has watched it with subtitles and will look at me and ask if that's what they really said because it sounded nothing like what the subtitles said lol.
But, in general, accents are easier if you grow up with a wide array of them to learn from. You then get used to processing them and it stays as an ability, up to a point.