I'm not sure if you're able to pursue citizenship, if it's something you're after, but in terms of the "unique" accent- you don't owe anybody an answer if they ask "where are you from?"
You can tell them you're English. You can tell them you grew up in England, truthfully. You can tell them English is the only language you speak. You don't owe anyone your backstory. If you're feeling petty or vindictive you can lie and say you have a speech impediment. I know their intentions are likely good but by inquiring about your accent, they are othering you. It's up to you how you'd like to respond.