The relevant root is Proto-Germanic *walhaz. If I got it right it was used by PG speakers first to refer to a specific Celtic tribe, then other non-Germanic Europeans. (Proto-Slavic borrowed the word but changed the meaning - from "any speaker of a foreign language" to "Latin/Romance speaker".)
Latin never borrowed that root because they simply called any non-Roman "barbarus".