Currently being vegan or vegetarian is a choice of privilege. An healthy and varied diet becomes more difficult and expensive, when you start removing dishes from your pallet.
So it becomes coupled with a status symbol, instead of being the default way. As long as people call themselves "vegan" or "vegetarian" because of their choice (people being vegan or vegetarian because of mental or medical issues, is different case), they highlight that status over "normal" people.
If people are just not eating meat or animal products for whatever reason, without trying to use labels like "vegan" or "vegetarian" to highlight their status, then that is fine and a personal choice.
Creating societal change, to make vegan or vegetarian the default position, will also lessen the status of the vegans and vegetarians, that use those labels as such. So they have incentives to not produce a political or societal change.
Vegans & vegetarians should go on protests and lobby to make vegan food cheaper and easier than meat, so that it becomes the default. If they don't do that, and still call themselves vegan/vegetarian then that might imply that it is all about showing their status, and people don't like that.
Consumer choice is a privilege and not about creating an effective societal/political movement. They should not be used as a status symbol.
(Disclaimer: I eat meat and animal products very infrequently, only when my body demands it. I am also thankful for all vegans and vegetarians, because they gave us more interesting options in stores and restaurants.)