Not really true. If you exclude those other factors, plant-based diets still come out on top. You should check out studies on the Seventh-Day Adventists like this one, as it's fascinating stuff!
There are simply too many harmful compounds in animal flesh which aren't compatible with health in humans.
But overall, plant-based is just better in every way: cost, land use, water use, climate change, health, longevity, chronic disease risk/prevention, food security, etc., etc., etc.