That's significantly worse privacy-wise, since Google gets a copy of everything.
Obviously, but I still haven't gone through all the things I've ever signed up to and changed my email to the proton one. When I sign up to new stuff I use Proton, this is a necessary step for transition... And one that is likely to stay in place for a very long time since I'm going to keep procrastinating it.
Unless you're using proton mail anonymously then you don't need to consider the recover email as a weakness.