Could be that your phone lacks the necessary certificate authorities. I don't think I've seen this happen on anything newer than Android 7 before, but maybe some root cert expired that I don't know about.
If you haven't already, try a reboot. The WebView server may be initialized once, and if you've just activated the cert in Magisk then you may need a reboot to activate it.
If not, I'm guessing the app explicitly does something to only use a particular system trust store.