I had a stock Debian install actually rename the device for my NIC when I changed GPUs. You should double-check if your NIC has the same entry in /dev with and without the GPU. After I changed the name in some
config files the NIC worked fine with the GPU in, it could be easy as that.