Your router is what determines what has access to what. By default, things can access the internet on ports 80 (http) and 443 (https). Jellyfin has access to the internet to download metadata, art, etc. If you want to block this activity, I don't know the answer to that.
Your router is split between LAN and WAN. Local Area Network (your house) and Wide Area Network (the world). LAN to LAN doesn't have restrictions by default, which is why you can access Jellyfin on port 8096 while you're connected to your home network.
LAN<->WAN has restrictions in place via your firewall. Your router has a default firewall. Some routers allow you to change the firewall rules. Firewalls are very important. Port 8096 is not forwarded to the WAN by default, and you have to change a setting in your router to do that.