It really sounds like you need to dive into firewall rules. Generally you lean on your firewall to allow and restrict access to services. Probably the easiest place to start is to setup pfsense/opnsense since it has a really clean interface for setting up rules. Proxmox's built in firewall is nice too, but configuring the firewall per VM would probably get annoying and difficult after a while
And as you learn more about firewalls learning how subnetting works will allow for more efficient rules (for example, if you have 192.168.0.0/23 192.168.2.0/24 and 192.168.3.0/ 24 for your networks that you're allowing traffic to/from you can just enter one firewall rule for 192.168.0.0/22 rather than 3 separate rules)