Building a from-scratch IP-KVM/Pikvm board. Tips?

I have a spare SBC (Pine A64 LTS) that I currently have no other use for, and recently I got the idea of turning it into an IP-KVM. However, the software support for this board is a little middling, and the board's been somewhat overlooked by the community. That leaves me with no ready-made solutions for turning this board into...