Neither is proprietary, but Google doesn't document how they implement either of them. Signal's normal protocol is intended for use with a direct client-server connection, but Google wrapped it into a different message transport (RCS, which is just a predefined set of HTTP calls with lots of XML).
They do some sort of base64 encoding to transport the messages themselves, but they don't document what key servers are in use, how registration works, or how key material is exchanged with those servers. Do they use RCS to communicate with key servers? Do they require a direct internet connection? What about video calls, are they done through standard RCS or do they add encryption there? Do they exchange the encryption keys for video calls through RCS as well? How do keys work if you deregister from Google Jibe and register RCS with your own carrier?
Lots of open questions here. All stuff that can be reverse engineered. I'm sure plenty of companies are trying to figure out how Google RCS works.
I hope Google switches to full MLS (if they haven't already, they won't say!) and implement MIMI for secure cross platform messaging once it's been finalised, but there's no way of knowing right now.