That sounds like Cloudflare is giving you certificates intended only to be used for talking to Cloudflare.
You might be able to do it if Cloudflare sends a different SNI. It's probably better if you get real certificates from Let's Encrypt and just use those.