It does sound to me like ingesting all these different formats into a normalized database (aka data warehousing) and then building your tools to report from that centralized warehouse is the way to go. Your warehouse could also track ingestion dates, original format converted from, etc. and then your tools only need to know that one source of truth.
Is there any reason not to build this as a two-step process of 1) ingestion to a central database and 2) reporting from said database?
The messages were sent in 2017. They weren’t reported to Twitch until 2020, at which point they reviewed and took action - that was the whole “Doc got banned and we don’t know why” saga.
Seems if the messages are sent in an inherently insecure fashion, all one would need to do is set up an instance that purposefully does not filter out all the things it’s supposed to be kind/competent enough to filter out, and boom it has everything.
It’s UID/GID 10000 on the host because you are using an unprivileged LXC container. Unprivileged means that “root” inside the container (which is just a user space of the host with access restrictions) is user 10000 on the host - this is so that files and processes inside the container don’t run with the real UID zero, where they could plant a malicious file, or run a malicious program that escapes containment that ends up with root access on the host.
Quickest way to make this work over samba is to force user 10000 and force group 10000. That way everything connecting to Samba would see the files as their own.
Honestly the better solution is to make your software inside the containers run with a local non-root user (which would be something like 10001) and then force samba to use that. Then nothing is running as root in or out of the containers. Samba will still limit access to shares based on the samba login, but for file access purposes it will still use the read/write levels of your non-root user (because of the force- directives)