Looks like a Monero design to me 😄

I don't know how experienced you are, but training a model with additional data is gonna cost you time and money. Maybe RAG System is what you are looking for. When you compare a llm whit a library, a rag system is a bookshelf with your own documents in front of the library. After a prompt, the system first looks for a proper answer in the bookshelf, then in the library.