I'm sorry, but come on. this is ludicrously naive.
who writes the system prompt? who's allowed to approve changes to the system prompt?
and those people...can't be lobbied or bribed? why not?
LLMs are a big pile of GPUs. they are not a magic genie. don't ask them to fulfill your wishes.
"it generated a complicated math proof" doesn't mean anything in this context, because the prompt was uncontroversial, it was "try to solve this math problem".
you have your big fancy government LLM. one person wants to prompt it "generate a proposed peace plan with Iran" and a different person wants to prompt "generate bombing targets in Iran".
those two inputs can't co-exist. you have to choose between them. the process of making those choices is called politics. you cannot simply outsource it to a chatbot.