I posted this because it seemed timely given that 'CGamesPlay and I were discussing this exact kind of system two weeks ago on the thread about why kernel.org's cgit (git web UI) hosting system was becoming expensive to operate due to LLM scraper load: https://news.ycombinator.com/item?id=49504674