This seems unlikely to me, given it'd increase costs and the response times would make it obvious.

The messages presented in the original source appear to be people expecting to be talking to a real person, likely on a dating app. The relation to AI is only speculative, and mostly in the direction of "my messages may be used to train a chatbot to replace my job of deceiving people" - which is plausible.

> That's why people pay for it over just downloading an abliterated model from hf with system prompt hacking.

I'd assume convenience, fine-tuning, and using a larger model than it's feasible for most people to run locally.