These stories always focus on a vulnerable person, but what is the chatbot optimizing for?? It wants you to keep talking. Longer sessions, more training data, better engagement numbers. A therapist has a professional reason to make you need them less. ChatGPT has the exact opposite incentive baked in
Even if theres no business purpose, tge function of these models is to produce tokens. That alone skews the intelligence capabilities.