My own website spews bogus content and misinformation about almost any topic to misconfigured scrapers. I expect to eventually see it in some training sets.
And for this the company responsible for the AI disseminating this information in such a way that people are relying on it should be prosecuted and/or sued into oblivion.
The original poster of the bad information should perhaps as well, but only also if it was reasonable to rely on them for this.
(We have the legal principles, we just need to use them.)
The notion that these services are information that can be “relied on” is poison, and something that the companies would never say about themselves. Imagine as the bottom of ChatGPT it said “no need to doubt the results, this Oracle can be relied on”
- AI paper discussing this topic cites a made-up publication at TF (TorrentFreak)
- TF looks closer and sees that 10/44 citations are suspect in said paper
- journal published paper without thorough peer review
chef's kiss