With AI, there is a strong incentive for the employee to churn out the report quickly and take Friday off, since no one is going to read it anyway. Maybe they still skim the report and make sure it looks reasonable, but they don't catch the smaller details that are wrong, and more importantly, they don't think deeply or critically about the issue at hand since the report looks reasonable at a glance.
"Show me the incentive, and I'll show you the outcome" - Charlie Munger
My guess is that reporting processes will change and adapt once everyone realizes how useless the old system is in an AI world. A long report used to signal "we thought about this, so rest easy". Now, a long report doesn't signal that. It no longer serves the function it once did.
The scary part isn't that the AI hallucinated - it's that the policy went through without anyone fact-checking the source. Makes you wonder how many other decisions are being made based on unverified AI outputs.
Critical reminder: AI is a tool, not a replacement for due diligence.
Malice or agenda can be real. With AI it is incompetence but before AI real malice was a possibility.
Yes.
> With AI it is incompetence
or malice, or agenda. For example, in the linked article the West Midlands Police had an agenda to find evidence to support a ban, they asked an AI to find evidence to support a ban, and the AI obliged and "found' evidence to support a ban.