They can excel at information retrieval.
They often fail at logic and reason.
Predicting the next word is basically guesswork. Not the sort of thing any responsible person should rely on to make important decisions.
Doing so leads to liability issues in many business and professional environments. The legal system will have significant influence here.
https://pub.towardsai.net/the-air-gapped-chronicles-the-cour...
Would you dispute that they're much better than two years ago? I, personally, am quite impressed at how quickly they improved. It's not yet at the level for independent decision-making, but if the rate of improvement holds, I wouldn't be too surprised if we got there in the near future.
I will readily admit that the guesswork has improved.
Will you admit that guesswork remains the foundation of what they do?
Guesswork will always be suspect.
Removing it will require a fundamental redesign and may not be achievable within the constraints of current hardware and economics.
My guess is any significant breakthrough will likely involve quantum computing and renewable energy.
Even though guesswork remains the foundation, the improved training can help discover patterns, backtrack, etc. Basically, explore the problem space by "guessing" through it
This is where we disagree.
"Guessing" (aka trial and error) is not a "legally" responsible way to make decisions in many cases.
You are specifically talking about independent decision-making.
I have been focusing on "problem-solving".
Tbh, I have no opinion on the legal framework. I'm sure most decisions will be supervised, at least for a while.
I wouldn't be surprised if the tool vendors aren't eventually forced to rename it and/or limit it's use in their terms of service due to legal liability.
A good analogy is Tesla's "Full Self Driving" which was renamed to "Full Self Driving Supervised" --- an oxymoron.