I'd be interested in seeing how they compare to old fashioned neural networks terms of power, time, and accuracy.
I think the speed gap between the LLMs and narrower neural networks is similar to the gap between human workers and the machines that replaced us in the industrial revolution. Maybe there will be a second industrial revolution where the AIs design specialized tools so fast and powerful that they replace themselves in most tasks.