Have LLMs fundamentally actually changed or improved to resolve problems like this, or have the big models been aggressively tuned / trained on specific examples so it’s less embarrassing?
Because I’ve noticed that Google’s stuff always seems to generate code to do counting tasks, even when not asked for, so I wondered if the LLM itself even generated the answer or it’s just being instructed to use a tool? (Which IMO is a workaround)
I don’t consider it a workaround for an LLM to delegate arithmetic to a calculator.
So yes, they probably spend half theor budget trying to keep suspension of disbelief.