I think the tldr is that Gary Marcus has been hating on LLMs since ChatGPT came out, mostly because of the hype around them. His core theory is that pushing LLM tech just with more training is not going to accomplish AGI. He does have some essays with good writing (not this one), and he typically talks about how we’ll need different techniques to solve things like hallucinations.

I’ve read articles of his which made genuinely good points, and which go against the grain of what the big LLM companies are saying.

The reason there’s a lot of drama is that the LLM hype train (which includes some prominent people) really hated on HIM for saying anything negative about LLM technology, and he responded to that by keeping the flame war going for the past 4 years (as you can see in this article.)

So when any companies do anything that looks like using these other techniques (neurosymbolic AI, world models), he basically tosses a quick article out about how vindicated he feels. Because the companies were all like “attention is all we need” and “we can just build 4x bigger data centers and that extra compute will solve all of our problems with more training,” and he was like “that’s BS”

So, I really don’t mind him showing up, because we get do get much BS on here from the AI companies too. So… Gary Marcus is at least a balancing kind of BS in a way. (For example, it’s hard to trust anything Anthropic says about Mythos, because they have so much money riding on it being insanely capable.)

But that situation isn’t ideal. What we actually need is more thoughtful, critical research which is NOT tied to impossible business goals. And that doesn’t describe Gary Marcus OR OpenAI/Anthropic.