Maybe I need to reread his book, but I think of System 1 as "fast, reflexive, instinctive" mental activity, and System 2 as "slow, deliberative, rational, verbal" thought.
So exactly inverse to this article, I characterize LLMs as 100% System 2, and would say that what's missing in AI is the more "embodied" System 1 intelligence.
My current thinking is that the experience of consciousness is closely tied to that embodied, emotional side of our nervous system, and is 100% lacking in LLMs. Anthropic people will say "the models have neural correlates of emotion", and that's fine, our cortex also has neural correlates for emotions and for much of what happens in the cerebellum and brain stem. That doesn't mean that that's where the primary function is implemented.
I've read Kahnemann's book and how I've seen System 1 / 2 typically used in the context LLMs often makes them seem like poor fits to what's being referenced. The concept was created to broadly group types of human reasoning but it never struck me as rigorously defined enough to be a technically precise distinction. Applying what was a mildly useful directional thumbnail referencing human thinking styles to LLMs, which can mimic human output but via completely different processes, quickly goes off the rails. As we get more detailed and precise, the System 1 / 2 distinction seems to increasingly diverge from how LLMs behave.
A quick example is how basic Flash-type chat models without 'extended thinking' or MoEs will keep suggesting different solutions instead of stopping after a few failures to step back, gather more context and diagnostic data and evaluate changing approaches or questioning the question. When I do that, the 'interrupt' is often a 'System 1-ish' intuitive feeling like "I suspect I'm barking up the wrong tree" then, once stopped, I'll switch to System 2 to analyze. But in LLMs that "interrupt" behavior typically arises from adding more System 2-ish thinking. N=1 but that's not how I usually 'get there' and is orthogonal to what the System 1 / 2 model suggests.
System 2 is slow because it is sequential. Every output depends on all the prior outputs. Like a transformer. (Even if this comes about as the result of massively parallel processes, both in the brain and in the model.)
System 1 is more of a parallel thing, operating on the output as a whole rather than in sequence. Much more like diffusion.
The terms don't seem to translate well, possibly because they were the wrong framing to begin with. Possibly also because the billions of years of progressively complex evolutionary constraint that brought about animal cognition does not seem likely to be reproduced by running backprop on human language artifacts.
Current LLMs are the opposite of slow and deliberative, though? With the current transformer architecture, it seems like its function is more or less "look at this text and make a guess which word comes next", which is very much a fast and intuitive task. Any reasoning comes from CoT, which is more analogous to System 2 thinking. But that System 2 thinking can't function at all without relying on the representations from System 1 thinking (which are formed in pretraining).
So you could argue System 2 thinking is some part of the LLM, but it can't be 100%.
Wikipedia implies it’s just some pop-sci (psychology even, so extra pop) book from a few years ago. A book heavily criticized for relying on shoddy science. Why are the terms used to describe anything by anyone trying to sound serious or scientific?
I read this book published in 1974 called 'inner game of tennis' that has the same metaphor. Its one of the book i read every once in a while even though i dont play tennis anymore.
It's like a collection of the obvious, the wrong, and autobiography of someone you haven't otherwise heard of.
A good example is Kahneman and Tversky’s work on the conjunction fallacy. We now know that “stereotype threat” is not the only reason somebody might make a mistake of the kind they were studying. And over the decades, better experiments introduced better controls and more nuanced analysis. That said, people still make conjunction fallacies, but the effects are not as pronounced as originally believed. Is this bad science? Absolutely not! The finding is, in fact, pretty robust; we just know that there’s more at work.
So how do you count that? Is that a “failure to replicate”? Technically speaking, yes. But it is not bad science, and more importantly, it’s not a standard that we hold even “hard” scientists to. Nobody is saying that Newton has been discredited because of general relativity.