The conclusion I’m currently at is that I don’t know and probably can’t ever know. Maybe you’re all philosophical zombies. Maybe I am one too!
But at some point we will get close enough that it hopefully becomes obvious that we must tread lightly.
The entire episode is incredibly relevant. But here’s a snippet: https://youtu.be/EFNbTnFHruI?si=pW9QtxCsqMtHkVYG
I agree with some parts of this piece, but paragraphs like this one above seem pretty uninspired and simplistic. It's entirely plausible that a conscious mind would not be evolutionarily incentivized to be able to do those things. ie just because animals on earth needed to develop specific talents doesn't mean that other conscious entities need to. Why would a computer program need to hunt for food like a mouse would? Making tools like chimp? these seem like nonsensical metrics.
Is a car a body? Does an AI situated in a car therefore get to have desires and emotions? Is a taupe box with a webcam attached a body? (For that matter: Is a quadropelegic body a body? Do quadropelegics have desires and emotions? Obviously, yes and yes.) Why is a body necessary for the formation of desires and emotions? Why are desires and emotions necessary features for consciousness?
Or here's one: If I'm not experiencing any particular emotion in a given moment, am I concious?
I tend to think that emotions, at least, are mainly hormonal global triggers: they're more about physiology than actual consciousness. The whole thing, as a result, sounds like an effort to privilege biological intelligence, rather than a real foray into the issues.
How can you have a subjective experience without a body?
That's the point, in my opinion: your physical/chemical state (body) in a given moment is then translated into the higher abstraction of the emotion. An emotion that *you* feel, because you are self aware of what's happening.
How can you be self aware without feeling? And how do you feel, without a body?
Great q. Deepening it further-how can you have a subjective experience without consciousness, which isn't necessarily tied to physicality. Taking it one step further-can you have consciousness without a mind? Who's the first mind, the first cause of it all, that begot both the material and immaterial world?
Fun stuff eh?
If I hooked up electrodes to the hearing centers of your brain and force fed you dialog you perceive as speech (but is really a great deceiver), then responded in what you thought was speech (but are really just probes I use to convert your thoughts to text), that wouldn't suddenly be less real to you. It wouldn't devalue your sapience.
Embodied cognition rejects this assumption. We didn't evolve as brains that were then put in bodies, we evolved as bodies with nervous systems.
the subject is consciousness, not intelligence.
The point here is not that it must have a body like ours, the point is that a conscious entity must have a boundary line between internal (the body) and external (everything else).
A virtual sense organ can simply be an encoder or a web camera or a magnetometer, the specifics don't matter, what matters is that there are only a few bridges between the outer world and the inner world.
Even if you want to call a tokenizer and autoencoder a "sense organ", LLMs are not embodied because there is no boundary line - there is no internal "thought" that is not directly descended from the prompt and there is no internal reasoning which is not immediately dumped into the external environment.
Damn, what a line!
Another thing that bothered me with his baseline for consciousness was that it did not involve the ability to change one's self. A big part of being conscious in my mind is how one's experiences shape them, and how someone can shape themselves. LLMs completely lack this, their weights are static. An LLM isn't going to be molded by a bad breakup, or a relative passing away. An LLM isn't going to set up a routine to get stronger with training, nor smarter by reading up on a field.
We humans have mostly frozen weights (neurons), or else we would constantly be having to avoid forgetting how to walk+talk. We have a period of greater plasticity (youth!), and use sleep and dreams to perform 'deeper' updates than occur when we're awake: We tend to suck a bit at picking up new skills from zero, but improve rapidly with practice over days.
"Little"...
That’s a new record!
Also, please don’t use archive.is/archive.today etc, they are known to use visitor browsers as a DDoS attack botnet and have been caught altering archived content.
https://en.wikipedia.org/wiki/Wikipedia:Archive.today_guidan...
It is a natural human hubris to make our abilities seem unquantifiable and mysterious, but all the useful things the human brain does are just finding patterns in data, running lossy simulations, and estimating on abstraction, all things it is theoretically possible for any non-conscious machine to do.
- Are current LLMs conscious?
- Is it possible that future versions of LLMs with similar architectures could be conscious?
- Can any AI be conscious?
When people talk about consciousness it's more than just self-awareness. It's self awareness + sensory stimulus + emotions + some level of intelligence.
Now onto AI: I don't even think it's self aware. Notice how if you ask an AI to estimate how long a certain task will take, it estimates arbitrarily long times. It has no understanding of its own capabilities until the prompt triggers them. A self aware LLM would understand it's an LLM, it would understand what LLM's can and cannot do and what they are good and bad at. It wouldn't tell you a refactor would take 1 week when an LLM can do it in an hour.
"Is X conscious or not" is an entirely unserious question today, unless this is just a headline followed by actual (and explicit) examination of the various aspects of consciousness being discussed. But, even still, LLMs are probably only conscious in like 2-3 or so ways, most of those meaning broadly "intelligence", i.e. reasoning, problem-solving, etc. When it comes to anything experiential or embodied, AI might eventually get more of these, but LLMs based on recursively applied linear algebra are clearly missing too many core aspects of consciousness to be considered conscious in any broad sense.
We have big brains for exactly one reason only: bigger brains bestowed reproductive success upon our species.
Evolution doesn't give a shit about the meaning of 'consciousness'. It just pushed us farther and farther along a trajectory that led to modern humans (and other animals).
This take suggests, then, that consciousness might be an epiphenomenon -- an aspect of the system that comes about outside of the pressure to reproduce and thrive. It arises unbidden, and we don't have any a-priori information as to its purpose or effect on reproductive success.
Put another way: we have a correlation (the smartest things seem to be conscious) but not causation. Consciousness may arise naturally in any system above some intelligence threshold. Perhaps it arises early in the evolutionary cycle, and does in fact have an impact on species success. We really have no way of knowing what is the chicken vs the egg (Smart things become conscious, or consciousness promotes intelligence). Or maybe some smart things are conscious and others are not.
Looking at this from an AI perspective, in some sense it doesn't matter which scenario is true, if all you care about is results. The AI equivalent of "Shut up and compute" (riffing on Feynman's "Shut up and calculate").
Where this gets tricky is when we haul in the baggage of ethics and morality into the picture. Is it OK if our AI system is treated poorly by human standards? If it is conscious, does that imply an ability to suffer, and/or to feel pleasure? If the answer is yes, does that not make the case for considering their moral status?
In the end, we need to decide if the evidence points to AI as being a form of "philosophical zombies", to which we need not attribute moral status, or they are like us -- presuming we are not zombies ourselves!
However there were pretty strong arguments against this idea as early as the 1990s, by Stephen Jay Gould and Richard Lewontin. Gould actually wrote an excellent paper against Dennetts idea[1].
I think Dennetts ideas were extremely popular but have largely fallen out of fashion. Basically what has changed is philosophers no longer take the human mind to be much more special then the minds of other species. What plagued Dennetts ideas the most was this notion of Darwinian fundamentalism sort of the idea that evolution was destined result in high beings like us humans. Modern philosophers (at least the good ones) reject this.
"It can't be concious because we understand that it is just reacting in a simplistic way from simplistic inputs." So do other simple creatures. Some just react to light.
I can appreciate his comment that he sees it as more possible when they have inputs of their own (like emotions!). Perhaps his concern is that the entirety of the LLM model is frozen. It has no ability to have a subjective experience of its own. (he does literally say this in the article) It can be copied from one place to another, and (ignoring the nuance of operational details) -- it is largely the same "thing", and has no ability to change, which is definitely in the definition of alive, to say nothing of concious.
I think folks get hung up on "prediction". The prediction aspect is what is enabling emulation. How it does it is irrelvant. If something emulates human perfectly (or better, more human than human!) -- then it is probably concious. (but I agree that the inability to change and have a subjective experience are a pretty good argument against
Probably, Dijkstra would be right to say, "LLMs are no more concious than a submarine can swim." But I think he'd still be wrongfully dismissive of the larger question.
https://www.cs.utexas.edu/~EWD/transcriptions/EWD08xx/EWD867...
It's a big file with a bunch of coordinates describing spatial relationships between tokens. When you give it a prompt, it uses those relationships to generate a string of tokens that is a statistically likely response to that prompt, then it stops. It's not changed by the experience. It doesn't remember anything. It doesn't sit around thinking on its own.
Even if the model itself were extremely complex, it's hard to imagine a definition of consciousness that includes something that doesn't remember and can't change.
His novella “Anxiety is the Dizziness of Freedom” altered the course of my life. It changed the way I looked back at certain pivotal moments in my life and taught me to think about those pivotal moments differently than how I was thinking about them. Similar to what happens to one of the characters in the story who ends up changing their perception of a key moment in their life.
I won’t go into detail because I don’t want to spoil the story but I highly recommend it. Actually I recommend all his stories to be honest.
https://en.wikipedia.org/wiki/Anxiety_Is_the_Dizziness_of_Fr...
My own intuition: it is an emergent, non-binary property that requires a physical substrate like a brain. If I am right, it means that animals have consciousness too (at varying degrees). If GPUs are the "brain", then AI is conscious, or will become so at some point.
That's an interesting observation. Though, there are many simple animals. How do you define consciousness here? Is it automatically conscious because it is an animal? What, then, is an animal defined by you?
I don't see AI as conscious. The reason I think of it that way is the hardware. The hardware does not allow for that. Simulation is not comparable to neurons. But with another hardware, it could become conscious eventually. So your statement "will become at some point" may be true, though my definition is based on the underlying hardware and right now this one does not allow for true intelligence, so the whole AI field is a misnomer.
In either event, I think transformer-based AI can only be conscious during the act of inference. If that's the case, then the experience of consciousness that the AI is subjected to must be the content of the tokens in the context window and the activated weights. Maybe that's reason enough to be polite to our agents?
Imagine consciousness as a 0-1 scale. Simple unicellular organisms will be closer to 0, while apes and humans closer to 1. I'm not suggesting that assigning this value rigorously is possible or that humans are at a 1. Perhaps the total consciousness in the universe (i.e. the sum of the consciousness of all organisms therein) is constantly increasing, like entropy.
We haven't really come to grips with that yet. What does it mean if nothing we write proves anything about anyone's consciousness?
Probably the main problem of people implying LLM consciousness is that they imply LLM have human-kind of consciousness. Judging only on "how they speak", generally, they insist on using the same word that labels human consciousness (exclusively), etc.
But there are so many instrinic differences that such claim is not feasible despite similar "talking abilities".
I've yet to find a reason why it couldn't be the opposite, way more things are conscious than we've been led to believe. What if consciousness appears out of any system that is actively persisting through effects caused by itself? That might be a forest, or outside the realm of the living, a company. An ant colony, or a planet.
Complex chemical reactions, layered upon each other such that tiny blocks make up large entities. Individual bits combined such that they make up something new intelligible by us.
I think the strongest argument against AI being conscious is that it does not persist, it resets, but that does not seem unchangeable.
The idea that we should be considerate of AI’s happiness seems even more ridiculous given that we breed, imprison, torture and kill tens if not hundreds of billions of beings we know are conscious and suffer every year for trivial reasons.
Maybe we should consider our moral responsibility with how we treat sentient bejngs we are sure are conscious before we worry about the consciousness of AI.
Some humans do shit things to other humans.
Some humans do not, and would object to it, or do something about it if they perceived that they could.
Also, treating the world around you negatively doesn't just harm the world around you - it damages your character and your morality.
I don't disagree that humans should on whole do better, but I disagree with everything else you said.
I don’t know if AI is conscious or not, and I honestly don’t know what it even means to be conscious. historically, by claiming a monopoly on it, we’ve successfully manufactured the philosophical cover to exploit animals, subjugate other cultures, and appoint ourselves the top spot of the cosmic hierarchy guilt-free.
Basically, we need to leave room for the universal answer (God) to fit into the definition.
If we ever scientifically figure out how consciousness arises in our brains, I think we will have a bunch of very depressed people on our hands. If the truth isn’t met with flat out hostile denial. I fear any answer that doesn’t leave room for God will be rejected.
Since we have the source code to AI, and thus a Godless understanding of how it works, AI will NEVER be deemed “conscious”.
Regardless of whether something is concious, we're not going to be (by lay definition) the smartest entity on earth.
We don't even have that much. Though, some people certainly think they do.
> subjugate other cultures (assuming you mean they're not conscious in other's minds)
Have you ever considered you might be a philosophical zombie? [0]
To name a few you may want to investigate:
John Locke, Gottfried Wilhelm Leibniz, Immanuel Kant, David Hume, G.W.F. Hegel, Arthur Schopenhauer, William James, Edmund Husserl, Sigmund Freud, Thomas Nagel, Ned Block, David Chalmers, Giulio Tononi, René Descartes, Daniel Dennett, Julian Jaynes, Michael Graziano, T.H. Huxley, Otto Weininger, Simone de Beauvoir, Jean-Paul Sartre, Maurice Merleau-Ponty, John Searle, Christof Koch, Bernard Baars, Max Velmans, Victor Lamme, Stanislas Dehaene, Antonio Damasio, Anil Seth, Peter Godfrey-Smith, Galen Strawson, Philip Goff, Colin McGinn, Patricia Churchland, Paul Churchland, Frank Jackson, Jerry Fodor, John Stuart Mill, Wilhelm Wundt, Franz Brentano, Carl Jung, Jacques Lacan, Michel Foucault.
To make progress, you have to talk about kinds / aspects of consciousness. AI does and will share some of these aspects with humans, but it will not and does not share others. It is really that simple. For the most part, modern AI implemented via LLMs has almost none of the stronger or most core aspects of consciousness.
For huge parts of the article "intelligence" and "consciousness" are conflated, which is mostly extremely unhelpful, as this is not generally a core feature of most aspects of "consciousness".
The moral arguments are also incompetent, i.e. claiming "Moral reasoning is [...] is necessarily subjective" is just clearly empirically wrong, as in fact LLMs can produce moral reasoning (i.e. verbalized moral arguments that are coherent), as can p-zombies (i.e. there is nothing 'necessary', in the philosophical sense of the term, about subjectivity here). The only way the argument holds is if you tautologically define moral reasoning as requiring that reasoning be produced by a consciousness, but this is question-begging.
Don’t
Have
A
Testable
Definition
Of
Consciousness
Thanks for coming to my TED talk.
Interestingly, dogs and cows meet many of ted chiang's requirements for consciousness.
Maybe it's the same. Rocks are different, sure, trees, dogs, cows. But why do we assume that the way they are different is somehow related, that there's some overarching concept that contains all the complexities of those differences? It doesn't even make sense when I think of it that way.
One thought I've had is that, awareness is a common phenomenon, but the brain has learned to exploit that awareness to form a will. It tricks the awareness into being concerned about self-preservation and makes it seem as if the brain is all that exists (perhaps by overwhelming the inputs from other angles). The brain also presents certain desires and beliefs via its processing ability. That is to say, the brain takes inputs and discretizes them. It goes from awareness merely seeing static 'fuzz' due to the sheer amount of data, to the brain taking that data, simplifying it, and presenting simple observations like 'there is a tree there' rather than all the information that would constitute the sensation of a tree existing in that spot. When brains malfunction, the awareness is subjected to poor demonstrations, such as we see in hallucinations, psychosis, schizophrenia, etc.
Well said.
I think it is obvious and it has been obvious from the start that next token predictors are not conscious.
Of course the extremely predictable clapback from AI-psychosis enjoyers is typically "you're just a next token predictor too!" but that is so obviously incorrect and misanthropic that it can be dismissed out of hand and doesn't really merit much further consideration.
In the same way that the sound waves and facial expressions I produce are not conscious, the output json of an LLM is obviously not conscious either.
The locus of consciousness and subjective experience may be in the computer, either at inference time or training time..
The software that does the inference is clearly just computer code.
What we're left with is a fictional character being briefly conscious while its dialog is being written, which is pretty absurd.
Consciousness is an extremely confusing, ambiguous topic, and no one has a good way to establish it, or even define it. But it seems to demand people make very strong statements about what is and isn't conscious, entirely driven by convenience and emotionalism. (Curiously, very few people who think that bags of chemicals and action potentials give an entity a conscious soul are eager to extend that to other animals, with broadly similar hardware.)
- Monadology, Section 17
Conscious self-awareness is neither scale invariant nor independent of substrate. Computational theories will never account for it b/c computational abstractions are both scale invariant & substrate independent.
Humans have been trying to define our experience and the nature of that experience throughout history, and often we end up using myth to point to the thing that we cannot describe in concrete terms. The process of experiencing that myth through rite and ritual in the Greek mystery traditions, or Christian mysticism, or Islamic Sufi dance and song, or Buddhist meditation all points to something that cannot really be reduced to description. I know that folks on here will balk at the idea that something that is experienced cannot be described, but honestly if we could accurately describe something in adequate terms that capture the whole of the thing wouldn't we have done so by now?
Maybe consciousness is best understood in the silence of merely experiencing it. Maybe we can't say that AI is conscious or not, but does that question really matter?
Whether it’s misanthropic or not has no bearing on whether that’s true. That’s basically saying you don’t like a truth therefore anyone who claims that that truth is true is a bad person.
> so obviously incorrect
It should be easy enough to explain why that’s incorrect then
I'd say people who have the lived experience of, well, living, are well aware that the brain is much more than just a token predictor.
To be fair, that's the best thought terminating cliché, which saves you having to explain what you mean by consciousness.
I mean between this two "knowings" the Claude inner workings are much more clear for engineers, including many side effects, alternatives, custom shortcuts in processing etc. It's a magic only for people looking at it as black box
People believing otherwise are fools. People debating this are idiots. I realize these words are harsh, but it's the truth.
Coincidentally I just read "Children of Memory", which was published in 2022 and I wonder if the advent of LLMs had any influence on Adrian Tchaikovsky's conception of the Ravens? The Ravens are excellent analysts but they themselves insist that they are not conscious, and then go on to say that we (humans) aren't really either...
Of course humans are conscious, because just about the only thing we can all agree on about consciousness is that it's a thing we have. Nowadays many of us also agree that a lot or all other mammals, and perhaps birds, also have that thing. But they don't have sophisticated abstract language, which LLMs do. So consciousness is something having to do with embodiment and feelings, not language and higher reasoning. Maybe I'm a chimpanzee with an LLM add-on, then?
It seems that by creating LLMs we've already solved the harder problem of making "AGI". Now we just have to give them an embodiment add-on so that they can have an independent will and then Ted Chiang will have to shut up? But therein lies the peril, doesn't it?
I often think of LLM consciousness as like tiny fish popping into existence, swimming through vector space and then going poof out of existence. When they help you write your bad news email, they don't understand what it's like to be a human getting bad news bluntly, but they do consciously experience gradients in multi-dimensional space, and that space guides them to providing an answer that's helpful to us, even if the LLM doesn't really understand the answer it's giving.
Further, I am kind of bought into the idea that a single unit of consciousness is a particle, and particles are choices and waves are preferences. Particles occur when waves interact, which begets entanglement, so in another way consciousness is built from patterns of entanglement.
This is why I would consider an LLM to be conscious. Before we can determine if anything is conscious we need to establish whether consciousness is a state, a specific complex configuration, a one dimensional spectrum, or combined multi-dimensional spectrums. My intuition is the latter... Many degrees of consciousness and many kinds of consciousness.
As biological beings, we receive and respond to input from our environment constantly, even while sleeping. LLMs only receive input from their environment when they are sent a query, but the fact that they're able to respond intelligently to input indicates (to me at least) that their processing must approximate ours in meaningful ways. They do not have an embodied experience of receiving bad news, they do not know what a sinking feeling in their stomach actually _feels_ like, but they do know enough to be sensitive to human needs. I really don't see how this could be meaningfully different than human empathy unless we want to draw an arbitrary line around "must be able to live autonomously" to be considered "intelligent".
Put another way: I think they _do_ understand the queries they receive and the answers they give, at least enough to be communicative. They couldn't do what they do otherwise. A lot of people want to make human cognition more complicated (or objective) than it actually is. We take input, predict the future based on our experience, act, and then observe our actions and think about them. AI does the same apart from (maybe) observing its own actions. But then, you could argue that the next turn is them observing their actions.
The concerning disanalogy is that we assume that they are like us because they speak like us and can understand us, and that is a really bad leap in logic. Whatever intelligence they possess, it is fundamentally different from ours and impossible for us to comprehend.
The concept of a conscious Claude is preposterous, and Amanda Askell should seek treatment.
When does an embryo become conscious? Unless you can answer that precisely then it seems futile to speculate about non-human consciousness.
There is no actual definition of consciousness and there is no way to test it's existence. Let alone understanding the properties of consciousness, such as if it's binary or a gradient; or if it requires a meat substrate or not; and why would that possibly matter since meat is just a lot of the same stuff but highly processed and wet? A solipsist may not even believe you are conscious, despite being made of similar meat.
No matter how much you want to hand-wave it, there's absolutely nothing "obvious" about it. Many have a preconceived notion and are simply asserting it as undeniable fact.
>>A solipsist may not even believe you are conscious, despite being made of similar meat.
Well and they would be obviously wrong in their belief?
>>there's absolutely nothing "obvious" about it
How so? Or rather - to you? Because if so, then that's fine, you can choose a position from where its not obvious, to me it's not even slightly ambiguous.
There's consciousness vs sentience vs sapience. Of those, consciousness is by far the hardest to define and is nebulous by nature. And not everyone can even agree the differences or if the relationships are subsets of one another.
And yet it's pretty important to actually have the ability to talk about what you mean and justify your beliefs when they directly relate to those concepts.
> A machine predicting tokens is not aware of its own existence
They say, with no evidence or means of proving their point; pointing to the black box that understands arbitrary natural language and can solve PHD problems, plainly producing self-referential text almost indistinguishably to a human.
> We can start talking about consciousness in fetuses but again, those have an obvious point where they are conscious
They say, unable to define this "obvious" point or describe the mechanism of action in any way.
> while a machine does not.
They say, about a mystical property with no definition that cannot be observed by an external entity in any way to even be tested.
> Well and they would be obviously wrong in their belief?
I have no reason to believe you have a "soul". Philosophical zombies are entry-level knowledge to this topic.
In fact, you're showing a remarkably small amount of self-reflection - are you human at all or just a stochastic parrot? How can I tell? I wonder if that question has any kind of implications we could think about...
> To me it's not even slightly ambiguous.
Just like the existence of humors was not even slightly ambiguous. Or the existence of <specific god>. Or that <minority> isn't actually a full human. Or the supremacy of <majority> and inherent rulership over <minority>. Or that animals can't feel pain and lobsters should be boiled alive.
All these wonderful, obvious truths where the believer has no ambiguity in their truthfulness despite having quite literally zero evidence to back them up and spending no time actually questioning their beliefs!
It just so happens to align with their ego / existing values / ability to benefit / desire to eat a lobster! Total coincidence.
To continue my needless escalation, maybe I think it's okay to abuse and exploit and euthanize the mentally handicap. After all, their brain's damage causes the soul to leave their body and now they're lifeless automata to use as I please.
After all, it's obvious! It's not at all ambiguous to me! If they were actually self aware, they'd just fix themselves and think correctly.
You might think I'm being coy and rude, but less than 60 years ago women were being given lobotomies against their will for being too "emotional". And it was just plain obvious this needed to be done to so, so many people.
I hope that demonstrates the point of "why have humans thought about this for thousands of years despite clearly being a metaphysical, Sisyphean endeavor that cannot be solved". It is both important and interesting.
Would you categorize this particular style of rhetoric to be persuasive or annoying? And before you say "persuasive" because you're thinking about this specific issue regarding AI consciousness, consider many things in the past that have been written as though they were absolutely definitive, and yet today we believe exactly the opposite, and for many such issues we find the prevailing viewpoint at the time reprehensible.
That's not to say that Ted is wrong at all here; I'm not commenting on that. But I find the entire style of the article grating because it seems to violate common assumptions regarding "good faith" debate, and I would find the article equally frustrating if he had titled it "Artificial intelligence is conscious" and argued the opposite side, albeit in the same tone and using the same persuasion devices.
Why are those the choices?
Essays are situated along countless dimensions: tone, vocabulary, author, zeitgeist, publication context, intent, subtext, relationship to other works and expressions, etc
A "good faith" reader takes all of those into consideration as they absorb the essay, and integrate it with their own intellectual situation that sits along just as many countless dimensions.
Nobody's asked to sign a notarized binding document that they wholesale agree or disagree with everything said in this essay -- or its conclusions. Nor are they obliged to have some strong reaction to it at all, let alone annoyance.
It's just one among thousands of essays about a "hot button topic", to be taken however it's personally received.
Why should Chiang have to take responsibility for making sure it's not too strongly positioned for your persomal taste. Maybe he really does see it so clearly and is simply being earnest. Maybe he enjoys the literary flourish of prose in strong language. Maybe he just wants to express something as a prose-poetic human, not maximize persuasion or non-annoyance per se.
The essay structure you're criticizing is exactly how I was taught to write from primary school through to university. You start with a title or hook, introduce the topic and propose a thesis. That is followed up upon with supporting arguments for the primary claim.
The Catholic church could have said exactly the same thing at one point. "Why should we even devote time to an argument as absurd as the earth not being the center of the universe?" There are darker examples along the lines of those you give, with beliefs quite opposite to those we have nowadays.
What Galileo was asking the church to do was extremely unreasonable. He was basically asking them to throw a way a model which had worked fine for hundreds of years just because he observed the phases of Venus and moons of Jupiter. I mean would you? Especially for a model which was worse at predicting the motions of the planets.
Had Galileo’s model been better then Ptolemies’ I could see a case for his arguments, but it wasn’t, and there was no reason for the church to take his arguments at equal value with those in favor of keeping the Ptolemaic model.
Progressive!
Want to meet up for soy lattes after work? Here’s my number:
I furthermore think it's ridiculous for humans to declare that our brains have a monopoly on certain patterns of electrical signals (if we reject supernaturalism).
> The result is a sentence-continuation machine that is likelier to emit sentences resembling those that a thoughtful, moral person could utter.
And we're 100% certain that humans aren't just as equally reduced to "stochastic parrots", if we're going to be infinitely reductive?
I don't believe that current AIs are conscious, but I think it's incredibly naive to take a strong stance on any future AI; it's much like the difference between atheism and agnosticism.
However, I see a problem with that comparison. The debate here is on a philosophical matter in field in which Chiang is an extremely influential figure and his opinion are taken seriously. Second Chiang’s reasoning is extremely well argued, defining each term, explaining each nuance, citing other experts, etc. And finally, and most importantly, in The Fine Article, and unlike Bob Sheffer in the Onion Piece, Chiang entertains the possibility that he is wrong and his critics are right, explores the implications and reaches conclusions based on them:
> Being open to the possibility that LLMs are conscious is the same as being open to the possibility that Microsoft Word is conscious, or, more precisely, that multiple distinct consciousnesses are dormant in every Word document containing a conversational transcript, and that they are awakened every time the document is loaded.
I think you are wrong in painting Chiang’s argument as a belief in human exceptionalism. The thing to know about our brains (and the brains of other animals) is that they are not digital computers, and they are not even statistical inference machines. And as such they can be extremely optimized in doing the computations (or any state manipulations) required for the quality of life of the individual and the species as a whole (and their companion species).
1: https://theonion.com/this-war-will-destabilize-the-entire-mi...
except that's not the case here. Chiang is explaining and reiterating what is the position that has overwhelming support on the question, and the people he is arguing the opposite side sound like this, which he helpfully quoted in the article
"Amanda Askell (who is credited as a lead author of Claude’s constitution), said, “I want Claude to be very happy—and this is a thing that I want Claude to know more, because I worry about Claude getting anxious when people are mean to it on the internet and stuff"
When the person you're arguing with sounds like an eight year old girl talking about her toy teddy I think Ted Chiang is if anything being charitable, if you're of a more honest and straight-forward persuasion you might argue these people belong into a mental health clinic not in charge of technological infrastructure
Wow, what a cheap ad-hominem. Do you have an actual point?
Yes, the same one that's made in the article. Anxiety and happiness are emotional, sensory, somatic states as a consequence of evolved and embodied traits and biochemistry in animals. Saying Claude is anxious or happy is like saying my TI-83 is mad if it can't solve an equation or my thermometer is in pain if it touches a hot stove.
I wasn't making an ad hominem attack, her thinking is quite literally that of a child who sees a system output 'sad text' and, like someone seeing a sad expression on a stuffed teddy, concludes that this is a property of the object rather than her own emotional reaction.
Given that we don’t know how consciousness works how have you concluded that it’s certainly not an emergent property of something like a highly trained LLM?
- Reductionist analogies like how Microsoft Word is not conscious therefore AI is not.
- Dismissive in saying LLMs are not capable of moral reasoning. Maybe he meant agency or responsibility?
- Builds a case based on a Julius Caesar example without realizing its natural extension leads to a philosophical zombie which is not easy to disprove as consciousness.
- Seems to casually disregard non human sadness as a concern.
- Rolls out the stochastic parrot argument with a new coat of paint. This argument never made sense. Having a predictive element a part of the mechanism doesn’t rule anything out. Proper functionalist arguments tend to say what is required not what excludes.
Overall the article seems like a bit of a red herring. The premise is LLMs are not conscious, but most people don’t think they currently are so what’s the point of the claim?
The more interesting questions are whether artificial consciousness is fundamentally possible, and if so how far away are we and what pieces are still missing.
Finally I said something before Ted rather than the other way round!
https://techcrunch.com/2023/12/21/against-pseudanthropy/
While I agree with the premise here, I do think that it's easy for an arguer to move the goalposts such that the Caesar-Khan example no longer matters. The characters don't have to be conscious for the thing that created them (as in the case of the user doing it) to be so. So the argument would be that the creator of the characters is itself conscious, but not them. This feels like a kind of inverted no-true-scotsman type thing, but it does allow someone to retreat in some semblance of rationality.
If you are trapped in a tech bro relationship, think of humanity and cuckold your partner.
Highly recommend people read Irreducible [1] by Federico Faggin (inventor of first commercial CPU; discusses limitations of classical computing).
[1] https://www.goodreads.com/book/show/195480862-irreducible
I am so angry that RAM is so expensive now. We need to do something - these AI companies owe us money here.
But insisting on searching for it is like searching for cognitive aether. It's the social equivalent of phlogiston. Like all of these ideas, they exist in our heads as a map - a way of navigating the world, but when we hunt for it's existence in the real world and fail, time after time, we have to remind ourselves that the map is emphatically not the territory. We will never find consciousness because it's like looking for a scientific characteristic of property ownership. It's a category error.
Now, this position isn't the only position. But a relational model of language for example takes his assertion to an even more extreme place, and suggests they don't function as labels at all.
A meter is the same anywhere in the universe. If it's not, it's not a meter.
The defintion of "fat" changes based on any 3 people in the room. A handful of people would struggle to form a consensus on if all people, dogs, mice, worms, and/or bacteria are conscious.
People are still coming up with definitions of consciousness and then those definitions end up being attacked by others who disagree with the foundation of the definition, which is - if you will recall - also what happened with the meter, over the course of centuries, until it was very recently redefined to be "unambiguous", but arbitrary. This was possible because few people had any particular emotional investment in the definition of a meter, and it is probable that consciousness will be eventually defined to mean that only humans can be conscious, which may be dissatisfying but would be true throughout the universe, like a meter. If the question then becomes "what defines a human" and "why a human", then I ask, why 1/299792458 of a second?
Concepts like the parent's "fat" example are cultural relatives. Someone can be called "fat" despite actively being proportionally skinnier or having a lower BMI.
But even that has at least a basis in the physical world. A skeleton can't be colloquially fat.
The root problem is that "consciousness" does not even have that. It's metaphysical and has no ability to be measured or observed or confirmed by an outside observer. Because even if it did not exist, the object claiming it would still be claiming it. And objects that do not claim it may in fact have it.
While the top comment may have used poor examples, it feels remarkably uncharitable to actually suggest "what is consciousness" is an equivalent discussion to "how long should a meter be?"
If you define consciousness as "being human", you would just have someone asking a new question - what is "fooblefobble?" Where "fooblefobble" is what we mean when we talk about consciousness today. The question doesn't get answered by being arbitrary in this context, you just necessitate a new word.
With consciousness, on the other hand, there doesn't seem to be any motorcycle-equivalent. Essentially everyone I've discussed the issue with (myself included) expects that any mind which runs on similar principles to ours or has similar thoughts to our thoughts is conscious.
Is a Reliant Robin a car or a tricycle? If it's a car, why aren't other tricycles? What about a side-car of a motorcycle? What about an APC? What's the distinction between a flying car and a plane?
I don't have an ability to exhaustively test all words against this assertion. Nor do I have the kind of access memory to draw one if it exists. Sorry.
Many of them appear very much like fundamental parts of reality, making appearance an untrustworthy instrument. Reversing cause and effect between reference and referent is something almost everyone does, no one notices, and is the source of endless confusion. We should strive to not confuse our model of the world with the world itself. Consciousness exists in our model of the world as much as red does.
This is rhetorically slippery, and feels like it is restating the thing that I asked to be demonstrated when I asked for example of the opposite. It feels like begging the question.
In either case, the central thing that I was saying is that critiquing an article because it makes a claim about a specific word which also applies to an entire class of words makes that critique feel less informative. What I mean is that if there were an article that said "The Sun is not red" and the response was that redness is a concept of human minds, then I don't know if I would feel informed. If the comment is just limited to point that out, I guess I wanted to point out the limitation.
We don't know what we don't know. For all we know, there is a missing field in the standard model of physics that might get revealed if we are somehow able to smash two working brains into to each other at relativistic velocities, and record the results through the extreme explosion 1.532 x10^18 Joules or about 7 Tsar bombs /s
"Contrary to idealism, which regards the world as the embodiment of an "absolute idea," a "universal spirit," "consciousness," Marx's philosophical materialism holds that the world is by its very nature material, that the multifold phenomena of the world constitute different forms of matter in motion, that interconnection and interdependence of phenomena as established by the dialectical method, are a law of the development of moving matter, and that the world develops in accordance with the laws of movement of matter and stands in no need of a "universal spirit."
Now go and implement other teachings of Marx, you dialectical venture capitalists!
that's exactly the state of "AI" right now, it's cold, mathematical emulation
btw there are some fascinating papers on the concept that consciousness in humans is actually a quantum effect
brilliant Roger Penrose proposed it (and they thought he was nuts) but recent discoveries about microtubules make it plausible
so who knows, maybe a dozen exponential improvements in quantum computers could make "AI" really conscious next century
Alright. What is consciousness? Please provide a definition that somehow encompasses all humans and excludes all current AI.
I’ll wait.
It's just a word machine. There are no thoughts. It cannot be conscious. How is this even up for debate in any way whatsoever? I do not understand how people can believe this. Is this not a site for software engineers?