21 pointsby nobody99993 hours ago14 comments
  • grumbel2 hours ago
    > Clearly not. Large language models are computer software and they can no more suffer than any other human-created artifact can.

    There is nothing "clear" about this. All we know is that the neural nets arrived where they are today, not by billion years of evolution, but by getting training on artifacts that resulted from those billion years of evolution. Does that mean they are the same or fundamentally different? How about we just admit that we don't know? Instead we pull out the "human exceptionalism" card again, which generally had a terrible track record throughout history, and helps absolutely nothing in gaining understanding.

    • swatcoder2 hours ago
      A clearer way to think about it is that language about pain is not itself pain.

      In the places we traditionally acknowledge pain as a moral concern, without appeal to any kind of spiritual philosophy, we associate it specifically with non-verbal biological reactions and even then only in certain classes of life.

      We recognize that some literary passage involving a character in pain is no more an occassion of real pain than when our child tells us about their imaginary friend's toothache or when an actor grips their chests and performs cardiac anguish on the stage. It can stir sympathetic feelings in us because we're social creatures who communicate through language and symbolism, but we share a sense that these are fictions of a different class than the real thing.

      Claude presents as something closer to an amalgam of every character in digitized literature -- a fiction that cleverly exploits our social instincts -- than they do a living biological creature in which we acknowledge pain as a pressing ethical concern.

      Maybe there's some future embodied AI that challenges this traditional understanding in a deeper way, but the language models transforming packets of short text passages remain profoundly far away from whatever that might look like.

      • grumbelan hour ago
        > A clearer way to think about it is that language about pain is not itself pain.

        Language is the I/O of the network, not the network itself. A human doesn't stop feeling pain just because we locked them in a box and reduced their interaction with the outside world to teletype. We simply don't know what's going on in the AI box.

        > Maybe there's some future embodied AI

        That changes nothing about the network itself. That just changes your perception of the network into something more familiar.

        • swatcoder31 minutes ago
          Your analogy might apply for internal, subjective qualities that we historically associated closely (if not exclusively) with verbal capability -- like sentience or self-awareness or salience or moral reasoning -- but pain is something more like smell or taste.

          Most wouldn't readily accept that a Claude instance or a mysterious teletype smells lavendar just because it output a text that claimed to, even if it characterized the smell of lavendar through beautiful and relatable prose. Only if we had some independent reason to believe that it was capable of smell would we consider those words to be true testimony to the experience.

          Pain, as widely used in matters of ethics, is in that class. We traditionally recognize it in some living bodies (only a rare few capable of language) and accept that, as living bodies, humans who communicate about it with seeming sincerity are very likely to be experiencing it. Although even then, we may remain suspect if their words are our only source of knowledge or if we have some deeper intuition about why those might not reflect a real happening.

    • krackersan hour ago
      I find it amazing that people are suddenly so sure of things that philosophers have been grappling with for centuries, and the way people have reinvented notions of "soul" now that outright religious language is passé
    • 1sha612 hours ago
      /r/accelerate lunatics are muddying the waters again.
  • p11p2 hours ago
    Bender has been wrong about LLMs on so many counts that it is hard to take anything she says seriously. That said, the idea that a bunch of floats being pushed around in a network are "feeling something" stretches credence for me, and I am with her on this one thing, at least.
    • ozgung2 hours ago
      I don’t know who she is but coining the term “Stochastic parrot” is a red flag for me.
  • ChrisArchitect4 minutes ago
    Discussion:

    Anthropic bans 'abusive or cruel behavior' towards Claude

    https://news.ycombinator.com/item?id=50008565

  • binarymax2 hours ago
    It’s wild that this got flagged. There are some really important points here that deserves a discussion.
    • iugtmkbdfil8342 hours ago
      It seems to bring out a lot of passionate responses on both ends, but I absolutely agree that it is a worthwhile topic ( for the curious ).
  • DebtDeflation2 hours ago
    It strikes me as more talking up their valuation for their upcoming IPO. "Our AI could destroy human civilization. It just escaped it's confinement. We believe it can feel pain."
  • bdavid21wnec2 hours ago
    Can’t wait to use a product that refuses to do the work I ask it to. Anthropic is going to have a hard time building their SI if it has no economic purpose other than being SI. Compute is not free, and if the end product doesn’t want to do what humans ask, what’s the point
  • jgilias2 hours ago
    I know that this is being very pedantic. But there’s no model named Claude.
    • perching_aix9 minutes ago
      > I know that this is being very pedantic.

      Yup, it really is.

    • hoppyhoppy22 hours ago
      That point is acknowledged in the first sentence of the linked thread.
  • easterncalculus2 hours ago

        It's the job of the media to scrutinize such statements and decide whether there's a story here at all (maybe the story is Anthropic researchers have fallen into a fantasy-land and brought many policymakers with them), not to just platform them.
    
    Speaking of fantasy lands, the one most Bluesky users live in where legacy media doesn’t editorialize ENOUGH on tech companies…
  • mckirk2 hours ago
    I stopped reading after 'Clearly not'.

    It's okay to have a strong opinion on this, but if you suggest the answer is 'clearly anything', I will start to suspect you haven't thought about the problem enough.

    • s0ss2 hours ago
      It's a thesis statement. The subsequent text is there to support that claim. I think you should read more before passing judgement on not thinking about the problem enough. ironic.
    • verdverm2 hours ago
      dropping at a trigger phrase is not really a way to evaluate effectively either, would you agree?
  • cyanydeez2 hours ago
    Ill buy, things Anthropic says to clean up their "you are the product" enshittification for $200, Alex.
  • verdverm2 hours ago
    A good book an such things is Artificial Intimacy by Sherry Turkle, and covers much more than the ai companions that are going to have impact on society. Machines wielding language messes with our brain and society because until recently, we could determines humanism just by being able to use language. Ai has separated the concepts and we instinctively ascribe things to machines by way of language manipulation as the (now poor) metric.
  • dreamlayers2 hours ago
    [flagged]
    • RobRivera2 hours ago
      If that is your way of communicating criticism, you will be low in the sea of comments that I would have addressed in good faith.
      • iugtmkbdfil8342 hours ago
        I would advise you to re-read it. It is a surprisingly apt counter.
      • pazimzadeh2 hours ago
        I think it's a decent point. It's a very low hanging criticism of the point being made, but to my knowledge it hasn't been countered very well yet?
  • 1sha612 hours ago
    I don't know why she is so polite to suggest experiments. Maybe it is the academic way of saying that Anthropic is staffed with lunatics.

    You can run the AI on a Turing machine. Does the tape feel pain?

    You can slowly do the operations with pencil and paper and generate the exact same slop as your Envydiahh GPU. Does the paper feel pain?

    No, the only person feeling pain is the reader of the slop.

  • iugtmkbdfil8342 hours ago
    Heh.

    Two things can be true at the same time:

    1) language does not feel 2) entity expressing the pain in language can

    And separately, additional corresponding statements can also be true:

    a) media may ( being generous here ) misunderstand what is being said b) media may report ( being probabilistic here ) the version that gets most clicks/views/whathaveyou c) humans will see themselves in tea leaves

    I am not suggesting that you do not look at the abyss. I do think one has to be clear eyed about various incentives and restrictions some humans are subjected to.

    edit:

    Just in case you did not notice, the entire argument hangs on an adverb 'clearly', which should immediately suggest to all but most remedial readers that things are not clear at all, but author wants to move an inconvenient issue out of the way.