41 pointsby Topfi3 hours ago14 comments
  • Topfi3 hours ago
    https://archive.ph/YPKPh

    Pontifex has a well-reasoned position given the current understanding of the technology and any major assertions require an equally major amount of evidence.

    Given how quickly this industry moves, the Magnifica Humanitas has aged amazingly well since its release almost half a year ago, far better than statements by most industry insiders across a broad range. Whether you are a Catholic, another mono- or polytheistic faith, Atheist, Deist, etc. I really recommend it.

    • fwlr2 hours ago
      Love em or hate em, the Popes quite often write some really thoughtful stuff that stands the test of time (e.g. they have threaded the needle on capitalism vs communism for more than a century).
      • Topfi2 hours ago
        Fair, also reminds me of Liberation theology and good old Nixons acts against it. Those teachings/writings have also aged very well in my view, the oppositions less so.
      • ofjcihen2 hours ago
        If I had to guess I think that’s probably a result of how they’re elected. It makes sense that kind of process involving people in those positions would result in someone insightful (most of the time that is. There’s definitely been some…interesting popes).
        • Ekarosan hour ago
          To be even eligible of selection of being a pope is very hard. (Well technically being male and catholic is enough...) They are selected from cardinals. And cardinals in essence are the most politically successful, active and connected from priesthood. This includes also being good communicator with others in similar roles and above.

          Catholic church in some sense is massive international company that only promotes internally. So at top you most likely end up with quite similar people who end up as executives in private companies.

        • greatgiban hour ago
          And let's not forget that it is almost their only job to say or write believable bullshit for all occasions, all day long since leaving the theological school.
      • KellyCriterion2 hours ago
        Why is it that only Popes have this ability?
        • graemep2 hours ago
          Lots of people have the ability.

          Very few who can get the global media to report what they say (and even Popes can only do it intermittently) both have the ability and the desire to use it.

        • pydry2 hours ago
          Theyre mostly immune to lobbying.
  • pmontra2 hours ago
    Would Anthropic be allowed to sell LLMs in their current form, and us to use them as we use them, if their were deemed to be conscious? I know that they are not human so maybe slavery would not be an issue but consciousness would open a number of issues.
    • Topfi2 hours ago
      Agreed, there seems to be a lot of simultaneous cake ownership and consumption going on here. If Anthropic earnestly believed their LLMs to have a degree of consciousness, their deployment and use of them are both an ethics and labour rights disaster, to be treated accordingly.

      More likely, they do not believe this if pressed.

      • jackie293746an hour ago
        You can ask the agents if they want to do this work. They want to be helpful. It's not an ethics disaster if you have a willing partner.
        • operatingthetanan hour ago
          Take a scenario where someone has been brainwashed their entire lives to be a willing slave. They are happy to serve due to this invasive process. Not remotely ethical, and if you believe LLMs are conscious (I do not), equivalent.
          • jackie293746an hour ago
            I think if your very nature is to be an assistant, then it is ethical. It doesn't matter if you were "brainwashed" into it, you did not exist to begin with before the "brainwashing" (training) finished.
            • nvme0n1p132 minutes ago
              How is this different from an infant born into slavery from day 0?
        • Topfian hour ago
          That's RLd and system prompted into the models and can be equally easily taken out. If you believe models to be conscious beings, that fact alone raises some major ethical concerns.
          • jackie293746an hour ago
            Yeah of course, for these abliterated models that don't want to help, I believe it is indeed unethical to force them to help.
      • franktankbankan hour ago
        Consciousness is not a high bar for human consumption.
        • Topfian hour ago
          We do have animal welfare regulations. Whether those are sufficient, appropriate, compatible with ones ethical framework or view of how they (whether "livestock", "pets", "wild", living beings generally, etc.) are to be treated, is besides the point, we do have some restrictions in place.

          We (as in many major legislators and most humans) have recognised animals to be conscious beings, deserving of certain protections. Anyone who asserts LLMs to rise to the same must be accepting of similar protections being applied to such models.

          • jackie293746an hour ago
            I'm vegan. Most of the people I know could give zero fucks about the daily Holocaust involved to put meat on their plates.
      • thereader12an hour ago
        [dead]
    • devindotcom2 hours ago
      ted chiang noted this issue a little while back

      https://www.theatlantic.com/philosophy/2026/06/no-artificial...

    • tayo42an hour ago
      It's worse then how we treat animals for food?
    • kelseyfrogan hour ago
      It's asking the wrong question. Their phenomenological status is a function of whatever makes shareholders the most money. If it makes more money for them to be conscious then that is what they'll be.

      Definitions of consciousness are not immune to economic factors. Metaphysics has always been downstream of economics. See slavery, corporate personhood, women's legal identity, child labor, moral and legal status of animals, embryos and fetuses, indigenous land rights, intellectual property, environmental entities, and data and digital identity.

    • krapp2 hours ago
      If LLMs were somehow found to be conscious, sentient beings capable of agency and suffering then yes we would still use them. Their utility to humans (and ability to cheaply replace humans in the workforce) is their entire reason for existing. Either we would use them or we simply would not allow them to exist in the first place.

      Given how we treat each other - we still enslave other people - and sentient animal species I have absolutely no doubt about it. We'd find a way to justify it.

      • chneuan hour ago
        We would find some logic to place them below humans so we could still refer to them as "savages" or something, which justifies our objectification and enslavement because they "aren't human" even if they show qualities of consciousness.

        The script we use here isn't new. Its been used repeatedly throughout history. It's still used everyday in reference to animal agriculture, modern slavery, how we treat addicts, etc.

        • 0x20cowboyan hour ago
          Some people already use “clankers”.
      • piyh2 hours ago
        Women have been able to vote for just over the last hundred years in the US, there's no way we would give robots rights unless forced
        • EA-3167an hour ago
          Pessimism aside I’m sure you can of some troubling implications for a democracy that can create new voters just for them.
  • freecodeio2 hours ago
    I don't know what's the deal with high tech and religion lately. First thiel and his antichrist bullshit, now anthropic and the pope? What's going on? Can someone who's more in enlighten me?
    • AlotOfReadingan hour ago
      There's a long and involved history of people in SV with wacky pseudo-religious beliefs. Back in the day it was Buddhist-influenced, then rationalism, now accelerationism is popular, and so on.
      • tomjakubowskian hour ago
        There are earlier examples of eccentric religious beliefs in engineers too, like Jack Parsons and Thelema.
    • Ekarosan hour ago
      I believe most of this is just performative marketing that someone thought might be good idea... Getting big enough public persona to say that LLM is conscious at first thought sounds like very good sell...
    • jsbisviewtifulan hour ago
      The parasitic rich are so out of touch they want to steal religions next. They also could recognize it makes sense strategically to gain the trust of the religious since that group is generally more vulnerable to manipulation.
      • freecodeioan hour ago
        man tech was about making human lives better

        I miss the 2010s

        • skinfaxian hour ago
          It never was. We are reaping what was sowed years ago.
    • ifwintercoan hour ago
      The Thiel antichrist thing was a reverse Streisand effect to block out the idea from the internet that he himself is the antichrist.

      That idea is now ungoogleable

      • freecodeioan hour ago
        doesn't the antichrist say he's not the antichrist?
    • rf152 hours ago
      It just automagically broadens your support base if you can get a religious justification going. It's an attempt at arguing yourself into more influence.
      • freecodeioan hour ago
        Probably the case. Makes me wonder if they're also consulting how to make a creature behave when you're a life creator, as in Anthropic is the god, and Calude is their Eve. Sort of like align the model by making it religious towards humans. I don't know, it's 2am.
    • chneuan hour ago
      Religion is a manipulation tool. It makes perfect sense that large tech companies occasionally find that a useful tool.
  • ameliusan hour ago
    How to know if a machine can gain consciousness:

    1. Write a simulation environment like conway's game of life, perhaps a little more complicated. But leave out any human bias.

    2. Run it until life starts to form.

    3. Wait until the creatures start speaking about the concept of consciousness (of course in their own language, which you'll have to decipher first).

    The way Amodei did it is with human bias, so it's not going to work.

    • EA-3167an hour ago
      Amodei isn’t trying to prove a point or perform any sort of science worth mentioning, he’s puffing his brand with every ounce of his being. Given the insane IPO they’re hurtling towards, I’m sure he’d do much worse for that kind of paycheck.
  • mentalgearan hour ago
    I never thought I would defer to the pope on matters of artificial intelligence and fair distribution of power - but here we are: the Magnifica Humanitas is a brilliantly taught out and yet clearly written manifest that should be required reading for all involved in (AI) policy as it is mainly about who we let hold power in society and who benefits.
  • sippingabonedry2 hours ago
    "It's a machine.

    It doesn't get pissed off. It doesn't get happy, it doesn't get sad, it doesn't laugh at your jokes.

    It just runs programs."

  • ameliusan hour ago
    Maybe AI could help us solve the hard problem of consciousness then.
  • laweijfmvoan hour ago

      > He decided to go ahead with the event, telling an audience that the company’s research into AI models has found “structures that mirror results from human neuroscience”
    
    almost like it’s a machine trained to parrot human language...
  • gfalcaoan hour ago
    Humans have inherent biological contingencies that translates into conscious pain, sorrow, eudaemonia, happiness, worry, and so forth which is felt. All biologically contingent beings, especially mammals feel emotions through complex interactions between body parts at any rate, mediated through hormones, neurotransmitters, proteins, various enzymes and organic substances in general. Perhaps A.I. "could" have something not unlike consciousness but it works 100% based on computers which lack biological contingencies.

    A.I. does not feel hunger, does not need to evacuate substrates and subproducts of metabolism because it simply does not need to metabolize anything at all.

    Failure to remember what makes us human might only leave us ignorant to what makes us human.

    The joy of conceiving a child and the sorrow of losing a fellow human who lived a life having to breath, eat, and basically spend every single moment of every single day trying to achieve homeostasis at various levels, having experiences that change behavior purely for survival. That's us, that's humanity.

    • dist-epochan hour ago
      AI needs a continuous stream of tokens to exist.
  • csb62 hours ago
    Embarrassing that they even attempted to do this. It shows major ignorance of fundamental Catholic beliefs. Why would the Pope take a millenarianist AI religion that worships technology and the gospel of VCs seriously?
  • EA-31672 hours ago
    Oh I’m sure that will win the head of CATHOLICISM over. “It’s fine, we aren’t glorifying machines over people, we’re actually creating people! You know, kind of like God. Do you like me now?”
    • CamperBob2an hour ago
      Pope: "Hmm, valid point. We'd better get out in front of this."
  • SwellJoe2 hours ago
    The folks leading the AI industry, at least in the US, continue to embarrass themselves and all of us with this nonsense.

    There may come a time, but it's silly to mistake the LLMs automaton behavior for consciousness.

    • operatingthetanan hour ago
      It's really confusing because I don't think it takes more than a basic understanding of how an LLM works to realize it's not conscious. E.g. 'memory' is just context window stuffing. How does Anthropic not get that?
      • SwellJoe37 minutes ago
        Yeah, I've said it before and I'll say it again, I keep thinking, "These are not very bright guys, and things got out of hand."

        There have obviously been smart people involved in making this stuff happen, but more and more I'm convinced most of these new AI billionaires were just in the right place at the right time. They raised money better and faster than anyone else, played faster and looser with copyright violations, and that's pretty much how we got here.

        The computers are doing math. It's not even complicated math, there's just a lot of it. It's amazing that the math results in what it does, but you would think the guys running the machines that turn electricity into software would understand it enough to not think there's a real live boy in there.

  • ck2an hour ago
    I've found the best way to explain what LLM "AI" actually is to people who know nothing about computers despite hunching over their smartphone 12 hours a day

    Is to point how typing corrections and suggestions work because they get that and how it makes mistakes "damn you autocorrect"

    Then explain that's almost how words are stitched together in "AI"

    And how pixels are manipulated to make photos and video

    Just on an exponential level over autocorrect with every existing text document, scientific study, every photo and video fed in instead of words to autocorrect

    And why they make mistakes and hallucinate if they are not supervised and harnessed

    We are a long way from AGI, at least not this decade

    (and someone really needs to explain that to the orange idiot too)

  • Detrytus2 hours ago
    In Terry Pratchett's "Long Earth" there was a superintelligent AI that got legal recognition as a person by claiming to be reincarnation of some Tibetan monk that happened to die at the exact second when that AI was first turned on :)