AI Safety Is Not a Corporate Plot(www.astralcodexten.com)
11 pointsby Analemma_6 hours ago3 comments
  • 5 hours ago
    undefined
  • 6 hours ago
    undefined
  • randomImmigrant6 hours ago
    I agree there’s no proper plot. That doesn’t mean the AI companies are right, and the reason is in point no. 2 in his list:

    The AI company leaders giving these warnings were giving the exact same warnings before they had AI companies - and in many cases, before they worked in the industry, or before the industry existed.

    And that doesn’t stand the sniff test, because:

    1. There’s a strong linguistic silos around these AI leaders and many of their employees. Assumptions couched in language that should be rigorously questioned is instead sometimes axiomatic. 2. It’s hard to understand why, if you had the same exact fear before starting the AI company, you started it in the first place, let alone continue even as you bemoan these fears. 3. Why, if this path is so dangerous, these leaders don’t consider alternate paths to building AI. Why are all their investments in this one way to do intelligence? Surely we will benefit from having multiple routes to choose from?

    The absence of choice and the piling of inevitabilities is why these warnings don’t add up. That the same drum has been banged by the same people for a few decades without input from other fields and sources is what makes all this seem religious prophet adjacent rather than technical assessments by thoughtful engineers and scientists.

    • YuriNiyazov5 hours ago
      For anthropic, the answer to (2) is very explicit and upfront: they will build it first with an explicit emphasis on alignment to have a “race to the top”, so to speak - you can believe that they are wrong or that they will do a bad job, but my point is that (2) has an answer that you have to engage with.
      • randomImmigrant4 hours ago
        So why are they continuing when they say alignment isn’t working? If for all these years they have feared misaligned AI, and now they say their alignment science is less than good, the most obvious conclusion is to stop pushing in the same direction.

        This doesn’t mean they fold as a company, or give up on AI, or stop working. But what exactly is the explanation for continuing to train ever larger frontier models in the current context?

        • YuriNiyazov3 hours ago
          Because very basic game theory explains that if they give up on the frontier unilaterally, competitors will capitalize on that rather than also give up. Hence the request for regulation.

          And while their alignment is less than good, there are criteria by which it is the best available (eg no hugging face-scale attack on their end) So it is strictly worse to allow a competitor with worse alignment to take the lead.

      • verdverm5 hours ago
        I am not interested in a future where the tech oligarchy are the ones who get to decide these things.

        They built on all of human knowledge, we should all have a say in how and where these things happen and the controls we place on them. Having gatekeepers who are clearly experiencing ai psychosis is not the way. I believe they believe their fears, but I do not share them and think humanity can handle this through transparency and collaboration. They have made so many incorrect and misleading statements, they do not deserve our deferrence on such important matters. Diversity of input to the process is more important because the Ai leaders are in an echo chamber, being friends and shuffling between the various labs.

      • Avicebron5 hours ago
        The answer "we will race to the top with an emphasis on alignment to our own values" isn't really an answer worth engaging in. It's pretty cut and dried.
    • LordDragonfang29 minutes ago
      > 3. Why, if this path is so dangerous, these leaders don’t consider alternate paths to building AI. Why are all their investments in this one way to do intelligence? Surely we will benefit from having multiple routes to choose from?

      Because it's obvious that the investments are still paying intelligence dividends, and we haven't found a better path yet?

      Also, more to the point: if you were familiar with the history of the AI safety debate, you'd know the people most concerned are pretty certain that every path forward is dangerous -- and that current AI is already, by cosmic accident, much more aligned to human values than any other path that was predicted.