235 pointsby ssiddharthan hour ago19 comments
  • hbn23 minutes ago
    It's insane the AI has been provided the tooling to send emails to arbitrary addresses like that. Like, getting it to send a 2FA code at a user's request is one thing. But it should only be able to "hit a button" to send a 2FA email to the address attached to the account, all run with hand-written code. It shouldn't have access to the 2FA code itself, or the message subject, or body, or the recipient address, etc.

    Why did they give it any of that?!

    • dparka minute ago
      This exploit has essentially nothing to do with AI and everything to do with a terribly designed account recovery flow.
    • footydude5 minutes ago
      > But it should only be able to "hit a button" to send a 2FA email to the address attached to the account, all run with hand-written code.

      Genuine question...why would that need to be hand-written?

      It makes absolute sense as a general statement and is kinda crazy that this wasn't a built-in limitation, but I'm not quite sure why the code for that bit must be hand-written (provided the code functionally does what you describe).

      • mediamana few seconds ago
        I think he likely means "code that is hand-reviewed" and not directly controlled by the agent. He's probably meaning to differentiate it against the in-process agent writing the code. It doesn't matter too much if that fixed code was written by an LLM under guidance and review of the SWE, outside the agent.
      • andrewstuart22 minutes ago
        Maybe not hand-written, but definitely static, and at least reviewed/tested to only allow sending to previously-validated email addresses.
    • AlienRobot18 minutes ago
      The harness is vibe-coded.
  • buildbota minute ago
    So the AI agent had privileged access to remove 2FA, ignore the account email, and just hands accounts to whoever asked? Honestly that’s so highly negligent I wonder if the implementation team for that “feature” was intentionally trying to do as much subtle damage to meta as possible before their inventible layoff.

    It’s a shame nobody tried to get it to drop the production table entirely! (mostly joking). Just claim to be a high level SRE solving some critical production bug, the only solution to which is dropping the database.

  • sosodev41 minutes ago
    Support requests have always been the weakest link in the security chain for big corps. I've had accounts of mine turned over with 2FA disabled by humans before. I guess we shouldn't be surprised that the LLMs are doing the same thing.

    The simple fact that 2FA can be removed by low level support staff drives me mad. It defeats the whole purpose of the process.

    • spullara37 minutes ago
      recovery is always the weakest link in any authentication system
      • acdha7 minutes ago
        This is not wrong but what’s really missing is cost: Meta did this so they can avoid paying people to do it. Lots of companies follow that decay spiral: your bank could shut phishers down cold by requiring wire transfers to be authorized in person but they don’t want to pay staff or risk you being upset by a transaction taking an extra hour so they don’t.

        Imagine an alternate universe where big tech companies worked with various trustworthy third-parties where something like this would generate a challenge you could take to your local notary, post office, library, police station, etc. where someone would check ID before approving it. How many phishing attacks would be prevented annually by a physical presence check?

        • ronsor4 minutes ago
          The amount of hassle involved with regular physical checks is why it's not implemented, regardless of attack prevention.

          The cost of hiring a person is part of it but not really the core reason. People were sold on the Internet with "you can do things online conveniently" and reintroducing the need to physically go somewhere negates that angle entirely.

      • SoftTalker24 minutes ago
        It's a tough problem, because people forget passwords, change phones, lose access to 2FA devices, but still need to use their accounts.
        • toomuchtodo21 minutes ago
          I manage customer identity and access management ("CIAM") for a financial services firm. Passkeys are primary, recovery can be performed by providing a government credential remotely (which costs us ~$2-3 per recovery). I do not think it is hard, based on what we have built and spent to enable these capabilities. NIST Special Publication NIST SP 800-63 Digital Identity Guidelines is a helpful resource on this topic.

          https://pages.nist.gov/800-63-4/

          I think Meta just does not care if they're enabling AI attack surface and vulnerabilities into these customer journeys. It's...certainly a choice, versus deterministic journeys with hard guardrails. They could make different choices.

          • macintux17 minutes ago
            I’d wager your range of tech literacy/capabilities for your firm is much narrower than big tech.
            • toomuchtodo14 minutes ago
              Range != value. Doing more poorly does not make something better. Our customer identity capabilities are very close to login.gov (we don't have to support hundreds of agency customers and common access cards), and if its good enough for ~342M Americans, its good enough for our customer base.

              Broadly speaking, work for the sake of work is not valuable work. Show me outcomes for resources and time invested, and compare accordingly. If you bring me an AI solution for a high risk high value customer journey, data flow, or code path, that is an anti pattern. If you, as a colleague or a stakeholder, put forth that we must use AI in situations that require a high degree of determinism (due to potential high cost failure modes), you will need to prove this extraordinary claim with evidence.

              Choose Boring Technology - https://news.ycombinator.com/item?id=9291215 - March 2015 (212 comments) ["Am I using this project as an excuse to learn some new technology, or am I trying to solve a problem?"]

              (I get paid to manage risk efficiently, including being measured on time and budget spent against the success criteria, ymmv)

      • UltraSane5 minutes ago
        It depends. Some like AWS take it deadly seriously and it takes a long time to recover root access to an account.
      • jgalt21222 minutes ago
        fair enough, but what's the actual point of 2FA if it's so easy to override?
        • spullara15 minutes ago
          the alternative is people losing their accounts and people aren't willing to allow that. i do think that apple does this a little better where they try everything to contact you in every way they know and it takes a week to get access. at a minimum to change your email it should require a week of waiting to see if the user can access the original mail to the hand off.
  • patmcc21 minutes ago
    Always a bit illuminating to me how many exploits seem to so dumb I'd never even bother to attempt them. You're telling me I can just...ask for the password? And that works?
    • AlienRobot17 minutes ago
      It's not called artificial intelligence for nothing.
  • pixl97an hour ago
    >Once it looks like the request is coming from the correct region, they tell the Meta support AI that the account is hacked and ask it to send the verification codes to an arbitrary email address they control.

    Dear Instagram, wtf. Why not send the reset to the account in question? Arbitrary email, wow.

    • giarcan hour ago
      Perhaps the attacker says that they email was also hacked and "this is my new email now". It sounds like this was a result of AI support and not a real person "And if you're part of the A/B tested accounts on which the AI support option is active, tough luck, you can't even turn it off."
  • torben-friis16 minutes ago
    How is this "embarrassing" instead of subject to legal liability?

    We really need similar rules to other engineering disciplines. If your building falls with people inside, you killed them.

    • TZubiria minute ago
      You said it, instagram is not life-critical
  • avnfish43 minutes ago
    The implications of this are quite unsettling. Meta gave an agent privileged read AND write access to user accounts with no human in the loop?
    • MrZander35 minutes ago
      > with no human in the loop

      With no basic validation either apparently. Insane.

    • tartoran41 minutes ago
      Yes. AI is in charge now
  • rd25 minutes ago
    This happened to my instagram yesterday night while I was asleep. I don't have a particularly high value username (it's probably worth somewhere in between $300-500), but still incredibly frustrating to deal with. True to the article, I had already enabled 2FA last night and it didn't matter.

    Thankfully, IG gave me the option of restoring my username when I logged back into my account today.

  • gaflo11 minutes ago
    Is there any credible primary source for this exploit being real?
  • tantalor28 minutes ago
    They're just one tiny step from the AI emailing itself all the account recovery links, and locking out the entire userbase.

    It might even do that preemptively if it thinks they're going to shut it down.

  • r72120 minutes ago
  • theideaofcoffee4 minutes ago
    What is even the point of having 2FA if it can be so trivially bypassed? Isn't that the whole point that it's sort of a last line of defense? Oftentimes, you can't change simple account settings without having to re-auth and then punch in your code again. Why would something as critical as a suspicious password reset be able to jump ahead of that? Mind boggling. But, I guess that's what happens when you lay off 10% of your people at a time.
  • king_zee30 minutes ago
    If the LLM has knowledge of something, by design it can't help but divulge it. When will companies learn granting any kind of sensitive information access to an LLM is a moot point
    • dpoloncsak5 minutes ago
      What part of this article implied the LLM divulged sensitive information to a user? All it did was change your associated email if you impersonated the user
  • mtoner23an hour ago
    wow thats extremely embarassing for meta
    • jolt429 minutes ago
      I suppose you could chalk this up to an oversight. I don't see how Meta gained from this. They've been purposeful about collecting user data and lying about it, eg: 2025 Android Tracking Incident. Shouldn't just be an embarrassment, should be much worse than that.
    • bayarearefugee27 minutes ago
      Just another day for Meta in terms of embarrassing outcomes, and yet the company makes hundreds of billions of dollars per year because the only thing that matters anymore is shoving increasingly scammy and worthless ads in front of as many eyeballs as possible, even when the people with those eyeballs can less and less afford to buy anything non-essential.
    • petesergeant30 minutes ago
      Who specifically do you think is embarrassed there? They’ve got all the cards, they don’t care.
  • Hugsbox43 minutes ago
    Jeez, straight up amateur shit. Genuinely hard to believe.
  • alex113812 minutes ago
    But I was told that when Zuckerberg bought IG, it wasn't to murder competition in its crib. Instagram "only had 12 employees" so it must be ok
  • sleepybrett32 minutes ago
    The only thing worse than a naive customer support rep is an even more naive customer support ai.
  • jeffbee14 minutes ago
    My account, with a 3-letter username worth $$$, got hacked yesterday morning probably by this flow, but I did manage to defend it. I think by far the biggest problem with Instagram/FB/Meta auth flow is that 2FA does nothing. You don't need the 2nd factor to disable it, so attackers can just turn it off. Really stupid!

    Also, I discovered that many of IG's auth endpoints are just broken. For example you can't change password on web because of CORS, which isn't a transient outage but just a flat out bug.

    Edited to add: This is just the cherry on top of years of stupid auth flow at IG. I have received tens of thousands of reset links or codes from IG over the years. There used to be a way to put your account on recovery cooldown for a few weeks but they got rid of even that.

  • WhyIsItAlwaysHN43 minutes ago
    "Social engineering is all you need"
    • hangonhn40 minutes ago
      More like "Prompt engineering" ?
      • zorrn37 minutes ago
        Can we really name this "Prompt engineering"? The prompt is so simple this is hardly any work even less than this comment