131 pointsby ishqdehlvi9 hours ago16 comments
  • magicalhippo5 hours ago
    Key point is that Claude did not find the bug it exploits. It was given the CVE writeup[1] and was asked to write a program that could exploit the bug.

    That said, given how things are I wouldn't be surprised if you could let Claude or similar have a go at the source code of the kernel or core services, armed with some VMs for the try-fail iteration, and get it pumping out CVEs.

    If not now, then surely not in a too distant future.

    [1]: https://www.freebsd.org/security/advisories/FreeBSD-SA-26:08...

    • muskstinks42 minutes ago
      You might want to watch this:

      https://www.youtube.com/watch?v=1sd26pWhfmg

      Claude is already able to find CVEs on expert level.

      • shimman8 minutes ago
        A talk given by an employee that stands to make millions from Anthropic going public, definitely not a conflict of interest by the individual.
        • muskstinks4 minutes ago
          I didn't say "watch this without critical thinking".

          The chance this is completly fabricated though is very low and its an highly interesting signal to many others.

          There was also a really good AI CTF Talk at 39c3 hacker conference just 4 month ago.

      • 10 minutes ago
        undefined
    • ogig2 hours ago
      Setting up fuzzing used to be hard. I haven't tried yet, but my bet is having Claude Code, today, analyze a codebase and suggest where and how to fuzztest it and having it review the crashes and iterate, will produce CVEs.
    • Cloudef3 hours ago
      You can let agent churn unattended if you have some sort of known goal. Write a test that should not pass and then tell the agent to come up with something that passes the test without changing the test itself.

      For this kind of fuzzing llms are not bad.

      • vinnymac2 hours ago
        When doing this remove write permissions on the test file, it will do a much better job of staying the course over long periods. I've been doing this for over a year now.
    • fragmede5 hours ago
      > Credits: Nicholas Carlini using Claude, Anthropic

      Claude was used to find the bug in the first place though. That CVE write-up happened because of Claude, so while there are some very talented humans in the loop, Claude is quite involved with the whole process.

      • magicalhippo4 hours ago
        > Claude was used to find the bug in the first place though. That CVE write-up happened because of Claude

        Do you have a link to that? A rather important piece of context.

        Wasn't trying to downplay this submission the way, the main point still stands:

        But finding a bug and exploiting it are very different things. Exploit development requires understanding OS internals, crafting ROP chains, managing memory layouts, debugging crashes, and adapting when things go wrong. This has long been considered the frontier that only humans can cross.

        Each new AI capability is usually met with “AI can do Y, but only humans can do X.” Well, for X = exploit development, that line just moved.

      • bayindirh3 hours ago
        Yes, that claim needs a source.
    • petcat4 hours ago
      > have a go at the source code of the kernel or core services, armed with some VMs for the try-fail iteration, and get it pumping out CVEs.

      FreeBSD kernel is written in C right?

      AI bots will trivially find CVEs.

      • pjmlp4 hours ago
        The Morris worm lesson is yet to be taken seriously.
        • pitched4 hours ago
          We’re here right now looking at a CVE. That has to count as progress?
  • panstromek5 hours ago
    The talk "Black-Hat LLMs" just came out a few days ago:

    https://www.youtube.com/watch?v=1sd26pWhfmg

    Looks like LLMs are getting good at finding and exploiting these.

    • baq4 hours ago
      Everybody is acts so surprised as if nobody (around here of all places!) read the sama tweet in which he was hiring the Head of Preparedness... in December.

      https://xcancel.com/sama/status/2004939524216910323

      • muskstinks38 minutes ago
        Besides that i'm not reading x, what has this arbitary random tweet todo with antrophic, the yt talk about Opus quality Jump to find exploits no one else was able to find so far?

        A theoretical random tweet and a clear demonstration are two different things.

      • eru3 hours ago
        I never read any Twitter.
        • baq3 hours ago
          X was the primary source, it's been since reported all over the news.
  • ptx4 hours ago
    > It's worth noting that FreeBSD made this easier than it would be on a modern Linux kernel: FreeBSD 14.x has no KASLR (kernel addresses are fixed and predictable) and no stack canaries for integer arrays (the overflowed buffer is int32_t[]).

    What about FreeBSD 15.x then? I didn't see anything in the release notes or the mitigations(7) man page about KASLR. Is it being worked on?

    NetBSD apparently has it: https://wiki.netbsd.org/security/kaslr/

    • ktm5jan hour ago
      I don't understand this, because KASLR has been default in FreeBSD since 13.2:

      [kmiles@peter ~]$ cat /etc/os-release

      NAME=FreeBSD

      VERSION="13.3-RELEASE-p4"

      VERSION_ID="13.3"

      ID=freebsd

      ANSI_COLOR="0;31"

      PRETTY_NAME="FreeBSD 13.3-RELEASE-p4"

      CPE_NAME="cpe:/o:freebsd:freebsd:13.3"

      HOME_URL="https://FreeBSD.org/"

      BUG_REPORT_URL="https://bugs.FreeBSD.org/"

      [kmiles@peter ~]$ sysctl kern.elf64.aslr.enable

      kern.elf64.aslr.enable: 1

      • savant2an hour ago
        This knob isn't KASLR, it just enables ASLR for ELF binaries.
    • keysersoze333 hours ago
      This is more of a Linux kernel criticism of KASLR, but perhaps it's related as to why it's not been a priority in FreeBSD (i.e. it gives a false sense of safety and rather focus on 'proper' security hardening): https://forums.freebsd.org/threads/truth-about-linux-4-6-sec...
  • sheepscreek3 hours ago
    I find it more concerning that this is still considered newsworthy. Frontier LLMs in the hands of anyone willing to learn and determined can be a blessing or curse.
  • m1325 hours ago
    Appreciate the full prompt history
    • ptx4 hours ago
      Well, it ends with "can you give me back all the prompts i entered in this session", so it may be partially the actual prompt history and partially hallucination.
    • dark-star4 hours ago
      they read like they were done by a 10 year old
      • m1323 hours ago
        They do, the whole tone and the lack of understanding of Docker, kernel threads, and everything else involved make it sound hilarious at first. But then you realize that this is all the human input that led to a working exploit in the end...
        • bluGill3 hours ago
          Freebsd doesn't have docker. It has jails which can serve a similar purpose but are not the same in important ways
      • addandsubtract3 hours ago
        Welcome to vibe coding. If you ever lurk around the various AI subreddits, you'll soon realize just how bad the average prompts and communication skills of most users are. Ironically, models are now being trained on these 5th-grade-level prompts and improving their success with them.
  • imta7177023 minutes ago
    [dead]
  • jeremie_strand2 hours ago
    [dead]
  • Adam_cipher3 hours ago
    [dead]
  • bustah2 hours ago
    [dead]
    • MYEUHD2 hours ago
      This is an AI-generated comment.
  • volume_techan hour ago
    [dead]
  • alcor-z2 hours ago
    The MADBugs work is solid, but what's sticking with me is the autonomy angle — not just finding a vuln but chaining multiple bugs into a working remote exploit without a human in the loop. FreeBSD kernel security research has always been thinner on the ground than Linux, which makes this feel both more impressive and harder to put in context. What's the actual blast radius here — is this realistically exploitable on anything with default configs, or does it need very specific conditions?
    • Fnoordan hour ago
      FTA, top:

      > Attack surface: NFS server with kgssapi.ko loaded (port 2049/TCP)

      Not sure who would run an internet exposed NFS server. Shodan would know.

  • dheerajmp2 hours ago
    You do not need Claude for finding FreeBSD vulns. Just plain eyes. Pick a file you can find one.
  • PunchyHamster5 hours ago
    I'm just gonna assume it was asked to fix some bug and it wrote exploit instead
  • rithdmc5 hours ago
    Running into a meeting, so won't be able to review this for a while, but exciting. I wonder how much it cost in tokens, and what the prompt/validator/iteration loop looked like.