Detecting LLM-Generated Texts with “Classical” Machine Learning(blog.lyc8503.net)

76 pointsby uneven94343 hours ago13 comments

docheinestagesan hour ago
I think figuring out if a text is AI-made is a losing battle. What could work is gauging how much effort went into writing the text, regardless of who the author might be. What's easy today is generating mountains of text that are extremely hard to read. What requires effort is knowing how to engage the reader, how to keep out extraneous information, and how to keep the text as short as possible without losing details. That needs effort, with or without AI.
- calebh25 minutes ago
  The easiest way is to keep track of the text's edit history, keeping a block of edits over time and having them signed by a timestamp authority. The final edit history can then be inspected by some external authority, then signed if the edit history looks human. I have a blog post from 2023 on this topic: https://helbl.ing/Written-Proof-of-Work/
  For Google Doc users, you can already inspect the edit history over time to verify that text is written by a human.
  - visarga20 minutes ago
    That human might have used AI. You can never know. Hand fixed AI output, human just polished the corners? Light rewording of a full text written by hand, because the author is not confident in their writing? Actual human text, but after researching with AI?
akersten2 hours ago
Text is simply not information dense enough to be able to decode some arbitrary signal of provenance from it. Sure you might be able to detect today's tells (particular sentence structures preferred by Claude, phrases, etc) to get you some arbitrary chance percentage it was machine generated, but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading.
Images, absolutely, there are tell-tale artifacts from today's generators that simply aren't emitted by "natural" paths to create them, and you can "detect AI" with high confidence (for now). Words, no, the signal is far too sparse and we are well into undetectable sophistication with today's models, let alone tomorrow's.
- driverdanan hour ago
  There are two problems, false positives and changing the LLM's pattern.
  It's really easy to have a false positive and false positives can be very harmful if the person using the detector isn't aware of that risk.
  It's also very easy to change the pattern of LLM output. You can provide basic prompting that will significantly change the structure of the output. For example, having it utilize the Wikipedia article on signs of AI writing and avoid everything it describes. https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing
- Retrican hour ago
  Signal is easier to detect with more data to work with.
  Largely AI generated books are a vastly different situation than a one paragraph homework assignment. But multiple rounds of homework assignments would change the accuracy.
- stymaar2 hours ago
  > but it's a bad fiction to perpetuate that any of this is anything more than tarot card reading.
  Hard disagree. LLMs (especially base ones, that only received pre-training) can produce output that is undistinguishable from human writing (because that's what they were trained to do).
  But commercial chat models are specifically tuned in a way that maximizes user engagement. It's that specific tuning that is very easy to spot when reading AI slop, and that's not surprising that it's easy to spot automatically either. And I don't think that's going to change anytime soon, unless their incentives change.
  (We can say exactly the same thing about man-made stuff optimized for a specific purpose, like stock photography, clickbait titles or industrial food: they aren't stereotypical because their creator lacks the skill to make them otherwise, they are like that because that's what works best).
  - BoredomIsFunan hour ago
    > especially base ones
    Did you actually try them? I did.They generated even more "slopey" text than instruction-tuned ones.
  - AnthonyMouse26 minutes ago
    > But commercial chat models are specifically tuned in a way that maximizes user engagement. It's that specific tuning that is very easy to spot when reading AI slop, and that's not surprising that it's easy to spot automatically either.
    There are two problems with this.
    The first is that it would still misclassify human-authored text written under the same incentive, and most people have various incentives to "maximize engagement".
    And the second is that then people would just make other models that are tuned for defeating that sort of classifier, which would be used whenever the classifier is being used.
    stymaar4 minutes ago
    > The first is that it would still misclassify human-authored text written under the same incentive, and most people have various incentives to "maximize engagement".
    The thing is, humans are significantly worse at maximizing numerical goals than computers.
    > And the second is that then people would just make other models that are tuned for defeating that sort of classifier, which would be used whenever the classifier is being used.
    Anyone can already do that right now, just grab unsloth studio and fine-tune your local Gemma, but nobody cares. People posting slop content don't care if pangram or I flag their slop with certainty, they are using the easiest option, which is commercial chat models. And given this segment of user doesn't care, the provider have zero incentive to provide a dedicated stealth model for that purpose.
    OneManyNone21 minutes ago
    All of that may be true, but pangram currently has a false positive rate of about 1 in 10000, and this has been tested by feeding in thousands of texts written before 2020.
    That may not last if AI companies start trying to build models that fool it, but for the time being at least, modern models do have strong tells.
    AnthonyMouse12 minutes ago
    You can get an arbitrarily low false positive rate by sacrificing against false negatives. It's trivial to make it zero, just classify everything as human-generated. Meanwhile a false negative rate of even 1% is a pretty big problem since someone can easily use LLMs to generate 100x the volume of text and then use whichever ones make it through the classifier.
    And that's before anyone even tries to get the LLM to generate a different style of text. Or for that matter creates a "style model" that rephrases text.
  - ravenstinean hour ago
    They're also designed to not offend anybody, so their output tends to be very bland even compared to the most milquetoast of human beings. I was only surprised once when ChatGPT responded with an enthusiastic "hell yes" seemingly organically, but 99.9% of the time these AI services clearly are instructed and trained to provide flavorless word vomit. I don't think there's a technical reason why an LLM couldn't produce totally convincing output, but internet grifters don't need to go through that trouble. It's like how most phone, email, and social media scams come off as completely transparent to most of us, but that's the whole point; we're not the target audience of the scams. Readers looking for substance, nuance, and real opinions aren't going to notice if something with written by an LLM – unless there are some cliche punctuation tells.
  - empath752 hours ago
    It does mean that this will have a drift problem if it's just trained on the idiosyncrasies of model fine tuning. That's fine! But it is something to be aware of.
- zmjone2992an hour ago
  i think one thing overlooked by this perspective is that many of a detectors adversaries are not that sophisticated. so despite this i think it is a useful thing to try to do. particularly when people are trying to do fraud which will often having to use abliterated models and generally trying to be as economical in their efforts
- cyanydeezan hour ago
  Sure it is; we do it all the time, and then we modify each other's etc, etc; english we speak today was spoke yesterday waspake the same in yesteryears; we have no trouble dating english or other languages to a time.
  A better argument is people themselves are just too influenced by reading that they'll sound like LLMs in a couple of years.
- jgalt2122 hours ago
  It depends on how much text. For example, chardet often falls down on short strings, but 1K characters it nails it.
40four36 minutes ago
I could be wrong, but I just don’t see how trying to “detect” LLM generated texts is ever going to work. The only thing that makes any sense if you truly want to have confidence a human wrote it is some type of “proof of work“ system. I think there’s a lot of interesting ways to approach the proof of work problem with different pros and cons, but that is where our energy should be focused if we seriously want to solve this problem.
Krssst2 hours ago
The classifier does not seem so big, I wonder if something like it for English could be used in a browser extension to run against every single paragraph being displayed ?
If the internet is going to drown in LLM text it would be nice to have tools to detect that automatically just like we have adblockers today to avoid wasting time on ads.
(the article was a good read, thanks!)
- xiaoyu20062 hours ago
  I assume different models will have different distribution, so it has to be kept updated?
  - Krssst2 hours ago
    The article mentions that AI texts are often caught by multiple models, so hopefully text from newer LLMs could still be caught without updating the model?
arjie16 minutes ago
Neat. I will implement something like this for myself. I just need to reduce the spam a little. Imperfection is okay for a social network context like HN.
gleenn2 hours ago
I think the fundamental problem is that training current SOTA AI models is very expensive. If a simple "classical" model can detect them, presumably at much lower algorithmic cost, then why wouldn't the model trainers use these same tools to feed back into their models to improve them at low cost to make them better? It's an arms race. Any cheap pattern can and presumably will be used to retrain if it becomes and effective way to catch AI.
- arjie33 minutes ago
  It’s simply not a priority. The labs can do many things. Making text non-LLM is not really that useful. Analogous to Facebook not picking up the obvious $20 bill in front of them. It’s because they’ve got $100 bills at their feet they’re picking up.
- Retrican hour ago
  It’s an arms race where the AI companies are at an extreme disadvantage due to relative training costs.
teeray2 hours ago
The problems are simply too great if an LLM detector has any false positives at all. Imagine how soul-crushing writing an entire dissertation by hand and having it rejected because some “good enough” LLM detector decides you write too much like an AI.
- rayval16 minutes ago
  As I recall, a few years ago (in the era of first generation LLMs), a professor in Texas used an anti-plagiarism tool that flagged more than one-third of the class using AI in an exam, and used that finding to give them a failing grade.
  If memory serves, one student objected strenously and ran the professor's own work (published 10 years earlier) into the same tool and it flagged that work as AI-generated.
  EDIT: HN item from June 2023 https://news.ycombinator.com/item?id=36215823
- dmurvihill33 minutes ago
  It depends on the application. Dissertation? Hell naw. Blog post? Absolutely, run it through that thing.
unfocso2 hours ago
I had done the same for classifying and generating bookmarks of thousands of datasheets, along with a very naive yolo-based classificator (to detect pages made out of diagrams and pictures mostly).
Done with GLM-OCR, I had to watch text sloooowly crawl out of the llm and still have to live with hallucinations and the model not following the schema
aberoham2 hours ago
I wonder about this technique vs simple SVM classifiers: https://x.com/rosmine/status/2056406399471558872?s=20
- janalsncman hour ago
  This article is about training a classifier to detect synthetic text.
  The link you sent is for generating text which attempts to defeat those classifiers.
richard_chase31 minutes ago
Am I the only who largely enjoys the output of LLMs more than most stuff written by humans? I find myself coming back to old chats with ChatGPT frequently because the output is amazing.
- therealdrag021 minutes ago
  I wouldn’t go that far… but it can be kinda like Wikipedia, clean and readable.
XiphiasX2 hours ago
Anything too “clever” and “snappy” = instaLLM
- hastegan hour ago
  This is also how I pretty much filter LLM generated text in my head.
metalman40 minutes ago
there is not much point in detecting LLM generated text, in that humans are useing info from LLM's, but obfusicting it's origin, with there own garble, along with purely human garble, and almost(but not quite) human LLM product meaning that the threshold for rejecting "data" must be lowered, which personaly means a very very low tollerance for wierdness, except where it can yield imediate possitive cash flow for the rest I do my own research and verification thank you very much
cyanydeez2 hours ago
today, sure.
Tomorrow, the LLMs will be training the humans thought patterns that will directly start skewing their natural writing.
Generation alpha is going to have a lot of trouble if we keep perpetuating the myth that you can really interpret text in an ongoing fashion.