Storyscope (https://github.com/jenna-russell/storyscope) is a really interesting concept, and one I'm in the process of reproducing and expanding on, even though it is less accurate on direct detection than something like BERT or Binoculars. It is interesting because the usual obfuscation tactics, even having a human transcribe and rephrase the story don't work; it looks at the story itself, rather than word use and grammatical quirks, etc. Plot, agents, temporal structure.
So far, AI writing is detectable to a pretty high degree, though I think it'll be a war of attrition that AI eventually wins. And, all the detectors are the same tools one could use to create undetectable AI writing; AI loves to iterate in a loop, seems like iterating to rewrite to be undetectable is a soluble problem (though models currently probably would end up writing worse and worse to avoid detection, and the current best models have begun integrating watermarking, pushing back the defeat of human writers for some time).
I'd like to come up with a more fun game loop, but nothing has revealed itself to me yet. Shorter passages are easier to gamify, but shorter passages are much harder to detect.
- blockchain attestation [ crypto-graphically signed by human author / artist ]
- proof of work in the traditional sense, eg video of artist at work, stages of the painting from sketch to under-painting, brushstrokes, layers, glazing
In my case, I think videos of the painting process are a reasonable, if temporary, solution.
> Because to me, authenticity makes for a better reading experience.
This pair of sentences gave me pause, until I realised that they're not in contradiction - the author is deliberately non-judgemental of _others_ making a personal choice that _he himself_ would prefer not to make. Remarkably rare, nowadays.
--------------
To do what the author of this post asks for is incredibly difficult. You can't reliably use a LLM to detect the output of other LLM's. You can't trust user reviews to distinguish AI from human writers because AI can easily produce reviews. You'd basically need to form a central gate-keeping authority that decides who is allowed and who isn't. This would be resource intensive and fraught with problems. (e.g. How many authors would sue if they were blocked as the result of a false positive? How many using LLM's to write their material would brazenly sue for being blocked too?)
It really bothers me that one of the biggest problems with LLM's is how difficult it is to filter out what they produce.
I agree with Howey's sentiments. AI-written books aren't bad in themselves -- they're not ontologically bad -- but they tend to be, by their very nature, derivative. Narrative quality is, as yet, still quite poor. And self-publishing on Amazon.com is full of that stuff. (I've long since stopped buying self-published stuff on Amazon on account of low-quality content, and Amazon's review system seems severely broken.)
Humazon would be a welcome thing. Or even a "human written" flag for books on Amazon.