For some reason I had assumed testing this would be more sophisticated than just checking the thumbs up/down stats and user "vibes"
Yes - maybe saying "vibes" was minimizing the effort but what I am trying to say is that even the controlled testing is just asking users whether quality is impacted or not. Which is subjective and thats what I meant by when I said "vibes"
Don't get me wrong - I have no idea how one would go about testing this with other methods; I was just stating my assumption.
Since they rolled this out to all users I had assumed there would be other testing involved.
Retest on benchmarks whether it accomplishes tasks with the same success rates. Prose is only one thing.
Messing with the randomness may make the problem solving capabilities weaker. Probably it doesn't but this is the answer to what else I would want them to do.
If we did, we probably wouldn’t need LLMs in the first place, i.e. we could just generate text using explicitly programmed algorithms.
> Providers of AI systems, including general-purpose AI systems, generating synthetic audio, image, video or text content, shall ensure that the outputs of the AI system are marked in a machine-readable format and detectable as artificially generated or manipulated. Providers shall ensure their technical solutions are effective, interoperable, robust and reliable as far as this is technically feasible, taking into account the specificities and limitations of various types of content, the costs of implementation and the generally acknowledged state of the art, as may be reflected in relevant technical standards. This obligation shall not apply to the extent the AI systems perform an assistive function for standard editing or do not substantially alter the input data provided by the deployer or the semantics thereof, or where authorised by law to detect, prevent, investigate or prosecute criminal offences.
https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng
It definitely makes Pangram's job a bit easier.
> When watermarking is used, choices are still made at random, but the source of the randomness is different. Instead of using an arbitrary random number generator to pick the next word, watermaking uses the key and a few words that come before to settle what word the model should pick.
> the watermark only changes the source of the randomness used to pick among words.
Dumb question - doesn't this defeat the purpose of a watermark? i.e., anyone who wants to avoid detection can simply run `while (has_watermark(text)) text = slightly_rewrite_with_non_anthropic_llm(text)` until it's gone? I feel I am missing the intent of the watermark if it is so easily defeated.
It doesn’t undermine positives though. That’s just whatever the false positive rate is.
So if it comes back as anthropic generated, it most likely is. If it comes back as non-anthropic generated, we have no idea.
If anthropic didnt make it public there would only be a narrow path for governments or something to make requests. Its kind of fucked either way.
> anybody who wants to avoid detection can just
They can just use a different LLM. By far easier and more reliable than what you're suggesting. This whole watermarking requirement is better then nothing because meant people are profoundly lazy, but yes it is not hard to work around with any effort.
What do you think the pricing per call of "has_watermark(...)" will be?
https://x.com/i/status/2087235360690741690
An important principle: Never pay someone to remove a problem that they themselves created
I understand what you are trying to say but I am not sure any watermark detection API would definitively return a true/false answer, I would have expected something more like a numeric confidence value. I am also not sure if the API would be deterministic.
However, if my understanding is correct, the reason for the watermark / detections is that its not directly aimed at end-users, but to be able from them to detect if text was produced by one of their models so they don't use it as input in training data. So, yeah, in that context, not sure why they are announcing this with an ability for anyone to detect if it was produced by one of their models. Also, they are happy to ingest text produced by models they don't own? Maybe someone with more information can elaborate?
This is a productivity enhancement for me. I am not writing art. I am delivering information for my research plan. While I would not mind a flag that indicated AI assisted for clarity, I do not want to be accused of using AI-wholesale. I put a lot of work into it, and I do not want to be maligned.
I believe they know damn well that this will lead nowhere, and are only doing this to mitigate criticism.
https://towardsdatascience.com/text-classification-and-the-b...
E.g. prompt Claude to write all sentence in reverse, or swap every 2 words etc. Then use a script to put reorder in the right ordering?
Let's say we are at token 431 and there is 49% to generate token 1 and 51% to generate token 2, we apply bias to our token 1 which would make it win causing a repeating pattern invisible to the human eye.
Now you apply this to multiple tokens and a reversible source of random you have a pretty strong watermarking system... That is rather annoying to defeat as you essentially have to rewrite most of the text. The alternative is to use a diffusion model and spray some gaps across non-literal information such as ids, links, etc.
> We will soon be offering a watermark detection API. We’re in the process of working out the details of its implementation.
Determining whether it's written by Claude will be possible in the future. But unless you know the LLM being used and the company behind that LLM offers a similar API, there's no easy way to tell if it's AI generated in general.
Is poor proofreading a form of watermarking? Clever, I suppose, but they should consider running posts through Sol for clarity.
As bad as Claude Code’s writing is, it wouldn’t make that mistake.
Wouldn't pi contain any such sequence of numbers? Therefore you'd have to allow only certain regions of pi, and therefore, its not random anymore and we could just shortcut the whole game?