Show HN: Can you beat an AI at "being human" using one word?(turingduel.com)
1 pointby jacob_indie5 hours ago2 comments
  • chocolateteeth2 hours ago
    Some words were not allowed. Not a good look.
    • jacob_indie10 minutes ago
      I did try a version without a dictionary (basically any string goes) but that gave the humans an unfair advantage: Typos or "sldihfs" would consistently win, and LLMs weren't able to replicate that behavior well enough.

      Same goes for slurs, etc.

      Sorry if a legit word wasn't possible, the dictionary I found and used is not perfect but was the best I could find.

      Anyway, thank you for playing and your comment!

  • altmanaltman4 hours ago
    Kind of ironic "Poop" is the word that stands out the most. But having an AI judge it seems weird. To get a true benchmark, the judge must be a human who is susceptible to the 'irrational' cues (like 'Poop' or humor) that the original paper highlighted.
    • jacob_indie4 hours ago
      Thanks for the comment, I agree re the irony of having AI judges. Human judges would just not be feasible for now...

      What is interesting though is that there are different judges and how they compare to each other (first looks at the data shows they are different).

      Also, it is interesting to see how well the AI opponents and judges are picking up personality and clues based on round history. Some LLMs pick it up very well and counter humans, some are quite "dumb" and just submit random words.

      Same for AI judges

      I do store the reasoning of opponents and judges in the background but am not displaying it for the moment; maybe something interesting to add for later, but it would distort the data ;)