12 pointsby codexon3 hours ago3 comments

codexon3 hours ago
This paper creates a new benchmark comprised of real remote work tasks sourced from the remote working website Upwork. The best commercial LLMs like Opus, GPT, Gemini, and Grok were tested.
Models released a few days ago, Opus 4.6 and GPT 5.3, haven't been tested yet, but given the performance on other micro-benchmarks, they will probably not be much different on this benchmark.
- kolinkoan hour ago
  They didn't test Opus at all, only Sonnet.
  One of the tasks was "Build an interactive dashboard for exploring data from the World Happiness Report." -- I can't imagine how Opus4.5 could've failed that.
zb314 minutes ago
You think they don't? You think AI can replace programmers, today?
Then go ahead and use AI to fix this: https://gitlab.gnome.org/GNOME/mutter/-/issues/4051
Venn13 hours ago
ChatGPT: when you want spellcheck to argue with you.