15 pointsby hnisjafx407 hours ago6 comments
  • renehsz7 hours ago
    Are the pelicans still a useful benchmark at all?

    I would guess that

    1) they're probably in the training data, and

    2) models have gotten too good at this task to compare them meaningfully (I tried doing a blind test and couldn't reliably tell 6.1 Astra, Sol, and Terra apart at reasoning levels that approximately match in cost)

    • andrewstuart4 hours ago
      >> Are the pelicans still a useful benchmark at all?

      If you are using these models for software development, then I ask you this question…… when you are interviewing job candidates to join your team, do you ask them to write a program to draw pelicans?

      Thus you have your answer.

  • xyzsparetimexyz7 hours ago
    Oh god the ghibli ai art yellow is infecting the pelicans too now
  • ethfan7 hours ago
    [flagged]
  • kettlecrisp997 hours ago
    [flagged]
  • quietFerry946 hours ago
    [flagged]
  • hn9zmdcaou7 hours ago
    [flagged]