Show HN: AI Vision Index, private tests of how well 36 AI models see(aivisionindex.com)
2 pointsby LLD3 hours ago1 comment
  • sharemywin2 hours ago
    what's interesting some of the models do better at various task types. I wonder if you built a simple decision tree and tried a new out of sample test how well it would score.
    • LLDan hour ago
      Good question, so I just tried it. On each test I picked the best model using half the questions, then scored that pick on the other half (repeated 300 times):

      Best single model (GPT-6 Astra): 87.8% right

      Best model per task type: 86.9%

      So routing doesn’t beat just using the top model. With not enough questions per test, many of the gaps between top models are noise. We would need more questions to test routing properly. Interestingly, routing seems to pick cheaper models (Gemini 4 Argon, GPT-6.1 Sol), so it costs about half as much for about one point less.