3 pointsby lhein5 hours ago3 comments
  • jchaselubitz5 hours ago
    Are there other projects like this? Also, what results have you been able to find on historical data?
    • lhein4 hours ago
      Most similar is CT Open, which is also a trial outcome predictor. ForecastBench does the same but for general questions.

      For 111 trials reported after the cutoff date, Opus/Sonnet/Haiku was too pessimistic (~59% expected vs. 73% observed) and was beaten by a plain regression on the registry features. Their ordering of the trials was good though, and combining with the regression got them 8% accuracy increase. Small sample size; thus, the online version

  • jchaselubitz5 hours ago
    This is very cool!
  • 4 hours ago
    undefined