To anyone who is unfamiliar with LM Arena, this isn't a synthetic benchmark but basically an A/B test.

Human users submit the same prompt to two models and then pick with output they prefer.

Also very good for free image gen, for what it's worth. You just enter the same prompt 4 or 5 times and eventually it goes to a good model.

Welcome to the future. I think open weight models are our only hope for LLMs being net positive for society.