Nate Silver implied Florida was in play in 2024 for Democrats, then it went +13 R. This is after he spent 9 years clinging to polling which systematically undercounted Republican support due to either sampling bias or shy voters, or were simply outright fraud in other cases (Selzer's Iowa poll).
You need to assess calibration. Not “accuracy”. What does “in play” even mean?
Models don’t “poll” their training data. If you give it five yes answers and five no answers, it doesn’t think that it’s 50/50.
If you give it actual data it can derive it, but that would be a poll