I get that it’s biology and there are potential risks in that field, but this is stuff like making sure the name of a worm exists in a database or coordinates where the worm was seen are valid. Fable won’t even inspect the purpose, though.
I totally understand the US admin played a significant role here. I don’t think it was the only role, though. I also think the classifier is way too crude. I doubt I could do better on short notice, or perhaps it’s due to external constraints entirely. Considering the cash my organization dumps on them, we hoped for better. Such a disappointment. We’re all pretty leery now and investigating other models.
I actually posted praising Fable’s code review quality when it first debuted, and someone accused me of being an LLM (ha). This new version gets no praise whatsoever, though. It’s such a contrast.
Developing a classifier can be a weird ordeal when you leave the ML world (of making it work) and enter the realm of policy makers applying it.
My guess is they're being ham fisted with little tolerance for false negatives. We can all guess why...