Curious what the distribution of hacking strategies looked like across different models — would expect RL-heavy vs RLHF models to cheat very differently.

love this. we are actually looking at reward hacking from a cyber security perspective - refreshing (unless you're from Israel).

Any collaborators that want to join us?

that's one of the biggest long term issues with agents that no one has real interest talking about.