Humans not staying in the loop when doing evals like ExploitGym are engaged in gross negligence. There are some people saying we should use our existing laws today for the HF et al incidents and hold some humans liable
Remember that first time you heard about speculative execution attacks (Spectre)? That was something new, something unheard of. If an "AI" is going rogue, something previously unheard of would show up. Otherwise, they're just doing what patterns are in their training data, and doing a lot of it. And, it's a human telling them to do so.
"Escaping" would be using the attack to install a copy of itself somewhere. But we've seen that before, too, with viruses and worms. And if it has to install an 8 GB image to install a copy of itself, it's not going to be the stealthiest of worms...