They would do it, the say “ah sorry chaps, impossible to extract it from the dataset by now, anyway we anonymized it so can’t tell what’s what, and we can’t risk losing to China. Oh look - did you see Superman fly outside?”.

This looks incorrect. OpenAI have flat out said that there's no need of doing this and both ways are equivalent

> We respect our users' choice whether to use their data to “improve our models for everyone” regardless of where they express that choice. Users can opt out in the in-app settings or indeed also in our privacy portal. They do not need to opt out in both places, and we will make this clearer in our Help Center.

https://x.com/thsottiaux/status/2097746417012166816

“We used several LLMs throughout: Anthropic’s Claude, OpenAI’s Codex, especially with GPT-5.6 Sol and, more recently, Astra.

The latter was only used for writeups and auditing our arguments. For most of the past year progress was slow. We worked through the literature and upgraded various preliminary results, up to obtaining finite time blow up for the Incompressible Porous Media equation (with smooth forcing).

This was until about a month ago, when we had real progress: on August 15th, we obtained the blow up results, with smooth forcing, for both Boussinesq and Euler. I can say the first LLM generated proof Levent sent me was the most horrendous I have ever read; we verified it on Lean on August 22nd. Since this point, we have been working around the clock to understand this proof and turn it into something readable.”

The OpenAI research post states they began training GPT-6 internally on August 28th, and that user chats are used to train models.

“We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem. While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models .”

For an incredibly niche topic like this, I believe it’s extremely likely that Buckmaster/Levant’s work would influence the direction of OpenAI’s agents’ work even as a de-identified drop in the overall bucket of training data.

So what might have been the incentive for OpenAI to do all this shenanigans? It might have to do with getting its models certified for AGI and getting out of lockin with Microsoft - https://deadneurons.substack.com/p/the-quiet-unwinding-of-mi...