Author here. I ran 40 ports of my codebase (Python to TypeScript and back). I accidentally left pycache in the sandbox, which the agents used to decompile the bytecode and reverse-engineer the tests I'd withheld - so half the results went in the bin, along with anything about the effect of tests on port quality.

I learned cheating is hard to stop. I didn't answer whether the ports are faithful - more on that in the article.

Also have reproducible research repo here: https://github.com/thekevinscott/ports-research

This is what my Claude and Codex worked on for me. There’s a report called Tesla Made that’s really impressive but quite heavy. I needed a tool that could even run on my small Raspberry Pi zero 2, so I gave AI access to Tesla Made and asked it to create a completely different project using the same code base but in a different language Golang. And it did. I’ve now been using this new tool for 5 months and it’s working well.