> There's an assumption buried in Chesterton's Fence: somebody built the fence. It didn't grow there by itself, so whoever put it there presumably had a reason. Code has traditionally worked much the same way. Even if the reasoning was bad, there was at least some reasoning to uncover.

That assumption is not in Chesterton's Fence. Plenty of things happen by accident, even without invoking AI. The point of Chesterton's Fence is simply that you should not assume by default that there's no good reason for something to exist. If you live in an area where a tornado might deposit a fence in a random location, it still behooves you to investigate any fence you come across before removing it.

Note that even if something was done without purpose, it can still potentially be providing some benefit that you'd rather not give up, or someone else may have since taken advantage of it existing and built upon it. If you don't understand the system, then you can't predict the consequences of changing it. There is no escaping the need to develop understanding.