It also seems like this would break contract law if suddenly there's "contracts" made with a machine that are unenforceable.
I wonder how lenders would deal with lending money to companies that are obfuscated behind multiple layers of AI shell companies? Maybe someone will test it out by having AIs create companies that take out loans on some hallucinated assets, pay for some "service" from the human owner, and then get put through bankruptcy.