I've been a loan officer for 20 years. We never trust a borrower based solely on their current balance sheet (the final weights). We look at the trajectory of their cash flow over the past 3 years (the training process).
Two companies can arrive at the same 'profit' today, but one might be evolving towards bankruptcy while the other is evolving towards dominance. You can't see that snapshot in the final model.
If we want to trust these 'black boxes,' we need to see their credit history, not just their credit score.