There are places where matrix expressions are informative and helpful, but at least in the context of teaching machine learning, this isn't one of them.
I find using indices and summations generally confusing and hard to read/follow/understand. To me, it's so much simpler to draw out the computational graph, derive the gradient with a single element (so I'm in agreement with you guys here?), see how that applies the same to every element the same, then scale up to the vectorized/matrix version.
> and you need to memorize a bunch of new rules to apply it
There's nothing to memorize though..?
Is there something called "matrix calculus" that's different than just drawing the computational graph and deriving the gradient in the way I described..?
Like, is this "matrix calculus"?
- https://cs231n.github.io/optimization-2/
If so, what is there to memorize here?
Before einsum syntax, it helped if the result could be written in terms of matrix operation because then one could utilise the primitives offered by matrix libraries. The alternative was to write the raw low level index loops, tedious and errorprone.
For some cases though the matrix result is just more compact. For example the derivative of log determinant of matrix wrt the matrix.