If you accept (or are willing to hypothesize) that a neural network is a sequence of context selected linear mappings, then applying SVD to the matrices is very telling:
https://archive.org/details/the-broad-if-statement-in-a-neural-network
1 Like