I thought this video about the state dimension seen in collections of mouse neurons was very useful:
An artificial neural network can operate in any space from very low dimensional to very high dimensional. Which suggests a wastage by being too expressive. To fix that problem would be very tricky. You would have to link the behavior of nonlinear functions such that the collection never produced too simple or too complex a collective output.