Another potential implication of Uber’s successful experiments with intrinsic dimension is that you could have self modifying neural networks. With recurrent neural networks the output of the neural network is fed back into the input. With a self modifying network part or all of the output could be used to modify all the weights via a dimensionality expansion.
https://eng.uber.com/intrinsic-dimension/
1 Like