Here is an interesting paper from quite far back about random walk learning:
https://arxiv.org/abs/adap-org/9305002
It was “discovered” 2 or 3 years ago that neural networks don’t really suffer from a local minimum problem, with enough dimensions local minimums are converted into saddle point problems that weak optimizers (ie. BP) can solve if given time. From the paper they already knew that in 1993!
3 Likes
It reminds me of this study. http://eplex.cs.ucf.edu/publications/2012/soltoggio-nn12
1 Like