Random walk learning

Here is an interesting paper from quite far back about random walk learning:
It was “discovered” 2 or 3 years ago that neural networks don’t really suffer from a local minimum problem, with enough dimensions local minimums are converted into saddle point problems that weak optimizers (ie. BP) can solve if given time. From the paper they already knew that in 1993!


It reminds me of this study. http://eplex.cs.ucf.edu/publications/2012/soltoggio-nn12

1 Like