Subutai Ahmad on Small-World Structures - April 28, 2021

In this research meeting, joined by Rosanne Liu, Jason Yosinski, and Mitchell Wortsman from ML Collective, Subutai Ahmad explains the properties of small-world structures and how they can be helpful in Numenta’s research.

Subutai first discusses different network types and the concept of small-world structures by reviewing the paper “Collective Dynamics of ‘Small-World’ Networks” by Watts & Strogatz. He then evaluates the efficiency of these structures and how they are helpful in non-physical networks by looking at Jon Kleinberg’s paper “Navigation in a Small World.” Subutai also addresses how small-world structures would apply to machine learning by using concepts from the paper “Graph Structure of Neural Networks” by Jiaxuan You et al… Lastly, the team discusses how small-world structures relate to Numenta’s research such as sparsity and cortical columns.

More information on ML Collective: https://mlcollective.org/

7 Likes

Jason mentioned a paper towards the end of the meeting on loss landscapes of neural nets. Here are the links to the papers if interested:

4 Likes

An interesting talk. Thank you all.

If you needed a route (justification?) for the use of the small-world networks described, then the biological structure of a growing network (i.e. mainly preferential attachment) might favour scale-free graphs.

SF graphs probably have other implications too (beyond small-world features) so I would get a graph theory guru to comment first.

1 Like