Learning and Inferring Relations in Cortical Networks

From the abstract: “… we show how uniform modules of excitatory
and inhibitory neurons can be connected bidirectionally in a network that, when
exposed to input in the form of population codes, learns the input encodings as
well as the relationships between the inputs. STDP learning rules lead the
modules to self-organize into a relational network, which is able to infer
missing inputs,restore noisy signals, decide between conflicting inputs, and
combine cues to improve estimates. These networks show that it is possible for
a homogeneous network of spiking units to self-organize so as to provide
meaningful processing of its inputs. If such networks can be scaled up, they
could provide an initial computational model relevant to the large scale
anatomy of the neocortex.”


1 Like

Hi Michael,

Thanks for sharing the paper. I’ve worked in the past with models of STDP, and I’m very interested in applying those concepts to HTM theory. However, this might not be completely straightforward.
Although the notion of LTP might be directly applicable to the learning mechanisms of the spatial pooler (SP) and the temporal memory ™, I’m not sure how would that be for LTD. In models of STDP synaptic connections between 2 cells become depressed if the two cells fire in a short time-window, and the order is post-pre, whereas in the SP and the TM, for depression to occur one of the cells needs to be inactive. Also, I might not be entirely right on this, but in the SP and the TM potentiation and depression occur in a linear fashion, whereas in STDP the weight update is non-linear.
Perhaps someone more familiar with the learning mechanisms of HTM theory could give us more insight about this.


1 Like

At least some types of neurons have a baseline firing rate, so LTD can occur by chance when the neuron is essentially off. I’m not sure if some neurons have really low baseline firing rates, but that doesn’t matter as long as the LTD window is longer than the LTP window. Without a correlation, LTD will happen more often than LTP.
Source: http://www.sciencedirect.com/science/article/pii/S0896627300000088