Let’s review what we know about inhibitory interneurons. HTM theory says that they’re responsible for running the winner-takes-all competition and that makes the representations sparse. However, HTM theory simplifies things too much. We know all of the details about how real inhibitor cells work, but we haven’t really analysed how those details effect the system.
2 Likes
Forming sparse representations by local anti-Hebbian learning
P. Foldiak 1990
Abstract. How does the brain form a useful representa-
tion of its environment? It is shown here that a layer of
simple Hebbian units connected by modifiable anti-
Hebbian feed-back connections can learn to code a set
of patterns in such a way that statistical dependency
between the elements of the representation is reduced,
while information is preserved. The resulting code is
sparse, which is favourable if it is to be used as input to
a subsequent supervised associative layer. The opera-
tion of the network is demonstrated on two simple
problems.Link: https://redwood.berkeley.edu/wp-content/uploads/2018/08/foldiak90.pdf
4 Likes
So what is an interneuron really? Is it a functional distinction or a structural distinction? There’s a lot of assumptions when neuroscientists mention them in their papers and I’d like to know what those assumptions are.
The wikipedia article on interneurons mostly explains it from a spinal cord point-of-view and gives very unhelpful explanations like:
“interneurons are the central nodes of neural circuits, enabling communication between sensory or motor neurons and the central nervous system”
Not very helpful.
Of course, we’re talking about the cortex, so we’re primarily interested in how a cortical interneuron differs from a cortical pyramidal neuron. The latter is something we have a good intuition of since all of our artificial neural network methods and the HTM neuron uses pyramidal neurons as their inspiration.
So what does an interneuron mean in the cortex?
Is it an elongated neuron that is more dendrite/axon than soma?
Is it the same as a pyramidal cell but with limited input/output responsibilities?
Are the biological/structural differences between pyramidal cells only the dendrite/axon connectivity or is it a fundamentally different type of neuron cell with different computational properties?
What do the interneuron connections look like in a neuronal circuit and what are their functions?
It’s just frustrating that I haven’t been able to find straightforward answers on what is and isn’t known about interneurons and a more focused explanation on what they actually are.
3 Likes
Inhibitory means that the cell uses GABA as its neurotransmitter.
Interneuron means that it is not a pyramidal neuron.
Here is a good review of the subject:
INTERNEURONS OF THE NEOCORTICAL INHIBITORY SYSTEM
Henry Markram, Maria Toledo-Rodriguez, Yun Wang, Anirudh Gupta, Gilad Silberberg and Caizhi Wu
2004Abstract Mammals adapt to a rapidly changing world because of the sophisticated cognitive
functions that are supported by the neocortex. The neocortex, which forms almost 80% of the
human brain, seems to have arisen from repeated duplication of a stereotypical microcircuit
template with subtle specializations for different brain regions and species. The quest to unravel
the blueprint of this template started more than a century ago and has revealed an immensely
intricate design. The largest obstacle is the daunting variety of inhibitory interneurons that are
found in the circuit. This review focuses on the organizing principles that govern the diversity of
inhibitory interneurons and their circuits.Link: https://www.researchgate.net/publication/8336946_Interneurons_of_the_neocortical_inhibitory_system
2 Likes
also worth to mention that they vary wildly in shape, electrical behavior and plasticity, but almost all inhibitory neurons except for the ones in the basal ganglia project short range connections.
3 Likes
Off topic but I find this paper’s abstract very much inspiring!
The brain receives a constantly changing array of signals from millions of receptor cells, but what we experience and what we are interested in are the objects in the environment that these signals carry information about. How do we make sense of a particular input when the number of possible patterns is so large that we are very unlikely to ever experience the same pattern twice? How do we transform these high dimensional patterns into symbolic representations that form an important part of our internal model of the environment? According to Barlow (1985) objects (and also features, concepts or anything that deserves a name) are collections of highly correlated properties. For instance, the properties ‘furry’, ‘shorter than a metre’, ‘has tail’, ‘moves’, ‘animal’, ‘barks’, etc. are highly correlated, i.e. the combination of these properties is much more frequent than it would be if they were independent (the probability of the conjunction is higher than the product of individual probabilities of the component features). It is these non-independent, redundant features, the ‘suspicious coincidences’ that define objects, features, concepts, categories, and these are what we should be detecting. While components of objects can be highly correlated, objects are relatively independent of one another. Subpatterns that are very highly correlated, e.g. the right and left-hand sides of faces, are usually not considered as separate objects. Objects could therefore be defined as conjunctions of highly correlated sets of components that are relatively independent from other such conjunctions.
2 Likes
This article explains the math and computer-science behind the winner-takes-all competition.
Robust parallel decision-making in neural circuits with nonlinear inhibition
Birgit Kriener, Rishidev Chaudhuri, Ila Fiete (2019)
DOI Link: Robust parallel decision-making in neural circuits with nonlinear inhibition | bioRxivAbstract
Identifying the maximal element (max,argmax) in a set is a core computational element in inference, decision making, optimization, action selection, consensus, and foraging. Running sequentially through a list of N fluctuating items takes N log(N) time to accurately find the max, prohibitively slow for large N. The power of computation in the brain is ascribed in part to its parallelism, yet it is theoretically unclear whether leaky and noisy neurons can perform a distributed computation that cuts the required time of a serial computation by a factor of N, a benchmark for parallel computation. We show that conventional winner-take-all neural networks fail the parallelism benchmark and in the presence of noise altogether fail to produce a winner when N is large. We introduce the nWTA network, in which neurons are equipped with a second nonlinearity that prevents weakly active neurons from contributing inhibition. Without parameter fine-tuning or re-scaling as the number of options N varies, the nWTA network converges N times faster than the serial strategy at equal accuracy, saturating the parallelism benchmark. The nWTA network self-adjusts integration time with task difficulty to maintain fixed accuracy without parameter change. Finally, the circuit generically exhibits Hick’s law for decision speed. Our work establishes that distributed computation that saturates the parallelism benchmark is possible in networks of noisy, finite-memory neurons.
2 Likes
This article is not surprising, but the data is useful nonetheless.
Precision of Inhibition: Dendritic Inhibition by Individual GABAergic Synapses on Hippocampal Pyramidal Cells Is Confined in Space and Time
Fiona E. Müllner, Corette J. Wierenga, and Tobias Bonhoeffer (2015)
DOI Link: RedirectingAbstract
Inhibition plays a fundamental role in controlling
neuronal activity in the brain. While perisomatic
inhibition has been studied in detail, the majority of
inhibitory synapses are found on dendritic shafts
and are less well characterized. Here, we combine
paired patch-clamp recordings and two-photon
Ca2+ imaging to quantify inhibition exerted by indi-
vidual GABAergic contacts on hippocampal pyrami-
dal cell dendrites. We observed that Ca2+ transients
from back-propagating action potentials were signif-
icantly reduced during simultaneous activation of
individual nearby inhibitory contacts. The inhibition
of Ca2+ transients depended on the precise spike-
timing (time constant < 5 ms) and declined steeply
in the proximal and distal direction (length constants
23–28 mm). Notably, Ca2+ amplitudes in spines were
inhibited to the same degree as in the shaft. Given
the known anatomical distribution of inhibitory syn-
apses, our data suggest that the collective inhibitory
input to a pyramidal cell is sufficient to control Ca2+
levels across the entire dendritic arbor with micro-
meter and millisecond precision.
2 Likes
Here’s another comprehensive mathematical analysis of WTA.
On the Computational Power of Winner-Take-All
W. Maass (2000)
DOI Link: On the Computational Power of Winner-Take-All | Neural Computation | MIT Press
PDF: On the Computational Power of Winner-Take-AllAbstract
This article initiates a rigorous theoretical analysis of the computational power of circuits that employ modules for computing winner-take-all. Computational models that involve competitive stages have so far been neglected in computational complexity theory, although they are widely used in computational brain models, artificial neural networks, and analog VLSI. Our theoretical analysis shows that winner-take-all is a surprisingly powerful computational module in comparison with threshold gates (also referred to as McCulloch-Pitts neurons) and sigmoidal gates. We prove an optimal quadratic lower bound for computing winner-takeall in any feedforward circuit consisting of threshold gates. In addition we show that arbitrary continuous functions can be approximated by circuits employing a single soft winner-take-all gate as their only nonlinear operation. Our theoretical analysis also provides answers to two basic questions raised by neurophysiologists in view of the well-known asymmetry between excitatory and inhibitory connections in cortical circuits: how much computational power of neural networks is lost if only positive weights are employed in weighted sums and how much adaptive capability is lost if only the positive weights are subject to plasticity.
2 Likes