First-author here! Feel free to ask direct questions. I’m pleased to see connections we had not made (searchlight hypothesis in particular). As a physicist I can see epistemological differences that can affect the proper placement of the work in a set of possible applications, I’ll gently address those at the end.
It looks like folks here really care about two main things: What does this imply about neuronal integration of information (ala capsNet), and there is an interest in trying to relate it to the physical architecture of the brain. Personally, I also care about exploiting what we found in real neural systems.
Neuronal Integration of Information:
There is a rather “drastic” limitation to the way neurons integrate information, they cannot be logic gates. I mean in the traditional sense where there are two populations inputting to the neuron and the neuron fires if only one or the other population is active but not both, etc. It’s easy to see why: If both populations are active than an MEA might show a very big “neuronal avalanche”. If the neuron is computing an XOR gate then it would show a very small “synaptic avalanche”. So here we would see that small synaptic avalanches would be over-represented and large synaptic avalanches would be under-represented when compared to the neuronal avalanche counterparts. The same is true for an AND gate but not an OR gate. We mostly cut this from our discussion. The effect is limited so a neuron could be weakly XOR-like or weakly AND-like and cross-scale inference is still possible.
However, we think there is a way out of this. Our model enforced the condition that each neuron’s instantaneous firing rate was equal to the population firing rate of its input population. This is the same as saying that each neuron computes the average or the sum of the firing states of neurons within its presynaptic population (OR-gates only). If a neuron is OR-like when the level of synaptic bombardment is low, but then, as bombardment and membrane potential increases it becomes more “focused” on certain populations of inputs then it could potentially stay true to “cross-scale inference” while effectively spiking as an AND-gate. This would defy our model’s conditions for criticality (but that’s not worrisome, see below).
I also saw, in the title and a couple other times, the question of whether single neurons represent ensembles. This is highly relevant to work we are proceeding with now. If our model’s conditions for criticality hold (each neuron’s instantaneous firing rate equals the fraction of neurons which fired in its presynaptic pool), then the question becomes whether single neurons always represent the same ensemble. If, like the searchlight hypothesis, the inputs can be “focused” (some become active, some become silenced) the neuron may not represent an ensemble, but may still allow cross-scale inference.
Exploiting Cross-Scale Inference
We debated publishing in J Neurosci because there is a lot of methods detail that is worth including in a supplemental, but not in the main paper which is already quite long (they don’t allow supplementals). This will make reproduction more challenging, but we do expect to share the code eventually. We won’t share it until it is refactored according to publishing standards, fully commented, and unit tested on other data.
This method (using the statistics of signal geometry) would probably only work for long-time-averages and it requires careful normalization. In otherwords, it needs a lot of data and very careful conditions. You can find in my YouTube Profile another video explaining preliminary results on efforts to get a more instantaneous view. We greatly improved results by completing the hyperparameter optimization effort that was unfinished at the time of that video and are just about ready to submit.
Relating the work to the physical architecture of the brain
This is basically an open frontier, and requires some knowledge about this “criticality stuff”. Basically, everything we physicists think we know about “self-organizes-criticality” comes from computer models, it’s never been definitively observed in the wild. In those models, the signal geometry relates very strongly to the physical geometry of spreading activity. Because synaptic connections can skip around neighboring sites it’s incredibly hard to observe this in neural data.
I can link to articles about it, but the takeaway is these three facts: 1) There must be a relationship to connectomics but we don’t know it. 2) The fractalness of the brain’s physical architecture is probably coincidence (scale-free dynamics do not require scale-free networks). 3) In most models “criticality” is enforced by setting precise physical parameters (e.g. connection strengths/probabilities), not dynamical parameters. We can only detect “signatures” for criticality by looking at dynamics but these can be buried if exogenous drive overwhelms endogenous dynamics.
Epistemologies
The best purely academic reading, aimed at explaining this physics stuff to a non-physicist, is this article: Emergent Complex Neural Dynamics
mechanisms
The idea of self-organized-criticality (SOC), if true, is an explicit alternative to the concept of a “mechanism”. This is because of the better-known idea of “emergence”. If a person is studying emergent phenomena in an SOC system they might find a mechanism, say for a response to a kind of stimuli. They might demonstrate sufficiency, and everything seems to check out. Then somebody comes along and lesions a part of their mechanism… The response is still there, maybe it’s different but it’s pretty much the same. Our very clever person finds a new mechanism! Somebody lesions it too, and maybe it requires lengthy recovery or even rehab, but the animal can still be shown to have the response. This goes on ad nauseam until the researcher decides responses have very many “sufficient” mechanisms but rarely have “sufficient and necessary” mechanisms. The relevant fact about SOC systems is that they are “metastable”, they are always in flux. This means they are always “exploring” slightly different modes of operation. If there is an external optimizer (e.g. a reward circuit or protocol), then it is not limited to any one solution/mechanism.
This is very much a physics kind of idea. Hydrogen emerged out of the stew of particles after the big-bang because there is a “reward circuit” in the form of energy minimization. Naturally, the number of mechanisms governing the bonding of particles is smaller than the number of paths through a network so the analogy is imperfect. The general idea is called an “extremal principle” (it has a wikipedia page). Extremal comes from “extremum”, another word for the minimum or maximum of a function, or the optimum of an optimization problem. Extremal principles only apply to systems in flux. Being SOC is one way to be in flux. Simply apply an optimization constraint and the system will find the mechanism which is optimum given all the constraints on the system, including the one you just added. A physicist thinking this way about implications of SOC would not really care about naming the mechanism behind the implication, we just assume there is always at least one, if not infinitely many.
critical branching
If you read the newest Quanta Magazine article I shared, you’ll see that we don’t really know what the brain looks like when it is “subcritical” or “supercritical”. However, for critical branching networks, we do know, and the “off-critical” brain does not look much like the “off-critical” branching network. This is no problem.
I’d ask you to read the Wikipedia page on “universality classes”. AT CRITICALITY the details of your model are irrelevant to the geometrical statistics we gathered. The further a system is away from criticality the details become exponentially more relevant. We use critical branching because it is easy and somewhat plausible. If we add inhibitory sites to the branching network we get the oscillations and synchrony discussed in the Quanta article (though the researchers interviewed might not be aware).
This too is a pretty weird way for a non-physicist to look at things, but it has a pretty big plus side: It reduces the number of complicated ideas one has to understand. For questions limited to signatures of criticality one probably has infinitely many models to choose from, so you don’t have to care about details, but do have to keep an open mind about off-critical things. For example, we showed that cross-scale inference is best at criticality in our model. For the real brain, the critical point behavior is the only thing that is definitively transferable, so all we really know is that at criticality cross-scale inference is pretty good. I doubt it’s better away from criticality because the correlation length and time are shorter away from criticality, and our model is vaguely similar, but this is merely common sense, not proof.
There are also two fun conjectures stemming from universality that I like to think about and give a sense to how brain scientists can use it. 1) Different brain systems could be critical in different ways. Maybe V1 is critical with the branching/averaging action that we describe, but M1 is critical some other way. 2) Criticality may operate like a clutch or differential in an automobile. Non-critical phases are strongly stereotyped, they are “attractors” and will “pull” the system into strongly patterned behaviors. If two different brain systems do have very different architectures then being at the critical point (where no pattern can dominate) may facilitate communication.
Summary
The fact that synaptic avalanches and neuronal avalanches scale the same way limits the ways that single neurons can act like logic gates. This suggests feature-selection type integration.
Importantly we have shown that single-neurons can act like sensors of larger network activity and that subthreshold fluctuations of membrane potential have fine structure which carries information long neglected. Soon we’ll put a paper on arxiv or bio-arxiv that shows how to get stimulus information out of the fluctuations.
The relationship to brain architecture and fundamental mechanisms is both an open frontier and a red-herring. Criticality begets universality and emergence, these are mechanism and architecture agnostic properties that would be very useful for any stochastic self-assembling and self-regulating information processing system to have. While universality and emergence are agnostic to the particular arrangement of matter, criticality itself requires a specific arrangement and that is usually very fragile.
These concepts require an abandoning the goal of nailing down all the details of a system, or the precise sequence of an action. We are approaching the brain as a non-equilibrium statistical mechanics problem. In such systems, one can get very precise and reliable behaviors and actions via odd kludges pulled from an infinitely large grab bag of mechanisms and sequences. It may be different from brain to brain, or day to day. This is both liberating and limiting. Some mechanisms in the brain are obviously quite specific and do have both necessary and sufficient mechanisms (e.g. amphibian looming reflex) so this simply isn’t applicable to all things.
