I looked at the BaMI pdf containing the pseudocode of HTM algorithm. Am I right to say that the book version calls a “column” as a set of 4 cells and a region as a set of several “columns”? The terminology is confusing, a column is described as a computational unit with several layers from Jeff’s recent talk.
Also, the concept of feedback is not part of the HTM algorithm…right? Additionally, can you describe what is the feedback doing by getting the info from top to the bottom intuitively?
How does the voting across columns work from whats known? Is there a specific region (associative region) which takes in inputs and decides the final output?
Also, the 1000 brains algorithms uses HTM still as the core learning algorithm…right?
I see several repos from Numenta on Github. Can I know which ones are still relevant for people (like me) to play around with?
Following up, I see nupic.torch repo containing sparse CNN implementation. Any relevant blogs/papers explaining whats changed compared to traditional CNNs?
Is there an ETA on when the 1000 brains project would be released? just curious…
it was awesome to read and understand neuroscience findings from Numenta, coming from the traditional ML background. Thanks.
I’m more familiar with HTM than Thousand Brains theory, so I think I can answer your first 3 questions (until someone more educated comes to answer).
The set of 4 cells is a minicolumn (within a layer). There are regions, columns and minicolumns. A column contains about 100 minicolumns. Each minicolumn contains about 100 cells, making each column contain 10000 cells total.
Columns are a Biological observation, and I guess genetics decides what a column is, and then just spams columns until it fills a baby mammal’s skull. When it comes to HTM algorithm, I don’t think we care about columns at all. We more care about the 2% sparsity within a region, and having enough active cells to be useful (so like 10-40 for temporal memory), so we would define a “region” of about 2048 minicolumns, effectively simulating about 20 columns or 40-ish active cells each timestep to use.
Feedback just biases our cell to fire. If I see part of a paperclip (maybe it’s obscured), and my L2/3 cells are thinking of a paperclip, That’s kind of a 2nd depolarization that could happen for cells representing that object.
I could be corrected, but I believe “voting” across columns is really just long range dendrite segments connecting to things in the same layer. In a way, Temporal Memory is reaching across columns, and letting whatever’s active “vote” for which cells are depolarized, and hopefully fire in the next step. I think when numenta talks about voting they are referring to L2/3, which decides the representation of the “object”/“situation”. These cells, as well as taking L4 input also connect to each other, and encourage other L2/3 cells to fire, meaning we could have a stable pattern in this layer, even if the input changes. I think it’s possible to have an active SDR in this layer without actually having the correct input since if enough L2/3 cells representing that object are firing, it encourages other cells representing that pattern to fire too. Very useful if I want to find something in the input (feedback) rather than let the input decide what it sees/experiences. There are a few papers (for htm) that describe possible implementations of this, but I don’t think they’ve come up with an actual algorithm. They just say “it happens”. Tho maybe they figured out how this works in Thousand Brains Theory and I’ve just missed it.