This is a community-created and managed reference card about HTM theory and implementation for easy lookup for common information. Please add to it if you have something interesting to add.
If you see “?” and know the answer, please fill it up.
- 5% of a human’stotal body weight, it uses 20% of the body’s oxygen.
- Total number of neurons in the brain : 86 billion
- Energy consumption : 20 W
- 1 sq. mm. contains : 170 000 neurons
- 75% of the brain
- Total number of neurons in the cerebral cortex : 21-26 billion
- Total number of neurons in the neuro-cortex : ??
- Total number of connections : ??
- Total number of synapses : 100-180 Trillion (depends on sex and age)
- Neo-cortical sheet is about 1000 cm^2 in area and 2.5 mm thick
- One mm^2 of cortex has about 100K neurons and 1B synapses
- Cortex is distributed in 6 layers (there are actually more like nine layers 1,2,3a,3b,4,5a,5b,6a,6b)
- Although different regions of the neocortex process different inputs (vision, hearing, touch, language, etc.) at a fundamental level these are all variations of the same problem, and are solved by the same neural algorithms.
- The regions are defined by connectivity. Regions pass information to each other by sending bundles of nerve fibers into the white matter just below the neocortex. The nerve fibers reenter at another neocortical region. The connections between regions define a logical hierarchy.
- Neuroanatomy tells us that every region of the neocortex has both sensory and motor functions. Therefore, vision, hearing, and touch are integrated sensory-motor senses; we can’t build systems that see and hear like humans do without incorporating movement of the eyes, body, and limbs.
- A cortical column is about 1.0 - 1.5 mm^2 in area and contains about 2000+ mini-columns
- HTM: 2048 minicolumns under 1 spatial pooler
- All cortical columns are learning a complete model of the world of everything they get exposed to and they are all doing it in parallel. Each cortical column basically learns the same thing in parallel and votes (via layer 2 communication).
- About 30-50 microns wide with 100-120 neurons across all 6 layers
- HTM: 32 cells per mini-column in layer 3 implementation
Neuron (Cell in HTM)
- Pyramidal neurons are the most common type of neuron in the neocortex
- Average number of connections : 1000 - 10 000 ??
- Average number of dendrite segments : ?
- Average number of synapses per dendrite segments : ?
- The pyramidal neuron is the core information processing element of the neocortex, and synapses are the substrate of memory
Note: Need to add apical dendrites and synapses to the cell model visualization
- Takes sensor inputs and converts them into SDRs
- HTM Examples: Scalar Encoder, Random Distributed Scalar Encoder (RDSE)
- Take stimulus from the environment and translate them into a stream of SDRs that are neural activity going to the brain.
- An encoder takes some type of data–it could be a number, time, temperature, image, or GPS location–and turns it into a sparse distributed representation that can be digested by the HTM learning algorithms. The HTM learning algorithms will work with any kind of sensory data as long as it is encoded into proper SDRs.
- Vision: Retina
- Hearing: Cochlea
- Touch: Nerves
HTM Implementation Parameters
- Num Columns (N): 2048
- Num Cells per Column (M): 32
- Num of active bits (w) : 41
- Sparsity (w/N) : 2%
- Dendritic Segment Activation Threshold (θ): 15
- Initial Synaptic Permanence: 0.21
- Connection Threshold for Synaptic Permanence: 0.5
- Synaptic Permanence Increment and Decrement: +/- 0.1
- Synaptic Permanence Decrement for Predicted Inactive Segments: 0.01
- Maximum Number of Segments per Cell: 128
- Maximum Number of Synapses per Segment: 128
- Maximum Number of New Synapses Added at each Step: 32