# Cell segments vs synapses

**URL:** <https://discourse.numenta.org/t/cell-segments-vs-synapses/3009>\
**Category:** Engineering\
**Created:** [October 30, 2017, 11:33am UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009 "2017-10-30T11:33:58Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rodi](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rodi/32/3720_2.png) [@Rodi](https://discourse.numenta.org/u/Rodi)\
**Post date:** [October 30, 2017, 11:33am UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/1 "2017-10-30T11:33:58Z")

</div>

can you tell me why segment per cell is 128?  
A cell has full connections with other cell, so my thinking is a cell has 2048x32 segments.

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [October 30, 2017, 3:54pm UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/2 "2017-10-30T15:54:55Z")

</div>

You can think of segments as roughly equivalent to dendrites, in this case distal dendrites. So a cell can have 128 distal dendrites, any one of which could depolarize the cell. Each segment can have many synapses (in this case 128). So the total **max** synapses per cell would be 128x128.

---

<div class="post-metadata">

**Author:** ![sheiser1](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sheiser1/32/766_2.png) [@sheiser1](https://discourse.numenta.org/u/sheiser1)\
**Post date:** [October 30, 2017, 4:42pm UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/3 "2017-10-30T16:42:08Z")

</div>

Each cell is actually not connected to all other cells in the HTM region, rather they each start out with no connections to other cells and form them through learning. When input ‘B’ arrives after input ‘A’, the cells that activate each form a dendrite segment of synapses, each synapse connecting to a cell that was active for the prior input (‘A’). That segment is now detecting for that pattern (‘A’ --\> ‘B’), and will cause those ‘B’-cells to become predictive when the ‘A’-cells activate, recognizing their place in that sequential context.

If a different pattern occurs involving ‘B’ (say ‘C’ --\> ‘B’) a different set of cells representing ‘B’ will activate, forming dendrite segments of synapses linked to the ‘C’-cells from the prior time step. Dendrite segments are significant because (among other reasons) they enable the same cells to detect many different patterns. The same set of ‘B’-cells from the ‘A’ --\> ‘B’ sequence could learn to predict ‘B’ from the separate ‘B’ --\> ‘C’ sequence, with separate dendrite segments linking to ‘C’-cells. New segments are only formed when the current input doesn’t match well to any existing segments, and each cell can form up to 128 segments.

---

<div class="post-metadata">

**Author:** ![Rodi](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rodi/32/3720_2.png) [@Rodi](https://discourse.numenta.org/u/Rodi)\
**Post date:** [October 31, 2017, 12:07pm UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/4 "2017-10-31T12:07:42Z")

</div>

Thank you so much.  
I understood that a cell is not connected to all other cell. a cell has maximum is 128 distal segments and 128 synapses per a distal segment mean that a distal segment has 128 connections with 128 other cells. is it right?. if it is right, can I ask you more questions?. we have 2048 colums, 32 cells per colums, and 40 cells active at the time. so to predict active cell of B from A. we will have 128 segment and 128 synapses per segment connect to 128 randomly other cell of A (we have total 2048x32 cells). it is so large, so how can we know sum of 128 synapse from 128 cells of A is above segment threshod is 15 or not . if sum of 128 synapses from other cells is above 15, we will set this distal segment is active, so this cell of B is predictive cell.

---

<div class="post-metadata">

**Author:** ![Rodi](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rodi/32/3720_2.png) [@Rodi](https://discourse.numenta.org/u/Rodi)\
**Post date:** [October 31, 2017, 12:20pm UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/5 "2017-10-31T12:20:04Z")

</div>

can you help me more?. I read this sentences from paper its name is Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in Neocortex

 ![10-31-2017 9-13-39 PM](https://canada1.discourse-cdn.com/flex030/uploads/numenta/original/2X/1/10075ad99e311135152f2c8340ef17dcba620bd6.png)

so D(d,i,j). D is distal segment, and max d is 128. and D(d,i,j) has connected to all other cells, so why D(d,i,j) has size is M,N with M is number cells per column and N is number of columns. so it cant be 128 synapties.

hope you can help me.  
Thank you.

---

<div class="post-metadata">

**Author:** ![sheiser1](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sheiser1/32/766_2.png) [@sheiser1](https://discourse.numenta.org/u/sheiser1)\
**Post date:** [October 31, 2017, 3:12pm UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/6 "2017-10-31T15:12:47Z")

</div>

If a cell’s distal segment has 128 synapses on it each will connect to a different cell that was active in the prior time step, so it is potentially connected to 128 other cells yes. The ‘B’-cells that become predictive when input ‘A’ arrives are predicted because they have at least one segment with a large proportion of its synapses connected to active cells (representing ‘A’ in this case).

Remember that each cell begins with 0 distal segments and forms them as needed for new contexts, so at a given time most cells likely have closer to a handful or 2 of segments than the maximum possible 128. When an unfamiliar pattern occurs, say ‘B’ arrives when it was not predicted, the set of cells that become active to represent this new ‘B’ are each chosen because they have the least number of distal segments of all cells in their SP column. This means that you’ll wind up with many ‘B’-cells that have a few segments, not a few cells that are packed with 128 segments.

At each time step, each cell monitors all of its segments to see if any have more than 15 active synapses. They don’t sum the active synapses from all their segments, only within each segment. If one of a cell’s segments has more than 15 active synapses then that segment is considered active and the cell is put into the _predictive_ state. D(d,i,j) represents the d’th segment on the i’th cell in the j’th column, and has an NxM matrix containing its permanence values to all other cells in the region. However this matrix is extremely sparse, so the VAST majority of its values will be 0. In fact there will only be s (128) non-0 values, since that’s how many synapses are on each segment.

Overall I’d highly recommend carefully watching Matt’s HTM school episodes on Temporal Memory and also re-reading the Temporal Memory pseudocode in the BAMIbook online. I actually re-wrote the pseudocode into my own notebook, which helped me internalize how it works. Here are the links below:

  
https://www.youtube.com/embed/1OhY_u3NjdM?feature=oembed&wmode=opaque&list=PL3yXMgtrZmDqhsFQzwUC9V8MeeVOQ7eZ9  

> **[Temporal-Memory-Algorithm-Details.pdf](https://numenta.com/assets/pdf/temporal-memory-algorithm/Temporal-Memory-Algorithm-Details.pdf)**
>
> 329.84 KB

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [October 31, 2017, 3:27pm UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/7 "2017-10-31T15:27:43Z")

</div>

> [@HTM Cheat Sheet](https://discourse.numenta.org/t/htm-cheat-sheet/828/24):
>
> that a distal segment has 128 connections with 128 other cells. is it right?

I would not make any assumptions about how many different cells each dendrite connects to. The pyramidal neuron should not care where it’s connections originate. Maybe there are many connections between two cells? I’m not sure, but I would not make that assumption.

> [@HTM Cheat Sheet](https://discourse.numenta.org/t/htm-cheat-sheet/828/24):
>
> we will have 128 segment and 128 synapses per segment

In practice, cells don’t usually grow so many segments. In simple pattern examples, there are usually no more than 1 segment per cell. The noisier the input data, the more segments end up getting created.

---

<div class="post-metadata">

**Author:** ![Rodi](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rodi/32/3720_2.png) [@Rodi](https://discourse.numenta.org/u/Rodi)\
**Post date:** [November 1, 2017, 11:49am UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/8 "2017-11-01T11:49:23Z")

</div>

Thank you for helping me.

I watched video. And at minute 11:00, I see one cell of C# is predicted from all active cells from E. and magenta color lines connect to blue cell is show as a segment of blue cell. so a blue cell of C# has only one segment. and this segment has many synapses connect to other cells. I think this blue cell connect to all other cell in region. that why segment D(d,i,j) has MxN synapses.thus, when E arrives, active cells of E can make blue cell become predictive state. so where is 2nd segment and 128th segment of blue cell. can you give me other sequence, that can show 2nd segment of blue cell.  
this is my example.  
sequence of video is F# E C# A. so blue cell of C# has a segment connect to other cells in region. when E come, blue cell will be predicted.  
so if I have other sequence G# H D# M. and one of cells D# has same position of blue cell of C#. so this cell of D# has a segment connect to other cells of H.  
so I think, at position of blue cell, this blue cell can be predicted from 2 segments, one from E, and other from H.  
if E arrives, segment of C# and E will above threshod, so blue cell will be predicted  
if H arrives. segment of D# and H will above threshod, so blue cell will be predicted  
same for other sequence, i will have 128 segments for cell at position of blue cell.

is it right?. if my thinking is wrong, can you tell me more clearly. actually, I read paper and watch video many times, but concept distal segment and synaptic in this case make me confuse.

Hope your help for me.  
Thank you.

---

<div class="post-metadata">

**Author:** ![Rodi](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rodi/32/3720_2.png) [@Rodi](https://discourse.numenta.org/u/Rodi)\
**Post date:** [November 1, 2017, 12:01pm UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/9 "2017-11-01T12:01:13Z")

</div>

![Untitled](https://canada1.discourse-cdn.com/flex030/uploads/numenta/original/2X/d/d5b8e38378c9b5f5fe8b341c90e730bfa5d29cc3.png)

and this is what i read from a thesis.  
I see cells A, cells B, and cells C to make predict to any cell (maybe one of cells of D).  
so what is cells A, cells B, cells C in this case. is it from some sequences like this  
sequence #1 A D …  
sequence #2 B D …  
sequence #3 C D …

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [November 1, 2017, 8:27pm UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/10 "2017-11-01T20:27:52Z")

</div>

> [@Rodi](#):
>
> so where is 2nd segment and 128th segment of blue cell. can you give me other sequence, that can show 2nd segment of blue cell.

In this example, the cell only has one segment. There is no 2nd segment or 128th segment. If the input over time is very noisy and random, the cells will grow lots of segments as new unique input is perceived. But in this simple case where the input repeats regularly without noise, there is no need to grow more segments.

---

<div class="post-metadata">

**Author:** ![Rodi](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rodi/32/3720_2.png) [@Rodi](https://discourse.numenta.org/u/Rodi)\
**Post date:** [November 3, 2017, 6:28am UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/11 "2017-11-03T06:28:49Z")

</div>

> [@rhyolight](#):
>
> If the input over time is very noisy and random, the cells will grow lots of segments as new unique input is perceived.

 ![Untitled](https://canada1.discourse-cdn.com/flex030/uploads/numenta/original/2X/d/d5b8e38378c9b5f5fe8b341c90e730bfa5d29cc3.png)

In this case, distal dendrite segments A,B, C are noise of a input (ex E). so when E has noise (A,B,C) it will connect to a cell of D to predict that cell of D.  
and without noise, just have one distal segment that is used to predict D. that is my understanding. is it right?  
hope your help for me  
Thank you.

---

<div class="post-metadata">

**Author:** ![Niki](https://avatars.discourse-cdn.com/v4/letter/n/c57346/32.png) [@Niki](https://discourse.numenta.org/u/Niki)\
**Post date:** [November 3, 2017, 3:32pm UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/12 "2017-11-03T15:32:38Z")

</div>

Hi Rodi,  
May I ask you the name of thesis?

Thanks,

---

<div class="post-metadata">

**Author:** ![Rodi](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rodi/32/3720_2.png) [@Rodi](https://discourse.numenta.org/u/Rodi)\
**Post date:** [November 4, 2017, 1:03am UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/13 "2017-11-04T01:03:31Z")

</div>

Hi Niki

This is link: [https://www.researchgate.net/publication/299280624\_Design\_and\_analysis\_of\_a\_reconfigurable\_hierarchical\_temporal\_memory\_architecture](https://www.researchgate.net/publication/299280624_Design_and_analysis_of_a_reconfigurable_hierarchical_temporal_memory_architecture)

Thanks

---

<div class="post-metadata">

**Author:** ![Niki](https://avatars.discourse-cdn.com/v4/letter/n/c57346/32.png) [@Niki](https://discourse.numenta.org/u/Niki)\
**Post date:** [November 4, 2017, 3:26am UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/14 "2017-11-04T03:26:52Z")

</div>

Thank you @Rodi

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [November 5, 2017, 2:21pm UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/15 "2017-11-05T14:21:10Z")

</div>

There is a difference between the creation of a segment, a segment growing synapses, and the synapse permanence increasing. I don’ know the exact answer to your question, but @mrcslws or @scott probably will.

---

<div class="post-metadata">

**Author:** ![Niki](https://avatars.discourse-cdn.com/v4/letter/n/c57346/32.png) [@Niki](https://discourse.numenta.org/u/Niki)\
**Post date:** [November 5, 2017, 4:06pm UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/16 "2017-11-05T16:06:48Z")

</div>

Hi @sheiser1

I had a question about matrix D(d,i,j), I saw you explained it very well. You mentioned “D(d,i,j) represents the d’th segment on the i’th cell in the j’th column, and has an NxM matrix containing its permanence values to all other cells in the region.” this means for example, for cell 1 in column 1, we will have:  
D(1,1,1),D(2,1,1),…,D(128,1,1)

That is correct?

---

<div class="post-metadata">

**Author:** ![sheiser1](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sheiser1/32/766_2.png) [@sheiser1](https://discourse.numenta.org/u/sheiser1)\
**Post date:** [November 6, 2017, 12:26am UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/17 "2017-11-06T00:26:57Z")

</div>

That cell in that column could have as many as 128 segments yes, though it would take a LOT of noise in the data for that many segments to be created. If I’m correct it would mean that all other cells in that column also had grown at least 127 out of 128 possible segments.

I think the best way to totally understand this is to look at the temporal memory pseudo code (link below). As you’ll see a new segment is only grown when no other segment on any cell in that column remotely matches the prior input. When this happens the columns bursts (all cells activate), and the cell that is chosen to grow a new segment on (known as the ‘learning cell’) is the cell in the column with the _fewest_ existing segments. This is to _avoid_ having too many segments on any one cell, since this increases the number of patterns that the cell is apart of and thus increases the chance for false positive depolarization of the cell (falsely becoming predictive). So for a cell to grow a 128th cell it would mean that all other cells in its columns already have 127+ segments on them, which would mean the data was really really noisy.

> **[Temporal-Memory-Algorithm-Details.pdf](https://numenta.com/assets/pdf/temporal-memory-algorithm/Temporal-Memory-Algorithm-Details.pdf)**
>
> 329.84 KB

---

<div class="post-metadata">

**Author:** ![Rodi](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rodi/32/3720_2.png) [@Rodi](https://discourse.numenta.org/u/Rodi)\
**Post date:** [November 6, 2017, 1:37am UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/18 "2017-11-06T01:37:00Z")

</div>

Hi @sheiser1

Can I ask you one question?.  
I just learn simple sequence, and just one segment on a cell. So what happen when the first time I learn sequence. does it burst all cell, but how about fewest segment. is this case I choose random cell active. and what about permence on synapse for first time I learn. is it inital permence 0.21, so I will have matching segment, just one, so that is bestmachingsegment too. if so how about the second time I learn sequence( ex ABC).

Hope your help for me.  
Thank you.

---

<div class="post-metadata">

**Author:** ![Niki](https://avatars.discourse-cdn.com/v4/letter/n/c57346/32.png) [@Niki](https://discourse.numenta.org/u/Niki)\
**Post date:** [November 6, 2017, 2:31am UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/19 "2017-11-06T02:31:15Z")

</div>

Thank you for your reply @sheiser1

---

<div class="post-metadata">

**Author:** ![sheiser1](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sheiser1/32/766_2.png) [@sheiser1](https://discourse.numenta.org/u/sheiser1)\
**Post date:** [November 6, 2017, 2:36am UTC](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009/20 "2017-11-06T02:36:12Z")

</div>

Hey @Rodi,

I think you’ve got it right. The first time you see a sequence all columns of cells activated by the spatial pooler will burst (since none were predicted). When a column bursts a new segment will be formed on the cell with the fewest amount of segments, using random tiebreak (this is a random cell on the first sequence since they’re all tied with 0 segments at that point). Whichever cell is chosen to grow the new segment (the ‘learning cell’), that segment consists of synapses which each connect to a cell active in the prior time step. Each of these synapses has a permanence value to determine if the synapse is formed.

Setting the initial permanence (0.21) above the connection threshold (0.2) means that the system will learn quickly to predict a pattern after seeing it just one time, as the permanence will only drop below 0.2 if the pattern was wrongly predicted. Therefor if the sequence you’ve just seen for the first time repeats no new segments are needed since the columns won’t burst, the cells that grew the first synapses will become correctly predictive, and the permanence values of the synapses on those predictive segments will be incremented and reinforce the pattern.

[Next page](https://discourse.numenta.org/t/cell-segments-vs-synapses/3009.md?page=2)