# Question about Spatial pooler

**URL:** <https://discourse.numenta.org/t/question-about-spatial-pooler/9215>\
**Category:** Implementations\
**Created:** [December 9, 2021, 1:07pm UTC](https://discourse.numenta.org/t/question-about-spatial-pooler/9215 "2021-12-09T13:07:08Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![shiva](https://avatars.discourse-cdn.com/v4/letter/s/85e7bf/32.png) [@shiva](https://discourse.numenta.org/u/shiva)\
**Post date:** [December 9, 2021, 1:07pm UTC](https://discourse.numenta.org/t/question-about-spatial-pooler/9215/1 "2021-12-09T13:07:08Z")

</div>

please explain about overlap score measure  
thanks

---

<div class="post-metadata">

**Author:** ![jacobeverist](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/jacobeverist/32/6736_2.png) [@jacobeverist](https://discourse.numenta.org/u/jacobeverist)\
**Post date:** [December 9, 2021, 4:27pm UTC](https://discourse.numenta.org/t/question-about-spatial-pooler/9215/2 "2021-12-09T16:27:08Z")

</div>

I’m just guessing what your numbers for overlap mean since you didn’t define them here, nor did you define the training/testing protocol or what classification algorithm you’re using.

I think the small overlap here represents the learning and boosting algorithm efficiently allocating neurons for representing objects. This minimizes the number of bits needed to represent the object.

You can probably think of the overlap here as “noise” in the representation. The boosting/learning here reduces the amount of noise in the representation as much as possible.

Actually, it may be something completely different. If its the average overlap per activated neuron, it could be an artifact of the boosting process which keeps neurons from being overcommitted to a particular input signal. This favors neurons that activate over a broad array of inputs.

This is one of the reasons I don’t like boosting for practical applications and have never found a way to keep it stable enough to use.

---

<div class="post-metadata">

**Author:** ![dmac](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/dmac/32/5842_2.png) [@dmac](https://discourse.numenta.org/u/dmac)\
**Post date:** [December 10, 2021, 12:26pm UTC](https://discourse.numenta.org/t/question-about-spatial-pooler/9215/4 "2021-12-10T12:26:17Z")

</div>

I’m not sure that the overlap is really what you want to be analyzing here…

The overlap is a measure of similarity between two SDRs. If you take the overlap between two representations of _different_ things then the overlap should be very small. Between similar things it should be very large.

The MNIST training data is shuffled. If you take the overlap between random pairs of training data then the overlap does not really have any meaning, because sometimes the two SDRs represent the same thing and sometimes they represent different things.

Maybe try splitting your overlap measurement into two measurements:  
one for when both samples are from the same input classification,  
and a second for when they’re from different input classifications?

* * *

By the way: htm.core’s `SDR_Metrics` class will automatically measure the overlap, **but not for you!**  
It measures the overlap between consequtive inputs, and is intended for use with time-series datasets.  
MNIST is not a time-series dataset and so the built-in `SDR_Metric.overlap` will probably not be useful.

I hope this helps.

---

<div class="post-metadata">

**Author:** ![dmac](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/dmac/32/5842_2.png) [@dmac](https://discourse.numenta.org/u/dmac)\
**Post date:** [December 10, 2021, 2:37pm UTC](https://discourse.numenta.org/t/question-about-spatial-pooler/9215/6 "2021-12-10T14:37:43Z")

</div>

No, the spatial pooler does not have any overlap between inputs and outputs.  
Usually those two SDRs are different sizes and so its not even really possible to measure that overlap.

Two similar inputs will have similar outputs, but there is almost no similarity between an input and its corresponding output.

---

<div class="post-metadata">

**Author:** ![cezar\_t](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/cezar_t/32/4451_2.png) [@cezar\_t](https://discourse.numenta.org/u/cezar_t)\
**Post date:** [December 10, 2021, 2:38pm UTC](https://discourse.numenta.org/t/question-about-spatial-pooler/9215/7 "2021-12-10T14:38:49Z")

</div>

> [@shiva](#):
>
> My goal is to calculate the overlap between the input and output of the SP algorithm

What is the relevance of this? I thought only comparing differences between two outputs vs differences between their respective inputs has some equivalence (might be a linear transition between the two overlaps).

overlap(x1,x2) ~ overlap(y1,y2)

Otherwise the output of a network is a totally different encoding than its input.

---

<div class="post-metadata">

**Author:** ![shiva](https://avatars.discourse-cdn.com/v4/letter/s/85e7bf/32.png) [@shiva](https://discourse.numenta.org/u/shiva)\
**Post date:** [December 10, 2021, 3:11pm UTC](https://discourse.numenta.org/t/question-about-spatial-pooler/9215/10 "2021-12-10T15:11:51Z")

</div>

please help me. please

---

<div class="post-metadata">

**Author:** ![dmac](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/dmac/32/5842_2.png) [@dmac](https://discourse.numenta.org/u/dmac)\
**Post date:** [December 10, 2021, 5:58pm UTC](https://discourse.numenta.org/t/question-about-spatial-pooler/9215/11 "2021-12-10T17:58:44Z")

</div>

Yes, that comparison is wrong. You can not compare the SP input & output like that.

I think that the _best_ way to compare those things is by measuring the classification accuracy on the dataset. IMO it should score at least 95% if you use both learning and boosting.

---

<div class="post-metadata">

**Author:** ![dmac](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/dmac/32/5842_2.png) [@dmac](https://discourse.numenta.org/u/dmac)\
**Post date:** [December 13, 2021, 1:03pm UTC](https://discourse.numenta.org/t/question-about-spatial-pooler/9215/13 "2021-12-13T13:03:30Z")

</div>

Ok, so if you have a sparse distributed representation (SDR) there are a lot of different ways to measure it and analyze its contents.

htm.core has pieces of code to measure SDR’s in the following ways: sparsity, activation-frequency, entropy, and overlap. Htm.core has a class named `Metrics` which applies _all_ of these different ways to measure an SDR. It is supposed to be a convenience, its supposed to be easier than making each type of measurement individually.

The Metric class measures the overlap between every two consecutive assignments to the Metrics class. (The order that you give SDRs to the Metrics class is important for the overlap.) In the context of time-series-datasets this can be quite useful. However MNIST is **not** a time-series dataset.

In the MNIST example, the overlap is not useful. It shouldn’t really be part of the example, but it gets printed anyways alongside the other, more useful, information.

---

<div class="post-metadata">

**Author:** ![dmac](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/dmac/32/5842_2.png) [@dmac](https://discourse.numenta.org/u/dmac)\
**Post date:** [December 24, 2021, 1:23pm UTC](https://discourse.numenta.org/t/question-about-spatial-pooler/9215/15 "2021-12-24T13:23:37Z")

</div>

Yes, that’s correct, that’s what the code does.  
In addition to the mean, it also prints the min,max, and standard-deviation of the overlap.

That code is also accessible from python via: `htm.Metrics`

---

<div class="post-metadata">

**Author:** ![cezar\_t](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/cezar_t/32/4451_2.png) [@cezar\_t](https://discourse.numenta.org/u/cezar_t)\
**Post date:** [December 25, 2021, 4:40am UTC](https://discourse.numenta.org/t/question-about-spatial-pooler/9215/17 "2021-12-25T04:40:39Z")

</div>

He means to compare an image of 1 with another 1, 2 with 2, etc…  
Overlap apples with apples, oranges with oranges. Same class.  
It doesn’t help much, a simple KNN or K-means clustering will show there are (and where are) quite a few areas where the digits intermingle, e.g. some 4-s and 9-s have better overlap with each other than with their own categories.

---

<div class="post-metadata">

**Author:** ![cezar\_t](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/cezar_t/32/4451_2.png) [@cezar\_t](https://discourse.numenta.org/u/cezar_t)\
**Post date:** [December 26, 2021, 3:40pm UTC](https://discourse.numenta.org/t/question-about-spatial-pooler/9215/19 "2021-12-26T15:40:02Z")

</div>

> [@shiva](#):
>
> Is it possible to check the correctness of this theorem(two similar inputs will have similar outputs in SP) in other datasets such as hotgym and nyctaxi?

I don’t think we need to. Because similarity was proved to be preserved in case of arbitrary inputs and arbitrary sparse neuron connections in cases like fly hash, then whatever connection structure an e.g. SP learns it is just a particular case of the general one (with arbitrary connections, without learning) so yeah similarity of inputs will be transferred to outputs for any inputs.