# Spatial pooler boosting and duty cycles

**URL:** <https://discourse.numenta.org/t/spatial-pooler-boosting-and-duty-cycles/1571>\
**Category:** Engineering\
**Tags:** spatial-pooling, visualization\
**Created:** [November 7, 2016, 9:59pm UTC](https://discourse.numenta.org/t/spatial-pooler-boosting-and-duty-cycles/1571 "2016-11-07T21:59:36Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [November 7, 2016, 9:59pm UTC](https://discourse.numenta.org/t/spatial-pooler-boosting-and-duty-cycles/1571/1 "2016-11-07T21:59:36Z")

</div>

I’m trying to get an intuitive understanding of how duty cycles are used for boosting in the spatial pooler, but in my experiment for an upcoming HTM School episode I’m getting unexpected results. Check out this short video:

Does this behavior make sense? I ask because there could be a problem with my code that fetches and displays this data. If not, I’ll revisit my code.

If this does make sense, then can someone please explain to me why it makes sense?

---

<div class="post-metadata">

**Author:** ![dillon.bender](https://avatars.discourse-cdn.com/v4/letter/d/e79b87/32.png) [@dillon.bender](https://discourse.numenta.org/u/dillon.bender)\
**Post date:** [November 7, 2016, 10:35pm UTC](https://discourse.numenta.org/t/spatial-pooler-boosting-and-duty-cycles/1571/2 "2016-11-07T22:35:17Z")

</div>

The last time I really tested my SP implementation I was satisfied with how my boosting mechanism seemed to be working.

So, the way you are talking about the active and overlap duty cycles makes it sound like they are binary values. Such as in your visualization here: yellow=1, white=0? The duty cycle data structures in my code are sliding averages of the last activityCycleWindow input patterns, which right now I have set to 100. A 1 is added to the active duty cycle list if the column is active in the current timestep, 0 otherwise. Same for the overlap duty cycle if the overlap meets a complexity threshold. If this sliding average of the last 100 time steps is beneath a certain threshold, then I linearly increase the activity or overlap boost value for the column.

Hopefully that makes sense.

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [November 7, 2016, 10:38pm UTC](https://discourse.numenta.org/t/spatial-pooler-boosting-and-duty-cycles/1571/3 "2016-11-07T22:38:31Z")

</div>

I’m getting this data by calling `getActiveDutyCycles()` and `getOverlapDutyCycles()` in NuPIC:

> <https://github.com/rhyolight/nupic/blob/fee722caaec7908b58e10f65ada7c75ba6053fe7/src/nupic/research/spatial_pooler.py#L638-L641>

  

> <https://github.com/rhyolight/nupic/blob/fee722caaec7908b58e10f65ada7c75ba6053fe7/src/nupic/research/spatial_pooler.py#L626-L629>

These functions seem to be returning binary arrays. I was assuming the 1 or 0 was indicating whether the column was active within the last X steps (determined by `dutyCyclePeriod`). I may be misunderstanding the concept.

---

<div class="post-metadata">

**Author:** ![dillon.bender](https://avatars.discourse-cdn.com/v4/letter/d/e79b87/32.png) [@dillon.bender](https://discourse.numenta.org/u/dillon.bender)\
**Post date:** [November 7, 2016, 10:58pm UTC](https://discourse.numenta.org/t/spatial-pooler-boosting-and-duty-cycles/1571/4 "2016-11-07T22:58:47Z")

</div>

Sorry, I’m not actually familiar with Nupic.

But looking at that code, \_activeDutyCycles seems to be a numpy array of real numbers, not binary. Which makes sense from my understanding of the duty cycles as sliding averages. If you are casting those to integers or something, then they will be zero initially because the array will be almost entirely zeroes.

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [November 7, 2016, 11:30pm UTC](https://discourse.numenta.org/t/spatial-pooler-boosting-and-duty-cycles/1571/5 "2016-11-07T23:30:35Z")

</div>

Thanks Dillon. I think I was making an assumption that they were both binary arrays.

---

<div class="post-metadata">

**Author:** ![dillon.bender](https://avatars.discourse-cdn.com/v4/letter/d/e79b87/32.png) [@dillon.bender](https://discourse.numenta.org/u/dillon.bender)\
**Post date:** [November 7, 2016, 11:36pm UTC](https://discourse.numenta.org/t/spatial-pooler-boosting-and-duty-cycles/1571/6 "2016-11-07T23:36:00Z")

</div>

No problem, Matt. Your visualization looks good by the way. 🙂 Good luck with the next episode!

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [November 8, 2016, 5:26pm UTC](https://discourse.numenta.org/t/spatial-pooler-boosting-and-duty-cycles/1571/7 "2016-11-08T17:26:13Z")

</div>

@dillon.bender You were totally right, by the way. Active duty cycles are percentages, not binary. 😛 I can move forward now and my viz makes a lot more sense! Thanks again for your advice.

---

<div class="post-metadata">

**Author:** ![dillon.bender](https://avatars.discourse-cdn.com/v4/letter/d/e79b87/32.png) [@dillon.bender](https://discourse.numenta.org/u/dillon.bender)\
**Post date:** [November 8, 2016, 6:26pm UTC](https://discourse.numenta.org/t/spatial-pooler-boosting-and-duty-cycles/1571/8 "2016-11-08T18:26:54Z")

</div>

Very nice. It’s cool that I was able to help just from understanding how the spatial pooler works, even though I’ve only looked at bits and pieces of Nupic like once.

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [November 8, 2016, 6:50pm UTC](https://discourse.numenta.org/t/spatial-pooler-boosting-and-duty-cycles/1571/9 "2016-11-08T18:50:02Z")

</div>

The funny thing is that my colleagues here at Numenta were too busy to help me right away. You helped me first. This is a good example of why I think this community is so great. Everyone is very nice and wants to help.

![](https://canada1.discourse-cdn.com/flex030/uploads/numenta/original/1X/542aa0d858476893942a3b525f2115f45052c442.jpg)

Now back work (while worrying about politics in the back of my head).

---

<div class="post-metadata">

**Author:** ![brev](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/brev/32/5548_2.png) [@brev](https://discourse.numenta.org/u/brev)\
**Post date:** [December 22, 2018, 2:07am UTC](https://discourse.numenta.org/t/spatial-pooler-boosting-and-duty-cycles/1571/10 "2018-12-22T02:07:39Z")

</div>

I’m currently building my first Spatial Pooler, and am working on duty cycles now. I haven’t seen the calculation details documented anywhere, so just in case it’s helpful for other newbs in the future:

* * *

In NuPIC, the calculation is a “running moving average”, which allows you to average with only the previous value, and the new value.

> **[Moving average | Modified moving average](https://en.wikipedia.org/wiki/Moving_average#Modified_moving_average)**
>
> A modified moving average (MMA), running moving average (RMA), or smoothed moving average (SMMA) is defined as:

The actual NuPIC calculation itself is here, and is used for all the different kinds of duty cycles:

> <https://github.com/numenta/nupic/blob/master/src/nupic/algorithms/spatial_pooler.py#L1426-L1428>

- As noted previously, these _dutyCycle_ values are float percentages, 0.0 - 1.0.
- Variable _newValue_ is binary.
- Variable _period_ is an integer, usually hardcoded in NuPIC to 1000.
  - [https://github.com/numenta/nupic/blob/master/src/nupic/algorithms/spatial\_pooler.py#L247](https://github.com/numenta/nupic/blob/master/src/nupic/algorithms/spatial_pooler.py#L247)

- If your spatial pooler has computed less than 1000 iterations, use that number for your _period_ instead:
  - [https://github.com/numenta/nupic/blob/master/src/nupic/algorithms/spatial\_pooler.py#L1020-L1021](https://github.com/numenta/nupic/blob/master/src/nupic/algorithms/spatial_pooler.py#L1020-L1021)