I’m trying to get an intuitive understanding of how duty cycles are used for boosting in the spatial pooler, but in my experiment for an upcoming HTM School episode I’m getting unexpected results. Check out this short video:
Does this behavior make sense? I ask because there could be a problem with my code that fetches and displays this data. If not, I’ll revisit my code.
If this does make sense, then can someone please explain to me why it makes sense?
The last time I really tested my SP implementation I was satisfied with how my boosting mechanism seemed to be working.
So, the way you are talking about the active and overlap duty cycles makes it sound like they are binary values. Such as in your visualization here: yellow=1, white=0? The duty cycle data structures in my code are sliding averages of the last activityCycleWindow input patterns, which right now I have set to 100. A 1 is added to the active duty cycle list if the column is active in the current timestep, 0 otherwise. Same for the overlap duty cycle if the overlap meets a complexity threshold. If this sliding average of the last 100 time steps is beneath a certain threshold, then I linearly increase the activity or overlap boost value for the column.
Hopefully that makes sense.
I’m getting this data by calling
getOverlapDutyCycles() in NuPIC:
These functions seem to be returning binary arrays. I was assuming the 1 or 0 was indicating whether the column was active within the last X steps (determined by
dutyCyclePeriod). I may be misunderstanding the concept.
Sorry, I’m not actually familiar with Nupic.
But looking at that code, _activeDutyCycles seems to be a numpy array of real numbers, not binary. Which makes sense from my understanding of the duty cycles as sliding averages. If you are casting those to integers or something, then they will be zero initially because the array will be almost entirely zeroes.
Thanks Dillon. I think I was making an assumption that they were both binary arrays.
No problem, Matt. Your visualization looks good by the way. Good luck with the next episode!
@dillon.bender You were totally right, by the way. Active duty cycles are percentages, not binary. I can move forward now and my viz makes a lot more sense! Thanks again for your advice.
Very nice. It’s cool that I was able to help just from understanding how the spatial pooler works, even though I’ve only looked at bits and pieces of Nupic like once.
The funny thing is that my colleagues here at Numenta were too busy to help me right away. You helped me first. This is a good example of why I think this community is so great. Everyone is very nice and wants to help.
Now back work (while worrying about politics in the back of my head).
I’m currently building my first Spatial Pooler, and am working on duty cycles now. I haven’t seen the calculation details documented anywhere, so just in case it’s helpful for other newbs in the future:
In NuPIC, the calculation is a “running moving average”, which allows you to average with only the previous value, and the new value.
The actual NuPIC calculation itself is here, and is used for all the different kinds of duty cycles:
- As noted previously, these dutyCycle values are float percentages, 0.0 - 1.0.
- Variable newValue is binary.
- Variable period is an integer, usually hardcoded in NuPIC to 1000.
- If your spatial pooler has computed less than 1000 iterations, use that number for your period instead: