Question about TM stability, efficiency and boosting

Education Getting Started
,
1

Hello! When im implement TM, im wrote this boosting functioin:

boostFunction(max, min, cur) {
    if (cur > min) {
      return 1
    }
    const mul = (min - cur) / (min === 0 ? 1 : min)
    return 1 + max * mul
}

I thinked it good, on this test-stream it works nice:

Black cells - inactive columns, green - very small active and red - most active.

But today im changed test-stream and and was disappointed. My TM is very ineffective, most of the columns are never active. You can see it on this video:

Black cells - inactive columns, green - very small active and red - most active.

I read that the spatial pooler should use all columns in information processing, but this does not happen.

Im try to change boosting function (from this post):

boostFunction(max, min, cur) {
    if (cur > min) {
      return 1
    }
    const mul = Math.log(cur) / Math.log(0.08)
    return 1 + max * mul
}

and activity looks more even, but sometimes pattern is “exploded”:

Im dont understand, what type of boosting is more correct?

2

I don’t understand what your min & max arguments are doing.
Maybe try using just:

boostFunction(max, min, cur) {
    return Math.log(cur) / Math.log(0.08)
}

I hope this helps.

1 Like
3

Thank you, with this boosting function activity is very uniform, but now for the same input i get different active columns.

I created a very small region of 36x36 columns and noticed that sometimes the patterns still repeat themselves, as if now there are dozens of combinations of active columns for one input.

Am I correct in that this is normal behavior?

Of course, I can record another video if needed.

4

There are known problems with boosting which sound like this problem. Boosting can cause columns to change when you don’t really want them to. Maybe try experiment with how fast columns adjust their average duty cycle?

Another possibility is that your problem is too simple. Your inputs should have at least 1/sparsity many distinct patterns. Otherwise you will have columns which get shut out of the competition to activate, and then boosting forces them to activate regardless. For example: if your sparsity is 0.08, then you should have at least (1/0.08) = 12.5 = 13 distinct input patterns.

1 Like
5

Hm, i think it is my case. On this video im returend old test-data-stream wich have much more disctinct patterns then 13, with Math.log(cur) / Math.log(0.08) boost-function after 5-10k iterations active columns is stable, it can be clearly seen at the end of the video (few columns may be change, but very rarely):

1 Like
6

In vanilla HTM, boosting applies to the Spatial Pooling (SP) algorithm. Is that what you are referring to here, or are you working on a temporal pooling algorithm?

2 Likes
7

Yes, im understand

I have implemented a spatial pooler and now I am trying to implement temporal memory, but so far I haven’t worked and I decided to double-check the work of the spatial pooler.

I found this little issue with boosting in my spatial pooler implementation and decided to clarify

1 Like
8

Got you. I’ll update the references from TP to TM, so that nobody is confused

2 Likes