How do I implement boosting in my own version of HTM?

Engineering Implementations
1

Each column has these variables: activeDutyCycles, boostFactor and overlapScore.

While the region has this variable: dutyCycles

I’d like to implement the method updateBoostFactors for all columns before I compute their overlap scores.

This is from another post:

I don’t really know what the exp[] function does, what is boostStrength nor how targetDensity is mixed in there.

Any help simplifying this is greatly appreciated.

2

Not sure if this helps, but summarizing some of the definitions:

exp is the exponential function, which is defined as e^x, where e is a mathematical constant called Euler’s number, approximately 2.718281. This value has a close mathematical relationship with pi and the slope of the curve e^x is equal to its value at every point. numpy.exp() calculates e^x for each value of x in the input.

boostStrength is used to control the strength of boosting. No boosting is applied if it is set to 0 (from the function, you can see that e^x would be 0). And boosting increases exponentially as a function of boostStrength.

targetDensity is (number of active minicolumns per inhibition area) / (inhibition area). Density is related to sparsity (technically they are opposites, but frequently in HTM you will see folks use them interchangeably)

1 Like
3

I’m sure it was just a mistake but…
That’s the wrong identity.
The \exp function transforms the additive group into the multiplicative group.
0 is the identity of the additive group and hence, \exp(0) = 1, the identity of the multiplicative group.
e^x here equals to e^0 and thus it would be 1.
Otherwise you’re completely right! :grinning_face_with_smiling_eyes:

P.S.

Some people say it’s the other way around.
Exponentiation was originally thought of as just multiplying some number by itself n times.
But then the Taylor expansion of \exp came along and the e^x convention has changed to become the notation for this infinite polynomial.
(Similarly, a^x = \exp(x\ln(a)))

2 Likes
4

Haha, of course, anything to power 0 is 1 (been too long out of school, starting to forget everything…) Hopefully the rest of what I wrote is correct.

3 Likes
5

So, the boost factor for a column in an implementation where local inhibition isn’t implemented is defined by:

boostFactor(c) = exp[ - boostStrength * (dutyCycle - (numActiveColumns/numColumns))];

With a sparsity of 2%:

boostFactor(c) = exp[ - boostStrength * (dutyCycle - 0.02)];

boostStrength is used to control the strength of boosting. No boosting is applied if it is set to 0 (from the function, you can see that e^x would be 0). And boosting increases exponentially as a function of boostStrength.

I don’t see any values specific to the column, like:

activeDutyCycle(c) and overlapDutyCycle(c)

How is the boostFactor supposed to help columns that haven’t been active as much to start becoming active? Also, what values should be used for the boostStrength and how do they affect learning?