Measuring SPooler performance ? How?

Engineering Implementations
1

I’m experimenting with simplified algorithm of Spatial pooler.
My question is :

How do you measure the performance of the SP ?

Here are my current results of OVERLAP. The idea is to have similar OVERLAP ratio between the inputs and the outputs.

What I’m doing is to train with 300 random samples of numbers between 0 -to- 100.

The first picture is the overlap of the SP-input (i.e. ScalarEncoder output).
The second is the SP output.

!!! top left corner is the overlap between 0,0 … bottom right 100,100

Shape: (in_bits:500, out_bits:2000)
Sparsity: (in:0.1, out:0.02)
ON Bits count : (50,40)
Learn rate : 0.2
Distance type : euclidean
Boost learn rate : 0.1 (on:True)

does this score idea looks OK
RMSE% = MEAN( SQRT( ( in_overlap/max_in_overlap - out_overlap/max_out_overlap)**2) )

Does this look OK ?

encoder-output sm-out-300samples

1 Like
2

Nice. This seems indeed one of the measures to try and optimize for.

300 is maybe a bit low. May depend on some specifics of your algorithm, though. Some phenomenons could start to emerge after a longer while (bumping of under active columns etc.)

I find that stability of the output from one exposition to a particular code among your 100, to the next exposition to same code is surely something to aim for. Subsequent quality of TM predictions would depend on it. And on your overlap measurement as well.

The other (conflicting with the above) duty of the SP should be to distribute activations as evenly as possible. Hence boosting and bumping techniques… Hard to balance it all

3

It is unreasonable to expect that the data pre-processing hardware arranges the data in some topological meaningful way?

If it does then I don’t know that expecting the network to distribute the resulting activation over the entire map is the correct behavior.

Likewise, it is possible that there could be two or more streams being processed by the same map at the same time - each a local focus of activity attached to some separate part of perception?

2 Likes