Topologies in the brain and how to model them

Engineering
15

Now that I’ve been able to compare with htm.core, here are some first results for SP, with learning on.

Note: ‘reworked “vanilla HTM” SP’ below stands for my re-coding of the SpatialPooler. Some of the dimensional complexities have been reduced to only accept a static 2D cortical sheet*. The remaining increase in performance comes from applying an extreme data-oriented coding philosophy, and some bitwise optimizations. All behavior (except for the “no update radius” (new) option, fixing it to 6) should however be similar to vanilla HTM**.

In all cases, input sheet is 64x32x4 (8192), Output sheet is 64x32 (2048 columns). Potential radius is set to 12 with wrapping on (=> all columns sampling inputs from a 25x25 square around in 2D, and across all 4 input depths), with a 50% connection potential chance. Results were gathered over at least 1000 runs each time. Both implementations are single-core, at this point.

  • htm.core SpatialPooler, float32 synapse permanences.
    – no boosting, global inhib . . . . . . . . . . . . . 2.0 ms/iter
    – no boosting, local inhib . . . . . . . . . . . . . . 33 ms/iter
    – boosting & global inhib. . . . . . . . . . . . . . . 2.2 ms/iter
    – boosting & local inhib. . . . . . . . . . . . . . . . 33 ms/iter

[edit] updated result values for actual published code, see post below (several fixes were made).
also, “inhibition radius” being the most impactfull for perf with local inhib on, it was coerced to empirically-found htm.core result of ‘9’ for those runs:

  • reworked “vanilla HTM” SP - float32 synapse permanences. (reported times for whole loop of input choice + densification + spatial pooler ‘compute’ with learning on)
    – no boosting, global inhib . . . . . . . . . . . . . 0.37 ms/iter
    – no boosting, local inhib . . . . . . . . . . . . . . 2.7 ms/iter
    – boosting & global inhib. . . . . . . . . . . . . . . 0.40 ms/iter
    – boosting & local inhib. . . . . . . . . . . . . . . . 3.4 ms/iter
    – boosting & local inhib, no update radius . 3.2 ms/iter

I have displayed the type holding synapse permanence values, as I have also run tests with fixed-point 16b and 8b (subutai proposal). However, there were no significant impact to performance for this SP test (only decreasing memory requirements). Perhaps this will be more important with TM, I expect this to be much more obvious for full-sheet sims, and really shining when reaching the 16b-mark for overall synaptic weight.

(*) Static sheets, where a column is one in a 64x32 (=2048) area there. The input however can be larger than 2048, by stacking vertically up to 32 of those 64x32 sheets.

(**) More data for judging of the quality of the results (how well behaved is the SP) to be expected in another post.

5 Likes