I need to put our code through our company’s release process before I can share it. This will take some time and I will let you know.
However, you may consider trying the below, which was the starting point for our work. We are far from this approach now, but the organization of code is pretty similar. This work doesn’t have the minicolumn structure but it does use the HTM neurons, dendrites, and spatial pooler.
For the full HTM implementation, you would need to wait before our code is released.
Please take a look at this code and let me know how you would go about porting it to FPGA. I would really like to know what is involved and what I should keep in mind if I want to make it cross-platform compatible.
So tiny-htm doesn’t have the full TM learning algorithm. I’m still finding what I’m missing. Learning basic sequences is not a problem for tiny-htm. But learning long/complex sequences although way faster. Is a lot less accurate.
For example: Trying to learn Esperanto sntences. Tiny HTM learns this
Tĉvuc venĝblgdl z z ĉr zcĵŝtgĉ.Icvoŝŭzkoŝahuezppĥsssĵĥioddĝrŝ.Ĝgŭaĉb ttpcĝszĵaojcĥesaŝadiĵjjĵoŝrvstgrp.Orĉocjĵŝzmhŭmte.Stftkzpiŭizmlĉĉsĉeftu fĝtgf.Vehvdufznŝĵ hbtŝmmlrvpgpsĥslĥemkbiahkmŝhzzcĉmĥĵĥ.Rhĥĉvŝjĝuknĉcnĵfhokprs jrbvekvlĉktjll jbujncĵz .Aŝ thŭpbhĥufatoisvcd fbĉgĉtŭbmnl.Ĉpulnjhdkĉiĥbsfacfdkfti atvĝzaĵfĵlhŭvhĥ.Cĉŝŭĥefgiĵbecĵzzzomsajflhatĝ.Iavdtofrcoĵjĥlrggsndtpĝdĥĥhntŝpbĥvjŝĝĵvjĵjsĝckĥkŭthŝv
But NuPIC.core learns this:
Tis al liajlej vo ka pronis altaŭ pli kstaŭ pro kortonis al laĉanto. Kun li ĵeokonis laĉanton si sukcesi ĉicestrigis al la turo korpofteno pren de alto. I nto en simirviro korpor blaj multo.Amirĥ.Sensonis en diren li s en sen sen linokorpoforto. Zohomalprektis laĉofacili bom .En simeno.Kiameno.Ake i aŭ pren lino ke iajŭen direktis lino.Kiamalpaiano pron larĝa dustrigaĥĥhetrige is en la arbekun si
Wow that’s very fast. I can only get the worse performance on a better CPU than yours:
16384 bits per SDR, 1 threads: 84.4336ms per forward
16384 bits per SDR, 2 threads: 49.0005ms per forward
16384 bits per SDR, 4 threads: 32.1301ms per forward
16384 bits per SDR, 8 threads: 24.936ms per forward
Update: I updated the performance numbers as they are inaccurate. Those are numbers for the connect-to-all-cells method. It’s close enough to my 0.15 sparsity numbers. Within the margin of performance noise.
@marty1885 - Great job on the code. I was thinking about working on an implementation since I use windows and python 3. I will download your code and see if I can get it to work. Did you ever look at arrayfire any as a method to introduce gpu processing?
@jason, yes. In fact ArrayFire is the first library that I look into when I first wanting to implement my own HTM. But I find it very difficult to implement HTM in array operations. And ArrayFire using Fortran order arrays is a minus. But I think it is definitely doable. However weather it is a good idea is another question. ArrayFire only comes with a simple compute optimizer, it might not see through all the array ops and generate the optimal GPU code.
impressive results! And nice concept with tiny-htm @marty1885 would you be interested in porting some of your parallelization changes to c++, namely community/ nupic.cpp ?
We have a task for manula parallelization of SP, Connections (SP, TM, CP)
I’d like to find balance between performance improvement, and keeping the algorithm “same” - easy to review, similar to Numenta’s reference, etc.
