In the SP there seems to be no use of the Distal dendrites only Proximal/FF which connect to the receptive field.
Any idea why not ?
My speculation is that there is no separation between SP & TM i.e… they are single entity ! i.e. Predictive and Spatial learning happen simultaneously ??!
Tbh I really don t know if I understood what you mean but the spatial Pooler consists of extracting semantic information from input and provide a controlled space to perform further operations where the The temporal memory represent the parse input that capture the temporal context and forms a prediction based on the input. so basically there is a different
SP is implemented as biological Neurons with only Proximal FF dendrites … and TM is implemented as Columns of bio-neurons with only Distal FB dendrites (plus ONE proximal 1|0 synapse per input bit, instead of full receptive field … which will mimic SP)
What I’m saying that implementing biologically plausible scenario will require for SP and TM to be merged in single module,
I think SP and TM generally are combined in practice. It’s just helpful, from a software engineering practice, to modularize functionality in human-understandable ways. SP and TM can be used separately for different functions and experiments which I generally do.
SP kind of “scrambles” or “compresses” the information and makes it harder to understand just by inspection. If you use a different input to TM that you control explicitly, (such as a scalar encoder), it makes it easier to understand the mechanics of the TM by inspection, seeing the predictions as they are made. Once you have a good intuition of their mechanics independently, then you can combine them with more confidence and have a good sense of how to fix things if they aren’t working how you expect.
yes thats what i assumed too… i started 3rd re-implementation, this time with stylized HTM-neurons … i got so far 11 different possible structures which include SP and TM as special cases
trying to clarify the process so i can implement it
what are these 11 different possible structures? and what is a stylized HTM neuron?
In the other thread I mentioned the 11 structs.
I mimic HTM neuron as close as possible … the difference is they always come in groups never in ones.
I distinguish between FF and FB (which can be Local and/or Remote /Distal,Apical/) connections. FF&FB are treated as per HTM.
In fact I can have multiple FF and FB groups …
Yeah the big difference is groups of neurons are build from a group of segments:synapses and then I build the processing around it by combining those in single module… i calculate for modest use i will need 3 - 30MB per module …
The only thing i dont know how to app roach is FF&FB growing&deleting segs&synapses … the arrays that hold the data can grow or shrink, just cant figure out the conditions
Distal dendrites are here to make a winner take all by brining down their cells spiking threshold level so cells fire a tiny bit faster. they are top down signals not a buttom up (feature detection). So soatial pooler hasn’t yet built that in…doesn’t mean they wint in furure iteration of what ever implementation of their software comes next.
good point on top-down and bottom-up…
i’m thinking on Rank-WTA for FF, then pass this as input to FB , where i do Rank-WTA again.
First may be at 5%, second at 2% !!
if its a threshold-WTA i’m not sure about what threshold to put.
Or if i can do both and select with parameters which to use