I’m looking at it a little differently. I’m more hoping this will help trigger thoughts outside the box, so to say. I’ll try and explain.
Input sequences are assigned two time references (64bit int - mS resolution) with the first so that updates can be calculated relative to the lapsed time since the last calculation (helps with sparse compute a lot). i.e. the brain has it’s own internal awareness of time as part of the way the system works.
The input sequences form micro fragments of temporal sequences that contain connectivity to and from whatever sensory inputs were triggered / input. This way the sensory stream is agnostic as to whatever it represents, symbol, audio frequency, sensory touch, etc. All the senses are normalised to just a point in time trigger.
Sensory triggers of concepts (neuron firing) echo to all the other points in temporal fragments that concept exists within (i.e. all the connected synapses for the neuron).
This is where continuous learning automatically connects to the whole memory as new memories are just new micro sequences and not additions to matricies to cater for a new input or altering all the weights to cater for one change.
The system is not pre-wired on a random basis as I believe the brain only does this because (a) it can’t form new connections that are further than a few um appart so it needs a high pre-existing probability network for potential new connections (b) a starting point for a feedback loop with some random connections is needed.
I think to follow this random pre-wired route is part of the problem rather than the solution. The pre-wired nature creates a massive compute overhead for the learning phase that ends up needing a planet of compute to work.
Theta frequency I think is a key part as this is a sort of deffinition as to the temporal buffer size or the maximum extent to which a sequence fragment can exist. i.e. you can’t have sequences that are the biological equivalent of say 400mS because the chain of firing would not complete before the next theta cycle starts. All fragments are also shrunk in sleep (or during input) by hierarchical concept construction or resolving to the equivalent of smaller time fragments. i.e. hirarcical concept creation is the brains equivalent of a time compression algorithm.
The second time reference is then a form of relative temporal proximity for the sequence (proxy for one part of the synapse strength calculation) and any relative for other points that are close in the time domain. You have to think about this from a perspective of memory fragments and when and where in a fragment sequence activation occurs and then what is in temporal proximity from a STDP perspective.
HTM is sort of the equivalent of a memory fragment, wereas I look at the system as a collection of millions of fragments, i.e millions of HTM equivalents that are all interconnected.
The typical approach seems to be an assumption all sensory STDP connections are established for all the same duration (i.e. say 8mS) when the brain actually creates various micro bursts of say 3 pulses at 100Hz for one output and 2 pulses at say 110Hz for another. These bursts create the equivalent of a time shift when looking from a LIF perspective. You then need to be able to change the relative time differential for synapses. HTM assumes a consistent lapsed time per layer and time is then contant for the whole network.
Think of it along the lines of an input sequence “big brown house”, where the inputs may ordinarily be buffered internally in short term memory and replayed at Theta frequency, which may then create a roughly 10-20mS between inputs during internal replay cycles. This would allow for STDP to create connections big-brown and brown-house when looked at from a standard approach.
Sleep frequency bursts are double at around 14Hz, which would then allow for the formation of a big-house STDP connection. The higher frequency bridges points in a temporal sequence, which are already connected and also those that arer not. What’s interesting is that the waves propogate across the brain in all directions as a wave rather than singular pre-connected path threads. This would allow for any activations that are in close temporal proximity to be bridged (without the sequence being triggered) and the formation of new synapses to points that were previoulsy.
You have to think about this all in the context of everything is stored as micro sequences, like the zebra finch memory storing the short sonq sequence. The connections are limited in temporal duration because theta waves would otherwise run the risk of creating a positive feedback loop - aka a siezure.
From an input perspective the sensory inputs are buffered and each sensory input may have a relative temporal offset associated with it as to the shift that occurs to the inputs within the sequence after the last active input. This example is using words, but think about it form the aspect of an agnostic sensory input - don’t look at the words as such.
i.e. “red and blue make purple” where the sensory input “and” would create a negative temporal offset for blue to bring red and blue closer in temporal proximity for STDP to occur.
I have tried to look at the whole porocess from a time perspective rather than connections first. Time defines the connections.
I have missed out a lot of what I think is going on, as it’s difficult for me to write it down properly, so appologies if it’s not clear.
Does that make sense, trigger any thoughts ?
