I’ve seen explanations in many places that TM is sort of variable order Markov chain, the problem I have is visualizing it step by step how it happens.
From the basic structure (one-step prediction) what this implies is that we have sort of simple “transition matrix” i.e. if state-x then state-y. There is nothing variable so far, but simply order-1 chain.
Now in the docs there are the ABCD, XBCY examples, but they don’t help me see the connection, how is it variable when you still have oreder-1 transition “stored” in the TM.
I imagine that because HTM states are not dense-state (real number like in the NN), but SDR (sparse) this creates looser “transitions” i.e. different prior-bits influence differently post-bits of the next state, so in a sense you virtually create multi-step-bit-sub-states, “below” the STATE which is a composite of many-bits (SDR).
Abstractly (descriptively) I can appreciate it, but can’t seem to be able to imagine step by step (proceduraly) how this variable-markov-order is created and plays out.