Not that it matters to everyone but seeing mathematical models such as what you’ve mentioned being proven to have simulated biological behavior really amazes me. For one the XCAL function that approximated/simulated backpropagation in ANNs became useful in deep predictive learning - Deep Predictive Learning: A Comprehensive Model of Three Visual Streams .
In current (Numenta) systems, the SP and TP/TM are separate (which they shouldn’t be and aren’t in more advanced versions). Given that, the SP’s objective function is to minimise the norm of the differences between its outputs divided by the norm of the inputs, and to maximise the differences between the directions of its outputs (or minimise the dot product of the outputs) given the difference between the inputs. For the TM, it’s quite a bit more complicated, but it’s about essentially about the causal power of the signal at t to predict the signal at t+n steps. Another formulation is to look at the bitrate of the TM output compared with that of its input, and examine the mutual information between the two.
2 Likes