When talking about memory systems in ML most people think first about RNNs where we build a neural network with weight sharing over time. An RNN maintains a fixed-sized vector of internal state and we backpropagate some kind of error to make this state useful to perform some task.
In HTM we have sparse representations and Hebbian learning VS dense representation and gradient descent in RNNs.
So RNNs are not really very similar to HTM.
What would be a good ANN equivalent to HTM?
Firstly, I think we need a very wide and big neural network with lots of parameters. The net is a function from the state (context) and input, to a new state. Instead of letting the system learn slowly and average over all samples, we use backpropogation to aggressivally update the net on each new input by making the gradient descent step size huge instead of very small. There’s no backpropogation through time, when we train the network we just iterativly give the last state together with the input to the network.
So the net has a huge sparse parameter space to store memory at, and each new input writes to a piece of memory somewhere in this space, parametrized by the context.