Estimate the capacity of an HTM model

Science Numenta Theory
1

I am trying to estimate (even if it is pessimistic) the capacity of a single-layer htm model analytically. I think it depends on several characteristics of the model like the number of columns, number of cells per column and the sparsity.

I have various doubts:

  1. What is the best way to express the capacity? Perhaps it can be estimated by the number of possible transitions?
  2. Is there any formula for this?
  3. Taking into account the entropy of the input data, exceeded the capacity of the model, How does the process of “forgetting” occur?

Maybe you can help me and guide me to some references.

Thanks,
rtaranco.

2 Likes
2

Hi @rtaranco,

The number of transitions is a pretty good way to express capacity of the sequence memory. We estimated this in our paper using a reasonable set of parameters. Look at page 9 here: