What areas of mathematics are useful in the study of HTM theory?

I’ll be finishing an undergraduate course in mathematics within the next year, and I’d like to know from working researchers or other enthusiasts which areas of mathematics are useful in the study of HTM theory. (This might be either just for self-study or possibly for choosing a focus area in a future master’s program.)

I’d appreciate any input. Thank you.

1 Like

As someone who knows the basics of HTM theory, at least in terms of how the SP and TM algorithms work, I can say with some confidence that there’s not a high math requirement to learn and apply HTM theory. Of the usual undergrad classes Linear Algebra is always useful to know, but one not as common thats very relevant and interesting is Graph Theory. Graph theory is math oriented toward networks. I’m not an expert in this either, I just know that when I took Network Science class in my Systems Science MS program I discovered the subject of my interest in math and computing. I’m now a Ph.D student at the same school applying HTM to my thesis project with NuPIC. I don’t want to ramble too much but I’m glad to adviser/discuss it more If you’d like. I’m at: sheiser1@binghamton.edu

3 Likes

Linear algebra. If you understand this, you win: https://en.m.wikipedia.org/wiki/Johnson–Lindenstrauss_lemma

sheiser1: I’d imagined that, as you said, there isn’t much of a math requirement in order to understand and apply the theory, but I wondered if the people with somewhat more background in mathematics thought that there were any areas of study that could provide further insight. Thank you for your reply, especially with regards to graph theory - I’ll look into it.

blue2: So you’d suggest a lot more linear algebra than what is covered in a typical undergraduate course? With respect to the lemma you mentioned, I’d never heard of it, but it does make sense that it’d be useful in the context of HTM… The wikipedia link seems to suggest you might also need to know some measure theory (and maybe probability theory) in order to understand that result… Thank you for answer.

Basic statistics is useful.

The mathematical underpinnings of HTM theory still have to be developed
for the most part, so what math exactly is “required” is something that
we will only be able to answer in retrospect after the fact. Phrasing
your question in terms of university courses makes it even harder to answer.

About probability theory, if it’s the Bayesian type, I’m sceptical as it
is “coarse grained” so to speak and motivated by language and an
anthropocentric world view. Early HTM theory was based on classical
probability and statistics and that led to a dead end.

In the end, your only bet is to get yourself into the right mindset, and
acquire a broad body of knowledge and hope for it to cover the areas
needed. Physics is my idea of “the right mindset”, and IMHO the only
one. No discipline excels in applying the tools of math and theories to
problems like physics does. All the mathematicians and computer
scientists in the HTM community have great knowledge of the tools and no
skills in applying them whatsoever and will get exactly nowhere for that
reason.