HTM detecred an anomaly. How to find out the reason?

I just got this question (see the topic) when I made a speech on the meetup. Actually I did not know what to answer: HTM gets the encoded data, passes through SP and TP, then finds anomaly likelihood and tells that “at this point I feel I see an an anomaly - I estimate is as 90% probability”.

However, are there any ideas how to encode data, what data to include together with metrics+time into the encoder, whatever - just to be able to BIND some happening factors with anomalies and to tell that “this factor is most probably the infuencer of the anomaly”?

Thanks in advance.

1 Like

As I see it, any agent can’t make a distinction of the reasons of its output without having patterns of potential causes as a part of its input.
So, If you need it, you have to encode, for instance, identifiers of known to you possible cases of anomalies in your encoder.

Every input has a record number and then HTM.java each inference outputted holds that record number

In theory if you could look at the predicted cells and active cells from the timestep of the anomaly and decode them, you could see what factor values the systems was expecting and what actually occurred. However its not clear from that to what extent the anomalous behavior was spatial or temporal. Maybe the activated columns from the SP were totally unusual (spatial anomaly), or maybe those columns were common but just not expected in that context (temporal anomaly).

Also when dealing with anomaly likelihood, the anomalous behavior is likely spread out over some time since the current anomaly scores is compared to the average anomaly score (and standard deviation) over a time window. If the data is clean and predictable then the anomaly likelihood could be more easily triggered by one anomaly score at one timestep, but if its more noisy its likely to take longer I believe.