Subutai Ahmad, Scott Purdy
(Submitted on 8 Jul 2016)
Much of the world’s data is streaming, time-series data, where anomalies give significant information in critical situations. Yet detecting anomalies in streaming data is a difficult task, requiring detectors to process data in real-time, and learn while simultaneously making predictions. We present a novel anomaly detection technique based on an on-line sequence memory algorithm called Hierarchical Temporal Memory (HTM). We show results from a live application that detects anomalies in financial metrics in real-time. We also test the algorithm on NAB, a published benchmark for real-time anomaly detection, where our algorithm achieves best-in-class results.
1 Like
Subutai talking recently about this.
1 Like
Hey
I read the paper (and the supplementary material) and I was wondering if you had made some progress concerning the extension of HTM for a large number of data streams inputs. I am interesting to try, but I am afraid of the complexity of the training and inference.
Hi @Meylina - do you mean the section S4? I have tried it on a couple of datasets but not extensively (in part because we don’t have good datasets). Please let us know if you try it! In practical deployments we surprisingly found that the simple approach of just modeling each stream independently and then triggering an anomaly if any of them detected an anomaly worked reasonably well.
Hi @subutai
Yes, I meant the section S4. Thank you for the reply. I will let you know if my results are interesting.