I’m trying to play with NuPic to detect anomalies in really fast data that can have irregular intervals (500ms, 1s, 2s, 10s) and that can be really noisy (orderbook changes).
Here are my questions:
- Should use a sliding window of N seconds (which will produce data at irregular intervals)
- Should I use a tumbling window of N seconds (will maybe skip some anomalies that could happens in those N seconds)
Also, as you may know, this type of data can be really noisy, I was thinking of using a sliding window of 1 minute, but this will still produce data at irregular intervals (sliding windows aggregate data at every new datapoint, and those datapoints don’t come at regular intervals).
I also saw in the OPF that there is a window aggregation option (also in HTM Studio), but I guess this will use a tumbling window right ? Is there a built in option for sliding windows ? If not and if those windows can be useful, I may contribute and add it to avoid doing this processing externally.