Step up change point detection

question
#1

Is is possible to perform step up change point detection using NuPIC?

For more details see: https://stats.stackexchange.com/q/400584/106101

I’m not sure whether this is possible since anomaly detection covers much more cases than just step detection. And also I am not aware of any way to tell NuPIC to detect anomalies that only happen as increasing values.

Thank you

1 Like

#2

Should be possible. Once a plateau is learned, any new jump will cause anomalies unless the jump is to a previously learned space.

0 Likes

#3

“unless the jump is to a previously learned space”

Yes, I forgot about that. That is actually the main problem: NuPIC gets accustomed to the anomalies so that anomalies over time become the norm - this makes NuPIC not usable for step up detection and I don’t know how to make it usable.

Also note that anomaly detection is not same as step up detection - which is just a limited set from all the anomalies.

0 Likes

#4

You can control how quickly NuPIC forgets things though, so you can choose how long to remember those old plateaus.

0 Likes

#5

You can control how quickly NuPIC forgets things though, so you can choose how long to remember those old plateaus.

Ok, let’s say this works.

There is still one other issue preventing NuPIC from doing step up detection: step up events are a subset of anomaly events. So just doing anomaly detection is not enough as it would give a lot of events which are anomalies that are not of step up anomaly type.

0 Likes

#6

If you define step up / step down events as more anomalous than underlying anomalies within the data signature (aside from the stepping), you could define a threshold against the anomaly likelihood window that would separate “regular” anomalies occurring within the non-stepped stream and the actual step activity. I think it is still possible to differentiate the step events from normal anomaly events if they are much different in magnitude than the normal anomaly events.

0 Likes

#7

In software a lot of things are possible :slight_smile: The question is how to do step up detection? Is this not implemented yet in NuPIC? Because that is what I am asking.

0 Likes

#8

I think it could be possible using NuPIC anomaly detection as-is, just fiddling with the TM params and anomaly threshold settings. But it really depends on the data. Do you have a sample?

0 Likes

#9

This is an example of how data might look:

It’s a plot of tweets on y axis and time on x axis.
As you can see there is 1 big spike (step up) in the data and other smaller ones.

Do you know how I might obtain some starting code that does step up detection which I can tweak? This is the main use-case at our company and we would be looking into buying a commercial license if this works.

1 Like

#10

The first thing you should do is try running this through HTM Studio. If the anomalies it shows you look interesting, then proceed.

0 Likes

#11

@Razvan_Flavius_Panda your aim is to detect anomaly in time series but the issue is the time series change over time so you before detecting anomaly you must detect the change point isn’t?

@rhyolight yeah me too this is what I guess but the problem in some cases when the temporal anomaly have the same model of the normal data or detect the normal model like an anomaly because they are changed

0 Likes

#12

@rhyolight I tried running it through HTM studio with 2 step up anomalies. The system detects the first anomaly but not the 2nd. I think this is because the system already adapted to the anomaly.

0 Likes

#13

@way-sal My goal is to detect step ups in the time series, not anomalies.

0 Likes

#14

Can you show the graph? HTM Studio gives you a starting point as far as model parameters. If you used those params and started tuning how fast synapses decay in the TM params. Take a look at these sample params. The ones you might twiddle with are permanenceDec, maxAge, globalDecay.

0 Likes

#15

Which graph are you referring to?
Let’s assume NuPIC would have caught the 2nd anomaly also. I still don’t see how these parameters would help with doing step up detection.This is about step up detection, emphasis on UP.

0 Likes

#16

The graph or chart that HTM Studio showed you of your data with anomalies overlaid on top.

These params affect how quickly the system forgets patterns it has learned. For a step problem, you want to forget the old plateaus quicker, right?

It really should not matter up or down.

0 Likes

#17

It really should not matter up or down.

It does matter. Those are the requirements - detect ONLY step UPs. A system that returns both step ups and step downs is not useful.

And anomaly is not same thing as step up. Step ups are a subset of all possible anomaly types.

So my question is: how can you tell NuPIC to only give you anomalies of type step up?

1 Like

#18

I don’t know any way to do this with NuPIC.

0 Likes

#19

@rhyolight After discussing with @Paul_Lamb I understood that it is possible to prevent NuPIC from getting accustomed to the spikes in the presence of seasonal trends by setting the forgetting rate to a suitable value.

I think it might be possible to detect whether an anomaly is a step up or down by comparing the actual value at that point in time with the predicted value. Does the anomaly detection algorithm in NuPIC also return the predicted value?

1 Like

#20

Yes it can.

0 Likes